From steven.charles.davis@REDACTED Fri Oct 1 03:12:28 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 30 Sep 2010 18:12:28 -0700 (PDT) Subject: Multicore not such a big issue? Message-ID: <8c42f0ef-a542-4120-8f4a-25ac0f1fe701@e14g2000yqe.googlegroups.com> According to Linux... http://www.physorg.com/news205050157.html ...more delays? /s From silvester.roessner@REDACTED Fri Oct 1 07:11:57 2010 From: silvester.roessner@REDACTED (Roessner, Silvester) Date: Fri, 1 Oct 2010 07:11:57 +0200 Subject: AW: [erlang-questions] Implementation of Fact (Belief) Base In-Reply-To: References: Message-ID: <2CEB6DA5040AED4F946351C85FAC87B505432235@DEJENSAPP01V1.vision.zeiss.org> Mercury could also be interesting for you: http://www.mercury.cs.mu.oz.au/backends.html Rosswart -----Urspr?ngliche Nachricht----- Von: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] Im Auftrag von zabrane Mikael Gesendet: Donnerstag, 30. September 2010 17:45 An: David Sergey Cc: Erlang Questions Betreff: Re: [erlang-questions] Implementation of Fact (Belief) Base Have a look to ERES : http://groups.google.com/group/erlang-programming/browse_thread/thread/6ea7e7de37274523 -- Regards Zabrane 2010/9/30 David Sergey : > Hello Everybody > > I'm researching ways in Erlang to create Prolog (or any other logical language like) fact/belif base. Point is - some data structures should be stored on process for later evaluation. > > Best Regards > David > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From zabrane3@REDACTED Fri Oct 1 09:03:30 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Fri, 1 Oct 2010 09:03:30 +0200 Subject: [erlang-questions] Implementation of Fact (Belief) Base In-Reply-To: <2CEB6DA5040AED4F946351C85FAC87B505432235@DEJENSAPP01V1.vision.zeiss.org> References: <2CEB6DA5040AED4F946351C85FAC87B505432235@DEJENSAPP01V1.vision.zeiss.org> Message-ID: or Erlog: http://www.trapexit.org/forum/viewtopic.php?t=6126&sid=cc8045f60370cac21dc1c8b11839fc47 Is there any new Erlog release Robert? -- Regards Zabrane 2010/10/1 Roessner, Silvester : > Mercury could also be interesting for you: > > http://www.mercury.cs.mu.oz.au/backends.html > > Rosswart > > > -----Urspr?ngliche Nachricht----- > Von: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] Im Auftrag von zabrane Mikael > Gesendet: Donnerstag, 30. September 2010 17:45 > An: David Sergey > Cc: Erlang Questions > Betreff: Re: [erlang-questions] Implementation of Fact (Belief) Base > > Have a look to ERES : > http://groups.google.com/group/erlang-programming/browse_thread/thread/6ea7e7de37274523 > > -- > Regards > Zabrane > > > > 2010/9/30 David Sergey : >> Hello Everybody >> >> I'm researching ways in Erlang to create Prolog (or any other logical language like) fact/belif base. Point is - some data structures should be stored on process for later evaluation. >> >> Best Regards >> David >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From zabrane3@REDACTED Fri Oct 1 09:07:45 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Fri, 1 Oct 2010 09:07:45 +0200 Subject: [erlang-questions] Implementation of Fact (Belief) Base In-Reply-To: References: <2CEB6DA5040AED4F946351C85FAC87B505432235@DEJENSAPP01V1.vision.zeiss.org> Message-ID: Erlog is on Github: http://github.com/rvirding/erlog -- Regards Zabrane 2010/10/1 zabrane Mikael : > or Erlog: > http://www.trapexit.org/forum/viewtopic.php?t=6126&sid=cc8045f60370cac21dc1c8b11839fc47 > > Is there any new Erlog release Robert? > -- > Regards > Zabrane > > 2010/10/1 Roessner, Silvester : >> Mercury could also be interesting for you: >> >> http://www.mercury.cs.mu.oz.au/backends.html >> >> Rosswart >> >> >> -----Urspr?ngliche Nachricht----- >> Von: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] Im Auftrag von zabrane Mikael >> Gesendet: Donnerstag, 30. September 2010 17:45 >> An: David Sergey >> Cc: Erlang Questions >> Betreff: Re: [erlang-questions] Implementation of Fact (Belief) Base >> >> Have a look to ERES : >> http://groups.google.com/group/erlang-programming/browse_thread/thread/6ea7e7de37274523 >> >> -- >> Regards >> Zabrane >> >> >> >> 2010/9/30 David Sergey : >>> Hello Everybody >>> >>> I'm researching ways in Erlang to create Prolog (or any other logical language like) fact/belif base. Point is - some data structures should be stored on process for later evaluation. >>> >>> Best Regards >>> David >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From csanto@REDACTED Fri Oct 1 09:37:50 2010 From: csanto@REDACTED (Corrado Santoro) Date: Fri, 1 Oct 2010 09:37:50 +0200 Subject: ERESYE: Change of license Message-ID: <9478046c53add3acc7a5e3e78dac4699.squirrel@webmail.cdc.unict.it> A new release of ERESYE has been posted in Sourceforge. Version of 1.2.5 is now released with BSD License. http://sourceforge.net/projects/eresye/ All the best, --Corrado From zabrane3@REDACTED Fri Oct 1 11:59:17 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Fri, 1 Oct 2010 11:59:17 +0200 Subject: [erlang-questions] ERESYE: Change of license In-Reply-To: <9478046c53add3acc7a5e3e78dac4699.squirrel@webmail.cdc.unict.it> References: <9478046c53add3acc7a5e3e78dac4699.squirrel@webmail.cdc.unict.it> Message-ID: Fantastic ! Thanks Corrado ... -- Regards Zabrane 2010/10/1 Corrado Santoro : > A new release of ERESYE has been posted in Sourceforge. Version of 1.2.5 > is now released with BSD License. > > http://sourceforge.net/projects/eresye/ > > All the best, > --Corrado > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From czinkos@REDACTED Fri Oct 1 12:33:53 2010 From: czinkos@REDACTED (Zsolt Czinkos) Date: Fri, 1 Oct 2010 12:33:53 +0200 Subject: mochiweb tutorial Message-ID: Hello I'm new to erlang and I'm looking for a tutorial on mochiweb. Can anybody point me to an URL? Thank you in advance Zsolt From kostis@REDACTED Fri Oct 1 16:24:09 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 01 Oct 2010 17:24:09 +0300 Subject: [erlang-questions] Dialyzer and parameterized modules In-Reply-To: <1285866041.1877.10.camel@army-desktop> References: <1285866041.1877.10.camel@army-desktop> Message-ID: <4CA5EF09.8050206@cs.ntua.gr> Fernando Benavides wrote: > Hi all, > > I know that since v2.2.0, dialyzer has support for parameterized > modules, but somehow it's not working for me. > Lets say that, for some unknown reason (o_O), I want a parameterized > module for atom queues. > I write this tiny parameterized module then: > .... CODE SNIPPED > > If I run dialyzer on it, no warnings are reported. But it has no specs > and I like to compile my code with warn_missing_spec. The compiler says > that the specs for pop/0 and push/1 are missing. So, I add the specs... > .... CODE SNIPPED > Compiler is not complaining anymore, which is great. But when I try to > dialyze the module, I get: > Contract for function that does not exist: atom_queue:pop/0 > Contract for function that does not exist: atom_queue:push/1 > > I change the specs to satisfy dialyzer, like this: > .... CODE SNIPPED > > But, of course, I can't compile that code: > spec for undefined function atom_queue:pop/1 > spec for undefined function atom_queue:push/2 > > So, basically the question is: How can I write a parameterized module > with specs that satisfies both dialyzer and erlc at the same time? First of all, let me congratulate you for your discipline in Erlang programming. However, I am afraid you have run into a problem here in what you want to do. The problem is that the infrastructure of parameterized modules was developed before -specs were introduced to Erlang. Consequently, the parse transform (or equivalent) that does the transformation into "usual" Erlang code is spec-agnostic. So, this is not a dialyzer issue but a problem in the translation of parameterized modules. I will leave this to somebody who understands that translation handle the issue. Kostis From daniel.goertzen@REDACTED Fri Oct 1 19:26:38 2010 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Fri, 1 Oct 2010 12:26:38 -0500 Subject: dialyzer/typer and exceptions Message-ID: While experimenting with dialyzer and typer, I discovered that "catch" acts as type inference barrier. As an example, gen_server:call/2 uses catch in its first line and the resulting inferred type signature is: -spec call(_,_) -> any(). Is there a way to get dialyzer/typer to ignore catches and assume the non-exception case, or is that opening pandora's box? Also, are there any other constructs that are known to confound dialyzer/typer? Thanks, Dan. From raould@REDACTED Fri Oct 1 20:02:30 2010 From: raould@REDACTED (Raoul Duke) Date: Fri, 1 Oct 2010 11:02:30 -0700 Subject: [erlang-questions] Multicore not such a big issue? In-Reply-To: <8c42f0ef-a542-4120-8f4a-25ac0f1fe701@e14g2000yqe.googlegroups.com> References: <8c42f0ef-a542-4120-8f4a-25ac0f1fe701@e14g2000yqe.googlegroups.com> Message-ID: On Thu, Sep 30, 2010 at 6:12 PM, Steve Davis wrote: > http://www.physorg.com/news205050157.html slightly hidden moral of the story: our concurrency debugging tools suck. From michael.santos@REDACTED Sat Oct 2 00:46:05 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 1 Oct 2010 18:46:05 -0400 Subject: [erlang-questions] VM segfault on exit with wx In-Reply-To: References: Message-ID: <20101001224605.GB30223@ecn.lan> On Tue, Sep 28, 2010 at 01:44:12PM -0400, Mark Scandariato wrote: > Seems like the VM segfaults on exit whenever I run anything that uses wx > (this is a single processor system). > > Let me know if you need any other info (or a core file or whatever). > > Mark. > > erlang@REDACTED:~/otp_src_R14B$ uname -a > Linux ubuntu 2.6.28-19-generic #65-Ubuntu SMP Thu Sep 16 14:14:28 UTC 2010 > i686 GNU/Linux > erlang@REDACTED:~/otp_src_R14B$ bin/cerl -debug -smp enable > Erlang R14B (erts-5.8.1) [source] [smp:1:1] [rq:1] [async-threads:0] [hipe] > [kernel-poll:false] [type-assertions] [debug-compiled] [lock-checking] > > Eshell V5.8.1 (abort with ^G) > 1> wx:demo(). > {wx_ref,35,wxFrame,<0.34.0>} > 2> q(). > ok > 3> Segmentation fault (core dumped) Looks as if beam is crashing printing out an error message. The attached patch seems to fix it. diff --git a/lib/wx/c_src/wxe_return.cpp b/lib/wx/c_src/wxe_return.cpp index 2c4f754..4c0e738 100644 --- a/lib/wx/c_src/wxe_return.cpp +++ b/lib/wx/c_src/wxe_return.cpp @@ -67,6 +67,6 @@ int wxeReturn::send() { if(res == -1) { wxString msg; - msg.Printf(wxT("Failed to send return or event msg")); - send_msg("internal_error", &msg); + if (msg.Printf(wxT("Failed to send return or event msg")) >= 0) + send_msg("internal_error", &msg); } reset(); From kostis@REDACTED Sat Oct 2 10:36:29 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 02 Oct 2010 11:36:29 +0300 Subject: [erlang-questions] dialyzer/typer and exceptions In-Reply-To: References: Message-ID: <4CA6EF0D.1070506@cs.ntua.gr> Daniel Goertzen wrote: > While experimenting with dialyzer and typer, I discovered that "catch" acts > as type inference barrier. Not sure what you mean by "type inference barrier". What the type inference algorithm does is to approximate the types (i.e., set of terms) for which a function will "succeed" (i.e., not raise an exception). What 'catch' does is it catches exceptions and thus allows functions to be used with more terms. For example, while the function: foo(X) -> X + 42. will return a value only for numbers and thus has type signature -spec foo(number()) -> number(). the function: bar() -> catch (X + 22). will return some value for all terms, not only for numbers, so it has the type signature: -spec bar(any()) -> any(). > As an example, gen_server:call/2 uses catch in > its first line and the resulting inferred type signature is: > > -spec call(_,_) -> any(). > > Is there a way to get dialyzer/typer to ignore catches and assume the > non-exception case, or is that opening pandora's box? The answer to your question is "of course there is such a way", but the question then is what exactly would this type inference be computing. The current one tries to model the operational semantics of Erlang, not those of some ideal(ized) language. > Also, are there any other constructs that are known to confound > dialyzer/typer? I think confound is a very strong term here, but the answer is that catch is the only such construct. What also destroys type information is catch-all clauses/cases. Independently of type inference though catch is a terrible construct and should be avoided in modern Erlang; try-catch is a much better construct for various reasons and dialyzer handles this much better. Kostis From boris.muehmer@REDACTED Sat Oct 2 10:49:41 2010 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Sat, 2 Oct 2010 10:49:41 +0200 Subject: [erlang-questions] The Book, lib_chan and IRC Lite Message-ID: I just found an old thread about the "IRC Lite" sample of "the book", because I also have some troubles getting this sample to run and I am still too inexperienced to locate the problems in the code... > The corrected versions of code/socket_dist/chat_client.erl and > code/socket_dist/chat_group.erl are attached to this mail. It looks like those attachements didn't manage it to get into the archive. And the current source archive from the Pragmatic site doesn't include those fixes. Could You please make them available? Thanks in advance, - boris From boris.muehmer@REDACTED Sat Oct 2 12:19:47 2010 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Sat, 2 Oct 2010 12:19:47 +0200 Subject: [erlang-questions] The Book, lib_chan and IRC Lite In-Reply-To: References: Message-ID: The code is correct and complete... _I_ made a mistake! I didn't start the server and client using the Makefile. Because of this "lib_md5" couldn't be found... On the other hand, this gave me my first "debugger" experience! Pretty cool! [After adding "+debug_info" to "erlc" in the Makefile and recompiling everything.] Almost like Visual Studio... ;-) - boris From michael.santos@REDACTED Sat Oct 2 20:46:29 2010 From: michael.santos@REDACTED (Michael Santos) Date: Sat, 2 Oct 2010 14:46:29 -0400 Subject: [erlang-questions] VM segfault on exit with wx In-Reply-To: <20101001224605.GB30223@ecn.lan> References: <20101001224605.GB30223@ecn.lan> Message-ID: <20101002184629.GA398@ecn.lan> On Fri, Oct 01, 2010 at 06:46:05PM -0400, Michael Santos wrote: > On Tue, Sep 28, 2010 at 01:44:12PM -0400, Mark Scandariato wrote: > > Seems like the VM segfaults on exit whenever I run anything that uses wx > > (this is a single processor system). > > > > Let me know if you need any other info (or a core file or whatever). > > > > Mark. > > > > erlang@REDACTED:~/otp_src_R14B$ uname -a > > Linux ubuntu 2.6.28-19-generic #65-Ubuntu SMP Thu Sep 16 14:14:28 UTC 2010 > > i686 GNU/Linux > > erlang@REDACTED:~/otp_src_R14B$ bin/cerl -debug -smp enable > > Erlang R14B (erts-5.8.1) [source] [smp:1:1] [rq:1] [async-threads:0] [hipe] > > [kernel-poll:false] [type-assertions] [debug-compiled] [lock-checking] > > > > Eshell V5.8.1 (abort with ^G) > > 1> wx:demo(). > > {wx_ref,35,wxFrame,<0.34.0>} > > 2> q(). > > ok > > 3> Segmentation fault (core dumped) > > Looks as if beam is crashing printing out an error message. The attached > patch seems to fix it. Wow, had another look and that "fix" doesn't help at all, that was just a heisenfix from running in gdb. Sorry for the noise! From boris.muehmer@REDACTED Sat Oct 2 23:31:28 2010 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Sat, 2 Oct 2010 23:31:28 +0200 Subject: "wx:demo()." / static example / possible bug on Ubuntu 10.04 / Erlang R13B04/R14B Message-ID: I just noticed on a Windows Server 2008 R2 system (actually running within a KVM virtual machine), that the "wx:demo()."/static-Example (using the R14B binary) gives a better result than on my (self-compiled/installed) Ubuntu 10.04 systems. On my windows systems the "wxStaticText" alignment is correct (left / center / right), but on my Ubuntu 10.04 system, all three text are on the left. This is true for Erlang R13B04 and R14B. Has anyone else experienced this behaviour? I noticed another interesting different behaviour between Windows and Ubuntu Erlang (R14B, possible other versions as well): "wx:demo()." doesn't return any warnings or errors on a windows box at all... on Ubuntu there are several Gtk-Warnings and an "WX INTERNAL ERROR: Failed to send return or event msg"... The warnings/errors on Ubuntu aren't around all the time; they also depend on the examples started. - boris From arjan@REDACTED Sun Oct 3 16:04:25 2010 From: arjan@REDACTED (Arjan Scherpenisse) Date: Sun, 03 Oct 2010 16:04:25 +0200 Subject: Zotonic 0.5.0 released Message-ID: <4CA88D69.1020903@scherpenisse.net> Dear Zotonic users and Erlang enthousiasts, After a long period of silence, I'm very pleased to announce the fifth release of Zotonic, the Erlang CMS [1]. As usual, the zip file can be downloaded from Google code: http://code.google.com/p/zotonic/downloads/list This release is a major feature release. As you can read in the release notes, we have been very actively developing Zotonic and are happy that we can present a new stable version. Please read on for the release notes. Kind regards, The zotonic team, Arjan Scherpenisse Marc Worrell Tim Benniks [1] More info on Zotonic: http://zotonic.com Release 0.5.0, released on 2010-10-03 ------------------------------------- * New features: ** Simpler module system Modules are simpler, do not have to be a fullblown gen_server. Registering z_notifier for modules is made more simpler by using erlang's introspection on modules. ** i18n support through gettext Gettext .po and .pot file support for translations. Templates can be translated per module. Pot files are automatically generated from the templates. ** Pluggable Access Control system The new ACL structure works through pluggable ACL modules. Two ACL modules are included as examples. mod_acl_adminonly, where all users are admins, and mod_acl_simple_roles, which implements a simple role based ACL system. ** Authentication can now be customized and extended. mod_authentication is the basic module used for authentication. This module can be extended. The mod_facebook is an (incomplete) example of such an extender. mod_authentication implements the username/password authentication, including logon and logoff. It also supports 'password forgotten' e-mails. ** User signup Non admin users can sign up using the mod_signup. This module works in harmony with the authentication module and authentication extenders. ** New OTP supervisor hierarchy. The PostgreSQL connection pool is now part of the individual sites. Sites are more isolated and can be individually started, restarted or stopped. It is possible to add and remove sites without restarting Zotonic. Modules are now isolated and the running status of a module is displayed in the admin's module overview. ** A status overview site, zotonic_status. zotonic_status shows the running status of all sites. When logged in, the user can start/stop/restart sites using his browser. It is also possible to do 'hg pull' updates of sites that contain a mercurial repo. ** New ErlyDTL tags: {% inherit %}, {% overrule %} ** New ErlyDTL support for multiple argument {% with %}: {% with a,b as c,d %} ** New ErlyDTL support for filters with multiple parameters. ** New ErlyDTL test set, including regression tests. ** New ErlyDTL filters group_by_title_firstchar, is_visible, pprint, urlize, without_embedded_media. ** Media preview enhancements {% image %} now supports the the following new arguments: 'extent' - create a larger image then the original not by scaling up but by adding a border to the image. 'removebg' - removes the image's background. It accepts an optional fuzziness parameter (range 0..100). 'upscale' - Forces a small image to scale up to the requested dimensions. ** Extended support for Websocket connections. The two newest protocols, as used by Chrome and Safari, are supported. ** mod_development improvements It now supports turning on or off the concatenation of {% lib %} includes as one file or separate files, and can give a live trace of translated templates, showing clearly the template inheritance and selections. ** mod_menu improvements It implements the menu now as a template, easing your own menu implementation. ** mod_emailer improvements It can now inline images into the e-mails ** New: mod_slideshow It can make a slideshow of any collection, you can add your own slide templates. ** New: mod_contact Simple contact form which gets sent over e-mail ** New: mod_facebook Facebook logon ** New: mod_imageclipper A simple javascript image-clipper bookmarklet for grabbing images from other websites. ** New: mod_logging A realtime log of debug messages and errors in the system. ** System wide configuration system (z_config) using a configuration file at 'priv/config' * Bugfixes: ** AllowHTML5 audio and video tags (#75) ** Typo in m_config, line 127. undefind -> undefined (#83) ** setting initial admin password does not work (#88) ** After upgrading the code to latest changeset admin authentication causes exception (#91) ** Menu module does not follow ACL rules (#92) ** Crash in start.sh using Erlang R14A on Mac OS X 10.6 (#93) ** Extra Atom Link (#95) ** Makefiles use rm GNUism (#96) ** z_email:split_name_email/1 does not what it says it should do (#97) ** dots in page paths are transformed into dashes (#98) ** attaching media to pages does not work correctly (#99) ** After a module crashes, the new dynamic observe_* methods are not re-initialized (#100) ** setting page path and unique name is broken (#101) ** IF statements on empty rsc_list structures (#104) ** When image is too small, providing only a width should not make the image very large (#105) ** And many various other fixes which users noted on the mailinglist and were fixed quickly. From pablo.platt@REDACTED Sun Oct 3 17:00:35 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Sun, 3 Oct 2010 08:00:35 -0700 (PDT) Subject: node.js compared to erlang Message-ID: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Hi I'm using erlang and I love it. I'm not trying to create an argument but to better understand the power of erlang. In what areas erlang dominates and what areas will you consider using node.js? For example, would you consider building something like rabbitmq or ejabberd in node.js? Or maybe you'll use node.js just for a simple single chat room but erlang for anything complicated than that. Are there a fundamental differences in performance or stability or use in distributed systems? Ulf Wiger commented on the comparison of erlang and node.js and said that erlang solves the problem of non blocking functions which might be very hard for other languages. http://journal.dedasys.com/2010/04/29/erlang-vs-node-js Thanks From max.lapshin@REDACTED Sun Oct 3 18:06:36 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 3 Oct 2010 20:06:36 +0400 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: <838303.44876.qm@web112615.mail.gq1.yahoo.com> References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: On Sun, Oct 3, 2010 at 7:00 PM, Pablo Platt wrote: > Hi > > I'm using erlang and I love it. > I'm not trying to create an argument but to better understand the power of > erlang. > > In what areas erlang dominates and what areas will you consider using node.js? I can't imagine area to use node.js? Awful language, lack of good and tested libraries. I can't see a reason to prefer ruby or python to Javascript. It doesn't solve any problem, that erlang solves. From gnoblin@REDACTED Sun Oct 3 19:43:43 2010 From: gnoblin@REDACTED (Slav Pankratov) Date: Sun, 3 Oct 2010 20:43:43 +0300 Subject: misultin rest example not working In-Reply-To: References: Message-ID: Hello! I am trying REST example here: http://code.google.com/p/misultin/wiki/ExamplesPage And when I try http://localhost:8080/users/roberto/messages I get an error: =ERROR REPORT==== 3-Oct-2010::20:35:20 === Error in process <0.36.0> with exit value: {function_clause,[{misultin_http,enc_headers,["This is ~s's messages page."]},{misultin_http,enc_headers,1},{misultin_http,call_mfa,2},{misultin_http,handle_get,2},{misultin_http,body,2}]} It's ok when I try - http://localhost:8080/ : You should read "Main home page."; - http://localhost:8080/users : You should read "Main users root."; but accessing users' folders and users' messages fail. What am I doing wrong? thanks! -Slav From rapsey@REDACTED Sun Oct 3 19:52:26 2010 From: rapsey@REDACTED (Rapsey) Date: Sun, 3 Oct 2010 19:52:26 +0200 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: <838303.44876.qm@web112615.mail.gq1.yahoo.com> References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: On Sun, Oct 3, 2010 at 5:00 PM, Pablo Platt wrote: > Hi > > I'm using erlang and I love it. > I'm not trying to create an argument but to better understand the power of > erlang. > > In what areas erlang dominates and what areas will you consider using > node.js? > For example, would you consider building something like rabbitmq or > ejabberd in > node.js? > Or maybe you'll use node.js just for a simple single chat room but erlang > for > anything complicated than that. > > Are there a fundamental differences in performance or stability or use in > distributed systems? > > Ulf Wiger commented on the comparison of erlang and node.js > and said that erlang solves the problem of non blocking functions which > might be > very hard for other languages. > http://journal.dedasys.com/2010/04/29/erlang-vs-node-js > > Thanks > > If you're proficient in Erlang, I don't see a single advantage in using node.js for anything. Sergej > > From roberto@REDACTED Sun Oct 3 20:02:12 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 3 Oct 2010 20:02:12 +0200 Subject: [erlang-questions] misultin rest example not working In-Reply-To: References: Message-ID: hi slav, it works perfectly fine on various configs here, misultin version 0.6.1. please state erlang and misultin version, it might help. r. 2010/10/3 Slav Pankratov : > Hello! > > I am trying REST example here: > http://code.google.com/p/misultin/wiki/ExamplesPage > > And when I try http://localhost:8080/users/roberto/messages > > I get an error: > > =ERROR REPORT==== 3-Oct-2010::20:35:20 === > Error in process <0.36.0> with exit value: > {function_clause,[{misultin_http,enc_headers,["This is ~s's messages > page."]},{misultin_http,enc_headers,1},{misultin_http,call_mfa,2},{misultin_http,handle_get,2},{misultin_http,body,2}]} > > It's ok when I try > > ? - http://localhost:8080/ : You should read "Main home page."; > ? - http://localhost:8080/users : You should read "Main users root."; > > but accessing users' folders and users' messages fail. > > What am I doing wrong? > > thanks! > > -Slav > -- ------------------------------------------------------------------- Roberto Ostinelli CTO, WideTag Inc. - Realtime, Social, Green widetag.com skype: rostinelli twitter: ostinelli mobile: +39 335 6 100 22 6 From max.lapshin@REDACTED Sun Oct 3 20:20:06 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 3 Oct 2010 22:20:06 +0400 Subject: Application config (.app) without app file Message-ID: Hi. Is it possible to start application without .app file on disk? It seems, that loading config from disk is hardcoded in load_application functionality. From erlang@REDACTED Sun Oct 3 20:40:20 2010 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 3 Oct 2010 20:40:20 +0200 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: <838303.44876.qm@web112615.mail.gq1.yahoo.com> References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: You have to ask why was erlang designed? why was node.js designed? I don't know why node.js ws desiged - I guess so you could write servers in js. Erlang was designed for building soft real-time fault-tolerant systems that could be upgraded without taking them out of service. These design criteria led to erlang features like: - fast per/process garbage collection - ability to change code on-the-fly (ie the module reload stuff, with the ability to run old and new module code at the same time) - several orthogonal error detection mechanisms (catch-throw/links/ ...) - cross platform error detection and recovery (ie to make something fault tolerant needs at least 2 machines, think the case when one machine crashes - the second machine must be able to take over) In the erlang system there is quite a lot going on behind the scenes to make sure this all happens without the user being aware of it - to first approximation you can spread processes and database tables over multiple nodes and it will behave in a reasonable manner ... I don't think things like have any correspondence in node.js - I guess if an entire node.js node crashes the user would not expect another node to take over in a seamless manner. The fun stuff in Erlang has to do with how the failure model interacts with code changing, moving code around, upgrading code without stopping the system and so on - these characteristics are extremely important if you want to build a 24x7 system with zero down time - less so if you just want to serve up pages as fast as possible and don't care if you take the system out of service for upgrades or errors. Erlang was designed for building fault-tolerant systems - node.js was not Cheers /Joe On Sun, Oct 3, 2010 at 5:00 PM, Pablo Platt wrote: > Hi > > I'm using erlang and I love it. > I'm not trying to create an argument but to better understand the power of > erlang. > > In what areas erlang dominates and what areas will you consider using node.js? > For example, would you consider building something like rabbitmq or ejabberd in > node.js? > Or maybe you'll use node.js just for a simple single chat room but erlang for > anything complicated than that. > > Are there a fundamental differences in performance or stability or use in > distributed systems? > > Ulf Wiger commented on the comparison of erlang and node.js > and said that erlang solves the problem of non blocking functions which might be > very hard for other languages. > http://journal.dedasys.com/2010/04/29/erlang-vs-node-js > > Thanks > > > From kaiduanx@REDACTED Sun Oct 3 21:07:25 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Sun, 3 Oct 2010 15:07:25 -0400 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: Joe, - cross platform error detection and recovery (i.e. to make something fault tolerant needs at least 2 machines, think the case when one machine crashes - the second machine must be able to take over) Can you give us a real-life example on this point? I have not seen an illustration for this in your book, and in Francesco/Simon's book. The upcoming Erlang/OTP-In-Action also does not touch this area. OTP supervisor structure seems focusing on re-start process on local machine. Of course Mnesia is a good example. I would like to see a code example from other projects too. Best regards, /Kaiduan On Sun, Oct 3, 2010 at 2:40 PM, Joe Armstrong wrote: > You have to ask why was erlang designed? why was node.js designed? > > I don't know why node.js ws desiged - I guess so you could write servers > in js. > > Erlang was designed for building soft real-time fault-tolerant systems > that could be > upgraded without taking them out of service. These design criteria led > to erlang features like: > > - fast per/process garbage collection > - ability to change code on-the-fly (ie the module reload stuff, with the > ?ability to run old and new module code at the same time) > - several orthogonal error detection mechanisms (catch-throw/links/ ...) > - cross platform error detection and recovery > ?(ie to make something fault tolerant needs at least 2 machines, > ? think the case when one machine crashes - the second machine must be > ?able to take over) > > In the erlang system there is quite a lot going on behind the scenes to make > sure this all happens without the user being aware of it - to first > approximation > you can spread processes and database tables over multiple nodes and > it will behave in a reasonable manner ... > > I don't think things like have any correspondence in node.js - I guess if > an entire node.js node crashes the user would not expect another node > to take over > in a seamless manner. > > The fun stuff in Erlang has to do with how the failure model interacts with > code changing, moving code around, upgrading code without stopping the system > and so on - these characteristics are extremely important if you want to > build a 24x7 system with zero down time - less so if you just want to serve up > pages as fast as possible and don't care if you take the system out of service > for upgrades or errors. > > Erlang was designed for building fault-tolerant systems - node.js was not > > Cheers > > /Joe > > > > > On Sun, Oct 3, 2010 at 5:00 PM, Pablo Platt wrote: >> Hi >> >> I'm using erlang and I love it. >> I'm not trying to create an argument but to better understand the power of >> erlang. >> >> In what areas erlang dominates and what areas will you consider using node.js? >> For example, would you consider building something like rabbitmq or ejabberd in >> node.js? >> Or maybe you'll use node.js just for a simple single chat room but erlang for >> anything complicated than that. >> >> Are there a fundamental differences in performance or stability or use in >> distributed systems? >> >> Ulf Wiger commented on the comparison of erlang and node.js >> and said that erlang solves the problem of non blocking functions which might be >> very hard for other languages. >> http://journal.dedasys.com/2010/04/29/erlang-vs-node-js >> >> Thanks >> >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ulrich.moritz@REDACTED Sun Oct 3 21:20:35 2010 From: ulrich.moritz@REDACTED (Moritz Ulrich) Date: Sun, 3 Oct 2010 21:20:35 +0200 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: There is a whole chapter in the erlang documentation about this: http://www.erlang.org/doc/design_principles/distributed_applications.html (It's an application-, not a supervisor-feature) On Sun, Oct 3, 2010 at 9:07 PM, Kaiduan Xie wrote: > Joe, > > - cross platform error detection and recovery > (i.e. to make something fault tolerant needs at least 2 machines, > think the case when one machine crashes - the second machine must be > able to take over) > > Can you give us a real-life example on this point? I have not seen an > illustration for this in your book, and in Francesco/Simon's book. The > upcoming Erlang/OTP-In-Action also does not touch this area. OTP > supervisor structure seems focusing on re-start process on local > machine. > > Of course Mnesia is a good example. I would like to see a code example > from other projects too. > > Best regards, > > /Kaiduan > On Sun, Oct 3, 2010 at 2:40 PM, Joe Armstrong wrote: >> You have to ask why was erlang designed? why was node.js designed? >> >> I don't know why node.js ws desiged - I guess so you could write servers >> in js. >> >> Erlang was designed for building soft real-time fault-tolerant systems >> that could be >> upgraded without taking them out of service. These design criteria led >> to erlang features like: >> >> - fast per/process garbage collection >> - ability to change code on-the-fly (ie the module reload stuff, with the >> ?ability to run old and new module code at the same time) >> - several orthogonal error detection mechanisms (catch-throw/links/ ...) >> - cross platform error detection and recovery >> ?(ie to make something fault tolerant needs at least 2 machines, >> ? think the case when one machine crashes - the second machine must be >> ?able to take over) >> >> In the erlang system there is quite a lot going on behind the scenes to make >> sure this all happens without the user being aware of it - to first >> approximation >> you can spread processes and database tables over multiple nodes and >> it will behave in a reasonable manner ... >> >> I don't think things like have any correspondence in node.js - I guess if >> an entire node.js node crashes the user would not expect another node >> to take over >> in a seamless manner. >> >> The fun stuff in Erlang has to do with how the failure model interacts with >> code changing, moving code around, upgrading code without stopping the system >> and so on - these characteristics are extremely important if you want to >> build a 24x7 system with zero down time - less so if you just want to serve up >> pages as fast as possible and don't care if you take the system out of service >> for upgrades or errors. >> >> Erlang was designed for building fault-tolerant systems - node.js was not >> >> Cheers >> >> /Joe >> >> >> >> >> On Sun, Oct 3, 2010 at 5:00 PM, Pablo Platt wrote: >>> Hi >>> >>> I'm using erlang and I love it. >>> I'm not trying to create an argument but to better understand the power of >>> erlang. >>> >>> In what areas erlang dominates and what areas will you consider using node.js? >>> For example, would you consider building something like rabbitmq or ejabberd in >>> node.js? >>> Or maybe you'll use node.js just for a simple single chat room but erlang for >>> anything complicated than that. >>> >>> Are there a fundamental differences in performance or stability or use in >>> distributed systems? >>> >>> Ulf Wiger commented on the comparison of erlang and node.js >>> and said that erlang solves the problem of non blocking functions which might be >>> very hard for other languages. >>> http://journal.dedasys.com/2010/04/29/erlang-vs-node-js >>> >>> Thanks >>> >>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Moritz Ulrich Programmer, Student, Almost normal Guy http://www.google.com/profiles/ulrich.moritz BB5F086F-C798-41D5-B742-494C1E9677E8 From erlang@REDACTED Sun Oct 3 22:25:48 2010 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 3 Oct 2010 22:25:48 +0200 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: On Sun, Oct 3, 2010 at 9:07 PM, Kaiduan Xie wrote: > Joe, > > - cross platform error detection and recovery > (i.e. to make something fault tolerant needs at least 2 machines, > think the case when one machine crashes - the second machine must be > able to take over) > > Can you give us a real-life example on this point? I have not seen an > illustration for this in your book, and in Francesco/Simon's book. When you spawn_link a process on a remote node it behaves just like a local process (apart from latency) - if the entire node or machine dies you'll get an exit. You can arrange that proceses in a supervisor tree are on remote nodes and thus try to restart them or restart them on a different node. In the books I guess we don't make the point strongly enough, but the link between processes is something that can exist across machine boundaries. So whatever applies on one machine (regarding error recovery) should work across machine boundaries. The process architectures on single nodes and in distributed systems are broadly similar with the exception that entire groups of processes will die if a node dies, even if they are not linked together. Most real life examples need only pretty simple things like replicated data in mnesia and a cross machine takeover mechanism. > The > upcoming Erlang/OTP-In-Action also does not touch this area. OTP > supervisor structure seems focusing on re-start process on local > machine. But there is no need why this should be so - supervisor structures could cross machine boundaries if necessary. The essential point is that spawn_link works over machine boundaries delivering exit signals across machine boundaries if processes die. The supervisors are just wrappers that encapsulate this mechanism in a convenient way. /Joe > > Of course Mnesia is a good example. I would like to see a code example > from other projects too. > > Best regards, > > /Kaiduan > On Sun, Oct 3, 2010 at 2:40 PM, Joe Armstrong wrote: >> You have to ask why was erlang designed? why was node.js designed? >> >> I don't know why node.js ws desiged - I guess so you could write servers >> in js. >> >> Erlang was designed for building soft real-time fault-tolerant systems >> that could be >> upgraded without taking them out of service. These design criteria led >> to erlang features like: >> >> - fast per/process garbage collection >> - ability to change code on-the-fly (ie the module reload stuff, with the >> ?ability to run old and new module code at the same time) >> - several orthogonal error detection mechanisms (catch-throw/links/ ...) >> - cross platform error detection and recovery >> ?(ie to make something fault tolerant needs at least 2 machines, >> ? think the case when one machine crashes - the second machine must be >> ?able to take over) >> >> In the erlang system there is quite a lot going on behind the scenes to make >> sure this all happens without the user being aware of it - to first >> approximation >> you can spread processes and database tables over multiple nodes and >> it will behave in a reasonable manner ... >> >> I don't think things like have any correspondence in node.js - I guess if >> an entire node.js node crashes the user would not expect another node >> to take over >> in a seamless manner. >> >> The fun stuff in Erlang has to do with how the failure model interacts with >> code changing, moving code around, upgrading code without stopping the system >> and so on - these characteristics are extremely important if you want to >> build a 24x7 system with zero down time - less so if you just want to serve up >> pages as fast as possible and don't care if you take the system out of service >> for upgrades or errors. >> >> Erlang was designed for building fault-tolerant systems - node.js was not >> >> Cheers >> >> /Joe >> >> >> >> >> On Sun, Oct 3, 2010 at 5:00 PM, Pablo Platt wrote: >>> Hi >>> >>> I'm using erlang and I love it. >>> I'm not trying to create an argument but to better understand the power of >>> erlang. >>> >>> In what areas erlang dominates and what areas will you consider using node.js? >>> For example, would you consider building something like rabbitmq or ejabberd in >>> node.js? >>> Or maybe you'll use node.js just for a simple single chat room but erlang for >>> anything complicated than that. >>> >>> Are there a fundamental differences in performance or stability or use in >>> distributed systems? >>> >>> Ulf Wiger commented on the comparison of erlang and node.js >>> and said that erlang solves the problem of non blocking functions which might be >>> very hard for other languages. >>> http://journal.dedasys.com/2010/04/29/erlang-vs-node-js >>> >>> Thanks >>> >>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From robert.virding@REDACTED Mon Oct 4 00:00:50 2010 From: robert.virding@REDACTED (Robert Virding) Date: Sun, 3 Oct 2010 22:00:50 +0000 (GMT) Subject: [erlang-questions] List comprehension filter expressions not throwing exceptions In-Reply-To: <1981926268.57981286142779976.JavaMail.root@zimbra> Message-ID: <784559543.58001286143250271.JavaMail.root@zimbra> The reason for this behaviour is that the compiler specially handles filters which are also valid guard tests. After a generator then all the *directly* following filters which are valid guard tests are handled as guards to the generators and not as "normal" tests. While this is more efficient it does lead to the inconsistencies you discovered in the case of errors in the filter tests. It seemed like a good idea at the time. I don't know if it is possible to change this today, if one wanted to. Robert ----- "Sam Bobroff" wrote: > Hi all, > > I've discovered (after some rather painful debugging) that badrecord > exceptions (and others, actually) are sometimes silently converted to > false when they're generated inside a list comprehension filter > expression (as if they were a guard?). > > I couldn't find any reference to this behaviour in the manual, so > perhaps it's a bug or perhaps there needs to be a note in the manual. > It > certainly surprised me. > > Here's some code to reproduce the behaviour: > > -module(lc). > -export([a/0, b/0]). > -record(rec, {x}). > > a() -> > [E || E <- [1], E#rec.x =:= 1]. > > b() -> > [E || E <- [1], check_element(E)]. > > check_element(E) -> > E#rec.x =:= 1. > > Tested on R13B04, R14A or R14B, I get: > > $ erl > Erlang R14A (erts-5.8) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.8 (abort with ^G) > 1> c(lc). > {ok,lc} > 2> lc:a(). > [] > 3> lc:b(). > ** exception error: {badrecord,rec} > in function lc:check_element/1 > in call from lc:'-b/0-lc$^0/1-0-'/1 > 4> > > Where I would expect both a() and b() to crash with badrecord. As the > code shows, executing the same expression wrapped in a function call > *does* cause the exception to be raised, so it's not as simple as all > exceptions being converted. > > Peace, > Sam. > -- > Sam Bobroff | sam@REDACTED | M5 Networks > Why does my email have those funny headers? Because I use PGP to sign > my email (and you should too!): that's how you know it's really from > me. > See: http://en.wikipedia.org/wiki/Pretty_Good_Privacy From robert.virding@REDACTED Mon Oct 4 00:09:54 2010 From: robert.virding@REDACTED (Robert Virding) Date: Sun, 3 Oct 2010 22:09:54 +0000 (GMT) Subject: [erlang-questions] Implementation of Fact (Belief) Base In-Reply-To: <1737187941.58051286143568640.JavaMail.root@zimbra> Message-ID: <671885214.58071286143794243.JavaMail.root@zimbra> No, Zabrane the latest release is the one on github. Erlog has not died or been shelved. It is dormant because what is really needed now is more work on the interface between erlang and erlog, and for that I need some users to provide input. The logic engine works fine within the limitations imposed by implementing it in erlang. If anyone is considering using erlog please contact me. Robert ----- "zabrane Mikael" wrote: > Erlog is on Github: > http://github.com/rvirding/erlog > > -- > Regards > Zabrane > > > 2010/10/1 zabrane Mikael : > > or Erlog: > > > http://www.trapexit.org/forum/viewtopic.php?t=6126&sid=cc8045f60370cac21dc1c8b11839fc47 > > > > Is there any new Erlog release Robert? > > -- > > Regards > > Zabrane > > > > 2010/10/1 Roessner, Silvester > : > >> Mercury could also be interesting for you: > >> > >> http://www.mercury.cs.mu.oz.au/backends.html > >> > >> Rosswart > >> > >> > >> -----Urspr?ngliche Nachricht----- > >> Von: erlang-questions@REDACTED > [mailto:erlang-questions@REDACTED] Im Auftrag von zabrane Mikael > >> Gesendet: Donnerstag, 30. September 2010 17:45 > >> An: David Sergey > >> Cc: Erlang Questions > >> Betreff: Re: [erlang-questions] Implementation of Fact (Belief) > Base > >> > >> Have a look to ERES : > >> > http://groups.google.com/group/erlang-programming/browse_thread/thread/6ea7e7de37274523 > >> > >> -- > >> Regards > >> Zabrane > >> > >> > >> > >> 2010/9/30 David Sergey : > >>> Hello Everybody > >>> > >>> I'm researching ways in Erlang to create Prolog (or any other > logical language like) fact/belif base. Point is - some data > structures should be stored on process for later evaluation. > >>> > >>> Best Regards > >>> David > >>> ________________________________________________________________ > >>> erlang-questions (at) erlang.org mailing list. > >>> See http://www.erlang.org/faq.html > >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>> > >>> > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > >> > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From masklinn@REDACTED Sun Oct 3 23:22:43 2010 From: masklinn@REDACTED (Masklinn) Date: Sun, 3 Oct 2010 23:22:43 +0200 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: <72D32663-8B02-482E-BA64-D11957A823A5@masklinn.net> On 2010-10-03, at 20:40 , Joe Armstrong wrote: > > I don't know why node.js ws desiged - I guess so you could write servers > in js. Yes. It's similar to Python's Twisted or Ruby's EventMachine. From zabrane3@REDACTED Mon Oct 4 01:58:51 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 4 Oct 2010 01:58:51 +0200 Subject: Gracefully timeout during erlang:port_call/3? Message-ID: Hi guys, Is there a way to gracefully timeout (i.e without crashing the port driver) during the synchronous call to "erlang:port_call/3"? The idea here is to try to timeout if the port takes long time to return. Any advice is welcome? -- Regards Zabrane From ok@REDACTED Mon Oct 4 06:24:00 2010 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 4 Oct 2010 17:24:00 +1300 Subject: [erlang-questions] List comprehension filter expressions not throwing exceptions In-Reply-To: <784559543.58001286143250271.JavaMail.root@zimbra> References: <784559543.58001286143250271.JavaMail.root@zimbra> Message-ID: On 4/10/2010, at 11:00 AM, Robert Virding wrote: > The reason for this behaviour is that the compiler specially handles filters which are also valid guard tests. After a generator then all the *directly* following filters which are valid guard tests are handled as guards to the generators and not as "normal" tests. While this is more efficient it does lead to the inconsistencies you discovered in the case of errors in the filter tests. > > It seemed like a good idea at the time. I don't know if it is possible to change this today, if one wanted to. It would be better to be consistent about this. If only bindings Pattern <- Expr and guard tests were allowed, general filters could still be hacked using true <- [Boolean_Expression] and everyone would know exactly where they stood. From ingela@REDACTED Mon Oct 4 09:11:25 2010 From: ingela@REDACTED (Ingela Andin) Date: Mon, 4 Oct 2010 09:11:25 +0200 Subject: SSL in R14B Message-ID: Hi! This mail is to inform you that we decided to change the ssl "verify_fun" option in a non backwards-compatible way compared to R14B. The reason is that we want to differentiate between a CA cert and the peer cert in the call to the "verify_fun" as some older certs may not include extensions that allows the user to make that distinction. If you are using R14A format of the "verify_fun" it will be converted in a compatible way so the problem is only for R14B users. So if you are using this option in R14B please use the latest versions of ssl and public_key applications ssl-4.1 and public_key-0.9 that are available at github. Regards Ingela Erlang/OTP team - Ericsson AB From als@REDACTED Mon Oct 4 11:23:44 2010 From: als@REDACTED (Anthony Shipman) Date: Mon, 4 Oct 2010 19:23:44 +1000 Subject: [erlang-questions] List comprehension filter expressions not throwing exceptions In-Reply-To: References: <784559543.58001286143250271.JavaMail.root@zimbra> Message-ID: <201010042023.45092.als@iinet.net.au> On Mon, 4 Oct 2010 03:24:00 pm Richard O'Keefe wrote: > On 4/10/2010, at 11:00 AM, Robert Virding wrote: > > The reason for this behaviour is that the compiler specially handles > > filters which are also valid guard tests. After a generator then all the > > *directly* following filters which are valid guard tests are handled as > > guards to the generators and not as "normal" tests. While this is more > > efficient it does lead to the inconsistencies you discovered in the case > > of errors in the filter tests. > > > > It seemed like a good idea at the time. I don't know if it is possible to > > change this today, if one wanted to. > > It would be better to be consistent about this. > If only bindings Pattern <- Expr and guard tests were allowed, > general filters could still be hacked using > true <- [Boolean_Expression] > and everyone would know exactly where they stood. > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED I think it would seriously reduce the usefulness if I couldn't have function calls in the filter tests. -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From hm@REDACTED Mon Oct 4 13:43:50 2010 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Mon, 4 Oct 2010 13:43:50 +0200 Subject: [erlang-questions] Application config (.app) without app file In-Reply-To: References: Message-ID: On Sun, Oct 3, 2010 at 8:20 PM, Max Lapshin wrote: > Hi. Is it possible to start application without .app file on disk? It > seems, that loading config from disk is hardcoded in load_application > functionality. You can have diskless Erlang nodes which loads code, .app-files etc. from a remote node. See the man pages for erl_prim_loader and erl_boot_server. /H?kan From djk121@REDACTED Mon Oct 4 13:57:09 2010 From: djk121@REDACTED (Dan Kelley) Date: Mon, 4 Oct 2010 07:57:09 -0400 Subject: high-volume logging via a gen_server Message-ID: I'm relatively new to erlang. I've been working on a simple router which connects to external servers which speak different protocols. When the router gets a packet, it decodes it, figures out the destination channel, encapsulates it in that channel's required format, and sends it along. This all works fine. I have a pretty standard-looking supervision tree setup for the processes which make up the router app. One of the processes is a gen_server which handles the logging for the overall app. It provides a simple interface (log:info(), log:debug(), etc) to the app. When clients use that interface, they end up using gen_server:cast to send the logging process a log line. I've moved as much of the CPU load as I can out to the functional interface so it's paid by the callers and not the logging process. Inside the logging gen_server, I'm not doing anything smarter than using io:format to write to disk: logit(Timestamp, FromPid, RequestId, LevelStr, Category, MessageKey, LogString, State) -> io:format(State#state.log_device, "~s,~s,rock,~s,~p,~s,~s,~s,~s,~s~n", [Timestamp, State#state.unix_pid, State#state.hostname, FromPid, LevelStr, RequestId, Category, MessageKey, LogString]). When I do performance tests of the overall app, it's invariably the logging process which is the limiting factor of the overall system. Usually what happens is that the mailbox for the process accumulates several hundred thousand messages, which causes the size of the VM to bloat until the host starts swapping. (I understand that I could use gen_server:call to make the logging synchronous, but that'd slow down all of the transactions, which I'd like to avoid if I can.) I'm pretty sure that the underlying disk is not saturated - I'm just not getting that many Mb of log data. My guess is that the single erlang process that the logging gen_server is using just can't both read from its mailbox and write to disk fast enough to keep up. So, what are good strategies to cope with a large incoming volume of messages that all need to wind up in the same logfile? Is there a more efficient way to write to disk than the simple io:format() call than I'm using above? What's a good way to parallelize the logging over multiple processes but keep all of the information in one file? Thanks, Dan From steve@REDACTED Mon Oct 4 13:59:59 2010 From: steve@REDACTED (Steven Gravell) Date: Mon, 4 Oct 2010 13:59:59 +0200 Subject: [erlang-questions] Application config (.app) without app file In-Reply-To: References: Message-ID: otherwise you could use the load function directly http://www.erlang.org/doc/man/application.html#load-1 /Steven Gravell http://mokele.co.uk/ 2010/10/4 H?kan Mattsson > On Sun, Oct 3, 2010 at 8:20 PM, Max Lapshin wrote: > > Hi. Is it possible to start application without .app file on disk? It > > seems, that loading config from disk is hardcoded in load_application > > functionality. > > You can have diskless Erlang nodes which loads code, .app-files etc. from a > remote node. See the man pages for erl_prim_loader and erl_boot_server. > > /H?kan > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Mon Oct 4 14:09:45 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 4 Oct 2010 16:09:45 +0400 Subject: [erlang-questions] Application config (.app) without app file In-Reply-To: References: Message-ID: 2010/10/4 Steven Gravell : > otherwise you could use the load function directly > http://www.erlang.org/doc/man/application.html#load-1 > /Steven Gravell > http://mokele.co.uk/ > Excelent! This is what I've missed from documentation. I can't use diskless nodes, because I have protocol that just exchanges terms. From attila.r.nohl@REDACTED Mon Oct 4 14:12:07 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 4 Oct 2010 14:12:07 +0200 Subject: [erlang-questions] high-volume logging via a gen_server In-Reply-To: References: Message-ID: 2010/10/4, Dan Kelley : [...] > So, what are good strategies to cope with a large incoming volume of > messages that all need to wind up in the same logfile? Is there a more > efficient way to write to disk than the simple io:format() call than I'm > using above? What's a good way to parallelize the logging over multiple > processes but keep all of the information in one file? I'm not sure if it's useful, but take a look at the disk_log module in Erlang/OTP. From chandrashekhar.mullaparthi@REDACTED Mon Oct 4 14:31:53 2010 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 4 Oct 2010 13:31:53 +0100 Subject: [erlang-questions] high-volume logging via a gen_server In-Reply-To: References: Message-ID: On 4 October 2010 12:57, Dan Kelley wrote: > So, what are good strategies to cope with a large incoming volume of > messages that all need to wind up in the same logfile? Is there a more > efficient way to write to disk than the simple io:format() call than I'm > using above? What's a good way to parallelize the logging over multiple > processes but keep all of the information in one file? > > What we do is to have one public ETS table per log file. All log entries are written directly to the ETS table by the calling process. Every few seconds, a dedicated logging process scans the ETS table, accumulates them, and dumps them to disk in one write operation. This works very well. cheers Chandru From cbenac@REDACTED Mon Oct 4 15:19:56 2010 From: cbenac@REDACTED (Clara Benac Earle) Date: Mon, 04 Oct 2010 15:19:56 +0200 Subject: New Erlang user group in Madrid In-Reply-To: <201009301239.37137.clist@uah.es> References: <4CA05A48.5030100@fi.upm.es> <201009301239.37137.clist@uah.es> Message-ID: <4CA9D47C.6020405@fi.upm.es> Dear all, We have created a mailing-list for the Erlang user group in Madrid. To subscribe go to: https://babel.ls.fi.upm.es/cgi-bin/mailman/listinfo/madrid-erlang-users We will shortly organize a Madrid Erlounge to get together and discuss Erlang over some beers and tapas. I will be sending the official announcement to the erlang-questions mailing list. Of course people from outside Madrid who want to participate and others who happen to be in Madrid are very welcomed! Cheers Clara PS: Meeting in other places like Tenerife, Marbella/Malaga and Coruna sounds very good too ;-) Maybe we can consider these as possible locations for future and enjoyable Erlounges ;-)? From hasan.veldstra@REDACTED Mon Oct 4 15:28:13 2010 From: hasan.veldstra@REDACTED (Hassy Veldstra) Date: Mon, 4 Oct 2010 14:28:13 +0100 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: <72D32663-8B02-482E-BA64-D11957A823A5@masklinn.net> References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> <72D32663-8B02-482E-BA64-D11957A823A5@masklinn.net> Message-ID: Node.js is to Erlang as Vespa is to BMW's whole line-up. V8 is a good JS engine, and Node.js is an interesting project and it can be a good platform for writing a certain kind of servers when you already know Javascript, but you don't get fault-tolerance, distribution, the amazing OTP libraries, and many other things. Node.js has good momentum (there's a lot of Javascript programmers out there). I think it will end up attracting more people to Erlang. On Sun, Oct 3, 2010 at 10:22 PM, Masklinn wrote: > On 2010-10-03, at 20:40 , Joe Armstrong wrote: >> >> I don't know why node.js ws desiged - I guess so you could write servers >> in js. > Yes. It's similar to Python's Twisted or Ruby's EventMachine. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From daniel.goertzen@REDACTED Mon Oct 4 15:32:41 2010 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Mon, 4 Oct 2010 08:32:41 -0500 Subject: [erlang-questions] dialyzer/typer and exceptions In-Reply-To: <4CA6EF0D.1070506@cs.ntua.gr> References: <4CA6EF0D.1070506@cs.ntua.gr> Message-ID: Thank you Kostis. I studied Haskell some time ago so I guess I was trying to make Erlang match up with that. I need to let go of that and recognize Erlang is different. Dan. On Sat, Oct 2, 2010 at 3:36 AM, Kostis Sagonas wrote: > Daniel Goertzen wrote: > >> While experimenting with dialyzer and typer, I discovered that "catch" >> acts >> as type inference barrier. >> > > Not sure what you mean by "type inference barrier". > > What the type inference algorithm does is to approximate the types (i.e., > set of terms) for which a function will "succeed" (i.e., not raise an > exception). What 'catch' does is it catches exceptions and thus allows > functions to be used with more terms. > > For example, while the function: > > foo(X) -> X + 42. > > will return a value only for numbers and thus has type signature > > -spec foo(number()) -> number(). > > the function: > > bar() -> catch (X + 22). > > will return some value for all terms, not only for numbers, so it has the > type signature: > > -spec bar(any()) -> any(). > > > As an example, gen_server:call/2 uses catch in >> its first line and the resulting inferred type signature is: >> >> -spec call(_,_) -> any(). >> >> Is there a way to get dialyzer/typer to ignore catches and assume the >> non-exception case, or is that opening pandora's box? >> > > The answer to your question is "of course there is such a way", but the > question then is what exactly would this type inference be computing. The > current one tries to model the operational semantics of Erlang, not those of > some ideal(ized) language. > > > Also, are there any other constructs that are known to confound >> dialyzer/typer? >> > > I think confound is a very strong term here, but the answer is that catch > is the only such construct. What also destroys type information is > catch-all clauses/cases. > > Independently of type inference though catch is a terrible construct and > should be avoided in modern Erlang; try-catch is a much better construct for > various reasons and dialyzer handles this much better. > > Kostis > From lcastro@REDACTED Mon Oct 4 15:25:20 2010 From: lcastro@REDACTED (Laura M. Castro) Date: Mon, 4 Oct 2010 15:25:20 +0200 Subject: [erlang-questions] New Erlang user group in Madrid In-Reply-To: <4CA9D47C.6020405@fi.upm.es> References: <4CA05A48.5030100@fi.upm.es> <201009301239.37137.clist@uah.es> <4CA9D47C.6020405@fi.upm.es> Message-ID: On 2010/10/4 Clara Benac Earle wrote: > PS: Meeting in other places like Tenerife, Marbella/Malaga and Coruna sounds > very good too ;-) Maybe we can consider these as possible locations for > future and enjoyable Erlounges ;-)? Please, do! :-) -- Laura M. Castro Computer Science Department University of A Coruna http://www.madsgroup.org/staff/laura/index_en.html From ivan060111ad@REDACTED Mon Oct 4 17:15:46 2010 From: ivan060111ad@REDACTED (Ivan Carmenates =?iso-8859-1?Q?Garc=EDa?=) Date: Mon, 04 Oct 2010 10:15:46 -0500 Subject: hi all Message-ID: hi, I have some kind of stupid question, when you say Erisson AB, that's means that it is a different company than Erisson Sony witch makes cellphones? From mevans@REDACTED Mon Oct 4 16:35:35 2010 From: mevans@REDACTED (Evans, Matthew) Date: Mon, 4 Oct 2010 10:35:35 -0400 Subject: [erlang-questions] high-volume logging via a gen_server In-Reply-To: References: Message-ID: Another option, and something we have implemented, is to extend this logic by having the log statement actually be a fun. For example: logger:funlog(fun() -> {"Some message ~p, with data ~p and ~p", [Message,Data,?MODULE:evaluate_data(Something)]} end). This function can squirrel the fun in an ETS table that can be quickly dumped to disc via a separate management process (e.g. ets:to_dets/2 or ets:tab2file/3). The dump process can, if needed, do the evaluation of the fun, or you can even defer that until some later time by reading the dumped data back into the VM (of course, you need to ensure the process that parses the fun has the same module versions as the process that created the fun). Doing it this way, as a fun, means expensive string operations are deferred until later. In effect giving you a form of lazy evaluation. Matt -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Chandru Sent: Monday, October 04, 2010 8:32 AM To: djk121@REDACTED Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] high-volume logging via a gen_server On 4 October 2010 12:57, Dan Kelley wrote: > So, what are good strategies to cope with a large incoming volume of > messages that all need to wind up in the same logfile? Is there a more > efficient way to write to disk than the simple io:format() call than I'm > using above? What's a good way to parallelize the logging over multiple > processes but keep all of the information in one file? > > What we do is to have one public ETS table per log file. All log entries are written directly to the ETS table by the calling process. Every few seconds, a dedicated logging process scans the ETS table, accumulates them, and dumps them to disk in one write operation. This works very well. cheers Chandru From kaiduanx@REDACTED Mon Oct 4 17:17:54 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Mon, 4 Oct 2010 11:17:54 -0400 Subject: [erlang-questions] high-volume logging via a gen_server In-Reply-To: References: Message-ID: If io:format is the bottleneck, you can move the formatting of log to the caller process, and then dump the formatted string to logger process. Also look the following links, http://blogtrader.net/blog/async_or_sync_log_in http://blogtrader.net/blog/a_case_study_of_scalable With erlang, you do not need too much log to analyze the system or to find bugs, use built-in trace mechanism. If the problem is re-producible, you can do selective log with the help of seq trace. For example, you can log the activities of the system for message from A to B only. This is very powerful to find bugs on a massive loaded live production system. Best regards, /Kaiduan On Mon, Oct 4, 2010 at 10:35 AM, Evans, Matthew wrote: > Another option, and something we have implemented, is to extend this logic by having the log statement actually be a fun. > > For example: > > logger:funlog(fun() -> > ? ? ? ?{"Some message ~p, with data ~p and ~p", > ? ? ? ?[Message,Data,?MODULE:evaluate_data(Something)]} end). > > This function can squirrel the fun in an ETS table that can be quickly dumped to disc via a separate management process (e.g. ets:to_dets/2 or ets:tab2file/3). The dump process can, if needed, do the evaluation of the fun, or you can even defer that until some later time by reading the dumped data back into the VM (of course, you need to ensure the process that parses the fun has the same module versions as the process that created the fun). Doing it this way, as a fun, means expensive string operations are deferred until later. In effect giving you a form of lazy evaluation. > > Matt > > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Chandru > Sent: Monday, October 04, 2010 8:32 AM > To: djk121@REDACTED > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] high-volume logging via a gen_server > > On 4 October 2010 12:57, Dan Kelley wrote: > >> So, what are good strategies to cope with a large incoming volume of >> messages that all need to wind up in the same logfile? ?Is there a more >> efficient way to write to disk than the simple io:format() call than I'm >> using above? ?What's a good way to parallelize the logging over multiple >> processes but keep all of the information in one file? >> >> > What we do is to have one public ETS table per log file. All log entries are > written directly to the ETS table by the calling process. Every few seconds, > a dedicated logging process scans the ETS table, accumulates them, and dumps > them to disk in one write operation. This works very well. > > cheers > Chandru > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kaiduanx@REDACTED Mon Oct 4 17:20:06 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Mon, 4 Oct 2010 11:20:06 -0400 Subject: [erlang-questions] hi all In-Reply-To: References: Message-ID: Sony Ericsson is a joint venture of Ericsson AB and Sony. 2010/10/4 Ivan Carmenates Garc?a : > hi, I have some kind of stupid question, when you say Erisson AB, that's > means that it is a different company than Erisson Sony witch makes > cellphones? > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From hynek@REDACTED Mon Oct 4 17:38:03 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Mon, 4 Oct 2010 17:38:03 +0200 Subject: [erlang-questions] high-volume logging via a gen_server In-Reply-To: References: Message-ID: On Mon, Oct 4, 2010 at 1:57 PM, Dan Kelley wrote: > I'm relatively new to erlang. ?I've been working on a simple router which > connects to external servers which speak different protocols. ?When the > router gets a packet, it decodes it, figures out the destination channel, > encapsulates it in that channel's required format, and sends it along. ?This > all works fine. > > I have a pretty standard-looking supervision tree setup for the processes > which make up the router app. ?One of the processes is a gen_server which > handles the logging for the overall app. ?It provides a simple interface > (log:info(), log:debug(), etc) ?to the app. ?When clients use that > interface, they end up using gen_server:cast to send the logging process a > log line. ?I've moved as much of the CPU load as I can out to the functional > interface so it's paid by the callers and not the logging process. > > Inside the logging gen_server, I'm not doing anything smarter than using > io:format to write to disk: > > logit(Timestamp, FromPid, RequestId, LevelStr, Category, MessageKey, > LogString, State) -> > ? ?io:format(State#state.log_device, > ? ? ?"~s,~s,rock,~s,~p,~s,~s,~s,~s,~s~n", [Timestamp, > ? State#state.unix_pid, > ? State#state.hostname, > ? FromPid, > ? LevelStr, > ? RequestId, > ? Category, > ? MessageKey, > ? LogString]). > > When I do performance tests of the overall app, it's invariably the logging > process which is the limiting factor of the overall system. ?Usually what > happens is that the mailbox for the process accumulates several hundred > thousand messages, which causes the size of the VM to bloat until the host > starts swapping. ?(I understand that I could use gen_server:call to make the > logging synchronous, but that'd slow down all of the transactions, which I'd > like to avoid if I can.) > > I'm pretty sure that the underlying disk is not saturated - I'm just not > getting that many Mb of log data. ?My guess is that the single erlang > process that the logging gen_server is using just can't both read from its > mailbox and write to disk fast enough to keep up. > > So, what are good strategies to cope with a large incoming volume of > messages that all need to wind up in the same logfile? ?Is there a more > efficient way to write to disk than the simple io:format() call than I'm > using above? ?What's a good way to parallelize the logging over multiple > processes but keep all of the information in one file? > > Thanks, > > Dan > The issue is that io:format/1,2,3 is synchronous call. Once message queue starts grow, this call becomes slower and slower and things goes wrong. It is spiral of death. Solution is change your code to asynchronous IO. See http://erlang.org/doc/apps/stdlib/io_protocol.html for more info. It can be this simple: logit(Timestamp, FromPid, RequestId, LevelStr, Category, MessageKey, LogString, State) -> State#state.log_device ! {io_request, self(), log, {put_chars, io_lib, format, [ "~s,~s,rock,~s,~p,~s,~s,~s,~s,~s~n", [Timestamp, State#state.unix_pid, State#state.hostname, FromPid, LevelStr, RequestId, Category, MessageKey, LogString]]}}. or logit(Timestamp, FromPid, RequestId, LevelStr, Category, MessageKey, LogString, State) -> State#state.log_device ! {io_request, self(), log, {put_chars, io_lib:format( "~s,~s,rock,~s,~p,~s,~s,~s,~s,~s~n", [Timestamp, State#state.unix_pid, State#state.hostname, FromPid, LevelStr, RequestId, Category, MessageKey, LogString])}}. And not forget catch {io_reply, log, Reply} in handle_info. But if you want avoid as much bottleneck as possible you can of course send this message from processes so then all values what you need centrally read from ets optimized for read locks. then just perform formating in caller and send it to io_server read from this ets and for better performance let some garbage process receive {io_reply, log, Reply}. (Don't forget that you can't use raw io_server in this mode if I remember right.) -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From rtrlists@REDACTED Mon Oct 4 18:00:57 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 4 Oct 2010 17:00:57 +0100 Subject: [erlang-questions] high-volume logging via a gen_server In-Reply-To: References: Message-ID: If you can move from a textual log file to a binary one and move the formatting from the logging point to the viewing part, then have a look at how error_logger and rb (report browser) do things. Logging in binary is pretty fast. I have found this provides a neat way of handling different logging levels without needing to restart things. I always log everything at debug level, and only upon viewing the log, do I filter out the correct level of detail to show. Disk space is cheap, right? (And yes, you can still get bogged down this way if there's too much traffic. It becomes an exercise in finding the right lowest level of log detail.) Robby From mihai@REDACTED Mon Oct 4 21:42:17 2010 From: mihai@REDACTED (Mihai Balea) Date: Mon, 4 Oct 2010 22:42:17 +0300 Subject: [erlang-questions] high-volume logging via a gen_server In-Reply-To: References: Message-ID: <2A8C7AB5-9F26-4D27-AEE8-029952B2728D@hates.ms> On Oct 4, 2010, at 2:57 PM, Dan Kelley wrote: > > So, what are good strategies to cope with a large incoming volume of > messages that all need to wind up in the same logfile? Is there a more > efficient way to write to disk than the simple io:format() call than I'm > using above? What's a good way to parallelize the logging over multiple > processes but keep all of the information in one file? Logging is a typical producer - consumer issue. When your system produces more data than your logger can consume, you basically have a number of choices: 1. Log less data: are you sure you need all that stuff in your log file at all times? Maybe you want a more flexible logging system that can enable and disable various log areas and levels on demand. 2. Speed up the logger: look into using buffered disk ops (the "file" module). I suspect unbuffered disk access is where you lose most of your performance. Also, look into using error_logger with the log_mf_h handler, it writes stuff in binary format and it is very fast. 3. Use synchronous logging. Slowing down the entire system a little is better than overloading the logger and having the VM crash due to out of memory conditions. 4. Decide if it is acceptable to lose log items and design you logger to drop stuff if it gets too busy. 5. Design your logger to spread the write load to several workers that log into separate files. If you add a timestamp or some other sort of index to log items you can merge them later into one temporally consistent file. This tends to get complicated though, I would try the other things first. Hope at least some of the above will be helpful :) Cheers, Mihai From erlang@REDACTED Mon Oct 4 22:08:19 2010 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 4 Oct 2010 22:08:19 +0200 Subject: [erlang-questions] high-volume logging via a gen_server In-Reply-To: References: Message-ID: disk_log was designed to do this > erl -man disk_log NAME disk_log - A disk based term logging facility DESCRIPTION disk_log is a disk based term logger which makes it possible to effi- ciently log items on files. Two types of logs are supported, halt logs and wrap logs .... disk_log is *unreasonably* fast :-) /Joe On Mon, Oct 4, 2010 at 1:57 PM, Dan Kelley wrote: > I'm relatively new to erlang. ?I've been working on a simple router which > connects to external servers which speak different protocols. ?When the > router gets a packet, it decodes it, figures out the destination channel, > encapsulates it in that channel's required format, and sends it along. ?This > all works fine. > > I have a pretty standard-looking supervision tree setup for the processes > which make up the router app. ?One of the processes is a gen_server which > handles the logging for the overall app. ?It provides a simple interface > (log:info(), log:debug(), etc) ?to the app. ?When clients use that > interface, they end up using gen_server:cast to send the logging process a > log line. ?I've moved as much of the CPU load as I can out to the functional > interface so it's paid by the callers and not the logging process. > > Inside the logging gen_server, I'm not doing anything smarter than using > io:format to write to disk: > > logit(Timestamp, FromPid, RequestId, LevelStr, Category, MessageKey, > LogString, State) -> > ? ?io:format(State#state.log_device, > ? ? ?"~s,~s,rock,~s,~p,~s,~s,~s,~s,~s~n", [Timestamp, > ? State#state.unix_pid, > ? State#state.hostname, > ? FromPid, > ? LevelStr, > ? RequestId, > ? Category, > ? MessageKey, > ? LogString]). > > When I do performance tests of the overall app, it's invariably the logging > process which is the limiting factor of the overall system. ?Usually what > happens is that the mailbox for the process accumulates several hundred > thousand messages, which causes the size of the VM to bloat until the host > starts swapping. ?(I understand that I could use gen_server:call to make the > logging synchronous, but that'd slow down all of the transactions, which I'd > like to avoid if I can.) > > I'm pretty sure that the underlying disk is not saturated - I'm just not > getting that many Mb of log data. ?My guess is that the single erlang > process that the logging gen_server is using just can't both read from its > mailbox and write to disk fast enough to keep up. > > So, what are good strategies to cope with a large incoming volume of > messages that all need to wind up in the same logfile? ?Is there a more > efficient way to write to disk than the simple io:format() call than I'm > using above? ?What's a good way to parallelize the logging over multiple > processes but keep all of the information in one file? > > Thanks, > > Dan > From juanjo@REDACTED Mon Oct 4 23:17:20 2010 From: juanjo@REDACTED (Juan Jose Comellas) Date: Mon, 4 Oct 2010 18:17:20 -0300 Subject: Seminario de Cloud Computing de CESSI Message-ID: Este mi?rcoles 6 de septiembre voy a estar exponiendo en el seminario de Cloud Computing de la CESSI que se llevar? a cabo en el Mini Auditorio de la UADE (Lima 717, Capital Federal). El t?tulo de la charla es "Servicios de Internet de Pr?xima Generaci?n utilizando Programaci?n Funcional con Erlang". Si alguno est? interesado en venir, el evento es gratuito y empieza a las 9:20 hs. Para inscribirse tienen que ir al sitio de la CESSI ( http://www.cessi.org.ar/). Saludos, Juanjo. From juanjo@REDACTED Mon Oct 4 23:18:08 2010 From: juanjo@REDACTED (Juan Jose Comellas) Date: Mon, 4 Oct 2010 18:18:08 -0300 Subject: Seminario de Cloud Computing de CESSI In-Reply-To: References: Message-ID: Sorry, wrong list. :( 2010/10/4 Juan Jose Comellas > Este mi?rcoles 6 de septiembre voy a estar exponiendo en el seminario de > Cloud Computing de la CESSI que se llevar? a cabo en el Mini Auditorio de la > UADE (Lima 717, Capital Federal). El t?tulo de la charla es "Servicios de > Internet de Pr?xima Generaci?n utilizando Programaci?n Funcional con > Erlang". > > Si alguno est? interesado en venir, el evento es gratuito y empieza a las > 9:20 hs. Para inscribirse tienen que ir al sitio de la CESSI ( > http://www.cessi.org.ar/). > > Saludos, Juanjo. > From ok@REDACTED Mon Oct 4 23:52:52 2010 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 5 Oct 2010 10:52:52 +1300 Subject: [erlang-questions] List comprehension filter expressions not throwing exceptions In-Reply-To: <201010042023.45092.als@iinet.net.au> References: <784559543.58001286143250271.JavaMail.root@zimbra> <201010042023.45092.als@iinet.net.au> Message-ID: <71C8C798-8F98-4A46-B268-ADD054DA1B59@cs.otago.ac.nz> On 4/10/2010, at 10:23 PM, Anthony Shipman wrote: > On Mon, 4 Oct 2010 03:24:00 pm Richard O'Keefe wrote: >> >> It would be better to be consistent about this. >> If only bindings Pattern <- Expr and guard tests were allowed, >> general filters could still be hacked using >> true <- [Boolean_Expression] >> and everyone would know exactly where they stood. >> > > I think it would seriously reduce the usefulness if I couldn't have function > calls in the filter tests. The text you quoted (repeated above) *proved* that it would not by showing a systematic way to eliminate such ugliness. I've just checked a bunch of other people's Erlang ocde. Functions that I saw used in list comprehensions include - whereis/1 -- could be allowed in guards - member/2 -- could be allowed in guards - keymember/3 -- could be allowed in guards where I'm happy to see them in list comprehensions because I'd be happy to see them in guards (yes, member/2 takes O(N) time, but so after all does length/1), plus - two that should have been eliminated by better data structure design - another that should have been a better algorithm - a case where the code would have been half as long had the function been inlined, and more efficient as well - a case of the form [X || X <- L, X /= f(L), ...] which takes quadratic time in its present form, but would have taken linear time had it been written FL = f(L), [X || X <- L, X /= FL, ...] and is arguably wrong even then - several things that could be handled easily by allowing (_;_) in list comprehension guards and certainly can be handled with an inlined orelse - two examples that at first I thought were good ones, but on closer inspection could have been handled by better data structure design ... This experience has forced me to two conclusions: (1) plain function calls in list comprehension filters are generally a bad code smell (2) nested functions in Erlang are also a bad code smell more often than not. From fritchie@REDACTED Tue Oct 5 00:36:14 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 04 Oct 2010 17:36:14 -0500 Subject: [erlang-questions] valid use for gen_server? In-Reply-To: Message of "Thu, 30 Sep 2010 08:38:14 PDT." <253554.95925.qm@web112607.mail.gq1.yahoo.com> Message-ID: <26193.1286231774@snookles.snookles.com> Pablo Platt wrote: pp> I see two options to implement the above, gen_event or custom event pp> module. What bothers me in gen_event is that it executes handlers pp> in its own context. If events are rare that shouldn't be a problem pp> but if each IM message will fire several events wouldn't this flood pp> the gen_event server? Using gen_event, if you want to apply backpressure, you have the option of sending the events synchronously. pp> I can use a custom event module that saves events and handlers in pp> ETS table and call functions dynamically with Module:Function(Args). pp> This also doesn't feel right because each IM message will require pp> several ETS lookups and dynamic functions calls which according to pp> the efficiency guide are are 6 times slower than normal calls. Er, sorry, but this sounds like premature worrying. If handling an event requires N reductions and therefore N units of time(*), then the cost of handling the same event using the dynamic functions calls would be (N - several) + (6 * several) ... which is likely to be really, *really* close to N. -Scott (*) Which isn't exactly true, but it's a good approximation. From freza@REDACTED Tue Oct 5 01:48:58 2010 From: freza@REDACTED (Jachym Holecek) Date: Tue, 5 Oct 2010 00:48:58 +0100 Subject: [erlang-questions] high-volume logging via a gen_server In-Reply-To: References: Message-ID: <20101004234858.GA2103@hanele> # Chandru 2010-10-04: > On 4 October 2010 12:57, Dan Kelley wrote: > > > So, what are good strategies to cope with a large incoming volume of > > messages that all need to wind up in the same logfile? Is there a more > > efficient way to write to disk than the simple io:format() call than I'm > > using above? What's a good way to parallelize the logging over multiple > > processes but keep all of the information in one file? > > > What we do is to have one public ETS table per log file. All log entries are > written directly to the ETS table by the calling process. Every few seconds, > a dedicated logging process scans the ETS table, accumulates them, and dumps > them to disk in one write operation. This works very well. Actually, this appears to be somewhat degenarate use case for ETS, under extreme loads at least. Passing log messages (ASCII text formatted in the context of calling process) to per-log gen_server in a synchronous call is at least as fast as the ETS approach, but possibly faster. Key point is to write the gen_server so that it avoids any unnecessary activity (especially: memory allocations & any kind of log data processing). Buffering can be achieved down at io device level with delayed_write option. Sorry to be a little vague, I didn't quite get around to analysing this in more detail (like: understand ETS locking protocol & run more measurements). I'm however sure that after I've changed a (proprietary) networked logging server from ETS-based approach to the one indicated above, write throughput went up from ~10-20MB/s to 100MB/s (ramdisk) or ~60-70MB/s (real disk). There were other changes along the way, but IIRC I've tried most of them with ETS first and it didn't help. Regards, -- Jachym From pablo.platt@REDACTED Tue Oct 5 01:02:31 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 4 Oct 2010 16:02:31 -0700 (PDT) Subject: [erlang-questions] valid use for gen_server? In-Reply-To: <26193.1286231774@snookles.snookles.com> References: <26193.1286231774@snookles.snookles.com> Message-ID: <817402.72396.qm@web112609.mail.gq1.yahoo.com> >Pablo Platt wrote: >pp> I see two options to implement the above, gen_event or custom event >pp> module. What bothers me in gen_event is that it executes handlers >pp> in its own context. If events are rare that shouldn't be a problem >pp> but if each IM message will fire several events wouldn't this flood >pp> the gen_event server? >Using gen_event, if you want to apply backpressure, you have the option >of sending the events synchronously. >pp> I can use a custom event module that saves events and handlers in >pp> ETS table and call functions dynamically with Module:Function(Args). >pp> This also doesn't feel right because each IM message will require >pp> several ETS lookups and dynamic functions calls which according to >pp> the efficiency guide are are 6 times slower than normal calls. >Er, sorry, but this sounds like premature worrying. If handling an >event requires N reductions and therefore N units of time(*), then the >cost of handling the same event using the dynamic functions calls would >be (N - several) + (6 * several) ... which is likely to be really, *>really* close to N. >-Scott >(*) Which isn't exactly true, but it's a good approximation. If I have ten events per message I need to make ten ETS calls and than add the dynamic function calls to it. On a busy system, can't this add up? From tony.arcieri@REDACTED Tue Oct 5 06:45:32 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Mon, 4 Oct 2010 22:45:32 -0600 Subject: If you could rename ports, what would you call them? Message-ID: I'm working on Reia, a Ruby-like language for the Erlang VM. Reia is an everything-is-an-object language, and the Erlang core types are all wrapped as objects. As part of this process I've wrapped ports. However, in my opinion port is a confusing name, particularly when dealing with a TCP port (in the Erlang sense) which represents a connection to a particular TCP port (in the TCP sense). Because of this sort of ambiguity, I'd like to rename ports. But what's a good replacement name? Some of the ones I've considered: SysPort, IOPort, Channel. What would you call ports if you had the opportunity to rename them? -- Tony Arcieri Medioh! A Kudelski Brand From g9414002.pccu.edu.tw@REDACTED Tue Oct 5 07:08:24 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Tue, 5 Oct 2010 13:08:24 +0800 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: Message-ID: Hi, Channel? Tunnel? Or ship? cheers, --- Yh.H. On Tue, Oct 5, 2010 at 12:45 PM, Tony Arcieri wrote: > I'm working on Reia, a Ruby-like language for the Erlang VM. Reia is an > everything-is-an-object language, and the Erlang core types are all wrapped > as objects. > > As part of this process I've wrapped ports. However, in my opinion port is > a > confusing name, particularly when dealing with a TCP port (in the Erlang > sense) which represents a connection to a particular TCP port (in the TCP > sense). > > Because of this sort of ambiguity, I'd like to rename ports. But what's a > good replacement name? Some of the ones I've considered: SysPort, IOPort, > Channel. > > What would you call ports if you had the opportunity to rename them? > > -- > Tony Arcieri > Medioh! A Kudelski Brand > From masklinn@REDACTED Tue Oct 5 07:17:26 2010 From: masklinn@REDACTED (Masklinn) Date: Tue, 5 Oct 2010 07:17:26 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: Message-ID: <7A26632E-02E0-4A2D-856E-32DFCA36142D@masklinn.net> On 2010-10-05, at 06:45 , Tony Arcieri wrote: > I'm working on Reia, a Ruby-like language for the Erlang VM. Reia is an > everything-is-an-object language, and the Erlang core types are all wrapped > as objects. > > As part of this process I've wrapped ports. However, in my opinion port is a > confusing name, particularly when dealing with a TCP port (in the Erlang > sense) which represents a connection to a particular TCP port (in the TCP > sense). > > Because of this sort of ambiguity, I'd like to rename ports. But what's a > good replacement name? Some of the ones I've considered: SysPort, IOPort, > Channel. > > What would you call ports if you had the opportunity to rename them? Foreign Process[ Interface]? From dgud@REDACTED Tue Oct 5 09:50:14 2010 From: dgud@REDACTED (Dan Gudmundsson) Date: Tue, 5 Oct 2010 09:50:14 +0200 Subject: [erlang-questions] "wx:demo()." / static example / possible bug on Ubuntu 10.04 / Erlang R13B04/R14B In-Reply-To: References: Message-ID: On Sat, Oct 2, 2010 at 11:31 PM, Boris M?hmer wrote: > I just noticed on a Windows Server 2008 R2 system (actually running > within a KVM virtual machine), that the "wx:demo()."/static-Example > (using the R14B binary) gives a better result than on my > (self-compiled/installed) Ubuntu 10.04 systems. > > On my windows systems the "wxStaticText" alignment is correct (left / > center / right), but on my Ubuntu 10.04 system, all three text are on > the left. This is true for Erlang R13B04 and R14B. Known issue see http://trac.wxwidgets.org/ticket/12539 > > Has anyone else experienced this behaviour? > > > I noticed another interesting different behaviour between Windows and > Ubuntu Erlang (R14B, possible other versions as well): "wx:demo()." > doesn't return any warnings or errors on a windows box at all... on > Ubuntu there are several Gtk-Warnings and an "WX INTERNAL ERROR: Failed to send return or event msg"... The warnings/errors on Ubuntu > aren't around all the time; they also depend on the examples started. > Wx internal error, means that the driver tries to send a message to a non existing process, the demo could probably be improved there. That ERROR should be a warning, I don't want remove the message completely since it can help the developer. Gtk warnings, I can't do much about them try starting firefox from the shell most gui programs on linux have but normally they go to /dev/null, they can depend on half implemented gtk/gnome themes and wxWidgets internal problems/hacks. /Dan > > ?- boris > From karol.skocik@REDACTED Tue Oct 5 10:16:33 2010 From: karol.skocik@REDACTED (karol skocik) Date: Tue, 5 Oct 2010 10:16:33 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: Message-ID: Pipe On Tue, Oct 5, 2010 at 6:45 AM, Tony Arcieri wrote: > I'm working on Reia, a Ruby-like language for the Erlang VM. Reia is an > everything-is-an-object language, and the Erlang core types are all wrapped > as objects. > > As part of this process I've wrapped ports. However, in my opinion port is a > confusing name, particularly when dealing with a TCP port (in the Erlang > sense) which represents a connection to a particular TCP port (in the TCP > sense). > > Because of this sort of ambiguity, I'd like to rename ports. But what's a > good replacement name? Some of the ones I've considered: SysPort, IOPort, > Channel. > > What would you call ports if you had the opportunity to rename them? > > -- > Tony Arcieri > Medioh! A Kudelski Brand > From erlang@REDACTED Tue Oct 5 10:30:00 2010 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 5 Oct 2010 10:30:00 +0200 Subject: bug in escript and a question Message-ID: This escript fails +------------------------------------- -record(b,{this=that}). main(_) -> io:format("B=~p~n",[#b{}]), halt(1). +----------------------------------- The code is between the lines marked +------ This is what happens $ escript b 12 b:3: record b undefined escript: There were compilation errors. If you add a single blank or comment before the record declartion then it works +------------------------------------- %% comment -record(b,{this=that}). main(_) -> io:format("B=~p~n",[#b{}]), halt(1). +----------------------------------- $ escript b 12 B={b,that} joe@REDACTED:~/work/c_compiler/bic_current/src$ Now it works Also -include(..) just doesn't work but -include_lib(...) does. I cannot think why this should be so - is there any good reason for this? I can't think of one. Cheers /Joe From erlang@REDACTED Tue Oct 5 10:34:35 2010 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 5 Oct 2010 10:34:35 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: Message-ID: Pids There shouldn't be ports in the language at all (sorry) - ports should in all circumstances behave and be indistinguishable from Pids. open_port(...) should return a Pid and not a port. We should reduce the number of concepts in the langauge not increase them - all in the name of conceptual integrity. /Joe On Tue, Oct 5, 2010 at 6:45 AM, Tony Arcieri wrote: > I'm working on Reia, a Ruby-like language for the Erlang VM. Reia is an > everything-is-an-object language, and the Erlang core types are all wrapped > as objects. > > As part of this process I've wrapped ports. However, in my opinion port is a > confusing name, particularly when dealing with a TCP port (in the Erlang > sense) which represents a connection to a particular TCP port (in the TCP > sense). > > Because of this sort of ambiguity, I'd like to rename ports. But what's a > good replacement name? Some of the ones I've considered: SysPort, IOPort, > Channel. > > What would you call ports if you had the opportunity to rename them? > > -- > Tony Arcieri > Medioh! A Kudelski Brand > From zabrane3@REDACTED Tue Oct 5 10:38:25 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Tue, 5 Oct 2010 10:38:25 +0200 Subject: [erlang-questions] bug in escript and a question In-Reply-To: References: Message-ID: Hi Joe, The "-include(...)" works fine like this: !/usr/bin/env escript %% -*- erlang -*- %%! -pz ../mycode ebin -include_lib("mycode/include/my.hrl"). Regards Zabrane 2010/10/5 Joe Armstrong : > This escript fails > > > ?+------------------------------------- > ? ?-record(b,{this=that}). > > ? ?main(_) -> > ? ? ? io:format("B=~p~n",[#b{}]), > ? ? ? halt(1). > ?+----------------------------------- > > The code is between the lines marked +------ > > This is what happens > > $ escript b 12 > b:3: record b undefined > escript: There were compilation errors. > > If you add a single blank or comment before the record declartion then it works > > ?+------------------------------------- > ? %% comment > ? ?-record(b,{this=that}). > > ? ?main(_) -> > ? ? ? io:format("B=~p~n",[#b{}]), > ? ? ? halt(1). > ?+----------------------------------- > $ escript b 12 > B={b,that} > joe@REDACTED:~/work/c_compiler/bic_current/src$ > > Now it works > > Also -include(..) just doesn't work but -include_lib(...) does. > I cannot think why this should be so - is there any good reason for this? > I can't think of one. > > Cheers > > /Joe > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Regards Zabrane From ds.erl@REDACTED Tue Oct 5 10:33:35 2010 From: ds.erl@REDACTED (DS) Date: Tue, 5 Oct 2010 10:33:35 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: Message-ID: <53024700-88F4-4C55-A1FE-76D43EE64AA3@sol42.com> On 5 Oct 2010, at 6:45, Tony Arcieri wrote: > What would you call ports if you had the opportunity to rename them? Minions :) From hsychla@REDACTED Tue Oct 5 11:02:49 2010 From: hsychla@REDACTED (Helge Sychla) Date: Tue, 5 Oct 2010 11:02:49 +0200 (CEST) Subject: ets documentation | reasons for badarag In-Reply-To: <15479836.1955.1286268817316.JavaMail.root@office.tpip.net> Message-ID: <2049920.1961.1286269369951.JavaMail.root@office.tpip.net> Hello, I'm new here so please excuse me if I state the obvious or am missing something. I'm working with Erlang for a few months now and had a rather frustrating problem yesterday: I was working with ets tables, several processes and ets:update_element/3. ets:update_element threw a badarg exception so I looked it up in the documentation and found "The function will fail with reason badarg if: * the table is not of type set or ordered_set, * Pos is less than 1 or greater than the object arity, or, * the element to update is also the key" so I checked my code for one of these three reasons but couldn't find any. Long story short the reason for the badarg exception was that my process did not have the necessary rights to write to the tables I wanted to edit. There is no mention on the whole ets page that a badarg exception is thrown without any further explanation if the process does not have the right rights. I think that should be added. While I will surely not make that mistake again, other beginners might and will be just as frustrated as me yesterday. Cheers Helge -- Dipl.-Ing. (FH) Helge Sychla From masklinn@REDACTED Tue Oct 5 11:02:33 2010 From: masklinn@REDACTED (Masklinn) Date: Tue, 5 Oct 2010 11:02:33 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: Message-ID: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> On 2010-10-05, at 10:34 , Joe Armstrong wrote: > Pids > > There shouldn't be ports in the language at all (sorry) - ports should > in all circumstances > behave and be indistinguishable from Pids. open_port(...) should > return a Pid and not a port. > > We should reduce the number of concepts in the langauge not increase > them - all in the > name of conceptual integrity. > > /Joe But they *are* two different concepts. Even if at the usage level they behave the same way from the POV of an external process, they're not created the same way (``open_port`` vs ``spawn`` and a bunch of control functions dumped into the ``erlang`` module), and you also need to distinguish them when *implementing* them. And I'm guessing that's where Tony is looking for a better name. If down the line ports and pids are better merged (in both external interface and implementation details) then maybe "ports" as a name/concept can disappear, but I don't think that's the case at the moment, you still need to refer to them as entities separate from pids when talking about implementing things in erlang. From als@REDACTED Tue Oct 5 11:23:41 2010 From: als@REDACTED (Anthony Shipman) Date: Tue, 5 Oct 2010 19:23:41 +1000 Subject: [erlang-questions] high-volume logging via a gen_server In-Reply-To: <2A8C7AB5-9F26-4D27-AEE8-029952B2728D@hates.ms> References: <2A8C7AB5-9F26-4D27-AEE8-029952B2728D@hates.ms> Message-ID: <201010052023.41849.als@iinet.net.au> On Tue, 5 Oct 2010 06:42:17 am Mihai Balea wrote: > On Oct 4, 2010, at 2:57 PM, Dan Kelley wrote: > > So, what are good strategies to cope with a large incoming volume of > > messages that all need to wind up in the same logfile? Is there a more > > efficient way to write to disk than the simple io:format() call than I'm > > using above? What's a good way to parallelize the logging over multiple > > processes but keep all of the information in one file? > You can have a buffer process ahead of the logger. The buffer process just has to accept and store log requests. A separate write process can pull them from the buffer (using an async protocol). If the buffer gets too full then you can have various policies such as discard oldest, youngest, low priority messages etc. -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From john.hughes@REDACTED Tue Oct 5 12:46:35 2010 From: john.hughes@REDACTED (John Hughes) Date: Tue, 5 Oct 2010 12:46:35 +0200 Subject: Talking about Erlang and QuickCheck at JAOO In-Reply-To: References: Message-ID: <0DD8077E6D56450F981415AA7AE73126@JohnsTablet2009> I'll be talking about testing dets and ejabberd at JAOO tomorrow--any Erlangers here, welcome to my session! (QuickCheck found 5 race conditions in dets over the past couple of weeks--see erlang-bugs for details--and I'll be explaining how). John Hughes From lars@REDACTED Tue Oct 5 13:39:53 2010 From: lars@REDACTED (Lars Thorsen) Date: Tue, 05 Oct 2010 13:39:53 +0200 Subject: [erlang-questions] bug in escript and a question In-Reply-To: References: Message-ID: <4CAB0E89.7060303@erix.ericsson.se> You can't have Erlang code on the first line in an escript, here are some lines from the man page ;). -------------------------------- The header of the Erlang script in the example differs from a normal Erlang module. The first line is intended to be the interpreter line, which invokes escript. However if you invoke the escript like this $ escript factorial 5 the contents of the first line does not matter, but it cannot contain Erlang code as it will be ignored. The second line in the example, contains an optional directive to the Emacs editor which causes it to enter the major mode for editing Erlang source files. If the directive is present it must be located on the second line. On the third line (or second line depending on the presence of the Emacs directive), it is possible to give arguments to the emulator, such as %%! -smp enable -sname factorial -mnesia debug verbose Such an argument line must start with %%! and the rest of the line will interpreted as arguments to the emulator. ------------------------------- ///Lars Joe Armstrong wrote: > This escript fails > > > +------------------------------------- > -record(b,{this=that}). > > main(_) -> > io:format("B=~p~n",[#b{}]), > halt(1). > +----------------------------------- > > The code is between the lines marked +------ > > This is what happens > > $ escript b 12 > b:3: record b undefined > escript: There were compilation errors. > > If you add a single blank or comment before the record declartion then it works > > +------------------------------------- > %% comment > -record(b,{this=that}). > > main(_) -> > io:format("B=~p~n",[#b{}]), > halt(1). > +----------------------------------- > $ escript b 12 > B={b,that} > joe@REDACTED:~/work/c_compiler/bic_current/src$ > > Now it works > > Also -include(..) just doesn't work but -include_lib(...) does. > I cannot think why this should be so - is there any good reason for this? > I can't think of one. > > Cheers > > /Joe > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From erlang@REDACTED Tue Oct 5 15:27:13 2010 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 5 Oct 2010 15:27:13 +0200 Subject: [erlang-questions] bug in escript and a question In-Reply-To: References: Message-ID: On Tue, Oct 5, 2010 at 10:38 AM, zabrane Mikael wrote: > Hi Joe, > > The "-include(...)" works fine like this: No no no ... I think you meant to say "-include(...) does not work, but there is a work-around involving "-include_lib(...)" which does work .. I just wondered why - it's a bit of a bother creating an extra directory for no apparent reason /Joe > > !/usr/bin/env escript > %% -*- erlang -*- > %%! -pz ../mycode ebin > > -include_lib("mycode/include/my.hrl"). > > Regards > Zabrane > > 2010/10/5 Joe Armstrong : >> This escript fails >> >> >> ?+------------------------------------- >> ? ?-record(b,{this=that}). >> >> ? ?main(_) -> >> ? ? ? io:format("B=~p~n",[#b{}]), >> ? ? ? halt(1). >> ?+----------------------------------- >> >> The code is between the lines marked +------ >> >> This is what happens >> >> $ escript b 12 >> b:3: record b undefined >> escript: There were compilation errors. >> >> If you add a single blank or comment before the record declartion then it works >> >> ?+------------------------------------- >> ? %% comment >> ? ?-record(b,{this=that}). >> >> ? ?main(_) -> >> ? ? ? io:format("B=~p~n",[#b{}]), >> ? ? ? halt(1). >> ?+----------------------------------- >> $ escript b 12 >> B={b,that} >> joe@REDACTED:~/work/c_compiler/bic_current/src$ >> >> Now it works >> >> Also -include(..) just doesn't work but -include_lib(...) does. >> I cannot think why this should be so - is there any good reason for this? >> I can't think of one. >> >> Cheers >> >> /Joe >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > > -- > Regards > Zabrane > From attila.r.nohl@REDACTED Tue Oct 5 15:30:57 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 5 Oct 2010 15:30:57 +0200 Subject: [erlang-questions] ets documentation | reasons for badarag In-Reply-To: <2049920.1961.1286269369951.JavaMail.root@office.tpip.net> References: <15479836.1955.1286268817316.JavaMail.root@office.tpip.net> <2049920.1961.1286269369951.JavaMail.root@office.tpip.net> Message-ID: 2010/10/5, Helge Sychla : > Hello, > > I'm new here so please excuse me if I state the obvious or am missing > something. > > I'm working with Erlang for a few months now and had a rather frustrating > problem yesterday: > I was working with ets tables, several processes and ets:update_element/3. > ets:update_element threw a badarg exception so I looked it up in the > documentation and found > > "The function will fail with reason badarg if: > > * the table is not of type set or ordered_set, > * Pos is less than 1 or greater than the object arity, or, > * the element to update is also the key" > > so I checked my code for one of these three reasons but couldn't find any. As far as I've noticed, ets will also return badarg if the table id is not valid (i.e. there's no table with that id). From erlang@REDACTED Tue Oct 5 15:33:44 2010 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 5 Oct 2010 15:33:44 +0200 Subject: [erlang-questions] bug in escript and a question In-Reply-To: <4CAB0E89.7060303@erix.ericsson.se> References: <4CAB0E89.7060303@erix.ericsson.se> Message-ID: On Tue, Oct 5, 2010 at 1:39 PM, Lars Thorsen wrote: > > You can't have Erlang code on the first line in an escript, here are some > lines from the man page ;). > > -------------------------------- > The header of the Erlang script in the example differs from a normal Erlang > module. The first line is intended to be the interpreter line, which invokes > escript. However if you invoke the escript like this > > $ escript factorial 5 > > the contents of the first line does not matter, but it cannot contain Erlang > code as it will be ignored. Goodness gracious me - you are indeed correct - My flabber has been gasted - this surely violates the principle of least astonishment, I need a nip of strong smelling-salts to recover. I thing it would be splendiferous if one could add a line of code in the first line of an escript file and if the documentation were to be revised to remove this rather bizaare restriction. /Joe > > The second line in the example, contains an optional directive to the Emacs > editor which causes it to enter the major mode for editing Erlang source > files. If the directive is present it must be located on the second line. > > On the third line (or second line depending on the presence of the Emacs > directive), it is possible to give arguments to the emulator, such as > > %%! -smp enable -sname factorial -mnesia debug verbose > > Such an argument line must start with %%! and the rest of the line will > interpreted as arguments to the emulator. > ------------------------------- > > > ///Lars > > > Joe Armstrong wrote: >> >> This escript fails >> >> >> ?+------------------------------------- >> ? ?-record(b,{this=that}). >> >> ? ?main(_) -> >> ? ? ? io:format("B=~p~n",[#b{}]), >> ? ? ? halt(1). >> ?+----------------------------------- >> >> The code is between the lines marked +------ >> >> This is what happens >> >> $ escript b 12 >> b:3: record b undefined >> escript: There were compilation errors. >> >> If you add a single blank or comment before the record declartion then it >> works >> >> ?+------------------------------------- >> ? %% comment >> ? ?-record(b,{this=that}). >> >> ? ?main(_) -> >> ? ? ? io:format("B=~p~n",[#b{}]), >> ? ? ? halt(1). >> ?+----------------------------------- >> $ escript b 12 >> B={b,that} >> joe@REDACTED:~/work/c_compiler/bic_current/src$ >> >> Now it works >> >> Also -include(..) just doesn't work but -include_lib(...) does. >> I cannot think why this should be so - is there any good reason for this? >> I can't think of one. >> >> Cheers >> >> /Joe >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > From hm@REDACTED Tue Oct 5 15:50:42 2010 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Tue, 5 Oct 2010 15:50:42 +0200 Subject: [erlang-questions] bug in escript and a question In-Reply-To: References: <4CAB0E89.7060303@erix.ericsson.se> Message-ID: > On Tue, Oct 5, 2010 at 1:39 PM, Lars Thorsen wrote: >> >> You can't have Erlang code on the first line in an escript, here are some >> lines from the man page ;). >> >> -------------------------------- >> The header of the Erlang script in the example differs from a normal Erlang >> module. The first line is intended to be the interpreter line, which invokes >> escript. However if you invoke the escript like this >> >> $ escript factorial 5 >> >> the contents of the first line does not matter, but it cannot contain Erlang >> code as it will be ignored. > > Goodness gracious me - you are indeed correct - > > My flabber has been gasted - this surely violates the principle of > least astonishment, I need a nip of strong smelling-salts to recover. > > I thing it would be splendiferous if one could add a line of code in > the first line of an escript file and if the documentation were to be > revised to remove this rather bizaare restriction. The "normal" way of writing escripts is to have "#!/usr/bin/env escript" as the first line. But if you find it bizarre, you may have the source code of a complete Erlang module (with the module attribute and export of main/1) in the escript file. The drawback of this is however that you cannot invoke the script with just its name (factorial 5) as you must invoke escript with your script name as argument (escript factorial 5). /H?kan From erlang@REDACTED Tue Oct 5 17:04:18 2010 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 5 Oct 2010 17:04:18 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> References: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> Message-ID: On Tue, Oct 5, 2010 at 11:02 AM, Masklinn wrote: > On 2010-10-05, at 10:34 , Joe Armstrong wrote: >> Pids >> >> There shouldn't be ports in the language at all (sorry) - ports should >> in all circumstances >> behave and be indistinguishable from Pids. ?open_port(...) should >> return a Pid and not a port. >> >> We should reduce the number of concepts in the langauge not increase >> them - all in the >> name of conceptual integrity. >> >> /Joe > > But they *are* two different concepts. But they shouldn't have been different - sure they have a differnt way of being created. There should be the notion of "creating a thing that behaves like a process" that is actually not a process, which which behaves like a process. Ets tables and ports should have been "things that behave like processes" - they should respond to messages and send exits when they die and so on. If they had behaved like processes then life would be easier and more regular ... If ports where pids then one would be able to access ports on remote nodes via their pids (just like remote pids in distributed erlang). Even if at the usage level they behave the same way from the POV of an external process, they're not created the same way (``open_port`` vs ``spawn`` and a bunch of control functions dumped into the ``erlang`` module), and you also need to distinguish them when *implementing* them. And I'm guessing that's where Tony is looking for a better name. > > If down the line ports and pids are better merged (in both external interface and implementation details) then maybe "ports" as a name/concept can disappear, but I don't think that's the case at the moment, you still need to refer to them as entities separate from pids when talking about implementing things in erlang. The erlang view of the world is "everything is a process" - all you should need to know is how to create a process and what protocol the process obeys - otherwise these things should obey "process semantics" this should be true for ets tables and ports ... /Joe From masklinn@REDACTED Tue Oct 5 17:11:02 2010 From: masklinn@REDACTED (Masklinn) Date: Tue, 5 Oct 2010 17:11:02 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> Message-ID: On 2010-10-05, at 17:04 , Joe Armstrong wrote: > On Tue, Oct 5, 2010 at 11:02 AM, Masklinn wrote: >> On 2010-10-05, at 10:34 , Joe Armstrong wrote: >>> Pids >>> >>> There shouldn't be ports in the language at all (sorry) - ports should >>> in all circumstances >>> behave and be indistinguishable from Pids. open_port(...) should >>> return a Pid and not a port. >>> >>> We should reduce the number of concepts in the langauge not increase >>> them - all in the >>> name of conceptual integrity. >>> >>> /Joe >> >> But they *are* two different concepts. > > But they shouldn't have been different - sure they have a differnt way > of being created. > > There should be the notion of "creating a thing that behaves like a > process" that is actually > not a process, which which behaves like a process. Some kind of "messageable" object protocol? Yeah that would be nice. But that wouldn't stop people from needing to refer to these messageables as what they are when discussing e.g. their implementation details. If you're implementing a port or a port driver, you have to talk about it as that, or go through hoops to express it. Which ends up the same. > Ets tables and > ports should have > been "things that behave like processes" - they should respond to messages and > send exits when they die and so on. If they had behaved like processes > then life would be > easier and more regular ... If ports where pids then one would be able > to access ports on > remote nodes via their pids (just like remote pids in distributed erlang). From the user's (of the process/port/table) point of view, sure, I completely agree, but that doesn't solve the implementor's problem. > The erlang view of the world is "everything is a process" - all you > should need to know > is how to create a process and what protocol the process obeys - > otherwise these things should > obey "process semantics" this should be true for ets tables and ports ? As I said, that's nice when you're using these objects/systems in implementing your program, but it's not feasible (or useless as you prefer) when you're implementing one of them: you don't implement a port the same way you implement a process, and you still might need to discuss them as that when discussing implementing them. Saying that everything is a process doesn't help there, it would just lead to people calling them "ets process" and "ports process" and we're back to square one. From attila.r.nohl@REDACTED Tue Oct 5 19:16:20 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 5 Oct 2010 19:16:20 +0200 Subject: Erlang SNMP agent coexisting with net-snmp? Message-ID: Hello! I have an Erlang application and I want to return some stats about the application and the host via SNMP. Some of the stats (for example disk usage or load average) is already implemented by the snmpd in the net-snmp package (or at least there's a bundled example configuration file which can provide this data). The application-related stats can be queried from the application. I was wondering if it's possible that use the net-snmp snmpd as the SNMP master agent and the Erlang application as a subagent, so a single SNMP interface would return all data, but I haven't found in the documentation that Erlang would support AgentX. This (of course) excludes the other way, that the Erlang is the master and the net-snmp is the sub. I can also implement the host-based counters in Erlang or use the 'pass' or 'exec' directives in net-snmp to get the application-based counters via an other interface from the application. Am I missing an other alternative? Maybe an AgentX implementation for Erlang? From anthonym@REDACTED Tue Oct 5 21:07:31 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Tue, 5 Oct 2010 12:07:31 -0700 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> Message-ID: <20101005190731.GA7108@alumni.caltech.edu> +1 How do we get ets and ports turned into Pids, I know it will take like 5 years to fully deprecate old usage, but why not start now? -Anthony On Tue, Oct 05, 2010 at 05:04:18PM +0200, Joe Armstrong wrote: > On Tue, Oct 5, 2010 at 11:02 AM, Masklinn wrote: > > On 2010-10-05, at 10:34 , Joe Armstrong wrote: > >> Pids > >> > >> There shouldn't be ports in the language at all (sorry) - ports should > >> in all circumstances > >> behave and be indistinguishable from Pids. ?open_port(...) should > >> return a Pid and not a port. > >> > >> We should reduce the number of concepts in the langauge not increase > >> them - all in the > >> name of conceptual integrity. > >> > >> /Joe > > > > But they *are* two different concepts. > > But they shouldn't have been different - sure they have a differnt way > of being created. > > There should be the notion of "creating a thing that behaves like a > process" that is actually > not a process, which which behaves like a process. Ets tables and > ports should have > been "things that behave like processes" - they should respond to messages and > send exits when they die and so on. If they had behaved like processes > then life would be > easier and more regular ... If ports where pids then one would be able > to access ports on > remote nodes via their pids (just like remote pids in distributed erlang). > > > Even if at the usage level they behave the same way from the POV of > an external process, they're not created the same way (``open_port`` > vs ``spawn`` and a bunch of control functions dumped into the > ``erlang`` module), and you also need to distinguish them when > *implementing* them. And I'm guessing that's where Tony is looking for > a better name. > > > > If down the line ports and pids are better merged (in both external interface and implementation details) then maybe "ports" as a name/concept can disappear, but I don't think that's the case at the moment, you still need to refer to them as entities separate from pids when talking about implementing things in erlang. > > The erlang view of the world is "everything is a process" - all you > should need to know > is how to create a process and what protocol the process obeys - > otherwise these things should > obey "process semantics" this should be true for ets tables and ports ... > > /Joe > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- ------------------------------------------------------------------------ Anthony Molinaro From tony@REDACTED Tue Oct 5 21:34:00 2010 From: tony@REDACTED (Tony Rogvall) Date: Tue, 5 Oct 2010 21:34:00 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> Message-ID: <96A72EDC-3AE9-480C-9F3B-39BEDE17EBC2@rogvall.se> On 5 okt 2010, at 17.04, Joe Armstrong wrote: > > But they shouldn't have been different - sure they have a differnt way > of being created. > FYI In the Erlang multiprocessor prototype by Pekka Hedqvist and me, implemented 1997, we moved towards this model. Not because it was pretty but because it was simple (and nice looking) ;-) One thing we did was to remove (nearly) all synchronous calls in the VM, even process_info was implemented with signals (primitive messages). Ports and processes both used a common structure (Task) and a common signal interface. If you have not read it, here is the link. I think it is nice reading. Note that multicore was not present at the time. We run it on a super expensive PC with 4 processors called four roses, thanks Bjarne ;-) http://www.erlang.se/publications/xjobb/0089-hedqvist.pdf It is fun to see that we thought that we should use Pascal like "pseudo code" to be political correct ;-) /Tony (R) > There should be the notion of "creating a thing that behaves like a > process" that is actually > not a process, which which behaves like a process. Ets tables and > ports should have > been "things that behave like processes" - they should respond to messages and > send exits when they die and so on. If they had behaved like processes > then life would be > easier and more regular ... If ports where pids then one would be able > to access ports on > remote nodes via their pids (just like remote pids in distributed erlang). > > > Even if at the usage level they behave the same way from the POV of > an external process, they're not created the same way (``open_port`` > vs ``spawn`` and a bunch of control functions dumped into the > ``erlang`` module), and you also need to distinguish them when > *implementing* them. And I'm guessing that's where Tony is looking for > a better name. >> >> If down the line ports and pids are better merged (in both external interface and implementation details) then maybe "ports" as a name/concept can disappear, but I don't think that's the case at the moment, you still need to refer to them as entities separate from pids when talking about implementing things in erlang. > > The erlang view of the world is "everything is a process" - all you > should need to know > is how to create a process and what protocol the process obeys - > otherwise these things should > obey "process semantics" this should be true for ets tables and ports ... > > /Joe > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From tony.arcieri@REDACTED Tue Oct 5 21:57:10 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Tue, 5 Oct 2010 13:57:10 -0600 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: Message-ID: As I'm building a language which is layered on top of Erlang itself, would it make sense to implement TCP sockets as Pids which point to a proxy process that actually talks to the socket or ETS table, relegating ports to the internal implementation? On Tue, Oct 5, 2010 at 2:34 AM, Joe Armstrong wrote: > Pids > > There shouldn't be ports in the language at all (sorry) - ports should > in all circumstances > behave and be indistinguishable from Pids. open_port(...) should > return a Pid and not a port. > > We should reduce the number of concepts in the langauge not increase > them - all in the > name of conceptual integrity. > > /Joe > > > On Tue, Oct 5, 2010 at 6:45 AM, Tony Arcieri > wrote: > > I'm working on Reia, a Ruby-like language for the Erlang VM. Reia is an > > everything-is-an-object language, and the Erlang core types are all > wrapped > > as objects. > > > > As part of this process I've wrapped ports. However, in my opinion port > is a > > confusing name, particularly when dealing with a TCP port (in the Erlang > > sense) which represents a connection to a particular TCP port (in the TCP > > sense). > > > > Because of this sort of ambiguity, I'd like to rename ports. But what's a > > good replacement name? Some of the ones I've considered: SysPort, IOPort, > > Channel. > > > > What would you call ports if you had the opportunity to rename them? > > > > -- > > Tony Arcieri > > Medioh! A Kudelski Brand > > > -- Tony Arcieri Medioh! A Kudelski Brand From erlang@REDACTED Tue Oct 5 22:12:19 2010 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 5 Oct 2010 22:12:19 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> Message-ID: On Tue, Oct 5, 2010 at 5:11 PM, Masklinn wrote: > On 2010-10-05, at 17:04 , Joe Armstrong wrote: >> On Tue, Oct 5, 2010 at 11:02 AM, Masklinn wrote: >>> On 2010-10-05, at 10:34 , Joe Armstrong wrote: >>>> Pids >>>> >>>> There shouldn't be ports in the language at all (sorry) - ports should >>>> in all circumstances >>>> behave and be indistinguishable from Pids. ?open_port(...) should >>>> return a Pid and not a port. >>>> >>>> We should reduce the number of concepts in the langauge not increase >>>> them - all in the >>>> name of conceptual integrity. >>>> >>>> /Joe >>> >>> But they *are* two different concepts. >> >> But they shouldn't have been different - sure they have a differnt way >> of being created. >> >> There should be the notion of "creating a thing that behaves like a >> process" that is actually >> not a process, which which behaves like a process. > Some kind of "messageable" object protocol? Yeah that would be nice. But that wouldn't stop people from needing to refer to these messageables as what they are when discussing e.g. their implementation details. If you're implementing a port or a port driver, you have to talk about it as that, or go through hoops to express it. Which ends up the same. > >> Ets tables and >> ports should have >> been "things that behave like processes" - they should respond to messages and >> send exits when they die and so on. If they had behaved like processes >> then life would be >> easier and more regular ... If ports where pids then one would be able >> to access ports on >> remote nodes via their pids (just like remote pids in distributed erlang). > From the user's (of the process/port/table) point of view, sure, I completely agree, but that doesn't solve the implementor's problem. > >> The erlang view of the world is "everything is a process" - all you >> should need to know >> is how to create a process and what protocol the process obeys - Good point - so we'd need a new BIF protocol(Pid) -> Atom which returns the name of the protocol that the Pid is supposed to obey It would be nice to say Pid = spawn(fun() -> end, Name) to embed the protcol name in the Pid - that would be nice (and useful) :-) So open_port(...) as before would return a Pid not a port but protocol(Pid) would return port and protocol(Pid) might return ets if Pid was a process that behaved like an Ets process ... regular processes would return 'undefined' and well behaved processes would return a name Then we could specify the protcol in CSP Name => in ! Type() -> out ! Type1() -> .... Nice ... /Joe >> otherwise these things should >> obey "process semantics" this should be true for ets tables and ports ? > As I said, that's nice when you're using these objects/systems in implementing your program, but it's not feasible (or useless as you prefer) when you're implementing one of them: you don't implement a port the same way you implement a process, and you still might need to discuss them as that when discussing implementing them. Saying that everything is a process doesn't help there, it would just lead to people calling them "ets process" and "ports process" and we're back to square one. From robert.virding@REDACTED Tue Oct 5 23:17:46 2010 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 05 Oct 2010 23:17:46 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: Message-ID: <4CAB95FA.6070804@erlang-solutions.com> I think that if something has process semantics, i.e. behaves like a process with asynch messages, exit signals, etc., then it's reference should be a pid. This irrespective of how it is started. Even today, with "normal" processes you don't have to know how they have been started to use them, just which protocol you need to communicate with them. Ports, of course, are a natural for pids as they already have process semantics. ( I won't go into a long tirade about how wrong the docs are when claiming that the port_XXXX BIFs have a cleaner and more logical behaviour than the message protocol. :-) ) ETS tables are also a natural to be described as pids, it is easy to define a message based protocol for them, and they already handle exit signals. The important thing with process "things" is that they HAVE process semantics and not HOW it is implemented under the covers. So for example, "sending a message" to an ETS pid could very well be implemented in a similar fashion as it is today. There should only be two types of things in Erlang, processes and immutable data. Robert On 2010-10-05 21.57, Tony Arcieri wrote: > As I'm building a language which is layered on top of Erlang itself, would > it make sense to implement TCP sockets as Pids which point to a proxy > process that actually talks to the socket or ETS table, relegating ports to > the internal implementation? > > On Tue, Oct 5, 2010 at 2:34 AM, Joe Armstrong wrote: > >> Pids >> >> There shouldn't be ports in the language at all (sorry) - ports should >> in all circumstances >> behave and be indistinguishable from Pids. open_port(...) should >> return a Pid and not a port. >> >> We should reduce the number of concepts in the langauge not increase >> them - all in the >> name of conceptual integrity. >> >> /Joe >> >> >> On Tue, Oct 5, 2010 at 6:45 AM, Tony Arcieri >> wrote: >>> I'm working on Reia, a Ruby-like language for the Erlang VM. Reia is an >>> everything-is-an-object language, and the Erlang core types are all >> wrapped >>> as objects. >>> >>> As part of this process I've wrapped ports. However, in my opinion port >> is a >>> confusing name, particularly when dealing with a TCP port (in the Erlang >>> sense) which represents a connection to a particular TCP port (in the TCP >>> sense). >>> >>> Because of this sort of ambiguity, I'd like to rename ports. But what's a >>> good replacement name? Some of the ones I've considered: SysPort, IOPort, >>> Channel. >>> >>> What would you call ports if you had the opportunity to rename them? >>> >>> -- >>> Tony Arcieri >>> Medioh! A Kudelski Brand >>> > > From mbj@REDACTED Wed Oct 6 00:19:32 2010 From: mbj@REDACTED (Martin Bjorklund) Date: Wed, 06 Oct 2010 00:19:32 +0200 (CEST) Subject: [erlang-questions] Erlang SNMP agent coexisting with net-snmp? In-Reply-To: References: Message-ID: <20101006.001932.137393757.mbj@tail-f.com> Hi, Attila Rajmund Nohl wrote: > Hello! > > I have an Erlang application and I want to return some stats about the > application and the host via SNMP. Some of the stats (for example disk > usage or load average) is already implemented by the snmpd in the > net-snmp package (or at least there's a bundled example configuration > file which can provide this data). The application-related stats can > be queried from the application. > > I was wondering if it's possible that use the net-snmp snmpd as the > SNMP master agent and the Erlang application as a subagent, so a > single SNMP interface would return all data, but I haven't found in > the documentation that Erlang would support AgentX. The SNMP agent in OTP does not support any standard subagent protocol (just the proprietary erlang-based one) but there is the application "snmp_ext" in jungerl which contains some code for three different "subagent" protocols - AgentX, SMUX, and SNMP. If all your MIBs are read-only, it is actually possible to use SNMP as the subagent protocol. net-snmp supports this as well, so you can run net-snmp as the master, and OTP as the subagent, or the other way around. > This (of course) > excludes the other way, that the Erlang is the master and the net-snmp > is the sub. > > I can also implement the host-based counters in Erlang or use the > 'pass' or 'exec' directives in net-snmp to get the application-based > counters via an other interface from the application. > > Am I missing an other alternative? Maybe an AgentX implementation for Erlang? The AgentX code in snmp_ext is not complete at all; IIRC it contains just the encode/decode functions. Feel free to use this code as a starting point for a proper AgentX implementation! I think it would be very useful to have this option for the OTP agent. /martin From jiansenhe@REDACTED Wed Oct 6 00:29:42 2010 From: jiansenhe@REDACTED (Jiansen He) Date: Tue, 5 Oct 2010 23:29:42 +0100 Subject: [erlang-questions] Problems when running distributed pingpong Message-ID: Hi, I'm new to Erlang. I tried to run the distributed pingpong program which could be found at http://www.erlang.org/doc/getting_started/conc_prog.html. The program works well when I run it in two terminals on the same machine. However, it doesn't work when I try to run it on two computers. I posted my testing result as follows. on Machine1 (Mac OS X Version 10.6.4) $ erl -sname pong Erlang R14B (erts-5.8.1) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) (pong@REDACTED)1> (pong@REDACTED)1> tut17:start_pong(). true On Machine2 (Windows 7) >werl -sname ping Erlang R14B (erts-5.8.1) [smp:2:2] [rq:2] [async-threads:0] Eshell V5.8.1 (abort with ^G) (ping@REDACTED)1> (ping@REDACTED)1> tut17:start_ping('pong@REDACTED'). <0.39.0> (ping@REDACTED)2> Nothing happened on both sides:( ========= To find the problem, I opened another terminal on both machines. on Windows ~>ping 95.175.140.5 Pinging 95.175.140.5 with 32 bytes of data: Reply from 95.175.140.5: bytes=32 time=4ms TTL=64 Reply from 95.175.140.5: bytes=32 time=1ms TTL=64 Reply from 95.175.140.5: bytes=32 time=34ms TTL=64 Reply from 95.175.140.5: bytes=32 time=10ms TTL=64 Ping statistics for 95.175.140.5: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 1ms, Maximum = 34ms, Average = 12ms on Mac dyn-95-175-140-5:~ $ ping -c4 95.175.140.3 PING 95.175.140.3 (95.175.140.3): 56 data bytes 64 bytes from 95.175.140.3: icmp_seq=0 ttl=128 time=1.491 ms 64 bytes from 95.175.140.3: icmp_seq=1 ttl=128 time=0.388 ms 64 bytes from 95.175.140.3: icmp_seq=2 ttl=128 time=0.399 ms 64 bytes from 95.175.140.3: icmp_seq=3 ttl=128 time=0.811 ms --- 95.175.140.3 ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.388/0.772/1.491/0.449 ms So, those two machines are connected to the same Ethernet and they can ping each other. I turned off the firewall but the program still doesn't work:( ======== To find more information about my network, I reopen the erlang system on Mac with -name option (pong@REDACTED)1> inet:gethostbyaddr('95.175.140.5'). {ok,{hostent,"dyn-95-175-140-5.chaledi.unite.inuknetworks.com", ["5.140.175.95.in-addr.arpa"], inet,4, [{95,175,140,5}]}} (pong@REDACTED)2> net_adm:dns_hostname('dyn-95-175-140-5'). {ok,"dyn-95-175-140-5"} (pong@REDACTED)3> net_adm:dns_hostname(' dyn-95-175-140-5.inuknetworks.com'). {ok,"dyn-95-175-140-5.inuknetworks.com"} (pong@REDACTED)4> net_adm:dns_hostname(' dyn-95-175-140-3.inuknetworks.com'). {error,'dyn-95-175-140-3.inuknetworks.com'} Then I open erlang system on Windows with -name option... On Mac (pong@REDACTED)5> inet:gethostbyaddr('95.175.140.3'). {ok,{hostent,"dyn-95-175-140-3.chaledi.unite.inuknetworks.com", ["3.140.175.95.in-addr.arpa"], inet,4, [{95,175,140,3}]}} (pong@REDACTED)6> net_adm:dns_hostname('dyn-95-175-140-3'). {ok,"dyn-95-175-140-19"} (pong@REDACTED)7> net_adm:dns_hostname(' dyn-95-175-140-3.inuknetworks.com'). {ok,"dyn-95-175-140-3.inuknetworks.com"} (pong@REDACTED)8> net_adm:dns_hostname(' dyn-95-175-140-3.chaledi.unite.inuknetworks.com'). {ok,"dyn-95-175-140-3.chaledi.unite.inuknetworks.com"} It is clear that the erlang system could only recognize another host where anthor erlang system is running. And, more importantly, I get host names that is known by my local DNS server. On Mac (pong@REDACTED)8> erlang:get_cookie(). 'erlang_cookie' On windows (ping@REDACTED)1> erlang:get_cookie(). 'erlang_cookie' It shows that the magic cookies are set correctly on both machines. on Mac (pong@REDACTED)9> pingpong:start_pong(). true (pong@REDACTED)10> net_adm:ping(' pong@REDACTED'). pong (pong@REDACTED)11> net_adm:ping(' pong@REDACTED'). pang (pong@REDACTED)12> net_adm:localhost(). "dyn-95-175-140-3.inuknetworks.com" It suggests that I should use hostname that has the form 'nodeName@REDACTED*.inuknetworks.com' on Windows (ping@REDACTED)2> net_adm:ping(' pong@REDACTED'). pang To summarize. my two computers are connected to the same Ethernet. The only difference between two IP addresses is the last digit. Firewalls are closed on both machines. Computers can ping each other from the outside of Erlang system. Inside the Erlang system, one machine could have at least 3 valid host names from DNS server's point of view. However, only one of them is valid for net_adm:ping/1, if Erlang is invoked with -name option. An Erlang system could detect another host where another Erlang system is running. Somehow, an Erlang system cannot communicate with an Erlang node in another Erlang system. Any suggestions? Jiansen From jws@REDACTED Wed Oct 6 02:14:23 2010 From: jws@REDACTED (Jeff Schultz) Date: Wed, 6 Oct 2010 11:14:23 +1100 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> Message-ID: <20101006001423.GA8623@mulga.csse.unimelb.edu.au> On Tue, Oct 05, 2010 at 05:04:18PM +0200, Joe Armstrong wrote: > There should be the notion of "creating a thing that behaves like a > process" that is actually not a process, which which behaves like a > process. Ets tables and ports should have been "things that behave > like processes" - they should respond to messages and send exits > when they die and so on. If they had behaved like processes then > life would be easier and more regular ... If ports where pids then > one would be able to access ports on remote nodes via their pids > (just like remote pids in distributed erlang). This sounds good to me, but I have a practical question: How much existing code would break if ets:new or open_port returned a pid? Is there a lot, or very little? Your protocol/1 idea would make it easy to adapt is_port to only succeed on pids created by open_port, but there's probably code somewhere that handles both ordinary processes and ports and that distinguishes them by is_pid. For that matter, what would we do with the named_table option to ets:new? Make it register the process? Jeff Schultz From djk121@REDACTED Wed Oct 6 05:09:33 2010 From: djk121@REDACTED (Dan Kelley) Date: Tue, 5 Oct 2010 23:09:33 -0400 Subject: [erlang-questions] high-volume logging via a gen_server In-Reply-To: <201010052023.41849.als@iinet.net.au> References: <2A8C7AB5-9F26-4D27-AEE8-029952B2728D@hates.ms> <201010052023.41849.als@iinet.net.au> Message-ID: On Tue, Oct 5, 2010 at 5:23 AM, Anthony Shipman wrote: > On Tue, 5 Oct 2010 06:42:17 am Mihai Balea wrote: > > On Oct 4, 2010, at 2:57 PM, Dan Kelley wrote: > > > So, what are good strategies to cope with a large incoming volume of > > > messages that all need to wind up in the same logfile? Is there a more > > > efficient way to write to disk than the simple io:format() call than > I'm > > > using above? What's a good way to parallelize the logging over > multiple > > > processes but keep all of the information in one file? > > > > You can have a buffer process ahead of the logger. The buffer process just > has > to accept and store log requests. A separate write process can pull them > from > the buffer (using an async protocol). If the buffer gets too full then you > can have various policies such as discard oldest, youngest, low priority > messages etc. One thing I was surprised to find while profiling yesterday was that much of the reason that logging was crippling my server was the CPU overhead of io:format, io_lib:format, and the various calls required to get a high-res timestamp. The way my router sets up right now, it takes ~15 microseconds to parse a reasonably complicated protocol, but about ~100 microseconds to compose and format the log line which says the parse has happened. Per what others have suggested in this thread, I had thought that a possible answer to my bottleneck would be to push as much of the logging work out to the clients. That turned out to make per-transaction latency very poor for the reason described above. So for now, I've profiled the path which generates the log line strings and have de-stupided my code a bit to bring the per-log-line latency down to ~60 microseconds. Taking into account all the responses on the thread (thanks!), and in conjunction with the formatting changes, I'm trying something like what you describe: because it's an easy first step, I'm just replacing writing to disk directly inside my logging gen_server with calls to disk_log:blog/2. So far that's working well. From bengt.kleberg@REDACTED Wed Oct 6 09:22:48 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 06 Oct 2010 09:22:48 +0200 Subject: [erlang-questions] Problems when running distributed pingpong In-Reply-To: References: Message-ID: <1286349768.5140.18.camel@seasc1137> Greetings, I see that you have tried to net_adm:ping/1 yourself (the node you are on). Have you tried the same command towards the other node? Preferably with all the 3 different addresses, both when the nodes have been started with -name and -sname. bengt On Wed, 2010-10-06 at 00:29 +0200, Jiansen He wrote: > Hi, > > I'm new to Erlang. I tried to run the distributed pingpong program which > could be found at http://www.erlang.org/doc/getting_started/conc_prog.html. > The program works well when I run it in two terminals on the same machine. > However, it doesn't work when I try to run it on two computers. I posted my > testing result as follows. > > on Machine1 (Mac OS X Version 10.6.4) > $ erl -sname pong > Erlang R14B (erts-5.8.1) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > (pong@REDACTED)1> > > (pong@REDACTED)1> tut17:start_pong(). > true > > On Machine2 (Windows 7) > >werl -sname ping > > Erlang R14B (erts-5.8.1) [smp:2:2] [rq:2] [async-threads:0] > > Eshell V5.8.1 (abort with ^G) > (ping@REDACTED)1> > > (ping@REDACTED)1> tut17:start_ping('pong@REDACTED'). > <0.39.0> > (ping@REDACTED)2> > > > Nothing happened on both sides:( > > ========= > > To find the problem, I opened another terminal on both machines. > > on Windows > ~>ping 95.175.140.5 > > Pinging 95.175.140.5 with 32 bytes of data: > Reply from 95.175.140.5: bytes=32 time=4ms TTL=64 > Reply from 95.175.140.5: bytes=32 time=1ms TTL=64 > Reply from 95.175.140.5: bytes=32 time=34ms TTL=64 > Reply from 95.175.140.5: bytes=32 time=10ms TTL=64 > > Ping statistics for 95.175.140.5: > Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), > Approximate round trip times in milli-seconds: > Minimum = 1ms, Maximum = 34ms, Average = 12ms > > > on Mac > dyn-95-175-140-5:~ $ ping -c4 95.175.140.3 > PING 95.175.140.3 (95.175.140.3): 56 data bytes > 64 bytes from 95.175.140.3: icmp_seq=0 ttl=128 time=1.491 ms > 64 bytes from 95.175.140.3: icmp_seq=1 ttl=128 time=0.388 ms > 64 bytes from 95.175.140.3: icmp_seq=2 ttl=128 time=0.399 ms > 64 bytes from 95.175.140.3: icmp_seq=3 ttl=128 time=0.811 ms > > --- 95.175.140.3 ping statistics --- > 4 packets transmitted, 4 packets received, 0.0% packet loss > round-trip min/avg/max/stddev = 0.388/0.772/1.491/0.449 ms > > So, those two machines are connected to the same Ethernet and they can ping > each other. I turned off the firewall but the program still doesn't work:( > > ======== > To find more information about my network, I reopen the erlang system on Mac > with -name option > (pong@REDACTED)1> > inet:gethostbyaddr('95.175.140.5'). > {ok,{hostent,"dyn-95-175-140-5.chaledi.unite.inuknetworks.com", > ["5.140.175.95.in-addr.arpa"], > inet,4, > [{95,175,140,5}]}} > (pong@REDACTED)2> > net_adm:dns_hostname('dyn-95-175-140-5'). > {ok,"dyn-95-175-140-5"} > (pong@REDACTED)3> net_adm:dns_hostname(' > dyn-95-175-140-5.inuknetworks.com'). > {ok,"dyn-95-175-140-5.inuknetworks.com"} > (pong@REDACTED)4> net_adm:dns_hostname(' > dyn-95-175-140-3.inuknetworks.com'). > {error,'dyn-95-175-140-3.inuknetworks.com'} > > Then I open erlang system on Windows with -name option... > > On Mac > (pong@REDACTED)5> > inet:gethostbyaddr('95.175.140.3'). > {ok,{hostent,"dyn-95-175-140-3.chaledi.unite.inuknetworks.com", > ["3.140.175.95.in-addr.arpa"], > inet,4, > [{95,175,140,3}]}} > (pong@REDACTED)6> > net_adm:dns_hostname('dyn-95-175-140-3'). > {ok,"dyn-95-175-140-19"} > (pong@REDACTED)7> net_adm:dns_hostname(' > dyn-95-175-140-3.inuknetworks.com'). > {ok,"dyn-95-175-140-3.inuknetworks.com"} > (pong@REDACTED)8> net_adm:dns_hostname(' > dyn-95-175-140-3.chaledi.unite.inuknetworks.com'). > {ok,"dyn-95-175-140-3.chaledi.unite.inuknetworks.com"} > > It is clear that the erlang system could only recognize another host where > anthor erlang system is running. And, more importantly, I get host names > that is known by my local DNS server. > > On Mac > (pong@REDACTED)8> erlang:get_cookie(). > 'erlang_cookie' > > On windows > (ping@REDACTED)1> erlang:get_cookie(). > 'erlang_cookie' > > It shows that the magic cookies are set correctly on both machines. > > on Mac > (pong@REDACTED)9> pingpong:start_pong(). > true > (pong@REDACTED)10> net_adm:ping(' > pong@REDACTED'). > pong > (pong@REDACTED)11> net_adm:ping(' > pong@REDACTED'). > pang > (pong@REDACTED)12> > net_adm:localhost(). "dyn-95-175-140-3.inuknetworks.com" > > It suggests that I should use hostname that has the form > 'nodeName@REDACTED*.inuknetworks.com' > > on Windows > (ping@REDACTED)2> net_adm:ping(' > pong@REDACTED'). > pang > > To summarize. my two computers are connected to the same Ethernet. The > only difference between two IP addresses is the last digit. Firewalls are > closed on both machines. Computers can ping each other from the outside of > Erlang system. Inside the Erlang system, one machine could have at least 3 > valid host names from DNS server's point of view. However, only one of them > is valid for net_adm:ping/1, if Erlang is invoked with -name option. An > Erlang system could detect another host where another Erlang system is > running. Somehow, an Erlang system cannot communicate with an Erlang node > in another Erlang system. > > Any suggestions? > > > Jiansen From erlang@REDACTED Wed Oct 6 09:53:09 2010 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 6 Oct 2010 09:53:09 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: <20101006001423.GA8623@mulga.csse.unimelb.edu.au> References: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> <20101006001423.GA8623@mulga.csse.unimelb.edu.au> Message-ID: On Wed, Oct 6, 2010 at 2:14 AM, Jeff Schultz wrote: > On Tue, Oct 05, 2010 at 05:04:18PM +0200, Joe Armstrong wrote: >> There should be the notion of "creating a thing that behaves like a >> process" that is actually not a process, which which behaves like a >> process. Ets tables and ports should have been "things that behave >> like processes" - they should respond to messages and send exits >> when they die and so on. If they had behaved like processes then >> life would be easier and more regular ... If ports where pids then >> one would be able to access ports on remote nodes via their pids >> (just like remote pids in distributed erlang). > > This sounds good to me, but I have a practical question: > > ? ?How much existing code would break if ets:new or open_port > ? ?returned a pid? ?Is there a lot, or very little? > > Your protocol/1 idea would make it easy to adapt is_port to only > succeed on pids created by open_port, but there's probably code > somewhere that handles both ordinary processes and ports and that > distinguishes them by is_pid. The more I think about this the more I like it. The problem with a Pid is you have no ideas what resource it represents and what protocol it obeys - this information is conveyed implicitly by the location of the Pid in some bit of code - you need to say "hello pid, which protocol do you obey" this cannot result in a network message but must be local to a pid. You need to know this information *before* you send a message, thus the implementation cannot involve any messaging. I think the Pid is just a heap pointer to a local data structure, so adding an extra field would be no bother ... You could use process_info(Pid) and look at the initial_call data but I assume process_info results in a network message to find out what's up with the process (is this correct?) > > > For that matter, what would we do with the named_table option to > ets:new? ?Make it register the process? But there shouldn't be named tables at all - there should only be pids and registerd processes one name registration mechanism not many ... > > > ? ?Jeff Schultz > From simon.hillerbring@REDACTED Wed Oct 6 10:16:34 2010 From: simon.hillerbring@REDACTED (Simon Hillerbring) Date: Wed, 6 Oct 2010 01:16:34 -0700 (PDT) Subject: Problems with erl_call Message-ID: <4550345a-956b-46b6-ac6a-fda355a75e29@f25g2000yqc.googlegroups.com> Hi, I have problems using erl_call with negative integers in the argument list for -a. Is there a way around this problem? /Simon Example: I have no problems with integer_to_list(1), but I get no reply at all for integer_to_list(-1): sol:~> erl_call -c COOKIE -sname test@REDACTED -a 'erlang integer_to_list [1]' "1"sol:~> erl_call -c COOKIE -sname test@REDACTED -a 'erlang integer_to_list [-1]' sol:~> I see no problems when I run it with -v, but the call seems to disappear: sol:~> erl_call -c COOKIE -sname test@REDACTED -a 'erlang integer_to_list [-1]' -v erl_call: node = test@REDACTED Cookie = COOKIE flags = verbosep module: name = , size = 0 apply = erlang integer_to_list [-1] erl_call: we are now connected to node "test@REDACTED" erl_call: module = erlang, function = integer_to_list, args = [-1] sol:~> From hynek@REDACTED Wed Oct 6 13:35:44 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Wed, 6 Oct 2010 13:35:44 +0200 Subject: Regular grammars and binary matching performance Message-ID: Hello, I have just send new regexdna solution using R14's binary module: http://shootout.alioth.debian.org/u32q/performance.php?test=regexdna http://shootout.alioth.debian.org/u64q/performance.php?test=regexdna It shows how big gain we would have get by limited regular expression support in binary module. I have written limited regular expression support in function compile_pattern/1 (very limited and poor man solution!) as demonstration there http://shootout.alioth.debian.org/u32q/program.php?test=regexdna&lang=hipe&id=7 for curiosity. I think it would be nice to have support for regular grammars in binary module or adapt http://code.google.com/p/re2/ as limited regular expression library because those limited regular expression (truly regular grammars, without sub-capturing may be and so) could be feasible in real projects. For comparison, re module is 15 times slower in this benchmark http://shootout.alioth.debian.org/u32q/program.php?test=regexdna&lang=hipe&id=6 Do you think it would be useful? Is anybody working on it already? Best regards -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From robert.virding@REDACTED Wed Oct 6 14:54:18 2010 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 06 Oct 2010 14:54:18 +0200 Subject: [erlang-questions] Regular grammars and binary matching performance In-Reply-To: References: Message-ID: <4CAC717A.9020806@erlang-solutions.com> Tuncer Ayaz has built an interface to the re2 library using NIFs, I think it works on both lists and binaries. Implementing true POSIX regular expressions in erlang is not that difficult, I have done one using the same algorithms as in re2 which works on lists and binaries (but not io_lists). I haven't measured it against re2 but I assume it is slower, but it does have the benefit of being guaranteed portable to all systems. Using capturing sub-expressions is costly though. While I agree that these regexps would definitely be useful, and easier to use because they are simpler, I don't think the interface should be included in the binary module. It would be better to have a separate module, for example re2 or pos_re, and have them work on both lists and binaries. The extra runtime cost would be negligible. One benefit of using re2, or something based on the same algorithms, is that you avoid having backtracking as in PCRE/Perl which can go pathological for some regular expressions. Robert On 2010-10-06 13.35, Hynek Vychodil wrote: > Hello, > I have just send new regexdna solution using R14's binary module: > http://shootout.alioth.debian.org/u32q/performance.php?test=regexdna > http://shootout.alioth.debian.org/u64q/performance.php?test=regexdna > It shows how big gain we would have get by limited regular expression > support in binary module. I have written limited regular expression > support in function compile_pattern/1 (very limited and poor man > solution!) as demonstration there > http://shootout.alioth.debian.org/u32q/program.php?test=regexdna&lang=hipe&id=7 > for curiosity. > I think it would be nice to have support for regular grammars in > binary module or adapthttp://code.google.com/p/re2/ as limited > regular expression library because those limited regular expression > (truly regular grammars, without sub-capturing may be and so) could be > feasible in real projects. For comparison, re module is 15 times > slower in this benchmark > http://shootout.alioth.debian.org/u32q/program.php?test=regexdna&lang=hipe&id=6 > > Do you think it would be useful? Is anybody working on it already? > > Best regards From vinoski@REDACTED Wed Oct 6 15:52:23 2010 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 6 Oct 2010 09:52:23 -0400 Subject: [erlang-questions] Problems with erl_call In-Reply-To: <4550345a-956b-46b6-ac6a-fda355a75e29@f25g2000yqc.googlegroups.com> References: <4550345a-956b-46b6-ac6a-fda355a75e29@f25g2000yqc.googlegroups.com> Message-ID: On Wed, Oct 6, 2010 at 4:16 AM, Simon Hillerbring wrote: > Hi, > I have problems using erl_call with negative integers in the argument > list for -a. > Is there a way around this problem? Definitely a bug. I assume erl_call is returning an exit status of -1 for you? Looks like the ei_format code isn't handling negative numbers for this case. It does an isdigit test on the minus sign character and gets an error. It seems to work if you use the -e option and feed erlang:integer_to_list(-1). to stdin, but the result in that case is {ok, "-1"} rather than just "-1" so you'd have a little parsing to do to extract the value. Or you can wait a few minutes for the patch I'll be submitting to erlang-patches to fix this. --steve From hynek@REDACTED Wed Oct 6 15:55:05 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Wed, 6 Oct 2010 15:55:05 +0200 Subject: [erlang-questions] Regular grammars and binary matching performance In-Reply-To: <4CAC717A.9020806@erlang-solutions.com> References: <4CAC717A.9020806@erlang-solutions.com> Message-ID: On Wed, Oct 6, 2010 at 2:54 PM, Robert Virding wrote: > ?Tuncer Ayaz has built an interface to the re2 library using NIFs, I think > it works on both lists and binaries. Implementing true POSIX regular > expressions in erlang is not that difficult, I have done one using the same > algorithms as in re2 which works on lists and binaries (but not io_lists). I > haven't measured it against re2 but I assume it is slower, but it does have > the benefit of being guaranteed portable to all systems. Using capturing > sub-expressions is costly though. I have been thinking about parse transformation and generation of code of DFA before binary module introduced. I thought that with HiPE compilation result should be competitive. But binary module changed game. When R14A released I have written above proof of concept and than haven't found time to try previous approach. I think you are right that Erlang native implementation would be nice. > > While I agree that these regexps would definitely be useful, and easier to > use because they are simpler, I don't think the interface should be included > in the binary module. It would be better to have a separate module, for > example re2 or pos_re, and have them work on both lists and binaries. The > extra runtime cost would be negligible. > > One benefit of using re2, or something based on the same algorithms, is that > you avoid having backtracking as in PCRE/Perl which can go pathological for > some regular expressions. > > Robert > > > On 2010-10-06 13.35, Hynek Vychodil wrote: >> >> Hello, >> I have just send new regexdna solution using R14's binary module: >> http://shootout.alioth.debian.org/u32q/performance.php?test=regexdna >> http://shootout.alioth.debian.org/u64q/performance.php?test=regexdna >> It shows how big gain we would have get by limited regular expression >> support in binary module. I have written limited regular expression >> support in function compile_pattern/1 (very limited and poor man >> solution!) as demonstration there >> >> http://shootout.alioth.debian.org/u32q/program.php?test=regexdna&lang=hipe&id=7 >> for curiosity. >> I think it would be nice to have support for regular grammars in >> binary module or adapthttp://code.google.com/p/re2/ ?as limited >> regular expression library because those limited regular expression >> (truly regular grammars, without sub-capturing may be and so) could be >> feasible in real projects. For comparison, re module is 15 times >> slower in this benchmark >> >> http://shootout.alioth.debian.org/u32q/program.php?test=regexdna&lang=hipe&id=6 >> >> Do you think it would be useful? Is anybody working on it already? >> >> Best regards > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From jiansenhe@REDACTED Wed Oct 6 17:22:55 2010 From: jiansenhe@REDACTED (Jiansen He) Date: Wed, 6 Oct 2010 16:22:55 +0100 Subject: [erlang-questions] Problems when running distributed pingpong In-Reply-To: <1286349768.5140.18.camel@seasc1137> References: <1286349768.5140.18.camel@seasc1137> Message-ID: Thank you for reply. I tried to net_adm:ping/1 towards nodes on both machines. When I start Erlang with -sname, fully qualified hostnames are rejected as soon as net_adm:ping/1 is called. net_adm:ping('pong@REDACTED*') returns pang if * represents another machine. I don't what else I can do under -sname mode. Since -name works no matter whether two machines are connected to the same Ethernet, most of tests are run on this mode. The problem with -name mode is finding out the valid hostname. I also did following tests today. (pong@REDACTED)1> net_adm:names(). {ok,[{"pong",49525}]} (pong@REDACTED)2> inet:gethostbyaddr('95.175.140.20'). {ok,{hostent,"dyn-95-175-140-20.chaledi.unite.inuknetworks.com", ["20.140.175.95.in-addr.arpa"], inet,4, [{95,175,140,20}]}} (pong@REDACTED)3> net_adm:ping('pong@REDACTED'). pang (pong@REDACTED)4> net_adm:ping(' pong@REDACTED'). pong Outside the Erlang ping -c4 dyn-95-175-140-20.chaledi.unite.inuknetworks.com PING dyn-95-175-140-20.chaledi.unite.inuknetworks.com (95.175.140.20): 56 data bytes 64 bytes from 95.175.140.20: icmp_seq=0 ttl=64 time=0.040 ms 64 bytes from 95.175.140.20: icmp_seq=1 ttl=64 time=0.121 ms 64 bytes from 95.175.140.20: icmp_seq=2 ttl=64 time=0.052 ms 64 bytes from 95.175.140.20: icmp_seq=3 ttl=64 time=0.123 ms --- dyn-95-175-140-20.chaledi.unite.inuknetworks.com ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.040/0.084/0.123/0.038 ms $ ping -c4 dyn-95-175-140-20.inuknetworks.com ping: cannot resolve dyn-95-175-140-20.inuknetworks.com: Unknown host $ ping -c4 "20.140.175.95.in-addr.arpa" ping: cannot resolve 20.140.175.95.in-addr.arpa: Unknown host In -name mode. I tried all 4 forms of hostname to ping both local machine and remote machine. Only hostname with the form "dyn-95-175-140-*. inuknetworks.com" works when I ping the same machine. None of them works when I ping another machine. I'm confused why I get so many hostnames for the same machine. All of them works when I feed them to net_adm:dns_hostname/1. However, the only valid hostname for net_adm:ping/1(dyn-95-175-140-20.inuknetworks.com) is different from the only valid hostname for my local DNS ( dyn-95-175-140-20.chaledi.unite.inuknetworks.com). Will there be anther possible hostname that I haven't found? Jiansen On Wed, Oct 6, 2010 at 8:22 AM, Bengt Kleberg wrote: > Greetings, > > I see that you have tried to net_adm:ping/1 yourself (the node you are > on). > Have you tried the same command towards the other node? Preferably with > all the 3 different addresses, both when the nodes have been started > with -name and -sname. > > > bengt > > On Wed, 2010-10-06 at 00:29 +0200, Jiansen He wrote: > > Hi, > > > > I'm new to Erlang. I tried to run the distributed pingpong program which > > could be found at > http://www.erlang.org/doc/getting_started/conc_prog.html. > > The program works well when I run it in two terminals on the same > machine. > > However, it doesn't work when I try to run it on two computers. I posted > my > > testing result as follows. > > > > on Machine1 (Mac OS X Version 10.6.4) > > $ erl -sname pong > > Erlang R14B (erts-5.8.1) [source] [smp:4:4] [rq:4] [async-threads:0] > [hipe] > > [kernel-poll:false] > > > > Eshell V5.8.1 (abort with ^G) > > (pong@REDACTED)1> > > > > (pong@REDACTED)1> tut17:start_pong(). > > true > > > > On Machine2 (Windows 7) > > >werl -sname ping > > > > Erlang R14B (erts-5.8.1) [smp:2:2] [rq:2] [async-threads:0] > > > > Eshell V5.8.1 (abort with ^G) > > (ping@REDACTED)1> > > > > (ping@REDACTED)1> tut17:start_ping('pong@REDACTED'). > > <0.39.0> > > (ping@REDACTED)2> > > > > > > Nothing happened on both sides:( > > > > ========= > > > > To find the problem, I opened another terminal on both machines. > > > > on Windows > > ~>ping 95.175.140.5 > > > > Pinging 95.175.140.5 with 32 bytes of data: > > Reply from 95.175.140.5: bytes=32 time=4ms TTL=64 > > Reply from 95.175.140.5: bytes=32 time=1ms TTL=64 > > Reply from 95.175.140.5: bytes=32 time=34ms TTL=64 > > Reply from 95.175.140.5: bytes=32 time=10ms TTL=64 > > > > Ping statistics for 95.175.140.5: > > Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), > > Approximate round trip times in milli-seconds: > > Minimum = 1ms, Maximum = 34ms, Average = 12ms > > > > > > on Mac > > dyn-95-175-140-5:~ $ ping -c4 95.175.140.3 > > PING 95.175.140.3 (95.175.140.3): 56 data bytes > > 64 bytes from 95.175.140.3: icmp_seq=0 ttl=128 time=1.491 ms > > 64 bytes from 95.175.140.3: icmp_seq=1 ttl=128 time=0.388 ms > > 64 bytes from 95.175.140.3: icmp_seq=2 ttl=128 time=0.399 ms > > 64 bytes from 95.175.140.3: icmp_seq=3 ttl=128 time=0.811 ms > > > > --- 95.175.140.3 ping statistics --- > > 4 packets transmitted, 4 packets received, 0.0% packet loss > > round-trip min/avg/max/stddev = 0.388/0.772/1.491/0.449 ms > > > > So, those two machines are connected to the same Ethernet and they can > ping > > each other. I turned off the firewall but the program still doesn't > work:( > > > > ======== > > To find more information about my network, I reopen the erlang system on > Mac > > with -name option > > (pong@REDACTED)1> > > inet:gethostbyaddr('95.175.140.5'). > > {ok,{hostent,"dyn-95-175-140-5.chaledi.unite.inuknetworks.com", > > ["5.140.175.95.in-addr.arpa"], > > inet,4, > > [{95,175,140,5}]}} > > (pong@REDACTED)2> > > net_adm:dns_hostname('dyn-95-175-140-5'). > > {ok,"dyn-95-175-140-5"} > > (pong@REDACTED)3> net_adm:dns_hostname(' > > dyn-95-175-140-5.inuknetworks.com'). > > {ok,"dyn-95-175-140-5.inuknetworks.com"} > > (pong@REDACTED)4> net_adm:dns_hostname(' > > dyn-95-175-140-3.inuknetworks.com'). > > {error,'dyn-95-175-140-3.inuknetworks.com'} > > > > Then I open erlang system on Windows with -name option... > > > > On Mac > > (pong@REDACTED)5> > > inet:gethostbyaddr('95.175.140.3'). > > {ok,{hostent,"dyn-95-175-140-3.chaledi.unite.inuknetworks.com", > > ["3.140.175.95.in-addr.arpa"], > > inet,4, > > [{95,175,140,3}]}} > > (pong@REDACTED)6> > > net_adm:dns_hostname('dyn-95-175-140-3'). > > {ok,"dyn-95-175-140-19"} > > (pong@REDACTED)7> net_adm:dns_hostname(' > > dyn-95-175-140-3.inuknetworks.com'). > > {ok,"dyn-95-175-140-3.inuknetworks.com"} > > (pong@REDACTED)8> net_adm:dns_hostname(' > > dyn-95-175-140-3.chaledi.unite.inuknetworks.com'). > > {ok,"dyn-95-175-140-3.chaledi.unite.inuknetworks.com"} > > > > It is clear that the erlang system could only recognize another host > where > > anthor erlang system is running. And, more importantly, I get host names > > that is known by my local DNS server. > > > > On Mac > > (pong@REDACTED)8> erlang:get_cookie(). > > 'erlang_cookie' > > > > On windows > > (ping@REDACTED)1> erlang:get_cookie(). > > 'erlang_cookie' > > > > It shows that the magic cookies are set correctly on both machines. > > > > on Mac > > (pong@REDACTED)9> pingpong:start_pong(). > > true > > (pong@REDACTED)10> net_adm:ping(' > > pong@REDACTED'). > > pong > > (pong@REDACTED)11> net_adm:ping(' > > pong@REDACTED'). > > pang > > (pong@REDACTED)12> > > net_adm:localhost(). "dyn-95-175-140-3.inuknetworks.com" > > > > It suggests that I should use hostname that has the form > > 'nodeName@REDACTED*.inuknetworks.com' > > > > on Windows > > (ping@REDACTED)2> net_adm:ping(' > > pong@REDACTED'). > > pang > > > > To summarize. my two computers are connected to the same Ethernet. The > > only difference between two IP addresses is the last digit. Firewalls > are > > closed on both machines. Computers can ping each other from the outside > of > > Erlang system. Inside the Erlang system, one machine could have at least > 3 > > valid host names from DNS server's point of view. However, only one of > them > > is valid for net_adm:ping/1, if Erlang is invoked with -name option. An > > Erlang system could detect another host where another Erlang system is > > running. Somehow, an Erlang system cannot communicate with an Erlang > node > > in another Erlang system. > > > > Any suggestions? > > > > > > Jiansen > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From cmcknight@REDACTED Wed Oct 6 17:57:28 2010 From: cmcknight@REDACTED (Charles McKnight) Date: Wed, 6 Oct 2010 11:57:28 -0400 Subject: Nested states in gen_fsm Message-ID: <5A01D349-6F4F-4997-A5A6-C45D21E5FC0A@pheonic.com> Hi All, I'm researching FSMs and wondered if anyone had an example of nested states using gen_fsm? I'm trying to demonstrate forking/joining of parallel processes from a "parent" process and cause a state transition to occur based on some subset of the "responses" from the parallel processes. Any examples appreciated. Best regards, Charles From tony.arcieri@REDACTED Wed Oct 6 19:58:09 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Wed, 6 Oct 2010 11:58:09 -0600 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: <838303.44876.qm@web112615.mail.gq1.yahoo.com> References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: On Sun, Oct 3, 2010 at 9:00 AM, Pablo Platt wrote: > In what areas erlang dominates and what areas will you consider using > node.js? > For example, would you consider building something like rabbitmq or > ejabberd in > node.js? Node makes you write everything in a fully asynchronous, event-driven manner, an upside down world of callbacks and inverted control. Erlang provides abstractions like processes and messages which allow things to appear synchronous when in fact they're fully asynchronous underneath. Erlang processes will hang on to state for you, so you can process incoming requests as a combination of the request and the current state, rather than having to backtrack and figure out what state you're in from a centralized I/O handler. Node may be nice for smaller servers that handle more trivial I/O problems. Erlang is better suited for building larger systems which are factored apart into several different processes. -- Tony Arcieri Medioh! A Kudelski Brand From raould@REDACTED Wed Oct 6 20:14:48 2010 From: raould@REDACTED (Raoul Duke) Date: Wed, 6 Oct 2010 11:14:48 -0700 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: On Wed, Oct 6, 2010 at 10:58 AM, Tony Arcieri wrote: > On Sun, Oct 3, 2010 at 9:00 AM, Pablo Platt wrote: >> In what areas erlang dominates and what areas will you consider using >> node.js? > Node may be nice for smaller servers that handle more trivial I/O problems. > Erlang is better suited for building larger systems which are factored apart > into several different processes. fundamentally, as a narrow-minded, often statically-typed-bigoted, jerk, i don't understand how anybody could see javascript and think "this is *so good* we *have* to extend its use to all other areas of programming endeavour!" i mean at least i get Dialyzer, QuickCheck in the Erlang world. :-) yeah, ok, all languages have problems, and javascript gets blamed for issues that are really in the DOM and the individual browsers, but even leaving those things aside, i still don't like it. :-) apologies, end of rant, shutting up. sincerely. From spawn.think@REDACTED Wed Oct 6 20:16:46 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Wed, 6 Oct 2010 20:16:46 +0200 Subject: [erlang-questions] Problems when running distributed pingpong In-Reply-To: References: <1286349768.5140.18.camel@seasc1137> Message-ID: It seems to me like a problem with your hosts file (on Mac /etc/hosts, on windows %systemroot%\system32\drivers\etc\ ) could you please post it here? On Wed, Oct 6, 2010 at 5:22 PM, Jiansen He wrote: > net:gethostbyaddr('95.175.140.20'). > -- Best Regards, - Ahmed http://nl.linkedin.com/in/adiaa From jiansenhe@REDACTED Wed Oct 6 20:53:26 2010 From: jiansenhe@REDACTED (Jiansen He) Date: Wed, 6 Oct 2010 19:53:26 +0100 Subject: [erlang-questions] Problems when running distributed pingpong In-Reply-To: References: <1286349768.5140.18.camel@seasc1137> Message-ID: Windows # Copyright (c) 1993-2009 Microsoft Corp. # # This is a sample HOSTS file used by Microsoft TCP/IP for Windows. # # This file contains the mappings of IP addresses to host names. Each # entry should be kept on an individual line. The IP address should # be placed in the first column followed by the corresponding host name. # The IP address and the host name should be separated by at least one # space. # # Additionally, comments (such as these) may be inserted on individual # lines or following the machine name denoted by a '#' symbol. # # For example: # # 102.54.94.97 rhino.acme.com # source server # 38.25.63.10 x.acme.com # x client host # localhost name resolution is handled within DNS itself. # 127.0.0.1 localhost # ::1 localhost Mac ## # Host Database # # localhost is used to configure the loopback interface # when the system is booting. Do not change this entry. ## 127.0.0.1 localhost 255.255.255.255 broadcasthost ::1 localhost fe80::1%lo0 localhost Shall I write something to the host file? Many thanks Jiansen On Wed, Oct 6, 2010 at 7:16 PM, Ahmed Omar wrote: > It seems to me like a problem with your hosts file (on Mac /etc/hosts, on > windows %systemroot%\system32\drivers\etc\ ) > > could you please post it here? > > On Wed, Oct 6, 2010 at 5:22 PM, Jiansen He wrote: > >> net:gethostbyaddr('95.175.140.20'). >> > > > > -- > Best Regards, > - Ahmed > http://nl.linkedin.com/in/adiaa > From spawn.think@REDACTED Wed Oct 6 21:04:34 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Wed, 6 Oct 2010 21:04:34 +0200 Subject: [erlang-questions] Problems when running distributed pingpong In-Reply-To: References: <1286349768.5140.18.camel@seasc1137> Message-ID: Yes, you could try adding both machines IPs/hostnames combinations there so something like : 95.175.140.5 dyn-95-175-140-3 dyn-95-175-140-3.inuknetworks.com 95.175.140.3 dyn-95-175-140-5 dyn-95-175-140-5.inuknetworks.com On Wed, Oct 6, 2010 at 8:53 PM, Jiansen He wrote: > > Windows > > # Copyright (c) 1993-2009 Microsoft Corp. > # > # This is a sample HOSTS file used by Microsoft TCP/IP for Windows. > # > # This file contains the mappings of IP addresses to host names. Each > # entry should be kept on an individual line. The IP address should > # be placed in the first column followed by the corresponding host name. > # The IP address and the host name should be separated by at least one > # space. > # > # Additionally, comments (such as these) may be inserted on individual > # lines or following the machine name denoted by a '#' symbol. > # > # For example: > # > # 102.54.94.97 rhino.acme.com # source server > # 38.25.63.10 x.acme.com # x client host > > # localhost name resolution is handled within DNS itself. > # 127.0.0.1 localhost > # ::1 localhost > > > Mac > > ## > # Host Database > # > # localhost is used to configure the loopback interface > # when the system is booting. Do not change this entry. > ## > 127.0.0.1 localhost > 255.255.255.255 broadcasthost > ::1 localhost > fe80::1%lo0 localhost > > > Shall I write something to the host file? > > Many thanks > Jiansen > > > > > > On Wed, Oct 6, 2010 at 7:16 PM, Ahmed Omar wrote: > >> It seems to me like a problem with your hosts file (on Mac /etc/hosts, on >> windows %systemroot%\system32\drivers\etc\ ) >> >> could you please post it here? >> >> On Wed, Oct 6, 2010 at 5:22 PM, Jiansen He wrote: >> >>> net:gethostbyaddr('95.175.140.20'). >>> >> >> >> >> -- >> Best Regards, >> - Ahmed >> http://nl.linkedin.com/in/adiaa >> > > -- Best Regards, - Ahmed http://nl.linkedin.com/in/adiaa From tony.arcieri@REDACTED Wed Oct 6 21:47:54 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Wed, 6 Oct 2010 13:47:54 -0600 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: On Wed, Oct 6, 2010 at 12:14 PM, Raoul Duke wrote: > fundamentally, as a narrow-minded, often statically-typed-bigoted, > jerk, i don't understand how anybody could see javascript and think > "this is *so good* we *have* to extend its use to all other areas of > programming endeavour!" The interesting thing about JavaScript is people are used to using it in an asynchronous, callback-driven manner where time is spent outside the interpreter doing other things, namely the web browser waiting on events for the JavaScript interpreter. Because of this, JavaScript is a "natural" environment for the sort of asynchronous, callback-driven programming that Node offers. I am not a fan of JavaScript as a language in general, though. -- Tony Arcieri Medioh! A Kudelski Brand From spawn.think@REDACTED Wed Oct 6 23:25:21 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Wed, 6 Oct 2010 23:25:21 +0200 Subject: [erlang-questions] Gracefully timeout during erlang:port_call/3? In-Reply-To: References: Message-ID: Not sure if there is a graceful way , but you can spawn the call in a process that will be terminated after some timeout. This will terminate the calling process, but if it will affect the port driver depends i guess it depends on the port timer:kill_after(Timeout, spawn(erlang, port_call, [Port, Operation, Data])). On Mon, Oct 4, 2010 at 1:58 AM, zabrane Mikael wrote: > Hi guys, > > Is there a way to gracefully timeout (i.e without crashing the port > driver) during the synchronous call to "erlang:port_call/3"? > The idea here is to try to timeout if the port takes long time to return. > > Any advice is welcome? > -- > Regards > Zabrane > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From jiansenhe@REDACTED Thu Oct 7 01:19:55 2010 From: jiansenhe@REDACTED (Jiansen He) Date: Thu, 7 Oct 2010 00:19:55 +0100 Subject: [erlang-questions] Problems when running distributed pingpong In-Reply-To: References: <1286349768.5140.18.camel@seasc1137> Message-ID: I tried adding both lines to both machines. It still doesn't work. Besides, can I bind two host names to the same IP in the same line? It doesn't looks like the form of sample binding. In addition, instead of changing the hosts file, could I "locally" change the IPs/hostnames binding inside erlang? Regards Jiansen On Wed, Oct 6, 2010 at 8:04 PM, Ahmed Omar wrote: > Yes, you could try adding both machines IPs/hostnames combinations there > so something like : > 95.175.140.5 dyn-95-175-140-3 dyn-95-175-140-3.inuknetworks.com > > 95.175.140.3 dyn-95-175-140-5 dyn-95-175-140-5.inuknetworks.com > > > On Wed, Oct 6, 2010 at 8:53 PM, Jiansen He wrote: > >> >> Windows >> >> # Copyright (c) 1993-2009 Microsoft Corp. >> # >> # This is a sample HOSTS file used by Microsoft TCP/IP for Windows. >> # >> # This file contains the mappings of IP addresses to host names. Each >> # entry should be kept on an individual line. The IP address should >> # be placed in the first column followed by the corresponding host name. >> # The IP address and the host name should be separated by at least one >> # space. >> # >> # Additionally, comments (such as these) may be inserted on individual >> # lines or following the machine name denoted by a '#' symbol. >> # >> # For example: >> # >> # 102.54.94.97 rhino.acme.com # source server >> # 38.25.63.10 x.acme.com # x client host >> >> # localhost name resolution is handled within DNS itself. >> # 127.0.0.1 localhost >> # ::1 localhost >> >> >> Mac >> >> ## >> # Host Database >> # >> # localhost is used to configure the loopback interface >> # when the system is booting. Do not change this entry. >> ## >> 127.0.0.1 localhost >> 255.255.255.255 broadcasthost >> ::1 localhost >> fe80::1%lo0 localhost >> >> >> Shall I write something to the host file? >> >> Many thanks >> Jiansen >> >> >> >> >> >> On Wed, Oct 6, 2010 at 7:16 PM, Ahmed Omar wrote: >> >>> It seems to me like a problem with your hosts file (on Mac /etc/hosts, on >>> windows %systemroot%\system32\drivers\etc\ ) >>> >>> could you please post it here? >>> >>> On Wed, Oct 6, 2010 at 5:22 PM, Jiansen He wrote: >>> >>>> net:gethostbyaddr('95.175.140.20'). >>>> >>> >>> >>> >>> -- >>> Best Regards, >>> - Ahmed >>> http://nl.linkedin.com/in/adiaa >>> >> >> > > > -- > Best Regards, > - Ahmed > http://nl.linkedin.com/in/adiaa > From bengt.kleberg@REDACTED Thu Oct 7 07:43:34 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 07 Oct 2010 07:43:34 +0200 Subject: [erlang-questions] Problems when running distributed pingpong In-Reply-To: References: <1286349768.5140.18.camel@seasc1137> Message-ID: <1286430214.5141.5.camel@seasc1137> I do not know what you are allowed to do on Windows, but the /etc/hosts file can handle multiple host names after after the IP address. Like this: IP_address canonical_hostname [aliases...] Just to check, but you do restart erlang when changing the files? bengt On Thu, 2010-10-07 at 01:19 +0200, Jiansen He wrote: > I tried adding both lines to both machines. It still doesn't work. > Besides, can I bind two host names to the same IP in the same line? It > doesn't looks like the form of sample binding. > > In addition, instead of changing the hosts file, could I "locally" change > the IPs/hostnames binding inside erlang? > > Regards > Jiansen > > > > On Wed, Oct 6, 2010 at 8:04 PM, Ahmed Omar wrote: > > > Yes, you could try adding both machines IPs/hostnames combinations there > > so something like : > > 95.175.140.5 dyn-95-175-140-3 dyn-95-175-140-3.inuknetworks.com > > > > 95.175.140.3 dyn-95-175-140-5 dyn-95-175-140-5.inuknetworks.com > > > > > > On Wed, Oct 6, 2010 at 8:53 PM, Jiansen He wrote: > > > >> > >> Windows > >> > >> # Copyright (c) 1993-2009 Microsoft Corp. > >> # > >> # This is a sample HOSTS file used by Microsoft TCP/IP for Windows. > >> # > >> # This file contains the mappings of IP addresses to host names. Each > >> # entry should be kept on an individual line. The IP address should > >> # be placed in the first column followed by the corresponding host name. > >> # The IP address and the host name should be separated by at least one > >> # space. > >> # > >> # Additionally, comments (such as these) may be inserted on individual > >> # lines or following the machine name denoted by a '#' symbol. > >> # > >> # For example: > >> # > >> # 102.54.94.97 rhino.acme.com # source server > >> # 38.25.63.10 x.acme.com # x client host > >> > >> # localhost name resolution is handled within DNS itself. > >> # 127.0.0.1 localhost > >> # ::1 localhost > >> > >> > >> Mac > >> > >> ## > >> # Host Database > >> # > >> # localhost is used to configure the loopback interface > >> # when the system is booting. Do not change this entry. > >> ## > >> 127.0.0.1 localhost > >> 255.255.255.255 broadcasthost > >> ::1 localhost > >> fe80::1%lo0 localhost > >> > >> > >> Shall I write something to the host file? > >> > >> Many thanks > >> Jiansen > >> > >> > >> > >> > >> > >> On Wed, Oct 6, 2010 at 7:16 PM, Ahmed Omar wrote: > >> > >>> It seems to me like a problem with your hosts file (on Mac /etc/hosts, on > >>> windows %systemroot%\system32\drivers\etc\ ) > >>> > >>> could you please post it here? > >>> > >>> On Wed, Oct 6, 2010 at 5:22 PM, Jiansen He wrote: > >>> > >>>> net:gethostbyaddr('95.175.140.20'). > >>>> > >>> > >>> > >>> > >>> -- > >>> Best Regards, > >>> - Ahmed > >>> http://nl.linkedin.com/in/adiaa > >>> > >> > >> > > > > > > -- > > Best Regards, > > - Ahmed > > http://nl.linkedin.com/in/adiaa > > From zabrane3@REDACTED Thu Oct 7 08:12:27 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Thu, 7 Oct 2010 08:12:27 +0200 Subject: [erlang-questions] Gracefully timeout during erlang:port_call/3? In-Reply-To: References: Message-ID: 2010/10/6 Ahmed Omar : > Not sure if there is a graceful way , but you can spawn the call in a > process that will be terminated after some timeout. That's the simple way to do it. > This will terminate the calling process, but if it will affect the port driver depends i guess it > depends on the port > timer:kill_after(Timeout, spawn(erlang, port_call, [Port, Operation, Data])). But this can crash my port program. Any other advice guys? -- Regards Zabrane > > On Mon, Oct 4, 2010 at 1:58 AM, zabrane Mikael wrote: >> >> Hi guys, >> >> Is there a way to gracefully timeout (i.e without crashing the port >> driver) during the synchronous call to "erlang:port_call/3"? >> The idea here is to try to timeout if the port takes long time to return. >> >> Any advice is welcome? >> -- >> Regards >> Zabrane >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > From raimo+erlang-questions@REDACTED Thu Oct 7 10:17:28 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 7 Oct 2010 10:17:28 +0200 Subject: [erlang-questions] Problems when running distributed pingpong In-Reply-To: References: <1286349768.5140.18.camel@seasc1137> Message-ID: <20101007081728.GA21316@erix.ericsson.se> Hi. Regarding name resolution Windows and MacOS X are two rather irregular platforms for Erlang. Congratulations. There are lots of tricks you can do by configuring Erlang's name resolution to compensate for environment glitches. See: http://www.erlang.org/doc/apps/erts/inet_cfg.html What often happens when you take this path is that all other tools in the OS thinks differently about name resolution and in a product that can be a problem. Changing the hosts file is a way to reconfigure the native resolver for the whole machine but only that machine. When more than one machine is involved you probably want to use what is in their common name resolving service. This is a trick that should work using strange longnames: erl -name pong@REDACTED erl -name ping@REDACTED 1> net_adm:ping('pong@REDACTED'). You should also be able to use the working name like this: erl -name pong@REDACTED erl -name ping@REDACTED 1> net_adm:ping('pong@REDACTED'). Both sides of the connection must aggree on their hostnames, and the hostname must work for creating the connection. To help debug your problem at hand the output of these is of interest: 2> io:format("~p.~n", [ets:tab2list(inet_db)]). 3> io:format("~p.~n", [ets:tab2list(inet_hosts_file_byname)]). 4> io:format("~p.~n", [ets:tab2list(inet_hosts_byname)]). 5> inet_gethost_native:gethostbyname("dyn-95-175-140-20"). 6> inet_gethost_native:gethostbyname("dyn-95-175-140-20.inuknetworks.com"). 7> inet_gethost_native:gethostbyname("dyn-95-175-140-20.chaledi.unite.inuknetworks.com"). On Thu, Oct 07, 2010 at 12:19:55AM +0100, Jiansen He wrote: > I tried adding both lines to both machines. It still doesn't work. > Besides, can I bind two host names to the same IP in the same line? It > doesn't looks like the form of sample binding. > > In addition, instead of changing the hosts file, could I "locally" change > the IPs/hostnames binding inside erlang? > > Regards > Jiansen > > > > On Wed, Oct 6, 2010 at 8:04 PM, Ahmed Omar wrote: > > > Yes, you could try adding both machines IPs/hostnames combinations there > > so something like : > > 95.175.140.5 dyn-95-175-140-3 dyn-95-175-140-3.inuknetworks.com > > > > 95.175.140.3 dyn-95-175-140-5 dyn-95-175-140-5.inuknetworks.com > > > > > > On Wed, Oct 6, 2010 at 8:53 PM, Jiansen He wrote: > > > >> > >> Windows > >> > >> # Copyright (c) 1993-2009 Microsoft Corp. > >> # > >> # This is a sample HOSTS file used by Microsoft TCP/IP for Windows. > >> # > >> # This file contains the mappings of IP addresses to host names. Each > >> # entry should be kept on an individual line. The IP address should > >> # be placed in the first column followed by the corresponding host name. > >> # The IP address and the host name should be separated by at least one > >> # space. > >> # > >> # Additionally, comments (such as these) may be inserted on individual > >> # lines or following the machine name denoted by a '#' symbol. > >> # > >> # For example: > >> # > >> # 102.54.94.97 rhino.acme.com # source server > >> # 38.25.63.10 x.acme.com # x client host > >> > >> # localhost name resolution is handled within DNS itself. > >> # 127.0.0.1 localhost > >> # ::1 localhost > >> > >> > >> Mac > >> > >> ## > >> # Host Database > >> # > >> # localhost is used to configure the loopback interface > >> # when the system is booting. Do not change this entry. > >> ## > >> 127.0.0.1 localhost > >> 255.255.255.255 broadcasthost > >> ::1 localhost > >> fe80::1%lo0 localhost > >> > >> > >> Shall I write something to the host file? > >> > >> Many thanks > >> Jiansen > >> > >> > >> > >> > >> > >> On Wed, Oct 6, 2010 at 7:16 PM, Ahmed Omar wrote: > >> > >>> It seems to me like a problem with your hosts file (on Mac /etc/hosts, on > >>> windows %systemroot%\system32\drivers\etc\ ) > >>> > >>> could you please post it here? > >>> > >>> On Wed, Oct 6, 2010 at 5:22 PM, Jiansen He wrote: > >>> > >>>> net:gethostbyaddr('95.175.140.20'). > >>>> > >>> > >>> > >>> > >>> -- > >>> Best Regards, > >>> - Ahmed > >>> http://nl.linkedin.com/in/adiaa > >>> > >> > >> > > > > > > -- > > Best Regards, > > - Ahmed > > http://nl.linkedin.com/in/adiaa > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From erlang@REDACTED Thu Oct 7 10:23:29 2010 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 7 Oct 2010 10:23:29 +0200 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: A thought struck me - people ask "what's the different between yaws and apache" or "erlang and node.js" It's the difference between having one web server and a few hundred thousand. An erlang web server will spawn a single process per connection - each process is in itself a complete little web server - if you are running mochiweb with 100K chat sessions you really have 100K webservers on your machine - not one. If one of these web servers crashes who cares. If Apache crashes it's a big deal - if an erlang process representing a web server crashes it's not a big deal. Erlang itself crashing (or yaws etc) is more akin to an entire machine failure. Think of an erlang system as providing hundreds of thousands of tiny isolated web servers. This argument is not entirely true and should not be pushed to far, but it does illustrate an essential difference in philosophy. /Joe On Wed, Oct 6, 2010 at 7:58 PM, Tony Arcieri wrote: > On Sun, Oct 3, 2010 at 9:00 AM, Pablo Platt wrote: > >> In what areas erlang dominates and what areas will you consider using >> node.js? >> For example, would you consider building something like rabbitmq or >> ejabberd in >> node.js? > > > Node makes you write everything in a fully asynchronous, event-driven > manner, an upside down world of callbacks and inverted control. Erlang > provides abstractions like processes and messages which allow things to > appear synchronous when in fact they're fully asynchronous underneath. > Erlang processes will hang on to state for you, so you can process incoming > requests as a combination of the request and the current state, rather than > having to backtrack and figure out what state you're in from a centralized > I/O handler. > > Node may be nice for smaller servers that handle more trivial I/O problems. > Erlang is better suited for building larger systems which are factored apart > into several different processes. > > -- > Tony Arcieri > Medioh! A Kudelski Brand > From max.lapshin@REDACTED Thu Oct 7 10:25:44 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 7 Oct 2010 12:25:44 +0400 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: In fact, erlang is not about performance, or compatibility with hundreds of thousands of legacy code lines. It is about management resources in runtime. Best way to manage resources in runtime. From kenneth.lundin@REDACTED Thu Oct 7 12:39:12 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Thu, 7 Oct 2010 12:39:12 +0200 Subject: [ANN] Erlang R14B bugfix (for Windows only) is now available for download Message-ID: Due to a number of users reporting problems with Erlang R14B on certain versions of Windows we now release a completely new prebuilt Windows self installable package. The corrected Windows package as well as a complete source package is available at http://erlang.org/download Please read the Readme below carefully. Bug fix release : otp_src_R14B_erts-5.8.1.1 Build date : 2010-10-06 This is a correction package for windows only. No other platform is affected. --- erts-5.8.1.1 -------------------------------------------------------------- OTP-8876 Windows 2003 and Windows XP pre SP3 would sometimes not start the Erlang R14B VM at all due to a bug in the cpu topology detection. The bug affects Windows only, no other platform is even remotely affected. The bug is now corrected. /Regards Kenneth, Erlang/OTP Ericsson From zvi.avraham@REDACTED Thu Oct 7 14:01:47 2010 From: zvi.avraham@REDACTED (Zvi) Date: Thu, 7 Oct 2010 05:01:47 -0700 (PDT) Subject: What's the most efficient method to broadcast a message to multiple (100Ks) of processes? Message-ID: <1a8c7fff-d999-49ba-b34b-cfa9c1868db9@c10g2000yqh.googlegroups.com> Hi, what's the most efficient method to broadcast a message to multiple (100Ks) of processes? The most naive way is: [ Pid ! Msg || Pid<-Pids ]. less naive method: [ spawn(fun() -> [Pid ! Msg || Pid<-PidsSublist] end) || PidsSublist <- partition(Pids, N) ]. where partition(L,N) splits list L into N sublists. Any ideas for better method? thanks, Zvi From joelr1@REDACTED Thu Oct 7 14:11:54 2010 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 7 Oct 2010 20:11:54 +0800 Subject: [erlang-questions] What's the most efficient method to broadcast a message to multiple (100Ks) of processes? In-Reply-To: <1a8c7fff-d999-49ba-b34b-cfa9c1868db9@c10g2000yqh.googlegroups.com> References: <1a8c7fff-d999-49ba-b34b-cfa9c1868db9@c10g2000yqh.googlegroups.com> Message-ID: http://groups.google.com/group/erlang-programming/browse_thread/thread/1931368998000836/b325e869a3eea26a Basically, raise priority to high around your sending loop and then get it back to normal right after. On Oct 7, 2010, at 8:01 PM, Zvi wrote: > Hi, > > what's the most efficient method to broadcast a message to multiple > (100Ks) of processes? > > The most naive way is: > > [ Pid ! Msg || Pid<-Pids ]. > > less naive method: > > [ spawn(fun() -> [Pid ! Msg || Pid<-PidsSublist] end) || > PidsSublist <- partition(Pids, N) ]. > > where partition(L,N) splits list L into N sublists. > > Any ideas for better method? --- http://twitter.com/wagerlabs From jiansenhe@REDACTED Thu Oct 7 15:03:10 2010 From: jiansenhe@REDACTED (Jiansen He) Date: Thu, 7 Oct 2010 14:03:10 +0100 Subject: [erlang-questions] Problems when running distributed pingpong In-Reply-To: <20101007081728.GA21316@erix.ericsson.se> References: <1286349768.5140.18.camel@seasc1137> <20101007081728.GA21316@erix.ericsson.se> Message-ID: Thank you so much. Your trick works. It is simple and useful! Jiansen On Thu, Oct 7, 2010 at 9:17 AM, Raimo Niskanen < raimo+erlang-questions@REDACTED > wrote: > Hi. > > Regarding name resolution Windows and MacOS X are two rather > irregular platforms for Erlang. Congratulations. > > There are lots of tricks you can do by configuring Erlang's name > resolution to compensate for environment glitches. See: > http://www.erlang.org/doc/apps/erts/inet_cfg.html > What often happens when you take this path is that all other > tools in the OS thinks differently about name resolution > and in a product that can be a problem. > > Changing the hosts file is a way to reconfigure the > native resolver for the whole machine but only that machine. > When more than one machine is involved you probably want > to use what is in their common name resolving service. > > This is a trick that should work using strange longnames: > > erl -name pong@REDACTED > > erl -name ping@REDACTED > > 1> net_adm:ping('pong@REDACTED'). > > > You should also be able to use the working name like this: > > erl -name pong@REDACTED > > erl -name ping@REDACTED > > 1> net_adm:ping('pong@REDACTED'). > > > Both sides of the connection must aggree on their hostnames, > and the hostname must work for creating the connection. > > > > To help debug your problem at hand the output of these is of interest: > 2> io:format("~p.~n", [ets:tab2list(inet_db)]). > 3> io:format("~p.~n", [ets:tab2list(inet_hosts_file_byname)]). > 4> io:format("~p.~n", [ets:tab2list(inet_hosts_byname)]). > 5> inet_gethost_native:gethostbyname("dyn-95-175-140-20"). > 6> inet_gethost_native:gethostbyname("dyn-95-175-140-20.inuknetworks.com > "). > 7> inet_gethost_native:gethostbyname(" > dyn-95-175-140-20.chaledi.unite.inuknetworks.com"). > > > On Thu, Oct 07, 2010 at 12:19:55AM +0100, Jiansen He wrote: > > I tried adding both lines to both machines. It still doesn't work. > > Besides, can I bind two host names to the same IP in the same line? It > > doesn't looks like the form of sample binding. > > > > In addition, instead of changing the hosts file, could I "locally" change > > the IPs/hostnames binding inside erlang? > > > > Regards > > Jiansen > > > > > > > > On Wed, Oct 6, 2010 at 8:04 PM, Ahmed Omar > wrote: > > > > > Yes, you could try adding both machines IPs/hostnames combinations > there > > > so something like : > > > 95.175.140.5 dyn-95-175-140-3 > dyn-95-175-140-3.inuknetworks.com > > > > > > 95.175.140.3 dyn-95-175-140-5 > dyn-95-175-140-5.inuknetworks.com > > > > > > > > > On Wed, Oct 6, 2010 at 8:53 PM, Jiansen He >wrote: > > > > > >> > > >> Windows > > >> > > >> # Copyright (c) 1993-2009 Microsoft Corp. > > >> # > > >> # This is a sample HOSTS file used by Microsoft TCP/IP for Windows. > > >> # > > >> # This file contains the mappings of IP addresses to host names. Each > > >> # entry should be kept on an individual line. The IP address should > > >> # be placed in the first column followed by the corresponding host > name. > > >> # The IP address and the host name should be separated by at least one > > >> # space. > > >> # > > >> # Additionally, comments (such as these) may be inserted on individual > > >> # lines or following the machine name denoted by a '#' symbol. > > >> # > > >> # For example: > > >> # > > >> # 102.54.94.97 rhino.acme.com # source server > > >> # 38.25.63.10 x.acme.com # x client host > > >> > > >> # localhost name resolution is handled within DNS itself. > > >> # 127.0.0.1 localhost > > >> # ::1 localhost > > >> > > >> > > >> Mac > > >> > > >> ## > > >> # Host Database > > >> # > > >> # localhost is used to configure the loopback interface > > >> # when the system is booting. Do not change this entry. > > >> ## > > >> 127.0.0.1 localhost > > >> 255.255.255.255 broadcasthost > > >> ::1 localhost > > >> fe80::1%lo0 localhost > > >> > > >> > > >> Shall I write something to the host file? > > >> > > >> Many thanks > > >> Jiansen > > >> > > >> > > >> > > >> > > >> > > >> On Wed, Oct 6, 2010 at 7:16 PM, Ahmed Omar > wrote: > > >> > > >>> It seems to me like a problem with your hosts file (on Mac > /etc/hosts, on > > >>> windows %systemroot%\system32\drivers\etc\ ) > > >>> > > >>> could you please post it here? > > >>> > > >>> On Wed, Oct 6, 2010 at 5:22 PM, Jiansen He >wrote: > > >>> > > >>>> net:gethostbyaddr('95.175.140.20'). > > >>>> > > >>> > > >>> > > >>> > > >>> -- > > >>> Best Regards, > > >>> - Ahmed > > >>> http://nl.linkedin.com/in/adiaa > > >>> > > >> > > >> > > > > > > > > > -- > > > Best Regards, > > > - Ahmed > > > http://nl.linkedin.com/in/adiaa > > > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > From ttom.kelly@REDACTED Thu Oct 7 16:07:10 2010 From: ttom.kelly@REDACTED (tom kelly) Date: Thu, 7 Oct 2010 15:07:10 +0100 Subject: Garbage collecting binaries Message-ID: Hello List, I need some help on garbage collecting binaries. I have an application that handles large binaries and under load it eats up all the available memory then falls over, even when I start all the data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}". Reading point 5.15 on the page: http://www.erlang.org/faq/how_do_i.html leads me to think that calling the garbage collector on the process that created a binary will clean it up but my shell experiment below shows me I'm wrong. It's now 10 minutes since I've done this experiment and the large binary is still in memory. Maybe I have to call the garbage collector of the process whose heap is storing the binary? If so, which process is it? Any pointers as to what I'm doing wrong or mis-understand will be greatly appreciated. I'm on a legacy R12B5 system. //Tom. 18> process_info(self(),total_heap_size). {total_heap_size,4181} 19> A = L(100000). [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118, 180,121,181,160,124,244,140,169,215,31,82,43|...] 20> process_info(self(),total_heap_size). {total_heap_size,1149851} 21> f(A). ok 22> erlang:garbage_collect(). true 23> process_info(self(),total_heap_size). {total_heap_size,3194} 24> memory(binary). 6536 25> A = B(100000). <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181, 218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>> 26> memory(binary). 106704 27> f(A). ok 28> memory(binary). 106704 29> erlang:garbage_collect(). true 30> memory(binary). 106560 From dmercer@REDACTED Thu Oct 7 16:13:23 2010 From: dmercer@REDACTED (David Mercer) Date: Thu, 7 Oct 2010 09:13:23 -0500 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: <838303.44876.qm@web112615.mail.gq1.yahoo.com> References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: <00a601cb6629$ce8eabe0$6bac03a0$@com> I might be the only person in the world whose two favorite languages are Javascript and Erlang. I had been writing full-blown server-side Javascript applications for 5 or 6 years until I discovered Erlang in '07. What had attracted me to Javascript was the ability to replace code on the fly (for instance, by loading the file into a string and eval-ing it). For distribution (the application was distributed over about 20 machines internally with about 100 additional smaller-scale remote servers communicating with the central hub), I used the file system and TCP (and HTTP for negotiating across firewalls), depending on the requirements. (I had also originally used MSMQ, but we phased that out over time in favor of more expedient methods that we developed ourselves.) The end result was a less robust version of an Erlang application, that worked pretty well. We had hot code loading, and redundancy gave us the fault-tolerance we required. When I discovered Erlang in 2007 (not sure why it took me that long), I immediately recognized that Erlang was built to handle all the problems we had written around, but it had a longer history and was certainly better at it. We had required so much of our redundancy (20 machines running about 7 processes each) not only to handle the load, but also to handle failures. Windows Script Host, which was our Javascript engine, is not as stable as you might think, and seemed to suffer from memory leaks which manifested itself over time. With 20 machines times 7 processes per machine and an MTBF of, say, a week, we had 10 or 20 failures a day (which were automatically handled). Honestly, the surprise for us was that WSH performed as well as it did. We had originally written our application in Javascript/WSH as a prototype, intending to rewrite in C/C++/Java/something else when we had completed our proof of concept. But as it turned out, processing speed wasn't really an issue for us -- network latency was usually the limiting issue, not CPU. However, it was surprising to us that WSH could, at times, run for days or weeks, processing millions of transactions before failing. WSH's multi-threading capabilities are nonexistent, and I tried various hacks to coax it into running multiple threads, but they tended to cause other problems (as you can imagine), so I wrote my own scheduler that time-sliced between various tasks. (It was actually a heap-based priority queue.) We isolated blocking operations (the main issue we had with blocking was that some asynchronous SQL queries were not working right, so we had to do them synchronously) in their own process so that they wouldn't block other tasks. However, Erlang does everything we wrote, and it does it better. Having written an equivalent system in a non-Erlang language, I really appreciate all the tools Erlang gives you to create manageable distributed server-side applications. Besides the interprocess messaging and hot code-loading, OTP is an excellent way of thinking about and framework for building your fault-tolerant servers. I did not know about Erlang and OTP when I started writing server-side Javascript, but if I had, I would have begun by rewriting Erlang and OTP (e.g., gen_server) in Javascript. It would have created a much better architecture, though it is not clear why I would write such a system in Javascript if I knew about Erlang. Node.js is only a small step up from WSH in that it is more portable and has better support for multi-tasking. I suspect you would find yourself rewriting a lot of Erlang/OTP in Javascript if you decided to build in Node.js your own server-side applications. That being said, as an old Javascript warrior myself, I'd be interested to hear how it goes. I, too, have considered trying out Node.js as my New Favorite Platform(TM), but haven't yet had the need. Today, I haven't done much programming in Javascript in a couple of years, but I gotta admit, as a language, it is very highly capable. I daresay, there was a time, 5-10 years ago, when I considered myself one of the best Javascript programmers in the world, and I would sometimes log into Experts Exchange or somewhere like that to help out, only to be dismayed that the Javascript questions were not about the language Javascript, but about manipulating the DOM in a web page. No-one ever seemed to ask the tricky questions about closures or object context, or performance of certain approaches etc. that was my expertise. You see more of that nowadays, as most of these concepts have gained traction in the mainstream; now there are probably tens of thousands of people with that knowledge, not just the hundreds of a few years ago. Thank-you for this walk down memory lane. Let me know how it goes with Node.js. I just might have to try it out... Cheers, DBM > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Pablo Platt > Sent: Sunday, October 03, 2010 10:01 AM > To: erlang-questions@REDACTED > Subject: [erlang-questions] node.js compared to erlang > > Hi > > I'm using erlang and I love it. > I'm not trying to create an argument but to better understand the power of > erlang. > > In what areas erlang dominates and what areas will you consider using > node.js? > For example, would you consider building something like rabbitmq or > ejabberd in > node.js? > Or maybe you'll use node.js just for a simple single chat room but erlang > for > anything complicated than that. > > Are there a fundamental differences in performance or stability or use in > distributed systems? > > Ulf Wiger commented on the comparison of erlang and node.js > and said that erlang solves the problem of non blocking functions which > might be > very hard for other languages. > http://journal.dedasys.com/2010/04/29/erlang-vs-node-js > > Thanks > > > From zeno490@REDACTED Thu Oct 7 16:57:19 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Thu, 7 Oct 2010 10:57:19 -0400 Subject: [erlang-questions] Garbage collecting binaries In-Reply-To: References: Message-ID: A bit hard to understand what is happening without seeing f(A)'s code. AFAIK, even if you don't keep a reference to a binary, if you keep a reference to a binary created from it, it'll keep it in full in memory. ie: A = << some large binary>>. <> = A. If you keep no references to A but you do to B, A will remain in memory because B will have been created as a pointer offset from A (as an optimization). Nicholas On Thu, Oct 7, 2010 at 10:07 AM, tom kelly wrote: > Hello List, > > I need some help on garbage collecting binaries. > I have an application that handles large binaries and under load it eats up > all the available memory then falls over, even when I start all the > data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}". > > Reading point 5.15 on the page: > http://www.erlang.org/faq/how_do_i.html > leads me to think that calling the garbage collector on the process that > created a binary will clean it up but my shell experiment below shows me > I'm > wrong. It's now 10 minutes since I've done this experiment and the large > binary is still in memory. > > Maybe I have to call the garbage collector of the process whose heap is > storing the binary? If so, which process is it? > > Any pointers as to what I'm doing wrong or mis-understand will be greatly > appreciated. > > I'm on a legacy R12B5 system. > > //Tom. > > > > 18> process_info(self(),total_heap_size). > {total_heap_size,4181} > > 19> A = L(100000). > [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118, > 180,121,181,160,124,244,140,169,215,31,82,43|...] > > 20> process_info(self(),total_heap_size). > {total_heap_size,1149851} > > 21> f(A). > ok > > 22> erlang:garbage_collect(). > true > > 23> process_info(self(),total_heap_size). > {total_heap_size,3194} > > > 24> memory(binary). > 6536 > > 25> A = B(100000). > <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181, > 218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>> > > 26> memory(binary). > 106704 > > 27> f(A). > ok > > 28> memory(binary). > 106704 > > 29> erlang:garbage_collect(). > true > > 30> memory(binary). > 106560 > From ttom.kelly@REDACTED Thu Oct 7 17:07:55 2010 From: ttom.kelly@REDACTED (tom kelly) Date: Thu, 7 Oct 2010 16:07:55 +0100 Subject: [erlang-questions] Garbage collecting binaries In-Reply-To: References: Message-ID: Hi Nicholas, This was just run in the erlang shell, A was a 100000 byte binary, "f(A)." told the shell to forget it, ie. set the reference count to zero. There were no other references to A in this case nor were other binaries created from it. It was gc'd after a while, presumably after 65535 reductions of the process that stored it (if that makes sense). My question is how can I reduce the fullsweep_after or explicitly force a gc on the heap where it is stored. //Tom. On Thu, Oct 7, 2010 at 3:57 PM, Nicholas Frechette wrote: > A bit hard to understand what is happening without seeing f(A)'s code. > AFAIK, even if you don't keep a reference to a binary, if you keep a > reference to a binary created from it, it'll keep it in full in memory. > ie: > A = << some large binary>>. > <> = A. > If you keep no references to A but you do to B, A will remain in memory > because B will have been created as a pointer offset from A (as an > optimization). > > Nicholas > > > On Thu, Oct 7, 2010 at 10:07 AM, tom kelly wrote: > >> Hello List, >> >> I need some help on garbage collecting binaries. >> I have an application that handles large binaries and under load it eats >> up >> all the available memory then falls over, even when I start all the >> data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}". >> >> Reading point 5.15 on the page: >> http://www.erlang.org/faq/how_do_i.html >> leads me to think that calling the garbage collector on the process that >> created a binary will clean it up but my shell experiment below shows me >> I'm >> wrong. It's now 10 minutes since I've done this experiment and the large >> binary is still in memory. >> >> Maybe I have to call the garbage collector of the process whose heap is >> storing the binary? If so, which process is it? >> >> Any pointers as to what I'm doing wrong or mis-understand will be greatly >> appreciated. >> >> I'm on a legacy R12B5 system. >> >> //Tom. >> >> >> >> 18> process_info(self(),total_heap_size). >> {total_heap_size,4181} >> >> 19> A = L(100000). >> [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118, >> 180,121,181,160,124,244,140,169,215,31,82,43|...] >> >> 20> process_info(self(),total_heap_size). >> {total_heap_size,1149851} >> >> 21> f(A). >> ok >> >> 22> erlang:garbage_collect(). >> true >> >> 23> process_info(self(),total_heap_size). >> {total_heap_size,3194} >> >> >> 24> memory(binary). >> 6536 >> >> 25> A = B(100000). >> <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181, >> 218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>> >> >> 26> memory(binary). >> 106704 >> >> 27> f(A). >> ok >> >> 28> memory(binary). >> 106704 >> >> 29> erlang:garbage_collect(). >> true >> >> 30> memory(binary). >> 106560 >> > > From ulf.wiger@REDACTED Thu Oct 7 17:34:49 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 7 Oct 2010 17:34:49 +0200 Subject: [erlang-questions] Garbage collecting binaries In-Reply-To: References: Message-ID: f(A) only unbinds the variable in the shell. The shell still has a history of commands and their results, so you would need to ensure that this is not taking up space as well. BR, Ulf W On 7 Oct 2010, at 17:07, tom kelly wrote: > Hi Nicholas, > > This was just run in the erlang shell, A was a 100000 byte binary, "f(A)." > told the shell to forget it, ie. set the reference count to zero. There were > no other references to A in this case nor were other binaries created from > it. It was gc'd after a while, presumably after 65535 reductions of the > process that stored it (if that makes sense). My question is how can I > reduce the fullsweep_after or explicitly force a gc on the heap where it is > stored. > > //Tom. > > > > On Thu, Oct 7, 2010 at 3:57 PM, Nicholas Frechette wrote: > >> A bit hard to understand what is happening without seeing f(A)'s code. >> AFAIK, even if you don't keep a reference to a binary, if you keep a >> reference to a binary created from it, it'll keep it in full in memory. >> ie: >> A = << some large binary>>. >> <> = A. >> If you keep no references to A but you do to B, A will remain in memory >> because B will have been created as a pointer offset from A (as an >> optimization). >> >> Nicholas >> >> >> On Thu, Oct 7, 2010 at 10:07 AM, tom kelly wrote: >> >>> Hello List, >>> >>> I need some help on garbage collecting binaries. >>> I have an application that handles large binaries and under load it eats >>> up >>> all the available memory then falls over, even when I start all the >>> data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}". >>> >>> Reading point 5.15 on the page: >>> http://www.erlang.org/faq/how_do_i.html >>> leads me to think that calling the garbage collector on the process that >>> created a binary will clean it up but my shell experiment below shows me >>> I'm >>> wrong. It's now 10 minutes since I've done this experiment and the large >>> binary is still in memory. >>> >>> Maybe I have to call the garbage collector of the process whose heap is >>> storing the binary? If so, which process is it? >>> >>> Any pointers as to what I'm doing wrong or mis-understand will be greatly >>> appreciated. >>> >>> I'm on a legacy R12B5 system. >>> >>> //Tom. >>> >>> >>> >>> 18> process_info(self(),total_heap_size). >>> {total_heap_size,4181} >>> >>> 19> A = L(100000). >>> [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118, >>> 180,121,181,160,124,244,140,169,215,31,82,43|...] >>> >>> 20> process_info(self(),total_heap_size). >>> {total_heap_size,1149851} >>> >>> 21> f(A). >>> ok >>> >>> 22> erlang:garbage_collect(). >>> true >>> >>> 23> process_info(self(),total_heap_size). >>> {total_heap_size,3194} >>> >>> >>> 24> memory(binary). >>> 6536 >>> >>> 25> A = B(100000). >>> <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181, >>> 218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>> >>> >>> 26> memory(binary). >>> 106704 >>> >>> 27> f(A). >>> ok >>> >>> 28> memory(binary). >>> 106704 >>> >>> 29> erlang:garbage_collect(). >>> true >>> >>> 30> memory(binary). >>> 106560 >>> >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From anders.nygren@REDACTED Thu Oct 7 17:36:04 2010 From: anders.nygren@REDACTED (Anders Nygren) Date: Thu, 7 Oct 2010 10:36:04 -0500 Subject: [erlang-questions] Garbage collecting binaries In-Reply-To: References: Message-ID: On Thu, Oct 7, 2010 at 10:07 AM, tom kelly wrote: > Hi Nicholas, > > This was just run in the erlang shell, A was a 100000 byte binary, "f(A)." > told the shell to forget it, ie. set the reference count to zero. There were > no other references to A in this case nor were other binaries created from > it. It was gc'd after a while, presumably after 65535 reductions of the > process that stored it (if that makes sense). My question is how can I > reduce the fullsweep_after or explicitly force a gc on the heap where it is > stored. > > //Tom. > Hi The shell keeps a history also. So even after f(A) the binary is still stored in the history. As an example 7> A=lists:seq(1,10). [1,2,3,4,5,6,7,8,9,10] 8> f(A). ok 9> A. * 1: variable 'A' is unbound 10> v(7). [1,2,3,4,5,6,7,8,9,10] The useful shell functions for this is h() -- history history(N) -- set how many previous commands to keep results(N) -- set how many previous command results to keep /Anders > > > On Thu, Oct 7, 2010 at 3:57 PM, Nicholas Frechette wrote: > >> A bit hard to understand what is happening without seeing f(A)'s code. >> AFAIK, even if you don't keep a reference to a binary, if you keep a >> reference to a binary created from it, it'll keep it in full in memory. >> ie: >> A = << some large binary>>. >> <> = A. >> If you keep no references to A but you do to B, A will remain in memory >> because B will have been created as a pointer offset from A (as an >> optimization). >> >> Nicholas >> >> >> On Thu, Oct 7, 2010 at 10:07 AM, tom kelly wrote: >> >>> Hello List, >>> >>> I need some help on garbage collecting binaries. >>> I have an application that handles large binaries and under load it eats >>> up >>> all the available memory then falls over, even when I start all the >>> data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}". >>> >>> Reading point 5.15 on the page: >>> http://www.erlang.org/faq/how_do_i.html >>> leads me to think that calling the garbage collector on the process that >>> created a binary will clean it up but my shell experiment below shows me >>> I'm >>> wrong. It's now 10 minutes since I've done this experiment and the large >>> binary is still in memory. >>> >>> Maybe I have to call the garbage collector of the process whose heap is >>> storing the binary? If so, which process is it? >>> >>> Any pointers as to what I'm doing wrong or mis-understand will be greatly >>> appreciated. >>> >>> I'm on a legacy R12B5 system. >>> >>> //Tom. >>> >>> >>> >>> 18> process_info(self(),total_heap_size). >>> {total_heap_size,4181} >>> >>> 19> A = L(100000). >>> [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118, >>> ?180,121,181,160,124,244,140,169,215,31,82,43|...] >>> >>> 20> process_info(self(),total_heap_size). >>> {total_heap_size,1149851} >>> >>> 21> f(A). >>> ok >>> >>> 22> erlang:garbage_collect(). >>> true >>> >>> 23> process_info(self(),total_heap_size). >>> {total_heap_size,3194} >>> >>> >>> 24> memory(binary). >>> 6536 >>> >>> 25> A = B(100000). >>> <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181, >>> ?218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>> >>> >>> 26> memory(binary). >>> 106704 >>> >>> 27> f(A). >>> ok >>> >>> 28> memory(binary). >>> 106704 >>> >>> 29> erlang:garbage_collect(). >>> true >>> >>> 30> memory(binary). >>> 106560 >>> >> >> > From djk121@REDACTED Thu Oct 7 17:51:48 2010 From: djk121@REDACTED (Dan Kelley) Date: Thu, 7 Oct 2010 11:51:48 -0400 Subject: confusing returns from httpc:request/4 Message-ID: Hi, I've got a simple router that's talking to a remote HTTP load balancer via httpc:request/4. For every request that I get into the router, I spawn a new process then attempt to connect to the HTTP LB. That works well most of the time, but occasionally I get {error, socket_closed_remotely} instead of results. I had assumed that these corresponded to the HTTP LB prematurely closing the connection, but when I look at the logs for that process, it always thinks that it's returning clean responses (HTTP 200 code, appropriately-formed bodies, etc). Anyone seen anything like this before? What else might I do to debug this? Thanks, Dan From ttom.kelly@REDACTED Thu Oct 7 18:32:04 2010 From: ttom.kelly@REDACTED (tom kelly) Date: Thu, 7 Oct 2010 17:32:04 +0100 Subject: [erlang-questions] Garbage collecting binaries In-Reply-To: References: Message-ID: Hi all, Thanks for all your quick replies! Looks like my demonstration wasn't as pure as I thought ;-) I still have my original problem of ~75% of my VMs memory being taken up by binaries, looks like I have a serious code-review ahead to see why this isn't being freed up. //Tom On Thu, Oct 7, 2010 at 4:34 PM, Ulf Wiger wrote: > > f(A) only unbinds the variable in the shell. > > The shell still has a history of commands and their results, so you > would need to ensure that this is not taking up space as well. > > BR, > Ulf W > > On 7 Oct 2010, at 17:07, tom kelly wrote: > > > Hi Nicholas, > > > > This was just run in the erlang shell, A was a 100000 byte binary, > "f(A)." > > told the shell to forget it, ie. set the reference count to zero. There > were > > no other references to A in this case nor were other binaries created > from > > it. It was gc'd after a while, presumably after 65535 reductions of the > > process that stored it (if that makes sense). My question is how can I > > reduce the fullsweep_after or explicitly force a gc on the heap where it > is > > stored. > > > > //Tom. > > > > > > > > On Thu, Oct 7, 2010 at 3:57 PM, Nicholas Frechette >wrote: > > > >> A bit hard to understand what is happening without seeing f(A)'s code. > >> AFAIK, even if you don't keep a reference to a binary, if you keep a > >> reference to a binary created from it, it'll keep it in full in memory. > >> ie: > >> A = << some large binary>>. > >> <> = A. > >> If you keep no references to A but you do to B, A will remain in memory > >> because B will have been created as a pointer offset from A (as an > >> optimization). > >> > >> Nicholas > >> > >> > >> On Thu, Oct 7, 2010 at 10:07 AM, tom kelly > wrote: > >> > >>> Hello List, > >>> > >>> I need some help on garbage collecting binaries. > >>> I have an application that handles large binaries and under load it > eats > >>> up > >>> all the available memory then falls over, even when I start all the > >>> data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}". > >>> > >>> Reading point 5.15 on the page: > >>> http://www.erlang.org/faq/how_do_i.html > >>> leads me to think that calling the garbage collector on the process > that > >>> created a binary will clean it up but my shell experiment below shows > me > >>> I'm > >>> wrong. It's now 10 minutes since I've done this experiment and the > large > >>> binary is still in memory. > >>> > >>> Maybe I have to call the garbage collector of the process whose heap is > >>> storing the binary? If so, which process is it? > >>> > >>> Any pointers as to what I'm doing wrong or mis-understand will be > greatly > >>> appreciated. > >>> > >>> I'm on a legacy R12B5 system. > >>> > >>> //Tom. > >>> > >>> > >>> > >>> 18> process_info(self(),total_heap_size). > >>> {total_heap_size,4181} > >>> > >>> 19> A = L(100000). > >>> [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118, > >>> 180,121,181,160,124,244,140,169,215,31,82,43|...] > >>> > >>> 20> process_info(self(),total_heap_size). > >>> {total_heap_size,1149851} > >>> > >>> 21> f(A). > >>> ok > >>> > >>> 22> erlang:garbage_collect(). > >>> true > >>> > >>> 23> process_info(self(),total_heap_size). > >>> {total_heap_size,3194} > >>> > >>> > >>> 24> memory(binary). > >>> 6536 > >>> > >>> 25> A = B(100000). > >>> <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181, > >>> 218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>> > >>> > >>> 26> memory(binary). > >>> 106704 > >>> > >>> 27> f(A). > >>> ok > >>> > >>> 28> memory(binary). > >>> 106704 > >>> > >>> 29> erlang:garbage_collect(). > >>> true > >>> > >>> 30> memory(binary). > >>> 106560 > >>> > >> > >> > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > From alexey.v.romanov@REDACTED Thu Oct 7 18:34:49 2010 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Thu, 7 Oct 2010 20:34:49 +0400 Subject: erl -make suggestion Message-ID: One slight problem with erl -make is that you can't supply different targets (e.g. one where TEST macro is defined and one where it isn't). Adding ability to define targets would be hard to do in a backwards-compatible way. So I suggest an option to pass a filename to be used instead of Emakefile. Currently the simplest workaround seems to be to create an Emakefile in a different directory, `cd` into it, and run erl -make there. Yours, Alexey Romanov From anthonym@REDACTED Thu Oct 7 20:00:57 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Thu, 7 Oct 2010 11:00:57 -0700 Subject: [erlang-questions] confusing returns from httpc:request/4 In-Reply-To: References: Message-ID: <20101007180057.GA18278@alumni.caltech.edu> Hi Dan, A quick websearch turned up http://stackoverflow.com/questions/2126630/erlangs-maximum-number-of-simultaneous-open-ports which seems to suggest you might be at the maximum allowable open file descriptors and need to update it. I've had less than stellar results with httpc (see old posts on this group http://groups.google.com/group/erlang-programming/browse_thread/thread/1d94595c83b21ef6/1b2416a346b65596?lnk=gst&q=httpc#1b2416a346b65596 http://groups.google.com/group/erlang-programming/browse_thread/thread/2f3684d1433c465d/2866bae1935138a2?lnk=gst&q=httpc#2866bae1935138a2). I would highly recommend ibrowse though, it's worked really well. -Anthony On Thu, Oct 07, 2010 at 11:51:48AM -0400, Dan Kelley wrote: > Hi, > > I've got a simple router that's talking to a remote HTTP load balancer via > httpc:request/4. For every request that I get into the router, I spawn a > new process then attempt to connect to the HTTP LB. > > That works well most of the time, but occasionally I get > {error, socket_closed_remotely} instead of results. I had assumed that > these corresponded to the HTTP LB prematurely closing the connection, but > when I look at the logs for that process, it always thinks that it's > returning clean responses (HTTP 200 code, appropriately-formed bodies, etc). > > Anyone seen anything like this before? What else might I do to debug this? > > Thanks, > > Dan -- ------------------------------------------------------------------------ Anthony Molinaro From dmercer@REDACTED Thu Oct 7 23:17:14 2010 From: dmercer@REDACTED (David Mercer) Date: Thu, 7 Oct 2010 16:17:14 -0500 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: Message-ID: <010101cb6665$0490a730$0db1f590$@com> On Tuesday, October 05, 2010, Joe Armstrong wrote: > There shouldn't be ports in the language at all (sorry) - ports should > in all circumstances > behave and be indistinguishable from Pids. open_port(...) should > return a Pid and not a port. > > We should reduce the number of concepts in the langauge not increase > them - all in the > name of conceptual integrity. I'm surprised ROK hasn't chimed in with my opinion on this. Since he hasn't, I am forced to agree with someone else. I'm with Armstrong on this. Cheers, DBM From youj@REDACTED Thu Oct 7 22:32:33 2010 From: youj@REDACTED (Jimmy) Date: Thu, 07 Oct 2010 15:32:33 -0500 Subject: port_command performs in cycle In-Reply-To: <1286481606.30861.ezmlm@erlang.org> References: <1286481606.30861.ezmlm@erlang.org> Message-ID: <4CEA870DD78D4F75B07CC239F2DB2AA4@fnal.gov> Hi Gurus, We have an Erlang program communicates to a C-port through the port_command(). We have messured the time taken to perform 6000 send/response at a time and repeated for 100 times. That is 600000 send/responses in total. The data are shown below. The first column is the return from timer:tc() for each 6000 send/responses. The second column is the elapsed time from timer:now_diff(). The data shows sawtooth-shape function. The response time decrease to a point and then jump up vertically and then decrease again. The period is around 60 seconds. We have attached the full test code (erl, C, makefile) for your review. We've experimented "+A 10", "+A 100". It does not seem to help. Thank you very much for your insight. Jimmy Erlang (BEAM) emulator version 5.6.5 [source] [64-bit] [smp:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> drv:test(100). {244061,[d]} elapseTime(us)= 245508 {222369,[d]} elapseTime(us)= 470812 {212453,[d]} elapseTime(us)= 683355 {203302,[d]} elapseTime(us)= 886736 {213917,[d]} elapseTime(us)= 1100729 {228699,[d]} elapseTime(us)= 1329508 {218659,[d]} elapseTime(us)= 1548250 {209391,[d]} elapseTime(us)= 1757724 {199847,[d]} elapseTime(us)= 1957645 {1947658,[d]} elapseTime(us)= 3905381 {2293637,[d]} elapseTime(us)= 6199105 {2195128,[d]} elapseTime(us)= 8394322 {2102996,[d]} elapseTime(us)= 10497397 {2011831,[d]} elapseTime(us)= 12509320 {1924136,[d]} elapseTime(us)= 14433533 {1841881,[d]} elapseTime(us)= 16275502 {1762773,[d]} elapseTime(us)= 18038356 {1687034,[d]} elapseTime(us)= 19725481 {1614767,[d]} elapseTime(us)= 21340324 {1545251,[d]} elapseTime(us)= 22885661 {1478884,[d]} elapseTime(us)= 24364620 {1415657,[d]} elapseTime(us)= 25780365 {1355062,[d]} elapseTime(us)= 27135500 {1229070,[d]} elapseTime(us)= 28364652 {1170889,[d]} elapseTime(us)= 29535615 {1120657,[d]} elapseTime(us)= 30656352 {1099148,[d]} elapseTime(us)= 31755575 {1097913,[d]} elapseTime(us)= 32853595 {1050709,[d]} elapseTime(us)= 33904385 {1005591,[d]} elapseTime(us)= 34910065 {962535,[d]} elapseTime(us)= 35872677 {920992,[d]} elapseTime(us)= 36793757 {881633,[d]} elapseTime(us)= 37675461 {844040,[d]} elapseTime(us)= 38519576 {807764,[d]} elapseTime(us)= 39327414 {773142,[d]} elapseTime(us)= 40100635 {739643,[d]} elapseTime(us)= 40840346 {708142,[d]} elapseTime(us)= 41548564 {677807,[d]} elapseTime(us)= 42226440 {648385,[d]} elapseTime(us)= 42874904 %%%%%%% drv.erl starts here %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -module(drv). -export([start/1, init/2, call_port/1, read/2, test/1, perftest/2]). %% import -import(timer, [tc/3, now_diff/2]). init(ExtPrg, DrvID) -> register(DrvID, self()), Port = open_port({spawn, ExtPrg}, [{packet, 2}, binary, exit_status]), loop(Port). start(DrvID) -> spawn_link(?MODULE, init, ["./mydrv.out", DrvID]). call_port(DrvID) -> DrvID ! {call, self()}, receive _ -> ok end. loop(Port) -> receive {call, Caller} -> erlang:port_command(Port, <<00>>), receive _ -> Caller ! ok end, loop(Port); stop -> erlang:port_close(Port), exit(normal) end. read(0,DrvID) -> [DrvID]; read(N,DrvID) -> call_port(DrvID), read((N-1),DrvID). perftest(0, {MS, S, US}) -> io:format("~p~celapseTime(us)=~10.B~n", [tc(drv, read, [6000,d]), 9, now_diff(now(), {MS, S, US})]); perftest(N, {MS, S, US}) -> io:format("~p~celapseTime(us)=~10.B~n", [tc(drv, read, [6000,d]), 9, now_diff(now(), {MS, S, US})]), perftest(N-1, {MS, S, US}). test(N) -> start(d), perftest(N, now()). %%%%%%% drv.erl ends here %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% /* **** fef_drver_lib.c starts here ******* */ #include #include #include #include #include #include int fef_read_cmd(char **buf, int *size); int fef_write_cmd(ei_x_buff* x); int fef_read_exact(char *buf, int len); int fef_write_exact(char *buf, int len); void fef_encode_ok(ei_x_buff* x) { const char* k_ok = "ok"; ei_x_encode_atom(x, k_ok); } void flush_free(ei_x_buff* x) { fef_write_cmd(x); ei_x_free(x); } int fef_read_cmd(char **buf, int *size) { int len, gotLen; gotLen = fef_read_exact(*buf, 2); if (gotLen != 2) { return(-1); } len = ((**((unsigned char**)buf)) << 8) | *(*((unsigned char**)buf)+1); if (len > *size) { char* tmp = (char *) realloc(*buf, len); if (tmp == NULL) { return -1; } else { *buf = tmp; } *size = len; } gotLen = fef_read_exact(*buf, len); return gotLen; } int fef_write_cmd(ei_x_buff *buff) { char li; li = (buff->index >> 8) & 0xff; fef_write_exact(&li, 1); li = buff->index & 0xff; fef_write_exact(&li, 1); return fef_write_exact(buff->buff, buff->index); } int fef_read_exact(char *buf, int len) { int i, got=0; do { if ((i = read(0, buf+got, len-got)) <= 0) { return i; } got += i; } while (got 0) { ei_x_new_with_version(&x); fef_encode_ok(&x); flush_free(&x); } return 0; } /* **** fef_drver_lib.c ends here ******* */ # #### Makefile starts here ############# THIS_PLATFORM:= $(shell uname -s) THIS_ARCH:= $(shell uname -p) TARGETS= mydrv.out drv.beam YCFLAGS+= -pipe -W -Wall \ -DTHIS_PLATFORM=\"${THIS_PLATFORM}\" -DTHIS_ARCH=\"${THIS_ARCH}\" \ -DTHIS_TARGET=${THIS_PLATFORM}_Target \ -I/usr/lib64/erlang/usr/include \ -I/usr/lib64/erlang/lib/erl_interface-3.5.9/include \ -L/usr/lib64/erlang/lib/erl_interface-3.5.9/lib \ -I . all : ${TARGETS} %.beam: %.erl erlc $< mydrv.out : fef_driver_lib.c cc ${YCFLAGS} -g -o $@ $^ -lpthread -L/usr/lib64/erlang/lib/erl_interface-3.5.9/lib -lerl_interface -lei .PHONY : clean clean : @rm -f ${TARGETS} *~ # #### Makefile ends here ############# From dale@REDACTED Fri Oct 8 01:18:32 2010 From: dale@REDACTED (Dale Harvey) Date: Fri, 8 Oct 2010 00:18:32 +0100 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: <00a601cb6629$ce8eabe0$6bac03a0$@com> References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> <00a601cb6629$ce8eabe0$6bac03a0$@com> Message-ID: You arent alone, I also pretty much entirely work with javascript and erlang. I enjoy programming with node in a way that I dont with erlang, stuff feels fun, I have lots of libraries that people have written for me, it already has a package manager which to be honest embarrasses the erlang side of me slightly. However using node I have had problems making very simple things be resilient to errors, async error handling is very very hard (as most erlang people know) and there seems to be little effort with nodes community to resolving that. So for now node is still a little fun toy to play with, but when I want to write something that I know stays up, I use erlang. However I am a little troubled by nodes popularity and the general dismissal of "well erlang does it better", people are flocking to node in a way that they are not with erlang, People are more and more needing to write the type of async servers that node and erlang provide, programming with an embedded server is also a huge benefit, people obviously want this but why are they embracing node in a way they arent with erlang? It took me 5 minutes from getting to the node website, to compiling it and having my first hello world server up and running, a few minutes later I had a package manager installed and some user contributed libraries that helped me write what I wanted. Installing erlang is mostly fine, however once I am there, there is no standard build tools which will let me write my first module, compile it (and by standard build tools I mean one command, not write my own emakefile, or type c(module) at the shell) and run it, there are a bunch of web servers to chose from which none are universally recommended and none are properly documented, I have absolutely no idea how to install any of these servers properly, to this day everyone had massively differing opinions on how libraries should be bundled. I think the erlang community really needs to sort out these problems once and for all, and before diving into emacs, these are political problems not technical ones. I have had this on my mind for a while and still dont know the best solutions, however, I am promising now that by the euc erlangotp.com will finish its redesign with one major goal, to get a programmer completely unfamiliar with erlang to get a "hello world" web server up and running in 5 minutes. I think its time to settle on and promote tools like rebar / mochiweb / learnyourselfsomeerlang / unknown package manager / and not wait for things to be officially blessed by the otp team, they are busy making erlang awesome, we as a community need to pick up the slack on that end. So yeh, bit of a rant, but if anyone else is interested in lowering the barriers to entry for erlang then shoot me an email / make a new thread, I certainly want as much of the community as possible involved. Cheers Dale On 7 October 2010 15:13, David Mercer wrote: > I might be the only person in the world whose two favorite languages are > Javascript and Erlang. I had been writing full-blown server-side > Javascript > applications for 5 or 6 years until I discovered Erlang in '07. What had > attracted me to Javascript was the ability to replace code on the fly (for > instance, by loading the file into a string and eval-ing it). For > distribution (the application was distributed over about 20 machines > internally with about 100 additional smaller-scale remote servers > communicating with the central hub), I used the file system and TCP (and > HTTP for negotiating across firewalls), depending on the requirements. (I > had also originally used MSMQ, but we phased that out over time in favor of > more expedient methods that we developed ourselves.) The end result was a > less robust version of an Erlang application, that worked pretty well. We > had hot code loading, and redundancy gave us the fault-tolerance we > required. > > > When I discovered Erlang in 2007 (not sure why it took me that long), I > immediately recognized that Erlang was built to handle all the problems we > had written around, but it had a longer history and was certainly better at > it. We had required so much of our redundancy (20 machines running about 7 > processes each) not only to handle the load, but also to handle failures. > Windows Script Host, which was our Javascript engine, is not as stable as > you might think, and seemed to suffer from memory leaks which manifested > itself over time. With 20 machines times 7 processes per machine and an > MTBF of, say, a week, we had 10 or 20 failures a day (which were > automatically handled). Honestly, the surprise for us was that WSH > performed as well as it did. We had originally written our application in > Javascript/WSH as a prototype, intending to rewrite in C/C++/Java/something > else when we had completed our proof of concept. But as it turned out, > processing speed wasn't really an issue for us -- network latency was > usually the limiting issue, not CPU. However, it was surprising to us that > WSH could, at times, run for days or weeks, processing millions of > transactions before failing. WSH's multi-threading capabilities are > nonexistent, and I tried various hacks to coax it into running multiple > threads, but they tended to cause other problems (as you can imagine), so I > wrote my own scheduler that time-sliced between various tasks. (It was > actually a heap-based priority queue.) We isolated blocking operations > (the > main issue we had with blocking was that some asynchronous SQL queries were > not working right, so we had to do them synchronously) in their own process > so that they wouldn't block other tasks. > > However, Erlang does everything we wrote, and it does it better. Having > written an equivalent system in a non-Erlang language, I really appreciate > all the tools Erlang gives you to create manageable distributed server-side > applications. Besides the interprocess messaging and hot code-loading, OTP > is an excellent way of thinking about and framework for building your > fault-tolerant servers. I did not know about Erlang and OTP when I started > writing server-side Javascript, but if I had, I would have begun by > rewriting Erlang and OTP (e.g., gen_server) in Javascript. It would have > created a much better architecture, though it is not clear why I would > write > such a system in Javascript if I knew about Erlang. Node.js is only a > small > step up from WSH in that it is more portable and has better support for > multi-tasking. I suspect you would find yourself rewriting a lot of > Erlang/OTP in Javascript if you decided to build in Node.js your own > server-side applications. > > That being said, as an old Javascript warrior myself, I'd be interested to > hear how it goes. I, too, have considered trying out Node.js as my New > Favorite Platform(TM), but haven't yet had the need. Today, I haven't done > much programming in Javascript in a couple of years, but I gotta admit, as > a > language, it is very highly capable. I daresay, there was a time, 5-10 > years ago, when I considered myself one of the best Javascript programmers > in the world, and I would sometimes log into Experts Exchange or somewhere > like that to help out, only to be dismayed that the Javascript questions > were not about the language Javascript, but about manipulating the DOM in a > web page. No-one ever seemed to ask the tricky questions about closures or > object context, or performance of certain approaches etc. that was my > expertise. You see more of that nowadays, as most of these concepts have > gained traction in the mainstream; now there are probably tens of thousands > of people with that knowledge, not just the hundreds of a few years ago. > > Thank-you for this walk down memory lane. Let me know how it goes with > Node.js. I just might have to try it out... > > Cheers, > > DBM > > > -----Original Message----- > > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] > On > > Behalf Of Pablo Platt > > Sent: Sunday, October 03, 2010 10:01 AM > > To: erlang-questions@REDACTED > > Subject: [erlang-questions] node.js compared to erlang > > > > Hi > > > > I'm using erlang and I love it. > > I'm not trying to create an argument but to better understand the power > of > > erlang. > > > > In what areas erlang dominates and what areas will you consider using > > node.js? > > For example, would you consider building something like rabbitmq or > > ejabberd in > > node.js? > > Or maybe you'll use node.js just for a simple single chat room but erlang > > for > > anything complicated than that. > > > > Are there a fundamental differences in performance or stability or use in > > distributed systems? > > > > Ulf Wiger commented on the comparison of erlang and node.js > > and said that erlang solves the problem of non blocking functions which > > might be > > very hard for other languages. > > http://journal.dedasys.com/2010/04/29/erlang-vs-node-js > > > > Thanks > > > > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ok@REDACTED Fri Oct 8 02:48:30 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 8 Oct 2010 13:48:30 +1300 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: <010101cb6665$0490a730$0db1f590$@com> References: <010101cb6665$0490a730$0db1f590$@com> Message-ID: On 8/10/2010, at 10:17 AM, David Mercer wrote: > > I'm surprised ROK hasn't chimed in with my opinion on this. Since he > hasn't, I am forced to agree with someone else. I'm with Armstrong on this. I didn't chime in because after Joe Armstrong had said his piece, I didn't have anything left to say. From robert.virding@REDACTED Fri Oct 8 03:01:49 2010 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 08 Oct 2010 03:01:49 +0200 Subject: [erlang-questions] Garbage collecting binaries In-Reply-To: References: Message-ID: <4CAE6D7D.4090100@erlang-solutions.com> Large binaries are not stored in any process heap, but are specially handled outside the normal process data. This allows them to be sent in messages between processes by reference instead copying, thereby saving the copying of potentially large amounts of data. Which is the main reason for doing it this way. However, this does cause problems for the gc. Once a large binary has been passed between processes it is no longer enough to collect just one process before freeing it. You must first collect a process before you can determine which references to large binaries it has had are no longer in use. This means you have to first collect at least each process through which the binary has passed before you can determine if there are any references to it, or if it can be freed. Which means that freeing of large binaries is delayed, which means it is possible to fill memory with unreferenced binaries which have not yet been collected. The fact that you can get references to large binaries even though you only reference parts of them just aggravate matters. Unfortunately there are few ways around this. As usual it is a matter of making trade-offs and hoping for the best. Removing the special handling of large binaries would kill many applications. Robert On 2010-10-07 16.07, tom kelly wrote: > Hello List, > > I need some help on garbage collecting binaries. > I have an application that handles large binaries and under load it eats up > all the available memory then falls over, even when I start all the > data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}". > > Reading point 5.15 on the page: > http://www.erlang.org/faq/how_do_i.html > leads me to think that calling the garbage collector on the process that > created a binary will clean it up but my shell experiment below shows me I'm > wrong. It's now 10 minutes since I've done this experiment and the large > binary is still in memory. > > Maybe I have to call the garbage collector of the process whose heap is > storing the binary? If so, which process is it? > > Any pointers as to what I'm doing wrong or mis-understand will be greatly > appreciated. > > I'm on a legacy R12B5 system. > > //Tom. > > > > 18> process_info(self(),total_heap_size). > {total_heap_size,4181} > > 19> A = L(100000). > [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118, > 180,121,181,160,124,244,140,169,215,31,82,43|...] > > 20> process_info(self(),total_heap_size). > {total_heap_size,1149851} > > 21> f(A). > ok > > 22> erlang:garbage_collect(). > true > > 23> process_info(self(),total_heap_size). > {total_heap_size,3194} > > > 24> memory(binary). > 6536 > > 25> A = B(100000). > <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181, > 218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>> > > 26> memory(binary). > 106704 > > 27> f(A). > ok > > 28> memory(binary). > 106704 > > 29> erlang:garbage_collect(). > true > > 30> memory(binary). > 106560 > From boris.muehmer@REDACTED Fri Oct 8 06:26:27 2010 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Fri, 8 Oct 2010 06:26:27 +0200 Subject: [erlang-questions] How to analyse why (w)erl.exe is not starting? In-Reply-To: References: <2D16B184-02B1-4757-B14D-8E54172A398C@gmail.com> Message-ID: Thanks to the new Windows R14B bugfix release everything is working now. - boris From kenneth.lundin@REDACTED Fri Oct 8 09:09:25 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Fri, 8 Oct 2010 09:09:25 +0200 Subject: [erlang-questions] Garbage collecting binaries In-Reply-To: <4CAE6D7D.4090100@erlang-solutions.com> References: <4CAE6D7D.4090100@erlang-solutions.com> Message-ID: Hi The GC of binaries has been significantly improved since R13B03. Before that it was quite easy to create situations where binaries where not released early enough during GC. >From R14 the new module binary was introduced. In that module the functions copy and referenced_byte_size can be helpful in situations where it is important to free unused binaries as soon as possible. See http://www.erlang.org/doc/man/binary.html#referenced_byte_size-1 and http://www.erlang.org/doc/man/binary.html#copy-1 /Kenneth, Erlang/OTP Ericsson On Fri, Oct 8, 2010 at 3:01 AM, Robert Virding wrote: > ?Large binaries are not stored in any process heap, but are specially > handled outside the normal process data. This allows them to be sent in > messages between processes by reference instead copying, thereby saving the > copying of potentially large amounts of data. Which is the main reason for > doing it this way. > > However, this does cause problems for the gc. Once a large binary has been > passed between processes it is no longer enough to collect just one process > before freeing it. You must first collect a process before you can determine > which references to large binaries it has had are no longer in use. This > means you have to first collect at least each process through which the > binary has passed before you can determine if there are any references to > it, or if it can be freed. Which means that freeing of large binaries is > delayed, which means it is possible to fill memory with unreferenced > binaries which have not yet been collected. > > The fact that you can get references to large binaries even though you only > reference parts of them just aggravate matters. > > Unfortunately there are few ways around this. As usual it is a matter of > making trade-offs and hoping for the best. Removing the special handling of > large binaries would kill many applications. > > Robert > > On 2010-10-07 16.07, tom kelly wrote: >> >> Hello List, >> >> I need some help on garbage collecting binaries. >> I have an application that handles large binaries and under load it eats >> up >> all the available memory then falls over, even when I start all the >> data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}". >> >> Reading point 5.15 on the page: >> http://www.erlang.org/faq/how_do_i.html >> leads me to think that calling the garbage collector on the process that >> created a binary will clean it up but my shell experiment below shows me >> I'm >> wrong. It's now 10 minutes since I've done this experiment and the large >> binary is still in memory. >> >> Maybe I have to call the garbage collector of the process whose heap is >> storing the binary? If so, which process is it? >> >> Any pointers as to what I'm doing wrong or mis-understand will be greatly >> appreciated. >> >> I'm on a legacy R12B5 system. >> >> //Tom. >> >> >> >> 18> ?process_info(self(),total_heap_size). >> {total_heap_size,4181} >> >> 19> ?A = L(100000). >> [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118, >> ?180,121,181,160,124,244,140,169,215,31,82,43|...] >> >> 20> ?process_info(self(),total_heap_size). >> {total_heap_size,1149851} >> >> 21> ?f(A). >> ok >> >> 22> ?erlang:garbage_collect(). >> true >> >> 23> ?process_info(self(),total_heap_size). >> {total_heap_size,3194} >> >> >> 24> ?memory(binary). >> 6536 >> >> 25> ?A = B(100000). >> <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181, >> ? 218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>> >> >> 26> ?memory(binary). >> 106704 >> >> 27> ?f(A). >> ok >> >> 28> ?memory(binary). >> 106704 >> >> 29> ?erlang:garbage_collect(). >> true >> >> 30> ?memory(binary). >> 106560 >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From arun.suresh@REDACTED Fri Oct 8 09:55:15 2010 From: arun.suresh@REDACTED (Arun Suresh) Date: Fri, 8 Oct 2010 00:55:15 -0700 Subject: Any particular reason Y dets tables have to always be "named" by an atom ? Message-ID: Hi I was wondering y it is required that dets tables have to always be "named" by an atom ? This becomes especially inconvenient when you have system for eg. where you'd have to process many short lived jobs. Ets provides you with an integer TableId.. so even if its named, you can still reuse the name and wont have to blow up the atom pool. I can pass the TableId around. Because dets has this limitation, I invariably end up having to create a pool for table names and then I have to worry bout writing code to synchronously acquire a table name and release the same when the process is done.. and ofcourse i have to worry about what happend the owning process dies.. etc. -Arun From krab@REDACTED Fri Oct 8 12:18:37 2010 From: krab@REDACTED (Kresten Krab Thorup) Date: Fri, 8 Oct 2010 12:18:37 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> <20101006001423.GA8623@mulga.csse.unimelb.edu.au> Message-ID: Further to this discussion, I would love to see a way that would allow writing a "port driver" in erlang. It would be a case to consider to make the resulting abstractions be more orthogonal to one another. For instance, with just a handful of NIFs you could implement the tcp_ip driver in erlang, and the result would likely be much more readable and manageable (but perhaps not so performant). You could actually do such an implementation now of course, but it would not work with port-specific functions like port_info, etc... Kresten On Oct 6, 2010, at 9:53 , Joe Armstrong wrote: > On Wed, Oct 6, 2010 at 2:14 AM, Jeff Schultz wrote: >> On Tue, Oct 05, 2010 at 05:04:18PM +0200, Joe Armstrong wrote: >>> There should be the notion of "creating a thing that behaves like a >>> process" that is actually not a process, which which behaves like a >>> process. Ets tables and ports should have been "things that behave >>> like processes" - they should respond to messages and send exits >>> when they die and so on. If they had behaved like processes then >>> life would be easier and more regular ... If ports where pids then >>> one would be able to access ports on remote nodes via their pids >>> (just like remote pids in distributed erlang). >> >> This sounds good to me, but I have a practical question: >> >> How much existing code would break if ets:new or open_port >> returned a pid? Is there a lot, or very little? >> >> Your protocol/1 idea would make it easy to adapt is_port to only >> succeed on pids created by open_port, but there's probably code >> somewhere that handles both ordinary processes and ports and that >> distinguishes them by is_pid. > > The more I think about this the more I like it. > > The problem with a Pid is you have no ideas what resource it represents and > what protocol it obeys - this information is conveyed implicitly by > the location of the Pid > in some bit of code - you need to say "hello pid, which protocol do you obey" > this cannot result in a network message but must be local to a pid. > > You need to know this information *before* you send a message, thus > the implementation > cannot involve any messaging. I think the Pid is just a heap pointer > to a local data > structure, so adding an extra field would be no bother ... > > You could use process_info(Pid) and look at the initial_call data > but I assume process_info results in a network message to find out > what's up with the > process (is this correct?) > >> >> >> For that matter, what would we do with the named_table option to >> ets:new? Make it register the process? > > But there shouldn't be named tables at all - there should only be pids > and registerd processes > one name registration mechanism not many ... > >> >> >> Jeff Schultz >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Kresten Krab Thorup, CTO, Trifork From erlang@REDACTED Fri Oct 8 13:16:26 2010 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 8 Oct 2010 13:16:26 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> <20101006001423.GA8623@mulga.csse.unimelb.edu.au> Message-ID: On Fri, Oct 8, 2010 at 12:18 PM, Kresten Krab Thorup wrote: > Further to this discussion, I would love to see a way that would allow writing a "port driver" in erlang. ?It would be a case to consider to make the resulting abstractions be more orthogonal to one another. > Indeed - in fact the Erlang scheduler and GC should be written in Erlang - this would have to be a strictly controlled sub-set of Erlang that created no garbage and had access to memory and registers - something that could be cross compiled to C (do pretty much like they did in smalltalk many years ago :-) /Joe > For instance, with just a handful of NIFs you could implement the tcp_ip driver in erlang, and the result would likely be much more readable and manageable (but perhaps not so performant). ?You could actually do such an implementation now of course, but it would not work with port-specific functions like port_info, etc... > > Kresten > > > On Oct 6, 2010, at 9:53 , Joe Armstrong wrote: > >> On Wed, Oct 6, 2010 at 2:14 AM, Jeff Schultz wrote: >>> On Tue, Oct 05, 2010 at 05:04:18PM +0200, Joe Armstrong wrote: >>>> There should be the notion of "creating a thing that behaves like a >>>> process" that is actually not a process, which which behaves like a >>>> process. Ets tables and ports should have been "things that behave >>>> like processes" - they should respond to messages and send exits >>>> when they die and so on. If they had behaved like processes then >>>> life would be easier and more regular ... If ports where pids then >>>> one would be able to access ports on remote nodes via their pids >>>> (just like remote pids in distributed erlang). >>> >>> This sounds good to me, but I have a practical question: >>> >>> ? ?How much existing code would break if ets:new or open_port >>> ? ?returned a pid? ?Is there a lot, or very little? >>> >>> Your protocol/1 idea would make it easy to adapt is_port to only >>> succeed on pids created by open_port, but there's probably code >>> somewhere that handles both ordinary processes and ports and that >>> distinguishes them by is_pid. >> >> The more I think about this the more I like it. >> >> The problem with a Pid is you have no ideas what resource it represents and >> what protocol it obeys - this information is conveyed implicitly by >> the location of the Pid >> in some bit of code - you need to say "hello pid, which protocol do you obey" >> this cannot result in a network message but must be local to a pid. >> >> You need to know this information *before* you send a message, thus >> the implementation >> cannot involve any messaging. I think the Pid is just a heap pointer >> to a local data >> structure, so adding an extra field would be no bother ... >> >> You could use process_info(Pid) and look at the initial_call data >> but I assume process_info results in a network message to find out >> what's up with the >> process (is this correct?) >> >>> >>> >>> For that matter, what would we do with the named_table option to >>> ets:new? ?Make it register the process? >> >> But there shouldn't be named tables at all - there should only be pids >> and registerd processes >> one name registration mechanism not many ... >> >>> >>> >>> ? ?Jeff Schultz >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > Kresten Krab Thorup, CTO, Trifork > > From masklinn@REDACTED Fri Oct 8 13:40:25 2010 From: masklinn@REDACTED (Masklinn) Date: Fri, 8 Oct 2010 13:40:25 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> <20101006001423.GA8623@mulga.csse.unimelb.edu.au> Message-ID: <72B0883B-0E4B-48C1-B3FD-08D6754CCAAA@masklinn.net> On 2010-10-08, at 13:16 , Joe Armstrong wrote: > On Fri, Oct 8, 2010 at 12:18 PM, Kresten Krab Thorup wrote: >> Further to this discussion, I would love to see a way that would allow writing a "port driver" in erlang. It would be a case to consider to make the resulting abstractions be more orthogonal to one another. >> > > Indeed - in fact the Erlang scheduler and GC should be written in Erlang > - this would have to be a strictly controlled sub-set of Erlang that created no > garbage and had access to memory and registers - something that could > be cross compiled to C (do pretty much like they did in smalltalk many > years ago :-) > > /Joe > This is also what PyPy tries to do for Python (they have rpython[0], a restricted and statically typed subset of Python) [0] http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html#restricted-python From ttom.kelly@REDACTED Fri Oct 8 13:45:16 2010 From: ttom.kelly@REDACTED (tom kelly) Date: Fri, 8 Oct 2010 12:45:16 +0100 Subject: [erlang-questions] Garbage collecting binaries In-Reply-To: References: <4CAE6D7D.4090100@erlang-solutions.com> Message-ID: Hi Robert, Thanks for that extra insight, it's a lot clearer now! We do pass the binaries between different processes so that's an important point for me to keep in mind while I'm debugging this. Also great work Kenneth & OTP team! I've been exploring this area of Erlang for the first time this week, nice to find out you're so far ahead of us! We were already switching to R14 for our next trunk release so I'm looking forward to experimenting with the new binary modules there. //Tom. On Fri, Oct 8, 2010 at 8:09 AM, Kenneth Lundin wrote: > Hi > > The GC of binaries has been significantly improved since R13B03. > Before that it was quite easy to create situations where binaries > where not released early enough > during GC. > > From R14 the new module binary was introduced. In that module the > functions copy and > referenced_byte_size can be helpful in situations where it is > important to free unused binaries as > soon as possible. > > See > http://www.erlang.org/doc/man/binary.html#referenced_byte_size-1 > and > http://www.erlang.org/doc/man/binary.html#copy-1 > > /Kenneth, Erlang/OTP Ericsson > > On Fri, Oct 8, 2010 at 3:01 AM, Robert Virding > wrote: > > Large binaries are not stored in any process heap, but are specially > > handled outside the normal process data. This allows them to be sent in > > messages between processes by reference instead copying, thereby saving > the > > copying of potentially large amounts of data. Which is the main reason > for > > doing it this way. > > > > However, this does cause problems for the gc. Once a large binary has > been > > passed between processes it is no longer enough to collect just one > process > > before freeing it. You must first collect a process before you can > determine > > which references to large binaries it has had are no longer in use. This > > means you have to first collect at least each process through which the > > binary has passed before you can determine if there are any references to > > it, or if it can be freed. Which means that freeing of large binaries is > > delayed, which means it is possible to fill memory with unreferenced > > binaries which have not yet been collected. > > > > The fact that you can get references to large binaries even though you > only > > reference parts of them just aggravate matters. > > > > Unfortunately there are few ways around this. As usual it is a matter of > > making trade-offs and hoping for the best. Removing the special handling > of > > large binaries would kill many applications. > > > > Robert > > > > On 2010-10-07 16.07, tom kelly wrote: > >> > >> Hello List, > >> > >> I need some help on garbage collecting binaries. > >> I have an application that handles large binaries and under load it eats > >> up > >> all the available memory then falls over, even when I start all the > >> data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}". > >> > >> Reading point 5.15 on the page: > >> http://www.erlang.org/faq/how_do_i.html > >> leads me to think that calling the garbage collector on the process that > >> created a binary will clean it up but my shell experiment below shows me > >> I'm > >> wrong. It's now 10 minutes since I've done this experiment and the large > >> binary is still in memory. > >> > >> Maybe I have to call the garbage collector of the process whose heap is > >> storing the binary? If so, which process is it? > >> > >> Any pointers as to what I'm doing wrong or mis-understand will be > greatly > >> appreciated. > >> > >> I'm on a legacy R12B5 system. > >> > >> //Tom. > >> > >> > >> > >> 18> process_info(self(),total_heap_size). > >> {total_heap_size,4181} > >> > >> 19> A = L(100000). > >> [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118, > >> 180,121,181,160,124,244,140,169,215,31,82,43|...] > >> > >> 20> process_info(self(),total_heap_size). > >> {total_heap_size,1149851} > >> > >> 21> f(A). > >> ok > >> > >> 22> erlang:garbage_collect(). > >> true > >> > >> 23> process_info(self(),total_heap_size). > >> {total_heap_size,3194} > >> > >> > >> 24> memory(binary). > >> 6536 > >> > >> 25> A = B(100000). > >> <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181, > >> 218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>> > >> > >> 26> memory(binary). > >> 106704 > >> > >> 27> f(A). > >> ok > >> > >> 28> memory(binary). > >> 106704 > >> > >> 29> erlang:garbage_collect(). > >> true > >> > >> 30> memory(binary). > >> 106560 > >> > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > From kostis@REDACTED Fri Oct 8 13:46:15 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 08 Oct 2010 14:46:15 +0300 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: <72B0883B-0E4B-48C1-B3FD-08D6754CCAAA@masklinn.net> References: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> <20101006001423.GA8623@mulga.csse.unimelb.edu.au> <72B0883B-0E4B-48C1-B3FD-08D6754CCAAA@masklinn.net> Message-ID: <4CAF0487.4000708@cs.ntua.gr> Masklinn wrote: > On 2010-10-08, at 13:16 , Joe Armstrong wrote: >> On Fri, Oct 8, 2010 at 12:18 PM, Kresten Krab Thorup wrote: >>> Further to this discussion, I would love to see a way that would allow writing a "port driver" in erlang. It would be a case to consider to make the resulting abstractions be more orthogonal to one another. >>> >> Indeed - in fact the Erlang scheduler and GC should be written in Erlang >> - this would have to be a strictly controlled sub-set of Erlang that created no >> garbage and had access to memory and registers - something that could >> be cross compiled to C (do pretty much like they did in smalltalk many >> years ago :-) >> >> /Joe >> > This is also what PyPy tries to do for Python (they have rpython[0], a restricted and statically typed subset of Python) > > > [0] http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html#restricted-python Right. And, out of curiosity, who is using that for anything serious? Kostis From mevans@REDACTED Fri Oct 8 14:10:42 2010 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 8 Oct 2010 08:10:42 -0400 Subject: Mnesia startup woes Message-ID: Hi, I have the following mnesia deployment. Two nodes (NodeA and NodeB) with disc_copies and disc_only_copies of tables (other nodes have ram copies of these tables too). NodeA starts first, followed by NodeB. NodeA is considered by the non-Erlang part of the system to be a master, and NodeB as the standby. When NodeB comes up (maybe much later) it calls the following code to get the schema from NodeA, join the party and copy the tables to itself (it will remove all files from the mnesia directory first): mnesia:change_config(extra_db_nodes, [NodeA]), mnesia:change_table_copy_type(schema, node(), disc_copies), Tabs = mnesia:system_info(tables) -- [schema], [mnesia:add_table_copy(Tab, node(), disc_copies) || Tab <- Tabs]. We can not control when NodeB starts, if ever. So it's likely that NodeA will be the only disc_only node when the tables are created. What I notice is that if NodeA comes up first, the data is unusable until NodeB comes up. This makes the system unusable for that period of time. Certainly I can understand why we do that (we need consistent data), but what I was after is a way to reverse mnesia:change_config/2 to remove NodeB? I can do this on NodeA: [ mnesia:force_load_table(T) || T <- mnesia:system_info (tables) ]. But was wondering if there is a better way? For example, to remove NodeB from NodeA's schema? Thanks From kenneth.lundin@REDACTED Fri Oct 8 14:16:36 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Fri, 8 Oct 2010 14:16:36 +0200 Subject: [erlang-questions] Garbage collecting binaries In-Reply-To: References: <4CAE6D7D.4090100@erlang-solutions.com> Message-ID: Hi Tom, Of course you should experiment with the new binary module in R14B but the GC of binaries is improved in general as well, so hopefully the freeing of binaries should work good enough for your case (and most cases) without need to use the additional possibilities available via the new binary module. /Kenneth Erlang/OTP Ericsson On Fri, Oct 8, 2010 at 1:45 PM, tom kelly wrote: > Hi Robert, > > Thanks for that extra insight, it's a lot clearer now! We do pass the > binaries between different processes so that's an important point for me to > keep in mind while I'm debugging this. > > Also great work Kenneth & OTP team! I've been exploring this area of Erlang > for the first time this week, nice to find out you're so far ahead of us! We > were already switching to R14 for our next trunk release so I'm looking > forward to experimenting with the new binary modules there. > > //Tom. > > > > On Fri, Oct 8, 2010 at 8:09 AM, Kenneth Lundin > wrote: >> >> Hi >> >> The GC of binaries has been significantly improved since R13B03. >> Before that it was quite easy to create situations where binaries >> where not released early enough >> during GC. >> >> From R14 the new module binary was introduced. In that module the >> functions copy and >> referenced_byte_size can be helpful in situations where it is >> important to free unused binaries as >> soon as possible. >> >> See >> http://www.erlang.org/doc/man/binary.html#referenced_byte_size-1 >> and >> http://www.erlang.org/doc/man/binary.html#copy-1 >> >> /Kenneth, Erlang/OTP Ericsson >> >> On Fri, Oct 8, 2010 at 3:01 AM, Robert Virding >> wrote: >> > ?Large binaries are not stored in any process heap, but are specially >> > handled outside the normal process data. This allows them to be sent in >> > messages between processes by reference instead copying, thereby saving >> > the >> > copying of potentially large amounts of data. Which is the main reason >> > for >> > doing it this way. >> > >> > However, this does cause problems for the gc. Once a large binary has >> > been >> > passed between processes it is no longer enough to collect just one >> > process >> > before freeing it. You must first collect a process before you can >> > determine >> > which references to large binaries it has had are no longer in use. This >> > means you have to first collect at least each process through which the >> > binary has passed before you can determine if there are any references >> > to >> > it, or if it can be freed. Which means that freeing of large binaries is >> > delayed, which means it is possible to fill memory with unreferenced >> > binaries which have not yet been collected. >> > >> > The fact that you can get references to large binaries even though you >> > only >> > reference parts of them just aggravate matters. >> > >> > Unfortunately there are few ways around this. As usual it is a matter of >> > making trade-offs and hoping for the best. Removing the special handling >> > of >> > large binaries would kill many applications. >> > >> > Robert >> > >> > On 2010-10-07 16.07, tom kelly wrote: >> >> >> >> Hello List, >> >> >> >> I need some help on garbage collecting binaries. >> >> I have an application that handles large binaries and under load it >> >> eats >> >> up >> >> all the available memory then falls over, even when I start all the >> >> data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}". >> >> >> >> Reading point 5.15 on the page: >> >> http://www.erlang.org/faq/how_do_i.html >> >> leads me to think that calling the garbage collector on the process >> >> that >> >> created a binary will clean it up but my shell experiment below shows >> >> me >> >> I'm >> >> wrong. It's now 10 minutes since I've done this experiment and the >> >> large >> >> binary is still in memory. >> >> >> >> Maybe I have to call the garbage collector of the process whose heap is >> >> storing the binary? If so, which process is it? >> >> >> >> Any pointers as to what I'm doing wrong or mis-understand will be >> >> greatly >> >> appreciated. >> >> >> >> I'm on a legacy R12B5 system. >> >> >> >> //Tom. >> >> >> >> >> >> >> >> 18> ?process_info(self(),total_heap_size). >> >> {total_heap_size,4181} >> >> >> >> 19> ?A = L(100000). >> >> [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118, >> >> ?180,121,181,160,124,244,140,169,215,31,82,43|...] >> >> >> >> 20> ?process_info(self(),total_heap_size). >> >> {total_heap_size,1149851} >> >> >> >> 21> ?f(A). >> >> ok >> >> >> >> 22> ?erlang:garbage_collect(). >> >> true >> >> >> >> 23> ?process_info(self(),total_heap_size). >> >> {total_heap_size,3194} >> >> >> >> >> >> 24> ?memory(binary). >> >> 6536 >> >> >> >> 25> ?A = B(100000). >> >> <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181, >> >> ? 218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>> >> >> >> >> 26> ?memory(binary). >> >> 106704 >> >> >> >> 27> ?f(A). >> >> ok >> >> >> >> 28> ?memory(binary). >> >> 106704 >> >> >> >> 29> ?erlang:garbage_collect(). >> >> true >> >> >> >> 30> ?memory(binary). >> >> 106560 >> >> >> > >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > See http://www.erlang.org/faq.html >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > >> > > > From krab@REDACTED Fri Oct 8 14:36:05 2010 From: krab@REDACTED (Kresten Krab Thorup) Date: Fri, 8 Oct 2010 14:36:05 +0200 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: <4CAB95FA.6070804@erlang-solutions.com> References: <4CAB95FA.6070804@erlang-solutions.com> Message-ID: On Oct 5, 2010, at 23:17 , Robert Virding wrote: > There should only be two types of things in Erlang, processes and > immutable data. Agreed. There is also the minor, but significant, case of handling "resource" objects; which are neither processes or immutable data .... - compiled regexes - compiled match specs - ets continuations - binary matchstates (never exposed outside beam) - NIF resource objects In implementing Erjang, I've run into issues with these several times because their semantics is not that well documented... Ideally, all these should IMHO be exposed with the same semantics across the board; in most cases these foreign objects match the empty binary, right? Perhaps they should also support some kind of "identical" operator? I.e. === for such resource objects could be used to see if they truly represent the same. Kresten From icarus.alive@REDACTED Fri Oct 8 15:22:31 2010 From: icarus.alive@REDACTED (Icarus Alive) Date: Fri, 8 Oct 2010 18:52:31 +0530 Subject: FreeBSD port for Erlang Message-ID: Hi, Anyone has any pointers or updates to the latest FreeBSD port for Erlang ? On searching online I found some old threads from Dec 2009, and a recent announcement about an unofficial availability or R14B01, but nothing further. Is nobody really using Erlang on FreeBSD ? PS> In case someone's curious, I do have mandate to "enhance and port" a piece of software that runs on Linux to FreeBSD. Am a Erlang newbie (if that's an excuse), so kindly be gentle. ~Icarus. From icarus.alive@REDACTED Fri Oct 8 15:27:27 2010 From: icarus.alive@REDACTED (Icarus Alive) Date: Fri, 8 Oct 2010 18:57:27 +0530 Subject: Any video tutorials for Erlang ? Message-ID: Hi, Am learning Erlang, and have extremely limited exposure to "Functional Programming", but some good-bit (15yrs) to Procedural and Object-Oriented programming. An ebook, called "Thinking in Erlang" has really been very helpful in my initial gentle rampup, and I think so far, am able to cope. That, alongwith the Erlang references, has been pretty decent. However, to hasten things up a bit, was wondering if anyone is aware of some good source of video tutorials for Erlang. These days, the youtube videos from the channels like google-io, on langs s.a. Python, RoR etc. are a-plenty, but inspite of significant buzz around Erlang, didn't find much around Erlang. So there some-place, I missed ? Just curious. BTW, if this is not the right place to ask this question, would appreciate pointer to the right forum. ~Icarus. From doug.fort@REDACTED Fri Oct 8 15:35:06 2010 From: doug.fort@REDACTED (Doug Fort) Date: Fri, 8 Oct 2010 09:35:06 -0400 Subject: [erlang-questions] Any video tutorials for Erlang ? In-Reply-To: References: Message-ID: The Pragmatic guys have have some screencasts that I found worthwhile: http://pragprog.com/screencasts/v-kserl/erlang-in-practice You have to pay for them, but IMO they are worth the money. On Fri, Oct 8, 2010 at 9:27 AM, Icarus Alive wrote: > Hi, > > Am learning Erlang, and have extremely limited exposure to "Functional > Programming", but some good-bit (15yrs) to Procedural and > Object-Oriented programming. An ebook, called "Thinking in Erlang" has > really been very helpful in my initial gentle rampup, and I think so > far, am able to cope. That, alongwith the Erlang references, has been > pretty decent. However, to hasten things up a bit, was wondering if > anyone is aware of some good source of video tutorials for Erlang. > These days, the youtube videos from the channels like google-io, on > langs s.a. Python, RoR etc. are a-plenty, but inspite of significant > buzz around Erlang, didn't find much around Erlang. So there > some-place, I missed ? Just curious. > > BTW, if this is not the right place to ask this question, would > appreciate pointer to the right forum. > > ~Icarus. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Doug Fort, Consulting Programmer http://www.dougfort.com From alexey.v.romanov@REDACTED Fri Oct 8 15:38:01 2010 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Fri, 8 Oct 2010 17:38:01 +0400 Subject: Dialyzer crash Message-ID: I've got this error when trying to run dialyzer (from R14B) on my project: aromanov@REDACTED:~/workspace/gm-controller$ dialyzer --verbose --build_plt --output_plt dialyzer/gmcontroller_1.plt -r ebin Compiling some key modules to native code... done in 1m41.75s Creating PLT dialyzer/gmcontroller_1.plt ... =ERROR REPORT==== 8-Oct-2010::17:29:59 === Error in process <0.2602.0> with exit value: {function_clause,[{erl_types,inf_tuples_in_sets,4},{erl_types,inf_tuple_sets,4},{erl_types,inf_tuple_sets,3},{erl_types,inf_union,5},{erl_types,t_inf_lists_strict,4},{erl_types,t_inf,3},{dialyzer_typesig,solve_one_c... dialyzer: Analysis failed with error: {function_clause,[{erl_types,inf_tuples_in_sets,4}, {erl_types,inf_tuple_sets,4}, {erl_types,inf_tuple_sets,3}, {erl_types,inf_union,5}, {erl_types,t_inf_lists_strict,4}, {erl_types,t_inf,3}, {dialyzer_typesig,solve_one_c,...}, {dialyzer_typesig,...}]} Last messages in the log cache: Typesig analysis for SCC: [{server_protocol_client,handle_call,3}] Typesig analysis for SCC: [{config,read_file,1}] Typesig analysis for SCC: [{config,merge_with_file,2}] Typesig analysis for SCC: [{config,read,1}] Typesig analysis for SCC: [{device,hardware_capabilities_test,0}] Typesig analysis for SCC: [{gmc_root_sup,init,1}] Typesig analysis for SCC: [{gmcontroller,start,2}] Typesig analysis for SCC: [{messages,recv,3}] Typesig analysis for SCC: [{messages,send,4}] Typesig analysis for SCC: [{server_protocol_client,do_request_play_lists,5}] I am not sure in what order the log messages are given, but both handle_call and do_request_play_lists specifications look fine to me and are accepted by the compiler: -spec do_request_play_lists(target_id(), [p_date()], key(), ivec(), ivec()) -> {ivec(), ivec()}. -spec handle_call(any(), pid(), state()) -> handle_call_return(state()). --verbose option (which is listed in dialyzer --help) also doesn't seem particularly useful here (there is quite a bit of time before the error comes up). Any ideas how the error can be fixed? Yours, Alexey Romanov From gleber.p@REDACTED Fri Oct 8 16:00:09 2010 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 8 Oct 2010 16:00:09 +0200 Subject: 'undef' while doing erl boot on release Message-ID: Hi I have the following structure (which is created automatically by scripts from dev environment): release/Vsn/App1/ebin/ /*.beam /*.app /App2/ebin/ /*.beam /*.app release/Vsn/system.rel system.rel contains all appropriate version info about apps. Boot script is done with systools:make_script/1 and release bundle is done with systools:make_tar/1 (i.e. without erts inside). This gives me a release bundle, which I copy to another place and extract. It gives me the following structure: $DIR/system/lib/App1/ebin/ /*.beam /*.app /App2/ebin/ /*.beam /*.app $DIR/system/releases/system.rel /Vsn/start.boot and if I do $ cd $DIR/system && erl -boot releases/Vsn/start.boot I get the following error: =CRASH REPORT==== 8-Oct-2010::15:46:26 === crasher: initial call: application_master:init/4 pid: <0.39.0> registered_name: [] exception exit: {bad_return, {{ibrowse_app,start,[normal,[]]}, {'EXIT', {undef, [{ibrowse_app,start,[normal,[]]}, {application_master,start_it_old,4}]}}}} in function application_master:init/4 ancestors: [<0.38.0>] messages: [{'EXIT',<0.40.0>,normal}] links: [<0.38.0>,<0.6.0>] dictionary: [] trap_exit: true status: running heap_size: 377 stack_size: 24 reductions: 105 neighbours: If I do $ cd $DIR/system && erl -pa lib/*/ebin Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> ibrowse_app:start(normal, []). {ok,<0.33.0>} Script file created previously by systools:make_script/1 does contain appropriate primLoad commands, application:load/1 and application:start_boot(ibrowse, permanent) commands. I've double checked - ibrowse is not installed anywhere else on the system. I'm out of ideas why this could happen. Any thoughts on this? Best, Gleb Peregud From icarus.alive@REDACTED Fri Oct 8 16:08:13 2010 From: icarus.alive@REDACTED (Icarus Alive) Date: Fri, 8 Oct 2010 19:38:13 +0530 Subject: [erlang-questions] Any video tutorials for Erlang ? In-Reply-To: References: Message-ID: Thanks for the excellent pointer. Just checked out the introductory video -- very promising, the best part being, the coverage of interaction with Python client -- exactly what I needed. Price seems just right, and video/voice quality was good. Wish there was a teaser of the actual tutorial, something like the first 10-15 seconds of 2-3 chapters. Anyhow, will certainly check out few tutorials. ~Icarus. On Fri, Oct 8, 2010 at 7:05 PM, Doug Fort wrote: > The Pragmatic guys have have some screencasts that I found worthwhile: > > http://pragprog.com/screencasts/v-kserl/erlang-in-practice > > You have to pay for them, but IMO they are worth the money. > > On Fri, Oct 8, 2010 at 9:27 AM, Icarus Alive wrote: >> Hi, >> >> Am learning Erlang, and have extremely limited exposure to "Functional >> Programming", but some good-bit (15yrs) to Procedural and >> Object-Oriented programming. An ebook, called "Thinking in Erlang" has >> really been very helpful in my initial gentle rampup, and I think so >> far, am able to cope. That, alongwith the Erlang references, has been >> pretty decent. However, to hasten things up a bit, was wondering if >> anyone is aware of some good source of video tutorials for Erlang. >> These days, the youtube videos from the channels like google-io, on >> langs s.a. Python, RoR etc. are a-plenty, but inspite of significant >> buzz around Erlang, didn't find much around Erlang. So there >> some-place, I missed ? Just curious. >> >> BTW, if this is not the right place to ask this question, would >> appreciate pointer to the right forum. >> >> ~Icarus. >> >> From kostis@REDACTED Fri Oct 8 16:07:39 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 08 Oct 2010 17:07:39 +0300 Subject: [erlang-questions] Dialyzer crash In-Reply-To: References: Message-ID: <4CAF25AB.4090301@cs.ntua.gr> Alexey Romanov wrote: > I've got this error when trying to run dialyzer (from R14B) on my project: > > aromanov@REDACTED:~/workspace/gm-controller$ dialyzer --verbose > --build_plt --output_plt dialyzer/gmcontroller_1.plt -r ebin > Compiling some key modules to native code... done in 1m41.75s > Creating PLT dialyzer/gmcontroller_1.plt ... > =ERROR REPORT==== 8-Oct-2010::17:29:59 === > ... SNIP > > dialyzer: Analysis failed with error: > {function_clause,[{erl_types,inf_tuples_in_sets,4}, > {erl_types,inf_tuple_sets,4}, > {erl_types,inf_tuple_sets,3}, > {erl_types,inf_union,5}, > {erl_types,t_inf_lists_strict,4}, > {erl_types,t_inf,3}, > {dialyzer_typesig,solve_one_c,...}, > {dialyzer_typesig,...}]} > Last messages in the log cache: > Typesig analysis for SCC: [{server_protocol_client,handle_call,3}] > Typesig analysis for SCC: [{config,read_file,1}] > Typesig analysis for SCC: [{config,merge_with_file,2}] > Typesig analysis for SCC: [{config,read,1}] > Typesig analysis for SCC: [{device,hardware_capabilities_test,0}] > Typesig analysis for SCC: [{gmc_root_sup,init,1}] > Typesig analysis for SCC: [{gmcontroller,start,2}] > Typesig analysis for SCC: [{messages,recv,3}] > Typesig analysis for SCC: [{messages,send,4}] > Typesig analysis for SCC: [{server_protocol_client,do_request_play_lists,5}] > > > I am not sure in what order the log messages are given, but both > handle_call and do_request_play_lists specifications look fine to me > and are accepted by the compiler: > > -spec do_request_play_lists(target_id(), [p_date()], key(), ivec(), ivec()) -> > {ivec(), ivec()}. > > -spec handle_call(any(), pid(), state()) -> handle_call_return(state()). > > --verbose option (which is listed in dialyzer --help) also doesn't > seem particularly useful here (there is quite a bit of time before the > error comes up). Any ideas how the error can be fixed? The error can be fixed by identifying a minimal set of modules that trigger the same crash and sending it to me -- off list is probably preferable. I might be wrong of course, but I suspect this is related to the handle_call_return(state()) type which is a parameterized one -- we have not seen many uses of those. Kostis From davidnwelton@REDACTED Fri Oct 8 16:10:39 2010 From: davidnwelton@REDACTED (David Welton) Date: Fri, 8 Oct 2010 16:10:39 +0200 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: > The fun stuff in Erlang has to do with how the failure model interacts with > code changing, moving code around, upgrading code without stopping the system > and so on - these characteristics are extremely important if you want to > build a 24x7 system with zero down time - less so if you just want to serve up > pages as fast as possible and don't care if you take the system out of service > for upgrades or errors. > > Erlang was designed for building fault-tolerant systems - node.js was not node.js was designed for creating mostly asynchronous servers, with most people using it for the web, a space where it competes with Erlang to some degree. I wrote the journal entry comparing node.js to Erlang referenced in this thread, and my intuition is that while node.js is not as "good" as Erlang, it's "good enough" for most web people doing asynch web stuff like comet. And, given its popularity and industry adoption, it will likely continue to improve, if it's managed well. I see Erlang getting eclipsed by node.js in the web space, and remaining something of a niche system for people who really do need all the fault-tolerant, 24/7, etc... stuff. Erlang's advantages there are obvious. However, mostly, the web world gets by without those things - they may be nice, but most people get by ok without them. It's a pity to see Erlang remain in that narrow niche, though, because Erlang would benefit from the "network effects" of increased popularity that its use in building web applications would bring, and, in the same vein, would likely make Erlang a more valuable skill for those of us who know it. -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From gleber.p@REDACTED Fri Oct 8 16:22:30 2010 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 8 Oct 2010 16:22:30 +0200 Subject: 'undef' while doing erl boot on release In-Reply-To: References: Message-ID: On Fri, Oct 8, 2010 at 16:00, Gleb Peregud wrote: > Hi > > I have the following structure (which is created automatically by > scripts from dev environment): > > release/Vsn/App1/ebin/ > ? ? ? ? ? ? ? ? ? ? /*.beam > ? ? ? ? ? ? ? ? ? ? /*.app > ? ? ? ? ? /App2/ebin/ > ? ? ? ? ? ? ? ? ? ? /*.beam > ? ? ? ? ? ? ? ? ? ? /*.app > release/Vsn/system.rel > > system.rel contains all appropriate version info about apps. Boot > script is done with systools:make_script/1 and release bundle is done > with systools:make_tar/1 (i.e. without erts inside). This gives me a > release bundle, which I copy to another place and extract. > > It gives me the following structure: > > $DIR/system/lib/App1/ebin/ > ? ? ? ? ? ? ? ? ? ? ? ? /*.beam > ? ? ? ? ? ? ? ? ? ? ? ? /*.app > ? ? ? ? ? ? ? /App2/ebin/ > ? ? ? ? ? ? ? ? ? ? ? ? /*.beam > ? ? ? ? ? ? ? ? ? ? ? ? /*.app > $DIR/system/releases/system.rel > ? ? ? ? ? ? ? ? ? ?/Vsn/start.boot > > and if I do > > $ cd $DIR/system && erl -boot releases/Vsn/start.boot > > I get the following error: > > =CRASH REPORT==== 8-Oct-2010::15:46:26 === > ?crasher: > ? ?initial call: application_master:init/4 > ? ?pid: <0.39.0> > ? ?registered_name: [] > ? ?exception exit: {bad_return, > ? ? ? ? ? ? ? ? ? ? ? ?{{ibrowse_app,start,[normal,[]]}, > ? ? ? ? ? ? ? ? ? ? ? ? {'EXIT', > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {undef, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [{ibrowse_app,start,[normal,[]]}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{application_master,start_it_old,4}]}}}} > ? ? ?in function ?application_master:init/4 > ? ?ancestors: [<0.38.0>] > ? ?messages: [{'EXIT',<0.40.0>,normal}] > ? ?links: [<0.38.0>,<0.6.0>] > ? ?dictionary: [] > ? ?trap_exit: true > ? ?status: running > ? ?heap_size: 377 > ? ?stack_size: 24 > ? ?reductions: 105 > ?neighbours: > > > If I do > > $ cd $DIR/system && erl -pa lib/*/ebin > Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.8.1 ?(abort with ^G) > 1> ibrowse_app:start(normal, []). > {ok,<0.33.0>} > > > Script file created previously by systools:make_script/1 does contain > appropriate primLoad commands, application:load/1 and > application:start_boot(ibrowse, permanent) commands. I've double > checked - ibrowse is not installed anywhere else on the system. > > I'm out of ideas why this could happen. Any thoughts on this? > > Best, > Gleb Peregud > Just to add more info. Starting ibrowse from console works perfectly well: $ cd $DIR/system && erl -pa lib/*/ebin Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> application:start(sasl). ok 2> =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === supervisor: {local,sasl_safe_sup} started: [{pid,<0.38.0>}, {name,alarm_handler}, {mfargs,{alarm_handler,start_link,[]}}, {restart_type,permanent}, {shutdown,2000}, {child_type,worker}] =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === supervisor: {local,sasl_safe_sup} started: [{pid,<0.39.0>}, {name,overload}, {mfargs,{overload,start_link,[]}}, {restart_type,permanent}, {shutdown,2000}, {child_type,worker}] =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === supervisor: {local,sasl_sup} started: [{pid,<0.37.0>}, {name,sasl_safe_sup}, {mfargs, {supervisor,start_link, [{local,sasl_safe_sup},sasl,safe]}}, {restart_type,permanent}, {shutdown,infinity}, {child_type,supervisor}] =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === supervisor: {local,sasl_sup} started: [{pid,<0.40.0>}, {name,release_handler}, {mfargs,{release_handler,start_link,[]}}, {restart_type,permanent}, {shutdown,2000}, {child_type,worker}] =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === application: sasl started_at: nonode@REDACTED 2> application:start(ibrowse). =PROGRESS REPORT==== 8-Oct-2010::16:21:30 === supervisor: {local,ibrowse_sup} started: [{pid,<0.47.0>}, {name,ibrowse}, {mfargs,{ibrowse,start_link,[]}}, {restart_type,permanent}, {shutdown,2000}, {child_type,worker}] =PROGRESS REPORT==== 8-Oct-2010::16:21:30 === application: ibrowse started_at: nonode@REDACTED ok 3> From ulf.wiger@REDACTED Fri Oct 8 16:35:51 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 8 Oct 2010 16:35:51 +0200 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> Message-ID: On 8 Oct 2010, at 16:10, David Welton wrote: > > I see Erlang getting eclipsed by node.js in the web space, and > remaining something of a niche system for people who really do need > all the fault-tolerant, 24/7, etc... stuff. Erlang's advantages there > are obvious. However, mostly, the web world gets by without those > things - they may be nice, but most people get by ok without them. > > It's a pity to see Erlang remain in that narrow niche, though, because > Erlang would benefit from the "network effects" of increased > popularity that its use in building web applications would bring, and, > in the same vein, would likely make Erlang a more valuable skill for > those of us who know it. In a sense, this is a problem plaguing all technologies that require a bit of extra effort up front in order to save time and money down the road. Erlang comes off as being strange by definition, since it is not OO, Java- like, or basically similar to anything mainstream. Also, the advantages of Erlang are perhaps not so readily apparent, as principles for massive concurrency and the issues of exception flows and exception handling in concurrent systems are not part of the regular curriculum. Add to that the "pioneering nature" of much in the Web space, and you will have a large group of people who go for the low-hanging fruit, and defer any problems of scale, robustness and complexity until later. It will be a combination of knowingly deferring some problems, and of not even realising what lies ahead. I don't necessarily view this as a bad thing. Much innovation might have been stifled by too much knowledge of the difficulties lurking around the corner. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From lukas@REDACTED Fri Oct 8 16:44:10 2010 From: lukas@REDACTED (Lukas Larsson) Date: Fri, 08 Oct 2010 16:44:10 +0200 Subject: [erlang-questions] Re: 'undef' while doing erl boot on release In-Reply-To: References: Message-ID: <1286549050.5557.15.camel@ancalagon.du.uab.ericsson.se> Hi, You probably have an OS env set to point to your erlang installation and not to your own release (the erl script does this by default if you look into $PATH_TO_ERLANG_INSTALL/bin/erl). By setting OTP_TOP to $DIR/system or LIB_DIRS to $DIR/system/lib before executing erl -boot it should work. See code path section of http://www.erlang.org/doc/man/code.html for details on what these settings do and http://www.erlang.org/doc/man/erl.html for details about which OS envs effect the erlang VM. Lukas On Fri, 2010-10-08 at 16:22 +0200, Gleb Peregud wrote: > On Fri, Oct 8, 2010 at 16:00, Gleb Peregud wrote: > > Hi > > > > I have the following structure (which is created automatically by > > scripts from dev environment): > > > > release/Vsn/App1/ebin/ > > /*.beam > > /*.app > > /App2/ebin/ > > /*.beam > > /*.app > > release/Vsn/system.rel > > > > system.rel contains all appropriate version info about apps. Boot > > script is done with systools:make_script/1 and release bundle is done > > with systools:make_tar/1 (i.e. without erts inside). This gives me a > > release bundle, which I copy to another place and extract. > > > > It gives me the following structure: > > > > $DIR/system/lib/App1/ebin/ > > /*.beam > > /*.app > > /App2/ebin/ > > /*.beam > > /*.app > > $DIR/system/releases/system.rel > > /Vsn/start.boot > > > > and if I do > > > > $ cd $DIR/system && erl -boot releases/Vsn/start.boot > > > > I get the following error: > > > > =CRASH REPORT==== 8-Oct-2010::15:46:26 === > > crasher: > > initial call: application_master:init/4 > > pid: <0.39.0> > > registered_name: [] > > exception exit: {bad_return, > > {{ibrowse_app,start,[normal,[]]}, > > {'EXIT', > > {undef, > > [{ibrowse_app,start,[normal,[]]}, > > {application_master,start_it_old,4}]}}}} > > in function application_master:init/4 > > ancestors: [<0.38.0>] > > messages: [{'EXIT',<0.40.0>,normal}] > > links: [<0.38.0>,<0.6.0>] > > dictionary: [] > > trap_exit: true > > status: running > > heap_size: 377 > > stack_size: 24 > > reductions: 105 > > neighbours: > > > > > > If I do > > > > $ cd $DIR/system && erl -pa lib/*/ebin > > Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] > > [hipe] [kernel-poll:false] > > > > Eshell V5.8.1 (abort with ^G) > > 1> ibrowse_app:start(normal, []). > > {ok,<0.33.0>} > > > > > > Script file created previously by systools:make_script/1 does contain > > appropriate primLoad commands, application:load/1 and > > application:start_boot(ibrowse, permanent) commands. I've double > > checked - ibrowse is not installed anywhere else on the system. > > > > I'm out of ideas why this could happen. Any thoughts on this? > > > > Best, > > Gleb Peregud > > > > Just to add more info. Starting ibrowse from console works perfectly well: > > $ cd $DIR/system && erl -pa lib/*/ebin > Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > 1> application:start(sasl). > ok > 2> > =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > supervisor: {local,sasl_safe_sup} > started: [{pid,<0.38.0>}, > {name,alarm_handler}, > {mfargs,{alarm_handler,start_link,[]}}, > {restart_type,permanent}, > {shutdown,2000}, > {child_type,worker}] > > =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > supervisor: {local,sasl_safe_sup} > started: [{pid,<0.39.0>}, > {name,overload}, > {mfargs,{overload,start_link,[]}}, > {restart_type,permanent}, > {shutdown,2000}, > {child_type,worker}] > > =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > supervisor: {local,sasl_sup} > started: [{pid,<0.37.0>}, > {name,sasl_safe_sup}, > {mfargs, > {supervisor,start_link, > [{local,sasl_safe_sup},sasl,safe]}}, > {restart_type,permanent}, > {shutdown,infinity}, > {child_type,supervisor}] > > =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > supervisor: {local,sasl_sup} > started: [{pid,<0.40.0>}, > {name,release_handler}, > {mfargs,{release_handler,start_link,[]}}, > {restart_type,permanent}, > {shutdown,2000}, > {child_type,worker}] > > =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > application: sasl > started_at: nonode@REDACTED > 2> application:start(ibrowse). > > =PROGRESS REPORT==== 8-Oct-2010::16:21:30 === > supervisor: {local,ibrowse_sup} > started: [{pid,<0.47.0>}, > {name,ibrowse}, > {mfargs,{ibrowse,start_link,[]}}, > {restart_type,permanent}, > {shutdown,2000}, > {child_type,worker}] > > =PROGRESS REPORT==== 8-Oct-2010::16:21:30 === > application: ibrowse > started_at: nonode@REDACTED > ok > 3> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From spawn.think@REDACTED Fri Oct 8 16:44:56 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Fri, 8 Oct 2010 16:44:56 +0200 Subject: [erlang-questions] Re: 'undef' while doing erl boot on release In-Reply-To: References: Message-ID: Check the script file for the paths , could you post the part of ibrowse? On Fri, Oct 8, 2010 at 4:22 PM, Gleb Peregud wrote: > On Fri, Oct 8, 2010 at 16:00, Gleb Peregud wrote: > > Hi > > > > I have the following structure (which is created automatically by > > scripts from dev environment): > > > > release/Vsn/App1/ebin/ > > /*.beam > > /*.app > > /App2/ebin/ > > /*.beam > > /*.app > > release/Vsn/system.rel > > > > system.rel contains all appropriate version info about apps. Boot > > script is done with systools:make_script/1 and release bundle is done > > with systools:make_tar/1 (i.e. without erts inside). This gives me a > > release bundle, which I copy to another place and extract. > > > > It gives me the following structure: > > > > $DIR/system/lib/App1/ebin/ > > /*.beam > > /*.app > > /App2/ebin/ > > /*.beam > > /*.app > > $DIR/system/releases/system.rel > > /Vsn/start.boot > > > > and if I do > > > > $ cd $DIR/system && erl -boot releases/Vsn/start.boot > > > > I get the following error: > > > > =CRASH REPORT==== 8-Oct-2010::15:46:26 === > > crasher: > > initial call: application_master:init/4 > > pid: <0.39.0> > > registered_name: [] > > exception exit: {bad_return, > > {{ibrowse_app,start,[normal,[]]}, > > {'EXIT', > > {undef, > > [{ibrowse_app,start,[normal,[]]}, > > {application_master,start_it_old,4}]}}}} > > in function application_master:init/4 > > ancestors: [<0.38.0>] > > messages: [{'EXIT',<0.40.0>,normal}] > > links: [<0.38.0>,<0.6.0>] > > dictionary: [] > > trap_exit: true > > status: running > > heap_size: 377 > > stack_size: 24 > > reductions: 105 > > neighbours: > > > > > > If I do > > > > $ cd $DIR/system && erl -pa lib/*/ebin > > Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] > > [hipe] [kernel-poll:false] > > > > Eshell V5.8.1 (abort with ^G) > > 1> ibrowse_app:start(normal, []). > > {ok,<0.33.0>} > > > > > > Script file created previously by systools:make_script/1 does contain > > appropriate primLoad commands, application:load/1 and > > application:start_boot(ibrowse, permanent) commands. I've double > > checked - ibrowse is not installed anywhere else on the system. > > > > I'm out of ideas why this could happen. Any thoughts on this? > > > > Best, > > Gleb Peregud > > > > Just to add more info. Starting ibrowse from console works perfectly well: > > $ cd $DIR/system && erl -pa lib/*/ebin > Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > 1> application:start(sasl). > ok > 2> > =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > supervisor: {local,sasl_safe_sup} > started: [{pid,<0.38.0>}, > {name,alarm_handler}, > {mfargs,{alarm_handler,start_link,[]}}, > {restart_type,permanent}, > {shutdown,2000}, > {child_type,worker}] > > =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > supervisor: {local,sasl_safe_sup} > started: [{pid,<0.39.0>}, > {name,overload}, > {mfargs,{overload,start_link,[]}}, > {restart_type,permanent}, > {shutdown,2000}, > {child_type,worker}] > > =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > supervisor: {local,sasl_sup} > started: [{pid,<0.37.0>}, > {name,sasl_safe_sup}, > {mfargs, > {supervisor,start_link, > [{local,sasl_safe_sup},sasl,safe]}}, > {restart_type,permanent}, > {shutdown,infinity}, > {child_type,supervisor}] > > =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > supervisor: {local,sasl_sup} > started: [{pid,<0.40.0>}, > {name,release_handler}, > {mfargs,{release_handler,start_link,[]}}, > {restart_type,permanent}, > {shutdown,2000}, > {child_type,worker}] > > =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > application: sasl > started_at: nonode@REDACTED > 2> application:start(ibrowse). > > =PROGRESS REPORT==== 8-Oct-2010::16:21:30 === > supervisor: {local,ibrowse_sup} > started: [{pid,<0.47.0>}, > {name,ibrowse}, > {mfargs,{ibrowse,start_link,[]}}, > {restart_type,permanent}, > {shutdown,2000}, > {child_type,worker}] > > =PROGRESS REPORT==== 8-Oct-2010::16:21:30 === > application: ibrowse > started_at: nonode@REDACTED > ok > 3> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From gleber.p@REDACTED Fri Oct 8 17:05:31 2010 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 8 Oct 2010 17:05:31 +0200 Subject: [erlang-questions] Re: 'undef' while doing erl boot on release In-Reply-To: <1286549050.5557.15.camel@ancalagon.du.uab.ericsson.se> References: <1286549050.5557.15.camel@ancalagon.du.uab.ericsson.se> Message-ID: Indeed using ERL_LIBS fixes it: $ cd $DIR/system $ ERL_LIBS=`pwd`/lib/ erl -boot releases/0.0.78/start Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] .............. =PROGRESS REPORT==== 8-Oct-2010::17:03:11 === supervisor: {local,ibrowse_sup} started: [{pid,<0.42.0>}, {name,ibrowse}, {mfargs,{ibrowse,start_link,[]}}, {restart_type,permanent}, {shutdown,2000}, {child_type,worker}] =PROGRESS REPORT==== 8-Oct-2010::17:03:11 === application: ibrowse started_at: nonode@REDACTED Eshell V5.8.1 (abort with ^G) 1> I thought that boot script does set up paths. Why doesn't it? Can I make it to do so? Thanks a lot! Gleb On Fri, Oct 8, 2010 at 16:44, Lukas Larsson wrote: > Hi, > > You probably have an OS env set to point to your erlang installation and > not to your own release (the erl script does this by default if you look > into $PATH_TO_ERLANG_INSTALL/bin/erl). By setting OTP_TOP to $DIR/system > or LIB_DIRS to $DIR/system/lib before executing erl -boot it should > work. > > See code path section of http://www.erlang.org/doc/man/code.html for > details on what these settings do and > http://www.erlang.org/doc/man/erl.html for details about which OS envs > effect the erlang VM. > > Lukas > > On Fri, 2010-10-08 at 16:22 +0200, Gleb Peregud wrote: >> On Fri, Oct 8, 2010 at 16:00, Gleb Peregud wrote: >> > Hi >> > >> > I have the following structure (which is created automatically by >> > scripts from dev environment): >> > >> > release/Vsn/App1/ebin/ >> > ? ? ? ? ? ? ? ? ? ? /*.beam >> > ? ? ? ? ? ? ? ? ? ? /*.app >> > ? ? ? ? ? /App2/ebin/ >> > ? ? ? ? ? ? ? ? ? ? /*.beam >> > ? ? ? ? ? ? ? ? ? ? /*.app >> > release/Vsn/system.rel >> > >> > system.rel contains all appropriate version info about apps. Boot >> > script is done with systools:make_script/1 and release bundle is done >> > with systools:make_tar/1 (i.e. without erts inside). This gives me a >> > release bundle, which I copy to another place and extract. >> > >> > It gives me the following structure: >> > >> > $DIR/system/lib/App1/ebin/ >> > ? ? ? ? ? ? ? ? ? ? ? ? /*.beam >> > ? ? ? ? ? ? ? ? ? ? ? ? /*.app >> > ? ? ? ? ? ? ? /App2/ebin/ >> > ? ? ? ? ? ? ? ? ? ? ? ? /*.beam >> > ? ? ? ? ? ? ? ? ? ? ? ? /*.app >> > $DIR/system/releases/system.rel >> > ? ? ? ? ? ? ? ? ? ?/Vsn/start.boot >> > >> > and if I do >> > >> > $ cd $DIR/system && erl -boot releases/Vsn/start.boot >> > >> > I get the following error: >> > >> > =CRASH REPORT==== 8-Oct-2010::15:46:26 === >> > ?crasher: >> > ? ?initial call: application_master:init/4 >> > ? ?pid: <0.39.0> >> > ? ?registered_name: [] >> > ? ?exception exit: {bad_return, >> > ? ? ? ? ? ? ? ? ? ? ? ?{{ibrowse_app,start,[normal,[]]}, >> > ? ? ? ? ? ? ? ? ? ? ? ? {'EXIT', >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {undef, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [{ibrowse_app,start,[normal,[]]}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{application_master,start_it_old,4}]}}}} >> > ? ? ?in function ?application_master:init/4 >> > ? ?ancestors: [<0.38.0>] >> > ? ?messages: [{'EXIT',<0.40.0>,normal}] >> > ? ?links: [<0.38.0>,<0.6.0>] >> > ? ?dictionary: [] >> > ? ?trap_exit: true >> > ? ?status: running >> > ? ?heap_size: 377 >> > ? ?stack_size: 24 >> > ? ?reductions: 105 >> > ?neighbours: >> > >> > >> > If I do >> > >> > $ cd $DIR/system && erl -pa lib/*/ebin >> > Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] >> > [hipe] [kernel-poll:false] >> > >> > Eshell V5.8.1 ?(abort with ^G) >> > 1> ibrowse_app:start(normal, []). >> > {ok,<0.33.0>} >> > >> > >> > Script file created previously by systools:make_script/1 does contain >> > appropriate primLoad commands, application:load/1 and >> > application:start_boot(ibrowse, permanent) commands. I've double >> > checked - ibrowse is not installed anywhere else on the system. >> > >> > I'm out of ideas why this could happen. Any thoughts on this? >> > >> > Best, >> > Gleb Peregud >> > >> >> Just to add more info. Starting ibrowse from console works perfectly well: >> >> $ cd $DIR/system && erl -pa lib/*/ebin >> Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] >> [hipe] [kernel-poll:false] >> >> Eshell V5.8.1 ?(abort with ^G) >> 1> application:start(sasl). >> ok >> 2> >> =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === >> ? ? ? ? ? supervisor: {local,sasl_safe_sup} >> ? ? ? ? ? ? ?started: [{pid,<0.38.0>}, >> ? ? ? ? ? ? ? ? ? ? ? ?{name,alarm_handler}, >> ? ? ? ? ? ? ? ? ? ? ? ?{mfargs,{alarm_handler,start_link,[]}}, >> ? ? ? ? ? ? ? ? ? ? ? ?{restart_type,permanent}, >> ? ? ? ? ? ? ? ? ? ? ? ?{shutdown,2000}, >> ? ? ? ? ? ? ? ? ? ? ? ?{child_type,worker}] >> >> =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === >> ? ? ? ? ? supervisor: {local,sasl_safe_sup} >> ? ? ? ? ? ? ?started: [{pid,<0.39.0>}, >> ? ? ? ? ? ? ? ? ? ? ? ?{name,overload}, >> ? ? ? ? ? ? ? ? ? ? ? ?{mfargs,{overload,start_link,[]}}, >> ? ? ? ? ? ? ? ? ? ? ? ?{restart_type,permanent}, >> ? ? ? ? ? ? ? ? ? ? ? ?{shutdown,2000}, >> ? ? ? ? ? ? ? ? ? ? ? ?{child_type,worker}] >> >> =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === >> ? ? ? ? ? supervisor: {local,sasl_sup} >> ? ? ? ? ? ? ?started: [{pid,<0.37.0>}, >> ? ? ? ? ? ? ? ? ? ? ? ?{name,sasl_safe_sup}, >> ? ? ? ? ? ? ? ? ? ? ? ?{mfargs, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ?{supervisor,start_link, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[{local,sasl_safe_sup},sasl,safe]}}, >> ? ? ? ? ? ? ? ? ? ? ? ?{restart_type,permanent}, >> ? ? ? ? ? ? ? ? ? ? ? ?{shutdown,infinity}, >> ? ? ? ? ? ? ? ? ? ? ? ?{child_type,supervisor}] >> >> =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === >> ? ? ? ? ? supervisor: {local,sasl_sup} >> ? ? ? ? ? ? ?started: [{pid,<0.40.0>}, >> ? ? ? ? ? ? ? ? ? ? ? ?{name,release_handler}, >> ? ? ? ? ? ? ? ? ? ? ? ?{mfargs,{release_handler,start_link,[]}}, >> ? ? ? ? ? ? ? ? ? ? ? ?{restart_type,permanent}, >> ? ? ? ? ? ? ? ? ? ? ? ?{shutdown,2000}, >> ? ? ? ? ? ? ? ? ? ? ? ?{child_type,worker}] >> >> =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === >> ? ? ? ? ?application: sasl >> ? ? ? ? ? started_at: nonode@REDACTED >> 2> application:start(ibrowse). >> >> =PROGRESS REPORT==== 8-Oct-2010::16:21:30 === >> ? ? ? ? ? supervisor: {local,ibrowse_sup} >> ? ? ? ? ? ? ?started: [{pid,<0.47.0>}, >> ? ? ? ? ? ? ? ? ? ? ? ?{name,ibrowse}, >> ? ? ? ? ? ? ? ? ? ? ? ?{mfargs,{ibrowse,start_link,[]}}, >> ? ? ? ? ? ? ? ? ? ? ? ?{restart_type,permanent}, >> ? ? ? ? ? ? ? ? ? ? ? ?{shutdown,2000}, >> ? ? ? ? ? ? ? ? ? ? ? ?{child_type,worker}] >> >> =PROGRESS REPORT==== 8-Oct-2010::16:21:30 === >> ? ? ? ? ?application: ibrowse >> ? ? ? ? ? started_at: nonode@REDACTED >> ok >> 3> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > From lukas@REDACTED Fri Oct 8 17:13:58 2010 From: lukas@REDACTED (Lukas Larsson) Date: Fri, 08 Oct 2010 17:13:58 +0200 Subject: [erlang-questions] Re: 'undef' while doing erl boot on release In-Reply-To: References: <1286549050.5557.15.camel@ancalagon.du.uab.ericsson.se> Message-ID: <1286550838.5557.21.camel@ancalagon.du.uab.ericsson.se> Good thing that it worked! You might try adding {path,["lib/*/ebin"]} as an argument to your make_script call. You also might want to have a look at the reltool application and see what it can do for you. It is designed to take care of all these things for you. For examples of how to use it see: http://www.erlang.org/doc/apps/reltool/reltool_examples.html Lukas On Fri, 2010-10-08 at 17:05 +0200, Gleb Peregud wrote: > Indeed using ERL_LIBS fixes it: > > $ cd $DIR/system > $ ERL_LIBS=`pwd`/lib/ erl -boot releases/0.0.78/start > > Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > > .............. > > =PROGRESS REPORT==== 8-Oct-2010::17:03:11 === > supervisor: {local,ibrowse_sup} > started: [{pid,<0.42.0>}, > {name,ibrowse}, > {mfargs,{ibrowse,start_link,[]}}, > {restart_type,permanent}, > {shutdown,2000}, > {child_type,worker}] > > =PROGRESS REPORT==== 8-Oct-2010::17:03:11 === > application: ibrowse > started_at: nonode@REDACTED > Eshell V5.8.1 (abort with ^G) > 1> > > I thought that boot script does set up paths. Why doesn't it? Can I > make it to do so? > > Thanks a lot! > Gleb > > On Fri, Oct 8, 2010 at 16:44, Lukas Larsson wrote: > > Hi, > > > > You probably have an OS env set to point to your erlang installation and > > not to your own release (the erl script does this by default if you look > > into $PATH_TO_ERLANG_INSTALL/bin/erl). By setting OTP_TOP to $DIR/system > > or LIB_DIRS to $DIR/system/lib before executing erl -boot it should > > work. > > > > See code path section of http://www.erlang.org/doc/man/code.html for > > details on what these settings do and > > http://www.erlang.org/doc/man/erl.html for details about which OS envs > > effect the erlang VM. > > > > Lukas > > > > On Fri, 2010-10-08 at 16:22 +0200, Gleb Peregud wrote: > >> On Fri, Oct 8, 2010 at 16:00, Gleb Peregud wrote: > >> > Hi > >> > > >> > I have the following structure (which is created automatically by > >> > scripts from dev environment): > >> > > >> > release/Vsn/App1/ebin/ > >> > /*.beam > >> > /*.app > >> > /App2/ebin/ > >> > /*.beam > >> > /*.app > >> > release/Vsn/system.rel > >> > > >> > system.rel contains all appropriate version info about apps. Boot > >> > script is done with systools:make_script/1 and release bundle is done > >> > with systools:make_tar/1 (i.e. without erts inside). This gives me a > >> > release bundle, which I copy to another place and extract. > >> > > >> > It gives me the following structure: > >> > > >> > $DIR/system/lib/App1/ebin/ > >> > /*.beam > >> > /*.app > >> > /App2/ebin/ > >> > /*.beam > >> > /*.app > >> > $DIR/system/releases/system.rel > >> > /Vsn/start.boot > >> > > >> > and if I do > >> > > >> > $ cd $DIR/system && erl -boot releases/Vsn/start.boot > >> > > >> > I get the following error: > >> > > >> > =CRASH REPORT==== 8-Oct-2010::15:46:26 === > >> > crasher: > >> > initial call: application_master:init/4 > >> > pid: <0.39.0> > >> > registered_name: [] > >> > exception exit: {bad_return, > >> > {{ibrowse_app,start,[normal,[]]}, > >> > {'EXIT', > >> > {undef, > >> > [{ibrowse_app,start,[normal,[]]}, > >> > {application_master,start_it_old,4}]}}}} > >> > in function application_master:init/4 > >> > ancestors: [<0.38.0>] > >> > messages: [{'EXIT',<0.40.0>,normal}] > >> > links: [<0.38.0>,<0.6.0>] > >> > dictionary: [] > >> > trap_exit: true > >> > status: running > >> > heap_size: 377 > >> > stack_size: 24 > >> > reductions: 105 > >> > neighbours: > >> > > >> > > >> > If I do > >> > > >> > $ cd $DIR/system && erl -pa lib/*/ebin > >> > Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] > >> > [hipe] [kernel-poll:false] > >> > > >> > Eshell V5.8.1 (abort with ^G) > >> > 1> ibrowse_app:start(normal, []). > >> > {ok,<0.33.0>} > >> > > >> > > >> > Script file created previously by systools:make_script/1 does contain > >> > appropriate primLoad commands, application:load/1 and > >> > application:start_boot(ibrowse, permanent) commands. I've double > >> > checked - ibrowse is not installed anywhere else on the system. > >> > > >> > I'm out of ideas why this could happen. Any thoughts on this? > >> > > >> > Best, > >> > Gleb Peregud > >> > > >> > >> Just to add more info. Starting ibrowse from console works perfectly well: > >> > >> $ cd $DIR/system && erl -pa lib/*/ebin > >> Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] > >> [hipe] [kernel-poll:false] > >> > >> Eshell V5.8.1 (abort with ^G) > >> 1> application:start(sasl). > >> ok > >> 2> > >> =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > >> supervisor: {local,sasl_safe_sup} > >> started: [{pid,<0.38.0>}, > >> {name,alarm_handler}, > >> {mfargs,{alarm_handler,start_link,[]}}, > >> {restart_type,permanent}, > >> {shutdown,2000}, > >> {child_type,worker}] > >> > >> =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > >> supervisor: {local,sasl_safe_sup} > >> started: [{pid,<0.39.0>}, > >> {name,overload}, > >> {mfargs,{overload,start_link,[]}}, > >> {restart_type,permanent}, > >> {shutdown,2000}, > >> {child_type,worker}] > >> > >> =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > >> supervisor: {local,sasl_sup} > >> started: [{pid,<0.37.0>}, > >> {name,sasl_safe_sup}, > >> {mfargs, > >> {supervisor,start_link, > >> [{local,sasl_safe_sup},sasl,safe]}}, > >> {restart_type,permanent}, > >> {shutdown,infinity}, > >> {child_type,supervisor}] > >> > >> =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > >> supervisor: {local,sasl_sup} > >> started: [{pid,<0.40.0>}, > >> {name,release_handler}, > >> {mfargs,{release_handler,start_link,[]}}, > >> {restart_type,permanent}, > >> {shutdown,2000}, > >> {child_type,worker}] > >> > >> =PROGRESS REPORT==== 8-Oct-2010::16:21:27 === > >> application: sasl > >> started_at: nonode@REDACTED > >> 2> application:start(ibrowse). > >> > >> =PROGRESS REPORT==== 8-Oct-2010::16:21:30 === > >> supervisor: {local,ibrowse_sup} > >> started: [{pid,<0.47.0>}, > >> {name,ibrowse}, > >> {mfargs,{ibrowse,start_link,[]}}, > >> {restart_type,permanent}, > >> {shutdown,2000}, > >> {child_type,worker}] > >> > >> =PROGRESS REPORT==== 8-Oct-2010::16:21:30 === > >> application: ibrowse > >> started_at: nonode@REDACTED > >> ok > >> 3> > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > From gleber.p@REDACTED Fri Oct 8 17:23:02 2010 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 8 Oct 2010 17:23:02 +0200 Subject: [erlang-questions] Re: 'undef' while doing erl boot on release In-Reply-To: <1286550838.5557.21.camel@ancalagon.du.uab.ericsson.se> References: <1286549050.5557.15.camel@ancalagon.du.uab.ericsson.se> <1286550838.5557.21.camel@ancalagon.du.uab.ericsson.se> Message-ID: On Fri, Oct 8, 2010 at 17:13, Lukas Larsson wrote: > You might try adding {path,["lib/*/ebin"]} as an argument to your > make_script call. It doesn't work out-of-the-box for me, but it's a good path - I'll investigate it. > You also might want to have a look at the reltool application and see > what it can do for you. It is designed to take care of all these things > for you. For examples of how to use it see: > > http://www.erlang.org/doc/apps/reltool/reltool_examples.html I found it a bit too hard to wrap my mind around it. Especially since structure of my project consists of 5 apps and about 10 dependencies apps and all of them are in different places and I want them to be loaded and selectively started. So I prepared few scripts which creates releases for me. But I'll give it another try at some point in future when things are more structured. Best, Gleb From ds.erl@REDACTED Fri Oct 8 18:36:42 2010 From: ds.erl@REDACTED (DS) Date: Fri, 8 Oct 2010 18:36:42 +0200 Subject: [erlang-questions] FreeBSD port for Erlang In-Reply-To: References: Message-ID: On 8 Oct 2010, at 15:22, Icarus Alive wrote: > Anyone has any pointers or updates to the latest FreeBSD port for > Erlang ? $ su - # cd /usr/ports/lang/erlang # make install Regards. -DS From fritchie@REDACTED Sat Oct 9 00:34:31 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Fri, 08 Oct 2010 17:34:31 -0500 Subject: [erlang-questions] Garbage collecting binaries In-Reply-To: Message of "Fri, 08 Oct 2010 12:45:16 BST." Message-ID: <20874.1286577271@snookles.snookles.com> Tom, one other thing to keep in mind ... if you pull in a huge binary, and then use pattern matching to "pick apart" that binary, the pattern matching is likely to create sub-binaries that still reference the original. So then the chain of who's got a reference to the huge binary gets more difficult to untangle: if you keep the small sub-binary around forever (e.g. as part of a long-lived process's state), then you'll keep the underlying "heap" binary around forever also. -Scott From anthonym@REDACTED Sat Oct 9 01:45:03 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Fri, 8 Oct 2010 16:45:03 -0700 Subject: [erlang-questions] node.js compared to erlang In-Reply-To: References: <838303.44876.qm@web112615.mail.gq1.yahoo.com> <00a601cb6629$ce8eabe0$6bac03a0$@com> Message-ID: <20101008234503.GB22424@alumni.caltech.edu> On Fri, Oct 08, 2010 at 12:18:32AM +0100, Dale Harvey wrote: > I think its time to settle on and promote tools like rebar / mochiweb / > learnyourselfsomeerlang / unknown package manager / and not wait for things > to be officially blessed by the otp team, they are busy making erlang > awesome, we as a community need to pick up the slack on that end. Just wanted to mention framewerk (http://code.google.com/p/fwtemplates/) and erlrc (http://code.google.com/p/erlrc/). The former provides an erlang template which allows you to quickly get a development package up and running which includes the following - generates a hello world style app when you initialize a new project - provides the building of .erl files using make - automatically builds .app and .appup files according to appup cookbook - has eunit, dialyzer and cover integration - builds debs, rpms or just tar.gz's. - manages dependencies and maps those to the packaging system you are using - integrates with your version control system to tag when you release a package It also integrates which erlc which makes deb and rpm attempt to use hot code loading when you install packages. I use this currently by running an erlang node on a machine and having everything I install packaged as an rpm. If a new rpm is available I can use yum/rpm install and it will attempt a hot code upgrade during the installation. It still has a few problems on RHEL but they are usually a result of broken code_change handlers. It's a great model, because you can have multiple applications versioned separately all running in the same node. If you need to launch to multiple nodes it ends up being as easy running for h in host1 ... host2 ; do ssh $h "yum install new_package" ; done. I've ended up repackaging many open source erlang projects as framewerk projects just so I can get erlrc and rpm integration. I've not played with rebar, but it seems to want to make a single application which is fine for a download and test sort of setup, but when deploying different applications to 100s of production machines, using standard packaging managers works really well. Anyway, I like it, and I think it brings something to the table in that it sort of skips the "lets build a package management system different from all other package management systems", and instead tries to work with the most common systems (rpm and deb). It's also possible to add other package management systems, so you could for instance get it working with faxien as an alternative packaging system, making it possible to release packages in many forms easily. -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From per@REDACTED Sat Oct 9 02:10:24 2010 From: per@REDACTED (Per Hedeland) Date: Sat, 9 Oct 2010 02:10:24 +0200 (CEST) Subject: [erlang-questions] FreeBSD port for Erlang In-Reply-To: Message-ID: <201010090010.o990AOE9049213@pluto.hedeland.org> DS wrote: > >On 8 Oct 2010, at 15:22, Icarus Alive wrote: >> Anyone has any pointers or updates to the latest FreeBSD port for >> Erlang ? > >$ su - ># cd /usr/ports/lang/erlang ># make install Indeed - the port was updated to R14B on Sept 27, a mere 12 days after the OTP release, I believe. Many thanks to the tireless maintainer, Jimmy Olgeni! --Per Hedeland From masklinn@REDACTED Sat Oct 9 09:28:57 2010 From: masklinn@REDACTED (Masklinn) Date: Sat, 9 Oct 2010 09:28:57 +0200 Subject: [erlang-questions] Garbage collecting binaries In-Reply-To: <20874.1286577271@snookles.snookles.com> References: <20874.1286577271@snookles.snookles.com> Message-ID: On 9 oct. 2010, at 00:34, Scott Lystig Fritchie wrote: > Tom, one other thing to keep in mind ... if you pull in a huge binary, > and then use pattern matching to "pick apart" that binary, the pattern > matching is likely to create sub-binaries that still reference the > original. So then the chain of who's got a reference to the huge binary > gets more difficult to untangle: if you keep the small sub-binary around > forever (e.g. as part of a long-lived process's state), then you'll keep > the underlying "heap" binary around forever also. > > -Scott On that subject, is there any way to force Erlang to copy the binary chunk in order to free it from its dependency on the bigger binary, trading a bit of CPU now for a lit of memory soon? From bob@REDACTED Sat Oct 9 09:42:02 2010 From: bob@REDACTED (Bob Ippolito) Date: Sat, 9 Oct 2010 15:42:02 +0800 Subject: [erlang-questions] Garbage collecting binaries In-Reply-To: References: <20874.1286577271@snookles.snookles.com> Message-ID: On Sat, Oct 9, 2010 at 3:28 PM, Masklinn wrote: > On 9 oct. 2010, at 00:34, Scott Lystig Fritchie wrote: >> Tom, one other thing to keep in mind ... if you pull in a huge binary, >> and then use pattern matching to "pick apart" that binary, the pattern >> matching is likely to create sub-binaries that still reference the >> original. ?So then the chain of who's got a reference to the huge binary >> gets more difficult to untangle: if you keep the small sub-binary around >> forever (e.g. as part of a long-lived process's state), then you'll keep >> the underlying "heap" binary around forever also. >> >> -Scott > On that subject, is there any way to force Erlang to copy the binary chunk in order to free it from its dependency on the bigger binary, trading a bit of CPU now for a lit of memory soon? http://erlang.org/doc/man/binary.html#copy-1 From m-garanin@REDACTED Sat Oct 9 09:19:42 2010 From: m-garanin@REDACTED (Garanin Michael) Date: Sat, 09 Oct 2010 11:19:42 +0400 Subject: misultin: handle disconnect Message-ID: <4CB0178E.2010107@yandex.ru> Hello! I try use misultin as comet-server. I copy code from post http://groups.google.com/group/erlang-programming/browse_thread/thread/60ad90aacb1091d/31af9240d4a92e54?lnk=gst&q=misultin+#31af9240d4a92e54 and simple adapted for keep request 10 minutes. I need handle disconnect (for example client close browser). I try use "link" but "handle_http" not died after client disconnect. Q: how i can handle disconnect ? Thanks! /and sorry my bad english/ ////////////////////////////////////////////////////// handle_http(Req) -> % spawn external process Pid = spawn(?MODULE, external_proc, [self()]), % send req Pid ! Req, % wait receive Response -> Response end. external_proc(ControllerPid) -> .. after 10 minutes .... ControllerPid ! Req:ok("OK"). ////////////////////////////////////////////////////// From djk121@REDACTED Sat Oct 9 17:25:38 2010 From: djk121@REDACTED (Dan Kelley) Date: Sat, 9 Oct 2010 11:25:38 -0400 Subject: [erlang-questions] confusing returns from httpc:request/4 In-Reply-To: <20101007180057.GA18278@alumni.caltech.edu> References: <20101007180057.GA18278@alumni.caltech.edu> Message-ID: On Thu, Oct 7, 2010 at 2:00 PM, Anthony Molinaro < anthonym@REDACTED> wrote: > Hi Dan, > > A quick websearch turned up > > > http://stackoverflow.com/questions/2126630/erlangs-maximum-number-of-simultaneous-open-ports > > which seems to suggest you might be at the maximum allowable open file > descriptors and need to update it. > Yeah, I had found that same thing. I did some testing with lsof (I'm on linux), and could never see the number of open filehandles go over ~200, and my ulimit is set to 1024. > > I've had less than stellar results with httpc (see old posts on this group > > http://groups.google.com/group/erlang-programming/browse_thread/thread/1d94595c83b21ef6/1b2416a346b65596?lnk=gst&q=httpc#1b2416a346b65596 > > http://groups.google.com/group/erlang-programming/browse_thread/thread/2f3684d1433c465d/2866bae1935138a2?lnk=gst&q=httpc#2866bae1935138a2 > ). > > I would highly recommend ibrowse though, it's worked really well. > .. and this turned out to be the answer for me too. Based on your advice, I switched to ibrowse, which has worked great. Thanks! Dan > > -Anthony > > On Thu, Oct 07, 2010 at 11:51:48AM -0400, Dan Kelley wrote: > > Hi, > > > > I've got a simple router that's talking to a remote HTTP load balancer > via > > httpc:request/4. For every request that I get into the router, I spawn a > > new process then attempt to connect to the HTTP LB. > > > > That works well most of the time, but occasionally I get > > {error, socket_closed_remotely} instead of results. I had assumed that > > these corresponded to the HTTP LB prematurely closing the connection, but > > when I look at the logs for that process, it always thinks that it's > > returning clean responses (HTTP 200 code, appropriately-formed bodies, > etc). > > > > Anyone seen anything like this before? What else might I do to debug > this? > > > > Thanks, > > > > Dan > > -- > ------------------------------------------------------------------------ > Anthony Molinaro > From roberto@REDACTED Sat Oct 9 18:31:54 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 9 Oct 2010 18:31:54 +0200 Subject: [erlang-questions] misultin: handle disconnect In-Reply-To: <4CB0178E.2010107@yandex.ru> References: <4CB0178E.2010107@yandex.ru> Message-ID: hi michael, try to use the code in the /examples directory instead, since the one you are providing here is a specific user case, not recommended unless you know exactly what you are doing. that being said, misultin is not built to empower a comet application. it does not provide events, like sockets being closed, to the http handle_loop. instead of comet, it has been preferred to support websockets, something misultin is quite good at. if you need comet instead, you will be better off using misultin core and build your own comet application over that. cheers, r. 2010/10/9 Garanin Michael : > Hello! > I try use misultin as comet-server. > I copy code from post > ?http://groups.google.com/group/erlang-programming/browse_thread/thread/60ad90aacb1091d/31af9240d4a92e54?lnk=gst&q=misultin+#31af9240d4a92e54 > and simple adapted for keep request 10 minutes. > I need handle disconnect (for example client close browser). I try use > "link" but "handle_http" not died after client disconnect. > > Q: how i can handle disconnect ? > > > Thanks! > /and sorry my bad english/ > > ////////////////////////////////////////////////////// > > handle_http(Req) -> ? ? ? ? % spawn external process > ? ? ? Pid = spawn(?MODULE, external_proc, [self()]), > ? ? ? % send req > ? ? ? Pid ! Req, > ? ? ? % wait > ? ? ? receive > ? ? ? ? ? ? ? Response -> Response > ? ? ? end. > > external_proc(ControllerPid) -> > > ? ? ? .. after 10 minutes .... > ? ? ?ControllerPid ! Req:ok("OK"). > > > ////////////////////////////////////////////////////// > > > From icarus.alive@REDACTED Sat Oct 9 18:34:39 2010 From: icarus.alive@REDACTED (Icarus Alive) Date: Sat, 9 Oct 2010 22:04:39 +0530 Subject: [erlang-questions] FreeBSD port for Erlang In-Reply-To: <201010090010.o990AOE9049213@pluto.hedeland.org> References: <201010090010.o990AOE9049213@pluto.hedeland.org> Message-ID: Appreciate the reponse and info on how to go about this. Thank you, Per and DS. On Sat, Oct 9, 2010 at 5:40 AM, Per Hedeland wrote: > DS wrote: >> >>On 8 Oct 2010, at 15:22, Icarus Alive wrote: >>> Anyone has any pointers or updates to the latest FreeBSD port for >>> Erlang ? >> >>$ su - >># cd /usr/ports/lang/erlang >># make install > > Indeed - the port was updated to R14B on Sept 27, a mere 12 days after > the OTP release, I believe. Many thanks to the tireless maintainer, > Jimmy Olgeni! > > --Per Hedeland > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From sperka@REDACTED Mon Oct 11 14:02:44 2010 From: sperka@REDACTED (=?UTF-8?B?U3ZhdG9wbHVrIMWgcGVya2E=?=) Date: Mon, 11 Oct 2010 14:02:44 +0200 Subject: Shared/Hybrid Heap Message-ID: <4CB2FCE4.1080304@gmail.com> Hi, does anyone know if there's any progress concerning shared/hybrid heap in Erlang ? I'm prototyping data-intensive app in Erlang and suspect that using hybrid/shared heap would have huge impact at least in terms of memory consumption and. Btw. do I understand it right that "shared" here means that everything is allocated on shared heap and "hybrid" that only what is analysed to be sent is allocated there (as described in R. Carlsson et al.: Message analysis for concurrent languages) ? Thanks. Svatopluk ?perka From krab@REDACTED Mon Oct 11 17:03:38 2010 From: krab@REDACTED (Kresten Krab Thorup) Date: Mon, 11 Oct 2010 17:03:38 +0200 Subject: [erlang-questions] Shared/Hybrid Heap In-Reply-To: <4CB2FCE4.1080304@gmail.com> References: <4CB2FCE4.1080304@gmail.com> Message-ID: <61034301-A5F3-4D5D-963A-DF0C3612EE52@trifork.com> It would be interesting to try running this with Erjang, because it could be in the sweet spot for Erjang given that it runs with a shared heap. What's the nature of your application? Let me know if you want some help to get started with Erjang. Kresten On Oct 11, 2010, at 14:02 , Svatopluk ?perka wrote: > Hi, > > does anyone know if there's any progress concerning shared/hybrid heap > in Erlang ? I'm prototyping data-intensive app in Erlang and suspect > that using hybrid/shared heap would have huge impact at least in terms > of memory consumption and. > > Btw. do I understand it right that "shared" here means that everything > is allocated on shared heap and "hybrid" that only what is analysed to > be sent is allocated there (as described in R. Carlsson et al.: Message > analysis for concurrent languages) ? > > Thanks. > > Svatopluk ?perka > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Kresten Krab Thorup, CTO, Trifork From mr.z.m.wu@REDACTED Mon Oct 11 17:20:53 2010 From: mr.z.m.wu@REDACTED (zhong ming wu) Date: Mon, 11 Oct 2010 11:20:53 -0400 Subject: trouble compiling : crypto.c In-Reply-To: References: Message-ID: Hello List I am an erlang newbie; my need for erlang is to install ejabberd. I have successfully compiled latest erlang on rhel 5.4/5.5. Now I need to do the same on rhel 4.8; one difference from the previous machines is that openssl is not a packaged version; it is installed from source at /usr/local/ssl ./configure --help says that I can supply --with-ssl=PATH for openssl lib and include dir. That said I tried giving it various combinations of lib and/or include dir to --with-ssl But "make" keeps failing with crypto.c not finding symbols. Can someone give me correct syntax to make the installation work? ./configure process does not seem to complain the options I gave in that at the end of ./configure "openssl not found" message is not issued with for example --with-ssl=/usr/local/ssl/lib option. But "make" fails on compiling crypto.c Thanks Mr. Wu From banibrata.dutta@REDACTED Mon Oct 11 17:23:38 2010 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Mon, 11 Oct 2010 20:53:38 +0530 Subject: Any Erlang BoF, SIG in India / Bangalore ? Message-ID: Admittedly, I am quite fascinated with Erlang as a language / framework, based on what I have read about it so far. Wondering if there are any other Erlang enthusiasts, practitioners from India, and specifically Bangalore. Would be quite interested in attending some BoF or like, if one happens. During my internet search, am yet to come accross any though. thanks & regards, BDutta From acton@REDACTED Mon Oct 11 17:41:00 2010 From: acton@REDACTED (Brian Acton) Date: Mon, 11 Oct 2010 08:41:00 -0700 Subject: [erlang-questions] Mnesia startup woes In-Reply-To: References: Message-ID: On Node A, mnesia:del_table_copy(schema, NodeB). You may only complete this operation if mnesia is stopped (or the cluster is down) Alternatively, you may want to explore mnesia:set_master_nodes() and utilizing this on node B. it may reduce the number of steps to recover NodeB as well as streamline the bring up --b On Fri, Oct 8, 2010 at 5:10 AM, Evans, Matthew wrote: > Hi, > > I have the following mnesia deployment. Two nodes (NodeA and NodeB) with > disc_copies and disc_only_copies of tables (other nodes have ram copies of > these tables too). > > NodeA starts first, followed by NodeB. NodeA is considered by the > non-Erlang part of the system to be a master, and NodeB as the standby. When > NodeB comes up (maybe much later) it calls the following code to get the > schema from NodeA, join the party and copy the tables to itself (it will > remove all files from the mnesia directory first): > > mnesia:change_config(extra_db_nodes, [NodeA]), > mnesia:change_table_copy_type(schema, node(), disc_copies), > Tabs = mnesia:system_info(tables) -- [schema], > [mnesia:add_table_copy(Tab, node(), disc_copies) || Tab <- Tabs]. > > We can not control when NodeB starts, if ever. So it's likely that NodeA > will be the only disc_only node when the tables are created. > > What I notice is that if NodeA comes up first, the data is unusable until > NodeB comes up. This makes the system unusable for that period of time. > Certainly I can understand why we do that (we need consistent data), but > what I was after is a way to reverse mnesia:change_config/2 to remove NodeB? > > I can do this on NodeA: > > [ mnesia:force_load_table(T) || T <- mnesia:system_info (tables) ]. > > But was wondering if there is a better way? For example, to remove NodeB > from NodeA's schema? > > Thanks > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From krab@REDACTED Mon Oct 11 18:23:09 2010 From: krab@REDACTED (Kresten Krab Thorup) Date: Mon, 11 Oct 2010 18:23:09 +0200 Subject: [erlang-questions] trouble compiling : crypto.c In-Reply-To: References: Message-ID: Hi, You need to use the otp_build script to build Erlang, not "make" directly. That confused me for a while. Perhaps that is the problem? run ./otp_build with no arguments to get a usage message displayed. Further instructions are in the file INSTALL.md in the top-level directory. Kresten On Oct 11, 2010, at 17:20 , zhong ming wu wrote: > Hello List > > I am an erlang newbie; my need for erlang is to install ejabberd. I have > successfully compiled latest erlang on rhel 5.4/5.5. Now I need to do the > same on rhel 4.8; one difference from the previous machines is that openssl > is not a packaged version; it is installed from source at /usr/local/ssl > > ./configure --help says that I can supply --with-ssl=PATH for openssl lib > and include dir. That said I tried giving it various combinations of lib > and/or include dir to --with-ssl But "make" keeps failing with crypto.c not > finding symbols. > > Can someone give me correct syntax to make the installation work? > ./configure process does not seem to complain the options I gave in that at > the end of ./configure "openssl not found" message is not issued with for > example --with-ssl=/usr/local/ssl/lib option. But "make" fails on compiling > crypto.c > > Thanks > > Mr. Wu Kresten Krab Thorup, CTO, Trifork From zeno490@REDACTED Mon Oct 11 19:02:52 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Mon, 11 Oct 2010 13:02:52 -0400 Subject: [erlang-questions] Shared/Hybrid Heap In-Reply-To: <4CB2FCE4.1080304@gmail.com> References: <4CB2FCE4.1080304@gmail.com> Message-ID: AFAIK, I believe the idea of a shared heap is that multiple processes, instead of having their own heap (or perhaps in addition to) would all allocate from a common heap. This would mean that any message passing from processes in that shared heap, can simply pass a pointer around and wouldn't need to copy it, hence the speed gain and lower memory usage. The idea of making it a hybrid system is that a default would be set (ie: current per process heap behavior) and then you would flag processes when you spawn them with which system/heap they should use. Obviously, messaging a process not in the common heap would require copying as it currently does. This obviously has important implications for garbage collection since either it will need to be performed progressively (as each process in the group stops, mark the memory they use, etc) or stop all the running processes at once while GC runs. Or maybe a more elaborate concurrent GC algorithm could work as well. Since the implications are far reaching, an EPP would probably be required and ultimately, it might be required to have it be a different binary (to prevent the currently used binary to grow in size from all the added VM/lib code which could be an issue on embedded devices). If you search in the mailing list history, you'll find a few posts on the subject explaining a few more details. 2cents Nicholas 2010/10/11 Svatopluk ?perka > Hi, > > does anyone know if there's any progress concerning shared/hybrid heap in > Erlang ? I'm prototyping data-intensive app in Erlang and suspect that using > hybrid/shared heap would have huge impact at least in terms of memory > consumption and. > > Btw. do I understand it right that "shared" here means that everything is > allocated on shared heap and "hybrid" that only what is analysed to be sent > is allocated there (as described in R. Carlsson et al.: Message analysis for > concurrent languages) ? > > Thanks. > > Svatopluk ?perka > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From abhijithg@REDACTED Mon Oct 11 20:07:04 2010 From: abhijithg@REDACTED (abhi) Date: Mon, 11 Oct 2010 23:37:04 +0530 Subject: [erlang-questions] Any Erlang BoF, SIG in India / Bangalore ? In-Reply-To: References: Message-ID: Hi, We currently don't have any Erlang group that meets regularly. But, there is a bangalore functional programmers group (bangalore-fp) and bangalore clojure users group ("bangalore-clj") here. Its been a long time since I have been to a functional programmers meet so I cannot tell you how often they meet. As for the Bangalore clojure users group, we meet first satuday of every month. There are a lot of people here who have worked with erlang here if you want to generally chat about it. Join the mailing list, leave a message or drop in for the next clojure meetup. We'll be happy to meet a fellow erlang enthusiast :) On Mon, Oct 11, 2010 at 8:53 PM, Banibrata Dutta wrote: > Admittedly, I am quite fascinated with Erlang as a language / framework, > based on what I have read about it so far. Wondering if there are any other > Erlang enthusiasts, practitioners from India, and specifically Bangalore. > Would be quite interested in attending some BoF or like, if one happens. > During my internet search, am yet to come accross any though. > > thanks & regards, > BDutta > -- Queer little twists and quirks go into the making of an individual. To suppress them all and follow clock and calendar and creed until the individual is lost in the neutral gray of the host is to be less than true to our inheritance. Life, that gorgeous quality of life, is not accomplished by following another man's rules. It is true we have the same hungers and same thirsts, but they are for different things and in different ways and in different seasons. Lay down your own day, follow it to its noon, or you will sit in an outer hall listening to the chimes but never reaching high enough to strike your own. From puzza007@REDACTED Mon Oct 11 20:12:12 2010 From: puzza007@REDACTED (Paul Oliver) Date: Mon, 11 Oct 2010 19:12:12 +0100 Subject: Distributed application problem Message-ID: Hi all, I'm having a problem with a distributed application where my app ends up running on two nodes. I've set up a test application using http://www.erlang.org/doc/design_principles/distributed_applications.html as a guide. The app is a simple dummy app that does nothing but start. The config files are as described in ?9.2: puzza@REDACTED:~/src/distr_test$ cat config/distr_test_cp[123].config %%-*-erlang-*- [ {kernel, [{distributed, [{distr_test, 5000, [cp1@REDACTED, {cp2@REDACTED, cp3@REDACTED}]}]}, {sync_nodes_mandatory, [cp2@REDACTED, cp2@REDACTED]}, % {sync_nodes_optional, [cp2@REDACTED, cp2@REDACTED]}, {sync_nodes_timeout, 5000} ] } ]. %%-*-erlang-*- [ {kernel, [{distributed, [{distr_test, 5000, [cp1@REDACTED, {cp2@REDACTED, cp3@REDACTED}]}]}, {sync_nodes_mandatory, [cp1@REDACTED, cp3@REDACTED]}, % {sync_nodes_optional, [cp1@REDACTED, cp3@REDACTED]}, {sync_nodes_timeout, 5000} ] } ]. %%-*-erlang-*- [ {kernel, [{distributed, [{distr_test, 5000, [cp1@REDACTED, {cp2@REDACTED, cp3@REDACTED}]}]}, {sync_nodes_mandatory, [cp1@REDACTED, cp2@REDACTED]}, % {sync_nodes_optional, [cp1@REDACTED, cp2@REDACTED]}, {sync_nodes_timeout, 5000} ] } ]. I'm starting the three nodes using: erl -sname cp1 -config config/distr_test_cp1 -pa ebin/ erl -sname cp2 -config config/distr_test_cp2 -pa ebin/ erl -sname cp3 -config config/distr_test_cp3 -pa ebin/ Then starting the app separately on all three nodes with application:start(distr_test). Now when I stop cp1 and cp2 the application is transferred to cp3 as expected. However, when I restart cp1 and cp2 along with the application, I find that it's running on both cp1 and cp3: (cp1@REDACTED)2> application:which_applications(). [{distr_test,"Distribution Test","0.1"}, {stdlib,"ERTS CXC 138 10","1.17.1"}, {kernel,"ERTS CXC 138 10","2.14.1"}] (cp1@REDACTED)3> (cp2@REDACTED)2> application:which_applications(). [{stdlib,"ERTS CXC 138 10","1.17.1"}, {kernel,"ERTS CXC 138 10","2.14.1"}] (cp2@REDACTED)3> (cp3@REDACTED)2> application:which_applications(). [{distr_test,"Distribution Test","0.1"}, {stdlib,"ERTS CXC 138 10","1.17.1"}, {kernel,"ERTS CXC 138 10","2.14.1"}] (cp3@REDACTED)3> I note that when I run start on cp1 it's a normal start rather than a takeover. Can anybody tell me what I'm doing wrong? Regards, Paul. From ulrich.moritz@REDACTED Mon Oct 11 21:24:24 2010 From: ulrich.moritz@REDACTED (Moritz Ulrich) Date: Mon, 11 Oct 2010 21:24:24 +0200 Subject: [erlang-questions] Distributed application problem In-Reply-To: References: Message-ID: Hello Paul, {sync_nodes_mandatory, [cp2@REDACTED, cp2@REDACTED]}, typo? (Notice the second cp2) On Mon, Oct 11, 2010 at 8:12 PM, Paul Oliver wrote: > {sync_nodes_mandatory, [cp2@REDACTED, cp2@REDACTED]}, -- Moritz Ulrich Programmer, Student, Almost normal Guy http://www.google.com/profiles/ulrich.moritz BB5F086F-C798-41D5-B742-494C1E9677E8 From puzza007@REDACTED Mon Oct 11 22:54:59 2010 From: puzza007@REDACTED (Paul Oliver) Date: Mon, 11 Oct 2010 21:54:59 +0100 Subject: [erlang-questions] Distributed application problem In-Reply-To: References: Message-ID: Thanks Moritz! On Mon, Oct 11, 2010 at 8:24 PM, Moritz Ulrich wrote: > Hello Paul, > > {sync_nodes_mandatory, [cp2@REDACTED, cp2@REDACTED]}, > > typo? (Notice the second cp2) > > On Mon, Oct 11, 2010 at 8:12 PM, Paul Oliver wrote: >> {sync_nodes_mandatory, [cp2@REDACTED, cp2@REDACTED]}, > > > > -- > Moritz Ulrich > Programmer, Student, Almost normal Guy > > http://www.google.com/profiles/ulrich.moritz > BB5F086F-C798-41D5-B742-494C1E9677E8 > From mr.z.m.wu@REDACTED Mon Oct 11 23:52:30 2010 From: mr.z.m.wu@REDACTED (zhong ming wu) Date: Mon, 11 Oct 2010 17:52:30 -0400 Subject: [erlang-questions] trouble compiling : crypto.c In-Reply-To: References: Message-ID: On Mon, Oct 11, 2010 at 12:23 PM, Kresten Krab Thorup wrote: > Hi, > > You need to use the otp_build script to build Erlang, not "make" Thanks for the reply but please ignore my post.. It seems that my openssl/include directory is missing some header files (I think I used make install but that didn't copy every header files). I force copied these files and erlang "make" seems to work ok now. > Kresten Krab Thorup, CTO, Trifork > > From erlang@REDACTED Tue Oct 12 00:59:30 2010 From: erlang@REDACTED (Stefan Marr) Date: Tue, 12 Oct 2010 00:59:30 +0200 Subject: [CfPart] BoF: Non-deterministic Programming for Manycore Systems at SPLASH'10 Message-ID: <0A1E4147-3831-45BA-969B-1B5D260FF75B@stefan-marr.de> ============================================================================ Call for Participation Non-deterministic Programming for Manycore Systems Birds of a Feather Tuesday, October 19, starting 5:30pm Reno/Tahoe, Nevada, 2010. Co-located with SPLASH 2010 http://soft.vub.ac.be/~smarr/bof_non-determenistic_programming.html ============================================================================ Non-deterministic Programming: A Solution for the Manycore Challenge? Informal Workshop on the Opportunities of Embracing Non-determinism in a Highly Parallel World Goal ---- This BoF is meant to bring together people who are interested in solving the challenges that the fine-grained parallelism of manycore and exascale systems present. Today, non-determinism is usually considered a problem researchers have to solve when they confronted with parallel programming. However, non-deterministic programming and its potential has been the subject of research for decades past. With the increasing and unavoidable degree of parallelism, it may be the time to discuss it from a new perspective. Thus, the main question of the workshop is, what if the downsides of non-determinism are outweighed by its benefits? We like to discuss the question of how a programming model for non-deterministic parallel programming could be designed. Thus, we want to think about what challenges need to be overcome and what research questions need to be formulated. Of interest are also which classic problems, for instance sorting, could be solved in a non-deterministic manner, and what kind of computational problems could benefit from speedups provided by additional parallelism even when trading off precision for speed. Attendance ---------- This Birds of a Feather is supposed to propose and discus new ideas, without any restriction of the creativity of its attendees. Every attendee is welcome to give a brief presentation, show a demo, or just engage into a discussion with the audience. Renaissance Project ------------------- This BoF will also be used as a venue to present the Renaissance Virtual Machine (RVM) of IBM Research[3]. On this occasion, the RVM will be opensourced and attendees will have the opportunity to get introduced to the technical details of it. The RVM is meant to provide a common foundation for research regarding non-deterministic parallel programming for manycore systems. It is a Squeak/Pharo compatible Smalltalk VM supporting a classic shared-memory programming model. Thus, the programmer has the illusion of a single object heap on which Smalltalk Processes can work in parallel. On top of the RVM, Ly was developed to embrace non-determinism and to harness emergence. The first results will be reported in an Onward! presentation on Thursday afternoon [1]: Harnessing Emergence for Manycore Programming: Early Experience Integrating Ensembles, Adverbs, and Object-based Inheritance by David Ungar, Sam S. Adams Confirmed Participants ---------------------- David Ungar, Doug Kimelman, IBM Research Brief introduction into the Renaissance project and demo of the current status of their Ly/Sly language and manycore virtual machine. Max OrHai, Andrew Black, Portland State University Giving an overview of an example application to test the Renaissance ensemble code as well as an outlook to a scalable non-deterministic parallel sorting algorithm. Theo D'Hondt, Vrije Universiteit Brussel Lessons from the past: Recapitulating the relations of non-deterministic programming, AI, and highly parallel systems like Connection Machine. Organizers ---------- Theo D'Hondt, Software Languages Lab, Vrije Universiteit Brussel Stefan Marr, Software Languages Lab, Vrije Universiteit Brussel [1] http://splashcon.org/program/onward-short-papers/154-onward-short-papers-b [2] http://splashcon.org/schedule/tuesday-oct-19-/235 [3] https://researcher.ibm.com/researcher/view_project.php?id=245 -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525 From steven.charles.davis@REDACTED Tue Oct 12 02:10:46 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 11 Oct 2010 17:10:46 -0700 (PDT) Subject: Shared/Hybrid Heap In-Reply-To: <4CB2FCE4.1080304@gmail.com> References: <4CB2FCE4.1080304@gmail.com> Message-ID: <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> "progress"? how would this be "progress"? ;-) /s On Oct 11, 7:02?am, Svatopluk ?perka wrote: > does anyone know if there's any progress concerning shared/hybrid heap > in Erlang ? From alexey.v.romanov@REDACTED Tue Oct 12 08:48:57 2010 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Tue, 12 Oct 2010 10:48:57 +0400 Subject: Dyalizer warnings for too wide return type Message-ID: One problem I have with Dialyzer is that it complains about cases where the return type specified is too wide. E.g. -type handle_cast_return() :: {noreply, tuple()} | {noreply, tuple(), integer()} | {stop, any(), tuple()}. -spec handle_cast(any(), tuple()) -> handle_cast_return(). handle_cast(_Msg, State) -> {noreply, State}. The definition says that handle_cast returns what it is supposed to return according to behaviour, but it gives this warning: sqlite3.erl:520: The specification for sqlite3:handle_cast/2 states that the function might also return {'noreply',tuple(),integer()} | {'stop',_,tuple()} but the inferred return is {'noreply',_} Given this definition of success typing (from "A Language for Specifying Type Contracts in Erlang"): "A success Typing of a function, f, is a type signature, (?) ? ? , such that whenever an application f(p) reduces to a value v, then p ? ? and v ? ?." the given specification satisfies it, and shouldn't result in a warning. Or at least, there should be an option to turn this set of warnings off, and I don't see one in `dialyzer -Whelp`. Obviously, I can narrow the return type, but then if implementation changes later, I have to review it. In some cases I _know_ that the implementation will change, which makes this even more annoying: -spec some_test() -> boolean(). some_test() -> false. %% TODO implement later %% in a different module foo() -> case m1:some_test() of true -> ... false -> ... end. In addition to the warning about some_test() I get warnings about foo(): "Pattern 'true' can never match type 'false' ", any functions called only from the 'true' branch are reported to be unused, etc. While these warnings are correct, they ignore my specification for some_test(). Can Dialyzer be forced to prefer the return type given in the spec over the` one it infers? Yours, Alexey Romanov From torben.lehoff@REDACTED Tue Oct 12 09:00:00 2010 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Tue, 12 Oct 2010 09:00:00 +0200 Subject: Extracting detailed information from TypEr/Dialyzer Message-ID: Hi, I want to extract the following information from a gen_fsm: - all the states - the incoming events - possible next states but I want to do it per function clause and not only on the function level as TypEr and Dialyzer does it. (I have to help a little by stating which functions are states in the gen_fsm, but that is so easy that I can live with the manual labour involved.) I have created a half-baked solution that is very specific to the code style I use so that I can extract the following about a gen_fsm: [{state_name(),{event(), [outgoing_messages()], {next_state,[state_name()]}] Each state has an entry in the list for each incoming event. (The format should probably be {state_name,event(),[outgoing_messages()],{next_state,[state_name()]}, but that is besides the point right now.) My problem is that my quick'n'dirty approach does not come out on top for all the next_state calculations since you can return from a function clause in many different ways. This kind of information must be available at some point during the TypEr and dialyzer processing of a file - I am okay with throwing my own code away if I get build on existing code and get the job done in a more general way than my current hack. Has anybody been down this road before me? If so, do you have some insights to share? For those who is questioning why I want to extract this information: I have modules which are several thousand lines long and no matter how readable and nice Erlang code is there is still a need to get a quick overview of what goes on in a module. The information above would be enough to create an SDL diagram of the main behaviour of a module which greatly helps when you have to communicate with non-Erlangers about the functionality of your code. This way I can acknowledge that the code is King and any manually created documentation is, well, at best a valiant Knight ;-) Thanks in advance, Torben -- http://www.linkedin.com/in/torbenhoffmann From ms@REDACTED Tue Oct 12 19:00:11 2010 From: ms@REDACTED (Martin Scholl) Date: Tue, 12 Oct 2010 19:00:11 +0200 Subject: Erlang Job in Germany, Marburg Message-ID: <4CB4941B.1070408@diskware.net> Hello all, we are seeking an Erlang Developer to join our software development team. The position is full-time and you will design and implement core parts of our software in Erlang as well as C/C++. If you are interested in distributed storage systems and data structures, please go ahead and apply: http://bit.ly/9ik0eB Thank you in advance and sorry for the noise else, Martin From ingela.andin@REDACTED Wed Oct 13 10:53:20 2010 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 13 Oct 2010 10:53:20 +0200 Subject: [erlang-questions] confusing returns from httpc:request/4 In-Reply-To: References: Message-ID: Hi! The only reason that the httpc-client will return the value {error, socket_closed_remotely} is that the connection is prematurely closed by the server. Unfortunately that something is sent on a tcp socket does not mean it has been sent on the wire yet, and depending on the circumstances it is possible for the socket to be closed before all data has been sent so occasionally a tcp_close can come before all data is delivered. When you use ibrowse do you verify that you always get a correct body? (compleate body) from what I can tell from the code ibrowse will on a tcp_closed message always return whatever happens to be in the buffer compleate or not. Regards Ingela Erlang/OTP team - Ericsson AB 2010/10/7 Dan Kelley : > Hi, > > I've got a simple router that's talking to a remote HTTP load balancer via > httpc:request/4. ?For every request that I get into the router, I spawn a > new process then attempt to connect to the HTTP LB. > > That works well most of the time, but occasionally I get > {error, socket_closed_remotely} instead of results. ?I had assumed that > these corresponded to the HTTP LB prematurely closing the connection, but > when I look at the logs for that process, it always thinks that it's > returning clean responses (HTTP 200 code, appropriately-formed bodies, etc). > > Anyone seen anything like this before? ?What else might I do to debug this? > > Thanks, > > Dan > From chandrashekhar.mullaparthi@REDACTED Wed Oct 13 11:05:04 2010 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Wed, 13 Oct 2010 10:05:04 +0100 Subject: [erlang-questions] confusing returns from httpc:request/4 In-Reply-To: References: Message-ID: On 13 October 2010 09:53, Ingela Andin wrote: > Hi! > > The only reason that the httpc-client will return the value {error, > socket_closed_remotely} > is that the connection is prematurely ?closed by the server. > Unfortunately ?that something > is sent on a tcp socket does not mean it has been sent on the wire > yet, and depending on > the circumstances it is possible for the socket to be closed before > all data has been sent > so occasionally a tcp_close can come before all data is delivered. > > When you use ibrowse do you verify that you always get a correct body? > (compleate body) from what I can tell from the code ibrowse will on a > tcp_closed ?message always return whatever happens to be in the buffer > compleate or not. > ibrowse does this only in the following cases: * server returned a "Connection: Close" * Server supports HTTP/0.9 or HTTP/1.0 In all other cases, it returns an error to the caller. regards, Chandru From sperka@REDACTED Wed Oct 13 11:26:02 2010 From: sperka@REDACTED (=?UTF-8?B?U3ZhdG9wbHVrIMWgcGVya2E=?=) Date: Wed, 13 Oct 2010 11:26:02 +0200 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> Message-ID: <4CB57B2A.7070703@gmail.com> Well, I believe there is a whole spectrum of software systems and although actor model is most natural model of computation I know, in its pure form it seems to be applicable only to a limited part of the spectrum - for practical reasons. For example data-intensive systems where there is huge amount of data managed by different processes and you need to compose results from different processes into a final result, pure share-nothing semantics is expensive, really expensive in terms of memory consumption and even CPU cycles. If you allow data to be shared by processes and its done behind the curtain by runtime, there's practically no difference for programmer, whole system just has different properties. What properties change and how is a complex problem, but I don't think it would be worse from all perspectives. Strict share-nothing is probably best choice for what Erlang is originally aimed for. But not best for everything. Svatopluk ?perka On 10/12/10 2:10 , Steve Davis wrote: > "progress"? how would this be "progress"? > > ;-) > > /s > > On Oct 11, 7:02 am, Svatopluk ?perka wrote: >> does anyone know if there's any progress concerning shared/hybrid heap >> in Erlang ? > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ingela.andin@REDACTED Wed Oct 13 11:58:09 2010 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 13 Oct 2010 11:58:09 +0200 Subject: [erlang-questions] confusing returns from httpc:request/4 In-Reply-To: References: Message-ID: 2010/10/13 Chandru : > On 13 October 2010 09:53, Ingela Andin wrote: >> Hi! >> >> The only reason that the httpc-client will return the value {error, >> socket_closed_remotely} >> is that the connection is prematurely ?closed by the server. >> Unfortunately ?that something >> is sent on a tcp socket does not mean it has been sent on the wire >> yet, and depending on >> the circumstances it is possible for the socket to be closed before >> all data has been sent >> so occasionally a tcp_close can come before all data is delivered. >> >> When you use ibrowse do you verify that you always get a correct body? >> (compleate body) from what I can tell from the code ibrowse will on a >> tcp_closed ?message always return whatever happens to be in the buffer >> compleate or not. >> > > ibrowse does this only in the following cases: > ?* server returned a "Connection: Close" > ?* Server supports HTTP/0.9 or HTTP/1.0 > > In all other cases, it returns an error to the caller. > > regards, > Chandru > Well I do not know if any of these things is true in Dans senario. But I do know that getting the problem with a tcp close message before all data is deliverd is a highly timing dependant problem and if it happens in the server there is nothing inets-client or ibrows can do about it except return an error. Regards Ingela Erlang/OTP team - Ericsson AB From mk@REDACTED Wed Oct 13 12:24:31 2010 From: mk@REDACTED (Morten Krogh) Date: Wed, 13 Oct 2010 12:24:31 +0200 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CB57B2A.7070703@gmail.com> References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> Message-ID: <4CB588DF.8090002@amberbio.com> Clearly, there is a case for both shared and copied data structures. It depends on the access patterns. If each process needs to access the data structure a lot, it is faster with copies, and vice versa. This could be true for read operations as well. It is not true, that an immutable, read only, data structure should always be shared. The reason is locality (cache misses, time of far away memory access etc.). I don't think it is possible to have the virtual machine solve this correctly in all cases. Only the programmer knows the future use of the data, except for very small programs, maybe. So this leads to the following proposal. Erlang processes should have internal threads. The programmer decides between spawning a process or a thread. Processes share almost nothing, threads almost everything. I am not saying Erlang should go that way, but it seems to be the solution to this issue. cheers, Morten Krogh. On 10/13/10 11:26 AM, Svatopluk ?perka wrote: > Well, I believe there is a whole spectrum of software systems and > although actor model is most natural model of computation I know, in > its pure form it seems to be applicable only to a limited part of the > spectrum - for practical reasons. > > For example data-intensive systems where there is huge amount of data > managed by different processes and you need to compose results from > different processes into a final result, pure share-nothing semantics > is expensive, really expensive in terms of memory consumption and even > CPU cycles. > > If you allow data to be shared by processes and its done behind the > curtain by runtime, there's practically no difference for programmer, > whole system just has different properties. What properties change and > how is a complex problem, but I don't think it would be worse from all > perspectives. > > Strict share-nothing is probably best choice for what Erlang is > originally aimed for. But not best for everything. > > Svatopluk ?perka > > On 10/12/10 2:10 , Steve Davis wrote: >> "progress"? how would this be "progress"? >> >> ;-) >> >> /s >> >> On Oct 11, 7:02 am, Svatopluk ?perka wrote: >>> does anyone know if there's any progress concerning shared/hybrid heap >>> in Erlang ? >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From dewa19@REDACTED Wed Oct 13 16:54:08 2010 From: dewa19@REDACTED (-sigit i.-) Date: Wed, 13 Oct 2010 16:54:08 +0200 Subject: [erlang-questions] Erlang user group in madrid? In-Reply-To: References: <4CA05A48.5030100@fi.upm.es> Message-ID: Hello everybody, I just started to dive into erlang. It is very exciting to know some erlanger here in Madrid. @Clara : definitely, I would attend Madrid Erlounge event. @Gonzalo : hands-on session during the event, thats a great idea especially for newbie like me :) @Madrid erlanger : its very nice to know you all :) Thx, -sigit i.- On 30 September 2010 10:05, Gonzalo wrote: > Hi Clara, > > It would be great to have an Erlang user group in Madrid. I have > little experience with the language/platform but sure I would attend > the Madrid Erlounge event. > > Saludos. > Gonzalo. > > > > > On Mon, Sep 27, 2010 at 10:48 AM, Clara Benac Earle > wrote: > > As far as I know, there is not an Erlang user group in Madrid, so let's > > start one! > > > > Those who have interest please send me an email, and I will organize a > > Madrid Erlounge asap. > > > > Nos vemos > > Clara > > > > > > -sigit i.- wrote: > >> > >> Hi all, > >> > >> Is anyone here have info on erlang user group in Madrid (their local > >> maillist, email, etc)? > >> I appreciate the info. > >> > >> Thxalot, > >> -sigit i.- > >> > >> > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > From ok@REDACTED Thu Oct 14 03:05:30 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 14 Oct 2010 14:05:30 +1300 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CB588DF.8090002@amberbio.com> References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> Message-ID: I came across an example where a shared heap would help recently. There is a rectangular grid of processes. Each process wants to be able to send messages to other processes, based on their coordinates. The simplest setup that would occur to me is - create the processes - build a data structure mapping (row,col) to pid - send it to each process But this results in O(N**2) space for N processes. With a shared heap, it would be O(N) space. The second simplest setup is to have a single process hold the data structure, and all the others ask it to relay messages. But this serialises all communication through a single chokepoint. The student who wrote the program did something that would never have occurred to me. He used the registry, doing Pid = whereis(list_to_atom([Row,Col])), Pid ! Message I don't know exactly what goes on in the emulator to make list_to_atom/1 and whereis/1 work; maybe they are lock-free and maybe they aren't. Of course there are intermediate approaches that work, like having a master (row,col)->pid server ("DNS") and having each of the processes in the grid keep a cache of the last k processes it talked to, which would work well in the problem at hand. From ok@REDACTED Thu Oct 14 03:21:20 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 14 Oct 2010 14:21:20 +1300 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CB588DF.8090002@amberbio.com> References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> Message-ID: <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> On 13/10/2010, at 11:24 PM, Morten Krogh wrote: > I don't think it is possible to have the virtual machine solve this correctly in all cases. Only the programmer knows the future use of the data, except for very small programs, maybe. > > So this leads to the following proposal. > > Erlang processes should have internal threads. > > The programmer decides between spawning a process or a thread. Processes share almost nothing, threads almost everything. > > > I am not saying Erlang should go that way, but it seems to be the solution to this issue. There is a tacit assumption here that if thread T1 has data structures D1 and D2 and it wants to share data structure D1 with thread T2, it wants to share D2 as well. This is not obviously the case. From mk@REDACTED Thu Oct 14 08:56:05 2010 From: mk@REDACTED (Morten Krogh) Date: Thu, 14 Oct 2010 08:56:05 +0200 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> Message-ID: <4CB6A985.4090701@amberbio.com> On 10/14/10 3:21 AM, Richard O'Keefe wrote: > On 13/10/2010, at 11:24 PM, Morten Krogh wrote: > >> I don't think it is possible to have the virtual machine solve this correctly in all cases. Only the programmer knows the future use of the data, except for very small programs, maybe. >> >> So this leads to the following proposal. >> >> Erlang processes should have internal threads. >> >> The programmer decides between spawning a process or a thread. Processes share almost nothing, threads almost everything. >> >> >> I am not saying Erlang should go that way, but it seems to be the solution to this issue. > There is a tacit assumption here that if thread T1 has data structures D1 and D2 > and it wants to share data structure D1 with thread T2, it wants to share D2 as > well. This is not obviously the case. > > Not as I see it. T2 doesn't need to even know about D2. The program goes like T1 is working with D1 and D2 ... _T2 = spawn_thread(init_function_t2, [D1]), ... now T1 and T2 are executing in parallel. T2 doesn't even know D2. Morten. From kostis@REDACTED Thu Oct 14 09:47:21 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 14 Oct 2010 10:47:21 +0300 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CB6A985.4090701@amberbio.com> References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> Message-ID: <4CB6B589.9050003@cs.ntua.gr> Morten Krogh wrote: > On 10/14/10 3:21 AM, Richard O'Keefe wrote: >> On 13/10/2010, at 11:24 PM, Morten Krogh wrote: >> >>> I don't think it is possible to have the virtual machine solve this >>> correctly in all cases. Only the programmer knows the future use of >>> the data, except for very small programs, maybe. >>> >>> So this leads to the following proposal. >>> >>> Erlang processes should have internal threads. >>> >>> The programmer decides between spawning a process or a thread. >>> Processes share almost nothing, threads almost everything. >>> >>> >>> I am not saying Erlang should go that way, but it seems to be the >>> solution to this issue. >> There is a tacit assumption here that if thread T1 has data structures >> D1 and D2 >> and it wants to share data structure D1 with thread T2, it wants to >> share D2 as >> well. This is not obviously the case. >> >> > > Not as I see it. T2 doesn't need to even know about D2. > > The program goes like > > T1 is working with D1 and D2 > ... > _T2 = spawn_thread(init_function_t2, [D1]), > ... > > now T1 and T2 are executing in parallel. T2 doesn't even know D2. You are missing the point -- or at least the point which I think Richard is making. In the scheme you propose, T2's execution will be influenced by a piece of data that _in principle_ is not shared. For example, T2 needs to be stopped executing or synchronized with T1 by garbage collection which might take place when process T1 allocates some big D2. How is that different than the two processes sharing everything? Kostis From mk@REDACTED Thu Oct 14 10:22:52 2010 From: mk@REDACTED (Morten Krogh) Date: Thu, 14 Oct 2010 10:22:52 +0200 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CB6B589.9050003@cs.ntua.gr> References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> <4CB6B589.9050003@cs.ntua.gr> Message-ID: <4CB6BDDC.3060900@amberbio.com> >> > > You are missing the point -- or at least the point which I think > Richard is making. In the scheme you propose, T2's execution will be > influenced by a piece of data that _in principle_ is not shared. For > example, T2 needs to be stopped executing or synchronized with T1 by > garbage collection which might take place when process T1 allocates > some big D2. > > How is that different than the two processes sharing everything? > > Kostis Kostis, I don't understand you. Are you saying that it is impossible to implement multithreading unless the threads share everything, and garbage collection will be slow? What about two erlang processes that share a binary. Isn't that how it works today? What about two erlang processes that share a nif resource? What about other languages and virtual machines. They have multithreading. We must be talking past each other here. By the way, I am not advocating such a change to erlang. I think you can get around shared memory by choosing the right processes, partition data correctly etc. It is only the most computationally intensive cases where you can't, and there one can use a nif resource or some external way of sharing data. From jameschurchman@REDACTED Thu Oct 14 12:34:32 2010 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 14 Oct 2010 11:34:32 +0100 Subject: [erlang-questions] If you could rename ports, what would you call them? In-Reply-To: References: <98BDC430-6D7F-423F-81A3-A079C08DAB29@masklinn.net> <20101006001423.GA8623@mulga.csse.unimelb.edu.au> Message-ID: <0B7E792F-73C4-4B92-A688-0E2DFA5C1FB5@gmail.com> Would this erlang subset not also have wider reaching applications allowing even the NIFs to sometimes be build in "erlang" if it had access to c standard headers somehow or provided very very fast code. Also would erlang lack of true "arrays" be a problem? (also really thinking ahead would it allow an OS to be built in erlang? :-) On 8 Oct 2010, at 12:16, Joe Armstrong wrote: > Indeed - in fact the Erlang scheduler and GC should be written in Erlang > - this would have to be a strictly controlled sub-set of Erlang that created no > garbage and had access to memory and registers - something that could > be cross compiled to C (do pretty much like they did in smalltalk many > years ago :-) > > /Joe From zeno490@REDACTED Thu Oct 14 16:47:46 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Thu, 14 Oct 2010 10:47:46 -0400 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CB6BDDC.3060900@amberbio.com> References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> <4CB6B589.9050003@cs.ntua.gr> <4CB6BDDC.3060900@amberbio.com> Message-ID: Sharing binaries is fine because they use a different heap altogether and a different garbage collection scheme (reference counting). Sharing a nif ressource implies you manage it manually through nifs, how you manage it is largely up to you. As far as other languages go, yes, most (all?) of them stop all threads to garbage collect (if those threads share memory). Java does this, C# does this, ruby, python, etc. C# recently or will shortly introduce a garbage collection algorithm that runs concurrently and thus doesn't stop all threads but from what I can remember, it isn't 100% guaranteed (if threads continue to allocate past a threshold, I believe they are still all stopped waiting for GC to complete). This an important reason why multi generational garbage collection algorithms are so popular: it keeps most GC cycles quick, thus stopping all threads for the least amount of time. The idea behind each process having its own heap is twofold: it helps memory locality and it isolates that heap such that it can be GCed on its own, without interrupting any other process. Processes are such essentially sandboxed in that regard. This also means that any two processes, or more, can run garbage collection concurrently without issues. While it sounds bad to stop all processes/threads sharing a piece of memory, in reality, for most desktop applications, it is a non issue. The story is different for server software however, where they tend to allocate a lot and have many threads. When you want to share some data between P1 and P2, if they do not share everything, a problem arises quite quickly: how do you mark allocations that should be shared and those that shouldn't? If you share everything, allocation is easy and goes in 1 heap (ignoring binaries here in their own heap). If you do not, you have to select those allocations. In both cases, because some memory is shared, the simplest way to GC will be to stop both processes. After all, you will need to inspect their respective stacks to see if they reference things in that shared heap and if the processes are running, the stack will keep changing. If even for the time of a memcpy of the stack, all processes will have to stop at some point or another. You also won't be able to run the GC concurrently on P1 and P2 if they share memory, or at least, it will be quite hard. Sharing memory and garbage collection is a non trivial problem to solve and as erlang currently does, the simplest, most elegant way to deal with the problem is to not share at all (and always copy) to avoid the problem. Nicholas On Thu, Oct 14, 2010 at 4:22 AM, Morten Krogh wrote: > > >>> >> You are missing the point -- or at least the point which I think Richard >> is making. In the scheme you propose, T2's execution will be influenced by a >> piece of data that _in principle_ is not shared. For example, T2 needs to be >> stopped executing or synchronized with T1 by garbage collection which might >> take place when process T1 allocates some big D2. >> >> How is that different than the two processes sharing everything? >> >> Kostis >> > > Kostis, > > I don't understand you. Are you saying that it is impossible to implement > multithreading unless the threads share everything, and garbage collection > will be slow? > > What about two erlang processes that share a binary. Isn't that how it > works today? > What about two erlang processes that share a nif resource? > What about other languages and virtual machines. They have multithreading. > > We must be talking past each other here. > > By the way, I am not advocating such a change to erlang. I think you can > get around shared memory by choosing the right processes, partition data > correctly etc. > It is only the most computationally intensive cases where you can't, and > there one can use a nif resource or some external way of sharing data. > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From sidorenko@REDACTED Thu Oct 14 16:59:00 2010 From: sidorenko@REDACTED (Andrey Sidorenko) Date: Thu, 14 Oct 2010 17:59:00 +0300 Subject: mnesia and simple records Message-ID: Hi All, Is it correct, that it is not possible to create mnesia table for the record which contains just one member? What is the reason for such kind of limitation? Eshell V5.7.5 (abort with ^G) 1> mnesia:create_schema([node()]). ok 2> mnesia:start(). ok 3> rd(test,{f1}). test 4> mnesia:create_table(test, [{attributes, record_info(fields, test)}]). {aborted,{bad_type,test,{attributes,[f1]}}} 5> rd(test2,{f1,f2}). test2 6> mnesia:create_table(test2, [{attributes, record_info(fields, test2)}]). {atomic,ok} 7> Thanks, Andrey From kenrobinsonster@REDACTED Thu Oct 14 17:15:30 2010 From: kenrobinsonster@REDACTED (Ken Robinson) Date: Fri, 15 Oct 2010 01:15:30 +1000 Subject: Staring more than one instance of a gen_server Message-ID: Hi All, If I wanted to start more than one instance of a gen_server what is the best practice? The gen_servers would be exactly the same except for the seeding parameters. From what I've found I could use the child spec and seed with specific names through the arguments like so: start_link(PortNumber, MySpecialName) -> gen_server:start_link({local, MySpecialName, ?MODULE, PortNumber, []). Would that work? -- regards, Ken Robinson From max.lapshin@REDACTED Thu Oct 14 19:02:10 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 14 Oct 2010 21:02:10 +0400 Subject: httpc problems Message-ID: Why httpc can make only one simultaneous request per profile? Maybe it would be good to fix this client in erlang. There is strange common problems among many languages: lack of good builtin http client. There is excelent tool ibrowse, maybe it (or something like it) should take it place inside OTP? From boris.muehmer@REDACTED Thu Oct 14 19:09:58 2010 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Thu, 14 Oct 2010 19:09:58 +0200 Subject: [erlang-questions] "wx:demo()." / static example / possible bug on Ubuntu 10.04 / Erlang R13B04/R14B In-Reply-To: References: Message-ID: >> On my windows systems the "wxStaticText" alignment is correct (left / >> center / right), but on my Ubuntu 10.04 system, all three text are on >> the left. This is true for Erlang R13B04 and R14B. > > Known issue see http://trac.wxwidgets.org/ticket/12539 Another "list" to check... :-) > Wx internal error, means that the driver tries to send a message to a > non existing process, the demo could probably be improved there. > That ERROR should be a warning, I don't want remove the message > completely since it can help the developer. Somehow I thought so... but thank You for explaining this behaviour. Recently I also saw that I get this Warning/Error on my Windows box as well, but not as often as on my Linux system. Maybe this is just a very subjective experience, because I use Linux most of the time, and only use Windows systems for some special things... > Gtk warnings, I can't do much about them try starting firefox from the > shell most gui programs on linux have but normally they go to > /dev/null, they can depend on half implemented gtk/gnome themes and > wxWidgets internal problems/hacks. I know those errors from other programs as well, because I normally use shells... it is just annoying... :-( - boris From phone.bytes@REDACTED Thu Oct 14 19:39:49 2010 From: phone.bytes@REDACTED (Phone) Date: Thu, 14 Oct 2010 11:39:49 -0600 Subject: Erlang MySQL ODBC Question Message-ID: <4CB74065.9060906@gmail.com> Newbie to Erlang. I have successfully connected to MySQL using ODBC, successful SELECT statement. Got result set back like this: {selected,["fieldname1", "fieldname2"],[{"fieldvalue1","fieldvalue2"}]} Not sure how this result set is stored and how to get access to the fieldvalues in my erlang program. Probably really dumb? Thanks From woody.peterson@REDACTED Thu Oct 14 19:40:30 2010 From: woody.peterson@REDACTED (Woody Peterson) Date: Thu, 14 Oct 2010 10:40:30 -0700 Subject: Is a process necessary in front of mnesia? Message-ID: Hi All, In the pragmatic screencasts' Erlang series, there is a gen_server in charge of writing to mnesia. In refactoring a small erlang system my company's building, we still have that boilerplate handling mnesia, but thought we should remove the gen_server entirely, as we could just use a function instead of a process as per erlang's best practices (http://www.erlang.se/doc/programming_rules.shtml#HDR16 ). Note that this gen_server holds no state, which is probably not a rule for when to not use a process, but seems like it could be a smell. I figure the gen_server in the screencasts could have just been there as a learning tool, so is there a reason beyond education to put a serializing process in front of mnesia as a rule? Are we right to assume we can have possibly tens of thousands of processes sending write messages to mnesia directly as opposed to a gen_server in front of it? Thanks, -Woody From ulrich.moritz@REDACTED Thu Oct 14 20:07:16 2010 From: ulrich.moritz@REDACTED (Moritz Ulrich) Date: Thu, 14 Oct 2010 20:07:16 +0200 Subject: [erlang-questions] Is a process necessary in front of mnesia? In-Reply-To: References: Message-ID: On Thu, Oct 14, 2010 at 7:40 PM, Woody Peterson wrote: > Are we right to assume we can have possibly tens of thousands of > processes sending write messages to mnesia directly as opposed to a > gen_server in front of it? I'm not very experienced with mnesia, but according to my understanding: Yes, you could get rid of the gen_server. Think of it: You have a powerful, distributed database as your backend, but you tunnel *all* you writes through a single gen_server instance. Every read, every write gets pushed through the message queue of a single bottleneck, just to forward it to mnesia. -- Moritz Ulrich Programmer, Student, Almost normal Guy http://www.google.com/profiles/ulrich.moritz BB5F086F-C798-41D5-B742-494C1E9677E8 From torben.lehoff@REDACTED Thu Oct 14 21:32:57 2010 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 14 Oct 2010 21:32:57 +0200 Subject: Extracting detailed information from TypEr/Dialyzer In-Reply-To: References: Message-ID: Hi again, For one of my gen_fsm's I have been able to deal with {next_state, Var, ...} by looking up the value of Var in the statements of the function clause - not a full solution, but enough to cover my code. Output example in CSP style: CALL_BUSY_ORIGINATING = iz ? some_message -> qsig ! {cc_progress_request, isi_queuing} -> CALL_BUSY_ORIGINATING [] iz ? another_message -> qsig ! {cc_progress_request, isi_connect_ack} -> CALL_ACTIVE_ORIGINATING_TX |~| CALL_ACTIVE_ORIGINATING_RX This corresponds to two function clauses in the call_busy_originating/2 function where the clauses differ on what message their receive on their iz interface. Then a message is sent on the qsig interface. The second clause does some calculation to figure out what the next state should be, but at this level I do not care about the details so I end the CSP clause with a non-deterministic choice between the two states that can be the next state. So now I have a nice compact overview of what my gen_fsm is doing which I can use to communicate with non-Erlangers. The initial response was "This looks fine!"... case closed. But I would still like to get some insights on how TypEr/Dialyzer works internally since I have a feeling that there must be code in there that can be used to derive a more generic solution. Cheers, Torben On Tue, Oct 12, 2010 at 09:00, Torben Hoffmann wrote: > Hi, > > I want to extract the following information from a gen_fsm: > > - all the states > - the incoming events > - possible next states > > but I want to do it per function clause and not only on the function level > as TypEr and Dialyzer does it. > (I have to help a little by stating which functions are states in the > gen_fsm, but that is so easy that I can live with the manual labour > involved.) > > I have created a half-baked solution that is very specific to the code > style I use so that I can extract the following about a gen_fsm: > [{state_name(),{event(), > [outgoing_messages()], > {next_state,[state_name()]}] > > Each state has an entry in the list for each incoming event. > (The format should probably be > {state_name,event(),[outgoing_messages()],{next_state,[state_name()]}, but > that is besides the point right now.) > > My problem is that my quick'n'dirty approach does not come out on top for > all the next_state calculations since you can return from a function clause > in many different ways. > > This kind of information must be available at some point during the TypEr > and dialyzer processing of a file - I am okay with throwing my own code away > if I get build on existing code and get the job done in a more general way > than my current hack. > > Has anybody been down this road before me? If so, do you have some insights > to share? > > For those who is questioning why I want to extract this information: I have > modules which are several thousand lines long and no matter how readable and > nice Erlang code is there is still a need to get a quick overview of what > goes on in a module. The information above would be enough to create an SDL > diagram of the main behaviour of a module which greatly helps when you have > to communicate with non-Erlangers about the functionality of your code. This > way I can acknowledge that the code is King and any manually created > documentation is, well, at best a valiant Knight ;-) > > Thanks in advance, > Torben > -- > http://www.linkedin.com/in/torbenhoffmann > -- http://www.linkedin.com/in/torbenhoffmann From spawn.think@REDACTED Thu Oct 14 22:01:23 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Thu, 14 Oct 2010 22:01:23 +0200 Subject: [erlang-questions] mnesia and simple records In-Reply-To: References: Message-ID: "{attributes, AtomList} a list of the attribute names for the records that are supposed to populate the table. The default value is [key, val]. *The table must have at least one extra attribute in addition to the key*." http://www.erlang.org/doc/man/mnesia.html#create_table-2 On Thu, Oct 14, 2010 at 4:59 PM, Andrey Sidorenko wrote: > Hi All, > > Is it correct, that it is not possible to create mnesia table for the > record > which contains just one member? What is the reason for such kind of > limitation? > > Eshell V5.7.5 (abort with ^G) > 1> mnesia:create_schema([node()]). > ok > 2> mnesia:start(). > ok > 3> rd(test,{f1}). > test > 4> mnesia:create_table(test, [{attributes, record_info(fields, test)}]). > {aborted,{bad_type,test,{attributes,[f1]}}} > 5> rd(test2,{f1,f2}). > test2 > 6> mnesia:create_table(test2, [{attributes, record_info(fields, test2)}]). > {atomic,ok} > 7> > > Thanks, > Andrey > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From spawn.think@REDACTED Thu Oct 14 22:31:38 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Thu, 14 Oct 2010 22:31:38 +0200 Subject: [erlang-questions] Staring more than one instance of a gen_server In-Reply-To: References: Message-ID: Hi Ken, It all depends on your purposes. If you want to start these processes as a part of a supervision tree, so you would probably like to use simple-one-for-one strategy and children dynamically supervisor:start_child(Sup, List) http://www.erlang.org/doc/design_principles/sup_princ.html#id67619 On Thu, Oct 14, 2010 at 5:15 PM, Ken Robinson wrote: > Hi All, > If I wanted to start more than one instance of a gen_server what is > the best practice? The gen_servers would be exactly the same except > for the seeding parameters. From what I've found I could use the child > spec and seed with specific names through the arguments like so: > start_link(PortNumber, MySpecialName) -> gen_server:start_link({local, > MySpecialName, ?MODULE, PortNumber, []). > Would that work? > -- > regards, > Ken Robinson > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From ok@REDACTED Thu Oct 14 23:09:01 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 15 Oct 2010 10:09:01 +1300 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CB6A985.4090701@amberbio.com> References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> Message-ID: <6A7A6432-58F1-405D-A17C-5A739151C99B@cs.otago.ac.nz> On 14/10/2010, at 7:56 PM, Morten Krogh wrote: > On 10/14/10 3:21 AM, Richard O'Keefe wrote: >> On 13/10/2010, at 11:24 PM, Morten Krogh wrote: >>> Erlang processes should have internal threads. >>> >>> The programmer decides between spawning a process or a thread. Processes share almost nothing, threads almost everything. >>> >> There is a tacit assumption here that if thread T1 has data structures D1 and D2 >> and it wants to share data structure D1 with thread T2, it wants to share D2 as >> well. This is not obviously the case. >> >> > > Not as I see it. T2 doesn't need to even know about D2. "Threads share almost everything". > > The program goes like > > T1 is working with D1 and D2 > ... > _T2 = spawn_thread(init_function_t2, [D1]), > ... > > now T1 and T2 are executing in parallel. T2 doesn't even know D2.\ But "threads share almost everything". T2 may not KNOW it's sharing D2 with T1, but D2 is *there* in its address space. That's a price. Why should T2 be forced to include D2 in its address space if it doesn't want it? A conceptually simpler scheme is a hybrid one, where some terms are private and some are shared, and send_shared(Pid, Msg) copies to the shared heap such parts of Msg as are not already there, after which both the sending and the receiving process refer to the shared version. This gives precise control over what gets shared and what doesn't. It still goes very much against the spirit of Erlang, because it wouldn't work very well if Pid referred to a process on another node. From hm@REDACTED Thu Oct 14 23:11:09 2010 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Thu, 14 Oct 2010 23:11:09 +0200 Subject: [erlang-questions] Is a process necessary in front of mnesia? In-Reply-To: References: Message-ID: On Thu, Oct 14, 2010 at 7:40 PM, Woody Peterson wrote: > Are we right to assume we can have possibly tens of thousands of > processes sending write messages to mnesia directly as opposed to a > gen_server in front of it? Yes. /H?kan From mevans@REDACTED Thu Oct 14 23:37:32 2010 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 14 Oct 2010 17:37:32 -0400 Subject: [erlang-questions] Is a process necessary in front of mnesia? In-Reply-To: References: Message-ID: If you mean remove your own gen_server, then yes it is. One thing that a "private" gen_server does give you is the ability to manage certain operations in a more efficient manner, while keeping a consistent API. For example I have an application that normally does quick key-based lookups. Sometimes however it is necessary to do select or match_object queries on large data-sets. Having a gen_server "front" these requests allows you to wrap the whole operation in a spawn, while providing a simple to use API for the user. e.g. handle_call({really_complex_search,SearchStuff},From,State) -> Pattern = #mnesia_table{some_element = SearchStuff, _ = '_'} spawn(fun()-> case mnesia:dirty_match_object(SearchStuff) of [Rec] -> gen_server:reply(From,Rec); _ -> gen_server:reply(From,error) end end) {noreply,State}; Certainly the calling application could do that too, but in our case the access method needs to be simple. Another thing I use a gen_server intermediately for is to prioritize certain operations over others. In one case we are supposed to process many writes as a low-priority task that doesn't affect the read rate. Having a gen_server in the way lets us do that. Matt -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Woody Peterson Sent: Thursday, October 14, 2010 1:41 PM To: Erlang-Questions Questions Subject: [erlang-questions] Is a process necessary in front of mnesia? Hi All, In the pragmatic screencasts' Erlang series, there is a gen_server in charge of writing to mnesia. In refactoring a small erlang system my company's building, we still have that boilerplate handling mnesia, but thought we should remove the gen_server entirely, as we could just use a function instead of a process as per erlang's best practices (http://www.erlang.se/doc/programming_rules.shtml#HDR16 ). Note that this gen_server holds no state, which is probably not a rule for when to not use a process, but seems like it could be a smell. I figure the gen_server in the screencasts could have just been there as a learning tool, so is there a reason beyond education to put a serializing process in front of mnesia as a rule? Are we right to assume we can have possibly tens of thousands of processes sending write messages to mnesia directly as opposed to a gen_server in front of it? Thanks, -Woody ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From djk121@REDACTED Fri Oct 15 05:47:29 2010 From: djk121@REDACTED (Dan Kelley) Date: Thu, 14 Oct 2010 23:47:29 -0400 Subject: hipe problems with R14B? Message-ID: Hi, I'm seeing the following when trying to compile with hipe in R14B on linux (centos 5.3, x86_64): ----- erlc +native -W hth.erl EXITED with reason {'trans_fun/2',{bif,binary_part,{f,0},[{x,0},{integer,0},{integer,1}],{x,1}}} @hipe_beam_to_icode:1144 =ERROR REPORT==== 14-Oct-2010::23:46:42 === Error in process <0.27.0> with exit value: {{badmatch,{'EXIT',{{hipe_beam_to_icode,1144,{'trans_fun/2',{bif,binary_part,{f,0},[{x,0},{integer,0},{integer,1}],{x,1}}}},[{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode... ./hth.erl:none: internal error in native_compile; crash reason: {{badmatch, {'EXIT', {{hipe_beam_to_icode,1144, {'trans_fun/2', {bif,binary_part, {f,0}, [{x,0},{integer,0},{integer,1}], {x,1}}}}, [{hipe_beam_to_icode,trans_fun,2}, {hipe_beam_to_icode,trans_fun,2}, {hipe_beam_to_icode,trans_fun,2}, {hipe_beam_to_icode,trans_fun,2}, {hipe_beam_to_icode,trans_fun,2}, {hipe_beam_to_icode,trans_fun,2}, {hipe_beam_to_icode,trans_op_call,6}, {hipe_beam_to_icode,trans_fun,2}]}}}, [{hipe,get_beam_icode,4},{hipe,'-run_compiler_1/3-fun-0-',4}]} ----- Am I doing something wrong, or is this a bug? Thanks, Dan From bengt.kleberg@REDACTED Fri Oct 15 07:57:45 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 15 Oct 2010 07:57:45 +0200 Subject: [erlang-questions] hipe problems with R14B? In-Reply-To: References: Message-ID: <1287122265.5114.0.camel@seasc1137> Greetings, Is it possible for you to supply the code? bengt On Fri, 2010-10-15 at 05:47 +0200, Dan Kelley wrote: > Hi, > > I'm seeing the following when trying to compile with hipe in R14B on linux > (centos 5.3, x86_64): > > ----- > erlc +native -W hth.erl > EXITED with reason > {'trans_fun/2',{bif,binary_part,{f,0},[{x,0},{integer,0},{integer,1}],{x,1}}} > @hipe_beam_to_icode:1144 > > =ERROR REPORT==== 14-Oct-2010::23:46:42 === > Error in process <0.27.0> with exit value: > {{badmatch,{'EXIT',{{hipe_beam_to_icode,1144,{'trans_fun/2',{bif,binary_part,{f,0},[{x,0},{integer,0},{integer,1}],{x,1}}}},[{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode... > > ./hth.erl:none: internal error in native_compile; > crash reason: {{badmatch, > {'EXIT', > {{hipe_beam_to_icode,1144, > {'trans_fun/2', > {bif,binary_part, > {f,0}, > [{x,0},{integer,0},{integer,1}], > {x,1}}}}, > [{hipe_beam_to_icode,trans_fun,2}, > {hipe_beam_to_icode,trans_fun,2}, > {hipe_beam_to_icode,trans_fun,2}, > {hipe_beam_to_icode,trans_fun,2}, > {hipe_beam_to_icode,trans_fun,2}, > {hipe_beam_to_icode,trans_fun,2}, > {hipe_beam_to_icode,trans_op_call,6}, > {hipe_beam_to_icode,trans_fun,2}]}}}, > > [{hipe,get_beam_icode,4},{hipe,'-run_compiler_1/3-fun-0-',4}]} > ----- > > Am I doing something wrong, or is this a bug? > > Thanks, > > Dan From ingela.andin@REDACTED Fri Oct 15 09:59:01 2010 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 15 Oct 2010 09:59:01 +0200 Subject: [erlang-questions] httpc problems In-Reply-To: References: Message-ID: Hi! 2010/10/14 Max Lapshin : > Why httpc can make only one simultaneous request per profile? Maybe it > would be good to fix this client in erlang. >From where did you get that impression? It is not so. > There is strange common problems among many languages: lack of good > builtin http client. > There is excelent tool ibrowse, maybe it (or something like it) should > take it place inside OTP? I think that ibrowse would have made a much better starting point than the client we started out with, however I do not think that now trying to replace the inets one with ibrowse would be less work than fixing possible problems with it. Regards Ingela Erlang/OTP team - Ericsson AB From rtrlists@REDACTED Fri Oct 15 11:32:57 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 15 Oct 2010 10:32:57 +0100 Subject: [erlang-questions] Erlang MySQL ODBC Question In-Reply-To: <4CB74065.9060906@gmail.com> References: <4CB74065.9060906@gmail.com> Message-ID: On Thu, Oct 14, 2010 at 6:39 PM, Phone wrote: > Newbie to Erlang. > > I have successfully connected to MySQL using ODBC, successful SELECT > statement. Got result set back like this: > > {selected,["fieldname1", "fieldname2"],[{"fieldvalue1","fieldvalue2"}]} > > Not sure how this result set is stored and how to get access to the > fieldvalues in my erlang program. > > Probably really dumb? > > Thanks > > > Kind of depends on what you want to do with your results. The plain result simple gives you the list of column names and the list of result set rows, where each row is a tuple of values. You can transform this into whatever your app requires by manipulating the lists. For example if you rather had a list of result set rows, where each row is a list of name-value pairs: Eshell V5.6.5 (abort with ^G) 1> RS = {selected,["field1", "field2"],[{"field1value1","field2value1"},{"field1value2","field2value2"}]}. {selected,["field1","field2"], [{"field1value1","field2value1"}, {"field1value2","field2value2"}]} 2> {selected, Cols, Rows} = RS. {selected,["field1","field2"], [{"field1value1","field2value1"}, {"field1value2","field2value2"}]} 3> [ lists:zip(Cols, tuple_to_list(Row)) || Row <- Rows ]. [[{"field1","field1value1"}, {"field2","field2value1"}], [{"field1","field1value2"}, {"field2","field2value2"}]] 4> So, it really just depends on what you want to do with your result set. Hope this helps a bit, Robby From vincent.dephily@REDACTED Fri Oct 15 11:45:03 2010 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Fri, 15 Oct 2010 11:45:03 +0200 Subject: [erlang-questions] Dyalizer warnings for too wide return type In-Reply-To: References: Message-ID: <201010151145.03649.vincent.dephily@mobile-devices.fr> On Tuesday 12 October 2010 08:48:57 Alexey Romanov wrote: > One problem I have with Dialyzer is that it complains about cases > where the return type specified is too wide. > > E.g. > > -type handle_cast_return() :: {noreply, tuple()} | {noreply, tuple(), > integer()} | {stop, any(), tuple()}. > -spec handle_cast(any(), tuple()) -> handle_cast_return(). > handle_cast(_Msg, State) -> > {noreply, State}. > > The definition says that handle_cast returns what it is supposed to > return according to behaviour, but it gives this warning: [...] > the given specification satisfies it, and shouldn't result in a > warning. Or at least, there should be an option to turn this set of > warnings off, and I don't see one in > `dialyzer -Whelp`. It sounds like -Wunderspecs or -Woverspecs would do that, but that doesn't seem to work. Actually it looks like (contrary to documentation) -Wunderspecs is on by default and we actually need a -Wno_underspecs option. But a global no_underspec for the program sounds dangerous too (you'll hide actual underspecs which should be fixed). Maybe dialyzer could ease the pain by handly behaviour spec specially ? > -spec some_test() -> boolean(). > some_test() -> false. %% TODO implement later > > %% in a different module > foo() -> > case m1:some_test() of > true -> ... > false -> ... > end. > > In addition to the warning about some_test() I get warnings about > foo(): "Pattern 'true' can never match type 'false' ", any functions > called only from the 'true' branch are reported to be unused, etc. > While these warnings are correct, they ignore my specification for > some_test(). Can Dialyzer be forced to prefer the return type given in > the spec over the` one it infers? Yes, that annoys me as well. I understand the warning for the some_spec function, but I think it shouldn't trickle down to the foo function. I have another use-case where I recompile a module at runtime but provide an initial implementation that always returns false. I'd like dializer to trust me when I declare my interface, even if the implementation doesn't match that. Actually, maybe we could make the case that non-exported functions must have an exact spec whereas exported function can have a spec that is more allowing than the implementation ? -- Vincent de Phily Mobile Devices +33 (0) 142 119 325 +353 (0) 85 710 6320 Warning This message (and any associated files) is intended only for the use of its intended recipient and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Any views or opinions presented are solely those of the author vincent.dephily@REDACTED and do not necessarily represent those of the company. Although the company has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. From matthias@REDACTED Fri Oct 15 12:21:09 2010 From: matthias@REDACTED (Matthias Lang) Date: Fri, 15 Oct 2010 12:21:09 +0200 Subject: [erlang-questions] limited error_logger dump (vm crashes because of generating dump) In-Reply-To: References: Message-ID: <20101015102109.GA5651@corelatus.se> Hi, I'm replying to an old-ish thread. Sorry, hard to keep up. On Monday, September 20, Max Lapshin wrote: > For several times I meet bad situation, when BEAM crashes due to > error_logger. It begins to dump laaarge state, take much more memory > than state was due to list nature (once, I've seen error_logger eating > 15GB of memory) and then crashes. You then solved it by writing io_lib_pretty_limited.erl. That's a good solution, I've successfully used the same approach in thousands of production systems for years: http://www.corelatus.com/~matthias/trunc_io.erl As you suspected, putting a format_state function in gen_server doesn't help much, you never know that you've taken care of every place where something sends a huge term to the error logger. The default error logger causes more trouble than it's worth. (For fun: how much memory does the VM need to print a 9 megabyte list? Ten times as much, i.e. 90 megabytes? Twenty times? Fifty times? No, more than 100 times: ~ >uname -a Linux contorpis 2.6.32-5-amd64 #1 SMP Fri Sep 17 21:50:19 UTC 2010 x86_64 GNU/Linux ~ >ulimit -v 1000000 %% limit to one gigabyte ~ >14erl Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> io_lib:fwrite("~p", [lists:duplicate(9000123, 34)]), ignore. Crash dump was written to: erl_crash.dump eheap_alloc: Cannot allocate 382630800 bytes of memory (of type "heap"). ) Matt From alexey.v.romanov@REDACTED Fri Oct 15 13:17:23 2010 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Fri, 15 Oct 2010 15:17:23 +0400 Subject: [erlang-questions] Dyalizer warnings for too wide return type In-Reply-To: <201010151145.03649.vincent.dephily@mobile-devices.fr> References: <201010151145.03649.vincent.dephily@mobile-devices.fr> Message-ID: Yours, Alexey Romanov On Fri, Oct 15, 2010 at 1:45 PM, Vincent de Phily wrote: > On Tuesday 12 October 2010 08:48:57 Alexey Romanov wrote: >> One problem I have with Dialyzer is that it complains about cases >> where the return type specified is too wide. >> >> E.g. >> >> -type handle_cast_return() :: {noreply, tuple()} | {noreply, tuple(), >> integer()} | {stop, any(), tuple()}. >> -spec handle_cast(any(), tuple()) -> handle_cast_return(). >> handle_cast(_Msg, State) -> >> ? ? {noreply, State}. >> >> The definition says that handle_cast returns what it is supposed to >> return according to behaviour, but it gives this warning: > [...] >> the given specification satisfies it, and shouldn't result in a >> warning. Or at least, there should be an option to turn this set of >> warnings off, and I don't see one in >> `dialyzer -Whelp`. > > It sounds like -Wunderspecs or -Woverspecs would do that, but that doesn't > seem to work. > Actually it looks like (contrary to documentation) -Wunderspecs > is on by default and we actually need a -Wno_underspecs option. If I enable -Wunderspecs, I get a lot more warnings, so it isn't on by default. These warnings don't seem to be considered underspecs for some reason. >> In addition to the warning about some_test() I get warnings about >> foo(): "Pattern 'true' can never match type 'false' ", any functions >> called only from the 'true' branch are reported to be unused, etc. >> While these warnings are correct, they ignore my specification for >> some_test(). Can Dialyzer be forced to prefer the return type given in >> the spec over the` one it infers? > > Yes, that annoys me as well. I understand the warning for the some_spec > function, but I think it shouldn't trickle down to the foo function. I have > another use-case where I recompile a module at runtime but provide an initial > implementation that always returns false. > > I'd like dializer to trust me when I declare my interface, even if the > implementation doesn't match that. Actually, maybe we could make the case that > non-exported functions must have an exact spec whereas exported function can > have a spec that is more allowing than the implementation ? This may be reasonable. > -- > Vincent de Phily > Mobile Devices > +33 (0) 142 119 325 > +353 (0) 85 710 6320 > > Warning > This message (and any associated files) is intended only for the use of its > intended recipient and may contain information that is confidential, subject > to copyright or constitutes a trade secret. If you are not the intended > recipient you are hereby notified that any dissemination, copying or > distribution of this message, or files associated with this message, is > strictly prohibited. If you have received this message in error, please > notify us immediately by replying to the message and deleting it from your > computer. Any views or opinions presented are solely those of the author > vincent.dephily@REDACTED and do not necessarily represent those of > the > company. Although the company has taken reasonable precautions to ensure no > viruses are present in this email, the company cannot accept responsibility > for any loss or damage arising from the use of this email or attachments. > From djk121@REDACTED Fri Oct 15 13:28:59 2010 From: djk121@REDACTED (Dan Kelley) Date: Fri, 15 Oct 2010 07:28:59 -0400 Subject: [erlang-questions] hipe problems with R14B? In-Reply-To: <1287122265.5114.0.camel@seasc1137> References: <1287122265.5114.0.camel@seasc1137> Message-ID: I can't, but this minimal reproducer shows the problem: ----- -module(example). -export([bar/1]). bar (Baz) -> binary_part(Baz, 0, 1). ----- That compiles without +native, but throws the same error with +native. While making up the reproducer, I tried this with the 2-arity version of binary_part (i.e., binary_part(Baz, {0,1}), and that seemed to work fine, FWIW. Thanks, Dan On Fri, Oct 15, 2010 at 1:57 AM, Bengt Kleberg wrote: > Greetings, > > Is it possible for you to supply the code? > > > bengt > > On Fri, 2010-10-15 at 05:47 +0200, Dan Kelley wrote: > > Hi, > > > > I'm seeing the following when trying to compile with hipe in R14B on > linux > > (centos 5.3, x86_64): > > > > ----- > > erlc +native -W hth.erl > > EXITED with reason > > > {'trans_fun/2',{bif,binary_part,{f,0},[{x,0},{integer,0},{integer,1}],{x,1}}} > > @hipe_beam_to_icode:1144 > > > > =ERROR REPORT==== 14-Oct-2010::23:46:42 === > > Error in process <0.27.0> with exit value: > > > {{badmatch,{'EXIT',{{hipe_beam_to_icode,1144,{'trans_fun/2',{bif,binary_part,{f,0},[{x,0},{integer,0},{integer,1}],{x,1}}}},[{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode... > > > > ./hth.erl:none: internal error in native_compile; > > crash reason: {{badmatch, > > {'EXIT', > > {{hipe_beam_to_icode,1144, > > {'trans_fun/2', > > {bif,binary_part, > > {f,0}, > > [{x,0},{integer,0},{integer,1}], > > {x,1}}}}, > > [{hipe_beam_to_icode,trans_fun,2}, > > {hipe_beam_to_icode,trans_fun,2}, > > {hipe_beam_to_icode,trans_fun,2}, > > {hipe_beam_to_icode,trans_fun,2}, > > {hipe_beam_to_icode,trans_fun,2}, > > {hipe_beam_to_icode,trans_fun,2}, > > {hipe_beam_to_icode,trans_op_call,6}, > > {hipe_beam_to_icode,trans_fun,2}]}}}, > > > > [{hipe,get_beam_icode,4},{hipe,'-run_compiler_1/3-fun-0-',4}]} > > ----- > > > > Am I doing something wrong, or is this a bug? > > > > Thanks, > > > > Dan > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From dmercer@REDACTED Fri Oct 15 15:17:09 2010 From: dmercer@REDACTED (David Mercer) Date: Fri, 15 Oct 2010 08:17:09 -0500 Subject: [erlang-questions] Is a process necessary in front of mnesia? In-Reply-To: References: Message-ID: <003001cb6c6b$476409b0$d62c1d10$@com> On Thursday, October 14, 2010, Evans, Matthew wrote: > Another thing I use a gen_server intermediately for is to prioritize > certain operations over others. In one case we are supposed to process > many writes as a low-priority task that doesn't affect the read rate. > Having a gen_server in the way lets us do that. For task prioritization, gen_server does not immediately leap out at me as the solution, since gen_servers normally process their requests in FIFO order. I would think something that utilized selective receive might be more appropriate. What is the solution pattern you use to adapt the gen_server to this purpose? Thanks, DBM From kostis@REDACTED Fri Oct 15 15:14:09 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 15 Oct 2010 16:14:09 +0300 Subject: [erlang-questions] hipe problems with R14B? In-Reply-To: References: <1287122265.5114.0.camel@seasc1137> Message-ID: <4CB853A1.3070303@cs.ntua.gr> Dan Kelley wrote: > I can't, but this minimal reproducer shows the problem: > > ----- > -module(example). > -export([bar/1]). > > bar (Baz) -> > binary_part(Baz, 0, 1). Thanks! This is more than sufficient. In fact, this is _exactly_ the information I would like to see. > That compiles without +native, but throws the same error with +native. > > While making up the reproducer, I tried this with the 2-arity version of > binary_part (i.e., binary_part(Baz, {0,1}), and that seemed to work fine. Well, it's not surprising. This is the first time that a BIF with three arguments was ever introduced to BEAM and the BEAM->ICode translation was never notified about it. It currently has cases for: %%--- bif0 --- trans_fun([{bif,BifName,nofail,[],Reg}|Instructions], Env) -> BifInst = trans_bif0(BifName,Reg), .... %%--- bif1 --- trans_fun([{bif,BifName,{f,Lbl},[_] = Args,Reg}|Instructions], Env) -> {BifInsts,Env1} = trans_bif(1,BifName,Lbl,Args,Reg,Env), ... %%--- bif2 --- trans_fun([{bif,BifName,{f,Lbl},[_,_] = Args,Reg}|Instructions], Env) -> {BifInsts,Env1} = trans_bif(2,BifName,Lbl,Args,Reg,Env), ... and there is no case for BIFs with more than two arguments. I will look into this. But, out of curiosity, is there really a need to have both binary_part/2 and binary_part/3 as BIFs in the language? Kostis PS. I also noticed that the manual contains a typo: 2> binary_part(Bin,{byte_size(Bin), -5)). The first ) should be } From mevans@REDACTED Fri Oct 15 15:33:06 2010 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 15 Oct 2010 09:33:06 -0400 Subject: [erlang-questions] Is a process necessary in front of mnesia? In-Reply-To: <003001cb6c6b$476409b0$d62c1d10$@com> References: ,<003001cb6c6b$476409b0$d62c1d10$@com> Message-ID: In our particular case read requests are processed immediately, while write requests are subjected to a token bucket mechanism within the gen_server to avoid overloading mnesia. Writes can, up to a certain limit, be queued if the token bucket denies the request, otherwise back-pressure is used to hold off the senders. Of course, this approach won't work in all cases but in our deployment it works just fine. Matt ________________________________________ From: David Mercer [dmercer@REDACTED] Sent: Friday, October 15, 2010 9:17 AM To: Evans, Matthew; 'Woody Peterson'; 'Erlang-Questions Questions' Subject: RE: [erlang-questions] Is a process necessary in front of mnesia? On Thursday, October 14, 2010, Evans, Matthew wrote: > Another thing I use a gen_server intermediately for is to prioritize > certain operations over others. In one case we are supposed to process > many writes as a low-priority task that doesn't affect the read rate. > Having a gen_server in the way lets us do that. For task prioritization, gen_server does not immediately leap out at me as the solution, since gen_servers normally process their requests in FIFO order. I would think something that utilized selective receive might be more appropriate. What is the solution pattern you use to adapt the gen_server to this purpose? Thanks, DBM From djk121@REDACTED Fri Oct 15 17:30:36 2010 From: djk121@REDACTED (Dan Kelley) Date: Fri, 15 Oct 2010 11:30:36 -0400 Subject: [erlang-questions] hipe problems with R14B? In-Reply-To: <4CB853A1.3070303@cs.ntua.gr> References: <1287122265.5114.0.camel@seasc1137> <4CB853A1.3070303@cs.ntua.gr> Message-ID: On Fri, Oct 15, 2010 at 9:14 AM, Kostis Sagonas wrote: > > But, out of curiosity, is there really a need to have both binary_part/2 > and binary_part/3 as BIFs in the language? > I'm sure I'm the wrong person to answer this, but for me, I don't have a need for both. I changed over to binary_part/2 earlier, and that works just fine. Dan From kostis@REDACTED Fri Oct 15 19:29:01 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 15 Oct 2010 20:29:01 +0300 Subject: [erlang-questions] hipe problems with R14B? In-Reply-To: <4CB853A1.3070303@cs.ntua.gr> References: <1287122265.5114.0.camel@seasc1137> <4CB853A1.3070303@cs.ntua.gr> Message-ID: <4CB88F5D.9070703@cs.ntua.gr> Kostis Sagonas wrote: > Dan Kelley wrote: >> I can't, but this minimal reproducer shows the problem: >> >> ----- >> -module(example). >> -export([bar/1]). >> >> bar (Baz) -> >> binary_part(Baz, 0, 1). > > .... SNIP > > and there is no case for BIFs with more than two arguments. I will look > into this. This can be fixed by the patch that appears in: http://github.com/kostis/otp/commit/c0933e3d8158a2e783548070e4edd2f0bbacd6bd the repository is: http://github.com/kostis/otp/tree/hipe-fixes Since this is very simple I trust that somebody from the OTP team will include it directly to 'dev'. Kostis From joaohf@REDACTED Fri Oct 15 21:14:02 2010 From: joaohf@REDACTED (=?UTF-8?Q?Jo=C3=A3o_Henrique_Freitas?=) Date: Fri, 15 Oct 2010 16:14:02 -0300 Subject: driver_send_term and receiver arg, expression to build pid Message-ID: Hi, In my free time I start to develop a driver to posix message queue for erlang (http://github.com/joaohf/emq_posix/). In the overall design, every mq can be used like a general file description. I use this behavior and lets the erts select when are some data in the fd to be read. When the data is read I need to know what is the callback processes (inside erlang VM) to send using the function: driver_send_term(ErlDrvPort port, ErlDrvTermData receiver, ErlDrvTermData* term, int n) This blog http://dudefrommangalore.blogspot.com/2009_03_01_archive.html (in my code http://github.com/joaohf/emq_posix/blob/master/c_src/emq_posix.c#L628), tell me to use a expression to 'receiver' argument. Works but I want to know if is right. Thanks PS: #define MAKE_CALLBACK_PID(x) ((ErlDrvTermData)( (( (*x).serial << 15 | (*x).num)) << 4 | (0x0 << 2 | 0x3)) ); -- ----------------------------------------------------------- Jo?o Henrique Freitas - joaohf_at_gmail.com Campinas-SP-Brasil BSD051283 LPI 1 http://www.joaohfreitas.eti.br From amilganju@REDACTED Sat Oct 16 10:08:52 2010 From: amilganju@REDACTED (Amil Ganju) Date: Sat, 16 Oct 2010 01:08:52 -0700 (PDT) Subject: Writing to Windows Eventlog Message-ID: <33c05cdc-b0d9-4473-99f7-af97e5727c58@g4g2000prj.googlegroups.com> Is there a library or any other technique to enable me to write to the Windows Eventlog from an erlang program? Thanks, Amil From schramm.ingo@REDACTED Sat Oct 16 15:05:45 2010 From: schramm.ingo@REDACTED (ingo.schramm) Date: Sat, 16 Oct 2010 06:05:45 -0700 (PDT) Subject: R14B failed to build on Ubuntu 9.04 Message-ID: <080b9e5e-ba66-476f-af0d-63b6e371fb60@26g2000yqv.googlegroups.com> R14B failed to build on Ubuntu 9.04 uname -a Linux hermes 2.6.28-19-generic #65-Ubuntu SMP Thu Sep 16 14:24:42 UTC 2010 x86_64 GNU/Linux gcc --version gcc (GCC) 4.2.4 (Ubuntu 4.2.4-5ubuntu1) ./configure --prefix=/opt/erlang/R14B make test X"$ERTS_SKIP_DEPEND" = X"true" || (cd erts/emulator && ERL_TOP=/ home/ingo/install/otp_src_R14B make generate depend) make[1]: Entering directory `/home/ingo/install/otp_src_R14B/erts/ emulator' make -f x86_64-unknown-linux-gnu/Makefile generate make[2]: Entering directory `/home/ingo/install/otp_src_R14B/erts/ emulator' gcc -g -O3 -I/home/ingo/install/otp_src_R14B/erts/x86_64-unknown- linux-gnu -D_GNU_SOURCE -DHAVE_CONFIG_H -Wall -Wstrict-prototypes - Wmissing-prototypes -Wdeclaration-after-statement -DUSE_THREADS - D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS - D_POSIX_THREAD_SAFE_FUNCTIONS -Ix86_64-unknown-linux-gnu/opt/plain - Ibeam -Isys/unix -Isys/common -Ix86_64-unknown-linux-gnu -Izlib - Ipcre -Ihipe -I../include -I../include/x86_64-unknown-linux-gnu -I../ include/internal -I../include/internal/x86_64-unknown-linux-gnu -c hipe/hipe_mkliterals.c -o obj/x86_64-unknown-linux-gnu/opt/plain/ hipe_mkliterals.o hipe/hipe_mkliterals.c:23:20: error: config.h: No such file or directory In file included from beam/sys.h:54, from hipe/hipe_mkliterals.c:30: sys/unix/erl_unix_sys.h:210: error: expected declaration specifiers or ?...? before ?*? token sys/unix/erl_unix_sys.h:210: warning: data definition has no type or storage class sys/unix/erl_unix_sys.h:210: warning: type defaults to ?int? in declaration of ?RETSIGTYPE? sys/unix/erl_unix_sys.h:210: error: ?RETSIGTYPE? declared as function returning a function sys/unix/erl_unix_sys.h:210: warning: function declaration isn?t a prototype In file included from hipe/hipe_mkliterals.c:30: beam/sys.h:223:2: error: #error Cannot handle this combination of int/ long/void*/size_t sizes beam/sys.h:233:2: error: #error Neither 32 nor 64 bit architecture beam/sys.h:314:2: error: #error Found no appropriate type to use for 'Uint32' and 'Sint32' beam/sys.h:324:2: error: #error Found no appropriate type to use for 'Uint16' and 'Sint16' In file included from beam/erl_threads.h:37, from beam/erl_smp.h:27, from beam/sys.h:341, from hipe/hipe_mkliterals.c:30: ../include/internal/ethread.h:30:37: error: ethread_header_config.h: No such file or directory ../include/internal/ethread.h:196:2: error: #error "No supported thread lib found" In file included from beam/erl_threads.h:37, from beam/erl_smp.h:27, from beam/sys.h:341, from hipe/hipe_mkliterals.c:30: ../include/internal/ethread.h:393: error: expected ?)? before ?*? token ../include/internal/ethread.h:394: error: expected ?)? before ?void? ../include/internal/ethread.h:395: warning: parameter names (without types) in function declaration ../include/internal/ethread.h:397: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?ethr_self? ../include/internal/ethread.h:398: warning: parameter names (without types) in function declaration ../include/internal/ethread.h:401: error: expected ?)? before ?*? token ../include/internal/ethread.h:402: warning: parameter names (without types) in function declaration ../include/internal/ethread.h:403: error: expected ?)? before ?void? ../include/internal/ethread.h:404: warning: parameter names (without types) in function declaration ../include/internal/ethread.h: In function ?ethr_spinlock_init?: ../include/internal/ethread.h:446: warning: implicit declaration of function ?pthread_mutex_init? ../include/internal/ethread.h: In function ?ethr_spinlock_destroy?: ../include/internal/ethread.h:461: warning: implicit declaration of function ?pthread_mutex_destroy? ../include/internal/ethread.h: In function ?ethr_spin_unlock?: ../include/internal/ethread.h:477: warning: implicit declaration of function ?pthread_mutex_unlock? ../include/internal/ethread.h: In function ?ethr_spin_lock?: ../include/internal/ethread.h:495: warning: implicit declaration of function ?pthread_mutex_lock? In file included from ../include/internal/ethread.h:815, from beam/erl_threads.h:37, from beam/erl_smp.h:27, from beam/sys.h:341, from hipe/hipe_mkliterals.c:30: ../include/internal/pthread/ethr_event.h: At top level: ../include/internal/pthread/ethr_event.h:144: error: expected ?)? before ?*? token ../include/internal/pthread/ethr_event.h:145: error: expected ?)? before ?*? token ../include/internal/pthread/ethr_event.h:146: error: expected ?)? before ?*? token ../include/internal/pthread/ethr_event.h:147: error: expected ?)? before ?*? token In file included from beam/erl_threads.h:37, from beam/erl_smp.h:27, from beam/sys.h:341, from hipe/hipe_mkliterals.c:30: ../include/internal/ethread.h:824: error: expected specifier-qualifier- list before ?ethr_event? In file included from ../include/internal/ethread.h:907, from beam/erl_threads.h:37, from beam/erl_smp.h:27, from beam/sys.h:341, from hipe/hipe_mkliterals.c:30: ../include/internal/ethr_mutex.h: In function ?ethr_mutex_trylock?: ../include/internal/ethr_mutex.h:420: warning: implicit declaration of function ?pthread_mutex_trylock? ../include/internal/ethr_mutex.h: In function ?ethr_rwmutex_tryrlock?: ../include/internal/ethr_mutex.h:460: warning: implicit declaration of function ?pthread_rwlock_tryrdlock? ../include/internal/ethr_mutex.h: In function ?ethr_rwmutex_rlock?: ../include/internal/ethr_mutex.h:469: warning: implicit declaration of function ?pthread_rwlock_rdlock? ../include/internal/ethr_mutex.h: In function ?ethr_rwmutex_runlock?: ../include/internal/ethr_mutex.h:477: warning: implicit declaration of function ?pthread_rwlock_unlock? ../include/internal/ethr_mutex.h: In function ?ethr_rwmutex_tryrwlock?: ../include/internal/ethr_mutex.h:485: warning: implicit declaration of function ?pthread_rwlock_trywrlock? ../include/internal/ethr_mutex.h: In function ?ethr_rwmutex_rwlock?: ../include/internal/ethr_mutex.h:494: warning: implicit declaration of function ?pthread_rwlock_wrlock? In file included from beam/erl_threads.h:40, from beam/erl_smp.h:27, from beam/sys.h:341, from hipe/hipe_mkliterals.c:30: beam/erl_term.h: At top level: beam/erl_term.h:417: error: expected specifier-qualifier-list before ?Uint16? beam/erl_term.h:735: error: expected specifier-qualifier-list before ?Uint32? beam/erl_term.h:867: error: expected specifier-qualifier-list before ?Uint32? In file included from beam/erl_smp.h:27, from beam/sys.h:341, from hipe/hipe_mkliterals.c:30: beam/erl_threads.h:57: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?erts_tid_t? beam/erl_threads.h:93: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?erts_tsd_key_t? beam/erl_threads.h:181: error: expected ?)? before ?*? token beam/erl_threads.h:183: error: expected ?)? before ?tid? beam/erl_threads.h:184: error: expected ?)? before ?tid? beam/erl_threads.h:187: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?erts_thr_self? beam/erl_threads.h:188: error: expected ?)? before ?x? beam/erl_threads.h:190: error: expected declaration specifiers or ?...? before ?Uint16? beam/erl_threads.h:267: error: expected declaration specifiers or ?...? before ?Uint16? beam/erl_threads.h:299: error: expected ?)? before ?*? token beam/erl_threads.h:300: error: expected ?)? before ?key? beam/erl_threads.h:301: error: expected ?)? before ?key? beam/erl_threads.h:302: error: expected ?)? before ?key? beam/erl_threads.h:344: error: expected ?)? before ?*? token beam/erl_threads.h:355: error: expected ?)? before ?tid? beam/erl_threads.h:366: error: expected ?)? before ?tid? beam/erl_threads.h:396: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?erts_thr_self? beam/erl_threads.h:407: error: expected ?)? before ?x? beam/erl_threads.h:433: error: expected declaration specifiers or ?...? before ?Uint16? beam/erl_threads.h:1162: error: expected declaration specifiers or ?...? before ?Uint16? beam/erl_threads.h:1444: error: expected ?)? before ?*? token beam/erl_threads.h:1454: error: expected ?)? before ?key? beam/erl_threads.h:1464: error: expected ?)? before ?key? beam/erl_threads.h:1474: error: expected ?)? before ?key? beam/erl_threads.h: In function ?erts_tse_fetch?: beam/erl_threads.h:1486: warning: implicit declaration of function ?ethr_get_ts_event? beam/erl_threads.h:1486: warning: cast to pointer from integer of different size beam/erl_threads.h: In function ?erts_tse_return?: beam/erl_threads.h:1495: warning: implicit declaration of function ?ethr_leave_ts_event? beam/erl_threads.h: In function ?erts_tse_set?: beam/erl_threads.h:1502: warning: implicit declaration of function ?ethr_event_set? beam/erl_threads.h:1502: error: ?ethr_ts_event? has no member named ?event? beam/erl_threads.h: In function ?erts_tse_reset?: beam/erl_threads.h:1509: warning: implicit declaration of function ?ethr_event_reset? beam/erl_threads.h:1509: error: ?ethr_ts_event? has no member named ?event? beam/erl_threads.h: In function ?erts_tse_wait?: beam/erl_threads.h:1516: warning: implicit declaration of function ?ethr_event_wait? beam/erl_threads.h:1516: error: ?ethr_ts_event? has no member named ?event? beam/erl_threads.h: In function ?erts_tse_swait?: beam/erl_threads.h:1525: warning: implicit declaration of function ?ethr_event_swait? beam/erl_threads.h:1525: error: ?ethr_ts_event? has no member named ?event? beam/erl_threads.h: In function ?erts_tse_is_tmp?: beam/erl_threads.h:1534: error: ?erts_tse_t? has no member named ?iflgs? In file included from hipe/hipe_mkliterals.c:30: beam/sys.h: At top level: beam/sys.h:827: error: expected ?)? before ?allowed_activities? beam/sys.h:829: error: expected declaration specifiers or ?...? before ?Uint32? beam/sys.h:859: error: expected ?)? before ?allowed_activities? beam/sys.h:867: error: expected declaration specifiers or ?...? before ?Uint32? In file included from hipe/hipe_mkliterals.c:31: beam/erl_vm.h:179: error: expected specifier-qualifier-list before ?Uint32? In file included from beam/erl_alloc.h:23, from beam/global.h:25, from hipe/hipe_mkliterals.c:32: x86_64-unknown-linux-gnu/opt/plain/erl_alloc_types.h:131: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?ErtsAlcType_t? x86_64-unknown-linux-gnu/opt/plain/erl_alloc_types.h:329: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?erts_alc_n2t? In file included from beam/erl_alloc.h:24, from beam/global.h:25, from hipe/hipe_mkliterals.c:32: beam/erl_alloc_util.h:36: error: expected specifier-qualifier-list before ?ErtsAlcType_t? beam/erl_alloc_util.h:114: error: expected ?)? before ?void? beam/erl_alloc_util.h:115: error: expected ?)? before ?void? beam/erl_alloc_util.h:116: error: expected ?)? before ?void? beam/erl_alloc_util.h:117: error: expected ?)? before ?void? beam/erl_alloc_util.h:119: error: expected ?)? before ?void? beam/erl_alloc_util.h:120: error: expected ?)? before ?void? beam/erl_alloc_util.h:121: error: expected ?)? before ?void? beam/erl_alloc_util.h:122: error: expected ?)? before ?void? beam/erl_alloc_util.h:123: error: expected ?)? before ?void? beam/erl_alloc_util.h:124: error: expected ?)? before ?void? beam/erl_alloc_util.h:125: error: expected ?)? before ?void? beam/erl_alloc_util.h:126: error: expected ?)? before ?void? beam/erl_alloc_util.h:127: error: expected ?)? before ?void? beam/erl_alloc_util.h:128: error: expected ?)? before ?void? beam/erl_alloc_util.h:129: error: expected ?)? before ?void? beam/erl_alloc_util.h:130: error: expected ?)? before ?void? In file included from beam/global.h:25, from hipe/hipe_mkliterals.c:32: beam/erl_alloc.h:47: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?erts_fix_core_allocator_ix? beam/erl_alloc.h:55: error: expected ?)? before ?void? beam/erl_alloc.h:56: error: expected ?)? before ?void? beam/erl_alloc.h:57: error: expected ?)? before ?void? beam/erl_alloc.h:61: warning: parameter names (without types) in function declaration beam/erl_alloc.h:62: error: expected ?)? before ?Uint? beam/erl_alloc.h:63: error: expected ?)? before ?ErtsFixInfo? beam/erl_alloc.h:64: error: expected ?)? before ?void? beam/erl_alloc.h:65: error: expected ?)? before ?void? beam/erl_alloc.h:66: error: expected ?)? before ?void? beam/erl_alloc.h:110: error: expected ?)? before ?void? beam/erl_alloc.h:111: error: expected ?;? before ?void? beam/erl_alloc.h:131: error: expected ?)? before ?Uint? beam/erl_alloc.h:133: error: expected ?)? before ?Uint? beam/erl_alloc.h:135: error: expected ?)? before ?void? beam/erl_alloc.h:137: error: expected ?)? before ?void? beam/erl_alloc.h:139: error: expected declaration specifiers or ?...? before ?ErtsAlcType_t? beam/erl_alloc.h:181: error: expected ?)? before ?type? beam/erl_alloc.h:194: error: expected ?)? before ?type? beam/erl_alloc.h:208: error: expected ?)? before ?type? beam/erl_alloc.h:218: error: expected ?)? before ?type? beam/erl_alloc.h:228: error: expected ?)? before ?type? In file included from beam/global.h:28, from hipe/hipe_mkliterals.c:32: beam/hash.h:71: error: expected specifier-qualifier-list before ?ErtsAlcType_t? beam/hash.h:81: error: expected ?)? before ?char? beam/hash.h:82: error: expected ?)? before ?Hash? In file included from beam/global.h:29, from hipe/hipe_mkliterals.c:32: beam/index.h:42: error: expected specifier-qualifier-list before ?ErtsAlcType_t? beam/index.h:53: error: expected ?)? before ?IndexTable? beam/index.h: In function ?erts_index_lookup?: beam/index.h:67: error: ?IndexTable? has no member named ?seg_table? In file included from beam/erl_process.h:36, from beam/beam_load.h:24, from beam/export.h:74, from beam/global.h:31, from hipe/hipe_mkliterals.c:32: beam/erl_process_lock.h: At top level: beam/erl_process_lock.h:57: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?ErtsProcLocks? beam/erl_process_lock.h:65: error: expected specifier-qualifier-list before ?ErtsProcLocks? In file included from beam/erl_process.h:41, from beam/beam_load.h:24, from beam/export.h:74, from beam/global.h:31, from hipe/hipe_mkliterals.c:32: beam/erl_message.h:46: error: expected specifier-qualifier-list before ?Uint64? In file included from beam/erl_node_tables.h:44, from beam/external.h:77, from beam/erl_message.h:49, from beam/erl_process.h:41, from beam/beam_load.h:24, from beam/export.h:74, from beam/global.h:31, from hipe/hipe_mkliterals.c:32: beam/erl_monitors.h:100: error: expected specifier-qualifier-list before ?Sint16? beam/erl_monitors.h:105: error: expected specifier-qualifier-list before ?Sint16? beam/erl_monitors.h:117: error: expected specifier-qualifier-list before ?Sint16? beam/erl_monitors.h:130: error: expected specifier-qualifier-list before ?Sint16? In file included from beam/erl_sys_driver.h:38, from beam/erl_port_task.h:28, from beam/erl_node_tables.h:47, from beam/external.h:77, from beam/erl_message.h:49, from beam/erl_process.h:41, from beam/beam_load.h:24, from beam/export.h:74, from beam/global.h:31, from hipe/hipe_mkliterals.c:32: beam/erl_driver.h:28:22: error: config.h: No such file or directory beam/erl_driver.h:55:34: error: erl_int_sizes_config.h: No such file or directory beam/erl_driver.h:199:2: error: #error No 64-bit integer type In file included from beam/external.h:77, from beam/erl_message.h:49, from beam/erl_process.h:41, from beam/beam_load.h:24, from beam/export.h:74, from beam/global.h:31, from hipe/hipe_mkliterals.c:32: beam/erl_node_tables.h:113: error: expected specifier-qualifier-list before ?Uint32? beam/erl_node_tables.h:153: error: expected specifier-qualifier-list before ?Uint32? beam/erl_node_tables.h: In function ?erts_smp_de_links_lock?: beam/erl_node_tables.h:249: error: ?DistEntry? has no member named ?lnk_mtx? beam/erl_node_tables.h: In function ?erts_smp_de_links_unlock?: beam/erl_node_tables.h:255: error: ?DistEntry? has no member named ?lnk_mtx? In file included from beam/erl_message.h:49, from beam/erl_process.h:41, from beam/beam_load.h:24, from beam/export.h:74, from beam/global.h:31, from hipe/hipe_mkliterals.c:32: beam/external.h: At top level: beam/external.h:97: error: expected specifier-qualifier-list before ?Uint32? beam/external.h:122: error: expected specifier-qualifier-list before ?Uint32? beam/external.h:155: error: expected declaration specifiers or ?...? before ?Uint32? beam/external.h:156: error: expected declaration specifiers or ?...? before ?Uint32? beam/external.h: In function ?erts_free_dist_ext_copy?: beam/external.h:207: warning: implicit declaration of function ?erts_free? In file included from beam/erl_process.h:41, from beam/beam_load.h:24, from beam/export.h:74, from beam/global.h:31, from hipe/hipe_mkliterals.c:32: beam/erl_message.h: At top level: beam/erl_message.h:223: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_message.h:224: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_message.h:225: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_message.h:226: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_message.h:234: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? In file included from beam/erl_process.h:46, from beam/beam_load.h:24, from beam/export.h:74, from beam/global.h:31, from hipe/hipe_mkliterals.c:32: beam/erl_bif_timer.h:32: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? In file included from beam/beam_load.h:24, from beam/export.h:74, from beam/global.h:31, from hipe/hipe_mkliterals.c:32: beam/erl_process.h:327: error: expected specifier-qualifier-list before ?Uint32? beam/erl_process.h:587: error: expected specifier-qualifier-list before ?Uint32? beam/erl_process.h:799: error: expected specifier-qualifier-list before ?Uint16? beam/erl_process.h: In function ?erts_heap_frag_shrink?: beam/erl_process.h:815: error: ?Process? has no member named ?mbuf? beam/erl_process.h: At top level: beam/erl_process.h:1063: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process.h:1113: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process.h:1114: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process.h:1120: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process.h:1124: error: expected declaration specifiers or ?...? before ?Uint32? beam/erl_process.h:1184: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process.h:1189: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process.h: In function ?erts_psd_get?: beam/erl_process.h:1212: error: ?Process? has no member named ?psd? beam/erl_process.h:1212: error: ?Process? has no member named ?psd? beam/erl_process.h: At top level: beam/erl_process.h:1220: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process.h: In function ?erts_psd_set?: beam/erl_process.h:1238: error: ?Process? has no member named ?psd? beam/erl_process.h:1239: error: ?Process? has no member named ?psd? beam/erl_process.h:1240: error: ?Process? has no member named ?psd? beam/erl_process.h:1247: error: ?plocks? undeclared (first use in this function) beam/erl_process.h:1247: error: (Each undeclared identifier is reported only once beam/erl_process.h:1247: error: for each function it appears in.) beam/erl_process.h:1247: warning: passing argument 3 of ?erts_psd_set_init? makes pointer from integer without a cast beam/erl_process.h:1247: error: too many arguments to function ?erts_psd_set_init? beam/erl_process.h: At top level: beam/erl_process.h:1274: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process.h:1291: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process.h: In function ?erts_proc_set_error_handler?: beam/erl_process.h:1297: error: ?plocks? undeclared (first use in this function) beam/erl_process.h:1297: error: too many arguments to function ?erts_psd_set? beam/erl_process.h: In function ?erts_get_current_pid?: beam/erl_process.h:1376: error: ?Process? has no member named ?id? In file included from beam/erl_process.h:1587, from beam/beam_load.h:24, from beam/export.h:74, from beam/global.h:31, from hipe/hipe_mkliterals.c:32: beam/erl_process_lock.h: At top level: beam/erl_process_lock.h:643: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process_lock.h:645: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process_lock.h:646: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process_lock.h:658: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process_lock.h:681: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process_lock.h:695: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process_lock.h:773: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process_lock.h:773: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process_lock.h:798: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process_lock.h:800: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/erl_process_lock.h: In function ?erts_pid2proc_opt?: beam/erl_process_lock.h:814: error: ?Process? has no member named ?id? beam/erl_process_lock.h:816: error: ?Process? has no member named ?status? In file included from beam/export.h:74, from beam/global.h:31, from hipe/hipe_mkliterals.c:32: beam/beam_load.h: At top level: beam/beam_load.h:26: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? In file included from beam/global.h:33, from hipe/hipe_mkliterals.c:32: beam/register.h:56: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/register.h:57: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/register.h:60: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/register.h:62: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/register.h:64: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? In file included from hipe/hipe_mkliterals.c:32: beam/global.h:184: error: expected specifier-qualifier-list before ?Uint32? beam/global.h: In function ?erts_prtsd_get?: beam/global.h:228: error: ?Port? has no member named ?psd? beam/global.h:228: error: ?Port? has no member named ?psd? beam/global.h: In function ?erts_prtsd_set?: beam/global.h:234: error: ?Port? has no member named ?psd? beam/global.h:235: error: ?Port? has no member named ?psd? beam/global.h:236: error: ?Port? has no member named ?psd? beam/global.h:240: error: ?Port? has no member named ?psd? beam/global.h:240: warning: implicit declaration of function ?erts_alloc? beam/global.h:241: error: ?Port? has no member named ?psd? beam/global.h: At top level: beam/global.h:363: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/global.h:367: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/global.h:370: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/global.h: In function ?erts_may_save_closed_port?: beam/global.h:543: error: ?Port? has no member named ?snapshot? beam/global.h: At top level: beam/global.h:847: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/global.h:861: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/global.h:1009: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/global.h:1065: error: expected specifier-qualifier-list before ?Uint32? beam/global.h:1244: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/global.h:1244: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h:1248: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?erts_portid2status? beam/global.h:1251: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h:1251: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h:1252: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h:1253: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h:1254: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h:1255: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?erts_port_status_get? beam/global.h:1260: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/global.h:1260: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h: In function ?erts_id2port_sflgs?: beam/global.h:1273: error: ?Port? has no member named ?status? beam/global.h:1273: error: ?sflgs? undeclared (first use in this function) beam/global.h: In function ?erts_port_release?: beam/global.h:1310: error: ?Port? has no member named ?status? beam/global.h:1310: error: ?Uint32? undeclared (first use in this function) beam/global.h: In function ?erts_drvport2port?: beam/global.h:1321: error: ?Port? has no member named ?status? beam/global.h:1321: error: ?Uint32? undeclared (first use in this function) beam/global.h: In function ?erts_drvportid2port?: beam/global.h:1336: error: ?Port? has no member named ?status? beam/global.h:1336: error: ?Uint32? undeclared (first use in this function) beam/global.h: At top level: beam/global.h:1345: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?erts_portid2status? beam/global.h: In function ?erts_is_port_alive?: beam/global.h:1367: warning: implicit declaration of function ?erts_portid2status? beam/global.h:1367: error: ?Uint32? undeclared (first use in this function) beam/global.h: In function ?erts_is_valid_tracer_port?: beam/global.h:1374: error: ?Uint32? undeclared (first use in this function) beam/global.h: At top level: beam/global.h:1378: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h:1379: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h: In function ?erts_port_status_bandor_set?: beam/global.h:1383: error: ?Port? has no member named ?status? beam/global.h:1383: error: ?band_status? undeclared (first use in this function) beam/global.h:1384: error: ?Port? has no member named ?status? beam/global.h:1384: error: ?bor_status? undeclared (first use in this function) beam/global.h: At top level: beam/global.h:1388: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h: In function ?erts_port_status_band_set?: beam/global.h:1392: error: ?Port? has no member named ?status? beam/global.h:1392: error: ?status? undeclared (first use in this function) beam/global.h: At top level: beam/global.h:1396: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h: In function ?erts_port_status_bor_set?: beam/global.h:1400: error: ?Port? has no member named ?status? beam/global.h:1400: error: ?status? undeclared (first use in this function) beam/global.h: At top level: beam/global.h:1404: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h: In function ?erts_port_status_set?: beam/global.h:1408: error: ?Port? has no member named ?status? beam/global.h:1408: error: ?status? undeclared (first use in this function) beam/global.h: At top level: beam/global.h:1412: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?erts_port_status_get? beam/global.h:1518: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?make_broken_hash? beam/global.h:1519: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?block_hash? beam/global.h:1520: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?make_hash2? beam/global.h:1521: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?make_hash? beam/global.h:1527: error: expected declaration specifiers or ?...? before ?Uint64? beam/global.h:1528: error: expected declaration specifiers or ?...? before ?Sint64? beam/global.h:1602: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/global.h:1624: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?erts_call_trace? beam/global.h:1795: error: expected declaration specifiers or ?...? before ?Uint32? beam/global.h:1827: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/global.h:1847: error: expected declaration specifiers or ?...? before ?ErtsProcLocks? beam/global.h: In function ?erts_alloc_message_heap?: beam/global.h:1868: error: ?Process? has no member named ?status? beam/global.h:1885: error: ?Process? has no member named ?off_heap? In file included from hipe/hipe_mkliterals.c:35: beam/erl_bits.h: At top level: beam/erl_bits.h:186: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?erts_bs_get_unaligned_uint32? In file included from hipe/hipe_mkliterals.c:142: beam/erl_binary.h:154: error: expected declaration specifiers or ?...? before ?ErtsAlcType_t? beam/erl_binary.h:181: error: expected declaration specifiers or ?...? before ?ErtsAlcType_t? In file included from hipe/hipe_mkliterals.c:142: beam/erl_binary.h: In function ?erts_get_aligned_binary_bytes?: beam/erl_binary.h:198: error: too many arguments to function ?erts_get_aligned_binary_bytes_extra? beam/erl_binary.h: At top level: beam/erl_binary.h:202: error: expected declaration specifiers or ?...? before ?ErtsAlcType_t? beam/erl_binary.h: In function ?erts_free_aligned_binary_bytes_extra?: beam/erl_binary.h:205: error: ?allocator? undeclared (first use in this function) beam/erl_binary.h: In function ?erts_free_aligned_binary_bytes?: beam/erl_binary.h:212: error: too many arguments to function ?erts_free_aligned_binary_bytes_extra? beam/erl_binary.h: In function ?erts_bin_drv_alloc_fnf?: beam/erl_binary.h:230: warning: implicit declaration of function ?erts_alloc_fnf? beam/erl_binary.h:230: warning: assignment makes pointer from integer without a cast beam/erl_binary.h: In function ?erts_bin_drv_alloc?: beam/erl_binary.h:240: warning: assignment makes pointer from integer without a cast beam/erl_binary.h: In function ?erts_bin_nrml_alloc?: beam/erl_binary.h:251: warning: assignment makes pointer from integer without a cast beam/erl_binary.h: In function ?erts_bin_realloc_fnf?: beam/erl_binary.h:263: warning: implicit declaration of function ?erts_realloc_fnf? beam/erl_binary.h:263: warning: assignment makes pointer from integer without a cast beam/erl_binary.h:265: warning: assignment makes pointer from integer without a cast beam/erl_binary.h: In function ?erts_bin_realloc?: beam/erl_binary.h:277: warning: assignment makes pointer from integer without a cast beam/erl_binary.h:279: warning: assignment makes pointer from integer without a cast beam/erl_binary.h:281: warning: implicit declaration of function ?erts_realloc_n_enomem? beam/erl_binary.h: In function ?erts_create_magic_binary?: beam/erl_binary.h:305: warning: initialization makes pointer from integer without a cast beam/erl_binary.h:307: warning: implicit declaration of function ?erts_alloc_n_enomem? hipe/hipe_mkliterals.c: At top level: hipe/hipe_mkliterals.c:264: error: ?struct process? has no member named ?off_heap? hipe/hipe_mkliterals.c:265: error: ?struct process? has no member named ?mbuf? hipe/hipe_mkliterals.c:266: error: ?struct process? has no member named ?id? hipe/hipe_mkliterals.c:267: error: ?struct process? has no member named ?flags? hipe/hipe_mkliterals.c:268: error: ?struct process? has no member named ?fvalue? hipe/hipe_mkliterals.c:269: error: ?struct process? has no member named ?freason? hipe/hipe_mkliterals.c:270: error: ?struct process? has no member named ?ftrace? hipe/hipe_mkliterals.c:371: error: ?struct process? has no member named ?msg? hipe/hipe_mkliterals.c:372: error: ?struct process? has no member named ?msg? hipe/hipe_mkliterals.c:459: error: ?struct process? has no member named ?off_heap? hipe/hipe_mkliterals.c:459: error: request for member ?first? in something not a structure or union hipe/hipe_mkliterals.c:461: error: initializer element is not constant hipe/hipe_mkliterals.c:461: error: (near initialization for ?rts_params[0].value?) make[2]: *** [obj/x86_64-unknown-linux-gnu/opt/plain/ hipe_mkliterals.o] Error 1 make[2]: Leaving directory `/home/ingo/install/otp_src_R14B/erts/ emulator' make[1]: *** [generate] Error 2 make[1]: Leaving directory `/home/ingo/install/otp_src_R14B/erts/ emulator' make: *** [depend] Error 2 From kenrobinsonster@REDACTED Sat Oct 16 15:34:37 2010 From: kenrobinsonster@REDACTED (Ken Robinson) Date: Sat, 16 Oct 2010 23:34:37 +1000 Subject: [erlang-questions] Staring more than one instance of a gen_server In-Reply-To: References: Message-ID: Hi Ahmed, Thanks for your reply. I read that part of the documentation. I feel I want some less dynamic. I read some of the gen_server doco (http://www.erlang.org/doc/man/gen_server.html) where it referred to the start_link function: start_link(ServerName, Module, Args, Options) -> Result ... ServerName = {local,Name} | {global,GlobalName} I'm going to try changing the Id to the module name plus a number then making the ServerName the same as that. Ken. {Id, StartFunc, Restart, Shutdown, Type, Modules} from the On Fri, Oct 15, 2010 at 6:31 AM, Ahmed Omar wrote: > Hi Ken, > It all depends on your purposes. If you want to start these processes as a > part of a supervision tree, so you would probably like to use > simple-one-for-one strategy and children dynamically > supervisor:start_child(Sup, List) > http://www.erlang.org/doc/design_principles/sup_princ.html#id67619 > On Thu, Oct 14, 2010 at 5:15 PM, Ken Robinson > wrote: >> >> Hi All, >> If I wanted to start more than one instance of a gen_server what is >> the best practice? The gen_servers would be exactly the same except >> for the seeding parameters. From what I've found I could use the child >> spec and seed with specific names through the arguments like so: >> start_link(PortNumber, MySpecialName) -> gen_server:start_link({local, >> MySpecialName, ?MODULE, PortNumber, []). >> Would that work? >> -- >> regards, >> Ken Robinson >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > -- regards, Ken Robinson Mob +61438681120 Home +61738523767 Work +61733063137 From mikpe@REDACTED Sat Oct 16 16:23:51 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Sat, 16 Oct 2010 16:23:51 +0200 Subject: [erlang-questions] R14B failed to build on Ubuntu 9.04 In-Reply-To: <080b9e5e-ba66-476f-af0d-63b6e371fb60@26g2000yqv.googlegroups.com> References: <080b9e5e-ba66-476f-af0d-63b6e371fb60@26g2000yqv.googlegroups.com> Message-ID: <19641.46455.874229.316340@pilspetsen.it.uu.se> On Sat, 16 Oct 2010 06:05:45 -0700 (PDT), "ingo.schramm" wrote: > R14B failed to build on Ubuntu 9.04 > > uname -a > Linux hermes 2.6.28-19-generic #65-Ubuntu SMP Thu Sep 16 14:24:42 UTC > 2010 x86_64 GNU/Linux > > gcc --version > gcc (GCC) 4.2.4 (Ubuntu 4.2.4-5ubuntu1) > > ./configure --prefix=3D/opt/erlang/R14B > > make > > test X"$ERTS_SKIP_DEPEND" =3D X"true" || (cd erts/emulator && ERL_TOP=3D/ > home/ingo/install/otp_src_R14B make generate depend) > make[1]: Entering directory `/home/ingo/install/otp_src_R14B/erts/ > emulator' > make -f x86_64-unknown-linux-gnu/Makefile generate > make[2]: Entering directory `/home/ingo/install/otp_src_R14B/erts/ > emulator' > gcc -g -O3 -I/home/ingo/install/otp_src_R14B/erts/x86_64-unknown- > linux-gnu -D_GNU_SOURCE -DHAVE_CONFIG_H -Wall -Wstrict-prototypes - > Wmissing-prototypes -Wdeclaration-after-statement -DUSE_THREADS - > D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS - > D_POSIX_THREAD_SAFE_FUNCTIONS -Ix86_64-unknown-linux-gnu/opt/plain - > Ibeam -Isys/unix -Isys/common -Ix86_64-unknown-linux-gnu -Izlib - > Ipcre -Ihipe -I../include -I../include/x86_64-unknown-linux-gnu -I../ > include/internal -I../include/internal/x86_64-unknown-linux-gnu -c > hipe/hipe_mkliterals.c -o obj/x86_64-unknown-linux-gnu/opt/plain/ > hipe_mkliterals.o > hipe/hipe_mkliterals.c:23:20: error: config.h: No such file or > directory This is the real error. > In file included from beam/sys.h:54, > from hipe/hipe_mkliterals.c:30: > sys/unix/erl_unix_sys.h:210: error: expected declaration specifiers or > =91...=92 before =91*=92 token > sys/unix/erl_unix_sys.h:210: warning: data definition has no type or > storage class The rest are just consequences of the first error. ./configure is supposed to create erts/x86_64-unknown-linux-gnu/config.h, but apparently it failed to do so for you. You need to figure out why, or at least post the output of the ./configure step for us to look at. FWIW, R14B builds just fine here on CentOS 5.5 / x86_64. /Mikael From anton.krasovsky@REDACTED Sat Oct 16 19:22:16 2010 From: anton.krasovsky@REDACTED (Anton Krasovsky) Date: Sat, 16 Oct 2010 18:22:16 +0100 Subject: [erlang-questions] Staring more than one instance of a gen_server In-Reply-To: References: Message-ID: Ken, the usual way of starting multiple instances of the same gen_server module is to use start_link/3 function. However, since start_link/3 will not register new process you'll need to hold on to Pid of newly created process and use it later when calling your server with gen_server:cast or gen_server:call. However, it's the best practice to use supervisors just as Ahmed suggested. Regards, Anton On Sat, Oct 16, 2010 at 2:34 PM, Ken Robinson wrote: > Hi Ahmed, > Thanks for your reply. > I read that part of the documentation. I feel I want some less dynamic. > I read some of the gen_server doco > (http://www.erlang.org/doc/man/gen_server.html) where it referred to > the start_link function: > start_link(ServerName, Module, Args, Options) -> Result > ... > ServerName = {local,Name} | {global,GlobalName} > I'm going to try changing the Id to the module name plus a number then > making the ServerName the same as that. > Ken. > > {Id, StartFunc, Restart, Shutdown, Type, Modules} from the > On Fri, Oct 15, 2010 at 6:31 AM, Ahmed Omar wrote: >> Hi Ken, >> It all depends on your purposes. If you want to start these processes as a >> part of a supervision tree, so you would probably like to use >> simple-one-for-one strategy and children dynamically >> supervisor:start_child(Sup, List) >> http://www.erlang.org/doc/design_principles/sup_princ.html#id67619 >> On Thu, Oct 14, 2010 at 5:15 PM, Ken Robinson >> wrote: >>> >>> Hi All, >>> If I wanted to start more than one instance of a gen_server what is >>> the best practice? The gen_servers would be exactly the same except >>> for the seeding parameters. From what I've found I could use the child >>> spec and seed with specific names through the arguments like so: >>> start_link(PortNumber, MySpecialName) -> gen_server:start_link({local, >>> MySpecialName, ?MODULE, PortNumber, []). >>> Would that work? >>> -- >>> regards, >>> Ken Robinson >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> >> >> -- >> Best Regards, >> - Ahmed Omar >> http://nl.linkedin.com/in/adiaa >> Follow me on twitter >> @spawn_think >> > > > > -- > regards, > Ken Robinson > Mob +61438681120 > Home +61738523767 > Work +61733063137 > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From boris.muehmer@REDACTED Sat Oct 16 22:27:11 2010 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Sat, 16 Oct 2010 22:27:11 +0200 Subject: [erlang-questions] R14B failed to build on Ubuntu 9.04 In-Reply-To: <19641.46455.874229.316340@pilspetsen.it.uu.se> References: <080b9e5e-ba66-476f-af0d-63b6e371fb60@26g2000yqv.googlegroups.com> <19641.46455.874229.316340@pilspetsen.it.uu.se> Message-ID: I did some tests, because I was courious... :-) My installation steps are documented at: http://borisukun.blogspot.com/2010/10/installing-erlangotp-r14b-on-ubuntu.html For the R14B / Ubuntu 9.04 test I used a virtual machine (KVM). The source was compiled without any problems. Using a "ssh -X" connection to the virtual system I could even run the "wx:demo/0". The only problem with the wx-demo was, when I tried the "gl" demo it resulted in a crash of the demo. When I tried to restart the demo, erl seg-faulted. q()-ing the eshell after wx-demo (just start and close) also resulted in erl seg-faulting. I also gave "debugger:start/0" a try... it showed up. But after I closed it (without any debugging, just start and close) and q()-ing the eshell, erl seg-faulted as well. I tried tv:start/0 and toolbar:start/0: nothing happend: 1) tv:start just returned a PID... nothing more 2) toolbar:start was silent for some time and than returned: ** exception exit: {startup_timeout,toolbar} in function toolbar:init_ok/1 - boris From mk@REDACTED Sun Oct 17 13:30:45 2010 From: mk@REDACTED (Morten Krogh) Date: Sun, 17 Oct 2010 13:30:45 +0200 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> <4CB6B589.9050003@cs.ntua.gr> <4CB6BDDC.3060900@amberbio.com> Message-ID: <4CBADE65.7010909@amberbio.com> Nicholas, nice post. I agree that fully automatic garbage collection probably will need to stop all threads at the same time. Actually, I still think binary sharing is an example of it. It is restricted relative to the general case, which makes it simpler. Alternatively, one could accept that the shared data should survive unless someone killed it. Or one could just accept that the programmer should accept that garbage collection is done in one sweep. I think it is insctructive to be precise about what the problem could be. Here is one example. A giant immutable key value store accesed by http read requests. To get highest performance, you would put the data in one tree, or hash table, shared by every erlang process. You would then spawn worker processes for each core, and let them handle http requests. To respond, they would access the tree, grab the data, and reply. This is completely equivalent to one process owning the tree, and all workers message passing into it, but performance is higher with the "pointer access" than through the message passing and context switching. Actually, you can make the API exactly like message passing. There would be no automatic garbage collection of the tree, and it would have to be killed explicitly, exactly like a process today. Shared memory and message passing to a data owner process is really the same thing semantically. The only difference is performance. Actually shared memory is a bad wording. It should be called "C like pointer access to the data". I guess ets is made based on this logic. Anyway, I regret using the word thread earlier. One word, process, is enough. The only real question is whether erlang should have a way for one process to read state of another process without the full send, put in message queue, context switch, recieve, lookup data, send, put in message queue, receive overhead. The semantics is the same. The programing model is the same. The speed is higher. Cheers, Morten. On 10/14/10 4:47 PM, Nicholas Frechette wrote: > Sharing binaries is fine because they use a different heap altogether and a > different garbage collection scheme (reference counting). > Sharing a nif ressource implies you manage it manually through nifs, how you > manage it is largely up to you. > As far as other languages go, yes, most (all?) of them stop all threads to > garbage collect (if those threads share memory). Java does this, C# does > this, ruby, python, etc. C# recently or will shortly introduce a garbage > collection algorithm that runs concurrently and thus doesn't stop all > threads but from what I can remember, it isn't 100% guaranteed (if threads > continue to allocate past a threshold, I believe they are still all stopped > waiting for GC to complete). This an important reason why multi generational > garbage collection algorithms are so popular: it keeps most GC cycles quick, > thus stopping all threads for the least amount of time. > > The idea behind each process having its own heap is twofold: it helps memory > locality and it isolates that heap such that it can be GCed on its own, > without interrupting any other process. Processes are such essentially > sandboxed in that regard. > This also means that any two processes, or more, can run garbage collection > concurrently without issues. > > While it sounds bad to stop all processes/threads sharing a piece of memory, > in reality, for most desktop applications, it is a non issue. The story is > different for server software however, where they tend to allocate a lot and > have many threads. > > When you want to share some data between P1 and P2, if they do not share > everything, a problem arises quite quickly: how do you mark allocations that > should be shared and those that shouldn't? If you share everything, > allocation is easy and goes in 1 heap (ignoring binaries here in their own > heap). If you do not, you have to select those allocations. In both cases, > because some memory is shared, the simplest way to GC will be to stop both > processes. After all, you will need to inspect their respective stacks to > see if they reference things in that shared heap and if the processes are > running, the stack will keep changing. If even for the time of a memcpy of > the stack, all processes will have to stop at some point or another. You > also won't be able to run the GC concurrently on P1 and P2 if they share > memory, or at least, it will be quite hard. > > Sharing memory and garbage collection is a non trivial problem to solve and > as erlang currently does, the simplest, most elegant way to deal with the > problem is to not share at all (and always copy) to avoid the problem. > > Nicholas > > On Thu, Oct 14, 2010 at 4:22 AM, Morten Krogh wrote: > >> >>> You are missing the point -- or at least the point which I think Richard >>> is making. In the scheme you propose, T2's execution will be influenced by a >>> piece of data that _in principle_ is not shared. For example, T2 needs to be >>> stopped executing or synchronized with T1 by garbage collection which might >>> take place when process T1 allocates some big D2. >>> >>> How is that different than the two processes sharing everything? >>> >>> Kostis >>> >> Kostis, >> >> I don't understand you. Are you saying that it is impossible to implement >> multithreading unless the threads share everything, and garbage collection >> will be slow? >> >> What about two erlang processes that share a binary. Isn't that how it >> works today? >> What about two erlang processes that share a nif resource? >> What about other languages and virtual machines. They have multithreading. >> >> We must be talking past each other here. >> >> By the way, I am not advocating such a change to erlang. I think you can >> get around shared memory by choosing the right processes, partition data >> correctly etc. >> It is only the most computationally intensive cases where you can't, and >> there one can use a nif resource or some external way of sharing data. >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From steven.charles.davis@REDACTED Sun Oct 17 17:52:57 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 17 Oct 2010 08:52:57 -0700 (PDT) Subject: Shared/Hybrid Heap In-Reply-To: <4CBADE65.7010909@amberbio.com> References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> <4CB6B589.9050003@cs.ntua.gr> <4CB6BDDC.3060900@amberbio.com> <4CBADE65.7010909@amberbio.com> Message-ID: <14a8a90f-2425-4e42-9519-849f0158ff22@h7g2000yqn.googlegroups.com> On Oct 17, 6:30?am, Morten Krogh wrote: > Actually shared memory is a bad wording. It should be called "C like > pointer access to the data". That sounds even worse! Have we learned nothing from all that history of thread-aches in mainstream computing? > The only real question is whether erlang should have a way for > one process to read state of another... I'm not sure you have fully thought through the implications of that suggestion! ;) More concretely, consider the meaning of the java keyword "volatile", what issue it points us to, and how that issue has deep implications for the entire Java platform. Best regs, /s From kostis@REDACTED Sun Oct 17 18:11:59 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Sun, 17 Oct 2010 19:11:59 +0300 Subject: [erlang-questions] Dyalizer warnings for too wide return type In-Reply-To: <201010151145.03649.vincent.dephily@mobile-devices.fr> References: <201010151145.03649.vincent.dephily@mobile-devices.fr> Message-ID: <4CBB204F.4000404@cs.ntua.gr> Vincent de Phily wrote: > On Tuesday 12 October 2010 08:48:57 Alexey Romanov wrote: >> One problem I have with Dialyzer is that it complains about cases >> where the return type specified is too wide. >> >> E.g. >> >> -type handle_cast_return() :: {noreply, tuple()} | {noreply, tuple(), >> integer()} | {stop, any(), tuple()}. >> -spec handle_cast(any(), tuple()) -> handle_cast_return(). >> handle_cast(_Msg, State) -> >> {noreply, State}. >> >> The definition says that handle_cast returns what it is supposed to >> return according to behaviour, but it gives this warning: > [...] >> the given specification satisfies it, and shouldn't result in a >> warning. Or at least, there should be an option to turn this set of >> warnings off, and I don't see one in >> `dialyzer -Whelp`. > > It sounds like -Wunderspecs or -Woverspecs would do that, but that doesn't > seem to work. Actually it looks like (contrary to documentation) -Wunderspecs > is on by default and we actually need a -Wno_underspecs option. > > But a global no_underspec for the program sounds dangerous too (you'll hide > actual underspecs which should be fixed). Maybe dialyzer could ease the pain > by handly behaviour spec specially ? > >> -spec some_test() -> boolean(). >> some_test() -> false. %% TODO implement later >> >> %% in a different module >> foo() -> >> case m1:some_test() of >> true -> ... >> false -> ... >> end. >> >> In addition to the warning about some_test() I get warnings about >> foo(): "Pattern 'true' can never match type 'false' ", any functions >> called only from the 'true' branch are reported to be unused, etc. >> While these warnings are correct, they ignore my specification for >> some_test(). Can Dialyzer be forced to prefer the return type given in >> the spec over the` one it infers? > > Yes, that annoys me as well. I understand the warning for the some_spec > function, but I think it shouldn't trickle down to the foo function. I have > another use-case where I recompile a module at runtime but provide an initial > implementation that always returns false. I could have written a much longer reply in this thread, but I'll keep this reply a bit short with the risk of sounding a bit impolite. This is not my intention. The whole discussion above is a bit naive, but perhaps this is primarily my fault because there is no document that explains dialyzer's decisions and philosophy in general terms. For a long time now I wanted to write a "dialyzer FAQ". I hope I can come around to that pretty soon. Anyway, the short answer is that dialyzer will never do what users may initially think it would be better if it did. The only information that dialyzer has to work with is the code and whenever it finds something that is a discrepancy between what the code does and what the programmer claims it does (e.g. in a spec) it spits a warning about it. Dialyzer cannot be in the minds of programmers and what's their ultimate intention even if they add "%% TODO implement later" comments :-) So, my reply to Alexey's message is that in a situation like the one he describes the proper action is to either comment out the true case in foo's code or realize that now is the time for this "later". Now, having written all these, there is a dialyzer option that we have primarily added for internal debugging purposes, but may come in handy in situations like that. If you want dialyzer to do its analysis based on success typings only and ignore the specs of the analyzed modules, you can use the option '--no_spec'. But there is no option "trust the specs, please" for cases where dialyzer can determine that they do not correspond to the code. Whenever you want to do that, simply don't use dialyzer! > I'd like dializer to trust me when I declare my interface, even if the > implementation doesn't match that. Actually, maybe we could make the case that > non-exported functions must have an exact spec whereas exported function can > have a spec that is more allowing than the implementation ? Sorry, but I firmly believe that this last part is wrong. Having published interfaces that do not correspond to the implementation is not a good idea. Among other things, what sort of confidence do you have that the client code works ok? How do you test things in this situation? Cheers, Kostis From boris.muehmer@REDACTED Sun Oct 17 20:58:26 2010 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Sun, 17 Oct 2010 20:58:26 +0200 Subject: [erlang-questions] R14B failed to build on Ubuntu 9.04 In-Reply-To: References: <080b9e5e-ba66-476f-af0d-63b6e371fb60@26g2000yqv.googlegroups.com> <19641.46455.874229.316340@pilspetsen.it.uu.se> Message-ID: Just a small correction concerning tv:start/0 and toolbar:start/0: 2010/10/16 Boris M?hmer : > I tried tv:start/0 and toolbar:start/0: nothing happend: > ?1) tv:start just returned a PID... nothing more > ?2) toolbar:start was silent for some time and than returned: > ? ? ? ?** exception exit: {startup_timeout,toolbar} > ? ? ? ? ? ? in function ?toolbar:init_ok/1 It turns out (tcl/)tk wasn't installed on those systems in question. After "sudo apt-get -y install tk-dev" tv and toolbar are working! - boris From mk@REDACTED Sun Oct 17 21:09:39 2010 From: mk@REDACTED (Morten Krogh) Date: Sun, 17 Oct 2010 21:09:39 +0200 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <14a8a90f-2425-4e42-9519-849f0158ff22@h7g2000yqn.googlegroups.com> References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> <4CB6B589.9050003@cs.ntua.gr> <4CB6BDDC.3060900@amberbio.com> <4CBADE65.7010909@amberbio.com> <14a8a90f-2425-4e42-9519-849f0158ff22@h7g2000yqn.googlegroups.com> Message-ID: <4CBB49F3.8060809@amberbio.com> Hi Steve On 10/17/10 5:52 PM, Steve Davis wrote: > On Oct 17, 6:30 am, Morten Krogh wrote: >> Actually shared memory is a bad wording. It should be called "C like >> pointer access to the data". > That sounds even worse! Have we learned nothing from all that history > of thread-aches in mainstream computing? > The point here is that all languages including Erlang has shared memory with any reasonably general definition of shared memory. Erlangers often make the claim that erlang doesn't have shared memory. I can only understand that point if shared memory is defined rather strictly as "C like pointers to an address". How would you define it such that Erlang doesn't have it but other languages do. The problem is of course that a process like this loop(State) -> receive {From, get, X} -> From ! {result, get(X, State)}, loop(State) etc..... Behaves exactly like shared memory seen from other processes point of view. Writing Pid ! {self(), get, 27} receive {result, Result} -> Result end. is like writing get(X, Pid) in C (the language) where Pid is pointer to a shared data structure in C in C. All issues with locks, deadlocks etc are the same. Locks are needed when you want to send two messages to a process without any one else touching it in between, or when the state of two processes must be coordinated. A deadlock occurs, for instance, when two processes are waiting for each other. An Erlang process contains more data than a single memory address, which is an advantage but it is qualitatively the same, and you can never serialize all data in one process. >> The only real question is whether erlang should have a way for >> one process to read state of another... > I'm not sure you have fully thought through the implications of that > suggestion! ;) More concretely, consider the meaning of the java > keyword "volatile", what issue it points us to, and how that issue has > deep implications for the entire Java platform. > Of course, I would never have all processes introspect all other processes. The example at hand was a large shared data structure, where directly reading state would save all the boilerplate above, and give you faster performance. You would have to say, that the pople who impemented ETS haven't thought tthings through as well or what? Do you think that Erlang is a worse language because of ETS? I think it is better, and Erlang could have much more of that for my part. Best regards, Morten. From freza@REDACTED Sun Oct 17 23:46:35 2010 From: freza@REDACTED (Jachym Holecek) Date: Sun, 17 Oct 2010 22:46:35 +0100 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CBB49F3.8060809@amberbio.com> References: <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> <4CB6B589.9050003@cs.ntua.gr> <4CB6BDDC.3060900@amberbio.com> <4CBADE65.7010909@amberbio.com> <14a8a90f-2425-4e42-9519-849f0158ff22@h7g2000yqn.googlegroups.com> <4CBB49F3.8060809@amberbio.com> Message-ID: <20101017214635.GA1570@hanele.lan> # Morten Krogh 2010-10-17: > The problem is of course that a process like this > > loop(State) -> > receive > {From, get, X} -> > From ! {result, get(X, State)}, > loop(State) > > etc..... > > Behaves exactly like shared memory seen from other processes point of view. > > Writing > > Pid ! {self(), get, 27} > receive > {result, Result} -> > Result > end. > > is like writing > > get(X, Pid) in C (the language) where Pid is pointer to a shared data > structure in C. "Apples and oranges" as they say. In your Erlang example: * Server explicitely agrees to provide some of its data. * Server is free to change this decision. * Client has to follow certain protocol to access this data. * All accesses to "shared" data are serialized. * It is "physically impossible" to obtain inconsistent snaphost of the data. * It is "physically impossible" to bypass server's control. * This is guaranteed at language level. * Your example is implemented completely within this well-defined semantics. Contrast that with C: * All threads have equal access right to whole process address space. * Sychronization can be done, but is a mere convention (spinlocks/mutexes). * Data consistency can be done, but is a mere convetion (memory barriers). * At language level memory access semantics is more or less undefined. * Mutexes/memory barriers have to be done in assembly, ie. outside C. A little more fun on the C side: * Compiler may reorder loads/stores (somewhat) unless you're careful. * CPU may reorder loads/stores (somewhat) unless you're careful. * Cache may do whatever it wants to (somewhat) unless you're careful. Normally these thing will be taken care of by whatever threads implementation you use (in architecture- and compiler-specific way), but if we're talking at language level then there's no way (as far as I know) to achieve sane shared memory semantics in C alone (if we admit the existence of concurrent threads of execution which the language knows nothing about, of course). But sure -- you can _model_ Erlang-like processes in C, and you can _model_ C-like shared memory in Erlang (including all of the quirks above if you really wanted to). But at that point you're imposing correspondence of some sort by brute force; as opposed to capturing inherent properties. Just my two pence... Regards, -- Jachym From mscame@REDACTED Mon Oct 18 08:01:01 2010 From: mscame@REDACTED (William Yangzhuwei) Date: Mon, 18 Oct 2010 14:01:01 +0800 Subject: When will print "*** Terminating erlang" Message-ID: hi there, I can start one erlang program with ` start_test.sh `(start_test.sh is just a script to start erl and passing some args). When I use `start_test.sh &` to let this job run in background, it cannot finish executing. I get some output like (test1@REDACTED)1> *** Terminating erlang (test1@REDACTED). I guess *** Terminating erlang is printed by otp not in this erlang program. How can I trace this trouble and any logging? -- Best Regards //William YangZhuwei From lukas@REDACTED Mon Oct 18 09:50:19 2010 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 18 Oct 2010 09:50:19 +0200 Subject: [erlang-questions] When will print "*** Terminating erlang" In-Reply-To: References: Message-ID: <1287388219.4639.3.camel@ancalagon.du.uab.ericsson.se> Hello! You might want to try starting sasl at startup and see what it prints, i.e. erl -boot start_sasl [more args] Lukas On Mon, 2010-10-18 at 14:01 +0800, William Yangzhuwei wrote: > hi there, > > I can start one erlang program with ` start_test.sh `(start_test.sh is just > a script to start erl and passing some args). > > > When I use `start_test.sh &` to let this job run in background, it cannot > finish executing. > > I get some output like > > (test1@REDACTED)1> *** Terminating erlang (test1@REDACTED). > > I guess *** Terminating erlang is printed by otp not in this erlang program. > > How can I trace this trouble and any logging? > > From mk@REDACTED Mon Oct 18 10:31:00 2010 From: mk@REDACTED (Morten Krogh) Date: Mon, 18 Oct 2010 10:31:00 +0200 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <20101017214635.GA1570@hanele.lan> References: <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> <4CB6B589.9050003@cs.ntua.gr> <4CB6BDDC.3060900@amberbio.com> <4CBADE65.7010909@amberbio.com> <14a8a90f-2425-4e42-9519-849f0158ff22@h7g2000yqn.googlegroups.com> <4CBB49F3.8060809@amberbio.com> <20101017214635.GA1570@hanele.lan> Message-ID: <4CBC05C4.1090701@amberbio.com> Hi Jachym There is one thing that I probably haven't made clear enough, and it will answer all points basically. Erlang has the loop receive loop process that serializes all access to the data. The equivalent in C is a single byte. The semantics of communicating with a single byre is well defined in C. A C thread communicates with the byte using message passing through the memory bus. The byte only allows get and put. Nothing else makes sense. The general picture is that you have some units of atomicity and beyond that you need to lock, coordinate access etc. Erlang gives you a higher control of the granularity than C, but nothing is fundamentally different. I just object to all the Erlang statements about locks, concurrency etc. If you want to get rid of locks and shared memory, you lose concurrency. You can't have both. You can serialize everything. In C, that would be one thread. In Erlang, one process. If you want concurrency, you need smaller blocks of data, which then needs to be coordinated. You can imagine an axis concurrency ~ locks, deadlocks etc ~ high performance in one end. serialized access ~ no locks ~ single threaded ~ simple programing model in the other end. C actually beats Erlang in both ends of the axis (not the programming model, of course, Erlang syntax is always simpler). Erlang's sweet spot is in the middle. It is so easy to package an amount of data into a serialized agent with controller logic. The most concurrent program is not an erlang program, that would be a C program (or even lower) with a loop on each core, where all iterations of all loops updated the memory in all kinds of ways. Each iteration of each loop is a "context switch". Erlang is of course very convenient. You get a whole "block of bytes" with serialized access, and a programming language to control the "gate" to the block of bytes. The hardware manufacturers could make something like Erlang, if they wanted to. Chop the memory into blocks, and give each block serialized access and a small programmable controller sitting at the gate of that block. They would lose performance and concurrency if they did. It is all the same. The only question is at what granularity you serialize your data. Jachym, I will not go through your points. You can imagine my response. It is basically : One byte in C is the equivalent of one Erlang process. One byte doesn't need any controller logic besides get and put, and a guaranteed atomicity of get and put. Cheers, Morten On 10/17/10 11:46 PM, Jachym Holecek wrote: > # Morten Krogh 2010-10-17: >> The problem is of course that a process like this >> >> loop(State) -> >> receive >> {From, get, X} -> >> From ! {result, get(X, State)}, >> loop(State) >> >> etc..... >> >> Behaves exactly like shared memory seen from other processes point of view. >> >> Writing >> >> Pid ! {self(), get, 27} >> receive >> {result, Result} -> >> Result >> end. >> >> is like writing >> >> get(X, Pid) in C (the language) where Pid is pointer to a shared data >> structure in C. > "Apples and oranges" as they say. In your Erlang example: > > * Server explicitely agrees to provide some of its data. > * Server is free to change this decision. > * Client has to follow certain protocol to access this data. > * All accesses to "shared" data are serialized. > * It is "physically impossible" to obtain inconsistent snaphost of the data. > * It is "physically impossible" to bypass server's control. > * This is guaranteed at language level. > * Your example is implemented completely within this well-defined semantics. > > Contrast that with C: > > * All threads have equal access right to whole process address space. > * Sychronization can be done, but is a mere convention (spinlocks/mutexes). > * Data consistency can be done, but is a mere convetion (memory barriers). > * At language level memory access semantics is more or less undefined. > * Mutexes/memory barriers have to be done in assembly, ie. outside C. > > A little more fun on the C side: > > * Compiler may reorder loads/stores (somewhat) unless you're careful. > * CPU may reorder loads/stores (somewhat) unless you're careful. > * Cache may do whatever it wants to (somewhat) unless you're careful. > > Normally these thing will be taken care of by whatever threads implementation > you use (in architecture- and compiler-specific way), but if we're talking at > language level then there's no way (as far as I know) to achieve sane shared > memory semantics in C alone (if we admit the existence of concurrent threads > of execution which the language knows nothing about, of course). > > But sure -- you can _model_ Erlang-like processes in C, and you can _model_ > C-like shared memory in Erlang (including all of the quirks above if you > really wanted to). But at that point you're imposing correspondence of some > sort by brute force; as opposed to capturing inherent properties. > > Just my two pence... > > Regards, > -- Jachym > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ulf.wiger@REDACTED Mon Oct 18 11:29:28 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 18 Oct 2010 10:29:28 +0100 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CBC05C4.1090701@amberbio.com> References: <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> <4CB6B589.9050003@cs.ntua.gr> <4CB6BDDC.3060900@amberbio.com> <4CBADE65.7010909@amberbio.com> <14a8a90f-2425-4e42-9519-849f0158ff22@h7g2000yqn.googlegroups.com> <4CBB49F3.8060809@amberbio.com> <20101017214635.GA1570@hanele.lan> <4CBC05C4.1090701@amberbio.com> Message-ID: <7455541B-BBDF-41C9-8924-EF63ED1DAE37@erlang-solutions.com> Morten, With all due respect, your argument is one that I've heard many times in different settings, and it misses a key point (which Jachym brought up). My experience from working in development of systems with complex concurrency patterns is that this argument breaks down completely and horribly expensively over time in larger projects. Saying that there are no fundamental differences between Erlang and C essentially means that you claim that "correct by design" and "correct by convention" are the same thing. By this token, Erlang and Haskell are fundamentally the same too, since it really doesn't matter that the Haskell compiler can verify many more aspects of a program at compile-time than Erlang can - you can simply avoid making type mistakes in Erlang and be done with it! Your line of reasoning also suggests that we can put great faith in micro-benchmarks, since, if you can make it work and make it really fast in a small example in C, you can do it equally well in a larger and more complex example. The place where Erlang and C become very different is when you start writing code that has "interesting" complexity. Automatic memory management may seem like a complete waste, until your program becomes sufficiently complex that you can no longer easily keep track of where you should lock, allocate and free memory. At this point, it is actually very difficult to outperform a good garbage collector with hand- written code. I've seen lots of real-world examples where small teams of expert C++ programmers have managed to convince management to go with a well-trimmed, elegant execution model in C++, only to run in all sorts of problems when lesser programmers were to start adding features - huge dips in performance and stability, raving memory leaks, and frantic running around and fire fighting for the experts, who are the only ones who seem to manage to stick to the conventions and not break stuff. The beginning mantra, "how hard can it be?", is later answered with "apparently, endlessly hard". And we haven't even discussed the problem of one of these processes crashing yet. With two concurrent contexts explicitly writing to the same block of memory, ensuring the integrity of that memory block if one of the contexts throws an exception will be very, very hard. This is where Erlang's "share nothing" philosophy really pays off. The VM may decide to share data under the hood, but as a programmer, you can be absolutely certain that the death of another process - even in the same memory space - cannot ruin the integrity of your own data. It really is as if you had your own distinct copies of everything. The problem of crashing processes is a particularly nasty _containment_ problem, but when you scrutinise many of the concurrency models out there you soon find that there are usually many issues that are not well contained. Encapsulation and containment are best handled in the programming model, rather than relying on convention, and this is exactly the area where there is a fundamental difference. E.F. Codd, who wrote the twelve (13, really) rules for relational databases, had this rule as No 12: Rule 12: The nonsubversion rule: If the system provides a low-level (record-at-a-time) interface, then that interface cannot be used to subvert the system, for example, bypassing a relational security or integrity constraint. (http://en.wikipedia.org/wiki/Codd's_12_rules) The importance of this rule is often underestimated, and it is quite common to ditch it early in favour of performance, since the consequences of subverting the consistency model are not readily apparent until much later, if even then. BR, Ulf W On 18 Oct 2010, at 09:31, Morten Krogh wrote: > Hi Jachym > > There is one thing that I probably haven't made clear enough, and it will answer all points basically. > > Erlang has the loop receive loop process that serializes all access to the data. > The equivalent in C is a single byte. The semantics of communicating with a single byre is well defined in C. > A C thread communicates with the byte using message passing through the memory bus. The byte only allows get and put. > Nothing else makes sense. > > The general picture is that you have some units of atomicity and beyond that you need to lock, coordinate access etc. > > Erlang gives you a higher control of the granularity than C, but nothing is fundamentally different. I just object to all the Erlang statements about locks, concurrency etc. > > If you want to get rid of locks and shared memory, you lose concurrency. You can't have both. You can serialize everything. In C, that would be one thread. In Erlang, > one process. If you want concurrency, you need smaller blocks of data, which then needs to be coordinated. > > You can imagine an axis > > concurrency ~ locks, deadlocks etc ~ high performance in one end. > > serialized access ~ no locks ~ single threaded ~ simple programing model in the other end. > > > C actually beats Erlang in both ends of the axis (not the programming model, of course, Erlang syntax is always simpler). > > Erlang's sweet spot is in the middle. It is so easy to package an amount of data into a serialized agent with controller logic. > > The most concurrent program is not an erlang program, that would be a C program (or even lower) with a loop on each core, where all iterations of all loops updated the memory in all kinds of ways. Each iteration of each loop is a "context switch". > > Erlang is of course very convenient. You get a whole "block of bytes" with serialized access, and a programming language to control the "gate" to the block of bytes. > > The hardware manufacturers could make something like Erlang, if they wanted to. Chop the memory into blocks, and give each block serialized access and a small programmable controller sitting at the gate of that block. They would lose performance and concurrency if they did. > > It is all the same. The only question is at what granularity you serialize your data. > > Jachym, I will not go through your points. You can imagine my response. It is basically : > > One byte in C is the equivalent of one Erlang process. One byte doesn't need any controller logic besides get and put, and a guaranteed atomicity of get and put. > > Cheers, > > Morten > > > > On 10/17/10 11:46 PM, Jachym Holecek wrote: >> # Morten Krogh 2010-10-17: >>> The problem is of course that a process like this >>> >>> loop(State) -> >>> receive >>> {From, get, X} -> >>> From ! {result, get(X, State)}, >>> loop(State) >>> >>> etc..... >>> >>> Behaves exactly like shared memory seen from other processes point of view. >>> >>> Writing >>> >>> Pid ! {self(), get, 27} >>> receive >>> {result, Result} -> >>> Result >>> end. >>> >>> is like writing >>> >>> get(X, Pid) in C (the language) where Pid is pointer to a shared data >>> structure in C. >> "Apples and oranges" as they say. In your Erlang example: >> >> * Server explicitely agrees to provide some of its data. >> * Server is free to change this decision. >> * Client has to follow certain protocol to access this data. >> * All accesses to "shared" data are serialized. >> * It is "physically impossible" to obtain inconsistent snaphost of the data. >> * It is "physically impossible" to bypass server's control. >> * This is guaranteed at language level. >> * Your example is implemented completely within this well-defined semantics. >> >> Contrast that with C: >> >> * All threads have equal access right to whole process address space. >> * Sychronization can be done, but is a mere convention (spinlocks/mutexes). >> * Data consistency can be done, but is a mere convetion (memory barriers). >> * At language level memory access semantics is more or less undefined. >> * Mutexes/memory barriers have to be done in assembly, ie. outside C. >> >> A little more fun on the C side: >> >> * Compiler may reorder loads/stores (somewhat) unless you're careful. >> * CPU may reorder loads/stores (somewhat) unless you're careful. >> * Cache may do whatever it wants to (somewhat) unless you're careful. >> >> Normally these thing will be taken care of by whatever threads implementation >> you use (in architecture- and compiler-specific way), but if we're talking at >> language level then there's no way (as far as I know) to achieve sane shared >> memory semantics in C alone (if we admit the existence of concurrent threads >> of execution which the language knows nothing about, of course). >> >> But sure -- you can _model_ Erlang-like processes in C, and you can _model_ >> C-like shared memory in Erlang (including all of the quirks above if you >> really wanted to). But at that point you're imposing correspondence of some >> sort by brute force; as opposed to capturing inherent properties. >> >> Just my two pence... >> >> Regards, >> -- Jachym >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From vincent.dephily@REDACTED Mon Oct 18 11:48:43 2010 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Mon, 18 Oct 2010 11:48:43 +0200 Subject: [erlang-questions] Dyalizer warnings for too wide return type In-Reply-To: <4CBB204F.4000404@cs.ntua.gr> References: <201010151145.03649.vincent.dephily@mobile-devices.fr> <4CBB204F.4000404@cs.ntua.gr> Message-ID: <201010181148.43453.vincent.dephily@mobile-devices.fr> On Sunday 17 October 2010 18:11:59 Kostis Sagonas wrote: > Vincent de Phily wrote: > I could have written a much longer reply in this thread, but I'll keep > this reply a bit short with the risk of sounding a bit impolite. This is > not my intention. Not at all, it's good to get an informed reply. > The whole discussion above is a bit naive, but perhaps this is primarily > my fault because there is no document that explains dialyzer's decisions > and philosophy in general terms. For a long time now I wanted to write a > "dialyzer FAQ". I hope I can come around to that pretty soon. That'd be nice. > Anyway, the short answer is that dialyzer will never do what users may > initially think it would be better if it did. The only information that > dialyzer has to work with is the code and whenever it finds something > that is a discrepancy between what the code does and what the programmer > claims it does (e.g. in a spec) it spits a warning about it. Dialyzer > cannot be in the minds of programmers and what's their ultimate > intention even if they add "%% TODO implement later" comments :-) > > So, my reply to Alexey's message is that in a situation like the one he > describes the proper action is to either comment out the true case in > foo's code or realize that now is the time for this "later". I never said (and neither did the OP I think) that dialyzer should ignore the spec/code discrepancy inside a module. For the obvious reason, as you say, that dialyzer can't read minds. What I thought (and it's quite possibly naive, but I'm not convinced yet) is that callers of the function could sometimes use the spec instead of the code to infer what the function returns. However, defining that "sometimes" may be tricky, a pandora's box we don't want to open. > Now, having written all these, there is a dialyzer option that we have > primarily added for internal debugging purposes, but may come in handy > in situations like that. If you want dialyzer to do its analysis based > on success typings only and ignore the specs of the analyzed modules, > you can use the option '--no_spec'. > > But there is no option "trust the specs, please" for cases where > dialyzer can determine that they do not correspond to the code. > Whenever you want to do that, simply don't use dialyzer! But I do want to use dialyzer, I like it (insert kiddy tantrum here). And I'm happy to get a dializer warning for that function, I just don't want the warning to propagate. If the warning is justified I'll correct it. If it isn't justified, I'll have less dialyzer noise to check and dismiss. > > I'd like dializer to trust me when I declare my interface, even if the > > implementation doesn't match that. Actually, maybe we could make the case > > that non-exported functions must have an exact spec whereas exported > > function can have a spec that is more allowing than the implementation ? > > Sorry, but I firmly believe that this last part is wrong. Having > published interfaces that do not correspond to the implementation is not > a good idea. Among other things, what sort of confidence do you have > that the client code works ok? How do you test things in this situation? I'll stand ground here, because it is a frequent and normal usecase (no buggy or unfinished code involved). Whenever you use pluggable implementations you're confronted with the "implementation does less that the interface states" problem. How many gen_server handle_call/3 did you write that could return all 8 possible Result types mentioned in the erlang docs ? Even for my own interfaces (of which I have tighter control than the standard OTP behaviours) I have that problem, because one module can return more types of results than another, but the declared interface is the same for both modules. Unless you say that specs should always follow the code exactly, but I think that misses an opportunity to make them more usefull. If the spec is just mirroring what dializer can infer automatically anyway, why do I keep writing them by hand (or at all) ? -- Vincent de Phily Mobile Devices +33 (0) 142 119 325 +353 (0) 85 710 6320 Warning This message (and any associated files) is intended only for the use of its intended recipient and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Any views or opinions presented are solely those of the author vincent.dephily@REDACTED and do not necessarily represent those of the company. Although the company has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. From ARoe@REDACTED Mon Oct 18 13:24:45 2010 From: ARoe@REDACTED (Adrian Roe) Date: Mon, 18 Oct 2010 04:24:45 -0700 Subject: Table viewer disappearing Message-ID: <0697C1D63C68ED4DA4EE16BFBB156ACD02AF0C29E0@IE2RD2XVS071.red002.local> Apologies for a potentially environment related question, but I have been happily developing an mnesia database subsystem on an Amazon EC2 instance running Ubuntu 10.10. As part of this I have used table viewer (tv:start()) and it has worked just fine (using NX/Nomachine & GNOME). No idea what has changed, but now when I do tv:start(), I get a PID returned but no window displayed. I've even reverted to machine images from when it was working but still no window spawned. Things like wx:demo() launch just fine. The docs around tv only talk about its start method and tv doesn?t appear to respond any messages sent to it via its PID. Has anyone else seen this / know what on earth might be going on. Thanks for any suggestions Adrian From mk@REDACTED Mon Oct 18 13:28:45 2010 From: mk@REDACTED (Morten Krogh) Date: Mon, 18 Oct 2010 13:28:45 +0200 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <7455541B-BBDF-41C9-8924-EF63ED1DAE37@erlang-solutions.com> References: <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> <4CB6B589.9050003@cs.ntua.gr> <4CB6BDDC.3060900@amberbio.com> <4CBADE65.7010909@amberbio.com> <14a8a90f-2425-4e42-9519-849f0158ff22@h7g2000yqn.googlegroups.com> <4CBB49F3.8060809@amberbio.com> <20101017214635.GA1570@hanele.lan> <4CBC05C4.1090701@amberbio.com> <7455541B-BBDF-41C9-8924-EF63ED1DAE37@erlang-solutions.com> Message-ID: <4CBC2F6D.1060509@amberbio.com> Ulf, I actually agree with most of what you say, and most of it is not at all in disagreement with my previous posts. I was talking about issues like concurrency, message passing, locks from a philosophical point of view, and I don't agree with the way Erlang is often presented as a way to have high concurrency without locks. I don't agree with the statement that shared memory is bad. You are talking about the Erlang platform, and its productivity. I like Erlang. I find it very easy to write programs, and have the processes talk to each other, monitor them etc. My philosophical rants were not against the language or platform. > > With all due respect, your argument is one that I've heard many times > in different settings, and it misses a key point (which Jachym brought > up). > My experience from working in development of systems with complex > concurrency patterns is that this argument breaks down completely > and horribly expensively over time in larger projects. > The argument breaks down? You mean that it becomes hard to write and maintain code? But I haven't made claims about that. I find Erlang programs much easier than C programs. > Saying that there are no fundamental differences between Erlang and C > essentially means that you claim that "correct by design" and "correct by > convention" are the same thing. By this token, Erlang and Haskell are > fundamentally the same too, since it really doesn't matter that the > Haskell > compiler can verify many more aspects of a program at compile-time than > Erlang can - you can simply avoid making type mistakes in Erlang and > be done with it! There are no fundamental difference between Erlang and C, when it comes to the concepts of concurrency, locks, deadlocks, shared memory. The only qualitative difference is between a sequential program and everything else. There are huge differences in actually implementing things, and the syntax, of course. > Your line of reasoning also suggests that we can put great faith in > micro-benchmarks, since, if you can make it work and make it really > fast in a small example in C, you can do it equally well in a larger and > more complex example. > I believe in modularity and simplicity. You need to choose the right level to work in. Erlang is just a layer on top of C. For many purposes, it is better to use Erlang than C. For other purposes, it is better to use electric currents in nand gates. Of course, you need higher level apis. Ulf, I agree totally with you. > The place where Erlang and C become very different is when you start > writing code that has "interesting" complexity. Automatic memory > management may seem like a complete waste, until your program > becomes sufficiently complex that you can no longer easily keep track > of where you should lock, allocate and free memory. At this point, it is > actually very difficult to outperform a good garbage collector with hand- > written code. > I agree. And you see what the next higher level will be. Automatic process collection, when a process can never be reached anymore. In 20 years, people will have "gigaprocesses", the older folks will remember that there was something called "gigabytes", and the next languages will have automatic process spawning and process collection. Erlang will be a low level language. What will such a unit of processes be called? A brain :) The brain languages will still have deadlocks, shared memory, locks etc, because those are philosophical/mathematical entities that are never "solved". > I've seen lots of real-world examples where > small teams of expert C++ programmers have managed to convince > management to go with a well-trimmed, elegant execution model in > C++, only to run in all sorts of problems when lesser programmers were > to start adding features - huge dips in performance and stability, raving > memory leaks, and frantic running around and fire fighting for the > experts, who are the only ones who seem to manage to stick to the > conventions and not break stuff. The beginning mantra, "how hard can > it be?", is later answered with "apparently, endlessly hard". > Interesting. By the way, are there examples of Erlang teams that have switched away from Erlang? > And we haven't even discussed the problem of one of these processes > crashing yet. With two concurrent contexts explicitly writing to the same > block of memory, ensuring the integrity of that memory block if one of > the contexts throws an exception will be very, very hard. This is where > Erlang's "share nothing" philosophy really pays off. The VM may decide > to share data under the hood, but as a programmer, you can be absolutely > certain that the death of another process - even in the same memory space > - cannot ruin the integrity of your own data. It really is as if you > had your own > distinct copies of everything. > This is where we disagree strongly. I am saying that you get the same problems again just at a higher level. The equivalent situation is this. P1 is a process, that keeps data, i.e. the so called shared memory. P2 and P3 sends messages to P1, many each. Remember a put message is like a mutation. If you want to serialize the updates from P2 and P3, a lock is needed. P1 for instance could wait for all messages from P2, before receiving from P3, incorporate them, release its lock and accept the messages from P3. What should P1 do ,if the messages from P2 stop early. Roll back, ignore it. The issues are exactly the same as at the lower byte level. Don't you see that these are deep principles, not something a language/VM can solve? Memory space in Erlang is not the same as memory space for C. Everything is lifted one level. The equivalent of memory space in Erlang is the set of running processes. The addresses are now Pids not RAM addresses. And if you want something interesting to happen, you must have some processes that accept messages and act on them. C address space does not exist for the Erlang programmer as little as physical positions of transistors exist for the C program. A C program does not crash if a transistor goes bad if there is a "virtualization layer" in between that redirects the current to a another transistor. > The problem of crashing processes is a particularly nasty _containment_ > problem, but when you scrutinise many of the concurrency models out > there you soon find that there are usually many issues that are not well > contained. Encapsulation and containment are best handled in the > programming model, rather than relying on convention, and this is > exactly the area where there is a fundamental difference. > Ulf, I agree that one should have higher level languages that encapsulate things, and maybe JAVA or C++ or whatever you are thinking about are too low level for big projects. Fine with me. On practical terms, I feel that Erlang some times gives up performance for unnecessary reasons. I am especially thinking about mutability. There is no reason to give up mutation within a process which is already a serializer. code like this loop(Dict) receive {put, K, V} -> Dict2 = dict:store(K, V, Dict), loop(Dict2) end. is just a performance wise inefficient way of doing loop(Mut) -> receive {put, K, V} -> put(K,V, Mut), loop(Mut) end. where Mut is a mutable data structure. In the dict exampe, there is a unncecessary work and garbage collection for the functional data structure Dict. And you gain nothing, since it is sequential code. ETS gives you an unnecessary copy. The process dictionary does the job, but there is only one of them and it must be a hash table. It would be better with PD1 = pd:new_hash_table(), PD2 = pd:new_tree(), etc. > E.F. Codd, who wrote the twelve (13, really) rules for relational > databases, had > this rule as No 12: > > Rule 12: The nonsubversion rule: > *If the system provides a low-level (record-at-a-time) interface, then > that interface * > *cannot be used to subvert the system, for example, bypassing a > relational security * > *or integrity constraint.* > (http://en.wikipedia.org/wiki/Codd's_12_rules > ) > > The importance of this rule is often underestimated, and it is quite > common to > ditch it early in favour of performance, since the consequences of > subverting > the consistency model are not readily apparent until much later, if > even then. > Interesting. I am not convinced. So basically, the programmer should be constrained, because he/she will make mistakes? I guess it is a question of finding the right balance. So, you think dirty_write was a mistake for mnesia? cheers, Morten. > BR, > Ulf W > > On 18 Oct 2010, at 09:31, Morten Krogh wrote: > >> Hi Jachym >> >> There is one thing that I probably haven't made clear enough, and it >> will answer all points basically. >> >> Erlang has the loop receive loop process that serializes all access >> to the data. >> The equivalent in C is a single byte. The semantics of communicating >> with a single byre is well defined in C. >> A C thread communicates with the byte using message passing through >> the memory bus. The byte only allows get and put. >> Nothing else makes sense. >> >> The general picture is that you have some units of atomicity and >> beyond that you need to lock, coordinate access etc. >> >> Erlang gives you a higher control of the granularity than C, but >> nothing is fundamentally different. I just object to all the Erlang >> statements about locks, concurrency etc. >> >> If you want to get rid of locks and shared memory, you lose >> concurrency. You can't have both. You can serialize everything. In C, >> that would be one thread. In Erlang, >> one process. If you want concurrency, you need smaller blocks of >> data, which then needs to be coordinated. >> >> You can imagine an axis >> >> concurrency ~ locks, deadlocks etc ~ high performance in one end. >> >> serialized access ~ no locks ~ single threaded ~ simple programing >> model in the other end. >> >> >> C actually beats Erlang in both ends of the axis (not the programming >> model, of course, Erlang syntax is always simpler). >> >> Erlang's sweet spot is in the middle. It is so easy to package an >> amount of data into a serialized agent with controller logic. >> >> The most concurrent program is not an erlang program, that would be a >> C program (or even lower) with a loop on each core, where all >> iterations of all loops updated the memory in all kinds of ways. Each >> iteration of each loop is a "context switch". >> >> Erlang is of course very convenient. You get a whole "block of bytes" >> with serialized access, and a programming language to control the >> "gate" to the block of bytes. >> >> The hardware manufacturers could make something like Erlang, if they >> wanted to. Chop the memory into blocks, and give each block >> serialized access and a small programmable controller sitting at the >> gate of that block. They would lose performance and concurrency if >> they did. >> >> It is all the same. The only question is at what granularity you >> serialize your data. >> >> Jachym, I will not go through your points. You can imagine my >> response. It is basically : >> >> One byte in C is the equivalent of one Erlang process. One byte >> doesn't need any controller logic besides get and put, and a >> guaranteed atomicity of get and put. >> >> Cheers, >> >> Morten >> >> >> >> On 10/17/10 11:46 PM, Jachym Holecek wrote: >>> # Morten Krogh 2010-10-17: >>>> The problem is of course that a process like this >>>> >>>> loop(State) -> >>>> receive >>>> {From, get, X} -> >>>> From ! {result, get(X, State)}, >>>> loop(State) >>>> >>>> etc..... >>>> >>>> Behaves exactly like shared memory seen from other processes point >>>> of view. >>>> >>>> Writing >>>> >>>> Pid ! {self(), get, 27} >>>> receive >>>> {result, Result} -> >>>> Result >>>> end. >>>> >>>> is like writing >>>> >>>> get(X, Pid) in C (the language) where Pid is pointer to a shared data >>>> structure in C. >>> "Apples and oranges" as they say. In your Erlang example: >>> >>> * Server explicitely agrees to provide some of its data. >>> * Server is free to change this decision. >>> * Client has to follow certain protocol to access this data. >>> * All accesses to "shared" data are serialized. >>> * It is "physically impossible" to obtain inconsistent snaphost of >>> the data. >>> * It is "physically impossible" to bypass server's control. >>> * This is guaranteed at language level. >>> * Your example is implemented completely within this well-defined >>> semantics. >>> >>> Contrast that with C: >>> >>> * All threads have equal access right to whole process address space. >>> * Sychronization can be done, but is a mere convention >>> (spinlocks/mutexes). >>> * Data consistency can be done, but is a mere convetion (memory >>> barriers). >>> * At language level memory access semantics is more or less undefined. >>> * Mutexes/memory barriers have to be done in assembly, ie. outside C. >>> >>> A little more fun on the C side: >>> >>> * Compiler may reorder loads/stores (somewhat) unless you're careful. >>> * CPU may reorder loads/stores (somewhat) unless you're careful. >>> * Cache may do whatever it wants to (somewhat) unless you're careful. >>> >>> Normally these thing will be taken care of by whatever threads >>> implementation >>> you use (in architecture- and compiler-specific way), but if we're >>> talking at >>> language level then there's no way (as far as I know) to achieve >>> sane shared >>> memory semantics in C alone (if we admit the existence of concurrent >>> threads >>> of execution which the language knows nothing about, of course). >>> >>> But sure -- you can _model_ Erlang-like processes in C, and you can >>> _model_ >>> C-like shared memory in Erlang (including all of the quirks above if you >>> really wanted to). But at that point you're imposing correspondence >>> of some >>> sort by brute force; as opposed to capturing inherent properties. >>> >>> Just my two pence... >>> >>> Regards, >>> -- Jachym >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > From ulf.wiger@REDACTED Mon Oct 18 15:18:00 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 18 Oct 2010 14:18:00 +0100 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CBC2F6D.1060509@amberbio.com> References: <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> <4CB6B589.9050003@cs.ntua.gr> <4CB6BDDC.3060900@amberbio.com> <4CBADE65.7010909@amberbio.com> <14a8a90f-2425-4e42-9519-849f0158ff22@h7g2000yqn.googlegroups.com> <4CBB49F3.8060809@amberbio.com> <20101017214635.GA1570@hanele.lan> <4CBC05C4.1090701@amberbio.com> <7455541B-BBDF-41C9-8924-EF63ED1DAE37@erlang-solutions.com> <4CBC2F6D.1060509@amberbio.com> Message-ID: On 18 Oct 2010, at 12:28, Morten Krogh wrote: >> And we haven't even discussed the problem of one of these processes >> crashing yet. With two concurrent contexts explicitly writing to the same >> block of memory, ensuring the integrity of that memory block if one of >> the contexts throws an exception will be very, very hard. This is where >> Erlang's "share nothing" philosophy really pays off. The VM may decide >> to share data under the hood, but as a programmer, you can be absolutely >> certain that the death of another process - even in the same memory space >> - cannot ruin the integrity of your own data. It really is as if you had your own >> distinct copies of everything. >> > This is where we disagree strongly. I am saying that you get the same problems again just at a higher level. Your claim was that there are no fundamental differences between Erlang and C in regard to shared memory. If you care to limit that claim to a particular set of problems that can arise in both environments, it's entirely uncontroversial. Otherwise, it becomes difficult to discuss the matter. Fundamentally, I'd say that Erlang is more similar to C programs running in separate OS processes, not sharing memory. Many of the problems you bring up can happen there too. What cannot happen is that one process directly modifies and corrupts the memory of another process. Also, even in that case, the two processes will be sharing physical memory at some level, but what is important is that the _user-level program_ cannot exploit this. This (fundamental) difference typically has the practical consequence that if you suffer a crash in a multithreaded shared-memory environment, all bets are off, whereas if you have memory protection boundaries between the processes, you may be able to recover. Needless to say, this is often not enough, and there may be inconsistencies and deadlocks at higher levels. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From vincent.dephily@REDACTED Mon Oct 18 15:29:34 2010 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Mon, 18 Oct 2010 15:29:34 +0200 Subject: Using bound variables in a fun's function clause Message-ID: <201010181529.34357.vincent.dephily@mobile-devices.fr> Hi all, consider the following snipnet : > #!/usr/bin/escript > main([Arg1, Arg2]) -> > F = fun(Arg1) -> match; > (_) -> nomatch > end, > io:format("~p~n", [F(Arg2)]). Compiling this (R13B04) will complain about ./test/funbind.es:3: Warning: variable 'Arg1' is unused ./test/funbind.es:4: Warning: variable 'Arg1' is unused ./test/funbind.es:4: Warning: variable 'Arg1' shadowed in 'fun' and create a fun where Arg1 is unbound. If I change the implementation to : > F = fun(Arg1_tmp) when Arg1_tmp == Arg1 -> match; > (_) -> nomatch > end, or even : > F = fun(Foo) -> > case Foo of > Arg1 -> match; > _ -> nomatch > end > end, I'll get the behavior I want (creating the fun with Arg1 bound) without any compile warning. Is this the expected behaviour ? If so, what's the rationale ? It's easy to work around, but it is quite surprising. -- Vincent de Phily Mobile Devices +33 (0) 142 119 325 +353 (0) 85 710 6320 Warning This message (and any associated files) is intended only for the use of its intended recipient and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Any views or opinions presented are solely those of the author vincent.dephily@REDACTED and do not necessarily represent those of the company. Although the company has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. From kenneth.lundin@REDACTED Mon Oct 18 15:30:45 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 18 Oct 2010 15:30:45 +0200 Subject: [erlang-questions] Table viewer disappearing In-Reply-To: <0697C1D63C68ED4DA4EE16BFBB156ACD02AF0C29E0@IE2RD2XVS071.red002.local> References: <0697C1D63C68ED4DA4EE16BFBB156ACD02AF0C29E0@IE2RD2XVS071.red002.local> Message-ID: Hi, tv is a gs based application and thus depends on the gs application which in turn depends on that you have tcl/tk installed. Try gs:start() in the Erlang shell, what happens? /Kenneth Erlang/OTP , Ericsson On Mon, Oct 18, 2010 at 1:24 PM, Adrian Roe wrote: > Apologies for a potentially environment related question, but I have been happily developing an mnesia database subsystem on an Amazon EC2 instance running Ubuntu 10.10. ?As part of this I have used table viewer (tv:start()) and it has worked just fine (using NX/Nomachine & GNOME). ?No idea what has changed, but now when I do tv:start(), I get a PID returned but no window displayed. ?I've even reverted to machine images from when it was working but still no window spawned. Things like wx:demo() launch just fine. > > The docs around tv only talk about its start method and tv doesn?t appear to respond any messages sent to it via its PID. ?Has anyone else seen this / know what on earth might be going on. > > Thanks for any suggestions > > Adrian > > From spawn.think@REDACTED Mon Oct 18 15:38:38 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Mon, 18 Oct 2010 15:38:38 +0200 Subject: [erlang-questions] Using bound variables in a fun's function clause In-Reply-To: <201010181529.34357.vincent.dephily@mobile-devices.fr> References: <201010181529.34357.vincent.dephily@mobile-devices.fr> Message-ID: Hi Vincent, Yes it's expected. It's about the variables scope You can read more here: http://www.erlang.org/doc/programming_examples/funs.html#id58554 On Mon, Oct 18, 2010 at 3:29 PM, Vincent de Phily < vincent.dephily@REDACTED> wrote: > Hi all, > > consider the following snipnet : > > > #!/usr/bin/escript > > main([Arg1, Arg2]) -> > > F = fun(Arg1) -> match; > > (_) -> nomatch > > end, > > io:format("~p~n", [F(Arg2)]). > > Compiling this (R13B04) will complain about > ./test/funbind.es:3: Warning: variable 'Arg1' is unused > ./test/funbind.es:4: Warning: variable 'Arg1' is unused > ./test/funbind.es:4: Warning: variable 'Arg1' shadowed in 'fun' > and create a fun where Arg1 is unbound. > > If I change the implementation to : > > F = fun(Arg1_tmp) when Arg1_tmp == Arg1 -> match; > > (_) -> nomatch > > end, > or even : > > F = fun(Foo) -> > > case Foo of > > Arg1 -> match; > > _ -> nomatch > > end > > end, > I'll get the behavior I want (creating the fun with Arg1 bound) without any > compile warning. > > > Is this the expected behaviour ? If so, what's the rationale ? It's easy to > work around, but it is quite surprising. > > -- > Vincent de Phily > Mobile Devices > +33 (0) 142 119 325 > +353 (0) 85 710 6320 > > Warning > This message (and any associated files) is intended only for the use of its > intended recipient and may contain information that is confidential, > subject > to copyright or constitutes a trade secret. If you are not the intended > recipient you are hereby notified that any dissemination, copying or > distribution of this message, or files associated with this message, is > strictly prohibited. If you have received this message in error, please > notify us immediately by replying to the message and deleting it from your > computer. Any views or opinions presented are solely those of the author > vincent.dephily@REDACTED and do not necessarily represent those > of > the > company. Although the company has taken reasonable precautions to ensure no > viruses are present in this email, the company cannot accept responsibility > for any loss or damage arising from the use of this email or attachments. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From vincent.dephily@REDACTED Mon Oct 18 15:50:21 2010 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Mon, 18 Oct 2010 15:50:21 +0200 Subject: [erlang-questions] Using bound variables in a fun's function clause In-Reply-To: References: <201010181529.34357.vincent.dephily@mobile-devices.fr> Message-ID: <201010181550.21455.vincent.dephily@mobile-devices.fr> On Monday 18 October 2010 15:38:38 Ahmed Omar wrote: > Hi Vincent, > Yes it's expected. It's about the variables scope > You can read more here: > http://www.erlang.org/doc/programming_examples/funs.html#id58554 Ah, I really should have done my homework better and RTFM (I'll blame tiredness). Thanks for the quick reply. -- Vincent de Phily Mobile Devices +33 (0) 142 119 325 +353 (0) 85 710 6320 Warning This message (and any associated files) is intended only for the use of its intended recipient and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Any views or opinions presented are solely those of the author vincent.dephily@REDACTED and do not necessarily represent those of the company. Although the company has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. From robert.virding@REDACTED Mon Oct 18 15:52:12 2010 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 18 Oct 2010 13:52:12 +0000 (GMT) Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <1804302357.107691287408975779.JavaMail.root@zimbra> Message-ID: <1311495418.107811287409932041.JavaMail.root@zimbra> I am bit late getting into this discussion but I have been away without good network connectivity. Of course you cannot have any form of useful concurrency without locks, share memory, etc at some level. There is however, one fundamental difference between doing it in C or in Erlang and that is *WHO* actually does all the locking, sharing, etc and *WHO* actually enforces the allowed access between processes. In C it is the user, either directly or through libraries, but it is only enforced by convention, there is nothing really *stopping* me from writing to any address I wish. In Erlang all locking, sharing etc is handled by the implementation and strictly enforced by having message passing as the only way of process communication for users. This is probably easier to implement as well, but it is by no means trivial to get it correct and efficient, as there is only a limited number of ways processes can communicate with each other. This is also a very powerful argument for immutable data. It is, of course, possible, and safe, to have mutable data and enforce message passing by copying (this is almost swearing in the church of Erlang) but it definitely limits the implementation. Data *must* be copied as it is mutable. With immutable data the implementation can itself decide how messages are actually "sent" between processes: either by copying as is done in the BEAM; or by having shared memory between processes; or by a combination of both. The user will not have to worry about it and not see the difference, except perhaps by performance. This also means that implementing a suggestion from earlier in this thread about adding threads becomes easier to implement and transparent to the user, just because data is immutable. It can in one sense be taken as a hint to the system as to how these processes are intended to interact with each other. With mutable data this is impossible. One quick comment on the original question about shared/hybrid heaps. Unfortunately, there is no "best" way of doing heaps: some applications will work better with shared heaps; some will work better with copying; and for many the difference will be negligible. For example having shared large binaries works well for some aplications but can cause others to crash. Also SMP's and multiple cores greatly increase the difficulty of shared heap implementation because of just problems with locking and synchronising memory access. Robert ----- "Morten Krogh" wrote: > Ulf, > > I actually agree with most of what you say, and most of it is not at > all > in disagreement with my previous posts. > > I was talking about issues like concurrency, message passing, locks > from > a philosophical point of view, and > I don't agree with the way Erlang is often presented as a way to have > > high concurrency without locks. > I don't agree with the statement that shared memory is bad. > > You are talking about the Erlang platform, and its productivity. I > like > Erlang. I find it very easy to write programs, and > have the processes talk to each other, monitor them etc. My > philosophical rants were not against the language or platform. > > > > > With all due respect, your argument is one that I've heard many > times > > in different settings, and it misses a key point (which Jachym > brought > > up). > > My experience from working in development of systems with complex > > concurrency patterns is that this argument breaks down completely > > and horribly expensively over time in larger projects. > > > > The argument breaks down? You mean that it becomes hard to write and > maintain code? But I > haven't made claims about that. I find Erlang programs much easier > than > C programs. > > > > Saying that there are no fundamental differences between Erlang and > C > > essentially means that you claim that "correct by design" and > "correct by > > convention" are the same thing. By this token, Erlang and Haskell > are > > fundamentally the same too, since it really doesn't matter that the > > > Haskell > > compiler can verify many more aspects of a program at compile-time > than > > Erlang can - you can simply avoid making type mistakes in Erlang > and > > be done with it! > There are no fundamental difference between Erlang and C, when it > comes to > the concepts of concurrency, locks, deadlocks, shared memory. > The only qualitative difference is between a sequential program and > everything else. > > There are huge differences in actually implementing things, and the > syntax, of course. > > > Your line of reasoning also suggests that we can put great faith in > > micro-benchmarks, since, if you can make it work and make it really > > fast in a small example in C, you can do it equally well in a larger > and > > more complex example. > > > I believe in modularity and simplicity. You need to choose the right > level to work in. > Erlang is just a layer on top of C. For many purposes, it is better to > > use Erlang than C. > For other purposes, it is better to use electric currents in nand > gates. > Of course, you need higher level apis. > Ulf, I agree totally with you. > > > The place where Erlang and C become very different is when you > start > > writing code that has "interesting" complexity. Automatic memory > > management may seem like a complete waste, until your program > > becomes sufficiently complex that you can no longer easily keep > track > > of where you should lock, allocate and free memory. At this point, > it is > > actually very difficult to outperform a good garbage collector with > hand- > > written code. > > > I agree. > > And you see what the next higher level will be. Automatic process > collection, when a process can never be reached anymore. > In 20 years, people will have "gigaprocesses", the older folks will > remember that there was something called "gigabytes", and the next > languages will have automatic > process spawning and process collection. Erlang will be a low level > language. What will such a unit of processes be called? A brain :) > > The brain languages will still have deadlocks, shared memory, locks > etc, > because those are philosophical/mathematical entities that are never > "solved". > > > > > > > I've seen lots of real-world examples where > > small teams of expert C++ programmers have managed to convince > > management to go with a well-trimmed, elegant execution model in > > C++, only to run in all sorts of problems when lesser programmers > were > > to start adding features - huge dips in performance and stability, > raving > > memory leaks, and frantic running around and fire fighting for the > > experts, who are the only ones who seem to manage to stick to the > > conventions and not break stuff. The beginning mantra, "how hard > can > > it be?", is later answered with "apparently, endlessly hard". > > > > Interesting. By the way, are there examples of Erlang teams that have > > switched away from Erlang? > > > And we haven't even discussed the problem of one of these processes > > crashing yet. With two concurrent contexts explicitly writing to the > same > > block of memory, ensuring the integrity of that memory block if one > of > > the contexts throws an exception will be very, very hard. This is > where > > Erlang's "share nothing" philosophy really pays off. The VM may > decide > > to share data under the hood, but as a programmer, you can be > absolutely > > certain that the death of another process - even in the same memory > space > > - cannot ruin the integrity of your own data. It really is as if you > > > had your own > > distinct copies of everything. > > > This is where we disagree strongly. I am saying that you get the same > > problems again just at a higher level. > The equivalent situation is this. > P1 is a process, that keeps data, i.e. the so called shared memory. > P2 and P3 sends messages to P1, many each. Remember a put message is > like a mutation. > If you want to serialize the updates from P2 and P3, a lock is needed. > > P1 for instance could wait for all messages from P2, before receiving > > from P3, > incorporate them, release its lock and accept the messages from P3. > What > should P1 do ,if the messages from P2 stop early. Roll back, ignore > it. > The issues are exactly the same as at the lower byte level. Don't you > > see that these are deep principles, not something a language/VM can > solve? > > Memory space in Erlang is not the same as memory space for C. > Everything > is lifted one level. The equivalent of memory space in Erlang is the > set > of running processes. > The addresses are now Pids not RAM addresses. And if you want > something > interesting to happen, you must have some processes that accept > messages > and act on them. > C address space does not exist for the Erlang programmer as little as > > physical positions of transistors exist for the C program. A C > program > does not crash if a transistor goes bad if > there is a "virtualization layer" in between that redirects the > current > to a another transistor. > > > > The problem of crashing processes is a particularly nasty > _containment_ > > problem, but when you scrutinise many of the concurrency models out > > there you soon find that there are usually many issues that are not > well > > contained. Encapsulation and containment are best handled in the > > programming model, rather than relying on convention, and this is > > exactly the area where there is a fundamental difference. > > > Ulf, I agree that one should have higher level languages that > encapsulate things, and maybe JAVA or C++ or whatever you are thinking > > about are too low level > for big projects. Fine with me. > > On practical terms, I feel that Erlang some times gives up performance > > for unnecessary reasons. I am especially thinking about mutability. > There is no reason to give up mutation within a process which is > already > a serializer. > > code like this > > loop(Dict) > receive > {put, K, V} -> > Dict2 = dict:store(K, V, Dict), > loop(Dict2) > end. > > > is just a performance wise inefficient way of doing > > loop(Mut) -> > receive > {put, K, V} -> > put(K,V, Mut), > loop(Mut) > end. > > where Mut is a mutable data structure. > In the dict exampe, there is a unncecessary work and garbage > collection > for the functional data structure Dict. > And you gain nothing, since it is sequential code. > > ETS gives you an unnecessary copy. The process dictionary does the > job, > but there is only one of them and it must be a hash table. It would be > > better with > PD1 = pd:new_hash_table(), > PD2 = pd:new_tree(), > etc. > > > > > E.F. Codd, who wrote the twelve (13, really) rules for relational > > databases, had > > this rule as No 12: > > > > Rule 12: The nonsubversion rule: > > *If the system provides a low-level (record-at-a-time) interface, > then > > that interface * > > *cannot be used to subvert the system, for example, bypassing a > > relational security * > > *or integrity constraint.* > > (http://en.wikipedia.org/wiki/Codd's_12_rules > > ) > > > > The importance of this rule is often underestimated, and it is quite > > > common to > > ditch it early in favour of performance, since the consequences of > > subverting > > the consistency model are not readily apparent until much later, if > > > even then. > > > > Interesting. I am not convinced. So basically, the programmer should > be > constrained, because he/she will make mistakes? > I guess it is a question of finding the right balance. So, you think > dirty_write was a mistake for mnesia? > > cheers, > > Morten. > > > > > > > BR, > > Ulf W > > > > On 18 Oct 2010, at 09:31, Morten Krogh wrote: > > > >> Hi Jachym > >> > >> There is one thing that I probably haven't made clear enough, and > it > >> will answer all points basically. > >> > >> Erlang has the loop receive loop process that serializes all access > > >> to the data. > >> The equivalent in C is a single byte. The semantics of > communicating > >> with a single byre is well defined in C. > >> A C thread communicates with the byte using message passing through > > >> the memory bus. The byte only allows get and put. > >> Nothing else makes sense. > >> > >> The general picture is that you have some units of atomicity and > >> beyond that you need to lock, coordinate access etc. > >> > >> Erlang gives you a higher control of the granularity than C, but > >> nothing is fundamentally different. I just object to all the Erlang > > >> statements about locks, concurrency etc. > >> > >> If you want to get rid of locks and shared memory, you lose > >> concurrency. You can't have both. You can serialize everything. In > C, > >> that would be one thread. In Erlang, > >> one process. If you want concurrency, you need smaller blocks of > >> data, which then needs to be coordinated. > >> > >> You can imagine an axis > >> > >> concurrency ~ locks, deadlocks etc ~ high performance in one > end. > >> > >> serialized access ~ no locks ~ single threaded ~ simple programing > > >> model in the other end. > >> > >> > >> C actually beats Erlang in both ends of the axis (not the > programming > >> model, of course, Erlang syntax is always simpler). > >> > >> Erlang's sweet spot is in the middle. It is so easy to package an > >> amount of data into a serialized agent with controller logic. > >> > >> The most concurrent program is not an erlang program, that would be > a > >> C program (or even lower) with a loop on each core, where all > >> iterations of all loops updated the memory in all kinds of ways. > Each > >> iteration of each loop is a "context switch". > >> > >> Erlang is of course very convenient. You get a whole "block of > bytes" > >> with serialized access, and a programming language to control the > >> "gate" to the block of bytes. > >> > >> The hardware manufacturers could make something like Erlang, if > they > >> wanted to. Chop the memory into blocks, and give each block > >> serialized access and a small programmable controller sitting at > the > >> gate of that block. They would lose performance and concurrency if > > >> they did. > >> > >> It is all the same. The only question is at what granularity you > >> serialize your data. > >> > >> Jachym, I will not go through your points. You can imagine my > >> response. It is basically : > >> > >> One byte in C is the equivalent of one Erlang process. One byte > >> doesn't need any controller logic besides get and put, and a > >> guaranteed atomicity of get and put. > >> > >> Cheers, > >> > >> Morten > >> > >> > >> > >> On 10/17/10 11:46 PM, Jachym Holecek wrote: > >>> # Morten Krogh 2010-10-17: > >>>> The problem is of course that a process like this > >>>> > >>>> loop(State) -> > >>>> receive > >>>> {From, get, X} -> > >>>> From ! {result, get(X, State)}, > >>>> loop(State) > >>>> > >>>> etc..... > >>>> > >>>> Behaves exactly like shared memory seen from other processes > point > >>>> of view. > >>>> > >>>> Writing > >>>> > >>>> Pid ! {self(), get, 27} > >>>> receive > >>>> {result, Result} -> > >>>> Result > >>>> end. > >>>> > >>>> is like writing > >>>> > >>>> get(X, Pid) in C (the language) where Pid is pointer to a shared > data > >>>> structure in C. > >>> "Apples and oranges" as they say. In your Erlang example: > >>> > >>> * Server explicitely agrees to provide some of its data. > >>> * Server is free to change this decision. > >>> * Client has to follow certain protocol to access this data. > >>> * All accesses to "shared" data are serialized. > >>> * It is "physically impossible" to obtain inconsistent snaphost > of > >>> the data. > >>> * It is "physically impossible" to bypass server's control. > >>> * This is guaranteed at language level. > >>> * Your example is implemented completely within this > well-defined > >>> semantics. > >>> > >>> Contrast that with C: > >>> > >>> * All threads have equal access right to whole process address > space. > >>> * Sychronization can be done, but is a mere convention > >>> (spinlocks/mutexes). > >>> * Data consistency can be done, but is a mere convetion (memory > > >>> barriers). > >>> * At language level memory access semantics is more or less > undefined. > >>> * Mutexes/memory barriers have to be done in assembly, ie. > outside C. > >>> > >>> A little more fun on the C side: > >>> > >>> * Compiler may reorder loads/stores (somewhat) unless you're > careful. > >>> * CPU may reorder loads/stores (somewhat) unless you're > careful. > >>> * Cache may do whatever it wants to (somewhat) unless you're > careful. > >>> > >>> Normally these thing will be taken care of by whatever threads > >>> implementation > >>> you use (in architecture- and compiler-specific way), but if we're > > >>> talking at > >>> language level then there's no way (as far as I know) to achieve > >>> sane shared > >>> memory semantics in C alone (if we admit the existence of > concurrent > >>> threads > >>> of execution which the language knows nothing about, of course). > >>> > >>> But sure -- you can _model_ Erlang-like processes in C, and you > can > >>> _model_ > >>> C-like shared memory in Erlang (including all of the quirks above > if you > >>> really wanted to). But at that point you're imposing > correspondence > >>> of some > >>> sort by brute force; as opposed to capturing inherent properties. > >>> > >>> Just my two pence... > >>> > >>> Regards, > >>> -- Jachym > >>> > >>> ________________________________________________________________ > >>> erlang-questions (at) erlang.org mailing > list. > >>> See http://www.erlang.org/faq.html > >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>> > >> > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > > http://erlang-solutions.com > > > > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From michael.santos@REDACTED Mon Oct 18 16:07:35 2010 From: michael.santos@REDACTED (Michael Santos) Date: Mon, 18 Oct 2010 10:07:35 -0400 Subject: [erlang-questions] When will print "*** Terminating erlang" In-Reply-To: References: Message-ID: <20101018140735.GA10291@ecn.lan> On Mon, Oct 18, 2010 at 02:01:01PM +0800, William Yangzhuwei wrote: > hi there, > > I can start one erlang program with ` start_test.sh `(start_test.sh is just > a script to start erl and passing some args). > > > When I use `start_test.sh &` to let this job run in background, it cannot > finish executing. > > I get some output like > > (test1@REDACTED)1> *** Terminating erlang (test1@REDACTED). > > I guess *** Terminating erlang is printed by otp not in this erlang program. > > How can I trace this trouble and any logging? Try passing the -detached or -noshell flags to erl. From mk@REDACTED Mon Oct 18 16:23:39 2010 From: mk@REDACTED (Morten Krogh) Date: Mon, 18 Oct 2010 16:23:39 +0200 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <1311495418.107811287409932041.JavaMail.root@zimbra> References: <1311495418.107811287409932041.JavaMail.root@zimbra> Message-ID: <0D96B9B0-B1BB-4B16-ACC5-063B5D1735C2@amberbio.com> Robert The user is not doing any C level locking in Erlang, but the user will lock at a higher level, the process message passing level. And that creates the same issues again with deadlocks etc. Morten. On Oct 18, 2010, at 3:52 PM, Robert Virding wrote: > I am bit late getting into this discussion but I have been away without good network connectivity. > > Of course you cannot have any form of useful concurrency without locks, share memory, etc at some level. There is however, one fundamental difference between doing it in C or in Erlang and that is *WHO* actually does all the locking, sharing, etc and *WHO* actually enforces the allowed access between processes. In C it is the user, either directly or through libraries, but it is only enforced by convention, there is nothing really *stopping* me from writing to any address I wish. In Erlang all locking, sharing etc is handled by the implementation and strictly enforced by having message passing as the only way of process communication for users. This is probably easier to implement as well, but it is by no means trivial to get it correct and efficient, as there is only a limited number of ways processes can communicate with each other. > > This is also a very powerful argument for immutable data. It is, of course, possible, and safe, to have mutable data and enforce message passing by copying (this is almost swearing in the church of Erlang) but it definitely limits the implementation. Data *must* be copied as it is mutable. With immutable data the implementation can itself decide how messages are actually "sent" between processes: either by copying as is done in the BEAM; or by having shared memory between processes; or by a combination of both. The user will not have to worry about it and not see the difference, except perhaps by performance. > > This also means that implementing a suggestion from earlier in this thread about adding threads becomes easier to implement and transparent to the user, just because data is immutable. It can in one sense be taken as a hint to the system as to how these processes are intended to interact with each other. With mutable data this is impossible. > > One quick comment on the original question about shared/hybrid heaps. Unfortunately, there is no "best" way of doing heaps: some applications will work better with shared heaps; some will work better with copying; and for many the difference will be negligible. For example having shared large binaries works well for some aplications but can cause others to crash. Also SMP's and multiple cores greatly increase the difficulty of shared heap implementation because of just problems with locking and synchronising memory access. > > Robert > > > ----- "Morten Krogh" wrote: > >> Ulf, >> >> I actually agree with most of what you say, and most of it is not at >> all >> in disagreement with my previous posts. >> >> I was talking about issues like concurrency, message passing, locks >> from >> a philosophical point of view, and >> I don't agree with the way Erlang is often presented as a way to have >> >> high concurrency without locks. >> I don't agree with the statement that shared memory is bad. >> >> You are talking about the Erlang platform, and its productivity. I >> like >> Erlang. I find it very easy to write programs, and >> have the processes talk to each other, monitor them etc. My >> philosophical rants were not against the language or platform. >> >>> >>> With all due respect, your argument is one that I've heard many >> times >>> in different settings, and it misses a key point (which Jachym >> brought >>> up). >>> My experience from working in development of systems with complex >>> concurrency patterns is that this argument breaks down completely >>> and horribly expensively over time in larger projects. >>> >> >> The argument breaks down? You mean that it becomes hard to write and >> maintain code? But I >> haven't made claims about that. I find Erlang programs much easier >> than >> C programs. >> >> >>> Saying that there are no fundamental differences between Erlang and >> C >>> essentially means that you claim that "correct by design" and >> "correct by >>> convention" are the same thing. By this token, Erlang and Haskell >> are >>> fundamentally the same too, since it really doesn't matter that the >> >>> Haskell >>> compiler can verify many more aspects of a program at compile-time >> than >>> Erlang can - you can simply avoid making type mistakes in Erlang >> and >>> be done with it! >> There are no fundamental difference between Erlang and C, when it >> comes to >> the concepts of concurrency, locks, deadlocks, shared memory. >> The only qualitative difference is between a sequential program and >> everything else. >> >> There are huge differences in actually implementing things, and the >> syntax, of course. >> >>> Your line of reasoning also suggests that we can put great faith in >>> micro-benchmarks, since, if you can make it work and make it really >>> fast in a small example in C, you can do it equally well in a larger >> and >>> more complex example. >>> >> I believe in modularity and simplicity. You need to choose the right >> level to work in. >> Erlang is just a layer on top of C. For many purposes, it is better to >> >> use Erlang than C. >> For other purposes, it is better to use electric currents in nand >> gates. >> Of course, you need higher level apis. >> Ulf, I agree totally with you. >> >>> The place where Erlang and C become very different is when you >> start >>> writing code that has "interesting" complexity. Automatic memory >>> management may seem like a complete waste, until your program >>> becomes sufficiently complex that you can no longer easily keep >> track >>> of where you should lock, allocate and free memory. At this point, >> it is >>> actually very difficult to outperform a good garbage collector with >> hand- >>> written code. >>> >> I agree. >> >> And you see what the next higher level will be. Automatic process >> collection, when a process can never be reached anymore. >> In 20 years, people will have "gigaprocesses", the older folks will >> remember that there was something called "gigabytes", and the next >> languages will have automatic >> process spawning and process collection. Erlang will be a low level >> language. What will such a unit of processes be called? A brain :) >> >> The brain languages will still have deadlocks, shared memory, locks >> etc, >> because those are philosophical/mathematical entities that are never >> "solved". >> >> >> >> >> >>> I've seen lots of real-world examples where >>> small teams of expert C++ programmers have managed to convince >>> management to go with a well-trimmed, elegant execution model in >>> C++, only to run in all sorts of problems when lesser programmers >> were >>> to start adding features - huge dips in performance and stability, >> raving >>> memory leaks, and frantic running around and fire fighting for the >>> experts, who are the only ones who seem to manage to stick to the >>> conventions and not break stuff. The beginning mantra, "how hard >> can >>> it be?", is later answered with "apparently, endlessly hard". >>> >> >> Interesting. By the way, are there examples of Erlang teams that have >> >> switched away from Erlang? >> >>> And we haven't even discussed the problem of one of these processes >>> crashing yet. With two concurrent contexts explicitly writing to the >> same >>> block of memory, ensuring the integrity of that memory block if one >> of >>> the contexts throws an exception will be very, very hard. This is >> where >>> Erlang's "share nothing" philosophy really pays off. The VM may >> decide >>> to share data under the hood, but as a programmer, you can be >> absolutely >>> certain that the death of another process - even in the same memory >> space >>> - cannot ruin the integrity of your own data. It really is as if you >> >>> had your own >>> distinct copies of everything. >>> >> This is where we disagree strongly. I am saying that you get the same >> >> problems again just at a higher level. >> The equivalent situation is this. >> P1 is a process, that keeps data, i.e. the so called shared memory. >> P2 and P3 sends messages to P1, many each. Remember a put message is >> like a mutation. >> If you want to serialize the updates from P2 and P3, a lock is needed. >> >> P1 for instance could wait for all messages from P2, before receiving >> >> from P3, >> incorporate them, release its lock and accept the messages from P3. >> What >> should P1 do ,if the messages from P2 stop early. Roll back, ignore >> it. >> The issues are exactly the same as at the lower byte level. Don't you >> >> see that these are deep principles, not something a language/VM can >> solve? >> >> Memory space in Erlang is not the same as memory space for C. >> Everything >> is lifted one level. The equivalent of memory space in Erlang is the >> set >> of running processes. >> The addresses are now Pids not RAM addresses. And if you want >> something >> interesting to happen, you must have some processes that accept >> messages >> and act on them. >> C address space does not exist for the Erlang programmer as little as >> >> physical positions of transistors exist for the C program. A C >> program >> does not crash if a transistor goes bad if >> there is a "virtualization layer" in between that redirects the >> current >> to a another transistor. >> >> >>> The problem of crashing processes is a particularly nasty >> _containment_ >>> problem, but when you scrutinise many of the concurrency models out >>> there you soon find that there are usually many issues that are not >> well >>> contained. Encapsulation and containment are best handled in the >>> programming model, rather than relying on convention, and this is >>> exactly the area where there is a fundamental difference. >>> >> Ulf, I agree that one should have higher level languages that >> encapsulate things, and maybe JAVA or C++ or whatever you are thinking >> >> about are too low level >> for big projects. Fine with me. >> >> On practical terms, I feel that Erlang some times gives up performance >> >> for unnecessary reasons. I am especially thinking about mutability. >> There is no reason to give up mutation within a process which is >> already >> a serializer. >> >> code like this >> >> loop(Dict) >> receive >> {put, K, V} -> >> Dict2 = dict:store(K, V, Dict), >> loop(Dict2) >> end. >> >> >> is just a performance wise inefficient way of doing >> >> loop(Mut) -> >> receive >> {put, K, V} -> >> put(K,V, Mut), >> loop(Mut) >> end. >> >> where Mut is a mutable data structure. >> In the dict exampe, there is a unncecessary work and garbage >> collection >> for the functional data structure Dict. >> And you gain nothing, since it is sequential code. >> >> ETS gives you an unnecessary copy. The process dictionary does the >> job, >> but there is only one of them and it must be a hash table. It would be >> >> better with >> PD1 = pd:new_hash_table(), >> PD2 = pd:new_tree(), >> etc. >> >> >> >>> E.F. Codd, who wrote the twelve (13, really) rules for relational >>> databases, had >>> this rule as No 12: >>> >>> Rule 12: The nonsubversion rule: >>> *If the system provides a low-level (record-at-a-time) interface, >> then >>> that interface * >>> *cannot be used to subvert the system, for example, bypassing a >>> relational security * >>> *or integrity constraint.* >>> (http://en.wikipedia.org/wiki/Codd's_12_rules >>> ) >>> >>> The importance of this rule is often underestimated, and it is quite >> >>> common to >>> ditch it early in favour of performance, since the consequences of >>> subverting >>> the consistency model are not readily apparent until much later, if >> >>> even then. >>> >> >> Interesting. I am not convinced. So basically, the programmer should >> be >> constrained, because he/she will make mistakes? >> I guess it is a question of finding the right balance. So, you think >> dirty_write was a mistake for mnesia? >> >> cheers, >> >> Morten. >> >> >> >> >> >>> BR, >>> Ulf W >>> >>> On 18 Oct 2010, at 09:31, Morten Krogh wrote: >>> >>>> Hi Jachym >>>> >>>> There is one thing that I probably haven't made clear enough, and >> it >>>> will answer all points basically. >>>> >>>> Erlang has the loop receive loop process that serializes all access >> >>>> to the data. >>>> The equivalent in C is a single byte. The semantics of >> communicating >>>> with a single byre is well defined in C. >>>> A C thread communicates with the byte using message passing through >> >>>> the memory bus. The byte only allows get and put. >>>> Nothing else makes sense. >>>> >>>> The general picture is that you have some units of atomicity and >>>> beyond that you need to lock, coordinate access etc. >>>> >>>> Erlang gives you a higher control of the granularity than C, but >>>> nothing is fundamentally different. I just object to all the Erlang >> >>>> statements about locks, concurrency etc. >>>> >>>> If you want to get rid of locks and shared memory, you lose >>>> concurrency. You can't have both. You can serialize everything. In >> C, >>>> that would be one thread. In Erlang, >>>> one process. If you want concurrency, you need smaller blocks of >>>> data, which then needs to be coordinated. >>>> >>>> You can imagine an axis >>>> >>>> concurrency ~ locks, deadlocks etc ~ high performance in one >> end. >>>> >>>> serialized access ~ no locks ~ single threaded ~ simple programing >> >>>> model in the other end. >>>> >>>> >>>> C actually beats Erlang in both ends of the axis (not the >> programming >>>> model, of course, Erlang syntax is always simpler). >>>> >>>> Erlang's sweet spot is in the middle. It is so easy to package an >>>> amount of data into a serialized agent with controller logic. >>>> >>>> The most concurrent program is not an erlang program, that would be >> a >>>> C program (or even lower) with a loop on each core, where all >>>> iterations of all loops updated the memory in all kinds of ways. >> Each >>>> iteration of each loop is a "context switch". >>>> >>>> Erlang is of course very convenient. You get a whole "block of >> bytes" >>>> with serialized access, and a programming language to control the >>>> "gate" to the block of bytes. >>>> >>>> The hardware manufacturers could make something like Erlang, if >> they >>>> wanted to. Chop the memory into blocks, and give each block >>>> serialized access and a small programmable controller sitting at >> the >>>> gate of that block. They would lose performance and concurrency if >> >>>> they did. >>>> >>>> It is all the same. The only question is at what granularity you >>>> serialize your data. >>>> >>>> Jachym, I will not go through your points. You can imagine my >>>> response. It is basically : >>>> >>>> One byte in C is the equivalent of one Erlang process. One byte >>>> doesn't need any controller logic besides get and put, and a >>>> guaranteed atomicity of get and put. >>>> >>>> Cheers, >>>> >>>> Morten >>>> >>>> >>>> >>>> On 10/17/10 11:46 PM, Jachym Holecek wrote: >>>>> # Morten Krogh 2010-10-17: >>>>>> The problem is of course that a process like this >>>>>> >>>>>> loop(State) -> >>>>>> receive >>>>>> {From, get, X} -> >>>>>> From ! {result, get(X, State)}, >>>>>> loop(State) >>>>>> >>>>>> etc..... >>>>>> >>>>>> Behaves exactly like shared memory seen from other processes >> point >>>>>> of view. >>>>>> >>>>>> Writing >>>>>> >>>>>> Pid ! {self(), get, 27} >>>>>> receive >>>>>> {result, Result} -> >>>>>> Result >>>>>> end. >>>>>> >>>>>> is like writing >>>>>> >>>>>> get(X, Pid) in C (the language) where Pid is pointer to a shared >> data >>>>>> structure in C. >>>>> "Apples and oranges" as they say. In your Erlang example: >>>>> >>>>> * Server explicitely agrees to provide some of its data. >>>>> * Server is free to change this decision. >>>>> * Client has to follow certain protocol to access this data. >>>>> * All accesses to "shared" data are serialized. >>>>> * It is "physically impossible" to obtain inconsistent snaphost >> of >>>>> the data. >>>>> * It is "physically impossible" to bypass server's control. >>>>> * This is guaranteed at language level. >>>>> * Your example is implemented completely within this >> well-defined >>>>> semantics. >>>>> >>>>> Contrast that with C: >>>>> >>>>> * All threads have equal access right to whole process address >> space. >>>>> * Sychronization can be done, but is a mere convention >>>>> (spinlocks/mutexes). >>>>> * Data consistency can be done, but is a mere convetion (memory >> >>>>> barriers). >>>>> * At language level memory access semantics is more or less >> undefined. >>>>> * Mutexes/memory barriers have to be done in assembly, ie. >> outside C. >>>>> >>>>> A little more fun on the C side: >>>>> >>>>> * Compiler may reorder loads/stores (somewhat) unless you're >> careful. >>>>> * CPU may reorder loads/stores (somewhat) unless you're >> careful. >>>>> * Cache may do whatever it wants to (somewhat) unless you're >> careful. >>>>> >>>>> Normally these thing will be taken care of by whatever threads >>>>> implementation >>>>> you use (in architecture- and compiler-specific way), but if we're >> >>>>> talking at >>>>> language level then there's no way (as far as I know) to achieve >>>>> sane shared >>>>> memory semantics in C alone (if we admit the existence of >> concurrent >>>>> threads >>>>> of execution which the language knows nothing about, of course). >>>>> >>>>> But sure -- you can _model_ Erlang-like processes in C, and you >> can >>>>> _model_ >>>>> C-like shared memory in Erlang (including all of the quirks above >> if you >>>>> really wanted to). But at that point you're imposing >> correspondence >>>>> of some >>>>> sort by brute force; as opposed to capturing inherent properties. >>>>> >>>>> Just my two pence... >>>>> >>>>> Regards, >>>>> -- Jachym >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions (at) erlang.org mailing >> list. >>>>> See http://www.erlang.org/faq.html >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>> >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>> >>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>> http://erlang-solutions.com >>> >>> >>> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From MARTIN.LOGAN@REDACTED Mon Oct 18 16:59:08 2010 From: MARTIN.LOGAN@REDACTED (Logan, Martin) Date: Mon, 18 Oct 2010 09:59:08 -0500 Subject: eDoc picking up -spec and -type; when? Message-ID: I hear this is going to happen. Do we know when? Specs are something I use all over the place in my code now and find them very helpful. The current issue is for those of us that make good use of @spec so that we can see type specifications in our edoc we now have to maintain specs in two places which is not tractable. Can anyone tell me when eDoc will start to pick up specs? Cheers, Martin From ARoe@REDACTED Mon Oct 18 17:02:07 2010 From: ARoe@REDACTED (Adrian Roe) Date: Mon, 18 Oct 2010 08:02:07 -0700 Subject: [erlang-questions] Table viewer disappearing In-Reply-To: References: <0697C1D63C68ED4DA4EE16BFBB156ACD02AF0C29E0@IE2RD2XVS071.red002.local> Message-ID: <0697C1D63C68ED4DA4EE16BFBB156ACD02AF0C2C5B@IE2RD2XVS071.red002.local> Kenneth That was just the ticket - the local copy of tk was corrupted. All working now thanks :) Adrian -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Kenneth Lundin Sent: 18 October 2010 14:31 To: Adrian Roe Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Table viewer disappearing Hi, tv is a gs based application and thus depends on the gs application which in turn depends on that you have tcl/tk installed. Try gs:start() in the Erlang shell, what happens? /Kenneth Erlang/OTP , Ericsson On Mon, Oct 18, 2010 at 1:24 PM, Adrian Roe wrote: > Apologies for a potentially environment related question, but I have been happily developing an mnesia database subsystem on an Amazon EC2 instance running Ubuntu 10.10. ?As part of this I have used table viewer (tv:start()) and it has worked just fine (using NX/Nomachine & GNOME). ?No idea what has changed, but now when I do tv:start(), I get a PID returned but no window displayed. ?I've even reverted to machine images from when it was working but still no window spawned. Things like wx:demo() launch just fine. > > The docs around tv only talk about its start method and tv doesn't appear to respond any messages sent to it via its PID. ?Has anyone else seen this / know what on earth might be going on. > > Thanks for any suggestions > > Adrian > > ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From jameschurchman@REDACTED Mon Oct 18 19:27:09 2010 From: jameschurchman@REDACTED (James Churchman) Date: Mon, 18 Oct 2010 18:27:09 +0100 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <0D96B9B0-B1BB-4B16-ACC5-063B5D1735C2@amberbio.com> References: <1311495418.107811287409932041.JavaMail.root@zimbra> <0D96B9B0-B1BB-4B16-ACC5-063B5D1735C2@amberbio.com> Message-ID: <349A2A56-678F-446D-8E1E-D4F33AECE74C@gmail.com> maybe ( i stress the maybe here.. ! ) it would be possible to produce some higher level construct (written in erlang) that acts more like a data base transaction for multiple gen_servers or processes, where you don't "lock" all the processes that may get used in the remaining run of code, but instead at the higher level you specify that all processes that have messages sent in the following run of code must not have any other messages sent to them until this chunk of code is run. This would further abstract you from doing your own high-locks although obviously would have to be based on some sort of intelligent locking mechanism at some level this would allow basic code like : process1 ! message1, process2 ! message2, process3 ! message3; (with other independent code that also messages those processes) & also more complex logic where you don't necessarily know which processes will get messages in a piece of running code, all with out "locks"? ps i am not in any way volunteering to write this, or even saying its practically possible On 18 Oct 2010, at 15:23, Morten Krogh wrote: > Robert > > The user is not doing any C level locking in Erlang, but the user will lock at a higher level, the process message passing level. > And that creates the same issues again with deadlocks etc. > > Morten. > > > > > On Oct 18, 2010, at 3:52 PM, Robert Virding wrote: > >> I am bit late getting into this discussion but I have been away without good network connectivity. >> >> Of course you cannot have any form of useful concurrency without locks, share memory, etc at some level. There is however, one fundamental difference between doing it in C or in Erlang and that is *WHO* actually does all the locking, sharing, etc and *WHO* actually enforces the allowed access between processes. In C it is the user, either directly or through libraries, but it is only enforced by convention, there is nothing really *stopping* me from writing to any address I wish. In Erlang all locking, sharing etc is handled by the implementation and strictly enforced by having message passing as the only way of process communication for users. This is probably easier to implement as well, but it is by no means trivial to get it correct and efficient, as there is only a limited number of ways processes can communicate with each other. >> >> This is also a very powerful argument for immutable data. It is, of course, possible, and safe, to have mutable data and enforce message passing by copying (this is almost swearing in the church of Erlang) but it definitely limits the implementation. Data *must* be copied as it is mutable. With immutable data the implementation can itself decide how messages are actually "sent" between processes: either by copying as is done in the BEAM; or by having shared memory between processes; or by a combination of both. The user will not have to worry about it and not see the difference, except perhaps by performance. >> >> This also means that implementing a suggestion from earlier in this thread about adding threads becomes easier to implement and transparent to the user, just because data is immutable. It can in one sense be taken as a hint to the system as to how these processes are intended to interact with each other. With mutable data this is impossible. >> >> One quick comment on the original question about shared/hybrid heaps. Unfortunately, there is no "best" way of doing heaps: some applications will work better with shared heaps; some will work better with copying; and for many the difference will be negligible. For example having shared large binaries works well for some aplications but can cause others to crash. Also SMP's and multiple cores greatly increase the difficulty of shared heap implementation because of just problems with locking and synchronising memory access. >> >> Robert >> >> >> ----- "Morten Krogh" wrote: >> >>> Ulf, >>> >>> I actually agree with most of what you say, and most of it is not at >>> all >>> in disagreement with my previous posts. >>> >>> I was talking about issues like concurrency, message passing, locks >>> from >>> a philosophical point of view, and >>> I don't agree with the way Erlang is often presented as a way to have >>> >>> high concurrency without locks. >>> I don't agree with the statement that shared memory is bad. >>> >>> You are talking about the Erlang platform, and its productivity. I >>> like >>> Erlang. I find it very easy to write programs, and >>> have the processes talk to each other, monitor them etc. My >>> philosophical rants were not against the language or platform. >>> >>>> >>>> With all due respect, your argument is one that I've heard many >>> times >>>> in different settings, and it misses a key point (which Jachym >>> brought >>>> up). >>>> My experience from working in development of systems with complex >>>> concurrency patterns is that this argument breaks down completely >>>> and horribly expensively over time in larger projects. >>>> >>> >>> The argument breaks down? You mean that it becomes hard to write and >>> maintain code? But I >>> haven't made claims about that. I find Erlang programs much easier >>> than >>> C programs. >>> >>> >>>> Saying that there are no fundamental differences between Erlang and >>> C >>>> essentially means that you claim that "correct by design" and >>> "correct by >>>> convention" are the same thing. By this token, Erlang and Haskell >>> are >>>> fundamentally the same too, since it really doesn't matter that the >>> >>>> Haskell >>>> compiler can verify many more aspects of a program at compile-time >>> than >>>> Erlang can - you can simply avoid making type mistakes in Erlang >>> and >>>> be done with it! >>> There are no fundamental difference between Erlang and C, when it >>> comes to >>> the concepts of concurrency, locks, deadlocks, shared memory. >>> The only qualitative difference is between a sequential program and >>> everything else. >>> >>> There are huge differences in actually implementing things, and the >>> syntax, of course. >>> >>>> Your line of reasoning also suggests that we can put great faith in >>>> micro-benchmarks, since, if you can make it work and make it really >>>> fast in a small example in C, you can do it equally well in a larger >>> and >>>> more complex example. >>>> >>> I believe in modularity and simplicity. You need to choose the right >>> level to work in. >>> Erlang is just a layer on top of C. For many purposes, it is better to >>> >>> use Erlang than C. >>> For other purposes, it is better to use electric currents in nand >>> gates. >>> Of course, you need higher level apis. >>> Ulf, I agree totally with you. >>> >>>> The place where Erlang and C become very different is when you >>> start >>>> writing code that has "interesting" complexity. Automatic memory >>>> management may seem like a complete waste, until your program >>>> becomes sufficiently complex that you can no longer easily keep >>> track >>>> of where you should lock, allocate and free memory. At this point, >>> it is >>>> actually very difficult to outperform a good garbage collector with >>> hand- >>>> written code. >>>> >>> I agree. >>> >>> And you see what the next higher level will be. Automatic process >>> collection, when a process can never be reached anymore. >>> In 20 years, people will have "gigaprocesses", the older folks will >>> remember that there was something called "gigabytes", and the next >>> languages will have automatic >>> process spawning and process collection. Erlang will be a low level >>> language. What will such a unit of processes be called? A brain :) >>> >>> The brain languages will still have deadlocks, shared memory, locks >>> etc, >>> because those are philosophical/mathematical entities that are never >>> "solved". >>> >>> >>> >>> >>> >>>> I've seen lots of real-world examples where >>>> small teams of expert C++ programmers have managed to convince >>>> management to go with a well-trimmed, elegant execution model in >>>> C++, only to run in all sorts of problems when lesser programmers >>> were >>>> to start adding features - huge dips in performance and stability, >>> raving >>>> memory leaks, and frantic running around and fire fighting for the >>>> experts, who are the only ones who seem to manage to stick to the >>>> conventions and not break stuff. The beginning mantra, "how hard >>> can >>>> it be?", is later answered with "apparently, endlessly hard". >>>> >>> >>> Interesting. By the way, are there examples of Erlang teams that have >>> >>> switched away from Erlang? >>> >>>> And we haven't even discussed the problem of one of these processes >>>> crashing yet. With two concurrent contexts explicitly writing to the >>> same >>>> block of memory, ensuring the integrity of that memory block if one >>> of >>>> the contexts throws an exception will be very, very hard. This is >>> where >>>> Erlang's "share nothing" philosophy really pays off. The VM may >>> decide >>>> to share data under the hood, but as a programmer, you can be >>> absolutely >>>> certain that the death of another process - even in the same memory >>> space >>>> - cannot ruin the integrity of your own data. It really is as if you >>> >>>> had your own >>>> distinct copies of everything. >>>> >>> This is where we disagree strongly. I am saying that you get the same >>> >>> problems again just at a higher level. >>> The equivalent situation is this. >>> P1 is a process, that keeps data, i.e. the so called shared memory. >>> P2 and P3 sends messages to P1, many each. Remember a put message is >>> like a mutation. >>> If you want to serialize the updates from P2 and P3, a lock is needed. >>> >>> P1 for instance could wait for all messages from P2, before receiving >>> >>> from P3, >>> incorporate them, release its lock and accept the messages from P3. >>> What >>> should P1 do ,if the messages from P2 stop early. Roll back, ignore >>> it. >>> The issues are exactly the same as at the lower byte level. Don't you >>> >>> see that these are deep principles, not something a language/VM can >>> solve? >>> >>> Memory space in Erlang is not the same as memory space for C. >>> Everything >>> is lifted one level. The equivalent of memory space in Erlang is the >>> set >>> of running processes. >>> The addresses are now Pids not RAM addresses. And if you want >>> something >>> interesting to happen, you must have some processes that accept >>> messages >>> and act on them. >>> C address space does not exist for the Erlang programmer as little as >>> >>> physical positions of transistors exist for the C program. A C >>> program >>> does not crash if a transistor goes bad if >>> there is a "virtualization layer" in between that redirects the >>> current >>> to a another transistor. >>> >>> >>>> The problem of crashing processes is a particularly nasty >>> _containment_ >>>> problem, but when you scrutinise many of the concurrency models out >>>> there you soon find that there are usually many issues that are not >>> well >>>> contained. Encapsulation and containment are best handled in the >>>> programming model, rather than relying on convention, and this is >>>> exactly the area where there is a fundamental difference. >>>> >>> Ulf, I agree that one should have higher level languages that >>> encapsulate things, and maybe JAVA or C++ or whatever you are thinking >>> >>> about are too low level >>> for big projects. Fine with me. >>> >>> On practical terms, I feel that Erlang some times gives up performance >>> >>> for unnecessary reasons. I am especially thinking about mutability. >>> There is no reason to give up mutation within a process which is >>> already >>> a serializer. >>> >>> code like this >>> >>> loop(Dict) >>> receive >>> {put, K, V} -> >>> Dict2 = dict:store(K, V, Dict), >>> loop(Dict2) >>> end. >>> >>> >>> is just a performance wise inefficient way of doing >>> >>> loop(Mut) -> >>> receive >>> {put, K, V} -> >>> put(K,V, Mut), >>> loop(Mut) >>> end. >>> >>> where Mut is a mutable data structure. >>> In the dict exampe, there is a unncecessary work and garbage >>> collection >>> for the functional data structure Dict. >>> And you gain nothing, since it is sequential code. >>> >>> ETS gives you an unnecessary copy. The process dictionary does the >>> job, >>> but there is only one of them and it must be a hash table. It would be >>> >>> better with >>> PD1 = pd:new_hash_table(), >>> PD2 = pd:new_tree(), >>> etc. >>> >>> >>> >>>> E.F. Codd, who wrote the twelve (13, really) rules for relational >>>> databases, had >>>> this rule as No 12: >>>> >>>> Rule 12: The nonsubversion rule: >>>> *If the system provides a low-level (record-at-a-time) interface, >>> then >>>> that interface * >>>> *cannot be used to subvert the system, for example, bypassing a >>>> relational security * >>>> *or integrity constraint.* >>>> (http://en.wikipedia.org/wiki/Codd's_12_rules >>>> ) >>>> >>>> The importance of this rule is often underestimated, and it is quite >>> >>>> common to >>>> ditch it early in favour of performance, since the consequences of >>>> subverting >>>> the consistency model are not readily apparent until much later, if >>> >>>> even then. >>>> >>> >>> Interesting. I am not convinced. So basically, the programmer should >>> be >>> constrained, because he/she will make mistakes? >>> I guess it is a question of finding the right balance. So, you think >>> dirty_write was a mistake for mnesia? >>> >>> cheers, >>> >>> Morten. >>> >>> >>> >>> >>> >>>> BR, >>>> Ulf W >>>> >>>> On 18 Oct 2010, at 09:31, Morten Krogh wrote: >>>> >>>>> Hi Jachym >>>>> >>>>> There is one thing that I probably haven't made clear enough, and >>> it >>>>> will answer all points basically. >>>>> >>>>> Erlang has the loop receive loop process that serializes all access >>> >>>>> to the data. >>>>> The equivalent in C is a single byte. The semantics of >>> communicating >>>>> with a single byre is well defined in C. >>>>> A C thread communicates with the byte using message passing through >>> >>>>> the memory bus. The byte only allows get and put. >>>>> Nothing else makes sense. >>>>> >>>>> The general picture is that you have some units of atomicity and >>>>> beyond that you need to lock, coordinate access etc. >>>>> >>>>> Erlang gives you a higher control of the granularity than C, but >>>>> nothing is fundamentally different. I just object to all the Erlang >>> >>>>> statements about locks, concurrency etc. >>>>> >>>>> If you want to get rid of locks and shared memory, you lose >>>>> concurrency. You can't have both. You can serialize everything. In >>> C, >>>>> that would be one thread. In Erlang, >>>>> one process. If you want concurrency, you need smaller blocks of >>>>> data, which then needs to be coordinated. >>>>> >>>>> You can imagine an axis >>>>> >>>>> concurrency ~ locks, deadlocks etc ~ high performance in one >>> end. >>>>> >>>>> serialized access ~ no locks ~ single threaded ~ simple programing >>> >>>>> model in the other end. >>>>> >>>>> >>>>> C actually beats Erlang in both ends of the axis (not the >>> programming >>>>> model, of course, Erlang syntax is always simpler). >>>>> >>>>> Erlang's sweet spot is in the middle. It is so easy to package an >>>>> amount of data into a serialized agent with controller logic. >>>>> >>>>> The most concurrent program is not an erlang program, that would be >>> a >>>>> C program (or even lower) with a loop on each core, where all >>>>> iterations of all loops updated the memory in all kinds of ways. >>> Each >>>>> iteration of each loop is a "context switch". >>>>> >>>>> Erlang is of course very convenient. You get a whole "block of >>> bytes" >>>>> with serialized access, and a programming language to control the >>>>> "gate" to the block of bytes. >>>>> >>>>> The hardware manufacturers could make something like Erlang, if >>> they >>>>> wanted to. Chop the memory into blocks, and give each block >>>>> serialized access and a small programmable controller sitting at >>> the >>>>> gate of that block. They would lose performance and concurrency if >>> >>>>> they did. >>>>> >>>>> It is all the same. The only question is at what granularity you >>>>> serialize your data. >>>>> >>>>> Jachym, I will not go through your points. You can imagine my >>>>> response. It is basically : >>>>> >>>>> One byte in C is the equivalent of one Erlang process. One byte >>>>> doesn't need any controller logic besides get and put, and a >>>>> guaranteed atomicity of get and put. >>>>> >>>>> Cheers, >>>>> >>>>> Morten >>>>> >>>>> >>>>> >>>>> On 10/17/10 11:46 PM, Jachym Holecek wrote: >>>>>> # Morten Krogh 2010-10-17: >>>>>>> The problem is of course that a process like this >>>>>>> >>>>>>> loop(State) -> >>>>>>> receive >>>>>>> {From, get, X} -> >>>>>>> From ! {result, get(X, State)}, >>>>>>> loop(State) >>>>>>> >>>>>>> etc..... >>>>>>> >>>>>>> Behaves exactly like shared memory seen from other processes >>> point >>>>>>> of view. >>>>>>> >>>>>>> Writing >>>>>>> >>>>>>> Pid ! {self(), get, 27} >>>>>>> receive >>>>>>> {result, Result} -> >>>>>>> Result >>>>>>> end. >>>>>>> >>>>>>> is like writing >>>>>>> >>>>>>> get(X, Pid) in C (the language) where Pid is pointer to a shared >>> data >>>>>>> structure in C. >>>>>> "Apples and oranges" as they say. In your Erlang example: >>>>>> >>>>>> * Server explicitely agrees to provide some of its data. >>>>>> * Server is free to change this decision. >>>>>> * Client has to follow certain protocol to access this data. >>>>>> * All accesses to "shared" data are serialized. >>>>>> * It is "physically impossible" to obtain inconsistent snaphost >>> of >>>>>> the data. >>>>>> * It is "physically impossible" to bypass server's control. >>>>>> * This is guaranteed at language level. >>>>>> * Your example is implemented completely within this >>> well-defined >>>>>> semantics. >>>>>> >>>>>> Contrast that with C: >>>>>> >>>>>> * All threads have equal access right to whole process address >>> space. >>>>>> * Sychronization can be done, but is a mere convention >>>>>> (spinlocks/mutexes). >>>>>> * Data consistency can be done, but is a mere convetion (memory >>> >>>>>> barriers). >>>>>> * At language level memory access semantics is more or less >>> undefined. >>>>>> * Mutexes/memory barriers have to be done in assembly, ie. >>> outside C. >>>>>> >>>>>> A little more fun on the C side: >>>>>> >>>>>> * Compiler may reorder loads/stores (somewhat) unless you're >>> careful. >>>>>> * CPU may reorder loads/stores (somewhat) unless you're >>> careful. >>>>>> * Cache may do whatever it wants to (somewhat) unless you're >>> careful. >>>>>> >>>>>> Normally these thing will be taken care of by whatever threads >>>>>> implementation >>>>>> you use (in architecture- and compiler-specific way), but if we're >>> >>>>>> talking at >>>>>> language level then there's no way (as far as I know) to achieve >>>>>> sane shared >>>>>> memory semantics in C alone (if we admit the existence of >>> concurrent >>>>>> threads >>>>>> of execution which the language knows nothing about, of course). >>>>>> >>>>>> But sure -- you can _model_ Erlang-like processes in C, and you >>> can >>>>>> _model_ >>>>>> C-like shared memory in Erlang (including all of the quirks above >>> if you >>>>>> really wanted to). But at that point you're imposing >>> correspondence >>>>>> of some >>>>>> sort by brute force; as opposed to capturing inherent properties. >>>>>> >>>>>> Just my two pence... >>>>>> >>>>>> Regards, >>>>>> -- Jachym >>>>>> >>>>>> ________________________________________________________________ >>>>>> erlang-questions (at) erlang.org mailing >>> list. >>>>>> See http://www.erlang.org/faq.html >>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>>> >>>>> >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions (at) erlang.org mailing list. >>>>> See http://www.erlang.org/faq.html >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>> >>>> >>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>>> http://erlang-solutions.com >>>> >>>> >>>> >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From vances@REDACTED Mon Oct 18 19:29:31 2010 From: vances@REDACTED (Vance Shipley) Date: Mon, 18 Oct 2010 13:29:31 -0400 Subject: [erlang-questions] Re: Change node name in mnesia In-Reply-To: References: <20100214031924.GM13976@h216-235-12-174.host.egate.net> <20100702222835.GC267@h216-235-12-169.host.egate.net> Message-ID: <20101018172930.GE21059@h216-235-12-174.host.egate.net> On Sat, Jul 03, 2010 at 09:09:31AM +0200, Dan Gudmundsson wrote: } Are you seeing those in the ets table? Yes I do see the headers in the ets table: 4> ets:i(foo). [...] <12 > {log_header,dcl_log,"1.0","4.4.10",'bar@REDACTED',{1287,322033,2 ... [...] } They should not be in the table, they are what they say the header of } a TABLE.DCL file. } } Either you or I have some improvements to do :-) I did another database migration today, changing long node name to short, on R13B04 yesterday and experienced the same results. These header records don't show up in the items passed through mnesia:traverse_backup/6 so it doesn't appear to be something I'm doing wrong. I'm going to work on demonstrating the problem as simply as possible. -- -Vance } On Sat, Jul 3, 2010 at 12:28 AM, Vance Shipley wrote: } > Is it valid to have records in an mnesia table of this form: } > } > ?{log_header,dcl_log,"1.0","4.4.10",'foo@REDACTED',{1266,113437,587068}}, } > } > The tables seem to be working fine even though these entries are } > there which obviously do not have the correct OID for the table. } > } > What I'd like to understand is if I've broken the tables myself } > or if this type of thing is valid and I should just deal with } > them when traversing a table with mnesia:foldl/3. } > } > ? ? ? ?-Vance } > } > On Sat, Feb 13, 2010 at 10:19:24PM -0500, Vance Shipley wrote: } > } ?I used the example in the Mnesia User's Guide to backup } > } ?and transform a database from a long node name to a short } > } ?one. ?That all seemed to go smoothly enough but now I'm } > } ?noticing these strange records in my tables: } > } } > } ? ? {log_header,dcl_log,"1.0","4.4.10",'foo@REDACTED',{1266,113437,587068}}, } > } } > } ?I discovered this using mnesia:foldl/3. ?Is this normal } > } ?or have I messed things up? From andrew@REDACTED Mon Oct 18 19:40:15 2010 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 18 Oct 2010 13:40:15 -0400 Subject: [erlang-questions] eDoc picking up -spec and -type; when? In-Reply-To: References: Message-ID: <20101018174015.GE19035@hijacked.us> On Mon, Oct 18, 2010 at 09:59:08AM -0500, Logan, Martin wrote: > I hear this is going to happen. Do we know when? Specs are something I use all over the place in my code now and find them very helpful. The current issue is for those of us that make good use of @spec so that we can see type specifications in our edoc we now have to maintain specs in two places which is not tractable. Can anyone tell me when eDoc will start to pick up specs? > Yes, I've been awaiting this feature for a long time too. I'd expected it before R14 but we're still waiting. Andrew From zeno490@REDACTED Tue Oct 19 00:33:43 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Mon, 18 Oct 2010 18:33:43 -0400 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <349A2A56-678F-446D-8E1E-D4F33AECE74C@gmail.com> References: <1311495418.107811287409932041.JavaMail.root@zimbra> <0D96B9B0-B1BB-4B16-ACC5-063B5D1735C2@amberbio.com> <349A2A56-678F-446D-8E1E-D4F33AECE74C@gmail.com> Message-ID: I think there is a definite case for having some form of shared memory in erlang however I do not think that that means compromising immutability. Processes sharing memory should still only allocate immutable objects. This means that, by sharing a pointer/reference under the hood by passing it in a message, the VM will be able to decide to copy or simply pass it depending on the receiving process (if it has access to said shared memory or not (as would be the case for a remote node PID) or if the cost is prohibitive (different core on a NUMA cpu perhaps)). This also means that by the time the receiving process receives the reference, the memory has been committed and as such even if the creating process died/crashed, the memory is still valid (1+ reference still alive). This will imply a GC pass that halts all processes sharing, however, it could potentially be avoided if you are clever about it. Handling it this way would be entirely transparent to the programmer (aside from selecting which processes share with which other processes) and would not create any consistency or race issues since it would reuse erlang communication primitives (messages). You could even add a profiling counter or metrics to diagnose cases where a lot of copying is happening that could be saved through shared memory. While it is easier to break a system based on convention in C++, if the libraries are sufficiently well thought out, it has personally never been a major/common issue where I have worked (now and in the past) even with junior programmers. This is typically why code reviews are in effect in most software houses in the first place. Breaking an erlang system is significantly harder but can still be done by writing your own drivers/nifs, just as you would in C++. The tools are perhaps easier to handle or rather, less prone to unintentional mistakes in erlang's case, as it is a higher level language after all. It is true as well, that to properly design a highly concurrent system in C++, one needs to be quite experienced if not an expert which isn't necessarily the case in erlang and thanks to which I was able, as a total erlang newbie, to write up server software that has been up for months now without a hitch, something that would have taken me a great deal more time in C++. IMO, keep immutability and simply add a flag to spawn_opt and be done with it. If people have a need for it, it'll be there, if not, everything will behave as it currently does. It might even make sense to make it a configurable value in the .app file, overridable when launching the application. 2cents Nicholas On Mon, Oct 18, 2010 at 1:27 PM, James Churchman wrote: > maybe ( i stress the maybe here.. ! ) it would be possible to produce some > higher level construct (written in erlang) that acts more like a data base > transaction for multiple gen_servers or processes, where you don't "lock" > all the processes that may get used in the remaining run of code, but > instead at the higher level you specify that all processes that have > messages sent in the following run of code must not have any other messages > sent to them until this chunk of code is run. This would further abstract > you from doing your own high-locks although obviously would have to be based > on some sort of intelligent locking mechanism at some level > > this would allow basic code like : > > process1 ! message1, > process2 ! message2, > process3 ! message3; > > (with other independent code that also messages those processes) > > & also more complex logic where you don't necessarily know which processes > will get messages in a piece of running code, all with out "locks"? > > ps i am not in any way volunteering to write this, or even saying its > practically possible > > On 18 Oct 2010, at 15:23, Morten Krogh wrote: > > > Robert > > > > The user is not doing any C level locking in Erlang, but the user will > lock at a higher level, the process message passing level. > > And that creates the same issues again with deadlocks etc. > > > > Morten. > > > > > > > > > > On Oct 18, 2010, at 3:52 PM, Robert Virding wrote: > > > >> I am bit late getting into this discussion but I have been away without > good network connectivity. > >> > >> Of course you cannot have any form of useful concurrency without locks, > share memory, etc at some level. There is however, one fundamental > difference between doing it in C or in Erlang and that is *WHO* actually > does all the locking, sharing, etc and *WHO* actually enforces the allowed > access between processes. In C it is the user, either directly or through > libraries, but it is only enforced by convention, there is nothing really > *stopping* me from writing to any address I wish. In Erlang all locking, > sharing etc is handled by the implementation and strictly enforced by having > message passing as the only way of process communication for users. This is > probably easier to implement as well, but it is by no means trivial to get > it correct and efficient, as there is only a limited number of ways > processes can communicate with each other. > >> > >> This is also a very powerful argument for immutable data. It is, of > course, possible, and safe, to have mutable data and enforce message passing > by copying (this is almost swearing in the church of Erlang) but it > definitely limits the implementation. Data *must* be copied as it is > mutable. With immutable data the implementation can itself decide how > messages are actually "sent" between processes: either by copying as is done > in the BEAM; or by having shared memory between processes; or by a > combination of both. The user will not have to worry about it and not see > the difference, except perhaps by performance. > >> > >> This also means that implementing a suggestion from earlier in this > thread about adding threads becomes easier to implement and transparent to > the user, just because data is immutable. It can in one sense be taken as a > hint to the system as to how these processes are intended to interact with > each other. With mutable data this is impossible. > >> > >> One quick comment on the original question about shared/hybrid heaps. > Unfortunately, there is no "best" way of doing heaps: some applications will > work better with shared heaps; some will work better with copying; and for > many the difference will be negligible. For example having shared large > binaries works well for some aplications but can cause others to crash. Also > SMP's and multiple cores greatly increase the difficulty of shared heap > implementation because of just problems with locking and synchronising > memory access. > >> > >> Robert > >> > >> > >> ----- "Morten Krogh" wrote: > >> > >>> Ulf, > >>> > >>> I actually agree with most of what you say, and most of it is not at > >>> all > >>> in disagreement with my previous posts. > >>> > >>> I was talking about issues like concurrency, message passing, locks > >>> from > >>> a philosophical point of view, and > >>> I don't agree with the way Erlang is often presented as a way to have > >>> > >>> high concurrency without locks. > >>> I don't agree with the statement that shared memory is bad. > >>> > >>> You are talking about the Erlang platform, and its productivity. I > >>> like > >>> Erlang. I find it very easy to write programs, and > >>> have the processes talk to each other, monitor them etc. My > >>> philosophical rants were not against the language or platform. > >>> > >>>> > >>>> With all due respect, your argument is one that I've heard many > >>> times > >>>> in different settings, and it misses a key point (which Jachym > >>> brought > >>>> up). > >>>> My experience from working in development of systems with complex > >>>> concurrency patterns is that this argument breaks down completely > >>>> and horribly expensively over time in larger projects. > >>>> > >>> > >>> The argument breaks down? You mean that it becomes hard to write and > >>> maintain code? But I > >>> haven't made claims about that. I find Erlang programs much easier > >>> than > >>> C programs. > >>> > >>> > >>>> Saying that there are no fundamental differences between Erlang and > >>> C > >>>> essentially means that you claim that "correct by design" and > >>> "correct by > >>>> convention" are the same thing. By this token, Erlang and Haskell > >>> are > >>>> fundamentally the same too, since it really doesn't matter that the > >>> > >>>> Haskell > >>>> compiler can verify many more aspects of a program at compile-time > >>> than > >>>> Erlang can - you can simply avoid making type mistakes in Erlang > >>> and > >>>> be done with it! > >>> There are no fundamental difference between Erlang and C, when it > >>> comes to > >>> the concepts of concurrency, locks, deadlocks, shared memory. > >>> The only qualitative difference is between a sequential program and > >>> everything else. > >>> > >>> There are huge differences in actually implementing things, and the > >>> syntax, of course. > >>> > >>>> Your line of reasoning also suggests that we can put great faith in > >>>> micro-benchmarks, since, if you can make it work and make it really > >>>> fast in a small example in C, you can do it equally well in a larger > >>> and > >>>> more complex example. > >>>> > >>> I believe in modularity and simplicity. You need to choose the right > >>> level to work in. > >>> Erlang is just a layer on top of C. For many purposes, it is better to > >>> > >>> use Erlang than C. > >>> For other purposes, it is better to use electric currents in nand > >>> gates. > >>> Of course, you need higher level apis. > >>> Ulf, I agree totally with you. > >>> > >>>> The place where Erlang and C become very different is when you > >>> start > >>>> writing code that has "interesting" complexity. Automatic memory > >>>> management may seem like a complete waste, until your program > >>>> becomes sufficiently complex that you can no longer easily keep > >>> track > >>>> of where you should lock, allocate and free memory. At this point, > >>> it is > >>>> actually very difficult to outperform a good garbage collector with > >>> hand- > >>>> written code. > >>>> > >>> I agree. > >>> > >>> And you see what the next higher level will be. Automatic process > >>> collection, when a process can never be reached anymore. > >>> In 20 years, people will have "gigaprocesses", the older folks will > >>> remember that there was something called "gigabytes", and the next > >>> languages will have automatic > >>> process spawning and process collection. Erlang will be a low level > >>> language. What will such a unit of processes be called? A brain :) > >>> > >>> The brain languages will still have deadlocks, shared memory, locks > >>> etc, > >>> because those are philosophical/mathematical entities that are never > >>> "solved". > >>> > >>> > >>> > >>> > >>> > >>>> I've seen lots of real-world examples where > >>>> small teams of expert C++ programmers have managed to convince > >>>> management to go with a well-trimmed, elegant execution model in > >>>> C++, only to run in all sorts of problems when lesser programmers > >>> were > >>>> to start adding features - huge dips in performance and stability, > >>> raving > >>>> memory leaks, and frantic running around and fire fighting for the > >>>> experts, who are the only ones who seem to manage to stick to the > >>>> conventions and not break stuff. The beginning mantra, "how hard > >>> can > >>>> it be?", is later answered with "apparently, endlessly hard". > >>>> > >>> > >>> Interesting. By the way, are there examples of Erlang teams that have > >>> > >>> switched away from Erlang? > >>> > >>>> And we haven't even discussed the problem of one of these processes > >>>> crashing yet. With two concurrent contexts explicitly writing to the > >>> same > >>>> block of memory, ensuring the integrity of that memory block if one > >>> of > >>>> the contexts throws an exception will be very, very hard. This is > >>> where > >>>> Erlang's "share nothing" philosophy really pays off. The VM may > >>> decide > >>>> to share data under the hood, but as a programmer, you can be > >>> absolutely > >>>> certain that the death of another process - even in the same memory > >>> space > >>>> - cannot ruin the integrity of your own data. It really is as if you > >>> > >>>> had your own > >>>> distinct copies of everything. > >>>> > >>> This is where we disagree strongly. I am saying that you get the same > >>> > >>> problems again just at a higher level. > >>> The equivalent situation is this. > >>> P1 is a process, that keeps data, i.e. the so called shared memory. > >>> P2 and P3 sends messages to P1, many each. Remember a put message is > >>> like a mutation. > >>> If you want to serialize the updates from P2 and P3, a lock is needed. > >>> > >>> P1 for instance could wait for all messages from P2, before receiving > >>> > >>> from P3, > >>> incorporate them, release its lock and accept the messages from P3. > >>> What > >>> should P1 do ,if the messages from P2 stop early. Roll back, ignore > >>> it. > >>> The issues are exactly the same as at the lower byte level. Don't you > >>> > >>> see that these are deep principles, not something a language/VM can > >>> solve? > >>> > >>> Memory space in Erlang is not the same as memory space for C. > >>> Everything > >>> is lifted one level. The equivalent of memory space in Erlang is the > >>> set > >>> of running processes. > >>> The addresses are now Pids not RAM addresses. And if you want > >>> something > >>> interesting to happen, you must have some processes that accept > >>> messages > >>> and act on them. > >>> C address space does not exist for the Erlang programmer as little as > >>> > >>> physical positions of transistors exist for the C program. A C > >>> program > >>> does not crash if a transistor goes bad if > >>> there is a "virtualization layer" in between that redirects the > >>> current > >>> to a another transistor. > >>> > >>> > >>>> The problem of crashing processes is a particularly nasty > >>> _containment_ > >>>> problem, but when you scrutinise many of the concurrency models out > >>>> there you soon find that there are usually many issues that are not > >>> well > >>>> contained. Encapsulation and containment are best handled in the > >>>> programming model, rather than relying on convention, and this is > >>>> exactly the area where there is a fundamental difference. > >>>> > >>> Ulf, I agree that one should have higher level languages that > >>> encapsulate things, and maybe JAVA or C++ or whatever you are thinking > >>> > >>> about are too low level > >>> for big projects. Fine with me. > >>> > >>> On practical terms, I feel that Erlang some times gives up performance > >>> > >>> for unnecessary reasons. I am especially thinking about mutability. > >>> There is no reason to give up mutation within a process which is > >>> already > >>> a serializer. > >>> > >>> code like this > >>> > >>> loop(Dict) > >>> receive > >>> {put, K, V} -> > >>> Dict2 = dict:store(K, V, Dict), > >>> loop(Dict2) > >>> end. > >>> > >>> > >>> is just a performance wise inefficient way of doing > >>> > >>> loop(Mut) -> > >>> receive > >>> {put, K, V} -> > >>> put(K,V, Mut), > >>> loop(Mut) > >>> end. > >>> > >>> where Mut is a mutable data structure. > >>> In the dict exampe, there is a unncecessary work and garbage > >>> collection > >>> for the functional data structure Dict. > >>> And you gain nothing, since it is sequential code. > >>> > >>> ETS gives you an unnecessary copy. The process dictionary does the > >>> job, > >>> but there is only one of them and it must be a hash table. It would be > >>> > >>> better with > >>> PD1 = pd:new_hash_table(), > >>> PD2 = pd:new_tree(), > >>> etc. > >>> > >>> > >>> > >>>> E.F. Codd, who wrote the twelve (13, really) rules for relational > >>>> databases, had > >>>> this rule as No 12: > >>>> > >>>> Rule 12: The nonsubversion rule: > >>>> *If the system provides a low-level (record-at-a-time) interface, > >>> then > >>>> that interface * > >>>> *cannot be used to subvert the system, for example, bypassing a > >>>> relational security * > >>>> *or integrity constraint.* > >>>> (http://en.wikipedia.org/wiki/Codd's_12_rules > >>>> ) > >>>> > >>>> The importance of this rule is often underestimated, and it is quite > >>> > >>>> common to > >>>> ditch it early in favour of performance, since the consequences of > >>>> subverting > >>>> the consistency model are not readily apparent until much later, if > >>> > >>>> even then. > >>>> > >>> > >>> Interesting. I am not convinced. So basically, the programmer should > >>> be > >>> constrained, because he/she will make mistakes? > >>> I guess it is a question of finding the right balance. So, you think > >>> dirty_write was a mistake for mnesia? > >>> > >>> cheers, > >>> > >>> Morten. > >>> > >>> > >>> > >>> > >>> > >>>> BR, > >>>> Ulf W > >>>> > >>>> On 18 Oct 2010, at 09:31, Morten Krogh wrote: > >>>> > >>>>> Hi Jachym > >>>>> > >>>>> There is one thing that I probably haven't made clear enough, and > >>> it > >>>>> will answer all points basically. > >>>>> > >>>>> Erlang has the loop receive loop process that serializes all access > >>> > >>>>> to the data. > >>>>> The equivalent in C is a single byte. The semantics of > >>> communicating > >>>>> with a single byre is well defined in C. > >>>>> A C thread communicates with the byte using message passing through > >>> > >>>>> the memory bus. The byte only allows get and put. > >>>>> Nothing else makes sense. > >>>>> > >>>>> The general picture is that you have some units of atomicity and > >>>>> beyond that you need to lock, coordinate access etc. > >>>>> > >>>>> Erlang gives you a higher control of the granularity than C, but > >>>>> nothing is fundamentally different. I just object to all the Erlang > >>> > >>>>> statements about locks, concurrency etc. > >>>>> > >>>>> If you want to get rid of locks and shared memory, you lose > >>>>> concurrency. You can't have both. You can serialize everything. In > >>> C, > >>>>> that would be one thread. In Erlang, > >>>>> one process. If you want concurrency, you need smaller blocks of > >>>>> data, which then needs to be coordinated. > >>>>> > >>>>> You can imagine an axis > >>>>> > >>>>> concurrency ~ locks, deadlocks etc ~ high performance in one > >>> end. > >>>>> > >>>>> serialized access ~ no locks ~ single threaded ~ simple programing > >>> > >>>>> model in the other end. > >>>>> > >>>>> > >>>>> C actually beats Erlang in both ends of the axis (not the > >>> programming > >>>>> model, of course, Erlang syntax is always simpler). > >>>>> > >>>>> Erlang's sweet spot is in the middle. It is so easy to package an > >>>>> amount of data into a serialized agent with controller logic. > >>>>> > >>>>> The most concurrent program is not an erlang program, that would be > >>> a > >>>>> C program (or even lower) with a loop on each core, where all > >>>>> iterations of all loops updated the memory in all kinds of ways. > >>> Each > >>>>> iteration of each loop is a "context switch". > >>>>> > >>>>> Erlang is of course very convenient. You get a whole "block of > >>> bytes" > >>>>> with serialized access, and a programming language to control the > >>>>> "gate" to the block of bytes. > >>>>> > >>>>> The hardware manufacturers could make something like Erlang, if > >>> they > >>>>> wanted to. Chop the memory into blocks, and give each block > >>>>> serialized access and a small programmable controller sitting at > >>> the > >>>>> gate of that block. They would lose performance and concurrency if > >>> > >>>>> they did. > >>>>> > >>>>> It is all the same. The only question is at what granularity you > >>>>> serialize your data. > >>>>> > >>>>> Jachym, I will not go through your points. You can imagine my > >>>>> response. It is basically : > >>>>> > >>>>> One byte in C is the equivalent of one Erlang process. One byte > >>>>> doesn't need any controller logic besides get and put, and a > >>>>> guaranteed atomicity of get and put. > >>>>> > >>>>> Cheers, > >>>>> > >>>>> Morten > >>>>> > >>>>> > >>>>> > >>>>> On 10/17/10 11:46 PM, Jachym Holecek wrote: > >>>>>> # Morten Krogh 2010-10-17: > >>>>>>> The problem is of course that a process like this > >>>>>>> > >>>>>>> loop(State) -> > >>>>>>> receive > >>>>>>> {From, get, X} -> > >>>>>>> From ! {result, get(X, State)}, > >>>>>>> loop(State) > >>>>>>> > >>>>>>> etc..... > >>>>>>> > >>>>>>> Behaves exactly like shared memory seen from other processes > >>> point > >>>>>>> of view. > >>>>>>> > >>>>>>> Writing > >>>>>>> > >>>>>>> Pid ! {self(), get, 27} > >>>>>>> receive > >>>>>>> {result, Result} -> > >>>>>>> Result > >>>>>>> end. > >>>>>>> > >>>>>>> is like writing > >>>>>>> > >>>>>>> get(X, Pid) in C (the language) where Pid is pointer to a shared > >>> data > >>>>>>> structure in C. > >>>>>> "Apples and oranges" as they say. In your Erlang example: > >>>>>> > >>>>>> * Server explicitely agrees to provide some of its data. > >>>>>> * Server is free to change this decision. > >>>>>> * Client has to follow certain protocol to access this data. > >>>>>> * All accesses to "shared" data are serialized. > >>>>>> * It is "physically impossible" to obtain inconsistent snaphost > >>> of > >>>>>> the data. > >>>>>> * It is "physically impossible" to bypass server's control. > >>>>>> * This is guaranteed at language level. > >>>>>> * Your example is implemented completely within this > >>> well-defined > >>>>>> semantics. > >>>>>> > >>>>>> Contrast that with C: > >>>>>> > >>>>>> * All threads have equal access right to whole process address > >>> space. > >>>>>> * Sychronization can be done, but is a mere convention > >>>>>> (spinlocks/mutexes). > >>>>>> * Data consistency can be done, but is a mere convetion (memory > >>> > >>>>>> barriers). > >>>>>> * At language level memory access semantics is more or less > >>> undefined. > >>>>>> * Mutexes/memory barriers have to be done in assembly, ie. > >>> outside C. > >>>>>> > >>>>>> A little more fun on the C side: > >>>>>> > >>>>>> * Compiler may reorder loads/stores (somewhat) unless you're > >>> careful. > >>>>>> * CPU may reorder loads/stores (somewhat) unless you're > >>> careful. > >>>>>> * Cache may do whatever it wants to (somewhat) unless you're > >>> careful. > >>>>>> > >>>>>> Normally these thing will be taken care of by whatever threads > >>>>>> implementation > >>>>>> you use (in architecture- and compiler-specific way), but if we're > >>> > >>>>>> talking at > >>>>>> language level then there's no way (as far as I know) to achieve > >>>>>> sane shared > >>>>>> memory semantics in C alone (if we admit the existence of > >>> concurrent > >>>>>> threads > >>>>>> of execution which the language knows nothing about, of course). > >>>>>> > >>>>>> But sure -- you can _model_ Erlang-like processes in C, and you > >>> can > >>>>>> _model_ > >>>>>> C-like shared memory in Erlang (including all of the quirks above > >>> if you > >>>>>> really wanted to). But at that point you're imposing > >>> correspondence > >>>>>> of some > >>>>>> sort by brute force; as opposed to capturing inherent properties. > >>>>>> > >>>>>> Just my two pence... > >>>>>> > >>>>>> Regards, > >>>>>> -- Jachym > >>>>>> > >>>>>> ________________________________________________________________ > >>>>>> erlang-questions (at) erlang.org mailing > >>> list. > >>>>>> See http://www.erlang.org/faq.html > >>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>>>>> > >>>>> > >>>>> > >>>>> ________________________________________________________________ > >>>>> erlang-questions (at) erlang.org mailing list. > >>>>> See http://www.erlang.org/faq.html > >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>>>> > >>>> > >>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. > >>>> http://erlang-solutions.com > >>>> > >>>> > >>>> > >>> > >>> > >>> ________________________________________________________________ > >>> erlang-questions (at) erlang.org mailing list. > >>> See http://www.erlang.org/faq.html > >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mscame@REDACTED Tue Oct 19 01:27:32 2010 From: mscame@REDACTED (William Yangzhuwei) Date: Tue, 19 Oct 2010 07:27:32 +0800 Subject: [erlang-questions] When will print "*** Terminating erlang" In-Reply-To: <20101018140735.GA10291@ecn.lan> References: <20101018140735.GA10291@ecn.lan> Message-ID: Thanks you guys, Finally,I found that it because erl process receive the suspend signal. I use -noshell to solves it. Really thanks you guys ? 2010-10-18 ??10:07?"Michael Santos" ??? > On Mon, Oct 18, 2010 at 02:01:01PM +0800, William Yangzhuwei wrote: >> hi there, >> >> I can start one erlang program with ` start_test.sh `(start_test.sh is just >> a script to start erl and passing some args). >> >> >> When I use `start_test.sh &` to let this job run in background, it cannot >> finish executing. >> >> I get some output like >> >> (test1@REDACTED)1> *** Terminating erlang (test1@REDACTED). >> >> I guess *** Terminating erlang is printed by otp not in this erlang program. >> >> How can I trace this trouble and any logging? > > Try passing the -detached or -noshell flags to erl. > From jameschurchman@REDACTED Tue Oct 19 02:59:53 2010 From: jameschurchman@REDACTED (James Churchman) Date: Tue, 19 Oct 2010 01:59:53 +0100 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: References: <1311495418.107811287409932041.JavaMail.root@zimbra> <0D96B9B0-B1BB-4B16-ACC5-063B5D1735C2@amberbio.com> <349A2A56-678F-446D-8E1E-D4F33AECE74C@gmail.com> Message-ID: <3E250BC0-F007-4C61-B750-942FD53A6BFA@gmail.com> going back to the original posts.. Kresten points out that Erjang uses a shared heap and copies all messages (and i imagine that the java virtual machine handles the reference counting etc.. ) so maybe a different strategy than altering the erlang vm heavily and (minorly) polluting the syntax with speed effecting only options (that may actually only make a few % difference in most cases) just have some rules on what VM to pick : 1) for : a) native code compilation with hipe b) very high number of processes "sharing" small messages c) "side-efect free timing" (eg soft realtime) Pick ErLang runtime 2) for : a) access to prebuilt jvm packages etc.. with no rpc serialisation timing and processing overhead b) lower number of processes passing huge chunks for data Pick ErJang runtime (when heavily proven and tested etc.. :-) Is it not just that simple? ( and if not and you Really have the performance need to outstrip either on their own? then "mix and match".. i heard that erlang networks quite effectively!!) On 18 Oct 2010, at 23:33, Nicholas Frechette wrote: > I think there is a definite case for having some form of shared memory in erlang however I do not think that that means compromising immutability. > Processes sharing memory should still only allocate immutable objects. This means that, by sharing a pointer/reference under the hood by passing it in a message, the VM will be able to decide to copy or simply pass it depending on the receiving process (if it has access to said shared memory or not (as would be the case for a remote node PID) or if the cost is prohibitive (different core on a NUMA cpu perhaps)). This also means that by the time the receiving process receives the reference, the memory has been committed and as such even if the creating process died/crashed, the memory is still valid (1+ reference still alive). This will imply a GC pass that halts all processes sharing, however, it could potentially be avoided if you are clever about it. Handling it this way would be entirely transparent to the programmer (aside from selecting which processes share with which other processes) and would not create any consistency or race issues since it would reuse erlang communication primitives (messages). You could even add a profiling counter or metrics to diagnose cases where a lot of copying is happening that could be saved through shared memory. > > While it is easier to break a system based on convention in C++, if the libraries are sufficiently well thought out, it has personally never been a major/common issue where I have worked (now and in the past) even with junior programmers. This is typically why code reviews are in effect in most software houses in the first place. Breaking an erlang system is significantly harder but can still be done by writing your own drivers/nifs, just as you would in C++. The tools are perhaps easier to handle or rather, less prone to unintentional mistakes in erlang's case, as it is a higher level language after all. It is true as well, that to properly design a highly concurrent system in C++, one needs to be quite experienced if not an expert which isn't necessarily the case in erlang and thanks to which I was able, as a total erlang newbie, to write up server software that has been up for months now without a hitch, something that would have taken me a great deal more time in C++. > > IMO, keep immutability and simply add a flag to spawn_opt and be done with it. If people have a need for it, it'll be there, if not, everything will behave as it currently does. > It might even make sense to make it a configurable value in the .app file, overridable when launching the application. > > 2cents > Nicholas > > On Mon, Oct 18, 2010 at 1:27 PM, James Churchman wrote: > maybe ( i stress the maybe here.. ! ) it would be possible to produce some higher level construct (written in erlang) that acts more like a data base transaction for multiple gen_servers or processes, where you don't "lock" all the processes that may get used in the remaining run of code, but instead at the higher level you specify that all processes that have messages sent in the following run of code must not have any other messages sent to them until this chunk of code is run. This would further abstract you from doing your own high-locks although obviously would have to be based on some sort of intelligent locking mechanism at some level > > this would allow basic code like : > > process1 ! message1, > process2 ! message2, > process3 ! message3; > > (with other independent code that also messages those processes) > > & also more complex logic where you don't necessarily know which processes will get messages in a piece of running code, all with out "locks"? > > ps i am not in any way volunteering to write this, or even saying its practically possible > > On 18 Oct 2010, at 15:23, Morten Krogh wrote: > > > Robert > > > > The user is not doing any C level locking in Erlang, but the user will lock at a higher level, the process message passing level. > > And that creates the same issues again with deadlocks etc. > > > > Morten. > > > > > > > > > > On Oct 18, 2010, at 3:52 PM, Robert Virding wrote: > > > >> I am bit late getting into this discussion but I have been away without good network connectivity. > >> > >> Of course you cannot have any form of useful concurrency without locks, share memory, etc at some level. There is however, one fundamental difference between doing it in C or in Erlang and that is *WHO* actually does all the locking, sharing, etc and *WHO* actually enforces the allowed access between processes. In C it is the user, either directly or through libraries, but it is only enforced by convention, there is nothing really *stopping* me from writing to any address I wish. In Erlang all locking, sharing etc is handled by the implementation and strictly enforced by having message passing as the only way of process communication for users. This is probably easier to implement as well, but it is by no means trivial to get it correct and efficient, as there is only a limited number of ways processes can communicate with each other. > >> > >> This is also a very powerful argument for immutable data. It is, of course, possible, and safe, to have mutable data and enforce message passing by copying (this is almost swearing in the church of Erlang) but it definitely limits the implementation. Data *must* be copied as it is mutable. With immutable data the implementation can itself decide how messages are actually "sent" between processes: either by copying as is done in the BEAM; or by having shared memory between processes; or by a combination of both. The user will not have to worry about it and not see the difference, except perhaps by performance. > >> > >> This also means that implementing a suggestion from earlier in this thread about adding threads becomes easier to implement and transparent to the user, just because data is immutable. It can in one sense be taken as a hint to the system as to how these processes are intended to interact with each other. With mutable data this is impossible. > >> > >> One quick comment on the original question about shared/hybrid heaps. Unfortunately, there is no "best" way of doing heaps: some applications will work better with shared heaps; some will work better with copying; and for many the difference will be negligible. For example having shared large binaries works well for some aplications but can cause others to crash. Also SMP's and multiple cores greatly increase the difficulty of shared heap implementation because of just problems with locking and synchronising memory access. > >> > >> Robert > >> > >> > >> ----- "Morten Krogh" wrote: > >> > >>> Ulf, > >>> > >>> I actually agree with most of what you say, and most of it is not at > >>> all > >>> in disagreement with my previous posts. > >>> > >>> I was talking about issues like concurrency, message passing, locks > >>> from > >>> a philosophical point of view, and > >>> I don't agree with the way Erlang is often presented as a way to have > >>> > >>> high concurrency without locks. > >>> I don't agree with the statement that shared memory is bad. > >>> > >>> You are talking about the Erlang platform, and its productivity. I > >>> like > >>> Erlang. I find it very easy to write programs, and > >>> have the processes talk to each other, monitor them etc. My > >>> philosophical rants were not against the language or platform. > >>> > >>>> > >>>> With all due respect, your argument is one that I've heard many > >>> times > >>>> in different settings, and it misses a key point (which Jachym > >>> brought > >>>> up). > >>>> My experience from working in development of systems with complex > >>>> concurrency patterns is that this argument breaks down completely > >>>> and horribly expensively over time in larger projects. > >>>> > >>> > >>> The argument breaks down? You mean that it becomes hard to write and > >>> maintain code? But I > >>> haven't made claims about that. I find Erlang programs much easier > >>> than > >>> C programs. > >>> > >>> > >>>> Saying that there are no fundamental differences between Erlang and > >>> C > >>>> essentially means that you claim that "correct by design" and > >>> "correct by > >>>> convention" are the same thing. By this token, Erlang and Haskell > >>> are > >>>> fundamentally the same too, since it really doesn't matter that the > >>> > >>>> Haskell > >>>> compiler can verify many more aspects of a program at compile-time > >>> than > >>>> Erlang can - you can simply avoid making type mistakes in Erlang > >>> and > >>>> be done with it! > >>> There are no fundamental difference between Erlang and C, when it > >>> comes to > >>> the concepts of concurrency, locks, deadlocks, shared memory. > >>> The only qualitative difference is between a sequential program and > >>> everything else. > >>> > >>> There are huge differences in actually implementing things, and the > >>> syntax, of course. > >>> > >>>> Your line of reasoning also suggests that we can put great faith in > >>>> micro-benchmarks, since, if you can make it work and make it really > >>>> fast in a small example in C, you can do it equally well in a larger > >>> and > >>>> more complex example. > >>>> > >>> I believe in modularity and simplicity. You need to choose the right > >>> level to work in. > >>> Erlang is just a layer on top of C. For many purposes, it is better to > >>> > >>> use Erlang than C. > >>> For other purposes, it is better to use electric currents in nand > >>> gates. > >>> Of course, you need higher level apis. > >>> Ulf, I agree totally with you. > >>> > >>>> The place where Erlang and C become very different is when you > >>> start > >>>> writing code that has "interesting" complexity. Automatic memory > >>>> management may seem like a complete waste, until your program > >>>> becomes sufficiently complex that you can no longer easily keep > >>> track > >>>> of where you should lock, allocate and free memory. At this point, > >>> it is > >>>> actually very difficult to outperform a good garbage collector with > >>> hand- > >>>> written code. > >>>> > >>> I agree. > >>> > >>> And you see what the next higher level will be. Automatic process > >>> collection, when a process can never be reached anymore. > >>> In 20 years, people will have "gigaprocesses", the older folks will > >>> remember that there was something called "gigabytes", and the next > >>> languages will have automatic > >>> process spawning and process collection. Erlang will be a low level > >>> language. What will such a unit of processes be called? A brain :) > >>> > >>> The brain languages will still have deadlocks, shared memory, locks > >>> etc, > >>> because those are philosophical/mathematical entities that are never > >>> "solved". > >>> > >>> > >>> > >>> > >>> > >>>> I've seen lots of real-world examples where > >>>> small teams of expert C++ programmers have managed to convince > >>>> management to go with a well-trimmed, elegant execution model in > >>>> C++, only to run in all sorts of problems when lesser programmers > >>> were > >>>> to start adding features - huge dips in performance and stability, > >>> raving > >>>> memory leaks, and frantic running around and fire fighting for the > >>>> experts, who are the only ones who seem to manage to stick to the > >>>> conventions and not break stuff. The beginning mantra, "how hard > >>> can > >>>> it be?", is later answered with "apparently, endlessly hard". > >>>> > >>> > >>> Interesting. By the way, are there examples of Erlang teams that have > >>> > >>> switched away from Erlang? > >>> > >>>> And we haven't even discussed the problem of one of these processes > >>>> crashing yet. With two concurrent contexts explicitly writing to the > >>> same > >>>> block of memory, ensuring the integrity of that memory block if one > >>> of > >>>> the contexts throws an exception will be very, very hard. This is > >>> where > >>>> Erlang's "share nothing" philosophy really pays off. The VM may > >>> decide > >>>> to share data under the hood, but as a programmer, you can be > >>> absolutely > >>>> certain that the death of another process - even in the same memory > >>> space > >>>> - cannot ruin the integrity of your own data. It really is as if you > >>> > >>>> had your own > >>>> distinct copies of everything. > >>>> > >>> This is where we disagree strongly. I am saying that you get the same > >>> > >>> problems again just at a higher level. > >>> The equivalent situation is this. > >>> P1 is a process, that keeps data, i.e. the so called shared memory. > >>> P2 and P3 sends messages to P1, many each. Remember a put message is > >>> like a mutation. > >>> If you want to serialize the updates from P2 and P3, a lock is needed. > >>> > >>> P1 for instance could wait for all messages from P2, before receiving > >>> > >>> from P3, > >>> incorporate them, release its lock and accept the messages from P3. > >>> What > >>> should P1 do ,if the messages from P2 stop early. Roll back, ignore > >>> it. > >>> The issues are exactly the same as at the lower byte level. Don't you > >>> > >>> see that these are deep principles, not something a language/VM can > >>> solve? > >>> > >>> Memory space in Erlang is not the same as memory space for C. > >>> Everything > >>> is lifted one level. The equivalent of memory space in Erlang is the > >>> set > >>> of running processes. > >>> The addresses are now Pids not RAM addresses. And if you want > >>> something > >>> interesting to happen, you must have some processes that accept > >>> messages > >>> and act on them. > >>> C address space does not exist for the Erlang programmer as little as > >>> > >>> physical positions of transistors exist for the C program. A C > >>> program > >>> does not crash if a transistor goes bad if > >>> there is a "virtualization layer" in between that redirects the > >>> current > >>> to a another transistor. > >>> > >>> > >>>> The problem of crashing processes is a particularly nasty > >>> _containment_ > >>>> problem, but when you scrutinise many of the concurrency models out > >>>> there you soon find that there are usually many issues that are not > >>> well > >>>> contained. Encapsulation and containment are best handled in the > >>>> programming model, rather than relying on convention, and this is > >>>> exactly the area where there is a fundamental difference. > >>>> > >>> Ulf, I agree that one should have higher level languages that > >>> encapsulate things, and maybe JAVA or C++ or whatever you are thinking > >>> > >>> about are too low level > >>> for big projects. Fine with me. > >>> > >>> On practical terms, I feel that Erlang some times gives up performance > >>> > >>> for unnecessary reasons. I am especially thinking about mutability. > >>> There is no reason to give up mutation within a process which is > >>> already > >>> a serializer. > >>> > >>> code like this > >>> > >>> loop(Dict) > >>> receive > >>> {put, K, V} -> > >>> Dict2 = dict:store(K, V, Dict), > >>> loop(Dict2) > >>> end. > >>> > >>> > >>> is just a performance wise inefficient way of doing > >>> > >>> loop(Mut) -> > >>> receive > >>> {put, K, V} -> > >>> put(K,V, Mut), > >>> loop(Mut) > >>> end. > >>> > >>> where Mut is a mutable data structure. > >>> In the dict exampe, there is a unncecessary work and garbage > >>> collection > >>> for the functional data structure Dict. > >>> And you gain nothing, since it is sequential code. > >>> > >>> ETS gives you an unnecessary copy. The process dictionary does the > >>> job, > >>> but there is only one of them and it must be a hash table. It would be > >>> > >>> better with > >>> PD1 = pd:new_hash_table(), > >>> PD2 = pd:new_tree(), > >>> etc. > >>> > >>> > >>> > >>>> E.F. Codd, who wrote the twelve (13, really) rules for relational > >>>> databases, had > >>>> this rule as No 12: > >>>> > >>>> Rule 12: The nonsubversion rule: > >>>> *If the system provides a low-level (record-at-a-time) interface, > >>> then > >>>> that interface * > >>>> *cannot be used to subvert the system, for example, bypassing a > >>>> relational security * > >>>> *or integrity constraint.* > >>>> (http://en.wikipedia.org/wiki/Codd's_12_rules > >>>> ) > >>>> > >>>> The importance of this rule is often underestimated, and it is quite > >>> > >>>> common to > >>>> ditch it early in favour of performance, since the consequences of > >>>> subverting > >>>> the consistency model are not readily apparent until much later, if > >>> > >>>> even then. > >>>> > >>> > >>> Interesting. I am not convinced. So basically, the programmer should > >>> be > >>> constrained, because he/she will make mistakes? > >>> I guess it is a question of finding the right balance. So, you think > >>> dirty_write was a mistake for mnesia? > >>> > >>> cheers, > >>> > >>> Morten. > >>> > >>> > >>> > >>> > >>> > >>>> BR, > >>>> Ulf W > >>>> > >>>> On 18 Oct 2010, at 09:31, Morten Krogh wrote: > >>>> > >>>>> Hi Jachym > >>>>> > >>>>> There is one thing that I probably haven't made clear enough, and > >>> it > >>>>> will answer all points basically. > >>>>> > >>>>> Erlang has the loop receive loop process that serializes all access > >>> > >>>>> to the data. > >>>>> The equivalent in C is a single byte. The semantics of > >>> communicating > >>>>> with a single byre is well defined in C. > >>>>> A C thread communicates with the byte using message passing through > >>> > >>>>> the memory bus. The byte only allows get and put. > >>>>> Nothing else makes sense. > >>>>> > >>>>> The general picture is that you have some units of atomicity and > >>>>> beyond that you need to lock, coordinate access etc. > >>>>> > >>>>> Erlang gives you a higher control of the granularity than C, but > >>>>> nothing is fundamentally different. I just object to all the Erlang > >>> > >>>>> statements about locks, concurrency etc. > >>>>> > >>>>> If you want to get rid of locks and shared memory, you lose > >>>>> concurrency. You can't have both. You can serialize everything. In > >>> C, > >>>>> that would be one thread. In Erlang, > >>>>> one process. If you want concurrency, you need smaller blocks of > >>>>> data, which then needs to be coordinated. > >>>>> > >>>>> You can imagine an axis > >>>>> > >>>>> concurrency ~ locks, deadlocks etc ~ high performance in one > >>> end. > >>>>> > >>>>> serialized access ~ no locks ~ single threaded ~ simple programing > >>> > >>>>> model in the other end. > >>>>> > >>>>> > >>>>> C actually beats Erlang in both ends of the axis (not the > >>> programming > >>>>> model, of course, Erlang syntax is always simpler). > >>>>> > >>>>> Erlang's sweet spot is in the middle. It is so easy to package an > >>>>> amount of data into a serialized agent with controller logic. > >>>>> > >>>>> The most concurrent program is not an erlang program, that would be > >>> a > >>>>> C program (or even lower) with a loop on each core, where all > >>>>> iterations of all loops updated the memory in all kinds of ways. > >>> Each > >>>>> iteration of each loop is a "context switch". > >>>>> > >>>>> Erlang is of course very convenient. You get a whole "block of > >>> bytes" > >>>>> with serialized access, and a programming language to control the > >>>>> "gate" to the block of bytes. > >>>>> > >>>>> The hardware manufacturers could make something like Erlang, if > >>> they > >>>>> wanted to. Chop the memory into blocks, and give each block > >>>>> serialized access and a small programmable controller sitting at > >>> the > >>>>> gate of that block. They would lose performance and concurrency if > >>> > >>>>> they did. > >>>>> > >>>>> It is all the same. The only question is at what granularity you > >>>>> serialize your data. > >>>>> > >>>>> Jachym, I will not go through your points. You can imagine my > >>>>> response. It is basically : > >>>>> > >>>>> One byte in C is the equivalent of one Erlang process. One byte > >>>>> doesn't need any controller logic besides get and put, and a > >>>>> guaranteed atomicity of get and put. > >>>>> > >>>>> Cheers, > >>>>> > >>>>> Morten > >>>>> > >>>>> > >>>>> > >>>>> On 10/17/10 11:46 PM, Jachym Holecek wrote: > >>>>>> # Morten Krogh 2010-10-17: > >>>>>>> The problem is of course that a process like this > >>>>>>> > >>>>>>> loop(State) -> > >>>>>>> receive > >>>>>>> {From, get, X} -> > >>>>>>> From ! {result, get(X, State)}, > >>>>>>> loop(State) > >>>>>>> > >>>>>>> etc..... > >>>>>>> > >>>>>>> Behaves exactly like shared memory seen from other processes > >>> point > >>>>>>> of view. > >>>>>>> > >>>>>>> Writing > >>>>>>> > >>>>>>> Pid ! {self(), get, 27} > >>>>>>> receive > >>>>>>> {result, Result} -> > >>>>>>> Result > >>>>>>> end. > >>>>>>> > >>>>>>> is like writing > >>>>>>> > >>>>>>> get(X, Pid) in C (the language) where Pid is pointer to a shared > >>> data > >>>>>>> structure in C. > >>>>>> "Apples and oranges" as they say. In your Erlang example: > >>>>>> > >>>>>> * Server explicitely agrees to provide some of its data. > >>>>>> * Server is free to change this decision. > >>>>>> * Client has to follow certain protocol to access this data. > >>>>>> * All accesses to "shared" data are serialized. > >>>>>> * It is "physically impossible" to obtain inconsistent snaphost > >>> of > >>>>>> the data. > >>>>>> * It is "physically impossible" to bypass server's control. > >>>>>> * This is guaranteed at language level. > >>>>>> * Your example is implemented completely within this > >>> well-defined > >>>>>> semantics. > >>>>>> > >>>>>> Contrast that with C: > >>>>>> > >>>>>> * All threads have equal access right to whole process address > >>> space. > >>>>>> * Sychronization can be done, but is a mere convention > >>>>>> (spinlocks/mutexes). > >>>>>> * Data consistency can be done, but is a mere convetion (memory > >>> > >>>>>> barriers). > >>>>>> * At language level memory access semantics is more or less > >>> undefined. > >>>>>> * Mutexes/memory barriers have to be done in assembly, ie. > >>> outside C. > >>>>>> > >>>>>> A little more fun on the C side: > >>>>>> > >>>>>> * Compiler may reorder loads/stores (somewhat) unless you're > >>> careful. > >>>>>> * CPU may reorder loads/stores (somewhat) unless you're > >>> careful. > >>>>>> * Cache may do whatever it wants to (somewhat) unless you're > >>> careful. > >>>>>> > >>>>>> Normally these thing will be taken care of by whatever threads > >>>>>> implementation > >>>>>> you use (in architecture- and compiler-specific way), but if we're > >>> > >>>>>> talking at > >>>>>> language level then there's no way (as far as I know) to achieve > >>>>>> sane shared > >>>>>> memory semantics in C alone (if we admit the existence of > >>> concurrent > >>>>>> threads > >>>>>> of execution which the language knows nothing about, of course). > >>>>>> > >>>>>> But sure -- you can _model_ Erlang-like processes in C, and you > >>> can > >>>>>> _model_ > >>>>>> C-like shared memory in Erlang (including all of the quirks above > >>> if you > >>>>>> really wanted to). But at that point you're imposing > >>> correspondence > >>>>>> of some > >>>>>> sort by brute force; as opposed to capturing inherent properties. > >>>>>> > >>>>>> Just my two pence... > >>>>>> > >>>>>> Regards, > >>>>>> -- Jachym > >>>>>> > >>>>>> ________________________________________________________________ > >>>>>> erlang-questions (at) erlang.org mailing > >>> list. > >>>>>> See http://www.erlang.org/faq.html > >>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>>>>> > >>>>> > >>>>> > >>>>> ________________________________________________________________ > >>>>> erlang-questions (at) erlang.org mailing list. > >>>>> See http://www.erlang.org/faq.html > >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>>>> > >>>> > >>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. > >>>> http://erlang-solutions.com > >>>> > >>>> > >>>> > >>> > >>> > >>> ________________________________________________________________ > >>> erlang-questions (at) erlang.org mailing list. > >>> See http://www.erlang.org/faq.html > >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mk@REDACTED Tue Oct 19 11:00:38 2010 From: mk@REDACTED (Morten Krogh) Date: Tue, 19 Oct 2010 11:00:38 +0200 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <3E250BC0-F007-4C61-B750-942FD53A6BFA@gmail.com> References: <1311495418.107811287409932041.JavaMail.root@zimbra> <0D96B9B0-B1BB-4B16-ACC5-063B5D1735C2@amberbio.com> <349A2A56-678F-446D-8E1E-D4F33AECE74C@gmail.com> <3E250BC0-F007-4C61-B750-942FD53A6BFA@gmail.com> Message-ID: <4CBD5E36.2040909@amberbio.com> Hi How does the process dictionary work when it comes to copying. Suppose we have a giant list of integers, L. Now where is the list copied? 1) put(a, L). 1) X = get(a). 2) Y = erase(a). Cheers, Morten. From mk@REDACTED Tue Oct 19 11:04:32 2010 From: mk@REDACTED (Morten Krogh) Date: Tue, 19 Oct 2010 11:04:32 +0200 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CBD5E36.2040909@amberbio.com> References: <1311495418.107811287409932041.JavaMail.root@zimbra> <0D96B9B0-B1BB-4B16-ACC5-063B5D1735C2@amberbio.com> <349A2A56-678F-446D-8E1E-D4F33AECE74C@gmail.com> <3E250BC0-F007-4C61-B750-942FD53A6BFA@gmail.com> <4CBD5E36.2040909@amberbio.com> Message-ID: <4CBD5F20.7090601@amberbio.com> Oh, sorry I guess it is never copied. On 10/19/10 11:00 AM, Morten Krogh wrote: > Hi > > How does the process dictionary work when it comes to copying. > > Suppose we have a giant list of integers, L. > > Now where is the list copied? > > 1) put(a, L). > > 1) X = get(a). > > 2) Y = erase(a). > > > Cheers, > > Morten. > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From robert.virding@REDACTED Tue Oct 19 12:05:39 2010 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 19 Oct 2010 10:05:39 +0000 (GMT) Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CBD5F20.7090601@amberbio.com> Message-ID: <1933245410.115681287482739116.JavaMail.root@zimbra> No, it's never copied. That is one of the benefits/problems with the process dictionary is that it keeps its data on the normal process heap. There is no need to copy data as with ETS but large amounts of data negatively affect GC times. Apart from the fact that it is mutable and therefore naughty to use. :-) Robert ----- "Morten Krogh" wrote: > Oh, sorry I guess it is never copied. > > > > On 10/19/10 11:00 AM, Morten Krogh wrote: > > Hi > > > > How does the process dictionary work when it comes to copying. > > > > Suppose we have a giant list of integers, L. > > > > Now where is the list copied? > > > > 1) put(a, L). > > > > 1) X = get(a). > > > > 2) Y = erase(a). > > > > > > Cheers, > > > > Morten. > > > > > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From bengt.kleberg@REDACTED Tue Oct 19 15:46:34 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 19 Oct 2010 15:46:34 +0200 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: <4CBADE65.7010909@amberbio.com> References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> <4CB6B589.9050003@cs.ntua.gr> <4CB6BDDC.3060900@amberbio.com> <4CBADE65.7010909@amberbio.com> Message-ID: <1287495994.5199.35.camel@seasc1137> Greetings, When you write that garbage collection needs all threads to stop, is that in the context of the current Erlang garbage collector? It seems to me (http://www.cs.wustl.edu/~mdeters/doc/slides/rtgc-history.pdf) that there are concurrent/parallel collectors at least for other languages. bengt On Sun, 2010-10-17 at 13:30 +0200, Morten Krogh wrote: > Nicholas, nice post. > > I agree that fully automatic garbage collection probably will need to > stop all threads at the same time. > > Actually, I still think binary sharing is an example of it. It is > restricted relative to the general case, which makes it simpler. > > Alternatively, one could accept that the shared data should survive > unless someone killed it. > Or one could just accept that the programmer should accept that garbage > collection is done in one sweep. > > > I think it is insctructive to be precise about what the problem could > be. Here is one example. > A giant immutable key value store accesed by http read requests. To get > highest performance, you would > put the data in one tree, or hash table, shared by every erlang process. > You would then spawn worker processes for each core, > and let them handle http requests. To respond, they would access the > tree, grab the data, and reply. > This is completely equivalent to one process owning the tree, and all > workers message passing into it, but performance is higher with the > "pointer access" > than through the message passing and context switching. Actually, you > can make the API exactly like message passing. There would be no > automatic garbage collection of > the tree, and it would have to be killed explicitly, exactly like a > process today. > > Shared memory and message passing to a data owner process is really the > same thing semantically. The only difference is performance. > Actually shared memory is a bad wording. It should be called "C like > pointer access to the data". > > I guess ets is made based on this logic. > > Anyway, I regret using the word thread earlier. One word, process, is > enough. The only real question is whether erlang should have a way for > one process to read state of another > process without the full send, put in message queue, context switch, > recieve, lookup data, send, put in message queue, receive overhead. > The semantics is the same. The programing model is the same. The speed > is higher. > > Cheers, > > Morten. > > > > > On 10/14/10 4:47 PM, Nicholas Frechette wrote: > > Sharing binaries is fine because they use a different heap altogether and a > > different garbage collection scheme (reference counting). > > Sharing a nif ressource implies you manage it manually through nifs, how you > > manage it is largely up to you. > > As far as other languages go, yes, most (all?) of them stop all threads to > > garbage collect (if those threads share memory). Java does this, C# does > > this, ruby, python, etc. C# recently or will shortly introduce a garbage > > collection algorithm that runs concurrently and thus doesn't stop all > > threads but from what I can remember, it isn't 100% guaranteed (if threads > > continue to allocate past a threshold, I believe they are still all stopped > > waiting for GC to complete). This an important reason why multi generational > > garbage collection algorithms are so popular: it keeps most GC cycles quick, > > thus stopping all threads for the least amount of time. > > > > The idea behind each process having its own heap is twofold: it helps memory > > locality and it isolates that heap such that it can be GCed on its own, > > without interrupting any other process. Processes are such essentially > > sandboxed in that regard. > > This also means that any two processes, or more, can run garbage collection > > concurrently without issues. > > > > While it sounds bad to stop all processes/threads sharing a piece of memory, > > in reality, for most desktop applications, it is a non issue. The story is > > different for server software however, where they tend to allocate a lot and > > have many threads. > > > > When you want to share some data between P1 and P2, if they do not share > > everything, a problem arises quite quickly: how do you mark allocations that > > should be shared and those that shouldn't? If you share everything, > > allocation is easy and goes in 1 heap (ignoring binaries here in their own > > heap). If you do not, you have to select those allocations. In both cases, > > because some memory is shared, the simplest way to GC will be to stop both > > processes. After all, you will need to inspect their respective stacks to > > see if they reference things in that shared heap and if the processes are > > running, the stack will keep changing. If even for the time of a memcpy of > > the stack, all processes will have to stop at some point or another. You > > also won't be able to run the GC concurrently on P1 and P2 if they share > > memory, or at least, it will be quite hard. > > > > Sharing memory and garbage collection is a non trivial problem to solve and > > as erlang currently does, the simplest, most elegant way to deal with the > > problem is to not share at all (and always copy) to avoid the problem. > > > > Nicholas > > > > On Thu, Oct 14, 2010 at 4:22 AM, Morten Krogh wrote: > > > >> > >>> You are missing the point -- or at least the point which I think Richard > >>> is making. In the scheme you propose, T2's execution will be influenced by a > >>> piece of data that _in principle_ is not shared. For example, T2 needs to be > >>> stopped executing or synchronized with T1 by garbage collection which might > >>> take place when process T1 allocates some big D2. > >>> > >>> How is that different than the two processes sharing everything? > >>> > >>> Kostis > >>> > >> Kostis, > >> > >> I don't understand you. Are you saying that it is impossible to implement > >> multithreading unless the threads share everything, and garbage collection > >> will be slow? > >> > >> What about two erlang processes that share a binary. Isn't that how it > >> works today? > >> What about two erlang processes that share a nif resource? > >> What about other languages and virtual machines. They have multithreading. > >> > >> We must be talking past each other here. > >> > >> By the way, I am not advocating such a change to erlang. I think you can > >> get around shared memory by choosing the right processes, partition data > >> correctly etc. > >> It is only the most computationally intensive cases where you can't, and > >> there one can use a nif resource or some external way of sharing data. > >> > >> > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From vances@REDACTED Tue Oct 19 19:51:44 2010 From: vances@REDACTED (Vance Shipley) Date: Tue, 19 Oct 2010 13:51:44 -0400 Subject: Extracting detailed information from TypEr/Dialyzer In-Reply-To: References: Message-ID: <20101019175144.GF21059@h216-235-12-174.host.egate.net> Torben, I went down this road ten years ago. At the time I was constantly diagraming my FSMs to reason about them and diagnose problems. I decided I needed to automate that process. The result: http://www.erlang.org/pipermail/erlang-questions/2001-October/003716.html Unfortunately the links to attachments in the above don't seem to work. I have attached the code I used for this however I haven't used it in many year so your mileage may vary. It was also fairly dependent on coding style. On Tue, Oct 12, 2010 at 09:00:00AM +0200, Torben Hoffmann wrote: } I want to extract the following information from a gen_fsm: } } - all the states } - the incoming events } - possible next states [...] } Has anybody been down this road before me? If so, do you have some insights } to share? -- -Vance -------------- next part -------------- %% graph_fsm.erl %% %% Author: Vance Shipley, Motivity Telecom Inc. %% Date: November, 2000 %% %% %% This library is free software; you can redistribute it and/or %% modify it under the terms of the GNU Lesser General Public %% License as published by the Free Software Foundation; either %% version 2 of the License, or (at your option) any later %% version. %% %% This library is distributed in the hope that it will be useful, %% but WITHOUT ANY WARRANTY; without even the implied warranty of %% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the %% GNU Lesser General Public License for more details. %% %% You should have received a copy of the GNU Lesser General %% Public License along with this library; if not, write to the %% Free Software Foundation, Inc., 59 Temple Place, Suite 330, %% Boston, MA 02111-1307 USA %% %% ------------------------------------------------------------------------ %% %% This module creates a graph description file, suitable for use with %% dot, describing the gen_fsm behaviour source file given. The output %% file can be fed to dot and friends to automatically create a %% postscript/gif %% picture of the state transitions within your state %% machine. %% %% Get dot at http://www.research.att.com/sw/tools/graphviz. %% %% Ouput is to a file named 'file.dot' where 'file.erl' is the source file %% %% Run the resulting graph specification file through dot: %% dot -Tgif file.dot > file.gif %% or dot -Tps file.dot > file.ps %% %% ------------------------------------------------------------------------ %% %% set tabstops to 3 (in vi :set tabstop=3) -module(graph_fsm). -vsn('1.3'). -author('vances@REDACTED'). -export([parse/1, parse/2]). % change these to set the font size in the output -define(FONT_GRAPH, 18). -define(FONT_EDGE, 10). parse(File) when atom(File) -> % construct the full filename Filename = atom_to_list(File) ++ ".erl", Includes = filename:dirname(filename:absname(Filename)), parse(Filename, [Includes]); parse(File) when list(File) -> % construct the full filename Base = filename:basename(File, ".erl"), Filename = Base ++ ".erl", Includes = filename:dirname(filename:absname(Filename)), parse(Filename, [Includes]). parse(File, Includes) when atom(File) -> % construct the full filename Filename = atom_to_list(File) ++ ".erl", parse(Filename, Includes); parse(File, Includes) -> % construct the full filename Base = filename:basename(File, ".erl"), Filename = Base ++ ".erl", {ok, Form} = epp:parse_file(Filename, Includes, []), % make sure it has gen_fsm behavior case get_attribute(Form, behaviour) of {behaviour, gen_fsm} -> true; {behaviour, _} -> exit({error, 'wrong behaviour'}); not_found -> exit({error, 'no behaviour'}) end, % get the module name {module, Module} = get_attribute(Form, module), % open a file to write the graph named after the module {ok, IoDevice} = file:open(Base ++ ".dot", write), % write the header for the graph io:fwrite(IoDevice, "digraph ~w {~n", [Module]), % set the label for the graph and it's fontsize io:fwrite(IoDevice, " label=\"~w\";~n fontsize=~w;~n", [Module, ?FONT_GRAPH]), % get the list of exported functions {export, Exports} = get_attribute(Form, export), % parse the exported functions parse_exports(Form, IoDevice, Exports), % close the graph file file:close(IoDevice), % return the base name of the file written {ok, filename:basename(Filename, ".erl")}. % find the value of a given attribute get_attribute(Form, Attribute) -> get_attribute(Form, Attribute, []). get_attribute([], Attribute, Values) -> {Attribute, Values}; get_attribute([H|T], Attribute, Values) -> % get the attribute's value case H of {attribute, _Line, Attribute, Value} -> get_attribute(T, Attribute, lists:append(Values, Value)); {eof, _Line} -> {Attribute, Values}; _ -> get_attribute(T, Attribute, Values) end. % ignore the callbacks which don't represent states parse_exports(Form, IoDevice, [{init,_}|T]) -> parse_exports(Form, IoDevice, T); parse_exports(Form, IoDevice, [{handle_event,_}|T]) -> parse_exports(Form, IoDevice, T); parse_exports(Form, IoDevice, [{handle_sync_event,_}|T]) -> parse_exports(Form, IoDevice, T); parse_exports(Form, IoDevice, [{handle_info,_}|T]) -> parse_exports(Form, IoDevice, T); parse_exports(Form, IoDevice, [{terminate,_}|T]) -> parse_exports(Form, IoDevice, T); parse_exports(Form, IoDevice, [{code_change,_}|T]) -> parse_exports(Form, IoDevice, T); % state handlers should have two or three arguments parse_exports(Form, IoDevice, [{StateName, Arity}|T]) when Arity < 4, Arity > 1 -> % get the function's Abstract Form case get_function(Form, StateName, Arity) of {StateName, Arity, Function} -> parse_function(IoDevice, StateName, Function), parse_exports(Form, IoDevice, T); not_found -> % the function doesn't exist, ignore parse_exports(Form, IoDevice, T) end; % we'll ignore anything which doesn't fit the above parse_exports(Form, IoDevice, [_H|T]) -> parse_exports(Form, IoDevice, T); % an empty list means we're done, close up shop parse_exports(_Form, IoDevice, []) -> % write out the closing stuff to the graph file io:fwrite(IoDevice, "}~n", []). % retrieve a function's AbsForm by name get_function([], _, _) -> not_found; get_function([H|T], FunctionName, Arity) -> case H of {function, _Line, FunctionName, Arity, Function} -> {FunctionName, Arity, Function}; {eof, _Line} -> not_found; _ -> get_function(T, FunctionName, Arity) end. % % parse_function(IoDevice, StateName, []) % % this clause matches if the event is bound to a variable % e.g idle(Event, StateData) -> % parse_function(IoDevice, StateName, [{clause,_Line,[{var,_Le,EventVar}|_], Guard, Body} | Clauses]) -> case find_nextstate(Body) of {var, NextStateVar} -> % state handler returns a variable, we'll have to find % out where it was defined (assuming in a case statement) case get_case(EventVar, Body) of {ok, Case} -> parse_case(IoDevice, StateName, NextStateVar, Case); not_found -> io:fwrite("A state handler [~w] clause returns a variable " "[~s] we assumed it was in a case statement but could " "not find one.~n", [StateName, NextStateVar]), {error, 'no case found'} end; {atom, NextState} -> % state handler returns a hard coded atom case parse_guard(Guard) of [] -> % there was no guard label created so this must be a % catch all for undefined events which we name "*" as % is done in SDL write_line(IoDevice, StateName, NextState, {atom,0,'*'}, []); GuardLabel -> write_line(IoDevice, StateName, NextState, [], GuardLabel) end, % we'll look for a case as well as there may be more returns case get_case(EventVar, Body) of {ok, Case} -> parse_case(IoDevice, StateName, NextState, Case); not_found -> % none found so ignore none end; not_found -> % there must be a case statement which has immediate returns case get_case(EventVar, Body) of {ok, Case} -> parse_case(IoDevice, StateName, bogosity, Case); % must not be a state handler at all, ignore not_found -> ok end end, parse_function(IoDevice, StateName, Clauses); % % this clause matches if the event is matched against a variable % e.g. idle(Event = {foo, bar}, StateData) -> % parse_function(IoDevice, StateName, [{clause, Line, [{match,_,{var,_,_EventVar},EventForm}|T], Guard, Body} | Clauses]) -> parse_function(IoDevice, StateName, [{clause, Line, [EventForm|T], Guard, Body} | Clauses]); % % this clause matches if the event is a term % e.g. idle(foo, StateData) -> % or idle({foo, bar}, StateData) -> % parse_function(IoDevice, StateName, [{clause,_Line,[EventForm|_], Guard, Body} | Clauses]) -> case find_nextstate(Body) of {atom, NextState} -> % we now know what we need to know write_line(IoDevice, StateName, NextState, EventForm, parse_guard(Guard)); not_found-> none % must not be a state handler at all, ignore end, parse_function(IoDevice, StateName, Clauses); % if the clause list is empty then we're done parse_function(_IoDevice, _StateName, []) -> ok; % any other function is not a state handler parse_function(_,_,_) -> ok. % % parse a clause guard % % no guard parse_guard([]) -> []; parse_guard([Guard]) -> io:fwrite("Guard=~p~n", [Guard]), parse_guard([], Guard). % % this case handles tests on record fields, probably StateData % e.g. idle(Event, StateData) when StateData#statedata.t3 > 0 -> % parse_guard([], [{op,_,Operator,{record_field,_,_,_,{_,_,Field}}, {_,_,Value}}|T]) -> NewLabel = io_lib:write(Field) ++ atom_to_list(Operator) ++ io_lib:write(Value), parse_guard(NewLabel, T); % % this case handles further tests on record fields % parse_guard(Label, [{op,_,Operator,{record_field,_,_,_,{_,_,Field}}, {_,_,Value}}|T]) -> NewLabel = "," ++ io_lib:write(Field) ++ atom_to_list(Operator) ++ io_lib:write(Value), parse_guard(Label ++ NewLabel, T); % % this case handles BIF guard tests on record fields, probably StateData % parse_guard([], [{call,_,{atom,_,Test}, [{record_field,_,_,_,{atom,_,Value}}]}|T]) -> NewLabel = io_lib:write(Test) ++ "(" ++ io_lib:write(Value) ++ ")", parse_guard(NewLabel, T); % % this case handles further BIF guard tests on record fields % parse_guard(Label, [{call,_,{atom,_,Test}, [{record_field,_,_,_,{atom,_,Value}}]}|T]) -> NewLabel = "," ++ io_lib:write(Test) ++ "(" ++ io_lib:write(Value) ++ ")", parse_guard(Label ++ NewLabel, T); parse_guard(Label, []) -> Label; % % this case handles further BIF guard tests on record fields % parse_guard(Label, [{call,_,{atom,_,Test}, [{record_field,_,_,_,{atom,_,Value}}]}|T]) -> NewLabel = "," ++ io_lib:write(Test) ++ "(" ++ io_lib:write(Value) ++ ")", parse_guard(Label ++ NewLabel, T); parse_guard(Label, []) -> Label. % find the case statement which operates on the passed (Event) variable get_case(EventVar, [H | T]) -> case H of {'case',_,{var,_,EventVar},Body} -> {ok, Body}; _ -> get_case(EventVar, T) end; get_case(_EventVar, []) -> not_found. % find the name of the variable which will be used for next_state find_nextstate([H|T]) -> case H of % we're looking for a gen_fsm defined return value {tuple,_,[{atom,_,next_state},{var,_,NextState},_|_]} -> {var, NextState}; {tuple,_,[{atom,_,next_state},{atom,_,NextState},_|_]} -> {atom, NextState}; {tuple,_,[{atom,_,stop},{atom,_,_NextState},_]} -> {atom, stop}; _ -> find_nextstate(T) end; find_nextstate([]) -> not_found. % cycle through all the clauses in the case statement % % this clause applies when the event is bound to a variable % which would be a catchall case which we name as "*" like in SDL parse_case(IoDevice, StateName, NextStateVar, [{clause, _, [{var, _, _EventVar}], _Guard, Body} | T]) -> case get_match(IoDevice, StateName, {atom,0,'*'}, NextStateVar, Body) of ok -> parse_case(IoDevice, StateName, NextStateVar, T); not_found -> {error, 'no case found'} end; % in this clause we catch atoms being matched against the event % which should be the normal case and represent the event name parse_case(IoDevice, StateName, NextStateVar, [{clause, _, [EventForm], _Guard, Body} | T]) -> case get_match(IoDevice, StateName, EventForm, NextStateVar, Body) of ok -> parse_case(IoDevice, StateName, NextStateVar, T); not_found -> {error, 'no case found'} end; parse_case(_IoDevice, _StateName, _NextStateVar, []) -> ok. % find the place where the specified (NextStateVar) variable is assigned get_match(IoDevice, StateName, EventForm, NextStateVar, [H|T]) -> case H of % we previously determined what variable name is used in the % return from this state handler so we will look to see where % it is bound {match, _, {var, _, NextStateVar}, {atom, _, NextState}} -> % ... and finally we do the real work! write_line(IoDevice, StateName, NextState, EventForm, []); {tuple,_,[{atom,_,next_state},{atom,_,NextState},_|_]} -> % hmmm ... they didn't use the variable after all write_line(IoDevice, StateName, NextState, EventForm, []); _ -> get_match(IoDevice, StateName, EventForm, NextStateVar, T) end; get_match(_IoDevice, _StateName, _EventForm, _NextStateVar, []) -> not_found. % write out the spec line to the file write_line(IoDevice, StateName, NextState, {atom,_,'*'}, []) -> io:fwrite(IoDevice, " ~w -> ~w [label=\"*\", fontsize=~w];~n", [StateName, NextState, ?FONT_EDGE]); write_line(IoDevice, StateName, NextState, {atom,_,'*'}, Guard) -> io:fwrite(IoDevice, " ~w -> ~w [label=\"*\\n[~s]\", fontsize=~w];~n", [StateName, NextState, Guard, ?FONT_EDGE]); write_line(IoDevice, StateName, NextState, EventForm, []) -> io:fwrite(IoDevice, " ~w -> ~w [label=\"~w\", fontsize=~w];~n", [StateName, NextState, normalize(EventForm), ?FONT_EDGE]); write_line(IoDevice, StateName, NextState, EventForm, Guard) -> io:fwrite(IoDevice, " ~w -> ~w [label=\"~w\\n[~s]\", fontsize=~w];~n", [StateName, NextState, normalize(EventForm), Guard, ?FONT_EDGE]). normalize({tuple,_,Tuple}) -> list_to_tuple(normalize(Tuple)); normalize(AbsTerm = {bin,_,_}) -> list_to_atom(lists:flatten(erl_pp:expr(AbsTerm))); normalize({_,_,Term}) -> Term; normalize([H|T]) -> [normalize(H)|normalize(T)]; normalize([]) -> []. From erlang@REDACTED Tue Oct 19 20:07:59 2010 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 19 Oct 2010 20:07:59 +0200 Subject: erl -pa buggy behavior Message-ID: The manual page for "erl" says -pa Dir1 Dir2 ... : Adds the specified directories to the beginning of the code path ... But it doesn't - not if you have a .erlang start-up file. When you have a startup file it is consulted first, *then* the -pa flag is obeyed. To make life worse the .erlang file isn't documented in the erl man page at all. /Joe From gianfranco.alongi@REDACTED Tue Oct 19 20:44:29 2010 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Tue, 19 Oct 2010 18:44:29 +0000 (GMT) Subject: [erlang-questions] Re: Extracting detailed information from TypEr/Dialyzer In-Reply-To: <20101019175144.GF21059@h216-235-12-174.host.egate.net> Message-ID: <40065321.117971287513869038.JavaMail.root@zimbra> Hi, I've been doing something _very_ similar through a direct parse_transform lib, but unfortunately came to the conclusion that the much repetitive pattern state_function/[2/3] code.... next_state_tuple/stop-tuple is very dependent the coding style, just like Vance says. The analyzer often outputs loopback nodes where the outoing arc goes to the next state NextState or another hidden state in a Variable :\ To remedy this, you can enchance your analyzer with runtime logging to a file, the logging could then contain which function was entered and which state the next state was. Then you could analyze that file with an external tool. At the moment, I'm working on this, with some extra bells and whistles, as data-generator by gen_fsm function head argument analysis, such that it will force the gen_fsm to cover as much as possible. Cheers /G ----- Original Message ----- From: "Vance Shipley" To: "Torben Hoffmann" Cc: "erlang-questions" Sent: Tuesday, October 19, 2010 6:51:44 PM GMT +00:00 GMT Britain, Ireland, Portugal Subject: [erlang-questions] Re: Extracting detailed information from TypEr/Dialyzer Torben, I went down this road ten years ago. At the time I was constantly diagraming my FSMs to reason about them and diagnose problems. I decided I needed to automate that process. The result: http://www.erlang.org/pipermail/erlang-questions/2001-October/003716.html Unfortunately the links to attachments in the above don't seem to work. I have attached the code I used for this however I haven't used it in many year so your mileage may vary. It was also fairly dependent on coding style. On Tue, Oct 12, 2010 at 09:00:00AM +0200, Torben Hoffmann wrote: } I want to extract the following information from a gen_fsm: } } - all the states } - the incoming events } - possible next states [...] } Has anybody been down this road before me? If so, do you have some insights } to share? -- -Vance ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From rzezeski@REDACTED Tue Oct 19 23:57:04 2010 From: rzezeski@REDACTED (Ryan Zezeski) Date: Tue, 19 Oct 2010 17:57:04 -0400 Subject: Access JSON in Erlang like you do in JavaScript Message-ID: I feel like a lot of people tend to feel pain when working with JSON in Erlang. Especially when trying to access nested values. I wrote a blog post about a way you can access JSON in Erlang using the same syntax you would in JavaScript. http://www.progski.net/blog/2010/destructuring_json_in_erlang_made_easy.html -Ryan From max.lapshin@REDACTED Wed Oct 20 00:25:53 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 20 Oct 2010 01:25:53 +0300 Subject: ERTS_FP_CHECK_INIT at 0x100fd758: detected unhandled FPE at 0x1 Message-ID: I have repeating messages in console about FP_CHECK_INIT How should I treat it? From tony.arcieri@REDACTED Wed Oct 20 00:29:31 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Tue, 19 Oct 2010 16:29:31 -0600 Subject: [erlang-questions] Re: Shared/Hybrid Heap In-Reply-To: References: <4CB2FCE4.1080304@gmail.com> <127d5a9b-d552-4238-bfb9-9bc1cd409b51@28g2000yqm.googlegroups.com> <4CB57B2A.7070703@gmail.com> <4CB588DF.8090002@amberbio.com> <28167138-3C02-4D75-9A22-2B74BF6C3321@cs.otago.ac.nz> <4CB6A985.4090701@amberbio.com> <4CB6B589.9050003@cs.ntua.gr> <4CB6BDDC.3060900@amberbio.com> Message-ID: On Thu, Oct 14, 2010 at 8:47 AM, Nicholas Frechette wrote: > As far as other languages go, yes, most (all?) of them stop all threads to > garbage collect (if those threads share memory). Java does this, C# does > this, ruby, python, etc. C# recently or will shortly introduce a garbage > collection algorithm that runs concurrently and thus doesn't stop all > threads but from what I can remember, it isn't 100% guaranteed (if threads > continue to allocate past a threshold, I believe they are still all stopped > waiting for GC to complete). This an important reason why multi > generational > garbage collection algorithms are so popular: it keeps most GC cycles > quick, > thus stopping all threads for the least amount of time. There are many garbage collectors available for Java, and some of them are completely pauseless, for example the garbage collection used in Azul's Vega appliances, which is now generally available in the Zing Java VM: http://www.azulsystems.com/products/zing/virtual-machine The Sun/Oracle JVM also has some options for concurrent "low pause" garbage collection. -- Tony Arcieri Medioh! A Kudelski Brand From jesper.louis.andersen@REDACTED Wed Oct 20 02:24:52 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 20 Oct 2010 02:24:52 +0200 Subject: [erlang-questions] Access JSON in Erlang like you do in JavaScript In-Reply-To: References: Message-ID: On Tue, Oct 19, 2010 at 11:57 PM, Ryan Zezeski wrote: > I feel like a lot of people tend to feel pain when working with JSON in > Erlang. ?Especially when trying to access nested values. ?I wrote a blog > post about a way you can access JSON in Erlang using the same syntax you > would in JavaScript. Have you considered wrapping the code into a parse transform so you can "embed" the javascript notation into Erlang directly? Another option is to write an xpath-like query tool, perhaps with a Zipper construction on the parse tree so you have a neat continuation for the next match, should you need it. This is probably my solution, if I need to process JSON documents in the future. You already made the first venture into this - think jQuery DOM selectors on steroids for selection into JSON structures. -- J. From rzezeski@REDACTED Wed Oct 20 04:12:17 2010 From: rzezeski@REDACTED (Ryan Zezeski) Date: Tue, 19 Oct 2010 22:12:17 -0400 Subject: [erlang-questions] Access JSON in Erlang like you do in JavaScript In-Reply-To: References: Message-ID: On Tue, Oct 19, 2010 at 8:24 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > Have you considered wrapping the code into a parse transform so you > can "embed" the javascript notation into Erlang directly? > > I have, and my first attempt was to use a parse transform. However, a parse transform requires that it's valid Erlang syntax and I don't believe something like "Obj.post.title" is considered valid Erlang syntax--correct? You could do something like "Obj/post/title" but at that point it looks more similar to XPath. Maybe I'll play around with this more. Another option is to write an xpath-like query tool, perhaps with a > Zipper construction on the parse tree so you have a neat continuation > for the next match, should you need it. This is probably my solution, > if I need to process JSON documents in the future. You already made > the first venture into this - think jQuery DOM selectors on steroids > for selection into JSON structures. > Huet's Zipper was definitely in my mind but I didn't do anything with it. I think I like the sound of jQuery for JSON but I'm not sure what that would look like. Has anyone done this in the JavaScript world? Let's say I had an object like so: var Obj = { posts: [{title: "Foo", body: "..."}, {title: "Bar", body: "..."}, ...]}; Are you talking about having something like the following... $(Obj, "title").each( function() { alert($(this).value); }; Pardon if my syntax is off, it's been a while since I've done JavaScript/jQuery development. From robert.virding@REDACTED Wed Oct 20 04:37:53 2010 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 20 Oct 2010 02:37:53 +0000 (GMT) Subject: [erlang-questions] Access JSON in Erlang like you do in JavaScript In-Reply-To: Message-ID: <901918617.118361287542273395.JavaMail.root@zimbra> You could always write it in the form "{Obj,post,title}" which is valid syntax. Or do it without parse transforms and call functions with a syntax like: fetch(Obj, {post,title}) This type of syntax, nesting using tuples, is used in OTP in places to access fields in nested property lists, and it is both legal erlang and not too ugly. Also much easier to implement function than parse transform. Robert ----- "Ryan Zezeski" wrote: > On Tue, Oct 19, 2010 at 8:24 PM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > > > > > Have you considered wrapping the code into a parse transform so you > > can "embed" the javascript notation into Erlang directly? > > > > > I have, and my first attempt was to use a parse transform. However, a > parse > transform requires that it's valid Erlang syntax and I don't believe > something like "Obj.post.title" is considered valid Erlang > syntax--correct? > You could do something like "Obj/post/title" but at that point it > looks > more similar to XPath. Maybe I'll play around with this more. > > Another option is to write an xpath-like query tool, perhaps with a > > Zipper construction on the parse tree so you have a neat > continuation > > for the next match, should you need it. This is probably my > solution, > > if I need to process JSON documents in the future. You already made > > the first venture into this - think jQuery DOM selectors on > steroids > > for selection into JSON structures. > > > > Huet's Zipper was definitely in my mind but I didn't do anything with > it. I > think I like the sound of jQuery for JSON but I'm not sure what that > would > look like. Has anyone done this in the JavaScript world? > > Let's say I had an object like so: > > var Obj = { posts: [{title: "Foo", body: "..."}, {title: "Bar", body: > "..."}, ...]}; > > Are you talking about having something like the following... > > $(Obj, "title").each( function() { alert($(this).value); }; > > Pardon if my syntax is off, it's been a while since I've done > JavaScript/jQuery development. From ok@REDACTED Wed Oct 20 04:59:59 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 20 Oct 2010 15:59:59 +1300 Subject: Pretty-printing Erlang Message-ID: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> I have been marking some student Erlang code and could have used a pretty-printer. There's a crude one I wrote, for tidying up basically good code, but something a bit stronger seems to be appropriate. Any recommendations? From ok@REDACTED Wed Oct 20 07:22:40 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 20 Oct 2010 18:22:40 +1300 Subject: [erlang-questions] Access JSON in Erlang like you do in JavaScript In-Reply-To: References: Message-ID: On 20/10/2010, at 10:57 AM, Ryan Zezeski wrote: > I feel like a lot of people tend to feel pain when working with JSON in > Erlang. Especially when trying to access nested values. I wrote a blog > post about a way you can access JSON in Erlang using the same syntax you > would in JavaScript. > > http://www.progski.net/blog/2010/destructuring_json_in_erlang_made_easy.html In Javascript you would access a single item using something like thingy[i1][i2][i3] where i1, i2, i3 are non-negative integer array indices or string hash keys. Suppose you have an implementation of EEP 18 (http://www.erlang.org/eeps/eep-0018.html). Then at(JSON = [H|_], Index) when is_tuple(H) -> if is_atom(Index) ; is_binary(Index) -> {value,{Index,Item}} = lists:keysearch(Index, 1, JSON), Item end; at(JSON, Index) when is_list(JSON), is_integer(Index) -> lists:nth(Index + 1, JSON). at(JSON, I1, I2) -> at(at(JSON, I1), I2). at(JSON, I1, I2, I3) -> at(at(JSON, I1), I2, I3). at(JSON, I1, I2, I3, I4) -> at(at(JSON, I1), I2, I3, I4). at_path(JSON, []) -> JSON; at_path(JSON, [X|Xs]) -> at_path(at(JSON, X), Xs). and now thingy[i1][i2][i3] becomes at(Thingy, I1, I2, I3) or at_path(Thingy, [I1,I2,I3]). One example in the link would be at(Obj, <<"post">>, <<"title">>) Another would be just at(Obj2, <<"person">>, <<"friends">>, 1) Adjusting this to the data structure used by mochijson2 would be simple too. It hardly seems worth adding special syntax and transformations to Erlang for something this simple. So what am I missing? From amit.murthy@REDACTED Wed Oct 20 08:00:54 2010 From: amit.murthy@REDACTED (amitm) Date: Tue, 19 Oct 2010 23:00:54 -0700 (PDT) Subject: confusing returns from httpc:request/4 In-Reply-To: References: Message-ID: <8ecc3ba2-3f82-4d55-a46f-063f41012ce4@p37g2000pra.googlegroups.com> FWIW, I had a similar problem. It went away when I disabled pipelining completely by executing ok = httpc:set_options([{max_keep_alive_length, 0}, {max_pipeline_length, 0}, {max_sessions, 0} ]), before my http calls. I used to usually see it on the 4th consecutive call to the same host when pipelining was enabled. The URL in question in my case was an Amazon EC2 url. The set of URLs in my case were "http://169.254.169.254/2009-04-04/meta-data/instance-id", "http://169.254.169.254/2009-04-04/meta-data/local-hostname", "http://169.254.169.254/2009-04-04/meta-data/local-ipv4", "http://169.254.169.254/2009-04-04/meta-data/public-hostname", "http://169.254.169.254/2009-04-04/meta-data/public-ipv4", "http://169.254.169.254/2009-04-04/meta-data/ami-id", While executing this set, with pipelining enabled, it used to always bomb on the 4th URL, i.e. for 'public-hostname' in 25% of the cases with pipelining disabled, the problem disappears. Amit On Oct 13, 2:58?pm, Ingela Andin wrote: > 2010/10/13 Chandru : > > > > > > > On 13 October 2010 09:53, Ingela Andin wrote: > >> Hi! > > >> The only reason that the httpc-client will return the value {error, > >> socket_closed_remotely} > >> is that the connection is prematurely ?closed by the server. > >> Unfortunately ?that something > >> is sent on a tcp socket does not mean it has been sent on the wire > >> yet, and depending on > >> the circumstances it is possible for the socket to be closed before > >> all data has been sent > >> so occasionally a tcp_close can come before all data is delivered. > > >> When you use ibrowse do you verify that you always get a correct body? > >> (compleate body) from what I can tell from the code ibrowse will on a > >> tcp_closed ?message always return whatever happens to be in the buffer > >> compleate or not. > > > ibrowse does this only in the following cases: > > ?* server returned a "Connection: Close" > > ?* Server supports HTTP/0.9 or HTTP/1.0 > > > In all other cases, it returns an error to the caller. > > > regards, > > Chandru > > Well I do not know if any of these things is true in Dans senario. But > I do know that > getting the problem with a tcp close message before all data is > deliverd is a highly timing dependant problem > and if it happens in the server there is nothing inets-client or > ibrows can do about it except return an error. > > Regards Ingela Erlang/OTP team - Ericsson AB > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From bengt.kleberg@REDACTED Wed Oct 20 09:03:42 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 20 Oct 2010 09:03:42 +0200 Subject: [erlang-questions] Pretty-printing Erlang In-Reply-To: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> References: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> Message-ID: <1287558222.4859.4.camel@seasc1137> Greetings, I have used the module erl_prettypr (http://www.erlang.org/doc/man/erl_prettypr.html) to achieve the emacs layout prescribed here at work. The code I write has 1 tab per level instead of the variable amount of space/tab that emacs prescribes. bengt On Wed, 2010-10-20 at 04:59 +0200, Richard O'Keefe wrote: > I have been marking some student Erlang code and could have used > a pretty-printer. There's a crude one I wrote, for tidying up > basically good code, but something a bit stronger seems to be > appropriate. > > Any recommendations? > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From bengt.kleberg@REDACTED Wed Oct 20 09:09:42 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 20 Oct 2010 09:09:42 +0200 Subject: clarify: Why are there two zlibs? Message-ID: <1287558582.4859.10.camel@seasc1137> Greetings, When I was looking at the online documentation for the first time in a while I noticed that there are two zlibs at the bottom of this page (http://www.erlang.org/doc/man_index.html). It would be nice with an explanation of the reason for this. bengt From mikpe@REDACTED Wed Oct 20 09:36:31 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Wed, 20 Oct 2010 09:36:31 +0200 Subject: [erlang-questions] ERTS_FP_CHECK_INIT at 0x100fd758: detected unhandled FPE at 0x1 In-Reply-To: References: Message-ID: <19646.39935.788301.514513@pilspetsen.it.uu.se> Max Lapshin writes: > I have repeating messages in console about FP_CHECK_INIT > > How should I treat it? As a bug, it signifies a condition that's potentially harmful. You should tell us about the environment where this occurs, including: - the exact CPU type - the operating system type & version - the erlang version If you can provide a test case that would be great too. If you're not running the latest R14 release then please check if the problem occurs with R14 or not. From spawn.think@REDACTED Wed Oct 20 13:32:09 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Wed, 20 Oct 2010 13:32:09 +0200 Subject: [erlang-questions] clarify: Why are there two zlibs? In-Reply-To: <1287558582.4859.10.camel@seasc1137> References: <1287558582.4859.10.camel@seasc1137> Message-ID: Yeah and they are linked to the same page! I think this is because zlib used to be in the kernel app then it's moved to erts at least that's what's mentioned in lib/kernel/doc/src/zlib_stub.xml "The module zlib is moved to the runtime system application" On Wed, Oct 20, 2010 at 9:09 AM, Bengt Kleberg wrote: > Greetings, > > When I was looking at the online documentation for the first time in a > while I noticed that there are two zlibs at the bottom of this page > (http://www.erlang.org/doc/man_index.html). It would be nice with an > explanation of the reason for this. > > > bengt > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From dangud@REDACTED Wed Oct 20 13:45:31 2010 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 20 Oct 2010 13:45:31 +0200 Subject: [erlang-questions] VM segfault on exit with wx In-Reply-To: <20101002184629.GA398@ecn.lan> References: <20101001224605.GB30223@ecn.lan> <20101002184629.GA398@ecn.lan> Message-ID: I had a quick look at this, and I have an eternal loop there. Since erlang is going down it kills all processes, the wx thread can't send an event to a non existing process and it tries to send an error msg to a non existing process which fails and it tries to send ... sigh Will fix in next release. /Dan On Sat, Oct 2, 2010 at 8:46 PM, Michael Santos wrote: > On Fri, Oct 01, 2010 at 06:46:05PM -0400, Michael Santos wrote: >> On Tue, Sep 28, 2010 at 01:44:12PM -0400, Mark Scandariato wrote: >> > Seems like the VM segfaults on exit whenever I run anything that uses wx >> > (this is a single processor system). >> > >> > Let me know if you need any other info (or a core file or whatever). >> > >> > Mark. >> > >> > erlang@REDACTED:~/otp_src_R14B$ uname -a >> > Linux ubuntu 2.6.28-19-generic #65-Ubuntu SMP Thu Sep 16 14:14:28 UTC 2010 >> > i686 GNU/Linux >> > erlang@REDACTED:~/otp_src_R14B$ bin/cerl -debug -smp enable >> > Erlang R14B (erts-5.8.1) [source] [smp:1:1] [rq:1] [async-threads:0] [hipe] >> > [kernel-poll:false] [type-assertions] [debug-compiled] [lock-checking] >> > >> > Eshell V5.8.1 ?(abort with ^G) >> > 1> wx:demo(). >> > {wx_ref,35,wxFrame,<0.34.0>} >> > 2> q(). >> > ok >> > 3> Segmentation fault (core dumped) >> >> Looks as if beam is crashing printing out an error message. The attached >> patch seems to fix it. > > Wow, had another look and that "fix" doesn't help at all, that was just > a heisenfix from running in gdb. Sorry for the noise! > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mazen.harake@REDACTED Wed Oct 20 13:59:53 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Wed, 20 Oct 2010 13:59:53 +0200 Subject: [erlang-questions] clarify: Why are there two zlibs? In-Reply-To: References: <1287558582.4859.10.camel@seasc1137> Message-ID: <4CBED9B9.8060103@erlang-solutions.com> I think it is the same symptom as the "erlang" module. it is also seen twice. On 20/10/2010 13:32, Ahmed Omar wrote: > Yeah and they are linked to the same page! I think this is because zlib used > to be in the kernel app then it's moved to erts > at least that's what's mentioned in lib/kernel/doc/src/zlib_stub.xml > > "The module zlib is moved to the runtime system > application" > > On Wed, Oct 20, 2010 at 9:09 AM, Bengt Kleberg > wrote: > >> Greetings, >> >> When I was looking at the online documentation for the first time in a >> while I noticed that there are two zlibs at the bottom of this page >> (http://www.erlang.org/doc/man_index.html). It would be nice with an >> explanation of the reason for this. >> >> >> bengt >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From alexey.v.romanov@REDACTED Wed Oct 20 15:58:58 2010 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Wed, 20 Oct 2010 17:58:58 +0400 Subject: Unexpected messages from a port driver Message-ID: [Duplicating my question on StackOverflow here.] Erlang-sqlite3 uses a port driver to connect with the SQLite database, and receives messages from the port in this way (http://github.com/alexeyr/erlang-sqlite3/blob/56bfbe484d0c5b97d7b1f08677def3cc388787be/src/sqlite3.erl#L609): wait_result(Port) -> receive {Port, Reply} -> % io:format("Reply: ~p~n", [Reply]), Reply; {error, Reason} -> io:format("Error: ~p~n", [Reason]), {error, Reason}; _Else -> io:format("Else: ~p~n", [_Else]), _Else end. According to the manual (http://www.erlang.org/doc/reference_manual/ports.html), the following messages can be received from a port: {Port,{data,Data}} Data is received from the external program. {Port,closed} Reply to Port ! {Pid,close}. {Port,connected} Reply to Port ! {Pid,{connect,NewPid}} {'EXIT',Port,Reason} If the port has terminated for some reason. So, when uncommenting the `io:format` line in `{Port, Reply}` clause, I should expect to see `{data, ...}` for actual replies. I don't; instead I see (for `test.erl`) Reply: {ok,101} Reply: [{columns,["name"]},{rows,[{<<"user">>}]}] Reply: [{columns,["sql"]}, {rows,[{<<"CREATE TABLE user (id INTEGER PRIMARY KEY, name TEXT, age INTEGER, wage INTEGER)">>}]}] Reply: {id,1} Reply: {id,2} Reply: [{columns,["id","name","age","wage"]}, {rows,[{1,<<"abby">>,20,2000},{2,<<"marge">>,30,2000}]}] Reply: [{columns,["id","name","age","wage"]},{rows,[{1,<<"abby">>,20,2000}]}] Reply: [{columns,["id","name","age","wage"]}, {rows,[{1,<<"abby">>,20,2000},{2,<<"marge">>,30,2000}]}] Reply: {ok,101} Reply: [{columns,["id","name","age","wage"]},{rows,[{1,<<"abby">>,20,2000}]}] Reply: {ok,101} 1. Am I missing something obvious, or is message format given in the manual wrong? 2. In the second case, what messages actually can be received from the port? In particular, on error is the message `{'EXIT',Port,Reason}` or something else? Yours, Alexey Romanov From joelr1@REDACTED Wed Oct 20 16:17:43 2010 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 20 Oct 2010 15:17:43 +0100 Subject: multiple processes using a raw mode file Message-ID: <38E5DFB6-9C4A-4287-9A71-070639C7D842@gmail.com> Why is there a limitation of only the process that opened a raw file being able to use that file? file:open returns a port, shouldn't any process be able to use that port? I'd like multiple processes to be able to use a file opened in raw mode. Thanks, Joel --- http://twitter.com/wagerlabs From jesper.louis.andersen@REDACTED Thu Oct 21 00:18:42 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 21 Oct 2010 00:18:42 +0200 Subject: [erlang-questions] Access JSON in Erlang like you do in JavaScript In-Reply-To: References: Message-ID: On Wed, Oct 20, 2010 at 4:12 AM, Ryan Zezeski wrote: > Are you talking about having something like the following... > $(Obj, "title").each( function() { alert($(this).value); }; Don't mind if the syntax is right or wrong. Yes, this is somewhat the idea I had in mind. You want some kind of generic access on the data-tree because otherwise you end up in a maze of small pattern matches, all alike. -- J. From vances@REDACTED Thu Oct 21 00:41:57 2010 From: vances@REDACTED (Vance Shipley) Date: Wed, 20 Oct 2010 18:41:57 -0400 Subject: Change node name in mnesia In-Reply-To: <20101018172930.GE21059@h216-235-12-174.host.egate.net> References: <20100214031924.GM13976@h216-235-12-174.host.egate.net> <20100702222835.GC267@h216-235-12-169.host.egate.net> <20101018172930.GE21059@h216-235-12-174.host.egate.net> Message-ID: <20101020224157.GA3472@h216-235-12-169.host.egate.net> Dan, I am able to reproduce the problem with the attached code and the procedure detailed below. In a nutshell the process outlined in The Mnesia User's Guide for renaming a database node in a backup file results in a corrupt database. http://erlang.org/doc/apps/mnesia/Mnesia_chap7.html#id74479 I used the attached test code, derived from the above example, to create a sample database, rename the node name from long form to short and verify it's integrity. Start a node with a long node name: $ erl -name long Eshell V5.8.1 (abort with ^G) (long@REDACTED)1> rr("db_transform.erl"). [acc,bar,baz,foo] Build an example data base: (long@REDACTED)2> db_transform:install([node()], 4000, 500, 25). {ok,#acc{foo = 3994,bar = 500,baz = 25}} Back up the example database into the file "long": (long@REDACTED)3> mnesia:backup(long). ok Start a node with a short node name: $ erl -sname short Eshell V5.8.1 (abort with ^G) (short@REDACTED)1> rr("db_transform.erl"). [acc,bar,baz,foo] Change the node name in the backup file "long" and create backup "short": (short@REDACTED)2> db_transform:rename('long@REDACTED', 'short@REDACTED', long, short). {ok,#acc{foo = 3994,bar = 500,baz = 25}} Create a new database from the backup "short": (short@REDACTED)3> mnesia:create_schema([node()]). ok (short@REDACTED)4> mnesia:start(). ok (short@REDACTED)5> mnesia:restore(short, [{default_op, recreate_tables}]). {atomic,[foo,baz,bar]} Test the database integrity: (short@REDACTED)6> db_transform:verify([foo, bar, baz]). #acc{foo = 3994,bar = 500,baz = 25} The above proves the backup, transform and restore process works. So far so good. Now below is where the problem can be reproduced. (short@REDACTED)7> q(). ok $ erl -sname short (short@REDACTED)1> mnesia:start(). ok (short@REDACTED)2> db_transform:verify([foo, bar, baz]). ** exception exit: {function_clause,[{db_transform,'-verify/2-fun-0-', [{log_header,dcl_log,"1.0","4.4.15",'short@REDACTED', {1287,613266,224499}}, 1899]}, {lists,foldl,3}, {mnesia,do_foldl,8}, {mnesia,foldl,6}, {mnesia_tm,apply_fun,3}, {mnesia_tm,execute_transaction,5}, {db_transform,count,2}, {db_transform,verify,2}]} in function db_transform:count/2 in call from db_transform:verify/2 Restarting the node is what triggers the problem. (short@REDACTED)3> ets:i(baz). <1 > {baz,711590, <<76,16,157,97,48,225,132,80,36,196,3,199,194,239 ... <2 > {baz,411561, <<173,175,232,135,28,118,233,210,111,49,169,19,88 ... <3 > {baz,909239, <<135,214,14,32,8,121,144,64,196,35,186,167,63,24 ... <4 > {baz,463433, <<66,156,188,32,49,157,54,160,133,241,196,78,22,2 ... <5 > {baz,997452, <<248,253,234,156,76,130,48,212,89,197,186,210,21 ... <6 > {baz,797390, <<25,1,27,228,30,234,2,147,80,80,167,58,230,54,11 ... <7 > {baz,879889, <<249,45,35,209,179,173,217,158,208,227,220,67,20 ... <8 > {baz,604804, <<120,75,101,145,118,116,39,218,204,246,198,162,3 ... <9 > {baz,807967, <<58,209,110,188,212,128,136,15,133,56,166,13,30, ... <10 > {baz,388388, <<143,53,248,42,115,146,218,222,246,67,10,173,185 ... <11 > {baz,890638, <<11,137,19,231,155,97,254,94,79,68,94,82,156,190 ... <12 > {baz,865475, <<169,173,174,222,235,89,94,90,11,192,50,72,37,43 ... <13 > {baz,595736, <<43,98,245,177,0,79,54,225,36,201,24,188,195,240 ... <14 > {log_header,dcl_log,"1.0","4.4.15",'short@REDACTED', ... <15 > {baz,771466, <<39,59,205,76,71,40,80,85,65,229,224,198,109,132 ... <16 > {baz,834821, <<76,65,143,50,217,159,155,39,129,56,67,59,73,95, ... <17 > {baz,534651, <<72,192,141,91,154,171,201,126,135,7,213,143,13, ... <18 > {baz,551117, <<85,185,77,144,114,113,203,93,206,29,122,106,52, ... <19 > {baz,224818, <<120,27,23,224,251,51,254,69,215,104,11,93,135,5 ... <20 > {baz,358442, <<255,165,10,214,6,121,98,31,231,82,52,194,27,94, ... <21 > {baz,382522, <<188,190,108,27,51,81,186,175,215,95,168,203,128 ... <22 > {baz,991628, <<232,191,166,252,143,87,251,139,145,217,227,85,5 ... <23 > {baz,748772, <<210,38,226,254,243,76,211,180,233,33,160,144,25 ... <24 > {baz,563034, <<52,120,167,54,135,139,205,40,57,250,28,86,34,10 ... <25 > {baz,765113, <<189,143,122,1,82,185,173,20,161,90,145,16,225,1 ... <26 > {baz,510427, <<193,192,140,177,72,25,44,199,190,71,144,24,242, ... EOT (q)uit (p)Digits (k)ill /Regexp --> For some reason the header of a TABLE.DCL file is getting stuck into the database in between stopping the node, restarting the node and starting mnesia again. -Vance On Mon, Oct 18, 2010 at 01:29:30PM -0400, Vance Shipley wrote: } On Sat, Jul 03, 2010 at 09:09:31AM +0200, Dan Gudmundsson wrote: } } Are you seeing those in the ets table? } } Yes I do see the headers in the ets table: } } 4> ets:i(foo). } [...] } <12 > {log_header,dcl_log,"1.0","4.4.10",'bar@REDACTED',{1287,322033,2 ... } [...] } } } They should not be in the table, they are what they say the header of } } a TABLE.DCL file. } } } } Either you or I have some improvements to do :-) } } I did another database migration today, changing long node name } to short, on R13B04 yesterday and experienced the same results. } These header records don't show up in the items passed through } mnesia:traverse_backup/6 so it doesn't appear to be something } I'm doing wrong. } } I'm going to work on demonstrating the problem as simply as possible. } } -- } -Vance } } } On Sat, Jul 3, 2010 at 12:28 AM, Vance Shipley wrote: } } > Is it valid to have records in an mnesia table of this form: } } > } } > ?{log_header,dcl_log,"1.0","4.4.10",'foo@REDACTED',{1266,113437,587068}}, } } > } } > The tables seem to be working fine even though these entries are } } > there which obviously do not have the correct OID for the table. } } > } } > What I'd like to understand is if I've broken the tables myself } } > or if this type of thing is valid and I should just deal with } } > them when traversing a table with mnesia:foldl/3. } } > } } > ? ? ? ?-Vance } } > } } > On Sat, Feb 13, 2010 at 10:19:24PM -0500, Vance Shipley wrote: } } > } ?I used the example in the Mnesia User's Guide to backup } } > } ?and transform a database from a long node name to a short } } > } ?one. ?That all seemed to go smoothly enough but now I'm } } > } ?noticing these strange records in my tables: } } > } } } > } ? ? {log_header,dcl_log,"1.0","4.4.10",'foo@REDACTED',{1266,113437,587068}}, } } > } } } > } ?I discovered this using mnesia:foldl/3. ?Is this normal } } > } ?or have I messed things up? -- -Vance -------------- next part -------------- -module(db_transform). -export([install/4, rename/4, info/1, verify/1]). -record(foo, {key, value}). -record(bar, {key, value}). -record(baz, {key, value}). -record(acc, {foo = 0, bar = 0, baz = 0}). install(Nodes, SizeFoo, SizeBar, SizeBaz) -> try case mnesia:create_schema(Nodes) of ok -> ok; {error, CReason} -> throw(CReason) end, case mnesia:start() of ok -> ok; {error, SReason} -> throw(SReason) end, case mnesia:wait_for_tables([schema], 10000) of ok -> ok; SchemaResult-> throw(SchemaResult) end, case mnesia:create_table(foo, [{disc_copies, Nodes}, {attributes, record_info(fields, foo)}]) of {atomic, ok} -> ok; T1Result -> throw(T1Result) end, case mnesia:create_table(bar, [{disc_copies, Nodes}, {attributes, record_info(fields, bar)}]) of {atomic, ok} -> ok; T2Result -> throw(T2Result) end, case mnesia:create_table(baz, [{disc_copies, Nodes}, {attributes, record_info(fields, baz)}]) of {atomic, ok} -> ok; T4Result -> throw(T4Result) end, Tables = [foo, bar, baz], case mnesia:wait_for_tables(Tables, 10000) of ok -> ok; TablesResult -> throw(TablesResult) end, crypto:start(), Threshold = mnesia:system_info(dump_log_write_threshold) - 1, initialize(Tables, Threshold, SizeFoo, SizeBar, SizeBaz), {ok, info(Tables)} of Result -> Result catch throw:Error -> mnesia:error_description(Error) end. initialize([], _Threshold, _SizeFoo, _SizeBar, _SizeBaz) -> ok; initialize([foo | Rest], Threshold, SizeFoo, SizeBar, SizeBaz) -> batch(foo, SizeFoo, Threshold), initialize(Rest, Threshold, SizeFoo, SizeBar, SizeBaz); initialize([bar | Rest], Threshold, SizeFoo, SizeBar, SizeBaz) -> batch(bar, SizeBar, Threshold), initialize(Rest, Threshold, SizeFoo, SizeBar, SizeBaz); initialize([baz | Rest], Threshold, SizeFoo, SizeBar, SizeBaz) -> batch(baz, SizeBaz, Threshold), initialize(Rest, Threshold, SizeFoo, SizeBar, SizeBaz). batch(_Table, N, _Threshold) when N =< 0 -> ok; batch(Table, N, Threshold) when N =< Threshold -> insert(Table, N), mnesia:dump_log(), ok; batch(Table, N, Threshold) when N > Threshold -> insert(Table, Threshold), mnesia:dump_log(), batch(Table, N - Threshold, Threshold). insert(_Table, 0) -> ok; insert(Table, N) -> Key = crypto:rand_uniform(0, 1000000), Value = crypto:rand_bytes(128), Rec = {Table, Key, Value}, mnesia:dirty_write(Table, Rec), insert(Table, N - 1). rename(From, To, Source, Target) -> Fswitch = fun(Node) when Node == From -> To; (Node) when Node == To -> throw({error, already_exists}); (Other) -> Other end, Fconvert = fun({schema, db_nodes, Nodes}, Acc) -> {[{schema, db_nodes, lists:map(Fswitch,Nodes)}], Acc}; ({schema, Tab, CreateList}, Acc) -> Keys = [ram_copies, disc_copies, disc_only_copies, cookie], FoptSwitch = fun({cookie, {Now, Node}}) when Node == From -> {cookie, {Now, To}}; ({Key, Val}) -> case lists:member(Key, Keys) of true -> {Key, lists:map(Fswitch, Val)}; false-> {Key, Val} end end, {[{schema, Tab, lists:map(FoptSwitch, CreateList)}], Acc}; (#foo{} = Foo, #acc{foo = Count} = Acc) -> {[Foo], Acc#acc{foo = Count + 1}}; (#bar{} = Foo, #acc{bar = Count} = Acc) -> {[Foo], Acc#acc{bar = Count + 1}}; (#baz{} = Foo, #acc{baz = Count} = Acc) -> {[Foo], Acc#acc{baz = Count + 1}} end, mnesia:traverse_backup(Source, Target, Fconvert, #acc{}). info(Tables) -> info(Tables, #acc{}). info([foo | Rest], Acc) -> Size = mnesia:table_info(foo, size), info(Rest, Acc#acc{foo = Size}); info([bar | Rest], Acc) -> Size = mnesia:table_info(bar, size), info(Rest, Acc#acc{bar = Size}); info([baz | Rest], Acc) -> Size = mnesia:table_info(baz, size), info(Rest, Acc#acc{baz = Size}); info([], Acc) -> Acc. verify(Tables) -> verify(Tables, #acc{}). verify([foo | Rest], Acc) -> Fun = fun(#foo{}, Count) -> Count + 1 end, Size = count(Fun, foo), verify(Rest, Acc#acc{foo = Size}); verify([bar | Rest], Acc) -> Fun = fun(#bar{}, Count) -> Count + 1 end, Size = count(Fun, bar), verify(Rest, Acc#acc{bar = Size}); verify([baz | Rest], Acc) -> Fun = fun(#baz{}, Count) -> Count + 1 end, Size = count(Fun, baz), verify(Rest, Acc#acc{baz = Size}); verify([], Acc) -> Acc. count(Facc, Table) -> Ftrans = fun() -> mnesia:foldl(Facc, 0, Table) end, case mnesia:transaction(Ftrans) of {atomic, Result} -> Result; {aborted, Reason} -> exit(Reason) end. From vances@REDACTED Thu Oct 21 00:45:21 2010 From: vances@REDACTED (Vance Shipley) Date: Wed, 20 Oct 2010 18:45:21 -0400 Subject: Change node name in mnesia In-Reply-To: <20101020224157.GA3472@h216-235-12-169.host.egate.net> References: <20100214031924.GM13976@h216-235-12-174.host.egate.net> <20100702222835.GC267@h216-235-12-169.host.egate.net> <20101018172930.GE21059@h216-235-12-174.host.egate.net> <20101020224157.GA3472@h216-235-12-169.host.egate.net> Message-ID: <20101020224521.GB3472@h216-235-12-169.host.egate.net> The interesting thing is that the database keeps on working. I only noticed the problem because I have code which uses mnesia:foldl/3 function to iterate over all the table entries and that ciode expects all the entries to be valid records for the table. -Vance On Wed, Oct 20, 2010 at 06:41:57PM -0400, Vance Shipley wrote: } For some reason the header of a TABLE.DCL file is getting stuck into } the database in between stopping the node, restarting the node and } starting mnesia again. From robert.virding@REDACTED Thu Oct 21 00:51:29 2010 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 20 Oct 2010 22:51:29 +0000 (GMT) Subject: [erlang-questions] Access JSON in Erlang like you do in JavaScript In-Reply-To: Message-ID: <1035648520.126651287615089427.JavaMail.root@zimbra> A maze of twisty small pattern matches even. Robert ----- "Jesper Louis Andersen" wrote: > On Wed, Oct 20, 2010 at 4:12 AM, Ryan Zezeski > wrote: > > > Are you talking about having something like the following... > > $(Obj, "title").each( function() { alert($(this).value); }; > > Don't mind if the syntax is right or wrong. Yes, this is somewhat the > idea I had in mind. You want some kind of generic access on the > data-tree because otherwise you end up in a maze of small pattern > matches, all alike. > > > -- > J. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From jiansenhe@REDACTED Thu Oct 21 02:42:10 2010 From: jiansenhe@REDACTED (Jiansen He) Date: Thu, 21 Oct 2010 01:42:10 +0100 Subject: [erlang-questions] code mobility Message-ID: Hello, I am thinking to what extend Erlang may support (weak) code mobility. Directly send a function to another node will not work because only function signatures are sent. I tried sending functions in 3 ways: 1) send a string to a remote node, where a erlang interpreter is invoked. 2) get the representation of function by calling erlang:fun_info/1, send the representation, then reconstruct the function at remote node. 3) force the remote node loading the module where the function is contained, so that function could be called by its name. Notice that none of the above solutions are satisfactory if another helper function is needed. If this is the case, then I have to send all potentially involving functions by hand. Could anyone offer a better solution to send a function in Erlang. Or, perhaps, some efforts have been done before? I think code mobility will became more and more important for future languages. Jiansen From ok@REDACTED Thu Oct 21 02:59:08 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 21 Oct 2010 13:59:08 +1300 Subject: [erlang-questions] Pretty-printing Erlang In-Reply-To: <1287558222.4859.4.camel@seasc1137> References: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> <1287558222.4859.4.camel@seasc1137> Message-ID: <207A14F9-852E-4C93-BF21-206F6B9616F9@cs.otago.ac.nz> On 20/10/2010, at 8:03 PM, Bengt Kleberg wrote: > Greetings, > > I have used the module erl_prettypr > (http://www.erlang.org/doc/man/erl_prettypr.html) to achieve the emacs > layout prescribed here at work. That's not exactly what I wanted, but it did lead me to erl_tidy:dir(), which was. There are several things I'd like erl_tidy to do for me which I can't see any options for. This isn't actually the original code here. It's the code after _my_ el cheapo pretty printer finished with it. -module(agent). -export([setup_agents/2]). setup_agents(Agents, 0) -> Agents; setup_agents(Agents, Num) -> Agent = init(), setup_agents(Agents ++ Agent, Num - 1). init() -> Vision = random:uniform(3), Hunger = random:uniform(5), Food = random:uniform(8), [{agent, Vision, Hunger, Food}]. Here's the code after erl_tidy:dir() finished with it. -module(agent). -export([setup_agents/2]). setup_agents(Agents, 0) -> Agents; setup_agents(Agents, Num) -> Agent = init(), setup_agents(Agents ++ Agent, Num - 1). init() -> Vision = random:uniform(3), Hunger = random:uniform(5), Food = random:uniform(8), [{agent, Vision, Hunger, Food}]. erl_tidy: has introduced two kinds of ugliness: (1) It has removed a line break after the first arrow, making the placement of the body code inconsistent between the clauses. (2) There were two body-level expressions in the second clause of setup_agents/2 which were well and properly separated, but erl_tidy: has put them on the same line, making it rather untidy. This could be addressed by providing {break_after_arrow,boolean()} {break_after_comma,boolean()} options. It's exam time here, so it will be a while before I can get around to doing anything about this, but before I even _think_ of peering inside erl_tidy:, do other people agree that this would be an improvement? Of course the best code here would be something like -module(agent). -export([set_up_agents/1]). set_up_agents(N) -> [new_agent() || _ <- lists:seq(1, N)]. new_agent() -> Vision = random:uniform(3), Hunger = random:uniform(5), Food = random:uniform(8), {agent,Vision,Hunger,Food}. but it is beyond the scope of a pretty-printer to make changes like that. Here's some sample code from a student assignment, after erl_tidy:dir() was done with it. setup_agents(Agents, 0) -> Agents; setup_agents(Agents, Num) -> Agent = init(), setup_agents(Agents ++ Agent, Num - 1). (1) I'd like {break_after_arrow,boolean()} to get consistent placement of code after arrows. (2) I'd like {break_after_comma,boolean()} to put the two expressions in the second clause on separate lines. setup_agents(Agents, 0) -> Agents; setup_agents(Agents, Num) -> Agent = init(), setup_agents(Agents ++ Agent, Num - 1). It is beyond the power of a > > The code I write has 1 tab per level instead of the variable amount of > space/tab that emacs prescribes. > > > bengt > > On Wed, 2010-10-20 at 04:59 +0200, Richard O'Keefe wrote: >> I have been marking some student Erlang code and could have used >> a pretty-printer. There's a crude one I wrote, for tidying up >> basically good code, but something a bit stronger seems to be >> appropriate. >> >> Any recommendations? >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ok@REDACTED Thu Oct 21 04:00:37 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 21 Oct 2010 15:00:37 +1300 Subject: [erlang-questions] code mobility In-Reply-To: References: Message-ID: On 21/10/2010, at 1:42 PM, Jiansen He wrote: > Hello, > > I am thinking to what extend Erlang may support (weak) code mobility. > Directly send a function to another node will not work because only > function signatures are sent. There are languages that do this, like Kali Scheme. For that matter, dear old Interlisp-D could send S-expressions around the network, and compiling S-expressions to byte code was fast enough that network transmission was the dominant cost. Obliq (from Luca Cardelli at DEC) could send objects around the net. without losing their methods. > Could anyone offer a better solution to send a function in Erlang. > Or, perhaps, some efforts have been done before? I think code > mobility will became more and more important for future languages. Code mobility was the coming thing several decades ago. Does anyone else remember Emerald? There are issues with security and so on that make it attractive only within a trust boundary. > From dima@REDACTED Thu Oct 21 06:09:30 2010 From: dima@REDACTED (Dmitry Vasiliev) Date: Thu, 21 Oct 2010 08:09:30 +0400 Subject: [erlang-questions] code mobility In-Reply-To: References: Message-ID: <4CBFBCFA.60502@hlabs.org> 21.10.2010 04:42, Jiansen He ?????: > Could anyone offer a better solution to send a function in Erlang. > Or, perhaps, some efforts have been done before? I think code > mobility will became more and more important for future languages. What about to send a module with code:load_binary/3? -- Dmitry Vasiliev http://hlabs.org http://twitter.com/hdima From rapsey@REDACTED Thu Oct 21 07:29:08 2010 From: rapsey@REDACTED (Rapsey) Date: Thu, 21 Oct 2010 07:29:08 +0200 Subject: [erlang-questions] code mobility In-Reply-To: References: Message-ID: Why cant you have the same beam files on all nodes? Sergej On Thu, Oct 21, 2010 at 2:42 AM, Jiansen He wrote: > Hello, > > I am thinking to what extend Erlang may support (weak) code mobility. > Directly send a function to another node will not work because only > function signatures are sent. I tried sending functions in 3 ways: > > 1) send a string to a remote node, where a erlang interpreter is > invoked. > 2) get the representation of function by calling erlang:fun_info/1, > send the representation, then reconstruct the function at remote node. > 3) force the remote node loading the module where the function is > contained, so that function could be called by its name. > > Notice that none of the above solutions are satisfactory if another > helper function is needed. If this is the case, then I have to send > all potentially involving functions by hand. > > Could anyone offer a better solution to send a function in Erlang. > Or, perhaps, some efforts have been done before? I think code > mobility will became more and more important for future languages. > > Jiansen > From bengt.kleberg@REDACTED Thu Oct 21 07:33:11 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 21 Oct 2010 07:33:11 +0200 Subject: [erlang-questions] Pretty-printing Erlang In-Reply-To: <207A14F9-852E-4C93-BF21-206F6B9616F9@cs.otago.ac.nz> References: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> <1287558222.4859.4.camel@seasc1137> <207A14F9-852E-4C93-BF21-206F6B9616F9@cs.otago.ac.nz> Message-ID: <1287639191.5435.9.camel@seasc1137> The clean up effect you describe as the beyond the scope of a pretty printer might be achieve by tidier (http://www.erlang-factory.com/conference/London2009/speakers/kostissagonas). The problem with several body level expressions on one line is a problem that I either avoid by having long variable/function names, or fix manually after wards. It would be nice to get it done automatically. bengt On Thu, 2010-10-21 at 02:59 +0200, Richard O'Keefe wrote: > On 20/10/2010, at 8:03 PM, Bengt Kleberg wrote: > > > Greetings, > > > > I have used the module erl_prettypr > > (http://www.erlang.org/doc/man/erl_prettypr.html) to achieve the emacs > > layout prescribed here at work. > > That's not exactly what I wanted, but it did lead me to > erl_tidy:dir(), which was. > > There are several things I'd like erl_tidy to do for me > which I can't see any options for. > > This isn't actually the original code here. It's the > code after _my_ el cheapo pretty printer finished with it. > > -module(agent). > -export([setup_agents/2]). > > setup_agents(Agents, 0) -> > Agents; > setup_agents(Agents, Num) -> > Agent = init(), > setup_agents(Agents ++ Agent, Num - 1). > > init() -> > Vision = random:uniform(3), > Hunger = random:uniform(5), > Food = random:uniform(8), > [{agent, Vision, Hunger, Food}]. > > Here's the code after erl_tidy:dir() finished with it. > > -module(agent). > > -export([setup_agents/2]). > > setup_agents(Agents, 0) -> Agents; > setup_agents(Agents, Num) -> > Agent = init(), setup_agents(Agents ++ Agent, Num - 1). > > init() -> > Vision = random:uniform(3), > Hunger = random:uniform(5), > Food = random:uniform(8), > [{agent, Vision, Hunger, Food}]. > > erl_tidy: has introduced two kinds of ugliness: > (1) It has removed a line break after the first arrow, > making the placement of the body code inconsistent > between the clauses. > > (2) There were two body-level expressions in the second > clause of setup_agents/2 which were well and properly > separated, but erl_tidy: has put them on the same > line, making it rather untidy. > > This could be addressed by providing > {break_after_arrow,boolean()} > {break_after_comma,boolean()} > options. > > It's exam time here, so it will be a while before I can > get around to doing anything about this, but before I even > _think_ of peering inside erl_tidy:, do other people > agree that this would be an improvement? > > > > Of course the best code here would be something like > > -module(agent). > -export([set_up_agents/1]). > > set_up_agents(N) -> > [new_agent() || _ <- lists:seq(1, N)]. > > new_agent() -> > Vision = random:uniform(3), > Hunger = random:uniform(5), > Food = random:uniform(8), > {agent,Vision,Hunger,Food}. > > but it is beyond the scope of a pretty-printer to make > changes like that. > > Here's some sample > code from a student assignment, after erl_tidy:dir() was > done with it. > > setup_agents(Agents, 0) -> Agents; > setup_agents(Agents, Num) -> > Agent = init(), setup_agents(Agents ++ Agent, Num - 1). > > (1) I'd like {break_after_arrow,boolean()} > to get consistent placement of code after arrows. > > (2) I'd like {break_after_comma,boolean()} > to put the two expressions in the second clause > on separate lines. > > setup_agents(Agents, 0) -> > Agents; > setup_agents(Agents, Num) -> > Agent = init(), > setup_agents(Agents ++ Agent, Num - 1). > > It is beyond the power of a > > > > > > > The code I write has 1 tab per level instead of the variable amount of > > space/tab that emacs prescribes. > > > > > > bengt > > > > On Wed, 2010-10-20 at 04:59 +0200, Richard O'Keefe wrote: > >> I have been marking some student Erlang code and could have used > >> a pretty-printer. There's a crude one I wrote, for tidying up > >> basically good code, but something a bit stronger seems to be > >> appropriate. > >> > >> Any recommendations? > >> > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > From rzezeski@REDACTED Thu Oct 21 07:38:31 2010 From: rzezeski@REDACTED (Ryan Zezeski) Date: Thu, 21 Oct 2010 01:38:31 -0400 Subject: [erlang-questions] Access JSON in Erlang like you do in JavaScript In-Reply-To: References: Message-ID: On Wed, Oct 20, 2010 at 1:22 AM, Richard O'Keefe wrote: > > On 20/10/2010, at 10:57 AM, Ryan Zezeski wrote: > > One example in the link would be > > at(Obj, <<"post">>, <<"title">>) > > Another would be just > > at(Obj2, <<"person">>, <<"friends">>, 1) > > Adjusting this to the data structure used by mochijson2 would be simple > too. > It hardly seems worth adding special syntax and transformations to Erlang > for something this simple. > > So what am I missing? > I don't think you're missing anything. I think you and Robert both make great points. Although, there is a part of me that still likes the fact that I can use JavaScript syntax with my solution. I agree that using parse transformations for this is probably not the best idea, but in the case of using Neotoma I'm only using parse transformations during the parsing of the PEG. The output is just another Erlang module. Of course, I'm assuming you already know this and you were referring to the other ideas on the table. Just wanted to clarify for anyone else reading this. Maybe there is room for multiple solutions. You know...different strokes for different folks. I wanted to share my finding as I hadn't seen anything like it before. Thanks for the input, everyone. From bengt.kleberg@REDACTED Thu Oct 21 07:41:37 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 21 Oct 2010 07:41:37 +0200 Subject: [erlang-questions] multiple processes using a raw mode file In-Reply-To: <38E5DFB6-9C4A-4287-9A71-070639C7D842@gmail.com> References: <38E5DFB6-9C4A-4287-9A71-070639C7D842@gmail.com> Message-ID: <1287639697.5435.17.camel@seasc1137> Greetings, The return value of file:open/2 is said to be a process (presumably that means a pid). The pid returned for a raw file open could be just a file descriptor (a guess). It would not be usable outside of the Erlang node. A real pid can be sent to any node. The file descriptor would only work for Erlang processes inside the same node, if my guess is right. bengt On Wed, 2010-10-20 at 16:17 +0200, Joel Reymont wrote: > Why is there a limitation of only the process that opened a raw file being able to use that file? > > file:open returns a port, shouldn't any process be able to use that port? > > I'd like multiple processes to be able to use a file opened in raw mode. > > Thanks, Joel > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From rzezeski@REDACTED Thu Oct 21 07:47:32 2010 From: rzezeski@REDACTED (Ryan Zezeski) Date: Thu, 21 Oct 2010 01:47:32 -0400 Subject: [erlang-questions] Pretty-printing Erlang In-Reply-To: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> References: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> Message-ID: On Tue, Oct 19, 2010 at 10:59 PM, Richard O'Keefe wrote: > I have been marking some student Erlang code and could have used > a pretty-printer. There's a crude one I wrote, for tidying up > basically good code, but something a bit stronger seems to be > appropriate. > > Any recommendations? > > Perhaps a new assignment for the students :) -Ryan From joelr1@REDACTED Thu Oct 21 08:56:40 2010 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 21 Oct 2010 07:56:40 +0100 Subject: [erlang-questions] multiple processes using a raw mode file In-Reply-To: <1287639697.5435.17.camel@seasc1137> References: <38E5DFB6-9C4A-4287-9A71-070639C7D842@gmail.com> <1287639697.5435.17.camel@seasc1137> Message-ID: What I'm seeing is that a descriptor opened in raw mode is only useful to the process that opened it. Position and truncate, for example, have no effect if I try them in a spawned process. I'd like for other processes on the same node to be able to use the same raw descriptor. While we are at it, an "atomic" (within the port driver) file:position+truncate would be most awesome! On Oct 21, 2010, at 6:41 AM, Bengt Kleberg wrote: > Greetings, > > The return value of file:open/2 is said to be a process (presumably that > means a pid). The pid returned for a raw file open could be just a file > descriptor (a guess). It would not be usable outside of the Erlang node. > A real pid can be sent to any node. The file descriptor would only work > for Erlang processes inside the same node, if my guess is right. --- http://twitter.com/wagerlabs From chinson.h@REDACTED Thu Oct 21 10:00:39 2010 From: chinson.h@REDACTED (Chao He) Date: Thu, 21 Oct 2010 16:00:39 +0800 Subject: [erlang-questions] Pretty-printing Erlang In-Reply-To: References: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> Message-ID: <1EB3D6F3-FE34-4B68-AF11-D16BEE0631B5@gmail.com> On Oct 21, 2010, at 1:47 PM, Ryan Zezeski wrote: > Perhaps a new assignment for the students :) > > -Ryan Great idea! Chinson From raimo+erlang-questions@REDACTED Thu Oct 21 11:41:53 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 21 Oct 2010 11:41:53 +0200 Subject: [erlang-questions] multiple processes using a raw mode file In-Reply-To: <38E5DFB6-9C4A-4287-9A71-070639C7D842@gmail.com> References: <38E5DFB6-9C4A-4287-9A71-070639C7D842@gmail.com> Message-ID: <20101021094153.GB32286@erix.ericsson.se> On Wed, Oct 20, 2010 at 03:17:43PM +0100, Joel Reymont wrote: > Why is there a limitation of only the process that opened a raw file being able to use that file? > > file:open returns a port, shouldn't any process be able to use that port? > > I'd like multiple processes to be able to use a file opened in raw mode. That would require some rewriting of efile_drv. prim_file and efile_drv assume the old port model where only the port owner communicates with the driver. The driver replies to the port owner. Later prim_file has been rewritten to use port_command instead of Port ! {command,...} to improve error detection. But efile_drv still replies to the port owner. If you use raw filehandle from another process I actually do not know what happens. Maybe efile_drv gets the the request and replies to the owner. Nevertheless it does not behave correctly. For it to behave correctly it would have to reply to the requester _and_ be rewritten for simultaneous requests from any number of processes, presumably a queue to serialize the requests, making it annoyingly similar to the "port owner does all requests" model. > > Thanks, Joel > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From nrapesh09@REDACTED Thu Oct 21 11:46:56 2010 From: nrapesh09@REDACTED (Nrapesh Khamesra) Date: Thu, 21 Oct 2010 15:16:56 +0530 Subject: [erlang-questions]not able to make two nodes communicate on mac os Message-ID: Hi, I am currently using mac os x 10.6.4 and I installed erlang version R13B04 while configuring i enabled the support for 64 bit adding the option '--enable-darwin-64bit' .It got installed perfectly fine however when I ran erlang using command erl -sname -test1 -setcookie abc and then started another elrang shell using command erl -sname -test2 -setcookie abc and tried to ping using the command net:adm,it wasn't able to ping the test1 node,however when I tried to ping the same node from my linux machine it was able to ping the node test1 on my mac. Can you please help me with this issue. Regards, From vladdu55@REDACTED Thu Oct 21 12:03:49 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 21 Oct 2010 12:03:49 +0200 Subject: [erlang-questions] Pretty-printing Erlang In-Reply-To: <207A14F9-852E-4C93-BF21-206F6B9616F9@cs.otago.ac.nz> References: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> <1287558222.4859.4.camel@seasc1137> <207A14F9-852E-4C93-BF21-206F6B9616F9@cs.otago.ac.nz> Message-ID: Hi! On Thu, Oct 21, 2010 at 02:59, Richard O'Keefe wrote: > On 20/10/2010, at 8:03 PM, Bengt Kleberg wrote: >> I have used the module erl_prettypr >> (http://www.erlang.org/doc/man/erl_prettypr.html) to achieve the emacs >> layout prescribed here at work. > > That's not exactly what I wanted, but it did lead me to > erl_tidy:dir(), which was. [not really an answer to the original question, sorry] Pretty-printing is a tricky business. The parsed Erlang syntax trees don't keep all the information in the source code. The epp_dodger can take care of well-behaved macros, but still for example integers get converted to their decimal value while in the source they could be in hex or octal. The "a_string " "another_string" construct gets merged into one string in the parse tree, too. With the advent of refactoring and other source code transformation tools, a pretty-printer that keeps track of how the original looks like is badly needed. Many tools have their own version of pretty-printer, but I'm not aware of any that goes all the way. The closest reference I know of is ParsErl (http://www.erlang.se/euc/07/papers/1200ParsErl.pdf), but I can't find any publicly available tool based on it. And as a final blow, we still have the case of badly-behaved macros, which are very difficult to handle in a meaningful way by any tool. I have had a plan to implement a pretty-printer for use with erlide since a long time ago, but there are many other things more urgent and it doesn't look like it's going to happen soon. If anyone starts working on this project, please feel free to contact me. best regards, Vlad From matthias@REDACTED Thu Oct 21 11:51:26 2010 From: matthias@REDACTED (Matthias Lang) Date: Thu, 21 Oct 2010 11:51:26 +0200 Subject: mostly a git question: how do I get my github repository back in sync? Message-ID: <20101021095126.GA5279@corelatus.se> Hi, I have a fork of erlang/otp on github, here: http://github.com/matthiasl/otp I made a branch a few months ago, add-os-pid-to-port-info. Bj?rn has since merged that branch into erlang/otp pu. Now, I want to get matthiasl/otp back up to date with erlang/otp so that I can make another change, starting with R14B. Q1: Is there a way to bring a github repository up to date from the github web interface? I can't find one. Maybe I missed it, maybe it's a dumb idea. Q2: Is this the recommended non-web way to do it? git clone http://github.com/erlang/otp.git git push git@REDACTED:matthiasl/otp.git or should I just give up and delete git@REDACTED:matthiasl/otp.git and then fork a new repository? That feels wrong. Matt From vladdu55@REDACTED Thu Oct 21 12:46:01 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 21 Oct 2010 12:46:01 +0200 Subject: [erlang-questions] mostly a git question: how do I get my github repository back in sync? In-Reply-To: <20101021095126.GA5279@corelatus.se> References: <20101021095126.GA5279@corelatus.se> Message-ID: On Thu, Oct 21, 2010 at 11:51, Matthias Lang wrote: > ?Q2: Is this the recommended non-web way to do it? > ? ? ? ?git clone http://github.com/erlang/otp.git > ? ? ? ?git push git@REDACTED:matthiasl/otp.git Hi, this will update only the master branch, use git push --all Depending on the state of your previous repo, an additional "-f" flag might be needed to force the update. regards, Vlad From joelr1@REDACTED Thu Oct 21 13:57:07 2010 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 21 Oct 2010 12:57:07 +0100 Subject: code path profiler Message-ID: <7621E8A1-0254-442D-A4C5-D30E08E79BE9@gmail.com> Existing tools (eprof, fprof) aggregate by process. This may be useful to analyze some systems but not server apps I usually deal with. In server apps it's useful to understand where the performance dips are along a path that starts with a request coming in and ends with the response going out. This sets up a timeline and I'd like to know what's happening alongside it. The timeline includes function calls and messages passed from one process to another. It also includes what the rest of the system is doing during that time (garbage collection?). Basically, I'd like to aggregate profiling information along a given repeatable code path, rather than per process. Then, I'd like to know the variation between the time taken to server requests during benchmarking. Does this make sense? Has anyone done anything along these lines? Thanks, Joel --- http://twitter.com/wagerlabs From ds.erl@REDACTED Thu Oct 21 13:00:28 2010 From: ds.erl@REDACTED (DS) Date: Thu, 21 Oct 2010 13:00:28 +0200 Subject: [erlang-questions]not able to make two nodes communicate on mac os In-Reply-To: References: Message-ID: <3D9C6E5C-8242-48D6-8743-24BEC2BA58B9@sol42.com> On 21 Oct 2010, at 11:46, Nrapesh Khamesra wrote: > erl -sname -test1 -setcookie abc Try "test1", without the leading dash. -DS From alessandro.sivieri@REDACTED Thu Oct 21 14:09:40 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Thu, 21 Oct 2010 14:09:40 +0200 Subject: JSON encoding of arrays of arrays Message-ID: Hi all, I'm struggling to find out how to properly write a structure that my MochiWeb server has to respond to a client (in JSON). The final JSON (if I have correctly written it) should be as follows: [ { "words": [ {"word": "someword1", "frequency": 10}, {"word": "someword2", "frequency": 8}, {"word": "someword3", "frequency": 3} ] }, { "words": [ {"word": "someword1", "frequency": 2}, {"word": "someword2", "frequency": 1}, {"word": "someword3", "frequency": 5} ] }, { "words": [ {"word": "someword1", "frequency": 4}, {"word": "someword2", "frequency": 9}, {"word": "someword3", "frequency": 12} ] } ] It comes from a word frequency analysis of some documents: each "words" element represents the output of a single document, and for each of them there is a list of words with their frequencies. Now, how should I write the struct in Erlang (given the fact that I am starting from a list of tuples for each document), so that mochijson2 produces the previous JSON? I have searched on the Web, many resources point to a video tutorial but it does not use a list of lists, so I have not been able to find out a way to write it... -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From joelr1@REDACTED Thu Oct 21 15:08:12 2010 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 21 Oct 2010 14:08:12 +0100 Subject: reading traces from a file in c/c++ Message-ID: What is the easiest way to incrementally read through a file of Erlang traces outside of Erlang? I could not see any functions for reading terms from file in the ei library. Is it a matter of always reading in a chunk big enough for any term, trying ei decoding functions and reading more of the same chunk if decoding fails? Thanks, Joel --- http://twitter.com/wagerlabs From ulf.wiger@REDACTED Thu Oct 21 15:27:40 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 21 Oct 2010 15:27:40 +0200 Subject: [erlang-questions] Pretty-printing Erlang In-Reply-To: References: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> <1287558222.4859.4.camel@seasc1137> <207A14F9-852E-4C93-BF21-206F6B9616F9@cs.otago.ac.nz> Message-ID: Actually, erl_scan now supports the options 'return_whitespace' and 'return_comments', so it should at least be possible to preserve formatting and comments in future tools. BR, Ulf W On 21 Oct 2010, at 12:03, Vlad Dumitrescu wrote: > With the advent of refactoring and other source code transformation > tools, a pretty-printer that keeps track of how the original looks > like is badly needed. Many tools have their own version of > pretty-printer, but I'm not aware of any that goes all the way. The > closest reference I know of is ParsErl > (http://www.erlang.se/euc/07/papers/1200ParsErl.pdf), but I can't find > any publicly available tool based on it. Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ingela.andin@REDACTED Thu Oct 21 15:32:22 2010 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 21 Oct 2010 15:32:22 +0200 Subject: [erlang-questions] Re: confusing returns from httpc:request/4 In-Reply-To: <8ecc3ba2-3f82-4d55-a46f-063f41012ce4@p37g2000pra.googlegroups.com> References: <8ecc3ba2-3f82-4d55-a46f-063f41012ce4@p37g2000pra.googlegroups.com> Message-ID: Hi! Humm ... what settings did you have when you had the problem? Did you set the pipeline timout ? What webserver did you connect to (apache, yaws, ...)? What version of inets? If there is some bug in the pipeling or persistenconnection code of course we would like to know so that we can fix it. Regards Ingela Erlang/OTP team - Ericsson AB 2010/10/20 amitm : > FWIW, I had a similar problem. It went away when I disabled pipelining > completely by executing > > ? ?ok = httpc:set_options([{max_keep_alive_length, 0}, > {max_pipeline_length, 0}, {max_sessions, 0} ]), > > before my http calls. I used to usually see it on the 4th consecutive > call to the same host when pipelining was enabled. The URL in question > in my case was an Amazon EC2 url. > > The set of URLs in my case were > > "http://169.254.169.254/2009-04-04/meta-data/instance-id", > "http://169.254.169.254/2009-04-04/meta-data/local-hostname", > "http://169.254.169.254/2009-04-04/meta-data/local-ipv4", > "http://169.254.169.254/2009-04-04/meta-data/public-hostname", > "http://169.254.169.254/2009-04-04/meta-data/public-ipv4", > "http://169.254.169.254/2009-04-04/meta-data/ami-id", > > While executing this set, with pipelining enabled, it used to always > bomb on the 4th URL, i.e. for 'public-hostname' in 25% of the cases > > with pipelining disabled, the problem disappears. > > ?Amit > > On Oct 13, 2:58?pm, Ingela Andin wrote: >> 2010/10/13 Chandru : >> >> >> >> >> >> > On 13 October 2010 09:53, Ingela Andin wrote: >> >> Hi! >> >> >> The only reason that the httpc-client will return the value {error, >> >> socket_closed_remotely} >> >> is that the connection is prematurely ?closed by the server. >> >> Unfortunately ?that something >> >> is sent on a tcp socket does not mean it has been sent on the wire >> >> yet, and depending on >> >> the circumstances it is possible for the socket to be closed before >> >> all data has been sent >> >> so occasionally a tcp_close can come before all data is delivered. >> >> >> When you use ibrowse do you verify that you always get a correct body? >> >> (compleate body) from what I can tell from the code ibrowse will on a >> >> tcp_closed ?message always return whatever happens to be in the buffer >> >> compleate or not. >> >> > ibrowse does this only in the following cases: >> > ?* server returned a "Connection: Close" >> > ?* Server supports HTTP/0.9 or HTTP/1.0 >> >> > In all other cases, it returns an error to the caller. >> >> > regards, >> > Chandru >> >> Well I do not know if any of these things is true in Dans senario. But >> I do know that >> getting the problem with a tcp close message before all data is >> deliverd is a highly timing dependant problem >> and if it happens in the server there is nothing inets-client or >> ibrows can do about it except return an error. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> Seehttp://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From steven.charles.davis@REDACTED Thu Oct 21 15:32:43 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 21 Oct 2010 06:32:43 -0700 (PDT) Subject: JSON encoding of arrays of arrays In-Reply-To: References: Message-ID: <48c4b938-8c14-4926-8b60-800236f3a16c@w19g2000yqb.googlegroups.com> The json syntax looks good, but I think this is an array of objects which contain arrays of objects. One way to encode an object would be to map it as a proplist, so an array of objects ends up as a list of proplists: [[{<<"words">>,[[{<<"word">>,<<"someword1">>}, {<<"frequency">>,10}], [{<<"word">>,<<"someword2">>},{<<"frequency">>,8}], [{<<"word">>,<<"someword3">>},{<<"frequency">>,3}]]}], [{<<"words">>,[[{<<"word">>,<<"someword1">>},{<<"frequency">>, 2}], [{<<"word">>,<<"someword2">>},{<<"frequency">>,1}], [{<<"word">>,<<"someword3">>},{<<"frequency">>,5}]]}], [{<<"words">>,[[{<<"word">>,<<"someword1">>},{<<"frequency">>, 4}], [{<<"word">>,<<"someword2">>},{<<"frequency">>,9}], [{<<"word">>,<<"someword3">>},{<<"frequency">>, 12}]]}]]. /s On Oct 21, 7:09?am, Alessandro Sivieri wrote: > Hi all, > > I'm struggling to find out how to properly write a structure that my > MochiWeb server has to respond to a client (in JSON). The final JSON (if I > have correctly written it) should be as follows: > > [ > ? ? { > ? ? ? ? "words": > ? [ > ? ? ? {"word": "someword1", "frequency": 10}, > ? ? ? ?{"word": "someword2", "frequency": 8}, > ? ? ? {"word": "someword3", "frequency": 3} > ? ] > ? ? }, > { > ? ? ? ? "words": > ? ? ? ? ? [ > ? ? ? ? ? ? ? {"word": "someword1", "frequency": 2}, > ? ? ? ? ? ? ? {"word": "someword2", "frequency": 1}, > ? ? ? ? ? ? ? {"word": "someword3", "frequency": 5} > ? ? ? ? ? ] > ? ? }, > { > ? ? ? ? "words": > ? ? ? ? ? [ > ? ? ? ? ? ? ? {"word": "someword1", "frequency": 4}, > ? ? ? ? ? ? ? {"word": "someword2", "frequency": 9}, > ? ? ? ? ? ? ? {"word": "someword3", "frequency": 12} > ? ? ? ? ? ] > ? ? } > ] > > It comes from a word frequency analysis of some documents: each "words" > element represents the output of a single document, and for each of them > there is a list of words with their frequencies. Now, how should I write the > struct in Erlang (given the fact that I am starting from a list of tuples > for each document), so that mochijson2 produces the previous JSON? > I have searched on the Web, many resources point to a video tutorial but it > does not use a list of lists, so I have not been able to find out a way to > write it... > > -- > Sivieri Alessandro > alessandro.sivi...@REDACTED://www.chimera-bellerofonte.eu/http://www.poul.org/ From raimo+erlang-questions@REDACTED Thu Oct 21 15:34:09 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 21 Oct 2010 15:34:09 +0200 Subject: [erlang-questions] mostly a git question: how do I get my github repository back in sync? In-Reply-To: <20101021095126.GA5279@corelatus.se> References: <20101021095126.GA5279@corelatus.se> Message-ID: <20101021133409.GA8838@erix.ericsson.se> On Thu, Oct 21, 2010 at 11:51:26AM +0200, Matthias Lang wrote: > Hi, > > I have a fork of erlang/otp on github, here: > > http://github.com/matthiasl/otp > > I made a branch a few months ago, add-os-pid-to-port-info. Bj?rn has > since merged that branch into erlang/otp pu. > > Now, I want to get matthiasl/otp back up to date with erlang/otp so > that I can make another change, starting with R14B. > > Q1: Is there a way to bring a github repository up to date from the > github web interface? > > I can't find one. Maybe I missed it, maybe it's a dumb idea. > > Q2: Is this the recommended non-web way to do it? > > git clone http://github.com/erlang/otp.git > git push git@REDACTED:matthiasl/otp.git That would only push the 'master' branch. And it would re-create and download a whole new local repository. If you append which branches to push it should work, the -f flag might also be needed. Using an existing repository: For branch 'dev': git checkout dev git pull origin git push upstream dev providing origin is erlang/otp and upstream is matthiasl/otp. The push might need a -f flag. Other variant, more forceful, 3 branches: git checkout dev git fetch origin git reset --hard origin/dev git branch -f master origin/master git branch -f pu origin/pu git push -f upstream dev master pu For dev, git reset is used since it is checked out. > > or should I just give up and delete git@REDACTED:matthiasl/otp.git > and then fork a new repository? That feels wrong. > > Matt > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From vladdu55@REDACTED Thu Oct 21 15:43:00 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 21 Oct 2010 15:43:00 +0200 Subject: [erlang-questions] Pretty-printing Erlang In-Reply-To: References: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> <1287558222.4859.4.camel@seasc1137> <207A14F9-852E-4C93-BF21-206F6B9616F9@cs.otago.ac.nz> Message-ID: On Thu, Oct 21, 2010 at 15:27, Ulf Wiger wrote: > > Actually, erl_scan now supports the options 'return_whitespace' and > 'return_comments', so it should at least be possible to preserve > formatting and comments in future tools. Yes, the scanner is now almost good enough for this purpose (I believe that instead of column information it should use offset from the start of the buffer, because it makes tools further up the chain simpler) . The next step is to adapt the parser. More precisely, I think that a separate parser is a better idea, because the requirements between parsing-before-compiling and parsing-before-processing-source are different. In the latter case, besides the points I mentioned before, one has to have a better error handling too, so that one can even handle incomplete or malformed code (something like the preprocessor does with "weird" macros, treating them as text, but at a higher level). best regards, Vlad From amit.murthy@REDACTED Thu Oct 21 16:06:46 2010 From: amit.murthy@REDACTED (amitm) Date: Thu, 21 Oct 2010 07:06:46 -0700 (PDT) Subject: confusing returns from httpc:request/4 In-Reply-To: References: <8ecc3ba2-3f82-4d55-a46f-063f41012ce4@p37g2000pra.googlegroups.com> Message-ID: <3e9cd45e-4e88-42cb-8a7f-b2666b069d2c@w9g2000prc.googlegroups.com> > Humm ... what settings did you have when you had the problem? The function making the call in the situation when the problem manifests itself is: mkGetReq2(GetUrl) -> GetRequest = {GetUrl, []}, GetHttpOptions = [{autoredirect, true}], Options = [ {sync,true}, {headers_as_is,true}, {body_format, binary} ], httpc:request(get, GetRequest, GetHttpOptions, Options). No other options were set, which means they run with their default values. > Did you set the pipeline timout ? No pipeline timeout was set. > What webserver did you connect to (apache, yaws, ...)? The webserver is one used by Amazon internally on EC2. Curl in verbose mode, as executed by the command "curl -v http://169.254.169.254/2009-04-04/meta-data/instance-id " prints out * About to connect() to 169.254.169.254 port 80 (#0) * Trying 169.254.169.254... connected * Connected to 169.254.169.254 (169.254.169.254) port 80 (#0) > GET /2009-04-04/meta-data/instance-id HTTP/1.1 > User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/ 0.9.8k zlib/1.2.3.3 libidn/1.15 > Host: 169.254.169.254 > Accept: */* > * HTTP 1.0, assume close after body < HTTP/1.0 200 OK < Content-Type: text/plain < Accept-Ranges: bytes < ETag: "-1124024050" < Last-Modified: Tue, 19 Oct 2010 13:58:54 GMT < Content-Length: 10 < Connection: close < Date: Thu, 21 Oct 2010 13:48:19 GMT < Server: EC2ws < * Closing connection #0 > What version of inets? erl -v prints "Erlang R14B (erts-5.8.1) [source] [rq:1] [async-threads:0] [kernel- poll:false]" Hope it helps, Amit On Oct 21, 6:32?pm, Ingela Andin wrote: > Hi! > > Humm ... what settings did you have when you had the problem? > Did you set the pipeline timout ? What webserver did you connect to > (apache, yaws, ...)? What > version of inets? If there is some bug in the pipeling or > persistenconnection code > of course we would like to know so that we can fix it. > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2010/10/20 amitm : > > > > > > > FWIW, I had a similar problem. It went away when I disabled pipelining > > completely by executing > > > ? ?ok = httpc:set_options([{max_keep_alive_length, 0}, > > {max_pipeline_length, 0}, {max_sessions, 0} ]), > > > before my http calls. I used to usually see it on the 4th consecutive > > call to the same host when pipelining was enabled. The URL in question > > in my case was an Amazon EC2 url. > > > The set of URLs in my case were > > > "http://169.254.169.254/2009-04-04/meta-data/instance-id", > > "http://169.254.169.254/2009-04-04/meta-data/local-hostname", > > "http://169.254.169.254/2009-04-04/meta-data/local-ipv4", > > "http://169.254.169.254/2009-04-04/meta-data/public-hostname", > > "http://169.254.169.254/2009-04-04/meta-data/public-ipv4", > > "http://169.254.169.254/2009-04-04/meta-data/ami-id", > > > While executing this set, with pipelining enabled, it used to always > > bomb on the 4th URL, i.e. for 'public-hostname' in 25% of the cases > > > with pipelining disabled, the problem disappears. > > > ?Amit > > > On Oct 13, 2:58?pm, Ingela Andin wrote: > >> 2010/10/13 Chandru : > > >> > On 13 October 2010 09:53, Ingela Andin wrote: > >> >> Hi! > > >> >> The only reason that the httpc-client will return the value {error, > >> >> socket_closed_remotely} > >> >> is that the connection is prematurely ?closed by the server. > >> >> Unfortunately ?that something > >> >> is sent on a tcp socket does not mean it has been sent on the wire > >> >> yet, and depending on > >> >> the circumstances it is possible for the socket to be closed before > >> >> all data has been sent > >> >> so occasionally a tcp_close can come before all data is delivered. > > >> >> When you use ibrowse do you verify that you always get a correct body? > >> >> (compleate body) from what I can tell from the code ibrowse will on a > >> >> tcp_closed ?message always return whatever happens to be in the buffer > >> >> compleate or not. > > >> > ibrowse does this only in the following cases: > >> > ?* server returned a "Connection: Close" > >> > ?* Server supports HTTP/0.9 or HTTP/1.0 > > >> > In all other cases, it returns an error to the caller. > > >> > regards, > >> > Chandru > > >> Well I do not know if any of these things is true in Dans senario. But > >> I do know that > >> getting the problem with a tcp close message before all data is > >> deliverd is a highly timing dependant problem > >> and if it happens in the server there is nothing inets-client or > >> ibrows can do about it except return an error. > > >> Regards Ingela Erlang/OTP team - Ericsson AB > > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> Seehttp://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > Seehttp://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From jiansenhe@REDACTED Thu Oct 21 17:27:29 2010 From: jiansenhe@REDACTED (Jiansen He) Date: Thu, 21 Oct 2010 16:27:29 +0100 Subject: [erlang-questions] code mobility In-Reply-To: References: Message-ID: Thank you for your reply, code:load_binary/3 is what I used in one of my demo programs. My code looks like this: --server.erl -module(server7). -export([start_server/0, server/0]). server() -> io:format("Server initializing...~n", []), receive {FUN, I, Client} -> io:format("Server:message received~n"), io:format("erlang:fun_info: ~w~n", [erlang:fun_info(FUN)]), io:format("result for ~w ~w is ~w~n",[FUN, I,FUN(I)]), Client ! FUN(I), io:format("result sent~n"), server() end. % initialize start_server() -> register(server, spawn(server7, server, [])). --client.erl -module(client). -export([start_client/1, client/3, fact/1, isEven/1, isOdd/1]). % client side code fact(0) -> 1; fact(N) -> N * fact(N-1). isEven(0) -> true; isEven(N) -> isOdd(N-1). isOdd(0) -> false; isOdd(N) -> isEven(N-1). start_client(Server_Node) -> {Mod, Bin, File} = code:get_object_code(client), rpc:multicall(code, load_binary, [Mod, File, Bin]), spawn(client, client, [Server_Node, fun client:fact/1, 4]). The above code will load client.beam to all nodes. If I only want send client.beam to server, where the code is really executed, I can use rpc:call/4 instead. There are two other trivial consideration in the above implementation. For one thing, I send the whole file to the server. The function is sent with other irrelevant functions. For another thing, what if I want to send a complex function which employs functions from other modules? Shall I know every details about the complex function and send involved files myself? For example, isEven/1 is defined in E.erl and isOdd is defined in O.erl. Both files are well documented because the functionality and usage of each function are clearly specified. Programmers could use isEven and isOdd as APIs. What users don't know (and don't need to know) is, how isOdd and isEven is implemented. Regards Jiansen From ingela.andin@REDACTED Thu Oct 21 17:54:02 2010 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 21 Oct 2010 17:54:02 +0200 Subject: [erlang-questions] Re: confusing returns from httpc:request/4 In-Reply-To: <3e9cd45e-4e88-42cb-8a7f-b2666b069d2c@w9g2000prc.googlegroups.com> References: <8ecc3ba2-3f82-4d55-a46f-063f41012ce4@p37g2000pra.googlegroups.com> <3e9cd45e-4e88-42cb-8a7f-b2666b069d2c@w9g2000prc.googlegroups.com> Message-ID: Hi! Yes thank you for the information. I think I have located the bug. It seems that to avoid crash reports the return value when receiving a tcp_closed was changed and this altered the terminate clause that is run, alas breaking the automatic retries that should be done if the server terminates a connection before serving all pipelined requests. This will be fixed in an upcoming release. Regards Ingela Erlang/OTP-team, Ericsson AB 2010/10/21 amitm : > >> Humm ... what settings did you have when you had the problem? > > The function making the call in the situation when the problem > manifests itself is: > > mkGetReq2(GetUrl) -> > ? ?GetRequest = {GetUrl, []}, > ? ?GetHttpOptions = [{autoredirect, true}], > ? ?Options = [ {sync,true}, {headers_as_is,true}, {body_format, > binary} ], > ? ?httpc:request(get, GetRequest, GetHttpOptions, Options). > > No other options were set, which means they run with their default > values. > >> Did you set the pipeline timout ? > > No pipeline timeout was set. > >> What webserver did you connect to (apache, yaws, ...)? > > The webserver is one used by Amazon internally on EC2. Curl in verbose > mode, as executed by the command > "curl -v http://169.254.169.254/2009-04-04/meta-data/instance-id " > prints out > > ?* About to connect() to 169.254.169.254 port 80 (#0) > ?* ? Trying 169.254.169.254... connected > ?* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0) > ?> GET /2009-04-04/meta-data/instance-id HTTP/1.1 > ?> User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/ > 0.9.8k zlib/1.2.3.3 libidn/1.15 > ?> Host: 169.254.169.254 > ?> Accept: */* > ?> > ?* HTTP 1.0, assume close after body > ?< HTTP/1.0 200 OK > ?< Content-Type: text/plain > ?< Accept-Ranges: bytes > ?< ETag: "-1124024050" > ?< Last-Modified: Tue, 19 Oct 2010 13:58:54 GMT > ?< Content-Length: 10 > ?< Connection: close > ?< Date: Thu, 21 Oct 2010 13:48:19 GMT > ?< Server: EC2ws > ?< > ?* Closing connection #0 > >> What version of inets? > > erl -v prints > "Erlang R14B (erts-5.8.1) [source] [rq:1] [async-threads:0] [kernel- > poll:false]" > > Hope it helps, > > ?Amit > > > On Oct 21, 6:32?pm, Ingela Andin wrote: >> Hi! >> >> Humm ... what settings did you have when you had the problem? >> Did you set the pipeline timout ? What webserver did you connect to >> (apache, yaws, ...)? What >> version of inets? If there is some bug in the pipeling or >> persistenconnection code >> of course we would like to know so that we can fix it. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> 2010/10/20 amitm : >> >> >> >> >> >> > FWIW, I had a similar problem. It went away when I disabled pipelining >> > completely by executing >> >> > ? ?ok = httpc:set_options([{max_keep_alive_length, 0}, >> > {max_pipeline_length, 0}, {max_sessions, 0} ]), >> >> > before my http calls. I used to usually see it on the 4th consecutive >> > call to the same host when pipelining was enabled. The URL in question >> > in my case was an Amazon EC2 url. >> >> > The set of URLs in my case were >> >> > "http://169.254.169.254/2009-04-04/meta-data/instance-id", >> > "http://169.254.169.254/2009-04-04/meta-data/local-hostname", >> > "http://169.254.169.254/2009-04-04/meta-data/local-ipv4", >> > "http://169.254.169.254/2009-04-04/meta-data/public-hostname", >> > "http://169.254.169.254/2009-04-04/meta-data/public-ipv4", >> > "http://169.254.169.254/2009-04-04/meta-data/ami-id", >> >> > While executing this set, with pipelining enabled, it used to always >> > bomb on the 4th URL, i.e. for 'public-hostname' in 25% of the cases >> >> > with pipelining disabled, the problem disappears. >> >> > ?Amit >> >> > On Oct 13, 2:58?pm, Ingela Andin wrote: >> >> 2010/10/13 Chandru : >> >> >> > On 13 October 2010 09:53, Ingela Andin wrote: >> >> >> Hi! >> >> >> >> The only reason that the httpc-client will return the value {error, >> >> >> socket_closed_remotely} >> >> >> is that the connection is prematurely ?closed by the server. >> >> >> Unfortunately ?that something >> >> >> is sent on a tcp socket does not mean it has been sent on the wire >> >> >> yet, and depending on >> >> >> the circumstances it is possible for the socket to be closed before >> >> >> all data has been sent >> >> >> so occasionally a tcp_close can come before all data is delivered. >> >> >> >> When you use ibrowse do you verify that you always get a correct body? >> >> >> (compleate body) from what I can tell from the code ibrowse will on a >> >> >> tcp_closed ?message always return whatever happens to be in the buffer >> >> >> compleate or not. >> >> >> > ibrowse does this only in the following cases: >> >> > ?* server returned a "Connection: Close" >> >> > ?* Server supports HTTP/0.9 or HTTP/1.0 >> >> >> > In all other cases, it returns an error to the caller. >> >> >> > regards, >> >> > Chandru >> >> >> Well I do not know if any of these things is true in Dans senario. But >> >> I do know that >> >> getting the problem with a tcp close message before all data is >> >> deliverd is a highly timing dependant problem >> >> and if it happens in the server there is nothing inets-client or >> >> ibrows can do about it except return an error. >> >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> >> ________________________________________________________________ >> >> erlang-questions (at) erlang.org mailing list. >> >> Seehttp://www.erlang.org/faq.html >> >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >> >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > Seehttp://www.erlang.org/faq.html >> > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> Seehttp://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From fabiomiranda@REDACTED Thu Oct 21 18:11:01 2010 From: fabiomiranda@REDACTED (Fabio Miranda) Date: Thu, 21 Oct 2010 11:11:01 -0500 Subject: Programming a recursive function that executes n times Message-ID: Hello, I want to simulate a time consuming task on Erlang, let's say I want to execute 10 000 times the above line: test_mnesia:busqueda(cars). How can I accomplish this on Erlang? thanks, fabio. From allanwegan@REDACTED Thu Oct 21 18:22:30 2010 From: allanwegan@REDACTED (Allan Wegan) Date: Thu, 21 Oct 2010 18:22:30 +0200 Subject: [erlang-questions] Programming a recursive function that executes n times In-Reply-To: References: Message-ID: <4CC068C6.1080308@allanwegan.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > I want to simulate a time consuming task on Erlang, let's say I want to execute 10 000 times the above line: > > test_mnesia:busqueda(cars). > > How can I accomplish this on Erlang? Try: fun test() -> Test_fun = fun (F, 0) -> 'undefined'; (F, N) -> test_mnesia:busqueda(cars), F(F, N - 1) end, Test_fun(Test_fun, 10000) . - -- Allan Wegan Jabber: allanwegan@REDACTED ICQ: 209459114 Phone: +49 40 6732962 Sch?neberger Stra?e 60, 22149 Hamburg -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (MingW32) iQEcBAEBAgAGBQJMwGjGAAoJENm5axHh7AcxkQ4IALfcClXsAdPZXos3cEHDgVCB SOtjB9evtY8FSTC/yx0d3iHcyD2qIDUWv0ekqNEBr9MR21KS5eT6NxusHZTS+xk6 yZXpW9ja9nOAiDkjjvLBVBx6O3V+0JZ2Xgx1L1UuXjq0LfBbpbTUgbIBOODk7Z01 oDyv7wTo/igjxhrMcHSMAlGIoXAEmAzPanfq3zKxJXQPYRYvc7Uh5MMZAQp8F62f +MMGUYCnlGonXA4rSTWCF6lathY+r/DfLgcehJu4GDxAM9q6k1yVVzLQTCUahKY6 D26oWvisWZoqUR8VANiW9xdZuYKPFee6PIRatRITnUVEass5gyQbUKRJCppBE4Q= =o1Lg -----END PGP SIGNATURE----- From dima@REDACTED Thu Oct 21 18:35:03 2010 From: dima@REDACTED (Dmitry Vasiliev) Date: Thu, 21 Oct 2010 20:35:03 +0400 Subject: [erlang-questions] code mobility In-Reply-To: References: Message-ID: <4CC06BB7.1070309@hlabs.org> 21.10.2010 19:27, Jiansen He ?????: > Thank you for your reply, > > code:load_binary/3 is what I used in one of my demo programs. My code looks > like this: [skip] > The above code will load client.beam to all nodes. If I only want send > client.beam to server, where the code is really executed, I can use > rpc:call/4 instead. There are two other trivial consideration in the above > implementation. > > For one thing, I send the whole file to the server. The function is sent > with other irrelevant functions. > > For another thing, what if I want to send a complex function which employs > functions from other modules? Shall I know every details about the complex > function and send involved files myself? For example, isEven/1 is defined > in E.erl and isOdd is defined in O.erl. Both files are well documented > because the functionality and usage of each function are clearly specified. > Programmers could use isEven and isOdd as APIs. What users don't know (and > don't need to know) is, how isOdd and isEven is implemented. Maybe you need to take a look at erl_boot_server module or implement some "on demand" remote module loading yourself? -- Dmitry Vasiliev http://hlabs.org http://twitter.com/hdima From fritchie@REDACTED Thu Oct 21 18:41:19 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 21 Oct 2010 11:41:19 -0500 Subject: [erlang-questions] code path profiler In-Reply-To: Message of "Thu, 21 Oct 2010 12:57:07 BST." <7621E8A1-0254-442D-A4C5-D30E08E79BE9@gmail.com> Message-ID: <68707.1287679279@snookles.snookles.com> Joel Reymont wrote: jr> Basically, I'd like to aggregate profiling information along a given jr> repeatable code path, rather than per process. Then, I'd like to jr> know the variation between the time taken to server requests during jr> benchmarking. Joel, it might be possible to have the profile tracing based on a sequential tracing token rather than per process, but I'm not aware of any OTP tool that can do that out of the box. The kernel app's 'seq_trace' module describes how it works. There's a short section "Advanced topics - combining with seq_trace" in the reference for the 'dbg' app. The 'ttb' docs also very briefly mention sequential tracing tokens. -Scott From spawn.think@REDACTED Thu Oct 21 21:18:15 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Thu, 21 Oct 2010 21:18:15 +0200 Subject: [erlang-questions] Programming a recursive function that executes n times In-Reply-To: <4CC068C6.1080308@allanwegan.de> References: <4CC068C6.1080308@allanwegan.de> Message-ID: Time consuming sounds a bit vage , do you want it to be "busy" during that time? or just "idle"? - To consume time just "waiting", you can user timer:sleep/1 , or do a receive - To repeat a task for N of times you can , lists:foreach(fun(_)-> test_mnesia:busqueda(cars) end, lists:seq(1,N)). Hope you found what you need On Thu, Oct 21, 2010 at 6:22 PM, Allan Wegan wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > I want to simulate a time consuming task on Erlang, let's say I want > to execute 10 000 times the above line: > > > > test_mnesia:busqueda(cars). > > > > How can I accomplish this on Erlang? > > Try: > fun test() -> > Test_fun = fun (F, 0) -> 'undefined'; (F, N) -> > test_mnesia:busqueda(cars), F(F, N - 1) end, > Test_fun(Test_fun, 10000) > . > > - -- > Allan Wegan > Jabber: allanwegan@REDACTED > ICQ: 209459114 > Phone: +49 40 6732962 > Sch?neberger Stra?e 60, 22149 Hamburg > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.14 (MingW32) > > iQEcBAEBAgAGBQJMwGjGAAoJENm5axHh7AcxkQ4IALfcClXsAdPZXos3cEHDgVCB > SOtjB9evtY8FSTC/yx0d3iHcyD2qIDUWv0ekqNEBr9MR21KS5eT6NxusHZTS+xk6 > yZXpW9ja9nOAiDkjjvLBVBx6O3V+0JZ2Xgx1L1UuXjq0LfBbpbTUgbIBOODk7Z01 > oDyv7wTo/igjxhrMcHSMAlGIoXAEmAzPanfq3zKxJXQPYRYvc7Uh5MMZAQp8F62f > +MMGUYCnlGonXA4rSTWCF6lathY+r/DfLgcehJu4GDxAM9q6k1yVVzLQTCUahKY6 > D26oWvisWZoqUR8VANiW9xdZuYKPFee6PIRatRITnUVEass5gyQbUKRJCppBE4Q= > =o1Lg > -----END PGP SIGNATURE----- > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From serge@REDACTED Thu Oct 21 21:50:55 2010 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 21 Oct 2010 15:50:55 -0400 Subject: [erlang-questions] reading traces from a file in c/c++ In-Reply-To: References: Message-ID: <4CC0999F.8040605@aleynikov.org> Hi Joel, I don't have an immediate solution to your question, but this has been on my to-do list for a while as I am in the process of rewriting the EPI project (http://www.erlang.org/pipermail/erlang-questions/2005-March/014930.html, http://code.google.com/p/epi/) to make certain Erlang-like features available in C++. What I got working so far is: 1. Support for all types of terms: long/double/string/atom/pid/port/ref via reference counting or copying depending on object's footprint. All terms are pretty intelligent and lightweight in a way that you never need to call new/delete in your code. 2. Support for an atom table. 3. All the marshaling/demarshaling functionality. 4. Distributed client-side connectivity to Erlang nodes and appropriate epmd connectivity for port discovery. 5. Support for mailboxes with fully asynchronous API allowing making non-blocking mailbox::recv() calls with completion notifications. 6. The entire application can run in a single thread (everything is non-blocking) or many threads if needed (currently using boost::asio for event looping, dispatching and networking). The next item on my to do list is to add server-side support for creating a C++ Erlang node with epmd registration. Reading Erlang terms from a file can be accomplished by writing a boost::spirit based parser, and a similar example is available here: http://svn.boost.org/svn/boost/trunk/libs/property_tree/examples/info_grammar_spirit.cpp I am planning to release the project to public once it's in somewhat more complete and working stage. However, if you wish to contribute, and write a parser to read and transform a text file in Erlang terms from my C++ library, there's room for some joint work. Serge On 10/21/2010 9:08 AM, Joel Reymont wrote: > What is the easiest way to incrementally read through a file of Erlang traces outside of Erlang? > > I could not see any functions for reading terms from file in the ei library. > > Is it a matter of always reading in a chunk big enough for any term, trying ei decoding functions and reading more of the same chunk if decoding fails? > > Thanks, Joel > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From rgowka1@REDACTED Thu Oct 21 21:56:12 2010 From: rgowka1@REDACTED (rgowka1) Date: Thu, 21 Oct 2010 15:56:12 -0400 Subject: map over bitstring Message-ID: Hi - How do I map a function over a bitstring without converting into string or list. For example - how can I replace << "a" >> with << "X" >> and any other character to << "Y" >>?? InBin = << "abcaa" >> OutBin = << "XYYXX" >> thanks. From spawn.think@REDACTED Thu Oct 21 21:58:19 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Thu, 21 Oct 2010 21:58:19 +0200 Subject: [erlang-questions] Programming a recursive function that executes n times In-Reply-To: References: <4CC068C6.1080308@allanwegan.de> Message-ID: ah Sorry i missed the title "Recursive" ! On Thu, Oct 21, 2010 at 9:18 PM, Ahmed Omar wrote: > Time consuming sounds a bit vage , do you want it to be "busy" during that > time? or just "idle"? > - To consume time just "waiting", you can user timer:sleep/1 , or do a > receive > - To repeat a task for N of times you can , lists:foreach(fun(_)-> test_mnesia:busqueda(cars) > end, lists:seq(1,N)). > > Hope you found what you need > > On Thu, Oct 21, 2010 at 6:22 PM, Allan Wegan wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> > I want to simulate a time consuming task on Erlang, let's say I want >> to execute 10 000 times the above line: >> > >> > test_mnesia:busqueda(cars). >> > >> > How can I accomplish this on Erlang? >> >> Try: >> fun test() -> >> Test_fun = fun (F, 0) -> 'undefined'; (F, N) -> >> test_mnesia:busqueda(cars), F(F, N - 1) end, >> Test_fun(Test_fun, 10000) >> . >> >> - -- >> Allan Wegan >> Jabber: allanwegan@REDACTED >> ICQ: 209459114 >> Phone: +49 40 6732962 >> Sch?neberger Stra?e 60, 22149 Hamburg >> >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v2.0.14 (MingW32) >> >> iQEcBAEBAgAGBQJMwGjGAAoJENm5axHh7AcxkQ4IALfcClXsAdPZXos3cEHDgVCB >> SOtjB9evtY8FSTC/yx0d3iHcyD2qIDUWv0ekqNEBr9MR21KS5eT6NxusHZTS+xk6 >> yZXpW9ja9nOAiDkjjvLBVBx6O3V+0JZ2Xgx1L1UuXjq0LfBbpbTUgbIBOODk7Z01 >> oDyv7wTo/igjxhrMcHSMAlGIoXAEmAzPanfq3zKxJXQPYRYvc7Uh5MMZAQp8F62f >> +MMGUYCnlGonXA4rSTWCF6lathY+r/DfLgcehJu4GDxAM9q6k1yVVzLQTCUahKY6 >> D26oWvisWZoqUR8VANiW9xdZuYKPFee6PIRatRITnUVEass5gyQbUKRJCppBE4Q= >> =o1Lg >> -----END PGP SIGNATURE----- >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From jesper.louis.andersen@REDACTED Thu Oct 21 22:10:23 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 21 Oct 2010 22:10:23 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: References: Message-ID: On Thu, Oct 21, 2010 at 9:56 PM, rgowka1 wrote: > Hi - > > How do I map a function over a bitstring without converting into string or list. > > For example - how can I replace << "a" >> with << "X" >> and any other > character to << "Y" >>?? > > InBin = << "abcaa" >> > > OutBin = << "XYYXX" >> The trick is Binary Comprehensions or by using the (R14B) binary module. Here is a solution using the Binary Comprehensions: 3> << < $X; _ -> $Y end>> || <> <= <<"abcaa" >> >>. <<"XYYXX">> -- J. From gregory.haskins@REDACTED Thu Oct 21 22:39:06 2010 From: gregory.haskins@REDACTED (Gregory Haskins) Date: Thu, 21 Oct 2010 16:39:06 -0400 Subject: struggling with port_command() Message-ID: <5EDD7341-13CB-4D95-9E97-0A40D92190D1@gmail.com> Hi All, I've been struggling all afternoon with what is perhaps a completely trivial problem, but I can't seem to get past it. Ultimately, I want to be able to have a simple external port program transform a string (XML, in fact, but that shouldn't matter). So I want to have my erlang program pump a string in on stdin, and then get a string back out on stdout. Simple stuff, but it just will not work for me. I scoured the net for examples and/or threads on problems in doing this, but was unable to figure out what I am doing wrong. Here is an escript which exemplifies the issue: #!/usr/bin/env escript pipe_stdin(Port) -> case io:get_line("") of eof -> %port_close(Port), ok; Data -> true = port_command(Port, Data), pipe_stdin(Port) end. main([Cmd]) -> io:format("Executing ~s~n", [Cmd]), Port = erlang:open_port({spawn, Cmd}, [stream, use_stdio, exit_status]), ok = pipe_stdin(Port), {ok, _Status, Output} = cmd_receive(Port, ""), io:format("-----~s~n", [Output]), ok. cmd_receive(Port, Acc) -> receive {Port, {data, Data}} -> cmd_receive(Port, string:concat(Acc, Data)); {Port, {exit_status, Status}} -> {ok, Status, Acc}; Else -> throw({unexpected_msg, Else}) end. The problem I am seeing is that the data never seems to reach the port program (with or without the port_close()). I pump the data in with port_command() and it returns "true" but nothing seems to come out the other side. Any help would be greatly appreciated. -Greg PS: As a secondary question, is the right way to signal EOF on the stdin to the port to use port_close()? -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 203 bytes Desc: This is a digitally signed message part URL: From ok@REDACTED Fri Oct 22 00:04:30 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Oct 2010 11:04:30 +1300 Subject: [erlang-questions] Pretty-printing Erlang In-Reply-To: <1287639191.5435.9.camel@seasc1137> References: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> <1287558222.4859.4.camel@seasc1137> <207A14F9-852E-4C93-BF21-206F6B9616F9@cs.otago.ac.nz> <1287639191.5435.9.camel@seasc1137> Message-ID: <032D4B20-14F2-4384-9D62-8302133CCB00@cs.otago.ac.nz> On 21/10/2010, at 6:33 PM, Bengt Kleberg wrote: > The clean up effect you describe as the beyond the scope of a pretty > printer might be achieve by tidier > (http://www.erlang-factory.com/conference/London2009/speakers/kostissagonas). I cannot connect to the http://tidier.softlab.ntua.gr/ site; is that still the correct site for tidier? > > The problem with several body level expressions on one line is a problem > that I either avoid by having long variable/function names, or fix > manually after wards. It would be nice to get it done automatically. The problem here is that the original code didn't *have* several expressions on one line; erl_tidy *un*tidied that code. From alessandro.sivieri@REDACTED Fri Oct 22 00:34:58 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Fri, 22 Oct 2010 00:34:58 +0200 Subject: [erlang-questions] JSON encoding of arrays of arrays In-Reply-To: References: Message-ID: 2010/10/22 Paul Oliver > The way I've seen it done by a guy at work is to reverse engineer it > by passing what you want back through mochijson2:decode > > Yes, I did exactly this in the end; thanks! -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From bob@REDACTED Fri Oct 22 03:11:18 2010 From: bob@REDACTED (Bob Ippolito) Date: Fri, 22 Oct 2010 09:11:18 +0800 Subject: [erlang-questions] JSON encoding of arrays of arrays In-Reply-To: References: Message-ID: I'm surprised that people are finding it to be complicated. A list of lists is literally an erlang list with erlang lists in it. It looks like the documentation does need some improvement though, it's only really obvious if you use it or if you read through the data types. {struct, [{binary(), json_term()}]} is an object [json_term()] is a list binary() is a string integer()/float() are number true/false/null atoms map to the same terms in json. On Fri, Oct 22, 2010 at 6:34 AM, Alessandro Sivieri wrote: > 2010/10/22 Paul Oliver > >> The way I've seen it done by a guy at work is to reverse engineer it >> by passing what you want back through mochijson2:decode >> >> > Yes, I did exactly this in the end; thanks! > > -- > Sivieri Alessandro > alessandro.sivieri@REDACTED > http://www.chimera-bellerofonte.eu/ > http://www.poul.org/ > From rgowka1@REDACTED Fri Oct 22 06:55:06 2010 From: rgowka1@REDACTED (rgowka1) Date: Fri, 22 Oct 2010 00:55:06 -0400 Subject: [erlang-questions] map over bitstring In-Reply-To: References: Message-ID: Thanks, that was really helpful. One more question, how do I zip on two bitstrings. On 10/21/10, Jesper Louis Andersen wrote: > On Thu, Oct 21, 2010 at 9:56 PM, rgowka1 wrote: >> Hi - >> >> How do I map a function over a bitstring without converting into string or >> list. >> >> For example - how can I replace << "a" >> with << "X" >> and any other >> character to << "Y" >>?? >> >> InBin = << "abcaa" >> >> >> OutBin = << "XYYXX" >> > > The trick is Binary Comprehensions or by using the (R14B) binary > module. Here is a solution using the Binary Comprehensions: > > 3> << < $X; _ -> $Y end>> || <> <= <<"abcaa" >> >>. > <<"XYYXX">> > > -- > J. > From bengt.kleberg@REDACTED Fri Oct 22 09:20:05 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 22 Oct 2010 09:20:05 +0200 Subject: [erlang-questions] Pretty-printing Erlang In-Reply-To: <032D4B20-14F2-4384-9D62-8302133CCB00@cs.otago.ac.nz> References: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> <1287558222.4859.4.camel@seasc1137> <207A14F9-852E-4C93-BF21-206F6B9616F9@cs.otago.ac.nz> <1287639191.5435.9.camel@seasc1137> <032D4B20-14F2-4384-9D62-8302133CCB00@cs.otago.ac.nz> Message-ID: <1287732005.7736.10.camel@seasc1137> You are correct, the site mentioned in the paper about tidier is not responding. I have no idea as to why, nor where tidier has gone. You are also correct (IMHO) about erl_tidy (and erl_prettypr) making things less tidy (pretty) when several body level expressions are placed on one line. bengt On Fri, 2010-10-22 at 00:04 +0200, Richard O'Keefe wrote: > On 21/10/2010, at 6:33 PM, Bengt Kleberg wrote: > > > The clean up effect you describe as the beyond the scope of a pretty > > printer might be achieve by tidier > > (http://www.erlang-factory.com/conference/London2009/speakers/kostissagonas). > > I cannot connect to the http://tidier.softlab.ntua.gr/ site; > is that still the correct site for tidier? > > > > > The problem with several body level expressions on one line is a problem > > that I either avoid by having long variable/function names, or fix > > manually after wards. It would be nice to get it done automatically. > > The problem here is that the original code didn't *have* several > expressions on one line; erl_tidy *un*tidied that code. > From raimo+erlang-questions@REDACTED Fri Oct 22 09:30:52 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 22 Oct 2010 09:30:52 +0200 Subject: [erlang-questions] Programming a recursive function that executes n times In-Reply-To: References: Message-ID: <20101022073052.GA16824@erix.ericsson.se> On Thu, Oct 21, 2010 at 11:11:01AM -0500, Fabio Miranda wrote: > > Hello, > > I want to simulate a time consuming task on Erlang, let's say I want to execute 10 000 times the above line: > > test_mnesia:busqueda(cars). > > > How can I accomplish this on Erlang? Er, is this an absolute beginners question, or is it an intricate expert question that I am missing the point in?... Assuming it is an absolute beginners question this might worth the reading: http://www.erlang.org/starting.html And here is an uncompiled program for you: -module(foo). -export([repeat/2]). repeat(Fun, N) when is_function(Fun, 0), is_integer(N), N >= 0 -> if N > 0 -> Fun(), repeat(Fun, N-1); true -> ok end. Call as: foo:repeat(fun () -> test_mnesia:busqueda(cars) end, 10000) Another poster gave you an equivalent answer using a shell fun, which is a wee bit more complicated but you it depends if you want to do it from the shell or from a program. If you want to collect the results another poster gave you a suggestion a'la (allready in stdlib, just call): [test_mnesia:busqueda(cars) || X <- lists:seq(1, 10000)] You get the results in a list, and can do it either from the shell or from a program. > > thanks, > > fabio. > > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Fri Oct 22 09:49:45 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 22 Oct 2010 09:49:45 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: References: Message-ID: <20101022074945.GB16824@erix.ericsson.se> On Fri, Oct 22, 2010 at 12:55:06AM -0400, rgowka1 wrote: > Thanks, that was really helpful. > > One more question, how do I zip on two bitstrings. For that you need to write a function of your own, matching out the heads of both bitstrings and building a new by appending. Uncompiled: zip(A, B) -> zip(A, B, <<>>). zip(<>, <>, R) -> zip(A, B, <>); zip(<<>>, <<>>, R) -> R. The comprehensions can not do it since they do all combinations of the generators and do not do them in parallel. > > On 10/21/10, Jesper Louis Andersen wrote: > > On Thu, Oct 21, 2010 at 9:56 PM, rgowka1 wrote: > >> Hi - > >> > >> How do I map a function over a bitstring without converting into string or > >> list. > >> > >> For example - how can I replace << "a" >> with << "X" >> and any other > >> character to << "Y" >>?? > >> > >> InBin = << "abcaa" >> > >> > >> OutBin = << "XYYXX" >> > > > > The trick is Binary Comprehensions or by using the (R14B) binary > > module. Here is a solution using the Binary Comprehensions: > > > > 3> << < $X; _ -> $Y end>> || <> <= <<"abcaa" >> >>. > > <<"XYYXX">> > > > > -- > > J. > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From alessandro.sivieri@REDACTED Fri Oct 22 10:54:35 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Fri, 22 Oct 2010 10:54:35 +0200 Subject: [erlang-questions] JSON encoding of arrays of arrays In-Reply-To: References: Message-ID: 2010/10/22 Bob Ippolito > I'm surprised that people are finding it to be complicated. A list of > lists is literally an erlang list with erlang lists in it. It looks > like the documentation does need some improvement though, it's only > really obvious if you use it or if you read through the data types. > > Well, maybe the documentation of mochijson2 is not *that* good (no offense here), I mean a couple of examples should be added I think. Also, the best API would be the one that takes the standard Erlang types (for example a list of lists) and produces a json, without the "struct" thing; I think there was a project like that somewhere on the Net, but it did not work with tuples... -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From rapsey@REDACTED Fri Oct 22 11:12:48 2010 From: rapsey@REDACTED (Rapsey) Date: Fri, 22 Oct 2010 11:12:48 +0200 Subject: [erlang-questions] JSON encoding of arrays of arrays In-Reply-To: References: Message-ID: On Fri, Oct 22, 2010 at 10:54 AM, Alessandro Sivieri < alessandro.sivieri@REDACTED> wrote: > 2010/10/22 Bob Ippolito > > > I'm surprised that people are finding it to be complicated. A list of > > lists is literally an erlang list with erlang lists in it. It looks > > like the documentation does need some improvement though, it's only > > really obvious if you use it or if you read through the data types. > > > > > Well, maybe the documentation of mochijson2 is not *that* good (no offense > here), I mean a couple of examples should be added I think. Also, the best > API would be the one that takes the standard Erlang types (for example a > list of lists) and produces a json, without the "struct" thing; I think > there was a project like that somewhere on the Net, but it did not work > with > tuples... > It is actually very easy to change mochijson2 module not to produce {struct,...} for decode (it's what I do). Encoding works without it as well. The only problem with encoding is when to interpret something as an array or object. mochijson2 supports explicitly specifying {array,[...]} as well as {struct,[..]}. Sergej From mk@REDACTED Fri Oct 22 11:25:13 2010 From: mk@REDACTED (Morten Krogh) Date: Fri, 22 Oct 2010 11:25:13 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: <20101022074945.GB16824@erix.ericsson.se> References: <20101022074945.GB16824@erix.ericsson.se> Message-ID: <4CC15879.2060904@amberbio.com> Hi, I would like to understand what happens under the hood with this zip function, especially <> Does this function work by copying R all the time, and then having the garbage collector collect the old R immediately, or can BEAM share R in some way. When I test it in the shell with 100MB: C = zip(A, B). A, B size 100 million. I get a quite fast run time indicating that copying R all the time does not take place. I get a peak memory consumption quite beyond the minimal 400 MB, however. So what is BEAM doing here? For the fun it, I wrote the map function in the spirit of Raimo's program, map(F, A) -> map(F, A, <<>>). map(F, <>, R) -> Mapped = F(X), map(F, A, <>); map(_F, <<>>, R) -> R. To my enormous surprise, this program is about 7 times faster than the binary comprehension. 18> A = binary:copy(<<"a">>, 10000000). 10 MB, that is. 19> T1 = now(), B = << <<"b">> || <> <= A>>, T2 = now(), timer:now_diff(T2,T1). 68860012 22> f(T1), f(T2), f(B). ok 23> T1 = now(), B = zip2:map(fun(X) -> 98 end, A), T2 = now(), timer:now_diff(T2,T1). 9284114 (zip2 is my module name for this test) What is going on? cheers, Morten. On 10/22/10 9:49 AM, Raimo Niskanen wrote: > On Fri, Oct 22, 2010 at 12:55:06AM -0400, rgowka1 wrote: >> Thanks, that was really helpful. >> >> One more question, how do I zip on two bitstrings. > For that you need to write a function of your own, matching > out the heads of both bitstrings and building a new by appending. > > Uncompiled: > zip(A, B) -> zip(A, B,<<>>). > zip(<>,<>, R) -> > zip(A, B,<>); > zip(<<>>,<<>>, R) -> R. > > The comprehensions can not do it since they do all combinations > of the generators and do not do them in parallel. > >> On 10/21/10, Jesper Louis Andersen wrote: >>> On Thu, Oct 21, 2010 at 9:56 PM, rgowka1 wrote: >>>> Hi - >>>> >>>> How do I map a function over a bitstring without converting into string or >>>> list. >>>> >>>> For example - how can I replace<< "a">> with<< "X">> and any other >>>> character to<< "Y">>?? >>>> >>>> InBin =<< "abcaa">> >>>> >>>> OutBin =<< "XYYXX">> >>> The trick is Binary Comprehensions or by using the (R14B) binary >>> module. Here is a solution using the Binary Comprehensions: >>> >>> 3> << < $X; _ -> $Y end>> ||<> <=<<"abcaa">> >>. >>> <<"XYYXX">> >>> >>> -- >>> J. >>> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From bob@REDACTED Fri Oct 22 11:29:07 2010 From: bob@REDACTED (Bob Ippolito) Date: Fri, 22 Oct 2010 17:29:07 +0800 Subject: [erlang-questions] JSON encoding of arrays of arrays In-Reply-To: References: Message-ID: On Fri, Oct 22, 2010 at 4:54 PM, Alessandro Sivieri wrote: > 2010/10/22 Bob Ippolito >> >> I'm surprised that people are finding it to be complicated. A list of >> lists is literally an erlang list with erlang lists in it. It looks >> like the documentation does need some improvement though, it's only >> really obvious if you use it or if you read through the data types. >> > > Well, maybe the documentation of mochijson2 is not *that* good (no offense > here), I mean a couple of examples should be added I think. Also, the best > API would be the one that takes the standard Erlang types (for example a > list of lists) and produces a json, without the "struct" thing; I think > there was a project like that somewhere on the Net, but it did not work with > tuples... I definitely didn't say that the documentation was very good, just that the library is generally not that complicated. The problem with your suggestion is that the "standard Erlang types" are very ambiguous because there's no way to distinguish a list of numbers and a string, and an empty proplist is indistinguishable from an empty list. On top of that, if you were to support tuples as JSON arrays then you would have another ambiguituy with the representation of a proplist. -bob From alessandro.sivieri@REDACTED Fri Oct 22 11:33:10 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Fri, 22 Oct 2010 11:33:10 +0200 Subject: [erlang-questions] JSON encoding of arrays of arrays In-Reply-To: References: Message-ID: 2010/10/22 Bob Ippolito > I definitely didn't say that the documentation was very good, just > that the library is generally not that complicated. > > Yes, but the requested data representation to be given to the encode method is not so clear (to me, at least); anyway, in the end taking the JSON representation and "decoding" it shows how to prepare the data, so having done that immediately would have given me what I wanted. -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From jesper.louis.andersen@REDACTED Fri Oct 22 12:11:51 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 22 Oct 2010 12:11:51 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: <4CC15879.2060904@amberbio.com> References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> Message-ID: On Fri, Oct 22, 2010 at 11:25 AM, Morten Krogh wrote: > Hi, > > I would like to understand what happens under the hood with this zip > function, especially > > <> > > Does this function work by copying R all the time, and then having the > garbage collector > collect the old R immediately, or can BEAM share R in some way. Binaries are put into their own Arena on the heap where they are ref-counted for their use. Since a binary can never hold a pointer to a binary, this idea is safe w.r.t cycles. Destructuring a binary can then be done with a triple, (P, O, L) where P is a pointer to the binary, O is the offset integer and L is the length integer. In effect, deconstructing the binary is fast. Constructing the binary is done in a way which gradually allocates more and more data for the binary (somewhat like a growing array) so that is also reasonably fast. This is the quick overview. The details, which you should peruse is at, http://www.erlang.org/doc/efficiency_guide/binaryhandling.html -- J. From norton@REDACTED Fri Oct 22 11:43:44 2010 From: norton@REDACTED (Joseph Wayne Norton) Date: Fri, 22 Oct 2010 18:43:44 +0900 Subject: [erlang-questions] JSON encoding of arrays of arrays In-Reply-To: References: Message-ID: Not sure if this is helpful or not to this discussion. For integrating the UBF framework with JSON over TCP/IP and JSON-RPC over HTTP, we choose a particular representation for mapping tuples, atoms, and records. The comments at the top of this erlang module describe the mapping. http://github.com/norton/ubf-jsonrpc/blob/master/src/jsf.erl There is also (some) notes in the README that describe the approach taken: http://github.com/norton/ubf-jsonrpc We are also using the mochijson2 encoder/decoder (thanks!). thanks, On Fri, 22 Oct 2010 18:29:07 +0900, Bob Ippolito wrote: > On Fri, Oct 22, 2010 at 4:54 PM, Alessandro Sivieri > wrote: >> 2010/10/22 Bob Ippolito >>> >>> I'm surprised that people are finding it to be complicated. A list of >>> lists is literally an erlang list with erlang lists in it. It looks >>> like the documentation does need some improvement though, it's only >>> really obvious if you use it or if you read through the data types. >>> >> >> Well, maybe the documentation of mochijson2 is not *that* good (no >> offense >> here), I mean a couple of examples should be added I think. Also, the >> best >> API would be the one that takes the standard Erlang types (for example a >> list of lists) and produces a json, without the "struct" thing; I think >> there was a project like that somewhere on the Net, but it did not work >> with >> tuples... > > I definitely didn't say that the documentation was very good, just > that the library is generally not that complicated. > > The problem with your suggestion is that the "standard Erlang types" > are very ambiguous because there's no way to distinguish a list of > numbers and a string, and an empty proplist is indistinguishable from > an empty list. On top of that, if you were to support tuples as JSON > arrays then you would have another ambiguituy with the representation > of a proplist. > > -bob > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- norton@REDACTED From mk@REDACTED Fri Oct 22 13:15:58 2010 From: mk@REDACTED (Morten Krogh) Date: Fri, 22 Oct 2010 13:15:58 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> Message-ID: <4CC1726E.1080606@amberbio.com> Hi Jesper Thanks for the link! It seems to me that the requirement that binaries must be contiguous in memory is quite strict. If that requirement was abandoned, binaries could be lists of ProcBins, or whatever they are called, instead. That would make it possible to avoid copying a binary both when there was no more space behind it, and when more than one append operation was to be performed. Of course, the run time system should be clever about when to expand the list and when to expand or copy the bytes. Anyway, I cannot get this whole explanation to fit with what I see using memory(binary). Simple example from my module zip2 append() -> A = binary:copy(<<"a">>, 100000000), B = <>, receive {From, get} -> From ! {A,B}; kill -> done end. Eshell V5.8.1 (abort with ^G) 1> memory(binary). 374360 2> Pid = spawn(zip2, append, []). <0.34.0> 3> memory(binary). 300086176 4> Pid ! kill. kill 5> memory(binary). 86320 Why is the memory usage 300 MB. It should have been 100MB, and in worst case, with naive copying, 200MB. But 300 MB. When I leave B out, the memory consumption is 100MB as it should be, so I don't think it is because I am using the shell incorrectly. Also, I was cautious not to keep anything in the shell. It is a spawned process. garbage_collect(Pid) doesn't change the 300MB either. Cheers, Morten. On 10/22/10 12:11 PM, Jesper Louis Andersen wrote: > On Fri, Oct 22, 2010 at 11:25 AM, Morten Krogh wrote: >> Hi, >> >> I would like to understand what happens under the hood with this zip >> function, especially >> >> <> >> >> Does this function work by copying R all the time, and then having the >> garbage collector >> collect the old R immediately, or can BEAM share R in some way. > Binaries are put into their own Arena on the heap where they are > ref-counted for their use. Since a binary can never hold a pointer to > a binary, this idea is safe w.r.t cycles. Destructuring a binary can > then be done with a triple, (P, O, L) where P is a pointer to the > binary, O is the offset integer and L is the length integer. In > effect, deconstructing the binary is fast. Constructing the binary is > done in a way which gradually allocates more and more data for the > binary (somewhat like a growing array) so that is also reasonably > fast. > > This is the quick overview. The details, which you should peruse is at, > > http://www.erlang.org/doc/efficiency_guide/binaryhandling.html > > > From mk@REDACTED Fri Oct 22 13:31:19 2010 From: mk@REDACTED (Morten Krogh) Date: Fri, 22 Oct 2010 13:31:19 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: <9F20C0E1-3B67-4761-B228-61148B0F30F4@rogvall.se> References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> <9F20C0E1-3B67-4761-B228-61148B0F30F4@rogvall.se> Message-ID: <4CC17607.8070601@amberbio.com> Yes, Tony. Now binary comprehension is 16 faster than the matching function. So the slow part is that it has to get back to the shell and get the <<"b">> (which is silly in this case because it is constant). cheers, Morten. On 10/22/10 1:16 PM, Tony Rogvall wrote: > > On 22 okt 2010, at 11.25, Morten Krogh wrote: > >> >> To my enormous surprise, this program is about 7 times faster than >> the binary comprehension. >> >> >> 18> A = binary:copy(<<"a">>, 10000000). >> >> 10 MB, that is. >> >> >> 19> T1 = now(), B = << <<"b">> || <> <= A>>, T2 = now(), >> timer:now_diff(T2,T1). >> 68860012 >> > Interpreted code !!! Put this code in a module and compile it, then > try again. > >> 22> f(T1), f(T2), f(B). >> ok >> 23> T1 = now(), B = zip2:map(fun(X) -> 98 end, A), T2 = now(), >> timer:now_diff(T2,T1). >> 9284114 >> > I guess zip2 is compiled. > >> (zip2 is my module name for this test) >> >> >> What is going on? >> > There is a big speed difference between interpreted code and compiled > code. > > /Tony > > /"Have run Make so many times I dunno what's installed anymore"/ > From mk@REDACTED Fri Oct 22 13:49:28 2010 From: mk@REDACTED (Morten Krogh) Date: Fri, 22 Oct 2010 13:49:28 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: <9F20C0E1-3B67-4761-B228-61148B0F30F4@rogvall.se> References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> <9F20C0E1-3B67-4761-B228-61148B0F30F4@rogvall.se> Message-ID: <4CC17A48.7020102@amberbio.com> Sorry, in my recent reply to Tony I tested the comprehension on the constant ><<"B"">>. Putting everything in the module and comparing the same mapped function ,f , the result is that the comprehension is 30% faster than my matching based map function. Cheers, Morten. On 10/22/10 1:16 PM, Tony Rogvall wrote: > > On 22 okt 2010, at 11.25, Morten Krogh wrote: > >> >> To my enormous surprise, this program is about 7 times faster than >> the binary comprehension. >> >> >> 18> A = binary:copy(<<"a">>, 10000000). >> >> 10 MB, that is. >> >> >> 19> T1 = now(), B = << <<"b">> || <> <= A>>, T2 = now(), >> timer:now_diff(T2,T1). >> 68860012 >> > Interpreted code !!! Put this code in a module and compile it, then > try again. > >> 22> f(T1), f(T2), f(B). >> ok >> 23> T1 = now(), B = zip2:map(fun(X) -> 98 end, A), T2 = now(), >> timer:now_diff(T2,T1). >> 9284114 >> > I guess zip2 is compiled. > >> (zip2 is my module name for this test) >> >> >> What is going on? >> > There is a big speed difference between interpreted code and compiled > code. > > /Tony > > /"Have run Make so many times I dunno what's installed anymore"/ > From tony@REDACTED Fri Oct 22 13:16:41 2010 From: tony@REDACTED (Tony Rogvall) Date: Fri, 22 Oct 2010 13:16:41 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: <4CC15879.2060904@amberbio.com> References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> Message-ID: <9F20C0E1-3B67-4761-B228-61148B0F30F4@rogvall.se> On 22 okt 2010, at 11.25, Morten Krogh wrote: > > To my enormous surprise, this program is about 7 times faster than the binary comprehension. > > > 18> A = binary:copy(<<"a">>, 10000000). > > 10 MB, that is. > > > 19> T1 = now(), B = << <<"b">> || <> <= A>>, T2 = now(), timer:now_diff(T2,T1). > 68860012 > Interpreted code !!! Put this code in a module and compile it, then try again. > 22> f(T1), f(T2), f(B). > ok > 23> T1 = now(), B = zip2:map(fun(X) -> 98 end, A), T2 = now(), timer:now_diff(T2,T1). > 9284114 > I guess zip2 is compiled. > (zip2 is my module name for this test) > > > What is going on? > There is a big speed difference between interpreted code and compiled code. /Tony "Have run Make so many times I dunno what's installed anymore" From jesper.louis.andersen@REDACTED Fri Oct 22 14:26:49 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 22 Oct 2010 14:26:49 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: <4CC1726E.1080606@amberbio.com> References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> <4CC1726E.1080606@amberbio.com> Message-ID: On Fri, Oct 22, 2010 at 1:15 PM, Morten Krogh wrote: > Hi Jesper > > Thanks for the link! > > It seems to me that the requirement that binaries must be contiguous in > memory is quite strict. > If that requirement was abandoned, binaries could be lists of ProcBins, or > whatever they are called, instead. This is essentially iolists you are proposing. They are already in the language, see http://prog21.dadgum.com/70.html for a quick discussion of them. > Simple example from my module zip2 [...] > Why is the memory usage 300 MB. It should have been 100MB, and in worst > case, with naive copying, 200MB. > But 300 MB. My guess is that B has some excess space allocated for appending which amount to around 100Mb (you should probably not go with a 100Mb binary in the first place, but rather an iolist of smaller binaries). B's extra space will only get shrunk away when the runtime decide to transfer the binary to another process and it is not before it is sent back to the shell as a message. My guess might be wrong though. -- J. From mk@REDACTED Fri Oct 22 14:56:32 2010 From: mk@REDACTED (Morten Krogh) Date: Fri, 22 Oct 2010 14:56:32 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> <4CC1726E.1080606@amberbio.com> Message-ID: <4CC18A00.5010708@amberbio.com> Hi Jesper On 10/22/10 2:26 PM, Jesper Louis Andersen wrote: > On Fri, Oct 22, 2010 at 1:15 PM, Morten Krogh wrote: >> Hi Jesper >> >> Thanks for the link! >> >> It seems to me that the requirement that binaries must be contiguous in >> memory is quite strict. >> If that requirement was abandoned, binaries could be lists of ProcBins, or >> whatever they are called, instead. > This is essentially iolists you are proposing. They are already in the > language, see > > http://prog21.dadgum.com/70.html > > for a quick discussion of them. > Yes, but my suggestion would let the VM handle lists of binaries instead of the application programmer. My lists only contain binaries. They are not full iolists, which I guess they could be. >> Simple example from my module zip2 > [...] > >> Why is the memory usage 300 MB. It should have been 100MB, and in worst >> case, with naive copying, 200MB. >> But 300 MB. > My guess is that B has some excess space allocated for appending which > amount to around 100Mb (you should probably not go with a 100Mb binary > in the first place, but rather an iolist of smaller binaries). B's > extra space will only get shrunk away when the runtime decide to > transfer the binary to another process and it is not before it is sent > back to the shell as a message. > > My guess might be wrong though. > It is actually 200 MB extra. Transfering the binaries to the shell, removes 100 MB, so you were right. But 200 MB in total is still allocated. It is still not clear to me. At B = <>, 200 MB is allocated behind A? Or somewhere else? And after message passing, 200 MB of extra space is shrunk to 100MB beyond the 100MB of actual data? Morten. From magnus@REDACTED Fri Oct 22 15:35:33 2010 From: magnus@REDACTED (Magnus Henoch) Date: Fri, 22 Oct 2010 14:35:33 +0100 Subject: Pretty-printing Erlang In-Reply-To: <032D4B20-14F2-4384-9D62-8302133CCB00@cs.otago.ac.nz> (Richard O'Keefe's message of "Fri, 22 Oct 2010 11:04:30 +1300") References: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> <1287558222.4859.4.camel@seasc1137> <207A14F9-852E-4C93-BF21-206F6B9616F9@cs.otago.ac.nz> <1287639191.5435.9.camel@seasc1137> <032D4B20-14F2-4384-9D62-8302133CCB00@cs.otago.ac.nz> Message-ID: <8462wu1hay.fsf@linux-b2a3.site> "Richard O'Keefe" writes: > I cannot connect to the http://tidier.softlab.ntua.gr/ site; > is that still the correct site for tidier? Seems to be down, but the web interface to Tidier is still up: http://tidier.softlab.ntua.gr:20000/tidier/getstarted -- Magnus Henoch, magnus@REDACTED Erlang Solutions http://www.erlang-solutions.com/ From simon@REDACTED Fri Oct 22 15:59:10 2010 From: simon@REDACTED (Simon MacMullen) Date: Fri, 22 Oct 2010 14:59:10 +0100 Subject: gen_server:call across nodes hangs indefinitely, net_adm:ping works Message-ID: <4CC198AE.7080009@rabbitmq.com> We're seeing a customer have some weird issues with RabbitMQ, and it's beginning to look like an Erlang problem. They have a two node system, with both nodes running R13B04 on virtualised CentOS 5. The VMs are both using iptables as a firewall, with the epmd port open, plus one other port for inter-node message passing (using inet_dist_listen_min / max to limit to that exact port). After the system has been up for a number of hours / days, they're seeing RabbitMQ hanging in various ways. Investigation of the system in this state shows that: * epmd is up and working * epmd -d -names gives the expected results * net_adm:ping/1 from either node to the other works * any Rabbit APIs that invoke gen_server:call/3 locally work * any Rabbit APIs that invoke gen_server:call/3 across nodes hang We've also seen a weird-looking error pop up around the same time (user_sup dies with "eio", see attached log), although I'm unclear as to whether this is a cause or a symptom. Unfortunately I'm a long way away from cutting this down to a minimal test case yet; I can't even replicate this myself. But does this look like anything anyone's ever seen before? Cheers, Simon -- Simon MacMullen Staff Engineer, RabbitMQ SpringSource, a division of VMware -------------- next part -------------- A non-text attachment was scrubbed... Name: node1-sasl.log Type: text/x-log Size: 982 bytes Desc: not available URL: From icfp.publicity@REDACTED Fri Oct 22 16:16:31 2010 From: icfp.publicity@REDACTED (Wouter Swierstra) Date: Fri, 22 Oct 2010 16:16:31 +0200 Subject: ICFP 2011: Call for Workshop Proposals Message-ID: CALL FOR WORKSHOP AND CO-LOCATED EVENT PROPOSALS ICFP 2011 16th ACM SIGPLAN International Conference on Functional Programming September 19 - 21, 2011 Tokyo, Japan http://www.icfpconference.org/icfp2011 The 16th ACM SIGPLAN International Conference on Functional Programming will be held in Tokyo, Japan on September 19-21, 2011. ICFP provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. Proposals are invited for workshops (and other co-located events, such as tutorials) to be affiliated with ICFP 2011 and sponsored by SIGPLAN. These events should be more informal and focused than ICFP itself, include sessions that enable interaction among the attendees, and be fairly low-cost. The preference is for one-day events, but other schedules can also be considered. NEW THIS YEAR: The workshops are scheduled to occur on September 18 (the day before ICFP) and September 22-24 (the three days after ICFP). ---------------------------------------------------------------------- Submission details Deadline for submission: November 19, 2010 Notification of acceptance: December 17, 2010 Prospective organizers of workshops or other co-located events are invited to submit a completed workshop proposal form in plain text format to the ICFP 2011 workshop co-chairs (Gabriele Keller and Derek Dreyer), via email to icfp11-workshops at mpi-sws.org by November 19, 2010. (For proposals of co-located events other than workshops, please fill in the workshop proposal form and just leave blank any sections that do not apply.) Please note that this is a firm deadline. Organizers will be notified if their event proposal is accepted by December 17, 2010, and if successful, depending on the event, they will be asked to produce a final report after the event has taken place that is suitable for publication in SIGPLAN Notices. The proposal form is available at: http://www.icfpconference.org/icfp2011/icfp11-workshops-form.txt Further information about SIGPLAN sponsorship is available at: http://acm.org/sigplan/sigplan_workshop_proposal.htm ---------------------------------------------------------------------- Selection committee The proposals will be evaluated by a committee comprising the following members of the ICFP 2010 organizing committee, together with the members of the SIGPLAN executive committee. Workshop Co-Chair: Gabriele Keller (University of New South Wales) Workshop Co-Chair: Derek Dreyer (MPI-SWS) General Co-Chair: Manuel Chakravarty (University of New South Wales) General Co-Chair: Zhenjiang Hu (National Institute of Informatics) Program Chair: Olivier Danvy (Aarhus University) ---------------------------------------------------------------------- Further information Any queries should be addressed to the workshop co-chairs (Gabriele Keller and Derek Dreyer), via email to icfp11-workshops at mpi-sws.org. From hynek@REDACTED Fri Oct 22 16:26:20 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Fri, 22 Oct 2010 16:26:20 +0200 Subject: [erlang-questions] struggling with port_command() In-Reply-To: <5EDD7341-13CB-4D95-9E97-0A40D92190D1@gmail.com> References: <5EDD7341-13CB-4D95-9E97-0A40D92190D1@gmail.com> Message-ID: On Thu, Oct 21, 2010 at 10:39 PM, Gregory Haskins wrote: > Hi All, > > I've been struggling all afternoon with what is perhaps a completely trivial problem, but I can't seem to get past it. > > Ultimately, I want to be able to have a simple external port program transform a string (XML, in fact, but that shouldn't matter). ?So I want to have my erlang program pump a string in on stdin, and then get a string back out on stdout. ?Simple stuff, but it just will not work for me. > > I scoured the net for examples and/or threads on problems in doing this, but was unable to figure out what I am doing wrong. ?Here is an escript which exemplifies the issue: > > #!/usr/bin/env escript > > pipe_stdin(Port) -> > ? ?case io:get_line("") of > ? ? ? ?eof -> > ? ? ? ? ? ?%port_close(Port), > ? ? ? ? ? ?ok; > ? ? ? ?Data -> > ? ? ? ? ? ?true = port_command(Port, Data), > ? ? ? ? ? ?pipe_stdin(Port) > ? ?end. > > main([Cmd]) -> > ? ?io:format("Executing ~s~n", [Cmd]), > ? ?Port = erlang:open_port({spawn, Cmd}, [stream, use_stdio, exit_status]), > > ? ?ok = pipe_stdin(Port), > ? ?{ok, _Status, Output} = cmd_receive(Port, ""), > > ? ?io:format("-----~s~n", [Output]), > > ? ?ok. > > cmd_receive(Port, Acc) -> > ? ?receive > ? ? ? ?{Port, {data, Data}} -> > ? ? ? ? ? ?cmd_receive(Port, string:concat(Acc, Data)); > ? ? ? ?{Port, {exit_status, Status}} -> > ? ? ? ? ? ?{ok, Status, Acc}; > ? ? ? ?Else -> > ? ? ? ? ? ?throw({unexpected_msg, Else}) > ? ?end. > > The problem I am seeing is that the data never seems to reach the port program (with or without the port_close()). ?I pump the data in with port_command() and it returns "true" but nothing seems to come out the other side. ?Any help would be greatly appreciated. > > -Greg > > PS: As a secondary question, is the right way to signal EOF on the stdin to the port to use port_close()? 1> Port = open_port({spawn, "cat"}, [stream]). #Port<0.522> 2> port_command(Port, "Hello World\n"). true 3> flush(). Shell got {#Port<0.522>,{data,{eol,"Hello World\n"}}} ok 4> port_close(Port). true Are you sure that data doesn't arrive to your port program? It seems that your port program is unable detect end of input data because there is not possible send eof (or I don't know how). When you use stream option it is up to you implement correct wire protocol. May be you have to make some wrapper around your port program. -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From mk@REDACTED Fri Oct 22 16:45:59 2010 From: mk@REDACTED (Morten Krogh) Date: Fri, 22 Oct 2010 16:45:59 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> <4CC1726E.1080606@amberbio.com> <4CC18A00.5010708@amberbio.com> Message-ID: <4CC1A3A7.1040404@amberbio.com> Thanks, Per. That was also the only reason I could see. And it doesn't really hurt to allocate too much, so it is a good idea. Morten. On 10/22/10 4:27 PM, Per Gustafsson wrote: > The reason for 300 MB is an optimizarion for building binaries which > allocates extra space for possible future additions to the binary. > > Per > > 2010/10/22 Morten Krogh > > > Hi Jesper > > > > On 10/22/10 2:26 PM, Jesper Louis Andersen wrote: > > On Fri, Oct 22, 2010 at 1:15 PM, Morten Krogh > wrote: > > Hi Jesper > > Thanks for the link! > > It seems to me that the requirement that binaries must be > contiguous in > memory is quite strict. > If that requirement was abandoned, binaries could be lists > of ProcBins, or > whatever they are called, instead. > > This is essentially iolists you are proposing. They are > already in the > language, see > > http://prog21.dadgum.com/70.html > > for a quick discussion of them. > > > Yes, but my suggestion would let the VM handle lists of binaries > instead of the application programmer. > My lists only contain binaries. They are not full iolists, which I > guess they could be. > > > > Simple example from my module zip2 > > [...] > > Why is the memory usage 300 MB. It should have been 100MB, > and in worst > case, with naive copying, 200MB. > But 300 MB. > > My guess is that B has some excess space allocated for > appending which > amount to around 100Mb (you should probably not go with a > 100Mb binary > in the first place, but rather an iolist of smaller binaries). B's > extra space will only get shrunk away when the runtime decide to > transfer the binary to another process and it is not before it > is sent > back to the shell as a message. > > My guess might be wrong though. > > It is actually 200 MB extra. > > Transfering the binaries to the shell, removes 100 MB, so you were > right. But 200 MB in total is still allocated. > > It is still not clear to me. At B = <>, 200 MB is > allocated behind A? Or somewhere else? > And after message passing, 200 MB of extra space is shrunk to > 100MB beyond the 100MB of actual data? > > Morten. > > > > > > > > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From mk@REDACTED Fri Oct 22 16:49:24 2010 From: mk@REDACTED (Morten Krogh) Date: Fri, 22 Oct 2010 16:49:24 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> <4CC1726E.1080606@amberbio.com> <4CC18A00.5010708@amberbio.com> Message-ID: <4CC1A474.1070107@amberbio.com> memory(binary) should probably just return the actually used memory, however. Morten. On 10/22/10 4:27 PM, Per Gustafsson wrote: > The reason for 300 MB is an optimizarion for building binaries which > allocates extra space for possible future additions to the binary. > > Per > > 2010/10/22 Morten Krogh > > > Hi Jesper > > > > On 10/22/10 2:26 PM, Jesper Louis Andersen wrote: > > On Fri, Oct 22, 2010 at 1:15 PM, Morten Krogh > wrote: > > Hi Jesper > > Thanks for the link! > > It seems to me that the requirement that binaries must be > contiguous in > memory is quite strict. > If that requirement was abandoned, binaries could be lists > of ProcBins, or > whatever they are called, instead. > > This is essentially iolists you are proposing. They are > already in the > language, see > > http://prog21.dadgum.com/70.html > > for a quick discussion of them. > > > Yes, but my suggestion would let the VM handle lists of binaries > instead of the application programmer. > My lists only contain binaries. They are not full iolists, which I > guess they could be. > > > > Simple example from my module zip2 > > [...] > > Why is the memory usage 300 MB. It should have been 100MB, > and in worst > case, with naive copying, 200MB. > But 300 MB. > > My guess is that B has some excess space allocated for > appending which > amount to around 100Mb (you should probably not go with a > 100Mb binary > in the first place, but rather an iolist of smaller binaries). B's > extra space will only get shrunk away when the runtime decide to > transfer the binary to another process and it is not before it > is sent > back to the shell as a message. > > My guess might be wrong though. > > It is actually 200 MB extra. > > Transfering the binaries to the shell, removes 100 MB, so you were > right. But 200 MB in total is still allocated. > > It is still not clear to me. At B = <>, 200 MB is > allocated behind A? Or somewhere else? > And after message passing, 200 MB of extra space is shrunk to > 100MB beyond the 100MB of actual data? > > Morten. > > > > > > > > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From jesper.louis.andersen@REDACTED Fri Oct 22 16:50:14 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 22 Oct 2010 16:50:14 +0200 Subject: [erlang-questions] gen_server:call across nodes hangs indefinitely, net_adm:ping works In-Reply-To: <4CC198AE.7080009@rabbitmq.com> References: <4CC198AE.7080009@rabbitmq.com> Message-ID: On Fri, Oct 22, 2010 at 3:59 PM, Simon MacMullen wrote: > We're seeing a customer have some weird issues with RabbitMQ, and it's > beginning to look like an Erlang problem. Personally, I would go aggressively for the network stack. When the system hangs, tcpdump the connection and see what kind of messages are passed. Are they received on the other end at all? For these kinds of problems, I like going from the copper and upwards, simply to establish a true knowledge about the problem. You might have built up a hypothetical knowledge about certain lower-level things. This must be acknowledged by inspection. * Is there a firewall state in iptables (assuming conntrack is used. /proc knows about the table)? * Are packets ending at the other machine at all (tcpdump) * Can you inspect the amount of network traffic going on at the incident? If you have a large setup, sysadmins tend to have nice graphs of this kind of stuff. * Duplex/speed negotiation problems in the network some place? * strace the erlang process for network communication stuff? * Kernel message logs (dmesg?) * Virtual machines? I've seen VMwares internal routing switch go nuts and throw away things or delay things. At least these are some points I would go for. It may be an Erlang bug, but I would guess it is somehow related to the infrastructure - otherwise others would probably have seen it. -- J. From spawn.think@REDACTED Fri Oct 22 17:39:24 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Fri, 22 Oct 2010 17:39:24 +0200 Subject: [erlang-questions] struggling with port_command() In-Reply-To: References: <5EDD7341-13CB-4D95-9E97-0A40D92190D1@gmail.com> Message-ID: not sure why you are doing the "io:get_line" thing for example if you commented out pipe stdin thing, run the script to execute ls it works fine On Fri, Oct 22, 2010 at 4:26 PM, Hynek Vychodil wrote: > On Thu, Oct 21, 2010 at 10:39 PM, Gregory Haskins > wrote: > > Hi All, > > > > I've been struggling all afternoon with what is perhaps a completely > trivial problem, but I can't seem to get past it. > > > > Ultimately, I want to be able to have a simple external port program > transform a string (XML, in fact, but that shouldn't matter). So I want to > have my erlang program pump a string in on stdin, and then get a string back > out on stdout. Simple stuff, but it just will not work for me. > > > > I scoured the net for examples and/or threads on problems in doing this, > but was unable to figure out what I am doing wrong. Here is an escript > which exemplifies the issue: > > > > #!/usr/bin/env escript > > > > pipe_stdin(Port) -> > > case io:get_line("") of > > eof -> > > %port_close(Port), > > ok; > > Data -> > > true = port_command(Port, Data), > > pipe_stdin(Port) > > end. > > > > main([Cmd]) -> > > io:format("Executing ~s~n", [Cmd]), > > Port = erlang:open_port({spawn, Cmd}, [stream, use_stdio, > exit_status]), > > > > ok = pipe_stdin(Port), > > {ok, _Status, Output} = cmd_receive(Port, ""), > > > > io:format("-----~s~n", [Output]), > > > > ok. > > > > cmd_receive(Port, Acc) -> > > receive > > {Port, {data, Data}} -> > > cmd_receive(Port, string:concat(Acc, Data)); > > {Port, {exit_status, Status}} -> > > {ok, Status, Acc}; > > Else -> > > throw({unexpected_msg, Else}) > > end. > > > > The problem I am seeing is that the data never seems to reach the port > program (with or without the port_close()). I pump the data in with > port_command() and it returns "true" but nothing seems to come out the other > side. Any help would be greatly appreciated. > > > > -Greg > > > > PS: As a secondary question, is the right way to signal EOF on the stdin > to the port to use port_close()? > > 1> Port = open_port({spawn, "cat"}, [stream]). > #Port<0.522> > 2> port_command(Port, "Hello World\n"). > true > 3> flush(). > Shell got {#Port<0.522>,{data,{eol,"Hello World\n"}}} > ok > 4> port_close(Port). > true > > Are you sure that data doesn't arrive to your port program? It seems > that your port program is unable detect end of input data because > there is not possible send eof (or I don't know how). When you use > stream option it is up to you implement correct wire protocol. May be > you have to make some wrapper around your port program. > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From rtrlists@REDACTED Fri Oct 22 21:53:31 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 22 Oct 2010 20:53:31 +0100 Subject: [erlang-questions] Writing to Windows Eventlog In-Reply-To: <33c05cdc-b0d9-4473-99f7-af97e5727c58@g4g2000prj.googlegroups.com> References: <33c05cdc-b0d9-4473-99f7-af97e5727c58@g4g2000prj.googlegroups.com> Message-ID: On Sat, Oct 16, 2010 at 9:08 AM, Amil Ganju wrote: > Is there a library or any other technique to enable me to write to the > Windows Eventlog from an erlang program? > > Thanks, > Amil > > I don't believe there is one yet. There is a lib to read the event log though; it's called nteventlog, part of the os_mon set of modules. Maybe that can act as a starting point for implementing the other direction? Robby From gregory.haskins@REDACTED Fri Oct 22 22:58:45 2010 From: gregory.haskins@REDACTED (Gregory Haskins) Date: Fri, 22 Oct 2010 16:58:45 -0400 Subject: [erlang-questions] struggling with port_command() In-Reply-To: References: <5EDD7341-13CB-4D95-9E97-0A40D92190D1@gmail.com> Message-ID: <4217BD02-5F50-41CD-8BB7-C802B69054DA@gmail.com> On Oct 22, 2010, at 10:26 AM, Hynek Vychodil wrote: > On Thu, Oct 21, 2010 at 10:39 PM, Gregory Haskins > wrote: >> Hi All, >> >> I've been struggling all afternoon with what is perhaps a completely trivial problem, but I can't seem to get past it. >> >> Ultimately, I want to be able to have a simple external port program transform a string (XML, in fact, but that shouldn't matter). So I want to have my erlang program pump a string in on stdin, and then get a string back out on stdout. Simple stuff, but it just will not work for me. >> >> I scoured the net for examples and/or threads on problems in doing this, but was unable to figure out what I am doing wrong. Here is an escript which exemplifies the issue: >> >> #!/usr/bin/env escript >> >> pipe_stdin(Port) -> >> case io:get_line("") of >> eof -> >> %port_close(Port), >> ok; >> Data -> >> true = port_command(Port, Data), >> pipe_stdin(Port) >> end. >> >> main([Cmd]) -> >> io:format("Executing ~s~n", [Cmd]), >> Port = erlang:open_port({spawn, Cmd}, [stream, use_stdio, exit_status]), >> >> ok = pipe_stdin(Port), >> {ok, _Status, Output} = cmd_receive(Port, ""), >> >> io:format("-----~s~n", [Output]), >> >> ok. >> >> cmd_receive(Port, Acc) -> >> receive >> {Port, {data, Data}} -> >> cmd_receive(Port, string:concat(Acc, Data)); >> {Port, {exit_status, Status}} -> >> {ok, Status, Acc}; >> Else -> >> throw({unexpected_msg, Else}) >> end. >> >> The problem I am seeing is that the data never seems to reach the port program (with or without the port_close()). I pump the data in with port_command() and it returns "true" but nothing seems to come out the other side. Any help would be greatly appreciated. >> >> -Greg >> >> PS: As a secondary question, is the right way to signal EOF on the stdin to the port to use port_close()? > > 1> Port = open_port({spawn, "cat"}, [stream]). > #Port<0.522> > 2> port_command(Port, "Hello World\n"). > true > 3> flush(). > Shell got {#Port<0.522>,{data,{eol,"Hello World\n"}}} > ok > 4> port_close(Port). > true > > Are you sure that data doesn't arrive to your port program? It turns out it was, but I was misdiagnosing the issue. > It seems > that your port program is unable detect end of input data because > there is not possible send eof (or I don't know how). When you use > stream option it is up to you implement correct wire protocol. Yeah, you are exactly right and this was my problem. The remote side of the connection was designed to wait for EOF and it is not possible to generate one with erlang:ports as you pointed out. It appeared to me that I wasn't even getting any data, but I think I was all along so it was a red-herring to be looking at the transmission itself. Once I realized the problem was w.r.t. communicating the packet length, the program started working. > May be > you have to make some wrapper around your port program. Yep, exactly. Thanks so much for the help, Appreciated. -Greg -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 203 bytes Desc: This is a digitally signed message part URL: From gregory.haskins@REDACTED Fri Oct 22 23:00:04 2010 From: gregory.haskins@REDACTED (Gregory Haskins) Date: Fri, 22 Oct 2010 17:00:04 -0400 Subject: [erlang-questions] struggling with port_command() In-Reply-To: References: <5EDD7341-13CB-4D95-9E97-0A40D92190D1@gmail.com> Message-ID: <9D593E2A-209B-40FA-836A-C6558B18BC5D@gmail.com> On Oct 22, 2010, at 11:39 AM, Ahmed Omar wrote: > not sure why you are doing the "io:get_line" thing > for example if you commented out pipe stdin thing, run the script to execute ls > it works fine Ah, that was just to feed data into the escript, which would then try to pass it along a port connection. If you comment out the pipe_stdin, the script doesn't have a purpose ;). But no bother, I have figured out the issue per my previous reply. Thanks for looking into it! Appreciated. -Greg > > > On Fri, Oct 22, 2010 at 4:26 PM, Hynek Vychodil wrote: > On Thu, Oct 21, 2010 at 10:39 PM, Gregory Haskins > wrote: > > Hi All, > > > > I've been struggling all afternoon with what is perhaps a completely trivial problem, but I can't seem to get past it. > > > > Ultimately, I want to be able to have a simple external port program transform a string (XML, in fact, but that shouldn't matter). So I want to have my erlang program pump a string in on stdin, and then get a string back out on stdout. Simple stuff, but it just will not work for me. > > > > I scoured the net for examples and/or threads on problems in doing this, but was unable to figure out what I am doing wrong. Here is an escript which exemplifies the issue: > > > > #!/usr/bin/env escript > > > > pipe_stdin(Port) -> > > case io:get_line("") of > > eof -> > > %port_close(Port), > > ok; > > Data -> > > true = port_command(Port, Data), > > pipe_stdin(Port) > > end. > > > > main([Cmd]) -> > > io:format("Executing ~s~n", [Cmd]), > > Port = erlang:open_port({spawn, Cmd}, [stream, use_stdio, exit_status]), > > > > ok = pipe_stdin(Port), > > {ok, _Status, Output} = cmd_receive(Port, ""), > > > > io:format("-----~s~n", [Output]), > > > > ok. > > > > cmd_receive(Port, Acc) -> > > receive > > {Port, {data, Data}} -> > > cmd_receive(Port, string:concat(Acc, Data)); > > {Port, {exit_status, Status}} -> > > {ok, Status, Acc}; > > Else -> > > throw({unexpected_msg, Else}) > > end. > > > > The problem I am seeing is that the data never seems to reach the port program (with or without the port_close()). I pump the data in with port_command() and it returns "true" but nothing seems to come out the other side. Any help would be greatly appreciated. > > > > -Greg > > > > PS: As a secondary question, is the right way to signal EOF on the stdin to the port to use port_close()? > > 1> Port = open_port({spawn, "cat"}, [stream]). > #Port<0.522> > 2> port_command(Port, "Hello World\n"). > true > 3> flush(). > Shell got {#Port<0.522>,{data,{eol,"Hello World\n"}}} > ok > 4> port_close(Port). > true > > Are you sure that data doesn't arrive to your port program? It seems > that your port program is unable detect end of input data because > there is not possible send eof (or I don't know how). When you use > stream option it is up to you implement correct wire protocol. May be > you have to make some wrapper around your port program. > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 203 bytes Desc: This is a digitally signed message part URL: From jiansenhe@REDACTED Sat Oct 23 00:24:58 2010 From: jiansenhe@REDACTED (Jiansen He) Date: Fri, 22 Oct 2010 23:24:58 +0100 Subject: [erlang-questions] performance on multicore computer Message-ID: Hi all, I write a small program to try parallel computing in Erlang. However, I find that it only use 110-120% CPU. I running a dual core iMac with hyper-threading technology. In Haskell, I can explicitly point out how many cores in my machine and I find my program consumes about 380% CPU. How can I let erlang use more CPU? Jiansen From jesper.louis.andersen@REDACTED Sat Oct 23 00:37:21 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 23 Oct 2010 00:37:21 +0200 Subject: [erlang-questions] performance on multicore computer In-Reply-To: References: Message-ID: On Sat, Oct 23, 2010 at 12:24 AM, Jiansen He wrote: > Hi all, > > I write a small program to try parallel computing in Erlang. ?However, I > find that it only use 110-120% CPU. ?I running a dual core iMac with > hyper-threading technology. When you run erl(1), how many cores does it report? Is your program parallel, or are you effectively waiting on something all the time so it is serial? -- J. From jiansenhe@REDACTED Sat Oct 23 00:42:40 2010 From: jiansenhe@REDACTED (Jiansen He) Date: Fri, 22 Oct 2010 23:42:40 +0100 Subject: [erlang-questions] performance on multicore computer In-Reply-To: References: Message-ID: where shall I run erl(1). I got this when I start erlang Erlang R14B (erts-5.8.1) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] My program should be parallel. I have a similar Haskell program which could consume 380% CPU! Jiansen On Fri, Oct 22, 2010 at 11:37 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Sat, Oct 23, 2010 at 12:24 AM, Jiansen He > wrote: > > Hi all, > > > > I write a small program to try parallel computing in Erlang. However, I > > find that it only use 110-120% CPU. I running a dual core iMac with > > hyper-threading technology. > > When you run erl(1), how many cores does it report? > > Is your program parallel, or are you effectively waiting on something > all the time so it is serial? > > > -- > J. > From jesper.louis.andersen@REDACTED Sat Oct 23 00:51:32 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 23 Oct 2010 00:51:32 +0200 Subject: [erlang-questions] performance on multicore computer In-Reply-To: References: Message-ID: On Sat, Oct 23, 2010 at 12:42 AM, Jiansen He wrote: > Erlang R14B (erts-5.8.1) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] > [kernel-poll:false] The smp:4:4 part should mean you have access to all (logical) CPUs. > My program should be parallel.? I have a similar Haskell program which could > consume 380% CPU! Haskell is different from Erlang, and its parallellism-constructions are much different. I wonder if you can give the program or at least tell us the strategy is uses for obtaining parallel evaluation. It may be here the problem lies (at least that is my hunch) -- J. From jiansenhe@REDACTED Sat Oct 23 01:07:39 2010 From: jiansenhe@REDACTED (Jiansen He) Date: Sat, 23 Oct 2010 00:07:39 +0100 Subject: [erlang-questions] performance on multicore computer In-Reply-To: References: Message-ID: Thank you so much. my program is actually 200 lines long. I posted related parts as follows. %% -- generate all sequences of length exactly n starting at index i or later %% gen_ :: Int -> Int -> [String] -> AssocList [String] Int gen_ (_, _, [], From) -> From ! {seq, []}; gen_ (I, N, [H | T], From) -> if N > length([H | T]) -> From ! {seq, []}; true -> gen_(I+1, N, T, self()), receive {seq, Seq} -> From ! {seq, insert({takeN(N, [H | T]), I}, Seq)}; Msg -> io:format("unrecognized message in gen_/4: ~n") end end. %% -- generate all sequences of length at most n %% gen :: Int -> [String] -> AssocList [String] Int gen(0, _, From) -> From ! {seq, []}; gen(N, L, From) -> spawn(concordance_1_1, gen_, [0, N, L, self()]), spawn(concordance_1_1, gen, [(N-1), L, self()]), gen_loop(2, [], From). gen_loop(0, Seq, From) -> From ! {seq, Seq}; gen_loop(N, Seq, From) -> receive {seq, Seq_} -> gen_loop(N-1, merge(Seq, Seq_), From); Msg -> io:format("f unrecognized message in gen_loop/3: ") end. In Haskell, I could simply write code like this: -- generate all sequences of length at most n gen :: Int -> [String] -> AssocList [String] Int gen 0 _ = [] --gen n l = merge sqs_len_n sqs_up_to_n1 gen n l = sqs_len_n `par` sqs_up_to_n1 `pseq` (merge sqs_len_n sqs_up_to_n1) where sqs_up_to_n1 = (gen (n-1) l) sqs_len_n = (gen' 0 n l) Please let me know if you have difficulty to understand my code. Many thanks again. Jiansen On Fri, Oct 22, 2010 at 11:51 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Sat, Oct 23, 2010 at 12:42 AM, Jiansen He > wrote: > > > Erlang R14B (erts-5.8.1) [source] [smp:4:4] [rq:4] [async-threads:0] > [hipe] > > [kernel-poll:false] > > The smp:4:4 part should mean you have access to all (logical) CPUs. > > > My program should be parallel. I have a similar Haskell program which > could > > consume 380% CPU! > > Haskell is different from Erlang, and its parallellism-constructions > are much different. I wonder if you can give the program or at least > tell us the strategy is uses for obtaining parallel evaluation. It may > be here the problem lies (at least that is my hunch) > > -- > J. > From jesper.louis.andersen@REDACTED Sat Oct 23 02:40:21 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 23 Oct 2010 02:40:21 +0200 Subject: [erlang-questions] performance on multicore computer In-Reply-To: References: Message-ID: On Sat, Oct 23, 2010 at 1:07 AM, Jiansen He wrote: This part: > %% -- generate all sequences of length exactly n starting at index i or > later > %% gen_ :: Int -> Int -> [String] -> AssocList [String] Int > gen_ (_, _, [], From) -> From ! {seq, []}; > gen_ (I, N, [H | T], From) -> > ?if > ??? N > length([H | T]) -> From ! {seq, []}; > ??? true -> > ?????? gen_(I+1, N, T, self()), > ?????? receive > ???????? {seq, Seq} -> > ?????????? From ! {seq, insert({takeN(N, [H | T]), I}, Seq)}; > ???????? Msg -> > ?????????? io:format("unrecognized message in gen_/4: ~n") > ?????? end > ?end. Seem a bit odd to me. If I read it right, you are essentially running a recursive loop by messaging yourself the result of the recursive calls. In other words I think you can rewrite this into a normal recursive loop. This will definitely be faster as the receives simply serialize into a long chain, the mailbox only containing a single message all the time. > %% -- generate all sequences of length at most n > %% gen :: Int -> [String] -> AssocList [String] Int > gen(0, _, From) -> > ??? From ! {seq, []}; > gen(N, L, From) -> > ??? spawn(concordance_1_1, gen_, [0, N, L, self()]), > ??? spawn(concordance_1_1, gen, [(N-1), L, self()]), > ??? gen_loop(2, [], From). > > gen_loop(0, Seq, From) -> > ??? From ! {seq, Seq}; > gen_loop(N, Seq, From) -> > ??? receive > ????? {seq, Seq_} -> > ??????? gen_loop(N-1, merge(Seq, Seq_), From); > ????? Msg -> > ?????????? io:format("f unrecognized message in gen_loop/3: ") > ??? end. These seem ok from a quick glance. And they seem to be in accordance with the Haskell code below. > In Haskell, I could simply write code like this: > > -- generate all sequences of length at most n > gen :: Int -> [String] -> AssocList [String] Int > gen 0 _ = [] > --gen n l = merge sqs_len_n sqs_up_to_n1 > gen n l = sqs_len_n `par` sqs_up_to_n1 `pseq` (merge sqs_len_n sqs_up_to_n1) > ??????????? where sqs_up_to_n1 = (gen (n-1) l) > ?????????????????????? sqs_len_n = (gen' 0 n l) The `par` combinator is perhaps mostly akin to the rpc:parallel_eval/1 call in the Erlang/OTP stdlib by the way. Note however that parallel_eval can span multiple machines (nodes in Erlang-terminology) in a distributed fashion, which `par` cannot. Also, by using parallel eval, you could eliminate the gen_loop from your code and fold the merge into gen like in the Haskell version. -- J. From matthias@REDACTED Sat Oct 23 08:52:15 2010 From: matthias@REDACTED (Matthias Lang) Date: Sat, 23 Oct 2010 08:52:15 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: <4CC18A00.5010708@amberbio.com> References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> <4CC1726E.1080606@amberbio.com> <4CC18A00.5010708@amberbio.com> Message-ID: <20101023065215.GA2991@corelatus.se> On Friday, October 22, Morten Krogh wrote: > Yes, but my suggestion would let the VM handle lists of binaries > instead of the application programmer. How is your suggestion different to "segmented binaries", described in "9.3 Segmented binaries" in http://dotat.at/tmp/binaries.pdf ? (IIRC, segmented binaries were dropped because there wasn't a clear performance gain.) Matt From boris.muehmer@REDACTED Sat Oct 23 15:46:56 2010 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Sat, 23 Oct 2010 15:46:56 +0200 Subject: Erlang Logo as SVG Message-ID: Is the Erlang Logo available as SVG version, please? - boris From kiszl@REDACTED Sat Oct 23 16:11:28 2010 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sat, 23 Oct 2010 16:11:28 +0200 Subject: [erlang-questions] Erlang Logo as SVG In-Reply-To: References: Message-ID: <4CC2ED10.3020001@tmit.bme.hu> It is available in eps format from here: http://www.erlang.org/promotion/ . Here is (for a while) the result of feeding it to an online svg converter: http://www2.online-convert.com/download-file/0b265969dcdfa071a6a04f94e81b66b7 . Z. On 10/23/2010 3:46 PM, Boris M?hmer wrote: > Is the Erlang Logo available as SVG version, please? > > > - boris > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From toby@REDACTED Sat Oct 23 17:44:32 2010 From: toby@REDACTED (Toby Thain) Date: Sat, 23 Oct 2010 11:44:32 -0400 Subject: [erlang-questions] Erlang Logo as SVG In-Reply-To: <4CC2ED10.3020001@tmit.bme.hu> References: <4CC2ED10.3020001@tmit.bme.hu> Message-ID: <4CC302E0.9000103@telegraphics.com.au> On 23/10/10 10:11 AM, Zoltan Lajos Kis wrote: > It is available in eps format from here: http://www.erlang.org/promotion/ . > > Here is (for a while) the result of feeding it to an online svg converter: > http://www2.online-convert.com/download-file/0b265969dcdfa071a6a04f94e81b66b7 > . Gone already. Try this Illustrator CS3 conversion to SVG 1.1: http://telegraphics.com.au/~toby/erlang-logo.svg > > Z. > > On 10/23/2010 3:46 PM, Boris M?hmer wrote: >> Is the Erlang Logo available as SVG version, please? >> >> >> - boris >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From boris.muehmer@REDACTED Sat Oct 23 19:42:25 2010 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Sat, 23 Oct 2010 19:42:25 +0200 Subject: [erlang-questions] Erlang Logo as SVG In-Reply-To: <4CC302E0.9000103@telegraphics.com.au> References: <4CC2ED10.3020001@tmit.bme.hu> <4CC302E0.9000103@telegraphics.com.au> Message-ID: Thank You very much! - boris From ulf.wiger@REDACTED Sun Oct 24 15:20:03 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 24 Oct 2010 15:20:03 +0200 Subject: github-flavored edoc Message-ID: <08B68DBF-DB94-4292-A907-4A05B12CEE4D@erlang-solutions.com> I pushed a small extension to edoc - called edown - to github: http://github.com/esl/edown The idea is to make the generated edoc directly readable on github, see e.g. http://github.com/esl/plain_fsm/blob/master/doc/plain_fsm.md (which BTW reveals some rendering glitches that someone should take a look at). The extension hooks in at the EDoc doclet level and below, even though it violates the framework in some respects. Rather than generating HTML framesets, it produces a README.md that github will display directly, and which combines overview, module index, etc. The way to use it is to call edoc:application(App, [{doclet, edown_doclet}]). Or, if you're using rebar, add the following in your rebar.config: {deps, [{edown, ".*", {git, "git://github.com/esl/edown.git", "HEAD"}}]}. {edoc_opts, [{doclet, edown_doclet}]}. Then, ./rebar get-deps ./rebar compile ./rebar doc I'm sure edown can be improved in many ways, and I welcome ideas and contributions. My first priority was to get something that at least does the job of making edoc more accessible in github projects. Also, I haven't tested that every project using edoc will render well in this manner. If you try it, and find problems, please let me know. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From joelr1@REDACTED Sun Oct 24 15:56:34 2010 From: joelr1@REDACTED (Joel Reymont) Date: Sun, 24 Oct 2010 14:56:34 +0100 Subject: couchdb performace 10x: using NIF for file io Message-ID: <9D7417E2-EB82-4648-861C-F86002B5BFD8@gmail.com> Simply switching to NIFs for file IO seems to have improved CouchDB write performance more than ten-fold. Compare the old graph http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f5a4 to the new graph http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f4d7 I was under the impression that the Erlang IO subsystem was highly optimized but there seems to be no limit to perfection. NIFs are a giant black hole that will subsume Erlang code as performance has to be improved. Start at the lowest level and keep moving up. All that will be left of Erlang in the end is 99.99999% uptime, fault tolerance and supervision... of optimized C code. It's swell and I'm all for it! Patch is here: http://github.com/wagerlabs/couchdb/commit/23527eb8165f81e63d47b230f3297d3072c88d83 --- http://twitter.com/wagerlabs From robert.virding@REDACTED Sun Oct 24 16:17:28 2010 From: robert.virding@REDACTED (Robert Virding) Date: Sun, 24 Oct 2010 14:17:28 +0000 (GMT) Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: <1561397215.551287929603003.JavaMail.root@zimbra> Message-ID: <977262664.571287929848268.JavaMail.root@zimbra> One intersting thing in the graphs is the fluctuations occurring in the NIF'ed version. The old version is much more consistent, but slower. I wonder why it is so. One problem with writing too much in C that there is a big risk that the code will contain more bugs. So while a system technically is "up" it will be less accessible if it has to restart sub-sections more often. Robert ----- "Joel Reymont" wrote: > Simply switching to NIFs for file IO seems to have improved CouchDB > write performance more than ten-fold. > > Compare the old graph > > http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f5a4 > > to the new graph > > http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f4d7 > > I was under the impression that the Erlang IO subsystem was highly > optimized but there seems to be no limit to perfection. > > NIFs are a giant black hole that will subsume Erlang code as > performance has to be improved. Start at the lowest level and keep > moving up. All that will be left of Erlang in the end is 99.99999% > uptime, fault tolerance and supervision... of optimized C code. It's > swell and I'm all for it! > > Patch is here: > > http://github.com/wagerlabs/couchdb/commit/23527eb8165f81e63d47b230f3297d3072c88d83 > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From ms@REDACTED Sun Oct 24 16:30:21 2010 From: ms@REDACTED (Martin Scholl) Date: Sun, 24 Oct 2010 16:30:21 +0200 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: <977262664.571287929848268.JavaMail.root@zimbra> References: <977262664.571287929848268.JavaMail.root@zimbra> Message-ID: <4CC442FD.8090103@diskware.net> I'd guess this is because the NIF blocks the current VM thread instead of a dedicated I/O thread. Just guessing, Martin On 10/24/2010 04:17 PM, Robert Virding wrote: > One intersting thing in the graphs is the fluctuations occurring in the NIF'ed version. The old version is much more consistent, but slower. I wonder why it is so. > > One problem with writing too much in C that there is a big risk that the code will contain more bugs. So while a system technically is "up" it will be less accessible if it has to restart sub-sections more often. > > Robert > > > ----- "Joel Reymont" wrote: > >> Simply switching to NIFs for file IO seems to have improved CouchDB >> write performance more than ten-fold. >> >> Compare the old graph >> >> http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f5a4 >> >> to the new graph >> >> http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f4d7 >> >> I was under the impression that the Erlang IO subsystem was highly >> optimized but there seems to be no limit to perfection. >> >> NIFs are a giant black hole that will subsume Erlang code as >> performance has to be improved. Start at the lowest level and keep >> moving up. All that will be left of Erlang in the end is 99.99999% >> uptime, fault tolerance and supervision... of optimized C code. It's >> swell and I'm all for it! >> >> Patch is here: >> >> http://github.com/wagerlabs/couchdb/commit/23527eb8165f81e63d47b230f3297d3072c88d83 >> >> --- >> http://twitter.com/wagerlabs >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From joelr1@REDACTED Sun Oct 24 16:43:16 2010 From: joelr1@REDACTED (Joel Reymont) Date: Sun, 24 Oct 2010 15:43:16 +0100 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: <4CC442FD.8090103@diskware.net> References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> Message-ID: <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> On Oct 24, 2010, at 3:30 PM, Martin Scholl wrote: > I'd guess this is because the NIF blocks the current VM thread instead > of a dedicated I/O thread. +A 4 is used by default and using +A 16 reduces the spikes http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c011c14 This is the latest 2.66ghz Core i7 MacBook Pro, 8gb memory and a 2 year-old 256gb Apple SSD. The number of cores is reported as 2 but, obviously, a lot of threads end up waiting. --- http://twitter.com/wagerlabs From rapsey@REDACTED Sun Oct 24 16:53:58 2010 From: rapsey@REDACTED (Rapsey) Date: Sun, 24 Oct 2010 16:53:58 +0200 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> Message-ID: Yep NIFs are awesome as hell. I've written a socket library and it works just beautifully. Once my streaming server is doing a few hundred mbit/s, it actually consumes less CPU than haproxy that sits in front of it. Sergej From zvi.avraham@REDACTED Sun Oct 24 19:10:46 2010 From: zvi.avraham@REDACTED (Zvi) Date: Sun, 24 Oct 2010 10:10:46 -0700 (PDT) Subject: Erlang-based opensource publish-subscribe servers Message-ID: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> Hi, I looking for the opensource examples of Erlang-based publish- subscribe servers. Any suggestions? Thanks in advance, Zvi From dizzyd@REDACTED Sun Oct 24 19:12:29 2010 From: dizzyd@REDACTED (Dave Smith) Date: Sun, 24 Oct 2010 11:12:29 -0600 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> Message-ID: On Sun, Oct 24, 2010 at 8:43 AM, Joel Reymont wrote: > > On Oct 24, 2010, at 3:30 PM, Martin Scholl wrote: > >> I'd guess this is because the NIF blocks the current VM thread instead >> of a dedicated I/O thread. > > +A 4 is used by default and using +A 16 reduces the spikes +A anything won't help you anymore. You are now doing the I/O on the VM scheduler thread and blocking ANY erlang code (on that scheduler) while the file op runs. This will certainly make the benchmarks look faster until typical circumstances; I'd be very curious to see what happens when you get into an overload situation and spend all your time blocked on file I/O while incoming client requests pile up. Since your erlang code won't be running all that often, things could get dicey. Nonetheless, an interesting experiment. D. > > http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c011c14 > > This is the latest 2.66ghz Core i7 MacBook Pro, 8gb memory and a 2 year-old 256gb Apple SSD. > > The number of cores is reported as 2 but, obviously, a lot of threads end up waiting. > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From joelr1@REDACTED Sun Oct 24 19:35:57 2010 From: joelr1@REDACTED (Joel Reymont) Date: Sun, 24 Oct 2010 18:35:57 +0100 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> Message-ID: Dave, On Oct 24, 2010, at 6:12 PM, Dave Smith wrote: > +A anything won't help you anymore. You are now doing the I/O on the > VM scheduler thread and blocking ANY erlang code (on that scheduler) > while the file op runs. Are you sure of this? Can you refer me to the spot in the VM code where I can learn more? The reason I'm surprised is that increasing the number of threads in the async thread pool had a clear effect on my benchmark. Spikes were, basically, eliminated. Thanks in advance, Joel --- http://twitter.com/wagerlabs From ulf.wiger@REDACTED Sun Oct 24 19:40:07 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 24 Oct 2010 19:40:07 +0200 Subject: [erlang-questions] Erlang-based opensource publish-subscribe servers In-Reply-To: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> Message-ID: Hi Zvi, One way to implement publish/subscribe is with gproc properties. -define(KEY, {?MODULE, subscr}). subscribe() -> gproc:add_local_property(?KEY, undefined). publish(Event) -> [P ! {self(), ?KEY, Event} || {P,_} <- gproc:lookup_local_properties(?KEY)]. Gproc takes care of monitoring the subscribers. Change local to global for global publish/subscribe. http://github.com/esl/gproc BR, Ulf W On 24 Oct 2010, at 19:10, Zvi wrote: > Hi, > > I looking for the opensource examples of Erlang-based publish- > subscribe servers. > Any suggestions? > > Thanks in advance, > Zvi > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From jiansenhe@REDACTED Sun Oct 24 19:48:09 2010 From: jiansenhe@REDACTED (Jiansen He) Date: Sun, 24 Oct 2010 18:48:09 +0100 Subject: [erlang-questions] performance on multicore computer In-Reply-To: References: Message-ID: Thank you, Jesper. my new code is as follows: %% -- generate all sequences of length exactly n starting at index i or later %% gen_ :: Int -> Int -> [String] -> AssocList [String] Int gen_ (_, _, []) -> []; gen_ (I, N, [H | T]) -> if N > length([H | T]) -> []; true -> insert({takeN(N, [H | T]), I}, gen_ (I+1, N, T)) end. %% -- generate all sequences of length at most n %% gen :: Int -> [String] -> AssocList [String] Int gen(0, _) -> []; gen(N, L) -> [Sqs_len_n, Sqs_up_to_n1] = rpc:parallel_eval([concordance_1_1_2, gen_, [0, N, L], concordance_1_1_2, gen, [(N-1), L]]), io:format("~p",[Sqs_len_n]). However, when I run this code, I got following error message: exception error: no function clause matching rpc:map_nodes([concordance_1_1_2,gen_, [0,2, ["the","manchester","marriage", "elizabeth","gaskell","mrs","openshaws", "christian","name","was","alice","and", "her","first","husband","had","been", "her","own","cousin","she", [...]|...]], concordance_1_1_2,gen, [1, ["the","manchester","marriage", "elizabeth","gaskell","mrs","openshaws", "christian","name","was","alice","and", "her","first","husband","had","been", "her","own", [...]|...]]], [nonode@REDACTED], [nonode@REDACTED]) in function rpc:parallel_eval/1 in call from concordance_1_1_2:gen/2 in call from concordance_1_1_2:concordance/2 in call from concordance_1_1_2:main/2 Can I use rpc:parallel_eval/1 on one machine only. In this program, I'm interested in how to utilize a multi-core machine, although rpc:parallel_eval/1 is definitely a useful machine for cluster computing. I tried net_adm:ping(nonode@REDACTED), and it give me pang. nodes() returns []. From kenneth.lundin@REDACTED Sun Oct 24 20:16:20 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Sun, 24 Oct 2010 20:16:20 +0200 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> Message-ID: The so called async threads selected with the +A flag are only used by the file driver that is in the original Erlang distribution from Ericsson (erlang.org). They can also be used by other user written drivers (I don't know if CouchDB has any driver like that). An implementation of file operations as NIF's will not make use of the asynch threads at all and starting Erlang with the +A flag should not make any difference at all for the execution of the NIFs. NIFs are executing in the scheduler thread and all other Erlang processes handled by the same scheduler are blocked while the NIF is executed. Erlang processes handled by other schedulers in an SMP setup can whoever execute as normal. Calling NIFs with potentially long execution times can easily destroy all multi processing capabilities and soft real time characteristics for an Erlang node. I wonder how responsive the system is to other events when running the benchmark. Of course it is possible to optimize the file operations if you know exactly what file system you are working towards, what file operations the application makes etc. but in the general case it is not that easy. /Kenneth Erlang/OTP , Ericsson On Sun, Oct 24, 2010 at 7:35 PM, Joel Reymont wrote: > Dave, > > On Oct 24, 2010, at 6:12 PM, Dave Smith wrote: > >> +A anything won't help you anymore. You are now doing the I/O on the >> VM scheduler thread and blocking ANY erlang code (on that scheduler) >> while the file op runs. > > Are you sure of this? Can you refer me to the spot in the VM code where I can learn more? > > The reason I'm surprised is that increasing the number of threads in the async thread pool had a clear effect on my benchmark. Spikes were, basically, eliminated. > > ? ? ? ?Thanks in advance, Joel > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From toby@REDACTED Sun Oct 24 21:15:56 2010 From: toby@REDACTED (Toby Thain) Date: Sun, 24 Oct 2010 15:15:56 -0400 Subject: [erlang-questions] Erlang-based opensource publish-subscribe servers In-Reply-To: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> Message-ID: <4CC485EC.7060503@telegraphics.com.au> On 24/10/10 1:10 PM, Zvi wrote: > Hi, > > I looking for the opensource examples of Erlang-based publish- > subscribe servers. > Any suggestions? Not sure if you mean applications -- if so, ejabberd implements the publish/subscribe parts of XMPP. --Toby > > Thanks in advance, > Zvi > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tony.arcieri@REDACTED Sun Oct 24 21:19:05 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Sun, 24 Oct 2010 13:19:05 -0600 Subject: [erlang-questions] Erlang-based opensource publish-subscribe servers In-Reply-To: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> Message-ID: On Sun, Oct 24, 2010 at 11:10 AM, Zvi wrote: > Hi, > > I looking for the opensource examples of Erlang-based publish- > subscribe servers. > Any suggestions? RabbitMQ also provides pubsub -- Tony Arcieri Medioh! A Kudelski Brand From zvi.avraham@REDACTED Sun Oct 24 21:22:17 2010 From: zvi.avraham@REDACTED (Zvi .) Date: Sun, 24 Oct 2010 21:22:17 +0200 Subject: [erlang-questions] Erlang-based opensource publish-subscribe servers In-Reply-To: <438091.32912.qm@web35908.mail.mud.yahoo.com> References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> <438091.32912.qm@web35908.mail.mud.yahoo.com> Message-ID: no, I need something lightweight, but able to work in a large cluster (10s of nodes). The ideas I have: 1. Use ets to store mappings of Subscriber Pids to TopicIDs - this ets will obviously be a bottleneck. 2. Have a process per TopicID, which hold list of all it's Subscriber Pids in it's state. Now the bottleneck is mapping TopicID to it's Pid 3. Use Ulf's gproc (looks it uses gen_leader abd requires patching Erlang?) I currently working on 1. Zvi On Sun, Oct 24, 2010 at 8:41 PM, Andrew Stone wrote: > If you want something powerful you can checkout rabbitmq. It may be > overkill but > it's pretty damn easy to get up and running. > > http://www.rabbitmq.com/ > > -Andrew > > > > ----- Original Message ---- > From: Zvi > To: erlang-questions@REDACTED > Sent: Sun, October 24, 2010 1:10:46 PM > Subject: [erlang-questions] Erlang-based opensource publish-subscribe > servers > > Hi, > > I looking for the opensource examples of Erlang-based publish- > subscribe servers. > Any suggestions? > > Thanks in advance, > Zvi > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Zvi Avraham, CTO Nivertech Ltd From stondage123@REDACTED Sun Oct 24 20:41:19 2010 From: stondage123@REDACTED (Andrew Stone) Date: Sun, 24 Oct 2010 11:41:19 -0700 (PDT) Subject: [erlang-questions] Erlang-based opensource publish-subscribe servers In-Reply-To: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> Message-ID: <438091.32912.qm@web35908.mail.mud.yahoo.com> If you want something powerful you can checkout rabbitmq. It may be overkill but it's pretty damn easy to get up and running. http://www.rabbitmq.com/ -Andrew ----- Original Message ---- From: Zvi To: erlang-questions@REDACTED Sent: Sun, October 24, 2010 1:10:46 PM Subject: [erlang-questions] Erlang-based opensource publish-subscribe servers Hi, I looking for the opensource examples of Erlang-based publish- subscribe servers. Any suggestions? Thanks in advance, Zvi ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From jesper.louis.andersen@REDACTED Sun Oct 24 22:13:48 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 24 Oct 2010 22:13:48 +0200 Subject: [erlang-questions] performance on multicore computer In-Reply-To: References: Message-ID: On Sun, Oct 24, 2010 at 7:48 PM, Jiansen He wrote: [..] > %% -- generate all sequences of length at most n > %% gen :: Int -> [String] -> AssocList [String] Int > gen(0, _) -> > ?? ?[]; > gen(N, L) -> > ?? ?[Sqs_len_n, Sqs_up_to_n1] = rpc:parallel_eval([concordance_1_1_2, gen_, > [0, N, L], concordance_1_1_2, gen, [(N-1), L]]), > ?? ?io:format("~p",[Sqs_len_n]). > > > However, when I run this code, I got following error message: The problem is that rpc:parallel_eval/1 takes values in another form. It expects rpc:parallel_eval([{M, F1, Args1}, {M, F2, Args2}]) in your case, but you omit the tupling { and } in your call. Hence what you are seeing is a runtime error on it. Here is a quick test, like yours: Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> rpc:parallel_eval([{dict, new, []}]). [{dict,0,16,16,8,80,48, {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}}] I am somewhat interested if the code will be fastest in Erlang or Haskell. I am guessing Haskell, but do beware laziness and profile the Haskell code a bit. It tends to bite :) -- J. From jiansenhe@REDACTED Sun Oct 24 22:55:22 2010 From: jiansenhe@REDACTED (Jiansen He) Date: Sun, 24 Oct 2010 21:55:22 +0100 Subject: [erlang-questions] performance on multicore computer In-Reply-To: References: Message-ID: Thanks again. I should notice this mistake. I may post my testing result when I further decomposed other parts of the program. I also guess that Haskell version will be faster in absolute time, but I'm more interested which will speed up closer to the 4X faster than their sequential version. So far, my Haskell code has very close to 4X even if I only changed one line code. whereas Erlang only speed up at small proportion. In fact, I also have a Scala version, which is almost identical to Haskell, but does speed up much neither :( Jiansen From joelr1@REDACTED Sun Oct 24 22:59:11 2010 From: joelr1@REDACTED (Joel Reymont) Date: Sun, 24 Oct 2010 21:59:11 +0100 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> Message-ID: <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote: > I wonder how responsive the system is to other events when running the > benchmark. The benchmark simulates several hundred clients hitting a (mochiweb) web server to read and write couchdb (json) documents. The system seems to stay -highly- responsive, 10x so compared to the same system not using NIFs at all. If low write response time is taken as a measure of system responsiveness then the first graph shows that the responsiveness of the system has increased dramatically. The write response here is the take taken to process a web request to write a couch document. --- http://twitter.com/wagerlabs From ulf.wiger@REDACTED Sun Oct 24 23:28:32 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 24 Oct 2010 23:28:32 +0200 Subject: [erlang-questions] Erlang-based opensource publish-subscribe servers In-Reply-To: References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> <438091.32912.qm@web35908.mail.mud.yahoo.com> Message-ID: On 24 Oct 2010, at 21:22, Zvi . wrote: > 3. Use Ulf's gproc (looks it uses gen_leader abd requires patching Erlang?) No, it requires no patches of Erlang. I patched Erlang in the first prototype, but only to demonstrate what it could look like if gproc were started as part of the kernel application, and the functionality integrated into the standard behaviours. It does use gen_leader, though. As far as I can tell, quite a few people actually use gen_leader in real systems nowadays. There are a few forks on github to choose from. BR, Ulf W > > I currently working on 1. > > Zvi > > On Sun, Oct 24, 2010 at 8:41 PM, Andrew Stone wrote: > >> If you want something powerful you can checkout rabbitmq. It may be >> overkill but >> it's pretty damn easy to get up and running. >> >> http://www.rabbitmq.com/ >> >> -Andrew >> >> >> >> ----- Original Message ---- >> From: Zvi >> To: erlang-questions@REDACTED >> Sent: Sun, October 24, 2010 1:10:46 PM >> Subject: [erlang-questions] Erlang-based opensource publish-subscribe >> servers >> >> Hi, >> >> I looking for the opensource examples of Erlang-based publish- >> subscribe servers. >> Any suggestions? >> >> Thanks in advance, >> Zvi >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > > -- > Zvi Avraham, CTO > Nivertech Ltd Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From surferjeff@REDACTED Sun Oct 24 23:37:23 2010 From: surferjeff@REDACTED (Jeffrey Rennie) Date: Sun, 24 Oct 2010 14:37:23 -0700 Subject: Ridiculuous newbie mnesia question. Message-ID: I seem to be stuck in a state where I can't create a table because it exists, but I can't delete the table because it doesn't exist! I searched the mnesia documentation, and I couldn't find a definition of the "Tab" arg passed to many mnesia functions. Here's what I tried: Eshell V5.7.5 (abort with ^G) 2> c(backgammon). {ok,backgammon} 5> c(backgammon). {ok,backgammon} 7> mnesia:start(). ok 8> backgammon:init(). {atomic,ok} 11> c(backgammon). {ok,backgammon} 12> backgammon:dumpfields(). [player_turn,dice,board] 13> c(backgammon). {ok,backgammon} 14> backgammon:init(). {aborted,{already_exists,backgammon_games}} 15> mnesia:info(). ---> Processes holding locks <--- ---> Processes waiting for locks <--- ---> Participant transactions <--- ---> Coordinator transactions <--- ---> Uncertain transactions <--- ---> Active tables <--- schema : with 2 records occupying 546 words of mem ===> System info in version "4.4.13", debug level = none <=== opt_disc. Directory "c:/Users/jeff/code/code/scratch/erlang/Mnesia.nonode@REDACTED" is NOT used. use fallback at restart = false running db nodes = [nonode@REDACTED] stopped db nodes = [] master node tables = [] remote = [backgammon_games] ram_copies = [schema] disc_copies = [] disc_only_copies = [] [] = [backgammon_games] [{nonode@REDACTED,ram_copies}] = [schema] 3 transactions committed, 1 aborted, 0 restarted, 0 logged to disc 0 held locks, 0 in queue; 0 local transactions, 0 remote 0 transactions waits for other nodes: [] ok %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% BEGIN CONFUSING BIT 17> mnesia:delete_table(backgammon_games). {aborted,{no_exists,backgammon_games}} 18> backgammon:init(). {aborted,{already_exists,backgammon_games}} 19> %% END CONFUSING BIT %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Here's my backgammon.erl: -module(backgammon). -compile(export_all). -include("backgammon.hrl"). create_tables(MnesiaNodes) -> mnesia:create_table(backgammon_games, [{attributes, record_info(fields, game_record)}, {ram_copies, MnesiaNodes}]). init() -> create_tables([a@REDACTED, b@REDACTED]). dumpfields() -> record_info(fields, game_state). From ulf.wiger@REDACTED Mon Oct 25 00:01:31 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 25 Oct 2010 00:01:31 +0200 Subject: [erlang-questions] Ridiculuous newbie mnesia question. In-Reply-To: References: Message-ID: Hi Jeffrey, The problem is that you have specified that the backgammon_games table should have copies on a@REDACTED and b@REDACTED, but your Erlang shell has the node name nonode@REDACTED (i.e. Erlang was started without distribution). One might argue whether it wouldn't have been better if mnesia had refused to create the table in the first place. As it is now, it believes it exists, because there is an entry in the schema, but it refuses to delete it because it has no active replicas. But since that's not what it complains about, I can understand your confusion. :) BR, Ulf W On 24 Oct 2010, at 23:37, Jeffrey Rennie wrote: > I seem to be stuck in a state where I can't create a table because it > exists, but I can't delete the table because it doesn't exist! > > I searched the mnesia documentation, and I couldn't find a definition > of the "Tab" arg passed to many mnesia functions. > > Here's what I tried: > > Eshell V5.7.5 (abort with ^G) > 2> c(backgammon). > {ok,backgammon} > 5> c(backgammon). > {ok,backgammon} > 7> mnesia:start(). > ok > 8> backgammon:init(). > {atomic,ok} > 11> c(backgammon). > {ok,backgammon} > 12> backgammon:dumpfields(). > [player_turn,dice,board] > 13> c(backgammon). > {ok,backgammon} > 14> backgammon:init(). > {aborted,{already_exists,backgammon_games}} > 15> mnesia:info(). > ---> Processes holding locks <--- > ---> Processes waiting for locks <--- > ---> Participant transactions <--- > ---> Coordinator transactions <--- > ---> Uncertain transactions <--- > ---> Active tables <--- > schema : with 2 records occupying 546 words of mem > ===> System info in version "4.4.13", debug level = none <=== > opt_disc. Directory > "c:/Users/jeff/code/code/scratch/erlang/Mnesia.nonode@REDACTED" is NOT > used. > use fallback at restart = false > running db nodes = [nonode@REDACTED] > stopped db nodes = [] > master node tables = [] > remote = [backgammon_games] > ram_copies = [schema] > disc_copies = [] > disc_only_copies = [] > [] = [backgammon_games] > [{nonode@REDACTED,ram_copies}] = [schema] > 3 transactions committed, 1 aborted, 0 restarted, 0 logged to disc > 0 held locks, 0 in queue; 0 local transactions, 0 remote > 0 transactions waits for other nodes: [] > ok > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > %% BEGIN CONFUSING BIT > 17> mnesia:delete_table(backgammon_games). > {aborted,{no_exists,backgammon_games}} > 18> backgammon:init(). > {aborted,{already_exists,backgammon_games}} > 19> > %% END CONFUSING BIT > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > > Here's my backgammon.erl: > -module(backgammon). > > -compile(export_all). > > -include("backgammon.hrl"). > > create_tables(MnesiaNodes) -> > mnesia:create_table(backgammon_games, > [{attributes, record_info(fields, game_record)}, > {ram_copies, MnesiaNodes}]). > > init() -> > create_tables([a@REDACTED, b@REDACTED]). > > dumpfields() -> > record_info(fields, game_state). > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ivan@REDACTED Sun Oct 24 23:34:26 2010 From: ivan@REDACTED (Ivan Uemlianin) Date: Sun, 24 Oct 2010 22:34:26 +0100 Subject: [erlang-questions] Erlang-based opensource publish-subscribe servers In-Reply-To: References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> <438091.32912.qm@web35908.mail.mud.yahoo.com> Message-ID: <4CC4A662.3040900@llaisdy.com> Dear Zvi gproc looks useful. And have you read Richard Jones' three-part tutorial, starting here: http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1/ Best Ivan On 24/10/2010 20:22, Zvi . wrote: > no, > I need something lightweight, but able to work in a large cluster (10s of > nodes). > > The ideas I have: > 1. Use ets to store mappings of Subscriber Pids to TopicIDs - this ets will > obviously be a bottleneck. > 2. Have a process per TopicID, which hold list of all it's Subscriber Pids > in it's state. Now the bottleneck is mapping TopicID to it's Pid > 3. Use Ulf's gproc (looks it uses gen_leader abd requires patching Erlang?) > > I currently working on 1. > > Zvi From krab@REDACTED Sun Oct 24 23:57:04 2010 From: krab@REDACTED (Kresten Krab Thorup) Date: Sun, 24 Oct 2010 23:57:04 +0200 Subject: [erlang-questions] Ridiculuous newbie mnesia question. In-Reply-To: References: Message-ID: <23EED039-AF62-4BDD-8473-0E6CC430656C@trifork.com> Looks like your node is called noname@REDACTED (ie you did not specify -name option to the erl command) while your script expects it to be called a@REDACTED? I'm no expert in mnesia, but what happens if you actually have nodes a@REDACTED and b@REDACTED running? Its obvious why the second create fails, but maybe you cannot delete because mnesia cannot find the named nodes? Kresten On 24/10/2010, at 23.38, "Jeffrey Rennie" wrote: > I seem to be stuck in a state where I can't create a table because it > exists, but I can't delete the table because it doesn't exist! > > I searched the mnesia documentation, and I couldn't find a definition > of the "Tab" arg passed to many mnesia functions. > > Here's what I tried: > > Eshell V5.7.5 (abort with ^G) > 2> c(backgammon). > {ok,backgammon} > 5> c(backgammon). > {ok,backgammon} > 7> mnesia:start(). > ok > 8> backgammon:init(). > {atomic,ok} > 11> c(backgammon). > {ok,backgammon} > 12> backgammon:dumpfields(). > [player_turn,dice,board] > 13> c(backgammon). > {ok,backgammon} > 14> backgammon:init(). > {aborted,{already_exists,backgammon_games}} > 15> mnesia:info(). > ---> Processes holding locks <--- > ---> Processes waiting for locks <--- > ---> Participant transactions <--- > ---> Coordinator transactions <--- > ---> Uncertain transactions <--- > ---> Active tables <--- > schema : with 2 records occupying 546 words of mem > ===> System info in version "4.4.13", debug level = none <=== > opt_disc. Directory > "c:/Users/jeff/code/code/scratch/erlang/Mnesia.nonode@REDACTED" is NOT > used. > use fallback at restart = false > running db nodes = [nonode@REDACTED] > stopped db nodes = [] > master node tables = [] > remote = [backgammon_games] > ram_copies = [schema] > disc_copies = [] > disc_only_copies = [] > [] = [backgammon_games] > [{nonode@REDACTED,ram_copies}] = [schema] > 3 transactions committed, 1 aborted, 0 restarted, 0 logged to disc > 0 held locks, 0 in queue; 0 local transactions, 0 remote > 0 transactions waits for other nodes: [] > ok > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > %% BEGIN CONFUSING BIT > 17> mnesia:delete_table(backgammon_games). > {aborted,{no_exists,backgammon_games}} > 18> backgammon:init(). > {aborted,{already_exists,backgammon_games}} > 19> > %% END CONFUSING BIT > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > > Here's my backgammon.erl: > -module(backgammon). > > -compile(export_all). > > -include("backgammon.hrl"). > > create_tables(MnesiaNodes) -> > mnesia:create_table(backgammon_games, > [{attributes, record_info(fields, game_record)}, > {ram_copies, MnesiaNodes}]). > > init() -> > create_tables([a@REDACTED, b@REDACTED]). > > dumpfields() -> > record_info(fields, game_state). > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From surferjeff@REDACTED Mon Oct 25 01:08:51 2010 From: surferjeff@REDACTED (Jeffrey Rennie) Date: Sun, 24 Oct 2010 16:08:51 -0700 Subject: [erlang-questions] Ridiculuous newbie mnesia question. In-Reply-To: References: Message-ID: <00b101cb73d0$6e1bab50$4a5301f0$@com> How did my first call to backgammon:init() succeed? 8> backgammon:init(). {atomic,ok} > -----Original Message----- > From: Ulf Wiger [mailto:ulf.wiger@REDACTED] > Sent: Sunday, October 24, 2010 3:02 PM > To: Jeffrey Rennie > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Ridiculuous newbie mnesia question. > > > Hi Jeffrey, > > The problem is that you have specified that the backgammon_games > table should have copies on a@REDACTED and b@REDACTED, but your > Erlang shell has the node name nonode@REDACTED (i.e. Erlang was > started without distribution). > > One might argue whether it wouldn't have been better if mnesia had > refused to create the table in the first place. As it is now, it > believes > it exists, because there is an entry in the schema, but it refuses to > delete it because it has no active replicas. But since that's not what > it > complains about, I can understand your confusion. :) > > BR, > Ulf W > > > On 24 Oct 2010, at 23:37, Jeffrey Rennie wrote: > > > I seem to be stuck in a state where I can't create a table because it > > exists, but I can't delete the table because it doesn't exist! > > > > I searched the mnesia documentation, and I couldn't find a definition > > of the "Tab" arg passed to many mnesia functions. > > > > Here's what I tried: > > > > Eshell V5.7.5 (abort with ^G) > > 2> c(backgammon). > > {ok,backgammon} > > 5> c(backgammon). > > {ok,backgammon} > > 7> mnesia:start(). > > ok > > 8> backgammon:init(). > > {atomic,ok} > > 11> c(backgammon). > > {ok,backgammon} > > 12> backgammon:dumpfields(). > > [player_turn,dice,board] > > 13> c(backgammon). > > {ok,backgammon} > > 14> backgammon:init(). > > {aborted,{already_exists,backgammon_games}} > > 15> mnesia:info(). > > ---> Processes holding locks <--- > > ---> Processes waiting for locks <--- > > ---> Participant transactions <--- > > ---> Coordinator transactions <--- > > ---> Uncertain transactions <--- > > ---> Active tables <--- > > schema : with 2 records occupying 546 words of > mem > > ===> System info in version "4.4.13", debug level = none <=== > > opt_disc. Directory > > "c:/Users/jeff/code/code/scratch/erlang/Mnesia.nonode@REDACTED" is NOT > > used. > > use fallback at restart = false > > running db nodes = [nonode@REDACTED] > > stopped db nodes = [] > > master node tables = [] > > remote = [backgammon_games] > > ram_copies = [schema] > > disc_copies = [] > > disc_only_copies = [] > > [] = [backgammon_games] > > [{nonode@REDACTED,ram_copies}] = [schema] > > 3 transactions committed, 1 aborted, 0 restarted, 0 logged to disc > > 0 held locks, 0 in queue; 0 local transactions, 0 remote > > 0 transactions waits for other nodes: [] > > ok > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > %% BEGIN CONFUSING BIT > > 17> mnesia:delete_table(backgammon_games). > > {aborted,{no_exists,backgammon_games}} > > 18> backgammon:init(). > > {aborted,{already_exists,backgammon_games}} > > 19> > > %% END CONFUSING BIT > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > > > > > Here's my backgammon.erl: > > -module(backgammon). > > > > -compile(export_all). > > > > -include("backgammon.hrl"). > > > > create_tables(MnesiaNodes) -> > > mnesia:create_table(backgammon_games, > > [{attributes, record_info(fields, game_record)}, > > {ram_copies, MnesiaNodes}]). > > > > init() -> > > create_tables([a@REDACTED, b@REDACTED]). > > > > dumpfields() -> > > record_info(fields, game_state). > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > From james.hague@REDACTED Mon Oct 25 02:12:05 2010 From: james.hague@REDACTED (James Hague) Date: Sun, 24 Oct 2010 19:12:05 -0500 Subject: constant floating point values Message-ID: Let's say I call an external function like this: my_module:add(1.0, 2.0) I think what happens right now--from looking at the BEAM code--is that 1.0 is created on the heap (12 bytes in the 32-bit emulator) and 2.0 is created on the heap (another 12 bytes). Then pointers to those two floats are passed as parameters to the external function. What's the technical reason that BEAM doesn't support constant floats, the same way as constant tuples and lists? (Note that in expressions where floating point math can be determined at compile time, such as "A * 2.5", then the value 2.5 is treated as a constant value and loaded directly into a virtual floating point register. Nice!) From max.lapshin@REDACTED Mon Oct 25 06:21:48 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 25 Oct 2010 08:21:48 +0400 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> Message-ID: There is other interesting question: are there plans to support asynchronous disk IO? It is already somehow implemented in FreeBSD and in progress in Linux. From surferjeff@REDACTED Mon Oct 25 07:30:31 2010 From: surferjeff@REDACTED (Jeffrey Rennie) Date: Sun, 24 Oct 2010 22:30:31 -0700 Subject: [erlang-questions] Ridiculuous newbie mnesia question. In-Reply-To: References: Message-ID: <00d701cb7405$bff89930$3fe9cb90$@com> I launched a new erl node with a proper -sname, and I'm still experiencing the same problem. Yes, nodes a@REDACTED and b@REDACTED are up and running. (emacs@REDACTED)17> backgammon:init(). {atomic,ok} (emacs@REDACTED)18> backgammon:insert_game(1, "Jeff"). {aborted,{no_exists,game_record}} (emacs@REDACTED)19> backgammon:init(). {aborted,{already_exists,game_record}} (emacs@REDACTED)20> mnesia:info(). ---> Processes holding locks <--- ---> Processes waiting for locks <--- ---> Participant transactions <--- ---> Coordinator transactions <--- ---> Uncertain transactions <--- ---> Active tables <--- schema : with 3 records occupying 656 words of mem ===> System info in version "4.4.13", debug level = none <=== opt_disc. Directory "c:/Users/jeff/code/code/scratch/erlang/Mnesia.emacs@REDACTED" is NOT used. use fallback at restart = false running db nodes = [emacs@REDACTED] stopped db nodes = [] master node tables = [] remote = [backgammon_games,game_record] ram_copies = [schema] disc_copies = [] disc_only_copies = [] [] = [backgammon_games,game_record] [{emacs@REDACTED,ram_copies}] = [schema] 4 transactions committed, 3 aborted, 0 restarted, 0 logged to disc 0 held locks, 0 in queue; 0 local transactions, 0 remote 0 transactions waits for other nodes: [] ok (emacs@REDACTED)21> mnesia:delete_table(backgammon_games). {aborted,{no_exists,backgammon_games}} (emacs@REDACTED)22> mnesia:delete_table(game_record). {aborted,{no_exists,game_record}} (emacs@REDACTED)23> backgammon:init(). {aborted,{already_exists,game_record}} (emacs@REDACTED)24> > -----Original Message----- > From: Ulf Wiger [mailto:ulf.wiger@REDACTED] > Sent: Sunday, October 24, 2010 3:02 PM > To: Jeffrey Rennie > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Ridiculuous newbie mnesia question. > > > Hi Jeffrey, > > The problem is that you have specified that the backgammon_games > table should have copies on a@REDACTED and b@REDACTED, but your > Erlang shell has the node name nonode@REDACTED (i.e. Erlang was > started without distribution). > > One might argue whether it wouldn't have been better if mnesia had > refused to create the table in the first place. As it is now, it > believes > it exists, because there is an entry in the schema, but it refuses to > delete it because it has no active replicas. But since that's not what > it > complains about, I can understand your confusion. :) > > BR, > Ulf W > > > On 24 Oct 2010, at 23:37, Jeffrey Rennie wrote: > > > I seem to be stuck in a state where I can't create a table because it > > exists, but I can't delete the table because it doesn't exist! > > > > I searched the mnesia documentation, and I couldn't find a definition > > of the "Tab" arg passed to many mnesia functions. > > > > Here's what I tried: > > > > Eshell V5.7.5 (abort with ^G) > > 2> c(backgammon). > > {ok,backgammon} > > 5> c(backgammon). > > {ok,backgammon} > > 7> mnesia:start(). > > ok > > 8> backgammon:init(). > > {atomic,ok} > > 11> c(backgammon). > > {ok,backgammon} > > 12> backgammon:dumpfields(). > > [player_turn,dice,board] > > 13> c(backgammon). > > {ok,backgammon} > > 14> backgammon:init(). > > {aborted,{already_exists,backgammon_games}} > > 15> mnesia:info(). > > ---> Processes holding locks <--- > > ---> Processes waiting for locks <--- > > ---> Participant transactions <--- > > ---> Coordinator transactions <--- > > ---> Uncertain transactions <--- > > ---> Active tables <--- > > schema : with 2 records occupying 546 words of > mem > > ===> System info in version "4.4.13", debug level = none <=== > > opt_disc. Directory > > "c:/Users/jeff/code/code/scratch/erlang/Mnesia.nonode@REDACTED" is NOT > > used. > > use fallback at restart = false > > running db nodes = [nonode@REDACTED] > > stopped db nodes = [] > > master node tables = [] > > remote = [backgammon_games] > > ram_copies = [schema] > > disc_copies = [] > > disc_only_copies = [] > > [] = [backgammon_games] > > [{nonode@REDACTED,ram_copies}] = [schema] > > 3 transactions committed, 1 aborted, 0 restarted, 0 logged to disc > > 0 held locks, 0 in queue; 0 local transactions, 0 remote > > 0 transactions waits for other nodes: [] > > ok > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > %% BEGIN CONFUSING BIT > > 17> mnesia:delete_table(backgammon_games). > > {aborted,{no_exists,backgammon_games}} > > 18> backgammon:init(). > > {aborted,{already_exists,backgammon_games}} > > 19> > > %% END CONFUSING BIT > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > > > > > Here's my backgammon.erl: > > -module(backgammon). > > > > -compile(export_all). > > > > -include("backgammon.hrl"). > > > > create_tables(MnesiaNodes) -> > > mnesia:create_table(backgammon_games, > > [{attributes, record_info(fields, game_record)}, > > {ram_copies, MnesiaNodes}]). > > > > init() -> > > create_tables([a@REDACTED, b@REDACTED]). > > > > dumpfields() -> > > record_info(fields, game_state). > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > From ulf.wiger@REDACTED Mon Oct 25 08:41:47 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 25 Oct 2010 08:41:47 +0200 Subject: [erlang-questions] Ridiculuous newbie mnesia question. In-Reply-To: <00d701cb7405$bff89930$3fe9cb90$@com> References: <00d701cb7405$bff89930$3fe9cb90$@com> Message-ID: <1AD7068E-BAE8-44BB-A77C-77BA32E08F49@erlang-solutions.com> Hi Jeffrey, Looking at the mnesia:info() output, there are a couple of things to note. 1. 'running db nodes = [emacs@REDACTED]' means that while a@REDACTED and b@REDACTED might be running at the erlang level, perhaps even with mnesia runnin, they are not connected to emacs@REDACTED 2. The line '"c:/.../Mnesia.emacs@REDACTED" is NOT used' means that you have not created a persistent schema. Normally, the persistent schema indicates which nodes are part of the mnesia cluster. Any other nodes that want to join should specify -mnesia extra_db_nodes Ns, where Ns includes at least some of the nodes listed in the persistent schema. To create a persistent schema, you run this before starting mnesia: mnesia:create_schema(Nodes). ...with the nodes in Nodes all up and running. You only need to call it from one node; it will create a schema on each node automatically. BR, Ulf W On 25 Oct 2010, at 07:30, Jeffrey Rennie wrote: > I launched a new erl node with a proper -sname, and I'm still experiencing > the same problem. Yes, nodes a@REDACTED and b@REDACTED are up and running. > > (emacs@REDACTED)17> backgammon:init(). > {atomic,ok} > (emacs@REDACTED)18> backgammon:insert_game(1, "Jeff"). > {aborted,{no_exists,game_record}} > (emacs@REDACTED)19> backgammon:init(). > {aborted,{already_exists,game_record}} > (emacs@REDACTED)20> mnesia:info(). > ---> Processes holding locks <--- > ---> Processes waiting for locks <--- > ---> Participant transactions <--- > ---> Coordinator transactions <--- > ---> Uncertain transactions <--- > ---> Active tables <--- > schema : with 3 records occupying 656 words of mem > ===> System info in version "4.4.13", debug level = none <=== > opt_disc. Directory > "c:/Users/jeff/code/code/scratch/erlang/Mnesia.emacs@REDACTED" is NOT used. > use fallback at restart = false > running db nodes = [emacs@REDACTED] > stopped db nodes = [] > master node tables = [] > remote = [backgammon_games,game_record] > ram_copies = [schema] > disc_copies = [] > disc_only_copies = [] > [] = [backgammon_games,game_record] > [{emacs@REDACTED,ram_copies}] = [schema] > 4 transactions committed, 3 aborted, 0 restarted, 0 logged to disc > 0 held locks, 0 in queue; 0 local transactions, 0 remote > 0 transactions waits for other nodes: [] > ok > (emacs@REDACTED)21> mnesia:delete_table(backgammon_games). > {aborted,{no_exists,backgammon_games}} > (emacs@REDACTED)22> mnesia:delete_table(game_record). > {aborted,{no_exists,game_record}} > (emacs@REDACTED)23> backgammon:init(). > {aborted,{already_exists,game_record}} > (emacs@REDACTED)24> > > >> -----Original Message----- >> From: Ulf Wiger [mailto:ulf.wiger@REDACTED] >> Sent: Sunday, October 24, 2010 3:02 PM >> To: Jeffrey Rennie >> Cc: erlang-questions@REDACTED >> Subject: Re: [erlang-questions] Ridiculuous newbie mnesia question. >> >> >> Hi Jeffrey, >> >> The problem is that you have specified that the backgammon_games >> table should have copies on a@REDACTED and b@REDACTED, but your >> Erlang shell has the node name nonode@REDACTED (i.e. Erlang was >> started without distribution). >> >> One might argue whether it wouldn't have been better if mnesia had >> refused to create the table in the first place. As it is now, it >> believes >> it exists, because there is an entry in the schema, but it refuses to >> delete it because it has no active replicas. But since that's not what >> it >> complains about, I can understand your confusion. :) >> >> BR, >> Ulf W >> >> >> On 24 Oct 2010, at 23:37, Jeffrey Rennie wrote: >> >>> I seem to be stuck in a state where I can't create a table because it >>> exists, but I can't delete the table because it doesn't exist! >>> >>> I searched the mnesia documentation, and I couldn't find a definition >>> of the "Tab" arg passed to many mnesia functions. >>> >>> Here's what I tried: >>> >>> Eshell V5.7.5 (abort with ^G) >>> 2> c(backgammon). >>> {ok,backgammon} >>> 5> c(backgammon). >>> {ok,backgammon} >>> 7> mnesia:start(). >>> ok >>> 8> backgammon:init(). >>> {atomic,ok} >>> 11> c(backgammon). >>> {ok,backgammon} >>> 12> backgammon:dumpfields(). >>> [player_turn,dice,board] >>> 13> c(backgammon). >>> {ok,backgammon} >>> 14> backgammon:init(). >>> {aborted,{already_exists,backgammon_games}} >>> 15> mnesia:info(). >>> ---> Processes holding locks <--- >>> ---> Processes waiting for locks <--- >>> ---> Participant transactions <--- >>> ---> Coordinator transactions <--- >>> ---> Uncertain transactions <--- >>> ---> Active tables <--- >>> schema : with 2 records occupying 546 words of >> mem >>> ===> System info in version "4.4.13", debug level = none <=== >>> opt_disc. Directory >>> "c:/Users/jeff/code/code/scratch/erlang/Mnesia.nonode@REDACTED" is NOT >>> used. >>> use fallback at restart = false >>> running db nodes = [nonode@REDACTED] >>> stopped db nodes = [] >>> master node tables = [] >>> remote = [backgammon_games] >>> ram_copies = [schema] >>> disc_copies = [] >>> disc_only_copies = [] >>> [] = [backgammon_games] >>> [{nonode@REDACTED,ram_copies}] = [schema] >>> 3 transactions committed, 1 aborted, 0 restarted, 0 logged to disc >>> 0 held locks, 0 in queue; 0 local transactions, 0 remote >>> 0 transactions waits for other nodes: [] >>> ok >>> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% >>> %% BEGIN CONFUSING BIT >>> 17> mnesia:delete_table(backgammon_games). >>> {aborted,{no_exists,backgammon_games}} >>> 18> backgammon:init(). >>> {aborted,{already_exists,backgammon_games}} >>> 19> >>> %% END CONFUSING BIT >>> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% >>> >>> >>> Here's my backgammon.erl: >>> -module(backgammon). >>> >>> -compile(export_all). >>> >>> -include("backgammon.hrl"). >>> >>> create_tables(MnesiaNodes) -> >>> mnesia:create_table(backgammon_games, >>> [{attributes, record_info(fields, game_record)}, >>> {ram_copies, MnesiaNodes}]). >>> >>> init() -> >>> create_tables([a@REDACTED, b@REDACTED]). >>> >>> dumpfields() -> >>> record_info(fields, game_state). >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From mazen.harake@REDACTED Mon Oct 25 09:56:37 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 25 Oct 2010 09:56:37 +0200 Subject: [erlang-questions] Ridiculuous newbie mnesia question. In-Reply-To: <1AD7068E-BAE8-44BB-A77C-77BA32E08F49@erlang-solutions.com> References: <00d701cb7405$bff89930$3fe9cb90$@com> <1AD7068E-BAE8-44BB-A77C-77BA32E08F49@erlang-solutions.com> Message-ID: <4CC53835.7040000@erlang-solutions.com> Although all these points are valid I would like to add a third point which should solve this particular situation; 3. The line '[] = [backgammon_games,game_record]' means that no node, which is a copy-holder of these tables, is connected. This means that you should first start one of those nodes (or a "fake" of one with the same nodename) and then start mnesia and do mnesia:change_config(extra_db_nodes, N) (as Ulf mentioned) where N is a copy-holder of those tables. When this is done the node name will show up like this: '[{foo@REDACTED, ram_copies}] = [backgammon_games,game_record]' and then it is safe to remove the table. The important part here is the "copy-holder" concept. An empty list means "no one is available", but the local node still knows about it so it "exists" but it doesn't exist in a place where you can change it (it is in a different dimension >:P). That is the reason why you can't create it nor delete it. /Mazen On 25/10/2010 08:41, Ulf Wiger wrote: > Hi Jeffrey, > > Looking at the mnesia:info() output, there are a couple of things > to note. > > 1. 'running db nodes = [emacs@REDACTED]' means that while a@REDACTED > and b@REDACTED might be running at the erlang level, perhaps even > with mnesia runnin, they are not connected to emacs@REDACTED > > 2. The line '"c:/.../Mnesia.emacs@REDACTED" is NOT used' means that > you have not created a persistent schema. Normally, the persistent > schema indicates which nodes are part of the mnesia cluster. Any > other nodes that want to join should specify -mnesia extra_db_nodes Ns, > where Ns includes at least some of the nodes listed in the persistent schema. > > To create a persistent schema, you run this before starting mnesia: > > mnesia:create_schema(Nodes). > > ...with the nodes in Nodes all up and running. You only need to call it > from one node; it will create a schema on each node automatically. > > BR, > Ulf W > > On 25 Oct 2010, at 07:30, Jeffrey Rennie wrote: > >> I launched a new erl node with a proper -sname, and I'm still experiencing >> the same problem. Yes, nodes a@REDACTED and b@REDACTED are up and running. >> >> (emacs@REDACTED)17> backgammon:init(). >> {atomic,ok} >> (emacs@REDACTED)18> backgammon:insert_game(1, "Jeff"). >> {aborted,{no_exists,game_record}} >> (emacs@REDACTED)19> backgammon:init(). >> {aborted,{already_exists,game_record}} >> (emacs@REDACTED)20> mnesia:info(). >> ---> Processes holding locks<--- >> ---> Processes waiting for locks<--- >> ---> Participant transactions<--- >> ---> Coordinator transactions<--- >> ---> Uncertain transactions<--- >> ---> Active tables<--- >> schema : with 3 records occupying 656 words of mem >> ===> System info in version "4.4.13", debug level = none<=== >> opt_disc. Directory >> "c:/Users/jeff/code/code/scratch/erlang/Mnesia.emacs@REDACTED" is NOT used. >> use fallback at restart = false >> running db nodes = [emacs@REDACTED] >> stopped db nodes = [] >> master node tables = [] >> remote = [backgammon_games,game_record] >> ram_copies = [schema] >> disc_copies = [] >> disc_only_copies = [] >> [] = [backgammon_games,game_record] >> [{emacs@REDACTED,ram_copies}] = [schema] >> 4 transactions committed, 3 aborted, 0 restarted, 0 logged to disc >> 0 held locks, 0 in queue; 0 local transactions, 0 remote >> 0 transactions waits for other nodes: [] >> ok >> (emacs@REDACTED)21> mnesia:delete_table(backgammon_games). >> {aborted,{no_exists,backgammon_games}} >> (emacs@REDACTED)22> mnesia:delete_table(game_record). >> {aborted,{no_exists,game_record}} >> (emacs@REDACTED)23> backgammon:init(). >> {aborted,{already_exists,game_record}} >> (emacs@REDACTED)24> >> >> >>> -----Original Message----- >>> From: Ulf Wiger [mailto:ulf.wiger@REDACTED] >>> Sent: Sunday, October 24, 2010 3:02 PM >>> To: Jeffrey Rennie >>> Cc: erlang-questions@REDACTED >>> Subject: Re: [erlang-questions] Ridiculuous newbie mnesia question. >>> >>> >>> Hi Jeffrey, >>> >>> The problem is that you have specified that the backgammon_games >>> table should have copies on a@REDACTED and b@REDACTED, but your >>> Erlang shell has the node name nonode@REDACTED (i.e. Erlang was >>> started without distribution). >>> >>> One might argue whether it wouldn't have been better if mnesia had >>> refused to create the table in the first place. As it is now, it >>> believes >>> it exists, because there is an entry in the schema, but it refuses to >>> delete it because it has no active replicas. But since that's not what >>> it >>> complains about, I can understand your confusion. :) >>> >>> BR, >>> Ulf W >>> >>> >>> On 24 Oct 2010, at 23:37, Jeffrey Rennie wrote: >>> >>>> I seem to be stuck in a state where I can't create a table because it >>>> exists, but I can't delete the table because it doesn't exist! >>>> >>>> I searched the mnesia documentation, and I couldn't find a definition >>>> of the "Tab" arg passed to many mnesia functions. >>>> >>>> Here's what I tried: >>>> >>>> Eshell V5.7.5 (abort with ^G) >>>> 2> c(backgammon). >>>> {ok,backgammon} >>>> 5> c(backgammon). >>>> {ok,backgammon} >>>> 7> mnesia:start(). >>>> ok >>>> 8> backgammon:init(). >>>> {atomic,ok} >>>> 11> c(backgammon). >>>> {ok,backgammon} >>>> 12> backgammon:dumpfields(). >>>> [player_turn,dice,board] >>>> 13> c(backgammon). >>>> {ok,backgammon} >>>> 14> backgammon:init(). >>>> {aborted,{already_exists,backgammon_games}} >>>> 15> mnesia:info(). >>>> ---> Processes holding locks<--- >>>> ---> Processes waiting for locks<--- >>>> ---> Participant transactions<--- >>>> ---> Coordinator transactions<--- >>>> ---> Uncertain transactions<--- >>>> ---> Active tables<--- >>>> schema : with 2 records occupying 546 words of >>> mem >>>> ===> System info in version "4.4.13", debug level = none<=== >>>> opt_disc. Directory >>>> "c:/Users/jeff/code/code/scratch/erlang/Mnesia.nonode@REDACTED" is NOT >>>> used. >>>> use fallback at restart = false >>>> running db nodes = [nonode@REDACTED] >>>> stopped db nodes = [] >>>> master node tables = [] >>>> remote = [backgammon_games] >>>> ram_copies = [schema] >>>> disc_copies = [] >>>> disc_only_copies = [] >>>> [] = [backgammon_games] >>>> [{nonode@REDACTED,ram_copies}] = [schema] >>>> 3 transactions committed, 1 aborted, 0 restarted, 0 logged to disc >>>> 0 held locks, 0 in queue; 0 local transactions, 0 remote >>>> 0 transactions waits for other nodes: [] >>>> ok >>>> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% >>>> %% BEGIN CONFUSING BIT >>>> 17> mnesia:delete_table(backgammon_games). >>>> {aborted,{no_exists,backgammon_games}} >>>> 18> backgammon:init(). >>>> {aborted,{already_exists,backgammon_games}} >>>> 19> >>>> %% END CONFUSING BIT >>>> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% >>>> >>>> >>>> Here's my backgammon.erl: >>>> -module(backgammon). >>>> >>>> -compile(export_all). >>>> >>>> -include("backgammon.hrl"). >>>> >>>> create_tables(MnesiaNodes) -> >>>> mnesia:create_table(backgammon_games, >>>> [{attributes, record_info(fields, game_record)}, >>>> {ram_copies, MnesiaNodes}]). >>>> >>>> init() -> >>>> create_tables([a@REDACTED, b@REDACTED]). >>>> >>>> dumpfields() -> >>>> record_info(fields, game_state). >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>> http://erlang-solutions.com >>> >> >> > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From tony@REDACTED Mon Oct 25 09:25:16 2010 From: tony@REDACTED (Tony Rogvall) Date: Mon, 25 Oct 2010 09:25:16 +0200 Subject: [erlang-questions] constant floating point values In-Reply-To: References: Message-ID: <8413866C-25BE-4EC7-8B31-965F7833B878@rogvall.se> You have to look a bit harder ;-) The beam loader is doing a lot of work under the hood. Have a look in emulator//opt/smp/beam_opcodes.c This file is generated by emulator/utils/beam_makeops (perl script) using the file emulator/beam/ops.tab as input. The beam_opcodes.c contains load scripts (transforms) that are emulated by emulator/beam/beam_load.c during the beam file loading. From what I understand most constants are today associated with the module in a constant pool. And operations that use them are transformed during load time. /Tony On 25 okt 2010, at 02.12, James Hague wrote: > Let's say I call an external function like this: > > my_module:add(1.0, 2.0) > > I think what happens right now--from looking at the BEAM code--is that 1.0 > is created on the heap (12 bytes in the 32-bit emulator) and 2.0 is created > on the heap (another 12 bytes). Then pointers to those two floats are passed > as parameters to the external function. > > What's the technical reason that BEAM doesn't support constant floats, the > same way as constant tuples and lists? > > (Note that in expressions where floating point math can be determined at > compile time, such as "A * 2.5", then the value 2.5 is treated as a constant > value and loaded directly into a virtual floating point register. Nice!) "Have run Make so many times I dunno what's installed anymore" From muharem@REDACTED Mon Oct 25 10:33:25 2010 From: muharem@REDACTED (Muharem Hrnjadovic) Date: Mon, 25 Oct 2010 10:33:25 +0200 Subject: EUnit setUp/tearDown question Message-ID: <4CC540D5.6070508@lbox.cc> Hello there! I am trying to set up a suite of tests where a temporary directory is created and torn down for each test. {{{ 1 -module(eunit_tests). 2 3 -include_lib("eunit.hrl"). 4 5 is_link(P) -> 6 {ok, FIT} = file:read_link_info(P), 7 element(3,FIT) =:= symlink. 8 9 test_is_link_with_regular(T) -> 10 P = filename:join(T, "a"), 11 ?cmd(io_lib:format("touch ~s", [P])), 12 ?assert(not is_link(P)). 13 14 test_is_link_with_link(T) -> 15 P = filename:join(T, "a"), 16 ?cmd(io_lib:format("touch ~s", [P])), 17 ?cmd(io_lib:format("ln -s ~s ~s.link", [P, P])), 18 ?assert(is_link(P ++ ".link")). 19 20 is_link_test_() -> 21 {foreach, 22 fun() -> ?cmd("mktemp -d") end, 23 fun(T) -> ?cmd("rm -rf " ++ T) end, 24 ... 25 }. }}} My question is: what should go into line 24 so that the two tests (test_is_link_with_regular() and test_is_link_with_link()) are called with T (the temporary directory path)? Or is there a better way to achieve the same thing? Best regards/Mit freundlichen Gr??en -- Muharem Hrnjadovic Public key id : B2BBFCFC Key fingerprint : A5A3 CC67 2B87 D641 103F 5602 219F 6B60 B2BB FCFC -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 900 bytes Desc: OpenPGP digital signature URL: From caox@REDACTED Mon Oct 25 11:02:39 2010 From: caox@REDACTED (caox) Date: Mon, 25 Oct 2010 17:02:39 +0800 Subject: problem of ei_decode_string Message-ID: <0522386E-9DB2-423D-9497-4FD5C8907D49@lightpole.net> hi I find there will be problem with this api when the length of list to be decoded is larger than 65535. Is this the limit of decoded string, or a bug? From gianfranco.alongi@REDACTED Mon Oct 25 11:08:39 2010 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Mon, 25 Oct 2010 09:08:39 +0000 (GMT) Subject: [erlang-questions] EUnit setUp/tearDown question In-Reply-To: <4CC540D5.6070508@lbox.cc> Message-ID: <18951558.1471287997719515.JavaMail.root@zimbra> Hi, -module(wth). -include_lib("eunit/include/eunit.hrl"). is_link_test_() -> {foreach, fun() -> ?cmd("mktemp -d temp") -- "\n" end, fun(T) -> ?cmd("rm -rf " ++ T) end, [ fun test_is_link_with_regular/1, fun test_is_link_with_link/1 ] }. % ------------------------------------------------------------------------------ is_link(P) -> {ok, FIT} = file:read_link_info(P), element(3,FIT) =:= symlink. test_is_link_with_regular(T) -> {"Test is link with regular", fun() -> P = filename:join(T, "a"), ?cmd("touch "++P), ?assert(not is_link(P)) end}. test_is_link_with_link(T) -> {"Test Is link with Link", fun() -> P = filename:join(T, "a"), ?cmd("touch "++P), ?cmd("ln -s "++P++" "++P++".link"), ?assert(is_link(P ++ ".link")) end}. zenon$ erlc wth.erl zenon$ erl -pa -eval 'eunit:test(wth,[verbose]).' Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> ======================== EUnit ======================== module 'wth' wth: test_is_link_with_regular (Test is link with regular)...[0.008 s] ok wth: test_is_link_with_link (Test Is link with Link)...[0.011 s] ok [done in 0.049 s] ======================================================= All 2 tests passed. 1> ----- Original Message ----- From: "Muharem Hrnjadovic" To: erlang-questions@REDACTED Sent: Monday, October 25, 2010 9:33:25 AM GMT +00:00 GMT Britain, Ireland, Portugal Subject: [erlang-questions] EUnit setUp/tearDown question Hello there! I am trying to set up a suite of tests where a temporary directory is created and torn down for each test. {{{ 1 -module(eunit_tests). 2 3 -include_lib("eunit.hrl"). 4 5 is_link(P) -> 6 {ok, FIT} = file:read_link_info(P), 7 element(3,FIT) =:= symlink. 8 9 test_is_link_with_regular(T) -> 10 P = filename:join(T, "a"), 11 ?cmd(io_lib:format("touch ~s", [P])), 12 ?assert(not is_link(P)). 13 14 test_is_link_with_link(T) -> 15 P = filename:join(T, "a"), 16 ?cmd(io_lib:format("touch ~s", [P])), 17 ?cmd(io_lib:format("ln -s ~s ~s.link", [P, P])), 18 ?assert(is_link(P ++ ".link")). 19 20 is_link_test_() -> 21 {foreach, 22 fun() -> ?cmd("mktemp -d") end, 23 fun(T) -> ?cmd("rm -rf " ++ T) end, 24 ... 25 }. }}} My question is: what should go into line 24 so that the two tests (test_is_link_with_regular() and test_is_link_with_link()) are called with T (the temporary directory path)? Or is there a better way to achieve the same thing? Best regards/Mit freundlichen Gr??en -- Muharem Hrnjadovic Public key id : B2BBFCFC Key fingerprint : A5A3 CC67 2B87 D641 103F 5602 219F 6B60 B2BB FCFC From mk@REDACTED Mon Oct 25 12:15:28 2010 From: mk@REDACTED (Morten Krogh) Date: Mon, 25 Oct 2010 12:15:28 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: <20101023065215.GA2991@corelatus.se> References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> <4CC1726E.1080606@amberbio.com> <4CC18A00.5010708@amberbio.com> <20101023065215.GA2991@corelatus.se> Message-ID: <4CC558C0.40203@amberbio.com> Matthias My suggestion is probably more or less the same. I will read that paper when I get time. It is possible that choosing between iolists and binaries should be left to the user. Morten. On 10/23/10 8:52 AM, Matthias Lang wrote: > On Friday, October 22, Morten Krogh wrote: > >> Yes, but my suggestion would let the VM handle lists of binaries >> instead of the application programmer. > How is your suggestion different to "segmented binaries", described > in "9.3 Segmented binaries" in http://dotat.at/tmp/binaries.pdf ? > > (IIRC, segmented binaries were dropped because there wasn't a clear > performance gain.) > > Matt From muharem@REDACTED Mon Oct 25 12:20:46 2010 From: muharem@REDACTED (Muharem Hrnjadovic) Date: Mon, 25 Oct 2010 12:20:46 +0200 Subject: [erlang-questions] EUnit setUp/tearDown question In-Reply-To: <18951558.1471287997719515.JavaMail.root@zimbra> References: <18951558.1471287997719515.JavaMail.root@zimbra> Message-ID: <4CC559FE.1060404@lbox.cc> Hello Gianfranco, thanks a million for the example below! It helped a lot :-) On 10/25/2010 11:08 AM, Gianfranco Alongi wrote: > Hi, > > > -module(wth). > -include_lib("eunit/include/eunit.hrl"). > > is_link_test_() -> > {foreach, > fun() -> ?cmd("mktemp -d temp") -- "\n" end, > fun(T) -> ?cmd("rm -rf " ++ T) end, > [ > fun test_is_link_with_regular/1, > fun test_is_link_with_link/1 > ] > }. > > % ------------------------------------------------------------------------------ > is_link(P) -> > {ok, FIT} = file:read_link_info(P), > element(3,FIT) =:= symlink. > > test_is_link_with_regular(T) -> > {"Test is link with regular", > fun() -> > P = filename:join(T, "a"), > ?cmd("touch "++P), > ?assert(not is_link(P)) > end}. > > test_is_link_with_link(T) -> > {"Test Is link with Link", > fun() -> > P = filename:join(T, "a"), > ?cmd("touch "++P), > ?cmd("ln -s "++P++" "++P++".link"), > ?assert(is_link(P ++ ".link")) > end}. > > > > zenon$ erlc wth.erl > zenon$ erl -pa -eval 'eunit:test(wth,[verbose]).' > > Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.7.5 (abort with ^G) > 1> ======================== EUnit ======================== > module 'wth' > wth: test_is_link_with_regular (Test is link with regular)...[0.008 s] ok > wth: test_is_link_with_link (Test Is link with Link)...[0.011 s] ok > [done in 0.049 s] > ======================================================= > All 2 tests passed. [..] Best regards/Mit freundlichen Gr??en -- Muharem Hrnjadovic Public key id : B2BBFCFC Key fingerprint : A5A3 CC67 2B87 D641 103F 5602 219F 6B60 B2BB FCFC -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 900 bytes Desc: OpenPGP digital signature URL: From mk@REDACTED Mon Oct 25 12:23:25 2010 From: mk@REDACTED (Morten Krogh) Date: Mon, 25 Oct 2010 12:23:25 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: <4CC54E1C.6070102@cs.ntua.gr> References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> <9F20C0E1-3B67-4761-B228-61148B0F30F4@rogvall.se> <4CC17A48.7020102@amberbio.com> <4CC54E1C.6070102@cs.ntua.gr> Message-ID: <4CC55A9D.1030708@amberbio.com> Kostis, I followed your suggestion and compiled it to native code with erlc +native zip2.erl, which improves the runtime for both methods. The run times are, for a 100 MB binary, and the map function X -> X + 1 hard coded into the module and into the map method itself (which is not quite realistic): Not native: comprehension : 10 s my own bit syntax function : 14s native: comprehension: 3.7 s my own bit syntax function: 6 s So, yes native is an improvement! Morten. On 10/25/10 11:30 AM, Kostis Sagonas wrote: > Morten Krogh wrote: >> Sorry, in my recent reply to Tony I tested the comprehension on the >> constant ><<"B"">>. >> >> Putting everything in the module and comparing the same mapped >> function ,f , the result is that the comprehension is 30% faster than >> my matching based map function. > > Morten, > > You may also want to try compiling the file to native code and report > whether or how this improves/changes the results. > > Cheers, > Kostis From gianfranco.alongi@REDACTED Mon Oct 25 12:37:17 2010 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Mon, 25 Oct 2010 10:37:17 +0000 (GMT) Subject: [erlang-questions] EUnit setUp/tearDown question In-Reply-To: <847659332.2471288002941543.JavaMail.root@zimbra> Message-ID: <1627147432.2491288003037662.JavaMail.root@zimbra> Hi Muharem, Maybe you already found your way to the User Guide, but I'll post the link in case, http://www.erlang.org/doc/apps/eunit/users_guide.html also you might find something useful on my Erlang Testing Blog. http://erlcode.wordpress.com/ Happy hacking. Cheers /G ----- Original Message ----- From: "Muharem Hrnjadovic" To: "Gianfranco Alongi" Cc: erlang-questions@REDACTED Sent: Monday, October 25, 2010 11:20:46 AM GMT +00:00 GMT Britain, Ireland, Portugal Subject: Re: [erlang-questions] EUnit setUp/tearDown question Hello Gianfranco, thanks a million for the example below! It helped a lot :-) On 10/25/2010 11:08 AM, Gianfranco Alongi wrote: > Hi, > > > -module(wth). > -include_lib("eunit/include/eunit.hrl"). > > is_link_test_() -> > {foreach, > fun() -> ?cmd("mktemp -d temp") -- "\n" end, > fun(T) -> ?cmd("rm -rf " ++ T) end, > [ > fun test_is_link_with_regular/1, > fun test_is_link_with_link/1 > ] > }. > > % ------------------------------------------------------------------------------ > is_link(P) -> > {ok, FIT} = file:read_link_info(P), > element(3,FIT) =:= symlink. > > test_is_link_with_regular(T) -> > {"Test is link with regular", > fun() -> > P = filename:join(T, "a"), > ?cmd("touch "++P), > ?assert(not is_link(P)) > end}. > > test_is_link_with_link(T) -> > {"Test Is link with Link", > fun() -> > P = filename:join(T, "a"), > ?cmd("touch "++P), > ?cmd("ln -s "++P++" "++P++".link"), > ?assert(is_link(P ++ ".link")) > end}. > > > > zenon$ erlc wth.erl > zenon$ erl -pa -eval 'eunit:test(wth,[verbose]).' > > Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.7.5 (abort with ^G) > 1> ======================== EUnit ======================== > module 'wth' > wth: test_is_link_with_regular (Test is link with regular)...[0.008 s] ok > wth: test_is_link_with_link (Test Is link with Link)...[0.011 s] ok > [done in 0.049 s] > ======================================================= > All 2 tests passed. [..] Best regards/Mit freundlichen Gr??en -- Muharem Hrnjadovic Public key id : B2BBFCFC Key fingerprint : A5A3 CC67 2B87 D641 103F 5602 219F 6B60 B2BB FCFC From michael.santos@REDACTED Mon Oct 25 12:40:59 2010 From: michael.santos@REDACTED (Michael Santos) Date: Mon, 25 Oct 2010 06:40:59 -0400 Subject: [erlang-questions] problem of ei_decode_string In-Reply-To: <0522386E-9DB2-423D-9497-4FD5C8907D49@lightpole.net> References: <0522386E-9DB2-423D-9497-4FD5C8907D49@lightpole.net> Message-ID: <20101025104059.GA2430@ecn.lan> On Mon, Oct 25, 2010 at 05:02:39PM +0800, caox wrote: > hi > > I find there will be problem with this api when the length of list to be decoded is larger than 65535. Is this the limit of decoded string, or a bug? A string in the external term format has a 2 byte length header. If the string length can't be represented in 2 bytes, it will be encoded as a list of small integers: 1> term_to_binary(lists:flatten(lists:duplicate(16#ffff-1, "n"))). <<131,107,255,254,110,110,110,110,110,110,110,110,110,110, 110,110,110,110,110,110,110,110,110,110,110,110,110,110, 110,...>> 131 = Erlang magic version number 107 = type is string 255,254 = length is 65534 bytes 110,... = "nnnn..." 2> term_to_binary(lists:flatten(lists:duplicate(16#ffff, "n"))). <<131,108,0,0,255,255,97,110,97,110,97,110,97,110,97,110, 97,110,97,110,97,110,97,110,97,110,97,110,97,...>> 131 = Erlang magic version number 108 = type is list 0,0,255,255 = length is 65535 bytes 97 = type is small integer 110 = "n" ... ei_decode_string() works with both formats. From muharem@REDACTED Mon Oct 25 12:59:50 2010 From: muharem@REDACTED (Muharem Hrnjadovic) Date: Mon, 25 Oct 2010 12:59:50 +0200 Subject: [erlang-questions] EUnit setUp/tearDown question In-Reply-To: <1627147432.2491288003037662.JavaMail.root@zimbra> References: <1627147432.2491288003037662.JavaMail.root@zimbra> Message-ID: <4CC56326.7090609@lbox.cc> On 10/25/2010 12:37 PM, Gianfranco Alongi wrote: > > Hi Muharem, Hello Gianfranco, > Maybe you already found your way to the User Guide, but I'll post the > link in case, > > http://www.erlang.org/doc/apps/eunit/users_guide.html I did pore over this for quite a while but could not figure how to set up the test suite described in the original email. The examples in the User Guide are nice but pretty simple. Adding something like the example you kindly provided to it would be very helpful IMHO. BTW, in the solution you provided "fun test_is_link_with_regular/1" would be an Instantiator, right? > also you might find something useful on my Erlang Testing Blog. > > http://erlcode.wordpress.com/ Oh, nice :-) Will take a look. > ----- Original Message ----- > From: "Muharem Hrnjadovic" > To: "Gianfranco Alongi" > Cc: erlang-questions@REDACTED > Sent: Monday, October 25, 2010 11:20:46 AM GMT +00:00 GMT Britain, Ireland, Portugal > Subject: Re: [erlang-questions] EUnit setUp/tearDown question > > Hello Gianfranco, > > thanks a million for the example below! It helped a lot :-) > > On 10/25/2010 11:08 AM, Gianfranco Alongi wrote: >> Hi, >> >> >> -module(wth). >> -include_lib("eunit/include/eunit.hrl"). >> >> is_link_test_() -> >> {foreach, >> fun() -> ?cmd("mktemp -d temp") -- "\n" end, >> fun(T) -> ?cmd("rm -rf " ++ T) end, >> [ >> fun test_is_link_with_regular/1, >> fun test_is_link_with_link/1 >> ] >> }. >> >> % ------------------------------------------------------------------------------ >> is_link(P) -> >> {ok, FIT} = file:read_link_info(P), >> element(3,FIT) =:= symlink. >> >> test_is_link_with_regular(T) -> >> {"Test is link with regular", >> fun() -> >> P = filename:join(T, "a"), >> ?cmd("touch "++P), >> ?assert(not is_link(P)) >> end}. >> >> test_is_link_with_link(T) -> >> {"Test Is link with Link", >> fun() -> >> P = filename:join(T, "a"), >> ?cmd("touch "++P), >> ?cmd("ln -s "++P++" "++P++".link"), >> ?assert(is_link(P ++ ".link")) >> end}. >> >> >> >> zenon$ erlc wth.erl >> zenon$ erl -pa -eval 'eunit:test(wth,[verbose]).' >> >> Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] >> >> Eshell V5.7.5 (abort with ^G) >> 1> ======================== EUnit ======================== >> module 'wth' >> wth: test_is_link_with_regular (Test is link with regular)...[0.008 s] ok >> wth: test_is_link_with_link (Test Is link with Link)...[0.011 s] ok >> [done in 0.049 s] >> ======================================================= >> All 2 tests passed. Best regards/Mit freundlichen Gr??en -- Muharem Hrnjadovic Public key id : B2BBFCFC Key fingerprint : A5A3 CC67 2B87 D641 103F 5602 219F 6B60 B2BB FCFC -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 900 bytes Desc: OpenPGP digital signature URL: From caox@REDACTED Mon Oct 25 14:09:02 2010 From: caox@REDACTED (caox) Date: Mon, 25 Oct 2010 20:09:02 +0800 Subject: [erlang-questions] problem of ei_decode_string In-Reply-To: <20101025104059.GA2430@ecn.lan> References: <0522386E-9DB2-423D-9497-4FD5C8907D49@lightpole.net> <20101025104059.GA2430@ecn.lan> Message-ID: <05878A2A-20BF-44E7-98F6-172F02ECF6B7@lightpole.net> thanks a lot. I know where the problem is. The return type of ei_get_type() will be ERL_LIST_EXT rather than ERL_STRING_EXT when string length is beyond 65535. So I should take care of this. ? 2010-10-25???6:40? Michael Santos ??? > On Mon, Oct 25, 2010 at 05:02:39PM +0800, caox wrote: >> hi >> >> I find there will be problem with this api when the length of list to be decoded is larger than 65535. Is this the limit of decoded string, or a bug? > > A string in the external term format has a 2 byte length header. If the > string length can't be represented in 2 bytes, it will be encoded as a > list of small integers: > > 1> term_to_binary(lists:flatten(lists:duplicate(16#ffff-1, "n"))). > <<131,107,255,254,110,110,110,110,110,110,110,110,110,110, > 110,110,110,110,110,110,110,110,110,110,110,110,110,110, > 110,...>> > > 131 = Erlang magic version number > 107 = type is string > 255,254 = length is 65534 bytes > 110,... = "nnnn..." > > 2> term_to_binary(lists:flatten(lists:duplicate(16#ffff, "n"))). > <<131,108,0,0,255,255,97,110,97,110,97,110,97,110,97,110, > 97,110,97,110,97,110,97,110,97,110,97,110,97,...>> > > 131 = Erlang magic version number > 108 = type is list > 0,0,255,255 = length is 65535 bytes > 97 = type is small integer > 110 = "n" > ... > > ei_decode_string() works with both formats. > > From gianfranco.alongi@REDACTED Mon Oct 25 14:16:04 2010 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Mon, 25 Oct 2010 12:16:04 +0000 (GMT) Subject: [erlang-questions] EUnit setUp/tearDown question In-Reply-To: <4CC56326.7090609@lbox.cc> Message-ID: <1485979649.2871288008964720.JavaMail.root@zimbra> Hi Muharem, > "BTW, in the solution you provided "fun test_is_link_with_regular/1" > would be an Instantiator, right?" Yes, you are correct. Cheers /G ----- Original Message ----- From: "Muharem Hrnjadovic" To: "Gianfranco Alongi" Cc: erlang-questions@REDACTED Sent: Monday, October 25, 2010 11:59:50 AM GMT +00:00 GMT Britain, Ireland, Portugal Subject: Re: [erlang-questions] EUnit setUp/tearDown question On 10/25/2010 12:37 PM, Gianfranco Alongi wrote: > > Hi Muharem, Hello Gianfranco, > Maybe you already found your way to the User Guide, but I'll post the > link in case, > > http://www.erlang.org/doc/apps/eunit/users_guide.html I did pore over this for quite a while but could not figure how to set up the test suite described in the original email. The examples in the User Guide are nice but pretty simple. Adding something like the example you kindly provided to it would be very helpful IMHO. BTW, in the solution you provided "fun test_is_link_with_regular/1" would be an Instantiator, right? > also you might find something useful on my Erlang Testing Blog. > > http://erlcode.wordpress.com/ Oh, nice :-) Will take a look. > ----- Original Message ----- > From: "Muharem Hrnjadovic" > To: "Gianfranco Alongi" > Cc: erlang-questions@REDACTED > Sent: Monday, October 25, 2010 11:20:46 AM GMT +00:00 GMT Britain, Ireland, Portugal > Subject: Re: [erlang-questions] EUnit setUp/tearDown question > > Hello Gianfranco, > > thanks a million for the example below! It helped a lot :-) > > On 10/25/2010 11:08 AM, Gianfranco Alongi wrote: >> Hi, >> >> >> -module(wth). >> -include_lib("eunit/include/eunit.hrl"). >> >> is_link_test_() -> >> {foreach, >> fun() -> ?cmd("mktemp -d temp") -- "\n" end, >> fun(T) -> ?cmd("rm -rf " ++ T) end, >> [ >> fun test_is_link_with_regular/1, >> fun test_is_link_with_link/1 >> ] >> }. >> >> % ------------------------------------------------------------------------------ >> is_link(P) -> >> {ok, FIT} = file:read_link_info(P), >> element(3,FIT) =:= symlink. >> >> test_is_link_with_regular(T) -> >> {"Test is link with regular", >> fun() -> >> P = filename:join(T, "a"), >> ?cmd("touch "++P), >> ?assert(not is_link(P)) >> end}. >> >> test_is_link_with_link(T) -> >> {"Test Is link with Link", >> fun() -> >> P = filename:join(T, "a"), >> ?cmd("touch "++P), >> ?cmd("ln -s "++P++" "++P++".link"), >> ?assert(is_link(P ++ ".link")) >> end}. >> >> >> >> zenon$ erlc wth.erl >> zenon$ erl -pa -eval 'eunit:test(wth,[verbose]).' >> >> Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] >> >> Eshell V5.7.5 (abort with ^G) >> 1> ======================== EUnit ======================== >> module 'wth' >> wth: test_is_link_with_regular (Test is link with regular)...[0.008 s] ok >> wth: test_is_link_with_link (Test Is link with Link)...[0.011 s] ok >> [done in 0.049 s] >> ======================================================= >> All 2 tests passed. Best regards/Mit freundlichen Gr??en -- Muharem Hrnjadovic Public key id : B2BBFCFC Key fingerprint : A5A3 CC67 2B87 D641 103F 5602 219F 6B60 B2BB FCFC From zvi.avraham@REDACTED Mon Oct 25 14:43:37 2010 From: zvi.avraham@REDACTED (Zvi) Date: Mon, 25 Oct 2010 05:43:37 -0700 (PDT) Subject: How to wake up hibernated process without sending message from external process Message-ID: <02b557c2-bc81-4451-ac42-c88e4dae55c4@l20g2000yqm.googlegroups.com> Hi, I have many hibernated erlang processes, which need to be periodically waken up. One way to do this, is to send wakeup messages from external process. Is there a scalable way for the hibernated process to send messages to itself via some kind of periodic timer? Also note, that "after" doesn't work for hibernated processes. In order to "after" clause to work, the process need to be "unhibernated". Thanks, Zvi ================================ -module(mymodule). -export([start/0, loop/1]). start() -> proc_lib:hibernate(?MODULE, loop, [0]); loop(State) -> receive {do_something, Msg} -> State2 = do_something(State, Msg), proc_lib:hibernate(?MODULE, loop, [State2]); wakeup -> loop(State) after Timeout -> do_cleanup() end. From spawn.think@REDACTED Mon Oct 25 14:51:48 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Mon, 25 Oct 2010 14:51:48 +0200 Subject: [erlang-questions] How to wake up hibernated process without sending message from external process In-Reply-To: <02b557c2-bc81-4451-ac42-c88e4dae55c4@l20g2000yqm.googlegroups.com> References: <02b557c2-bc81-4451-ac42-c88e4dae55c4@l20g2000yqm.googlegroups.com> Message-ID: You could use erlang:send_after(Time, Dest, Msg), e.g : erlang:send_after(1000, self(), wakeup). http://www.erlang.org/doc/man/erlang.html#send_after-3 On Mon, Oct 25, 2010 at 2:43 PM, Zvi wrote: > Hi, > > I have many hibernated erlang processes, which need to be periodically > waken up. > One way to do this, is to send wakeup messages from external process. > Is there a scalable way for the hibernated process to send messages to > itself via some kind of periodic timer? > > Also note, that "after" doesn't work for hibernated processes. In > order to "after" clause to work, the process need to be > "unhibernated". > > Thanks, > Zvi > > ================================ > -module(mymodule). > -export([start/0, loop/1]). > > start() -> > proc_lib:hibernate(?MODULE, loop, [0]); > > loop(State) -> > receive > {do_something, Msg} -> > State2 = do_something(State, Msg), > proc_lib:hibernate(?MODULE, loop, [State2]); > wakeup -> > loop(State) > after Timeout -> > do_cleanup() > end. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From ulf.wiger@REDACTED Mon Oct 25 14:54:08 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 25 Oct 2010 14:54:08 +0200 Subject: [erlang-questions] How to wake up hibernated process without sending message from external process In-Reply-To: <02b557c2-bc81-4451-ac42-c88e4dae55c4@l20g2000yqm.googlegroups.com> References: <02b557c2-bc81-4451-ac42-c88e4dae55c4@l20g2000yqm.googlegroups.com> Message-ID: I imagine that erlang:start_timer/3 ought to work well. It is pretty scalable, in that the runtime system can handle many thousands - most likely much more than that - of timers without noticeable degradation. There are two different timer BIFs: start_timer/3 and send_after/3. The main difference is that start_timer/3 includes the Ref returned by the function in the message itself, e.g. 1> TRef = erlang:start_timer(1000, self(), "hi"). #Ref<0.0.0.344> 2> receive {timeout,TRef,Msg} -> io:fwrite("Timer says ~p.~n", [Msg]) end. Timer says "hi". ok ...whereas send_after/3 simply sends whatever message you specify: 3> erlang:send_after(1000,self(),"ho"). #Ref<0.0.0.352> 4> flush(). Shell got "ho" ok Personally, I always use erlang:start_timer/3. BR, Ulf W On 25 Oct 2010, at 14:43, Zvi wrote: > Hi, > > I have many hibernated erlang processes, which need to be periodically > waken up. > One way to do this, is to send wakeup messages from external process. > Is there a scalable way for the hibernated process to send messages to > itself via some kind of periodic timer? > > Also note, that "after" doesn't work for hibernated processes. In > order to "after" clause to work, the process need to be > "unhibernated". > > Thanks, > Zvi > > ================================ > -module(mymodule). > -export([start/0, loop/1]). > > start() -> > proc_lib:hibernate(?MODULE, loop, [0]); > > loop(State) -> > receive > {do_something, Msg} -> > State2 = do_something(State, Msg), > proc_lib:hibernate(?MODULE, loop, [State2]); > wakeup -> > loop(State) > after Timeout -> > do_cleanup() > end. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From spawn.think@REDACTED Mon Oct 25 14:54:45 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Mon, 25 Oct 2010 14:54:45 +0200 Subject: [erlang-questions] How to wake up hibernated process without sending message from external process In-Reply-To: References: <02b557c2-bc81-4451-ac42-c88e4dae55c4@l20g2000yqm.googlegroups.com> Message-ID: Of course this is not periodical itself, but you can make it periodical by calling it every time before you hibernate Another way would be to use time:send_interval , but it's not a scalable way http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id52228 On Mon, Oct 25, 2010 at 2:51 PM, Ahmed Omar wrote: > You could use > erlang:send_after(Time, Dest, Msg), e.g : erlang:send_after(1000, self(), > wakeup). > http://www.erlang.org/doc/man/erlang.html#send_after-3 > > On Mon, Oct 25, 2010 at 2:43 PM, Zvi wrote: > >> Hi, >> >> I have many hibernated erlang processes, which need to be periodically >> waken up. >> One way to do this, is to send wakeup messages from external process. >> Is there a scalable way for the hibernated process to send messages to >> itself via some kind of periodic timer? >> >> Also note, that "after" doesn't work for hibernated processes. In >> order to "after" clause to work, the process need to be >> "unhibernated". >> >> Thanks, >> Zvi >> >> ================================ >> -module(mymodule). >> -export([start/0, loop/1]). >> >> start() -> >> proc_lib:hibernate(?MODULE, loop, [0]); >> >> loop(State) -> >> receive >> {do_something, Msg} -> >> State2 = do_something(State, Msg), >> proc_lib:hibernate(?MODULE, loop, [State2]); >> wakeup -> >> loop(State) >> after Timeout -> >> do_cleanup() >> end. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From ms@REDACTED Mon Oct 25 14:56:01 2010 From: ms@REDACTED (Martin Scholl) Date: Mon, 25 Oct 2010 14:56:01 +0200 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> Message-ID: <4CC57E61.6030105@diskware.net> On 10/25/2010 06:21 AM, Max Lapshin wrote: > There is other interesting question: are there plans to support > asynchronous disk IO? It is already somehow implemented in FreeBSD and > in progress in Linux. For linux-land I have written a NIF for it (using a thread for collecting the results via io_getevents/2). The performance difference isn't huge compared with blocking I/O -- but unlike blocking I/O you have to worry about alignment (I/O as well as memory) and other issues much more. For Linux-land I'd recommend to stick with Linus' recommendation: http://kerneltrap.org/node/7563 Martin From kenneth.lundin@REDACTED Mon Oct 25 15:27:06 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 25 Oct 2010 15:27:06 +0200 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: <9D7417E2-EB82-4648-861C-F86002B5BFD8@gmail.com> References: <9D7417E2-EB82-4648-861C-F86002B5BFD8@gmail.com> Message-ID: Hi, Have you tried the very same benchmark with just using the 'raw' option when opening the file. You should then get about the same efficiency as you get with NIFs with the difference that you can use asynch threads as well (i.e . the +A option will be effective). Without the +A you will get the same setup as you get with your NIF example. Another benefit with that is that you don't need to write any C-code. If the NIF example still is much faster (which I doubt) please let me know. Without use of the raw option all file operations is done via an extra Erlang process which explains the difference in speed. /Kenneth Erlang/OTP, Ericsson On Sun, Oct 24, 2010 at 3:56 PM, Joel Reymont wrote: > Simply switching to NIFs for file IO seems to have improved CouchDB write performance more than ten-fold. > > Compare the old graph > > http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f5a4 > > to the new graph > > http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f4d7 > > I was under the impression that the Erlang IO subsystem was highly optimized but there seems to be no limit to perfection. > > NIFs are a giant black hole that will subsume Erlang code as performance has to be improved. Start at the lowest level and keep moving up. All that will be left of Erlang in the end is 99.99999% uptime, fault tolerance and supervision... of optimized C code. It's swell and I'm all for it! > > Patch is here: > > http://github.com/wagerlabs/couchdb/commit/23527eb8165f81e63d47b230f3297d3072c88d83 > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From joelr1@REDACTED Mon Oct 25 15:41:22 2010 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 25 Oct 2010 14:41:22 +0100 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <9D7417E2-EB82-4648-861C-F86002B5BFD8@gmail.com> Message-ID: Kenneth, On Oct 25, 2010, at 2:27 PM, Kenneth Lundin wrote: > Have you tried the very same benchmark with just using the 'raw' > option when opening the file. Couch uses the raw option by default. Search for raw in the patch, I left the options untouched and ignore the ones I don't care about in the NIF. http://github.com/wagerlabs/couchdb/commit/23527eb8165f81e63d47b230f3297d3072c88d83 > You should then get about the same efficiency as you get with NIFs Not the case. > If the NIF example still is much faster (which I doubt) please let me know. NIF is still much faster! Note, however, that the difference on Linux is far less pronounced. It's a 2x speedup on Linux with a hard disk vs 10x on Mac OSX with SSD. --- http://twitter.com/wagerlabs From ulf.wiger@REDACTED Mon Oct 25 15:47:58 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 25 Oct 2010 15:47:58 +0200 Subject: BUG? expanding xml content... Message-ID: While debugging the edown component, I started noticing a weird thing about how XMERL expands content. The function is mainly called upon to expand 'simple' content, and does this well, but EDoc passes in a mix of simple content and xmerl records. First, the correct expansion of simple content: 9> xmerl_lib:expand_content([{a,[{b,[{c,[]}]}]}]). [#xmlElement{ name = a,... parents = [],... content = [ #xmlElement{ name = b,... parents = [{a,1}], content = [ #xmlElement{ name = c,... parents = [{b,1},{a,1}],...}] The thing to note is how the 'parents' attribute is faithfully maintained. Now, the same call with manually created records: 1> xmerl_lib:expand_content( [#xmlElement{ name=a, content= [#xmlElement{ name=b, content=[ #xmlElement{name=c}]}]}]). [#xmlElement{ name = a,... parents = [],... content = [ #xmlElement{ name = b,... parents = [],... content = [ #xmlElement{name = c,... parents = [],...}], ...}] The problem is that when it encounters xmerl records, it simply assumes that the parent attribute is correct, but in the case of manually constructed xmerl trees, it most likely isn't. Also, if the structures are transplanted from a parsed document, chances are that the 'parents' attribute is still wrong in the new context. Unless someone comes up with a strong defense in favour of the existing semantics, I will work on a patch that makes expand_content always update the 'parents' attribute. (Since the xmerl test suites are not yet available, that patch would unfortunately not contain updated tests. If someone on the OTP team feels like just fixing this, as it is a pretty simple change, please let me know.) BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From paul.joseph.davis@REDACTED Mon Oct 25 16:25:11 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Mon, 25 Oct 2010 10:25:11 -0400 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> Message-ID: On Sun, Oct 24, 2010 at 4:59 PM, Joel Reymont wrote: > > On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote: > >> I wonder how responsive the system is to other events when running the >> benchmark. > > The benchmark simulates several hundred clients hitting a (mochiweb) web server to read and write couchdb (json) documents. The system seems to stay -highly- responsive, 10x so compared to the same system not using NIFs at all. > > If low write response time is taken as a measure of system responsiveness then the first graph shows that the responsiveness of the system has increased dramatically. The write response here is the take taken to process a web request to write a couch document. > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > Joel, I think the issue here is the way that we sequester file io to a single Erlang process. In your benchmark you'd basically be sacrificing an Erlang scheduler thread to speed up the synchronous writes while the other schedulers are freely available to handle read/write requests. I'm not sure if Mikeal has written a test for Relaximation that runs a similar test that hits multiple databases. I think the way I'd try and show the issue would be to have a large number of clients attempting to write to their own database. Also, a trivial way to prove to your self that the NIF's are indeed called in the same scheduler threads is to create a NIF that has a function that just does "while(1){}". If you call that function in more processes than you have schedulers the VM will halt. HTH, Paul Davis From joelr1@REDACTED Mon Oct 25 16:55:08 2010 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 25 Oct 2010 15:55:08 +0100 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> Message-ID: <699D9254-337D-4EFC-96AD-D927AC2F042E@gmail.com> On Oct 25, 2010, at 3:25 PM, Paul Davis wrote: > In your benchmark you'd basically be > sacrificing an Erlang scheduler thread to speed up the synchronous > writes while the other schedulers are freely available to handle > read/write requests. I sacrifice the schedule to make -both- read and write requests since I replaced all calls to the Erlang file module with calls to my NIFs. Well, except for trivial bits like rename and delete. I think it should be fine to sacrifice a scheduler for small periods of time on a multicore machine. I'll ask Mikeal for multi-database tests. --- http://twitter.com/wagerlabs From paul.joseph.davis@REDACTED Mon Oct 25 17:21:37 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Mon, 25 Oct 2010 11:21:37 -0400 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: <699D9254-337D-4EFC-96AD-D927AC2F042E@gmail.com> References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> <699D9254-337D-4EFC-96AD-D927AC2F042E@gmail.com> Message-ID: On Mon, Oct 25, 2010 at 10:55 AM, Joel Reymont wrote: > > On Oct 25, 2010, at 3:25 PM, Paul Davis wrote: > >> In your benchmark you'd basically be >> sacrificing an Erlang scheduler thread to speed up the synchronous >> writes while the other schedulers are freely available to handle >> read/write requests. > > I sacrifice the schedule to make -both- read and write requests since I replaced all calls to the Erlang file module with calls to my NIFs. Well, except for trivial bits like rename and delete. > > I think it should be fine to sacrifice a scheduler for small periods of time on a multicore machine. > > I'll ask Mikeal for multi-database tests. > > --- > http://twitter.com/wagerlabs > > Joel, What's the final size of your database though? IIRC, the point of those tests wasn't really to test how fast the disk can read/write data, but to look at hour readers and writers interact, ie, do lots of readers make writes disproportionately slower? The point being that if you have enough RAM you could be caching extensive parts of the database in memory which would have the general effect of making most reads be roughly non-blocking. The proper way to test this would be to try and figure out a way to saturate disk io so that a large number of read/write calls are blocking and then try and do something that doesn't touch disk. I can't think of a very good way to set that up other than maybe to create a large number of large databases, compact each of them simultaneously, and then try and run the reader/writer tests or some such. HTH, Paul Davis From jodie.burch@REDACTED Mon Oct 25 17:28:54 2010 From: jodie.burch@REDACTED (Jodie Burch) Date: Mon, 25 Oct 2010 16:28:54 +0100 Subject: Erlang User Conference - 5 Days left at the Early Bird Rate Message-ID: Hi All Just to let you know that, if you haven?t registered already, there are only 5 days left on the Early Bird Discount for the Erlang User Conference in Stockholm on the 16th November which gives you a saving of SEK 250. All the information about the conference is here: http://www.erlang-factory.com/conference/ErlangUserConference2010 This year's conference is packed full of exciting talks, with great speakers such as Simon Thompson, Kenneth Lundin, Tino Breddin and Rusty Klophaus. There will also be a day of Erlang Tutorials on the 15th with limited spaces. We hope to see you all there! Thanks Jodie From joelr1@REDACTED Mon Oct 25 17:51:25 2010 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 25 Oct 2010 16:51:25 +0100 Subject: enif_send from a thread after NIF returns Message-ID: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Is it possible to use enif_send to asynchronously message back to Erlang after the NIF returns? I'm wondering if enif_thread_create could be used to create a thread that continues running and eventually invokes enif_send. A practical example of using the above would be asynchronous io. Thanks for any guidance, Joel --- http://twitter.com/wagerlabs From rapsey@REDACTED Mon Oct 25 17:58:31 2010 From: rapsey@REDACTED (Rapsey) Date: Mon, 25 Oct 2010 17:58:31 +0200 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: yes it is. Env = enif_alloc_env(); enif_send(NULL,&Process, Env,Msg); For every enif_send you need an enif_clear_env after it. enif_clear_env(Env); Sergej On Mon, Oct 25, 2010 at 5:51 PM, Joel Reymont wrote: > Is it possible to use enif_send to asynchronously message back to Erlang > after the NIF returns? > > I'm wondering if enif_thread_create could be used to create a thread that > continues running and eventually invokes enif_send. > > A practical example of using the above would be asynchronous io. > > Thanks for any guidance, Joel > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From paul.joseph.davis@REDACTED Mon Oct 25 18:00:49 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Mon, 25 Oct 2010 12:00:49 -0400 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: On Mon, Oct 25, 2010 at 11:51 AM, Joel Reymont wrote: > Is it possible to use enif_send to asynchronously message back to Erlang after the NIF returns? > > I'm wondering if enif_thread_create could be used to create a thread that continues running and eventually invokes enif_send. > > A practical example of using the above would be asynchronous io. > > ? ? ? ?Thanks for any guidance, Joel > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > Quite possible. There's a simple example of sending a term in my nif-examples [1] repo and a fairly more complex example in Emonk [2]. [1] http://github.com/davisp/nif-examples [2] http://github.com/davisp/emonk From joelr1@REDACTED Mon Oct 25 18:08:48 2010 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 25 Oct 2010 17:08:48 +0100 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: With a combination of NIF, detaching a thread and enif_send, why would you ever need a port driver? Are port drivers obsolete? --- http://twitter.com/wagerlabs From rapsey@REDACTED Mon Oct 25 18:55:40 2010 From: rapsey@REDACTED (Rapsey) Date: Mon, 25 Oct 2010 18:55:40 +0200 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: I see no reason to use them over NIFs. Sergej On Mon, Oct 25, 2010 at 6:08 PM, Joel Reymont wrote: > With a combination of NIF, detaching a thread and enif_send, why would you > ever need a port driver? > > Are port drivers obsolete? > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From paul.joseph.davis@REDACTED Mon Oct 25 19:51:38 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Mon, 25 Oct 2010 13:51:38 -0400 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: On Mon, Oct 25, 2010 at 12:08 PM, Joel Reymont wrote: > With a combination of NIF, detaching a thread and enif_send, why would you ever need a port driver? > > Are port drivers obsolete? > > --- > http://twitter.com/wagerlabs > > Its a different model really. NIF's have started to allow people to re-implement almost anything a port driver can do, as well as things they can't. Though, the question is more about technical debt at this point. For instance, if you had a situation that could be solved as a port using the async thread pool, why would you want to re-implement that as a NIF? NIF's are extremely awesome and its easy to assume they're better for everything but there are definitely cases where a port driver is a better fit. HTH, Paul Davis From joelr1@REDACTED Mon Oct 25 19:56:10 2010 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 25 Oct 2010 18:56:10 +0100 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: <139A5240-0C43-4B11-85AF-D5494E0F0F87@gmail.com> On Oct 25, 2010, at 6:51 PM, Paul Davis wrote: > For instance, if you had a situation that could be solved as a port > using the async thread pool, why would you want to re-implement that > as a NIF? I think it boils down to ease of use and to "worse is better". Yes, NIF requires reimplementing an async thread pool but NIF is so much easier to use! Want to debug your NIF? Attach a debugger and you are there. The layer between the Erlang VM and your C code is thin and transparent! Compare this to trying to debug a port driver, even if you manage to fully understand it's underlying mechanisms. --- http://twitter.com/wagerlabs From dizzyd@REDACTED Mon Oct 25 20:02:13 2010 From: dizzyd@REDACTED (Dave Smith) Date: Mon, 25 Oct 2010 12:02:13 -0600 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: <139A5240-0C43-4B11-85AF-D5494E0F0F87@gmail.com> References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> <139A5240-0C43-4B11-85AF-D5494E0F0F87@gmail.com> Message-ID: On Mon, Oct 25, 2010 at 11:56 AM, Joel Reymont wrote: > > Want to debug your NIF? Attach a debugger and you are there. The layer between the Erlang VM and your C code is thin and transparent! Compare this to trying to debug a port driver, even if you manage to fully understand it's underlying mechanisms. I've never had any problems debugging a port. As you say, attach a debugger, set a breakpoint and wait for it to fire... D. From joelr1@REDACTED Mon Oct 25 20:05:14 2010 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 25 Oct 2010 19:05:14 +0100 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> <139A5240-0C43-4B11-85AF-D5494E0F0F87@gmail.com> Message-ID: <512C963D-F783-4C8B-BBDB-703568F1B126@gmail.com> On Oct 25, 2010, at 7:02 PM, Dave Smith wrote: > I've never had any problems debugging a port. As you say, attach a > debugger, set a breakpoint and wait for it to fire... I'm talking about the way port drivers require you to structure and write your code. With NIF you just decode your arguments and you are there. I, for one, have never managed to fully grasp port driver mechanisms so I welcome our NIF overlords! --- http://twitter.com/wagerlabs From rgowka1@REDACTED Tue Oct 26 02:54:05 2010 From: rgowka1@REDACTED (rgowka1) Date: Mon, 25 Oct 2010 20:54:05 -0400 Subject: Accumulators.. Message-ID: Hi - I am reading Joe's book and on page 75 there is a description of accumulators. I really the idea of accumulators, especially the advantage to avoid having to traverse a list multiple times. My question is - Is there a way to generalize and extend the use of accumulators? Joe divides a list of numbers into Evens and Odds by using two empty lists are accumulators. Is there a way to extend this approach to N buckets. For example, traverse a big list just one - lists:seq(1,math:pow(2,32)) and group them into 100 buckets like all the numbers divisible by 2 in bucket 1, all the numbers divisible by 3 in bucket 2, all the numbers divisible by 5 in bucket 3... etc. odds_and_evens_acc(L) -> odds_and_evens_acc(L, [], []). odds_and_evens_acc([H|T], Odds, Evens) -> case (H rem 2) of 1 -> odds_and_evens_acc(T, [H|Odds], Evens); 0 -> odds_and_evens_acc(T, Odds, [H|Evens]) end; odds_and_evens_acc([], Odds, Evens) -> {Odds, Evens}. thanks. From vances@REDACTED Tue Oct 26 03:10:32 2010 From: vances@REDACTED (Vance Shipley) Date: Mon, 25 Oct 2010 21:10:32 -0400 Subject: [erlang-questions] Accumulators.. In-Reply-To: References: Message-ID: <20101026011032.GP21059@h216-235-12-174.host.egate.net> On Mon, Oct 25, 2010 at 08:54:05PM -0400, rgowka1 wrote: } My question is - Is there a way to generalize and extend the use of } accumulators? Your accumulator can be a more complex Erlang term than a list. foo(Arg) -> foo(Arg, gb_tree:empty()). foo(Arg, Acc) -> ... foo(Next, gb_tree:insert(Key, Val, Acc)). -- -Vance From augusto@REDACTED Tue Oct 26 04:04:09 2010 From: augusto@REDACTED (Augusto Becciu) Date: Mon, 25 Oct 2010 23:04:09 -0300 Subject: [erlang-questions] Erlang-based opensource publish-subscribe servers In-Reply-To: References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> <438091.32912.qm@web35908.mail.mud.yahoo.com> Message-ID: Here's my fork of gen_leader where I fixed all the bugs I encountered while running gproc in a distributed setting: http://github.com/abecciu/gen_leader_revival Augusto On Sun, Oct 24, 2010 at 6:28 PM, Ulf Wiger wrote: > > On 24 Oct 2010, at 21:22, Zvi . wrote: > > > 3. Use Ulf's gproc (looks it uses gen_leader abd requires patching > Erlang?) > > > No, it requires no patches of Erlang. I patched Erlang in the first > prototype, > but only to demonstrate what it could look like if gproc were started as > part of the kernel application, and the functionality integrated into the > standard behaviours. > > It does use gen_leader, though. As far as I can tell, quite a few people > actually use gen_leader in real systems nowadays. There are a few > forks on github to choose from. > > BR, > Ulf W > > > > > I currently working on 1. > > > > Zvi > > > > On Sun, Oct 24, 2010 at 8:41 PM, Andrew Stone > wrote: > > > >> If you want something powerful you can checkout rabbitmq. It may be > >> overkill but > >> it's pretty damn easy to get up and running. > >> > >> http://www.rabbitmq.com/ > >> > >> -Andrew > >> > >> > >> > >> ----- Original Message ---- > >> From: Zvi > >> To: erlang-questions@REDACTED > >> Sent: Sun, October 24, 2010 1:10:46 PM > >> Subject: [erlang-questions] Erlang-based opensource publish-subscribe > >> servers > >> > >> Hi, > >> > >> I looking for the opensource examples of Erlang-based publish- > >> subscribe servers. > >> Any suggestions? > >> > >> Thanks in advance, > >> Zvi > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > > > > > -- > > Zvi Avraham, CTO > > Nivertech Ltd > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From g9414002.pccu.edu.tw@REDACTED Tue Oct 26 04:34:56 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Tue, 26 Oct 2010 10:34:56 +0800 Subject: [erlang-questions] Accumulators.. In-Reply-To: References: Message-ID: On Tue, Oct 26, 2010 at 8:54 AM, rgowka1 wrote: > My question is - Is there a way to generalize and extend the use of > accumulators? Joe divides a list of numbers into Evens and Odds by > using two empty lists are accumulators. Is there a way to extend this > approach to N buckets. For example, traverse a big list just one - > lists:seq(1,math:pow(2,32)) and group them into 100 buckets like all > the numbers divisible by 2 in bucket 1, all the numbers divisible by 3 > in bucket 2, all the numbers divisible by 5 in bucket 3... etc. > > To process 100 buckets, you'll use another predicate, and it means to treat buckets accumulation another problem which uses accumulators to save computing efficiency. Best Regards. --- Y-H. Y. From rzezeski@REDACTED Tue Oct 26 05:30:56 2010 From: rzezeski@REDACTED (Ryan Zezeski) Date: Mon, 25 Oct 2010 23:30:56 -0400 Subject: [erlang-questions] Accumulators.. In-Reply-To: References: Message-ID: On Mon, Oct 25, 2010 at 8:54 PM, rgowka1 wrote: > Hi - > > > My question is - Is there a way to generalize and extend the use of > accumulators? > > odds_and_evens_acc([H|T], Odds, Evens) -> > case (H rem 2) of > 1 -> odds_and_evens_acc(T, [H|Odds], Evens); > 0 -> odds_and_evens_acc(T, Odds, [H|Evens]) > end; > odds_and_evens_acc([], Odds, Evens) -> > {Odds, Evens}. > > thanks. You could also make use of higher-order functions. The key is that your accumulator data structure is an arbitrary term and the accumulator function can be an arbitrary (i.e. user defined) function. -module(odds_and_evens). -export([odds_and_evens/1, odds_and_evens_2/1]). odds_and_evens(L) -> Counts = orddict:from_list([{evens,0}, {odds,0}]), lists:foldl(fun odds_and_evens_acc/2, Counts, L). odds_and_evens_acc(Num, Acc) when Num rem 2 =:= 0 -> orddict:update_counter(evens, 1, Acc); odds_and_evens_acc(_Num, Acc) -> orddict:update_counter(odds, 1, Acc). odds_and_evens_2(L) -> Evens = length(lists:filter(fun is_even/1, L)), [{evens,Evens}, {odds, length(L) - Evens}]. is_even(N) -> N rem 2 =:= 0. 1> c(odds_and_evens). {ok,odds_and_evens} 2> odds_and_evens:odds_and_evens([1,2,3]). [{evens,1},{odds,2}] 3> odds_and_evens:odds_and_evens_2([1,2,3]). [{evens,1},{odds,2}] 4> odds_and_evens:odds_and_evens(lists:seq(1,100)). [{evens,50},{odds,50}] 5> odds_and_evens:odds_and_evens_2(lists:seq(1,100)). [{evens,50},{odds,50}] 6> odds_and_evens:odds_and_evens(lists:seq(1,101)). [{evens,50},{odds,51}] 7> odds_and_evens:odds_and_evens_2(lists:seq(1,101)). [{evens,50},{odds,51}] 8> odds_and_evens:odds_and_evens(lists:duplicate(45,1)). [{evens,0},{odds,45}] 9> odds_and_evens:odds_and_evens_2(lists:duplicate(45,1)). [{evens,0},{odds,45}] I feel like, in general, Erlang style tends towards explicit recursion rather than HOFs but I don't understand why. Maybe because function composition and nesting is not as syntactically clean as it is in other languages? I'm specifically thinking of Haskell and Clojure which I got to know before Erlang. -Ryan From alisdairsullivan@REDACTED Tue Oct 26 05:50:28 2010 From: alisdairsullivan@REDACTED (Alisdair Sullivan) Date: Mon, 25 Oct 2010 20:50:28 -0700 Subject: Abstract Form for -type and -spec In-Reply-To: <08B68DBF-DB94-4292-A907-4A05B12CEE4D@erlang-solutions.com> References: <08B68DBF-DB94-4292-A907-4A05B12CEE4D@erlang-solutions.com> Message-ID: Is the abstract form for type and spec attributes (as returned by, for example, epp:parse_file/3) documented anywhere? What meaning does the list in the third position in -type specs have? Is it ever not an empty list? From muharem@REDACTED Tue Oct 26 07:43:26 2010 From: muharem@REDACTED (Muharem Hrnjadovic) Date: Tue, 26 Oct 2010 07:43:26 +0200 Subject: Passing command line options to erlang programs Message-ID: <4CC66A7E.3000509@lbox.cc> Hello, I am fairly new to erlang, so please forgive me if this is a stupid question :-) I am trying to pass command line options to the following erlang program: {{{ -module(demo). -export([main/1]). main(Args) -> io:format("~p~n", [Args]). }}} When only passing command line *arguments* everything works as expected: {{{ $ erl -noshell -s demo main a b c -s init stop [a,b,c] }}} But when passing command line *options* things break {{{ $ erl -noshell -s demo main a -b c -s init stop [a] $ erl -noshell -s demo main -a b c -s init stop {"init terminating in do_boot",{undef,[{demo,main,[]},{init,start_it,1},{init,start_em,1}]}} Crash dump was written to: erl_crash.dump init terminating in do_boot () }}} Is there another way to sneak options past erl? Or is the code above broken in some other way? If no, then maybe erl needs an option passing/escape mechanism like e.g. gcc's "-Wl" ..? When using an escript file BTW things works as one would expect. Best regards/Mit freundlichen Gr??en -- Muharem Hrnjadovic Public key id : B2BBFCFC Key fingerprint : A5A3 CC67 2B87 D641 103F 5602 219F 6B60 B2BB FCFC -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 900 bytes Desc: OpenPGP digital signature URL: From bengt.kleberg@REDACTED Tue Oct 26 08:52:43 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 26 Oct 2010 08:52:43 +0200 Subject: [erlang-questions] Passing command line options to erlang programs In-Reply-To: <4CC66A7E.3000509@lbox.cc> References: <4CC66A7E.3000509@lbox.cc> Message-ID: <1288075963.8675.2.camel@seasc1137> Greetings, Please look at the erl manual page: http://erlang.org/doc/man/erl.html There you will find that all flags (arguments with - first) are for the Erlang part of the runtime system. However, there are -extra and -- which both may be what you are looking for. bengt On Tue, 2010-10-26 at 07:43 +0200, Muharem Hrnjadovic wrote: > Hello, > > I am fairly new to erlang, so please forgive me if this is a stupid > question :-) > > I am trying to pass command line options to the following erlang program: > {{{ > -module(demo). > -export([main/1]). > > main(Args) -> io:format("~p~n", [Args]). > }}} > > When only passing command line *arguments* everything works as expected: > > {{{ > $ erl -noshell -s demo main a b c -s init stop > [a,b,c] > }}} > > But when passing command line *options* things break > > {{{ > $ erl -noshell -s demo main a -b c -s init stop > [a] > $ erl -noshell -s demo main -a b c -s init stop > {"init terminating in > do_boot",{undef,[{demo,main,[]},{init,start_it,1},{init,start_em,1}]}} > > Crash dump was written to: erl_crash.dump > init terminating in do_boot () > }}} > > Is there another way to sneak options past erl? Or is the code above > broken in some other way? > If no, then maybe erl needs an option passing/escape mechanism like e.g. > gcc's "-Wl" ..? > > When using an escript file BTW things works as one would expect. > > Best regards/Mit freundlichen Gr??en > From bengt.kleberg@REDACTED Tue Oct 26 08:57:22 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 26 Oct 2010 08:57:22 +0200 Subject: article: Erlang mentioned as one of 7 languages on the rise Message-ID: <1288076242.8675.4.camel@seasc1137> Greetings, No news here but Erlang gets a mentioning: http://infoworld.com/d/developer-world/7-programming-languages-the-rise-620?page=0,3 bengt From ulf.wiger@REDACTED Tue Oct 26 08:58:46 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 26 Oct 2010 08:58:46 +0200 Subject: [erlang-questions] Erlang-based opensource publish-subscribe servers In-Reply-To: References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> <438091.32912.qm@web35908.mail.mud.yahoo.com> Message-ID: <0AECA917-6716-43EB-91E8-85F05D60C549@erlang-solutions.com> Hi Augusto, Thanks. Could you explain your approach to handling the dynamic adding/removal of candidates, how you've tested it and why you believe it's safe? Note that I'm not suggesting it isn't safe. :) Just that it touches on the core algorithm, and I think many would feel more comfortable using it if the changes were thoroughly explained. BR, Ulf W On 26 Oct 2010, at 04:04, Augusto Becciu wrote: > Here's my fork of gen_leader where I fixed all the bugs I encountered while running gproc in a distributed setting: > > http://github.com/abecciu/gen_leader_revival > > > Augusto > > On Sun, Oct 24, 2010 at 6:28 PM, Ulf Wiger wrote: > > On 24 Oct 2010, at 21:22, Zvi . wrote: > > > 3. Use Ulf's gproc (looks it uses gen_leader abd requires patching Erlang?) > > > No, it requires no patches of Erlang. I patched Erlang in the first prototype, > but only to demonstrate what it could look like if gproc were started as > part of the kernel application, and the functionality integrated into the > standard behaviours. > > It does use gen_leader, though. As far as I can tell, quite a few people > actually use gen_leader in real systems nowadays. There are a few > forks on github to choose from. > > BR, > Ulf W > > > > > I currently working on 1. > > > > Zvi > > > > On Sun, Oct 24, 2010 at 8:41 PM, Andrew Stone wrote: > > > >> If you want something powerful you can checkout rabbitmq. It may be > >> overkill but > >> it's pretty damn easy to get up and running. > >> > >> http://www.rabbitmq.com/ > >> > >> -Andrew > >> > >> > >> > >> ----- Original Message ---- > >> From: Zvi > >> To: erlang-questions@REDACTED > >> Sent: Sun, October 24, 2010 1:10:46 PM > >> Subject: [erlang-questions] Erlang-based opensource publish-subscribe > >> servers > >> > >> Hi, > >> > >> I looking for the opensource examples of Erlang-based publish- > >> subscribe servers. > >> Any suggestions? > >> > >> Thanks in advance, > >> Zvi > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > > > > > -- > > Zvi Avraham, CTO > > Nivertech Ltd > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From rvg@REDACTED Tue Oct 26 11:09:15 2010 From: rvg@REDACTED (Rudolph van Graan) Date: Tue, 26 Oct 2010 10:09:15 +0100 Subject: Recommended disk block size/record size/page size for DETS and mnesia disc-only tables Message-ID: Hi, What is the recommended disk block size AKA record size AKA page size for Mnesia disc-only tables? Does anyone know how this is determined? I am basically after the information to match the sizes of the DETS slots/pages to the file system and ultimately the disks to reduce IO requirements to the minimum. Thanks. Rudolph van Graan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3822 bytes Desc: not available URL: From alessandro.sivieri@REDACTED Tue Oct 26 12:15:06 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Tue, 26 Oct 2010 12:15:06 +0200 Subject: Mochiweb and SSL Message-ID: Hi all, here again with a MochiWeb question: I'm trying to launch an instance of MochiWeb with SSL support (which should be possible, for what I have found on the Web about it); these are the options that I have added to myserver_sup.erl: WebConfig = [ {ip, Ip}, {docroot, test_deps:local_path(["priv", "www"])}, {port, 443}, {ssl, true}, {ssl_opts, [ {certfile, "/somepath/certs/01.pem"}, {keyfile, "/somepath/private/01.key"}, {verify, verify_peer}, {cacertfile, "/somepath/cacert.pem"}, {fail_if_no_peer_cert, true}, {verify_fun, fun(_) -> false end} ]}], The three paths point to the CA certificate and to the certificate and key created for the local server; Erlang R13 then crashes with the following: http://pastebin.com/Mc8JvCgw While Erlang R14 crashes with the following: http://pastebin.com/4xnwkfU5 What should be the correct options to pass to the mochiweb_http:start() function? -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From joelr1@REDACTED Tue Oct 26 14:08:07 2010 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Oct 2010 13:08:07 +0100 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: On Oct 25, 2010, at 5:00 PM, Paul Davis wrote: > Quite possible. There's a simple example of sending a term in my > nif-examples [1] repo and a fairly more complex example in Emonk [2]. It seems that the calling code must wait for the spawned thread to finish (join). The documentation states that it's impossible to create detached threads and that all threads must be joined. How do you spawn a thread from a NIF then, such that it continues to run after the spawning NIF returns? Thanks, Joel --- http://twitter.com/wagerlabs From joelr1@REDACTED Tue Oct 26 14:25:45 2010 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Oct 2010 13:25:45 +0100 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: Also, what is the point of launching threads from a NIF if it has to wait for launched threads to finish? Does the VM scheduler continue running (not block) while the NIF is waiting in enif_thread_join? Thanks, Joel --- http://twitter.com/wagerlabs From rapsey@REDACTED Tue Oct 26 15:09:08 2010 From: rapsey@REDACTED (Rapsey) Date: Tue, 26 Oct 2010 15:09:08 +0200 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: On Tue, Oct 26, 2010 at 2:08 PM, Joel Reymont wrote: > > On Oct 25, 2010, at 5:00 PM, Paul Davis wrote: > > > Quite possible. There's a simple example of sending a term in my > > nif-examples [1] repo and a fairly more complex example in Emonk [2]. > > It seems that the calling code must wait for the spawned thread to finish > (join). The documentation states that it's impossible to create detached > threads and that all threads must be joined. > > How do you spawn a thread from a NIF then, such that it continues to run > after the spawning NIF returns? > Where in the documentation does it say that? I launch my socket accept thread in load. Sergej From joelr1@REDACTED Tue Oct 26 15:12:48 2010 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Oct 2010 14:12:48 +0100 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: <3C7429BD-E142-4D89-B3B4-14F37D6FFA92@gmail.com> On Oct 26, 2010, at 2:09 PM, Rapsey wrote: > Where in the documentation does it say that? I launch my socket accept thread in load. man erl_driver /erl_drv_thread_create --- The driver creating the thread has the respon- sibility of joining the thread, via erl_drv_thread_join() , before the driver is unloaded. It is not possible to create "detached" threads, i.e., threads that don't need to be joined. Warning: All created threads need to be joined by the driver before it is unloaded. If the driver fails to join all threads created before it is unloaded, the runtime system will most likely crash when the code of the driver is unloaded. --- In the context of a NIF, "driver unloaded" seems to be "NIF exits". The VM consistently crashes when I skip the join after creating threads. --- http://twitter.com/wagerlabs From rapsey@REDACTED Tue Oct 26 15:22:20 2010 From: rapsey@REDACTED (Rapsey) Date: Tue, 26 Oct 2010 15:22:20 +0200 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: <3C7429BD-E142-4D89-B3B4-14F37D6FFA92@gmail.com> References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> <3C7429BD-E142-4D89-B3B4-14F37D6FFA92@gmail.com> Message-ID: On Tue, Oct 26, 2010 at 3:12 PM, Joel Reymont wrote: > > On Oct 26, 2010, at 2:09 PM, Rapsey wrote: > > > Where in the documentation does it say that? I launch my socket accept > thread in load. > > man erl_driver > > /erl_drv_thread_create > > --- > The driver creating the thread has the respon- > sibility of joining the thread, via erl_drv_thread_join() , before > the > driver is unloaded. It is not possible to create "detached" > threads, > i.e., threads that don't need to be joined. > > Warning: > All created threads need to be joined by the driver before it > is > unloaded. If the driver fails to join all threads created before it > is > unloaded, the runtime system will most likely crash when the code > of > the driver is unloaded. > --- > > In the context of a NIF, "driver unloaded" seems to be "NIF exits". The VM > consistently crashes when I skip the join after creating threads. > Does unload/reload/upgrade get called? I have printf in those so that I know what's going on. Sergej From joelr1@REDACTED Tue Oct 26 15:23:54 2010 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Oct 2010 14:23:54 +0100 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: On Oct 26, 2010, at 2:09 PM, Rapsey wrote: > Where in the documentation does it say that? I launch my socket accept thread in load. Do you launch your socket accept thread from a NIF? Where do you join it? --- http://twitter.com/wagerlabs From rapsey@REDACTED Tue Oct 26 15:29:21 2010 From: rapsey@REDACTED (Rapsey) Date: Tue, 26 Oct 2010 15:29:21 +0200 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: On Tue, Oct 26, 2010 at 3:23 PM, Joel Reymont wrote: > > On Oct 26, 2010, at 2:09 PM, Rapsey wrote: > > > Where in the documentation does it say that? I launch my socket accept > thread in load. > > Do you launch your socket accept thread from a NIF? > > Where do you join it? > I launch it in the load function of the nif (that gets called automatically). I don't join it anywhere. Sergej From joelr1@REDACTED Tue Oct 26 15:29:44 2010 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Oct 2010 14:29:44 +0100 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> <3C7429BD-E142-4D89-B3B4-14F37D6FFA92@gmail.com> Message-ID: On Oct 26, 2010, at 2:22 PM, Rapsey wrote: > Does unload/reload/upgrade get called? I have printf in those so that I know what's going on. I'm sure it does, although I'm not using it. Thanks for the tip, though. I now understand that the unload "callback" corresponds to driver unload and that's where threads must be joined. I'm trying to implement asynchronous fsync, though, so I spawn a thread for each fsync. Joining all those threads on unload in a long-running server would require me to keep a -huge- stack of thread ids, one per fsync call. I realize that it's probably a bad idea to launch a thread for each fsync but I'm just experimenting. How do you work around this in your network server? Do you use a thread pool? Thanks, Joel --- http://twitter.com/wagerlabs From joelr1@REDACTED Tue Oct 26 15:32:06 2010 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Oct 2010 14:32:06 +0100 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> Message-ID: <10AED3C7-16EF-4090-894B-FA73CAB13CF7@gmail.com> On Oct 26, 2010, at 2:29 PM, Rapsey wrote: > I launch it in the load function of the nif (that gets called automatically). I don't join it anywhere. What I'm trying to point out is that if you do this from a function and don't join then you'll crash the VM very quickly. I don't understand why and how it works in your case. My guess is that unload is never called and thus no join is called for. Again, the docs state that detached threads are impossible and joining is a must. --- http://twitter.com/wagerlabs From rapsey@REDACTED Tue Oct 26 15:37:46 2010 From: rapsey@REDACTED (Rapsey) Date: Tue, 26 Oct 2010 15:37:46 +0200 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> <3C7429BD-E142-4D89-B3B4-14F37D6FFA92@gmail.com> Message-ID: On Tue, Oct 26, 2010 at 3:29 PM, Joel Reymont wrote: > > On Oct 26, 2010, at 2:22 PM, Rapsey wrote: > > > Does unload/reload/upgrade get called? I have printf in those so that I > know what's going on. > > I'm sure it does, although I'm not using it. Thanks for the tip, though. I > now understand that the unload "callback" corresponds to driver unload and > that's where threads must be joined. > > I'm trying to implement asynchronous fsync, though, so I spawn a thread for > each fsync. Joining all those threads on unload in a long-running server > would require me to keep a -huge- stack of thread ids, one per fsync call. I > realize that it's probably a bad idea to launch a thread for each fsync but > I'm just experimenting. > > How do you work around this in your network server? Do you use a thread > pool? > I don't use fsync anywhere :) I have only one accept thread and sockets are non-blocking. Sergej From rapsey@REDACTED Tue Oct 26 15:41:50 2010 From: rapsey@REDACTED (Rapsey) Date: Tue, 26 Oct 2010 15:41:50 +0200 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: <10AED3C7-16EF-4090-894B-FA73CAB13CF7@gmail.com> References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> <10AED3C7-16EF-4090-894B-FA73CAB13CF7@gmail.com> Message-ID: On Tue, Oct 26, 2010 at 3:32 PM, Joel Reymont wrote: > > On Oct 26, 2010, at 2:29 PM, Rapsey wrote: > > > I launch it in the load function of the nif (that gets called > automatically). I don't join it anywhere. > > What I'm trying to point out is that if you do this from a function and > don't join then you'll crash the VM very quickly. > > I don't understand why and how it works in your case. My guess is that > unload is never called and thus no join is called for. Again, the docs state > that detached threads are impossible and joining is a must. > Oh sorry, just noticed (it's been a while since I've implemented this). I join the thread in unload function. Sergej From joelr1@REDACTED Tue Oct 26 15:42:03 2010 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Oct 2010 14:42:03 +0100 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> <3C7429BD-E142-4D89-B3B4-14F37D6FFA92@gmail.com> Message-ID: <8328BA89-647D-4980-AEC4-796D7DCA3692@gmail.com> On Oct 26, 2010, at 2:37 PM, Rapsey wrote: > I don't use fsync anywhere :) > I have only one accept thread and sockets are non-blocking. Do you see unload being called? Can you trigger it somehow and see if the VM crashes? The manual says you must join before unloading (while unloading?). It's a bug if you are seeing different. --- http://twitter.com/wagerlabs From pablo.platt@REDACTED Tue Oct 26 15:48:29 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 26 Oct 2010 06:48:29 -0700 (PDT) Subject: How does gproc register global names? Message-ID: <8076.10780.qm@web112601.mail.gq1.yahoo.com> Hi I'm using gproc to register local names and properties. I've read that gen_leader is needed to use gproc in the global scope. How does the data get synced across all nodes? Why isn't mnesia being used for that? How do you solve race conditions and failures? Thanks From sverker@REDACTED Tue Oct 26 16:25:48 2010 From: sverker@REDACTED (Sverker Eriksson) Date: Tue, 26 Oct 2010 16:25:48 +0200 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: <3C7429BD-E142-4D89-B3B4-14F37D6FFA92@gmail.com> References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> <3C7429BD-E142-4D89-B3B4-14F37D6FFA92@gmail.com> Message-ID: <4CC6E4EC.1070807@erix.ericsson.se> Joel Reymont wrote: > On Oct 26, 2010, at 2:09 PM, Rapsey wrote: > > >> Where in the documentation does it say that? I launch my socket accept thread in load. >> > > man erl_driver > > /erl_drv_thread_create > > --- > The driver creating the thread has the respon- > sibility of joining the thread, via erl_drv_thread_join() , before the > driver is unloaded. It is not possible to create "detached" threads, > i.e., threads that don't need to be joined. > > Warning: > All created threads need to be joined by the driver before it is > unloaded. If the driver fails to join all threads created before it is > unloaded, the runtime system will most likely crash when the code of > the driver is unloaded. > --- > > In the context of a NIF, "driver unloaded" seems to be "NIF exits". The VM consistently crashes when I skip the join after creating threads. > You do not have to join a created thread before the nif returns. If the VM is crashing when a thread-creating nif returns then you are probably doing something wrong. "driver unloaded" correspond to "NIF library unloaded". That is quite natural. Bad things will happen if a dynamic library is unloaded (driver or nif) while existing threads execute code in that library. A NIF library is only unloaded as the result of a module upgrade where the old module gets purged OR if you replace a NIF library by making repeated calls to erlang:load_nif from the same module. A safe way to make sure that your thread is joined before the library is unloaded is to create a resource object that acts as a handle to your thread. The destructor of the resource can then do join. Resource objects has a protection mechanism that postpone the unloading of a nif library until the last resource object with a destructor in that library is garbage collected. Maybe nif-created threads should have a similar protection mechanism. Have to think about that... /Sverker, Erlang/OTP From max.lapshin@REDACTED Tue Oct 26 16:38:36 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 26 Oct 2010 18:38:36 +0400 Subject: stop async ibrowse request Message-ID: I make several simultaneous requests to one HTTP URL: {ibrowse_req_id, Stream} = ibrowse:send_req(URL, Headers, get, [], [{stream_to,self()},{response_format,binary},{stream_chunk_size,4096}]), Headers contains different Range for I've got data, that I need from one request and want to stop it. How to do it? From joelr1@REDACTED Tue Oct 26 16:40:43 2010 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Oct 2010 15:40:43 +0100 Subject: [erlang-questions] enif_send from a thread after NIF returns In-Reply-To: <4CC6E4EC.1070807@erix.ericsson.se> References: <87FABC08-6CB8-46EE-B60B-ABB46D73A5B3@gmail.com> <3C7429BD-E142-4D89-B3B4-14F37D6FFA92@gmail.com> <4CC6E4EC.1070807@erix.ericsson.se> Message-ID: <0900B38A-E755-4572-A43F-B20584548E28@gmail.com> On Oct 26, 2010, at 3:25 PM, Sverker Eriksson wrote: > A safe way to make sure that your thread is joined before the library is unloaded is to create a resource object that acts as a handle to your thread. The destructor of the resource can then do join. That makes total sense and I know understand why emonk does what it does. Thanks, Joel [1] http://github.com/davisp/emonk --- http://twitter.com/wagerlabs From ulf.wiger@REDACTED Tue Oct 26 17:31:24 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 26 Oct 2010 17:31:24 +0200 Subject: [erlang-questions] How does gproc register global names? In-Reply-To: <8076.10780.qm@web112601.mail.gq1.yahoo.com> References: <8076.10780.qm@web112601.mail.gq1.yahoo.com> Message-ID: It is correct that gproc uses gen_leader. All registration reguests are passed on to the leader process, and it broadcasts the results to the other candidates & workers. Monitoring of registered processes is done by the local gproc instance. When a new leader is elected, it sends its list of registered names to the others; if they happen to have names that the new leader doesn't know about, they will send them to the new leader. This is not terribly unlike how global does things, although it uses a multicall mechanism to handle atomicity. In measurements that I've made, the gproc method is much more efficient. OTOH, global has facilities to handle partitioned networks. Gproc doesn't... yet. Using mnesia would not give better consistency, unless transactions are used - but that would kill performance. And there are no persistency reasons to use it either. Gproc uses ets tables, which are rigged to survive even if the gproc process dies; the restarted gproc scans the table and sets new monitors. BR, Ulf W On 26 Oct 2010, at 15:48, Pablo Platt wrote: > Hi > > I'm using gproc to register local names and properties. > I've read that gen_leader is needed to use gproc in the global scope. > > How does the data get synced across all nodes? > Why isn't mnesia being used for that? > How do you solve race conditions and failures? > > Thanks > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From max.lapshin@REDACTED Tue Oct 26 17:48:44 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 26 Oct 2010 08:48:44 -0700 (PDT) Subject: stop async ibrowse request In-Reply-To: References: Message-ID: It seems, that answer for my question is in fork http://github.com/textendo/ibrowse/commit/aec26ace6d2641dcee31d9a3db154dc57dab52e2 From g@REDACTED Tue Oct 26 18:16:39 2010 From: g@REDACTED (Garrett Smith) Date: Tue, 26 Oct 2010 11:16:39 -0500 Subject: [erlang-questions] Passing command line options to erlang programs In-Reply-To: <1288075963.8675.2.camel@seasc1137> References: <4CC66A7E.3000509@lbox.cc> <1288075963.8675.2.camel@seasc1137> Message-ID: You might also consider using escript, which is better suited for running console based Erlang apps, IMO. Here's a simple tutorial that uses getopt, but also covers escript: http://erlware.blogspot.com/2010/10/console-applications-in-erlang.html On Tue, Oct 26, 2010 at 1:52 AM, Bengt Kleberg wrote: > Greetings, > > Please look at the erl manual page: > http://erlang.org/doc/man/erl.html > > There you will find that all flags (arguments with - first) are for the > Erlang part of the runtime system. > > However, there are -extra and -- which both may be what you are looking > for. > > > bengt > > > On Tue, 2010-10-26 at 07:43 +0200, Muharem Hrnjadovic wrote: >> Hello, >> >> I am fairly new to erlang, so please forgive me if this is a stupid >> question :-) >> >> I am trying to pass command line options to the following erlang program: >> {{{ >> -module(demo). >> -export([main/1]). >> >> main(Args) -> io:format("~p~n", [Args]). >> }}} >> >> When only passing command line *arguments* everything works as expected: >> >> {{{ >> $ erl -noshell -s demo main a b c -s init stop >> [a,b,c] >> }}} >> >> But when passing command line *options* things break >> >> {{{ >> $ erl -noshell -s demo main a -b c -s init stop >> [a] >> $ erl -noshell -s demo main -a b c -s init stop >> {"init terminating in >> do_boot",{undef,[{demo,main,[]},{init,start_it,1},{init,start_em,1}]}} >> >> Crash dump was written to: erl_crash.dump >> init terminating in do_boot () >> }}} >> >> Is there another way to sneak options past erl? Or is the code above >> broken in some other way? >> If no, then maybe erl needs an option passing/escape mechanism like e.g. >> gcc's "-Wl" ..? >> >> When using an escript file BTW things works as one would expect. >> >> Best regards/Mit freundlichen Gr??en >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From pablo.platt@REDACTED Tue Oct 26 18:49:26 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 26 Oct 2010 09:49:26 -0700 (PDT) Subject: [erlang-questions] How does gproc register global names? In-Reply-To: References: <8076.10780.qm@web112601.mail.gq1.yahoo.com> Message-ID: <480449.61818.qm@web112615.mail.gq1.yahoo.com> Why doesn't mnesia have a 'light weight' transaction that works like gproc? When a new leader is elected, how does other nodes knows which names it doesn't know about? I'm using gproc to register gen_server processes. It might be useful to have gproc:cast and gproc:call that use gen_server:cast and gen_server:call in addition to gproc:send. Is it possible to count all registered processes without using a counter? Is it possible to page thorough all registered processes? Thanks ________________________________ From: Ulf Wiger To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Tue, October 26, 2010 5:31:24 PM Subject: Re: [erlang-questions] How does gproc register global names? It is correct that gproc uses gen_leader. All registration reguests are passed on to the leader process, and it broadcasts the results to the other candidates & workers. Monitoring of registered processes is done by the local gproc instance. When a new leader is elected, it sends its list of registered names to the others; if they happen to have names that the new leader doesn't know about, they will send them to the new leader. This is not terribly unlike how global does things, although it uses a multicall mechanism to handle atomicity. In measurements that I've made, the gproc method is much more efficient. OTOH, global has facilities to handle partitioned networks. Gproc doesn't... yet. Using mnesia would not give better consistency, unless transactions are used - but that would kill performance. And there are no persistency reasons to use it either. Gproc uses ets tables, which are rigged to survive even if the gproc process dies; the restarted gproc scans the table and sets new monitors. BR, Ulf W On 26 Oct 2010, at 15:48, Pablo Platt wrote: > Hi > > I'm using gproc to register local names and properties. > I've read that gen_leader is needed to use gproc in the global scope. > > How does the data get synced across all nodes? > Why isn't mnesia being used for that? > How do you solve race conditions and failures? > > Thanks > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From ulf.wiger@REDACTED Tue Oct 26 19:27:45 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 26 Oct 2010 19:27:45 +0200 Subject: [erlang-questions] How does gproc register global names? In-Reply-To: <480449.61818.qm@web112615.mail.gq1.yahoo.com> References: <8076.10780.qm@web112601.mail.gq1.yahoo.com> <480449.61818.qm@web112615.mail.gq1.yahoo.com> Message-ID: On 26 Oct 2010, at 18:49, Pablo Platt wrote: > Why doesn't mnesia have a 'light weight' transaction that works like gproc? Mnesia has a slightly different charter, ensuring consistent persistency. > When a new leader is elected, how does other nodes knows which names it doesn't know about? In the Mod:elected() callback, the newly elected leader can pass data to the others, which they receive in Mod:surrender(). > I'm using gproc to register gen_server processes. > It might be useful to have gproc:cast and gproc:call that use gen_server:cast and gen_server:call in addition to gproc:send. True, but my idea was that OTP integrate gproc and make gen.erl aware of gproc-registered names. :) > Is it possible to count all registered processes without using a counter? It would be possible if the function gproc:pattern/2 were exported, or a function gproc:select_count/2 were added, similar to select/2 et all, which are already there. > Is it possible to page thorough all registered processes? Yes, with either QLC (using gproc:table/1) or using gproc:select/3 (using the Limit argument to fetch a give number of objects at a time. BR, Ulf W > > Thanks > > > > From: Ulf Wiger > To: Pablo Platt > Cc: erlang-questions@REDACTED > Sent: Tue, October 26, 2010 5:31:24 PM > Subject: Re: [erlang-questions] How does gproc register global names? > > > It is correct that gproc uses gen_leader. > All registration reguests are passed on to the leader process, > and it broadcasts the results to the other candidates & workers. > Monitoring of registered processes is done by the local gproc > instance. > > When a new leader is elected, it sends its list of registered names > to the others; if they happen to have names that the new leader > doesn't know about, they will send them to the new leader. > > This is not terribly unlike how global does things, although it uses > a multicall mechanism to handle atomicity. In measurements that > I've made, the gproc method is much more efficient. > > OTOH, global has facilities to handle partitioned networks. Gproc > doesn't... yet. > > Using mnesia would not give better consistency, unless transactions > are used - but that would kill performance. And there are no persistency > reasons to use it either. Gproc uses ets tables, which are rigged to survive > even if the gproc process dies; the restarted gproc scans the table and > sets new monitors. > > BR, > Ulf W > > On 26 Oct 2010, at 15:48, Pablo Platt wrote: > > > Hi > > > > I'm using gproc to register local names and properties. > > I've read that gen_leader is needed to use gproc in the global scope. > > > > How does the data get synced across all nodes? > > Why isn't mnesia being used for that? > > How do you solve race conditions and failures? > > > > Thanks > > > > > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From andrewmmc@REDACTED Tue Oct 26 19:41:08 2010 From: andrewmmc@REDACTED (Andrew M) Date: Tue, 26 Oct 2010 19:41:08 +0200 Subject: R14B compile failure on Ubuntu 9.04 Message-ID: <87C83191517A48FE8F1FD7BC53FD21A9@gmail.com> Hi there,I have been trying to compile R14B, ending in the following failure:=== Entering application hipemake[3]: Entering directory `/usr/src/otp_src_R14B/lib/hipe/misc'erlc -W ?+debug_info +warn_exported_vars +warn_missing_spec +warn_untyped_record -o../ebin hipe_consttab.erlpthread/ethr_event.c:92: Fatal error in wait__(): Function not implemented (38)make[3]: *** [../ebin/hipe_consttab.beam] Abortedmake[3]: Leaving directory `/usr/src/otp_src_R14B/lib/hipe/misc'make[2]: *** [opt] Error 2make[2]: Leaving directory `/usr/src/otp_src_R14B/lib/hipe'make[1]: *** [opt] Error 2make[1]: Leaving directory `/usr/src/otp_src_R14B/lib'make: *** [secondary_bootstrap_build] Error 2Line 92 calls ETHR_FATAL_ERROR__()Am I missing a library, if so, which one?Thanks for your help,Andrew From chandrashekhar.mullaparthi@REDACTED Wed Oct 27 00:38:45 2010 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 26 Oct 2010 23:38:45 +0100 Subject: [erlang-questions] Re: stop async ibrowse request In-Reply-To: References: Message-ID: On 26 October 2010 16:48, Max Lapshin wrote: > It seems, that answer for my question is in fork > > http://github.com/textendo/ibrowse/commit/aec26ace6d2641dcee31d9a3db154dc57dab52e2 I'm incorporating that patch into the next release of ibrowse. I was in touch with that author just a few days ago. cheers Chandru From pablo.platt@REDACTED Wed Oct 27 00:58:19 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 26 Oct 2010 15:58:19 -0700 (PDT) Subject: [erlang-questions] How does gproc register global names? In-Reply-To: References: <8076.10780.qm@web112601.mail.gq1.yahoo.com> <480449.61818.qm@web112615.mail.gq1.yahoo.com> Message-ID: <829521.64112.qm@web112603.mail.gq1.yahoo.com> > True, but my idea was that OTP integrate gproc and make gen.erl > aware of gproc-registered names. :) Why doesn't OTP integrate gproc? It is already used and tested by several people. The code and api are very good. Can someone from the Erlang team pick up the glove? ________________________________ From: Ulf Wiger To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Tue, October 26, 2010 7:27:45 PM Subject: Re: [erlang-questions] How does gproc register global names? On 26 Oct 2010, at 18:49, Pablo Platt wrote: > Why doesn't mnesia have a 'light weight' transaction that works like gproc? Mnesia has a slightly different charter, ensuring consistent persistency. > When a new leader is elected, how does other nodes knows which names it doesn't >know about? In the Mod:elected() callback, the newly elected leader can pass data to the others, which they receive in Mod:surrender(). > I'm using gproc to register gen_server processes. > It might be useful to have gproc:cast and gproc:call that use gen_server:cast >and gen_server:call in addition to gproc:send. True, but my idea was that OTP integrate gproc and make gen.erl aware of gproc-registered names. :) > Is it possible to count all registered processes without using a counter? It would be possible if the function gproc:pattern/2 were exported, or a function gproc:select_count/2 were added, similar to select/2 et all, which are already there. > Is it possible to page thorough all registered processes? Yes, with either QLC (using gproc:table/1) or using gproc:select/3 (using the Limit argument to fetch a give number of objects at a time. BR, Ulf W > > Thanks > > > > From: Ulf Wiger > To: Pablo Platt > Cc: erlang-questions@REDACTED > Sent: Tue, October 26, 2010 5:31:24 PM > Subject: Re: [erlang-questions] How does gproc register global names? > > > It is correct that gproc uses gen_leader. > All registration reguests are passed on to the leader process, > and it broadcasts the results to the other candidates & workers. > Monitoring of registered processes is done by the local gproc > instance. > > When a new leader is elected, it sends its list of registered names > to the others; if they happen to have names that the new leader > doesn't know about, they will send them to the new leader. > > This is not terribly unlike how global does things, although it uses > a multicall mechanism to handle atomicity. In measurements that > I've made, the gproc method is much more efficient. > > OTOH, global has facilities to handle partitioned networks. Gproc > doesn't... yet. > > Using mnesia would not give better consistency, unless transactions > are used - but that would kill performance. And there are no persistency > reasons to use it either. Gproc uses ets tables, which are rigged to survive > even if the gproc process dies; the restarted gproc scans the table and > sets new monitors. > > BR, > Ulf W > > On 26 Oct 2010, at 15:48, Pablo Platt wrote: > > > Hi > > > > I'm using gproc to register local names and properties. > > I've read that gen_leader is needed to use gproc in the global scope. > > > > How does the data get synced across all nodes? > > Why isn't mnesia being used for that? > > How do you solve race conditions and failures? > > > > Thanks > > > > > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From pablo.platt@REDACTED Wed Oct 27 00:54:58 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 26 Oct 2010 15:54:58 -0700 (PDT) Subject: [erlang-questions] How does gproc register global names? In-Reply-To: References: <8076.10780.qm@web112601.mail.gq1.yahoo.com> <480449.61818.qm@web112615.mail.gq1.yahoo.com> Message-ID: <361924.86594.qm@web112615.mail.gq1.yahoo.com> Thank you for the answer. ________________________________ From: Ulf Wiger To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Tue, October 26, 2010 7:27:45 PM Subject: Re: [erlang-questions] How does gproc register global names? On 26 Oct 2010, at 18:49, Pablo Platt wrote: > Why doesn't mnesia have a 'light weight' transaction that works like gproc? Mnesia has a slightly different charter, ensuring consistent persistency. > When a new leader is elected, how does other nodes knows which names it doesn't >know about? In the Mod:elected() callback, the newly elected leader can pass data to the others, which they receive in Mod:surrender(). > I'm using gproc to register gen_server processes. > It might be useful to have gproc:cast and gproc:call that use gen_server:cast >and gen_server:call in addition to gproc:send. True, but my idea was that OTP integrate gproc and make gen.erl aware of gproc-registered names. :) > Is it possible to count all registered processes without using a counter? It would be possible if the function gproc:pattern/2 were exported, or a function gproc:select_count/2 were added, similar to select/2 et all, which are already there. > Is it possible to page thorough all registered processes? Yes, with either QLC (using gproc:table/1) or using gproc:select/3 (using the Limit argument to fetch a give number of objects at a time. BR, Ulf W > > Thanks > > > > From: Ulf Wiger > To: Pablo Platt > Cc: erlang-questions@REDACTED > Sent: Tue, October 26, 2010 5:31:24 PM > Subject: Re: [erlang-questions] How does gproc register global names? > > > It is correct that gproc uses gen_leader. > All registration reguests are passed on to the leader process, > and it broadcasts the results to the other candidates & workers. > Monitoring of registered processes is done by the local gproc > instance. > > When a new leader is elected, it sends its list of registered names > to the others; if they happen to have names that the new leader > doesn't know about, they will send them to the new leader. > > This is not terribly unlike how global does things, although it uses > a multicall mechanism to handle atomicity. In measurements that > I've made, the gproc method is much more efficient. > > OTOH, global has facilities to handle partitioned networks. Gproc > doesn't... yet. > > Using mnesia would not give better consistency, unless transactions > are used - but that would kill performance. And there are no persistency > reasons to use it either. Gproc uses ets tables, which are rigged to survive > even if the gproc process dies; the restarted gproc scans the table and > sets new monitors. > > BR, > Ulf W > > On 26 Oct 2010, at 15:48, Pablo Platt wrote: > > > Hi > > > > I'm using gproc to register local names and properties. > > I've read that gen_leader is needed to use gproc in the global scope. > > > > How does the data get synced across all nodes? > > Why isn't mnesia being used for that? > > How do you solve race conditions and failures? > > > > Thanks > > > > > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From steven.charles.davis@REDACTED Wed Oct 27 01:34:33 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 26 Oct 2010 16:34:33 -0700 (PDT) Subject: Mochiweb and SSL In-Reply-To: References: Message-ID: <2558e383-4d2c-489a-aa6b-000ce993a367@e14g2000yqe.googlegroups.com> What happens when you change the port to 8443 (i.e. above 1024)? >From the second example it looks like you don't have fdserv available that would allow you to run on a well-known port. From augusto@REDACTED Wed Oct 27 03:02:53 2010 From: augusto@REDACTED (Augusto Becciu) Date: Tue, 26 Oct 2010 22:02:53 -0300 Subject: [erlang-questions] Erlang-based opensource publish-subscribe servers In-Reply-To: <0AECA917-6716-43EB-91E8-85F05D60C549@erlang-solutions.com> References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> <438091.32912.qm@web35908.mail.mud.yahoo.com> <0AECA917-6716-43EB-91E8-85F05D60C549@erlang-solutions.com> Message-ID: Hi Ulf, On Tue, Oct 26, 2010 at 3:58 AM, Ulf Wiger wrote: > > Hi Augusto, > Thanks. Could you explain your approach to handling the dynamic > adding/removal of candidates, how you've tested it and why you > believe it's safe? The algorithm to add nodes dynamically, along with some failure scenarios, is explained here: http://gist.github.com/499825 I haven't worked on dynamic removal, but you can just shut down nodes and they will be put in the 'down' list. I haven't done rigorous analysis or testing on it. Just tested the cases I documented in the gist linked above and some other similar ones. I started working on a test suite using eunit and mcErlang but ran out of time in the last months. I'm still interested in developing it though, and hope to get more free time to finish it by December. I very much appreciate any feedback you can provide on this. :) > > Note that I'm not suggesting it isn't safe. :) > Just that it touches on the core algorithm, and I think many would feel > more comfortable using it if the changes were thoroughly explained. It doesn't modify the core algorithm. It works like an extension to it. The code is only used if you add the option {seed, Node} to start the node. I encourage people to use my fork not because of the possibility to add nodes dynamically, but because it contains many fixes (major and minor), cleaned up code, and proper project organization with rebarized build system. Of particular interest to gproc_dist users are these two fixes: 1) http://github.com/abecciu/gen_leader_revival/commit/686da14f9a349ce4891ca910ba997e30b46b3860 Without this fix, when a leader goes down and other node takes over leadership, all the elements registered by the old leader won't be removed. 2) http://github.com/abecciu/gen_leader_revival/commit/64903e4d4a765ce8e9ca765f0a1a8ce59f833627 Without this fix, any request made to gproc_dist during the election process will make it crash. Cheers, Augusto > BR, > Ulf W > On 26 Oct 2010, at 04:04, Augusto Becciu wrote: > > Here's my fork of gen_leader where I fixed all the bugs I encountered while running gproc in a distributed setting: > http://github.com/abecciu/gen_leader_revival > > Augusto > > On Sun, Oct 24, 2010 at 6:28 PM, Ulf Wiger wrote: >> >> On 24 Oct 2010, at 21:22, Zvi . wrote: >> >> > 3. Use Ulf's gproc (looks it uses gen_leader abd requires patching Erlang?) >> >> >> No, it requires no patches of Erlang. I patched Erlang in the first prototype, >> but only to demonstrate what it could look like if gproc were started as >> part of the kernel application, and the functionality integrated into the >> standard behaviours. >> >> It does use gen_leader, though. As ?far as I can tell, quite a few people >> actually use gen_leader in real systems nowadays. There are a few >> forks on github to choose from. >> >> BR, >> Ulf W >> >> > >> > I currently working on 1. >> > >> > Zvi >> > >> > On Sun, Oct 24, 2010 at 8:41 PM, Andrew Stone wrote: >> > >> >> If you want something powerful you can checkout rabbitmq. It may be >> >> overkill but >> >> it's pretty damn easy to get up and running. >> >> >> >> http://www.rabbitmq.com/ >> >> >> >> -Andrew >> >> >> >> >> >> >> >> ----- Original Message ---- >> >> From: Zvi >> >> To: erlang-questions@REDACTED >> >> Sent: Sun, October 24, 2010 1:10:46 PM >> >> Subject: [erlang-questions] Erlang-based opensource publish-subscribe >> >> servers >> >> >> >> Hi, >> >> >> >> I looking for the opensource examples of Erlang-based publish- >> >> subscribe servers. >> >> Any suggestions? >> >> >> >> Thanks in advance, >> >> Zvi >> >> >> >> ________________________________________________________________ >> >> erlang-questions (at) erlang.org mailing list. >> >> See http://www.erlang.org/faq.html >> >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> > >> > >> > >> > -- >> > Zvi Avraham, CTO >> > Nivertech Ltd >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > Ulf Wiger,?CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > From surferjeff@REDACTED Wed Oct 27 04:56:47 2010 From: surferjeff@REDACTED (Jeffrey Rennie) Date: Tue, 26 Oct 2010 19:56:47 -0700 Subject: [erlang-questions] Ridiculuous newbie mnesia question. In-Reply-To: <4CC53835.7040000@erlang-solutions.com> References: <00d701cb7405$bff89930$3fe9cb90$@com> <1AD7068E-BAE8-44BB-A77C-77BA32E08F49@erlang-solutions.com> <4CC53835.7040000@erlang-solutions.com> Message-ID: <032101cb7582$9c740ac0$d55c2040$@com> Thank you Mazen. Where can I find a book or document that contains such detailed explanations? I'm having a hard time grasping the big picture for mnesia. I've read all the docs that are part of the erlang distribution. > -----Original Message----- > From: Mazen Harake [mailto:mazen.harake@REDACTED] > Sent: Monday, October 25, 2010 12:57 AM > To: erlang-questions@REDACTED > Cc: Jeffrey Rennie > Subject: Re: [erlang-questions] Ridiculuous newbie mnesia question. > > Although all these points are valid I would like to add a third point > which should solve this particular situation; > > 3. The line '[] = [backgammon_games,game_record]' means that no node, > which is a copy-holder of these tables, is connected. This means that > you should first start one of those nodes (or a "fake" of one with the > same nodename) and then start mnesia and do > mnesia:change_config(extra_db_nodes, N) (as Ulf mentioned) where N is a > copy-holder of those tables. When this is done the node name will show > up like this: '[{foo@REDACTED, ram_copies}] = > [backgammon_games,game_record]' and then it is safe to remove the > table. > > The important part here is the "copy-holder" concept. An empty list > means "no one is available", but the local node still knows about it so > it "exists" but it doesn't exist in a place where you can change it (it > is in a different dimension >:P). That is the reason why you can't > create it nor delete it. > > /Mazen > > On 25/10/2010 08:41, Ulf Wiger wrote: > > Hi Jeffrey, > > > > Looking at the mnesia:info() output, there are a couple of things > > to note. > > > > 1. 'running db nodes = [emacs@REDACTED]' means that while a@REDACTED > > and b@REDACTED might be running at the erlang level, perhaps even > > with mnesia runnin, they are not connected to emacs@REDACTED > > > > 2. The line '"c:/.../Mnesia.emacs@REDACTED" is NOT used' means that > > you have not created a persistent schema. Normally, the > persistent > > schema indicates which nodes are part of the mnesia cluster. Any > > other nodes that want to join should specify -mnesia > extra_db_nodes Ns, > > where Ns includes at least some of the nodes listed in the > persistent schema. > > > > To create a persistent schema, you run this before starting mnesia: > > > > mnesia:create_schema(Nodes). > > > > ...with the nodes in Nodes all up and running. You only need to call > it > > from one node; it will create a schema on each node automatically. > > > > BR, > > Ulf W > > > > On 25 Oct 2010, at 07:30, Jeffrey Rennie wrote: > > > >> I launched a new erl node with a proper -sname, and I'm still > experiencing > >> the same problem. Yes, nodes a@REDACTED and b@REDACTED are up and > running. > >> > >> (emacs@REDACTED)17> backgammon:init(). > >> {atomic,ok} > >> (emacs@REDACTED)18> backgammon:insert_game(1, "Jeff"). > >> {aborted,{no_exists,game_record}} > >> (emacs@REDACTED)19> backgammon:init(). > >> {aborted,{already_exists,game_record}} > >> (emacs@REDACTED)20> mnesia:info(). > >> ---> Processes holding locks<--- > >> ---> Processes waiting for locks<--- > >> ---> Participant transactions<--- > >> ---> Coordinator transactions<--- > >> ---> Uncertain transactions<--- > >> ---> Active tables<--- > >> schema : with 3 records occupying 656 words of > mem > >> ===> System info in version "4.4.13", debug level = none<=== > >> opt_disc. Directory > >> "c:/Users/jeff/code/code/scratch/erlang/Mnesia.emacs@REDACTED" is NOT > used. > >> use fallback at restart = false > >> running db nodes = [emacs@REDACTED] > >> stopped db nodes = [] > >> master node tables = [] > >> remote = [backgammon_games,game_record] > >> ram_copies = [schema] > >> disc_copies = [] > >> disc_only_copies = [] > >> [] = [backgammon_games,game_record] > >> [{emacs@REDACTED,ram_copies}] = [schema] > >> 4 transactions committed, 3 aborted, 0 restarted, 0 logged to disc > >> 0 held locks, 0 in queue; 0 local transactions, 0 remote > >> 0 transactions waits for other nodes: [] > >> ok > >> (emacs@REDACTED)21> mnesia:delete_table(backgammon_games). > >> {aborted,{no_exists,backgammon_games}} > >> (emacs@REDACTED)22> mnesia:delete_table(game_record). > >> {aborted,{no_exists,game_record}} > >> (emacs@REDACTED)23> backgammon:init(). > >> {aborted,{already_exists,game_record}} > >> (emacs@REDACTED)24> > >> > >> > >>> -----Original Message----- > >>> From: Ulf Wiger [mailto:ulf.wiger@REDACTED] > >>> Sent: Sunday, October 24, 2010 3:02 PM > >>> To: Jeffrey Rennie > >>> Cc: erlang-questions@REDACTED > >>> Subject: Re: [erlang-questions] Ridiculuous newbie mnesia question. > >>> > >>> > >>> Hi Jeffrey, > >>> > >>> The problem is that you have specified that the backgammon_games > >>> table should have copies on a@REDACTED and b@REDACTED, but your > >>> Erlang shell has the node name nonode@REDACTED (i.e. Erlang was > >>> started without distribution). > >>> > >>> One might argue whether it wouldn't have been better if mnesia had > >>> refused to create the table in the first place. As it is now, it > >>> believes > >>> it exists, because there is an entry in the schema, but it refuses > to > >>> delete it because it has no active replicas. But since that's not > what > >>> it > >>> complains about, I can understand your confusion. :) > >>> > >>> BR, > >>> Ulf W > >>> > >>> > >>> On 24 Oct 2010, at 23:37, Jeffrey Rennie wrote: > >>> > >>>> I seem to be stuck in a state where I can't create a table because > it > >>>> exists, but I can't delete the table because it doesn't exist! > >>>> > >>>> I searched the mnesia documentation, and I couldn't find a > definition > >>>> of the "Tab" arg passed to many mnesia functions. > >>>> > >>>> Here's what I tried: > >>>> > >>>> Eshell V5.7.5 (abort with ^G) > >>>> 2> c(backgammon). > >>>> {ok,backgammon} > >>>> 5> c(backgammon). > >>>> {ok,backgammon} > >>>> 7> mnesia:start(). > >>>> ok > >>>> 8> backgammon:init(). > >>>> {atomic,ok} > >>>> 11> c(backgammon). > >>>> {ok,backgammon} > >>>> 12> backgammon:dumpfields(). > >>>> [player_turn,dice,board] > >>>> 13> c(backgammon). > >>>> {ok,backgammon} > >>>> 14> backgammon:init(). > >>>> {aborted,{already_exists,backgammon_games}} > >>>> 15> mnesia:info(). > >>>> ---> Processes holding locks<--- > >>>> ---> Processes waiting for locks<--- > >>>> ---> Participant transactions<--- > >>>> ---> Coordinator transactions<--- > >>>> ---> Uncertain transactions<--- > >>>> ---> Active tables<--- > >>>> schema : with 2 records occupying 546 words of > >>> mem > >>>> ===> System info in version "4.4.13", debug level = none<=== > >>>> opt_disc. Directory > >>>> "c:/Users/jeff/code/code/scratch/erlang/Mnesia.nonode@REDACTED" is > NOT > >>>> used. > >>>> use fallback at restart = false > >>>> running db nodes = [nonode@REDACTED] > >>>> stopped db nodes = [] > >>>> master node tables = [] > >>>> remote = [backgammon_games] > >>>> ram_copies = [schema] > >>>> disc_copies = [] > >>>> disc_only_copies = [] > >>>> [] = [backgammon_games] > >>>> [{nonode@REDACTED,ram_copies}] = [schema] > >>>> 3 transactions committed, 1 aborted, 0 restarted, 0 logged to disc > >>>> 0 held locks, 0 in queue; 0 local transactions, 0 remote > >>>> 0 transactions waits for other nodes: [] > >>>> ok > >>>> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > >>>> %% BEGIN CONFUSING BIT > >>>> 17> mnesia:delete_table(backgammon_games). > >>>> {aborted,{no_exists,backgammon_games}} > >>>> 18> backgammon:init(). > >>>> {aborted,{already_exists,backgammon_games}} > >>>> 19> > >>>> %% END CONFUSING BIT > >>>> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > >>>> > >>>> > >>>> Here's my backgammon.erl: > >>>> -module(backgammon). > >>>> > >>>> -compile(export_all). > >>>> > >>>> -include("backgammon.hrl"). > >>>> > >>>> create_tables(MnesiaNodes) -> > >>>> mnesia:create_table(backgammon_games, > >>>> [{attributes, record_info(fields, game_record)}, > >>>> {ram_copies, MnesiaNodes}]). > >>>> > >>>> init() -> > >>>> create_tables([a@REDACTED, b@REDACTED]). > >>>> > >>>> dumpfields() -> > >>>> record_info(fields, game_state). > >>>> > >>>> ________________________________________________________________ > >>>> erlang-questions (at) erlang.org mailing list. > >>>> See http://www.erlang.org/faq.html > >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>>> > >>> Ulf Wiger, CTO, Erlang Solutions, Ltd. > >>> http://erlang-solutions.com > >>> > >> > >> > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > > http://erlang-solutions.com > > > > > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From surferjeff@REDACTED Wed Oct 27 05:37:38 2010 From: surferjeff@REDACTED (Jeffrey Rennie) Date: Tue, 26 Oct 2010 20:37:38 -0700 Subject: [erlang-questions] Ridiculuous newbie mnesia question. In-Reply-To: <1AD7068E-BAE8-44BB-A77C-77BA32E08F49@erlang-solutions.com> References: <00d701cb7405$bff89930$3fe9cb90$@com> <1AD7068E-BAE8-44BB-A77C-77BA32E08F49@erlang-solutions.com> Message-ID: <032401cb7588$4f5f5180$ee1df480$@com> Thank you for your patience. I still haven't escaped from this wet paper bag: > Looking at the mnesia:info() output, there are a couple of things > to note. > > 1. 'running db nodes = [emacs@REDACTED]' means that while a@REDACTED > and b@REDACTED might be running at the erlang level, perhaps even > with mnesia runnin, they are not connected to emacs@REDACTED > Why aren't they connected? How do I connect them? I can ping them (see below.) > 2. The line '"c:/.../Mnesia.emacs@REDACTED" is NOT used' means that > you have not created a persistent schema. Normally, the persistent > schema indicates which nodes are part of the mnesia cluster. Any > other nodes that want to join should specify -mnesia extra_db_nodes > Ns, > where Ns includes at least some of the nodes listed in the > persistent schema. > > To create a persistent schema, you run this before starting mnesia: > > mnesia:create_schema(Nodes). > > ...with the nodes in Nodes all up and running. You only need to call it > from one node; it will create a schema on each node automatically. I tried that. I didn't specify the other nodes because the documentation for mnesia:create_schema says to specify disk copy nodes only, and I want ram copy nodes. But mnesia:info() still shows me I'm not connected to anything: -module(backgammon). -compile(export_all). -include("backgammon.hrl"). create_tables(MnesiaNodes) -> mnesia:create_table(game_record, [{attributes, record_info(fields, game_record)}, {ram_copies, MnesiaNodes}]). init() -> mnesia:create_schema([]), mnesia:start(), create_tables([a@REDACTED, b@REDACTED]). (emacs@REDACTED)4> backgammon:init(). {atomic,ok} (emacs@REDACTED)5> mnesia:info(). ---> Processes holding locks <--- ---> Processes waiting for locks <--- ---> Participant transactions <--- ---> Coordinator transactions <--- ---> Uncertain transactions <--- ---> Active tables <--- schema : with 2 records occupying 546 words of mem ===> System info in version "4.4.13", debug level = none <=== opt_disc. Directory "c:/Users/jeff/code/code/scratch/erlang/Mnesia.emacs@REDACTED" is used. use fallback at restart = false running db nodes = [emacs@REDACTED] stopped db nodes = [] master node tables = [] remote = [game_record] ram_copies = [] disc_copies = [schema] disc_only_copies = [] [] = [game_record] [{emacs@REDACTED,disc_copies}] = [schema] 3 transactions committed, 0 aborted, 0 restarted, 1 logged to disc 0 held locks, 0 in queue; 0 local transactions, 0 remote 0 transactions waits for other nodes: [] ok (emacs@REDACTED)6> net_adm:ping(a@REDACTED). pong (emacs@REDACTED)7> net_adm:ping(b@REDACTED). pong (emacs@REDACTED)8> From max.lapshin@REDACTED Wed Oct 27 07:02:58 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 27 Oct 2010 09:02:58 +0400 Subject: [erlang-questions] Re: stop async ibrowse request In-Reply-To: References: Message-ID: On Wed, Oct 27, 2010 at 2:38 AM, Chandru wrote: > I'm incorporating that patch into the next release of ibrowse. I was > in touch with that author just a few days ago. > Great. That patch has one problem: if I start ibrowse as {stream_to, self()}, stream_close doesn't work. Works only is {stream_to, {self(), once}}. Also there are maaany forks about rebar. It would be good to elliminate them, by merging =) Will you accept patch with debian installation package? From amit.murthy@REDACTED Wed Oct 27 07:25:56 2010 From: amit.murthy@REDACTED (amitm) Date: Tue, 26 Oct 2010 22:25:56 -0700 (PDT) Subject: confusing returns from httpc:request/4 In-Reply-To: References: <8ecc3ba2-3f82-4d55-a46f-063f41012ce4@p37g2000pra.googlegroups.com> <3e9cd45e-4e88-42cb-8a7f-b2666b069d2c@w9g2000prc.googlegroups.com> Message-ID: <864cbd1d-8b1f-41e3-a670-740b9506ee03@g25g2000yqn.googlegroups.com> Sorry, I missed out on stating that to make the problem go away, I also set the HTTP version as "HTTP/1.0" So you may also want to look at the code for handling persistent connections in general, especially when the server only supports HTTP 1.0 . Amit On Oct 21, 8:54?pm, Ingela Andin wrote: > Hi! > > Yes thank you for the information. I think I have located the bug. It seems > that to avoid crash reports the return value when receiving a tcp_closed > was changed and this altered the terminate clause that is run, ?alas breaking > the automatic retries that should be done if the server terminates a connection > before serving all pipelined requests. This will be fixed in an > upcoming release. > > Regards Ingela Erlang/OTP-team, ?Ericsson AB > > 2010/10/21amitm: > > > > > > > > >> Humm ... what settings did you have when you had the problem? > > > The function making the call in the situation when the problem > > manifests itself is: > > > mkGetReq2(GetUrl) -> > > ? ?GetRequest = {GetUrl, []}, > > ? ?GetHttpOptions = [{autoredirect, true}], > > ? ?Options = [ {sync,true}, {headers_as_is,true}, {body_format, > > binary} ], > > ? ?httpc:request(get, GetRequest, GetHttpOptions, Options). > > > No other options were set, which means they run with their default > > values. > > >> Did you set the pipeline timout ? > > > No pipeline timeout was set. > > >> What webserver did you connect to (apache, yaws, ...)? > > > The webserver is one used by Amazon internally on EC2. Curl in verbose > > mode, as executed by the command > > "curl -vhttp://169.254.169.254/2009-04-04/meta-data/instance-id" > > prints out > > > ?* About to connect() to 169.254.169.254 port 80 (#0) > > ?* ? Trying 169.254.169.254... connected > > ?* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0) > > ?> GET /2009-04-04/meta-data/instance-id HTTP/1.1 > > ?> User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/ > > 0.9.8k zlib/1.2.3.3 libidn/1.15 > > ?> Host: 169.254.169.254 > > ?> Accept: */* > > ?> > > ?* HTTP 1.0, assume close after body > > ?< HTTP/1.0 200 OK > > ?< Content-Type: text/plain > > ?< Accept-Ranges: bytes > > ?< ETag: "-1124024050" > > ?< Last-Modified: Tue, 19 Oct 2010 13:58:54 GMT > > ?< Content-Length: 10 > > ?< Connection: close > > ?< Date: Thu, 21 Oct 2010 13:48:19 GMT > > ?< Server: EC2ws > > ?< > > ?* Closing connection #0 > > >> What version of inets? > > > erl -v prints > > "Erlang R14B (erts-5.8.1) [source] [rq:1] [async-threads:0] [kernel- > > poll:false]" > > > Hope it helps, > > > ?Amit > > > On Oct 21, 6:32?pm, Ingela Andin wrote: > >> Hi! > > >> Humm ... what settings did you have when you had the problem? > >> Did you set the pipeline timout ? What webserver did you connect to > >> (apache, yaws, ...)? What > >> version of inets? If there is some bug in the pipeling or > >> persistenconnection code > >> of course we would like to know so that we can fix it. > > >> Regards Ingela Erlang/OTP team - Ericsson AB > > >> 2010/10/20amitm: > > >> > FWIW, I had a similar problem. It went away when I disabled pipelining > >> > completely by executing > > >> > ? ?ok = httpc:set_options([{max_keep_alive_length, 0}, > >> > {max_pipeline_length, 0}, {max_sessions, 0} ]), > > >> > before my http calls. I used to usually see it on the 4th consecutive > >> > call to the same host when pipelining was enabled. The URL in question > >> > in my case was an Amazon EC2 url. > > >> > The set of URLs in my case were > > >> > "http://169.254.169.254/2009-04-04/meta-data/instance-id", > >> > "http://169.254.169.254/2009-04-04/meta-data/local-hostname", > >> > "http://169.254.169.254/2009-04-04/meta-data/local-ipv4", > >> > "http://169.254.169.254/2009-04-04/meta-data/public-hostname", > >> > "http://169.254.169.254/2009-04-04/meta-data/public-ipv4", > >> > "http://169.254.169.254/2009-04-04/meta-data/ami-id", > > >> > While executing this set, with pipelining enabled, it used to always > >> > bomb on the 4th URL, i.e. for 'public-hostname' in 25% of the cases > > >> > with pipelining disabled, the problem disappears. > > >> > ?Amit > > >> > On Oct 13, 2:58?pm, Ingela Andin wrote: > >> >> 2010/10/13 Chandru : > > >> >> > On 13 October 2010 09:53, Ingela Andin wrote: > >> >> >> Hi! > > >> >> >> The only reason that the httpc-client will return the value {error, > >> >> >> socket_closed_remotely} > >> >> >> is that the connection is prematurely ?closed by the server. > >> >> >> Unfortunately ?that something > >> >> >> is sent on a tcp socket does not mean it has been sent on the wire > >> >> >> yet, and depending on > >> >> >> the circumstances it is possible for the socket to be closed before > >> >> >> all data has been sent > >> >> >> so occasionally a tcp_close can come before all data is delivered. > > >> >> >> When you use ibrowse do you verify that you always get a correct body? > >> >> >> (compleate body) from what I can tell from the code ibrowse will on a > >> >> >> tcp_closed ?message always return whatever happens to be in the buffer > >> >> >> compleate or not. > > >> >> > ibrowse does this only in the following cases: > >> >> > ?* server returned a "Connection: Close" > >> >> > ?* Server supports HTTP/0.9 or HTTP/1.0 > > >> >> > In all other cases, it returns an error to the caller. > > >> >> > regards, > >> >> > Chandru > > >> >> Well I do not know if any of these things is true in Dans senario. But > >> >> I do know that > >> >> getting the problem with a tcp close message before all data is > >> >> deliverd is a highly timing dependant problem > >> >> and if it happens in the server there is nothing inets-client or > >> >> ibrows can do about it except return an error. > > >> >> Regards Ingela Erlang/OTP team - Ericsson AB > > >> >> ________________________________________________________________ > >> >> erlang-questions (at) erlang.org mailing list. > >> >> Seehttp://www.erlang.org/faq.html > >> >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > >> > ________________________________________________________________ > >> > erlang-questions (at) erlang.org mailing list. > >> > Seehttp://www.erlang.org/faq.html > >> > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> Seehttp://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > Seehttp://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From ulf.wiger@REDACTED Wed Oct 27 09:04:06 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 27 Oct 2010 09:04:06 +0200 Subject: [erlang-questions] Ridiculuous newbie mnesia question. In-Reply-To: <032401cb7588$4f5f5180$ee1df480$@com> References: <00d701cb7405$bff89930$3fe9cb90$@com> <1AD7068E-BAE8-44BB-A77C-77BA32E08F49@erlang-solutions.com> <032401cb7588$4f5f5180$ee1df480$@com> Message-ID: On 27 Oct 2010, at 05:37, Jeffrey Rennie wrote: > Thank you for your patience. I still haven't escaped from this wet paper > bag: > >> Looking at the mnesia:info() output, there are a couple of things >> to note. >> >> 1. 'running db nodes = [emacs@REDACTED]' means that while a@REDACTED >> and b@REDACTED might be running at the erlang level, perhaps even >> with mnesia runnin, they are not connected to emacs@REDACTED >> > > Why aren't they connected? How do I connect them? I can ping them (see > below.) Use the extra_db_nodes parameter for the nodes that do not have a persistent copy of the schema. For example, if you have nodes A, B, and C, where A and B have a persistent schema, you would initially create the schema from either A or B: mnesia:create_schema([A,B]). ...making sure that both nodes A and B are reachable at the time. Then when starting mnesia on C, you can either do this: mnesia:start([{extra_db_nodes,[A,B]}]) or add it to the erl command line: erl -sname c -mnesia extra_db_nodes \[a@REDACTED,b@REDACTED\] note the shell-related escaping issues. If your node names have special characters, you have to include the single quotes and escape them too. You can also create a sys.config file, where you insert application environment variables: sys.config: [{mnesia, [ {extra_db_nodes, [a@REDACTED, b@REDACTED]} ] }]. then start Erlang with erl -sname c -config sys The extra_db_nodes parameter tells mnesia where to go and find the persistent schema, so you would usually include all nodes that are known to have a persistent schema in the list. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From hans.bolinder@REDACTED Wed Oct 27 09:26:58 2010 From: hans.bolinder@REDACTED (Hans Bolinder) Date: Wed, 27 Oct 2010 09:26:58 +0200 Subject: [erlang-questions] eDoc picking up -spec and -type; when? In-Reply-To: References: Message-ID: <19655.54338.625703.700891@ornendil.du.uab.ericsson.se> [Logan, Martin:] > Can anyone tell me when eDoc will start to pick up specs? The work in on-going but interrupted from time to time by other tasks. There is a chance that a working version will be finished before R14B01, but R14B02 is more likely. The latest version can be found at http://github.com/uabboli/EDoc. It is not stable, but as long as one is aware of that, it can be used for testing. A typical invocation that activates the new features: edoc:files([File], [{dialyzer_specs,all},{report_missing_type,true}, {report_type_mismatch,true},{pretty_print,erl_pp}, {preprocess,true}]). Best regards, Hans Bolinder, Erlang/OTP team, Ericsson From ebegumisa@REDACTED Wed Oct 27 09:46:01 2010 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 27 Oct 2010 18:46:01 +1100 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> Message-ID: Hi, I hope the Couch team isn't planning on doing this by default -- something about it makes me nervous... When CouchDB is on it's own, it might not be alarming/noticeable, but I'm using CouchDB "embedded" in a wider Erlang/OTP application stack (i.e. where Couch is just one of many OTP apps running in the *SAME* VM -- I have a few hacks for avoiding socket communication.) I too worry about the potential for NIF-endowed couch io disturbing the balance of Erlang's scheduling. It would be good to see similar benchmarking with the VM concurrently doing things other than handling couch-related requests (which are implicitly synchronised in your case.) - Edmond - On Mon, 25 Oct 2010 07:59:11 +1100, Joel Reymont wrote: > > On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote: > >> I wonder how responsive the system is to other events when running the >> benchmark. > > The benchmark simulates several hundred clients hitting a (mochiweb) web > server to read and write couchdb (json) documents. The system seems to > stay -highly- responsive, 10x so compared to the same system not using > NIFs at all. > > If low write response time is taken as a measure of system > responsiveness then the first graph shows that the responsiveness of the > system has increased dramatically. The write response here is the take > taken to process a web request to write a couch document. > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Wed Oct 27 09:49:56 2010 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 27 Oct 2010 18:49:56 +1100 Subject: [erlang-questions] non-trivial supervisor idioms? -- An open-source plea! In-Reply-To: References: <4CA0AAD3.3030209@erlang-solutions.com> Message-ID: That puts a nice perspective on things... obvious now I was trying to use included apps for something they were not designed for. But your story really got me thinking... Sometimes I wish our friends at Ericsson would open-source one of their larger Erlang projects. Those of us who have NOT picked up Erlang from industrial environments have a problem of finding code we can reliably read for inspiration. I (and I think many), heard about Erlang from somewhere, went and read Joe's book (a superb introduction which covers the 'spirit' of Erlang), identified with the problems Erlang tries to solve, maybe read Cesarini and Thompson as a follow up (an excellent reference.) The problem is no matter how good these books are, there's only so much a book can teach you. A book certainly cannot advise you on how to write a complete application or how to avoid writing programs that become unwieldy as they grow. The question is what next? The obvious answer is to look at some open-source code. But when most open-source projects have learned Erlang the same way it becomes a case of the blind leading the blind. Problems that must have already been experienced and dealt with at Ericsson are repeated. A large production-ready open-source project written by battle-hardened experienced Erlang programmers would really fill in that void. Something big and sufficiently complex like a soft-switch or something. - Edmond - On Tue, 28 Sep 2010 17:07:16 +1000, Ulf Wiger wrote: > > On 27 Sep 2010, at 20:05, Edmond Begumisa wrote: > >> Hi again Ulf, >> >> It's great to get 'the guy' on this subject online so I'm going to take >> full advantage and ask two more questions that have been dogging me... >> >> Firstly, is there an open-source project you know of that uses >> included-applications and/or start phases properly that I could take a >> peek at? Maybe in OTP source itself? > > Off the top of my head, I really can't think of any. :) > > The area where start phases really come in handy is when your > application needs > to support failover/takeover behaviour. This is also when the StartType > argument > is needed. One can implement takeover by writing a special start phase > that instructs > the processes to take over processing from the other side. In general, > it is best to do > this at a point where all processes have been started and are ready to > process > incoming requests. > > The initial reason for start phases was that the complex call-handling > applications > at Ericsson had some pretty horrendous dependencies to sort out before > they > could start accepting calls, and doing this work in the init function of > the processes > simply wasn't feasible. Also, when a process dies in the init function, > this is > interpreted as a start error, and the application start will fail, > whereas individual > processes have proper supervision while they are responding to requests > from > the start phase code (which runs in the application_starter process). > > Included applications were mainly introduced since the same call-handling > applications needed to move as one during failover and takeover, and > starting > a dozen or so top applications made that much more difficult. It was > just too much > code and too many modules to integrate into one single application > without one > more structuring layer. > > Initially, I wrote some code that read .appSrc files in each > sub-application and > integrated them into one larger application, using a top-level resource > file - I > think it had the extension .appLm (as in load module - never mind; it > made sense > at Ericsson, and it was so long ago that I may be remembering wrong). > This was > later generalised by OTP into included_applications. > > The O&M applications also had a problem during takeover: The snmp code > assumed that the snmp agent was locally registered on the same node, > which > wasn't necessarily the case during the transition - either on the node > taking over > or on the node where it ran before. We then created a wrapper > application that > included all the O&M applications, and called the individual start > functions for > each included app. > > Later, we moved away from that, as we had to also support applications > that > were written according to a different timeline, and therefore couldn't > be integrated > the same way as our other apps. I came up with a solution for starting > and stopping > included apps and plugging in their start phase hooks in the right > places in the > startup flow, but for some reason people found it complicated... :) > > The better solution was to make use of the fact that the application > controller now > had a message passing interface for controlling the starting and > stopping of apps. > We were already using this in our cluster controller, so we could extend > it by > specifying distributed start dependencies and which applications needed > to do > takeover in parallel. This way, the cluster controller knew in which > order to move > applications during takeover, and in which order to terminate them, once > migrated. > Unfortunately, all this code is proprietary. It's on my long list of > things I'd like to do, > but that list just keeps growing, without much ever being removed from > it... > > A long time ago, I made a prototype (and sent to OTP) that introduced > start phase > dependencies. This would IMHO make it much easier to specify dependencies > between applications. As an example, mnesia loads tables in the > background, so > when the application:start() function returns, one cannot assume that > tables are > loaded, and has to call mnesia:wait_for_tables() (which can time out, > and has some > corner cases where tables will never be loaded without intervention - > not that the > function itself will tell you when they occur). It might be better if > mnesia had a > load_tables start phase, which other applications could depend on. > > BR, > Ulf W > >> >> Secondly, I've always liked the idea of using included applications not >> necessarily for start phases but as a delayed/start-on-demand mechanism >> (taking advantage of the fact that included apps are automatically >> loaded but not started.) That is, manually calling >> application:start(foo) only if a particular feature of my app is used. >> But I have one query that made attempts for such use short-lived... the >> fact that an application can only be included by one other application. >> I think this limitation makes it harder to use included apps and start >> phases especially if you're using apps that are not in-house. >> >> For example, lets say CouchDB starts using mnesia (ok that's dumb >> but...) and decide to start it up using start phases (and therefore add >> it as an included application in couch.app) Then I have my FunkyApp >> that's been using mnesia too as included application. I then decide to >> use CouchDB for a new funky feature of FunkyApp. Now things break >> because mnesia is being used by both FunkyApp and CouchDB. To fix this, >> I not only have to modify my in-house app I have to modify the >> out-house CouchDB too. >> >> Is there an obvious fix to this I've been missing? >> >> - Edmond - >> >> On Tue, 28 Sep 2010 03:19:29 +1000, Ulf Wiger >> wrote: >> >> >> On 27 Sep 2010, at 18:14, Edmond Begumisa wrote: >> >> Ulf, >> >> I've been doing such initialisation in the init function of a worker >> manager process. Using Daniel's example, I might have a gen_server >> child of the main supervisor called db_mgr and set up the mnesia schema >> in db_mgr:init >> >> Have I been doing the 'wrong' thing OTP-wise? >> >> Not necessarily, but my personal preference is to cleanly separate >> setup code >> from application startup. This is in part because I used to work on a >> very complex >> product, where the setup was decidedly non-trivial, and the startup >> process had >> to be optimised in several steps. >> >> Still, even there, I believe that the setup logic was bootstrapped into >> the startup >> phase, but the code was still kept cleanly separated. The only thing >> that was >> part of the startup was a simple check to see if the setup code had >> been run. >> >> BR, >> Ulf W >> >> >> >> - Edmond - >> >> On Tue, 28 Sep 2010 00:31:47 +1000, Ulf Wiger >> wrote: >> >> On 27/09/2010 16:15, Daniel Goertzen wrote: >> I've read the documentation on supervision and have seen a few >> tutorials, >> but they don't seem to move beyond the core concepts. For example, what >> happens if you want to check and optionally setup an mnesia schema >> during >> startup...where should this code go? In the supervisor init() or >> start_link() function? Should I have my supervisor create a worker >> process >> whole sole job is to do this kind of setup and then dynamically add >> other >> workers (or supervisors) to the supervisor with start_child()? >> >> I strongly recommend doing that sort of thing in a separate procedure, >> rather than in the startup phase. >> >> If you want your application to be able to bootstrap itself, I would >> suggest that you either: >> >> - create a special application that runs before your other apps, >> and verifies that the installation is ok. To this end, it might be >> useful to know that you can pre-sort the .rel file. The systools lib >> will only change the sort order if needed to respect start >> dependencies. >> - Introduce start_phases, then do minimal work in the init function, >> and push the rest to functions that are called from start phase >> hooks. This also has the advantage that you know that your processes >> are all started and ready to respond during the init phase. >> >> Start phases are documented in >> http://www.erlang.org/doc/apps/kernel/application.html#Module:start_phase-3 >> >> BR, >> Ulf W >> >> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ulf.wiger@REDACTED Wed Oct 27 09:58:27 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 27 Oct 2010 09:58:27 +0200 Subject: [erlang-questions] eDoc picking up -spec and -type; when? In-Reply-To: <19655.54338.625703.700891@ornendil.du.uab.ericsson.se> References: <19655.54338.625703.700891@ornendil.du.uab.ericsson.se> Message-ID: Hi Hans, Since you're working on EDoc... :) While playing with my edown extension to EDoc (where I also had some issues with the imperative nature and implicit assumptions in the plugin framework, but that's OT), I thought a bit about how EDoc handles external references. The small initial test case I had was the docs in edown_doclet.erl [1], which is basically copied from edoc_doclet.erl. I noticed that the link references e.g. {@link //edoc/edoc:layout/2} didn't expand to a usable (online) link. By default, edoc finds the local copy of the app and inserts the file path, but it also provides an option {app_default, URI} for other targets. I thought the reasonable configuration would be something like: {edoc_opts, [{doclet, edown_doclet}, {app_default,"http://www.erlang.org/doc/man"}]}. ...in rebar.config This changes the link, but to http://www.erlang.org/doc/man/edoc/doc/edoc.html#layout-2 The "edoc/doc/ path is something edoc inserts. The correct link should be http://www.erlang.org/doc/man/edoc_doclet.html#layout-2 One can spend significant time with the edoc docs without grasping how to get edoc to correctly reference the online OTP man pages. Is there some obvious trick that I'm missing? There are some other options, where edoc can be instructed to fetch edoc-info files, but I'd rather have something that can be statically resolved. BR, Ulf W [1] http://github.com/esl/edown/blob/master/doc/edown_doclet.md On 27 Oct 2010, at 09:26, Hans Bolinder wrote: > [Logan, Martin:] >> Can anyone tell me when eDoc will start to pick up specs? > > The work in on-going but interrupted from time to time by other tasks. > > There is a chance that a working version will be finished before > R14B01, but R14B02 is more likely. > > The latest version can be found at http://github.com/uabboli/EDoc. It > is not stable, but as long as one is aware of that, it can be used for > testing. > > A typical invocation that activates the new features: > > edoc:files([File], [{dialyzer_specs,all},{report_missing_type,true}, > {report_type_mismatch,true},{pretty_print,erl_pp}, > {preprocess,true}]). > > Best regards, > > Hans Bolinder, Erlang/OTP team, Ericsson > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ulf.wiger@REDACTED Wed Oct 27 10:17:43 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 27 Oct 2010 10:17:43 +0200 Subject: [erlang-questions] non-trivial supervisor idioms? -- An open-source plea! In-Reply-To: References: <4CA0AAD3.3030209@erlang-solutions.com> Message-ID: <36BEDEE4-D66A-4FD4-9793-B0B444C35236@erlang-solutions.com> Edmond, One big problem for a company releasing large and complex, battle-tested software as Open Source is that Ericsson - with a ~30% share of the global mobile infrastructure market - is a juicy IPR target. Since it's entirely possible - nay, almost a certainty - that code of that size will infringe on some existing patents*, and given that it is a practical impossibility to figure out which (unless you happen to be the owner of the patent, and know what to look for), it is unlikely to happen. * most of which would probably not hold up in court, but challenging them in court is still a risky and expensive process - not least with huge opportunity cost. There are a few feeble protections against being sued for patent infringement: - Use a fringe technology. This invalidates many claims, as they are often fairly language-specific. - Do not search the patent database. This may seem absurd, but you probably won't find anything anyway, but if you did search, you may eventually get sued for wilful infringement, which will increase the penalty significantly. - Do not release your source code, so that it can be indexed and subjected to automated search programs by patent raiders. - Stay small and uninteresting, so no one will bother suing you. - Keep a really big patent portfolio, so you can sue them right back. Ericsson is not small and uninteresting, but they _can_ exercise caution in releasing source code. Given this, it is actually no small matter that they maintain Erlang/OTP in the open. :) But hey, it's their decision. I'm not saying it's necessarily a bad idea - just guessing at some of the possible impediments. BR, Ulf W On 27 Oct 2010, at 09:49, Edmond Begumisa wrote: > That puts a nice perspective on things... obvious now I was trying to use included apps for something they were not designed for. But your story really got me thinking... > > Sometimes I wish our friends at Ericsson would open-source one of their larger Erlang projects. Those of us who have NOT picked up Erlang from industrial environments have a problem of finding code we can reliably read for inspiration. > > I (and I think many), heard about Erlang from somewhere, went and read Joe's book (a superb introduction which covers the 'spirit' of Erlang), identified with the problems Erlang tries to solve, maybe read Cesarini and Thompson as a follow up (an excellent reference.) The problem is no matter how good these books are, there's only so much a book can teach you. A book certainly cannot advise you on how to write a complete application or how to avoid writing programs that become unwieldy as they grow. The question is what next? > > The obvious answer is to look at some open-source code. But when most open-source projects have learned Erlang the same way it becomes a case of the blind leading the blind. Problems that must have already been experienced and dealt with at Ericsson are repeated. A large production-ready open-source project written by battle-hardened experienced Erlang programmers would really fill in that void. Something big and sufficiently complex like a soft-switch or something. > > - Edmond - > > > On Tue, 28 Sep 2010 17:07:16 +1000, Ulf Wiger wrote: > >> >> On 27 Sep 2010, at 20:05, Edmond Begumisa wrote: >> >>> Hi again Ulf, >>> >>> It's great to get 'the guy' on this subject online so I'm going to take full advantage and ask two more questions that have been dogging me... >>> >>> Firstly, is there an open-source project you know of that uses included-applications and/or start phases properly that I could take a peek at? Maybe in OTP source itself? >> >> Off the top of my head, I really can't think of any. :) >> >> The area where start phases really come in handy is when your application needs >> to support failover/takeover behaviour. This is also when the StartType argument >> is needed. One can implement takeover by writing a special start phase that instructs >> the processes to take over processing from the other side. In general, it is best to do >> this at a point where all processes have been started and are ready to process >> incoming requests. >> >> The initial reason for start phases was that the complex call-handling applications >> at Ericsson had some pretty horrendous dependencies to sort out before they >> could start accepting calls, and doing this work in the init function of the processes >> simply wasn't feasible. Also, when a process dies in the init function, this is >> interpreted as a start error, and the application start will fail, whereas individual >> processes have proper supervision while they are responding to requests from >> the start phase code (which runs in the application_starter process). >> >> Included applications were mainly introduced since the same call-handling >> applications needed to move as one during failover and takeover, and starting >> a dozen or so top applications made that much more difficult. It was just too much >> code and too many modules to integrate into one single application without one >> more structuring layer. >> >> Initially, I wrote some code that read .appSrc files in each sub-application and >> integrated them into one larger application, using a top-level resource file - I >> think it had the extension .appLm (as in load module - never mind; it made sense >> at Ericsson, and it was so long ago that I may be remembering wrong). This was >> later generalised by OTP into included_applications. >> >> The O&M applications also had a problem during takeover: The snmp code >> assumed that the snmp agent was locally registered on the same node, which >> wasn't necessarily the case during the transition - either on the node taking over >> or on the node where it ran before. We then created a wrapper application that >> included all the O&M applications, and called the individual start functions for >> each included app. >> >> Later, we moved away from that, as we had to also support applications that >> were written according to a different timeline, and therefore couldn't be integrated >> the same way as our other apps. I came up with a solution for starting and stopping >> included apps and plugging in their start phase hooks in the right places in the >> startup flow, but for some reason people found it complicated... :) >> >> The better solution was to make use of the fact that the application controller now >> had a message passing interface for controlling the starting and stopping of apps. >> We were already using this in our cluster controller, so we could extend it by >> specifying distributed start dependencies and which applications needed to do >> takeover in parallel. This way, the cluster controller knew in which order to move >> applications during takeover, and in which order to terminate them, once migrated. >> Unfortunately, all this code is proprietary. It's on my long list of things I'd like to do, >> but that list just keeps growing, without much ever being removed from it... >> >> A long time ago, I made a prototype (and sent to OTP) that introduced start phase >> dependencies. This would IMHO make it much easier to specify dependencies >> between applications. As an example, mnesia loads tables in the background, so >> when the application:start() function returns, one cannot assume that tables are >> loaded, and has to call mnesia:wait_for_tables() (which can time out, and has some >> corner cases where tables will never be loaded without intervention - not that the >> function itself will tell you when they occur). It might be better if mnesia had a >> load_tables start phase, which other applications could depend on. >> >> BR, >> Ulf W >> >>> >>> Secondly, I've always liked the idea of using included applications not necessarily for start phases but as a delayed/start-on-demand mechanism (taking advantage of the fact that included apps are automatically loaded but not started.) That is, manually calling application:start(foo) only if a particular feature of my app is used. But I have one query that made attempts for such use short-lived... the fact that an application can only be included by one other application. I think this limitation makes it harder to use included apps and start phases especially if you're using apps that are not in-house. >>> >>> For example, lets say CouchDB starts using mnesia (ok that's dumb but...) and decide to start it up using start phases (and therefore add it as an included application in couch.app) Then I have my FunkyApp that's been using mnesia too as included application. I then decide to use CouchDB for a new funky feature of FunkyApp. Now things break because mnesia is being used by both FunkyApp and CouchDB. To fix this, I not only have to modify my in-house app I have to modify the out-house CouchDB too. >>> >>> Is there an obvious fix to this I've been missing? >>> >>> - Edmond - >>> >>> On Tue, 28 Sep 2010 03:19:29 +1000, Ulf Wiger wrote: >>> >>> >>> On 27 Sep 2010, at 18:14, Edmond Begumisa wrote: >>> >>> Ulf, >>> >>> I've been doing such initialisation in the init function of a worker manager process. Using Daniel's example, I might have a gen_server child of the main supervisor called db_mgr and set up the mnesia schema in db_mgr:init >>> >>> Have I been doing the 'wrong' thing OTP-wise? >>> >>> Not necessarily, but my personal preference is to cleanly separate setup code >>> from application startup. This is in part because I used to work on a very complex >>> product, where the setup was decidedly non-trivial, and the startup process had >>> to be optimised in several steps. >>> >>> Still, even there, I believe that the setup logic was bootstrapped into the startup >>> phase, but the code was still kept cleanly separated. The only thing that was >>> part of the startup was a simple check to see if the setup code had been run. >>> >>> BR, >>> Ulf W >>> >>> >>> >>> - Edmond - >>> >>> On Tue, 28 Sep 2010 00:31:47 +1000, Ulf Wiger wrote: >>> >>> On 27/09/2010 16:15, Daniel Goertzen wrote: >>> I've read the documentation on supervision and have seen a few tutorials, >>> but they don't seem to move beyond the core concepts. For example, what >>> happens if you want to check and optionally setup an mnesia schema during >>> startup...where should this code go? In the supervisor init() or >>> start_link() function? Should I have my supervisor create a worker process >>> whole sole job is to do this kind of setup and then dynamically add other >>> workers (or supervisors) to the supervisor with start_child()? >>> >>> I strongly recommend doing that sort of thing in a separate procedure, >>> rather than in the startup phase. >>> >>> If you want your application to be able to bootstrap itself, I would >>> suggest that you either: >>> >>> - create a special application that runs before your other apps, >>> and verifies that the installation is ok. To this end, it might be >>> useful to know that you can pre-sort the .rel file. The systools lib >>> will only change the sort order if needed to respect start >>> dependencies. >>> - Introduce start_phases, then do minimal work in the init function, >>> and push the rest to functions that are called from start phase >>> hooks. This also has the advantage that you know that your processes >>> are all started and ready to respond during the init phase. >>> >>> Start phases are documented in >>> http://www.erlang.org/doc/apps/kernel/application.html#Module:start_phase-3 >>> >>> BR, >>> Ulf W >>> >>> >>> >>> -- >>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>> >>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>> http://erlang-solutions.com >>> >>> >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >>> >>> -- >>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From alessandro.sivieri@REDACTED Wed Oct 27 10:44:02 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Wed, 27 Oct 2010 10:44:02 +0200 Subject: [erlang-questions] Re: Mochiweb and SSL In-Reply-To: <2558e383-4d2c-489a-aa6b-000ce993a367@e14g2000yqe.googlegroups.com> References: <2558e383-4d2c-489a-aa6b-000ce993a367@e14g2000yqe.googlegroups.com> Message-ID: 2010/10/27 Steve Davis > What happens when you change the port to 8443 (i.e. above 1024)? > > Yes, I had to use a port above 1024... anyway, I tried with 8443, and here it is the output: http://pastebin.com/6H67CUhP It looks like something is running, but then when I try to access https://IP:8443 it crashes... I have not changed anything else in the Mochiweb generated application, except the parameters cited above, so I don't know if anything else has to be changed. -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From ebegumisa@REDACTED Wed Oct 27 11:03:36 2010 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 27 Oct 2010 20:03:36 +1100 Subject: [erlang-questions] non-trivial supervisor idioms? -- An open-source plea! In-Reply-To: <36BEDEE4-D66A-4FD4-9793-B0B444C35236@erlang-solutions.com> References: <4CA0AAD3.3030209@erlang-solutions.com> <36BEDEE4-D66A-4FD4-9793-B0B444C35236@erlang-solutions.com> Message-ID: I suppose you're right. Perhaps there's another solution. My problem is this... if you do a google search for "hex encode in Erlang" you'll see a large number of really bad code snippets. Yet the authors present these as solutions. I for one used one of these 'solutions' for a while when just starting out with Erlang. As I got more proficient, one day I looked back at that code and thought "that's horrible" (Erlang is unique this way in that it can take you a really long time to learn how to recognise bad code.) Now, multiply and this by all the functions required to write an application and what you end up with is some not-so-good code released as open-source that other programmers relatively new to Erlang look to as a 'how-to' reference. This cannot be good for the language as a whole. Maybe Ericsson could oversee the development of a new open source tool. I would personally love to see an Asterisk in Erlang, but I have purely selfish reasons. - Edmond - On Wed, 27 Oct 2010 19:17:43 +1100, Ulf Wiger wrote: > > Edmond, > > One big problem for a company releasing large and complex, battle-tested > software > as Open Source is that Ericsson - with a ~30% share of the global mobile > infrastructure > market - is a juicy IPR target. Since it's entirely possible - nay, > almost a certainty - that > code of that size will infringe on some existing patents*, and given > that it is a practical > impossibility to figure out which (unless you happen to be the owner of > the patent, > and know what to look for), it is unlikely to happen. > > * most of which would probably not hold up in court, but challenging > them in court > is still a risky and expensive process - not least with huge > opportunity cost. > > There are a few feeble protections against being sued for patent > infringement: > > - Use a fringe technology. This invalidates many claims, as they are > often fairly > language-specific. > - Do not search the patent database. This may seem absurd, but you > probably > won't find anything anyway, but if you did search, you may eventually > get sued > for wilful infringement, which will increase the penalty significantly. > - Do not release your source code, so that it can be indexed and > subjected to > automated search programs by patent raiders. > - Stay small and uninteresting, so no one will bother suing you. > - Keep a really big patent portfolio, so you can sue them right back. > > Ericsson is not small and uninteresting, but they _can_ exercise caution > in releasing > source code. Given this, it is actually no small matter that they > maintain Erlang/OTP > in the open. :) > > But hey, it's their decision. I'm not saying it's necessarily a bad idea > - just guessing > at some of the possible impediments. > > BR, > Ulf W > > On 27 Oct 2010, at 09:49, Edmond Begumisa wrote: > >> That puts a nice perspective on things... obvious now I was trying to >> use included apps for something they were not designed for. But your >> story really got me thinking... >> >> Sometimes I wish our friends at Ericsson would open-source one of their >> larger Erlang projects. Those of us who have NOT picked up Erlang from >> industrial environments have a problem of finding code we can reliably >> read for inspiration. >> >> I (and I think many), heard about Erlang from somewhere, went and read >> Joe's book (a superb introduction which covers the 'spirit' of Erlang), >> identified with the problems Erlang tries to solve, maybe read Cesarini >> and Thompson as a follow up (an excellent reference.) The problem is no >> matter how good these books are, there's only so much a book can teach >> you. A book certainly cannot advise you on how to write a complete >> application or how to avoid writing programs that become unwieldy as >> they grow. The question is what next? >> >> The obvious answer is to look at some open-source code. But when most >> open-source projects have learned Erlang the same way it becomes a case >> of the blind leading the blind. Problems that must have already been >> experienced and dealt with at Ericsson are repeated. A large >> production-ready open-source project written by battle-hardened >> experienced Erlang programmers would really fill in that void. >> Something big and sufficiently complex like a soft-switch or something. >> >> - Edmond - >> >> >> On Tue, 28 Sep 2010 17:07:16 +1000, Ulf Wiger >> wrote: >> >>> >>> On 27 Sep 2010, at 20:05, Edmond Begumisa wrote: >>> >>>> Hi again Ulf, >>>> >>>> It's great to get 'the guy' on this subject online so I'm going to >>>> take full advantage and ask two more questions that have been dogging >>>> me... >>>> >>>> Firstly, is there an open-source project you know of that uses >>>> included-applications and/or start phases properly that I could take >>>> a peek at? Maybe in OTP source itself? >>> >>> Off the top of my head, I really can't think of any. :) >>> >>> The area where start phases really come in handy is when your >>> application needs >>> to support failover/takeover behaviour. This is also when the >>> StartType argument >>> is needed. One can implement takeover by writing a special start phase >>> that instructs >>> the processes to take over processing from the other side. In general, >>> it is best to do >>> this at a point where all processes have been started and are ready to >>> process >>> incoming requests. >>> >>> The initial reason for start phases was that the complex call-handling >>> applications >>> at Ericsson had some pretty horrendous dependencies to sort out before >>> they >>> could start accepting calls, and doing this work in the init function >>> of the processes >>> simply wasn't feasible. Also, when a process dies in the init >>> function, this is >>> interpreted as a start error, and the application start will fail, >>> whereas individual >>> processes have proper supervision while they are responding to >>> requests from >>> the start phase code (which runs in the application_starter process). >>> >>> Included applications were mainly introduced since the same >>> call-handling >>> applications needed to move as one during failover and takeover, and >>> starting >>> a dozen or so top applications made that much more difficult. It was >>> just too much >>> code and too many modules to integrate into one single application >>> without one >>> more structuring layer. >>> >>> Initially, I wrote some code that read .appSrc files in each >>> sub-application and >>> integrated them into one larger application, using a top-level >>> resource file - I >>> think it had the extension .appLm (as in load module - never mind; it >>> made sense >>> at Ericsson, and it was so long ago that I may be remembering wrong). >>> This was >>> later generalised by OTP into included_applications. >>> >>> The O&M applications also had a problem during takeover: The snmp code >>> assumed that the snmp agent was locally registered on the same node, >>> which >>> wasn't necessarily the case during the transition - either on the node >>> taking over >>> or on the node where it ran before. We then created a wrapper >>> application that >>> included all the O&M applications, and called the individual start >>> functions for >>> each included app. >>> >>> Later, we moved away from that, as we had to also support applications >>> that >>> were written according to a different timeline, and therefore couldn't >>> be integrated >>> the same way as our other apps. I came up with a solution for starting >>> and stopping >>> included apps and plugging in their start phase hooks in the right >>> places in the >>> startup flow, but for some reason people found it complicated... :) >>> >>> The better solution was to make use of the fact that the application >>> controller now >>> had a message passing interface for controlling the starting and >>> stopping of apps. >>> We were already using this in our cluster controller, so we could >>> extend it by >>> specifying distributed start dependencies and which applications >>> needed to do >>> takeover in parallel. This way, the cluster controller knew in which >>> order to move >>> applications during takeover, and in which order to terminate them, >>> once migrated. >>> Unfortunately, all this code is proprietary. It's on my long list of >>> things I'd like to do, >>> but that list just keeps growing, without much ever being removed from >>> it... >>> >>> A long time ago, I made a prototype (and sent to OTP) that introduced >>> start phase >>> dependencies. This would IMHO make it much easier to specify >>> dependencies >>> between applications. As an example, mnesia loads tables in the >>> background, so >>> when the application:start() function returns, one cannot assume that >>> tables are >>> loaded, and has to call mnesia:wait_for_tables() (which can time out, >>> and has some >>> corner cases where tables will never be loaded without intervention - >>> not that the >>> function itself will tell you when they occur). It might be better if >>> mnesia had a >>> load_tables start phase, which other applications could depend on. >>> >>> BR, >>> Ulf W >>> >>>> >>>> Secondly, I've always liked the idea of using included applications >>>> not necessarily for start phases but as a delayed/start-on-demand >>>> mechanism (taking advantage of the fact that included apps are >>>> automatically loaded but not started.) That is, manually calling >>>> application:start(foo) only if a particular feature of my app is >>>> used. But I have one query that made attempts for such use >>>> short-lived... the fact that an application can only be included by >>>> one other application. I think this limitation makes it harder to use >>>> included apps and start phases especially if you're using apps that >>>> are not in-house. >>>> >>>> For example, lets say CouchDB starts using mnesia (ok that's dumb >>>> but...) and decide to start it up using start phases (and therefore >>>> add it as an included application in couch.app) Then I have my >>>> FunkyApp that's been using mnesia too as included application. I then >>>> decide to use CouchDB for a new funky feature of FunkyApp. Now things >>>> break because mnesia is being used by both FunkyApp and CouchDB. To >>>> fix this, I not only have to modify my in-house app I have to modify >>>> the out-house CouchDB too. >>>> >>>> Is there an obvious fix to this I've been missing? >>>> >>>> - Edmond - >>>> >>>> On Tue, 28 Sep 2010 03:19:29 +1000, Ulf Wiger >>>> wrote: >>>> >>>> >>>> On 27 Sep 2010, at 18:14, Edmond Begumisa wrote: >>>> >>>> Ulf, >>>> >>>> I've been doing such initialisation in the init function of a worker >>>> manager process. Using Daniel's example, I might have a gen_server >>>> child of the main supervisor called db_mgr and set up the mnesia >>>> schema in db_mgr:init >>>> >>>> Have I been doing the 'wrong' thing OTP-wise? >>>> >>>> Not necessarily, but my personal preference is to cleanly separate >>>> setup code >>>> from application startup. This is in part because I used to work on a >>>> very complex >>>> product, where the setup was decidedly non-trivial, and the startup >>>> process had >>>> to be optimised in several steps. >>>> >>>> Still, even there, I believe that the setup logic was bootstrapped >>>> into the startup >>>> phase, but the code was still kept cleanly separated. The only thing >>>> that was >>>> part of the startup was a simple check to see if the setup code had >>>> been run. >>>> >>>> BR, >>>> Ulf W >>>> >>>> >>>> >>>> - Edmond - >>>> >>>> On Tue, 28 Sep 2010 00:31:47 +1000, Ulf Wiger >>>> wrote: >>>> >>>> On 27/09/2010 16:15, Daniel Goertzen wrote: >>>> I've read the documentation on supervision and have seen a few >>>> tutorials, >>>> but they don't seem to move beyond the core concepts. For example, >>>> what >>>> happens if you want to check and optionally setup an mnesia schema >>>> during >>>> startup...where should this code go? In the supervisor init() or >>>> start_link() function? Should I have my supervisor create a worker >>>> process >>>> whole sole job is to do this kind of setup and then dynamically add >>>> other >>>> workers (or supervisors) to the supervisor with start_child()? >>>> >>>> I strongly recommend doing that sort of thing in a separate procedure, >>>> rather than in the startup phase. >>>> >>>> If you want your application to be able to bootstrap itself, I would >>>> suggest that you either: >>>> >>>> - create a special application that runs before your other apps, >>>> and verifies that the installation is ok. To this end, it might be >>>> useful to know that you can pre-sort the .rel file. The systools lib >>>> will only change the sort order if needed to respect start >>>> dependencies. >>>> - Introduce start_phases, then do minimal work in the init function, >>>> and push the rest to functions that are called from start phase >>>> hooks. This also has the advantage that you know that your processes >>>> are all started and ready to respond during the init phase. >>>> >>>> Start phases are documented in >>>> http://www.erlang.org/doc/apps/kernel/application.html#Module:start_phase-3 >>>> >>>> BR, >>>> Ulf W >>>> >>>> >>>> >>>> -- >>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>>> >>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>>> http://erlang-solutions.com >>>> >>>> >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >>>> >>>> -- >>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>> >>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>> http://erlang-solutions.com >>> >>> >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From geoffrey.biggs@REDACTED Wed Oct 27 11:05:59 2010 From: geoffrey.biggs@REDACTED (Geoffrey Biggs) Date: Wed, 27 Oct 2010 18:05:59 +0900 Subject: [erlang-questions] non-trivial supervisor idioms? -- An open-source plea! In-Reply-To: References: <4CA0AAD3.3030209@erlang-solutions.com> <36BEDEE4-D66A-4FD4-9793-B0B444C35236@erlang-solutions.com> Message-ID: <4CC7EB77.9040106@aist.go.jp> I'd rather see an Erlang Cookbook. The Perl and Python ones are wonderful. Geoff On 27/10/10 18:03, Edmond Begumisa wrote: > I suppose you're right. > > Perhaps there's another solution. My problem is this... if you do a > google search for "hex encode in Erlang" you'll see a large number of > really bad code snippets. Yet the authors present these as solutions. I > for one used one of these 'solutions' for a while when just starting out > with Erlang. As I got more proficient, one day I looked back at that > code and thought "that's horrible" (Erlang is unique this way in that it > can take you a really long time to learn how to recognise bad code.) > Now, multiply and this by all the functions required to write an > application and what you end up with is some not-so-good code released > as open-source that other programmers relatively new to Erlang look to > as a 'how-to' reference. This cannot be good for the language as a whole. > > Maybe Ericsson could oversee the development of a new open source tool. > I would personally love to see an Asterisk in Erlang, but I have purely > selfish reasons. > > - Edmond - > > > On Wed, 27 Oct 2010 19:17:43 +1100, Ulf Wiger > wrote: > >> >> Edmond, >> >> One big problem for a company releasing large and complex, >> battle-tested software >> as Open Source is that Ericsson - with a ~30% share of the global >> mobile infrastructure >> market - is a juicy IPR target. Since it's entirely possible - nay, >> almost a certainty - that >> code of that size will infringe on some existing patents*, and given >> that it is a practical >> impossibility to figure out which (unless you happen to be the owner >> of the patent, >> and know what to look for), it is unlikely to happen. >> >> * most of which would probably not hold up in court, but challenging >> them in court >> is still a risky and expensive process - not least with huge >> opportunity cost. >> >> There are a few feeble protections against being sued for patent >> infringement: >> >> - Use a fringe technology. This invalidates many claims, as they are >> often fairly >> language-specific. >> - Do not search the patent database. This may seem absurd, but you >> probably >> won't find anything anyway, but if you did search, you may >> eventually get sued >> for wilful infringement, which will increase the penalty significantly. >> - Do not release your source code, so that it can be indexed and >> subjected to >> automated search programs by patent raiders. >> - Stay small and uninteresting, so no one will bother suing you. >> - Keep a really big patent portfolio, so you can sue them right back. >> >> Ericsson is not small and uninteresting, but they _can_ exercise >> caution in releasing >> source code. Given this, it is actually no small matter that they >> maintain Erlang/OTP >> in the open. :) >> >> But hey, it's their decision. I'm not saying it's necessarily a bad >> idea - just guessing >> at some of the possible impediments. >> >> BR, >> Ulf W >> >> On 27 Oct 2010, at 09:49, Edmond Begumisa wrote: >> >>> That puts a nice perspective on things... obvious now I was trying to >>> use included apps for something they were not designed for. But your >>> story really got me thinking... >>> >>> Sometimes I wish our friends at Ericsson would open-source one of >>> their larger Erlang projects. Those of us who have NOT picked up >>> Erlang from industrial environments have a problem of finding code we >>> can reliably read for inspiration. >>> >>> I (and I think many), heard about Erlang from somewhere, went and >>> read Joe's book (a superb introduction which covers the 'spirit' of >>> Erlang), identified with the problems Erlang tries to solve, maybe >>> read Cesarini and Thompson as a follow up (an excellent reference.) >>> The problem is no matter how good these books are, there's only so >>> much a book can teach you. A book certainly cannot advise you on how >>> to write a complete application or how to avoid writing programs that >>> become unwieldy as they grow. The question is what next? >>> >>> The obvious answer is to look at some open-source code. But when most >>> open-source projects have learned Erlang the same way it becomes a >>> case of the blind leading the blind. Problems that must have already >>> been experienced and dealt with at Ericsson are repeated. A large >>> production-ready open-source project written by battle-hardened >>> experienced Erlang programmers would really fill in that void. >>> Something big and sufficiently complex like a soft-switch or something. >>> >>> - Edmond - >>> >>> >>> On Tue, 28 Sep 2010 17:07:16 +1000, Ulf Wiger >>> wrote: >>> >>>> >>>> On 27 Sep 2010, at 20:05, Edmond Begumisa wrote: >>>> >>>>> Hi again Ulf, >>>>> >>>>> It's great to get 'the guy' on this subject online so I'm going to >>>>> take full advantage and ask two more questions that have been >>>>> dogging me... >>>>> >>>>> Firstly, is there an open-source project you know of that uses >>>>> included-applications and/or start phases properly that I could >>>>> take a peek at? Maybe in OTP source itself? >>>> >>>> Off the top of my head, I really can't think of any. :) >>>> >>>> The area where start phases really come in handy is when your >>>> application needs >>>> to support failover/takeover behaviour. This is also when the >>>> StartType argument >>>> is needed. One can implement takeover by writing a special start >>>> phase that instructs >>>> the processes to take over processing from the other side. In >>>> general, it is best to do >>>> this at a point where all processes have been started and are ready >>>> to process >>>> incoming requests. >>>> >>>> The initial reason for start phases was that the complex >>>> call-handling applications >>>> at Ericsson had some pretty horrendous dependencies to sort out >>>> before they >>>> could start accepting calls, and doing this work in the init >>>> function of the processes >>>> simply wasn't feasible. Also, when a process dies in the init >>>> function, this is >>>> interpreted as a start error, and the application start will fail, >>>> whereas individual >>>> processes have proper supervision while they are responding to >>>> requests from >>>> the start phase code (which runs in the application_starter process). >>>> >>>> Included applications were mainly introduced since the same >>>> call-handling >>>> applications needed to move as one during failover and takeover, and >>>> starting >>>> a dozen or so top applications made that much more difficult. It was >>>> just too much >>>> code and too many modules to integrate into one single application >>>> without one >>>> more structuring layer. >>>> >>>> Initially, I wrote some code that read .appSrc files in each >>>> sub-application and >>>> integrated them into one larger application, using a top-level >>>> resource file - I >>>> think it had the extension .appLm (as in load module - never mind; >>>> it made sense >>>> at Ericsson, and it was so long ago that I may be remembering >>>> wrong). This was >>>> later generalised by OTP into included_applications. >>>> >>>> The O&M applications also had a problem during takeover: The snmp code >>>> assumed that the snmp agent was locally registered on the same node, >>>> which >>>> wasn't necessarily the case during the transition - either on the >>>> node taking over >>>> or on the node where it ran before. We then created a wrapper >>>> application that >>>> included all the O&M applications, and called the individual start >>>> functions for >>>> each included app. >>>> >>>> Later, we moved away from that, as we had to also support >>>> applications that >>>> were written according to a different timeline, and therefore >>>> couldn't be integrated >>>> the same way as our other apps. I came up with a solution for >>>> starting and stopping >>>> included apps and plugging in their start phase hooks in the right >>>> places in the >>>> startup flow, but for some reason people found it complicated... :) >>>> >>>> The better solution was to make use of the fact that the application >>>> controller now >>>> had a message passing interface for controlling the starting and >>>> stopping of apps. >>>> We were already using this in our cluster controller, so we could >>>> extend it by >>>> specifying distributed start dependencies and which applications >>>> needed to do >>>> takeover in parallel. This way, the cluster controller knew in which >>>> order to move >>>> applications during takeover, and in which order to terminate them, >>>> once migrated. >>>> Unfortunately, all this code is proprietary. It's on my long list of >>>> things I'd like to do, >>>> but that list just keeps growing, without much ever being removed >>>> from it... >>>> >>>> A long time ago, I made a prototype (and sent to OTP) that >>>> introduced start phase >>>> dependencies. This would IMHO make it much easier to specify >>>> dependencies >>>> between applications. As an example, mnesia loads tables in the >>>> background, so >>>> when the application:start() function returns, one cannot assume >>>> that tables are >>>> loaded, and has to call mnesia:wait_for_tables() (which can time >>>> out, and has some >>>> corner cases where tables will never be loaded without intervention >>>> - not that the >>>> function itself will tell you when they occur). It might be better >>>> if mnesia had a >>>> load_tables start phase, which other applications could depend on. >>>> >>>> BR, >>>> Ulf W >>>> >>>>> >>>>> Secondly, I've always liked the idea of using included applications >>>>> not necessarily for start phases but as a delayed/start-on-demand >>>>> mechanism (taking advantage of the fact that included apps are >>>>> automatically loaded but not started.) That is, manually calling >>>>> application:start(foo) only if a particular feature of my app is >>>>> used. But I have one query that made attempts for such use >>>>> short-lived... the fact that an application can only be included by >>>>> one other application. I think this limitation makes it harder to >>>>> use included apps and start phases especially if you're using apps >>>>> that are not in-house. >>>>> >>>>> For example, lets say CouchDB starts using mnesia (ok that's dumb >>>>> but...) and decide to start it up using start phases (and therefore >>>>> add it as an included application in couch.app) Then I have my >>>>> FunkyApp that's been using mnesia too as included application. I >>>>> then decide to use CouchDB for a new funky feature of FunkyApp. Now >>>>> things break because mnesia is being used by both FunkyApp and >>>>> CouchDB. To fix this, I not only have to modify my in-house app I >>>>> have to modify the out-house CouchDB too. >>>>> >>>>> Is there an obvious fix to this I've been missing? >>>>> >>>>> - Edmond - >>>>> >>>>> On Tue, 28 Sep 2010 03:19:29 +1000, Ulf Wiger >>>>> wrote: >>>>> >>>>> >>>>> On 27 Sep 2010, at 18:14, Edmond Begumisa wrote: >>>>> >>>>> Ulf, >>>>> >>>>> I've been doing such initialisation in the init function of a >>>>> worker manager process. Using Daniel's example, I might have a >>>>> gen_server child of the main supervisor called db_mgr and set up >>>>> the mnesia schema in db_mgr:init >>>>> >>>>> Have I been doing the 'wrong' thing OTP-wise? >>>>> >>>>> Not necessarily, but my personal preference is to cleanly separate >>>>> setup code >>>>> from application startup. This is in part because I used to work on >>>>> a very complex >>>>> product, where the setup was decidedly non-trivial, and the startup >>>>> process had >>>>> to be optimised in several steps. >>>>> >>>>> Still, even there, I believe that the setup logic was bootstrapped >>>>> into the startup >>>>> phase, but the code was still kept cleanly separated. The only >>>>> thing that was >>>>> part of the startup was a simple check to see if the setup code had >>>>> been run. >>>>> >>>>> BR, >>>>> Ulf W >>>>> >>>>> >>>>> >>>>> - Edmond - >>>>> >>>>> On Tue, 28 Sep 2010 00:31:47 +1000, Ulf Wiger >>>>> wrote: >>>>> >>>>> On 27/09/2010 16:15, Daniel Goertzen wrote: >>>>> I've read the documentation on supervision and have seen a few >>>>> tutorials, >>>>> but they don't seem to move beyond the core concepts. For example, >>>>> what >>>>> happens if you want to check and optionally setup an mnesia schema >>>>> during >>>>> startup...where should this code go? In the supervisor init() or >>>>> start_link() function? Should I have my supervisor create a worker >>>>> process >>>>> whole sole job is to do this kind of setup and then dynamically add >>>>> other >>>>> workers (or supervisors) to the supervisor with start_child()? >>>>> >>>>> I strongly recommend doing that sort of thing in a separate procedure, >>>>> rather than in the startup phase. >>>>> >>>>> If you want your application to be able to bootstrap itself, I would >>>>> suggest that you either: >>>>> >>>>> - create a special application that runs before your other apps, >>>>> and verifies that the installation is ok. To this end, it might be >>>>> useful to know that you can pre-sort the .rel file. The systools lib >>>>> will only change the sort order if needed to respect start >>>>> dependencies. >>>>> - Introduce start_phases, then do minimal work in the init function, >>>>> and push the rest to functions that are called from start phase >>>>> hooks. This also has the advantage that you know that your processes >>>>> are all started and ready to respond during the init phase. >>>>> >>>>> Start phases are documented in >>>>> http://www.erlang.org/doc/apps/kernel/application.html#Module:start_phase-3 >>>>> >>>>> >>>>> BR, >>>>> Ulf W >>>>> >>>>> >>>>> >>>>> -- >>>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>>>> >>>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>>>> http://erlang-solutions.com >>>>> >>>>> >>>>> >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions (at) erlang.org mailing list. >>>>> See http://www.erlang.org/faq.html >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>> >>>>> >>>>> >>>>> -- >>>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>>> >>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>>> http://erlang-solutions.com >>>> >>>> >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>> >>> >>> -- >>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > From kostis@REDACTED Wed Oct 27 11:35:03 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 27 Oct 2010 12:35:03 +0300 Subject: [erlang-questions] Pretty-printing Erlang In-Reply-To: <032D4B20-14F2-4384-9D62-8302133CCB00@cs.otago.ac.nz> References: <5AE1A072-0537-4AC3-9820-2267A6CCF4C7@cs.otago.ac.nz> <1287558222.4859.4.camel@seasc1137> <207A14F9-852E-4C93-BF21-206F6B9616F9@cs.otago.ac.nz> <1287639191.5435.9.camel@seasc1137> <032D4B20-14F2-4384-9D62-8302133CCB00@cs.otago.ac.nz> Message-ID: <4CC7F247.5080702@cs.ntua.gr> Richard O'Keefe wrote: > On 21/10/2010, at 6:33 PM, Bengt Kleberg wrote: > >> The clean up effect you describe as the beyond the scope of a pretty >> printer might be achieve by tidier >> (http://www.erlang-factory.com/conference/London2009/speakers/kostissagonas). > > I cannot connect to the http://tidier.softlab.ntua.gr/ site; > is that still the correct site for tidier? The website for tidier is up and running again for those that want to use it. (We have had serious trouble with the air conditioning system in the lab and several disks have been hit or mounted read only since last Friday.) >> The problem with several body level expressions on one line is a problem >> that I either avoid by having long variable/function names, or fix >> manually after wards. It would be nice to get it done automatically. > > The problem here is that the original code didn't *have* several > expressions on one line; erl_tidy *un*tidied that code. FYI: Tidier is using the same infrastructure as erl_tidy, namely the erl_prettypr module, and its output is untidied in similar respects, I am afraid. Not much can be done about it, at least until the parser is adapted to also maintain comments and white spaces. Kostis From spawn.think@REDACTED Wed Oct 27 11:46:51 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Wed, 27 Oct 2010 11:46:51 +0200 Subject: [ANN] LinkedIn EUGs (Erlang Users' Groups) Message-ID: Dear all, In order to facilitate better communication and easier collaboration for Erlang User Groups, I'm glad to announce the newly created location-oriented subgroups under Erlang/OTP professionals group . Benefits : You can easily start discussions , share news, announcements and more, all localized and organized professionally. Currently available: EUG-Sweden EUG-UK EUG-USA EUG-Spain We can add up to 20 subgroups. Please if you want to add your EUG to the list, send me an email with number of members you expect. Note : To join any of the EUGs you have to be a member of Erlang/OTP professionals group., so make sure to join if your are not already there. -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From spawn.think@REDACTED Wed Oct 27 12:39:34 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Wed, 27 Oct 2010 12:39:34 +0200 Subject: [Job] Erlang Developer, Netherlands Message-ID: *Position: **Erlang Developer* *Location*: Rotterdam, The Netherlands *You:* Love Erlang, enjoy challenges, not afraid of learning about new technologies and passionate about building great software. *We:* At Nimbuzz Core team, we are working behind the scenes. Our mission is to deliver a scalable, fault tolerant and stable system. Our daily work includes fixing bugs, optimizing performance and developing new features. We are also the 2nd line of defense for production issues. Our core system is based on ejabberd. *Qualifications: * - BSc Computer Sciences or equivalent degree is a must. - 2+ years of Erlang experience is a must. - Unix/Linux Experience is a must. - English fluency is a must. - Ejabberd experience is a huge pre. - Experience with testing frameworks is a pre. - Production support experience is a pre. - Experience with version control systems is a pre. - Java experience is a pre. - C experience is a pre.. - Scrum experience is a pre. *How to Apply:* Please send your CV to rabia@REDACTED, the subject should be : Erlang Developer *More about the company: * *Nimbuzz * -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From zvi.avraham@REDACTED Wed Oct 27 15:10:23 2010 From: zvi.avraham@REDACTED (Zvi) Date: Wed, 27 Oct 2010 06:10:23 -0700 (PDT) Subject: Erlang-based opensource publish-subscribe servers In-Reply-To: References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> <438091.32912.qm@web35908.mail.mud.yahoo.com> Message-ID: <362eacca-fb1e-4464-838a-92487c6fdbc6@c20g2000yqj.googlegroups.com> Hi, Just found about pg module (not pg2). http://erldocs.com/R14B/stdlib/pg.html?i=18&search=pg Looks like it can be used to implement pubsub. For each topic create process group (pg:create/1), then each subsriber's process joins topic's process group (pg:join/2). Publish maps cleanly to pg:send/2. Any ideas, why not to use this module? I guess process groups use registered names, so that's a disadvantage. Another problem is, that docs call this module "experimental". thanks, Zvi On Oct 24, 9:22?pm, "Zvi ." wrote: > no, > I need something lightweight, but able to work in a large cluster (10s of > nodes). > > The ideas I have: > 1. Use ets to store mappings of Subscriber Pids to TopicIDs - this ets will > obviously be a bottleneck. > 2. Have a process per TopicID, which hold list of all it's Subscriber Pids > in it's state. Now the bottleneck is mapping TopicID to it's Pid > 3. Use Ulf's gproc (looks it uses gen_leader abd requires patching Erlang?) > > I currently working on 1. > > Zvi > From banibrata.dutta@REDACTED Wed Oct 27 15:42:32 2010 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Wed, 27 Oct 2010 19:12:32 +0530 Subject: [erlang-questions] Re: Erlang-based opensource publish-subscribe servers In-Reply-To: <362eacca-fb1e-4464-838a-92487c6fdbc6@c20g2000yqj.googlegroups.com> References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> <438091.32912.qm@web35908.mail.mud.yahoo.com> <362eacca-fb1e-4464-838a-92487c6fdbc6@c20g2000yqj.googlegroups.com> Message-ID: But then, why not pg2 ? It says: "In pg, each message is sent to all members in the group. In this module, each message may be sent to one, some, *or all members*. " With the option of sending to all-members, it looks like something that is quite similar to pg, and isn't tagged "experimental" and also more proven thanks to it being used in the disk_log module !! On Wed, Oct 27, 2010 at 6:40 PM, Zvi wrote: > Hi, > > Just found about pg module (not pg2). > http://erldocs.com/R14B/stdlib/pg.html?i=18&search=pg > > Looks like it can be used to implement pubsub. > For each topic create process group (pg:create/1), then each > subsriber's process joins topic's process group (pg:join/2). > Publish maps cleanly to pg:send/2. > > Any ideas, why not to use this module? > I guess process groups use registered names, so that's a > disadvantage. > Another problem is, that docs call this module "experimental". > > thanks, > Zvi > > On Oct 24, 9:22 pm, "Zvi ." wrote: > > no, > > I need something lightweight, but able to work in a large cluster (10s of > > nodes). > > > > The ideas I have: > > 1. Use ets to store mappings of Subscriber Pids to TopicIDs - this ets > will > > obviously be a bottleneck. > > 2. Have a process per TopicID, which hold list of all it's Subscriber > Pids > > in it's state. Now the bottleneck is mapping TopicID to it's Pid > > 3. Use Ulf's gproc (looks it uses gen_leader abd requires patching > Erlang?) > > > > I currently working on 1. > > > > Zvi > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- regards, Banibrata http://www.linkedin.com/in/bdutta http://twitter.com/edgeliving From mail@REDACTED Wed Oct 27 16:00:33 2010 From: mail@REDACTED (Tim Fletcher) Date: Wed, 27 Oct 2010 07:00:33 -0700 (PDT) Subject: non-trivial supervisor idioms? -- An open-source plea! In-Reply-To: <4CC7EB77.9040106@aist.go.jp> References: <4CA0AAD3.3030209@erlang-solutions.com> <36BEDEE4-D66A-4FD4-9793-B0B444C35236@erlang-solutions.com> <4CC7EB77.9040106@aist.go.jp> Message-ID: <77d8ed1f-1bff-40ad-bf85-07893bac3d72@f33g2000yqh.googlegroups.com> > I'd rather see an Erlang Cookbook. The Perl and Python ones are wonderful. There's this: http://www.trapexit.org/Category:CookBook Tim From bile@REDACTED Wed Oct 27 16:05:05 2010 From: bile@REDACTED (bile@REDACTED) Date: Wed, 27 Oct 2010 10:05:05 -0400 Subject: [erlang-questions] Re: Erlang-based opensource publish-subscribe servers In-Reply-To: <362eacca-fb1e-4464-838a-92487c6fdbc6@c20g2000yqj.googlegroups.com> References: <9ef7e8ca-da16-45c8-86d0-c830f6561991@u10g2000yqk.googlegroups.com> <438091.32912.qm@web35908.mail.mud.yahoo.com> <362eacca-fb1e-4464-838a-92487c6fdbc6@c20g2000yqj.googlegroups.com> Message-ID: <20101027100505.5beadd78@otis> Speaking of pg. pg:members/1 claims to accept a term() as pg:create/1 but in fact doesn't. It only handles pids and atoms. Other functions have the same issue. Meaning you can create a process group which can't be joined or queried. On Wed, 27 Oct 2010 06:10:23 -0700 (PDT) Zvi wrote: > Hi, > > Just found about pg module (not pg2). > http://erldocs.com/R14B/stdlib/pg.html?i=18&search=pg > > Looks like it can be used to implement pubsub. > For each topic create process group (pg:create/1), then each > subsriber's process joins topic's process group (pg:join/2). > Publish maps cleanly to pg:send/2. > > Any ideas, why not to use this module? > I guess process groups use registered names, so that's a > disadvantage. > Another problem is, that docs call this module "experimental". > > thanks, > Zvi > > On Oct 24, 9:22?pm, "Zvi ." wrote: > > no, > > I need something lightweight, but able to work in a large cluster > > (10s of nodes). > > > > The ideas I have: > > 1. Use ets to store mappings of Subscriber Pids to TopicIDs - this > > ets will obviously be a bottleneck. > > 2. Have a process per TopicID, which hold list of all it's > > Subscriber Pids in it's state. Now the bottleneck is mapping > > TopicID to it's Pid 3. Use Ulf's gproc (looks it uses gen_leader > > abd requires patching Erlang?) > > > > I currently working on 1. > > > > Zvi > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From paul.joseph.davis@REDACTED Wed Oct 27 16:19:18 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Wed, 27 Oct 2010 10:19:18 -0400 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> Message-ID: On Wed, Oct 27, 2010 at 3:46 AM, Edmond Begumisa wrote: > Hi, > > I hope the Couch team isn't planning on doing this by default -- something > about it makes me nervous... > > When CouchDB is on it's own, it might not be alarming/noticeable, but I'm > using CouchDB "embedded" in a wider Erlang/OTP application stack (i.e. where > Couch is just one of many OTP apps running in the *SAME* VM -- I have a few > hacks for avoiding socket communication.) I too worry about the potential > for NIF-endowed couch io disturbing the balance of Erlang's scheduling. > > It would be good to see similar benchmarking with the VM concurrently doing > things other than handling couch-related requests (which are implicitly > synchronised in your case.) > > - Edmond - > Edmond, You don't have to worry about us changing away from the default configuration without making sure that different types of loads are either unaffected for similarly improved. I've already had to address this specific issue when integrating Emonk in an experimental branch so it was the first thing I noticed about the file descriptor patch. Out of curiosity, do you have another Erlang app that would be a good candidate for using to test that other parts of the VM remain responsive. I was going to suggest various parts of CouchDB that don't touch IO as a smoke screen, but an app doing something real that we can measure in and out of couch and with and without the new file io would be a good help. Paul Davis > On Mon, 25 Oct 2010 07:59:11 +1100, Joel Reymont wrote: > >> >> On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote: >> >>> I wonder how responsive the system is to other events when running the >>> benchmark. >> >> The benchmark simulates several hundred clients hitting a (mochiweb) web >> server to read and write couchdb (json) documents. The system seems to stay >> -highly- responsive, 10x so compared to the same system not using NIFs at >> all. >> >> If low write response time is taken as a measure of system responsiveness >> then the first graph shows that the responsiveness of the system has >> increased dramatically. The write response here is the take taken to process >> a web request to write a couch document. >> >> --- >> http://twitter.com/wagerlabs >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From chandrashekhar.mullaparthi@REDACTED Wed Oct 27 17:06:08 2010 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Wed, 27 Oct 2010 16:06:08 +0100 Subject: [erlang-questions] Re: stop async ibrowse request In-Reply-To: References: Message-ID: On 27 October 2010 06:02, Max Lapshin wrote: > On Wed, Oct 27, 2010 at 2:38 AM, Chandru > wrote: >> I'm incorporating that patch into the next release of ibrowse. I was >> in touch with that author just a few days ago. >> > > Great. That patch has one problem: if I start ibrowse as {stream_to, > self()}, stream_close doesn't work. Works only is {stream_to, {self(), > once}}. Ok, I'll look into this. Thanks for reporting it. > > Also there are maaany forks about rebar. It would be good to > elliminate them, by merging =) > > Will you accept patch with debian installation package? > Yes, I did look at rebar once and it didn't work for me. Someone is helping me out with this. In case that doesn't work out, I'll ask for your help. cheers Chandru From vincenzo.di.somma@REDACTED Wed Oct 27 17:32:07 2010 From: vincenzo.di.somma@REDACTED (Vincenzo Di Somma) Date: Wed, 27 Oct 2010 17:32:07 +0200 Subject: Simple port forwarding tool Message-ID: <1288193527.2975.28.camel@cyclone> Hello, newby question, can you suggest me a port forwarding tool/snippet/example? Thanks, vds -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From rgowka1@REDACTED Wed Oct 27 17:40:43 2010 From: rgowka1@REDACTED (rgowka1) Date: Wed, 27 Oct 2010 11:40:43 -0400 Subject: Browser based bittorrent application in Erlang? Message-ID: Hi - What would you use to develop a Browser based bittorrent application in Erlang. Something along the lines of http://www.bitlet.org. I am guessing that, Bitlet uses a Java Applet running in the browser to send and receive data. What would be a more suited technology to use with Erlang? Is there something in Erlang that could be used to run a small VM in the browser? thanks, From matthias@REDACTED Wed Oct 27 18:06:36 2010 From: matthias@REDACTED (Matthias Lang) Date: Wed, 27 Oct 2010 18:06:36 +0200 Subject: [erlang-questions] Simple port forwarding tool In-Reply-To: <1288193527.2975.28.camel@cyclone> References: <1288193527.2975.28.camel@cyclone> Message-ID: <20101027160636.GA4576@corelatus.se> On Wednesday, October 27, Vincenzo Di Somma wrote: > Hello, > newby question, can you suggest me a port forwarding > tool/snippet/example? It only does TCP port forwarding, but maybe that's enough: http://github.com/matthiasl/eproxy The forwarding is done in loop_body(), see: http://github.com/matthiasl/eproxy/blob/master/reflect.erl Matt From elliot.murphy@REDACTED Wed Oct 27 18:54:44 2010 From: elliot.murphy@REDACTED (Elliot Murphy) Date: Wed, 27 Oct 2010 12:54:44 -0400 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> Message-ID: On Wed, Oct 27, 2010 at 10:19 AM, Paul Davis wrote: > Out of curiosity, do you have another Erlang app that would be a good > candidate for using to test that other parts of the VM remain > responsive. I was going to suggest various parts of CouchDB that don't > touch IO as a smoke screen, but an app doing something real that we > can measure in and out of couch and with and without the new file io > would be a good help. I don't know how realistic it is to run both on the same server, but RabbitMQ is another Erlang app that I deploy in the same sites as CouchDB. -elliot From paul.joseph.davis@REDACTED Wed Oct 27 19:17:32 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Wed, 27 Oct 2010 13:17:32 -0400 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> Message-ID: On Wed, Oct 27, 2010 at 12:54 PM, Elliot Murphy wrote: > On Wed, Oct 27, 2010 at 10:19 AM, Paul Davis > wrote: >> Out of curiosity, do you have another Erlang app that would be a good >> candidate for using to test that other parts of the VM remain >> responsive. I was going to suggest various parts of CouchDB that don't >> touch IO as a smoke screen, but an app doing something real that we >> can measure in and out of couch and with and without the new file io >> would be a good help. > > I don't know how realistic it is to run both on the same server, but > RabbitMQ is another Erlang app that I deploy in the same sites as > CouchDB. > > -elliot > That's a bit more heavy weight than I was thinking. After a bit of Googling I might take a look at writing something with egd to do CPU saturation without being too synthetic. Paul From thomasl_erlang@REDACTED Wed Oct 27 18:58:46 2010 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 27 Oct 2010 09:58:46 -0700 (PDT) Subject: [erlang-questions] non-trivial supervisor idioms? -- An open-source plea! In-Reply-To: References: <4CA0AAD3.3030209@erlang-solutions.com> Message-ID: <249668.71494.qm@web111413.mail.gq1.yahoo.com> ----- Original Message ---- > The obvious answer is to look at some open-source code. But when most >open-source projects have learned Erlang the same way it becomes a case of the >blind leading the blind. Problems that must have already been experienced and >dealt with at Ericsson are repeated. A large production-ready open-source >project written by battle-hardened experienced Erlang programmers would really >fill in that void. Something big and sufficiently complex like a soft-switch or >something. > Hi Edmond You can find a lot of useful stuff in the Erlang/OTP distribution itself. (Or read and learn from the plentiful open source out there, rabbitmq, riak, ejabberd, yaws, mochiweb, etc etc.) Regarding code quality, I'd say after having seen a large fraction of the "old school" industrial-size erlang products that there's plenty of blindness in there too. Most of those programmers were just regular guys. Interrogating Ulf Wiger is probably your best bet in this area. Best regards, Thomas From boris.muehmer@REDACTED Wed Oct 27 19:46:03 2010 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Wed, 27 Oct 2010 19:46:03 +0200 Subject: [erlang-questions] Re: non-trivial supervisor idioms? -- An open-source plea! In-Reply-To: <77d8ed1f-1bff-40ad-bf85-07893bac3d72@f33g2000yqh.googlegroups.com> References: <4CA0AAD3.3030209@erlang-solutions.com> <36BEDEE4-D66A-4FD4-9793-B0B444C35236@erlang-solutions.com> <4CC7EB77.9040106@aist.go.jp> <77d8ed1f-1bff-40ad-bf85-07893bac3d72@f33g2000yqh.googlegroups.com> Message-ID: 2010/10/27 Tim Fletcher : >> I'd rather see an Erlang Cookbook. The Perl and Python ones are wonderful. > > There's this: http://www.trapexit.org/Category:CookBook Thr current result is: phpBB : Critical Error Could not connect to the database Hmmm... sometimes I prefer paper-books. - boris From rgowka1@REDACTED Wed Oct 27 20:24:58 2010 From: rgowka1@REDACTED (rgowka1) Date: Wed, 27 Oct 2010 14:24:58 -0400 Subject: [erlang-questions] map over bitstring In-Reply-To: <4CC55A9D.1030708@amberbio.com> References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> <9F20C0E1-3B67-4761-B228-61148B0F30F4@rogvall.se> <4CC17A48.7020102@amberbio.com> <4CC54E1C.6070102@cs.ntua.gr> <4CC55A9D.1030708@amberbio.com> Message-ID: Hi - How can I convert a bitstring that has only characters $1 and $0 into a binary. <<0,1,0,0,1,0,0,1,0>> to << 2#0100100100 >> to << 292 >> something similar to io_lib:fread("~2u", [$0,$1,$0,$0,$1,$0,$0,$1,$0]) but that works on bitstring instead of string. thanks. On Mon, Oct 25, 2010 at 6:23 AM, Morten Krogh wrote: > Kostis, > > I followed your suggestion and compiled it to native code with erlc +native > zip2.erl, which improves the runtime for > both methods. > > The run times are, for a 100 MB binary, and the map function X -> X + 1 hard > coded into the module and into the map method itself (which is not quite > realistic): > > Not native: > > comprehension : ? ? 10 s > my own bit syntax function : 14s > > native: > > comprehension: 3.7 s > my own bit syntax function: 6 s > > So, yes native is an improvement! > > Morten. > > > > On 10/25/10 11:30 AM, Kostis Sagonas wrote: >> >> Morten Krogh wrote: >>> >>> Sorry, in my recent reply to Tony I tested the comprehension on the >>> constant ><<"B"">>. >>> >>> Putting everything in the module and comparing the same mapped function >>> ,f , the result is that the comprehension is 30% faster than my matching >>> based map function. >> >> Morten, >> >> You may also want to try compiling the file to native code and report >> whether or how this improves/changes the results. >> >> Cheers, >> Kostis > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kenneth.lundin@REDACTED Wed Oct 27 21:13:26 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 27 Oct 2010 21:13:26 +0200 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> Message-ID: My and the OTP teams view in this matter is: - Reimplementing standard functionality already written in C (but as a driver and with asynch thread support) as NIFs is generally a bad idea) - Implementing potentially blocking function calls with NIFs is a bad idea. - You should have VERY strong reasons for writing NIFs at all. It is a big point in not writing anything in C if it can be avoided. - The implementation of NIFs is more modern than the driver concept and among other things the passing of data between Erlang and C-code is more efficient for NIFs than for drivers. The driver concept does still have its place and advantages especially for handling external asynchronous input to Erlang processes. We plan to improve the driver mechanisms and make solutions from the NIFs to be available when writing drivers as well. If it is correct that NIFs for file operations is 2 times faster than the current file operations in raw mode we will do something about it because that difference is not justified. But first we must investigate if that really is the case and where the performance is lost. /Kenneth Erlang/OTP Ericsson On Wed, Oct 27, 2010 at 9:46 AM, Edmond Begumisa wrote: > Hi, > > I hope the Couch team isn't planning on doing this by default -- something > about it makes me nervous... > > When CouchDB is on it's own, it might not be alarming/noticeable, but I'm > using CouchDB "embedded" in a wider Erlang/OTP application stack (i.e. where > Couch is just one of many OTP apps running in the *SAME* VM -- I have a few > hacks for avoiding socket communication.) I too worry about the potential > for NIF-endowed couch io disturbing the balance of Erlang's scheduling. > > It would be good to see similar benchmarking with the VM concurrently doing > things other than handling couch-related requests (which are implicitly > synchronised in your case.) > > - Edmond - > > On Mon, 25 Oct 2010 07:59:11 +1100, Joel Reymont wrote: > >> >> On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote: >> >>> I wonder how responsive the system is to other events when running the >>> benchmark. >> >> The benchmark simulates several hundred clients hitting a (mochiweb) web >> server to read and write couchdb (json) documents. The system seems to stay >> -highly- responsive, 10x so compared to the same system not using NIFs at >> all. >> >> If low write response time is taken as a measure of system responsiveness >> then the first graph shows that the responsiveness of the system has >> increased dramatically. The write response here is the take taken to process >> a web request to write a couch document. >> >> --- >> http://twitter.com/wagerlabs >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > From tony@REDACTED Wed Oct 27 21:15:27 2010 From: tony@REDACTED (Tony Rogvall) Date: Wed, 27 Oct 2010 21:15:27 +0200 Subject: [erlang-questions] map over bitstring In-Reply-To: References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> <9F20C0E1-3B67-4761-B228-61148B0F30F4@rogvall.se> <4CC17A48.7020102@amberbio.com> <4CC54E1C.6070102@cs.ntua.gr> <4CC55A9D.1030708@amberbio.com> Message-ID: <6A8B3626-7A81-4B2D-B86E-3BD692AD6185@rogvall.se> I love the one-liners (can't help it) Close to what you need? << <<(X-$0):1>> || <> <= <<$1,$0,$0,$1,$0,$0,$1,$0>> >>. How many bits did you need ;-) 8, 9 or 10 ? << 292 >> is truncated into <<36>> (or <<"$">>) since it's more bits than 8. Ps Start an Erlang shell (erl) and poke your expression in, and see what the results are. /Tony On 27 okt 2010, at 20.24, rgowka1 wrote: > Hi - > > How can I convert a bitstring that has only characters $1 and $0 into a binary. > > <<0,1,0,0,1,0,0,1,0>> to << 2#0100100100 >> to << 292 >> > > something similar to io_lib:fread("~2u", [$0,$1,$0,$0,$1,$0,$0,$1,$0]) > but that works on bitstring instead of string. > > thanks. > > On Mon, Oct 25, 2010 at 6:23 AM, Morten Krogh wrote: >> Kostis, >> >> I followed your suggestion and compiled it to native code with erlc +native >> zip2.erl, which improves the runtime for >> both methods. >> >> The run times are, for a 100 MB binary, and the map function X -> X + 1 hard >> coded into the module and into the map method itself (which is not quite >> realistic): >> >> Not native: >> >> comprehension : 10 s >> my own bit syntax function : 14s >> >> native: >> >> comprehension: 3.7 s >> my own bit syntax function: 6 s >> >> So, yes native is an improvement! >> >> Morten. >> >> >> >> On 10/25/10 11:30 AM, Kostis Sagonas wrote: >>> >>> Morten Krogh wrote: >>>> >>>> Sorry, in my recent reply to Tony I tested the comprehension on the >>>> constant ><<"B"">>. >>>> >>>> Putting everything in the module and comparing the same mapped function >>>> ,f , the result is that the comprehension is 30% faster than my matching >>>> based map function. >>> >>> Morten, >>> >>> You may also want to try compiling the file to native code and report >>> whether or how this improves/changes the results. >>> >>> Cheers, >>> Kostis >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > "Have run Make so many times I dunno what's installed anymore" From dmercer@REDACTED Wed Oct 27 21:17:12 2010 From: dmercer@REDACTED (David Mercer) Date: Wed, 27 Oct 2010 14:17:12 -0500 Subject: [erlang-questions] map over bitstring In-Reply-To: References: <20101022074945.GB16824@erix.ericsson.se> <4CC15879.2060904@amberbio.com> <9F20C0E1-3B67-4761-B228-61148B0F30F4@rogvall.se> <4CC17A48.7020102@amberbio.com> <4CC54E1C.6070102@cs.ntua.gr> <4CC55A9D.1030708@amberbio.com> Message-ID: <013a01cb760b$9016eb10$b044c130$@com> On Wednesday, October 27, 2010, rgowka1 wrote: > How can I convert a bitstring that has only characters $1 and $0 into a > binary. > > <<0,1,0,0,1,0,0,1,0>> to << 2#0100100100 >> to << 292 >> > > something similar to io_lib:fread("~2u", [$0,$1,$0,$0,$1,$0,$0,$1,$0]) > but that works on bitstring instead of string. Not entirely sure what you are trying to do. << 292 >> is an 8-bit byte equivalent to << 36 >> or << "$" >>; <<0,1,0,0,1,0,0,1,0>> is a 9-byte binary; and 2#0100100100 is an integer specified by a 10-digit base-2 number. Assuming that you erroneously added a 0 to the base-2 number, 2#010010010 = 146. You can get that number as follows: Bits = <<0,1,0,0,1,0,0,1,0>>, NumBits = size(Bits), << Val:NumBits >> = << <> || <> <= Bits >>, Val. Cheers, David From joelr1@REDACTED Wed Oct 27 21:28:03 2010 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 27 Oct 2010 20:28:03 +0100 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> Message-ID: <19DA2795-8BE2-4926-8048-3007CECD33E2@gmail.com> Kenneth, The difference on the Mac is 10x for writes. It's at least 2x on Linux. How do you propose to investigate? Sent from my iPhone On 27/10/2010, at 20:13, Kenneth Lundin wrote: > My and the OTP teams view in this matter is: > > - Reimplementing standard functionality already written in C (but as a > driver and with asynch thread support) as NIFs is generally a bad > idea) > - Implementing potentially blocking function calls with NIFs is a bad idea. > - You should have VERY strong reasons for writing NIFs at all. It is a > big point in not writing anything in C if it can be avoided. > - The implementation of NIFs is more modern than the driver concept > and among other things the passing of data between Erlang and C-code > is more efficient for NIFs than for drivers. The driver concept does > still have its place and advantages especially for > handling external asynchronous input to Erlang processes. We plan to > improve the driver mechanisms and make solutions > from the NIFs to be available when writing drivers as well. > > If it is correct that NIFs for file operations is 2 times faster than > the current file operations in raw mode we will do something about it > because that difference is not justified. But first we must > investigate if that really is the case and where the performance is > lost. > > /Kenneth Erlang/OTP Ericsson > > On Wed, Oct 27, 2010 at 9:46 AM, Edmond Begumisa > wrote: >> Hi, >> >> I hope the Couch team isn't planning on doing this by default -- something >> about it makes me nervous... >> >> When CouchDB is on it's own, it might not be alarming/noticeable, but I'm >> using CouchDB "embedded" in a wider Erlang/OTP application stack (i.e. where >> Couch is just one of many OTP apps running in the *SAME* VM -- I have a few >> hacks for avoiding socket communication.) I too worry about the potential >> for NIF-endowed couch io disturbing the balance of Erlang's scheduling. >> >> It would be good to see similar benchmarking with the VM concurrently doing >> things other than handling couch-related requests (which are implicitly >> synchronised in your case.) >> >> - Edmond - >> >> On Mon, 25 Oct 2010 07:59:11 +1100, Joel Reymont wrote: >> >>> >>> On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote: >>> >>>> I wonder how responsive the system is to other events when running the >>>> benchmark. >>> >>> The benchmark simulates several hundred clients hitting a (mochiweb) web >>> server to read and write couchdb (json) documents. The system seems to stay >>> -highly- responsive, 10x so compared to the same system not using NIFs at >>> all. >>> >>> If low write response time is taken as a measure of system responsiveness >>> then the first graph shows that the responsiveness of the system has >>> increased dramatically. The write response here is the take taken to process >>> a web request to write a couch document. >>> >>> --- >>> http://twitter.com/wagerlabs >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> From brobinson@REDACTED Wed Oct 27 21:32:47 2010 From: brobinson@REDACTED (Bryan Robinson [ICG-MKTS]) Date: Wed, 27 Oct 2010 15:32:47 -0400 Subject: Formatting integers with io_lib Message-ID: <4CC87E5F.5080606@atdesk.com> I may have missed or misunderstood something in the io_lib and io library documentation, but is there a way to use io_lib:format() to insert leading zero characters on small integers? For example, in the Erlang shell: {_,Time} = calendar:local_time(). {H,M,S} = Time. Msg = io_lib:format("~w:~w:~w", [H,M,S]"). io_format("~s", Msg). I get "15:2:9". I would like to get back "15:02:09", and of course when the integer is two digits I would not like the leading zero. Is there a similar way to do this like with the C stdlib and the %0n where n=2 formatter? printf("%02d:%02d:%02d", h, m, t); I would appreciate any pointers or tips. Thank you! -- - Bryan From ulf.wiger@REDACTED Wed Oct 27 21:41:13 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 27 Oct 2010 21:41:13 +0200 Subject: [erlang-questions] Formatting integers with io_lib In-Reply-To: <4CC87E5F.5080606@atdesk.com> References: <4CC87E5F.5080606@atdesk.com> Message-ID: <66FCE1C3-0760-4475-A54A-DC37D64F3852@erlang-solutions.com> 1> {H,M,S} = time(). {21,37,1} 2> io:fwrite("~2.2.0w.~2.2.0w.~2.2.0w~n",[H,M,S]). 21.37.01 BR, Ulf W On 27 Oct 2010, at 21:32, Bryan Robinson [ICG-MKTS] wrote: > I may have missed or misunderstood something in the io_lib and io > library documentation, but is there a way to use io_lib:format() to > insert leading zero characters on small integers? For example, in the > Erlang shell: > > {_,Time} = calendar:local_time(). > {H,M,S} = Time. > Msg = io_lib:format("~w:~w:~w", [H,M,S]"). > io_format("~s", Msg). > > I get "15:2:9". I would like to get back "15:02:09", and of course when > the integer is two digits I would not like the leading zero. > > Is there a similar way to do this like with the C stdlib and the %0n > where n=2 formatter? > > printf("%02d:%02d:%02d", h, m, t); > > I would appreciate any pointers or tips. Thank you! > > -- > > - Bryan > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From dizzyd@REDACTED Wed Oct 27 22:18:19 2010 From: dizzyd@REDACTED (Dave Smith) Date: Wed, 27 Oct 2010 14:18:19 -0600 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: <19DA2795-8BE2-4926-8048-3007CECD33E2@gmail.com> References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> <19DA2795-8BE2-4926-8048-3007CECD33E2@gmail.com> Message-ID: On Wed, Oct 27, 2010 at 1:28 PM, Joel Reymont wrote: > Kenneth, > > The difference on the Mac is 10x for writes. It's at least 2x on Linux. Just to be pedantic, that 10x # is for a SSD, no? Otherwise, I've never seen OS X/Darwin win any awards for I/O speed... :) D. From joelr1@REDACTED Wed Oct 27 22:28:48 2010 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 27 Oct 2010 21:28:48 +0100 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> <19DA2795-8BE2-4926-8048-3007CECD33E2@gmail.com> Message-ID: <437DEE88-3652-49F9-A2CE-DF28BC8A1FF4@gmail.com> Writing to SSD, yes. There's no award for speed, it's a relative 10x improvement, Darwin vs Darwin. Sent from my iPhone On 27/10/2010, at 21:18, Dave Smith wrote: > On Wed, Oct 27, 2010 at 1:28 PM, Joel Reymont wrote: >> Kenneth, >> >> The difference on the Mac is 10x for writes. It's at least 2x on Linux. > > Just to be pedantic, that 10x # is for a SSD, no? Otherwise, I've > never seen OS X/Darwin win any awards for I/O speed... :) > > D. From rgowka1@REDACTED Thu Oct 28 02:37:14 2010 From: rgowka1@REDACTED (rgowka1) Date: Wed, 27 Oct 2010 20:37:14 -0400 Subject: zlib compress and uncompress on bit string << 255, 3:2 >>. Message-ID: Hi - I am using zlib:compress and uncompress on bitstring. The expression zlib:uncompress( zlib:compress ( << 255, 3:2 >> )) =:= << 255, 3:2 >> is evaluating to false. zlib:uncompress( zlib:compress ( << 255, 3:2 >> )). <<"?">> << << X >> || << X:1 >> <= <<"?">> >>. <<1,1,0,0,0,0,1,1,1,0,1,1,1,1,1,1>> << << X >> || << X:1 >> <= << 255, 3:2 >> >>. <<1,1,1,1,1,1,1,1,1,1>> Is this a bug with the zlib library or am I missing something basic here? thanks From kostis@REDACTED Thu Oct 28 02:54:01 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 28 Oct 2010 03:54:01 +0300 Subject: [erlang-questions] zlib compress and uncompress on bit string << 255, 3:2 >>. In-Reply-To: References: Message-ID: <4CC8C9A9.8070500@cs.ntua.gr> rgowka1 wrote: > Hi - > > I am using zlib:compress and uncompress on bitstring. > > The expression zlib:uncompress( zlib:compress ( << 255, 3:2 >> )) =:= > << 255, 3:2 >> is evaluating to false. > > zlib:uncompress( zlib:compress ( << 255, 3:2 >> )). > <<"?">> > > << << X >> || << X:1 >> <= <<"?">> >>. > <<1,1,0,0,0,0,1,1,1,0,1,1,1,1,1,1>> > > << << X >> || << X:1 >> <= << 255, 3:2 >> >>. > <<1,1,1,1,1,1,1,1,1,1>> > > Is this a bug with the zlib library or am I missing something basic here? What exactly makes you consider it a bug? The manual clearly mentions that zlib:uncompress/1 returns a binary(). Why do you expect that equality of this term with a bitstring() which is not a binary() should succeed? Kostis From rgowka1@REDACTED Thu Oct 28 03:15:00 2010 From: rgowka1@REDACTED (rgowka1) Date: Wed, 27 Oct 2010 21:15:00 -0400 Subject: [erlang-questions] zlib compress and uncompress on bit string << 255, 3:2 >>. In-Reply-To: <4CC8C9A9.8070500@cs.ntua.gr> References: <4CC8C9A9.8070500@cs.ntua.gr> Message-ID: Intuitively, when one uses a loss-less compression algorithm the input should match the decompressed compressed input. In this case I was naively thinking that zlib:uncompress( zlib:compress ( << 255, 3:2 >> )) would give me back << 255, 3:2 >>. I agree with you about the expected input and output being binary and that makes sense. But somehow I still feel that the basic tenet of loss-less compression is not true here. Again all of this is based on my very limited knowledge of Erlang and the theory of compression.. Anyway, are there any functions to compress and uncompress bitstrings? On Wed, Oct 27, 2010 at 8:54 PM, Kostis Sagonas wrote: > rgowka1 wrote: >> >> Hi - >> >> I am using zlib:compress and uncompress on bitstring. >> >> The expression zlib:uncompress( zlib:compress ( << 255, 3:2 >> ?)) =:= >> << 255, 3:2 >> is evaluating to false. >> >> zlib:uncompress( zlib:compress ( << 255, 3:2 >> ?)). >> <<"?">> >> >> << << X >> || << X:1 >> ?<= <<"?">> ?>>. >> <<1,1,0,0,0,0,1,1,1,0,1,1,1,1,1,1>> >> >> ?<< << X >> || << X:1 >> ?<= << 255, 3:2 >> ?>>. >> <<1,1,1,1,1,1,1,1,1,1>> >> >> Is this a bug with the zlib library or am I missing something basic here? > > What exactly makes you consider it a bug? > > The manual clearly mentions that zlib:uncompress/1 returns a binary(). > Why do you expect that equality of this term with a bitstring() which is not > a binary() should succeed? > > Kostis > From ebegumisa@REDACTED Thu Oct 28 09:23:49 2010 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Thu, 28 Oct 2010 18:23:49 +1100 Subject: [erlang-questions] non-trivial supervisor idioms? -- An open-source plea! In-Reply-To: <249668.71494.qm@web111413.mail.gq1.yahoo.com> References: <4CA0AAD3.3030209@erlang-solutions.com> <249668.71494.qm@web111413.mail.gq1.yahoo.com> Message-ID: On Thu, 28 Oct 2010 03:58:46 +1100, Thomas Lindgren wrote: > > > > > ----- Original Message ---- > >> The obvious answer is to look at some open-source code. But when most >> open-source projects have learned Erlang the same way it becomes a >> case of the >> blind leading the blind. Problems that must have already been >> experienced and >> dealt with at Ericsson are repeated. A large production-ready >> open-source >> project written by battle-hardened experienced Erlang programmers >> would really >> fill in that void. Something big and sufficiently complex like a >> soft-switch or >> something. >> > > > Hi Edmond > > You can find a lot of useful stuff in the Erlang/OTP distribution itself. This is true, I have been doing this. For instance the hex-encoding I use now is a modification of the base64 encoder in stdlib. > (Or read and learn from the plentiful open source out there, rabbitmq, > riak, > ejabberd, yaws, mochiweb, etc etc.) > I have found yaws particularly useful, in both design and quality of code. I wonder whether this is by accident! The primary yaws developer is a very experienced Erlanger who has roots in the OTP team. > Regarding code quality, I'd say after having seen a large fraction of > the "old > school" industrial-size erlang products that there's plenty of blindness > in > there too. Most of those programmers were just regular guys. This is an interesting revelation. > Interrogating Ulf > Wiger is probably your best bet in this area. Something else that needs to be mentioned more is the older Erlang book from 96 "Concurrent programming in Erlang". The first few chapters offer very good guides on what to do and what not to do. I discovered that part i pdf on erlang.org very late long after I had picked up some very bad habits from the Internet. - Edmond - > Best regards, > Thomas > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From mazen.harake@REDACTED Thu Oct 28 09:39:29 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Thu, 28 Oct 2010 09:39:29 +0200 Subject: [erlang-questions] Formatting integers with io_lib In-Reply-To: <66FCE1C3-0760-4475-A54A-DC37D64F3852@erlang-solutions.com> References: <4CC87E5F.5080606@atdesk.com> <66FCE1C3-0760-4475-A54A-DC37D64F3852@erlang-solutions.com> Message-ID: <4CC928B1.9060201@erlang-solutions.com> Relevant documentation: http://www.erlang.org/doc/man/io.html#format-2 On 27/10/2010 21:41, Ulf Wiger wrote: > 1> {H,M,S} = time(). > {21,37,1} > 2> io:fwrite("~2.2.0w.~2.2.0w.~2.2.0w~n",[H,M,S]). > 21.37.01 > > BR, > Ulf W > > On 27 Oct 2010, at 21:32, Bryan Robinson [ICG-MKTS] wrote: > >> I may have missed or misunderstood something in the io_lib and io >> library documentation, but is there a way to use io_lib:format() to >> insert leading zero characters on small integers? For example, in the >> Erlang shell: >> >> {_,Time} = calendar:local_time(). >> {H,M,S} = Time. >> Msg = io_lib:format("~w:~w:~w", [H,M,S]"). >> io_format("~s", Msg). >> >> I get "15:2:9". I would like to get back "15:02:09", and of course when >> the integer is two digits I would not like the leading zero. >> >> Is there a similar way to do this like with the C stdlib and the %0n >> where n=2 formatter? >> >> printf("%02d:%02d:%02d", h, m, t); >> >> I would appreciate any pointers or tips. Thank you! >> >> -- >> >> - Bryan >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ebegumisa@REDACTED Thu Oct 28 09:37:25 2010 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Thu, 28 Oct 2010 18:37:25 +1100 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> Message-ID: You could try embedding yaws serving both static and dynamic content (completely unrelated to couch's requests.) You could then also use hovercraft for couch io -- I imagine this would eliminate yaws + couch's mochiweb sharing the socket driver thereby giving a better picture of how NIF io affects the yaws side. - Edmond - On Thu, 28 Oct 2010 01:19:18 +1100, Paul Davis wrote: > On Wed, Oct 27, 2010 at 3:46 AM, Edmond Begumisa > wrote: >> Hi, >> >> I hope the Couch team isn't planning on doing this by default -- >> something >> about it makes me nervous... >> >> When CouchDB is on it's own, it might not be alarming/noticeable, but >> I'm >> using CouchDB "embedded" in a wider Erlang/OTP application stack (i.e. >> where >> Couch is just one of many OTP apps running in the *SAME* VM -- I have a >> few >> hacks for avoiding socket communication.) I too worry about the >> potential >> for NIF-endowed couch io disturbing the balance of Erlang's scheduling. >> >> It would be good to see similar benchmarking with the VM concurrently >> doing >> things other than handling couch-related requests (which are implicitly >> synchronised in your case.) >> >> - Edmond - >> > > Edmond, > > You don't have to worry about us changing away from the default > configuration without making sure that different types of loads are > either unaffected for similarly improved. I've already had to address > this specific issue when integrating Emonk in an experimental branch > so it was the first thing I noticed about the file descriptor patch. > > Out of curiosity, do you have another Erlang app that would be a good > candidate for using to test that other parts of the VM remain > responsive. I was going to suggest various parts of CouchDB that don't > touch IO as a smoke screen, but an app doing something real that we > can measure in and out of couch and with and without the new file io > would be a good help. > > Paul Davis > >> On Mon, 25 Oct 2010 07:59:11 +1100, Joel Reymont >> wrote: >> >>> >>> On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote: >>> >>>> I wonder how responsive the system is to other events when running the >>>> benchmark. >>> >>> The benchmark simulates several hundred clients hitting a (mochiweb) >>> web >>> server to read and write couchdb (json) documents. The system seems to >>> stay >>> -highly- responsive, 10x so compared to the same system not using NIFs >>> at >>> all. >>> >>> If low write response time is taken as a measure of system >>> responsiveness >>> then the first graph shows that the responsiveness of the system has >>> increased dramatically. The write response here is the take taken to >>> process >>> a web request to write a couch document. >>> >>> --- >>> http://twitter.com/wagerlabs >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Thu Oct 28 10:06:25 2010 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Thu, 28 Oct 2010 19:06:25 +1100 Subject: [erlang-questions] couchdb performace 10x: using NIF for file io In-Reply-To: <19DA2795-8BE2-4926-8048-3007CECD33E2@gmail.com> References: <977262664.571287929848268.JavaMail.root@zimbra> <4CC442FD.8090103@diskware.net> <7EF55591-924B-482E-941A-1F6BEC4B8A32@gmail.com> <718AF205-3505-4BF0-BB91-2F7F89857877@gmail.com> <19DA2795-8BE2-4926-8048-3007CECD33E2@gmail.com> Message-ID: As a devout Couch user, what I'd like to see is a pragmatic approach as the couch team investigates this -- possibly targeting an optional configuration with a clear description of the trade-offs to Couch users. For example, the Couch team could advise using this new feature strictly in a private emulator devoted to couch since the benchmarks show that NIF io doesn't seem to have a detrimental effect on the rest of couch. Personally, for my current uses, Couch's existing io performance is sufficient. The use of hovercraft, the Erlang view server and some workarounds to avoid mochiweb have been effective in giving me performance boosts where I need them. That said, it cannot be denied that for a database server, a 2-10 fold increment in io performance is a good enough reason to do things that you should not really be doing or that are potentially disruptive. For some couch users, the pros might outweigh the cons. What I'd like to see is a clear description of what those cons are so I can decide for myself. - Edmond - On Thu, 28 Oct 2010 06:28:03 +1100, Joel Reymont wrote: > Kenneth, > > The difference on the Mac is 10x for writes. It's at least 2x on Linux. > > How do you propose to investigate? > > Sent from my iPhone > > On 27/10/2010, at 20:13, Kenneth Lundin wrote: > >> My and the OTP teams view in this matter is: >> >> - Reimplementing standard functionality already written in C (but as a >> driver and with asynch thread support) as NIFs is generally a bad >> idea) >> - Implementing potentially blocking function calls with NIFs is a bad >> idea. >> - You should have VERY strong reasons for writing NIFs at all. It is a >> big point in not writing anything in C if it can be avoided. >> - The implementation of NIFs is more modern than the driver concept >> and among other things the passing of data between Erlang and C-code >> is more efficient for NIFs than for drivers. The driver concept does >> still have its place and advantages especially for >> handling external asynchronous input to Erlang processes. We plan to >> improve the driver mechanisms and make solutions >> from the NIFs to be available when writing drivers as well. >> >> If it is correct that NIFs for file operations is 2 times faster than >> the current file operations in raw mode we will do something about it >> because that difference is not justified. But first we must >> investigate if that really is the case and where the performance is >> lost. >> >> /Kenneth Erlang/OTP Ericsson >> >> On Wed, Oct 27, 2010 at 9:46 AM, Edmond Begumisa >> wrote: >>> Hi, >>> >>> I hope the Couch team isn't planning on doing this by default -- >>> something >>> about it makes me nervous... >>> >>> When CouchDB is on it's own, it might not be alarming/noticeable, but >>> I'm >>> using CouchDB "embedded" in a wider Erlang/OTP application stack (i.e. >>> where >>> Couch is just one of many OTP apps running in the *SAME* VM -- I have >>> a few >>> hacks for avoiding socket communication.) I too worry about the >>> potential >>> for NIF-endowed couch io disturbing the balance of Erlang's scheduling. >>> >>> It would be good to see similar benchmarking with the VM concurrently >>> doing >>> things other than handling couch-related requests (which are implicitly >>> synchronised in your case.) >>> >>> - Edmond - >>> >>> On Mon, 25 Oct 2010 07:59:11 +1100, Joel Reymont >>> wrote: >>> >>>> >>>> On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote: >>>> >>>>> I wonder how responsive the system is to other events when running >>>>> the >>>>> benchmark. >>>> >>>> The benchmark simulates several hundred clients hitting a (mochiweb) >>>> web >>>> server to read and write couchdb (json) documents. The system seems >>>> to stay >>>> -highly- responsive, 10x so compared to the same system not using >>>> NIFs at >>>> all. >>>> >>>> If low write response time is taken as a measure of system >>>> responsiveness >>>> then the first graph shows that the responsiveness of the system has >>>> increased dramatically. The write response here is the take taken to >>>> process >>>> a web request to write a couch document. >>>> >>>> --- >>>> http://twitter.com/wagerlabs >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>> >>> >>> -- >>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>> -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ingela@REDACTED Thu Oct 28 11:51:51 2010 From: ingela@REDACTED (Ingela Andin) Date: Thu, 28 Oct 2010 11:51:51 +0200 Subject: [erlang-questions] Re: confusing returns from httpc:request/4 In-Reply-To: <864cbd1d-8b1f-41e3-a670-740b9506ee03@g25g2000yqn.googlegroups.com> References: <8ecc3ba2-3f82-4d55-a46f-063f41012ce4@p37g2000pra.googlegroups.com> <3e9cd45e-4e88-42cb-8a7f-b2666b069d2c@w9g2000prc.googlegroups.com> <864cbd1d-8b1f-41e3-a670-740b9506ee03@g25g2000yqn.googlegroups.com> Message-ID: Hi! Thank you for pointing that out, and yes the piepline code and the persistent connection code has a lot of common code so it is quite natural that the bug affects them both. Regards Ingela Erlang/OTP team - Ericsson AB 2010/10/27 amitm : > Sorry, I missed out on stating that to make the problem go away, I > also set the HTTP version as "HTTP/1.0" > > So you may also want to look at the code for handling persistent > connections in general, especially when the server only supports HTTP > 1.0 . > > > ?Amit > > > On Oct 21, 8:54?pm, Ingela Andin wrote: >> Hi! >> >> Yes thank you for the information. I think I have located the bug. It seems >> that to avoid crash reports the return value when receiving a tcp_closed >> was changed and this altered the terminate clause that is run, ?alas breaking >> the automatic retries that should be done if the server terminates a connection >> before serving all pipelined requests. This will be fixed in an >> upcoming release. >> >> Regards Ingela Erlang/OTP-team, ?Ericsson AB >> >> 2010/10/21amitm: >> >> >> >> >> >> >> >> >> Humm ... what settings did you have when you had the problem? >> >> > The function making the call in the situation when the problem >> > manifests itself is: >> >> > mkGetReq2(GetUrl) -> >> > ? ?GetRequest = {GetUrl, []}, >> > ? ?GetHttpOptions = [{autoredirect, true}], >> > ? ?Options = [ {sync,true}, {headers_as_is,true}, {body_format, >> > binary} ], >> > ? ?httpc:request(get, GetRequest, GetHttpOptions, Options). >> >> > No other options were set, which means they run with their default >> > values. >> >> >> Did you set the pipeline timout ? >> >> > No pipeline timeout was set. >> >> >> What webserver did you connect to (apache, yaws, ...)? >> >> > The webserver is one used by Amazon internally on EC2. Curl in verbose >> > mode, as executed by the command >> > "curl -vhttp://169.254.169.254/2009-04-04/meta-data/instance-id" >> > prints out >> >> > ?* About to connect() to 169.254.169.254 port 80 (#0) >> > ?* ? Trying 169.254.169.254... connected >> > ?* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0) >> > ?> GET /2009-04-04/meta-data/instance-id HTTP/1.1 >> > ?> User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/ >> > 0.9.8k zlib/1.2.3.3 libidn/1.15 >> > ?> Host: 169.254.169.254 >> > ?> Accept: */* >> > ?> >> > ?* HTTP 1.0, assume close after body >> > ?< HTTP/1.0 200 OK >> > ?< Content-Type: text/plain >> > ?< Accept-Ranges: bytes >> > ?< ETag: "-1124024050" >> > ?< Last-Modified: Tue, 19 Oct 2010 13:58:54 GMT >> > ?< Content-Length: 10 >> > ?< Connection: close >> > ?< Date: Thu, 21 Oct 2010 13:48:19 GMT >> > ?< Server: EC2ws >> > ?< >> > ?* Closing connection #0 >> >> >> What version of inets? >> >> > erl -v prints >> > "Erlang R14B (erts-5.8.1) [source] [rq:1] [async-threads:0] [kernel- >> > poll:false]" >> >> > Hope it helps, >> >> > ?Amit >> >> > On Oct 21, 6:32?pm, Ingela Andin wrote: >> >> Hi! >> >> >> Humm ... what settings did you have when you had the problem? >> >> Did you set the pipeline timout ? What webserver did you connect to >> >> (apache, yaws, ...)? What >> >> version of inets? If there is some bug in the pipeling or >> >> persistenconnection code >> >> of course we would like to know so that we can fix it. >> >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> >> 2010/10/20amitm: >> >> >> > FWIW, I had a similar problem. It went away when I disabled pipelining >> >> > completely by executing >> >> >> > ? ?ok = httpc:set_options([{max_keep_alive_length, 0}, >> >> > {max_pipeline_length, 0}, {max_sessions, 0} ]), >> >> >> > before my http calls. I used to usually see it on the 4th consecutive >> >> > call to the same host when pipelining was enabled. The URL in question >> >> > in my case was an Amazon EC2 url. >> >> >> > The set of URLs in my case were >> >> >> > "http://169.254.169.254/2009-04-04/meta-data/instance-id", >> >> > "http://169.254.169.254/2009-04-04/meta-data/local-hostname", >> >> > "http://169.254.169.254/2009-04-04/meta-data/local-ipv4", >> >> > "http://169.254.169.254/2009-04-04/meta-data/public-hostname", >> >> > "http://169.254.169.254/2009-04-04/meta-data/public-ipv4", >> >> > "http://169.254.169.254/2009-04-04/meta-data/ami-id", >> >> >> > While executing this set, with pipelining enabled, it used to always >> >> > bomb on the 4th URL, i.e. for 'public-hostname' in 25% of the cases >> >> >> > with pipelining disabled, the problem disappears. >> >> >> > ?Amit >> >> >> > On Oct 13, 2:58?pm, Ingela Andin wrote: >> >> >> 2010/10/13 Chandru : >> >> >> >> > On 13 October 2010 09:53, Ingela Andin wrote: >> >> >> >> Hi! >> >> >> >> >> The only reason that the httpc-client will return the value {error, >> >> >> >> socket_closed_remotely} >> >> >> >> is that the connection is prematurely ?closed by the server. >> >> >> >> Unfortunately ?that something >> >> >> >> is sent on a tcp socket does not mean it has been sent on the wire >> >> >> >> yet, and depending on >> >> >> >> the circumstances it is possible for the socket to be closed before >> >> >> >> all data has been sent >> >> >> >> so occasionally a tcp_close can come before all data is delivered. >> >> >> >> >> When you use ibrowse do you verify that you always get a correct body? >> >> >> >> (compleate body) from what I can tell from the code ibrowse will on a >> >> >> >> tcp_closed ?message always return whatever happens to be in the buffer >> >> >> >> compleate or not. >> >> >> >> > ibrowse does this only in the following cases: >> >> >> > ?* server returned a "Connection: Close" >> >> >> > ?* Server supports HTTP/0.9 or HTTP/1.0 >> >> >> >> > In all other cases, it returns an error to the caller. >> >> >> >> > regards, >> >> >> > Chandru >> >> >> >> Well I do not know if any of these things is true in Dans senario. But >> >> >> I do know that >> >> >> getting the problem with a tcp close message before all data is >> >> >> deliverd is a highly timing dependant problem >> >> >> and if it happens in the server there is nothing inets-client or >> >> >> ibrows can do about it except return an error. >> >> >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> >> >> ________________________________________________________________ >> >> >> erlang-questions (at) erlang.org mailing list. >> >> >> Seehttp://www.erlang.org/faq.html >> >> >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >> >> >> > ________________________________________________________________ >> >> > erlang-questions (at) erlang.org mailing list. >> >> > Seehttp://www.erlang.org/faq.html >> >> > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >> >> >> ________________________________________________________________ >> >> erlang-questions (at) erlang.org mailing list. >> >> Seehttp://www.erlang.org/faq.html >> >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >> >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > Seehttp://www.erlang.org/faq.html >> > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> Seehttp://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Thu Oct 28 15:27:35 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 28 Oct 2010 17:27:35 +0400 Subject: debian & ubuntu R14B Message-ID: Ubuntu package is frozen at R13B04 and Debian on R13. Also, it would be very cool to see builds for 9.10, 10.04 and lenny. Maybe maintainer is here and he just needs help with building it? Or the easiest way for me is to make debian folder for builds for exact platforms, which must be stored near erlyvideo? From brobinson@REDACTED Thu Oct 28 15:54:37 2010 From: brobinson@REDACTED (Bryan Robinson [ICG-MKTS]) Date: Thu, 28 Oct 2010 09:54:37 -0400 Subject: [erlang-questions] Formatting integers with io_lib In-Reply-To: <4CC928B1.9060201@erlang-solutions.com> References: <4CC87E5F.5080606@atdesk.com> <66FCE1C3-0760-4475-A54A-DC37D64F3852@erlang-solutions.com> <4CC928B1.9060201@erlang-solutions.com> Message-ID: <4CC9809D.2060606@atdesk.com> Ulf, Thank you, I plugged that in and it worked. Mazen, Thank you, that was the documentation I was looking at, but I missed the "~F.P.PadModC" part. When I opened your link there it was right before me! Thanks. - Bryan On 10/28/10 03:39, Mazen Harake wrote: > Relevant documentation: > http://www.erlang.org/doc/man/io.html#format-2 > > On 27/10/2010 21:41, Ulf Wiger wrote: >> 1> {H,M,S} = time(). >> {21,37,1} >> 2> io:fwrite("~2.2.0w.~2.2.0w.~2.2.0w~n",[H,M,S]). >> 21.37.01 >> >> BR, >> Ulf W >> >> On 27 Oct 2010, at 21:32, Bryan Robinson [ICG-MKTS] wrote: >> >>> I may have missed or misunderstood something in the io_lib and io >>> library documentation, but is there a way to use io_lib:format() to >>> insert leading zero characters on small integers? For example, in the >>> Erlang shell: >>> >>> {_,Time} = calendar:local_time(). >>> {H,M,S} = Time. >>> Msg = io_lib:format("~w:~w:~w", [H,M,S]"). >>> io_format("~s", Msg). >>> >>> I get "15:2:9". I would like to get back "15:02:09", and of course when >>> the integer is two digits I would not like the leading zero. >>> >>> Is there a similar way to do this like with the C stdlib and the %0n >>> where n=2 formatter? >>> >>> printf("%02d:%02d:%02d", h, m, t); >>> >>> I would appreciate any pointers or tips. Thank you! >>> >>> -- >>> >>> - Bryan >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > -- ////////////////////////////////////// Bryan Robinson Automated Trading Desk, A Citi Company +1.843.789.2287 brobinson@REDACTED Automated Trading Desk, LLC (ATD) is the owner of Automated Trading Desk Financial Services, LLC (AUTO) and Automated Trading Desk Brokerage Services, LLC (ATDB), both FINRA members and Members SIPC. ATD does not offer any brokerage services and is not a FINRA member. All brokerage services, trading functions, execution of order flow and related matters are performed through AUTO and ATDB utilizing ATD's proprietary technology and software. Any reference to ATD trading, ATD trading services, ATD trading performance, ATD orders, we, us, our or other such usage refers to the services and trading activities of AUTO and ATDB utilizing ATD's proprietary technology and software. Periods of market volatility or other system delays may adversely affect trade execution and related services. ////////////////////////////////////// From vincenzo.di.somma@REDACTED Thu Oct 28 16:05:32 2010 From: vincenzo.di.somma@REDACTED (Vincenzo Di Somma) Date: Thu, 28 Oct 2010 16:05:32 +0200 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: References: Message-ID: <1288274732.2975.54.camel@cyclone> We discussed a bit about packaging Erlang at the Erlang Factory in London this year. One very interesting suggestion was to improve the separation of the runtime environment from the development environment. vds On Thu, 2010-10-28 at 17:27 +0400, Max Lapshin wrote: > Ubuntu package is frozen at R13B04 and Debian on R13. Also, it would > be very cool to see builds for 9.10, 10.04 and lenny. > Maybe maintainer is here and he just needs help with building it? > > Or the easiest way for me is to make debian folder for builds for > exact platforms, which must be stored near erlyvideo? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From gleber.p@REDACTED Thu Oct 28 16:07:30 2010 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 28 Oct 2010 16:07:30 +0200 Subject: Looking for an algorithm for reliable broadcast Message-ID: Hello all I'm looking for algorithm of broadcasting events to a group of processes in a distributed setting where nodes may fail. Here are requirements: 1) Speed is the major factor 2) Reliable (no events may be lost, even if it was delivered just to one node) 3) "Weak" consistency: a) Events sent by the same actor are delivered to other actors in the same order as sent b) Events sent by different actors may be delivered to an actor in unspecified order (delivery order may vary at different recipients) even if sent with considerable time difference I'd prefer an algorithm that works in optimistic mode (at most amortized O(n) messages), while falling back to less-efficient mode (used to fix local view and/or retransmit missing events) if errors (missing events, bad ordering, etc.) are detected. Any ideas? I'd be glad for any pointers here. I've found a lot of related algorithms out there, but it seems to be a very active field of research that it's hard to find one which suits the problem best. Have you heard about an efficient "optimistic" algorithm which solves this kind of problem? N.B. Additionally each actor has has to maintain a cache of last N received events, it should be kept consistent (with the same requirements as message delivery) and when new actor joins a group it's cache should to be filled. This problem seems to be related but secondary to the former. Thanks! Gleb Peregud From max.lapshin@REDACTED Thu Oct 28 16:17:38 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 28 Oct 2010 18:17:38 +0400 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: <1288274732.2975.54.camel@cyclone> References: <1288274732.2975.54.camel@cyclone> Message-ID: On Thu, Oct 28, 2010 at 6:05 PM, Vincenzo Di Somma wrote: > We discussed a bit about packaging Erlang at the Erlang Factory in > London this year. > One very interesting suggestion was to improve the separation of the > runtime environment from the development environment. > Maybe it is a good idea to split into erlang and erlang-dev, maybe not, but when there are no fresh packages for months and it still requires some knowledge hidden inside one person to build a package, everything ends in ./configure && make && make install on production server. I tried to build R14B for squeeze, but failed: too complex build setup required. If situation doesn't change, I will have to refuse from building debian package because of no erlang in debian or ubuntu. From dafgar@REDACTED Thu Oct 28 16:30:08 2010 From: dafgar@REDACTED (Sergei Fomin) Date: Thu, 28 Oct 2010 18:30:08 +0400 Subject: crypto rsa encryption Message-ID: <4CC988F0.4050800@yandex-team.ru> Hello. I'm trying to get rsa encryption with crypto module. With PKCS1 padding scheme it works ok, but 'rsa_no_padding' option makes crypto:rsa_public_encrypt to raise: ** exception error: encrypt_failed. My system: otp: R13B03, R13B04, R14B Ubuntu 9.10 OpenSSL 0.9.8g 19 Oct 2007 Any ideas what the problem is? -- Sergei Fomin From joelr1@REDACTED Thu Oct 28 17:13:28 2010 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 28 Oct 2010 16:13:28 +0100 Subject: ei: decoding negative values Message-ID: When using ei, how do I know whether to decode signed or unsigned values? I'm not sure if I should use ei_decode_long or ei_decode_ulong and ei_get_type is not very helpful here. My goal is to display the value that was encoded. Thanks, Joel --- http://twitter.com/wagerlabs From joelr1@REDACTED Thu Oct 28 18:49:29 2010 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 28 Oct 2010 17:49:29 +0100 Subject: why a timestamp 3-tuple Message-ID: I was wondering about the best way to convert the output from erlang:now(), e.g. {A, B, C} to a single number. It appears that I can just do (A * 1000000 + B) * 1000000 + C. What is the advantage to keeping timestamps as a 3-tuple? Thanks, Joel --- http://twitter.com/wagerlabs From mononcqc@REDACTED Thu Oct 28 18:55:55 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 28 Oct 2010 12:55:55 -0400 Subject: [erlang-questions] why a timestamp 3-tuple In-Reply-To: References: Message-ID: If I recall, that representation dates back before Erlang could actually represent it as one large number. I don't think there's much of an advantage with the 3-tuple. On Thu, Oct 28, 2010 at 12:49 PM, Joel Reymont wrote: > I was wondering about the best way to convert the output from erlang:now(), > e.g. {A, B, C} to a single number. It appears that I can just do (A * > 1000000 + B) * 1000000 + C. > > What is the advantage to keeping timestamps as a 3-tuple? > > Thanks, Joel > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From joelr1@REDACTED Thu Oct 28 19:30:24 2010 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 28 Oct 2010 18:30:24 +0100 Subject: cp in erlang traces Message-ID: <19592971-52A6-438E-AA88-E6B38890FA3B@gmail.com> Assume a trace from erlang:trace that looks like this: {trace_ts, Pid, call, {_M, _F, Arity} = Func, {cp, CP}, TS} What does CP stand for here and how would you use it if you were building your own profiler? This does not seem to be documented in the man page for erlang:trace. Thanks, Joel --- http://twitter.com/wagerlabs From thomasl_erlang@REDACTED Thu Oct 28 19:41:03 2010 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 28 Oct 2010 10:41:03 -0700 (PDT) Subject: [erlang-questions] non-trivial supervisor idioms? -- An open-source plea! In-Reply-To: References: <4CA0AAD3.3030209@erlang-solutions.com> <249668.71494.qm@web111413.mail.gq1.yahoo.com> Message-ID: <716334.78231.qm@web111415.mail.gq1.yahoo.com> ----- Original Message ---- > From: Edmond Begumisa > > > Regarding code quality, I'd say after having seen a large fraction of the >"old > > school" industrial-size erlang products that there's plenty of blindness in > > there too. Most of those programmers were just regular guys. > > This is an interesting revelation. Well, Erlang was intended to be easy to learn and use, and these products have had dozens or even hundreds of programmers working on them over many years. It's pretty natural, isn't it? > > Interrogating Ulf > > Wiger is probably your best bet in this area. > > Something else that needs to be mentioned more is the older Erlang book from >96 "Concurrent programming in Erlang". The first few chapters offer very good >guides on what to do and what not to do. I discovered that part i pdf on >erlang.org very late long after I had picked up some very bad habits from the >Internet. Oh, I see -- I thought you were basically looking for advice on "programming erlang in the large". But in this case, let me say you're not alone in liking the good old Erlang book :-) Best, Thomas From trevorw@REDACTED Thu Oct 28 19:40:57 2010 From: trevorw@REDACTED (trevorw@REDACTED) Date: Thu, 28 Oct 2010 19:40:57 +0200 (SAST) Subject: [erlang-questions] crypto rsa encryption In-Reply-To: <4CC988F0.4050800@yandex-team.ru> References: <4CC988F0.4050800@yandex-team.ru> Message-ID: > Hello. > > I'm trying to get rsa encryption with crypto module. With PKCS1 padding > scheme it works ok, but 'rsa_no_padding' option makes > crypto:rsa_public_encrypt to raise: > ** exception error: encrypt_failed. > > My system: > otp: R13B03, R13B04, R14B > Ubuntu 9.10 > OpenSSL 0.9.8g 19 Oct 2007 > > Any ideas what the problem is? > > -- > Sergei Fomin > Hi Sergei With RSA_NO_PADDING the size of your input needs to be exactly the same size as your RSA key, otherwise you need to add padding yourself. Regards, Trevor From anthonym@REDACTED Thu Oct 28 20:48:52 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Thu, 28 Oct 2010 11:48:52 -0700 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: References: <1288274732.2975.54.camel@cyclone> Message-ID: <20101028184852.GA33595@alumni.caltech.edu> It's not debian, but for the last few versions of erlang, I've been updating the EPEL spec file and patches. The nice thing is that several patches became unnecessary between R12 and R13, most of the rest disappeared with R14 (the one remaining patch just disables the formating of man pages). So right now I have a pretty vanilla setup, very simple .spec and a single patch (you can build the whole thing with rpmbuild -ba erlang.spec), I attached the spec and patch (both derived from those included in the EPEL SRPM of R12). It may be that its easier to package when you have a single package for the whole thing? Anyway, maybe it will help in packaging for debian/ubuntu, to see an rpm example? -Anthony On Thu, Oct 28, 2010 at 06:17:38PM +0400, Max Lapshin wrote: > On Thu, Oct 28, 2010 at 6:05 PM, Vincenzo Di Somma > wrote: > > We discussed a bit about packaging Erlang at the Erlang Factory in > > London this year. > > One very interesting suggestion was to improve the separation of the > > runtime environment from the development environment. > > > > Maybe it is a good idea to split into erlang and erlang-dev, maybe > not, but when there are no fresh packages for months and it still > requires some knowledge hidden inside one person to build a package, > everything ends in ./configure && make && make install on production > server. > > I tried to build R14B for squeeze, but failed: too complex build setup > required. If situation doesn't change, I will have to refuse from > building debian package because of no erlang in debian or ubuntu. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- ------------------------------------------------------------------------ Anthony Molinaro -------------- next part -------------- %define ver R14B %define rel 00 Name: erlang Version: %{ver} Release: %{rel}.1%{?dist} Summary: General-purpose programming language and runtime environment Group: Development/Languages License: ERPL URL: http://www.erlang.org Source: http://www.erlang.org/download/otp_src_R14B.tar.gz Source1: http://www.erlang.org/download/otp_doc_html_R14B.tar.gz Source2: http://www.erlang.org/download/otp_doc_man_R14B.tar.gz Patch1: otp-R14B-00-0001-Do-not-format-man-pages.patch BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n) BuildRequires: ncurses-devel BuildRequires: openssl-devel BuildRequires: unixODBC-devel BuildRequires: tcl-devel BuildRequires: tk-devel BuildRequires: gd-devel BuildRequires: jdk BuildRequires: flex BuildRequires: m4 Requires: tk # Added virtual Provides for each erlang module Provides: erlang-appmon = %{version}-%{release} Provides: erlang-asn1 = %{version}-%{release} Provides: erlang-common_test = %{version}-%{release} Provides: erlang-compiler = %{version}-%{release} Provides: erlang-cosEvent = %{version}-%{release} Provides: erlang-cosEventDomain = %{version}-%{release} Provides: erlang-cosFileTransfer = %{version}-%{release} Provides: erlang-cosNotification = %{version}-%{release} Provides: erlang-cosProperty = %{version}-%{release} Provides: erlang-cosTime = %{version}-%{release} Provides: erlang-cosTransactions = %{version}-%{release} Provides: erlang-crypto = %{version}-%{release} Provides: erlang-debugger = %{version}-%{release} Provides: erlang-dialyzer = %{version}-%{release} Provides: erlang-docbuilder = %{version}-%{release} Provides: erlang-edoc = %{version}-%{release} Provides: erlang-erts = %{version}-%{release} Provides: erlang-et = %{version}-%{release} Provides: erlang-eunit = %{version}-%{release} Provides: erlang-gs = %{version}-%{release} Provides: erlang-hipe = %{version}-%{release} Provides: erlang-ic = %{version}-%{release} Provides: erlang-inets = %{version}-%{release} Provides: erlang-inviso = %{version}-%{release} Provides: erlang-kernel = %{version}-%{release} Provides: erlang-megaco = %{version}-%{release} Provides: erlang-mnesia = %{version}-%{release} Provides: erlang-observer = %{version}-%{release} Provides: erlang-odbc = %{version}-%{release} Provides: erlang-orber = %{version}-%{release} Provides: erlang-os_mon = %{version}-%{release} Provides: erlang-otp_mibs = %{version}-%{release} Provides: erlang-parsetools = %{version}-%{release} Provides: erlang-percept = %{version}-%{release} Provides: erlang-pman = %{version}-%{release} Provides: erlang-public_key = %{version}-%{release} Provides: erlang-runtime_tools = %{version}-%{release} Provides: erlang-sasl = %{version}-%{release} Provides: erlang-snmp = %{version}-%{release} Provides: erlang-ssh = %{version}-%{release} Provides: erlang-ssl = %{version}-%{release} Provides: erlang-stdlib = %{version}-%{release} Provides: erlang-syntax_tools = %{version}-%{release} Provides: erlang-test_server = %{version}-%{release} Provides: erlang-toolbar = %{version}-%{release} Provides: erlang-tools = %{version}-%{release} Provides: erlang-tv = %{version}-%{release} Provides: erlang-typer = %{version}-%{release} Provides: erlang-webtool = %{version}-%{release} Provides: erlang-xmerl = %{version}-%{release} %description Erlang is a general-purpose programming language and runtime environment. Erlang has built-in support for concurrency, distribution and fault tolerance. Erlang is used in several large telecommunication systems from Ericsson. %package doc Summary: Erlang documentation Group: Development/Languages %description doc Documentation for Erlang. %prep #%setup -q -n otp_src_%{ver}%{rel} %setup -q -n otp_src_%{ver} %patch1 -p1 -b .manpages %build # WARN: --enable-dynamic-ssl-lib needed for preventing strange messages about missing dependencies on EPEL # see https://bugzilla.redhat.com/458646 and https://bugzilla.redhat.com/499525 %ifarch sparcv9 sparc64 CFLAGS="$RPM_OPT_FLAGS -mcpu=ultrasparc -fno-strict-aliasing" %configure --enable-dynamic-ssl-lib %else CFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing" %configure --enable-dynamic-ssl-lib %endif chmod -R u+w . make %install rm -rf $RPM_BUILD_ROOT make INSTALL_PREFIX=$RPM_BUILD_ROOT install # clean up find $RPM_BUILD_ROOT%{_libdir}/erlang -perm 0775 | xargs chmod 755 find $RPM_BUILD_ROOT%{_libdir}/erlang -name Makefile | xargs chmod 644 find $RPM_BUILD_ROOT%{_libdir}/erlang -name \*.o | xargs chmod 644 find $RPM_BUILD_ROOT%{_libdir}/erlang -name \*.bat | xargs rm -f find $RPM_BUILD_ROOT%{_libdir}/erlang -name index.txt.old | xargs rm -f # doc mkdir -p erlang_doc tar -C erlang_doc -zxf %{SOURCE1} tar -C $RPM_BUILD_ROOT/%{_libdir}/erlang -zxf %{SOURCE2} # make links to binaries #mkdir -p $RPM_BUILD_ROOT/%{_bindir} #cd $RPM_BUILD_ROOT/%{_bindir} #for file in erl erlc escript dialyzer #do # ln -sf ../%{_lib}/erlang/bin/$file . #done # remove buildroot from installed files cd $RPM_BUILD_ROOT/%{_libdir}/erlang sed -i "s|$RPM_BUILD_ROOT||" erts*/bin/{erl,start} releases/RELEASES bin/{erl,start} %clean rm -rf $RPM_BUILD_ROOT %files %defattr(-,root,root) %doc AUTHORS EPLICENCE README.md %{_bindir}/* %{_libdir}/erlang %files doc %defattr(-,root,root) %doc erlang_doc/* %post %{_libdir}/erlang/Install -minimal %{_libdir}/erlang >/dev/null 2>/dev/null %changelog * Wed Sep 29 2010 Anthony Molinaro - R14B-00.1 - modified R13B04 spec to work with R14B * Wed Jul 7 2010 Anthony Molinaro - R13B-04.1 - modified the R12B spec to work with R13B04 * Mon Jun 7 2010 Peter Lemenkov - R12B-5.10 - Added missing virtual provides erlang-erts * Tue May 25 2010 Peter Lemenkov - R12B-5.9 - Use java-1.4.2 only for EL-[45] - Added virtual provides for each erlang module - Small typo fix * Mon Apr 19 2010 Peter Lemenkov - R12B-5.8 - Patches rebased - Added patches 6,7 from trunk - Use %%configure * Tue Apr 21 2009 Debarshi Ray R12B-5.7 - Updated rpath patch. - Fixed configure to respect $RPM_OPT_FLAGS. * Sun Mar 1 2009 Gerard Milmeister - R12B-5.6 - new release R12B-5 - link escript and dialyzer to %{_bindir} * Tue Feb 24 2009 Fedora Release Engineering - R12B-5.5 - Rebuilt for https://fedoraproject.org/wiki/Fedora_11_Mass_Rebuild * Sat Feb 14 2009 Dennis Gilmore - R12B-4.5 - fix sparc arches to compile * Fri Jan 16 2009 Tomas Mraz - R12B-4.4 - rebuild with new openssl * Sat Oct 25 2008 Gerard Milmeister - R12B-4.1 - new release R12B-4 * Fri Sep 5 2008 Gerard Milmeister - R12B-3.3 - fixed sslrpath patch * Thu Jul 17 2008 Tom "spot" Callaway - R12B-3.2 - fix license tag * Sun Jul 6 2008 Gerard Milmeister - R12B-3.1 - new release R12B-3 * Thu Mar 27 2008 Gerard Milmeister - R12B-1.1 - new release R12B-1 * Sat Feb 23 2008 Gerard Milmeister - R12B-0.3 - disable strict aliasing optimization * Mon Feb 18 2008 Fedora Release Engineering - R12B-0.2 - Autorebuild for GCC 4.3 * Sat Dec 8 2007 Gerard Milmeister - R12B-0.1 - new release R12B-0 * Wed Dec 05 2007 Release Engineering - R11B-6 - Rebuild for deps * Sun Aug 19 2007 Gerard Milmeister - R11B-5.3 - fix some permissions * Sat Aug 18 2007 Gerard Milmeister - R11B-5.2 - enable dynamic linking for ssl * Sat Aug 18 2007 Gerard Milmeister - R11B-5.1 - new release R11B-5 * Sat Mar 24 2007 Thomas Fitzsimmons - R11B-2.4 - Require java-1.5.0-gcj-devel for build. * Sun Dec 31 2006 Gerard Milmeister - R11B-2.3 - remove buildroot from installed files * Sat Dec 30 2006 Gerard Milmeister - R11B-2.2 - added patch for compiling with glibc 2.5 * Sat Dec 30 2006 Gerard Milmeister - R11B-2.1 - new version R11B-2 * Mon Aug 28 2006 Gerard Milmeister - R11B-0.3 - Rebuild for FE6 * Wed Jul 5 2006 Gerard Milmeister - R11B-0.2 - add BR m4 * Thu May 18 2006 Gerard Milmeister - R11B-0.1 - new version R11B-0 * Wed May 3 2006 Gerard Milmeister - R10B-10.3 - added patch for run_erl by Knut-H??vard Aksnes * Mon Mar 13 2006 Gerard Milmeister - R10B-10.1 - new version R10B-10 * Thu Dec 29 2005 Gerard Milmeister - R10B-9.1 - New Version R10B-9 * Sat Oct 29 2005 Gerard Milmeister - R10B-8.2 - updated rpath patch * Sat Oct 29 2005 Gerard Milmeister - R10B-8.1 - New Version R10B-8 * Sat Oct 1 2005 Gerard Milmeister - R10B-6.4 - Added tk-devel and tcl-devel to buildreq - Added tk to req * Tue Sep 6 2005 Gerard Milmeister - R10B-6.3 - Remove perl BuildRequires * Tue Aug 30 2005 Gerard Milmeister - R10B-6.2 - change /usr/lib to %%{_libdir} - redirect output in %%post to /dev/null - add unixODBC-devel to BuildRequires - split doc off to erlang-doc package * Sat Jun 25 2005 Gerard Milmeister - R10B-6.1 - New Version R10B-6 * Sun Feb 13 2005 Gerard Milmeister - R10B-3.1 - New Version R10B-3 * Mon Dec 27 2004 Gerard Milmeister - 0:R10B-2-0.fdr.1 - New Version R10B-2 * Wed Oct 6 2004 Gerard Milmeister - 0:R10B-0.fdr.1 - New Version R10B * Thu Oct 16 2003 Gerard Milmeister - 0:R9B-1.fdr.1 - First Fedora release -------------- next part -------------- diff -ur a/erts/etc/unix/Install.src b/erts/etc/unix/Install.src --- a/erts/etc/unix/Install.src 2010-02-19 10:03:27.000000000 -0800 +++ b/erts/etc/unix/Install.src 2010-07-07 04:41:13.000000000 -0700 @@ -139,14 +139,4 @@ cp -p $Name.boot start.boot cp -p ../releases/%I_SYSTEM_VSN%/$Name.script start.script -# -# Fixing the man pages -# - -if [ -d $ERL_ROOT/man ] -then - cd $ERL_ROOT - ./misc/format_man_pages $ERL_ROOT -fi - exit 0 From banibrata.dutta@REDACTED Thu Oct 28 21:01:55 2010 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Fri, 29 Oct 2010 00:31:55 +0530 Subject: OFF-TOPIC: Anyone knows if trapexit.org site is down ? Message-ID: Apologies, as I didn't find any other means to ask this question. Since past 4-5 hours, I find that I cannot reach any of the trapexit.org sites / subsites s.a. planet, forums etc. Anyone has information on whether it's a planned outage ? -- regards, Banibrata http://www.linkedin.com/in/bdutta http://twitter.com/edgeliving From studiomaggi@REDACTED Thu Oct 28 21:24:38 2010 From: studiomaggi@REDACTED (Pietro Maggi) Date: Thu, 28 Oct 2010 21:24:38 +0200 Subject: [erlang-questions] OFF-TOPIC: Anyone knows if trapexit.org site is down ? In-Reply-To: References: Message-ID: >From here it is down. BTW you can check with something like: http://downforeveryoneorjustme.com/trapexit.org regards Pietro On Thu, Oct 28, 2010 at 9:01 PM, Banibrata Dutta wrote: > Apologies, as I didn't find any other means to ask this question. Since past > 4-5 hours, I find that I cannot reach any of the trapexit.org sites / > subsites s.a. planet, forums etc. Anyone has information on whether it's a > planned outage ? > > -- > regards, > Banibrata > http://www.linkedin.com/in/bdutta > http://twitter.com/edgeliving > From attila.r.nohl@REDACTED Thu Oct 28 21:54:19 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 28 Oct 2010 21:54:19 +0200 Subject: [erlang-questions] non-trivial supervisor idioms? -- An open-source plea! In-Reply-To: <716334.78231.qm@web111415.mail.gq1.yahoo.com> References: <4CA0AAD3.3030209@erlang-solutions.com> <249668.71494.qm@web111413.mail.gq1.yahoo.com> <716334.78231.qm@web111415.mail.gq1.yahoo.com> Message-ID: 2010/10/28, Thomas Lindgren : [...] > Well, Erlang was intended to be easy to learn and use, and these products > have > had dozens or even hundreds of programmers working on them over many years. > It's > pretty natural, isn't it? On the other hand I think this is true for nearly all programming languages, the C/Java projects have similar turnabouts. By the way, I had a colleague who has working on AXD, but he said he didn't wrote a single line of new code, he just copied and pasted some earlier written code - and was genuinely surprised that it worked. From robert.virding@REDACTED Thu Oct 28 23:23:24 2010 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 28 Oct 2010 21:23:24 +0000 (GMT) Subject: [erlang-questions] map over bitstring In-Reply-To: Message-ID: <1508410223.18111288301004359.JavaMail.root@zimbra> Seeing binaries contain bytes getting the binary << 292 >> is impossible. You have to put it in more that 8 bits. Robert ----- "rgowka1" wrote: > Hi - > > How can I convert a bitstring that has only characters $1 and $0 into > a binary. > > <<0,1,0,0,1,0,0,1,0>> to << 2#0100100100 >> to << 292 >> > > something similar to io_lib:fread("~2u", > [$0,$1,$0,$0,$1,$0,$0,$1,$0]) > but that works on bitstring instead of string. > > thanks. > > On Mon, Oct 25, 2010 at 6:23 AM, Morten Krogh > wrote: > > Kostis, > > > > I followed your suggestion and compiled it to native code with erlc > +native > > zip2.erl, which improves the runtime for > > both methods. > > > > The run times are, for a 100 MB binary, and the map function X -> X > + 1 hard > > coded into the module and into the map method itself (which is not > quite > > realistic): > > > > Not native: > > > > comprehension : ? ? 10 s > > my own bit syntax function : 14s > > > > native: > > > > comprehension: 3.7 s > > my own bit syntax function: 6 s > > > > So, yes native is an improvement! > > > > Morten. > > > > > > > > On 10/25/10 11:30 AM, Kostis Sagonas wrote: > >> > >> Morten Krogh wrote: > >>> > >>> Sorry, in my recent reply to Tony I tested the comprehension on > the > >>> constant ><<"B"">>. > >>> > >>> Putting everything in the module and comparing the same mapped > function > >>> ,f , the result is that the comprehension is 30% faster than my > matching > >>> based map function. > >> > >> Morten, > >> > >> You may also want to try compiling the file to native code and > report > >> whether or how this improves/changes the results. > >> > >> Cheers, > >> Kostis > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From spawn.think@REDACTED Fri Oct 29 00:39:02 2010 From: spawn.think@REDACTED (Ahmed Omar) Date: Fri, 29 Oct 2010 00:39:02 +0200 Subject: [erlang-questions] cp in erlang traces In-Reply-To: <19592971-52A6-438E-AA88-E6B38890FA3B@gmail.com> References: <19592971-52A6-438E-AA88-E6B38890FA3B@gmail.com> Message-ID: *CP*The continuation pointer, i.e. the return address for the current call. Usually useless for other than runtime system developers. This may be followed by the function into which the CP points, which is the function calling the current function. http://www.erlang.org/doc/apps/erts/crash_dump.html#id68876 i think, someone correct me please, fprof does already use it actually to determine the order of the call in the stack. On Thu, Oct 28, 2010 at 7:30 PM, Joel Reymont wrote: > Assume a trace from erlang:trace that looks like this: > > {trace_ts, Pid, call, {_M, _F, Arity} = Func, {cp, CP}, TS} > > What does CP stand for here and how would you use it if you were building > your own profiler? > > This does not seem to be documented in the man page for erlang:trace. > > Thanks, Joel > > --- > http://twitter.com/wagerlabs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From kino@REDACTED Fri Oct 29 04:42:20 2010 From: kino@REDACTED (satoshi kinoshita) Date: Fri, 29 Oct 2010 11:42:20 +0900 Subject: base64:mime_decode/1 should strip invalid equal signes? In-Reply-To: <1288318788.375.ezmlm@erlang.org> References: <1288318788.375.ezmlm@erlang.org> Message-ID: Hi, According to the reference manual of base64 module, I'm expecting mime_decode/1 strips "illegal characters". It looks fine for stripping "illegal characters" for most of bad characters. But it does not strip bad equals. I dont' know this is a bug or not. Invalid equal signs are not supposed to be stripped as "illegal characters"? ------------------------------------------------------------------------------- http://www.erlang.org/doc/man/base64.html "mime_decode/1 and mime_decode_to_string/1 strips away illegal characters, while decode/1 and decode_to_string/1 only strips away whitespace characters." ------------------------------------------------------------------------------- [kino@REDACTED ~] erl Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> base64:encode("abc"). <<"YWJj">> 2> base64:mime_decode(<<"YWJj">>). <<"abc">> 3> base64:mime_decode(<<"YWJj=">>). ** exception error: no match of right hand side value <<"abc">> in function base64:mime_decode_binary/2 4> base64:mime_decode(<<"YWJj==">>). ** exception error: no match of right hand side value <<"abc">> in function base64:mime_decode_binary/2 5> base64:mime_decode(<<"YW.Jj">>). <<"abc">> 6> base64:mime_decode(<<"YW..Jj">>). <<"abc">> 7> base64:mime_decode(<<"YW...Jj">>). <<"abc">> 8> base64:mime_decode(<<"YW...Jj....">>). <<"abc">> From banibrata.dutta@REDACTED Fri Oct 29 05:37:21 2010 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Fri, 29 Oct 2010 09:07:21 +0530 Subject: [erlang-questions] OFF-TOPIC: Anyone knows if trapexit.org site is down ? In-Reply-To: References: Message-ID: Thanks Pietro, for confirming that it's down too, and a very useful link. The HTTP GET succeeds, but in reality it contains an error message, so health problems seem to still plague the site... phpBB : *Critical Error* Could not connect to the database Hope someone from trapexit notices it. On Fri, Oct 29, 2010 at 12:54 AM, Pietro Maggi wrote: > From here it is down. > BTW you can check with something like: > > http://downforeveryoneorjustme.com/trapexit.org > > regards > Pietro > > On Thu, Oct 28, 2010 at 9:01 PM, Banibrata Dutta > wrote: > > Apologies, as I didn't find any other means to ask this question. Since > past > > 4-5 hours, I find that I cannot reach any of the trapexit.org sites / > > subsites s.a. planet, forums etc. Anyone has information on whether it's > a > > planned outage ? > > > > -- > > regards, > > Banibrata > > http://www.linkedin.com/in/bdutta > > http://twitter.com/edgeliving > > > -- regards, Banibrata http://www.linkedin.com/in/bdutta http://twitter.com/edgeliving From mjtruog@REDACTED Fri Oct 29 05:56:04 2010 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 28 Oct 2010 20:56:04 -0700 Subject: [erlang-questions] OFF-TOPIC: Anyone knows if trapexit.org site is down ? In-Reply-To: References: Message-ID: <4CCA45D4.9040906@gmail.com> Has there been effort to host trapexit running with an Erlang framework? possibly with PHP integration? On 10/28/2010 08:37 PM, Banibrata Dutta wrote: > Thanks Pietro, for confirming that it's down too, and a very useful link. > > The HTTP GET succeeds, but in reality it contains an error message, so > health problems seem to still plague the site... > > phpBB : *Critical Error* > Could not connect to the database > > Hope someone from trapexit notices it. > > On Fri, Oct 29, 2010 at 12:54 AM, Pietro Maggi wrote: > > >> From here it is down. >> BTW you can check with something like: >> >> http://downforeveryoneorjustme.com/trapexit.org >> >> regards >> Pietro >> >> On Thu, Oct 28, 2010 at 9:01 PM, Banibrata Dutta >> wrote: >> >>> Apologies, as I didn't find any other means to ask this question. Since >>> >> past >> >>> 4-5 hours, I find that I cannot reach any of the trapexit.org sites / >>> subsites s.a. planet, forums etc. Anyone has information on whether it's >>> >> a >> >>> planned outage ? >>> >>> -- >>> regards, >>> Banibrata >>> http://www.linkedin.com/in/bdutta >>> http://twitter.com/edgeliving >>> >>> >> > > > From alessandro.sivieri@REDACTED Fri Oct 29 10:51:08 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Fri, 29 Oct 2010 10:51:08 +0200 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: <20101028184852.GA33595@alumni.caltech.edu> References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> Message-ID: We could try to backport packages from the next Ubuntu release, assuming that it will have the R14B release; the first alpha of 11.04 should come out in a few days, if I remember correctly... -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From matthias@REDACTED Fri Oct 29 12:40:41 2010 From: matthias@REDACTED (Matthias Lang) Date: Fri, 29 Oct 2010 12:40:41 +0200 Subject: limitations of erlang:open_port() and os:cmd() Message-ID: <20101029104041.GA7590@corelatus.se> Hi, I know of a few limitations when running external programs from Erlang, and I know a few workarounds. Does anyone know of limitations and/or workarounds I don't know about? Would be interesting to hear about them. The limitations I know of are: 1. There's no way to do flow control. 2. There's no way to kill a misbehaving process 3. There's no way to send eof _and_ get the process' exit status Expanding on those: -------------------- 1. There's no way to do flow control Example: 1> os:cmd("cat /dev/zero"). Crash dump was written to: erl_crash.dump eheap_alloc: Cannot allocate 1048656 bytes of memory (of type "heap_frag"). Workaround 1.1: 1> os:cmd("cat /dev/zero | netcat -l -p 49152"). User switch command --> s --> c 1> {ok, S} = gen_tcp:connect("localhost", 49152, [{active, once}]). {ok,#Port<0.607>} 2> flush(). Shell got {tcp,#Port<0.607>, [0,0,0,0,0,0,0,0,0,0 Suckage: depends on netcat, I'm not sure which process' exit status you get -------------------- 2. There's no way to kill a misbehaving process Example: 1> os:cmd("wc /dev/zero"). Workaround 2.1: Write a wrapper program which echoes the unix pid before exec()ing the actual command. Workaround 2.2: Write a wrapper program which forks() and copies IO to the child, terminating the child on eof. Suckage: you have to write a wrapper -------------------- 3. There's no way to send eof _and_ get the process' exit status Example: 1> {ok, Bin} = file:read_file("/tmp/mml.tgz"). {ok,<<31,139,8,0,100,129,202,76,0,3,236,93,9,120,20,85, 2> P = open_port({spawn, "tar -xzf -"}, [exit_status, binary]). #Port<0.535> 3> port_command(P, Bin). true 4> flush(). ok 5> port_close(P). true 6> flush(). ok Workaround 3.1: Abuse strace, using knowledge of the unix pid gained somehow: os:cmd("strace -p 7974 -e trace=process"). Workaround 3.2: Go via netcat again, e.g. 2> P = open_port({spawn, "netcat -l -p 49152 | tar -xzf -"}, [exit_status, binary]). 3> {ok, S} = gen_tcp:connect("localhost", 49152, []). 4> gen_tcp:send(S, Bin), gen_tcp:close(S), flush(). Suckage: probably only works on unix, requires netcat Matt From jesper.louis.andersen@REDACTED Fri Oct 29 13:59:19 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 29 Oct 2010 13:59:19 +0200 Subject: [erlang-questions] OFF-TOPIC: Anyone knows if trapexit.org site is down ? In-Reply-To: <4CCA45D4.9040906@gmail.com> References: <4CCA45D4.9040906@gmail.com> Message-ID: On Fri, Oct 29, 2010 at 5:56 AM, Michael Truog wrote: > Has there been effort to host trapexit running with an Erlang > framework? ?possibly with PHP integration? I think, and may be wrong, that trapexit predates the numerous Erlang frameworks popping up lately. My guess is that time resources and constraints are what is keeping the site back from being moved to an Erlang-based installation. Redoing years of work is always hard. Some adventurous soul could try to give one of the Erlang Web servers PHP support however. -- J. From garazdawi@REDACTED Fri Oct 29 17:28:26 2010 From: garazdawi@REDACTED (Lukas Larsson) Date: Fri, 29 Oct 2010 17:28:26 +0200 Subject: [erlang-questions] OFF-TOPIC: Anyone knows if trapexit.org site is down ? In-Reply-To: References: <4CCA45D4.9040906@gmail.com> Message-ID: There is work being done by us (Erlang Solutions) right now to convert it to use the Erlang Web framework. I don't know when that will be ready, or if it has even left the planning stage. Though as there is quite a lot of things to redo and there are not that many ready components which can be just plugged in to get everything working it will be quite a while before the migration will be completed. Trapexit.org seems to be up for me now anyways. Lukas On Fri, Oct 29, 2010 at 1:59 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Fri, Oct 29, 2010 at 5:56 AM, Michael Truog wrote: > > Has there been effort to host trapexit running with an Erlang > > framework? possibly with PHP integration? > > I think, and may be wrong, that trapexit predates the numerous Erlang > frameworks popping up lately. My guess is that time resources and > constraints are what is keeping the site back from being moved to an > Erlang-based installation. Redoing years of work is always hard. Some > adventurous soul could try to give one of the Erlang Web servers PHP > support however. > > -- > J. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From francesco@REDACTED Fri Oct 29 17:30:04 2010 From: francesco@REDACTED (Francesco Cesarini) Date: Fri, 29 Oct 2010 16:30:04 +0100 Subject: [erlang-questions] OFF-TOPIC: Anyone knows if trapexit.org site is down ? In-Reply-To: References: <4CCA45D4.9040906@gmail.com> Message-ID: <4CCAE87C.5050702@erlang-solutions.com> Henrik and Henry from our Uppsala Office are running the last year project term with the computer science department at Uppsala University. Eight final year students are creating a flexible and scalable framework using RabbitMQ, CouchDB, Nitrogen and the Erlang web. The goal of the project is for the students to learn how to gather requirements, and manage a project, pick the tools, etc... not migrate the whole website. We hope the effort will be a step in the right direction. Our initial thoughts are to break up trapexit into smaller and more manageable sites (similar to tryerlang.org), releasing everything as open source, and using trapexit.org as a portal aggregating information form the various sites. The first sprint ends today. We'll let you know if there is anything to report. Rgds, Francesco On 29/10/2010 12:59, Jesper Louis Andersen wrote: > On Fri, Oct 29, 2010 at 5:56 AM, Michael Truog wrote: >> Has there been effort to host trapexit running with an Erlang >> framework? possibly with PHP integration? > I think, and may be wrong, that trapexit predates the numerous Erlang > frameworks popping up lately. My guess is that time resources and > constraints are what is keeping the site back from being moved to an > Erlang-based installation. Redoing years of work is always hard. Some > adventurous soul could try to give one of the Erlang Web servers PHP > support however. > -- Erlang Solutions Ltd. http://www.erlang-solutions.com From dafgar@REDACTED Fri Oct 29 17:52:10 2010 From: dafgar@REDACTED (Sergei Fomin) Date: Fri, 29 Oct 2010 19:52:10 +0400 Subject: [erlang-questions] crypto rsa encryption In-Reply-To: References: <4CC988F0.4050800@yandex-team.ru> Message-ID: <4CCAEDAA.60808@yandex-team.ru> It really works. Thank you. -- Sergei Fomin trevorw@REDACTED wrote: >> Hello. >> >> I'm trying to get rsa encryption with crypto module. With PKCS1 padding >> scheme it works ok, but 'rsa_no_padding' option makes >> crypto:rsa_public_encrypt to raise: >> ** exception error: encrypt_failed. >> >> My system: >> otp: R13B03, R13B04, R14B >> Ubuntu 9.10 >> OpenSSL 0.9.8g 19 Oct 2007 >> >> Any ideas what the problem is? >> >> -- >> Sergei Fomin >> > > Hi Sergei > > With RSA_NO_PADDING the size of your input needs to be exactly the same > size as your RSA key, otherwise you need to add padding yourself. > > Regards, > Trevor > > > From trevorw@REDACTED Fri Oct 29 18:11:04 2010 From: trevorw@REDACTED (Trevor Woollacott) Date: Fri, 29 Oct 2010 18:11:04 +0200 (SAST) Subject: [erlang-questions] crypto rsa encryption In-Reply-To: <4CCAEDAA.60808@yandex-team.ru> References: <4CC988F0.4050800@yandex-team.ru> <4CCAEDAA.60808@yandex-team.ru> Message-ID: > It really works. > Thank you. > > -- > Sergei Fomin > > trevorw@REDACTED wrote: >>> Hello. >>> >>> I'm trying to get rsa encryption with crypto module. With PKCS1 padding >>> scheme it works ok, but 'rsa_no_padding' option makes >>> crypto:rsa_public_encrypt to raise: >>> ** exception error: encrypt_failed. >>> >>> My system: >>> otp: R13B03, R13B04, R14B >>> Ubuntu 9.10 >>> OpenSSL 0.9.8g 19 Oct 2007 >>> >>> Any ideas what the problem is? >>> >>> -- >>> Sergei Fomin >>> >> >> Hi Sergei >> >> With RSA_NO_PADDING the size of your input needs to be exactly the same >> size as your RSA key, otherwise you need to add padding yourself. >> >> Regards, >> Trevor >> >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > Ok, great:) Just a note: if you aren't using one of the usual cryptographic padding schemes and you use RSA_NO_PADDING, then you still need to implement some other secure padding scheme in your application code to make sure that your encrypted value is secure. Regards, Trevor From demeshchuk@REDACTED Sat Oct 30 10:03:52 2010 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Sat, 30 Oct 2010 12:03:52 +0400 Subject: Erlang arithmetics Message-ID: Greetings. I'm writing an article comparing Erlang and Node.js and I stumbled upon the performance question. My initial goal was to compare some basic arithmetics speed, like the total distance between randomly distributed points. So, I have written the following code for Erlang: ===================================================== -module(arith_speed). -export([ test/1 ]). test(N) -> L = lists:seq(1, N), [{X0, Y0} | Points] = [{random:uniform(1000), random:uniform(1000)} || _ <- L], Now = now(), lists:foldl(fun move_to/2, {0, {X0, Y0}}, Points), timer:now_diff(now(), Now). move_to({X, Y}, {Sum, {X0, Y0}}) -> {Sum + math:sqrt((X - X0) * (X - X0) + (Y - Y0) * (Y - Y0)), {X, Y}}. ====================================================== and the following code for Node.js: ====================================================== var a = []; for(var i = 0; i < 1000000; i++) { a[i] = {}; a[i].x = Math.floor(Math.random() * 1000); a[i].y = Math.floor(Math.random() * 1000); } var sum = 0; var start = (new Date()).valueOf(); for(var i = 1; i < 1000000; i++) { var prev = a[i-1]; sum += Math.sqrt((a[i].x - prev.x) * (a[i].x - prev.x) + (a[i].y - prev.y) * (a[i].y - prev.y)); } var end = (new Date()).valueOf(); console.log(end - start); ============================================ There was no special tuning for Erlang and Node, both using the latest versions. But "arith_speed:test(1000000)." from Erlang console and "node test.js" have given me very different results: about 413 milliseconds for Erlang and 124 milliseconds for Node. So, the difference was about 4 times! I tried to change the total number of points, and the overall result remained the same. Both Erlang and V8 (Google's engine that is used by Node) use IEEE 754-2008 implementation, so that's not about float type representation. So, for now I have several probable explanations: 1. I've done something wrong and my tests suck (but that may mean that the difference in performance may be even more significant) 2. Erlang uses type overflow check on each computation to determine if it's time to switch from smallint to bigint. 3. Some more reasons that I don't know about or don't consider. Also, I'm still not sure if this kind of test is good for arithmetics comparison. On one side, it uses only pretty basic operations (summing, multiplying and square root) but on the other side it may involve some special computation mechanisms for Erlang that may slow it down. So, any help in this research is very appreciated. I understand that this involves another platform too, but since Erlang appeared to be slower I want to start from it first. Thanks in advance. -- Best regards, Dmitry Demeshchuk From mk@REDACTED Sat Oct 30 10:52:10 2010 From: mk@REDACTED (Morten Krogh) Date: Sat, 30 Oct 2010 10:52:10 +0200 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: References: Message-ID: <4CCBDCBA.5030602@amberbio.com> Hi Dimitry You need to answer a lot of questions before you can even start the comparison. 1. Should the implementations keep a list of all points, or can they just calculate a running sum and discard the points immediately. Right now, your erlang solution keeps two lists, and your js program keep one object (hash table). 2. Is parallelizing the computation allowed?. If so, this would give erlang an edge since you can parallelize this problem by simple message passing. That would require more work in js. 3. What types do you want for the points. In erlang you use random:uniform(1000) which returns integers, in js Math.random() which returns floats. The type issue also includes the question of checking for overflow, as you mention. 4. Should the program be short and simple? In the extreme case, the erlang program could use nifs. 5. How should they calculate random numbers? There could be a huge difference in speed based on the algorithm of course. Actually, I think a speed test between languages should not involve random numbers, unless you make sure they use the same random number generating algorithm. Cheers, Morten. On 10/30/10 10:03 AM, Dmitry Demeshchuk wrote: > Greetings. > > I'm writing an article comparing Erlang and Node.js and I stumbled > upon the performance question. > > My initial goal was to compare some basic arithmetics speed, like the > total distance between randomly distributed points. So, I have written > the following code for Erlang: > > ===================================================== > > -module(arith_speed). > -export([ > test/1 > ]). > > test(N) -> > L = lists:seq(1, N), > [{X0, Y0} | Points] = [{random:uniform(1000), > random:uniform(1000)} || _<- L], > Now = now(), > lists:foldl(fun move_to/2, {0, {X0, Y0}}, Points), > timer:now_diff(now(), Now). > > move_to({X, Y}, {Sum, {X0, Y0}}) -> > {Sum + math:sqrt((X - X0) * (X - X0) + (Y - Y0) * (Y - Y0)), {X, Y}}. > > ====================================================== > > and the following code for Node.js: > > ====================================================== > > var a = []; > for(var i = 0; i< 1000000; i++) { > a[i] = {}; > a[i].x = Math.floor(Math.random() * 1000); > a[i].y = Math.floor(Math.random() * 1000); > } > > var sum = 0; > > var start = (new Date()).valueOf(); > > for(var i = 1; i< 1000000; i++) { > var prev = a[i-1]; > sum += Math.sqrt((a[i].x - prev.x) * (a[i].x - prev.x) + (a[i].y - > prev.y) * (a[i].y - prev.y)); > } > > var end = (new Date()).valueOf(); > > console.log(end - start); > > ============================================ > > There was no special tuning for Erlang and Node, both using the latest versions. > But "arith_speed:test(1000000)." from Erlang console and "node > test.js" have given me very different results: about 413 milliseconds > for Erlang and 124 milliseconds for Node. So, the difference was about > 4 times! I tried to change the total number of points, and the overall > result remained the same. > > Both Erlang and V8 (Google's engine that is used by Node) use IEEE > 754-2008 implementation, so that's not about float type > representation. So, for now I have several probable explanations: > > 1. I've done something wrong and my tests suck (but that may mean that > the difference in performance may be even more significant) > 2. Erlang uses type overflow check on each computation to determine if > it's time to switch from smallint to bigint. > 3. Some more reasons that I don't know about or don't consider. > > Also, I'm still not sure if this kind of test is good for arithmetics > comparison. On one side, it uses only pretty basic operations > (summing, multiplying and square root) but on the other side it may > involve some special computation mechanisms for Erlang that may slow > it down. > > So, any help in this research is very appreciated. I understand that > this involves another platform too, but since Erlang appeared to be > slower I want to start from it first. > > Thanks in advance. > From bjarne@REDACTED Sat Oct 30 11:04:55 2010 From: bjarne@REDACTED (=?iso-8859-1?Q?Bjarne_D=E4cker?=) Date: Sat, 30 Oct 2010 11:04:55 +0200 Subject: ACM SIGPLAN Erlang Workshop Steering Committee Message-ID: <73EA6145029C4CA8BB9548B1FA081DD1@D2N1LG2J> Dear Erlang Friends The following Steering Committee for the ACM SIGPLAN Erlang Workshops has been created Simon Thompson, chairperson Clara Benac Earle Lars-?ke Fredlund Scott Lystig Fritchie Zol?n Horv?th Francesco Cesarini (adjunct) Bjarne D?cker (adjunct) They are all recent workshop chairpersons or programme chairpersons. The idea is that new chairpersons will join the committee and older ones successively drop out. Up to now the final discussion at a workshop has ended by electing the two chairpersons for the coming workshop. They have then had to turn to their predecessors to learn about thier duties. The Steering Committee will firm up the routines around the workshops and also be able to give new chairpersons better support. The next step is to prepare for a workshop in connection with ICFP September 19-21, 2011 in Tokyo. http://www.icfpconference.org/icfp2011/ Best wishes Bjarne D?cker (manager of CSLab at Ericsson 1984-2002 where Erlang was invented) From masklinn@REDACTED Sat Oct 30 10:59:51 2010 From: masklinn@REDACTED (Masklinn) Date: Sat, 30 Oct 2010 10:59:51 +0200 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: References: Message-ID: <0ADB9D72-DF59-4544-8799-95A4A654534C@masklinn.net> On 2010-10-30, at 10:03 , Dmitry Demeshchuk wrote: > > 1. I've done something wrong and my tests suck (but that may mean that > the difference in performance may be even more significant) > 2. Erlang uses type overflow check on each computation to determine if > it's time to switch from smallint to bigint. > 3. Some more reasons that I don't know about or don't consider. Might not be all of it, but I'll go with third for now: you haven't indicated which Erlang runtime you're using (or your compilation options), so I can only assume it's the BEAM interpreter. V8 is based on a quite modern JIT, and even fairly trivial JITs are extremely efficient on highly repetitive, highly-numerical code[0], and will therefore perform very, very well on your code which is precisely that. BEAM, on the other hand, is a fairly basic interpreter, and the overhead of numerical code is therefore quite large compared to a JIT. I would suggest first checking you're using HiPE. [0] Python's third-party JIT, Psyco, will usually net a 2-4x speedup on "normal" Python code (at the cost of memory), but in algorithmical, numerical code the gains are generally above 10x and can reach two orders of magnitude (100x). You can also see this effect when comparing PyPy with CPython [1] [1] PyPy is a complete Python reimplementation using, among other things, a JIT (which CPython doesn't). In the Alioth shootout[2], the difference between numerical tests and string-based ones is flagrant: PyPy is 4 times faster than CPython on fannkuch-redux or spectral-norm but 10 times slower on reverse-complement or regex-dna (using the exact same code) [3] [2] http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=pypy&lang2=python [3] pidigits doesn't count, as the CPython version uses gmpy From hvjunk@REDACTED Sat Oct 30 11:29:07 2010 From: hvjunk@REDACTED (Hendrik Visage) Date: Sat, 30 Oct 2010 11:29:07 +0200 Subject: Horses for courses (Erlang vs. node.js, etc.) Message-ID: Hi there, One of my pet "peefs" in life, is the language/browser/editor/OS wars. And as such, when a language/browser/OS/editor wants to become everything to everybody, you see the Emacs/Windows/Eclipse/etc. bloat happening (Yes, I *DO* use Emacs, and vi, and cat, and ex and notepad and Eclipse) Now on to Erlang vs. Node.js: Each have it's place, and you have to consider what/where/how you need to deploy something, and based on that you will make your decisions on the language, OS, VM etc. Do not take me wrong, as I will find several places I will not use Erlang, but then there are places I can't use Java either, and javascript have a specific place too. So: Yes, I do like Erlang, and yes I do like to see it improved... BUT I still subscribe to the view that a system/specification is complete, not when you can't add anything anymore, but when you can't remove anything anymore. Hendrik From mk@REDACTED Sat Oct 30 11:55:17 2010 From: mk@REDACTED (Morten Krogh) Date: Sat, 30 Oct 2010 11:55:17 +0200 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: <4CCBDCBA.5030602@amberbio.com> References: <4CCBDCBA.5030602@amberbio.com> Message-ID: <4CCBEB85.8050209@amberbio.com> Sorry, I just realized that you are only timing the inner loop, not the random number generation. But why bother with the random numbers then? Shouldn't multiplication and addition run with the same speed for all values of numbers of a given type? Morten. On 10/30/10 10:52 AM, Morten Krogh wrote: > Hi Dimitry > > You need to answer a lot of questions before you can even start the > comparison. > > 1. Should the implementations keep a list of all points, or can they > just calculate a running sum and discard the points immediately. > Right now, your erlang solution keeps two lists, and your js program > keep one object (hash table). > > 2. Is parallelizing the computation allowed?. If so, this would give > erlang an edge since you can parallelize this problem by simple > message passing. > That would require more work in js. > > 3. What types do you want for the points. In erlang you use > random:uniform(1000) which returns integers, in js Math.random() which > returns floats. > The type issue also includes the question of checking for > overflow, as you mention. > > 4. Should the program be short and simple? In the extreme case, the > erlang program could use nifs. > > 5. How should they calculate random numbers? There could be a huge > difference in speed based on the algorithm of course. > Actually, I think a speed test between languages should not > involve random numbers, unless you make sure they use the same random > number generating algorithm. > > Cheers, > > Morten. > > > > > > On 10/30/10 10:03 AM, Dmitry Demeshchuk wrote: >> Greetings. >> >> I'm writing an article comparing Erlang and Node.js and I stumbled >> upon the performance question. >> >> My initial goal was to compare some basic arithmetics speed, like the >> total distance between randomly distributed points. So, I have written >> the following code for Erlang: >> >> ===================================================== >> >> -module(arith_speed). >> -export([ >> test/1 >> ]). >> >> test(N) -> >> L = lists:seq(1, N), >> [{X0, Y0} | Points] = [{random:uniform(1000), >> random:uniform(1000)} || _<- L], >> Now = now(), >> lists:foldl(fun move_to/2, {0, {X0, Y0}}, Points), >> timer:now_diff(now(), Now). >> >> move_to({X, Y}, {Sum, {X0, Y0}}) -> >> {Sum + math:sqrt((X - X0) * (X - X0) + (Y - Y0) * (Y - Y0)), {X, >> Y}}. >> >> ====================================================== >> >> and the following code for Node.js: >> >> ====================================================== >> >> var a = []; >> for(var i = 0; i< 1000000; i++) { >> a[i] = {}; >> a[i].x = Math.floor(Math.random() * 1000); >> a[i].y = Math.floor(Math.random() * 1000); >> } >> >> var sum = 0; >> >> var start = (new Date()).valueOf(); >> >> for(var i = 1; i< 1000000; i++) { >> var prev = a[i-1]; >> sum += Math.sqrt((a[i].x - prev.x) * (a[i].x - prev.x) + (a[i].y - >> prev.y) * (a[i].y - prev.y)); >> } >> >> var end = (new Date()).valueOf(); >> >> console.log(end - start); >> >> ============================================ >> >> There was no special tuning for Erlang and Node, both using the >> latest versions. >> But "arith_speed:test(1000000)." from Erlang console and "node >> test.js" have given me very different results: about 413 milliseconds >> for Erlang and 124 milliseconds for Node. So, the difference was about >> 4 times! I tried to change the total number of points, and the overall >> result remained the same. >> >> Both Erlang and V8 (Google's engine that is used by Node) use IEEE >> 754-2008 implementation, so that's not about float type >> representation. So, for now I have several probable explanations: >> >> 1. I've done something wrong and my tests suck (but that may mean that >> the difference in performance may be even more significant) >> 2. Erlang uses type overflow check on each computation to determine if >> it's time to switch from smallint to bigint. >> 3. Some more reasons that I don't know about or don't consider. >> >> Also, I'm still not sure if this kind of test is good for arithmetics >> comparison. On one side, it uses only pretty basic operations >> (summing, multiplying and square root) but on the other side it may >> involve some special computation mechanisms for Erlang that may slow >> it down. >> >> So, any help in this research is very appreciated. I understand that >> this involves another platform too, but since Erlang appeared to be >> slower I want to start from it first. >> >> Thanks in advance. >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From mathiasstalas@REDACTED Sat Oct 30 12:44:47 2010 From: mathiasstalas@REDACTED (Mathias) Date: Sat, 30 Oct 2010 12:44:47 +0200 Subject: Erlang re:run regular exp, match problrm Message-ID: Hi there, I'm trying to figure out how Erlangs re:run module works. When executing this:: 1> re:run("", ""). {match,[{0,54}]} I can see that it gives me a match on the complete XML representation {match,[{0,54}]}. But what I really would like to do is for it to give me a subset of matches on each entity similar to {match,[{0,26},{27, 26}]}. so the output would yield something like this: 0-26 gives the first xml entity complete with it's attributes and match 27,26 gives the remaining entity. If anyone can spot why my regexp: is failing and guide me in the right direction closer to find the solution it will be greatly appreciated. I know about xmerl but for my trivial case it seems like overkill. Thx in advance. BR, Mathias Stal?s From erlangdlf@REDACTED Sat Oct 30 13:03:44 2010 From: erlangdlf@REDACTED (dlfen) Date: Sat, 30 Oct 2010 19:03:44 +0800 Subject: [erlang-questions] Erlang re:run regular exp, match problrm In-Reply-To: References: Message-ID: <23BB6CB9-D0EA-4053-9630-627294D6D79B@gmail.com> try this. re:run("", "",[global]). ? 2010-10-30???6:44? Mathias ??? > Hi there, > > I'm trying to figure out how Erlangs re:run module works. > > When executing this:: > 1> re:run(" z=\"14\"/>", ""). > {match,[{0,54}]} > > I can see that it gives me a match on the complete XML representation > {match,[{0,54}]}. > > But what I really would like to do is for it to give me a subset of matches > on each entity similar to {match,[{0,26},{27, 26}]}. > > so the output would yield something like this: > 0-26 gives the first xml entity complete with it's attributes y="2" z="4"/> and > match 27,26 gives the remaining entity. > > If anyone can spot why my regexp: is failing and guide me > in the right direction closer to find the solution it will be greatly > appreciated. > > I know about xmerl but for my trivial case it seems like overkill. > > Thx in advance. > > BR, > Mathias Stal?s From mathiasstalas@REDACTED Sat Oct 30 13:34:08 2010 From: mathiasstalas@REDACTED (Mathias) Date: Sat, 30 Oct 2010 13:34:08 +0200 Subject: [erlang-questions] Erlang re:run regular exp, match problrm In-Reply-To: <23BB6CB9-D0EA-4053-9630-627294D6D79B@gmail.com> References: <23BB6CB9-D0EA-4053-9630-627294D6D79B@gmail.com> Message-ID: Works like a charm! Many thanks dlfen! BR, Mathias On Sat, Oct 30, 2010 at 1:03 PM, dlfen wrote: > try this. > re:run(" z=\"14\"/>", "",[global]). > > > ? 2010-10-30???6:44? Mathias ??? > > > Hi there, > > > > I'm trying to figure out how Erlangs re:run module works. > > > > When executing this:: > > 1> re:run(" > z=\"14\"/>", ""). > > {match,[{0,54}]} > > > > I can see that it gives me a match on the complete XML representation > > {match,[{0,54}]}. > > > > But what I really would like to do is for it to give me a subset of > matches > > on each entity similar to {match,[{0,26},{27, 26}]}. > > > > so the output would yield something like this: > > 0-26 gives the first xml entity complete with it's attributes x="12" > > y="2" z="4"/> and > > match 27,26 gives the remaining entity. > > > > If anyone can spot why my regexp: is failing and guide > me > > in the right direction closer to find the solution it will be greatly > > appreciated. > > > > I know about xmerl but for my trivial case it seems like overkill. > > > > Thx in advance. > > > > BR, > > Mathias Stal?s > > From demeshchuk@REDACTED Sat Oct 30 14:10:36 2010 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Sat, 30 Oct 2010 16:10:36 +0400 Subject: [erlang-questions] Horses for courses (Erlang vs. node.js, etc.) In-Reply-To: References: Message-ID: Let me please explain the situation with my question regarding Node.js and Erlang. The main goal of my article is to show just that you describe. Each platform has its own qualities that make it better for some tasks and worse for the other (or maybe impossible to be used for specific projects). I was going to explain the article purpose and contents in details but after some lines realized that I would just write a short summary of the article. :) In short, it compares both of the platforms and concludes when and where each platform is more convenient. But without any benchmarks, he comparison will definitely seem incomplete. This is a too obvious criteria to just ignore it. That's why I asked my question about arithmetics. It's just a basic test that needs, however, some explanation. On Sat, Oct 30, 2010 at 1:29 PM, Hendrik Visage wrote: > Hi there, > > ?One of my pet "peefs" in life, is the language/browser/editor/OS wars. And > as such, when a language/browser/OS/editor wants to become everything to > everybody, you see the Emacs/Windows/Eclipse/etc. bloat happening (Yes, I > *DO* use Emacs, and vi, and cat, and ex and notepad and Eclipse) > > ?Now on to Erlang vs. Node.js: > > ?Each have it's place, and you have to consider what/where/how you need to > deploy something, and based on that you will make your decisions on the > language, OS, VM etc. > > ?Do not take me wrong, as I will find several places I will not use Erlang, > but then there are places I can't use Java either, and javascript have a > specific place too. > > ?So: Yes, I do like Erlang, and yes I do like to see it improved... BUT I > still subscribe to the view that a system/specification is complete, not > when you can't add anything anymore, but when you can't remove anything > anymore. > > Hendrik > -- Best regards, Dmitry Demeshchuk From demeshchuk@REDACTED Sat Oct 30 14:23:42 2010 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Sat, 30 Oct 2010 16:23:42 +0400 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: <4CCBDCBA.5030602@amberbio.com> References: <4CCBDCBA.5030602@amberbio.com> Message-ID: On Sat, Oct 30, 2010 at 12:52 PM, Morten Krogh wrote: > Hi Dimitry > > You need to answer a lot of questions before you can even start the > comparison. > > 1. Should the implementations keep a list of all points, or can they just > calculate a running sum and discard the points immediately. > Right now, your erlang solution keeps two lists, and your js program keep > one object (hash table). No, garbage collecting time shouldn't be considered, as well as memory usage. I understand that this isn't very fair (for example, one platform may be much faster on garbage collecting but slower on arithmetic operations) but the purpose of the test is to measure some average productivity of basic arithmetic operations. > > 2. Is parallelizing the computation allowed?. If so, this would give erlang > an edge since you can parallelize this problem by simple message passing. > That would require more work in js. No, if I wanted to use it, I would definitely do it. :) But, first of all, Node.js will have a significant overhead on IPC (to run multi-core task you'll have to run several processes and communicate between them). And, that is also important, my goal was to compare arithmetics abilities, not abilities of parallelizing (where Node will be extremely slower). > > 3. What types do you want for the points. In erlang you use > random:uniform(1000) which returns integers, in js Math.random() which > returns floats. > ? ?The type issue also includes the question of checking for overflow, as > you mention. Well, that's why I used Math.floor for JS ;) > > 4. Should the program be short and simple? ?In the extreme case, the erlang > program could use nifs. Sure, I could use a port or even a driver. But the subject of interest is Erlang built-in arithmetics, not C. In my article, I mention that ports can be used for some expensive operations, but these calculations will be limited by the port itself then Erlang performance. > > 5. How should they calculate random numbers? There could be a huge > difference in speed based on the algorithm of course. > ? ?Actually, I think a speed test between languages should not involve > random numbers, unless you make sure they use the same random number > generating algorithm. As you can see, my tests don't measure random numbers generation time. Yes, I was going to try using the same pseudo-random generation algorithm on both platforms for another test (Erlang will use process dictionary for this, which should slow down the randomizer) but, again, my subject there is just arithmetic operations. > > Cheers, > > Morten. > > > > > > On 10/30/10 10:03 AM, Dmitry Demeshchuk wrote: >> >> Greetings. >> >> I'm writing an article comparing Erlang and Node.js and I stumbled >> upon the performance question. >> >> My initial goal was to compare some basic arithmetics speed, like the >> total distance between randomly distributed points. So, I have written >> the following code for Erlang: >> >> ===================================================== >> >> -module(arith_speed). >> -export([ >> ? ? test/1 >> ]). >> >> test(N) -> >> ? ? L = lists:seq(1, N), >> ? ? [{X0, Y0} | Points] = [{random:uniform(1000), >> random:uniform(1000)} || _<- L], >> ? ? Now = now(), >> ? ? lists:foldl(fun move_to/2, {0, {X0, Y0}}, Points), >> ? ? timer:now_diff(now(), Now). >> >> move_to({X, Y}, {Sum, {X0, Y0}}) -> >> ? ? {Sum + math:sqrt((X - X0) * (X - X0) + (Y - Y0) * (Y - Y0)), {X, Y}}. >> >> ====================================================== >> >> and the following code for Node.js: >> >> ====================================================== >> >> var a = []; >> for(var i = 0; i< ?1000000; i++) { >> ? ? a[i] = {}; >> ? ? a[i].x = Math.floor(Math.random() * 1000); >> ? ? a[i].y = Math.floor(Math.random() * 1000); >> } >> >> var sum = 0; >> >> var start = (new Date()).valueOf(); >> >> for(var i = 1; i< ?1000000; i++) { >> ? ? var prev = a[i-1]; >> ? ? sum += Math.sqrt((a[i].x - prev.x) * (a[i].x - prev.x) + (a[i].y - >> prev.y) * (a[i].y - prev.y)); >> } >> >> var end = (new Date()).valueOf(); >> >> console.log(end - start); >> >> ============================================ >> >> There was no special tuning for Erlang and Node, both using the latest >> versions. >> But "arith_speed:test(1000000)." from Erlang console and "node >> test.js" have given me very different results: about 413 milliseconds >> for Erlang and 124 milliseconds for Node. So, the difference was about >> 4 times! I tried to change the total number of points, and the overall >> result remained the same. >> >> Both Erlang and V8 (Google's engine that is used by Node) use IEEE >> 754-2008 implementation, so that's not about float type >> representation. So, for now I have several probable explanations: >> >> 1. I've done something wrong and my tests suck (but that may mean that >> the difference in performance may be even more significant) >> 2. Erlang uses type overflow check on each computation to determine if >> it's time to switch from smallint to bigint. >> 3. Some more reasons that I don't know about or don't consider. >> >> Also, I'm still not sure if this kind of test is good for arithmetics >> comparison. On one side, it uses only pretty basic operations >> (summing, multiplying and square root) but on the other side it may >> involve some special computation mechanisms for Erlang that may slow >> it down. >> >> So, any help in this research is very appreciated. I understand that >> this involves another platform too, but since Erlang appeared to be >> slower I want to start from it first. >> >> Thanks in advance. >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best regards, Dmitry Demeshchuk From mk@REDACTED Sat Oct 30 14:37:52 2010 From: mk@REDACTED (Morten Krogh) Date: Sat, 30 Oct 2010 14:37:52 +0200 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: References: <4CCBDCBA.5030602@amberbio.com> Message-ID: <4CCC11A0.8080004@amberbio.com> Hi Dimitry Yes, I first thought that the random number generation was timed as well, so some of my points are irrelevant. But why don't you compare very short programs instead that just loops or recurses on the number N? like this in js: sum = 0; for (i =0; i < 1000000; i++) { sum += 37*i*(i - 5) + i + 23; } and the same in erlang. Do you need the lists of random numbers for your basic arithmetic? Morten. On 10/30/10 2:23 PM, Dmitry Demeshchuk wrote: > On Sat, Oct 30, 2010 at 12:52 PM, Morten Krogh wrote: >> Hi Dimitry >> >> You need to answer a lot of questions before you can even start the >> comparison. >> >> 1. Should the implementations keep a list of all points, or can they just >> calculate a running sum and discard the points immediately. >> Right now, your erlang solution keeps two lists, and your js program keep >> one object (hash table). > No, garbage collecting time shouldn't be considered, as well as memory > usage. I understand that this isn't very fair (for example, one > platform may be much faster on garbage collecting but slower on > arithmetic operations) but the purpose of the test is to measure some > average productivity of basic arithmetic operations. > >> 2. Is parallelizing the computation allowed?. If so, this would give erlang >> an edge since you can parallelize this problem by simple message passing. >> That would require more work in js. > No, if I wanted to use it, I would definitely do it. :) But, first of > all, Node.js will have a significant overhead on IPC (to run > multi-core task you'll have to run several processes and communicate > between them). And, that is also important, my goal was to compare > arithmetics abilities, not abilities of parallelizing (where Node will > be extremely slower). > >> 3. What types do you want for the points. In erlang you use >> random:uniform(1000) which returns integers, in js Math.random() which >> returns floats. >> The type issue also includes the question of checking for overflow, as >> you mention. > Well, that's why I used Math.floor for JS ;) > >> 4. Should the program be short and simple? In the extreme case, the erlang >> program could use nifs. > Sure, I could use a port or even a driver. But the subject of interest > is Erlang built-in arithmetics, not C. In my article, I mention that > ports can be used for some expensive operations, but these > calculations will be limited by the port itself then Erlang > performance. > >> 5. How should they calculate random numbers? There could be a huge >> difference in speed based on the algorithm of course. >> Actually, I think a speed test between languages should not involve >> random numbers, unless you make sure they use the same random number >> generating algorithm. > As you can see, my tests don't measure random numbers generation time. > Yes, I was going to try using the same pseudo-random generation > algorithm on both platforms for another test (Erlang will use process > dictionary for this, which should slow down the randomizer) but, > again, my subject there is just arithmetic operations. > >> Cheers, >> >> Morten. >> >> >> >> >> >> On 10/30/10 10:03 AM, Dmitry Demeshchuk wrote: >>> Greetings. >>> >>> I'm writing an article comparing Erlang and Node.js and I stumbled >>> upon the performance question. >>> >>> My initial goal was to compare some basic arithmetics speed, like the >>> total distance between randomly distributed points. So, I have written >>> the following code for Erlang: >>> >>> ===================================================== >>> >>> -module(arith_speed). >>> -export([ >>> test/1 >>> ]). >>> >>> test(N) -> >>> L = lists:seq(1, N), >>> [{X0, Y0} | Points] = [{random:uniform(1000), >>> random:uniform(1000)} || _<- L], >>> Now = now(), >>> lists:foldl(fun move_to/2, {0, {X0, Y0}}, Points), >>> timer:now_diff(now(), Now). >>> >>> move_to({X, Y}, {Sum, {X0, Y0}}) -> >>> {Sum + math:sqrt((X - X0) * (X - X0) + (Y - Y0) * (Y - Y0)), {X, Y}}. >>> >>> ====================================================== >>> >>> and the following code for Node.js: >>> >>> ====================================================== >>> >>> var a = []; >>> for(var i = 0; i< 1000000; i++) { >>> a[i] = {}; >>> a[i].x = Math.floor(Math.random() * 1000); >>> a[i].y = Math.floor(Math.random() * 1000); >>> } >>> >>> var sum = 0; >>> >>> var start = (new Date()).valueOf(); >>> >>> for(var i = 1; i< 1000000; i++) { >>> var prev = a[i-1]; >>> sum += Math.sqrt((a[i].x - prev.x) * (a[i].x - prev.x) + (a[i].y - >>> prev.y) * (a[i].y - prev.y)); >>> } >>> >>> var end = (new Date()).valueOf(); >>> >>> console.log(end - start); >>> >>> ============================================ >>> >>> There was no special tuning for Erlang and Node, both using the latest >>> versions. >>> But "arith_speed:test(1000000)." from Erlang console and "node >>> test.js" have given me very different results: about 413 milliseconds >>> for Erlang and 124 milliseconds for Node. So, the difference was about >>> 4 times! I tried to change the total number of points, and the overall >>> result remained the same. >>> >>> Both Erlang and V8 (Google's engine that is used by Node) use IEEE >>> 754-2008 implementation, so that's not about float type >>> representation. So, for now I have several probable explanations: >>> >>> 1. I've done something wrong and my tests suck (but that may mean that >>> the difference in performance may be even more significant) >>> 2. Erlang uses type overflow check on each computation to determine if >>> it's time to switch from smallint to bigint. >>> 3. Some more reasons that I don't know about or don't consider. >>> >>> Also, I'm still not sure if this kind of test is good for arithmetics >>> comparison. On one side, it uses only pretty basic operations >>> (summing, multiplying and square root) but on the other side it may >>> involve some special computation mechanisms for Erlang that may slow >>> it down. >>> >>> So, any help in this research is very appreciated. I understand that >>> this involves another platform too, but since Erlang appeared to be >>> slower I want to start from it first. >>> >>> Thanks in advance. >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > From demeshchuk@REDACTED Sat Oct 30 14:48:49 2010 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Sat, 30 Oct 2010 16:48:49 +0400 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: <4CCC11A0.8080004@amberbio.com> References: <4CCBDCBA.5030602@amberbio.com> <4CCC11A0.8080004@amberbio.com> Message-ID: Hi, Morten You have a point, random numbers aren't necessary. :) My goal was just to have some more or less uniform distribution that can be reached using randomized data. Of course, another variant was to choose some static arithmetic operation. To my own defense, I'll say that despite of the random numbers the results were very similar, so the difference in numbers didn't affect them much. But still you are right, static numbers are a bit more reliable. However, that doesn't answer the question "why so big difference in speed for the two platforms?". It's well-known that Erlang arithmetics is slow but I still cannot find a clear answer that describes why exactly... Hope, some of Erlang fathers will be able to help with this... On Sat, Oct 30, 2010 at 4:37 PM, Morten Krogh wrote: > Hi Dimitry > > Yes, I first thought that the random number generation was timed as well, so > some of my points are irrelevant. > > But why don't you compare very short programs instead that just loops or > recurses on the number N? > > like this in js: > > sum = 0; > for (i =0; i < 1000000; i++) { > ? ?sum += 37*i*(i - 5) + i + 23; > } > > and the same in erlang. Do you need the lists of random numbers for your > basic arithmetic? > > Morten. > > > On 10/30/10 2:23 PM, Dmitry Demeshchuk wrote: >> >> On Sat, Oct 30, 2010 at 12:52 PM, Morten Krogh ?wrote: >>> >>> Hi Dimitry >>> >>> You need to answer a lot of questions before you can even start the >>> comparison. >>> >>> 1. Should the implementations keep a list of all points, or can they just >>> calculate a running sum and discard the points immediately. >>> Right now, your erlang solution keeps two lists, and your js program keep >>> one object (hash table). >> >> No, garbage collecting time shouldn't be considered, as well as memory >> usage. I understand that this isn't very fair (for example, one >> platform may be much faster on garbage collecting but slower on >> arithmetic operations) but the purpose of the test is to measure some >> average productivity of basic arithmetic operations. >> >>> 2. Is parallelizing the computation allowed?. If so, this would give >>> erlang >>> an edge since you can parallelize this problem by simple message passing. >>> That would require more work in js. >> >> No, if I wanted to use it, I would definitely do it. :) But, first of >> all, Node.js will have a significant overhead on IPC (to run >> multi-core task you'll have to run several processes and communicate >> between them). And, that is also important, my goal was to compare >> arithmetics abilities, not abilities of parallelizing (where Node will >> be extremely slower). >> >>> 3. What types do you want for the points. In erlang you use >>> random:uniform(1000) which returns integers, in js Math.random() which >>> returns floats. >>> ? ?The type issue also includes the question of checking for overflow, as >>> you mention. >> >> Well, that's why I used Math.floor for JS ;) >> >>> 4. Should the program be short and simple? ?In the extreme case, the >>> erlang >>> program could use nifs. >> >> Sure, I could use a port or even a driver. But the subject of interest >> is Erlang built-in arithmetics, not C. In my article, I mention that >> ports can be used for some expensive operations, but these >> calculations will be limited by the port itself then Erlang >> performance. >> >>> 5. How should they calculate random numbers? There could be a huge >>> difference in speed based on the algorithm of course. >>> ? ?Actually, I think a speed test between languages should not involve >>> random numbers, unless you make sure they use the same random number >>> generating algorithm. >> >> As you can see, my tests don't measure random numbers generation time. >> Yes, I was going to try using the same pseudo-random generation >> algorithm on both platforms for another test (Erlang will use process >> dictionary for this, which should slow down the randomizer) but, >> again, my subject there is just arithmetic operations. >> >>> Cheers, >>> >>> Morten. >>> >>> >>> >>> >>> >>> On 10/30/10 10:03 AM, Dmitry Demeshchuk wrote: >>>> >>>> Greetings. >>>> >>>> I'm writing an article comparing Erlang and Node.js and I stumbled >>>> upon the performance question. >>>> >>>> My initial goal was to compare some basic arithmetics speed, like the >>>> total distance between randomly distributed points. So, I have written >>>> the following code for Erlang: >>>> >>>> ===================================================== >>>> >>>> -module(arith_speed). >>>> -export([ >>>> ? ? test/1 >>>> ]). >>>> >>>> test(N) -> >>>> ? ? L = lists:seq(1, N), >>>> ? ? [{X0, Y0} | Points] = [{random:uniform(1000), >>>> random:uniform(1000)} || _<- L], >>>> ? ? Now = now(), >>>> ? ? lists:foldl(fun move_to/2, {0, {X0, Y0}}, Points), >>>> ? ? timer:now_diff(now(), Now). >>>> >>>> move_to({X, Y}, {Sum, {X0, Y0}}) -> >>>> ? ? {Sum + math:sqrt((X - X0) * (X - X0) + (Y - Y0) * (Y - Y0)), {X, >>>> Y}}. >>>> >>>> ====================================================== >>>> >>>> and the following code for Node.js: >>>> >>>> ====================================================== >>>> >>>> var a = []; >>>> for(var i = 0; i< ? ?1000000; i++) { >>>> ? ? a[i] = {}; >>>> ? ? a[i].x = Math.floor(Math.random() * 1000); >>>> ? ? a[i].y = Math.floor(Math.random() * 1000); >>>> } >>>> >>>> var sum = 0; >>>> >>>> var start = (new Date()).valueOf(); >>>> >>>> for(var i = 1; i< ? ?1000000; i++) { >>>> ? ? var prev = a[i-1]; >>>> ? ? sum += Math.sqrt((a[i].x - prev.x) * (a[i].x - prev.x) + (a[i].y - >>>> prev.y) * (a[i].y - prev.y)); >>>> } >>>> >>>> var end = (new Date()).valueOf(); >>>> >>>> console.log(end - start); >>>> >>>> ============================================ >>>> >>>> There was no special tuning for Erlang and Node, both using the latest >>>> versions. >>>> But "arith_speed:test(1000000)." from Erlang console and "node >>>> test.js" have given me very different results: about 413 milliseconds >>>> for Erlang and 124 milliseconds for Node. So, the difference was about >>>> 4 times! I tried to change the total number of points, and the overall >>>> result remained the same. >>>> >>>> Both Erlang and V8 (Google's engine that is used by Node) use IEEE >>>> 754-2008 implementation, so that's not about float type >>>> representation. So, for now I have several probable explanations: >>>> >>>> 1. I've done something wrong and my tests suck (but that may mean that >>>> the difference in performance may be even more significant) >>>> 2. Erlang uses type overflow check on each computation to determine if >>>> it's time to switch from smallint to bigint. >>>> 3. Some more reasons that I don't know about or don't consider. >>>> >>>> Also, I'm still not sure if this kind of test is good for arithmetics >>>> comparison. On one side, it uses only pretty basic operations >>>> (summing, multiplying and square root) but on the other side it may >>>> involve some special computation mechanisms for Erlang that may slow >>>> it down. >>>> >>>> So, any help in this research is very appreciated. I understand that >>>> this involves another platform too, but since Erlang appeared to be >>>> slower I want to start from it first. >>>> >>>> Thanks in advance. >>>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best regards, Dmitry Demeshchuk From mk@REDACTED Sat Oct 30 14:55:10 2010 From: mk@REDACTED (Morten Krogh) Date: Sat, 30 Oct 2010 14:55:10 +0200 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: References: <4CCBDCBA.5030602@amberbio.com> <4CCC11A0.8080004@amberbio.com> Message-ID: <4CCC15AE.4040608@amberbio.com> Hi Dimitry The results should be the same. That was my point too, so you readers will get a clearer picture if you keep the code as short as possible. Anyway, about why erlang is slower? How did you compile it in erlang? with or without native? erlc +native program.erl On 10/30/10 2:48 PM, Dmitry Demeshchuk wrote: > Hi, Morten > > You have a point, random numbers aren't necessary. :) My goal was just > to have some more or less uniform distribution that can be reached > using randomized data. Of course, another variant was to choose some > static arithmetic operation. > > To my own defense, I'll say that despite of the random numbers the > results were very similar, so the difference in numbers didn't affect > them much. But still you are right, static numbers are a bit more > reliable. > > However, that doesn't answer the question "why so big difference in > speed for the two platforms?". It's well-known that Erlang arithmetics > is slow but I still cannot find a clear answer that describes why > exactly... > > Hope, some of Erlang fathers will be able to help with this... > > On Sat, Oct 30, 2010 at 4:37 PM, Morten Krogh wrote: >> Hi Dimitry >> >> Yes, I first thought that the random number generation was timed as well, so >> some of my points are irrelevant. >> >> But why don't you compare very short programs instead that just loops or >> recurses on the number N? >> >> like this in js: >> >> sum = 0; >> for (i =0; i< 1000000; i++) { >> sum += 37*i*(i - 5) + i + 23; >> } >> >> and the same in erlang. Do you need the lists of random numbers for your >> basic arithmetic? >> >> Morten. >> >> >> On 10/30/10 2:23 PM, Dmitry Demeshchuk wrote: >>> On Sat, Oct 30, 2010 at 12:52 PM, Morten Krogh wrote: >>>> Hi Dimitry >>>> >>>> You need to answer a lot of questions before you can even start the >>>> comparison. >>>> >>>> 1. Should the implementations keep a list of all points, or can they just >>>> calculate a running sum and discard the points immediately. >>>> Right now, your erlang solution keeps two lists, and your js program keep >>>> one object (hash table). >>> No, garbage collecting time shouldn't be considered, as well as memory >>> usage. I understand that this isn't very fair (for example, one >>> platform may be much faster on garbage collecting but slower on >>> arithmetic operations) but the purpose of the test is to measure some >>> average productivity of basic arithmetic operations. >>> >>>> 2. Is parallelizing the computation allowed?. If so, this would give >>>> erlang >>>> an edge since you can parallelize this problem by simple message passing. >>>> That would require more work in js. >>> No, if I wanted to use it, I would definitely do it. :) But, first of >>> all, Node.js will have a significant overhead on IPC (to run >>> multi-core task you'll have to run several processes and communicate >>> between them). And, that is also important, my goal was to compare >>> arithmetics abilities, not abilities of parallelizing (where Node will >>> be extremely slower). >>> >>>> 3. What types do you want for the points. In erlang you use >>>> random:uniform(1000) which returns integers, in js Math.random() which >>>> returns floats. >>>> The type issue also includes the question of checking for overflow, as >>>> you mention. >>> Well, that's why I used Math.floor for JS ;) >>> >>>> 4. Should the program be short and simple? In the extreme case, the >>>> erlang >>>> program could use nifs. >>> Sure, I could use a port or even a driver. But the subject of interest >>> is Erlang built-in arithmetics, not C. In my article, I mention that >>> ports can be used for some expensive operations, but these >>> calculations will be limited by the port itself then Erlang >>> performance. >>> >>>> 5. How should they calculate random numbers? There could be a huge >>>> difference in speed based on the algorithm of course. >>>> Actually, I think a speed test between languages should not involve >>>> random numbers, unless you make sure they use the same random number >>>> generating algorithm. >>> As you can see, my tests don't measure random numbers generation time. >>> Yes, I was going to try using the same pseudo-random generation >>> algorithm on both platforms for another test (Erlang will use process >>> dictionary for this, which should slow down the randomizer) but, >>> again, my subject there is just arithmetic operations. >>> >>>> Cheers, >>>> >>>> Morten. >>>> >>>> >>>> >>>> >>>> >>>> On 10/30/10 10:03 AM, Dmitry Demeshchuk wrote: >>>>> Greetings. >>>>> >>>>> I'm writing an article comparing Erlang and Node.js and I stumbled >>>>> upon the performance question. >>>>> >>>>> My initial goal was to compare some basic arithmetics speed, like the >>>>> total distance between randomly distributed points. So, I have written >>>>> the following code for Erlang: >>>>> >>>>> ===================================================== >>>>> >>>>> -module(arith_speed). >>>>> -export([ >>>>> test/1 >>>>> ]). >>>>> >>>>> test(N) -> >>>>> L = lists:seq(1, N), >>>>> [{X0, Y0} | Points] = [{random:uniform(1000), >>>>> random:uniform(1000)} || _<- L], >>>>> Now = now(), >>>>> lists:foldl(fun move_to/2, {0, {X0, Y0}}, Points), >>>>> timer:now_diff(now(), Now). >>>>> >>>>> move_to({X, Y}, {Sum, {X0, Y0}}) -> >>>>> {Sum + math:sqrt((X - X0) * (X - X0) + (Y - Y0) * (Y - Y0)), {X, >>>>> Y}}. >>>>> >>>>> ====================================================== >>>>> >>>>> and the following code for Node.js: >>>>> >>>>> ====================================================== >>>>> >>>>> var a = []; >>>>> for(var i = 0; i< 1000000; i++) { >>>>> a[i] = {}; >>>>> a[i].x = Math.floor(Math.random() * 1000); >>>>> a[i].y = Math.floor(Math.random() * 1000); >>>>> } >>>>> >>>>> var sum = 0; >>>>> >>>>> var start = (new Date()).valueOf(); >>>>> >>>>> for(var i = 1; i< 1000000; i++) { >>>>> var prev = a[i-1]; >>>>> sum += Math.sqrt((a[i].x - prev.x) * (a[i].x - prev.x) + (a[i].y - >>>>> prev.y) * (a[i].y - prev.y)); >>>>> } >>>>> >>>>> var end = (new Date()).valueOf(); >>>>> >>>>> console.log(end - start); >>>>> >>>>> ============================================ >>>>> >>>>> There was no special tuning for Erlang and Node, both using the latest >>>>> versions. >>>>> But "arith_speed:test(1000000)." from Erlang console and "node >>>>> test.js" have given me very different results: about 413 milliseconds >>>>> for Erlang and 124 milliseconds for Node. So, the difference was about >>>>> 4 times! I tried to change the total number of points, and the overall >>>>> result remained the same. >>>>> >>>>> Both Erlang and V8 (Google's engine that is used by Node) use IEEE >>>>> 754-2008 implementation, so that's not about float type >>>>> representation. So, for now I have several probable explanations: >>>>> >>>>> 1. I've done something wrong and my tests suck (but that may mean that >>>>> the difference in performance may be even more significant) >>>>> 2. Erlang uses type overflow check on each computation to determine if >>>>> it's time to switch from smallint to bigint. >>>>> 3. Some more reasons that I don't know about or don't consider. >>>>> >>>>> Also, I'm still not sure if this kind of test is good for arithmetics >>>>> comparison. On one side, it uses only pretty basic operations >>>>> (summing, multiplying and square root) but on the other side it may >>>>> involve some special computation mechanisms for Erlang that may slow >>>>> it down. >>>>> >>>>> So, any help in this research is very appreciated. I understand that >>>>> this involves another platform too, but since Erlang appeared to be >>>>> slower I want to start from it first. >>>>> >>>>> Thanks in advance. >>>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > From demeshchuk@REDACTED Sat Oct 30 15:20:14 2010 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Sat, 30 Oct 2010 17:20:14 +0400 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: <4CCC15AE.4040608@amberbio.com> References: <4CCBDCBA.5030602@amberbio.com> <4CCC11A0.8080004@amberbio.com> <4CCC15AE.4040608@amberbio.com> Message-ID: Hi, Morten No, I didn't use HiPE. Compiling with HiPE and default options gave about twice better performance. But I would prefer not to mention HiPE in the article, it will rather confuse the readers than help them to understand the basic qualities of the platform. I used all default settings of erl and erlc, so that people will be able to reproduce my steps and check the difference themselves. Of course, due to the differences in hardware and system configuration, they will get different results but I believe that the average Node:Erlang ratio will remain approximately the same. But the option about making Erlang give approximately the same performance as Node doesn't seem bad to me. At least, in that case I won't have to explain the difference in results :) On Sat, Oct 30, 2010 at 4:55 PM, Morten Krogh wrote: > Hi Dimitry > > The results should be the same. That was my point too, so you readers will > get a clearer picture if you keep the code as short as possible. > > Anyway, about why erlang is slower? > > How did you compile it in erlang? > with or without ?native? > > erlc +native program.erl > > > > > On 10/30/10 2:48 PM, Dmitry Demeshchuk wrote: >> >> Hi, Morten >> >> You have a point, random numbers aren't necessary. :) My goal was just >> to have some more or less uniform distribution that can be reached >> using randomized data. Of course, another variant was to choose some >> static arithmetic operation. >> >> To my own defense, I'll say that despite of the random numbers the >> results were very similar, so the difference in numbers didn't affect >> them much. But still you are right, static numbers are a bit more >> reliable. >> >> However, that doesn't answer the question "why so big difference in >> speed for the two platforms?". It's well-known that Erlang arithmetics >> is slow but I still cannot find a clear answer that describes why >> exactly... >> >> Hope, some of Erlang fathers will be able to help with this... >> >> On Sat, Oct 30, 2010 at 4:37 PM, Morten Krogh ?wrote: >>> >>> Hi Dimitry >>> >>> Yes, I first thought that the random number generation was timed as well, >>> so >>> some of my points are irrelevant. >>> >>> But why don't you compare very short programs instead that just loops or >>> recurses on the number N? >>> >>> like this in js: >>> >>> sum = 0; >>> for (i =0; i< ?1000000; i++) { >>> ? ?sum += 37*i*(i - 5) + i + 23; >>> } >>> >>> and the same in erlang. Do you need the lists of random numbers for your >>> basic arithmetic? >>> >>> Morten. >>> >>> >>> On 10/30/10 2:23 PM, Dmitry Demeshchuk wrote: >>>> >>>> On Sat, Oct 30, 2010 at 12:52 PM, Morten Krogh >>>> ?wrote: >>>>> >>>>> Hi Dimitry >>>>> >>>>> You need to answer a lot of questions before you can even start the >>>>> comparison. >>>>> >>>>> 1. Should the implementations keep a list of all points, or can they >>>>> just >>>>> calculate a running sum and discard the points immediately. >>>>> Right now, your erlang solution keeps two lists, and your js program >>>>> keep >>>>> one object (hash table). >>>> >>>> No, garbage collecting time shouldn't be considered, as well as memory >>>> usage. I understand that this isn't very fair (for example, one >>>> platform may be much faster on garbage collecting but slower on >>>> arithmetic operations) but the purpose of the test is to measure some >>>> average productivity of basic arithmetic operations. >>>> >>>>> 2. Is parallelizing the computation allowed?. If so, this would give >>>>> erlang >>>>> an edge since you can parallelize this problem by simple message >>>>> passing. >>>>> That would require more work in js. >>>> >>>> No, if I wanted to use it, I would definitely do it. :) But, first of >>>> all, Node.js will have a significant overhead on IPC (to run >>>> multi-core task you'll have to run several processes and communicate >>>> between them). And, that is also important, my goal was to compare >>>> arithmetics abilities, not abilities of parallelizing (where Node will >>>> be extremely slower). >>>> >>>>> 3. What types do you want for the points. In erlang you use >>>>> random:uniform(1000) which returns integers, in js Math.random() which >>>>> returns floats. >>>>> ? ?The type issue also includes the question of checking for overflow, >>>>> as >>>>> you mention. >>>> >>>> Well, that's why I used Math.floor for JS ;) >>>> >>>>> 4. Should the program be short and simple? ?In the extreme case, the >>>>> erlang >>>>> program could use nifs. >>>> >>>> Sure, I could use a port or even a driver. But the subject of interest >>>> is Erlang built-in arithmetics, not C. In my article, I mention that >>>> ports can be used for some expensive operations, but these >>>> calculations will be limited by the port itself then Erlang >>>> performance. >>>> >>>>> 5. How should they calculate random numbers? There could be a huge >>>>> difference in speed based on the algorithm of course. >>>>> ? ?Actually, I think a speed test between languages should not involve >>>>> random numbers, unless you make sure they use the same random number >>>>> generating algorithm. >>>> >>>> As you can see, my tests don't measure random numbers generation time. >>>> Yes, I was going to try using the same pseudo-random generation >>>> algorithm on both platforms for another test (Erlang will use process >>>> dictionary for this, which should slow down the randomizer) but, >>>> again, my subject there is just arithmetic operations. >>>> >>>>> Cheers, >>>>> >>>>> Morten. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 10/30/10 10:03 AM, Dmitry Demeshchuk wrote: >>>>>> >>>>>> Greetings. >>>>>> >>>>>> I'm writing an article comparing Erlang and Node.js and I stumbled >>>>>> upon the performance question. >>>>>> >>>>>> My initial goal was to compare some basic arithmetics speed, like the >>>>>> total distance between randomly distributed points. So, I have written >>>>>> the following code for Erlang: >>>>>> >>>>>> ===================================================== >>>>>> >>>>>> -module(arith_speed). >>>>>> -export([ >>>>>> ? ? test/1 >>>>>> ]). >>>>>> >>>>>> test(N) -> >>>>>> ? ? L = lists:seq(1, N), >>>>>> ? ? [{X0, Y0} | Points] = [{random:uniform(1000), >>>>>> random:uniform(1000)} || _<- L], >>>>>> ? ? Now = now(), >>>>>> ? ? lists:foldl(fun move_to/2, {0, {X0, Y0}}, Points), >>>>>> ? ? timer:now_diff(now(), Now). >>>>>> >>>>>> move_to({X, Y}, {Sum, {X0, Y0}}) -> >>>>>> ? ? {Sum + math:sqrt((X - X0) * (X - X0) + (Y - Y0) * (Y - Y0)), {X, >>>>>> Y}}. >>>>>> >>>>>> ====================================================== >>>>>> >>>>>> and the following code for Node.js: >>>>>> >>>>>> ====================================================== >>>>>> >>>>>> var a = []; >>>>>> for(var i = 0; i< ? ? ?1000000; i++) { >>>>>> ? ? a[i] = {}; >>>>>> ? ? a[i].x = Math.floor(Math.random() * 1000); >>>>>> ? ? a[i].y = Math.floor(Math.random() * 1000); >>>>>> } >>>>>> >>>>>> var sum = 0; >>>>>> >>>>>> var start = (new Date()).valueOf(); >>>>>> >>>>>> for(var i = 1; i< ? ? ?1000000; i++) { >>>>>> ? ? var prev = a[i-1]; >>>>>> ? ? sum += Math.sqrt((a[i].x - prev.x) * (a[i].x - prev.x) + (a[i].y - >>>>>> prev.y) * (a[i].y - prev.y)); >>>>>> } >>>>>> >>>>>> var end = (new Date()).valueOf(); >>>>>> >>>>>> console.log(end - start); >>>>>> >>>>>> ============================================ >>>>>> >>>>>> There was no special tuning for Erlang and Node, both using the latest >>>>>> versions. >>>>>> But "arith_speed:test(1000000)." from Erlang console and "node >>>>>> test.js" have given me very different results: about 413 milliseconds >>>>>> for Erlang and 124 milliseconds for Node. So, the difference was about >>>>>> 4 times! I tried to change the total number of points, and the overall >>>>>> result remained the same. >>>>>> >>>>>> Both Erlang and V8 (Google's engine that is used by Node) use IEEE >>>>>> 754-2008 implementation, so that's not about float type >>>>>> representation. So, for now I have several probable explanations: >>>>>> >>>>>> 1. I've done something wrong and my tests suck (but that may mean that >>>>>> the difference in performance may be even more significant) >>>>>> 2. Erlang uses type overflow check on each computation to determine if >>>>>> it's time to switch from smallint to bigint. >>>>>> 3. Some more reasons that I don't know about or don't consider. >>>>>> >>>>>> Also, I'm still not sure if this kind of test is good for arithmetics >>>>>> comparison. On one side, it uses only pretty basic operations >>>>>> (summing, multiplying and square root) but on the other side it may >>>>>> involve some special computation mechanisms for Erlang that may slow >>>>>> it down. >>>>>> >>>>>> So, any help in this research is very appreciated. I understand that >>>>>> this involves another platform too, but since Erlang appeared to be >>>>>> slower I want to start from it first. >>>>>> >>>>>> Thanks in advance. >>>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions (at) erlang.org mailing list. >>>>> See http://www.erlang.org/faq.html >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>> >>>>> >>>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best regards, Dmitry Demeshchuk From masklinn@REDACTED Sat Oct 30 16:17:01 2010 From: masklinn@REDACTED (Masklinn) Date: Sat, 30 Oct 2010 16:17:01 +0200 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: References: <4CCBDCBA.5030602@amberbio.com> <4CCC11A0.8080004@amberbio.com> <4CCC15AE.4040608@amberbio.com> Message-ID: <38D7260A-7C80-40BA-A780-762389BE9315@masklinn.net> On 2010-10-30, at 15:20 , Dmitry Demeshchuk wrote: > Hi, Morten > > No, I didn't use HiPE. Compiling with HiPE and default options gave > about twice better performance. > But I would prefer not to mention HiPE in the article, it will rather > confuse the readers than help them to understand the basic qualities > of the platform. given crunching numbers is nowhere near the "basic qualities of the erlang platform" (especially without HiPE), I fail to see the point of this test then. From freza@REDACTED Sat Oct 30 17:31:50 2010 From: freza@REDACTED (Jachym Holecek) Date: Sat, 30 Oct 2010 16:31:50 +0100 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: References: Message-ID: <20101030153150.GA6761@hanele.lan> Hello, This one: -module(b). -export([test/1]). test(N) -> L = [random_point(1000, 1000) || _ <- lists:seq(1, N)], Now = now(), distance(L, 0.0), timer:now_diff(now(), Now). distance([{X0, Y0} | [{X1, Y1} | _] = L], Acc) -> Dst = math:sqrt(square(X0 - X1) + square(Y0 - Y1)), distance(L, Dst + Acc); distance([_], Acc) -> Acc. square(X) -> X * X. random_point(Xmax, Ymax) -> {random:uniform(Xmax), random:uniform(Ymax)}. is a bit faster than your original version (and maybe a bit easier on beginner's eyes): %% 'a' -- original, 'b' -- the above, results from 6 consecutive runs. a:test(1000000) => [288317, 324549, 327840, 386648, 388363, 390389] b:test(1000000) => [210855, 230395, 238020, 246694, 253710, 261157] Speedup may be because 'b' avoids a little allocation overhead, ie. building the lists:foldl/3 accumulator tuple. The floating point value used as Acc in case 'b' will still be an on-heap object though -- compiler can unbox floats locally, but not across function calls it seems? You're right in that integer arith ops have to check for smallint -> bigint overflow, BTW. I don't have Node.js or HiPE around for comparison... Have fun, -- Jachym From hynek@REDACTED Sat Oct 30 20:47:04 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Sat, 30 Oct 2010 20:47:04 +0200 Subject: [erlang-questions] Erlang re:run regular exp, match problrm In-Reply-To: References: <23BB6CB9-D0EA-4053-9630-627294D6D79B@gmail.com> Message-ID: I would not thanks on your place. It doesn't do what you want but works only by accident in this particular example. [^", "",[global]) but anyway you should use xml parser for xml parsing because xml is not parseable by regular grammar so regular expression is not proper tool for do it. You will end up with error prone solution. On Sat, Oct 30, 2010 at 1:34 PM, Mathias wrote: > Works like a charm! > > Many thanks dlfen! > > BR, > Mathias > > On Sat, Oct 30, 2010 at 1:03 PM, dlfen wrote: > >> try this. >> re:run("> z=\"14\"/>", "",[global]). >> >> >> ? 2010-10-30???6:44? Mathias ??? >> >> > Hi there, >> > >> > I'm trying to figure out how Erlangs re:run module works. >> > >> > When executing this:: >> > 1> re:run("> > z=\"14\"/>", ""). >> > {match,[{0,54}]} >> > >> > I can see that it gives me a match on the complete XML representation >> > {match,[{0,54}]}. >> > >> > But what I really would like to do is for it to give me a subset of >> matches >> > on each entity similar to {match,[{0,26},{27, 26}]}. >> > >> > so the output would yield ?something like this: >> > 0-26 gives the first xml entity complete with it's attributes > x="12" >> > y="2" z="4"/> and >> > match 27,26 gives the remaining entity. >> > >> > If anyone can spot why my regexp: is failing and guide >> me >> > in the right direction closer to find the solution it will be greatly >> > appreciated. >> > >> > I know about xmerl but for my trivial case it seems like overkill. >> > >> > Thx in advance. >> > >> > BR, >> > Mathias Stal?s >> >> > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From vincenzo.di.somma@REDACTED Sat Oct 30 22:25:42 2010 From: vincenzo.di.somma@REDACTED (Vincenzo Di Somma) Date: Sat, 30 Oct 2010 16:25:42 -0400 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> Message-ID: <1288470342.15939.6.camel@blackrain> The next Ubuntu release, 11.04, will be out on April 2011 and unless something changes it will probably ship the same version that has been shipped so far. I haven't worked on the .deb packages myself but I've heard it's a quite complicated by the big number of dependencies the erlang tarball has. I'm under the impression that splitting the runtime and the development environment will make the packaging and the maintenance of the packages a lot easier. Thanks, vds On Fri, 2010-10-29 at 10:51 +0200, Alessandro Sivieri wrote: > We could try to backport packages from the next Ubuntu release, assuming > that it will have the R14B release; the first alpha of 11.04 should come out > in a few days, if I remember correctly... > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From vincenzo.di.somma@REDACTED Sat Oct 30 22:27:59 2010 From: vincenzo.di.somma@REDACTED (Vincenzo Di Somma) Date: Sat, 30 Oct 2010 16:27:59 -0400 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: <20101028184852.GA33595@alumni.caltech.edu> References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> Message-ID: <1288470480.15939.8.camel@blackrain> On Thu, 2010-10-28 at 11:48 -0700, Anthony Molinaro wrote: > It's not debian, but for the last few versions of erlang, I've been updating > the EPEL spec file and patches. The nice thing is that several patches > became unnecessary between R12 and R13, most of the rest disappeared with > R14 (the one remaining patch just disables the formating of man pages). > So right now I have a pretty vanilla setup, very simple .spec and a single > patch (you can build the whole thing with rpmbuild -ba erlang.spec), I > attached the spec and patch (both derived from those included in the EPEL > SRPM of R12). > > It may be that its easier to package when you have a single package for > the whole thing? I think it will make things much worse. > Anyway, maybe it will help in packaging for debian/ubuntu, to see an > rpm example? If it's one big huge rmp, nope it's not going to help. :) Thanks, vds > On Thu, Oct 28, 2010 at 06:17:38PM +0400, Max Lapshin wrote: > > On Thu, Oct 28, 2010 at 6:05 PM, Vincenzo Di Somma > > wrote: > > > We discussed a bit about packaging Erlang at the Erlang Factory in > > > London this year. > > > One very interesting suggestion was to improve the separation of the > > > runtime environment from the development environment. > > > > > > > Maybe it is a good idea to split into erlang and erlang-dev, maybe > > not, but when there are no fresh packages for months and it still > > requires some knowledge hidden inside one person to build a package, > > everything ends in ./configure && make && make install on production > > server. > > > > I tried to build R14B for squeeze, but failed: too complex build setup > > required. If situation doesn't change, I will have to refuse from > > building debian package because of no erlang in debian or ubuntu. > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From mjtruog@REDACTED Sat Oct 30 21:38:19 2010 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 30 Oct 2010 12:38:19 -0700 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: <20101030153150.GA6761@hanele.lan> References: <20101030153150.GA6761@hanele.lan> Message-ID: <4CCC742B.6030407@gmail.com> You need to use HiPE with guards on the function arguments (integers, floats) if you want the arithmetic to be as fast as possible within Erlang code. Exclusively testing arithmetic within Erlang seems to be otherwise pointless, since Erlang did not focus on numerical processing. On 10/30/2010 08:31 AM, Jachym Holecek wrote: > Hello, > > This one: > > -module(b). > -export([test/1]). > > test(N) -> > L = [random_point(1000, 1000) || _ <- lists:seq(1, N)], > Now = now(), > distance(L, 0.0), > timer:now_diff(now(), Now). > > distance([{X0, Y0} | [{X1, Y1} | _] = L], Acc) -> > Dst = math:sqrt(square(X0 - X1) + square(Y0 - Y1)), > distance(L, Dst + Acc); > distance([_], Acc) -> > Acc. > > square(X) -> > X * X. > > random_point(Xmax, Ymax) -> > {random:uniform(Xmax), random:uniform(Ymax)}. > > is a bit faster than your original version (and maybe a bit easier on > beginner's eyes): > > %% 'a' -- original, 'b' -- the above, results from 6 consecutive runs. > a:test(1000000) => [288317, 324549, 327840, 386648, 388363, 390389] > b:test(1000000) => [210855, 230395, 238020, 246694, 253710, 261157] > > Speedup may be because 'b' avoids a little allocation overhead, ie. building > the lists:foldl/3 accumulator tuple. The floating point value used as Acc in > case 'b' will still be an on-heap object though -- compiler can unbox floats > locally, but not across function calls it seems? You're right in that integer > arith ops have to check for smallint -> bigint overflow, BTW. > > I don't have Node.js or HiPE around for comparison... > > Have fun, > -- Jachym > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From max.lapshin@REDACTED Sat Oct 30 22:33:09 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 31 Oct 2010 00:33:09 +0400 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: <1288470480.15939.8.camel@blackrain> References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> <1288470480.15939.8.camel@blackrain> Message-ID: Frankly speaking, I'm going to create monolitic packages (only one erlang package) for two versions of debian and ubuntu. Just for those, who is not going to wait two years for official build. From max.lapshin@REDACTED Sat Oct 30 22:42:31 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 31 Oct 2010 00:42:31 +0400 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: <4CCC742B.6030407@gmail.com> References: <20101030153150.GA6761@hanele.lan> <4CCC742B.6030407@gmail.com> Message-ID: Would you, kindly, provide an example with such optimized code? I couldn't reproduce it with my benchmarks. From anthonym@REDACTED Sat Oct 30 23:07:34 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Sat, 30 Oct 2010 14:07:34 -0700 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> <1288470480.15939.8.camel@blackrain> Message-ID: I think monolithic is the way to go. OTP is released all as one thing so I don't see how splitting it would help you over time, maintenance would be a pain. With a monolithic package its very minimal to update, R14B took me about an hour to package and push to our yum repo. -Anthony On Oct 30, 2010, at 1:33 PM, Max Lapshin wrote: > Frankly speaking, I'm going to create monolitic packages (only one > erlang package) for two versions of debian and ubuntu. Just for those, > who is not going to wait two years for official build. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From max.lapshin@REDACTED Sat Oct 30 23:17:51 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 31 Oct 2010 01:17:51 +0400 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> <1288470480.15939.8.camel@blackrain> Message-ID: On Sun, Oct 31, 2010 at 1:07 AM, Anthony Molinaro wrote: > I think monolithic is the way to go. OTP is released all as one thing so I don't see how splitting it would help you over time, maintenance would be a pain. ?With a monolithic package its very minimal to update, R14B took me about an hour to package and push to our yum repo. > I don't understand what for to split erlang into many several subpackages. Who in the hell will ever install only erlang-corba without, for example, erlang-megaco? From jesper.louis.andersen@REDACTED Sat Oct 30 23:22:50 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 30 Oct 2010 23:22:50 +0200 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: References: <20101030153150.GA6761@hanele.lan> <4CCC742B.6030407@gmail.com> Message-ID: On Sat, Oct 30, 2010 at 10:42 PM, Max Lapshin wrote: > Would you, kindly, provide an example with such optimized code? I > couldn't reproduce it with my benchmarks. If type inference already figures the type information out correctly, it should definitely be the same speed these two run with. Another point from the original: The Javascript version creates an Array which is filled before the initial run. In contrast, the Erlang version stores this in a singly linked list. So when the main calculation body runs, the Erlang versions suffers from having to walk the list whereas the Java version might be able to utilize that the array has a more cache-friendly behaviour. When you are doing benchmarks this synthetic, I think it may have a large effect on the speed. Also, this is a pet peeve of mine: Measuring the usefulness of a language by a few benchmarks is probably not going to give a fair survey of the question. As many people have already pointed out, they would solve the problem differently. I, for instance, would never first build up a singly-linked list of random numbers and then consume it afterwards. In Haskell I would, because fusion optimizations will trigger in Haskell, but in Erlang I would try to avoid generation/consumption constructions. As for Node.js vs Erlang, the real trouble with "Number Crunching problems" is that Ocaml, C++, Java, C or Fortran is an order of magnitude faster, usually. It would therefore be much smarter, for Node.js or Erlang, to utilize the effort put into these languages for the problem at hand and interface with them. -- J. From vincenzo.di.somma@REDACTED Sat Oct 30 23:23:19 2010 From: vincenzo.di.somma@REDACTED (Vincenzo Di Somma) Date: Sat, 30 Oct 2010 17:23:19 -0400 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> <1288470480.15939.8.camel@blackrain> Message-ID: <1288473799.15939.10.camel@blackrain> On Sun, 2010-10-31 at 01:17 +0400, Max Lapshin wrote: > On Sun, Oct 31, 2010 at 1:07 AM, Anthony Molinaro > wrote: > > I think monolithic is the way to go. OTP is released all as one thing so I don't see how splitting it would help you over time, maintenance would be a pain. With a monolithic package its very minimal to update, R14B took me about an hour to package and push to our yum repo. > > > > I don't understand what for to split erlang into many several > subpackages. Who in the hell will ever install only erlang-corba > without, for example, erlang-megaco? It's actually the other way around, why would I have erlang-corba on my machine if I only run say CouchDB? Thanks, vds -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From max.lapshin@REDACTED Sat Oct 30 23:24:41 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 31 Oct 2010 01:24:41 +0400 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: <1288473799.15939.10.camel@blackrain> References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> <1288470480.15939.8.camel@blackrain> <1288473799.15939.10.camel@blackrain> Message-ID: On Sun, Oct 31, 2010 at 1:23 AM, Vincenzo Di Somma wrote: > > It's actually the other way around, why would I have erlang-corba on my > machine if I only run say CouchDB? > Thanks, > ? ? ? ? ? ? ? ? ? ? ? ?vds I'm not speaking about "what can be in ideal world". I'm speaking about: have you ever deselected erlang-corba, when running only couchdb? From max.lapshin@REDACTED Sat Oct 30 23:29:13 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 31 Oct 2010 01:29:13 +0400 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: References: <20101030153150.GA6761@hanele.lan> <4CCC742B.6030407@gmail.com> Message-ID: Can this trivial code be optimized with HiPE? -module(a). -export([z/0]). z() -> T1 = erlang:now(), V = z(100000, 0), T2 = erlang:now(), put(v, V), io:format("S: ~p~n", [timer:now_diff(T2, T1)]), ok. z(0, S) -> S; z(N, S) when is_number(N) andalso is_number(S) -> z(N - 1, S*2 + 3). From vincenzo.di.somma@REDACTED Sat Oct 30 23:40:17 2010 From: vincenzo.di.somma@REDACTED (Vincenzo Di Somma) Date: Sat, 30 Oct 2010 17:40:17 -0400 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> <1288470480.15939.8.camel@blackrain> <1288473799.15939.10.camel@blackrain> Message-ID: <1288474817.15939.16.camel@blackrain> On Sun, 2010-10-31 at 01:24 +0400, Max Lapshin wrote: > On Sun, Oct 31, 2010 at 1:23 AM, Vincenzo Di Somma > wrote: > > > > It's actually the other way around, why would I have erlang-corba on my > > machine if I only run say CouchDB? > > Thanks, > > vds > > I'm not speaking about "what can be in ideal world". I'm speaking > about: have you ever deselected erlang-corba, when running only > couchdb? Sorry I am speaking of an ideal world, because I don't see it very far. In Ubuntu, at the moment we have around 50 packages out of one single Erlang tarball. If we can only move the effort a bit the from creating distribution packages to releasing a tarball that can be packaged more easily, it will pay a lot in the long term. Thanks, vds -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From elliot.murphy@REDACTED Sun Oct 31 00:19:46 2010 From: elliot.murphy@REDACTED (Elliot Murphy) Date: Sat, 30 Oct 2010 18:19:46 -0400 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> <1288470480.15939.8.camel@blackrain> <1288473799.15939.10.camel@blackrain> Message-ID: On Oct 30, 2010, at 5:24 PM, Max Lapshin wrote: > On Sun, Oct 31, 2010 at 1:23 AM, Vincenzo Di Somma > wrote: >> >> It's actually the other way around, why would I have erlang-corba on my >> machine if I only run say CouchDB? >> Thanks, >> vds > > I'm not speaking about "what can be in ideal world". I'm speaking > about: have you ever deselected erlang-corba, when running only > couchdb? Uh, yes. This is not ideal world, this is a real problem that affects end users. Ubuntu includes CouchDB on the livecd, and absolutely does not want all of OTP dragged in as a dependency. Every release there is pressure to remove Erlang and CouchDB to free up more space on the CD. Every few K helps - I want these technologies available in more environments such as mobile and tablet, where space is even more important. -elliot From alessandro.sivieri@REDACTED Sun Oct 31 00:23:28 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Sun, 31 Oct 2010 00:23:28 +0200 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: <1288474817.15939.16.camel@blackrain> References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> <1288470480.15939.8.camel@blackrain> <1288473799.15939.10.camel@blackrain> <1288474817.15939.16.camel@blackrain> Message-ID: This is something like a similar discussion happened in the ruby mailing list: Debian (and consequently Ubuntu) has the package separation as a general philosophy, so that each user can install only what it needs. For example, I installed Erlang in a minimal Debian installation, and I avoided the all metapackage and installed only the core and some other packages, so I avoided installing X packages (which are needed by some erlang-*). So I think we will never see one single package of the whole distribution in those, at least not officially released. -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From max.lapshin@REDACTED Sun Oct 31 00:27:53 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 31 Oct 2010 02:27:53 +0400 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> <1288470480.15939.8.camel@blackrain> <1288473799.15939.10.camel@blackrain> <1288474817.15939.16.camel@blackrain> Message-ID: I can understand separation erlang package to erlang-nox, erlang-java (because it can't be installed on 256MB VPS) and erlang-x11. But I can't understand any other separation, because I've never heard about anyone, who _has_ installed only what he wants. From kennethstone@REDACTED Sun Oct 31 01:54:12 2010 From: kennethstone@REDACTED (Kenny Stone) Date: Sat, 30 Oct 2010 18:54:12 -0500 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: References: <20101030153150.GA6761@hanele.lan> <4CCC742B.6030407@gmail.com> Message-ID: If you're comparing node.js and erlang, it's probably for web services. It seems to me that arithmetic is fairly pointless measure of comparison (and something neither does that well as others have pointed out). Comparisons that would interest me if I was reading your article: benchmarks for handling many concurrent connections, availability of open source toolsets for solving common web problems, ease of distributing or scaling. On Sat, Oct 30, 2010 at 4:29 PM, Max Lapshin wrote: > Can this trivial code be optimized with HiPE? > > > -module(a). > > -export([z/0]). > > z() -> > T1 = erlang:now(), > V = z(100000, 0), > T2 = erlang:now(), > put(v, V), > io:format("S: ~p~n", [timer:now_diff(T2, T1)]), > ok. > > > z(0, S) -> > S; > > z(N, S) when is_number(N) andalso is_number(S) -> > z(N - 1, S*2 + 3). > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From igouy2@REDACTED Sun Oct 31 01:57:16 2010 From: igouy2@REDACTED (Isaac Gouy) Date: Sat, 30 Oct 2010 16:57:16 -0700 (PDT) Subject: [erlang-questions] Erlang arithmetics Message-ID: <168426.71864.qm@web65609.mail.ac4.yahoo.com> On 10/30/10 10:03 AM, Dmitry Demeshchuk wrote: > Greetings. > > I'm writing an article comparing Erlang and Node.js and I stumbled > upon the performance question. > > My initial goal was to compare some basic arithmetics speed, like the > total distance between randomly distributed points. Or like n-body? http://shootout.alioth.debian.org/u64/program.php?test=nbody&lang=hipe&id=3 http://shootout.alioth.debian.org/u64/benchmark.php?test=nbody&lang=v8 Note: TraceMonkey does better http://shootout.alioth.debian.org/u64/program.php?test=nbody&lang=tracemonkey&id=1 Or like spectral-norm? http://shootout.alioth.debian.org/u64/benchmark.php?test=spectralnorm&lang=all From g9414002.pccu.edu.tw@REDACTED Sun Oct 31 02:03:18 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Sun, 31 Oct 2010 09:03:18 +0800 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: References: Message-ID: Hi, Dmitry. I say "No" to what you're doing. Do not compare the difference of execution time between programs in Erlang and in JavaScript. It ought to be on Erlang basis to measure efficiency of Erlang, and it ought be on the corresponding basis of JavaScript for measuring. Besides, an Erlang program running on HIPE shares benefits of the platform. Node.js depends on the JavaScript engine of some browser. As you know, they are specific implementations. You would not discuss how differently those codes work just about two specific implementations in the article. You're not doing tests for either HIPE or Node.js. Find general cases to discuss which writing fashion causes an Erlang program better than other one in the same language and so does Node.js. Thus, you need test some Erlang functions, those perform the same arithmetic computation, and compare time- difference between them. And do the same work on JavaScript test. Finally you could find how difference of enhancement between some rewriting skill on Erlang programming and on JavaScript programming. -- Best Regards. --- Y-H. H. On Sat, Oct 30, 2010 at 4:03 PM, Dmitry Demeshchuk wrote: > Greetings. > > I'm writing an article comparing Erlang and Node.js and I stumbled > upon the performance question. > > My initial goal was to compare some basic arithmetics speed, like the > total distance between randomly distributed points. So, I have written > the following code for Erlang: > > ===================================================== > > -module(arith_speed). > -export([ > test/1 > ]). > > test(N) -> > L = lists:seq(1, N), > [{X0, Y0} | Points] = [{random:uniform(1000), > random:uniform(1000)} || _ <- L], > Now = now(), > lists:foldl(fun move_to/2, {0, {X0, Y0}}, Points), > timer:now_diff(now(), Now). > > move_to({X, Y}, {Sum, {X0, Y0}}) -> > {Sum + math:sqrt((X - X0) * (X - X0) + (Y - Y0) * (Y - Y0)), {X, Y}}. > > ====================================================== > > and the following code for Node.js: > > ====================================================== > > var a = []; > for(var i = 0; i < 1000000; i++) { > a[i] = {}; > a[i].x = Math.floor(Math.random() * 1000); > a[i].y = Math.floor(Math.random() * 1000); > } > > var sum = 0; > > var start = (new Date()).valueOf(); > > for(var i = 1; i < 1000000; i++) { > var prev = a[i-1]; > sum += Math.sqrt((a[i].x - prev.x) * (a[i].x - prev.x) + (a[i].y - > prev.y) * (a[i].y - prev.y)); > } > > var end = (new Date()).valueOf(); > > console.log(end - start); > > ============================================ > > There was no special tuning for Erlang and Node, both using the latest > versions. > But "arith_speed:test(1000000)." from Erlang console and "node > test.js" have given me very different results: about 413 milliseconds > for Erlang and 124 milliseconds for Node. So, the difference was about > 4 times! I tried to change the total number of points, and the overall > result remained the same. > > Both Erlang and V8 (Google's engine that is used by Node) use IEEE > 754-2008 implementation, so that's not about float type > representation. So, for now I have several probable explanations: > > 1. I've done something wrong and my tests suck (but that may mean that > the difference in performance may be even more significant) > 2. Erlang uses type overflow check on each computation to determine if > it's time to switch from smallint to bigint. > 3. Some more reasons that I don't know about or don't consider. > > Also, I'm still not sure if this kind of test is good for arithmetics > comparison. On one side, it uses only pretty basic operations > (summing, multiplying and square root) but on the other side it may > involve some special computation mechanisms for Erlang that may slow > it down. > > So, any help in this research is very appreciated. I understand that > this involves another platform too, but since Erlang appeared to be > slower I want to start from it first. > > Thanks in advance. > > -- > Best regards, > Dmitry Demeshchuk > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From g9414002.pccu.edu.tw@REDACTED Sun Oct 31 05:22:16 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Sun, 31 Oct 2010 12:22:16 +0800 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: References: Message-ID: Hi, Dmitry. Your code in JavaScript with Node.js is not equivalent to the Erlang one. You could take some operation-equivalent program: for example, I modified your JavaScript code, and make it follow fashion of your Erlang code, as // ================================== // Modified Erlang code test() -> Now = now(), seq(1, 1000), timer:now_diff(now(), Now). seq(M, N) -> L = lists:seq(M, N), [{X0,Y0}|Ps] = [ {random:uniform(1000),random:uniform(1000)} || _ <- L ], {Total,_} = lists:foldr(fun move_to/2, {0, {X0,Y0}}, Ps), Total. move_to({X, Y}, {Sum, {X0, Y0}}) -> {Sum + math:sqrt((X - X0) * (X - X0) + (Y - Y0) * (Y - Y0)), {X, Y}}. // ==================================== // Modified JavaScript code with Node.js, // following Erlang fashion function move_to(xy, sum, xy0) { var result = []; result[0] = {}; result[0] = sum + Math.sqrt( (xy[0]-xy0[0])*(xy[0]-xy0[0]) + (xy[1]-xy0[1])*(xy[1]-xy0[1]) ); result[1] = {}; result[1] = xy; return result; } function fold(f, base, xy0, list) { if (list.length == 0) { var result = []; result[0] = {}; result[0] = base; result[1] = {}; result[1] = xy0; return result; } var p = list[list.length-1]; list.length = list.length-1; var sum_xy = move_to(p, base, xy0); return fold(f, sum_xy[0], sum_xy[1], list); } function list_seq(m, n) { var result = []; var i; for (i=m; i<=n; i++) { result[i-m] = {}; result[i-m] = m; } return result; } function list_rand(seq) { var result = []; for (var i in seq) { result[i] = {}; result[i] = []; result[i][0] = Math.floor(Math.random() * 1000 + 1); result[i][1] = Math.floor(Math.random() * 1000 + 1); } return result; } function seq(m, n) { var list = list_seq(m, n); var list1 = list_rand(list); var head = list1[list1.length-1]; list1.length--; var result = fold(move_to, 0, head, list1); return result[0]; } var start = (new Date()).valueOf(); console.log(seq(1,1000)); var end = (new Date()).valueOf(); console.log(end-start); // =============================== In my computer with Ubuntu 10.10, the JavaScript program performed in 24~25 milliseconds, and the Erlang one performs about 8.5 milliseconds. Result is meaningful. The JavaScript one used some functions, but no recursive functions. Almost the same operations were performed, and the Erlang works fine. It may be persuasive. -- Best Regards, --- Y-H. H. On Sat, Oct 30, 2010 at 4:03 PM, Dmitry Demeshchuk wrote: > Greetings. > > I'm writing an article comparing Erlang and Node.js and I stumbled > upon the performance question. > > My initial goal was to compare some basic arithmetics speed, like the > total distance between randomly distributed points. So, I have written > the following code for Erlang: > > and the following code for Node.js: > > ====================================================== > > var a = []; > for(var i = 0; i < 1000000; i++) { > a[i] = {}; > a[i].x = Math.floor(Math.random() * 1000); > a[i].y = Math.floor(Math.random() * 1000); > } > > var sum = 0; > > var start = (new Date()).valueOf(); > > for(var i = 1; i < 1000000; i++) { > var prev = a[i-1]; > sum += Math.sqrt((a[i].x - prev.x) * (a[i].x - prev.x) + (a[i].y - > prev.y) * (a[i].y - prev.y)); > } > > var end = (new Date()).valueOf(); > > console.log(end - start); > > ============================================ > > From g9414002.pccu.edu.tw@REDACTED Sun Oct 31 05:36:48 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Sun, 31 Oct 2010 12:36:48 +0800 Subject: [erlang-questions] Erlang arithmetics In-Reply-To: References: Message-ID: First, I'm sorry for an mistake in my previous message as "No recursion used." Actually I have a recursive function `fold' in JavaScript code, that made the JavaScript version slow. Additional Information: when testing code like these, it's obviously that using Node.js won't overcome stack-overflow problem. It cannot beat some situation that constructing 1,000,000 array elements, each contains two elements and one sub-array. In data structure JavaScript even with Node.js is weak. --- Y-H. H. 2010/10/31 ??? (Yau-Hsien Huang) > Hi, Dmitry. Your code in JavaScript with Node.js is not equivalent to > the Erlang one. You could take some operation-equivalent program: > for example, I modified your JavaScript code, and make it follow fashion > of your Erlang code, as > > // ================================== > // Modified Erlang code > > test() -> > Now = now(), > seq(1, 1000), > timer:now_diff(now(), Now). > > seq(M, N) -> > L = lists:seq(M, N), > [{X0,Y0}|Ps] = > [ {random:uniform(1000),random:uniform(1000)} || _ <- L ], > {Total,_} = > lists:foldr(fun move_to/2, {0, {X0,Y0}}, Ps), > Total. > > move_to({X, Y}, {Sum, {X0, Y0}}) -> > {Sum + math:sqrt((X - X0) * (X - X0) + (Y - Y0) * (Y - Y0)), {X, Y}}. > > // ==================================== > // Modified JavaScript code with Node.js, > // following Erlang fashion > > function move_to(xy, sum, xy0) { > var result = []; > result[0] = {}; > result[0] = sum + > Math.sqrt( > (xy[0]-xy0[0])*(xy[0]-xy0[0]) > + (xy[1]-xy0[1])*(xy[1]-xy0[1]) > ); > result[1] = {}; > result[1] = xy; > return result; > } > > function fold(f, base, xy0, list) { > if (list.length == 0) { > var result = []; > result[0] = {}; > result[0] = base; > result[1] = {}; > result[1] = xy0; > return result; > } > var p = list[list.length-1]; > list.length = list.length-1; > var sum_xy = move_to(p, base, xy0); > return fold(f, sum_xy[0], sum_xy[1], list); > } > > function list_seq(m, n) { > var result = []; > var i; > for (i=m; i<=n; i++) { > result[i-m] = {}; > result[i-m] = m; > } > return result; > } > > function list_rand(seq) { > var result = []; > for (var i in seq) { > result[i] = {}; > result[i] = []; > result[i][0] = Math.floor(Math.random() * 1000 + 1); > result[i][1] = Math.floor(Math.random() * 1000 + 1); > } > return result; > } > > function seq(m, n) { > var list = list_seq(m, n); > var list1 = list_rand(list); > var head = list1[list1.length-1]; > list1.length--; > var result = fold(move_to, 0, head, list1); > return result[0]; > } > > var start = (new Date()).valueOf(); > console.log(seq(1,1000)); > var end = (new Date()).valueOf(); > console.log(end-start); > // =============================== > > In my computer with Ubuntu 10.10, the JavaScript program performed in 24~25 > milliseconds, and the Erlang one performs about 8.5 milliseconds. Result is > meaningful. > The JavaScript one used some functions, but no recursive functions. Almost > the same > operations were performed, and the Erlang works fine. It may be persuasive. > > > -- > > Best Regards, > > --- Y-H. H. > > > On Sat, Oct 30, 2010 at 4:03 PM, Dmitry Demeshchuk wrote: > >> Greetings. >> >> I'm writing an article comparing Erlang and Node.js and I stumbled >> upon the performance question. >> >> My initial goal was to compare some basic arithmetics speed, like the >> total distance between randomly distributed points. So, I have written >> the following code for Erlang: >> > > >> and the following code for Node.js: >> >> ====================================================== >> >> var a = []; >> for(var i = 0; i < 1000000; i++) { >> a[i] = {}; >> a[i].x = Math.floor(Math.random() * 1000); >> a[i].y = Math.floor(Math.random() * 1000); >> } >> >> var sum = 0; >> >> var start = (new Date()).valueOf(); >> >> for(var i = 1; i < 1000000; i++) { >> var prev = a[i-1]; >> sum += Math.sqrt((a[i].x - prev.x) * (a[i].x - prev.x) + (a[i].y - >> prev.y) * (a[i].y - prev.y)); >> } >> >> var end = (new Date()).valueOf(); >> >> console.log(end - start); >> >> ============================================ >> >> -- Best Regards. --- Y-H. H. From francesco@REDACTED Sun Oct 31 10:54:11 2010 From: francesco@REDACTED (Francesco Cesarini) Date: Sun, 31 Oct 2010 09:54:11 +0000 Subject: 2010 Erlang User Conference Early Bird Registration Deadline Tonight (31/10)! Message-ID: <4CCD3CC3.2060406@erlang-solutions.com> Hi all, a reminder that today is the Early Bird deadline for the 2010 Erlang User Conference. You can register on the user conference website: https://www.erlang-factory.com/conference/ErlangUserConference2010/register With talks from companies such as Google, Motorola, Ericsson, Basho, Ericsson, Klarna, Lambda Stream, Quviq, Gemini Mobile, Erlang Solutions, Pikko Tekk, Mobile Interactive Group as well as research institutions including the University Kent, Eotvos Lorand, Universidad Politecnica de Madrid, it is an event not to be missed. http://www.erlang-factory.com/conference/ErlangUserConference2010/talks We are well over the 250 delegate record from 2009, and have reconfigured the seating arrangement adding more space. But places are limited, so if you are sure you want to attend, sign up before we sell out again. Francesco -- Erlang Solutions Ltd. http://www.erlang-solutions.com From alessandro.sivieri@REDACTED Sun Oct 31 12:38:16 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Sun, 31 Oct 2010 12:38:16 +0100 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> <1288470480.15939.8.camel@blackrain> <1288473799.15939.10.camel@blackrain> <1288474817.15939.16.camel@blackrain> Message-ID: I also want to add that I think that Ubuntu 11.04 will have R14, because Debian testing already has it, so I cannot see any good reason for not having it; of course, if you are the official maintainer, then you may have good reasons for it :) -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From vincenzo.di.somma@REDACTED Sun Oct 31 14:48:37 2010 From: vincenzo.di.somma@REDACTED (Vincenzo Di Somma) Date: Sun, 31 Oct 2010 09:48:37 -0400 Subject: [erlang-questions] debian & ubuntu R14B In-Reply-To: References: <1288274732.2975.54.camel@cyclone> <20101028184852.GA33595@alumni.caltech.edu> <1288470480.15939.8.camel@blackrain> <1288473799.15939.10.camel@blackrain> <1288474817.15939.16.camel@blackrain> Message-ID: <1288532917.6645.0.camel@blackrain> I'm not the maintainer, I'm just very interested in having the best possible Erlang story in both Ubuntu and Debian. vds On Sun, 2010-10-31 at 12:38 +0100, Alessandro Sivieri wrote: > I also want to add that I think that Ubuntu 11.04 will have R14, > because Debian testing already has it, so I cannot see any good reason > for not having it; of course, if you are the official maintainer, then > you may have good reasons for it :) > > -- > Sivieri Alessandro > alessandro.sivieri@REDACTED > http://www.chimera-bellerofonte.eu/ > http://www.poul.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From mathiasstalas@REDACTED Sun Oct 31 17:41:31 2010 From: mathiasstalas@REDACTED (Mathias) Date: Sun, 31 Oct 2010 17:41:31 +0100 Subject: [erlang-questions] Erlang re:run regular exp, match problrm In-Reply-To: References: <23BB6CB9-D0EA-4053-9630-627294D6D79B@gmail.com> Message-ID: Hi, That expression was actually the first one I tried out, with the only difference that I did It from within my application. This was before I knew that file:read_file and UTF-8 don't blend well. I used file:read_file to read in my UTF-8 encoded file... I only tried dilfen's expression from the CLI, and there it succeeded. At the point of posting I had tried so many different solutions that I was tired and just wanted my rather simple laboratory application to work. Later I found out that neither did that sample he gave work from my programs application scope or the one proposed form you. Further investigation has lead me to believe that re:run/3 has either an issue with strings lacking of linefeed. Putting some non valid xml in a file and using my rather simple program always yields([[{0,162}]]) the first char and the last points(see attached doc) ending char '>' as a match, it doesn't split them up as expected which is either the expected module behaviour which I find a bit odd or a(god forbid) programming fault from my part, Here is the code I use: -module(mock). -export([start/0, read_file/1, decode_data/1, find_pattern/3]). start() -> Bin = read_file("point.xml"), UnicodeString = decode_data(Bin), NodeList = find_pattern(UnicodeString, "", [unicode, global]), NodeList. read_file(File) -> case file:read_file(File) of {ok, Bin} -> Bin; _ -> [] end. decode_data(Data) -> case unicode:characters_to_list(Data, utf8) of {error, Encoded, Rest} -> io:format("Caught Error~w~n", Encoded, Rest), []; List -> List end. find_pattern(Str, Pattern, Options) -> case re:run(Str, Pattern, Options) of {match, Part} -> io:format("find_pattern: ~w~n", [Part]), Part; nomatch -> [] end. However, adding a linefeed '\n' after each entity in the doc will give the expected result: [[{0,27}],[{28,27}],[{56,27}],[{84,27}],[{112,27}],[{140,27}]] which to me looks strange. Haven't read up on the re module that much but this is my experience. I have resigned to using xmerl_xpath which seems to do the job. I guess me coming from the Java world is a bit spoiled with strong support for string manipulation and doing the above would have taken men less then 10 min. Anyway Thank you both for the effort. BR, Mathias Stal?s On Sat, Oct 30, 2010 at 8:47 PM, Hynek Vychodil wrote: > I would not thanks on your place. It doesn't do what you want but > works only by accident in this particular example. [^ any char except of '<', 'p', 'o', 'i', 'n', 't'. [^<]* would work in > same way in this particular example. > > This would work much more generally > > re:run(" y=\"2\"z=\"14\"/>", "",[global]) > > but anyway you should use xml parser for xml parsing because xml is > not parseable by regular grammar so regular expression is not proper > tool for do it. You will end up with error prone solution. > > On Sat, Oct 30, 2010 at 1:34 PM, Mathias wrote: > > Works like a charm! > > > > Many thanks dlfen! > > > > BR, > > Mathias > > > > On Sat, Oct 30, 2010 at 1:03 PM, dlfen wrote: > > > >> try this. > >> re:run(" >> z=\"14\"/>", "",[global]). > >> > >> > >> ? 2010-10-30???6:44? Mathias ??? > >> > >> > Hi there, > >> > > >> > I'm trying to figure out how Erlangs re:run module works. > >> > > >> > When executing this:: > >> > 1> re:run(" >> > z=\"14\"/>", ""). > >> > {match,[{0,54}]} > >> > > >> > I can see that it gives me a match on the complete XML representation > >> > {match,[{0,54}]}. > >> > > >> > But what I really would like to do is for it to give me a subset of > >> matches > >> > on each entity similar to {match,[{0,26},{27, 26}]}. > >> > > >> > so the output would yield something like this: > >> > 0-26 gives the first xml entity complete with it's attributes >> x="12" > >> > y="2" z="4"/> and > >> > match 27,26 gives the remaining entity. > >> > > >> > If anyone can spot why my regexp: is failing and > guide > >> me > >> > in the right direction closer to find the solution it will be greatly > >> > appreciated. > >> > > >> > I know about xmerl but for my trivial case it seems like overkill. > >> > > >> > Thx in advance. > >> > > >> > BR, > >> > Mathias Stal?s > >> > >> > > > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: point.xml Type: text/xml Size: 163 bytes Desc: not available URL: From sigmastar@REDACTED Sun Oct 31 17:58:04 2010 From: sigmastar@REDACTED (Jesse Gumm) Date: Sun, 31 Oct 2010 11:58:04 -0500 Subject: [erlang-questions] Erlang re:run regular exp, match problrm In-Reply-To: References: <23BB6CB9-D0EA-4053-9630-627294D6D79B@gmail.com> Message-ID: The problem in that last example is that by default * is greedy and . doesn't match the linefeed (which is why putting \n at the end of each thing worked) "" will match from the first instance of "" Changing it to: "" will make * act "ungreedy" and only match at the first instance it finds, then end. Alternatively, you could use "]*\/>" then you don't really have to worry about greediness or not. -Jesse On Sun, Oct 31, 2010 at 11:41 AM, Mathias wrote: > Hi, > > That expression was actually the first one I tried out, with the only > difference that I did It from within my application. This was before I knew > that file:read_file and UTF-8 don't blend well. I used file:read_file to > read in my UTF-8 encoded file... I only tried dilfen's expression from the > CLI, and there it succeeded. At the point of posting I had tried so many > different solutions that I was tired and just wanted my rather simple > laboratory application to work. Later I found out that neither did that > sample he gave work from my programs application scope or the one proposed > form you. > > Further investigation has lead me to believe that re:run/3 has either an > issue with strings lacking of linefeed. > Putting some non valid xml in a file and using my rather simple program > always yields([[{0,162}]]) the first char and the last points(see attached > doc) ending char '>' as a match, it doesn't split them up as expected which > is either the expected module behaviour which I find a bit odd or a(god > forbid) programming fault from my part, Here is the code I use: > > -module(mock). > -export([start/0, read_file/1, decode_data/1, find_pattern/3]). > > start() -> > Bin = read_file("point.xml"), > UnicodeString = decode_data(Bin), > NodeList = find_pattern(UnicodeString, "", [unicode, > global]), > NodeList. > > read_file(File) -> > case file:read_file(File) of > {ok, Bin} -> Bin; > _ -> [] > end. > > decode_data(Data) -> > case unicode:characters_to_list(Data, utf8) of > {error, Encoded, Rest} -> > io:format("Caught Error~w~n", Encoded, Rest), > []; > List -> > List > end. > > find_pattern(Str, Pattern, Options) -> > case re:run(Str, Pattern, Options) of > {match, Part} -> > io:format("find_pattern: ~w~n", [Part]), > Part; > nomatch -> [] > end. > > However, adding a linefeed '\n' after each entity in the doc will give the > expected result: > [[{0,27}],[{28,27}],[{56,27}],[{84,27}],[{112,27}],[{140,27}]] > which to me looks strange. Haven't read up on the re module that much but > this is my experience. > > I have resigned to using xmerl_xpath which seems to do the job. I guess me > coming from the Java world is a bit spoiled with strong support for string > manipulation and doing the above would have taken men less then 10 min. > > Anyway Thank you both for the effort. > > BR, > Mathias Stal?s > > > > > On Sat, Oct 30, 2010 at 8:47 PM, Hynek Vychodil wrote: > >> I would not thanks on your place. It doesn't do what you want but >> works only by accident in this particular example. [^> any char except of '<', 'p', 'o', 'i', 'n', 't'. [^<]* would work in >> same way in this particular example. >> >> This would work much more generally >> >> re:run("> y=\"2\"z=\"14\"/>", "",[global]) >> >> but anyway you should use xml parser for xml parsing because xml is >> not parseable by regular grammar so regular expression is not proper >> tool for do it. You will end up with error prone solution. >> >> On Sat, Oct 30, 2010 at 1:34 PM, Mathias wrote: >> > Works like a charm! >> > >> > Many thanks dlfen! >> > >> > BR, >> > Mathias >> > >> > On Sat, Oct 30, 2010 at 1:03 PM, dlfen wrote: >> > >> >> try this. >> >> re:run("> >> z=\"14\"/>", "",[global]). >> >> >> >> >> >> ? 2010-10-30???6:44? Mathias ??? >> >> >> >> > Hi there, >> >> > >> >> > I'm trying to figure out how Erlangs re:run module works. >> >> > >> >> > When executing this:: >> >> > 1> re:run("> >> > z=\"14\"/>", ""). >> >> > {match,[{0,54}]} >> >> > >> >> > I can see that it gives me a match on the complete XML representation >> >> > {match,[{0,54}]}. >> >> > >> >> > But what I really would like to do is for it to give me a subset of >> >> matches >> >> > on each entity similar to {match,[{0,26},{27, 26}]}. >> >> > >> >> > so the output would yield something like this: >> >> > 0-26 gives the first xml entity complete with it's attributes > >> x="12" >> >> > y="2" z="4"/> and >> >> > match 27,26 gives the remaining entity. >> >> > >> >> > If anyone can spot why my regexp: is failing and >> guide >> >> me >> >> > in the right direction closer to find the solution it will be greatly >> >> > appreciated. >> >> > >> >> > I know about xmerl but for my trivial case it seems like overkill. >> >> > >> >> > Thx in advance. >> >> > >> >> > BR, >> >> > Mathias Stal?s >> >> >> >> >> > >> >> >> >> -- >> --Hynek (Pichi) Vychodil >> >> Analyze your data in minutes. Share your insights instantly. Thrill >> your boss. Be a data hero! >> Try GoodData now for free: www.gooddata.com >> > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Jesse Gumm Sigma Star Systems 414.940.4866 gumm@REDACTED http://www.sigma-star.com From mathiasstalas@REDACTED Sun Oct 31 18:09:25 2010 From: mathiasstalas@REDACTED (Mathias) Date: Sun, 31 Oct 2010 18:09:25 +0100 Subject: [erlang-questions] Erlang re:run regular exp, match problrm In-Reply-To: References: <23BB6CB9-D0EA-4053-9630-627294D6D79B@gmail.com> Message-ID: Thank you for the clarification Jesse. Nicely explained. I have tried it and it worked. BR, Mathias Stal?s On Sun, Oct 31, 2010 at 5:58 PM, Jesse Gumm wrote: > The problem in that last example is that by default * is greedy and . > doesn't match the linefeed (which is why putting \n at the end of each thing > worked) > > "" will match from the first instance of " last "/>" > > Changing it to: > > "" will make * act "ungreedy" and only match at the first > instance it finds, then end. > > Alternatively, you could use > > "]*\/>" then you don't really have to worry about greediness or > not. > > -Jesse > > > On Sun, Oct 31, 2010 at 11:41 AM, Mathias wrote: > >> Hi, >> >> That expression was actually the first one I tried out, with the only >> difference that I did It from within my application. This was before I knew >> that file:read_file and UTF-8 don't blend well. I used file:read_file to >> read in my UTF-8 encoded file... I only tried dilfen's expression from the >> CLI, and there it succeeded. At the point of posting I had tried so many >> different solutions that I was tired and just wanted my rather simple >> laboratory application to work. Later I found out that neither did that >> sample he gave work from my programs application scope or the one proposed >> form you. >> >> Further investigation has lead me to believe that re:run/3 has either an >> issue with strings lacking of linefeed. >> Putting some non valid xml in a file and using my rather simple program >> always yields([[{0,162}]]) the first char and the last points(see attached >> doc) ending char '>' as a match, it doesn't split them up as expected which >> is either the expected module behaviour which I find a bit odd or a(god >> forbid) programming fault from my part, Here is the code I use: >> >> -module(mock). >> -export([start/0, read_file/1, decode_data/1, find_pattern/3]). >> >> start() -> >> Bin = read_file("point.xml"), >> UnicodeString = decode_data(Bin), >> NodeList = find_pattern(UnicodeString, "", [unicode, >> global]), >> NodeList. >> >> read_file(File) -> >> case file:read_file(File) of >> {ok, Bin} -> Bin; >> _ -> [] >> end. >> >> decode_data(Data) -> >> case unicode:characters_to_list(Data, utf8) of >> {error, Encoded, Rest} -> >> io:format("Caught Error~w~n", Encoded, Rest), >> []; >> List -> >> List >> end. >> >> find_pattern(Str, Pattern, Options) -> >> case re:run(Str, Pattern, Options) of >> {match, Part} -> >> io:format("find_pattern: ~w~n", [Part]), >> Part; >> nomatch -> [] >> end. >> >> However, adding a linefeed '\n' after each entity in the doc will give the >> expected result: >> [[{0,27}],[{28,27}],[{56,27}],[{84,27}],[{112,27}],[{140,27}]] >> which to me looks strange. Haven't read up on the re module that much but >> this is my experience. >> >> I have resigned to using xmerl_xpath which seems to do the job. I guess me >> coming from the Java world is a bit spoiled with strong support for string >> manipulation and doing the above would have taken men less then 10 min. >> >> Anyway Thank you both for the effort. >> >> BR, >> Mathias Stal?s >> >> >> >> >> On Sat, Oct 30, 2010 at 8:47 PM, Hynek Vychodil wrote: >> >>> I would not thanks on your place. It doesn't do what you want but >>> works only by accident in this particular example. [^>> any char except of '<', 'p', 'o', 'i', 'n', 't'. [^<]* would work in >>> same way in this particular example. >>> >>> This would work much more generally >>> >>> re:run(">> y=\"2\"z=\"14\"/>", "",[global]) >>> >>> but anyway you should use xml parser for xml parsing because xml is >>> not parseable by regular grammar so regular expression is not proper >>> tool for do it. You will end up with error prone solution. >>> >>> On Sat, Oct 30, 2010 at 1:34 PM, Mathias >>> wrote: >>> > Works like a charm! >>> > >>> > Many thanks dlfen! >>> > >>> > BR, >>> > Mathias >>> > >>> > On Sat, Oct 30, 2010 at 1:03 PM, dlfen wrote: >>> > >>> >> try this. >>> >> re:run(">> >> z=\"14\"/>", "",[global]). >>> >> >>> >> >>> >> ? 2010-10-30???6:44? Mathias ??? >>> >> >>> >> > Hi there, >>> >> > >>> >> > I'm trying to figure out how Erlangs re:run module works. >>> >> > >>> >> > When executing this:: >>> >> > 1> re:run(">> >> > z=\"14\"/>", ""). >>> >> > {match,[{0,54}]} >>> >> > >>> >> > I can see that it gives me a match on the complete XML >>> representation >>> >> > {match,[{0,54}]}. >>> >> > >>> >> > But what I really would like to do is for it to give me a subset of >>> >> matches >>> >> > on each entity similar to {match,[{0,26},{27, 26}]}. >>> >> > >>> >> > so the output would yield something like this: >>> >> > 0-26 gives the first xml entity complete with it's attributes >> >> x="12" >>> >> > y="2" z="4"/> and >>> >> > match 27,26 gives the remaining entity. >>> >> > >>> >> > If anyone can spot why my regexp: is failing and >>> guide >>> >> me >>> >> > in the right direction closer to find the solution it will be >>> greatly >>> >> > appreciated. >>> >> > >>> >> > I know about xmerl but for my trivial case it seems like overkill. >>> >> > >>> >> > Thx in advance. >>> >> > >>> >> > BR, >>> >> > Mathias Stal?s >>> >> >>> >> >>> > >>> >>> >>> >>> -- >>> --Hynek (Pichi) Vychodil >>> >>> Analyze your data in minutes. Share your insights instantly. Thrill >>> your boss. Be a data hero! >>> Try GoodData now for free: www.gooddata.com >>> >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > > -- > Jesse Gumm > Sigma Star Systems > 414.940.4866 > gumm@REDACTED > http://www.sigma-star.com > From tomas.abrahamsson@REDACTED Sun Oct 31 19:58:35 2010 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Sun, 31 Oct 2010 19:58:35 +0100 Subject: [erlang-questions] Extracting detailed information from TypEr/Dialyzer In-Reply-To: References: Message-ID: On Tue, Oct 12, 2010 at 09:00, Torben Hoffmann wrote: > Hi, > > I want to extract the following information from a gen_fsm: > > ? - all the states > ? - the incoming events > ? - possible next states > > but I want to do it per function clause and not only on the function level > as TypEr and Dialyzer does it. One way to do that is to copy each function clause into a separate function. Then one can run TypEr on the expanded code. Attached is an escript that does such an expansion of exported functions of arity 2 and 3, and also of handle_event, handle_sync_event and handle_info. The script accepts -I and -D options for include paths and macro definitions, just like erlc,and prints the expanded program on stdout. I've tried plotting state machines from the output of TypEr when run on the expanded results, and it seems to do reasonably well. I haven't done any thorough verification. It doesn't solve all of your problems: it doesn't find any outgoing messages, and it has the same limitations you already indicated when returning from a function clause in different ways, so I guess the results are roughly what you already seems to have, or what Vance's drawing tool already produces. Somehow being able to make more use of TypEr's knowledge about a file would be really interesting, for example to be able to query it for type information given various execution paths through the program, if it would be possible. BRs Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: fsm_clause_separator.es Type: application/ecmascript Size: 6080 bytes Desc: not available URL: From mk@REDACTED Sun Oct 31 22:08:38 2010 From: mk@REDACTED (Morten Krogh) Date: Sun, 31 Oct 2010 22:08:38 +0100 Subject: [erlang-questions] Erlang re:run regular exp, match problrm In-Reply-To: References: <23BB6CB9-D0EA-4053-9630-627294D6D79B@gmail.com> Message-ID: <4CCDDAD6.90805@amberbio.com> Hi Take care, '>' is an allowed character in attribute values, e.g., is valid xml. If you control the input yourself it is no problem, of course. Morten. On 10/31/10 5:58 PM, Jesse Gumm wrote: > The problem in that last example is that by default * is greedy and . > doesn't match the linefeed (which is why putting \n at the end of each thing > worked) > > "" will match from the first instance of" last "/>" > > Changing it to: > > "" will make * act "ungreedy" and only match at the first > instance it finds, then end. > > Alternatively, you could use > > "]*\/>" then you don't really have to worry about greediness or > not. > > -Jesse > > > On Sun, Oct 31, 2010 at 11:41 AM, Mathias wrote: > >> Hi, >> >> That expression was actually the first one I tried out, with the only >> difference that I did It from within my application. This was before I knew >> that file:read_file and UTF-8 don't blend well. I used file:read_file to >> read in my UTF-8 encoded file... I only tried dilfen's expression from the >> CLI, and there it succeeded. At the point of posting I had tried so many >> different solutions that I was tired and just wanted my rather simple >> laboratory application to work. Later I found out that neither did that >> sample he gave work from my programs application scope or the one proposed >> form you. >> >> Further investigation has lead me to believe that re:run/3 has either an >> issue with strings lacking of linefeed. >> Putting some non valid xml in a file and using my rather simple program >> always yields([[{0,162}]]) the first char and the last points(see attached >> doc) ending char '>' as a match, it doesn't split them up as expected which >> is either the expected module behaviour which I find a bit odd or a(god >> forbid) programming fault from my part, Here is the code I use: >> >> -module(mock). >> -export([start/0, read_file/1, decode_data/1, find_pattern/3]). >> >> start() -> >> Bin = read_file("point.xml"), >> UnicodeString = decode_data(Bin), >> NodeList = find_pattern(UnicodeString, "", [unicode, >> global]), >> NodeList. >> >> read_file(File) -> >> case file:read_file(File) of >> {ok, Bin} -> Bin; >> _ -> [] >> end. >> >> decode_data(Data) -> >> case unicode:characters_to_list(Data, utf8) of >> {error, Encoded, Rest} -> >> io:format("Caught Error~w~n", Encoded, Rest), >> []; >> List -> >> List >> end. >> >> find_pattern(Str, Pattern, Options) -> >> case re:run(Str, Pattern, Options) of >> {match, Part} -> >> io:format("find_pattern: ~w~n", [Part]), >> Part; >> nomatch -> [] >> end. >> >> However, adding a linefeed '\n' after each entity in the doc will give the >> expected result: >> [[{0,27}],[{28,27}],[{56,27}],[{84,27}],[{112,27}],[{140,27}]] >> which to me looks strange. Haven't read up on the re module that much but >> this is my experience. >> >> I have resigned to using xmerl_xpath which seems to do the job. I guess me >> coming from the Java world is a bit spoiled with strong support for string >> manipulation and doing the above would have taken men less then 10 min. >> >> Anyway Thank you both for the effort. >> >> BR, >> Mathias Stal?s >> >> >> >> >> On Sat, Oct 30, 2010 at 8:47 PM, Hynek Vychodilwrote: >> >>> I would not thanks on your place. It doesn't do what you want but >>> works only by accident in this particular example. [^>> any char except of '<', 'p', 'o', 'i', 'n', 't'. [^<]* would work in >>> same way in this particular example. >>> >>> This would work much more generally >>> >>> re:run(">> y=\"2\"z=\"14\"/>","",[global]) >>> >>> but anyway you should use xml parser for xml parsing because xml is >>> not parseable by regular grammar so regular expression is not proper >>> tool for do it. You will end up with error prone solution. >>> >>> On Sat, Oct 30, 2010 at 1:34 PM, Mathias wrote: >>>> Works like a charm! >>>> >>>> Many thanks dlfen! >>>> >>>> BR, >>>> Mathias >>>> >>>> On Sat, Oct 30, 2010 at 1:03 PM, dlfen wrote: >>>> >>>>> try this. >>>>> re:run(">>>> z=\"14\"/>","",[global]). >>>>> >>>>> >>>>> ? 2010-10-30???6:44? Mathias ??? >>>>> >>>>>> Hi there, >>>>>> >>>>>> I'm trying to figure out how Erlangs re:run module works. >>>>>> >>>>>> When executing this:: >>>>>> 1> re:run(">>>>> z=\"14\"/>",""). >>>>>> {match,[{0,54}]} >>>>>> >>>>>> I can see that it gives me a match on the complete XML representation >>>>>> {match,[{0,54}]}. >>>>>> >>>>>> But what I really would like to do is for it to give me a subset of >>>>> matches >>>>>> on each entity similar to {match,[{0,26},{27, 26}]}. >>>>>> >>>>>> so the output would yield something like this: >>>>>> 0-26 gives the first xml entity complete with it's attributes>>>> x="12" >>>>>> y="2" z="4"/> and >>>>>> match 27,26 gives the remaining entity. >>>>>> >>>>>> If anyone can spot why my regexp: is failing and >>> guide >>>>> me >>>>>> in the right direction closer to find the solution it will be greatly >>>>>> appreciated. >>>>>> >>>>>> I know about xmerl but for my trivial case it seems like overkill. >>>>>> >>>>>> Thx in advance. >>>>>> >>>>>> BR, >>>>>> Mathias Stal?s >>>>> >>> >>> >>> -- >>> --Hynek (Pichi) Vychodil >>> >>> Analyze your data in minutes. Share your insights instantly. Thrill >>> your boss. Be a data hero! >>> Try GoodData now for free: www.gooddata.com >>> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > From mathiasstalas@REDACTED Sun Oct 31 23:05:31 2010 From: mathiasstalas@REDACTED (Mathias) Date: Sun, 31 Oct 2010 23:05:31 +0100 Subject: [erlang-questions] Erlang re:run regular exp, match problrm In-Reply-To: <4CCDDAD6.90805@amberbio.com> References: <23BB6CB9-D0EA-4053-9630-627294D6D79B@gmail.com> <4CCDDAD6.90805@amberbio.com> Message-ID: I'm just goofing around at the moment so no worries. I found xmerl_xpath to be a good friend at the moment. It's a pity(IMOP) that the documentation on how to use these library is gently speaking sparse. On the other side I'm glad that they exist, and there are some good examples in other open source projects that have some nice coding where one can pick up some bits and pieces and figure things out. BR, Mathias Stal?s On Sun, Oct 31, 2010 at 10:08 PM, Morten Krogh wrote: > Hi > > Take care, '>' is an allowed character in attribute values, e.g., > > > > is valid xml. > > If you control the input yourself it is no problem, of course. > > Morten. > > > > On 10/31/10 5:58 PM, Jesse Gumm wrote: > >> The problem in that last example is that by default * is greedy and . >> doesn't match the linefeed (which is why putting \n at the end of each >> thing >> worked) >> >> "" will match from the first instance of"> last "/>" >> >> Changing it to: >> >> "" will make * act "ungreedy" and only match at the first >> instance it finds, then end. >> >> Alternatively, you could use >> >> "]*\/>" then you don't really have to worry about greediness or >> not. >> >> -Jesse >> >> >> On Sun, Oct 31, 2010 at 11:41 AM, Mathias >> wrote: >> >> Hi, >>> >>> That expression was actually the first one I tried out, with the only >>> difference that I did It from within my application. This was before I >>> knew >>> that file:read_file and UTF-8 don't blend well. I used file:read_file to >>> read in my UTF-8 encoded file... I only tried dilfen's expression from >>> the >>> CLI, and there it succeeded. At the point of posting I had tried so many >>> different solutions that I was tired and just wanted my rather simple >>> laboratory application to work. Later I found out that neither did that >>> sample he gave work from my programs application scope or the one >>> proposed >>> form you. >>> >>> Further investigation has lead me to believe that re:run/3 has either an >>> issue with strings lacking of linefeed. >>> Putting some non valid xml in a file and using my rather simple program >>> always yields([[{0,162}]]) the first char and the last points(see >>> attached >>> doc) ending char '>' as a match, it doesn't split them up as expected >>> which >>> is either the expected module behaviour which I find a bit odd or a(god >>> forbid) programming fault from my part, Here is the code I use: >>> >>> -module(mock). >>> -export([start/0, read_file/1, decode_data/1, find_pattern/3]). >>> >>> start() -> >>> Bin = read_file("point.xml"), >>> UnicodeString = decode_data(Bin), >>> NodeList = find_pattern(UnicodeString, "", [unicode, >>> global]), >>> NodeList. >>> >>> read_file(File) -> >>> case file:read_file(File) of >>> {ok, Bin} -> Bin; >>> _ -> [] >>> end. >>> >>> decode_data(Data) -> >>> case unicode:characters_to_list(Data, utf8) of >>> {error, Encoded, Rest} -> >>> io:format("Caught Error~w~n", Encoded, Rest), >>> []; >>> List -> >>> List >>> end. >>> >>> find_pattern(Str, Pattern, Options) -> >>> case re:run(Str, Pattern, Options) of >>> {match, Part} -> >>> io:format("find_pattern: ~w~n", [Part]), >>> Part; >>> nomatch -> [] >>> end. >>> >>> However, adding a linefeed '\n' after each entity in the doc will give >>> the >>> expected result: >>> [[{0,27}],[{28,27}],[{56,27}],[{84,27}],[{112,27}],[{140,27}]] >>> which to me looks strange. Haven't read up on the re module that much but >>> this is my experience. >>> >>> I have resigned to using xmerl_xpath which seems to do the job. I guess >>> me >>> coming from the Java world is a bit spoiled with strong support for >>> string >>> manipulation and doing the above would have taken men less then 10 min. >>> >>> Anyway Thank you both for the effort. >>> >>> BR, >>> Mathias Stal?s >>> >>> >>> >>> >>> On Sat, Oct 30, 2010 at 8:47 PM, Hynek Vychodil>> >wrote: >>> >>> I would not thanks on your place. It doesn't do what you want but >>>> works only by accident in this particular example. [^>>> any char except of '<', 'p', 'o', 'i', 'n', 't'. [^<]* would work in >>>> same way in this particular example. >>>> >>>> This would work much more generally >>>> >>>> re:run(">>> y=\"2\"z=\"14\"/>","",[global]) >>>> >>>> but anyway you should use xml parser for xml parsing because xml is >>>> not parseable by regular grammar so regular expression is not proper >>>> tool for do it. You will end up with error prone solution. >>>> >>>> On Sat, Oct 30, 2010 at 1:34 PM, Mathias >>>> wrote: >>>> >>>>> Works like a charm! >>>>> >>>>> Many thanks dlfen! >>>>> >>>>> BR, >>>>> Mathias >>>>> >>>>> On Sat, Oct 30, 2010 at 1:03 PM, dlfen wrote: >>>>> >>>>> try this. >>>>>> re:run(">>>>> z=\"14\"/>","",[global]). >>>>>> >>>>>> >>>>>> ? 2010-10-30???6:44? Mathias ??? >>>>>> >>>>>> Hi there, >>>>>>> >>>>>>> I'm trying to figure out how Erlangs re:run module works. >>>>>>> >>>>>>> When executing this:: >>>>>>> 1> re:run(">>>>>> z=\"14\"/>",""). >>>>>>> {match,[{0,54}]} >>>>>>> >>>>>>> I can see that it gives me a match on the complete XML representation >>>>>>> {match,[{0,54}]}. >>>>>>> >>>>>>> But what I really would like to do is for it to give me a subset of >>>>>>> >>>>>> matches >>>>>> >>>>>>> on each entity similar to {match,[{0,26},{27, 26}]}. >>>>>>> >>>>>>> so the output would yield something like this: >>>>>>> 0-26 gives the first xml entity complete with it's attributes>>>>>> >>>>>> x="12" >>>>>> >>>>>>> y="2" z="4"/> and >>>>>>> match 27,26 gives the remaining entity. >>>>>>> >>>>>>> If anyone can spot why my regexp: is failing and >>>>>>> >>>>>> guide >>>> >>>>> me >>>>>> >>>>>>> in the right direction closer to find the solution it will be greatly >>>>>>> appreciated. >>>>>>> >>>>>>> I know about xmerl but for my trivial case it seems like overkill. >>>>>>> >>>>>>> Thx in advance. >>>>>>> >>>>>>> BR, >>>>>>> Mathias Stal?s >>>>>>> >>>>>> >>>>>> >>>> >>>> -- >>>> --Hynek (Pichi) Vychodil >>>> >>>> Analyze your data in minutes. Share your insights instantly. Thrill >>>> your boss. Be a data hero! >>>> Try GoodData now for free: www.gooddata.com >>>> >>>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >