From alexander.uvarov@REDACTED Tue Sep 1 01:07:29 2009 From: alexander.uvarov@REDACTED (Alexander Uvarov) Date: Tue, 1 Sep 2009 05:07:29 +0600 Subject: [erlang-questions] Efficient way to select a range of data between 2 values? In-Reply-To: <97619b170908310935q4a90bb7ag644e68dcfd33d604@mail.gmail.com> References: <97619b170908310935q4a90bb7ag644e68dcfd33d604@mail.gmail.com> Message-ID: <6B57F916-67C9-49E6-B9FE-3C9448F675A3@gmail.com> On Aug 31, 2009, at 10:35 PM, Rapsey wrote: That's awesome! Finished in 18.878169 seconds. # 2.000.000 lookups Finished in 14.843154 seconds. # 1000 lookups (using ets) > > Result for 100k: > geoip:loop(100000, T). > {1235,667762,860389} > {1235,667763,299972} > About half a second. Pretty good no? > > > Sergej > > On Mon, Aug 31, 2009 at 6:25 PM, Alexander Uvarov < > alexander.uvarov@REDACTED> wrote: From yogishb@REDACTED Tue Sep 1 00:17:22 2009 From: yogishb@REDACTED (Yogish Baliga) Date: Mon, 31 Aug 2009 15:17:22 -0700 (PDT) Subject: gen_server terminate stopping other application like mnesia Message-ID: <860894.74152.qm@web31808.mail.mud.yahoo.com> My app has one supervisor which in turn start few gen_server (standard OTP). start_link of one of the gen_server, start several other applications. To ensure the clean shutdown, this gen_server's terminate function call application:stop(..) on the dependent applications. While shutting down the whole VM via init:stop(), my app's supervisor get's brutally killed as the gen_server 's terminate function does not return. It gets stuck in application:stop(..) of dependent application (including mnesia). Is this the desired behavior? Thanx, -- baliga "Point of view is worth 80 IQ points" --Alan Kay http://dudefrommangalore.blogspot.com/ From rvirding@REDACTED Tue Sep 1 01:27:51 2009 From: rvirding@REDACTED (Robert Virding) Date: Tue, 1 Sep 2009 01:27:51 +0200 Subject: [erlang-questions] Efficient way to select a range of data between 2 values? In-Reply-To: <6B57F916-67C9-49E6-B9FE-3C9448F675A3@gmail.com> References: <97619b170908310935q4a90bb7ag644e68dcfd33d604@mail.gmail.com> <6B57F916-67C9-49E6-B9FE-3C9448F675A3@gmail.com> Message-ID: <3dbc6d1c0908311627q458d8298jffae5bf3c6a61ba2@mail.gmail.com> Sorry, I can't stop myself from making a comment here. This is a wonderful example of the importance of choosing the right data structures and that no structure is good for everything. I know this is a trivial, self-evident and well-known fact, but it is so trivial that we tend to forget it. Anyway, enough moralising, Robert 2009/9/1 Alexander Uvarov > > On Aug 31, 2009, at 10:35 PM, Rapsey wrote: > > That's awesome! > > Finished in 18.878169 seconds. # 2.000.000 lookups > Finished in 14.843154 seconds. # 1000 lookups (using ets) > > >> Result for 100k: >> geoip:loop(100000, T). >> {1235,667762,860389} >> {1235,667763,299972} >> About half a second. Pretty good no? >> >> >> Sergej >> >> On Mon, Aug 31, 2009 at 6:25 PM, Alexander Uvarov < >> alexander.uvarov@REDACTED> wrote: >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From nem@REDACTED Tue Sep 1 02:51:15 2009 From: nem@REDACTED (Geoff Cant) Date: Tue, 01 Sep 2009 02:51:15 +0200 Subject: [erlang-questions] gen_server terminate stopping other application like mnesia In-Reply-To: <860894.74152.qm@web31808.mail.mud.yahoo.com> (Yogish Baliga's message of "Mon, 31 Aug 2009 15:17:22 -0700 (PDT)") References: <860894.74152.qm@web31808.mail.mud.yahoo.com> Message-ID: Yogish Baliga writes: > My app has one supervisor which in turn start few gen_server (standard OTP). start_link of one of the gen_server, start several other applications. To ensure the clean shutdown, this gen_server's terminate function call application:stop(..) on the dependent applications. > > While shutting down the whole VM via init:stop(), my app's supervisor get's brutally killed as the gen_server 's terminate function does not return. It gets stuck in application:stop(..) of dependent application (including mnesia). > > Is this the desired behavior? > I think that at system shutdown time (after calling init:stop()), the application_controller kills all running applications anyway. It does this by calling exit(AppTopLevelPid, shutdown) and waiting for the exit signal {'Exit', AppTopLevelPid, _}. At this point the application_controller won't respond to any application API calls. You don't need to and shouldn't try to shutdown applications at init:stop() time. You could provide a your_app:stop() method to stop your app without shutting down the entire node. It could call application:stop(YourApp), then application:stop/1 on all your dependent apps. It won't be called by the application system and it won't be called during system shutdown, so you won't run into deadlock. Cheers, -- Geoff Cant From ngocdaothanh@REDACTED Tue Sep 1 04:53:26 2009 From: ngocdaothanh@REDACTED (ngocdaothanh) Date: Mon, 31 Aug 2009 19:53:26 -0700 (PDT) Subject: Use Mnesia backup on other machines In-Reply-To: <6a3ae47e0908311034h59390e5rd4e5cf0dd5c89c62@mail.gmail.com> References: <6a3ae47e0908311034h59390e5rd4e5cf0dd5c89c62@mail.gmail.com> Message-ID: Thank you for the answers. > Is this about the actual file movement, or about whether a mnesia backup done on cluster A can be restored on cluster B? I mean for a same version of Erlang, is the default backup file cross- platform and can be restored anywhere just like a PostgreSQL backup file? How can I create backup file on a remote server running Ubuntu, then restore it to my local Snow Leopard laptop? I heard somewhere that once created a Mnesia DB will be fixed to a node name, simply copying it doesn't work. > until_next_time() function from crone crone is very interesting. Where can I find the latest version of it? Regards. On Sep 1, 2:34?am, Robert Raschke wrote: > On Mon, Aug 31, 2009 at 5:43 PM, Paul Mineiro wrote: > > > > > Inline. > > > -- p > > > On Sun, 30 Aug 2009, ngocdaothanh wrote: > > > > Hi everyone, > > > > I would like to ask about Mnesia backup: > > > 1. How to periodically create backup of Mnesia DB of a node? > > > You can have a gen_server which uses timer:send_interval/2 combined with > > handle_info/2 to trigger the backups. ?You can back up everything with > > mnesia:backup/1. ?In case the backup takes a very long time you should > > twiddle something in the gen_server state to prevent starting a new backup > > while an old one is happening. > > No need for the timer though, just use the gen_server's own timeout return > value. Combine this with calculating a timeout value using the > until_next_time() function from crone and you can have nicely scheduled > backups. > > Minimal, untested and not even compiled example, started with something like > mybackup:start_link("DB.BAK", {daily, {every, {1, hr}, {between, {12, 0, 0, > am}, {11, 59, 59, pm}}}}) . > > -module(mybackup). > > -behaviour(gen_server). > > -export([start_link/2, init/1, handle_info/2]). > > start_link(Backup_File, Backup_When) -> > ? ? gen_server:start_link({local, ?MODULE}, ?MODULE, [Backup_File, > Backup_When], []). > > init([Backup_File, Backup_When]) -> > ? ? {ok, {Backup_File, Backup_When}, > 1000*crone:until_next_time(Backup_When)}. > > handle_info(timeout, {Backup_File, Backup_When} = State) -> > ? ? mnesia:backup(Backup_File), > ? ? % HACK: Just in case it took less than a second, make sure we don't do > it again right away. > ? ? timer:sleep(1000), > ? ? {noreply, State, 1000*crone:until_next_time(Backup_When)}. > > Robby From shenyute@REDACTED Tue Sep 1 05:52:19 2009 From: shenyute@REDACTED (Yu-Teh Shen) Date: Tue, 1 Sep 2009 11:52:19 +0800 Subject: about socket problem Message-ID: <65055d7c0908312052x16dfffb4sa031d36629b0e047@mail.gmail.com> I got a question about erlang socket. Now I have a server(written by erlang) and a client(written by C). Client create a socket connection to server and then idle for a long time. After that, I kill client process, but I found out that the server didn't know client was been killed! And I use netstat -pan in server, the connection is still exist! Is there anyway, the server can detect client had been killed? Like using poll function in C socket (error code POLLERR or POLLHUP) Thanks a lot! Shen, Yu-Teh From gene.tani@REDACTED Tue Sep 1 10:07:37 2009 From: gene.tani@REDACTED (Gene Tani) Date: Tue, 1 Sep 2009 01:07:37 -0700 (PDT) Subject: Efficient way to select a range of data between 2 values? In-Reply-To: References: Message-ID: <36cfc87c-96c2-40eb-9465-cabc23fc4c4d@r34g2000vba.googlegroups.com> On Aug 31, 9:25?am, Alexander Uvarov wrote: > I have ~100K ip2country records like {ip_address_map, {range_from, ? > range_to, country_code}} and I am searching for fastest way to lookup ? > country_code. There should be matching guard like range_from =< IP >= ? > range_to. Any ideas, guys? I am not satisfied with ets (lookup time ? > from 0.06s up to 1.6s). There's another option, this project in Google code (B Ippolito) http://code.google.com/p/egeoip/ From paf@REDACTED Tue Sep 1 10:51:10 2009 From: paf@REDACTED (Paulo Alexandre Ferreira) Date: Tue, 1 Sep 2009 09:51:10 +0100 Subject: Cross-compiling Erlang? (once again) Message-ID: <20090901085021.M17562@dei.isep.ipp.pt> Any recent advice/files on cross-compiling Erlang? I have seen: 1) the mails from Dec 2008, 2) the links from Brian Zhou: ?http://trac.nslu2-linux.org/optware/browser/trunk/make/erlang.mk ?http://trac.nslu2-linux.org/optware/browser/trunk/sources/erlang 3) The questions from Corrado Santoro and the patches from Matthias Lang in Set 2008 4) The tutorial on trapexit. More questions: Where is --without-java switch in compiling R13? Has anyone seen the sources of erlang-lite anywhere? Sorry if my troubles are somewhere between the chair and the keyboard... Thanks in advance. Paulo Ferreira From rapsey@REDACTED Tue Sep 1 10:58:26 2009 From: rapsey@REDACTED (Rapsey) Date: Tue, 1 Sep 2009 10:58:26 +0200 Subject: [erlang-questions] about socket problem In-Reply-To: <65055d7c0908312052x16dfffb4sa031d36629b0e047@mail.gmail.com> References: <65055d7c0908312052x16dfffb4sa031d36629b0e047@mail.gmail.com> Message-ID: <97619b170909010158tee9bd85v588607bf1b717b02@mail.gmail.com> That is just how sockets work I'm afraid. No matter what language you are using. If your server/client are on the same computer you will get a notification that the socket was closed, but not when you're communicating over the internet. The only way around is to ping the other side every once in a while. Sergej On Tue, Sep 1, 2009 at 5:52 AM, Yu-Teh Shen wrote: > I got a question about erlang socket. > > Now I have a server(written by erlang) and a client(written by C). > > Client create a socket connection to server and then idle for a long time. > > After that, I kill client process, but I found out that the server didn't > know client was been killed! > > And I use netstat -pan in server, the connection is still exist! > > Is there anyway, the server can detect client had been killed? Like using > poll function in C socket (error code POLLERR or POLLHUP) > > Thanks a lot! > > > Shen, Yu-Teh > From egil@REDACTED Tue Sep 1 11:07:05 2009 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Tue, 01 Sep 2009 11:07:05 +0200 Subject: [erlang-questions] Cross-compiling Erlang? (once again) In-Reply-To: <20090901085021.M17562@dei.isep.ipp.pt> References: <20090901085021.M17562@dei.isep.ipp.pt> Message-ID: <4A9CE439.30303@erix.ericsson.se> There is a readme-file in the xcomp directory in the erlang source root. Basically it is setting an environment with a template and then building normally. for example, export ERL_TOP= cd $ERL_TOP eval `./otp_build env_cross $ERL_TOP/xcomp/erl-xcomp-some-env.conf` ./otp_build autoconf ./otp_build configure ./otp_build boot ./otp_build release or in the manner you want. Regards, Bj?rn-Egil Erlang/OTP Paulo Alexandre Ferreira wrote: > > > Any recent advice/files on cross-compiling Erlang? > > > I have seen: > > 1) the mails from Dec 2008, > 2) the links from Brian Zhou: > > http://trac.nslu2-linux.org/optware/browser/trunk/make/erlang.mk > http://trac.nslu2-linux.org/optware/browser/trunk/sources/erlang > > > 3) The questions from Corrado Santoro and the patches from Matthias Lang in Set 2008 > > > 4) The tutorial on trapexit. > > > > More questions: > > Where is --without-java switch in compiling R13? > > Has anyone seen the sources of erlang-lite anywhere? > > > Sorry if my troubles are somewhere between the chair and the keyboard... > > > Thanks in advance. > > Paulo Ferreira > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org From chandrashekhar.mullaparthi@REDACTED Tue Sep 1 12:08:16 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 1 Sep 2009 11:08:16 +0100 Subject: [erlang-questions] Re: Use Mnesia backup on other machines In-Reply-To: References: <6a3ae47e0908311034h59390e5rd4e5cf0dd5c89c62@mail.gmail.com> Message-ID: 2009/9/1 ngocdaothanh > Thank you for the answers. > > > Is this about the actual file movement, or about whether a mnesia backup > done on cluster A can be restored on cluster B? > > I mean for a same version of Erlang, is the default backup file cross- > platform and can be restored anywhere just like a PostgreSQL backup > file? How can I create backup file on a remote server running Ubuntu, > then restore it to my local Snow Leopard laptop? I heard somewhere > that once created a Mnesia DB will be fixed to a node name, simply > copying it doesn't work. > If you just want the contents of some/all tables restored, you can use the backup anywhere. If you want the entire database rebuilt from scratch using the backup, the node names have to match. There is a way to change node names in the backup file if you need the latter. Refer to the Mnesia user guide for details. cheers Chandru From rtrlists@REDACTED Tue Sep 1 12:29:02 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 1 Sep 2009 11:29:02 +0100 Subject: erlsrv start doesn't complete sometimes Message-ID: <6a3ae47e0909010329s406da005o383a8f8f3ab82297@mail.gmail.com> Hi, I'm trying to use erlsrv on Windows to run my app as a service. Mostly it works fine. But I have some strange behaviour when the start of the node takes a bit longer. I have not yet managed to produce a small example. Has anyone else seen issues with an OTP style release not getting started correctly under erlsrv? >From the logs it looks like it just gets stuck in the middle of the application startup phase at some point. I've looked at the erlsrv code and noticed that it keeps a connection to the node's shell. What I am wondering is, could it be possible that the shell needs to be ready for erlsrv within a certain period of time? Because I only get the stuck Node when the startup takes a bit longer (~ 1 minute), thus making the shell appear quite late. If I add -noshell (as told not to by the erlsrv documentation) then everything starts up, but I obviously can't stop it through the Services interface. I'll try and find time to make a small app that exhibits this behaviour, but if anyone has come across this already I'd be happy to hear about it. Thanks, Robby From koug@REDACTED Tue Sep 1 13:31:51 2009 From: koug@REDACTED (John Kougoulos) Date: Tue, 1 Sep 2009 14:31:51 +0300 (GTB Daylight Time) Subject: [erlang-questions] about socket problem In-Reply-To: <97619b170909010158tee9bd85v588607bf1b717b02@mail.gmail.com> References: <65055d7c0908312052x16dfffb4sa031d36629b0e047@mail.gmail.com> <97619b170909010158tee9bd85v588607bf1b717b02@mail.gmail.com> Message-ID: you could also open the socket with SO_KEEPALIVE John On Tue, 1 Sep 2009, Rapsey wrote: > That is just how sockets work I'm afraid. No matter what language you are > using. If your server/client are on the same computer you will get a > notification that the socket was closed, but not when you're communicating > over the internet. The only way around is to ping the other side every once > in a while. > > > Sergej > > > On Tue, Sep 1, 2009 at 5:52 AM, Yu-Teh Shen wrote: > >> I got a question about erlang socket. >> >> Now I have a server(written by erlang) and a client(written by C). >> >> Client create a socket connection to server and then idle for a long time. >> >> After that, I kill client process, but I found out that the server didn't >> know client was been killed! >> >> And I use netstat -pan in server, the connection is still exist! >> >> Is there anyway, the server can detect client had been killed? Like using >> poll function in C socket (error code POLLERR or POLLHUP) >> >> Thanks a lot! >> >> >> Shen, Yu-Teh >> > From joelr1@REDACTED Tue Sep 1 14:04:24 2009 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 1 Sep 2009 13:04:24 +0100 Subject: global:set_lock/1 vs gen_leader & gen_paxos Message-ID: Suppose I need a single node (process) in a cluster to write to a file on Amazon S3. I don't care which node does it since all I'm doing is writing a list of nodes to the file. I could try to elect a leader with gen_leader or come to consensus with gen_paxos. What are the drawbacks of using global:set_lock/1 instead, though? Thanks, Joel --- fastest mac firefox! http://wagerlabs.com From paul-trapexit@REDACTED Tue Sep 1 16:23:08 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Tue, 1 Sep 2009 07:23:08 -0700 (PDT) Subject: [erlang-questions] global:set_lock/1 vs gen_leader & gen_paxos In-Reply-To: References: Message-ID: I used global:set_lock/1 for just this purpose, but it was an operation that happened infrequently (daily) and the number of nodes in my cluster never exceeded 24. It worked so I never tested the alternatives. -- p On Tue, 1 Sep 2009, Joel Reymont wrote: > Suppose I need a single node (process) in a cluster to write to a file > on Amazon S3. I don't care which node does it since all I'm doing is > writing a list of nodes to the file. > > I could try to elect a leader with gen_leader or come to consensus > with gen_paxos. What are the drawbacks of using global:set_lock/1 > instead, though? > > Thanks, Joel > > --- > fastest mac firefox! > http://wagerlabs.com > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From francesco@REDACTED Tue Sep 1 16:17:24 2009 From: francesco@REDACTED (Francesco Cesarini (Erlang Training and Consulting)) Date: Tue, 01 Sep 2009 15:17:24 +0100 Subject: Reminder: Erlang e-Learning Position Vacancy Message-ID: <4A9D2CF4.1090504@erlang-consulting.com> Hi, just a note to say that we are closing the application for the position as an Erlang e-learning developer tomorrow. If you are interested, please register on the University of Kent's website http://jobs.kent.ac.uk finding the job using reference /STM0111. /For information is in the email I sent out the 11th and in the job site. If you have questions, feel free to drop me a line so we can discuss it. This is a perfect opportunity for a graduate to work with ETC and the e-learning department at the University of Kent, bringing together the best of both worlds. Francesco -- http://www.erlang-consulting.com -------- Original Message -------- Subject: [erlang-questions] Erlang e-Learning Position Date: Tue, 11 Aug 2009 12:25:58 +0100 From: Francesco Cesarini (Erlang Training and Consulting) To: erlang-questions@REDACTED Job: Erlang e-Learning Developer Location: London, UK Type: 2 year contract Salary: Competitive package based on experience Description: We are looking for a high-flying graduate to work with Erlang Training and Consulting (ETC) to transform their classroom training materials for Erlang into a state-of-the-art e-learning programme. ETC are the principal providers of Erlang training worldwide, and this programme will allow them to significantly expand their global business. You will have a good honours degree in computing, multimedia, or an IT-related subject. You will be committed to communicating technical ideas through different media: text, presentation, video, and Instant Messaging. You will be confident in finding out about, learning and using and integrating new technologies, including web systems, core multimedia systems as well as the basics of Erlang itself. This position is supported by the Knowledge Transfer Partnerships (KTP) programme, and as part of that you will receive management and other skills training, much like graduate training programmes in larger companies, leading to a Diploma in Management. You will also, if you wish, be able to register for a research degree in the School of Computing at Kent. Closing date for receipt of applications : Wednesday 02 Sep 2009. Interviews are likely to be held : Friday 11 Sep 2009. To apply for this position, you need to be eligible to work in the EU. We will not be sponsoring any visa applications. You can apply on our site at http://www.erlang-consulting.com/jobs/listing Regards, Francesco -- http://www.erlang-consulting.com ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From puzza007@REDACTED Tue Sep 1 16:38:06 2009 From: puzza007@REDACTED (Paul Oliver) Date: Tue, 1 Sep 2009 15:38:06 +0100 Subject: OTP R13B01 odbc Message-ID: Hi all, I'm experiencing an odd issue with the odbc application under OTP R13B. If I run start a connection *after* starting pman, it works. However, if I don't run pman, I get a connection timeout! See my erl shell session below. I've added logging to the odbcserver port and in the latter case it doesn't seem to start at all. Does anyone know what the problem could be? Thanks, Paul. Eshell V5.7.2 (abort with ^G) 1> odbc:start(). ok 2> odbc:connect("DSN=inftest",[]). =ERROR REPORT==== 1-Sep-2009::14:20:24 === ** Generic server <0.40.0> terminating ** Last message in was {<0.33.0>, {connect,[1,1,2,1,1,"DSN=inftest"],on,on}, infinity} ** When Server state == {state,#Port<0.547>, {<0.33.0>,#Ref<0.0.0.37>}, <0.33.0>,undefined,on,undefined,undefined,on, connecting,undefined,0, [#Port<0.545>,#Port<0.546>], undefined,undefined} ** Reason for termination == ** timeout {error,connection_closed} 3> But: 1> odbc:start(). ok 2> pman:start(). <0.42.0> 3> odbc:connect("DSN=inftest",[]). {ok,<0.54.0>} From joelr1@REDACTED Tue Sep 1 16:31:03 2009 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 1 Sep 2009 15:31:03 +0100 Subject: [erlang-questions] global:set_lock/1 vs gen_leader & gen_paxos In-Reply-To: References: Message-ID: <5AF12A5C-D8D2-467B-BB73-3ABA42423E12@gmail.com> I'm looking at 30+ nodes in the cluster and locking every minute or so. Experts tell me that even then the time to acquire the global lock should not exceed a few seconds. On Sep 1, 2009, at 3:23 PM, Paul Mineiro wrote: > I used global:set_lock/1 for just this purpose, but it was an > operation > that happened infrequently (daily) and the number of nodes in my > cluster > never exceeded 24. --- fastest mac firefox! http://wagerlabs.com From per@REDACTED Tue Sep 1 23:32:04 2009 From: per@REDACTED (Per Hedeland) Date: Tue, 1 Sep 2009 23:32:04 +0200 (CEST) Subject: [erlang-questions] about socket problem In-Reply-To: <97619b170909010158tee9bd85v588607bf1b717b02@mail.gmail.com> Message-ID: <200909012132.n81LW4vs033059@pluto.hedeland.org> Rapsey wrote: > >That is just how sockets work I'm afraid. No matter what language you are >using. If your server/client are on the same computer you will get a >notification that the socket was closed, but not when you're communicating >over the internet. The only way around is to ping the other side every once >in a while. This isn't correct. There are cases of non-local TCP connections (doesn't matter if it's over the Internet or a LAN) where there is no notification, basically if someone pulls the power on the remote host or pulls the wire in between, but any form of "normal" close when both hosts keep running and the network keeps functioning will generate the exact same notification as in the local case. UDP is a completely different story, of course. >On Tue, Sep 1, 2009 at 5:52 AM, Yu-Teh Shen wrote: >> >> After that, I kill client process, but I found out that the server didn't >> know client was been killed! >> >> And I use netstat -pan in server, the connection is still exist! >> >> Is there anyway, the server can detect client had been killed? Like using >> poll function in C socket (error code POLLERR or POLLHUP) Of course this is what normally happens (modulo the case above), but it is dependant on the behaviour of your Erlang application. If it hasn't "expressed interest" in what happens on the socket, by using either {active, true} or {active, once} mode, or calling gen_tcp:recv(), the VM won't even look at it. If the application has done either of those things, it will receive a {tcp_closed, Socket} message or a {error, closed} return value, respectively. (You could potentially get a {tcp_error, Socket, Reason} message or a {error, posix()} return value instead - this shouldn't happen when the client just closes, but you should handle it.) At this point it is up to the application to actually close the socket by calling gen_tcp:close(). --Per Hedeland From discoloda@REDACTED Wed Sep 2 05:22:04 2009 From: discoloda@REDACTED (Vincent Adam Burns) Date: Tue, 1 Sep 2009 20:22:04 -0700 Subject: Bmula Message-ID: <3de26b2b0909012022v1e5c9f06oef118db1be73b3fd@mail.gmail.com> http://code.google.com/p/bmula/ A generic library with usefull tools for building websites and game engines. 1. Active DB (working title) - I imagined this as a database where to save IO time; rows that are active are really just Erlang processes that die after 2 minutes of inactivity and while alive use a backend database engine to store (such as dets, Berkly DB etc..) every 10 minutes. Ended up as a API that is like a gen_server that uses a key to differentiate between rows. 2. Email Server/Client. 3. gen_server for random number and word generation, uses a markov chain to read in a bunch of words and then can randomly build up words. 4. A stats system so that you can keep track of different stats of a program and then build graphs using the data. Bmula uses this to track how many random numbers are generated and emails sent/failed to send. -- --- You know you've achieved perfection in design, not when you have nothing more to add, but when you have nothing more to take away. From rtrlists@REDACTED Wed Sep 2 11:40:06 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 2 Sep 2009 10:40:06 +0100 Subject: open_error -192 on erl_ddll:load_driver(crypto_drv) Message-ID: <6a3ae47e0909020240h666d6e25jb808fa1b02a24ae5@mail.gmail.com> Hiya, does anyone know why I might be getting a failure to load crypto_drv on a specific Windows XP box? It appears to work fine in lots of other ones. I have tracked it down to the call of erl_ddll:load_driver(LibDir, crypto_drv) in crypto_server.erl, where the LibDir does contain the correct path to the folder that contains crypto_drv.dll and my environment has the PATH set correctly, so that libeay32.dll can be found. If I try loading the driver manually (with the above erl_ddll:load_driver function), I get the following error: {error, {open_error, -192}} Does anyone have any idea where to look what this actually means? Thanks for any pointers, Robby From magnus@REDACTED Wed Sep 2 12:35:38 2009 From: magnus@REDACTED (Magnus Henoch) Date: Wed, 02 Sep 2009 11:35:38 +0100 Subject: open_error -192 on erl_ddll:load_driver(crypto_drv) In-Reply-To: <6a3ae47e0909020240h666d6e25jb808fa1b02a24ae5@mail.gmail.com> (Robert Raschke's message of "Wed, 2 Sep 2009 10:40:06 +0100") References: <6a3ae47e0909020240h666d6e25jb808fa1b02a24ae5@mail.gmail.com> Message-ID: <84k50hprb9.fsf@linux-b2a3.site> Robert Raschke writes: > If I try loading the driver manually (with the above erl_ddll:load_driver > function), I get the following error: > {error, {open_error, -192}} > > Does anyone have any idea where to look what this actually means? erl_ddll:format_error({open_error, -192}) would give a clearer message. Note that 'due to peculiarities in the dynamic loading interfaces on different platform, the returned string is only guaranteed to describe the correct error if format_error/1 is called in the same instance of the erlang virtual machine as the error appeared in (meaning the same operating system process)'. -- Magnus Henoch, magnus@REDACTED Erlang Training and Consulting http://www.erlang-consulting.com/ From rtrlists@REDACTED Wed Sep 2 12:40:34 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 2 Sep 2009 11:40:34 +0100 Subject: [erlang-questions] Re: open_error -192 on erl_ddll:load_driver(crypto_drv) In-Reply-To: <84k50hprb9.fsf@linux-b2a3.site> References: <6a3ae47e0909020240h666d6e25jb808fa1b02a24ae5@mail.gmail.com> <84k50hprb9.fsf@linux-b2a3.site> Message-ID: <6a3ae47e0909020340m235a76fdre051c5a6ea31b2ee@mail.gmail.com> On Wed, Sep 2, 2009 at 11:35 AM, Magnus Henoch wrote: > Robert Raschke writes: > > > If I try loading the driver manually (with the above erl_ddll:load_driver > > function), I get the following error: > > {error, {open_error, -192}} > > > > Does anyone have any idea where to look what this actually means? > > erl_ddll:format_error({open_error, -192}) would give a clearer message. > Note that 'due to peculiarities in the dynamic loading interfaces on > different platform, the returned string is only guaranteed to describe > the correct error if format_error/1 is called in the same instance of > the erlang virtual machine as the error appeared in (meaning the same > operating system process)'. > Turns out this is a Windows specific error. I have dug in the C sources and have found that the error codes returned from the Windows LoadLibrary() call are returned as (-10 - Windows error code). So that makes LoadLibrary() fail with 182 = ERROR_INVALID_ORDINAL . And that in turn appears to mean that Windows couldn't load the DLL, which is not at all helpful. At least it means it could read it, it just couldn't do what it's meant to do with it. Since the paths are all set up correctly, I can only assume that there is some other underlying reason why a DLL cannot get loaded. Probably something to do with versioning and "side by side". Robby From rtrlists@REDACTED Wed Sep 2 16:31:01 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 2 Sep 2009 15:31:01 +0100 Subject: [erlang-questions] open_error -192 on erl_ddll:load_driver(crypto_drv) In-Reply-To: <4A9E7048.9000104@savageminds.com> References: <6a3ae47e0909020240h666d6e25jb808fa1b02a24ae5@mail.gmail.com> <4A9E7048.9000104@savageminds.com> Message-ID: <6a3ae47e0909020731y689f3976g93d32731dd80c57@mail.gmail.com> On Wed, Sep 2, 2009 at 2:16 PM, Rob Charlton wrote: > Hi, > > Googling for libeay32.dll and error 182 brings up quite a lot of hits - > mostly related to an application picking up an older copy of the DLL > somewhere on the machine. Seems to be fixable by juggling entries in the > path around. Have you searched for other copies of libeay32.dll on the PC? > > Cheers > > Rob > > Looks like we have a winner. It turns out that on that pesky machine, there is an extemely old (2005) libeay32.dll in C:\WINDOWS\system32 . Because that directory is the SystemDirectory and Windows looks for DLLs there first when loading DLLs, and because it is loaded as a side-effect of loading the crypt_drv.dll (i.e., not using an absolute path), that really old file ends up as the libeay32.dll that gets loaded. Because it is so old, it does not have all the necessary exported functions, and therefore triggers the ERROR_INVALID_ORDINAL. I wonder which really anti-social piece of SW thought it would be handy to dump a DLL in the Windows system folder? Probably loads of apps that still do this, aargh. Thanks for the help, Robby From rob.charlton@REDACTED Wed Sep 2 15:16:56 2009 From: rob.charlton@REDACTED (Rob Charlton) Date: Wed, 02 Sep 2009 14:16:56 +0100 Subject: [erlang-questions] open_error -192 on erl_ddll:load_driver(crypto_drv) In-Reply-To: <6a3ae47e0909020240h666d6e25jb808fa1b02a24ae5@mail.gmail.com> References: <6a3ae47e0909020240h666d6e25jb808fa1b02a24ae5@mail.gmail.com> Message-ID: <4A9E7048.9000104@savageminds.com> Hi, Googling for libeay32.dll and error 182 brings up quite a lot of hits - mostly related to an application picking up an older copy of the DLL somewhere on the machine. Seems to be fixable by juggling entries in the path around. Have you searched for other copies of libeay32.dll on the PC? Cheers Rob Robert Raschke wrote: > Hiya, > > does anyone know why I might be getting a failure to load crypto_drv on a > specific Windows XP box? It appears to work fine in lots of other ones. > > I have tracked it down to the call of erl_ddll:load_driver(LibDir, > crypto_drv) in crypto_server.erl, where the LibDir does contain the correct > path to the folder that contains crypto_drv.dll and my environment has the > PATH set correctly, so that libeay32.dll can be found. > > If I try loading the driver manually (with the above erl_ddll:load_driver > function), I get the following error: > {error, {open_error, -192}} > > Does anyone have any idea where to look what this actually means? > > Thanks for any pointers, > Robby > > From vinoski@REDACTED Wed Sep 2 17:10:47 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 2 Sep 2009 11:10:47 -0400 Subject: wx fails to build on Snow Leopard Message-ID: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> I'm seeing that the wx part of the Erlang/OTP build doesn't work on OS X 10.6 (Snow Leopard). I don't use wx so it's no big deal to me, but I'm seeing the g++ compiler appear to hang when compiling lib/wx/c_src/gen/wxe_funcs.cpp. I let it run quite awhile and it never got past that point, but my laptop fans were running full blast so I wonder if the compilation got stuck in an infinite loop. If I manually edit wx out of the lib/Makefile, then everything builds and seems to work as normal. There are also lots of compilation warnings in the wx area due to deprecation of some APIs being used and due to string constants being passed as char*. --steve From kamiseq@REDACTED Wed Sep 2 19:43:21 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Wed, 2 Sep 2009 19:43:21 +0200 Subject: -compile() directive Message-ID: hi, I cant find any description about -compile directive on the net. thanks for help pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From mononcqc@REDACTED Wed Sep 2 19:52:39 2009 From: mononcqc@REDACTED (Fred Hebert (MononcQc)) Date: Wed, 2 Sep 2009 13:52:39 -0400 Subject: [erlang-questions] -compile() directive In-Reply-To: References: Message-ID: <8b9ee55b0909021052l362a6e96p3b53598796ba94ed@mail.gmail.com> 2009/9/2 pawe? kami?ski > hi, > I cant find any description about -compile directive on the net. thanks for > help > > pozdrawiam > Pawe? Kami?ski > > kamiseq@REDACTED > pkaminski.prv@REDACTED > ______________________ > If you're talking about the -compile() module attribute, the arguments you can give it are the flags that can be used when compiling anything. See http://erlang.org/doc/man/compile.html#file-2 It can take a single argument only, so either (as an example): -compile(export_all). or -compile([export_all, 'E']). Some flags will make it so the compiler doesn't save a .beam file to disk you can use compile:output_generated(FlagList) to see if it will or not. If you're NOT talking about the -compile() module attribute, then sorry for the useless mail! Hope this helps. From kamiseq@REDACTED Wed Sep 2 20:04:33 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Wed, 2 Sep 2009 20:04:33 +0200 Subject: [erlang-questions] -compile() directive In-Reply-To: <8b9ee55b0909021052l362a6e96p3b53598796ba94ed@mail.gmail.com> References: <8b9ee55b0909021052l362a6e96p3b53598796ba94ed@mail.gmail.com> Message-ID: no I was thinking about the same thing, thanks I will read about that pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From kamiseq@REDACTED Wed Sep 2 21:15:12 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Wed, 2 Sep 2009 21:15:12 +0200 Subject: implementing annotation in erlang Message-ID: hi, I need to implement simple command dispatcher like this *[{cmdA, module_name, funA},{cmdB, module_name, funB}, ...] * and then use it with each call to my process with specific command name. But then every time I change or add function I need to remember to manually change cmd-fun list. normally I would annotate a function with command name (ie @command(name="getDevTimeCmd")) and then use the information to build dispatcher map while initializing object. I know I can use *module_info()* to get list of all exported function in the module, but then I can't decide which function is command's callback nor define a name of the command(and use only function name instead). The promising solution I'm thinking about is to use macro *-define(command(Name, Fun), assignCommandCallback(Name, Fun)).* and *?command("getDevTimeCmd", getTime). getTime(Args)->... .* but I'm not really sure how to use the information provided that way, when the assignCommandFun/2 will be executed??? tanks for any suggestions pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From igorrs@REDACTED Wed Sep 2 21:47:22 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Wed, 2 Sep 2009 16:47:22 -0300 Subject: [erlang-questions] implementing annotation in erlang In-Reply-To: References: Message-ID: >From what I understand, you want to associate names with functions, so that you can decide which function to run just by "calling its name". If I am right, one way to implement what you want is to have a dict holding those associations (Name -> Function). That dict could be the responsibility of a named process, that would provide a wrapper (a message-passing "API") for the dict operations. Or you could use ets. Igor. 2009/9/2 pawe? kami?ski : > hi, > I need to implement simple command dispatcher like this > > *[{cmdA, module_name, funA},{cmdB, module_name, funB}, ...] * > > and then use it with each call to my process with specific command name. But > then every time I change or add function I need to remember to manually > change cmd-fun list. > normally I would annotate a function with command name (ie > @command(name="getDevTimeCmd")) and then use the information to build > dispatcher map while initializing object. > > I know I can use *module_info()* ?to get list of all exported function in > the module, but then I can't decide which function is command's callback nor > define a name of the command(and use only function name instead). > > The promising solution I'm thinking about is to use macro > *-define(command(Name, Fun), assignCommandCallback(Name, Fun)).* > and > *?command("getDevTimeCmd", getTime). > getTime(Args)->... .* > > but I'm not really sure how to use the information provided that way, when > the assignCommandFun/2 will be executed??? > > tanks for any suggestions > > pozdrawiam > Pawe? Kami?ski > > kamiseq@REDACTED > pkaminski.prv@REDACTED > ______________________ > From diginux@REDACTED Wed Sep 2 22:13:50 2009 From: diginux@REDACTED (Jordan Wilberding) Date: Wed, 2 Sep 2009 15:13:50 -0500 Subject: Chicago Erlang Usergroup Meeting 9/9/09 Message-ID: <5756037b0909021313i7992f0able4b9c3333023ad33@mail.gmail.com> What: Chicago Erlang Usergroup Meeting 9/9/09 When: September 9, 2009 6:00 AM Where: We are confirmed for that Wednesday at 6:00. Orbitz has agreed again to let us use their office space for the meetup. All the names of the people that will attend are needed ahead of time; therefore; please email Martin Logan at martinjlogan@REDACTED to RSVP. The Talk: Coding for the web and Nitrogen by Tristan Sloughter. Thanks! Jordan Wilberding From kamiseq@REDACTED Thu Sep 3 00:12:02 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Thu, 3 Sep 2009 00:12:02 +0200 Subject: [erlang-questions] implementing annotation in erlang In-Reply-To: References: Message-ID: yes sure, but you need to somehow insert the association into the dict/list/ets whatever, I dont want to do it manually, I would like to mark my function (like in java, yes Im aware that erlang is not java:) with some metadata that is far simpler to maintain in development cycle. so the problem is how to put name->fun mapping into some structure rather than what type of the structure to use. the simplest solution would be to have a fun in the module like *commandMap()-> NewMap = dict:new(), Map1 = dict:store("**getDevTimeCmd", **getTime, NewMap), Map2 = dict:store("**getDevTimeCmd", **getTime, Map1), .. MapN .* but then every time I change my function name or command name or add or delete something I need to update commandMap as well in every command module, I would rather write one general function that gets info about function and generates insert to the structure so I can focus on more interesting things. pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ 2009/9/2 Igor Ribeiro Sucupira > From what I understand, you want to associate names with functions, so > that you can decide which function to run just by "calling its name". > > If I am right, one way to implement what you want is to have a dict > holding those associations (Name -> Function). That dict could be the > responsibility of a named process, that would provide a wrapper (a > message-passing "API") for the dict operations. > > Or you could use ets. > > Igor. > > 2009/9/2 pawe? kami?ski : > > hi, > > I need to implement simple command dispatcher like this > > > > *[{cmdA, module_name, funA},{cmdB, module_name, funB}, ...] * > > > > and then use it with each call to my process with specific command name. > But > > then every time I change or add function I need to remember to manually > > change cmd-fun list. > > normally I would annotate a function with command name (ie > > @command(name="getDevTimeCmd")) and then use the information to build > > dispatcher map while initializing object. > > > > I know I can use *module_info()* to get list of all exported function in > > the module, but then I can't decide which function is command's callback > nor > > define a name of the command(and use only function name instead). > > > > The promising solution I'm thinking about is to use macro > > *-define(command(Name, Fun), assignCommandCallback(Name, Fun)).* > > and > > *?command("getDevTimeCmd", getTime). > > getTime(Args)->... .* > > > > but I'm not really sure how to use the information provided that way, > when > > the assignCommandFun/2 will be executed??? > > > > tanks for any suggestions > > > > pozdrawiam > > Pawe? Kami?ski > > > > kamiseq@REDACTED > > pkaminski.prv@REDACTED > > ______________________ > > > From rvirding@REDACTED Thu Sep 3 01:00:40 2009 From: rvirding@REDACTED (Robert Virding) Date: Thu, 3 Sep 2009 01:00:40 +0200 Subject: [erlang-questions] implementing annotation in erlang In-Reply-To: References: Message-ID: <3dbc6d1c0909021600t148a992dse080f3c0b10c9d91@mail.gmail.com> You have a problem in that Erlang has practically no global data stored implicitly by the system. Everythig is in ets tables, processes or in modules. So however you decide to store your data you will have to do yourself, you will have to actually build the table somewhere. To save having to keep multiple copies of the table you can either put it an ets table, which can be made accessible to all processes, or use a dict in a named dispatch process which all process could then access. This is what I understand Igor was saying. An alternative would be to add dispatch tables as code in the actual modules and have a central dispatch module to call the right module. The dispatch table would effectively be stored in code. To change the table you would need to modify and recompile the module. This is doable. It's tough but if you don't have any global data then you don't, Robert 2009/9/3 pawe? kami?ski > yes sure, but you need to somehow insert the association into the > dict/list/ets whatever, I dont want to do it manually, I would like to mark > my function (like in java, yes Im aware that erlang is not java:) with some > metadata that is far simpler to maintain in development cycle. so the > problem is how to put name->fun mapping into some structure rather than > what > type of the structure to use. > > the simplest solution would be to have a fun in the module like > > *commandMap()-> > NewMap = dict:new(), > Map1 = dict:store("**getDevTimeCmd", **getTime, NewMap), > Map2 = dict:store("**getDevTimeCmd", **getTime, Map1), > .. > MapN > .* > > but then every time I change my function name or command name or add or > delete something I need to update commandMap as well in every command > module, I would rather write one general function that gets info about > function and generates insert to the structure so I can focus on more > interesting things. > > pozdrawiam > Pawe? Kami?ski > > kamiseq@REDACTED > pkaminski.prv@REDACTED > ______________________ > > > 2009/9/2 Igor Ribeiro Sucupira > > > From what I understand, you want to associate names with functions, so > > that you can decide which function to run just by "calling its name". > > > > If I am right, one way to implement what you want is to have a dict > > holding those associations (Name -> Function). That dict could be the > > responsibility of a named process, that would provide a wrapper (a > > message-passing "API") for the dict operations. > > > > Or you could use ets. > > > > Igor. > > > > 2009/9/2 pawe? kami?ski : > > > hi, > > > I need to implement simple command dispatcher like this > > > > > > *[{cmdA, module_name, funA},{cmdB, module_name, funB}, ...] * > > > > > > and then use it with each call to my process with specific command > name. > > But > > > then every time I change or add function I need to remember to manually > > > change cmd-fun list. > > > normally I would annotate a function with command name (ie > > > @command(name="getDevTimeCmd")) and then use the information to build > > > dispatcher map while initializing object. > > > > > > I know I can use *module_info()* to get list of all exported function > in > > > the module, but then I can't decide which function is command's > callback > > nor > > > define a name of the command(and use only function name instead). > > > > > > The promising solution I'm thinking about is to use macro > > > *-define(command(Name, Fun), assignCommandCallback(Name, Fun)).* > > > and > > > *?command("getDevTimeCmd", getTime). > > > getTime(Args)->... .* > > > > > > but I'm not really sure how to use the information provided that way, > > when > > > the assignCommandFun/2 will be executed??? > > > > > > tanks for any suggestions > > > > > > pozdrawiam > > > Pawe? Kami?ski > > > > > > kamiseq@REDACTED > > > pkaminski.prv@REDACTED > > > ______________________ > > > > > > From mononcqc@REDACTED Thu Sep 3 01:39:38 2009 From: mononcqc@REDACTED (Fred Hebert (MononcQc)) Date: Wed, 2 Sep 2009 19:39:38 -0400 Subject: [erlang-questions] implementing annotation in erlang In-Reply-To: References: Message-ID: <8b9ee55b0909021639v3472c750u156dbb0397720248@mail.gmail.com> This solution might not be the nicest and will be slower than most, but it should require less maintenance and code to use: We'll use a custom module attribute to get the functions we want to map. Let's create and compile the following module to test our stuff: -module(chk). -export([test/0,fake/1]). -command_map([test/0]). fake(N) -> N+1. test() -> test. We compile it and can fetch its module attributes with: 10> proplists:get_value(command_map, chk:module_info(attributes)). [{test,0}] Maybe that can help you at least getting the names of the functions where you need them. If you'd like a central way to find all the functions inside command_map module attributes, you might want to try the following: -module(mappings). -export([find/0]). -define(ATTR, command_map). find() -> [ {Mod, get_mapped(Mod)} || {Mod,_Path} <- code:all_loaded(), is_mapped(Mod)]. is_mapped(Mod) -> is_list(proplists:get_value(?ATTR, Mod:module_info(attributes))). get_mapped(Mod) -> proplists:get_value(?ATTR, Mod:module_info(attributes)). Which will scan through all loaded modules (yeah, they need to be active already...) and fetch the mapped functions. It'll return the whole list under the form [{Mod1, [{Fun1,Arity1},{Fun2,Arity2}, ...,{FunN,ArityN}]}, {ModN, ...}], Which should make it easier to call stuff. The scheme is easy enough to modify, process further to attach to a keyword and whatnot. I've attached both files just in case. Hope this helps. -------------- next part -------------- A non-text attachment was scrubbed... Name: chk.erl Type: application/octet-stream Size: 103 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mappings.erl Type: application/octet-stream Size: 340 bytes Desc: not available URL: From steven.charles.davis@REDACTED Thu Sep 3 02:37:38 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 2 Sep 2009 17:37:38 -0700 (PDT) Subject: implementing annotation in erlang In-Reply-To: <8b9ee55b0909021639v3472c750u156dbb0397720248@mail.gmail.com> References: <8b9ee55b0909021639v3472c750u156dbb0397720248@mail.gmail.com> Message-ID: <9087d40e-76ac-4513-bd27-85ce76539e9e@k39g2000yqe.googlegroups.com> I think you are trying to force a "java way" onto the platform. Have you considered using records defined in the header (hrl) for those arguments? i.e. if you are using annotations in java you are recompiling source in any case when you make the change. If you make them into records then you'll equivalently be recompiling across the entire application code, which appears to me to be an equivalent way of making the change. ...or did I misunderstand the problem? /s From igorrs@REDACTED Thu Sep 3 05:05:46 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Thu, 3 Sep 2009 00:05:46 -0300 Subject: [erlang-questions] Re: implementing annotation in erlang In-Reply-To: <9087d40e-76ac-4513-bd27-85ce76539e9e@k39g2000yqe.googlegroups.com> References: <8b9ee55b0909021639v3472c750u156dbb0397720248@mail.gmail.com> <9087d40e-76ac-4513-bd27-85ce76539e9e@k39g2000yqe.googlegroups.com> Message-ID: Hi, Steve. I believe Pawel is not worried about having to recompile. The key (most difficult) functionality he needs is to be able to change the name of a function without having to change *anything* more. So, for example, if command "getDevTimeCmd" is associated with fun some_module:getTime/0 and you then decide to change the name of getTime to getLocalTime (and recompile), you would automagically have "getDevTimeCmd" associated to fun some_module:getLocalTime/0. I don't think this can be done in Erlang (but what do I know? :)). Fred's neat solution got close, but you still have to change somewhere else if you rename fun test/0 to fun run_test/0, for example. However, at least the change would be in the same module, which is already an advantage over what I and other people have proposed. Igor. On Wed, Sep 2, 2009 at 9:37 PM, Steve Davis wrote: > I think you are trying to force a "java way" onto the platform. > > Have you considered using records defined in the header (hrl) for > those arguments? i.e. if you are using annotations in java you are > recompiling source in any case when you make the change. If you make > them into records then you'll equivalently be recompiling across the > entire application code, which appears to me to be an equivalent way > of making the change. > > ...or did I misunderstand the problem? > > /s > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From kamiseq@REDACTED Thu Sep 3 08:37:41 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Thu, 3 Sep 2009 08:37:41 +0200 Subject: [erlang-questions] Re: implementing annotation in erlang In-Reply-To: References: <8b9ee55b0909021639v3472c750u156dbb0397720248@mail.gmail.com> <9087d40e-76ac-4513-bd27-85ce76539e9e@k39g2000yqe.googlegroups.com> Message-ID: 2009/9/3 Igor Ribeiro Sucupira > Hi, Steve. > > I believe Pawel is not worried about having to recompile. > > The key (most difficult) functionality he needs is to be able to > change the name of a function without having to change *anything* > more. > hurray, finally somebody got it :). I will look closely at Fred's solution. anyway in arlang web 1.3 there are annotation, but I dont understand them at the moment how they are created and mapped to some fun invocation. is there a reason that erlang dont have more advanced (over module_info()) reflection api?? pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From kagato@REDACTED Thu Sep 3 09:42:28 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 3 Sep 2009 00:42:28 -0700 Subject: [erlang-questions] Re: implementing annotation in erlang In-Reply-To: References: <8b9ee55b0909021639v3472c750u156dbb0397720248@mail.gmail.com> <9087d40e-76ac-4513-bd27-85ce76539e9e@k39g2000yqe.googlegroups.com> Message-ID: Erlang doesn't have a more extensive reflection API largely because it's so easy to roll something equivalent (or better in some cases). Without the overhead of objects and class hierarchies, introspection doesn't really mean the same thing anyway. Perhaps the best pattern here is to have a central registry using processes to hold the state. For example: > -module(cmd_mgr). > -behavior(gen_server). > -export([init/1,handle_call/3,handle_cast/2,handle_info/2,terminate/ > 2,code_change/3]). > -export([start_link/0,register_command/2,execute_command/2]). > > -record(cmd_mgr_state,{commands}). > > % API > start_link() -> gen_server:start_link({global,cmd_mgr},? > MODULE,default_commands(),[]). > > default_commands() -> []. % Add any "default" commands here. > > register_command(CommandName,CommandFun) -> > gen_server:call({global,cmd_mgr},{register, > {CommandName,CommandFun}}). > > execute_command(CommandName,Args) -> > case gen_server:call({global,cmd_mgr},{command,CommandName}) of > command_not_found -> bad_command; > {found,Fun} -> apply(Fun,Args) > end. > > % Callbacks > init(Commands) -> {ok,#cmd_mgr_state{commands=Commands}}. > > handle_call({register,{_,_} = Command},_From,#cmd_mgr_state{commands > = Commands} = State) -> > { reply, added, State#cmd_mgr_state{commands = [Command | > Commands]} }; > > handle_call({command,CmdName},_From,#cmd_mgr_state{commands = > Commands} = State) -> > case proplists:get_value(CmdName, Commands) of > undefined -> {reply,command_not_found,State}; > Fun when is_function(Fun) -> {reply,{found,Fun},State} > end. > > handle_cast(not_implemented,State) -> {noreply,State}. > handle_info(not_implemented,State) -> {noreply,State}. > code_change(_OldVsn,State,_Extra) -> {ok,State}. > terminate(_Reason,_State) -> terminated. Here's a terminal session where I use this manager: > Erlang R13B (erts-5.7.1) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.7.1 (abort with ^G) > 1> {ok,Mgr} = cmd_mgr:start_link(). > {ok,<0.35.0>} > 2> cmd_mgr:register_command(foo,fun () -> ok end). > added > 3> cmd_mgr:register_command(bar,fun () -> qux end). > added > 4> cmd_mgr:register_command(reverse,fun lists:reverse/1). > added > 5> cmd_mgr:execute_command(foo,[]). > ok > 6> cmd_mgr:execute_command(bar,[]). > qux > 7> cmd_mgr:execute_command(reverse,[ [1,2,3,4,5] ]). % Note here, > this list is an argument list, and [1,2,3,4,5] is the first argument > to the call. > [5,4,3,2,1] > 8> init:stop(). The above code is a simple gen_server that starts up with a set of "default commands". The Command List is just a proplist that maps a set of commands to functions. You can register additional commands using register_command/2. When you execute a command, it returns the function and calls it in the process of the caller, so the manager does not get blocked by the calls. Also, if you are dead-set on using the module:function notation instead of funs, you could store the M,F and return it, instead of returning the actual function--so the pattern should still be good there too. Note that I hold this data "in process", but it's quite possible to store them in ets or mnesia if you want this process to be able to crash and recover its state. That said, if the command plugins and command manager all run under the same supervisor, it can be a "one_for_all" supervisor and the state will refresh if anybody dies (as the whole thing gets recreated). There are a few functions that offer the ability to register an arbitrary fun as a command or to that command. To take advantage of this, you should write your command modules as gen_servers and plug them into the hierarchy in such a way that they are started. I know that this may seem unnatural (i.e. why do I need to start a process, can't I autodetect it, etc). This is the Erlang way and I dare say that it is easier in the long run. While the processes might seem to be a waste now, it's just a matter of time until they start becoming the state-containers for your command state data (just the way you might have used the class to stash state data in Ruby/Python/Java). Similarly, default_commands/0 could load a configuration file, allowing you to handle command registration more flexibly. You could, conceivably, walk the code_path and find modules to autoload commands, I would humbly suggest that it is highly likely that it's easier to maintain a command list than it will be to retrofit the system when you should no longer register all commands in all situations. Explicit is really not that bad in this case, and Erlang's app deployment story makes it easy to configure this per deployment (which is invariably where you'll end up). Just adopt the convention to convert any sort of global state management using object / class registration behavior with starting a process. It will serve you well if you write a lot of Erlang. I think I'm going to blog about this. On Sep 2, 2009, at 11:37 PM, pawe? kami?ski wrote: > 2009/9/3 Igor Ribeiro Sucupira > >> Hi, Steve. >> >> I believe Pawel is not worried about having to recompile. >> >> The key (most difficult) functionality he needs is to be able to >> change the name of a function without having to change *anything* >> more. >> > > hurray, finally somebody got it :). I will look closely at Fred's > solution. > anyway in arlang web 1.3 there are annotation, but I dont understand > them at > the moment how they are created and mapped to some fun invocation. > > is there a reason that erlang dont have more advanced (over > module_info()) > reflection api?? > > pozdrawiam > Pawe? Kami?ski > > kamiseq@REDACTED > pkaminski.prv@REDACTED > ______________________ -- Jayson Vantuyl kagato@REDACTED From vinoski@REDACTED Thu Sep 3 09:48:00 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Thu, 3 Sep 2009 03:48:00 -0400 Subject: [erlang-questions] Re: implementing annotation in erlang In-Reply-To: References: <8b9ee55b0909021639v3472c750u156dbb0397720248@mail.gmail.com> <9087d40e-76ac-4513-bd27-85ce76539e9e@k39g2000yqe.googlegroups.com> Message-ID: <65b2728e0909030048l1796188ek14391e34c8bc76b8@mail.gmail.com> 2009/9/3 pawe? kami?ski > 2009/9/3 Igor Ribeiro Sucupira > > > Hi, Steve. > > > > I believe Pawel is not worried about having to recompile. > > > > The key (most difficult) functionality he needs is to be able to > > change the name of a function without having to change *anything* > > more. > > > > hurray, finally somebody got it :). I will look closely at Fred's solution. > anyway in arlang web 1.3 there are annotation, but I dont understand them > at > the moment how they are created and mapped to some fun invocation. > > is there a reason that erlang dont have more advanced (over module_info()) > reflection api?? > I believe you could easily take Fred's custom attribute solution but write a parse transform to replace the attribute with a generated exported function in the module that would return the necessary mapping information. If you gave that function the same name in every module, perhaps giving it an uppercase name like 'COMMAND_MAP' so it would be unlikely to clash with normal function names, you could use Fred's technique of walking the modules at runtime but either use beam_lib:chunks(module, [exports]) to see if a given module has such a function, or just try to invoke it under a catch and ignore the ones that result in errors. --steve From puzza007@REDACTED Thu Sep 3 12:24:23 2009 From: puzza007@REDACTED (Paul Oliver) Date: Thu, 3 Sep 2009 11:24:23 +0100 Subject: OTP R13B01 odbc In-Reply-To: References: Message-ID: More info in case it rings any bells... Using strace to monitor erl/odbcserver causes odbc:connect to *work*. The odbcserver process starts up and connects to both the supervisor prot and the worker port. Adding logging to odbcserver seems to show that when not being viewed with strace (or having pman run), the main function in odbcserver is never run! Is there a reason it wouldn't even get this far? The logging we've added to odbcserver.c is: int main(void) { FILE *fh; byte *msg = NULL; char *temp = NULL, *supervisor_port = NULL, *odbc_port = NULL; size_t length; #ifdef WIN32 _setmode(_fileno( stdin), _O_BINARY); #endif * fh = fopen("/tmp/odbc.log", "a"); fprintf(fh, "Start main...\n"); fflush(fh); fclose(fh);* [snip] #if defined WIN32 || defined USE_IPV4 /* Currently only an old windows compiler is supported so we do not have ipv6 capabilities */ static SOCKET connect_to_erlang(const char *port, int is_sup) { SOCKET sock; struct sockaddr_in sin; FILE *fh; int conn_ret; * if (is_sup) fh = fopen("/tmp/sup.log", "a"); else fh = fopen("/tmp/odbc.log", "a"); fprintf(fh, "connect_to_erlang call to \"socket\"\n"); fflush(fh); * I'm running on Linux foobar 2.6.24-24-server #1 SMP Tue Aug 18 16:51:43 UTC 2009 x86_64 GNU/Linux Thanks! Paul. On Tue, Sep 1, 2009 at 3:38 PM, Paul Oliver wrote: > Hi all, > > I'm experiencing an odd issue with the odbc application under OTP R13B. > > If I run start a connection *after* starting pman, it works. However, > if I don't run pman, I get a connection timeout! See my erl shell > session below. I've added logging to the odbcserver port and in the > latter case it doesn't seem to start at all. > > Does anyone know what the problem could be? > > Thanks, > Paul. > > Eshell V5.7.2 (abort with ^G) > 1> odbc:start(). > ok > 2> odbc:connect("DSN=inftest",[]). > > =ERROR REPORT==== 1-Sep-2009::14:20:24 === > ** Generic server <0.40.0> terminating > ** Last message in was {<0.33.0>, > {connect,[1,1,2,1,1,"DSN=inftest"],on,on}, > infinity} > ** When Server state == {state,#Port<0.547>, > {<0.33.0>,#Ref<0.0.0.37>}, > <0.33.0>,undefined,on,undefined,undefined,on, > connecting,undefined,0, > [#Port<0.545>,#Port<0.546>], > undefined,undefined} > ** Reason for termination == > ** timeout > {error,connection_closed} > 3> > > But: > > 1> odbc:start(). > ok > 2> pman:start(). > <0.42.0> > 3> odbc:connect("DSN=inftest",[]). > {ok,<0.54.0>} > From zerthurd@REDACTED Thu Sep 3 12:29:10 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Thu, 3 Sep 2009 17:29:10 +0700 Subject: [erlang-questions] Proposal for -warning() directive Message-ID: Hello Is it possible to add -warning(String) directive in Code Preprocessor or Compiler? It must be similar to CPP #warning directive which prints warning message in compilation time. Thank you -- Maxim Treskin From james.hague@REDACTED Thu Sep 3 17:36:26 2009 From: james.hague@REDACTED (James Hague) Date: Thu, 3 Sep 2009 10:36:26 -0500 Subject: proposal: expand definition of filename type Message-ID: I've found that sometimes it's handy to have filenames as lists--which is the Erlang standard--and sometimes it's nice to treat them as binaries. So here's a proposal: In file access functions, define a filename to be a binary or an IO list. Examples: 1. file:file_open(<<"circus/images/clown.png">>, []). 2. file:file_open([Directory, BaseName, ".dat"], []). Advantages: * Filenames, which can potentially be long, don't expand into 8 or 16 bytes per character. (Think of code that manipulates a huge directory tree, for example.) * It's easy to add path names and extensions without copying entire lists. Open questions: * Just keep the filename module operating on lists? * Should there be an alternate version of file:list_dir that returns a list of binaries? Note that there shouldn't be any compatibility issues, because lists are a subset of IO lists. From puzza007@REDACTED Thu Sep 3 17:36:20 2009 From: puzza007@REDACTED (Paul Oliver) Date: Thu, 3 Sep 2009 16:36:20 +0100 Subject: OTP R13B01 odbc In-Reply-To: References: Message-ID: Found it! For suture reference, it was snoopy.so (a logging package installed on a dev server I'm using http://sourceforge.net/projects/snoopylogger/). ?It screws up the vforked process because it does stuff (logging around calls to execve) before the execve is called. ?Obviously for vfork this is a problem - the manpage states "The vfork() function has the same effect as fork(), except that the behaviour is undefined if the process created by vfork() either modifies any data other than a variable of type pid_t used to store the return value from vfork(), or returns from the function in which vfork() was called, or calls any other function before successfully calling _exit() or one of the exec family of functions.". Please see the strace output below. BTW, is there any reason that vfork isn't disabled by default on Linux?? Given that fork is implemented using copy-on-write is there any benefit in using vfork? 11:51:29 read(13, "\7\0\0\0\300V\0\0pts/262\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 384) = 384 11:51:29 read(13, "\7\0\0\0\334a\0\0pts/264\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 384) = 384 11:51:29 read(13, "\7\0\0\0\337f\0\0pts/265\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 384) = 384 11:51:29 fcntl(13, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0 11:51:29 alarm(0) = 1 11:51:29 rt_sigaction(SIGALRM, {SIG_DFL}, NULL, 8) = 0 11:51:29 close(13) = 0 11:51:29 getsid(0) = 26338 11:51:29 getuid() = 1084 11:51:29 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3661, ...}) = 0 11:51:29 getpid() = 21217 11:51:29 sendto(11, "<86>Sep 3 11:51:29 snoopy[21217"..., 97, MSG_NOSIGNAL, NULL, 0) = -1 ENOTSOCK (Socket operation on non-socket) 11:51:29 close(11) = 0 11:51:29 socket(PF_FILE, SOCK_DGRAM, 0) = 11 11:51:29 fcntl(11, F_SETFD, FD_CLOEXEC) = 0 11:51:29 connect(11, {sa_family=AF_FILE, path="/dev/log"}, 110) = 0 11:51:29 sendto(11, "<86>Sep 3 11:51:29 snoopy[21217"..., 97, MSG_NOSIGNAL, NULL, 0) = 97 11:51:29 execve("/space/jpurring/erlR13B01/lib/erlang/erts-5.7.2/bin/child_setup", ["/space/jpurring/erlR13B01/lib/er"..., "false", ".", "exec /space/jpurring/erlR13B01/l"..., "3:1023", "10:1", "11:0", "-"], [/* 55 vars */]) = 0 11:51:29 brk(0) = 0x603000 11:51:29 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3b4e541000 ... 11:51:29 dup2(10, 1) = 1 11:51:29 dup2(11, 0) = -1 EBADF (Bad file descriptor) 11:51:29 exit_group(1) = ? On Thu, Sep 3, 2009 at 11:24 AM, Paul Oliver wrote: > > More info in case it rings any bells... > > Using strace to monitor erl/odbcserver causes odbc:connect to *work*.? The odbcserver process starts up and connects to both the supervisor prot and the worker port.? Adding logging to odbcserver seems to show that when not being viewed with strace (or having pman run), the main function in odbcserver is never run!? Is there a reason it wouldn't even get this far? > > The logging we've added to odbcserver.c is: > > int main(void) > { > ??? FILE *fh; > ??? byte *msg = NULL; > ??? char *temp = NULL, *supervisor_port = NULL, *odbc_port = NULL; > ??? size_t length; > #ifdef WIN32 > ??? _setmode(_fileno( stdin),? _O_BINARY); > #endif > > ????? fh = fopen("/tmp/odbc.log", "a"); > ??? fprintf(fh, "Start main...\n"); > ??? fflush(fh); > ??? fclose(fh); > > [snip] > > #if defined WIN32 || defined USE_IPV4 > /* Currently only an old windows compiler is supported so we do not have ipv6 > ? capabilities */ > static SOCKET connect_to_erlang(const char *port, int is_sup) > { > ??? SOCKET sock; > ??? struct sockaddr_in sin; > ??? FILE *fh; > ??? int conn_ret; > > ??? if (is_sup) > ????? fh = fopen("/tmp/sup.log", "a"); > ??? else > ????? fh = fopen("/tmp/odbc.log", "a"); > > ??? fprintf(fh, "connect_to_erlang call to \"socket\"\n"); > ??? fflush(fh); > > I'm running on Linux foobar 2.6.24-24-server #1 SMP Tue Aug 18 16:51:43 UTC 2009 x86_64 GNU/Linux > > Thanks! > Paul. > > On Tue, Sep 1, 2009 at 3:38 PM, Paul Oliver wrote: > > Hi all, > > > > I'm experiencing an odd issue with the odbc application under OTP R13B. > > > > If I run start a connection *after* starting pman, it works. ?However, > > if I don't run pman, I get a connection timeout! ?See my erl shell > > session below. ?I've added logging to the odbcserver port and in the > > latter case it doesn't seem to start at all. > > > > Does anyone know what the problem could be? > > > > Thanks, > > Paul. > > > > Eshell V5.7.2 ?(abort with ^G) > > 1> ?odbc:start(). > > ok > > 2> odbc:connect("DSN=inftest",[]). > > > > =ERROR REPORT==== 1-Sep-2009::14:20:24 === > > ** Generic server <0.40.0> terminating > > ** Last message in was {<0.33.0>, > > ? ? ? ? ? ? ? ? ? ? ? ?{connect,[1,1,2,1,1,"DSN=inftest"],on,on}, > > ? ? ? ? ? ? ? ? ? ? ? ?infinity} > > ** When Server state == {state,#Port<0.547>, > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {<0.33.0>,#Ref<0.0.0.37>}, > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? <0.33.0>,undefined,on,undefined,undefined,on, > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? connecting,undefined,0, > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [#Port<0.545>,#Port<0.546>], > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined} > > ** Reason for termination == > > ** timeout > > {error,connection_closed} > > 3> > > > > But: > > > > 1> odbc:start(). > > ok > > 2> pman:start(). > > <0.42.0> > > 3> ?odbc:connect("DSN=inftest",[]). > > {ok,<0.54.0>} > > > From erlang@REDACTED Thu Sep 3 18:02:46 2009 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 3 Sep 2009 18:02:46 +0200 Subject: [erlang-questions] proposal: expand definition of filename type In-Reply-To: References: Message-ID: <9b08084c0909030902g27c8d80ej934c3fda934a143e@mail.gmail.com> Sorry James, but I disagree... I think the library arguments like this should be moved in exactly the oppose direction - I dislike polymorphic arguments like this. I think the first argument to to file:file_open (or whatever) should be a string and nothing else. If you want polymorphic behaviour then you should define you own private interface library james_lib:file_open/2 which defines whatever conventions you like for the arguments. The current practise of allowing polymorphic arguments is a symptom of "can't make your mid up ism" - it makes type checking a lot more difficult and the logic of programs a lot more difficult to follow. It make programming more difficult because I have to choose which convention to use, also it will trip me up later since I'll end up thinking that all filenames in all modules in every application can be represented in this way. To achieve what you want I suggest having a single and very large library james_hague_lib_misc.erl where you put a large number of unrelated functions that merely modify the behaviour of the standard functions. (this is what I did in the Erlang book and I have continued with the practise since then) I think the standard libraries should have a restricted number of types and that deviations from these belong in private libraries. If a particular set of customisations is useful then it can be published in its own right as a library. Cheers /Joe On Thu, Sep 3, 2009 at 5:36 PM, James Hague wrote: > I've found that sometimes it's handy to have filenames as lists--which > is the Erlang standard--and sometimes it's nice to treat them as > binaries. ?So here's a proposal: In file access functions, define a > filename to be a binary or an IO list. > > Examples: > 1. file:file_open(<<"circus/images/clown.png">>, []). > 2. file:file_open([Directory, BaseName, ".dat"], []). > > Advantages: > * Filenames, which can potentially be long, don't expand into 8 or 16 > bytes per character. ?(Think of code that manipulates a huge directory > tree, for example.) > * It's easy to add path names and extensions without copying entire lists. > > Open questions: > * Just keep the filename module operating on lists? > * Should there be an alternate version of file:list_dir that returns a > list of binaries? > > Note that there shouldn't be any compatibility issues, because lists > are a subset of IO lists. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From john.erickson@REDACTED Thu Sep 3 19:51:41 2009 From: john.erickson@REDACTED (Erickson, John) Date: Thu, 3 Sep 2009 11:51:41 -0600 Subject: EPMD and multi-user distributed computation Message-ID: <3C44D46958109E408F5848593B05A1D8C2E50444@rrsmsx505.amr.corp.intel.com> I have an erlang program that is run similar to an MPI program, where it is deployed over many nodes and erlang starts up on those nodes and must be shut down when the program terminates. I cannot leave erlang or epmd running on these computers. Furthermore, many users would like to be able to use this program simultaneously. I am having problems with epmd run by different users conflicting on the available ports. Currently I am statically allocating these ports to users with ERL_EPMD_PORT, but this is a pretty cumbersome solution. Is there a better way to do this? I could dynamically allocate the port on a single node, but since epmd has to be run on every node in the system, I don't see an easy way to do this. Why does there need to be an epmd on every node anyway? Wouldn't a single one suffice? From mjtruog@REDACTED Thu Sep 3 20:28:35 2009 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 03 Sep 2009 11:28:35 -0700 Subject: [erlang-questions] EPMD and multi-user distributed computation In-Reply-To: <3C44D46958109E408F5848593B05A1D8C2E50444@rrsmsx505.amr.corp.intel.com> References: <3C44D46958109E408F5848593B05A1D8C2E50444@rrsmsx505.amr.corp.intel.com> Message-ID: <4AA00AD3.5050404@gmail.com> Erickson, John wrote: > I have an erlang program that is run similar to an MPI program, where it is deployed over many nodes and erlang starts up on those nodes and must be shut down when the program terminates. I cannot leave erlang or epmd running on these computers. Furthermore, many users would like to be able to use this program simultaneously. I am having problems with epmd run by different users conflicting on the available ports. Currently I am statically allocating these ports to users with ERL_EPMD_PORT, but this is a pretty cumbersome solution. > > Is there a better way to do this? I could dynamically allocate the port on a single node, but since epmd has to be run on every node in the system, I don't see an easy way to do this. Why does there need to be an epmd on every node anyway? Wouldn't a single one suffice? > You might want to try automating the startup/teardown process with the slave module in stdlib, or something similar. You would be able to pass ERL_EPMD_PORT as an argument and might need a separate step to manage starting/stopping epmd with ssh/rsh, depending on how you are doing things. If you pass ERL_EPMD_PORT to the first VM on the command line, you will not be able to find it in the command line through a module function, so you should probably use some configuration item that sets ERL_EPMD_PORT, in addition to specifying it on the command line (i.e., ERL_EPMD_PORT configuration is driven from the configuration item, not the environmental variable or the specification of an environmental variable on the command line). From sgolovan@REDACTED Thu Sep 3 21:01:03 2009 From: sgolovan@REDACTED (Sergei Golovan) Date: Thu, 3 Sep 2009 23:01:03 +0400 Subject: [erlang-questions] EPMD and multi-user distributed computation In-Reply-To: <3C44D46958109E408F5848593B05A1D8C2E50444@rrsmsx505.amr.corp.intel.com> References: <3C44D46958109E408F5848593B05A1D8C2E50444@rrsmsx505.amr.corp.intel.com> Message-ID: On Thu, Sep 3, 2009 at 9:51 PM, Erickson, John wrote: > I have an erlang program that is run similar to an MPI program, where it is deployed over many nodes and erlang starts up on those nodes and must be shut down when the program terminates. ?I cannot leave erlang or epmd running on these computers. ?Furthermore, many users would like to be able to use this program simultaneously. ?I am having problems with epmd run by different users conflicting on the available ports. ?Currently I am statically allocating these ports to users with ERL_EPMD_PORT, but this is a pretty cumbersome solution. > > Is there a better way to do this? ?I could dynamically allocate the port on a single node, but since epmd has to be run on every node in the system, I don't see an easy way to do this. ?Why does there need to be an epmd on every node anyway? ?Wouldn't a single one suffice? You don't have to run one EPMD daemon per user. One instance is sufficient. Just run it at startup and let all users use the standard EPMD port. Moreover, having one EPMD per user doesn't increase security at all, as any user may easily stop all epmd processes. So, EPMD process must run in a trusted environment anyway. Cheers! -- Sergei Golovan From paul-trapexit@REDACTED Thu Sep 3 21:54:02 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Thu, 3 Sep 2009 12:54:02 -0700 (PDT) Subject: [erlang-questions] proposal: expand definition of filename type In-Reply-To: <9b08084c0909030902g27c8d80ej934c3fda934a143e@mail.gmail.com> References: <9b08084c0909030902g27c8d80ej934c3fda934a143e@mail.gmail.com> Message-ID: The point is well taken generally, but for iolists in particular, I find that I'm often disappointed that various portions of stdlib only take strings, since building up iolists incrementally is natural and many useful functions (io_lib:format/2, mochijson2:encode/1) return iolists. Example: you cannot use an iolist as the body of a request for http:request/4 and friends. I generally only discover empirically when testing that I have to insert flattening code to make things work. -- p On Thu, 3 Sep 2009, Joe Armstrong wrote: > Sorry James, but I disagree... > > I think the library arguments like this should be moved in exactly the > oppose direction - I dislike polymorphic arguments like this. I think > the first argument to to file:file_open (or whatever) should be a > string and nothing else. > > If you want polymorphic behaviour then you should define you own > private interface library > james_lib:file_open/2 which defines whatever conventions you like for > the arguments. > > The current practise of allowing polymorphic arguments is a symptom of > "can't make your mid up ism" - it makes type checking a lot more > difficult and the logic of programs a lot > more difficult to follow. It make programming more difficult because I > have to choose which convention to use, also it will trip me up later > since I'll end up thinking that all filenames > in all modules in every application can be represented in this way. > > To achieve what you want I suggest having a single and very large library > james_hague_lib_misc.erl where you put a large number of unrelated > functions that merely modify the behaviour of the standard functions. > (this is what I did in the Erlang book and I have continued with the > practise since then) > > I think the standard libraries should have a restricted number of > types and that deviations > from these belong in private libraries. If a particular set of > customisations is useful then it can be published in its own right as > a library. > > Cheers > > /Joe > > On Thu, Sep 3, 2009 at 5:36 PM, James Hague wrote: > > I've found that sometimes it's handy to have filenames as lists--which > > is the Erlang standard--and sometimes it's nice to treat them as > > binaries. ?So here's a proposal: In file access functions, define a > > filename to be a binary or an IO list. > > > > Examples: > > 1. file:file_open(<<"circus/images/clown.png">>, []). > > 2. file:file_open([Directory, BaseName, ".dat"], []). > > > > Advantages: > > * Filenames, which can potentially be long, don't expand into 8 or 16 > > bytes per character. ?(Think of code that manipulates a huge directory > > tree, for example.) > > * It's easy to add path names and extensions without copying entire lists. > > > > Open questions: > > * Just keep the filename module operating on lists? > > * Should there be an alternate version of file:list_dir that returns a > > list of binaries? > > > > Note that there shouldn't be any compatibility issues, because lists > > are a subset of IO lists. > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From kagato@REDACTED Fri Sep 4 00:22:40 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 3 Sep 2009 15:22:40 -0700 Subject: [erlang-questions] Proposal for -warning() directive In-Reply-To: References: Message-ID: <995D7894-610D-468C-83FE-59F992C39785@souja.net> I've had a case where this would have been useful. Not to have a directive explosion, I might also recommend -deprecated(). In fact, I would probably find that more useful than -warning. Or maybe there should be a directive more like: -emit(warning,"Something bad about this module."). -emit(deprecated,"This disappears next version."). Or I might be overthinking it. But I do think this functionality in some form is useful. On Sep 3, 2009, at 3:29 AM, Maxim Treskin wrote: > Hello > > Is it possible to add -warning(String) directive in Code > Preprocessor or > Compiler? > It must be similar to CPP #warning directive which prints warning > message in > compilation time. > > Thank you > > -- > Maxim Treskin -- Jayson Vantuyl kagato@REDACTED From dizzyd@REDACTED Fri Sep 4 00:24:34 2009 From: dizzyd@REDACTED (Dave Smith) Date: Thu, 3 Sep 2009 16:24:34 -0600 Subject: Bug in epmd_srv.c? Message-ID: Hi all, I've been working on a helper module to enable usage of epmd for things other than just nodes (i.e. poor man's zeroconf service). In the process, I've found some discrepancies in the epmd documentation and server code. I _think_ there is a bug or two there, but would appreciate validation by the OTP team -- it could just be a lack of understanding on my part. :) First off, the docs for the distribution protocol state that the ALIVE2_REQ has the field "LowestVersion" followed by "HighestVersion". However, the implementation in erl_epmd.erl has them reversed as does the documentation for the PORT2_RESP packet. This is, I believe, a minor documentation bug. Secondly, in erts/epmd/src/epmd_srv.c:628 (R13B01), there is the following line: offset += (strlen(node->extra)-1); Unfortunately, if the length of the extra field is zero, this causes the last byte of the packet to disappear as it winds up decrementing "offset". Reading through epmd, it looks like the expectation is that the extra field is null-terminated --- but the docs make no mention of it. This is somewhat odd to me as that field is prefixed with a 2 byte length value in the protocol, so I see no reason for the implementation to require the data to be a C string. Of course, this may be for backwards compatibility purposes; it's hard to know for sure. Any guidance/illumination on these issues would be appreciated. Thanks, D. From allan.merolla@REDACTED Fri Sep 4 02:07:34 2009 From: allan.merolla@REDACTED (Allan Merolla) Date: Fri, 4 Sep 2009 10:07:34 +1000 Subject: Mnesia Top 1? Message-ID: Hi, I have a massive table containing time-series (values versus time) for different object. Table structure is like ObjectID, Time, Value. {ObjectID, Time} is the key. I have a function that returns interpolated values (e.g. a value for each 10min) for a time range and object ID called getList(ObjId, FromTime, ToTime). I use the following qlc query to get un-interpolated data. qlc:q([X<-mnesia:table(Tbl), X#tbl.objId = ObjId andalso X#tbl.time>=FromTime andalso X#tbl.time http://www.paceits.com Notice: if you are not an authorised recipient of this e-mail, please contact the sender by return e-mail. In this case, you should not read, print, re-transmit, store, act or rely on this e-mail or any attachments, and should destroy all copies of them. This e-mail and attachments are confidential and may contain copyright material of the sender or other parties. You should only re-transmit, distribute or commercialise the material if you are authorised to do so. This notice should not be removed. From kagato@REDACTED Fri Sep 4 02:35:27 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 3 Sep 2009 17:35:27 -0700 Subject: [erlang-questions] Mnesia Top 1? In-Reply-To: References: Message-ID: <5BBB7F23-E074-4D07-9B87-59EBBEEAAFEE@souja.net> I think QLC will do this. Something like: TH = mnesia:table(Tbl), Q = qlc:q([X<-TH, X#tbl.objId = ObjId, X#tbl.time < FromTime]), QS = qlc:keysort(#tbl.time,Q,[{order,descending}]), QC = qlc:cursor(QS), Row = case qlc:next_answers(QC,1) of [ Top1 ] -> Top1; [] -> undefined; Error -> Error end. Basically, query for those records, apply the descending sort on Time, get a cursor, and grab the first answer. Use a similar process to get the upper bound as well. On Sep 3, 2009, at 5:07 PM, Allan Merolla wrote: > Hi, > I have a massive table containing time-series (values versus time) > for different object. Table structure is like ObjectID, Time, Value. > {ObjectID, Time} is the key. > I have a function that returns interpolated values (e.g. a value for > each 10min) for a time range and object ID called getList(ObjId, > FromTime, ToTime). > > I use the following qlc query to get un-interpolated data. > qlc:q([X<-mnesia:table(Tbl), X#tbl.objId = ObjId andalso > X#tbl.time>=FromTime andalso X#tbl.time so far but the problem is that I need to know what had been the > value for ObjId for a record just before FromTime and also a record > just after ToTime. > I am new to mnesia, but in SQL the answer is: > SELECT TOP1 * FROM Tbl WHERE [ObjectID]=ObjID AND [Time] ORDER BY [Time] DESC. > > I know in sql if I put an index on the [Time] column, the above > operation would be fast even in a massive table. The question is: > What is the most efficient way to do the above top1 query in mnesia? > I found that there is a function called prev(key) in mnesia that > returns the key for the previous record but it won't help me since > previous record may belong to another Object. > > Best Regards// > > > > > Allan Merolla > Analyst Programmer > > One Team, One Dream, Zero! > Incidents & Non Conformances > PACE ITS > Level 2, 120 Wickham Street > Fortitude Valley QLD 4006 > PO Box 141, Albion QLD 4010 > Phone: +61 7 3257 4711 > Fax: +61 7 3257 4788 > Mobile: 0430 300 277 > allan.merolla@REDACTED > http://www.paceits.com > Notice: if you are not an authorised recipient of this e-mail, > please contact the sender by return e-mail. In this case, you should > not read, print, re-transmit, store, act or rely on this e-mail or > any attachments, and should destroy all copies of them. This e-mail > and attachments are confidential and may contain copyright material > of the sender or other parties. You should only re-transmit, > distribute or commercialise the material if you are authorised to do > so. This notice should not be removed. > > -- Jayson Vantuyl kagato@REDACTED From james.hague@REDACTED Fri Sep 4 04:03:10 2009 From: james.hague@REDACTED (James Hague) Date: Thu, 3 Sep 2009 21:03:10 -0500 Subject: [erlang-questions] proposal: expand definition of filename type In-Reply-To: References: <9b08084c0909030902g27c8d80ej934c3fda934a143e@mail.gmail.com> Message-ID: On Thu, Sep 3, 2009 at 2:54 PM, Paul Mineiro wrote: > The point is well taken generally, but for iolists in particular, I find > that I'm often disappointed that various portions of stdlib only take > strings, I admit to being enamored with IO lists (and especially the functions that accept a plain binary, too). It's handy to be able to mix and match binaries and lists to fit the situation, without littering my code with conversions. James From attila.r.nohl@REDACTED Fri Sep 4 11:08:01 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 4 Sep 2009 11:08:01 +0200 Subject: [erlang-questions] Proposal for -warning() directive In-Reply-To: <995D7894-610D-468C-83FE-59F992C39785@souja.net> References: <995D7894-610D-468C-83FE-59F992C39785@souja.net> Message-ID: <401d3ba30909040208q1c1a7113tcdeaf26df872442@mail.gmail.com> 2009/9/4, Jayson Vantuyl : > I've had a case where this would have been useful. Not to have a > directive explosion, I might also recommend -deprecated(). In fact, I > would probably find that more useful than -warning. > > Or maybe there should be a directive more like: > > -emit(warning,"Something bad about this module."). > -emit(deprecated,"This disappears next version."). > > Or I might be overthinking it. But I do think this functionality in > some form is useful. I suggest that you should check the regexp.erl file in the stdlib in OTP R13. It contains this text: -deprecated([sh_to_awk/1,parse/1,format_error/1,match/2,first_match/2,matches/2]). -deprecated([sub/3,gsub/3,split/2]). From kagato@REDACTED Fri Sep 4 11:12:34 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 4 Sep 2009 02:12:34 -0700 Subject: [erlang-questions] Proposal for -warning() directive In-Reply-To: <401d3ba30909040208q1c1a7113tcdeaf26df872442@mail.gmail.com> References: <995D7894-610D-468C-83FE-59F992C39785@souja.net> <401d3ba30909040208q1c1a7113tcdeaf26df872442@mail.gmail.com> Message-ID: <43046AE5-FD59-43D0-A191-C4C05219B350@souja.net> So it does. Gotta love Erlang. It's got so many nooks and crannies. Randomly in that vein, if anyone can explain to me how to use the sofs module, I'd appreciate it. :-D On Sep 4, 2009, at 2:08 AM, Attila Rajmund Nohl wrote: > 2009/9/4, Jayson Vantuyl : >> I've had a case where this would have been useful. Not to have a >> directive explosion, I might also recommend -deprecated(). In >> fact, I >> would probably find that more useful than -warning. >> >> Or maybe there should be a directive more like: >> >> -emit(warning,"Something bad about this module."). >> -emit(deprecated,"This disappears next version."). >> >> Or I might be overthinking it. But I do think this functionality in >> some form is useful. > > I suggest that you should check the regexp.erl file in the stdlib in > OTP R13. It contains this text: > > -deprecated([sh_to_awk/1,parse/1,format_error/1,match/2,first_match/ > 2,matches/2]). > -deprecated([sub/3,gsub/3,split/2]). > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From silvester.roessner@REDACTED Fri Sep 4 11:33:36 2009 From: silvester.roessner@REDACTED (Roessner, Silvester) Date: Fri, 4 Sep 2009 11:33:36 +0200 Subject: AW: [erlang-questions] Handling UTF-8 data when parsing XML using xmerl In-Reply-To: <20090819161636.GC380@ziti.local> References: <20090819161636.GC380@ziti.local> Message-ID: <2CEB6DA5040AED4F946351C85FAC87B5038DE7E5@DEJENSAPP01V1.vision.zeiss.org> Hi Seth, I had also problems with unicode support in xmerl. My solution is to convert the list containing unicode code-points (which I get in my case from .NET) into a UTF-8 string xmerl can handle. fix_unicode(XmlString) -> Binary = unicode:characters_to_binary(XmlString, unicode), binary_to_list(Binary). Since I do only parse XML data with xmerl but use my own functions to output xml I can't tell you if this fix your problem. Hope my answer isn't too late - and that it is really related to your problem ;-) Silvester ____________________ Carl Zeiss Vision GmbH S i l v e s t e r R ? ? n e r Corporate IT Solutions Center Team Head Calculation Engine / VI - IS5 Telefon / Phone: +49 (7361) 591 831 Fax: +49 (7361) 591 498 mailto:silvester.roessner@REDACTED http://www.vision.zeiss.com Carl Zeiss Vision GmbH, Turnstr. 27, 73430 Aalen Gesch?ftsf?hrer: Dr. Raymund Heinen, Thomas Radke Sitz der Gesellschaft: 73430 Aalen, Deutschland Amtsgericht Ulm, HRB 501574, USt.-IdNr:DE 237 102 722 > This message is intended for a particular addressee only and may contain business or company secrets. If you have received this email in error, please contact the sender and delete the message immediately. Any use of this email, including saving, publishing, copying, replication or forwarding of the message or the contents is not permitted. -----Urspr?ngliche Nachricht----- > Von: erlang-questions@REDACTED > [mailto:erlang-questions@REDACTED] Im Auftrag von Seth Falcon > Gesendet: Mittwoch, 19. August 2009 18:17 > An: Erlang Questions > Betreff: [erlang-questions] Handling UTF-8 data when parsing > XML using xmerl > > Hi all, > > I'm using xmerl to parse Atom feed data and have encountered > some surprising behavior with respect to how UTF-8 encoded > data is handled. > > The problem I started to solve is as follows: > > Consider this XML: > > > > blah > > > > The goal is to extract the contents of the node as a > single string. So parse_content(Xml) should return " href='/blah'>blah". > > The approach I took was to use xmerl to parse the entire > document, and then use xmerl:export_simple/2 on the children > of to recapture the text. But in testing with > UTF-8 data, I'm finding that while xmerl will parse UTF-8 > data, it cannot later handle the representation it creates > when calling xmerl:export_simple. > > Here's an example of what I'm seeing: > > First, here's the contents of file simple.xml (pasting the > UTF-8 below, crossing fingers that it comes across in email). > The body of the title tag can be reproduced in an Erlang session as: > > HiThere = [72,105,32,8230,32,116,104,101,114,101]. > > > %% simple.xml: > > Hi ? there > > %% now here's what I see: > > 2> {Xml, _} = xmerl_scan:file("simple.xml"). > {{xmlElement,title,title,[], > {xmlNamespace,[],[]}, > [],1,[], > [{xmlText,[{title,1}], > 1,[], > [72,105,32,8230,32,116,104,101,114,101], > text}], > [],".",undeclared}, > []} > > 3> Exported = lists:flatten(xmerl:export_simple([Xml], xmerl_xml)). > [60,63,120,109,108,32,118,101,114,115,105,111,110,61,34,49, > 46,48,34,63,62,60,116,105,116,108,101,62,72|...] > > 4> xmerl_scan:string(Exported). > 3265- fatal: > {error,{wfc_Legal_Character,{error,{bad_character,8230}}}} > ** exception exit: {fatal, > > {{error,{wfc_Legal_Character,{error,{bad_character,8230}}}}, > {file,file_name_unknown}, > {line,1}, > {col,34}}} > in function xmerl_scan:fatal/2 > in call from xmerl_scan:scan_char_data/5 > in call from xmerl_scan:scan_content/11 > in call from xmerl_scan:scan_element/12 > in call from xmerl_scan:scan_document/2 > in call from xmerl_scan:string/2 > > > %% If I make the following transformation, things work again: > > 5> > xmerl_scan:string(binary_to_list(unicode:characters_to_binary( > Exported))). > {{xmlElement,title,title,[], > {xmlNamespace,[],[]}, > [],1,[], > [{xmlText,[{title,1}], > 1,[], > [72,105,32,8230,32,116,104,101,114,101], > text}], > [],"/opt/seth/EVRI/sg/GIT/zgst/rods",undeclared}, > []} > > > %% and strangely, given that I think I do have valid UTF-8, > this also %% works: > > 6> xmerl_scan:string(Exported, [{encoding, latin1}]). > > > Questions: > > * Is this the expected behavior? > > * Suggestions for a better way of doing the parsing or handling the > UTF-8 strings? > > Thanks, > > + seth > > ________________________________________________________________ > erlang-questions mailing list. See > http://www.erlang.org/faq.html erlang-questions (at) erlang.org > > From csanto@REDACTED Fri Sep 4 12:08:10 2009 From: csanto@REDACTED (Corrado Santoro) Date: Fri, 4 Sep 2009 12:08:10 +0200 (CEST) Subject: [ANN] new release of AMNESIA Message-ID: <57595.151.97.19.54.1252058890.squirrel@webmail.cdc.unict.it> Dear Erlangers, I would like to announce a new release (1.4.0) of AMNESIA, the erlang wrapper for SQL DBMSs. New features: - support for aggregated queries - cursors - booleans (thanks Juan!) default values in table definitions Any contrib (e.g. a driver for Postgres) is welcome, as well as any suggestion of bug reporting :-) Amnesia can be found at: http://amnesia.sourceforge.net Feel free to contact any of the authors for problems and/or suggestions. All the best, --Corrado From koops.j@REDACTED Fri Sep 4 13:04:17 2009 From: koops.j@REDACTED (Jeroen Koops) Date: Fri, 4 Sep 2009 13:04:17 +0200 Subject: [JOB] Looking for an Erlang/OTP developer in Amsterdam Message-ID: <331a9abb0909040404v3addfab4ydde19495e9600517@mail.gmail.com> Texprezzo, a privately owned startup based in Amsterdam, the Netherlands, is looking for a full-time Erlang/OTP developer. What we ask: - Several years of professional programming experience, not necessarily in Erlang. - Strong interest in, and some experience with, developing in and learning about Erlang/OTP. - Ability to work unsupervised - good problem-solving skills. - Good communication skills in English. Knowledge of any of the following is a plus: - HTML, Javascript, PHP, Linux, Solaris - Telecom-related technologies, especially relating to SMS What we offer: - Opportunity to be among the first employees in a startup - Competitive renumeration package - Earn a living doing what you love doing: programming Erlang! Note: This is a permanent position, at this moment we are not interested in contractors/freelancers. To apply, please send a CV to jeroen.koops@REDACTED. You can also contact me for more information at the same address, or by telephone on +31-20-3120528. *Company information:* *The Texprezzo Group was founded in 2008 by a team of experienced telecom professionals with a wide variety of backgrounds with experience in various areas related to mobile messaging. Their goal was to create new value with the existing messaging infrastructures as currently deployed by mobile operators today, thus stimulating a new wave of revenues and end user experience. The main investors behind Texprezzo Group BV have a long track record in establishing and creating new technology businesses. An example of which is the recently sold mBalance Group, the leading SMS Routing infrastructure company.* From tuncer.ayaz@REDACTED Fri Sep 4 15:30:55 2009 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 4 Sep 2009 15:30:55 +0200 Subject: [erlang-questions] wx fails to build on Snow Leopard In-Reply-To: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> References: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> Message-ID: <4ac8254d0909040630o43cc6beo3988d9e2fc24d3bb@mail.gmail.com> On Wed, Sep 2, 2009 at 5:10 PM, Steve Vinoski wrote: > I'm seeing that the wx part of the Erlang/OTP build doesn't work on OS X > 10.6 (Snow Leopard). I don't use wx so it's no big deal to me, but I'm > seeing the g++ compiler appear to hang when compiling > lib/wx/c_src/gen/wxe_funcs.cpp. I let it run quite awhile and it never got > past that point, but my laptop fans were running full blast so I wonder if > the compilation got stuck in an infinite loop. If I manually edit wx out of > the lib/Makefile, then everything builds and seems to work as normal. There > are also lots of compilation warnings in the wx area due to deprecation of > some APIs being used and due to string constants being passed as char*. The build continues once that one long compile step has finished. What was "quite a while"? 10 minutes? 30 minutes? Can you try to build again and wait a little longer? This does not mean that the wx driver now automatically works on OSX: "WX Failed loading "wxe_driver"@[...]". Also, by default on OSX.6 it's -m64, so: wxe_driver.so: Mach-O 64-bit bundle x86_64.). The wx driver not easily working is an issue specific to Darwin, there are no problems like that on linux. For the record, Snow Leopard seems to bundle wx-2.8.8 and the on 10.5 it?also failed with 2.8.10. That's why I didn't try to use a 2.8.10 install of wxMac for the driver, yet. From tuncer.ayaz@REDACTED Fri Sep 4 15:32:35 2009 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 4 Sep 2009 15:32:35 +0200 Subject: [erlang-questions] Native code for Windows/MacOS? In-Reply-To: <4ac8254d0906301141j7f5e84f2n7621a2d11c40630@mail.gmail.com> References: <33D06C00-5F12-4410-8343-FA69ADFB15F3@cs.otago.ac.nz> <88FDA2AE-D877-4417-8633-75D5FA619D56@cs.otago.ac.nz> <91ECEC2C-E0A1-4D4B-A9D8-8CCA1F166B40@cs.otago.ac.nz> <4ac8254d0906260649r1af5b434p9800d227f173c782@mail.gmail.com> <4A49E1FC.9090600@cs.ntua.gr> <4ac8254d0906300328w1d48ace6wbb9a86f34794c0a6@mail.gmail.com> <4A4A13DE.4000305@cs.ntua.gr> <4ac8254d0906301141j7f5e84f2n7621a2d11c40630@mail.gmail.com> Message-ID: <4ac8254d0909040632p7bb539cco9966291bbfc7f350@mail.gmail.com> On Tue, Jun 30, 2009 at 8:41 PM, Tuncer Ayaz wrote: > On Tue, Jun 30, 2009 at 3:32 PM, Kostis Sagonas wrote: >> Tuncer Ayaz wrote: >>> >>> On Tue, Jun 30, 2009 at 10:24 AM, Joel Reymont wrote: >>>> >>>> On Jun 30, 2009, at 10:59 AM, Kostis Sagonas wrote: >>>> >>>>> Since, as it appears from the post above, HiPE works OK when Xcode is >>>>> installed if one uses --enable-hipe, >>>> >>>> It doesn't or doesn't always. Richard O'Keefe had to install mach >>>> override >>>> and I wasn't able to build at all. I don't know how it can possibly work >>>> for >>>> Tuncer. >>> >>> It may very well be that the repl says HiPE is enabled but is not going to >>> work >>> if I try to use it at runtime. Need to test that. >> >> I somehow doubt that HiPE says enabled without actually being so. >> One easy way to test that is to fire up erl and issue the following command: >> >> ?1> hipe:c(lists, [verbose]). >> >> If you see something like: >> >> ? Compiling: lists >> ? Options: [verbose,load,icode_ssa_const_prop,...]. >> ? Compiling {lists,'-concat/1-fun-0-',1} >> ? Compiling {lists,'-filter/2-lc$^0/1-0-',2} >> ? Compiling {lists,module_info,1} >> ? Compiling {lists,module_info,0} >> ? Compiling {lists,rufmerge2_2,6} >> ?... >> ?{ok,lists} >> >> The hipe compiler is working OK. ?For extra sanity, repeat the previous >> command and if it crashes with a message: >> >> ?=ERROR REPORT==== 30-Jun-2009::16:29:05 === >> ?Module lists must be purged before loading >> >> then you can be pretty sure that things are *really* working as they should. >> ?(This is not a joke.) > > I can confirm that it *really* works :). > > Any idea what's missing for x86_64 HiPE on OSX? > FYI, on Snow Leopard the compiler does (naturally) default to -m64 and therefore we get a x86_64 beam without HiPE. From seth@REDACTED Fri Sep 4 16:11:23 2009 From: seth@REDACTED (Seth Falcon) Date: Fri, 4 Sep 2009 07:11:23 -0700 Subject: [erlang-questions] Handling UTF-8 data when parsing XML using xmerl In-Reply-To: <2CEB6DA5040AED4F946351C85FAC87B5038DE7E5@DEJENSAPP01V1.vision.zeiss.org> References: <20090819161636.GC380@ziti.local> <2CEB6DA5040AED4F946351C85FAC87B5038DE7E5@DEJENSAPP01V1.vision.zeiss.org> Message-ID: <20090904141123.GJ12397@ziti.local> Hi Silvester, * On 2009-09-04 at 11:33 +0200 Roessner, Silvester wrote: > I had also problems with unicode support in xmerl. > > My solution is to convert the list containing unicode code-points > (which I get in my case from .NET) > into a UTF-8 string xmerl can handle. > > fix_unicode(XmlString) -> > Binary = unicode:characters_to_binary(XmlString, unicode), > binary_to_list(Binary). That looks quite similar to the workaround included in my original post. The need for such a transformation lacks some elegance IMO -- especially in the context of the original post where the string is coming from data that xmerl has already parsed once. So my question is really whether that's how things are "supposed to be" or whether there is a different and cleaner approach or a way to fix xmerl to handle this better. But I'm glad to know that someone else has seen a similar issue and come to a similar work around. Thanks, + seth -- Seth Falcon | @sfalcon | http://userprimary.net/user From colm.dougan@REDACTED Fri Sep 4 16:21:48 2009 From: colm.dougan@REDACTED (Colm Dougan) Date: Fri, 4 Sep 2009 15:21:48 +0100 Subject: Munging ets continuations Message-ID: <24d4f39c0909040721j6c6a2f92j97881b1aa5bff7cc@mail.gmail.com> Hi, I know that I can do : {_, C} = ets:match_object(T, '_', 5). And the continuation will looks something like this : {16397,5,[],5,<<>>,[],0,0} ... where it looks like the first element is the ets table reference and that the two 5s are the next offset and limit. I realize that the continuation contains an opaque reference and so I need to use repair_continuation to "bless" an arbitrary tuple (like the above) to make it a usable continuation. My question is: is there a programmatic way to create continuations for a given offset/length? Or is there a way I can munge it myself? I have tried this : ets:select( ets:repair_continuation( {Tab, Offset, [], Limit, <<>>, [], 0, 0}, [{Query,[],['$_']}] )) Initially results were encouraging but it didn't appear to work in some cases and I know it may be encapsulation breaching and evil. My use-case is that I have an ordered_set table with (say) 1000 entries and I want to look at a given offset within them, e.g. entries 700 to 720 without having to pull back the whole list or walk along it in chunks until I reach 700. What i want to do is use something like continuations to give me the appropriate slice in one shot. Maybe continuations are not the answer - if not then I'd like to hear about any alternative solutions to this. Any help appreciated. Thanks, Colm From nicolas@REDACTED Fri Sep 4 17:02:09 2009 From: nicolas@REDACTED (Nicolas Niclausse) Date: Fri, 04 Sep 2009 17:02:09 +0200 Subject: [erlang-questions] State of the Union: FFI In-Reply-To: <1239876259.10508.29.camel@gnatziu> References: <2812B7B9-6B8C-4CDE-A966-96624AF61625@me.com> <22831017.post@talk.nabble.com> <01D2B728-D19B-4923-A5A8-DE08CC70A908@me.com> <23021754.post@talk.nabble.com> <23063501.post@talk.nabble.com> <1239876259.10508.29.camel@gnatziu> Message-ID: <4AA12BF1.2080308@niclux.org> Alceste Scalas ecrivait le 16/04/2009 12:04: > Il giorno mer, 15/04/2009 alle 10.32 -0700, Zvi ha scritto: >> There is no ArithAlloc macro in R13A, so I changed it to HAlloc. >> There is digit_t type in R13A so I defined it as ErtsHalfDigit (typedef >> Uint16 digit_t;) >> Fortunattely there not too much changes: [...] >> >> Can you prepare a clean diff against R13A sources? >> Maybe Alceste will help us. > > > Thank you for your work! > > I'm a bit busy right now, but I'll try your changes here and generate > some clean diffs (I'm managing the FFI stuff in a Mercurial repository, > I'll need to rebase everything on R13A in order to keep the history). Hi, Any news on FFI for R13B ? And what is the status of the EEP ? -- Nicolas From harmlessbystandard@REDACTED Fri Sep 4 20:54:56 2009 From: harmlessbystandard@REDACTED (Davichi pavel) Date: Fri, 4 Sep 2009 11:54:56 -0700 (PDT) Subject: iPhone_Erlang Static Library In-Reply-To: <4AA12BF1.2080308@niclux.org> References: <2812B7B9-6B8C-4CDE-A966-96624AF61625@me.com> <22831017.post@talk.nabble.com> <01D2B728-D19B-4923-A5A8-DE08CC70A908@me.com> <23021754.post@talk.nabble.com> <23063501.post@talk.nabble.com> <1239876259.10508.29.camel@gnatziu> <4AA12BF1.2080308@niclux.org> Message-ID: <268682.75514.qm@web31403.mail.mud.yahoo.com> So I have been working on turning Erlang into a static library for the iPhone, and I have almost finished. However, I have two errors that I don't know how to fix: multi_drv.c:47:0 Variable 'multi_driver_entry' has initializer but incomplete type mem_drv.c:38:0 Variable 'mem_driver_entry' has initializer but incomplete type I am not proficient enough in C or C++ to figure these out. Can someone give me some advice about this? Here's the code where its throwing the error: struct driver_entry multi_driver_entry = { multi_init, multi_start, multi_stop, multi_erlang_read, NULL, NULL, "multi" }; const struct driver_entry mem_driver_entry = { mem_init, mem_start, mem_stop, mem_command, NULL, NULL, "mem_drv" }; Samuel Barney From joelr1@REDACTED Fri Sep 4 22:18:29 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 4 Sep 2009 21:18:29 +0100 Subject: gathering error reports on a single node Message-ID: Is there a recipe for gathering error_logger output from several nodes on a single one? I would like to redirect the error_logger output for my cluster to the shell prompt of the node I'm connected to and looking at. Thanks, Joel --- fastest mac firefox! http://wagerlabs.com From mjtruog@REDACTED Fri Sep 4 22:51:09 2009 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 04 Sep 2009 13:51:09 -0700 Subject: [erlang-questions] gathering error reports on a single node In-Reply-To: References: Message-ID: <4AA17DBD.2010204@gmail.com> That is suppose to be taken care of for you with the erlang:group_leader/0. By default the error_logger uses the tty option to use stdout which should find its way to your shell, as long as all processes doing the error_logger-ing have the same group_leader. (at least that is the way I understand it) Joel Reymont wrote: > Is there a recipe for gathering error_logger output from several nodes > on a single one? > > I would like to redirect the error_logger output for my cluster to the > shell prompt of the node I'm connected to and looking at. > > Thanks, Joel > > --- > fastest mac firefox! > http://wagerlabs.com > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From joelr1@REDACTED Fri Sep 4 23:11:33 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 4 Sep 2009 22:11:33 +0100 Subject: [erlang-questions] gathering error reports on a single node In-Reply-To: <4AA17DBD.2010204@gmail.com> References: <4AA17DBD.2010204@gmail.com> Message-ID: <3FB8FFCE-1AF3-4463-B2DC-DE49EBEE38A3@gmail.com> So something like this from the shell then? [ group_leader(group_leader(), rpc:call(Node, erlang, whereis, [error_logger])) || Node <- nodes()]. On Sep 4, 2009, at 9:51 PM, Michael Truog wrote: > That is suppose to be taken care of for you with the > erlang:group_leader/0. By default the error_logger uses the tty > option > to use stdout which should find its way to your shell, as long as all > processes doing the error_logger-ing have the same group_leader. (at > least that is the way I understand it) --- fastest mac firefox! http://wagerlabs.com From mjtruog@REDACTED Sat Sep 5 01:03:56 2009 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 04 Sep 2009 16:03:56 -0700 Subject: [erlang-questions] gathering error reports on a single node In-Reply-To: <3FB8FFCE-1AF3-4463-B2DC-DE49EBEE38A3@gmail.com> References: <4AA17DBD.2010204@gmail.com> <3FB8FFCE-1AF3-4463-B2DC-DE49EBEE38A3@gmail.com> Message-ID: <4AA19CDC.1050903@gmail.com> I think that should work since error_logger depends on io:format/3 as shown in lib/erlang/lib/stdlib-1.16.2/src/error_logger_tty_h.erl. If the group_leader isn't set in other processes, they would need to depend on error_logger output. If you care about sasl, it seems like it would need to also have the group_leader set. It is easier if you set the group_leader at the top of the process hierarchy though... if that is possible. Joel Reymont wrote: > So something like this from the shell then? > > [ group_leader(group_leader(), rpc:call(Node, erlang, whereis, > [error_logger])) || Node <- nodes()]. > > On Sep 4, 2009, at 9:51 PM, Michael Truog wrote: > >> That is suppose to be taken care of for you with the >> erlang:group_leader/0. By default the error_logger uses the tty option >> to use stdout which should find its way to your shell, as long as all >> processes doing the error_logger-ing have the same group_leader. (at >> least that is the way I understand it) > > --- > fastest mac firefox! > http://wagerlabs.com > > > > > From leaf.pub1@REDACTED Sat Sep 5 05:15:28 2009 From: leaf.pub1@REDACTED (Leaf Petersen) Date: Fri, 4 Sep 2009 20:15:28 -0700 Subject: DAMP 2010 Final Call For Papers Message-ID: Apologies if this is not the appropriate place to post this. We would love to get more submissions and participation from the Erlang community this year - if you are working on something that might be of interest please consider submitting! Thanks, Leaf Petersen (General Chair) DAMP 2010: Workshop Declarative Aspects of Multicore Programming Madrid, SPAIN (colocated with POPL 2010) January 19, 2010 damp10.cs.nmsu.edu SUBMISSION DEADLINE: SEPTEMBER 21, 2009 The advent of multicore architectures has profoundly increased the importance of research in parallel computing. Modern platforms are becoming more complex and heterogenous and novel solutions are needed to account for their peculiarities. Multicore architectures will differ in significant ways from their multisocket predecessors. For example, the communication to compute bandwidth ratio is likely to be higher, which will positively impact performance. More generally, multicore architectures introduce several new dimensions of variability in both performance guarantees and architectural contracts, such as the memory model, that may not stabilize for several generations of product. Programs written in functional or (constraint-)logic programming languages, or in other highly declarative languages with a controlled use of side effects, can greatly simplify parallel programming. Such declarative programming allows for a deterministic semantics even when the underlying implementation might be highly non-deterministic. In addition to simplifying programming this can simplify debugging and analyzing correctness. DAMP 2010 is the fifth in a series of one-day workshops seeking to explore ideas in declarative programming language design that will greatly simplify programming for multicore architectures, and more generally for tightly coupled parallel architectures. The emphasis will be on (constraint-)logic and functional programming, but any declarative programming language ideas that aim to raise the level of abstraction are welcome. DAMP seeks to gather together researchers in declarative approaches to parallel programming and to foster cross fertilization across different approaches. Specific topics include, but are not limited to: * investigation of applications of logic, constraint logic, and functional programing to multicore programing * run-time issues of exploitation of parallelism using declarative programming approaches (e.g., garbage collection, scheduling) * architectural impact on exploitation of parallelism from declarative languages * type systems and analysis for accurately detecting dependencies, aliasing, side effects, and impure features * language level declarative constructs for expressing parallelism * declarative language specification for the description of data placement and distribution * compilation and static analysis techniques to support exploitation of parallelism from declarative languages (e.g., granularity control) * practical experiences and challenges arising from parallel declarative programming * technology for debugging parallel programs * design and implementation of domain-specific declarative languages for multicore programming Submission: Submitted papers papers should not exceed 10 pages in ACM SIGPLAN conference format. Submission is electronic via: http://www.easychair.org/conferences/?conf=damp10 Accepted papers will be published in the ACM Digital Library and in a physical proceedings. Papers must adhere to the SIGPLAN Republication Policy: http://www.sigplan.org/republicationpolicy.htm Concurrent submissions to other conferences, workshops, journals, or similar forums of publication are not allowed. However, DAMP is intended to be a venue for discussion and exploration of works-in-progress, and so publication of a paper at DAMP 2010 is not intended to preclude later publication as appropriate. Additional information about the submission process can be found at the conference web site. Important dates: Abstract submission: Sept. 21 Paper submission: Sept. 25 Notification to authors: Oct. 26 Camera ready: Nov. 9 Program Chair: Enrico Pontelli New Mexico State University General Chairs: Leaf Petersen Intel Corporation Santa Clara, CA, USA Program Committee: Manuel Carro Universidad Politecnica de Madrid Clemens Grelck University of Hertfordshire Haifeng Guo University of Nebraska at Omaha Gabriele Keller University of New South Wales Hans-Wolfgang Loidl Ludwig-Maximilians-Universitat Munchen Leaf Petersen Intel Corporation John Reppy University of Chicago Ricardo Rocha University of Porto Kostis Sagonas National Technical University of Athens Vitor Santos Costa University of Porto Satnam Singh Microsoft Research Philip Trinder Heriot-Watt University Pascal Van Hentenryck Brown University URL: http://damp10.cs.nmsu.edu From harmlessbystandard@REDACTED Sat Sep 5 05:04:36 2009 From: harmlessbystandard@REDACTED (Davichi pavel) Date: Fri, 4 Sep 2009 20:04:36 -0700 (PDT) Subject: [erlang-questions] iPhone_Erlang Static Library In-Reply-To: <268682.75514.qm@web31403.mail.mud.yahoo.com> References: <2812B7B9-6B8C-4CDE-A966-96624AF61625@me.com> <22831017.post@talk.nabble.com> <01D2B728-D19B-4923-A5A8-DE08CC70A908@me.com> <23021754.post@talk.nabble.com> <23063501.post@talk.nabble.com> <1239876259.10508.29.camel@gnatziu> <4AA12BF1.2080308@niclux.org> <268682.75514.qm@web31403.mail.mud.yahoo.com> Message-ID: <237792.77057.qm@web31405.mail.mud.yahoo.com> I have managed to compile erts into a static library. However there are a few warnings that I have to get rid of. Here they are: "Initialization from incompatible pointer" This happens in bin_drv.c, multi_drv.c, and sys.c. All of these come from the contents of structs based off of the struct erl_drv_entry. Any help would be nessasary. Here is a bit of the code that I have changed but throws the errors: struct erl_drv_e multi_driver_entry = { multi_init, multi_start, # Throws the warning multi_stop, #throws the warning multi_erlang_read, #Throws the warning NULL, NULL, "multi" }; const struct erl_drv_entry bin_driver = { bin_init, bin_start, #Throws the warning bin_stop, #Throws the warning bin_erlang_read, #Throws that warning NULL, NULL, "binary_filer" }; Sometimes you have to stand in the mud to get anything done -Loki- ________________________________ From: Davichi pavel To: erlang-questions@REDACTED Sent: Friday, September 4, 2009 12:54:56 PM Subject: [erlang-questions] iPhone_Erlang Static Library So I have been working on turning Erlang into a static library for the iPhone, and I have almost finished. However, I have two errors that I don't know how to fix: multi_drv.c:47:0 Variable 'multi_driver_entry' has initializer but incomplete type mem_drv.c:38:0 Variable 'mem_driver_entry' has initializer but incomplete type I am not proficient enough in C or C++ to figure these out. Can someone give me some advice about this? Here's the code where its throwing the error: struct driver_entry multi_driver_entry = { multi_init, multi_start, multi_stop, multi_erlang_read, NULL, NULL, "multi" }; const struct driver_entry mem_driver_entry = { mem_init, mem_start, mem_stop, mem_command, NULL, NULL, "mem_drv" }; Samuel Barney From nick@REDACTED Sat Sep 5 19:40:07 2009 From: nick@REDACTED (Niclas Eklund) Date: Sat, 5 Sep 2009 19:40:07 +0200 (CEST) Subject: [erlang-bugs] Erlang R13B01 ssh-1.1.3 cipher key matching bug and documentation errors In-Reply-To: <20090712025646.GA44385@k2r.org> References: <20090712025646.GA44385@k2r.org> Message-ID: Hello! Thank you for the input. At some parts will make it into R13B02. Niclas @Erlang/OTP On Sun, 12 Jul 2009, Kenji Rikitake wrote: > Here's a list of bugs/documentation errors of ssh-1.1.3 for R13B01 which > I experienced yesterday. > > * [bug] ssh:shell/3 and ssh:connect/3 do not crash immediately even if > they fail to negotiate the cipher to use, and hang forever > > How to reproduce: > set NOT to accept 3des-cbc as a cipher on the server > (in OpenSSH, set Ciphers directive at sshd_config, *excluding* 3des-cbc) > > Possible reason: failure of finding a matching cipher does not throw > an exception immediately (I haven't tested yet). > > FYI: on Portable OpenSSH 5.1 for FreeBSD slogin client, it will turn > out to be something like the following: > > -- quote -- > debug1: match: OpenSSH_5.1p1 FreeBSD-20080901 pat OpenSSH* > debug1: Enabling compatibility mode for protocol 2.0 > debug1: Local version string SSH-2.0-OpenSSH_5.1p1 FreeBSD-20080901 > debug1: SSH2_MSG_KEXINIT sent > debug1: SSH2_MSG_KEXINIT received > no matching cipher found: client 3des-cbc server aes128-ctr,blowfish-cbc,aes128-cbc,aes192-cbc,aes256-cbc > -- unquote -- > > * [documentation error] ssh manual should include that the current ssh > module only supports the following crypto parameters of SSH Version 2 > protocol: (my opinion follows later in this message) > > cipher: 3des-cbc only > MACs: hmac-sha1 only > > * [documentation error] ssh manual should include that only an > *unencrypted* private key is supported for ssh_rsa public key > authentication. > > The manual should also note that private keys for public key > authentication used for interactive logins are mostly encrypted so > cannot be used for the time being. > > * [documentation error] ssh:connect/1 and ssh:connect/2 no longer exist, > but still documented. Description for those old functions should be > eliminated, and requirement to use ssh:connect/3 instead should be > described. > > * [my opinion] I personally think only supporting 3des-cbc is *archaic* > and insufficient; implementing at least stronger ciphers such as > aes128-cbc and aes256-cbc, or even blowfish-cbc, should be considered > ASAP, regarding the strength of the ciphers. > > Regards, > Kenji Rikitake > > > ________________________________________________________________ > erlang-bugs mailing list. See http://www.erlang.org/faq.html > erlang-bugs (at) erlang.org > From dmitriid@REDACTED Sun Sep 6 21:47:39 2009 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Sun, 6 Sep 2009 22:47:39 +0300 Subject: Reanimation of elryvideo Message-ID: A member of the Erlang Russian mailing list (erlang-russian@REDACTED ) has decided to continue working on erlyvideo (http://code.google.com/p/erlyvideo/ ) The new repo is located here: http://github.com/maxlapshin/erlyvideo The new version can now send notifications to flash clients (something like a push channel) and can read (but not yet play, possibly due to a faulty handshake) mp4 files. Any help would be greatly appreciated From kunthar@REDACTED Sun Sep 6 22:25:19 2009 From: kunthar@REDACTED (Kunthar) Date: Sun, 6 Sep 2009 23:25:19 +0300 Subject: [erlang-questions] Reanimation of elryvideo In-Reply-To: References: Message-ID: <9a09ca9a0909061325w3cb18005x93f1b61af6afa1c4@mail.gmail.com> Really happy to see this. I would like join in as soon as have time for it. Kunth On Sun, Sep 6, 2009 at 10:47 PM, Dmitrii Dimandt wrote: > A member of the Erlang Russian mailing list > (erlang-russian@REDACTED) has decided to continue working on > erlyvideo (http://code.google.com/p/erlyvideo/) > > The new repo is located here: http://github.com/maxlapshin/erlyvideo > > The new version can now send notifications to flash clients (something like > a push channel) and can read (but not yet play, possibly due to a faulty > handshake) mp4 files. > > Any help would be greatly appreciated From rapsey@REDACTED Sun Sep 6 22:34:07 2009 From: rapsey@REDACTED (Rapsey) Date: Sun, 6 Sep 2009 22:34:07 +0200 Subject: [erlang-questions] Reanimation of elryvideo In-Reply-To: References: Message-ID: <97619b170909061334u115bf497u4d22ed8488ac631c@mail.gmail.com> The problem with playing mp4 files is quite possibly a lack of AVC sequence header (check the official flash file format spec). The AVC sequence header needs to be sent to the client before any h264 video frame, otherwise flash will not display the video. This is something that needs to be generated from the encoded video frames, because it is not present in the mp4 file format. Look at the ffmpeg source code to see how to generate the avc sequence header. It's in libavformat/flvenc.c a call to ff_isom_write_avcc(...) does and it is defined in libavformat/avc.c Sergej On Sun, Sep 6, 2009 at 9:47 PM, Dmitrii Dimandt wrote: > A member of the Erlang Russian mailing list ( > erlang-russian@REDACTED) has decided to continue working on > erlyvideo (http://code.google.com/p/erlyvideo/) > > The new repo is located here: http://github.com/maxlapshin/erlyvideo > > The new version can now send notifications to flash clients (something like > a push channel) and can read (but not yet play, possibly due to a faulty > handshake) mp4 files. > > Any help would be greatly appreciated From dgud@REDACTED Mon Sep 7 08:00:18 2009 From: dgud@REDACTED (Dan Gudmundsson) Date: Mon, 07 Sep 2009 08:00:18 +0200 Subject: [erlang-bugs] Re: [erlang-questions] wx fails to build on Snow Leopard In-Reply-To: <4ac8254d0909040630o43cc6beo3988d9e2fc24d3bb@mail.gmail.com> References: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> <4ac8254d0909040630o43cc6beo3988d9e2fc24d3bb@mail.gmail.com> Message-ID: <4AA4A172.1040809@erix.ericsson.se> I don't think wxWidgets 2.8.* works in 64-bit, it uses the old gui api. They are working on fixing that in wxWidgets 2.9 and 3.0. But the problem remains, for me the emulator crashes with a SIGTRAP when loading the driver. /Dan Tuncer Ayaz wrote: > On Wed, Sep 2, 2009 at 5:10 PM, Steve Vinoski wrote: >> I'm seeing that the wx part of the Erlang/OTP build doesn't work on OS X >> 10.6 (Snow Leopard). I don't use wx so it's no big deal to me, but I'm >> seeing the g++ compiler appear to hang when compiling >> lib/wx/c_src/gen/wxe_funcs.cpp. I let it run quite awhile and it never got >> past that point, but my laptop fans were running full blast so I wonder if >> the compilation got stuck in an infinite loop. If I manually edit wx out of >> the lib/Makefile, then everything builds and seems to work as normal. There >> are also lots of compilation warnings in the wx area due to deprecation of >> some APIs being used and due to string constants being passed as char*. > > The build continues once that one long compile step has finished. > What was "quite a while"? 10 minutes? 30 minutes? > > Can you try to build again and wait a little longer? > > This does not mean that the wx driver now automatically works on OSX: > "WX Failed loading "wxe_driver"@[...]". > Also, by default on OSX.6 it's -m64, so: > wxe_driver.so: Mach-O 64-bit bundle x86_64.). > > The wx driver not easily working is an issue specific to Darwin, > there are no problems like that on linux. > > For the record, Snow Leopard seems to bundle wx-2.8.8 and the > on 10.5 it also failed with 2.8.10. That's why I didn't try to use > a 2.8.10 install of wxMac for the driver, yet. > > ________________________________________________________________ > erlang-bugs mailing list. See http://www.erlang.org/faq.html > erlang-bugs (at) erlang.org > > From paul.joseph.davis@REDACTED Mon Sep 7 08:18:17 2009 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Mon, 7 Sep 2009 02:18:17 -0400 Subject: [erlang-questions] wx fails to build on Snow Leopard In-Reply-To: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> References: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> Message-ID: On Wed, Sep 2, 2009 at 11:10 AM, Steve Vinoski wrote: > I'm seeing that the wx part of the Erlang/OTP build doesn't work on OS X > 10.6 (Snow Leopard). I don't use wx so it's no big deal to me, but I'm > seeing the g++ compiler appear to hang when compiling > lib/wx/c_src/gen/wxe_funcs.cpp. I let it run quite awhile and it never got > past that point, but my laptop fans were running full blast so I wonder if > the compilation got stuck in an infinite loop. If I manually edit wx out of > the lib/Makefile, then everything builds and seems to work as normal. There > are also lots of compilation warnings in the wx area due to deprecation of > some APIs being used and due to string constants being passed as char*. > --steve > I just thought I'd pipe up and say that while this part of the build does take an awfully long time, it eventually finishes. I've been working through build issues with a couple projects on Mac OS X 10.6 all weekend and this one eventually concerned me until I realized it was just a bit slow. I can confirm that calling wx:new() throws an error that mentions undefined symbols. I've pasted it below. I'm not actively using wx but maybe the trace helps someone somewhere. Paul Davis $ ./bin/erl Erlang R13B01 (erts-5.7.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) 1> wx:new(). =ERROR REPORT==== 7-Sep-2009::02:12:56 === WX Failed loading "wxe_driver"@"/Users/davisp/tests/otp/otp_src_R13B01/lib/wx/priv/i386-apple-darwin10.0.0" ** exception error: {load_driver,"dlopen(/Users/davisp/tests/otp/otp_src_R13B01/lib/wx/priv/i386-apple-darwin10.0.0/wxe_driver.so, 2): Symbol not found: __ZN10wxBoxSizer11RecalcSizesEv\n Referenced from: /Users/davisp/tests/otp/otp_src_R13B01/lib/wx/priv/i386-apple-darwin10.0.0/wxe_driver.so\n Expected in: flat namespace\n in /Users/davisp/tests/otp/otp_src_R13B01/lib/wx/priv/i386-apple-darwin10.0.0/wxe_driver.so"} in function wxe_server:start/0 in call from wx:new/1 2> From ulf.wiger@REDACTED Mon Sep 7 10:25:58 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 07 Sep 2009 10:25:58 +0200 Subject: [erlang-questions] State of the Union: FFI In-Reply-To: <4AA12BF1.2080308@niclux.org> References: <2812B7B9-6B8C-4CDE-A966-96624AF61625@me.com> <22831017.post@talk.nabble.com> <01D2B728-D19B-4923-A5A8-DE08CC70A908@me.com> <23021754.post@talk.nabble.com> <23063501.post@talk.nabble.com> <1239876259.10508.29.camel@gnatziu> <4AA12BF1.2080308@niclux.org> Message-ID: <4AA4C396.50805@erlang-consulting.com> Nicolas Niclausse wrote: > > Hi, > > Any news on FFI for R13B ? And what is the status of the EEP ? At the Erlang Workshop this weekend, Kenneth Lundin listed some things to come in the near future. Among those things were dynamically loadable BIFs. When asked, Kenneth said that they ought to be released within a year, hopefully within the next 6 months. Kenneth is vacationing in the Scottish Highlands this week, so I thought I'd give this account of what he said. Obviously, I don't speak for the OTP team, so you'll have to wait for them to give you the official version. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ulf.wiger@REDACTED Mon Sep 7 10:37:43 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 07 Sep 2009 10:37:43 +0200 Subject: [erlang-questions] Munging ets continuations In-Reply-To: <24d4f39c0909040721j6c6a2f92j97881b1aa5bff7cc@mail.gmail.com> References: <24d4f39c0909040721j6c6a2f92j97881b1aa5bff7cc@mail.gmail.com> Message-ID: <4AA4C657.9090107@erlang-consulting.com> Colm Dougan wrote: > My question is: is there a programmatic way to create continuations > for a given offset/length? Or is there a way I can munge it myself? I imagine that you would have to be prepared to meditate regularly on the VM internals in order to maintain this. :) > My use-case is that I have an ordered_set table with (say) 1000 > entries and I want to look at a given offset within them, e.g. > entries 700 to 720 without having to pull back the whole list or walk > along it in chunks until I reach 700. Presumably, you don't know the value of the key of entry 700, or you would simply pass that to ets:next/2 and iterate from there. It seems to me that ets:slot/2 is what you need. From the manual: "If the table is of type ordered_set, the function returns a list containing the I:th object in Erlang term order." (It starts with slot 0). BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From mml6364@REDACTED Mon Sep 7 11:11:52 2009 From: mml6364@REDACTED (mliu) Date: Mon, 7 Sep 2009 02:11:52 -0700 (PDT) Subject: sinan/faxien otp live upgrade Message-ID: Hi everyone, New to the list...wondering if sinan/faxien users know whether it can handle hot upgrades of applications/release? I'm starting a new otp project and am experimenting with sinan/faxien for the dev/build/test cycle, and wondering if others are using it for deployment and release upgrade maintenance. I've been able to successfully publish my otp app to a local repository, and then install the release using faxien. What I'm wondering is, how does faxien deal with a live system? Does it take into appup/relup files in the release? Can it automatically issue the required release_handler calls? If not, is there something out there that will help with the live release upgrade process? Thanks! Mike From tuncer.ayaz@REDACTED Mon Sep 7 11:24:12 2009 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 7 Sep 2009 11:24:12 +0200 Subject: [erlang-bugs] Re: [erlang-questions] wx fails to build on Snow Leopard In-Reply-To: <4AA4A172.1040809@erix.ericsson.se> References: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> <4ac8254d0909040630o43cc6beo3988d9e2fc24d3bb@mail.gmail.com> <4AA4A172.1040809@erix.ericsson.se> Message-ID: <4ac8254d0909070224p11db54f6idaad05b6fb6ad0ab@mail.gmail.com> On Mon, Sep 7, 2009 at 8:00 AM, Dan Gudmundsson wrote: > I don't think wxWidgets 2.8.* works in 64-bit, it uses the old gui api. > They are working on fixing that in wxWidgets 2.9 and 3.0. http://wiki.wxwidgets.org/Development:_wxMac#Building_under_10.6_Snow_Leopard I'll try the WX_2_9_0_BRANCH branch and/or trunk. > But the problem remains, for me the emulator crashes with a SIGTRAP when > loading the driver. > > /Dan > > Tuncer Ayaz wrote: >> >> On Wed, Sep 2, 2009 at 5:10 PM, Steve Vinoski wrote: >>> >>> I'm seeing that the wx part of the Erlang/OTP build doesn't work on OS X >>> 10.6 (Snow Leopard). I don't use wx so it's no big deal to me, but I'm >>> seeing the g++ compiler appear to hang when compiling >>> lib/wx/c_src/gen/wxe_funcs.cpp. I let it run quite awhile and it never >>> got >>> past that point, but my laptop fans were running full blast so I wonder >>> if >>> the compilation got stuck in an infinite loop. If I manually edit wx out >>> of >>> the lib/Makefile, then everything builds and seems to work as normal. >>> There >>> are also lots of compilation warnings in the wx area due to deprecation >>> of >>> some APIs being used and due to string constants being passed as char*. >> >> The build continues once that one long compile step has finished. >> What was "quite a while"? 10 minutes? 30 minutes? >> >> Can you try to build again and wait a little longer? >> >> This does not mean that the wx driver now automatically works on OSX: >> ?"WX Failed loading "wxe_driver"@[...]". >> Also, by default on OSX.6 it's -m64, so: >> ?wxe_driver.so: Mach-O 64-bit bundle x86_64.). >> >> The wx driver not easily working is an issue specific to Darwin, >> there are no problems like that on linux. >> >> For the record, Snow Leopard seems to bundle wx-2.8.8 and the >> on 10.5 it also failed with 2.8.10. That's why I didn't try to use >> a 2.8.10 install of wxMac for the driver, yet. >> >> ________________________________________________________________ >> erlang-bugs mailing list. See http://www.erlang.org/faq.html >> erlang-bugs (at) erlang.org >> >> > From kagato@REDACTED Mon Sep 7 11:55:47 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Mon, 7 Sep 2009 02:55:47 -0700 Subject: [erlang-questions] Re: [erlang-bugs] Re: [erlang-questions] wx fails to build on Snow Leopard In-Reply-To: <4ac8254d0909070224p11db54f6idaad05b6fb6ad0ab@mail.gmail.com> References: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> <4ac8254d0909040630o43cc6beo3988d9e2fc24d3bb@mail.gmail.com> <4AA4A172.1040809@erix.ericsson.se> <4ac8254d0909070224p11db54f6idaad05b6fb6ad0ab@mail.gmail.com> Message-ID: <6BB01A6C-5AD2-45D9-8A5A-EEF8612F77F3@souja.net> >>> On Wed, Sep 2, 2009 at 5:10 PM, Steve Vinoski >>> wrote: >>> What was "quite a while"? 10 minutes? 30 minutes? A small aside. You should really leave compilations like this for an hour before giving up (and watch the memory usage in the meantime). If it's taking up CPU, it's probably just pathological compilation behavior. In my halcyon days at university, bootstrapping GCC on the default AIX compiler on one of the departmental machines would snag on a certain file. I eventually discovered that it would take 45 minutes to compile. The second time around (when it rebuilds the compiler with itself) would still take 6 minutes. I killed that build a dozen times before I figured out that it was really working. -- Jayson Vantuyl kagato@REDACTED From vinoski@REDACTED Mon Sep 7 15:54:37 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Mon, 7 Sep 2009 09:54:37 -0400 Subject: [erlang-questions] Re: [erlang-bugs] Re: [erlang-questions] wx fails to build on Snow Leopard In-Reply-To: <6BB01A6C-5AD2-45D9-8A5A-EEF8612F77F3@souja.net> References: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> <4ac8254d0909040630o43cc6beo3988d9e2fc24d3bb@mail.gmail.com> <4AA4A172.1040809@erix.ericsson.se> <4ac8254d0909070224p11db54f6idaad05b6fb6ad0ab@mail.gmail.com> <6BB01A6C-5AD2-45D9-8A5A-EEF8612F77F3@souja.net> Message-ID: <65b2728e0909070654x59379508m7735c2c8a1d7bc90@mail.gmail.com> On Mon, Sep 7, 2009 at 5:55 AM, Jayson Vantuyl wrote: > On Wed, Sep 2, 2009 at 5:10 PM, Steve Vinoski wrote: >>>> What was "quite a while"? 10 minutes? 30 minutes? >>>> >>> A small aside. You should really leave compilations like this for an > hour before giving up (and watch the memory usage in the meantime). If it's > taking up CPU, it's probably just pathological compilation behavior. > No need to lecture. Anytime I hear my laptop fans kick into high gear and run that way for more than 5 minutes, I hunt down the source of the issue and make it stop. Let it run for an hour? No thanks. I have no interest in potentially overheating my laptop and unnecessarily shortening its lifespan trying to compile something like this. The only reason I posted the information was in case others might find it helpful, plus I thought the wx authors might want to track down the cause of this compilation behavior and fix it in their source code. --steve From masklinn@REDACTED Mon Sep 7 16:05:22 2009 From: masklinn@REDACTED (Masklinn) Date: Mon, 7 Sep 2009 16:05:22 +0200 Subject: [erlang-questions] Re: [erlang-bugs] Re: [erlang-questions] wx fails to build on Snow Leopard In-Reply-To: <65b2728e0909070654x59379508m7735c2c8a1d7bc90@mail.gmail.com> References: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> <4ac8254d0909040630o43cc6beo3988d9e2fc24d3bb@mail.gmail.com> <4AA4A172.1040809@erix.ericsson.se> <4ac8254d0909070224p11db54f6idaad05b6fb6ad0ab@mail.gmail.com> <6BB01A6C-5AD2-45D9-8A5A-EEF8612F77F3@souja.net> <65b2728e0909070654x59379508m7735c2c8a1d7bc90@mail.gmail.com> Message-ID: On 7 Sep 2009, at 15:54 , Steve Vinoski wrote: On Mon, Sep 7, 2009 at 5:55 AM, Jayson Vantuyl wrote: > >> On Wed, Sep 2, 2009 at 5:10 PM, Steve Vinoski >> wrote: >>>>> What was "quite a while"? 10 minutes? 30 minutes? >>>>> >>>> A small aside. You should really leave compilations like this >>>> for an >> hour before giving up (and watch the memory usage in the >> meantime). If it's >> taking up CPU, it's probably just pathological compilation behavior. > No need to lecture. Anytime I hear my laptop fans kick into high > gear and > run that way for more than 5 minutes, I hunt down the source of the > issue > and make it stop. Let it run for an hour? No thanks. I have no > interest in > potentially overheating my laptop and unnecessarily shortening its > lifespan > trying to compile something like this. Any modern CPU (especially in laptops) has built-in temperature management and overheating protection, unless your laptop has pretty severe hardware defects there's no reason for it to overheat (let alone get a shortened lifespan out of that). In any correctly built laptop, the temperatures reached by the components will be well within the hardware's tolerances (though not your thighs's). And re erlang compilation, though I have no doubt it's malfunctioning under SL, it used to take a good half hour to hour under leopard on a 2GHz C2D macbook. From magnus@REDACTED Mon Sep 7 16:11:31 2009 From: magnus@REDACTED (Magnus Henoch) Date: Mon, 07 Sep 2009 15:11:31 +0100 Subject: ssl:listen/2, ssl:port/1: docs vs reality Message-ID: <84ljkqrgj0.fsf@linux-b2a3.site> Hi, In R13B01, the documentation for ssl:listen/2 says: > Sets up a socket to listen on port Port at the local host. If Port is > zero, listen/2 picks an available port number (use port/1 to retreive > it). However, ssl:port/1 was removed in R13, so the documentation should probably be updated. Also, what is the new recommended way to get the port number from an SSL listening socket? Magnus From colm.dougan@REDACTED Mon Sep 7 16:41:01 2009 From: colm.dougan@REDACTED (Colm Dougan) Date: Mon, 7 Sep 2009 15:41:01 +0100 Subject: [erlang-questions] Munging ets continuations In-Reply-To: <4AA4C657.9090107@erlang-consulting.com> References: <24d4f39c0909040721j6c6a2f92j97881b1aa5bff7cc@mail.gmail.com> <4AA4C657.9090107@erlang-consulting.com> Message-ID: <24d4f39c0909070741i211f8b06xb15bbaee2e43003c@mail.gmail.com> On Mon, Sep 7, 2009 at 9:37 AM, Ulf Wiger wrote: > Colm Dougan wrote: >> My use-case is that I have an ordered_set table with (say) 1000 entries >> and I want to look at a given offset within them, e.g. >> entries 700 to 720 without having to pull back the whole list or walk >> along it in chunks until I reach 700. > > Presumably, you don't know the value of the key of entry 700, > or you would simply pass that to ets:next/2 and iterate from there. > > It seems to me that ets:slot/2 is what you need. > From the manual: > > "If the table is of type ordered_set, the function returns a list containing > the I:th object in Erlang term order." > > (It starts with slot 0). Thanks Ulf. ets:slot does exactly what I need. Thanks, Colm From dmitriid@REDACTED Mon Sep 7 16:56:04 2009 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Mon, 7 Sep 2009 17:56:04 +0300 Subject: [erlang-questions] Reanimation of elryvideo In-Reply-To: <97619b170909061334u115bf497u4d22ed8488ac631c@mail.gmail.com> References: <97619b170909061334u115bf497u4d22ed8488ac631c@mail.gmail.com> Message-ID: <69350BC6-AD14-42D4-B5D8-0D09988FCB36@gmail.com> Thank you! I'll relay this to the author. On Sep 6, 2009, at 11:34 PM, Rapsey wrote: > The problem with playing mp4 files is quite possibly a lack of AVC > sequence > header (check the official flash file format spec). The AVC sequence > header > needs to be sent to the client before any h264 video frame, > otherwise flash > will not display the video. > This is something that needs to be generated from the encoded video > frames, > because it is not present in the mp4 file format. > Look at the ffmpeg source code to see how to generate the avc sequence > header. It's in libavformat/flvenc.c a call to > ff_isom_write_avcc(...) does > and it is defined in libavformat/avc.c > > > Sergej > > On Sun, Sep 6, 2009 at 9:47 PM, Dmitrii Dimandt > wrote: > >> A member of the Erlang Russian mailing list ( >> erlang-russian@REDACTED) has decided to continue working on >> erlyvideo (http://code.google.com/p/erlyvideo/) >> >> The new repo is located here: http://github.com/maxlapshin/erlyvideo >> >> The new version can now send notifications to flash clients >> (something like >> a push channel) and can read (but not yet play, possibly due to a >> faulty >> handshake) mp4 files. >> >> Any help would be greatly appreciated From vinoski@REDACTED Mon Sep 7 17:04:33 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Mon, 7 Sep 2009 11:04:33 -0400 Subject: [erlang-questions] Re: [erlang-bugs] Re: [erlang-questions] wx fails to build on Snow Leopard In-Reply-To: References: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> <4ac8254d0909040630o43cc6beo3988d9e2fc24d3bb@mail.gmail.com> <4AA4A172.1040809@erix.ericsson.se> <4ac8254d0909070224p11db54f6idaad05b6fb6ad0ab@mail.gmail.com> <6BB01A6C-5AD2-45D9-8A5A-EEF8612F77F3@souja.net> <65b2728e0909070654x59379508m7735c2c8a1d7bc90@mail.gmail.com> Message-ID: <65b2728e0909070804w12897ff6t624ee16c32c777fd@mail.gmail.com> On Mon, Sep 7, 2009 at 10:05 AM, Masklinn wrote: > On 7 Sep 2009, at 15:54 , Steve Vinoski wrote: > On Mon, Sep 7, 2009 at 5:55 AM, Jayson Vantuyl wrote: > >> >> On Wed, Sep 2, 2009 at 5:10 PM, Steve Vinoski wrote: >>> >>>> What was "quite a while"? 10 minutes? 30 minutes? >>>>>> >>>>>> A small aside. You should really leave compilations like this for an >>>>> >>>> hour before giving up (and watch the memory usage in the meantime). If >>> it's >>> taking up CPU, it's probably just pathological compilation behavior. >>> >> No need to lecture. Anytime I hear my laptop fans kick into high gear and >> run that way for more than 5 minutes, I hunt down the source of the issue >> and make it stop. Let it run for an hour? No thanks. I have no interest in >> potentially overheating my laptop and unnecessarily shortening its >> lifespan >> trying to compile something like this. >> > > Any modern CPU (especially in laptops) has built-in temperature management > and overheating protection, unless your laptop has pretty severe hardware > defects there's no reason for it to overheat (let alone get a shortened > lifespan out of that). In any correctly built laptop, the temperatures > reached by the components will be well within the hardware's tolerances > (though not your thighs's). > Tell that to my old PowerBook, which recently gave up the ghost after a couple months of my son constantly watching video game videos from YouTube, causing the fans to be running on high all the time and the body to constantly be pretty hot. Like I said, no need to lecture. I used to do chip testing. We'd literally bake them, we'd freeze them with liquid nitrogen, we'd run them for weeks under extreme temperatures. I'm well aware of what they can handle, but I'm also well aware of what even non-extreme temperatures can mean for device lifespan. So thanks but I'll stand by my concerns. And re erlang compilation, though I have no doubt it's malfunctioning under > SL, it used to take a good half hour to hour under leopard on a 2GHz C2D > macbook. The entire compilation of Erlang/OTP took that long, yes, but here we're talking about a single file taking that long to compile -- quite a difference. --steve From info@REDACTED Mon Sep 7 17:28:19 2009 From: info@REDACTED (info) Date: Mon, 7 Sep 2009 17:28:19 +0200 Subject: crone.erl: tasks never executed Message-ID: <200909071728194536455@its3.ch> Hi all, Who has already used the crone.erl application ? I compiled it under erlide. The tasks are never executed ! Run with the command : crone_test:start(). From baryluk@REDACTED Mon Sep 7 17:49:29 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Mon, 07 Sep 2009 17:49:29 +0200 Subject: [erlang-questions] crone.erl: tasks never executed In-Reply-To: <200909071728194536455@its3.ch> References: <200909071728194536455@its3.ch> Message-ID: <1252338569.5961.196.camel@sredniczarny> Dnia 2009-09-07, pon o godzinie 17:28 +0200, info pisze: > Hi all, > Who has already used the crone.erl application ? > I compiled it under erlide. The tasks are never executed ! > Run with the command : crone_test:start(). You just need to wait long enough... Read crone_test.erl to see when jobs are started :) You can also uncomment second line in loop_task/1 to get some debugging. -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From info@REDACTED Mon Sep 7 18:20:24 2009 From: info@REDACTED (info) Date: Mon, 7 Sep 2009 18:20:24 +0200 Subject: [erlang-questions] crone.erl: tasks never executed References: <200909071728194536455@its3.ch>, <1252338569.5961.196.camel@sredniczarny> Message-ID: <200909071820244778311@its3.ch> I changed jobs in order to see their starting but ... nothing ! Dnia 2009-09-07, pon o godzinie 17:28 +0200, info pisze: > Hi all, > Who has already used the crone.erl application ? > I compiled it under erlide. The tasks are never executed ! > Run with the command : crone_test:start(). You just need to wait long enough... Read crone_test.erl to see when jobs are started :) You can also uncomment second line in loop_task/1 to get some debugging. -- Witold Baryluk From baryluk@REDACTED Mon Sep 7 18:32:34 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Mon, 07 Sep 2009 18:32:34 +0200 Subject: [erlang-questions] crone.erl: tasks never executed In-Reply-To: <200909071820244778311@its3.ch> References: <200909071728194536455@its3.ch> , <1252338569.5961.196.camel@sredniczarny> <200909071820244778311@its3.ch> Message-ID: <1252341154.5961.249.camel@sredniczarny> Dnia 2009-09-07, pon o godzinie 18:20 +0200, info pisze: > I changed jobs in order to see their starting but ... nothing ! so... i will say... nothing. Maybe you should read my previous post again or provide some information, something more than... nothing. ;) -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From paul-trapexit@REDACTED Mon Sep 7 18:48:38 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Mon, 7 Sep 2009 09:48:38 -0700 (PDT) Subject: [erlang-questions] sinan/faxien otp live upgrade In-Reply-To: References: Message-ID: We wrote something called erlrc to help with this. We run Ubuntu so we integrated it with aptitude. In theory, it could be integrated with sinan/faxien, but I haven't done that work. http://code.google.com/p/erlrc/ -- p On Mon, 7 Sep 2009, mliu wrote: > Hi everyone, > > New to the list...wondering if sinan/faxien users know whether it can > handle hot upgrades of applications/release? > > I'm starting a new otp project and am experimenting with sinan/faxien > for the dev/build/test cycle, and wondering if others are using it for > deployment and release upgrade maintenance. I've been able to > successfully publish my otp app to a local repository, and then > install the release using faxien. > > What I'm wondering is, how does faxien deal with a live system? Does > it take into appup/relup files in the release? Can it automatically > issue the required release_handler calls? If not, is there something > out there that will help with the live release upgrade process? > > Thanks! > Mike > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From info@REDACTED Mon Sep 7 18:58:00 2009 From: info@REDACTED (info) Date: Mon, 7 Sep 2009 18:58:00 +0200 Subject: [erlang-questions] crone.erl: tasks never executed References: <200909071728194536455@its3.ch>, <1252338569.5961.196.camel@sredniczarny>, <200909071820244778311@its3.ch>, <1252341154.5961.249.camel@sredniczarny> Message-ID: <200909071857598620292@its3.ch> Really NOTHING: no "sleeping for ..." messages. Dnia 2009-09-07, pon o godzinie 18:20 +0200, info pisze: > I changed jobs in order to see their starting but ... nothing ! so... i will say... nothing. Maybe you should read my previous post again or provide some information, something more than... nothing. ;) -- Witold Baryluk From rob@REDACTED Mon Sep 7 19:11:12 2009 From: rob@REDACTED (Rob Charlton) Date: Mon, 07 Sep 2009 18:11:12 +0100 Subject: [erlang-questions] crone.erl: tasks never executed In-Reply-To: <200909071857598620292@its3.ch> References: <200909071728194536455@its3.ch>, <1252338569.5961.196.camel@sredniczarny>, <200909071820244778311@its3.ch>, <1252341154.5961.249.camel@sredniczarny> <200909071857598620292@its3.ch> Message-ID: <4AA53EB0.2070301@erlang-consulting.com> info wrote: > Really NOTHING: no "sleeping for ..." messages. > > Dnia 2009-09-07, pon o godzinie 18:20 +0200, info pisze: > >> I changed jobs in order to see their starting but ... nothing ! >> Try going back to a copy of the source before you edited it. I just pulled down and compiled crone.erl and crone_test.erl, uncommented the io:fwrite in loop_task and got the following output: 3> crone_test:start(). {{daily,{every,{23,sec},{between,{3,pm},{3,30,pm}}}}, {io,fwrite,["Hello, world!~n"]}}: sleeping for 20:53:03 {{daily,{3,30,pm}},{io,fwrite,["It's three thirty~n"]}}: sleeping for 21:23:03 {{daily,[{1,10,am},{1,7,30,am}]},{io,fwrite,["Bing~n"]}}: sleeping for 7:00:33 [<0.50.0>,<0.49.0>,<0.48.0>,<0.47.0>,<0.46.0>,<0.45.0>, <0.44.0>,<0.43.0>] {{weekly,thu,{2,am}},{io,fwrite,["It's 2 Thursday morning~n"]}}: sleeping for 79:53:03 {{weekly,wed,{2,am}},{io,fwrite,["It's 2 Wednesday morning~n"]}}: sleeping for 55:53:03 {{weekly,fri,{2,am}},{io,fwrite,["It's 2 Friday morning~n"]}}: sleeping for 103:53:03 {{monthly,1,{2,am}},{io,fwrite,["First of the month!~n"]}}: sleeping for 583:53:03 {{monthly,4,{2,am}},{io,fwrite,["Fourth of the month!~n"]}}: sleeping for 655:53:03 -- Erlang Training and Consulting Ltd www.erlang-consulting.com From info@REDACTED Mon Sep 7 19:20:43 2009 From: info@REDACTED (info) Date: Mon, 7 Sep 2009 19:20:43 +0200 Subject: [erlang-questions] crone.erl: tasks never executed References: <200909071728194536455@its3.ch>, <1252338569.5961.196.camel@sredniczarny>, <200909071820244778311@its3.ch>, <1252341154.5961.249.camel@sredniczarny>, <200909071857598620292@its3.ch>, <4AA53EB0.2070301@erlang-consulting.com> Message-ID: <200909071920425097558@its3.ch> Try with erlide/eclipse ... info wrote: > Really NOTHING: no "sleeping for ..." messages. > > Dnia 2009-09-07, pon o godzinie 18:20 +0200, info pisze: > > > I changed jobs in order to see their starting but ... nothing ! > > Try going back to a copy of the source before you edited it. I just pulled down and compiled crone.erl and crone_test.erl, uncommented the io:fwrite in loop_task and got the following output: 3 > crone_test:start(). {{daily,{every,{23,sec},{between,{3,pm},{3,30,pm}}}}, {io,fwrite,["Hello, world!~n"]}}: sleeping for 20:53:03 {{daily,{3,30,pm}},{io,fwrite,["It's three thirty~n"]}}: sleeping for 21:23:03 {{daily,[{1,10,am},{1,7,30,am}]},{io,fwrite,["Bing~n"]}}: sleeping for 7:00:33 [ <0.50.0 >, <0.49.0 >, <0.48.0 >, <0.47.0 >, <0.46.0 >, <0.45.0 >, <0.44.0 >, <0.43.0 >] {{weekly,thu,{2,am}},{io,fwrite,["It's 2 Thursday morning~n"]}}: sleeping for 79:53:03 {{weekly,wed,{2,am}},{io,fwrite,["It's 2 Wednesday morning~n"]}}: sleeping for 55:53:03 {{weekly,fri,{2,am}},{io,fwrite,["It's 2 Friday morning~n"]}}: sleeping for 103:53:03 {{monthly,1,{2,am}},{io,fwrite,["First of the month!~n"]}}: sleeping for 583:53:03 {{monthly,4,{2,am}},{io,fwrite,["Fourth of the month!~n"]}}: sleeping for 655:53:03 -- Erlang Training and Consulting Ltd www.erlang-consulting.com ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From vladdu55@REDACTED Mon Sep 7 19:50:23 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 7 Sep 2009 19:50:23 +0200 Subject: [erlang-questions] crone.erl: tasks never executed In-Reply-To: <200909071920425097558@its3.ch> References: <200909071728194536455@its3.ch> <1252338569.5961.196.camel@sredniczarny> <200909071820244778311@its3.ch> <1252341154.5961.249.camel@sredniczarny> <200909071857598620292@its3.ch> <4AA53EB0.2070301@erlang-consulting.com> <200909071920425097558@its3.ch> Message-ID: <95be1d3b0909071050y3f259bb9q7f93debc541ca6c1@mail.gmail.com> On Mon, Sep 7, 2009 at 19:20, info wrote: > Try with erlide/eclipse ... > info wrote: >> Really NOTHING: no "sleeping for ..." messages. > Try going back to a copy of the source before you edited it. I just > pulled down and compiled crone.erl and crone_test.erl, uncommented the > io:fwrite in loop_task and got the following output: > > 3 > crone_test:start(). > {{daily,{every,{23,sec},{between,{3,pm},{3,30,pm}}}}, Hi, The console in erlide is not yet ready for prime time, there are things that it doesn't display. If you're using the latest nightly builds, it should display everything unless there's some bug, which might happen. The console is in the process of redesign, until it's ready one can start an erlang node manually and configure erlide to use it when executing the code (by using the node's name and cookie in the run configuration dialog). best regards, Vlad From tuncer.ayaz@REDACTED Mon Sep 7 20:01:46 2009 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 7 Sep 2009 20:01:46 +0200 Subject: [erlang-bugs] Re: [erlang-questions] wx fails to build on Snow Leopard In-Reply-To: <4ac8254d0909070224p11db54f6idaad05b6fb6ad0ab@mail.gmail.com> References: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> <4ac8254d0909040630o43cc6beo3988d9e2fc24d3bb@mail.gmail.com> <4AA4A172.1040809@erix.ericsson.se> <4ac8254d0909070224p11db54f6idaad05b6fb6ad0ab@mail.gmail.com> Message-ID: <4ac8254d0909071101g6e4210a7g16dbeb1ef7d09107@mail.gmail.com> On Mon, Sep 7, 2009 at 11:24 AM, Tuncer Ayaz wrote: > On Mon, Sep 7, 2009 at 8:00 AM, Dan Gudmundsson wrote: >> I don't think wxWidgets 2.8.* works in 64-bit, it uses the old gui api. >> They are working on fixing that in wxWidgets 2.9 and 3.0. > > http://wiki.wxwidgets.org/Development:_wxMac#Building_under_10.6_Snow_Leopard > > I'll try the WX_2_9_0_BRANCH branch and/or trunk. fyi, 2.9 branch didn't build completely. trunk built but it looks like gen/wxe_funcs.cpp would need to be adapted to changes in wx trunk. >> But the problem remains, for me the emulator crashes with a SIGTRAP when >> loading the driver. >> >> /Dan >> >> Tuncer Ayaz wrote: >>> >>> On Wed, Sep 2, 2009 at 5:10 PM, Steve Vinoski wrote: >>>> >>>> I'm seeing that the wx part of the Erlang/OTP build doesn't work on OS X >>>> 10.6 (Snow Leopard). I don't use wx so it's no big deal to me, but I'm >>>> seeing the g++ compiler appear to hang when compiling >>>> lib/wx/c_src/gen/wxe_funcs.cpp. I let it run quite awhile and it never >>>> got >>>> past that point, but my laptop fans were running full blast so I wonder >>>> if >>>> the compilation got stuck in an infinite loop. If I manually edit wx out >>>> of >>>> the lib/Makefile, then everything builds and seems to work as normal. >>>> There >>>> are also lots of compilation warnings in the wx area due to deprecation >>>> of >>>> some APIs being used and due to string constants being passed as char*. >>> >>> The build continues once that one long compile step has finished. >>> What was "quite a while"? 10 minutes? 30 minutes? >>> >>> Can you try to build again and wait a little longer? >>> >>> This does not mean that the wx driver now automatically works on OSX: >>> ?"WX Failed loading "wxe_driver"@[...]". >>> Also, by default on OSX.6 it's -m64, so: >>> ?wxe_driver.so: Mach-O 64-bit bundle x86_64.). >>> >>> The wx driver not easily working is an issue specific to Darwin, >>> there are no problems like that on linux. >>> >>> For the record, Snow Leopard seems to bundle wx-2.8.8 and the >>> on 10.5 it also failed with 2.8.10. That's why I didn't try to use >>> a 2.8.10 install of wxMac for the driver, yet. >>> >>> ________________________________________________________________ >>> erlang-bugs mailing list. See http://www.erlang.org/faq.html >>> erlang-bugs (at) erlang.org >>> >>> >> > From ngocdaothanh@REDACTED Tue Sep 8 04:53:47 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Tue, 8 Sep 2009 11:53:47 +0900 Subject: Log level of error_logger Message-ID: <5c493e530909071953w5a475b8cl6d39a25e9ccf1c25@mail.gmail.com> Hi, As far as I know error_logger is the only "standard" log library of Erlang/OTP. Is there a way to specify log level (debug, info, warn, error, fatal) and log destination (console, file) of error_logger? I'm using log4erl which has these features. I'm curious if error_logger has these. Thanks. From dgud@REDACTED Tue Sep 8 08:50:16 2009 From: dgud@REDACTED (Dan Gudmundsson) Date: Tue, 08 Sep 2009 08:50:16 +0200 Subject: [erlang-questions] Re: [erlang-bugs] Re: [erlang-questions] wx fails to build on Snow Leopard In-Reply-To: <65b2728e0909070804w12897ff6t624ee16c32c777fd@mail.gmail.com> References: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> <4ac8254d0909040630o43cc6beo3988d9e2fc24d3bb@mail.gmail.com> <4AA4A172.1040809@erix.ericsson.se> <4ac8254d0909070224p11db54f6idaad05b6fb6ad0ab@mail.gmail.com> <6BB01A6C-5AD2-45D9-8A5A-EEF8612F77F3@souja.net> <65b2728e0909070654x59379508m7735c2c8a1d7bc90@mail.gmail.com> <65b2728e0909070804w12897ff6t624ee16c32c777fd@mail.gmail.com> Message-ID: <4AA5FEA8.4030905@erix.ericsson.se> It depends on gcc version (and hardware). I who compile it daily, though not on the mac, have no problem with it, so I havn't split up the file to make the code more complex and harder to debug. It takes 6.5 minutes to compile on my old 1.83 Ghz mac mini, with 1GB memory. /Dan Steve Vinoski wrote: > On Mon, Sep 7, 2009 at 10:05 AM, Masklinn wrote: > >> On 7 Sep 2009, at 15:54 , Steve Vinoski wrote: >> On Mon, Sep 7, 2009 at 5:55 AM, Jayson Vantuyl wrote: >> >>> On Wed, Sep 2, 2009 at 5:10 PM, Steve Vinoski wrote: >>>>> What was "quite a while"? 10 minutes? 30 minutes? >>>>>>> A small aside. You should really leave compilations like this for an >>>>> hour before giving up (and watch the memory usage in the meantime). If >>>> it's >>>> taking up CPU, it's probably just pathological compilation behavior. >>>> >>> No need to lecture. Anytime I hear my laptop fans kick into high gear and >>> run that way for more than 5 minutes, I hunt down the source of the issue >>> and make it stop. Let it run for an hour? No thanks. I have no interest in >>> potentially overheating my laptop and unnecessarily shortening its >>> lifespan >>> trying to compile something like this. >>> >> Any modern CPU (especially in laptops) has built-in temperature management >> and overheating protection, unless your laptop has pretty severe hardware >> defects there's no reason for it to overheat (let alone get a shortened >> lifespan out of that). In any correctly built laptop, the temperatures >> reached by the components will be well within the hardware's tolerances >> (though not your thighs's). >> > > Tell that to my old PowerBook, which recently gave up the ghost after a > couple months of my son constantly watching video game videos from YouTube, > causing the fans to be running on high all the time and the body to > constantly be pretty hot. > > Like I said, no need to lecture. I used to do chip testing. We'd literally > bake them, we'd freeze them with liquid nitrogen, we'd run them for weeks > under extreme temperatures. I'm well aware of what they can handle, but I'm > also well aware of what even non-extreme temperatures can mean for device > lifespan. So thanks but I'll stand by my concerns. > > And re erlang compilation, though I have no doubt it's malfunctioning under >> SL, it used to take a good half hour to hour under leopard on a 2GHz C2D >> macbook. > > > The entire compilation of Erlang/OTP took that long, yes, but here we're > talking about a single file taking that long to compile -- quite a > difference. > > --steve > From dgud@REDACTED Tue Sep 8 08:52:04 2009 From: dgud@REDACTED (Dan Gudmundsson) Date: Tue, 08 Sep 2009 08:52:04 +0200 Subject: [erlang-bugs] Re: [erlang-questions] wx fails to build on Snow Leopard In-Reply-To: <4ac8254d0909071101g6e4210a7g16dbeb1ef7d09107@mail.gmail.com> References: <65b2728e0909020810j19107120r40db7344b7fbc278@mail.gmail.com> <4ac8254d0909040630o43cc6beo3988d9e2fc24d3bb@mail.gmail.com> <4AA4A172.1040809@erix.ericsson.se> <4ac8254d0909070224p11db54f6idaad05b6fb6ad0ab@mail.gmail.com> <4ac8254d0909071101g6e4210a7g16dbeb1ef7d09107@mail.gmail.com> Message-ID: <4AA5FF14.2050406@erix.ericsson.se> Tuncer Ayaz wrote: > On Mon, Sep 7, 2009 at 11:24 AM, Tuncer Ayaz wrote: >> On Mon, Sep 7, 2009 at 8:00 AM, Dan Gudmundsson wrote: >>> I don't think wxWidgets 2.8.* works in 64-bit, it uses the old gui api. >>> They are working on fixing that in wxWidgets 2.9 and 3.0. >> http://wiki.wxwidgets.org/Development:_wxMac#Building_under_10.6_Snow_Leopard >> >> I'll try the WX_2_9_0_BRANCH branch and/or trunk. > > fyi, > > 2.9 branch didn't build completely. trunk built but it looks like > gen/wxe_funcs.cpp would need to be adapted to changes in > wx trunk. > I'm not surprised, I havn't had time to look at the dev version yet. /Dan From rtrlists@REDACTED Tue Sep 8 10:35:58 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 8 Sep 2009 09:35:58 +0100 Subject: [erlang-questions] Log level of error_logger In-Reply-To: <5c493e530909071953w5a475b8cl6d39a25e9ccf1c25@mail.gmail.com> References: <5c493e530909071953w5a475b8cl6d39a25e9ccf1c25@mail.gmail.com> Message-ID: <6a3ae47e0909080135u1a0a63daq81bcc0d43578e901@mail.gmail.com> On Tue, Sep 8, 2009 at 3:53 AM, Ngoc Dao wrote: > As far as I know error_logger is the only "standard" log library of > Erlang/OTP. Is there a way to specify log level (debug, info, warn, > error, fatal) and log destination (console, file) of error_logger? > > I'm using log4erl which has these features. I'm curious if > error_logger has these. > In the standard distro there are two error loggers available. One plain text one that you can configure using some sys.config file settings (see the kernel doc page, http://www.erlang.org/doc/man/kernel_app.html); but it justs bangs everything into the one big log. Then there's the error logger that gets installed when you start SASL (see the sasl doc page, http://www.erlang.org/doc/man/sasl_app.html). That one allows you to specify a rolling log, for example. And the report browser (rb) allows you to view log entries at specific levels. I am using the sasl error logging and am reasonably happy with it at the moment. But I do get the slight impression that these error loggers are there as a base to build upon. There's also a module disk_log, that looks like it would be a good fit for implementing your own more tailored file logging. Logging comes up once in a while in the mailing list, so a search through the archives can probably be very enlightening. Robby From rdoering@REDACTED Tue Sep 8 12:21:31 2009 From: rdoering@REDACTED (Ralf Doering) Date: Tue, 08 Sep 2009 12:21:31 +0200 Subject: xmerl and selecting attributes Message-ID: <87ab157n4k.fsf@netsys-it.de> Hello, seems I've got a problem selecting the value of an attribute from an XML tag. Consider the following short example: --8<---------------cut here---------------start------------->8--- -module(xmerl_test). -include_lib("xmerl/include/xmerl.hrl"). -import(xmerl_xs, [ xslapply/2, value_of/1, select/2, built_in_rules/2 ]). -export([start/0]). start() -> {XmlElement, _Rest} = xmerl_scan:string(""), template(XmlElement). template( E = #xmlElement{name='test'}) -> Attr = value_of(select("@myattr", E)), io:format("MyAttribute: ~w~n", [Attr]), xslapply(fun template/1,E), E; template(E) -> built_in_rules( fun template/1, E). --8<---------------cut here---------------end--------------->8--- When running this, I get: (emacs@REDACTED)13> xmerl_test:start(). MyAttribute: [] [] I was expecting that value_of(select("@myattr"),E) would extract "123" as attribute value. Am I wrong here, do I use the wrong XPath for this? TIA, Ralf -- Ralf D?ring | NetSys.IT GbR Networking & Security Solutions | www.netsys-it.de rdoering@REDACTED | Tel.: +49 (3677) 20 35 15 From rtrlists@REDACTED Tue Sep 8 12:43:15 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 8 Sep 2009 11:43:15 +0100 Subject: [erlang-questions] xmerl and selecting attributes In-Reply-To: <87ab157n4k.fsf@netsys-it.de> References: <87ab157n4k.fsf@netsys-it.de> Message-ID: <6a3ae47e0909080343r5d996134y70667458e029f75f@mail.gmail.com> On Tue, Sep 8, 2009 at 11:21 AM, Ralf Doering wrote: > > Hello, > > seems I've got a problem selecting the value of an attribute from an XML > tag. > > Consider the following short example: > > --8<---------------cut here---------------start------------->8--- > -module(xmerl_test). > > -include_lib("xmerl/include/xmerl.hrl"). > > -import(xmerl_xs, > [ xslapply/2, value_of/1, select/2, built_in_rules/2 ]). > > > -export([start/0]). > > start() -> > {XmlElement, _Rest} = xmerl_scan:string(""), > template(XmlElement). > > > template( E = #xmlElement{name='test'}) -> > Attr = value_of(select("@myattr", E)), > io:format("MyAttribute: ~w~n", [Attr]), > xslapply(fun template/1,E), > E; > > template(E) -> > built_in_rules( fun template/1, E). > > --8<---------------cut here---------------end--------------->8--- > > > When running this, I get: > > (emacs@REDACTED)13> xmerl_test:start(). > MyAttribute: [] > [] > > I was expecting that value_of(select("@myattr"),E) would extract "123" as > attribute value. Am I wrong here, do I use the wrong XPath for this? > > I think the xmerl_xs:value_of/1 function will only ever return the text nodes of the element list you supply. The xmerl_xs:select/2 invocation you have is fine: Erlang (BEAM) emulator version 5.6.5 [smp:2] [async-threads:0] Eshell V5.6.5 (abort with ^G) 1> rr(xmerl). [xmerl_event,xmerl_fun_states,xmerl_scanner,xmlAttribute, xmlComment,xmlContext,xmlDecl,xmlDocument,xmlElement, xmlNamespace,xmlNode,xmlNsNode,xmlObj,xmlPI,xmlText] 2> {E,_} = xmerl_scan:string(""). {#xmlElement{ name = test,expanded_name = test,nsinfo = [], namespace = #xmlNamespace{default = [],nodes = []}, parents = [],pos = 1, attributes = [#xmlAttribute{ name = myattr,expanded_name = [],nsinfo = [],namespace = [], parents = [],pos = 1,language = [],value = "123", normalized = false}], content = [],language = [],xmlbase = "C:/Robby", elementdef = undeclared}, []} 3> xmerl_xs:select("@myattr", E). [#xmlAttribute{name = myattr,expanded_name = [],nsinfo = [], namespace = [],parents = [],pos = 1,language = [], value = "123",normalized = false}] So, I guess you have to get the value through the record itself. Robby From rtrlists@REDACTED Tue Sep 8 13:49:42 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 8 Sep 2009 12:49:42 +0100 Subject: Erlang JAM (!) files make OpenOffice not print on Tuesdays Message-ID: <6a3ae47e0909080449u7910003bt76058ea046e67c69@mail.gmail.com> A bit off topic, but I thought this was quite amusing. Just came across it in catching up with my comp.risks reading (http://catless.ncl.ac.uk/Risks/). Robby >Date: Sun, 16 Aug 2009 11:27:21 +1000 >From: phil colbourn >Subject: Cannot print on Tuesdays! Today I came across an interesting bug mentioned on a blog. The problem was that printing for some people failed occasionally. Later someone noted that his Wife had been complaining that she couldn't print on Tuesdays! In reading through the bug report people were initially claiming that it must be an OpenOffice bug since all other applications printed fine. Others noted that it comes and goes. One user found a solution: To remove and purge the system of OpenOffice and re-install (any easy task on Ubuntu). He reported on a Thursday that this fixed his printing problem. Two weeks later he reported (on a Tuesday) that his solution did not work after-all. Nearly 4 months later the Wife of a Ubuntu hacker complained that OpenOffice would not print on Tuesdays. I can imagine the scenario: Wife: Steve, the printer will not work on Tuesdays. Steve: That's the printer's day off - Of course it will not print on Tuesdays. Wife: No, I'm serious! I can not print from OpenOffice on Tuesdays. Steve: (Unbelieving..) Ok... Show me. Wife: I can't show you. Steve: (Rolling eyes..) Why? Wife: It's Wednesday! Steve: (Nods. He says slowly...) Right. The problem seemed to be tracked down to a program called 'file'. This *NIX utility uses patterns to detect file types. eg. if the file starts with '%!' followed by 'PS-Adobe-' then it is a PostScript file. It seems that OpenOffice writes the date to the postscript file. On Tuesdays it takes the form of %%CreationDate: (Tue MMM D hh:mm:...) An error in the pattern for an Erlang JAM file meant that 'Tue' in the PostScript file was being recognised as an Erlang JAM file and so, presumably, it was not being sent to the printer. The Erlang JAM file pattern is: 4 string Tue Jan 22 14:32:44 MET 1991 Erlang JAM file - version 4.2 It should have been 4 string Tue\ Jan\ 22\ 14:32:44\ MET\ 1991 Erlang JAM file - version 4.2 With the large number of files types that this program attempts to match (over 1600) it is not surprising that errors are made in the patterns, but also the order of matching could mean that false positives are common. In this case, an Erlang JAM file was matched before the PostScript match occurred. References: http://mdzlog.alcor.net/2009/08/15/bohrbugs-openoffice-org-wont-print-on-tuesdays/ Reported as this bug: https://bugs.edge.launchpad.net/ubuntu/+source/cupsys/+bug/255161 Later made a duplicate to this bug: https://bugs.edge.launchpad.net/ubuntu/+source/file/+bug/248619. http://www.blaxlandweather.com/ http://philatwarrimoo.blogspot.com From rdoering@REDACTED Tue Sep 8 14:37:39 2009 From: rdoering@REDACTED (Ralf Doering) Date: Tue, 08 Sep 2009 14:37:39 +0200 Subject: [erlang-questions] xmerl and selecting attributes In-Reply-To: <6a3ae47e0909080343r5d996134y70667458e029f75f@mail.gmail.com> (Robert Raschke's message of "Tue, 8 Sep 2009 11:43:15 +0100") References: <87ab157n4k.fsf@netsys-it.de> <6a3ae47e0909080343r5d996134y70667458e029f75f@mail.gmail.com> Message-ID: <87tyzd6298.fsf@netsys-it.de> Hello Robert, Robert Raschke writes: > On Tue, Sep 8, 2009 at 11:21 AM, Ralf Doering wrote: >> I was expecting that value_of(select("@myattr"),E) would extract "123" as >> attribute value. Am I wrong here, do I use the wrong XPath for this? >> >> > I think the xmerl_xs:value_of/1 function will only ever return the text > nodes of the element list you supply. [...] > So, I guess you have to get the value through the record itself. Thank you for your reply. I already guessed that, and with your hints it works perfectly now. Thx, Ralf -- Ralf D?ring NetSys.IT GbR www.netsys-it.de From info@REDACTED Tue Sep 8 16:07:24 2009 From: info@REDACTED (info) Date: Tue, 8 Sep 2009 16:07:24 +0200 Subject: erlang and google maps api Message-ID: <200909081607238660294@its3.ch> Hi, Is it possible to call the google maps api from an erlang application ? Api are written in javascript and don't exist in another language. I don't imagine to rewrite them ... From dizzyd@REDACTED Tue Sep 8 16:53:08 2009 From: dizzyd@REDACTED (Dave Smith) Date: Tue, 8 Sep 2009 08:53:08 -0600 Subject: Bug in inets HTTP Client w/ Chunked Encoding Message-ID: Greetings, I stumbled across what I believe to be a bug in the inets HTTP client, when used with HTTP chunked encoding and having the client save directly to a file. The distilled test case is attached. >From my reading of the client source code, it looks like there is a rather severe impedance mismatch between the httpc_handler and http_chunk modules. The http_chunk modules correctly decodes the chunks and calls back into httpc_manager:stream, but then fails to pass the modified HTTP request back out on completion. It's not clear to me exactly how one would fix this without a fair bit of rework. Of note, it's possible to work around the failure by having the library stream the chunks to the calling process. However, it's kinda of silly to have to do that when the HTTP client provides a feature for this very use case. Please let me know if I can do anything to further elucidate this bug. D. -------------- next part -------------- A non-text attachment was scrubbed... Name: httpc_bug.erl Type: application/octet-stream Size: 882 bytes Desc: not available URL: From dizzyd@REDACTED Tue Sep 8 16:56:11 2009 From: dizzyd@REDACTED (Dave Smith) Date: Tue, 8 Sep 2009 08:56:11 -0600 Subject: Bug in inets HTTP Client w/ Chunked Encoding In-Reply-To: References: Message-ID: Argh...I failed basic training of bug reporting apparently... O/S Platform: OS X 10.6 and 10.5 Erlang Version: R13B01 (64 and 32-bit versions) D. On Tue, Sep 8, 2009 at 8:53 AM, Dave Smith wrote: > Greetings, > > I stumbled across what I believe to be a bug in the inets HTTP client, > when used with HTTP chunked encoding and having the client save > directly to a file. The distilled test case is attached. > > From my reading of the client source code, it looks like there is a > rather severe impedance mismatch between the httpc_handler and > http_chunk modules. The http_chunk modules correctly decodes the > chunks and calls back into httpc_manager:stream, but then fails to > pass the modified HTTP request back out on completion. It's not clear > to me exactly how one would fix this without a fair bit of rework. > > Of note, it's possible to work around the failure by having the > library stream the chunks to the calling process. However, it's kinda > of silly to have to do that when the HTTP client provides a feature > for this very use case. > > Please let me know if I can do anything to further elucidate this bug. > > D. > From seancribbs@REDACTED Tue Sep 8 17:10:23 2009 From: seancribbs@REDACTED (Sean Cribbs) Date: Tue, 08 Sep 2009 11:10:23 -0400 Subject: [erlang-questions] erlang and google maps api In-Reply-To: <200909081607238660294@its3.ch> References: <200909081607238660294@its3.ch> Message-ID: <4AA673DF.9010904@gmail.com> info wrote: > Hi, > Is it possible to call the google maps api from an erlang application ? > Api are written in javascript and don't exist in another language. I don't imagine to rewrite them ... > > There are some pieces, for example the Geocoding API, that could be useful in Erlang. My initial Google and github searches don't come up with anything, however. Sean From info@REDACTED Tue Sep 8 18:48:25 2009 From: info@REDACTED (info) Date: Tue, 8 Sep 2009 18:48:25 +0200 Subject: [erlang-questions] erlang and google maps api References: <200909081607238660294@its3.ch>, <4AA673DF.9010904@gmail.com> Message-ID: <200909081848246743997@its3.ch> That means there is nothing in Erlang excepted erl-geocode.erl ? info wrote: > Hi, > Is it possible to call the google maps api from an erlang application ? > Api are written in javascript and don't exist in another language. I don't imagine to rewrite them ... > > There are some pieces, for example the Geocoding API, that could be useful in Erlang. My initial Google and github searches don't come up with anything, however. Sean ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From steven.charles.davis@REDACTED Wed Sep 9 01:55:57 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 8 Sep 2009 16:55:57 -0700 (PDT) Subject: erlang and google maps api In-Reply-To: <200909081607238660294@its3.ch> References: <200909081607238660294@its3.ch> Message-ID: <732e01ca-d18b-4c59-9ad5-9cca63effb9f@o41g2000yqb.googlegroups.com> Take a look at the Nitrogen web framework which has a google api implementation. The way Nitrogen approaches it may well be fairly easy to adapt to your particular needs. (the "api" it is after all just a very long query string on the url). /s On Sep 8, 9:07?am, "info" wrote: > Hi, > Is it possible to call the google maps api from an erlang application ? > Api are written in javascript and don't exist in another language. I don't imagine to rewrite them ... From kruegger@REDACTED Wed Sep 9 06:18:09 2009 From: kruegger@REDACTED (Stephen Han) Date: Tue, 8 Sep 2009 21:18:09 -0700 Subject: [erlang-questions] R12B erlang node restart after system clock change In-Reply-To: <4A94F25A.4080802@erlang-consulting.com> References: <86f1f5350908251843jf7d768fwed81c75c23bdd951@mail.gmail.com> <4A94F25A.4080802@erlang-consulting.com> Message-ID: <86f1f5350909082118t46e5dfafqe954fed0400821af@mail.gmail.com> Ulf Thanks for the information. I was able to resolve the issue right after see your email and I am getting the same behavior as R10B. I thought it was my mistake during the code upgrade. However, today I download the fresh R13B01 for my own fun and found out that same typo in the code again. Then I realized it was not really my mistake from the beginning. I am still using Linux 2.4 kernel so that is why I am getting this issue easily. In heart.c #if defined(CORRET_USING_TIMES) || defined(GETHRTIME_WITH_CLOCK_GETTIME) # define HEART_CORRECT_USING_TIMES 1 #endif The CORRET_USING_TIMES is a typo, I did not bother to look when it was introduced but it should be CORRECT_USING_TIMES, otherwise the system clock change will restart the node as default behavior in 2.4 system regards, On Wed, Aug 26, 2009 at 1:29 AM, Ulf Wiger wrote: > > The documentation for heart does say: > > "It should be noted that if the system clock is adjusted with more than > HEART_BEAT_TIMEOUT seconds, heart will timeout and try to reboot the system. > This can happen, for example, if the system clock is adjusted automatically > by use of NTP (Network Time Protocol)." > > (...even in R10B). > > However, the reason why you're not seeing this in R10B is, I think, > that heart.c has been re-written to use the system timestamp by > default, whereas it derived timestamps from system ticks in R10. > > One relevant difference in the code seems to be: > > /* > * Implement time correction using times() call even on Linuxes > * that can simulate gethrtime with clock_gettime, no use implementing > * a phony gethrtime in this file as the time questions are so infrequent. > */ > #if defined(CORRET_USING_TIMES) || defined(GETHRTIME_WITH_CLOCK_GETTIME) > # define HEART_CORRECT_USING_TIMES 1 > #endif > > > Timestamps are still simulated on WIN32 or if HAVE_GETHRTIME is not > defined, but HEART_CORRECT_USING_TIMES is. > > (Please verify for yourself by reading erts/etc/common/heart.c, > as this is not documented, from what I can tell, and you should > never draw conclusions based solely on my sloppy reading of C code). > > Perhaps using ticks whenever possible would be the best strategy > for heart.c, as it is hardly a feature that it goes bezerk if > someone dabbles with the system clock. It doesn't need hi-res > timestamps to begin with, as no one in their right mind would > set HEART_BEAT_TIMEOUT to something in the millisecond range > (I don't really recommend anything less than a minute, actually, > as heart is just a last resort, and /will/ interfere will > crash dump generation too, if given a chance). > > BR, > Ulf W > > > Stephen Han wrote: > >> Hi >> >> I am facing an issue where erlang node is restarted by "heart" whenever I >> change the system clock forward. It seems beam got KILL signal and the >> "heart" restarting the node. The node got restarted even I move forward >> the >> system clock for 1 minute. >> >> FYI, I am using OTP R12B-3. >> >> The problem is I am not even sure whether the node got restarted by our >> application or Erlang/OTP. >> However, this is also not reproducible in our old software which used to >> use >> R10B-8. >> >> Is there any changes have been made to post R10B where Erlang node should >> restart if the system clock move forward? >> >> Can you suggest any good method to debugging this kind of problem? >> >> regards, >> >> > > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > From raimo+erlang-questions@REDACTED Wed Sep 9 09:54:00 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 9 Sep 2009 09:54:00 +0200 Subject: Is the old mailing list archive permanently nuked? In-Reply-To: <20090909050456.GA4883@corelatus.se> References: <20090909050456.GA4883@corelatus.se> Message-ID: <20090909075400.GA25623@erix.ericsson.se> On Wed, Sep 09, 2009 at 07:04:56AM +0200, Matthias Lang wrote: > Hi, > > The mailing list archive used to be at URLs such as > > http://www.erlang.org/ml-archive/erlang-questions/200301/msg00103.html > > URLs like that are now all 404s. It seems like the archive is now > > http://erlang.org/pipermail/erlang-questions/2003-January/006683.html > > Was that intentional? Yes. That was the old Majordomo archive, I did not copy it along in the web server upgrade June 1 2009. All mail in that archive is in the (now also old) Mailman (Pipermail) archive you refer to. And also, all mail is in the new Ezmlm archive at e.g: http://www.erlang.org/cgi-bin/ezmlm-cgi/4/6682 I have problems getting Google to index the Ezmlm archive, so the Pipermail archive will be around for a while. > > (Someone has been nice enough to fix all the links in the FAQ for me. > I want to check that breaking the old archive URLs was deliberate > before I pull the change.) > > Matthias -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From rtrlists@REDACTED Wed Sep 9 14:29:42 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 9 Sep 2009 13:29:42 +0100 Subject: Finding nodes Message-ID: <6a3ae47e0909090529h7211ab1bm6613627b42d4343b@mail.gmail.com> Hi, I am thinking about how to get Erlang Nodes within a LAN to find each other, without having to have a known central server. Has anyone already done something along the lines of service discovery in Erlang? Like using something like DNS-SD to find other Erlang Nodes on your subnet, for example? Or multicast ping on known port? Or ...? I'd be interested in prior art, before plunging head first into a new territory. Googling didn't turn up anything, but so far I've only spent about 30 minutes on searching; more to follow. Thanks for any pointers, Robby From dizzyd@REDACTED Wed Sep 9 14:47:23 2009 From: dizzyd@REDACTED (Dave Smith) Date: Wed, 9 Sep 2009 06:47:23 -0600 Subject: [erlang-questions] Finding nodes In-Reply-To: <6a3ae47e0909090529h7211ab1bm6613627b42d4343b@mail.gmail.com> References: <6a3ae47e0909090529h7211ab1bm6613627b42d4343b@mail.gmail.com> Message-ID: Assuming you are OK with the inherent security implications of allowing ANY node to exchange messages with any other node, I'd just use a multicast ping packet (< 20 lines of Erlang) which would then allow me to see what hosts are running EPMD. From there, you can talk to EPMD and find all the nodes running on a particular host. DNS-SD would be really nice to have someday, but that's a fair bit of code to knock together last time I looked. EPMD gets you most of the way there, since all distributed erlang nodes start it by default -- you just need to identify those hosts. You can talk to EPMD via erl_epmd (undoc'd, in the kernel app) or (BLATANT PLUG) via the epmd_api library I wrote: http://github.com/dizzyd/epmd_api As noted previously however, there are SIGNIFICANT security implications to what you want to do -- are you sure you want to do this? :) D. On Wed, Sep 9, 2009 at 6:29 AM, Robert Raschke wrote: > Hi, > > I am thinking about how to get Erlang Nodes within a LAN to find each other, > without having to have a known central server. > > Has anyone already done something along the lines of service discovery in > Erlang? Like using something like DNS-SD to find other Erlang Nodes on your > subnet, for example? Or multicast ping on known port? Or ...? > > I'd be interested in prior art, before plunging head first into a new > territory. Googling didn't turn up anything, but so far I've only spent > about 30 minutes on searching; more to follow. > > Thanks for any pointers, > Robby > From paul-trapexit@REDACTED Wed Sep 9 15:33:11 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Wed, 9 Sep 2009 06:33:11 -0700 (PDT) Subject: [erlang-questions] Finding nodes In-Reply-To: <6a3ae47e0909090529h7211ab1bm6613627b42d4343b@mail.gmail.com> References: <6a3ae47e0909090529h7211ab1bm6613627b42d4343b@mail.gmail.com> Message-ID: http://code.google.com/p/nodefinder/ -- p On Wed, 9 Sep 2009, Robert Raschke wrote: > Hi, > > I am thinking about how to get Erlang Nodes within a LAN to find each other, > without having to have a known central server. > > Has anyone already done something along the lines of service discovery in > Erlang? Like using something like DNS-SD to find other Erlang Nodes on your > subnet, for example? Or multicast ping on known port? Or ...? > > I'd be interested in prior art, before plunging head first into a new > territory. Googling didn't turn up anything, but so far I've only spent > about 30 minutes on searching; more to follow. > > Thanks for any pointers, > Robby > From rtrlists@REDACTED Wed Sep 9 16:03:02 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 9 Sep 2009 15:03:02 +0100 Subject: [erlang-questions] Finding nodes In-Reply-To: References: <6a3ae47e0909090529h7211ab1bm6613627b42d4343b@mail.gmail.com> Message-ID: <6a3ae47e0909090703i48c64b7du687732a45169b970@mail.gmail.com> Thank you Dave and Paul. This will give me a enough info to proceed with my endeavour. Fabulous. Weird that Google didn't point me at nodefinder when I typed in "discover erlang node" earlier today, and now it's there at the top. And I will have to think hard about security, of course. But most likely I will restrict myself to a known subnet and combine that with a node naming strategy. Robby From vances@REDACTED Wed Sep 9 19:31:49 2009 From: vances@REDACTED (Vance Shipley) Date: Wed, 9 Sep 2009 13:31:49 -0400 Subject: [erlang-questions] Finding nodes In-Reply-To: <6a3ae47e0909090703i48c64b7du687732a45169b970@mail.gmail.com> References: <6a3ae47e0909090529h7211ab1bm6613627b42d4343b@mail.gmail.com> <6a3ae47e0909090703i48c64b7du687732a45169b970@mail.gmail.com> Message-ID: <20090909173149.GC48044@h216-235-12-172.host.egate.net> On Wed, Sep 09, 2009 at 03:03:02PM +0100, Robert Raschke wrote: } And I will have to think hard about security, of course. 1> Ethernet =:= Internet. false -- -Vance From s.merle@REDACTED Wed Sep 9 19:40:18 2009 From: s.merle@REDACTED (Sebastien Merle) Date: Wed, 9 Sep 2009 19:40:18 +0200 Subject: Supervised pool of remote processes and process state keeping Message-ID: <102d3f890909091040l3ac964c2w59a63f6f3458d433@mail.gmail.com> Hi, How a supervised pool of remote processes should be designed with OTP ? I was thinking about having a simple-one-for-one supervisor starting local processes. These processes would then spawn_link a remote one. Is there a better way ? Another thing I don't have clear, is who is responsible to keep state information to be able to restart jobs lost when the supervisor restart a process ? For example, I have a job scheduler, a worker pool and workers. With OTP principles, I would have a main supervisor that starts the scheduler (gen_server) and the pool (simple-one-for-one supervisor) the pool then starts the workers (gen_server). The scheduler send job requests to the pool, and then the pool distribute the requests amongst its workers. The problems comes when one of the worker need to be restarted. What append of the pending jobs of this worker ? Is it the supervisor (worker pool) task to keep track of each job requests given to each worker and redistribute them if a worker is restarted ? What the OTP way of handling this kind of state information ? Is it OK for the worker pool to be a supervisor ? It doesn't feel quite good that a gen_server is sending job request to a supervisor... Regards, -- Sebastien Merle. From james.hague@REDACTED Wed Sep 9 20:54:33 2009 From: james.hague@REDACTED (James Hague) Date: Wed, 9 Sep 2009 13:54:33 -0500 Subject: How binaries are implemented Message-ID: I'm puzzling over some details in section 4.1 of the Efficiency Guide, "How binaries are implemented." When a greater than 64 byte binary is created, both a RefC binary (on the global heap) and a ProcBin (on the process heap) come into existence. Also, any time a smaller binary is matched out of that binary, a sub binary is created. According to the docs: "All ProcBin objects in a process are part of a linked list, so that the garbage collector can keep track of them and decrement the reference counters in the binary when a ProcBin disappears. " Is a sub-binary essentially the same as a ProcBin, in that it gets tracked by being added to a linked list? I'd also be interested in more details about when the reference counter for RefC binaries gets incremented and decremented. I'm trying to get a handle on if extreme use of sub binaries is a bad case for the emulator. Imagine an XML parser that keeps the entire document in a big binary, then has tens of thousands of sub binaries, which are really just pointers and lengths into the master binary. It sounds like it would be cheap, but is the runtime designed for that? Or would it be exploiting binaries in ways that cause additional expense? James From kamiseq@REDACTED Wed Sep 9 23:51:39 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Wed, 9 Sep 2009 23:51:39 +0200 Subject: [erlang-questions] Re: implementing annotation in erlang In-Reply-To: <65b2728e0909030048l1796188ek14391e34c8bc76b8@mail.gmail.com> References: <8b9ee55b0909021639v3472c750u156dbb0397720248@mail.gmail.com> <9087d40e-76ac-4513-bd27-85ce76539e9e@k39g2000yqe.googlegroups.com> <65b2728e0909030048l1796188ek14391e34c8bc76b8@mail.gmail.com> Message-ID: hi, finally I went through parse_transform but constructing even simple format from scratch is a nightmare. to prepare something like cmdCallbackList()-> [{"cmdDevSetTime",setTime,1},{"cmdFpSetStatus",setStatus,2}]. I need to build {function,19,cmdCallbackList,0, [{clause,19,[],[], [{cons,20, {tuple,20, [{atom,20,"cmdDevSetTime"}, {atom,20,setTime}, {integer,20,1}]}, {cons,20, {tuple,20, [{atom,20,"cmdFpSetStatus"}, {atom,20,setStatus}, {integer,20,2}]}, {nil,20}}}]}]} is there a way to construct code blocks in more elegant way, something like create_fun(LineNo, Name, Arity, Body, ..)-> would produce function block create_tuple(LineNo, Tuple) create_list(...) and so on and still Im thinking that extracting information from code in runtime is far more productive (elegant, sufficient, just simpler) then transforming code to something different. generally all I need is just to obtain information that this function is annoted with metadata and store that or make additional tasks. with parse transform I need to build and add new function to parsed module that other can call and get those information or create whole new file with well known name that other modules could refer to. anyway thanks for help pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From corticalcomputer@REDACTED Thu Sep 10 02:24:02 2009 From: corticalcomputer@REDACTED (G.S.) Date: Wed, 9 Sep 2009 17:24:02 -0700 Subject: DDE and Erlang. Message-ID: <2a67d3ff0909091724g325010can56a7d230cb36281a@mail.gmail.com> Hello, I'm trying to interface my Erlang program with another application over DDE. Has anyone ever done this? (I can't find anything useful online). Regards, -Gene From vinoski@REDACTED Thu Sep 10 06:17:31 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Thu, 10 Sep 2009 00:17:31 -0400 Subject: [erlang-questions] Re: implementing annotation in erlang In-Reply-To: References: <8b9ee55b0909021639v3472c750u156dbb0397720248@mail.gmail.com> <9087d40e-76ac-4513-bd27-85ce76539e9e@k39g2000yqe.googlegroups.com> <65b2728e0909030048l1796188ek14391e34c8bc76b8@mail.gmail.com> Message-ID: <65b2728e0909092117p4eafe8bo8ad8a5a42f6f4c9a@mail.gmail.com> 2009/9/9 pawe? kami?ski > hi, > finally I went through parse_transform but constructing even simple format > from scratch is a nightmare. > It's not all that bad once you get used to it. > to prepare something like > cmdCallbackList()-> > [{"cmdDevSetTime",setTime,1},{"cmdFpSetStatus",setStatus,2}]. > > I need to build > {function,19,cmdCallbackList,0, > [{clause,19,[],[], > [{cons,20, > {tuple,20, > [{atom,20,"cmdDevSetTime"}, > {atom,20,setTime}, > {integer,20,1}]}, > {cons,20, > {tuple,20, > [{atom,20,"cmdFpSetStatus"}, > {atom,20,setStatus}, > {integer,20,2}]}, > {nil,20}}}]}]} > > is there a way to construct code blocks in more elegant way, something like > create_fun(LineNo, Name, Arity, Body, ..)-> would produce function block > create_tuple(LineNo, Tuple) > create_list(...) > and so on > You can write a function like the one you're trying to create in straight Erlang, making sure you compile it with debug_info, and then use code:get_object_code(module_name) to get the beam code and then use beam_lib:chunks(Beam,[abstract_code]) to get the equivalent Abstract Format to give you a nice starting point for what you're trying to produce. Another way to do it is to use erl_parse:abstract() -- you pass in the Erlang data and it spits out the Abstract Format equivalent. See also the erl_syntax module, including the revert() function that converts any syntax trees you build with the erl_syntax functions into Abstract Format. Passing normal Erlang data into erl_syntax:abstract() and then passing that result to erl_syntax:revert() gives you Abstract Format, for example, pretty much the equivalent of erl_parse:abstract(). and still Im thinking that extracting information from code in runtime is > far more productive (elegant, sufficient, just simpler) then transforming > code to something different. generally all I need is just to obtain > information that this function is annoted with metadata and store that or > make additional tasks. with parse transform I need to build and add new > function to parsed module that other can call and get those information or > create whole new file with well known name that other modules could refer > to. > To extract something at runtime, you gotta get it into the beam code to begin with. Seems to me that parse transforms are a great way to do just that. With the Abstract Format, you get the "code is data, data is code" equivalency that makes code augmentation easy, and with beam_lib you get the ability to look through the compiled code, checking modules to see if they export the functions you generated with the parse transform. Once you find them, calling them to get the metadata you want is easy. --steve From ulf.wiger@REDACTED Thu Sep 10 08:41:39 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 10 Sep 2009 08:41:39 +0200 Subject: [erlang-questions] Re: implementing annotation in erlang In-Reply-To: References: <8b9ee55b0909021639v3472c750u156dbb0397720248@mail.gmail.com> <9087d40e-76ac-4513-bd27-85ce76539e9e@k39g2000yqe.googlegroups.com> <65b2728e0909030048l1796188ek14391e34c8bc76b8@mail.gmail.com> Message-ID: <4AA89FA3.6030401@erlang-consulting.com> pawe? kami?ski wrote: > to prepare something like > cmdCallbackList()-> > [{"cmdDevSetTime",setTime,1},{"cmdFpSetStatus",setStatus,2}]. > > I need to build > {function,19,cmdCallbackList,0, > [{clause,19,[],[], > [{cons,20, > {tuple,20, > [{atom,20,"cmdDevSetTime"}, > {atom,20,setTime}, > {integer,20,1}]}, > {cons,20, > {tuple,20, > [{atom,20,"cmdFpSetStatus"}, > {atom,20,setStatus}, > {integer,20,2}]}, > {nil,20}}}]}]} > > is there a way to construct code blocks in more elegant way, something like > create_fun(LineNo, Name, Arity, Body, ..)-> would produce function block > create_tuple(LineNo, Tuple) > create_list(...) > and so on To begin with, using erl_parse:abstract([{"cmdDevSetTime",setTime,1}...]) is a significant simplification. When generating a constant term in a parse_transform, it is easiest to just build the term, and then calling abstract/1 to turn it into forms. I wouldn't worry about the line numbers. If you do, there is always abstract/2. Most of the time when dealing with parse transforms, it isn't worth it, and the compiler doesn't really care. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From john.hughes@REDACTED Thu Sep 10 08:54:18 2009 From: john.hughes@REDACTED (John Hughes) Date: Thu, 10 Sep 2009 08:54:18 +0200 Subject: [erlang-questions] Re: implementing annotation in erlang In-Reply-To: References: <8b9ee55b0909021639v3472c750u156dbb0397720248@mail.gmail.com> <9087d40e-76ac-4513-bd27-85ce76539e9e@k39g2000yqe.googlegroups.com> <65b2728e0909030048l1796188ek14391e34c8bc76b8@mail.gmail.com> Message-ID: I usually add the line numbers automatically anyway--build generated code without them, then traverse it and insert the same line number into each node. It's easy to do and it helps a bit. John From: "pawe? kami?ski" finally I went through parse_transform but constructing even simple format from scratch is a nightmare. to prepare something like cmdCallbackList()-> [{"cmdDevSetTime",setTime,1},{"cmdFpSetStatus",setStatus,2}]. I need to build {function,19,cmdCallbackList,0, [{clause,19,[],[], [{cons,20, {tuple,20, [{atom,20,"cmdDevSetTime"}, {atom,20,setTime}, {integer,20,1}]}, {cons,20, {tuple,20, [{atom,20,"cmdFpSetStatus"}, {atom,20,setStatus}, {integer,20,2}]}, {nil,20}}}]}]} is there a way to construct code blocks in more elegant way, something like create_fun(LineNo, Name, Arity, Body, ..)-> would produce function block create_tuple(LineNo, Tuple) create_list(...) and so on From rtrlists@REDACTED Thu Sep 10 10:50:12 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 10 Sep 2009 09:50:12 +0100 Subject: [erlang-questions] Finding nodes In-Reply-To: <20090909173149.GC48044@h216-235-12-172.host.egate.net> References: <6a3ae47e0909090529h7211ab1bm6613627b42d4343b@mail.gmail.com> <6a3ae47e0909090703i48c64b7du687732a45169b970@mail.gmail.com> <20090909173149.GC48044@h216-235-12-172.host.egate.net> Message-ID: <6a3ae47e0909100150w54b5212bsfc6fdcf06483797c@mail.gmail.com> On Wed, Sep 9, 2009 at 6:31 PM, Vance Shipley wrote: > On Wed, Sep 09, 2009 at 03:03:02PM +0100, Robert Raschke wrote: > } And I will have to think hard about security, of course. > > 1> Ethernet =:= Internet. > false > > > Yeah. What I meant is that I have to decide if security is something I need to actually worry about or not. I don't think I will have to worry about laying open Erlang Nodes to abuse through my use of discovery, but I have to think hard about that, in order to make sure my conclusion is correct :-) Retrofitting any measures is likely to be an impossible (or rather, improbably complicated) task. Robby From eric.l.2046@REDACTED Thu Sep 10 11:46:50 2009 From: eric.l.2046@REDACTED (Eric Liang) Date: Thu, 10 Sep 2009 17:46:50 +0800 Subject: Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] Message-ID: <4AA8CB0A.1020800@gmail.com> Hi all, To build a erlang IDE, I've tried Distel in emacs22. but unfortunately it does not work well. What I've done is exactly in accordance with Bill Clementson's blog post: http://bc.tech.coop/blog/070528.html They are: 1. Download the latest code from : http://distel.googlecode.com/svn/trunk/ 2. Modify the .emacs. I did confirm the load-path and distel-dir settings are correct. Here is it: ;; This is needed for Erlang mode setup (setq erlang-root-dir "/usr/local/lib/erlang") (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.5.1/emacs" load-path)) (setq exec-path (cons "/usr/local/lib/erlang/bin" exec-path)) (require 'erlang-start) ;; This is needed for Distel setup (let ((distel-dir "/Users/bc/Projects/distel/elisp")) (unless (member distel-dir load-path) ;; Add distel-dir to the end of load-path (setq load-path (append load-path (list distel-dir))))) (require 'distel) (distel-setup) ;; Some Erlang customizations (add-hook 'erlang-mode-hook (lambda () ;; when starting an Erlang shell in Emacs, default in the node name (setq inferior-erlang-machine-options '("-sname" "emacs")) ;; add Erlang functions to an imenu menu (imenu-add-to-menubar "imenu"))) ;; A number of the erlang-extended-mode key bindings are useful in the shell too (defconst distel-shell-keys '(("\C-\M-i" erl-complete) ("\M-?" erl-complete) ("\M-." erl-find-source-under-point) ("\M-," erl-find-source-unwind) ("\M-*" erl-find-source-unwind) ) "Additional keys to bind when in Erlang shell.") (add-hook 'erlang-shell-mode-hook (lambda () ;; add some Distel bindings to the Erlang shell (dolist (spec distel-shell-keys) (define-key erlang-shell-mode-map (car spec) (cadr spec))))) When I open a .erl file, it works well and Distel dose exist in the Erlang tag of the emacs' menubar. But when I type C-M-i behind the word "cr" to complete the function name,the emacs received this message: Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] I checked the code in Distel, it looks like that it will call completions() which will call the get() by the argument:A an empty string("") in otp_doc.erl. The function get() will call to_list() on the argument:A,which will get the error: *** ERROR: Shell process terminated! *** =ERROR REPORT==== 8-Sep-2009::15:02:42 === Error in process <0.66.0> on node 'emacs@REDACTED' with exit value: {undef,[{lib,format_exception,[4,error,undef,[{erl_internal,bif,[to_list,1]},{erl_lint,expr,3},{erl_lint,exprs,3},{erl_lint,exprs_opt,3},{erl_eval,check_command,2},{shell,exprs,6},{shell,eval_exprs,6},{shell,eval_loop.. I guess this is the reason for the completion failure, but have no idea how to fix it. Can anyone here give me some advice? As a newbie in Erlang, it's very possible I've stepped in the wrong way.However, any suggestions will be appreciated. Thanks in advance. Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From egil@REDACTED Thu Sep 10 14:59:27 2009 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Thu, 10 Sep 2009 14:59:27 +0200 Subject: [erlang-questions] How binaries are implemented In-Reply-To: References: Message-ID: <4AA8F82F.6000009@erix.ericsson.se> James Hague wrote: > I'm puzzling over some details in section 4.1 of the Efficiency Guide, > "How binaries are implemented." > > When a greater than 64 byte binary is created, both a RefC binary (on > the global heap) and a ProcBin (on the process heap) come into > existence. Also, any time a smaller binary is matched out of that > binary, a sub binary is created. According to the docs: > > "All ProcBin objects in a process are part of a linked list, so that > the garbage collector can keep track of them and decrement the > reference counters in the binary when a ProcBin disappears. " > I think the terminology is a bit confusing and Bj?rn has to correct me if I'm wrong here. A ProcBin is the (Eterm) header that points to the reference counted offheap binary, the refc binary. > Is a sub-binary essentially the same as a ProcBin, in that it gets > tracked by being added to a linked list? No. Only ProcBin binaries headers are included in the list which points to the RefC binary. The list is there for performance. Instead of scanning the heap for unreferenced ProcBins after a gc, only the list has to be traversed. Also, the sub-binaries are process bound. If you send a binary, which is a sub-binary, to another process a new ProcBin is created for that process which points to the refc binary and reference counter is increased. > > I'd also be interested in more details about when the reference > counter for RefC binaries gets incremented and decremented. > The counter is in direct correlation to the number of ProcBins (Eterm headers) to that offheap binary. When the unreferenced ProcBins gets garbage collected the reference counters are decreased and when new ProcBins are created to the binary the reference counter is increased. Also, drivers may increase and decrease the reference counter for an offheap binary. Sub-binaries will point to the ProcBins and keep them alive until they themselves become dead and collected. > I'm trying to get a handle on if extreme use of sub binaries is a bad > case for the emulator. Imagine an XML parser that keeps the entire > document in a big binary, then has tens of thousands of sub binaries, > which are really just pointers and lengths into the master binary. It > sounds like it would be cheap, but is the runtime designed for that? > Or would it be exploiting binaries in ways that cause additional > expense? Yes, it is cheap and the runtime system is designed for it. Additional expense? No, but some words of caution. A subbin will reference its master so its not only the subpart of the binary that is live, its the whole binary. If subbins or procbins survives to the old_heap (generational heap) the binary might be more longlived than intended and more expensive in terms of memory. Explicit calls erlang:garbage_collect/0|1, which will do a fullsweep of the process, will remedy this. This is a drawback which is being studied to improve current gc strategies. Regards, Bj?rn-Egil Erlang/OTP From james.hague@REDACTED Thu Sep 10 16:11:22 2009 From: james.hague@REDACTED (James Hague) Date: Thu, 10 Sep 2009 09:11:22 -0500 Subject: [erlang-questions] How binaries are implemented In-Reply-To: References: Message-ID: Good info--thank you! From erik@REDACTED Thu Sep 10 17:15:10 2009 From: erik@REDACTED (Erik Rigtorp) Date: Thu, 10 Sep 2009 17:15:10 +0200 Subject: Latency with tcp active mode Message-ID: Hi! I'm working on an application where latency is very important. It listens to messages on some TCP sockets, processes them and sends them out on some TCP sockets. The problem is that I can measure a huge round trip latency between sending a message to the application and receiving the response. After substracting the network round trip i can see insignificant delays to up to 400ms delays. Nagling is of course disabled. Apperently there is some significant delay between the NIC and until i receive the message in an erlang process. I'm using active mode for the sockets but I've noticed that in RabbitMQ they use prim_inet:async_recv() to do async tcp receive. Anyone know why they do that? I've been carefully studying the code in inet_drv.c and I can't see much difference between the code paths for using prim_inet:async_recv() and active mode. Where might this delay come from? I might try to instrument the runtime with dtrace. Erik Rigtorp From magnus.falk@REDACTED Thu Sep 10 18:04:00 2009 From: magnus.falk@REDACTED (Magnus Falk) Date: Thu, 10 Sep 2009 18:04:00 +0200 Subject: How can I determine if a list is a just a string or a list of strings? Message-ID: <41086e200909100904g78f27e71le88e868d30b66671@mail.gmail.com> Hi all, I have a variable that can either contain a list of strings or a just a string. Is there a good way to tell what kind I'm, dealing with? "192.168.1.18" vs. ["192.168.1.18", "192.168.1.19"] Cheers, Magnus P.S. I posted this question on Stackoverflow as well: http://stackoverflow.com/questions/1406173/how-can-i-determine-if-a-list-is-a-just-a-string-or-a-list-of-stringsD.S. From ulf.wiger@REDACTED Thu Sep 10 18:10:44 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 10 Sep 2009 18:10:44 +0200 Subject: [erlang-questions] How can I determine if a list is a just a string or a list of strings? In-Reply-To: <41086e200909100904g78f27e71le88e868d30b66671@mail.gmail.com> References: <41086e200909100904g78f27e71le88e868d30b66671@mail.gmail.com> Message-ID: <4AA92504.7000608@erlang-consulting.com> Magnus Falk wrote: > Hi all, > > I have a variable that can either contain a list of strings or a just a > string. Is there a good way to tell what kind I'm, dealing with? > > "192.168.1.18" vs. ["192.168.1.18", "192.168.1.19"] The classic 'dirty trick' test is is_integer(hd(String)). BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From rvirding@REDACTED Thu Sep 10 18:18:03 2009 From: rvirding@REDACTED (Robert Virding) Date: Thu, 10 Sep 2009 18:18:03 +0200 Subject: [erlang-questions] How can I determine if a list is a just a string or a list of strings? In-Reply-To: <4AA92504.7000608@erlang-consulting.com> References: <41086e200909100904g78f27e71le88e868d30b66671@mail.gmail.com> <4AA92504.7000608@erlang-consulting.com> Message-ID: <3dbc6d1c0909100918j5f09fe23y6c2ef4625fe4ea82@mail.gmail.com> Hehe, I suggested is_list(hd(String)) on stackoverflow, Robert 2009/9/10 Ulf Wiger > Magnus Falk wrote: > >> Hi all, >> >> I have a variable that can either contain a list of strings or a just a >> string. Is there a good way to tell what kind I'm, dealing with? >> >> "192.168.1.18" vs. ["192.168.1.18", "192.168.1.19"] >> > > The classic 'dirty trick' test is is_integer(hd(String)). > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From rapsey@REDACTED Thu Sep 10 19:09:35 2009 From: rapsey@REDACTED (Rapsey) Date: Thu, 10 Sep 2009 19:09:35 +0200 Subject: [erlang-questions] How can I determine if a list is a just a string or a list of strings? In-Reply-To: <4AA92504.7000608@erlang-consulting.com> References: <41086e200909100904g78f27e71le88e868d30b66671@mail.gmail.com> <4AA92504.7000608@erlang-consulting.com> Message-ID: <97619b170909101009g3d829aaek71632607ade0f981@mail.gmail.com> I think this method is faster: matches lists in lists, test first [[_|_]|_] = L mathches all lists, test second: [_|_] = L Sergej On Thu, Sep 10, 2009 at 6:10 PM, Ulf Wiger wrote: > Magnus Falk wrote: > >> Hi all, >> >> I have a variable that can either contain a list of strings or a just a >> string. Is there a good way to tell what kind I'm, dealing with? >> >> "192.168.1.18" vs. ["192.168.1.18", "192.168.1.19"] >> > > The classic 'dirty trick' test is is_integer(hd(String)). > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From juhani@REDACTED Thu Sep 10 19:45:33 2009 From: juhani@REDACTED (=?ISO-8859-1?Q?Juhani_R=E4nkimies?=) Date: Thu, 10 Sep 2009 20:45:33 +0300 Subject: "Bad value on output port 'tcp_inet'" for null terminated messages Message-ID: Hi, ok = gen_tcp:send(Socket, "message text"), seems to work (the receiving end is of course confused), but ok = gen_tcp:send(Socket, "message text\0"), gives =ERROR REPORT==== 10-Sep-2009::20:21:21 === Bad value on output port 'tcp_inet' How can I send the trailing 0-byte? This is erl5.7.2 on XP. Thanks, -juhani From chandrashekhar.mullaparthi@REDACTED Thu Sep 10 21:44:49 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Thu, 10 Sep 2009 20:44:49 +0100 Subject: [erlang-questions] "Bad value on output port 'tcp_inet'" for null terminated messages In-Reply-To: References: Message-ID: 2009/9/10 Juhani R?nkimies > Hi, > > ok = gen_tcp:send(Socket, "message text"), > > seems to work (the receiving end is of course confused), but > > ok = gen_tcp:send(Socket, "message text\0"), > > gives > > =ERROR REPORT==== 10-Sep-2009::20:21:21 === > Bad value on output port 'tcp_inet' > > Have you modified the data for the purpose of this email? There is nothing wrong with the data you are sending in the second case. > How can I send the trailing 0-byte? > gen_tcp:send(Socket, ["message text", 0]). cheers Chandru From kaczmarek.w@REDACTED Thu Sep 10 22:25:31 2009 From: kaczmarek.w@REDACTED (Wojciech Kaczmarek) Date: Thu, 10 Sep 2009 22:25:31 +0200 Subject: passing an include path to escript Message-ID: <31ae10910909101325l4f95f442m94066bfb2c3dd211@mail.gmail.com> Hi! It there a way to instruct escript where to look for files to be included? Right now it only seem to accept absolute paths or paths relative to a lib_dir of an installed library. It would be good to allow escript to include the files located in a place relative to where the script is being run. Real world example: escript is used to run etap[1] tests in a make test phase. As this is testing, no files are installed in an erlang tree yet, sadly it's impossible to use hrl files existing in the source (which would be installed thus accessible but after the tests pass..). Catch 22, isn't it? cheers, Wojtek [1] http://github.com/ngerakines/etap/tree/master From kagato@REDACTED Thu Sep 10 22:49:01 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 10 Sep 2009 13:49:01 -0700 Subject: [erlang-questions] passing an include path to escript In-Reply-To: <31ae10910909101325l4f95f442m94066bfb2c3dd211@mail.gmail.com> References: <31ae10910909101325l4f95f442m94066bfb2c3dd211@mail.gmail.com> Message-ID: <45A1888F-CEC0-4F21-B8A7-1476822F9CB2@souja.net> According to the docs (and my tests), escript will only use - include_lib, not a regular -include. The way to make that work is if you have the file in the code_path (hence, why it works for installed modules). Since any modifications within the script happen after the -include_lib would be processed, you'll have to have the escript program set this path itself. I can't find a way to do this. I tried just giving it the flag that "erl" would take, didn't work. I tried adding the flag via an environmental variable, also didn't work. (ERL_AFLAGS="-pa mypath") When I use etap, I have my actual tests in a compiled module, so I don't have this problem. That said, I don't know of any way to make this work. I think the best way would be to add "-patha(...)" and "-pathz" to escript. Anybody know how to get escript to modify its path? Also, can anyone explain why -include() isn't allowed? On Sep 10, 2009, at 1:25 PM, Wojciech Kaczmarek wrote: > Hi! > > It there a way to instruct escript where to look for files to be > included? > Right now it only seem to accept absolute paths or paths relative to a > lib_dir of an installed library. > It would be good to allow escript to include the files located in a > place relative to where the script is being run. > > Real world example: > > escript is used to run etap[1] tests in a make test phase. As this is > testing, no files are installed in an erlang tree yet, sadly it's > impossible to use hrl files existing in the source (which would be > installed thus accessible but after the tests pass..). Catch 22, isn't > it? > > cheers, > Wojtek > > [1] http://github.com/ngerakines/etap/tree/master > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From launoja@REDACTED Thu Sep 10 20:20:44 2009 From: launoja@REDACTED (Jani Launonen) Date: Thu, 10 Sep 2009 21:20:44 +0300 Subject: [erlang-questions] "Bad value on output port 'tcp_inet'" for null terminated messages In-Reply-To: References: Message-ID: > Hi, > > ok = gen_tcp:send(Socket, "message text"), > > seems to work (the receiving end is of course confused), but > > ok = gen_tcp:send(Socket, "message text\0"), > > gives > > =ERROR REPORT==== 10-Sep-2009::20:21:21 === > Bad value on output port 'tcp_inet' > > > How can I send the trailing 0-byte? How about: ok = gen_tcp:send(Socket, ["message text", 0]). I haven't tested that this actually works (my erlang environment is missing after a update and with 2.5G data downloading one is taking quite lot of time) :) But: "An I/O list is a binary or a (possibly deep) list of binaries or integers in the range 0..255." Says Erlang Reference Manual about Port BIFs. > This is erl5.7.2 on XP. > > > Thanks, > -juhani You're most welcomed, Jani > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From per@REDACTED Fri Sep 11 00:27:10 2009 From: per@REDACTED (Per Hedeland) Date: Fri, 11 Sep 2009 00:27:10 +0200 (CEST) Subject: [erlang-questions] "Bad value on output port 'tcp_inet'" for null terminated messages In-Reply-To: Message-ID: <200909102227.n8AMRAXS026040@pluto.hedeland.org> Jani Launonen wrote: [Juhani R?nkimies wrote:] >> >> ok = gen_tcp:send(Socket, "message text"), >> >> seems to work (the receiving end is of course confused), but Why "of course"? Sending NUL-terminated strings over TCP is rather uncommon. >> ok = gen_tcp:send(Socket, "message text\0"), >> >> gives >> >> =ERROR REPORT==== 10-Sep-2009::20:21:21 === >> Bad value on output port 'tcp_inet' >> >> >> How can I send the trailing 0-byte? It should work as above, and does when I try it: Eshell V5.7.2 (abort with ^G) 1> {ok,S0}=gen_tcp:listen(5678,[]). {ok,#Port<0.431>} 2> {ok,S1}=gen_tcp:connect("localhost",5678,[]). {ok,#Port<0.438>} 3> {ok,S2}=gen_tcp:accept(S0). {ok,#Port<0.439>} 4> gen_tcp:send(S1,"message text\0"). ok 5> flush(). Shell got {tcp,#Port<0.439>,[109,101,115,115,97,103,101,32,116,101,120,116,0]} ok Are you using some "strange" options? I can't think of any that would cause that result though. Or, as Chandru asked, are you really describing the problem correctly? >How about: >ok = gen_tcp:send(Socket, ["message text", 0]). That should work too.:-) >"An I/O list is a binary or a (possibly deep) list of binaries or integers in the range 0..255." >Says Erlang Reference Manual about Port BIFs. Yes - it doesn't say that it is *required* to be deep, which is the only difference between the "integer lists" "message text\0" and ["message text", 0]. 6> gen_tcp:send(S1,["message text",0]). ok 7> flush(). Shell got {tcp,#Port<0.439>,[109,101,115,115,97,103,101,32,116,101,120,116,0]} ok >> This is erl5.7.2 on XP. I wouldn't think such breakage could be caused by Windows, but who knows - maybe it has "binary" and "text" sockets... Above test is on FreeBSD. --Per Hedeland. From per@REDACTED Fri Sep 11 00:35:59 2009 From: per@REDACTED (Per Hedeland) Date: Fri, 11 Sep 2009 00:35:59 +0200 (CEST) Subject: [erlang-questions] "Bad value on output port 'tcp_inet'" for null terminated messages In-Reply-To: <200909102227.n8AMRAXS026040@pluto.hedeland.org> Message-ID: <200909102235.n8AMZx9K026261@pluto.hedeland.org> Per Hedeland wrote: > >I wouldn't think such breakage could be caused by Windows, but who >knows - maybe it has "binary" and "text" sockets... Sorry, forgot the smiley there - of course it's the VM's I/O system that gives the error message, Windows can't be blamed. And it does that if you try to send something that isn't an I/O list, e.g. 8> gen_tcp:send(S1, [message, text, 0]). =ERROR REPORT==== 11-Sep-2009::00:29:12 === Bad value on output port 'tcp_inet' So again, you probably aren't describing the problem correctly. --Per From per@REDACTED Fri Sep 11 00:49:31 2009 From: per@REDACTED (Per Hedeland) Date: Fri, 11 Sep 2009 00:49:31 +0200 (CEST) Subject: [erlang-questions] Latency with tcp active mode In-Reply-To: Message-ID: <200909102249.n8AMnVpH026470@pluto.hedeland.org> Erik Rigtorp wrote: > >I'm working on an application where latency is very important. It >listens to messages on some TCP sockets, processes them and sends them >out on some TCP sockets. The problem is that I can measure a huge >round trip latency between sending a message to the application and >receiving the response. After substracting the network round trip i >can see insignificant delays to up to 400ms delays. Nagling is of >course disabled. Are you really sure about that? 200ms is the traditional Nagle delay, though it seems most stacks use a lower value these days. When I do a trivial test with loopback connections (separate send and receive), I see ~ 100ms round-trip delay with Nagle on (default, of course), 1-2ms with it off (seems rather high actually). > Apperently there is some significant delay between >the NIC and until i receive the message in an erlang process. How did you determine that - as opposed to a delay between a send and the NIC? What OS are you running on? Of course as has been discussed many times here, selective receive and a long message queue can give very bad results - could this be the case for you? If you are actually using {active, true} you might want to try {active, once} instead... --Per Hedeland From eric.l.2046@REDACTED Fri Sep 11 06:32:25 2009 From: eric.l.2046@REDACTED (Eric Liang) Date: Fri, 11 Sep 2009 12:32:25 +0800 Subject: Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] Message-ID: <4AA9D2D9.5040605@gmail.com> Hi all, To build a erlang IDE, I've tried Distel in emacs22. but unfortunately it does not work well. What I've done is exactly in accordance with Bill Clementson's blog post: http://bc.tech.coop/blog/070528.html They are: 1. Download the latest code from : http://distel.googlecode.com/svn/trunk/ 2. Modify the .emacs. I did confirm the load-path and distel-dir settings are correct. Here is it: ;; This is needed for Erlang mode setup (setq erlang-root-dir "/usr/local/lib/erlang") (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.5.1/emacs" load-path)) (setq exec-path (cons "/usr/local/lib/erlang/bin" exec-path)) (require 'erlang-start) ;; This is needed for Distel setup (let ((distel-dir "/Users/bc/Projects/distel/elisp")) (unless (member distel-dir load-path) ;; Add distel-dir to the end of load-path (setq load-path (append load-path (list distel-dir))))) (require 'distel) (distel-setup) ;; Some Erlang customizations (add-hook 'erlang-mode-hook (lambda () ;; when starting an Erlang shell in Emacs, default in the node name (setq inferior-erlang-machine-options '("-sname" "emacs")) ;; add Erlang functions to an imenu menu (imenu-add-to-menubar "imenu"))) ;; A number of the erlang-extended-mode key bindings are useful in the shell too (defconst distel-shell-keys '(("\C-\M-i" erl-complete) ("\M-?" erl-complete) ("\M-." erl-find-source-under-point) ("\M-," erl-find-source-unwind) ("\M-*" erl-find-source-unwind) ) "Additional keys to bind when in Erlang shell.") (add-hook 'erlang-shell-mode-hook (lambda () ;; add some Distel bindings to the Erlang shell (dolist (spec distel-shell-keys) (define-key erlang-shell-mode-map (car spec) (cadr spec))))) When I open a .erl file, it works well and Distel dose exist in the Erlang tag of the emacs' menubar. But when I type C-M-i behind the word "cr" to complete the function name,the emacs received this message: Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] I checked the code in Distel, it looks like that it will call completions() which will call the get() by the argument:A an empty string("") in otp_doc.erl. The function get() will call to_list() on the argument:A,which will get the error: *** ERROR: Shell process terminated! *** =ERROR REPORT==== 8-Sep-2009::15:02:42 === Error in process <0.66.0> on node 'emacs@REDACTED' with exit value: {undef,[{lib,format_exception,[4,error,undef,[{erl_internal,bif,[to_list,1]},{erl_lint,expr,3},{erl_lint,exprs,3},{erl_lint,exprs_opt,3},{erl_eval,check_command,2},{shell,exprs,6},{shell,eval_exprs,6},{shell,eval_loop.. I guess this is the reason for the completion failure, but have no idea how to fix it. Can anyone here give me some advice? As a newbie in Erlang, it's very possible I've stepped in the wrong way.However, any suggestions will be appreciated. Thanks in advance. Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From juhani@REDACTED Fri Sep 11 06:49:33 2009 From: juhani@REDACTED (=?ISO-8859-1?Q?Juhani_R=E4nkimies?=) Date: Fri, 11 Sep 2009 07:49:33 +0300 Subject: [erlang-questions] "Bad value on output port 'tcp_inet'" for null terminated messages In-Reply-To: <200909102235.n8AMZx9K026261@pluto.hedeland.org> References: <200909102227.n8AMRAXS026040@pluto.hedeland.org> <200909102235.n8AMZx9K026261@pluto.hedeland.org> Message-ID: On Fri, Sep 11, 2009 at 1:35 AM, Per Hedeland wrote: > Per Hedeland wrote: >> > So again, you probably aren't describing the problem correctly. > > --Per > Hello Jani, Chandru and Per, Yep, that's right. Sleep deprivation in action. All the necessary information to solve the problem was right there in from of my eyes all the time. I remember it. My brains just refused to process the information. A tuple had slipped in iolist of another send. Sorry and thanks, -juhani From ngocdaothanh@REDACTED Fri Sep 11 09:09:39 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 11 Sep 2009 16:09:39 +0900 Subject: [erlang-questions] Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] In-Reply-To: <4AA8CB0A.1020800@gmail.com> References: <4AA8CB0A.1020800@gmail.com> Message-ID: <5c493e530909110009m43c0c47emd49d86b477ab6a88@mail.gmail.com> I tried Distel from trunk yesterday and it did not work. The version from download did work: http://code.google.com/p/distel/downloads/list. On Thu, Sep 10, 2009 at 6:46 PM, Eric Liang wrote: > Hi all, > To build a erlang IDE, I've tried Distel in emacs22. but unfortunately it > does not work well. What I've done is exactly in accordance with Bill > Clementson's blog post: > > http://bc.tech.coop/blog/070528.html > > They are: > > Download the latest code from :? http://distel.googlecode.com/svn/trunk/ > Modify the .emacs. I did confirm the load-path and distel-dir settings are > correct. Here is it: > > ;; This is needed for Erlang mode setup > (setq erlang-root-dir "/usr/local/lib/erlang") > (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.5.1/emacs" > load-path)) > (setq exec-path (cons "/usr/local/lib/erlang/bin" exec-path)) > (require 'erlang-start) > > ;; This is needed for Distel setup > (let ((distel-dir "/Users/bc/Projects/distel/elisp")) > (unless (member distel-dir load-path) > ;; Add distel-dir to the end of load-path > (setq load-path (append load-path (list distel-dir))))) > > (require 'distel) > (distel-setup) > > ;; Some Erlang customizations > (add-hook 'erlang-mode-hook > (lambda () > ;; when starting an Erlang shell in Emacs, default in the node name > (setq inferior-erlang-machine-options '("-sname" "emacs")) > ;; add Erlang functions to an imenu menu > (imenu-add-to-menubar "imenu"))) > > ;; A number of the erlang-extended-mode key bindings are useful in the shell > too > (defconst distel-shell-keys > '(("\C-\M-i" erl-complete) > ("\M-?" erl-complete) > ("\M-." erl-find-source-under-point) > ("\M-," erl-find-source-unwind) > ("\M-*" erl-find-source-unwind) > ) > "Additional keys to bind when in Erlang shell.") > > (add-hook 'erlang-shell-mode-hook > (lambda () > ;; add some Distel bindings to the Erlang shell > (dolist (spec distel-shell-keys) > (define-key erlang-shell-mode-map (car spec) (cadr spec))))) > > When I open a .erl file, it works well and Distel dose exist in the Erlang > tag of the emacs' menubar. But when I type C-M-i behind the word "cr" to > complete the function name,the emacs received this message: > > Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel > modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] > > I checked the code in Distel, it looks like that it will call completions() > which will call the get() by the argument:A? an empty string("")? in > otp_doc.erl.? The function get() will call to_list() on the argument:A,which > will get the error: > > *** ERROR: Shell process terminated! *** > > =ERROR REPORT==== 8-Sep-2009::15:02:42 === > Error in process <0.66.0> on node 'emacs@REDACTED' with exit value: > {undef,[{lib,format_exception,[4,error,undef,[{erl_internal,bif,[to_list,1]},{erl_lint,expr,3},{erl_lint,exprs,3},{erl_lint,exprs_opt,3},{erl_eval,check_command,2},{shell,exprs,6},{shell,eval_exprs,6},{shell,eval_loop.. > > I guess this is the reason for the completion failure, but have no idea how > to fix it. Can anyone here give me some advice? > > As a newbie in Erlang, it's very possible I've stepped in the wrong > way.However, any suggestions will be appreciated. > > Thanks in advance. > > Eric From vladdu55@REDACTED Fri Sep 11 09:50:17 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 11 Sep 2009 09:50:17 +0200 Subject: global save_calls flag? Message-ID: <95be1d3b0909110050x27c99412ga82830877ebe420c@mail.gmail.com> Hi, Is there any way to globally set the save_calls flag on all processes in the system? Of course, this is for debugging only. The only way I found is to periodically poll all processes and set the flag on them, but that's very clunky. best regards, Vlad From eric.l.2046@REDACTED Fri Sep 11 13:04:14 2009 From: eric.l.2046@REDACTED (Eric Liang) Date: Fri, 11 Sep 2009 19:04:14 +0800 Subject: [erlang-questions] Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] In-Reply-To: <5c493e530909110009m43c0c47emd49d86b477ab6a88@mail.gmail.com> References: <4AA8CB0A.1020800@gmail.com> <5c493e530909110009m43c0c47emd49d86b477ab6a88@mail.gmail.com> Message-ID: <4AAA2EAE.8030408@gmail.com> Ngoc Dao wrote: > I tried Distel from trunk yesterday and it did not work. The version > from download did work: > http://code.google.com/p/distel/downloads/list. > > Thanks Ngoc, it works. But, still can't debug and profile, like Bill Clementson's introduction. When I type C-c C-d i, it looks like is interpreting forever, and can't set breakpoints to debug. I've checked the archive of this mailling list, there is only one unresolved thread here: http://groups.google.com/group/erlang-programming/browse_thread/thread/c34d5245998948c6 Do you have more suggestions? TIA. > On Thu, Sep 10, 2009 at 6:46 PM, Eric Liang wrote: > >> Hi all, >> To build a erlang IDE, I've tried Distel in emacs22. but unfortunately it >> does not work well. What I've done is exactly in accordance with Bill >> Clementson's blog post: >> >> http://bc.tech.coop/blog/070528.html >> >> They are: >> >> Download the latest code from : http://distel.googlecode.com/svn/trunk/ >> Modify the .emacs. I did confirm the load-path and distel-dir settings are >> correct. Here is it: >> >> ;; This is needed for Erlang mode setup >> (setq erlang-root-dir "/usr/local/lib/erlang") >> (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.5.1/emacs" >> load-path)) >> (setq exec-path (cons "/usr/local/lib/erlang/bin" exec-path)) >> (require 'erlang-start) >> >> ;; This is needed for Distel setup >> (let ((distel-dir "/Users/bc/Projects/distel/elisp")) >> (unless (member distel-dir load-path) >> ;; Add distel-dir to the end of load-path >> (setq load-path (append load-path (list distel-dir))))) >> >> (require 'distel) >> (distel-setup) >> >> ;; Some Erlang customizations >> (add-hook 'erlang-mode-hook >> (lambda () >> ;; when starting an Erlang shell in Emacs, default in the node name >> (setq inferior-erlang-machine-options '("-sname" "emacs")) >> ;; add Erlang functions to an imenu menu >> (imenu-add-to-menubar "imenu"))) >> >> ;; A number of the erlang-extended-mode key bindings are useful in the shell >> too >> (defconst distel-shell-keys >> '(("\C-\M-i" erl-complete) >> ("\M-?" erl-complete) >> ("\M-." erl-find-source-under-point) >> ("\M-," erl-find-source-unwind) >> ("\M-*" erl-find-source-unwind) >> ) >> "Additional keys to bind when in Erlang shell.") >> >> (add-hook 'erlang-shell-mode-hook >> (lambda () >> ;; add some Distel bindings to the Erlang shell >> (dolist (spec distel-shell-keys) >> (define-key erlang-shell-mode-map (car spec) (cadr spec))))) >> >> When I open a .erl file, it works well and Distel dose exist in the Erlang >> tag of the emacs' menubar. But when I type C-M-i behind the word "cr" to >> complete the function name,the emacs received this message: >> >> Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel >> modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] >> >> I checked the code in Distel, it looks like that it will call completions() >> which will call the get() by the argument:A an empty string("") in >> otp_doc.erl. The function get() will call to_list() on the argument:A,which >> will get the error: >> >> *** ERROR: Shell process terminated! *** >> >> =ERROR REPORT==== 8-Sep-2009::15:02:42 === >> Error in process <0.66.0> on node 'emacs@REDACTED' with exit value: >> {undef,[{lib,format_exception,[4,error,undef,[{erl_internal,bif,[to_list,1]},{erl_lint,expr,3},{erl_lint,exprs,3},{erl_lint,exprs_opt,3},{erl_eval,check_command,2},{shell,exprs,6},{shell,eval_exprs,6},{shell,eval_loop.. >> >> I guess this is the reason for the completion failure, but have no idea how >> to fix it. Can anyone here give me some advice? >> >> As a newbie in Erlang, it's very possible I've stepped in the wrong >> way.However, any suggestions will be appreciated. >> >> Thanks in advance. >> >> Eric >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From daniel.kwiecinski@REDACTED Fri Sep 11 13:44:43 2009 From: daniel.kwiecinski@REDACTED (Daniel Kwiecinski) Date: Fri, 11 Sep 2009 12:44:43 +0100 Subject: Is it possible to obtain all the records information at macros expanding time? E.g. Something like shell's rl(). Message-ID: Hi folks, Is it possible to obtain all the records information at macros expanding time? E.g. Something like shell's *rl()*. Thank you in advance. -- Kind Regards, Daniel Kwiecinski From leap@REDACTED Fri Sep 11 17:13:55 2009 From: leap@REDACTED (Michael Turner) Date: Fri, 11 Sep 2009 15:13:55 +0000 Subject: Artificial Neural Network example at trapexit.org - any successes? Message-ID: http://www.trapexit.org/Erlang_and_Neural_Networks The code isn't complete, and when I fill in the blanks as best I can -- well, I realized upon running it there doesn't seem to be any guarantee that the network is finished building itself before you do any operations on it. I've written the original blog author for help, but no reply yet. Does anyone have the complete code? Has anyone reproduced the results shown in the trapexit.org article? Searching around, I find some comments on smaller details (like whether the author implemented dot-product efficiently or idiomatically) but nobody talking about getting the whole thing working, successfully or not. An article that contained code that actually worked would, I think, redound somewhat more to the credit of Erlang. I liked looking at the pictures, though. -michael turner From eric.l.2046@REDACTED Fri Sep 11 18:09:36 2009 From: eric.l.2046@REDACTED (Eric Liang) Date: Sat, 12 Sep 2009 00:09:36 +0800 Subject: [erlang-questions] Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] In-Reply-To: <4AAA2EAE.8030408@gmail.com> References: <4AA8CB0A.1020800@gmail.com> <5c493e530909110009m43c0c47emd49d86b477ab6a88@mail.gmail.com> <4AAA2EAE.8030408@gmail.com> Message-ID: <4AAA7640.8090700@gmail.com> Eric Liang wrote: > Ngoc Dao wrote: >> I tried Distel from trunk yesterday and it did not work. The version >> from download did work: >> http://code.google.com/p/distel/downloads/list. >> >> > Thanks Ngoc, it works. But, still can't debug and profile, like Bill > Clementson's introduction. > > When I type C-c C-d i, it looks like is interpreting forever, and > can't set breakpoints to debug. I've checked the archive of this > mailling list, there is only one unresolved thread here: > > http://groups.google.com/group/erlang-programming/browse_thread/thread/c34d5245998948c6 > > Do you have more suggestions? TIA. I think I've resolved this. Just load module (C-c C-d L) before being interpreted. So simple. :-[ In Distel, load module will tell you nothing more than "load: [module test_ring]", except the first time is successful, in which scenario, it will tell you "Successfully uploaded backend modules into node". So, It's difficulty to know whether or not the module was loaded successfully. And, one module can be loaded only if it's beam has been compiled by the flag: debug_info. > >> On Thu, Sep 10, 2009 at 6:46 PM, Eric Liang wrote: >> >>> Hi all, >>> To build a erlang IDE, I've tried Distel in emacs22. but unfortunately it >>> does not work well. What I've done is exactly in accordance with Bill >>> Clementson's blog post: >>> >>> http://bc.tech.coop/blog/070528.html >>> >>> They are: >>> >>> Download the latest code from : http://distel.googlecode.com/svn/trunk/ >>> Modify the .emacs. I did confirm the load-path and distel-dir settings are >>> correct. Here is it: >>> >>> ;; This is needed for Erlang mode setup >>> (setq erlang-root-dir "/usr/local/lib/erlang") >>> (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.5.1/emacs" >>> load-path)) >>> (setq exec-path (cons "/usr/local/lib/erlang/bin" exec-path)) >>> (require 'erlang-start) >>> >>> ;; This is needed for Distel setup >>> (let ((distel-dir "/Users/bc/Projects/distel/elisp")) >>> (unless (member distel-dir load-path) >>> ;; Add distel-dir to the end of load-path >>> (setq load-path (append load-path (list distel-dir))))) >>> >>> (require 'distel) >>> (distel-setup) >>> >>> ;; Some Erlang customizations >>> (add-hook 'erlang-mode-hook >>> (lambda () >>> ;; when starting an Erlang shell in Emacs, default in the node name >>> (setq inferior-erlang-machine-options '("-sname" "emacs")) >>> ;; add Erlang functions to an imenu menu >>> (imenu-add-to-menubar "imenu"))) >>> >>> ;; A number of the erlang-extended-mode key bindings are useful in the shell >>> too >>> (defconst distel-shell-keys >>> '(("\C-\M-i" erl-complete) >>> ("\M-?" erl-complete) >>> ("\M-." erl-find-source-under-point) >>> ("\M-," erl-find-source-unwind) >>> ("\M-*" erl-find-source-unwind) >>> ) >>> "Additional keys to bind when in Erlang shell.") >>> >>> (add-hook 'erlang-shell-mode-hook >>> (lambda () >>> ;; add some Distel bindings to the Erlang shell >>> (dolist (spec distel-shell-keys) >>> (define-key erlang-shell-mode-map (car spec) (cadr spec))))) >>> >>> When I open a .erl file, it works well and Distel dose exist in the Erlang >>> tag of the emacs' menubar. But when I type C-M-i behind the word "cr" to >>> complete the function name,the emacs received this message: >>> >>> Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel >>> modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] >>> >>> I checked the code in Distel, it looks like that it will call completions() >>> which will call the get() by the argument:A an empty string("") in >>> otp_doc.erl. The function get() will call to_list() on the argument:A,which >>> will get the error: >>> >>> *** ERROR: Shell process terminated! *** >>> >>> =ERROR REPORT==== 8-Sep-2009::15:02:42 === >>> Error in process <0.66.0> on node 'emacs@REDACTED' with exit value: >>> {undef,[{lib,format_exception,[4,error,undef,[{erl_internal,bif,[to_list,1]},{erl_lint,expr,3},{erl_lint,exprs,3},{erl_lint,exprs_opt,3},{erl_eval,check_command,2},{shell,exprs,6},{shell,eval_exprs,6},{shell,eval_loop.. >>> >>> I guess this is the reason for the completion failure, but have no idea how >>> to fix it. Can anyone here give me some advice? >>> >>> As a newbie in Erlang, it's very possible I've stepped in the wrong >>> way.However, any suggestions will be appreciated. >>> >>> Thanks in advance. >>> >>> Eric >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From goran.bage@REDACTED Fri Sep 11 17:38:57 2009 From: goran.bage@REDACTED (=?ISO-8859-1?Q?G=F6ran_B=E5ge?=) Date: Fri, 11 Sep 2009 17:38:57 +0200 Subject: R9C on OSX Message-ID: <4AAA6F11.9070907@mobilearts.com> Hello, Is anyone running R9 on a mac? I'm moving from Linux to Mac as a development environment, but I need R9 as we have customer installations running R9. I found sources on the net (R9C-2) but configuration failed (curses missing). Anyone have any information that may help? Cheers, -- -- Goran ------------------------- May the Snow be with you ---- Goran Bage, MobileArts, www.mobilearts.com Tjarhovsgatan 56, SE-116 28 Stockholm, Sweden email:goran.bage@REDACTED, phone: +46 733 358405 From kaczmarek.w@REDACTED Fri Sep 11 18:37:17 2009 From: kaczmarek.w@REDACTED (Wojciech Kaczmarek) Date: Fri, 11 Sep 2009 18:37:17 +0200 Subject: [erlang-questions] Re: implementing annotation in erlang In-Reply-To: References: <8b9ee55b0909021639v3472c750u156dbb0397720248@mail.gmail.com> <9087d40e-76ac-4513-bd27-85ce76539e9e@k39g2000yqe.googlegroups.com> <65b2728e0909030048l1796188ek14391e34c8bc76b8@mail.gmail.com> Message-ID: <31ae10910909110937y62de8eaej2ffa02df794c51fd@mail.gmail.com> Hi Pawe?! 2009/9/9 pawe? kami?ski : > hi, > finally I went through parse_transform but constructing even simple format > from scratch is a nightmare. [cut] > is there a way to construct code blocks in more elegant way, something like > create_fun(LineNo, Name, Arity, Body, ..)-> would produce function block > create_tuple(LineNo, Tuple) > create_list(...) > and so on Whenever somebody tries to use parse_transform to implement some problem which sounds easy (but then the complexity explodes), I tend to immediately think about LFE - Lisp Flavoured Erlang. It worked for me many times and I didn't have to bother with parse_transforms. So, in LFE you could define a simple macro which would run your code at compile time when you define a function. Eg. macro could add function name to a well known ets. It would be easy for you to do if you were exposed to any Lisp already. If not, that's another paradigm shift to handle. > and still Im thinking that extracting information from code in runtime is > far more productive (elegant, sufficient, just simpler) then transforming > code to something different. generally all I need is just to obtain > information that this function is annoted with metadata and store that or > make additional tasks. with parse transform I need to build and add new > function to parsed module that other can call and get those information or > create whole new file with well known name that other modules could refer > to. Erlang runtime is very simple and there you have only compiled modules containing functions. Even the presence of variable bindings in a shell is a hack. As it may seem constraining for coders coming from other languages, such simplicity makes robustness more easy. cheers, Wojtek From kagato@REDACTED Sat Sep 12 00:17:23 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 11 Sep 2009 15:17:23 -0700 Subject: [erlang-questions] Is it possible to obtain all the records information at macros expanding time? E.g. Something like shell's rl(). In-Reply-To: References: Message-ID: <0668F728-B3C0-4A13-9240-389F370D8285@souja.net> It's not identical to rl(), but there is record_info(). It's actually not even a real function, as it gets expanded like a macro at compile time. About the only place you see it much is creating Mnesia tables (to get the field names). Read all about it here: http://erlang.org/doc/reference_manual/records.html#8.7 8.7 is probably the more useful nooks in the Erlang Reference. It's helpful to know stuff like this: -record(moves,up,down,left,right). record_info(fields,moves) -> [up,down,left,right]; record_info(size,moves) -> 5. And most interesting: #moves.down =:= 3. X#moves.down =:= element(#moves.down, X). On Sep 11, 2009, at 4:44 AM, Daniel Kwiecinski wrote: > Hi folks, > > Is it possible to obtain all the records information at macros > expanding > time? E.g. Something like shell's *rl()*. > > Thank you in advance. > > -- > Kind Regards, > Daniel Kwiecinski -- Jayson Vantuyl kagato@REDACTED From steven.charles.davis@REDACTED Sat Sep 12 16:11:51 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 12 Sep 2009 07:11:51 -0700 (PDT) Subject: Is it possible to obtain all the records information at macros expanding time? E.g. Something like shell's rl(). In-Reply-To: <0668F728-B3C0-4A13-9240-389F370D8285@souja.net> References: <0668F728-B3C0-4A13-9240-389F370D8285@souja.net> Message-ID: <93fc444a-ea27-4335-9b15-ba0934d2030a@j4g2000yqa.googlegroups.com> ...and if you happen to be using records to define tables in mnesia, then you can get the field names back at run time by using mnesia:table_info(Table, attributes). /s On Sep 11, 5:17?pm, Jayson Vantuyl wrote: > It's not identical to rl(), but there is record_info(). ?It's actually ? > not even a real function, as it gets expanded like a macro at compile ? > time. ?About the only place you see it much is creating Mnesia tables ? > (to get the field names). From steven.charles.davis@REDACTED Sat Sep 12 17:03:01 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 12 Sep 2009 08:03:01 -0700 (PDT) Subject: Google Trends for Erlang Message-ID: <29e14dd2-93b4-4077-90b8-2f6fa2e3a636@a21g2000yqc.googlegroups.com> Some weekend "fun", and the results of this surprised me greatly: http://www.google.com/trends?q=erlang ...who would have thought that the most active region (at least by search rank) would be Korea and not Sweden, the UK or the USA? /sd From kiszl@REDACTED Sat Sep 12 17:40:33 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sat, 12 Sep 2009 17:40:33 +0200 Subject: [erlang-questions] Google Trends for Erlang In-Reply-To: <29e14dd2-93b4-4077-90b8-2f6fa2e3a636@a21g2000yqc.googlegroups.com> References: <29e14dd2-93b4-4077-90b8-2f6fa2e3a636@a21g2000yqc.googlegroups.com> Message-ID: <4AABC0F1.4080702@tmit.bme.hu> Would be nice to see what percentage of these searches are generated by mythologists, mathematicians and programmers, all searching for completely different "Erlangs". Z. Steve Davis wrote: > Some weekend "fun", and the results of this surprised me greatly: > > http://www.google.com/trends?q=erlang > > ...who would have thought that the most active region (at least by > search rank) would be Korea and not Sweden, the UK or the USA? > > /sd > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From steven.charles.davis@REDACTED Sat Sep 12 22:09:16 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 12 Sep 2009 15:09:16 -0500 Subject: [erlang-questions] Google Trends for Erlang In-Reply-To: <8209f740909121132l21b78a27ue87335e933c512ad@mail.gmail.com> References: <29e14dd2-93b4-4077-90b8-2f6fa2e3a636@a21g2000yqc.googlegroups.com> <4AABC0F1.4080702@tmit.bme.hu> <8209f740909121132l21b78a27ue87335e933c512ad@mail.gmail.com> Message-ID: <4AABFFEC.7040900@gmail.com> That would explain a lot! There's lies, damned lies and then statistics. Ulf Wiger wrote: > Something like this perhaps? :) > > http://www.theothereast.net/?p=1681 > > "...it seems that D?erlanger just released an announcement to their > fan club stating their intentions to play in South Korea." > From paul-trapexit@REDACTED Sun Sep 13 03:25:03 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Sat, 12 Sep 2009 18:25:03 -0700 (PDT) Subject: dets durability Message-ID: i'm having trouble finding the answer to this question. if i do a bunch of dets operations, e.g., dets:insert/2 and dets:delete/2, and then hard reboot the operating system without synchronizing the page cache (e.g., hit the power button), and then reopen the dets file: what guarantees do i have? some possible answers are (after the file is "repaired"): * every update is guaranteed to be reflected in the dets; or * you may not have every update, but the dets will be in a consistent state. no partial updates will be applied (no mangled records); or * no guarantees, due to some unlikely worst case. thanks in advance, -- p From mjtruog@REDACTED Sun Sep 13 04:08:06 2009 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 12 Sep 2009 19:08:06 -0700 Subject: [erlang-questions] dets durability In-Reply-To: References: Message-ID: <4AAC5406.9000002@gmail.com> I suspect "every update is guaranteed to be reflected in the dets" only if you use dets:sync/1 on the table after doing an dets:insert/2 or dets:delete/2 before a reboot. However, since that probably can not always consistently occur (can not assume a insert/sync or delete/sync is atomic) in reality I think "you may not have every update, but the dets will be in a consistent state. no partial updates will be applied (no mangled records)", is a more realistic assessment. Paul Mineiro wrote: > i'm having trouble finding the answer to this question. > > if i do a bunch of dets operations, e.g., dets:insert/2 and dets:delete/2, > and then hard reboot the operating system without synchronizing the > page cache (e.g., hit the power button), and then reopen the dets file: > what guarantees do i have? > > some possible answers are (after the file is "repaired"): > > * every update is guaranteed to be reflected in the dets; or > * you may not have every update, but the dets will be in a consistent > state. no partial updates will be applied (no mangled records); or > * no guarantees, due to some unlikely worst case. > > thanks in advance, > > -- p > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From eugoss@REDACTED Sun Sep 13 05:51:08 2009 From: eugoss@REDACTED (Eugene Ossintsev) Date: Sat, 12 Sep 2009 20:51:08 -0700 (PDT) Subject: Google Trends for Erlang In-Reply-To: <29e14dd2-93b4-4077-90b8-2f6fa2e3a636@a21g2000yqc.googlegroups.com> References: <29e14dd2-93b4-4077-90b8-2f6fa2e3a636@a21g2000yqc.googlegroups.com> Message-ID: <179157a9-7446-48c5-a7d0-bb22e0c2cdf8@s39g2000yqj.googlegroups.com> I'm not sure why South Korea is the number one. Maybe due to their incredible thirst for knowledge? #2 for Sweden and #3 for Russia are fair enough. No questions here. Why isn't the USA at the top? I don't know. Why sould it be? FP languages are not very popular in the USA, unfortunately. More in Europe and Russia. On Sep 12, 11:03?am, Steve Davis wrote: > Some weekend "fun", and the results of this surprised me greatly: > > http://www.google.com/trends?q=erlang > > ...who would have thought that the most active region (at least by > search rank) would be Korea and not Sweden, the UK or the USA? > > /sd > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From attila.r.nohl@REDACTED Sun Sep 13 11:28:17 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Sun, 13 Sep 2009 11:28:17 +0200 Subject: [erlang-questions] dets durability In-Reply-To: References: Message-ID: <401d3ba30909130228p5db5aa5ev4743d664da3d28f0@mail.gmail.com> 2009/9/13, Paul Mineiro : > i'm having trouble finding the answer to this question. > > if i do a bunch of dets operations, e.g., dets:insert/2 and dets:delete/2, > and then hard reboot the operating system without synchronizing the > page cache (e.g., hit the power button), and then reopen the dets file: > what guarantees do i have? > > some possible answers are (after the file is "repaired"): > > * every update is guaranteed to be reflected in the dets; or > * you may not have every update, but the dets will be in a consistent > state. no partial updates will be applied (no mangled records); or > * no guarantees, due to some unlikely worst case. I think it's not that unlikely that the hard drive has a write cache, so even if the operating system (and the Erlang VM) thinks that the data made through to the disk (which might not be the case when you hit the power button), in reality it didn't. So my guess would be "no guarantees". From ngocdaothanh@REDACTED Sun Sep 13 16:05:57 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Sun, 13 Sep 2009 23:05:57 +0900 Subject: [erlang-questions] Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] In-Reply-To: <4AAA7640.8090700@gmail.com> References: <4AA8CB0A.1020800@gmail.com> <5c493e530909110009m43c0c47emd49d86b477ab6a88@mail.gmail.com> <4AAA2EAE.8030408@gmail.com> <4AAA7640.8090700@gmail.com> Message-ID: <5c493e530909130705t6f7afd4dubf5fe95a30485129@mail.gmail.com> After many try-and-errors I suspect that Distel currently can only interpret a module if its .beam file is in the same directory with its .erl file, or in ../ebin. Is this true? How to overcome this problem? On Sat, Sep 12, 2009 at 1:09 AM, Eric Liang wrote: > Eric Liang wrote: > > Ngoc Dao wrote: > > I tried Distel from trunk yesterday and it did not work. The version > from download did work: > http://code.google.com/p/distel/downloads/list. > > > > Thanks Ngoc, it works. But, still can't debug and profile, like Bill > Clementson's introduction. > > When I type C-c C-d i, it looks like is interpreting forever, and can't set > breakpoints to debug. I've checked the archive of this mailling list, there > is only one unresolved thread here: > > http://groups.google.com/group/erlang-programming/browse_thread/thread/c34d5245998948c6 > > Do you have more suggestions? TIA. > > I think I've resolved this. Just load module (C-c C-d L) before being > interpreted. So simple. :-[ > > In Distel, load module will tell you nothing more than "load: [module > test_ring]", except the first time is successful, in which scenario, it will > tell you "Successfully uploaded backend modules into node". So, It's > difficulty to? know? whether or not the module was loaded? successfully. > > And, one module can be loaded only if it's beam has been compiled by the > flag: debug_info. > > > On Thu, Sep 10, 2009 at 6:46 PM, Eric Liang wrote: > > > Hi all, > To build a erlang IDE, I've tried Distel in emacs22. but unfortunately it > does not work well. What I've done is exactly in accordance with Bill > Clementson's blog post: > > http://bc.tech.coop/blog/070528.html > > They are: > > Download the latest code from :? http://distel.googlecode.com/svn/trunk/ > Modify the .emacs. I did confirm the load-path and distel-dir settings are > correct. Here is it: > > ;; This is needed for Erlang mode setup > (setq erlang-root-dir "/usr/local/lib/erlang") > (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.5.1/emacs" > load-path)) > (setq exec-path (cons "/usr/local/lib/erlang/bin" exec-path)) > (require 'erlang-start) > > ;; This is needed for Distel setup > (let ((distel-dir "/Users/bc/Projects/distel/elisp")) > (unless (member distel-dir load-path) > ;; Add distel-dir to the end of load-path > (setq load-path (append load-path (list distel-dir))))) > > (require 'distel) > (distel-setup) > > ;; Some Erlang customizations > (add-hook 'erlang-mode-hook > (lambda () > ;; when starting an Erlang shell in Emacs, default in the node name > (setq inferior-erlang-machine-options '("-sname" "emacs")) > ;; add Erlang functions to an imenu menu > (imenu-add-to-menubar "imenu"))) > > ;; A number of the erlang-extended-mode key bindings are useful in the shell > too > (defconst distel-shell-keys > '(("\C-\M-i" erl-complete) > ("\M-?" erl-complete) > ("\M-." erl-find-source-under-point) > ("\M-," erl-find-source-unwind) > ("\M-*" erl-find-source-unwind) > ) > "Additional keys to bind when in Erlang shell.") > > (add-hook 'erlang-shell-mode-hook > (lambda () > ;; add some Distel bindings to the Erlang shell > (dolist (spec distel-shell-keys) > (define-key erlang-shell-mode-map (car spec) (cadr spec))))) > > When I open a .erl file, it works well and Distel dose exist in the Erlang > tag of the emacs' menubar. But when I type C-M-i behind the word "cr" to > complete the function name,the emacs received this message: > > Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel > modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] > > I checked the code in Distel, it looks like that it will call completions() > which will call the get() by the argument:A? an empty string("")? in > otp_doc.erl.? The function get() will call to_list() on the argument:A,which > will get the error: > > *** ERROR: Shell process terminated! *** > > =ERROR REPORT==== 8-Sep-2009::15:02:42 === > Error in process <0.66.0> on node 'emacs@REDACTED' with exit value: > {undef,[{lib,format_exception,[4,error,undef,[{erl_internal,bif,[to_list,1]},{erl_lint,expr,3},{erl_lint,exprs,3},{erl_lint,exprs_opt,3},{erl_eval,check_command,2},{shell,exprs,6},{shell,eval_exprs,6},{shell,eval_loop.. > > I guess this is the reason for the completion failure, but have no idea how > to fix it. Can anyone here give me some advice? > > As a newbie in Erlang, it's very possible I've stepped in the wrong > way.However, any suggestions will be appreciated. > > Thanks in advance. > > Eric > > > From macfisherman@REDACTED Sun Sep 13 17:27:46 2009 From: macfisherman@REDACTED (Jeff Macdonald) Date: Sun, 13 Sep 2009 11:27:46 -0400 Subject: newbie having difficulty with inet_res:nslookup Message-ID: <45ae90370909130827y16daaef9ibe9fb9558beb9e30@mail.gmail.com> Hi All, I wrote a fairly simple program to look up MX records for domains and then look up the exhanger's A and AAAA records. I'm trying to see how many MX's have IPv6 capable hosts. My program simply reads a file which contains a domain per line and spawns a processes/thread/child for each domain. The spawned process then does the MX, A and AAAA lookups and prints the results to STDOUT as a erlang tupple. This seems to work fine for a small sample set of 10 rows but when trying 100 rows or more I get: =ERROR REPORT==== 12-Sep-2009::23:06:41 === Error in process <0.11161.0> with exit value: {function_clause,[{lists,map,[#Fun,emfile]},{mx,do_lookups,1}]} My program is below. Usage is: mx:init({192,168, 1, 1}). (replace that tupple with the IP of your DNS server) mx:process_file("file-with-domains"). I currently can't share a list of domains because it is customer data, but I sure there is some lists somewhere. I suspect that I'm overloading the DNS server at home. That is just whatever is running in my linksys router. Any other suggestions about the code welcomed. I know I could probably run the A and AAAA queries in parallel. I also can probably skip queries when the data is present in the MX results. For now, I'm just trying to get the basics. TIA %% Author: jeff %% Created: Sep 10, 2009 %% Description: TODO: Add description to mx -module(mx). %% %% Include files %% -include_lib("kernel/src/inet_dns.hrl"). %% %% Exported Functions %% -export([init/1, process_file/1, do_lookups/1]). %% %% API Functions %% init(Resolver) -> inet_db:add_ns(Resolver). process_file(Filename) -> {ok,Fh}=file:open(Filename, [read]), read_file(Fh). do_lookups(Domain) -> MX_list=lookup_mx(Domain), Address=lists:map(fun lookup_address/1, MX_list), io:format("~p:~n\t~p~n~n", [Domain, Address]). %% %% Local Functions %% read_file(Fh) -> case io:get_line(Fh, "") of eof -> done; {error, Reason} -> {error, Reason}; Data -> process_data(string:strip(Data,right,$\n)), read_file(Fh) end. process_data(Data) -> spawn(mx, do_lookups, [Data]). lookup_mx(Domain) -> case inet_res:nslookup(Domain, 1, mx) of {error, Reason} -> Reason; {ok, #dns_rec{ anlist = Ans }} -> lists:map(fun parse_mx/1, Ans) end. lookup_address(Host) -> { Host, {a, lookup_address(Host, a)}, {aaaa, lookup_address(Host, aaaa)} }. lookup_address(Host, a) -> case inet_res:nslookup(Host, 1, a) of {error, Reason} -> Reason; {ok, #dns_rec{ anlist = Ans }} -> lists:map(fun parse_a/1, Ans) end; lookup_address(Host, aaaa) -> case inet_res:nslookup(Host, 1, aaaa) of {error, Reason} -> Reason; {ok, #dns_rec{ anlist = Ans }} -> Host,lists:map(fun parse_aaaa/1, Ans) end. parse_mx( RR ) -> #dns_rr{ type=mx, data={ _Pref, Exchange} } = RR, Exchange. parse_a( RR ) -> #dns_rr{ type=a, data=Address } = RR, Address. parse_aaaa( RR ) -> #dns_rr{ type=aaaa, data=Address } = RR, Address. -- Jeff Macdonald Ayer, MA From macfisherman@REDACTED Sun Sep 13 17:29:16 2009 From: macfisherman@REDACTED (Jeff Macdonald) Date: Sun, 13 Sep 2009 11:29:16 -0400 Subject: newbie having difficulty with inet_res:nslookup In-Reply-To: <45ae90370909130827y16daaef9ibe9fb9558beb9e30@mail.gmail.com> References: <45ae90370909130827y16daaef9ibe9fb9558beb9e30@mail.gmail.com> Message-ID: <45ae90370909130829o523b16e9p69d6e605792eff15@mail.gmail.com> I forgot to say Mac OS X (.5), erlang 12B I think (what's the way to have the shell give the erlang version?). On Sun, Sep 13, 2009 at 11:27 AM, Jeff Macdonald wrote: > Hi All, > > I wrote a fairly simple program to look up MX records for domains and > then look up the exhanger's A and AAAA records. I'm trying to see how > many MX's have IPv6 capable hosts. > > My program simply reads a file which contains a domain per line and > spawns a processes/thread/child for each domain. The spawned process > then does the MX, A and AAAA lookups and prints the results to STDOUT > as a erlang tupple. This seems to work fine for a small sample set of > 10 rows but when trying 100 rows or more I get: > > =ERROR REPORT==== 12-Sep-2009::23:06:41 === > Error in process <0.11161.0> with exit value: > {function_clause,[{lists,map,[#Fun,emfile]},{mx,do_lookups,1}]} > > My program is below. Usage is: > > mx:init({192,168, 1, 1}). (replace that tupple with the IP of your DNS server) > mx:process_file("file-with-domains"). > > I currently can't share a list of domains because it is customer data, > but I sure there is some lists somewhere. > > I suspect that I'm overloading the DNS server at home. That is just > whatever is running in my linksys router. Any other suggestions about > the code welcomed. I know I could probably run the A and AAAA queries > in parallel. I also can probably skip queries when the data is present > in the MX results. For now, I'm just trying to get the basics. > > TIA > > > > %% Author: jeff > %% Created: Sep 10, 2009 > %% Description: TODO: Add description to mx > -module(mx). > > %% > %% Include files > %% > -include_lib("kernel/src/inet_dns.hrl"). > > %% > %% Exported Functions > %% > -export([init/1, process_file/1, do_lookups/1]). > > %% > %% API Functions > %% > > init(Resolver) ?-> > ? ? ? ?inet_db:add_ns(Resolver). > > process_file(Filename) ?-> > ? ? ? ?{ok,Fh}=file:open(Filename, [read]), > ? ? ? ?read_file(Fh). > > do_lookups(Domain) ? ? ?-> > ? ? ? ?MX_list=lookup_mx(Domain), > ? ? ? ?Address=lists:map(fun lookup_address/1, MX_list), > ? ?io:format("~p:~n\t~p~n~n", [Domain, Address]). > > %% > %% Local Functions > %% > > read_file(Fh) ? -> > ? ? ? ?case io:get_line(Fh, "") of > ? ? ? ? ? ? ? ?eof ? ? ? ? ? ? ? ? ? ? ? ? ? ? -> done; > ? ? ? ? ? ? ? ?{error, Reason} -> {error, Reason}; > ? ? ? ? ? ? ? ?Data ? ? ? ? ? ? ? ? ? ?-> > ? ? ? ? ? ? ? ? ? ? ? ?process_data(string:strip(Data,right,$\n)), > ? ? ? ? ? ? ? ? ? ? ? ?read_file(Fh) > ? ? ? ?end. > > process_data(Data) ? ? ?-> > ? ? ? ?spawn(mx, do_lookups, [Data]). > > > lookup_mx(Domain) ? ? ? -> > ? ? ? ?case inet_res:nslookup(Domain, 1, mx) of > ? ? ? ? ? ? ? ?{error, Reason} -> Reason; > ? ? ? ? ? ? ? ?{ok, #dns_rec{ anlist = Ans }} -> lists:map(fun parse_mx/1, Ans) > ? ? ? ?end. > > lookup_address(Host) ? ?-> > ? ? ? ?{ Host, {a, lookup_address(Host, a)}, > ? ? ? ? ? ? ? ? ? ? ? ?{aaaa, lookup_address(Host, aaaa)} }. > > lookup_address(Host, a) -> > ? ? ? ?case inet_res:nslookup(Host, 1, a) of > ? ? ? ? ? ? ? ?{error, Reason} -> Reason; > ? ? ? ? ? ? ? ?{ok, #dns_rec{ anlist = Ans }} -> lists:map(fun parse_a/1, Ans) > ? ? ? ?end; > > lookup_address(Host, aaaa) ? ? ?-> > ? ? ? ?case inet_res:nslookup(Host, 1, aaaa) of > ? ? ? ? ? ? ? ?{error, Reason} -> Reason; > ? ? ? ? ? ? ? ?{ok, #dns_rec{ anlist = Ans }} -> Host,lists:map(fun parse_aaaa/1, Ans) > ? ? ? ?end. > > parse_mx( RR ) ?-> > ? ? ? ?#dns_rr{ type=mx, data={ _Pref, Exchange} } = RR, > ? ? ? ?Exchange. > > parse_a( RR ) ? -> > ? ? ? ?#dns_rr{ type=a, data=Address } = RR, > ? ? ? ?Address. > > parse_aaaa( RR ) ? ? ? ?-> > ? ? ? ?#dns_rr{ type=aaaa, data=Address } = RR, > ? ? ? ?Address. > > > > > -- > Jeff Macdonald > Ayer, MA > -- Jeff Macdonald Ayer, MA From gleber.p@REDACTED Sun Sep 13 17:43:53 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Sun, 13 Sep 2009 17:43:53 +0200 Subject: [erlang-questions] Re: newbie having difficulty with inet_res:nslookup In-Reply-To: <45ae90370909130829o523b16e9p69d6e605792eff15@mail.gmail.com> References: <45ae90370909130827y16daaef9ibe9fb9558beb9e30@mail.gmail.com> <45ae90370909130829o523b16e9p69d6e605792eff15@mail.gmail.com> Message-ID: <14f0e3620909130843t511140f3p199f9caea62b9939@mail.gmail.com> Hello. Looks like function inet_res:nslookup/3 returns {error, emfile}, so function lookup_mx/1 returns atom 'emfile', which is passed in function do_lookups/1 to the lists:map/2 as a second parameter. But lists:map/2 accepts only lists as second argument. Error 'emfie' means that there is too many opened file descriptors, i.e. there are too many opened sockets. I'd suggest rewriting lookup_mx/1 in "fail fast" way, i.e.: lookup_mx(Domain) -> {ok, #dns_rec{ anlist = Ans }} = inet_res:nslookup(Domain, 1, mx), lists:map(fun parse_mx/1, Ans). Best, Gleb Peregud On Sun, Sep 13, 2009 at 17:29, Jeff Macdonald wrote: > I forgot to say Mac OS X (.5), erlang 12B I think (what's the way to > have the shell give the erlang version?). > > > On Sun, Sep 13, 2009 at 11:27 AM, Jeff Macdonald wrote: >> Hi All, >> >> I wrote a fairly simple program to look up MX records for domains and >> then look up the exhanger's A and AAAA records. I'm trying to see how >> many MX's have IPv6 capable hosts. >> >> My program simply reads a file which contains a domain per line and >> spawns a processes/thread/child for each domain. The spawned process >> then does the MX, A and AAAA lookups and prints the results to STDOUT >> as a erlang tupple. This seems to work fine for a small sample set of >> 10 rows but when trying 100 rows or more I get: >> >> =ERROR REPORT==== 12-Sep-2009::23:06:41 === >> Error in process <0.11161.0> with exit value: >> {function_clause,[{lists,map,[#Fun,emfile]},{mx,do_lookups,1}]} >> >> My program is below. Usage is: >> >> mx:init({192,168, 1, 1}). (replace that tupple with the IP of your DNS server) >> mx:process_file("file-with-domains"). >> >> I currently can't share a list of domains because it is customer data, >> but I sure there is some lists somewhere. >> >> I suspect that I'm overloading the DNS server at home. That is just >> whatever is running in my linksys router. Any other suggestions about >> the code welcomed. I know I could probably run the A and AAAA queries >> in parallel. I also can probably skip queries when the data is present >> in the MX results. For now, I'm just trying to get the basics. >> >> TIA >> >> >> >> %% Author: jeff >> %% Created: Sep 10, 2009 >> %% Description: TODO: Add description to mx >> -module(mx). >> >> %% >> %% Include files >> %% >> -include_lib("kernel/src/inet_dns.hrl"). >> >> %% >> %% Exported Functions >> %% >> -export([init/1, process_file/1, do_lookups/1]). >> >> %% >> %% API Functions >> %% >> >> init(Resolver) ?-> >> ? ? ? ?inet_db:add_ns(Resolver). >> >> process_file(Filename) ?-> >> ? ? ? ?{ok,Fh}=file:open(Filename, [read]), >> ? ? ? ?read_file(Fh). >> >> do_lookups(Domain) ? ? ?-> >> ? ? ? ?MX_list=lookup_mx(Domain), >> ? ? ? ?Address=lists:map(fun lookup_address/1, MX_list), >> ? ?io:format("~p:~n\t~p~n~n", [Domain, Address]). >> >> %% >> %% Local Functions >> %% >> >> read_file(Fh) ? -> >> ? ? ? ?case io:get_line(Fh, "") of >> ? ? ? ? ? ? ? ?eof ? ? ? ? ? ? ? ? ? ? ? ? ? ? -> done; >> ? ? ? ? ? ? ? ?{error, Reason} -> {error, Reason}; >> ? ? ? ? ? ? ? ?Data ? ? ? ? ? ? ? ? ? ?-> >> ? ? ? ? ? ? ? ? ? ? ? ?process_data(string:strip(Data,right,$\n)), >> ? ? ? ? ? ? ? ? ? ? ? ?read_file(Fh) >> ? ? ? ?end. >> >> process_data(Data) ? ? ?-> >> ? ? ? ?spawn(mx, do_lookups, [Data]). >> >> >> lookup_mx(Domain) ? ? ? -> >> ? ? ? ?case inet_res:nslookup(Domain, 1, mx) of >> ? ? ? ? ? ? ? ?{error, Reason} -> Reason; >> ? ? ? ? ? ? ? ?{ok, #dns_rec{ anlist = Ans }} -> lists:map(fun parse_mx/1, Ans) >> ? ? ? ?end. >> >> lookup_address(Host) ? ?-> >> ? ? ? ?{ Host, {a, lookup_address(Host, a)}, >> ? ? ? ? ? ? ? ? ? ? ? ?{aaaa, lookup_address(Host, aaaa)} }. >> >> lookup_address(Host, a) -> >> ? ? ? ?case inet_res:nslookup(Host, 1, a) of >> ? ? ? ? ? ? ? ?{error, Reason} -> Reason; >> ? ? ? ? ? ? ? ?{ok, #dns_rec{ anlist = Ans }} -> lists:map(fun parse_a/1, Ans) >> ? ? ? ?end; >> >> lookup_address(Host, aaaa) ? ? ?-> >> ? ? ? ?case inet_res:nslookup(Host, 1, aaaa) of >> ? ? ? ? ? ? ? ?{error, Reason} -> Reason; >> ? ? ? ? ? ? ? ?{ok, #dns_rec{ anlist = Ans }} -> Host,lists:map(fun parse_aaaa/1, Ans) >> ? ? ? ?end. >> >> parse_mx( RR ) ?-> >> ? ? ? ?#dns_rr{ type=mx, data={ _Pref, Exchange} } = RR, >> ? ? ? ?Exchange. >> >> parse_a( RR ) ? -> >> ? ? ? ?#dns_rr{ type=a, data=Address } = RR, >> ? ? ? ?Address. >> >> parse_aaaa( RR ) ? ? ? ?-> >> ? ? ? ?#dns_rr{ type=aaaa, data=Address } = RR, >> ? ? ? ?Address. >> >> >> >> >> -- >> Jeff Macdonald >> Ayer, MA >> > > > > -- > Jeff Macdonald > Ayer, MA > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From kiszl@REDACTED Sun Sep 13 17:49:14 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sun, 13 Sep 2009 17:49:14 +0200 Subject: [erlang-questions] Re: newbie having difficulty with inet_res:nslookup In-Reply-To: <45ae90370909130829o523b16e9p69d6e605792eff15@mail.gmail.com> References: <45ae90370909130827y16daaef9ibe9fb9558beb9e30@mail.gmail.com> <45ae90370909130829o523b16e9p69d6e605792eff15@mail.gmail.com> Message-ID: <4AAD147A.3010407@tmit.bme.hu> Hi, You don't handle errors properly in lookup_mx. One of your domain lookups result in an {error, emfile}, from which you return the atom 'emfile' to the call in do_lookups. This will cause error, when fed to the lists:map/2 function. Either let lookup_mx crash on error, or handle the error case in do_lookups as well. To check your version, type: 1> erlang:system_info(otp_release). "R13B01" Z. Jeff Macdonald wrote: > I forgot to say Mac OS X (.5), erlang 12B I think (what's the way to > have the shell give the erlang version?). > > > On Sun, Sep 13, 2009 at 11:27 AM, Jeff Macdonald wrote: > >> Hi All, >> >> I wrote a fairly simple program to look up MX records for domains and >> then look up the exhanger's A and AAAA records. I'm trying to see how >> many MX's have IPv6 capable hosts. >> >> My program simply reads a file which contains a domain per line and >> spawns a processes/thread/child for each domain. The spawned process >> then does the MX, A and AAAA lookups and prints the results to STDOUT >> as a erlang tupple. This seems to work fine for a small sample set of >> 10 rows but when trying 100 rows or more I get: >> >> =ERROR REPORT==== 12-Sep-2009::23:06:41 === >> Error in process <0.11161.0> with exit value: >> {function_clause,[{lists,map,[#Fun,emfile]},{mx,do_lookups,1}]} >> >> My program is below. Usage is: >> >> mx:init({192,168, 1, 1}). (replace that tupple with the IP of your DNS server) >> mx:process_file("file-with-domains"). >> >> I currently can't share a list of domains because it is customer data, >> but I sure there is some lists somewhere. >> >> I suspect that I'm overloading the DNS server at home. That is just >> whatever is running in my linksys router. Any other suggestions about >> the code welcomed. I know I could probably run the A and AAAA queries >> in parallel. I also can probably skip queries when the data is present >> in the MX results. For now, I'm just trying to get the basics. >> >> TIA >> >> >> >> %% Author: jeff >> %% Created: Sep 10, 2009 >> %% Description: TODO: Add description to mx >> -module(mx). >> >> %% >> %% Include files >> %% >> -include_lib("kernel/src/inet_dns.hrl"). >> >> %% >> %% Exported Functions >> %% >> -export([init/1, process_file/1, do_lookups/1]). >> >> %% >> %% API Functions >> %% >> >> init(Resolver) -> >> inet_db:add_ns(Resolver). >> >> process_file(Filename) -> >> {ok,Fh}=file:open(Filename, [read]), >> read_file(Fh). >> >> do_lookups(Domain) -> >> MX_list=lookup_mx(Domain), >> Address=lists:map(fun lookup_address/1, MX_list), >> io:format("~p:~n\t~p~n~n", [Domain, Address]). >> >> %% >> %% Local Functions >> %% >> >> read_file(Fh) -> >> case io:get_line(Fh, "") of >> eof -> done; >> {error, Reason} -> {error, Reason}; >> Data -> >> process_data(string:strip(Data,right,$\n)), >> read_file(Fh) >> end. >> >> process_data(Data) -> >> spawn(mx, do_lookups, [Data]). >> >> >> lookup_mx(Domain) -> >> case inet_res:nslookup(Domain, 1, mx) of >> {error, Reason} -> Reason; >> {ok, #dns_rec{ anlist = Ans }} -> lists:map(fun parse_mx/1, Ans) >> end. >> >> lookup_address(Host) -> >> { Host, {a, lookup_address(Host, a)}, >> {aaaa, lookup_address(Host, aaaa)} }. >> >> lookup_address(Host, a) -> >> case inet_res:nslookup(Host, 1, a) of >> {error, Reason} -> Reason; >> {ok, #dns_rec{ anlist = Ans }} -> lists:map(fun parse_a/1, Ans) >> end; >> >> lookup_address(Host, aaaa) -> >> case inet_res:nslookup(Host, 1, aaaa) of >> {error, Reason} -> Reason; >> {ok, #dns_rec{ anlist = Ans }} -> Host,lists:map(fun parse_aaaa/1, Ans) >> end. >> >> parse_mx( RR ) -> >> #dns_rr{ type=mx, data={ _Pref, Exchange} } = RR, >> Exchange. >> >> parse_a( RR ) -> >> #dns_rr{ type=a, data=Address } = RR, >> Address. >> >> parse_aaaa( RR ) -> >> #dns_rr{ type=aaaa, data=Address } = RR, >> Address. >> >> >> >> >> -- >> Jeff Macdonald >> Ayer, MA >> >> > > > > From steven.charles.davis@REDACTED Sun Sep 13 18:03:47 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 13 Sep 2009 09:03:47 -0700 (PDT) Subject: SCTP accept? Message-ID: The documented example for gen_sctp appears to run only in one process. The order being gen_sctp:open, then gen_sctp:listen(Socket, true), then goes straight into gen_sctp:recv... What happened to the "accept" step? i.e. how do you spin off a new client connection into its own process? Any comment gratefully received! /sd From praveen.ray@REDACTED Sun Sep 13 18:57:22 2009 From: praveen.ray@REDACTED (Praveen Ray) Date: Sun, 13 Sep 2009 12:57:22 -0400 Subject: extra parameters to -module Message-ID: This might be a basic erlang questions but I'm not sure..while reading ewgi source, I came across this line: -module(ewgi_yaws, [Appl]). Does anyone know what does the [Appl] parameter mean? Where is the reference documentation for 'module' directive? thanks From koops.j@REDACTED Sun Sep 13 19:01:47 2009 From: koops.j@REDACTED (Jeroen Koops) Date: Sun, 13 Sep 2009 19:01:47 +0200 Subject: [erlang-questions] extra parameters to -module In-Reply-To: References: Message-ID: <331a9abb0909131001m5c581f9awd6339740a7be92ed@mail.gmail.com> Hi Praveen, This is a parametrized module. Parametrized modules are a non-standard but widely used feature of Erlang. More information can be found at: http://ftp.sunet.se/pub/lang/erlang/workshop/2003/paper/p29-carlsson.pdf Regards, Jeroen On Sun, Sep 13, 2009 at 6:57 PM, Praveen Ray wrote: > This might be a basic erlang questions but I'm not sure..while reading ewgi > source, I came across this line: > > -module(ewgi_yaws, [Appl]). > > Does anyone know what does the [Appl] parameter mean? Where is the > reference > documentation for 'module' directive? > > thanks > From tsuraan@REDACTED Sun Sep 13 19:08:00 2009 From: tsuraan@REDACTED (tsuraan) Date: Sun, 13 Sep 2009 12:08:00 -0500 Subject: status of shared or hybrid heap? Message-ID: <84fb38e30909131008k62cd1f6cx585f09b93ee6d6d@mail.gmail.com> Erlang has support for -shared and -hybrid, which seem to be different ways of not having processes have totally separate heaps. Where can I find documentation on these options? Also, is there any plan for making one of the heap sharing options default at any point? I've run into problems with blindly applying pmap with closures with large amounts of data, and having my program run out of memory. Shared heaps would probably make this a lot less likely. From richardc@REDACTED Sun Sep 13 20:54:38 2009 From: richardc@REDACTED (Richard Carlsson) Date: Sun, 13 Sep 2009 20:54:38 +0200 Subject: [erlang-questions] status of shared or hybrid heap? In-Reply-To: <84fb38e30909131008k62cd1f6cx585f09b93ee6d6d@mail.gmail.com> References: <84fb38e30909131008k62cd1f6cx585f09b93ee6d6d@mail.gmail.com> Message-ID: <4AAD3FEE.9080109@it.uu.se> tsuraan wrote: > Erlang has support for -shared and -hybrid, which seem to be different > ways of not having processes have totally separate heaps. Where can I > find documentation on these options? Also, is there any plan for > making one of the heap sharing options default at any point? I've run > into problems with blindly applying pmap with closures with large > amounts of data, and having my program run out of memory. Shared > heaps would probably make this a lot less likely. To my knowledge, both variants have been broken (can segfault) since the SMP support was added to Erlang. Lack of manpower prevented these alternative (and still experimental) memory management models from being kept up to date. SMP required a lot of changes, and maintaining the shared/hybrid memory models is a highly specialized task; furthermore, the person who knew most about it has moved on to greener fields. A somewhat sad situation, but on the upside we did get good SMP support under the standard memory model. Also note that the shared heap model had its own problems, mostly to do with long global pause times during garbage collection, which is generally a big no-no in server applications. The hybrid model was more promising, but as mentioned, it is not in working order. /Richard From macfisherman@REDACTED Sun Sep 13 21:17:43 2009 From: macfisherman@REDACTED (Jeff Macdonald) Date: Sun, 13 Sep 2009 15:17:43 -0400 Subject: [erlang-questions] Re: newbie having difficulty with inet_res:nslookup In-Reply-To: <14f0e3620909130843t511140f3p199f9caea62b9939@mail.gmail.com> References: <45ae90370909130827y16daaef9ibe9fb9558beb9e30@mail.gmail.com> <45ae90370909130829o523b16e9p69d6e605792eff15@mail.gmail.com> <14f0e3620909130843t511140f3p199f9caea62b9939@mail.gmail.com> Message-ID: <45ae90370909131217r100c5398v451f14f39f6121d3@mail.gmail.com> On Sun, Sep 13, 2009 at 11:43 AM, Gleb Peregud wrote: > I'd suggest rewriting lookup_mx/1 in "fail fast" way, i.e.: > > lookup_mx(Domain) ? ? ? -> > ? ? ? {ok, #dns_rec{ anlist = Ans }} = inet_res:nslookup(Domain, 1, mx), > ? ? ? lists:map(fun parse_mx/1, Ans). excellent. Thanks. Next is figuring out how to increase file handles in Mac OS X and have Erlang use them. -- Jeff Macdonald Ayer, MA From john.hughes@REDACTED Sun Sep 13 22:27:42 2009 From: john.hughes@REDACTED (John Hughes) Date: Sun, 13 Sep 2009 22:27:42 +0200 Subject: [erlang-questions] status of shared or hybrid heap? In-Reply-To: <4AAD3FEE.9080109@it.uu.se> References: <84fb38e30909131008k62cd1f6cx585f09b93ee6d6d@mail.gmail.com> <4AAD3FEE.9080109@it.uu.se> Message-ID: <884BF39470C04B8A8BF299764140B0CA@HALL> > tsuraan wrote: >> Erlang has support for -shared and -hybrid, which seem to be different >> ways of not having processes have totally separate heaps. Where can I >> find documentation on these options? Also, is there any plan for >> making one of the heap sharing options default at any point? I've run >> into problems with blindly applying pmap with closures with large >> amounts of data, and having my program run out of memory. Shared >> heaps would probably make this a lot less likely. > Sounds like a problem I run into a lot: when closures are copied to a new heap, then all sharing is lost... and there can be a lot of sharing in the data structures representing a closure. There's been discussion of a new version of term_to_binary and binary_to_term that would preserve sharing, but so far no implementation. John From chandrashekhar.mullaparthi@REDACTED Mon Sep 14 00:03:52 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sun, 13 Sep 2009 23:03:52 +0100 Subject: [erlang-questions] Re: newbie having difficulty with inet_res:nslookup In-Reply-To: <45ae90370909131217r100c5398v451f14f39f6121d3@mail.gmail.com> References: <45ae90370909130827y16daaef9ibe9fb9558beb9e30@mail.gmail.com> <45ae90370909130829o523b16e9p69d6e605792eff15@mail.gmail.com> <14f0e3620909130843t511140f3p199f9caea62b9939@mail.gmail.com> <45ae90370909131217r100c5398v451f14f39f6121d3@mail.gmail.com> Message-ID: 2009/9/13 Jeff Macdonald > On Sun, Sep 13, 2009 at 11:43 AM, Gleb Peregud wrote: > > > I'd suggest rewriting lookup_mx/1 in "fail fast" way, i.e.: > > > > lookup_mx(Domain) -> > > {ok, #dns_rec{ anlist = Ans }} = inet_res:nslookup(Domain, 1, mx), > > lists:map(fun parse_mx/1, Ans). > > excellent. Thanks. Next is figuring out how to increase file handles > in Mac OS X and have Erlang use them. > > You could increase it using 'ulimit' before you start your erlang node. cheers Chandru From eric.l.2046@REDACTED Mon Sep 14 05:29:15 2009 From: eric.l.2046@REDACTED (Eric Liang) Date: Mon, 14 Sep 2009 11:29:15 +0800 Subject: [erlang-questions] Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] In-Reply-To: <5c493e530909130705t6f7afd4dubf5fe95a30485129@mail.gmail.com> References: <4AA8CB0A.1020800@gmail.com> <5c493e530909110009m43c0c47emd49d86b477ab6a88@mail.gmail.com> <4AAA2EAE.8030408@gmail.com> <4AAA7640.8090700@gmail.com> <5c493e530909130705t6f7afd4dubf5fe95a30485129@mail.gmail.com> Message-ID: <4AADB88B.8090700@gmail.com> Ngoc Dao wrote: > After many try-and-errors I suspect that Distel currently can only > interpret a module if its .beam file is in the same directory with its > .erl file, or in ../ebin. Is this true? How to overcome this problem? > > Maybe add_path will help you. @spec code:add_patha(Dir) => true | {error, bad_directory} And you can check this by code:get_path(). > On Sat, Sep 12, 2009 at 1:09 AM, Eric Liang wrote: > >> Eric Liang wrote: >> >> Ngoc Dao wrote: >> >> I tried Distel from trunk yesterday and it did not work. The version >> from download did work: >> http://code.google.com/p/distel/downloads/list. >> >> >> >> Thanks Ngoc, it works. But, still can't debug and profile, like Bill >> Clementson's introduction. >> >> When I type C-c C-d i, it looks like is interpreting forever, and can't set >> breakpoints to debug. I've checked the archive of this mailling list, there >> is only one unresolved thread here: >> >> http://groups.google.com/group/erlang-programming/browse_thread/thread/c34d5245998948c6 >> >> Do you have more suggestions? TIA. >> >> I think I've resolved this. Just load module (C-c C-d L) before being >> interpreted. So simple. :-[ >> >> In Distel, load module will tell you nothing more than "load: [module >> test_ring]", except the first time is successful, in which scenario, it will >> tell you "Successfully uploaded backend modules into node". So, It's >> difficulty to know whether or not the module was loaded successfully. >> >> And, one module can be loaded only if it's beam has been compiled by the >> flag: debug_info. >> >> >> On Thu, Sep 10, 2009 at 6:46 PM, Eric Liang wrote: >> >> >> Hi all, >> To build a erlang IDE, I've tried Distel in emacs22. but unfortunately it >> does not work well. What I've done is exactly in accordance with Bill >> Clementson's blog post: >> >> http://bc.tech.coop/blog/070528.html >> >> They are: >> >> Download the latest code from : http://distel.googlecode.com/svn/trunk/ >> Modify the .emacs. I did confirm the load-path and distel-dir settings are >> correct. Here is it: >> >> ;; This is needed for Erlang mode setup >> (setq erlang-root-dir "/usr/local/lib/erlang") >> (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.5.1/emacs" >> load-path)) >> (setq exec-path (cons "/usr/local/lib/erlang/bin" exec-path)) >> (require 'erlang-start) >> >> ;; This is needed for Distel setup >> (let ((distel-dir "/Users/bc/Projects/distel/elisp")) >> (unless (member distel-dir load-path) >> ;; Add distel-dir to the end of load-path >> (setq load-path (append load-path (list distel-dir))))) >> >> (require 'distel) >> (distel-setup) >> >> ;; Some Erlang customizations >> (add-hook 'erlang-mode-hook >> (lambda () >> ;; when starting an Erlang shell in Emacs, default in the node name >> (setq inferior-erlang-machine-options '("-sname" "emacs")) >> ;; add Erlang functions to an imenu menu >> (imenu-add-to-menubar "imenu"))) >> >> ;; A number of the erlang-extended-mode key bindings are useful in the shell >> too >> (defconst distel-shell-keys >> '(("\C-\M-i" erl-complete) >> ("\M-?" erl-complete) >> ("\M-." erl-find-source-under-point) >> ("\M-," erl-find-source-unwind) >> ("\M-*" erl-find-source-unwind) >> ) >> "Additional keys to bind when in Erlang shell.") >> >> (add-hook 'erlang-shell-mode-hook >> (lambda () >> ;; add some Distel bindings to the Erlang shell >> (dolist (spec distel-shell-keys) >> (define-key erlang-shell-mode-map (car spec) (cadr spec))))) >> >> When I open a .erl file, it works well and Distel dose exist in the Erlang >> tag of the emacs' menubar. But when I type C-M-i behind the word "cr" to >> complete the function name,the emacs received this message: >> >> Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel >> modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] >> >> I checked the code in Distel, it looks like that it will call completions() >> which will call the get() by the argument:A an empty string("") in >> otp_doc.erl. The function get() will call to_list() on the argument:A,which >> will get the error: >> >> *** ERROR: Shell process terminated! *** >> >> =ERROR REPORT==== 8-Sep-2009::15:02:42 === >> Error in process <0.66.0> on node 'emacs@REDACTED' with exit value: >> {undef,[{lib,format_exception,[4,error,undef,[{erl_internal,bif,[to_list,1]},{erl_lint,expr,3},{erl_lint,exprs,3},{erl_lint,exprs_opt,3},{erl_eval,check_command,2},{shell,exprs,6},{shell,eval_exprs,6},{shell,eval_loop.. >> >> I guess this is the reason for the completion failure, but have no idea how >> to fix it. Can anyone here give me some advice? >> >> As a newbie in Erlang, it's very possible I've stepped in the wrong >> way.However, any suggestions will be appreciated. >> >> Thanks in advance. >> >> Eric >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From ngocdaothanh@REDACTED Mon Sep 14 07:12:22 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Mon, 14 Sep 2009 14:12:22 +0900 Subject: [erlang-questions] Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] In-Reply-To: <4AADB88B.8090700@gmail.com> References: <4AA8CB0A.1020800@gmail.com> <5c493e530909110009m43c0c47emd49d86b477ab6a88@mail.gmail.com> <4AAA2EAE.8030408@gmail.com> <4AAA7640.8090700@gmail.com> <5c493e530909130705t6f7afd4dubf5fe95a30485129@mail.gmail.com> <4AADB88B.8090700@gmail.com> Message-ID: <5c493e530909132212r5b9c87a1td3b67a95ee379bec@mail.gmail.com> My project structure: ebin .beam files src web .erl files I do the following: * Setup cookie with ~/.erlang.cookie file * From console (not from Distel), sstart a node, paths are set with -pa option to the erl command * From Emacs open a src/web/my_module.erl file * From Distel, connect to the node, the connection is confirmed by listing all processes on the node * From Distel, load my_module * From Distel, intepret the module, this fails with "forever" message "Interpreting: my_module" * Breakpoint does not work because the module is not intepreted I'm using Distel 4.03 from http://code.google.com/p/distel/ with the Erlang R13B01 on Snow Leopard. On Mon, Sep 14, 2009 at 12:29 PM, Eric Liang wrote: > Ngoc Dao wrote: > > After many try-and-errors I suspect that Distel currently can only > interpret a module if its .beam file is in the same directory with its > .erl file, or in ../ebin. Is this true? How to overcome this problem? > > > > Maybe add_path will help you. > > @spec code:add_patha(Dir) => true | {error, bad_directory} > > And you can check this by code:get_path(). > > On Sat, Sep 12, 2009 at 1:09 AM, Eric Liang wrote: > > > Eric Liang wrote: > > Ngoc Dao wrote: > > I tried Distel from trunk yesterday and it did not work. The version > from download did work: > http://code.google.com/p/distel/downloads/list. > > > > Thanks Ngoc, it works. But, still can't debug and profile, like Bill > Clementson's introduction. > > When I type C-c C-d i, it looks like is interpreting forever, and can't set > breakpoints to debug. I've checked the archive of this mailling list, there > is only one unresolved thread here: > > http://groups.google.com/group/erlang-programming/browse_thread/thread/c34d5245998948c6 > > Do you have more suggestions? TIA. > > I think I've resolved this. Just load module (C-c C-d L) before being > interpreted. So simple. :-[ > > In Distel, load module will tell you nothing more than "load: [module > test_ring]", except the first time is successful, in which scenario, it will > tell you "Successfully uploaded backend modules into node". So, It's > difficulty to? know? whether or not the module was loaded? successfully. > > And, one module can be loaded only if it's beam has been compiled by the > flag: debug_info. > > > On Thu, Sep 10, 2009 at 6:46 PM, Eric Liang wrote: > > > Hi all, > To build a erlang IDE, I've tried Distel in emacs22. but unfortunately it > does not work well. What I've done is exactly in accordance with Bill > Clementson's blog post: > > http://bc.tech.coop/blog/070528.html > > They are: > > Download the latest code from :? http://distel.googlecode.com/svn/trunk/ > Modify the .emacs. I did confirm the load-path and distel-dir settings are > correct. Here is it: > > ;; This is needed for Erlang mode setup > (setq erlang-root-dir "/usr/local/lib/erlang") > (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.5.1/emacs" > load-path)) > (setq exec-path (cons "/usr/local/lib/erlang/bin" exec-path)) > (require 'erlang-start) > > ;; This is needed for Distel setup > (let ((distel-dir "/Users/bc/Projects/distel/elisp")) > (unless (member distel-dir load-path) > ;; Add distel-dir to the end of load-path > (setq load-path (append load-path (list distel-dir))))) > > (require 'distel) > (distel-setup) > > ;; Some Erlang customizations > (add-hook 'erlang-mode-hook > (lambda () > ;; when starting an Erlang shell in Emacs, default in the node name > (setq inferior-erlang-machine-options '("-sname" "emacs")) > ;; add Erlang functions to an imenu menu > (imenu-add-to-menubar "imenu"))) > > ;; A number of the erlang-extended-mode key bindings are useful in the shell > too > (defconst distel-shell-keys > '(("\C-\M-i" erl-complete) > ("\M-?" erl-complete) > ("\M-." erl-find-source-under-point) > ("\M-," erl-find-source-unwind) > ("\M-*" erl-find-source-unwind) > ) > "Additional keys to bind when in Erlang shell.") > > (add-hook 'erlang-shell-mode-hook > (lambda () > ;; add some Distel bindings to the Erlang shell > (dolist (spec distel-shell-keys) > (define-key erlang-shell-mode-map (car spec) (cadr spec))))) > > When I open a .erl file, it works well and Distel dose exist in the Erlang > tag of the emacs' menubar. But when I type C-M-i behind the word "cr" to > complete the function name,the emacs received this message: > > Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel > modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] > > I checked the code in Distel, it looks like that it will call completions() > which will call the get() by the argument:A? an empty string("")? in > otp_doc.erl.? The function get() will call to_list() on the argument:A,which > will get the error: > > *** ERROR: Shell process terminated! *** > > =ERROR REPORT==== 8-Sep-2009::15:02:42 === > Error in process <0.66.0> on node 'emacs@REDACTED' with exit value: > {undef,[{lib,format_exception,[4,error,undef,[{erl_internal,bif,[to_list,1]},{erl_lint,expr,3},{erl_lint,exprs,3},{erl_lint,exprs_opt,3},{erl_eval,check_command,2},{shell,exprs,6},{shell,eval_exprs,6},{shell,eval_loop.. > > I guess this is the reason for the completion failure, but have no idea how > to fix it. Can anyone here give me some advice? > > As a newbie in Erlang, it's very possible I've stepped in the wrong > way.However, any suggestions will be appreciated. > > Thanks in advance. > > Eric > > > > > From eric.l.2046@REDACTED Mon Sep 14 09:26:52 2009 From: eric.l.2046@REDACTED (Eric Liang) Date: Mon, 14 Sep 2009 15:26:52 +0800 Subject: [erlang-questions] Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] In-Reply-To: <5c493e530909132212r5b9c87a1td3b67a95ee379bec@mail.gmail.com> References: <4AA8CB0A.1020800@gmail.com> <5c493e530909110009m43c0c47emd49d86b477ab6a88@mail.gmail.com> <4AAA2EAE.8030408@gmail.com> <4AAA7640.8090700@gmail.com> <5c493e530909130705t6f7afd4dubf5fe95a30485129@mail.gmail.com> <4AADB88B.8090700@gmail.com> <5c493e530909132212r5b9c87a1td3b67a95ee379bec@mail.gmail.com> Message-ID: <4AADF03C.8030505@gmail.com> Ngoc Dao wrote: > My project structure: > ebin > .beam files > src > web > .erl files > > I do the following: > * Setup cookie with ~/.erlang.cookie file > * From console (not from Distel), sstart a node, paths are set with > -pa option to the erl command > * From Emacs open a src/web/my_module.erl file > * From Distel, connect to the node, the connection is confirmed by > listing all processes on the node > * From Distel, load my_module > * From Distel, intepret the module, this fails with "forever" message > "Interpreting: my_module" > * Breakpoint does not work because the module is not intepreted > > I'm using Distel 4.03 from http://code.google.com/p/distel/ with the > Erlang R13B01 on Snow Leopard. > > The module should be compiled with the flag: debug_info if you want to load it, IMO. And you may check this first. Whether or not you've noticed the message I've shown before in this thread, well, I think the problems are very like. > On Mon, Sep 14, 2009 at 12:29 PM, Eric Liang wrote: > >> Ngoc Dao wrote: >> >> After many try-and-errors I suspect that Distel currently can only >> interpret a module if its .beam file is in the same directory with its >> .erl file, or in ../ebin. Is this true? How to overcome this problem? >> >> >> >> Maybe add_path will help you. >> >> @spec code:add_patha(Dir) => true | {error, bad_directory} >> >> And you can check this by code:get_path(). >> >> On Sat, Sep 12, 2009 at 1:09 AM, Eric Liang wrote: >> >> >> Eric Liang wrote: >> >> Ngoc Dao wrote: >> >> I tried Distel from trunk yesterday and it did not work. The version >> from download did work: >> http://code.google.com/p/distel/downloads/list. >> >> >> >> Thanks Ngoc, it works. But, still can't debug and profile, like Bill >> Clementson's introduction. >> >> When I type C-c C-d i, it looks like is interpreting forever, and can't set >> breakpoints to debug. I've checked the archive of this mailling list, there >> is only one unresolved thread here: >> >> http://groups.google.com/group/erlang-programming/browse_thread/thread/c34d5245998948c6 >> >> Do you have more suggestions? TIA. >> >> I think I've resolved this. Just load module (C-c C-d L) before being >> interpreted. So simple. :-[ >> >> In Distel, load module will tell you nothing more than "load: [module >> test_ring]", except the first time is successful, in which scenario, it will >> tell you "Successfully uploaded backend modules into node". So, It's >> difficulty to know whether or not the module was loaded successfully. >> >> And, one module can be loaded only if it's beam has been compiled by the >> flag: debug_info. >> >> >> On Thu, Sep 10, 2009 at 6:46 PM, Eric Liang wrote: >> >> >> Hi all, >> To build a erlang IDE, I've tried Distel in emacs22. but unfortunately it >> does not work well. What I've done is exactly in accordance with Bill >> Clementson's blog post: >> >> http://bc.tech.coop/blog/070528.html >> >> They are: >> >> Download the latest code from : http://distel.googlecode.com/svn/trunk/ >> Modify the .emacs. I did confirm the load-path and distel-dir settings are >> correct. Here is it: >> >> ;; This is needed for Erlang mode setup >> (setq erlang-root-dir "/usr/local/lib/erlang") >> (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.5.1/emacs" >> load-path)) >> (setq exec-path (cons "/usr/local/lib/erlang/bin" exec-path)) >> (require 'erlang-start) >> >> ;; This is needed for Distel setup >> (let ((distel-dir "/Users/bc/Projects/distel/elisp")) >> (unless (member distel-dir load-path) >> ;; Add distel-dir to the end of load-path >> (setq load-path (append load-path (list distel-dir))))) >> >> (require 'distel) >> (distel-setup) >> >> ;; Some Erlang customizations >> (add-hook 'erlang-mode-hook >> (lambda () >> ;; when starting an Erlang shell in Emacs, default in the node name >> (setq inferior-erlang-machine-options '("-sname" "emacs")) >> ;; add Erlang functions to an imenu menu >> (imenu-add-to-menubar "imenu"))) >> >> ;; A number of the erlang-extended-mode key bindings are useful in the shell >> too >> (defconst distel-shell-keys >> '(("\C-\M-i" erl-complete) >> ("\M-?" erl-complete) >> ("\M-." erl-find-source-under-point) >> ("\M-," erl-find-source-unwind) >> ("\M-*" erl-find-source-unwind) >> ) >> "Additional keys to bind when in Erlang shell.") >> >> (add-hook 'erlang-shell-mode-hook >> (lambda () >> ;; add some Distel bindings to the Erlang shell >> (dolist (spec distel-shell-keys) >> (define-key erlang-shell-mode-map (car spec) (cadr spec))))) >> >> When I open a .erl file, it works well and Distel dose exist in the Erlang >> tag of the emacs' menubar. But when I type C-M-i behind the word "cr" to >> complete the function name,the emacs received this message: >> >> Unexpected reply: [rex [badrpc [EXIT [undef ([distel rpc_entry (distel >> modules ("cr"))] [rpc -handle_call/3-fun-0- 5])]]]] >> >> I checked the code in Distel, it looks like that it will call completions() >> which will call the get() by the argument:A an empty string("") in >> otp_doc.erl. The function get() will call to_list() on the argument:A,which >> will get the error: >> >> *** ERROR: Shell process terminated! *** >> >> =ERROR REPORT==== 8-Sep-2009::15:02:42 === >> Error in process <0.66.0> on node 'emacs@REDACTED' with exit value: >> {undef,[{lib,format_exception,[4,error,undef,[{erl_internal,bif,[to_list,1]},{erl_lint,expr,3},{erl_lint,exprs,3},{erl_lint,exprs_opt,3},{erl_eval,check_command,2},{shell,exprs,6},{shell,eval_exprs,6},{shell,eval_loop.. >> >> I guess this is the reason for the completion failure, but have no idea how >> to fix it. Can anyone here give me some advice? >> >> As a newbie in Erlang, it's very possible I've stepped in the wrong >> way.However, any suggestions will be appreciated. >> >> Thanks in advance. >> >> Eric >> >> >> >> >> >> Thanks Eric -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From joelr1@REDACTED Mon Sep 14 15:10:15 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 14 Sep 2009 14:10:15 +0100 Subject: large scale deployments and netsplits Message-ID: Has anyone tried deployments with 30-50 nodes? I see large numbers of net splits on EC2, perhaps because I'm using 3-4 global processes and that generates large amounts of traffic. The splits manifest themselves in nodes reporting lost connections to other nodes. I'm also starting 10k bots per node and these establish connections to another cluster of 30+ nodes. On that cluster I'm occasionally seeing a node or two peel off but nothing more than that. Certainly nothing on the scale of splits in the bot cluster where many nodes loose connections to others. --- fastest mac firefox! http://wagerlabs.com From joelr1@REDACTED Mon Sep 14 17:59:22 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 14 Sep 2009 16:59:22 +0100 Subject: [erlang-questions] large scale deployments and netsplits In-Reply-To: <8209f740909140848m4582290dka8f9e0b6e72321c@mail.gmail.com> References: <8209f740909140848m4582290dka8f9e0b6e72321c@mail.gmail.com> Message-ID: Whatever the default is. I don't tweak it. On Sep 14, 2009, at 4:48 PM, Ulf Wiger wrote: > What is your setting for net_ticktime? > Is it the same on all nodes? --- fastest mac firefox! http://wagerlabs.com From james.hague@REDACTED Mon Sep 14 18:22:04 2009 From: james.hague@REDACTED (James Hague) Date: Mon, 14 Sep 2009 11:22:04 -0500 Subject: data sharing is outside the semantics of Erlang, but it sure is useful Message-ID: I've run into several cases where enforcing the sharing of data resulted in a significant memory savings. I'm talking about a reduction in heap size from 60MB to under half that. By "enforcing the sharing of data" I mean making sure that identical elements in a data structure are actually referencing the same locations in memory. This is easy to do in Erlang, because the compiler is very literal: fix_tuple({H, H}) -> {H, H}; ... That ensures that identical looking elements in the tuple are sharing memory locations. But there is absolutely no reason the compiler has to do this. It would be perfectly valid to optimize away the entire function, just returning the original value. Would any existing standard library functions make this nicer? What I really want is to have a gb_trees:insert function that returns {NewTree, InsertedValue} where InsertedValue references existing data (unless it wasn't already in the tree; in that case, InsertedValue is exactly what I passed in). Then I can happily use InsertedValue, knowing data is being shared. James From joelr1@REDACTED Mon Sep 14 18:39:57 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 14 Sep 2009 17:39:57 +0100 Subject: [erlang-questions] large scale deployments and netsplits In-Reply-To: <8209f740909140848m4582290dka8f9e0b6e72321c@mail.gmail.com> References: <8209f740909140848m4582290dka8f9e0b6e72321c@mail.gmail.com> Message-ID: <0784F720-9D6A-44B1-8CCE-637FC8DD0621@gmail.com> On Sep 14, 2009, at 4:48 PM, Ulf Wiger wrote: > What is your setting for net_ticktime? Is there a way to set net_ticktime programmatically, once the node is up and running? --- fastest mac firefox! http://wagerlabs.com From ulf.wiger@REDACTED Mon Sep 14 19:28:43 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 14 Sep 2009 19:28:43 +0200 Subject: [erlang-questions] large scale deployments and netsplits In-Reply-To: <0784F720-9D6A-44B1-8CCE-637FC8DD0621@gmail.com> References: <8209f740909140848m4582290dka8f9e0b6e72321c@mail.gmail.com> <0784F720-9D6A-44B1-8CCE-637FC8DD0621@gmail.com> Message-ID: <4AAE7D4B.7020909@erlang-consulting.com> Joel Reymont wrote: > > On Sep 14, 2009, at 4:48 PM, Ulf Wiger wrote: > >> What is your setting for net_ticktime? > > Is there a way to set net_ticktime programmatically, > once the node is up and running? net_kernel:set_net_ticktime(TickTime [, TransitionPeriod]) The default tick timeout is 45 < T < 75, with TickTime being 60 seconds. See http://www.erlang.org/doc/man/kernel_app.html Normally, and especially for systems where response time is an issue, the default ticktime is too long - not too short. If you have network connections that are not responding within at least 45 seconds, it seems as if you have some rather severe overload on your links. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From james.hague@REDACTED Mon Sep 14 19:53:42 2009 From: james.hague@REDACTED (James Hague) Date: Mon, 14 Sep 2009 12:53:42 -0500 Subject: data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: References: Message-ID: > really want is to have a gb_trees:insert function that... I meant gb_sets. Something like gb_sets:add_shared(Term, Set). James From joelr1@REDACTED Mon Sep 14 20:05:22 2009 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 14 Sep 2009 19:05:22 +0100 Subject: [erlang-questions] large scale deployments and netsplits In-Reply-To: <4AAE7D4B.7020909@erlang-consulting.com> References: <8209f740909140848m4582290dka8f9e0b6e72321c@mail.gmail.com> <0784F720-9D6A-44B1-8CCE-637FC8DD0621@gmail.com> <4AAE7D4B.7020909@erlang-consulting.com> Message-ID: <5C31805E-2D9C-4A36-A6F7-46119D73CFE0@gmail.com> On Sep 14, 2009, at 6:28 PM, Ulf Wiger wrote: > If you have network connections that are not responding within > at least 45 seconds, it seems as if you have some rather > severe overload on your links. I'm attaching the source code to my "flash bot", one that uses socket connections to the server. The bot restarts whenever something goes wrong which happens often. For example, the bot may not connect to the server within 5 seconds or the other side may drop the connection, or the subscription ack may not come on time, etc. I'm starting 50 EC2 instances, 10k bots each (bot code below). I'm running a single Erlang VM per small instance. When a bot node comes up, it starts fetching a server list from Amazon S3 every 10 seconds and updating a local cache. Bots grab the list from the local cache. I also redirect local error_logger on every instance to the "mothership", the node that I'm launching the simulation from and that I'm staring at. Whenever a bot starts up, it will fetch the server list from the local cache, look up 2 globally registered processes (semaphore, sim_histo) and connect to the server. Once connected, bots subscribe to a topic, wait for an ack and then go into a wait mode. The "semaphore" process will exit when a given number of bots have "bumped" it and bots will then star waiting for a chunk of data. There's also a global "statistics" process, hidden in the stats module, that bots send async requests to periodically. 10 thousand bots per node and EC2 instance will repeat the above process but the nodes seem to loose connectivity before bots subscribe successfully. Notice that I have a random delay before each reconnect to lessen the burden on the application server. All of the above doesn't seem like it should burden the network links in theory. That's not what I'm seeing in practice, though. I'm thinking of complex schemes where 500k bots don't monitor a single process, etc. but I wonder if there's something simple that I'm missing. Thanks, Joel P.S. The transition bit is there to centralize error handling, since I want to restart from scratch any time something goes wrong. --- -module(flashbot). -behavior(gen_fsm). -export([start/1, status/1]). -export([not_connected/2, connected/2, packet/2, subscribed/2, network_split/2]). -export([init/1, handle_event/3, handle_sync_event/4, handle_info/3, terminate/3, code_change/4]). -define(MAX_FAILS, 60). -define(MIN_DELAY, 2000). -define(TIMEOUT, 5000). -record(state, { sim, histo, timer, semaphore, expected, socket, latency, fails, timeout, transition }). start(Expected) -> gen_fsm:start_link(?MODULE, [Expected], []). status(Ref) -> gen_fsm:sync_send_all_state_event(Ref, status). init([Expected]) -> put(state, pre_init), Sem = global:whereis_name(semaphore), Histo = global:whereis_name(sim_histo), init([Expected, Sem, Histo]); init([Expected, Sem, Histo]) when is_pid(Sem), is_pid(Histo) -> {A1, A2, A3} = now(), random:seed(A1, A2, A3), State = #state{ sim = node(Histo), histo = Histo, expected = Expected, semaphore = Sem, fails = -1, timeout = random:uniform(?MIN_DELAY) }, catch erlang:monitor(process, Sem), {_, _, State1} = reconnect(State), stats:add(flash_total_started, 1), stats:sum(flash_started, 1), put(state, completed_init), {ok, not_connected, State1}; init([Expected, _, _]) -> init([Expected]). network_split(rejoin, State) -> put(state, netsplit), case net_adm:ping(State#state.sim) of pong -> Sem = global:whereis_name(semaphore), Histo = global:whereis_name(sim_histo), put(state, rejoined), case {Sem, Histo} of {P1, P2} when is_pid(P1), is_pid(P2) -> stats:add(flash_total_rejoins, 1), State1 = State#state{semaphore = P1, histo = P2}, transition({error, netsplit}, network_split, State1); _ -> F = initially(rejoin, ?TIMEOUT), transition(ok, network_split, F, State) end; _ -> F = initially(rejoin, ?TIMEOUT), transition(ok, network_split, F, State) end; network_split(_, State) -> {next_state, network_split, State}. not_connected(connect, State) -> put(state, connecting), timer:sleep(random:uniform(State#state.timeout)), put(state, fetching_server_list), Result = case (catch sim_cache:server_list()) of {'EXIT', _} -> {error, server_list_timeout}; Other -> Other end, put(state, have_server_list), F = fun({ok, [{Host, _, Port, _}|_]}, S) -> Ref = gen_fsm:send_event_after(50, {connect, Host, Port}), S#state{timer = Ref} end, transition(Result, not_connected, not_connected, F, State); not_connected({connect, Host, Port}, State) -> Result = gen_tcp:connect(Host, Port, [binary, {packet, 0}, {active, true}, {reuseaddr, true} ], ?TIMEOUT), stats:add(flash_total_connected, 1), stats:sum(flash_connected, 1), F = fun({ok, Sock}, S) -> Ref = gen_fsm:send_event_after(50, packet2), S#state{socket = Sock, timer = Ref} end, transition(Result, not_connected, packet, F, State). packet(packet2, State) -> Result = send(<<255, 255, 255, 255>>, State), inet:setopts(State#state.socket, [{packet, 2}]), transition(Result, packet, initially(timeout, ?TIMEOUT), State); packet(timeout, State) -> transition({error, packet_ack_timeout}, packet, connected, State); packet(ack, State) -> transition(ok, packet, connected, initially(subscribe), State). connected(subscribe, State) -> JSON = mochijson2:encode({struct, [{action, <<"subscribe">>}, {data, <<"events">>} ]}), Result = send(JSON, State), if Result == ok -> stats:add(flash_total_sub_req, 1), stats:sum(flash_sub_req, 1); true -> ignore end, %% re-subscribe after a timeout transition(Result, connected, initially(timeout, ?TIMEOUT), State); connected(timeout, State) -> %% subscribe ack timed out transition({error, subscribe_timeout}, connected, subscribed, State); connected(ack, State) -> %% subscribe confirmation received catch gen_fsm:cancel_timer(State#state.timer), semaphore:up(State#state.semaphore), stats:add(flash_total_sub_ack, 1), stats:sum(flash_sub_ack, 1), transition(ok, connected, subscribed, State#state{timer = undefined}). subscribed(ready, State) -> F = fun(ok, S) -> Ref = gen_fsm:send_event_after(?TIMEOUT, timeout), S#state{timer = Ref} end, transition(ok, subscribed, subscribed, F, State); subscribed(timeout, State) -> sim_histo:failure(State#state.histo), {stop, normal, State}; subscribed(Expected, State) when Expected == State#state.expected -> sim_histo:success(State#state.histo, State#state.latency), JSON = mochijson2:encode({struct, [{action, <<"unsubscribe">>}, {data, <<"events">>}]}), send(JSON, State), {stop, normal, State}. handle_event(Event, Where, State) -> {stop, {unknown_event, Event, Where}, State}. handle_sync_event(status, _From, Where, State) -> {reply, State#state.transition, Where, State}; handle_sync_event(Event, From, Where, State) -> {stop, {unknown_sync_event, Event, From, Where}, State}. handle_info({'DOWN', _, process, Pid, noconnection}, _, State) when State#state.semaphore == Pid -> stats:add(flash_total_splits, 1), network_split(rejoin, State); handle_info({'DOWN', _, process, Pid, normal}, Where, State) when State#state.semaphore == Pid -> ?MODULE:Where(ready, State); handle_info({'DOWN', _, process, _, shutdown}, Where, State) -> {next_state, Where, State}; handle_info({tcp, _Sock, <<"ACK">>}, Where, State) -> ?MODULE:Where(ack, State); handle_info({tcp, _Sock, Bin}, Where, State) -> Now = now(), {Then, Bin1} = binary_to_term(Bin), JSON = mochijson2:decode(Bin1), %% grab the timestamp Delta = timer:now_diff(Now, Then), State1 = State#state{latency = Delta}, ?MODULE:Where(JSON, State1); handle_info(Error, Where, State = #state{}) when element(1, Error) == tcp_closed; element(1, Error) == tcp_error; element(1, Error) == error -> stats:add(flash_total_tcp_errors, 1), stats:sum(flash_tcp_errors, 1), transition(Error, Where, State); handle_info(Info, Where, State) -> {stop, {unknown_info, Info, Where}, State}. terminate(_Reason, _Where, State) -> catch gen_tcp:close(State#state.socket), stats:add(flash_total_stopped, 1), stats:sum(flash_stopped, 1), ok. code_change(_OldVsn, Where, State, _Extra) -> {ok, Where, State}. send(Bin, State = #state{}) -> gen_tcp:send(State#state.socket, Bin). reconnect(State = #state{fails = ?MAX_FAILS}) -> {stop, max_fails, State}; reconnect(State) -> flush(), catch gen_tcp:close(State#state.socket), catch gen_fsm:cancel_timer(State#state.timer), Ref = gen_fsm:send_event_after(50, connect), State1 = State#state{timer = Ref, fails = State#state.fails + 1}, %% Timeout = State1#state.fails * State1#state.timeout %% + random:uniform(?MIN_DELAY), Timeout = State1#state.timeout + random:uniform(?MIN_DELAY), State2 = State1#state{timeout = Timeout}, %% io:format("~p: reconnecting. fails: ~p, timeout: ~p~n", %% [self(), State2#state.fails, State#state.timeout]), stats:add(flash_total_fails, 1), {next_state, not_connected, State2}. flush() -> receive _ -> flush() after 0 -> ok end. transition(Error, Next, State = #state{}) when is_atom(Next) -> transition(Error, Next, Next, State). transition(Error, Next, F, State = #state{}) when is_atom(Next), is_function(F) -> transition(Error, Next, Next, F, State); transition(Error, Current, Next, State = #state{}) when is_atom(Current), is_atom(Next) -> transition(Error, Current, Next, fun(_, S) -> S end, State). transition(Good, Prev, Next, F, State = #state{}) when is_atom(Next), is_function(F), Good == ok; is_atom(Next), is_function(F), element(1, Good) == ok -> State1 = State#state{transition = {Prev, Next, Good}}, put(state, {transition, Prev, Next, Good}), {next_state, Next, F(Good, State1)}; transition(Error, subscribed, Next, F, State = #state{semaphore = Sem}) -> semaphore:down(Sem), transition(Error, undefined, Next, F, State); transition(Error, Prev, Next, _, State = #state{}) -> %% io:format("~p: ~p -> ~p = ~p~n", [self(), Prev, Next, Error]), stats:add(flash_errors, 1), State1 = State#state{transition = {Prev, Next, Error}}, put(state, {transition, Prev, Next, Error}), reconnect(State1). initially(Event) -> initially(Event, 50). initially(Event, Timeout) -> fun(ok, State = #state{}) -> catch gen_fsm:cancel_timer(State#state.timer), Ref = gen_fsm:send_event_after(Timeout, Event), State#state{timer = Ref} end. --- fastest mac firefox! http://wagerlabs.com From rvirding@REDACTED Mon Sep 14 22:17:36 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 14 Sep 2009 22:17:36 +0200 Subject: [erlang-questions] Re: data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: References: Message-ID: <3dbc6d1c0909141317v78417d1dqa466cc3b0eda6847@mail.gmail.com> 2009/9/14 James Hague > > really want is to have a gb_trees:insert function that... > > I meant gb_sets. Something like gb_sets:add_shared(Term, Set). > I am missing something here. gb_sets (nor sets, ordsets, rbsets) does not make a copy of the data which is put into the set. All that is copied is enough of the *tree* to insert the new element. There is no need to copy the new data as it is kept within the same process. Only ets makes a copy of the data. Robert From james.hague@REDACTED Mon Sep 14 22:36:18 2009 From: james.hague@REDACTED (James Hague) Date: Mon, 14 Sep 2009 15:36:18 -0500 Subject: [erlang-questions] Re: data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <3dbc6d1c0909141317v78417d1dqa466cc3b0eda6847@mail.gmail.com> References: <3dbc6d1c0909141317v78417d1dqa466cc3b0eda6847@mail.gmail.com> Message-ID: > I am missing something here. gb_sets (nor sets, ordsets, rbsets) does not > make a copy of the data which is put into the set. All that is copied is > enough of the *tree* to insert the new element. There is no need to copy the > new data as it is kept within the same process. Only ets makes a copy of the > data. Let's say you've got a long list of strings. Many of them duplicates. You don't just want to remove the duplicates because that will change the length of the list. The goals is to ensure that identical strings are shared, so there's only one copy in memory. What's a practical way of doing that? This is irrelevant most of the time, but there are some situations where it's a huge win. (My solution was to build a new list by adding each element to a binary tree. If a string is already in the tree, return the version that's already there (which is not something that gb_sets does). In the resulting list, elements are shared as much as possible. I'm clearly taking advantage of how the runtime works, but it shrunk the heap size by tens of megabytes.) From rvirding@REDACTED Mon Sep 14 23:06:29 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 14 Sep 2009 23:06:29 +0200 Subject: [erlang-questions] Re: data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: References: <3dbc6d1c0909141317v78417d1dqa466cc3b0eda6847@mail.gmail.com> Message-ID: <3dbc6d1c0909141406l18b3cb68r2fbd09d1023a71b3@mail.gmail.com> Ah ok, then I understand you. Well I would class that as an unnecessary misfeature, file a bug report and ask them to change it. (Unnecessary as you don't really need to put in the new element) I can just point out that neither sets, ordsets nor rbsets do that, they all leave the original element in. :-) Robert 2009/9/14 James Hague > > I am missing something here. gb_sets (nor sets, ordsets, rbsets) does not > > make a copy of the data which is put into the set. All that is copied is > > enough of the *tree* to insert the new element. There is no need to copy > the > > new data as it is kept within the same process. Only ets makes a copy of > the > > data. > > Let's say you've got a long list of strings. Many of them duplicates. > You don't just want to remove the duplicates because that will change > the length of the list. The goals is to ensure that identical strings > are shared, so there's only one copy in memory. What's a practical > way of doing that? > > This is irrelevant most of the time, but there are some situations > where it's a huge win. > > (My solution was to build a new list by adding each element to a > binary tree. If a string is already in the tree, return the version > that's already there (which is not something that gb_sets does). In > the resulting list, elements are shared as much as possible. I'm > clearly taking advantage of how the runtime works, but it shrunk the > heap size by tens of megabytes.) > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From mikpe@REDACTED Tue Sep 15 00:43:11 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Tue, 15 Sep 2009 00:43:11 +0200 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: References: Message-ID: <19118.50943.674579.952722@pilspetsen.it.uu.se> James Hague writes: > I've run into several cases where enforcing the sharing of data > resulted in a significant memory savings. I'm talking about a > reduction in heap size from 60MB to under half that. By "enforcing the > sharing of data" I mean making sure that identical elements in a data > structure are actually referencing the same locations in memory. > > This is easy to do in Erlang, because the compiler is very literal: > > fix_tuple({H, H}) -> {H, H}; > ... > > That ensures that identical looking elements in the tuple are sharing > memory locations. But there is absolutely no reason the compiler has > to do this. It would be perfectly valid to optimize away the entire > function, just returning the original value. > > Would any existing standard library functions make this nicer? What I > really want is to have a gb_trees:insert function that returns > {NewTree, InsertedValue} where InsertedValue references existing data > (unless it wasn't already in the tree; in that case, InsertedValue is > exactly what I passed in). Then I can happily use InsertedValue, > knowing data is being shared. Sounds like you want "hash consing". A hash table keeps track of all non-immediate terms seen so far. To "intern" a new term you recurse down to the leaves and compute hashes, on the way up you check if an equivalent node (e.g. cons/2 or tuple/N) has been seen, and if so you use that one otherwise you add the new node to the hash table. Either way you return the interned node and its hash on the way up. This may lose sharing with the input term, but interned terms will share everything's that's sharable. Erlang's default memory model doesn't allow same-node processes to share memory(*), so you will lose sharing in message sends. (*) Except in one binaries-related case which is irrelevant here. A major downside of hash consing is that it can leak memory: if an interned term becomes unreferenced in the application, the hash table will still keep a master copy of it, wasting memory. VMs with built-in support for hash consing usually also support "weak references" or "weak hashes" where the referenced data can be nuked if the GC determines that it should be dead. From bengt.tillman@REDACTED Tue Sep 15 09:25:06 2009 From: bengt.tillman@REDACTED (Bengt Tillman) Date: Tue, 15 Sep 2009 09:25:06 +0200 Subject: [erlang-questions] large scale deployments and netsplits In-Reply-To: <4AAE7D4B.7020909@erlang-consulting.com> References: <8209f740909140848m4582290dka8f9e0b6e72321c@mail.gmail.com> <0784F720-9D6A-44B1-8CCE-637FC8DD0621@gmail.com> <4AAE7D4B.7020909@erlang-consulting.com> Message-ID: <38E78E0735C49D448494F368109B801518F7AB@esealmw114.eemea.ericsson.se> In the NETSim product we simulate the operation and maintenance traffic of lots of different kinds of exchanges in GSM or 3G-networks. We can simulate thousands of exchanges and have to spread them over a large number of Erlang nodes (up to a couple of hundreds) executing on a number of computers (in the 10ths) in a local area network. We have had to set the net ticktime to 300 in order to keep the Erlang nodes from losing contact with each other. The response times between different Erlang nodes is not mission critical in our application - we communicate between Erlang nodes in order to administrate the network of exchanges and collect error reports. The mission critical traffic is TCP/IP (or other O&M protocols) between a simulated exchange and the system our customers want to test (OSS). BR, Bengt Tillman > Ulf Wiger wrote > > Joel Reymont wrote: > > > > On Sep 14, 2009, at 4:48 PM, Ulf Wiger wrote: > > > >> What is your setting for net_ticktime? > > > > Is there a way to set net_ticktime programmatically, once > the node is > > up and running? > > net_kernel:set_net_ticktime(TickTime [, TransitionPeriod]) > > The default tick timeout is 45 < T < 75, with TickTime being > 60 seconds. > > See http://www.erlang.org/doc/man/kernel_app.html > > Normally, and especially for systems where response time is > an issue, the default ticktime is too long - not too short. > If you have network connections that are not responding > within at least 45 seconds, it seems as if you have some > rather severe overload on your links. > > BR, > Ulf W From ulf.wiger@REDACTED Tue Sep 15 10:30:23 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 15 Sep 2009 10:30:23 +0200 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <19118.50943.674579.952722@pilspetsen.it.uu.se> References: <19118.50943.674579.952722@pilspetsen.it.uu.se> Message-ID: <4AAF509F.7050400@erlang-consulting.com> Mikael Pettersson wrote: > > Erlang's default memory model doesn't allow same-node processes to > share memory(*), so you will lose sharing in message sends. Yes, but there are two levels of sharing here. 1. The sharing of terms across processes, as with large binaries 2. The relative sharing within the term itself. The latter could be preserved using a sharing-preserving copy. As this is invariably more expensive than the current copying algorithm when there is no sharing to preserve (likely a very common case), it is reasonable that this isn't the default. The problem today is that you cannot make a sharing-preserving copy between processes at the Erlang level even if your life depended on it, and in some cases, this may indeed be the case, figuratively speaking. Some data structures simply cannot be passed in a message or in a spawn, since the loss of sharing leads to a memory explosion. One problem is of course that if a process that has such a term crashes, and EXIT messages are propagated that contain the term, the EXIT propagation will kill the node. Even without loss of sharing, this can happen, of course. I did it once by spawn-linking 100,000 processes from the shell and then mis-spelling length(processes()). This led to an EXIT message, containing a list of 100,000 pids, that was copied 100,000 times. My workstation was frozen for 10 minutes while the VM was trying to find a way to cope. No mystery there, of course, and no sharing either. All I could do was laugh at my stupidity and take an extra coffee break. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ulf.wiger@REDACTED Tue Sep 15 10:41:12 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 15 Sep 2009 10:41:12 +0200 Subject: [erlang-questions] large scale deployments and netsplits In-Reply-To: <38E78E0735C49D448494F368109B801518F7AB@esealmw114.eemea.ericsson.se> References: <8209f740909140848m4582290dka8f9e0b6e72321c@mail.gmail.com> <0784F720-9D6A-44B1-8CCE-637FC8DD0621@gmail.com> <4AAE7D4B.7020909@erlang-consulting.com> <38E78E0735C49D448494F368109B801518F7AB@esealmw114.eemea.ericsson.se> Message-ID: <4AAF5328.3070708@erlang-consulting.com> Hi Bengt, Bengt Tillman wrote: > > We have had to set the net ticktime to 300 in order to keep the Erlang > nodes from losing contact with each other. The response times between > different Erlang nodes is not mission critical in our application ... I will admit that I have meditated over the network tick algorithm in Erlang several times, without being any wiser for it. It's a very nice piece of code, but I can't help thinking that there is some fatal flaw buried deep within it. At AXD 301, we tried reducing the detection times as much as we could, but never could get below a net_ticktime of 10 without getting lots of false positives. In contrast, our own device processor supervision had shorter detection times (5-6 seconds, if memory serves) and practically never any false positives, using the very same communication network. The code wasn't nearly as elegant, though. :) To be fair, this was on an internal ATM network, so we could be fairly sure that the internal communication paths were never starved by other traffic. This is of course not true in general for TCP/IP networks. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From mikpe@REDACTED Tue Sep 15 12:12:57 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Tue, 15 Sep 2009 12:12:57 +0200 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <4AAF509F.7050400@erlang-consulting.com> References: <19118.50943.674579.952722@pilspetsen.it.uu.se> <4AAF509F.7050400@erlang-consulting.com> Message-ID: <19119.26793.491700.875344@pilspetsen.it.uu.se> Ulf Wiger writes: > Mikael Pettersson wrote: > > > > Erlang's default memory model doesn't allow same-node processes to > > share memory(*), so you will lose sharing in message sends. > > Yes, but there are two levels of sharing here. > > 1. The sharing of terms across processes, as with large binaries > 2. The relative sharing within the term itself. > > The latter could be preserved using a sharing-preserving copy. > As this is invariably more expensive than the current copying > algorithm when there is no sharing to preserve (likely a very > common case), it is reasonable that this isn't the default. > > The problem today is that you cannot make a sharing-preserving > copy between processes at the Erlang level even if your life > depended on it, and in some cases, this may indeed be the case, > figuratively speaking. Some data structures simply cannot be > passed in a message or in a spawn, since the loss of sharing > leads to a memory explosion. So a sharing-preserving term_to_binary might fix the worst problems, even if it requires some non-default option and requires users to explicitly wrap and unwrap parts of messages? Just trying to see if this is something worth pursuing or not. /Mikael From james.hague@REDACTED Tue Sep 15 15:50:12 2009 From: james.hague@REDACTED (James Hague) Date: Tue, 15 Sep 2009 08:50:12 -0500 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <19118.50943.674579.952722@pilspetsen.it.uu.se> References: <19118.50943.674579.952722@pilspetsen.it.uu.se> Message-ID: > Sounds like you want "hash consing". Hash consing is a heavyweight solution. It's got a fixed cost for something that's usually irrelevant. What I really want is a function that takes a data structure and returns a new version with maximal sharing. I can write special case versions of that in Erlang, but it's messy and feels like something that should be a general library service. From baryluk@REDACTED Tue Sep 15 21:40:51 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Tue, 15 Sep 2009 21:40:51 +0200 Subject: [erlang-questions] Re: data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: References: <3dbc6d1c0909141317v78417d1dqa466cc3b0eda6847@mail.gmail.com> Message-ID: <1253043651.24240.1928.camel@sredniczarny> Dnia 2009-09-14, pon o godzinie 15:36 -0500, James Hague pisze: > > I am missing something here. gb_sets (nor sets, ordsets, rbsets) does not > > make a copy of the data which is put into the set. All that is copied is > > enough of the *tree* to insert the new element. There is no need to copy the > > new data as it is kept within the same process. Only ets makes a copy of the > > data. > > Let's say you've got a long list of strings. Many of them duplicates. > You don't just want to remove the duplicates because that will change > the length of the list. The goals is to ensure that identical strings > are shared, so there's only one copy in memory. What's a practical > way of doing that? > > This is irrelevant most of the time, but there are some situations > where it's a huge win. > > (My solution was to build a new list by adding each element to a > binary tree. If a string is already in the tree, return the version > that's already there (which is not something that gb_sets does). In > the resulting list, elements are shared as much as possible. I'm > clearly taking advantage of how the runtime works, but it shrunk the > heap size by tens of megabytes.) It looks for me as quite good solution, you depend on memory saving, so explicitly manages to have shared strings: Any other way i can see is to have this add/retrivial from binary tree, to be performed implicitly on every step (via some sort of hashing) of computation to ensure that identical elements are only once in memory (of one process). But this have really big performance problems i think, and most of times you really don't need perfect sharing. You can limit this "hashing" to only terms which are complex enaugh, like long lists, or nested tuples. But still this is very costly. If you want to have some term (or list of many terms) and shrink them, by some magic, you can using following pseudo code: maximize_sharing(Term) -> Tree = tree:new(), NewTree = iterate_over_every_subterm_and_add_first_to_tree(Term), NewTerm = iterate_over_every_subterm_and_find_in_tree(Term, NewTree). Lack of sharing is also problematic when sending term_to_binary data, because term_to_binary essentially flattens term be repeatedly copying data. It would be nice to have term_to_binary which doesn't copies data more than once, which already is in the same term. Ie. > erlang:process_info(self(), heap_size). {heap_size,1597} > X = {2,2,2,2}, Y1 = {X,X,X,X}, Y2 = {Y1,Y1,Y1,Y1}, Y3 = {Y2,Y2,Y2,Y2}, Y4 = {Y3,Y3,Y3,Y3}, Y5 = {Y4,Y4,Y4,Y4}, Y6 = {Y5,Y5,Y5,Y5}, Y7 = {Y6,Y6,Y6,Y6}, Y8 = {Y7,Y7,Y7,Y7}, Y9 = {Y8,Y8,Y8,Y8}, ok. > erlang:process_info(self(), heap_size). {heap_size,4181} > In memory Y9 will be pretty small, and whole process memory consumption will be small, but term_to_binary(Y9) (binary with 2MB of data, but not counting to the heap_size), or just displaying using io_lib:format("~w", [Y9]) (2MB of data after flattening resulting list) or sending it to different node will be disaster. > erlang:process_info(self(), heap_size). % after io_lib:format(), ok. {heap_size,9707190} I don't know how it affects sending Y9 to process on the same node. As we know Y9 need to be copied to heap of the destination process (because sharing heaps between process destroy soft real-time semantic of garbage collection, but improves other performance metrics for big messages). But is it copied smart enough? > P = spawn(fun() -> P = receive P2 -> P2 end, io:format("~p ~n ~p ~n", [erlang:process_info(self(), heap_size), size(P)]), ok end), P ! Y9, ok. {heap_size,2103540} > Unfortunately no. First good step will be to have version of term_to_binary which preserves sharing of (already shared) subterms (term_to_binary have second argument for options, like minorversion or gzip compression), and binary_to_term which understand how to unpack them preserving sharing (also across multiple process in the same node, or compact storage in file/db). -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From paul-trapexit@REDACTED Tue Sep 15 23:25:19 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Tue, 15 Sep 2009 14:25:19 -0700 (PDT) Subject: [erlang-questions] Re: data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <1253043651.24240.1928.camel@sredniczarny> References: <3dbc6d1c0909141317v78417d1dqa466cc3b0eda6847@mail.gmail.com> <1253043651.24240.1928.camel@sredniczarny> Message-ID: i use erlang:term_to_binary to persist items in databases all the time, so this thread caught my attention. to stimulate discussion i pounded out some code that does hash consing (hashcons:share/1) and that does encoding and decoding suitable for composition with erlang:term_to_binary/2 (hashcons:encode/1) and erlang:binary_to_term/1 (hashcons:decode/1). encoded terms don't necessarily have a smaller wire footprint even in a very favorable case (see below), but should decode into a shared data structure (as opposed to calling hashcons:share/1 after calling erlang:binary_to_term/1, which could create a temporary which is unacceptably huge). attached. -- p % erl Erlang (BEAM) emulator version 5.6.5 [source] [async-threads:0] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> hashcons:encode (lists:duplicate (10, wazzup)). {[0,0,0,0,0,0,0,0,0,0],[{0,wazzup}]} 2> size (erlang:term_to_binary (hashcons:encode (lists:duplicate (10, wazzup)), [ compressed ])). 35 3> size (erlang:term_to_binary (lists:duplicate (10, wazzup), [ compressed ])). 31 4> % not always smaller 4> size (erlang:term_to_binary (hashcons:encode (lists:duplicate (100, wazzup)), [ compressed ])). 38 5> size (erlang:term_to_binary (lists:duplicate (100, wazzup), [ compressed ])). 37 -------------- next part -------------- -module (hashcons). -export ([ share/1, encode/1, decode/1 ]). -include_lib ("eunit/include/eunit.hrl"). %-=====================================================================- %- hash consing - %-=====================================================================- share (Term) -> element (1, share (Term, dict:new ())). share (Term, Dict) -> case dict:find (Term, Dict) of { ok, OrigTerm } -> { OrigTerm, Dict }; error -> { NewTerm, NewDict } = share_copy (Term, Dict), { NewTerm, dict:store (NewTerm, NewTerm, NewDict) } end. share_copy (Term, Dict) when is_list (Term) -> { NewTerm, NewDict } = lists:foldl (fun (T, { Acc, D }) -> { TPrime, DPrime } = share (T, D), { [ TPrime | Acc ], DPrime } end, { [], Dict }, Term), { lists:reverse (NewTerm), NewDict }; share_copy (Term, Dict) when is_tuple (Term) -> { ListTerm, NewDict } = share_copy (tuple_to_list (Term), Dict), { list_to_tuple (ListTerm), NewDict }; share_copy (Term, Dict) -> { Term, Dict }. %-=====================================================================- %- encoding, intended to be composed with term_to_binary %-=====================================================================- encode (Term) -> { Ref, Encoded, _, Codec } = encode (Term, dict:new (), dict:new ()), { Encoded, [ X || X <- dict:to_list (Codec), element (1, X) =/= Ref ] }. encode (Term, Dict, Codec) -> case dict:find (Term, Dict) of { ok, { Ref, OrigTerm } } -> { Ref, OrigTerm, Dict, Codec }; error -> { NewTerm, NewDict, NewCodec } = encode_copy (Term, Dict, Codec), Ref = dict:size (NewDict), { Ref, NewTerm, dict:store (NewTerm, { Ref, NewTerm }, NewDict), dict:store (Ref, NewTerm, NewCodec) } end. encode_copy (Term, Dict, Codec) when is_list (Term) -> { NewTerm, NewDict, NewCodec } = lists:foldl (fun (T, { Acc, D, C }) -> { Ref, _, DPrime, CPrime } = encode (T, D, C), { [ Ref | Acc ], DPrime, CPrime } end, { [], Dict, Codec }, Term), { lists:reverse (NewTerm), NewDict, NewCodec }; encode_copy (Term, Dict, Codec) when is_tuple (Term) -> { ListTerm, NewDict, NewCodec } = encode_copy (tuple_to_list (Term), Dict, Codec), { { t, ListTerm }, NewDict, NewCodec }; encode_copy (Term, Dict, Codec) when is_integer (Term) -> { { i, Term }, Dict, Codec }; encode_copy (Term, Dict, Codec) -> { Term, Dict, Codec }. %-=====================================================================- %- decoding, intended to be composed with binary_to_term %-=====================================================================- decode ({ Encoded, Codec }) -> decode (Encoded, dict:from_list (Codec)). decode (Encoded, Codec) when is_list (Encoded) -> [ decode (X, Codec) || X <- Encoded ]; decode ({ t, Encoded }, Codec) when is_list (Encoded) -> list_to_tuple (decode (Encoded, Codec)); decode ({ i, Literal }, _Codec) when is_integer (Literal) -> Literal; decode (Ref, Codec) when is_integer (Ref) -> decode (dict:fetch (Ref, Codec), Codec); decode (Literal, _Codec) -> Literal. %-=====================================================================- %- Tests - %-=====================================================================- % ok, don't want to depend upon quickcheck, so here's some cheese -define (FORALL (Var, Gen, Cond), fun (A) -> Var = (Gen) (A), Cond end). flasscheck (N, Limit, P) -> flasscheck (1, N, math:log (Limit), P). flasscheck (M, N, LogLimit, P) when M =< N -> Size = trunc (math:exp (LogLimit * M / N)), true = P (Size), io:format (".", []), flasscheck (M + 1, N, LogLimit, P); flasscheck (_, N, _, _) -> io:format ("~n~p tests passed~n", [ N ]), ok. random_integer () -> random:uniform (100000). random_float () -> random:uniform (). random_binary () -> list_to_binary ([ random:uniform (255) || _ <- lists:seq (1, 5) ]). random_atom () -> list_to_atom ([ random:uniform ($z - $a) + $a || _ <- lists:seq (1, 5) ]). random_list () -> [ random_term () || _ <- lists:seq (1, 3) ]. random_tuple () -> list_to_tuple (random_list ()). random_term () -> case random:uniform (10) of 1 -> random_integer (); 2 -> random_integer (); 3 -> random_float (); 4 -> random_float (); 5 -> random_binary (); 6 -> random_binary (); 7 -> random_atom (); 8 -> random_atom (); 9 -> random_list (); 10 -> random_tuple () end. share_test_ () -> F = fun () -> T = ?FORALL (X, fun (_) -> random_term () end, (fun (Term) -> Term = share (Term), true end) (X)), ok = flasscheck (10000, 10, T) end, { timeout, 60, F }. codec_test_ () -> F = fun () -> T = ?FORALL (X, fun (_) -> random_term () end, (fun (Term) -> Term = decode (encode (Term)), true end) (X)), ok = flasscheck (10000, 10, T) end, { timeout, 60, F }. From paul-trapexit@REDACTED Tue Sep 15 23:34:58 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Tue, 15 Sep 2009 14:34:58 -0700 (PDT) Subject: [erlang-questions] Re: data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: References: <3dbc6d1c0909141317v78417d1dqa466cc3b0eda6847@mail.gmail.com> <1253043651.24240.1928.camel@sredniczarny> Message-ID: It still passes the test either way (ugh, my test sucks), but I think this is a bugfix: -- p --- hashcons.erl.orig 2009-09-15 14:29:30.000000000 -0700 +++ hashcons.erl 2009-09-15 14:29:32.000000000 -0700 @@ -53,7 +53,7 @@ Ref = dict:size (NewDict), { Ref, NewTerm, - dict:store (NewTerm, { Ref, NewTerm }, NewDict), + dict:store (Term, { Ref, NewTerm }, NewDict), dict:store (Ref, NewTerm, NewCodec) } end. On Tue, 15 Sep 2009, Paul Mineiro wrote: > i use erlang:term_to_binary to persist items in databases all the time, so > this thread caught my attention. > > to stimulate discussion i pounded out some code that does hash consing > (hashcons:share/1) and that does encoding and decoding suitable for > composition with erlang:term_to_binary/2 (hashcons:encode/1) and > erlang:binary_to_term/1 (hashcons:decode/1). encoded terms don't > necessarily have a smaller wire footprint even in a very favorable case > (see below), but should decode into a shared data structure (as opposed to > calling hashcons:share/1 after calling erlang:binary_to_term/1, which > could create a temporary which is unacceptably huge). > > attached. > > -- p > > % erl > Erlang (BEAM) emulator version 5.6.5 [source] [async-threads:0] [kernel-poll:false] > > Eshell V5.6.5 (abort with ^G) > 1> hashcons:encode (lists:duplicate (10, wazzup)). > {[0,0,0,0,0,0,0,0,0,0],[{0,wazzup}]} > 2> size (erlang:term_to_binary (hashcons:encode (lists:duplicate (10, wazzup)), [ compressed ])). > 35 > 3> size (erlang:term_to_binary (lists:duplicate (10, wazzup), [ compressed ])). > 31 > 4> % not always smaller > 4> size (erlang:term_to_binary (hashcons:encode (lists:duplicate (100, wazzup)), [ compressed ])). > 38 > 5> size (erlang:term_to_binary (lists:duplicate (100, wazzup), [ compressed ])). > 37 > From tony@REDACTED Tue Sep 15 22:38:57 2009 From: tony@REDACTED (Tony Rogvall) Date: Tue, 15 Sep 2009 22:38:57 +0200 Subject: Fwd: [erlang-questions] Re: data sharing is outside the semantics of Erlang, but it sure is useful References: <962980AB-2FFA-4F6A-B26E-0BBB45178217@rogvall.se> Message-ID: <73E79269-24E9-43AF-A537-4C31BB0F79F8@rogvall.se> Forgot the "Reply All" .... Begin forwarded message: > From: Tony Rogvall > Date: ti 15 sep 2009 22.36.32 GMT+02:00 > To: Witold Baryluk > Subject: Re: [erlang-questions] Re: data sharing is outside the > semantics of Erlang, but it sure is useful > > Here is nice term encoder/decoder (below) that will solve the size > problem. > The algorithm is not efficient (in Erlang) but if/when implemented > in C it could be used as a bif. > Still it will produce decent results on (all) shared data. > > The encoding of Y9 (from previous example) is then: > > te:encode(Y9). > > {2, > {1,1,1,1}, > {2,2,2,2}, > {3,3,3,3}, > {4,4,4,4}, > {5,5,5,5}, > {6,6,6,6}, > {7,7,7,7}, > {8,8,8,8}, > {9,9,9,9}, > {10,10,10,10}, > 11} > > That when encoded with term_to_binary lead to a binary of size 107. > If coded straight it was 2796203 bytes!!! > > Other examples that reveal some properties of the algorithm: > > te:encode({[a,b],[a,b,c],[a,b,c,d]}). > > {a,b,c,d,[], > [4|5], > [3|6], > [2|7], > [1|8], > [3|5], > [2|10], > [1|11], > [2|5], > [1|13], > {14,12,9}, > 15} > > te:encode({[b,a],[c,b,a],[d,c,b,a]}). > > {d,c,b,a,[],[4|5],[3|6],[2|7],[1|8],{7,8,9},10} > > > applied to code (parse trees) you will be able to detect all common > sub expressions > (renaming bindings and removing line numbers helps a lot ;-) > > /Tony > > ------------- > -module(te). > > -export([encode/1, decode/1]). > -export([binary_to_term/1, term_to_binary/1]). > > %% > %% Decode a term given the pointer map > %% > binary_to_term(Bin) -> > decode(erlang:binary_to_term(Bin)). > > decode(Map) -> > decode(1,Map). > > %% When implement this in C the setelement is of course destructive > decode(I,Map) -> > E = element(I,Map), > if I == size(Map) -> > element(E,Map); > is_list(E), E =/= [] -> > H = element(hd(E),Map), > T = element(tl(E),Map), > decode(I+1, setelement(I, Map, [H|T])); > is_tuple(E), size(E) > 0 -> > T = decode_tuple(E,size(E),Map,[]), > decode(I+1, setelement(I, Map, T)); > true -> > decode(I+1, Map) > end. > > > decode_tuple(_T,0,_Map,Acc) -> > list_to_tuple(Acc); > decode_tuple(T,I,Map,Acc) -> > Ei = element(I,T), > E = element(Ei,Map), > decode_tuple(T,I-1,Map,[E|Acc]). > > %% > %% Do term compression/encoding > %% Encode bottom up while assinging pointer index > %% > term_to_binary(Term) -> > erlang:term_to_binary(encode(Term)). > > encode(Term) -> > {E,_K,_D, S} = encode(Term, 1, dict:new(), []), > list_to_tuple(lists:reverse([E|S])). > > encode(T, K, D, S) -> > if is_list(T), T =/= [] -> > {He,K1,D1,S1} = encode(hd(T), K, D, S), > {Te,K2,D2,S2} = encode(tl(T), K1, D1, S1), > insert([He|Te], K2, D2, S2); > is_tuple(T), size(T) > 0 -> > {Te,K1,D1,S1} = encode_tuple(T,size(T),K,D,S,[]), > insert(Te, K1, D1, S1); > true -> > insert(T, K, D, S) > end. > > encode_tuple(_T, 0, K, D, S, Acc) -> > {list_to_tuple(Acc), K, D, S}; > encode_tuple(T, I, K, D, S, Acc) -> > {E,K1,D1,S1} = encode(element(I,T), K, D, S), > encode_tuple(T,I-1, K1, D1,S1,[E|Acc]). > > > %% lookup and maybe insert > insert(Encoded, K, D, S) -> > try dict:fetch(Encoded, D) of > Ke -> {Ke,K,D,S} > catch > error:_ -> > De = dict:store(Encoded, K, D), > {K,K+1,De,[Encoded|S]} > end. > ------------------- > > > On 15 sep 2009, at 21.40, Witold Baryluk wrote: > >> Dnia 2009-09-14, pon o godzinie 15:36 -0500, James Hague pisze: >>>> I am missing something here. gb_sets (nor sets, ordsets, rbsets) >>>> does not >>>> make a copy of the data which is put into the set. All that is >>>> copied is >>>> enough of the *tree* to insert the new element. There is no need >>>> to copy the >>>> new data as it is kept within the same process. Only ets makes a >>>> copy of the >>>> data. >>> >>> Let's say you've got a long list of strings. Many of them >>> duplicates. >>> You don't just want to remove the duplicates because that will >>> change >>> the length of the list. The goals is to ensure that identical >>> strings >>> are shared, so there's only one copy in memory. What's a practical >>> way of doing that? >>> >>> This is irrelevant most of the time, but there are some situations >>> where it's a huge win. >>> >>> (My solution was to build a new list by adding each element to a >>> binary tree. If a string is already in the tree, return the version >>> that's already there (which is not something that gb_sets does). In >>> the resulting list, elements are shared as much as possible. I'm >>> clearly taking advantage of how the runtime works, but it shrunk the >>> heap size by tens of megabytes.) >> >> It looks for me as quite good solution, you depend on memory >> saving, so >> explicitly manages to have shared strings: >> >> Any other way i can see is to have this add/retrivial from binary >> tree, >> to be performed implicitly on every step (via some sort of hashing) >> of >> computation to ensure that identical elements are only once in memory >> (of one process). But this have really big performance problems i >> think, >> and most of times you really don't need perfect sharing. >> >> You can limit this "hashing" to only terms which are complex enaugh, >> like long lists, or nested tuples. But still this is very costly. >> >> If you want to have some term (or list of many terms) and shrink >> them, >> by some magic, you can using following pseudo code: >> >> maximize_sharing(Term) -> >> Tree = tree:new(), >> NewTree = iterate_over_every_subterm_and_add_first_to_tree(Term), >> NewTerm = iterate_over_every_subterm_and_find_in_tree(Term, NewTree). >> >> >> >> Lack of sharing is also problematic when sending term_to_binary data, >> because term_to_binary essentially flattens term be repeatedly >> copying >> data. It would be nice to have term_to_binary which doesn't copies >> data >> more than once, which already is in the same term. >> >> Ie. >>> erlang:process_info(self(), heap_size). >> {heap_size,1597} >> >>> X = {2,2,2,2}, >> Y1 = {X,X,X,X}, >> Y2 = {Y1,Y1,Y1,Y1}, >> Y3 = {Y2,Y2,Y2,Y2}, >> Y4 = {Y3,Y3,Y3,Y3}, >> Y5 = {Y4,Y4,Y4,Y4}, >> Y6 = {Y5,Y5,Y5,Y5}, >> Y7 = {Y6,Y6,Y6,Y6}, >> Y8 = {Y7,Y7,Y7,Y7}, >> Y9 = {Y8,Y8,Y8,Y8}, >> ok. >> >>> erlang:process_info(self(), heap_size). >> {heap_size,4181} >> >>> >> >> In memory Y9 will be pretty small, and whole process memory >> consumption >> will be small, but term_to_binary(Y9) (binary with 2MB of data, but >> not >> counting to the heap_size), or just displaying using >> io_lib:format("~w", >> [Y9]) (2MB of data after flattening resulting list) or sending it to >> different node will be disaster. >> >>> erlang:process_info(self(), heap_size). % after io_lib:format(), ok. >> {heap_size,9707190} >> >> >> I don't know how it affects sending Y9 to process on the same node. >> As we know Y9 need to be copied to heap of the destination process >> (because sharing heaps between process destroy soft real-time >> semantic >> of garbage collection, but improves other performance metrics for big >> messages). But is it copied smart enough? >> >> >>> P = spawn(fun() -> P = receive P2 -> P2 end, io:format("~p ~n ~p >>> ~n", >> [erlang:process_info(self(), heap_size), size(P)]), ok end), P ! >> Y9, ok. >> >> {heap_size,2103540} >>> >> >> Unfortunately no. >> >> First good step will be to have version of term_to_binary which >> preserves sharing of (already shared) subterms (term_to_binary have >> second argument for options, like minorversion or gzip >> compression), and >> binary_to_term which understand how to unpack them preserving sharing >> (also across >> multiple process in the same node, or compact storage in file/db). >> >> >> -- >> Witold Baryluk > From baryluk@REDACTED Tue Sep 15 21:53:38 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Tue, 15 Sep 2009 21:53:38 +0200 Subject: [erlang-questions] Re: data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <1253043651.24240.1928.camel@sredniczarny> References: <3dbc6d1c0909141317v78417d1dqa466cc3b0eda6847@mail.gmail.com> <1253043651.24240.1928.camel@sredniczarny> Message-ID: <1253044418.24240.1944.camel@sredniczarny> Dnia 2009-09-15, wto o godzinie 21:40 +0200, Witold Baryluk pisze: > Unfortunately no. > > First good step will be to have version of term_to_binary which > preserves sharing of (already shared) subterms (term_to_binary have > second argument for options, like minorversion or gzip compression), and > binary_to_term which understand how to unpack them preserving sharing > (also across > multiple process in the same node, or compact storage in file/db). > > btw. term_to_binary(Y9, [{compressed, 9}]) have size of about 15KB, and {compressed, 6} (default one) is about 25KB ! (but anyway after binary_to_term heap_size explodes to 2MB). -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From aqua.shim@REDACTED Wed Sep 16 04:36:40 2009 From: aqua.shim@REDACTED (Jason Shim) Date: Wed, 16 Sep 2009 11:36:40 +0900 Subject: [erlang-questions] Re: Google Trends for Erlang In-Reply-To: <179157a9-7446-48c5-a7d0-bb22e0c2cdf8@s39g2000yqj.googlegroups.com> References: <29e14dd2-93b4-4077-90b8-2f6fa2e3a636@a21g2000yqc.googlegroups.com> <179157a9-7446-48c5-a7d0-bb22e0c2cdf8@s39g2000yqj.googlegroups.com> Message-ID: Interesting.I'm a korean. It's so difficult to find korean document and people related 'Erlang'. (but doesn't means that no one knows erlang in korea) In korea, only one erlang book issued, Joe's Erlang book (translated version). On Sun, Sep 13, 2009 at 12:51 PM, Eugene Ossintsev wrote: > I'm not sure why South Korea is the number one. Maybe due to their > incredible thirst for knowledge? > > #2 for Sweden and #3 for Russia are fair enough. No questions here. > > Why isn't the USA at the top? I don't know. Why sould it be? FP > languages are not very popular in the USA, unfortunately. More in > Europe and Russia. > > On Sep 12, 11:03 am, Steve Davis > wrote: > > Some weekend "fun", and the results of this surprised me greatly: > > > > http://www.google.com/trends?q=erlang > > > > ...who would have thought that the most active region (at least by > > search rank) would be Korea and not Sweden, the UK or the USA? > > > > /sd > > > > ________________________________________________________________ > > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From nick@REDACTED Wed Sep 16 07:39:59 2009 From: nick@REDACTED (Nick Gerakines) Date: Tue, 15 Sep 2009 22:39:59 -0700 Subject: SF Erlounge, hosted by Rupture / Electronic Arts Message-ID: Thursday, September 24th, 2009 from 7:00 pm to 9:00 pm. I'm pleased to announce that Electronic Arts is hosting the next San Francisco Erlounge in the San Francisco / Rupture office. The event is open to the public. Erlang developers, functional programming folks and the curious are welcome to this social event. Light food and drinks will be provided. Rupture 400 2nd Street, Suite 350 San Francisco, CA 94107 As always, we greatly appreciate it when you RSVP. Please direct any questions or comments to nick+sferlounge@REDACTED I can also be reached on my mobile phone, +1 (415) 963-1165. There is parking available next to the building, but it isn't free. The Rupture / EASF office is also a few blocks away from CalTrain for anyone traveling from the south-bay. For those who can't make it to this one, I'll be reaching out to organize an event in the south-bay in mid to late October. Please contact me if interested in hosting. Also, check out http://sferlounge.com/ for more information and updates. # Nick Gerakines From leap@REDACTED Wed Sep 16 08:20:39 2009 From: leap@REDACTED (Michael Turner) Date: Wed, 16 Sep 2009 06:20:39 +0000 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: Message-ID: I'm a curious about how the subject line on this thread, which seems to me too-easily generalized from the very specific problem James brings up: saving space when you have a very long list of strings, with some strings repeated. If you're representing items of data of any kind in a very long list, I can only assume it's because the cost of linear access is a non-issue for your application. If you want to save space by storing only once the repeated elements in a long, mostly-serial-access list, well, that reminds of very much of the very general idea of "data compression", which is also usually used when linear-search access is not much of an issue, when memory is an issue, and when the data features significant repetition. So why not just use compression, if saving space is an important goal and reducing random-access time is not? Not sure that I'm totally sold on the Erlang Way of doing things, being pretty new to the language. But in this particular case (or even for generalizations of it), I don't see why the Erlang Way (insofar as I understand it) is necessarily inferior to anything requiring that the language break with its general shared-nothing approach. Did I miss something? -michael turner On 9/15/2009, "James Hague" wrote: >> Sounds like you want "hash consing". > >Hash consing is a heavyweight solution. It's got a fixed cost for >something that's usually irrelevant. What I really want is a function >that takes a data structure and returns a new version with maximal >sharing. I can write special case versions of that in Erlang, but >it's messy and feels like something that should be a general library >service. > >________________________________________________________________ >erlang-questions mailing list. See http://www.erlang.org/faq.html >erlang-questions (at) erlang.org > > From ulf.wiger@REDACTED Wed Sep 16 10:40:24 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 16 Sep 2009 10:40:24 +0200 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: References: Message-ID: <4AB0A478.4070007@erlang-consulting.com> Michael Turner wrote: > > Not sure that I'm totally sold on the Erlang Way of doing things, being > pretty new to the language. But in this particular case (or even for > generalizations of it), I don't see why the Erlang Way (insofar as I > understand it) is necessarily inferior to anything requiring that the > language break with its general shared-nothing approach. Did I miss > something? Again, two kinds of sharing here. 'The share-nothing approach' refers to the fact that there are no shared data structures between processes (conceptually, as the VM certainly makes use of the possibility to share stuff anytime it can do so safely.) The other type of sharing is the implicit sharing /within an Erlang term/. Consider: X = lists:seq(1,1000), Y = {X,X,X,X,X,X,X,X}. The runtime system will only allocate 2 words for the tuple + 8 words for the pointers to X. This is an extremely efficient way of building what's conceptually a structure of 8000 integers. This is used deliberately by certain modules. QuickCheck, for example, builds intricate structures that rely extensively on this type of sharing. Passing such a structure to another process, either by spawn or message passing, is generally fatal. http://erlang.org/pipermail/erlang-bugs/2007-November/000488.html (This post also contains a nice little program that could be used to benchmark different solutions.) I encountered this problem once in a very subtle way, where I had deliberately relied on implicit sharing in the state, but happened to pass the entire state by mistake inside a message. http://www.erlang.org/pipermail/erlang-questions/2005-November/017924.html The bug existed for a while, but wasn't noticeable as long as everything was on the process heap, since the deeply recursive data structure didn't take up much space. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From leap@REDACTED Wed Sep 16 15:59:55 2009 From: leap@REDACTED (Michael Turner) Date: Wed, 16 Sep 2009 13:59:55 +0000 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <4AB0A478.4070007@erlang-consulting.com> Message-ID: On 9/16/2009, "Ulf Wiger" wrote: >Michael Turner wrote: >> >> Not sure that I'm totally sold on the Erlang Way of doing things, being >> pretty new to the language. But in this particular case (or even for >> generalizations of it), I don't see why the Erlang Way (insofar as I >> understand it) is necessarily inferior to anything requiring that the >> language break with its general shared-nothing approach. Did I miss >> something? > >Again, two kinds of sharing here. [Snip] Interesting answer, but not an answer to my real question. (Maybe I went on for a paragraph too long.) It seems that James is interested in data structure sharing IN THIS PARTICULAR CASE because it gives him a kind of data compression. Again, if that's the goal, why is ordinary data compression *not* the answer here? If compression isn't the goal, what did I miss about the (implicit) problem statement, to which sharing semantics in Erlang *is* the answer? -michael turner From james.hague@REDACTED Wed Sep 16 18:31:34 2009 From: james.hague@REDACTED (James Hague) Date: Wed, 16 Sep 2009 11:31:34 -0500 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: References: <4AB0A478.4070007@erlang-consulting.com> Message-ID: > It seems that James is interested in data structure sharing IN THIS > PARTICULAR CASE because it gives him a kind of data compression. ?Again, > if that's the goal, why is ordinary data compression *not* the answer > here? Can you clarify what you mean by "ordinary data compression?" From kagato@REDACTED Wed Sep 16 18:43:18 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Wed, 16 Sep 2009 09:43:18 -0700 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: References: <4AB0A478.4070007@erlang-consulting.com> Message-ID: <9C09417A-DAE4-4B12-B791-447B01472D8B@souja.net> I think the comparison is "Statistical Data Compression" versus "Semantic Data Compression". I don't really think that it's useful to classify data sharing as data compression. Bottom line, there's an optimization (and a clearly important one) that Erlang isn't doing. Exporting the term to binary and compressing it isn't particularly fruitful, I suspect. In a (horrifying) XML example, I think it's the difference between:
123 Any St Any Town, AS USA
123 Any St Any Town, AS USA 123 Any St Any Town, AS USA The original poster wants to be able to build structures like the first. It's possible within a single process, but there's a technicality in that sending the structure via a message expands it. The second poster is suggesting that you just GZIP the latter. While this would undoubtedly work, it's extremely clunky, probably not actually kinder on memory usage (due to generating lots of binaries), and doesn't really address the problem that an existing optimization could be generalized to work between processes. On Sep 16, 2009, at 9:31 AM, James Hague wrote: >> It seems that James is interested in data structure sharing IN THIS >> PARTICULAR CASE because it gives him a kind of data compression. >> Again, >> if that's the goal, why is ordinary data compression *not* the answer >> here? > > Can you clarify what you mean by "ordinary data compression?" > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From joelr1@REDACTED Wed Sep 16 20:02:14 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 16 Sep 2009 19:02:14 +0100 Subject: [erlang-questions] large scale deployments and netsplits In-Reply-To: <4AAF5328.3070708@erlang-consulting.com> References: <8209f740909140848m4582290dka8f9e0b6e72321c@mail.gmail.com> <0784F720-9D6A-44B1-8CCE-637FC8DD0621@gmail.com> <4AAE7D4B.7020909@erlang-consulting.com> <38E78E0735C49D448494F368109B801518F7AB@esealmw114.eemea.ericsson.se> <4AAF5328.3070708@erlang-consulting.com> Message-ID: <084CEC73-BEF4-4C1F-8A84-B947F18711CD@gmail.com> This is with a default net tick time. =INFO REPORT==== 16-Sep-2009::17:44:55 === module: stats elapsed: 20.014404 {flash_errors,1}: 24350 {flash_total_connected,1}: 479678 {flash_total_fails,1}: 493159 {flash_total_started,1}: 468819 {flash_total_sub_ack,1}: 444984 {flash_total_sub_req,1}: 448042 {flash_total_tcp_errors,1}: 802 {flash_connected,1}: 290320 {flash_started,1}: 253302 {flash_sub_ack,1}: 270553 {flash_sub_req,1}: 270999 {flash_tcp_errors,1}: 280 {"flash_connected/sec",1}: 14505 {"flash_started/sec",1}: 12655 {"flash_sub_ack/sec",1}: 13517 {"flash_sub_req/sec",1}: 13540 {"flash_tcp_errors/sec",1}: 13 I have 468,819 bots started on 100 small EC2 instances, 1 VM per instance. 479,678 bots connected. The number is higher than started because bots can connect multiple times, e.g. when there's an error. 290,320 bots connected in just the last 20s. =ERROR REPORT==== 16-Sep-2009::17:45:15 === ** Node 'janus@REDACTED' not responding ** ** Removing (timedout) connection ** ** at node janus@REDACTED ** =INFO REPORT==== 16-Sep-2009::17:45:15 === netsplit: down: 'janus@REDACTED', latency: 3.0330ms ** at node janus@REDACTED ** I put in a piece of code that saves nodes that are up and pings them every 15s, watching out for pongs and saving the latency. I print the latency once the node is down. What I see is a latency of just 3ms (within EC2 of course) when the node splits. What could be causing this? Thanks, Joel --- fastest mac firefox! http://wagerlabs.com From joelr1@REDACTED Wed Sep 16 20:05:08 2009 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 16 Sep 2009 19:05:08 +0100 Subject: [erlang-questions] large scale deployments and netsplits In-Reply-To: <4AAF5328.3070708@erlang-consulting.com> References: <8209f740909140848m4582290dka8f9e0b6e72321c@mail.gmail.com> <0784F720-9D6A-44B1-8CCE-637FC8DD0621@gmail.com> <4AAE7D4B.7020909@erlang-consulting.com> <38E78E0735C49D448494F368109B801518F7AB@esealmw114.eemea.ericsson.se> <4AAF5328.3070708@erlang-consulting.com> Message-ID: <68999015-9BC8-410D-BE04-65175809BE9B@gmail.com> One other thing... I'm seeing a LOT of messages like this, but no 'node down' messages. Any comments on this? Thanks, Joel --- =ERROR REPORT==== 16-Sep-2009::18:03:56 === global: 'janus@REDACTED' failed to connect to 'janus@REDACTED' ** at node janus@REDACTED ** =ERROR REPORT==== 16-Sep-2009::18:03:56 === global: 'janus@REDACTED' failed to connect to 'janus@REDACTED' ** at node janus@REDACTED ** --- fastest mac firefox! http://wagerlabs.com From jason@REDACTED Wed Sep 16 20:34:05 2009 From: jason@REDACTED (Jason Rexilius) Date: Wed, 16 Sep 2009 13:34:05 -0500 Subject: SysV IPC Message-ID: <4AB12F9D.30906@hostedlabs.com> Hello! In my searching for SysV or POSIX IPC interfaces for erlang I found an earlier post to this list by "Tim Potter" who mentioned he was starting to work on one. I'm wondering if anyone knows anything about it? Tim if you're still on the list? What I'm trying to do is get an interface to SysV message queues in erlang to coordinate with other legacy applications that are already using them. Is this an easy thing and I'm just being dumb? -jason From kiszl@REDACTED Wed Sep 16 20:34:41 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Wed, 16 Sep 2009 20:34:41 +0200 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: References: Message-ID: <4AB12FC1.3040809@tmit.bme.hu> Michael Turner wrote: > It seems that James is interested in data structure sharing IN THIS > PARTICULAR CASE because it gives him a kind of data compression. Again, > if that's the goal, why is ordinary data compression *not* the answer > here? If compression isn't the goal, what did I miss about the > (implicit) problem statement, to which sharing semantics in Erlang *is* > the answer? > > -michael turner > > A bit of analogy, but this is like suggesting that creating a symlink to a file, is not different from copying the file, and then tar.gz'ing the two. The result might be the same space-wise, but there are certain differences in usability... Z. From paul-trapexit@REDACTED Wed Sep 16 20:43:40 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Wed, 16 Sep 2009 11:43:40 -0700 (PDT) Subject: [erlang-questions] SysV IPC In-Reply-To: <4AB12F9D.30906@hostedlabs.com> References: <4AB12F9D.30906@hostedlabs.com> Message-ID: If it hasn't been done already, one could make a linked-in driver to front the sysv API. One would have to use threads to avoid blocking the emulator in some cases, but nonblocking operations could be done with a port_control and could be reasonably zippy. It sounds straightforward, albeit tedious. -- p On Wed, 16 Sep 2009, Jason Rexilius wrote: > Hello! > > In my searching for SysV or POSIX IPC interfaces for erlang I found > an earlier post to this list by "Tim Potter" who mentioned he was > starting to work on one. > > I'm wondering if anyone knows anything about it? Tim if you're still > on the list? > > What I'm trying to do is get an interface to SysV message queues in > erlang to coordinate with other legacy applications that are already > using them. Is this an easy thing and I'm just being dumb? > > -jason > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From max.lapshin@REDACTED Wed Sep 16 20:55:21 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 16 Sep 2009 22:55:21 +0400 Subject: [erlang-questions] SysV IPC In-Reply-To: References: <4AB12F9D.30906@hostedlabs.com> Message-ID: I want to follow question: UNIX domain sockets has very interesting possibility to pass fd. It is very useful when small acceptor is on 80 port, that connects with worker on non-privileged ports. Accepted TCP socket with client connection is passed via UNIX socket. Is there any way to accept such socket in Erlang. If no, than what way is to implement such functionality? From paul-trapexit@REDACTED Wed Sep 16 21:07:16 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Wed, 16 Sep 2009 12:07:16 -0700 (PDT) Subject: [erlang-questions] SysV IPC In-Reply-To: References: <4AB12F9D.30906@hostedlabs.com> Message-ID: On Wed, 16 Sep 2009, Max Lapshin wrote: > I want to follow question: UNIX domain sockets has very interesting > possibility to pass fd. > It is very useful when small acceptor is on 80 port, that connects > with worker on non-privileged ports. > Accepted TCP socket with client connection is passed via UNIX socket. > > Is there any way to accept such socket in Erlang. If no, than what way > is to implement such functionality? Well you can get at the file descriptor with a linked-in driver. The heat would be to convert that into a gen_tcp instance but I don't see anything in the public API that would facilitate that. You can do pretty much anything with a linked-in driver, including corrupting memory, deadlocking or crashing the emulator, etc; so they are not to be taken lightly. For this particular problem, there are other solutions: http://erlanganswers.com/web/mcedemo/PrivilegedPort.html -- p From mjtruog@REDACTED Wed Sep 16 21:11:03 2009 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 16 Sep 2009 12:11:03 -0700 Subject: [erlang-questions] SysV IPC In-Reply-To: References: <4AB12F9D.30906@hostedlabs.com> Message-ID: <4AB13847.9020108@gmail.com> port_control doesn't do async operations. you would need to use the async thread pool with driver_async. You would then be limited by the size of the async thread pool, which has separate job queues for each thread (so long jobs clog it up), and the first usage of driver_async would lock the driver so it can not be unloaded (or hotswapped). Paul Mineiro wrote: > If it hasn't been done already, one could make a linked-in driver to front > the sysv API. One would have to use threads to avoid blocking the > emulator in some cases, but nonblocking operations could be done with a > port_control and could be reasonably zippy. > > It sounds straightforward, albeit tedious. > > -- p > > On Wed, 16 Sep 2009, Jason Rexilius wrote: > > >> Hello! >> >> In my searching for SysV or POSIX IPC interfaces for erlang I found >> an earlier post to this list by "Tim Potter" who mentioned he was >> starting to work on one. >> >> I'm wondering if anyone knows anything about it? Tim if you're still >> on the list? >> >> What I'm trying to do is get an interface to SysV message queues in >> erlang to coordinate with other legacy applications that are already >> using them. Is this an easy thing and I'm just being dumb? >> >> -jason >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From paul-trapexit@REDACTED Wed Sep 16 21:15:42 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Wed, 16 Sep 2009 12:15:42 -0700 (PDT) Subject: [erlang-questions] SysV IPC In-Reply-To: <4AB13847.9020108@gmail.com> References: <4AB12F9D.30906@hostedlabs.com> <4AB13847.9020108@gmail.com> Message-ID: I actually think the right design would be port_control with IPC_NOWAIT, and only going to a thread pool if blocking semantics are desired *and* the initial call would block. In that case the return value from port_control could signal the caller to expect a (Erlang) message from the port. -- p On Wed, 16 Sep 2009, Michael Truog wrote: > port_control doesn't do async operations. you would need to use the > async thread pool with driver_async. You would then be limited by the > size of the async thread pool, which has separate job queues for each > thread (so long jobs clog it up), and the first usage of driver_async > would lock the driver so it can not be unloaded (or hotswapped). > > Paul Mineiro wrote: > > If it hasn't been done already, one could make a linked-in driver to front > > the sysv API. One would have to use threads to avoid blocking the > > emulator in some cases, but nonblocking operations could be done with a > > port_control and could be reasonably zippy. > > > > It sounds straightforward, albeit tedious. > > > > -- p > > > > On Wed, 16 Sep 2009, Jason Rexilius wrote: > > > > > >> Hello! > >> > >> In my searching for SysV or POSIX IPC interfaces for erlang I found > >> an earlier post to this list by "Tim Potter" who mentioned he was > >> starting to work on one. > >> > >> I'm wondering if anyone knows anything about it? Tim if you're still > >> on the list? > >> > >> What I'm trying to do is get an interface to SysV message queues in > >> erlang to coordinate with other legacy applications that are already > >> using them. Is this an easy thing and I'm just being dumb? > >> > >> -jason > >> > >> > >> ________________________________________________________________ > >> erlang-questions mailing list. See http://www.erlang.org/faq.html > >> erlang-questions (at) erlang.org > >> > >> > >> > > > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From paul-trapexit@REDACTED Wed Sep 16 21:18:49 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Wed, 16 Sep 2009 12:18:49 -0700 (PDT) Subject: [erlang-questions] SysV IPC In-Reply-To: References: <4AB12F9D.30906@hostedlabs.com> <4AB13847.9020108@gmail.com> Message-ID: Also, re thread pool disabling reloading, I've run into this, but I don't entirely understand the limitation. I've had good luck implementing my own thread pools for linked-in drivers so that I could continue to hot upgrade them. -- p On Wed, 16 Sep 2009, Paul Mineiro wrote: > I actually think the right design would be port_control with IPC_NOWAIT, > and only going to a thread pool if blocking semantics are desired *and* > the initial call would block. In that case the return value from > port_control could signal the caller to expect a (Erlang) message from the > port. > > -- p > > On Wed, 16 Sep 2009, Michael Truog wrote: > > > port_control doesn't do async operations. you would need to use the > > async thread pool with driver_async. You would then be limited by the > > size of the async thread pool, which has separate job queues for each > > thread (so long jobs clog it up), and the first usage of driver_async > > would lock the driver so it can not be unloaded (or hotswapped). > > > > Paul Mineiro wrote: > > > If it hasn't been done already, one could make a linked-in driver to front > > > the sysv API. One would have to use threads to avoid blocking the > > > emulator in some cases, but nonblocking operations could be done with a > > > port_control and could be reasonably zippy. > > > > > > It sounds straightforward, albeit tedious. > > > > > > -- p > > > > > > On Wed, 16 Sep 2009, Jason Rexilius wrote: > > > > > > > > >> Hello! > > >> > > >> In my searching for SysV or POSIX IPC interfaces for erlang I found > > >> an earlier post to this list by "Tim Potter" who mentioned he was > > >> starting to work on one. > > >> > > >> I'm wondering if anyone knows anything about it? Tim if you're still > > >> on the list? > > >> > > >> What I'm trying to do is get an interface to SysV message queues in > > >> erlang to coordinate with other legacy applications that are already > > >> using them. Is this an easy thing and I'm just being dumb? > > >> > > >> -jason > > >> > > >> > > >> ________________________________________________________________ > > >> erlang-questions mailing list. See http://www.erlang.org/faq.html > > >> erlang-questions (at) erlang.org > > >> > > >> > > >> > > > > > > > > > ________________________________________________________________ > > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > > erlang-questions (at) erlang.org > > > > > > > > > > > > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > > From mjtruog@REDACTED Wed Sep 16 21:26:42 2009 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 16 Sep 2009 12:26:42 -0700 Subject: [erlang-questions] SysV IPC In-Reply-To: References: <4AB12F9D.30906@hostedlabs.com> <4AB13847.9020108@gmail.com> Message-ID: <4AB13BF2.9020203@gmail.com> The limitation is just good to know before-hand. A separate thread pool makes sense, but the resulting port driver is also competing with the Erlang VM for the OS scheduling. Paul Mineiro wrote: > Also, re thread pool disabling reloading, I've run into this, but I > don't entirely understand the limitation. I've had good luck implementing > my own thread pools for linked-in drivers so that I could continue to hot > upgrade them. > > -- p > > On Wed, 16 Sep 2009, Paul Mineiro wrote: > > >> I actually think the right design would be port_control with IPC_NOWAIT, >> and only going to a thread pool if blocking semantics are desired *and* >> the initial call would block. In that case the return value from >> port_control could signal the caller to expect a (Erlang) message from the >> port. >> >> -- p >> >> On Wed, 16 Sep 2009, Michael Truog wrote: >> >> >>> port_control doesn't do async operations. you would need to use the >>> async thread pool with driver_async. You would then be limited by the >>> size of the async thread pool, which has separate job queues for each >>> thread (so long jobs clog it up), and the first usage of driver_async >>> would lock the driver so it can not be unloaded (or hotswapped). >>> >>> Paul Mineiro wrote: >>> >>>> If it hasn't been done already, one could make a linked-in driver to front >>>> the sysv API. One would have to use threads to avoid blocking the >>>> emulator in some cases, but nonblocking operations could be done with a >>>> port_control and could be reasonably zippy. >>>> >>>> It sounds straightforward, albeit tedious. >>>> >>>> -- p >>>> >>>> On Wed, 16 Sep 2009, Jason Rexilius wrote: >>>> >>>> >>>> >>>>> Hello! >>>>> >>>>> In my searching for SysV or POSIX IPC interfaces for erlang I found >>>>> an earlier post to this list by "Tim Potter" who mentioned he was >>>>> starting to work on one. >>>>> >>>>> I'm wondering if anyone knows anything about it? Tim if you're still >>>>> on the list? >>>>> >>>>> What I'm trying to do is get an interface to SysV message queues in >>>>> erlang to coordinate with other legacy applications that are already >>>>> using them. Is this an easy thing and I'm just being dumb? >>>>> >>>>> -jason >>>>> >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>>> erlang-questions (at) erlang.org >>>>> >>>>> >>>>> >>>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>>> >>>> >>>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >>> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From vinoski@REDACTED Wed Sep 16 21:35:41 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 16 Sep 2009 15:35:41 -0400 Subject: [erlang-questions] SysV IPC In-Reply-To: References: <4AB12F9D.30906@hostedlabs.com> Message-ID: <65b2728e0909161235q62e131b4y9942306e93857628@mail.gmail.com> On Wed, Sep 16, 2009 at 3:07 PM, Paul Mineiro wrote: > Well you can get at the file descriptor with a linked-in driver. The heat > would be to convert that into a gen_tcp instance but I don't see anything > in the public API that would facilitate that. > You can pass {fd, Fd} as an option to gen_tcp:connect or gen_tcp:listen to use an existing file descriptor. Works great. --steve From fritchie@REDACTED Wed Sep 16 22:02:14 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 16 Sep 2009 15:02:14 -0500 Subject: [erlang-questions] large scale deployments and netsplits In-Reply-To: Message of "Wed, 16 Sep 2009 19:02:14 BST." <084CEC73-BEF4-4C1F-8A84-B947F18711CD@gmail.com> Message-ID: <95993.1253131334@snookles.snookles.com> Joel Reymont wrote: jr> I have 468,819 bots started on 100 small EC2 instances, 1 VM per jr> instance. Joel, just to rule out something that I know has bitten me (and is worth mentioning to others in the community)... ... are the sizes of the Erlang VM processes big enough to run the risk of paging virtual memory pages to disk? If yes, then all bets are off for timely responses to network distribution communication, including liveness checks. Many Linux kernels have a value of /proc/sys/vm/swappiness equal to 40. That's just damn annoying for anyone who want to have a single OS process grow to a significant fraction of total RAM without having some of your RAM pages flushed to disk for "no good reason". IMHO, most Linux servers should have swappiness's value to 0, period. -Scott From fritchie@REDACTED Wed Sep 16 22:04:47 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 16 Sep 2009 15:04:47 -0500 Subject: [erlang-questions] SysV IPC In-Reply-To: Message of "Wed, 16 Sep 2009 22:55:21 +0400." Message-ID: <96159.1253131487@snookles.snookles.com> Max Lapshin wrote: ml> I want to follow question: UNIX domain sockets has very interesting ml> possibility to pass fd. There's such a driver available in the Jungerl collection of Erlang code. See http://jungerl.sourceforge.net/ -Scott From kaiduanx@REDACTED Thu Sep 17 00:26:32 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 16 Sep 2009 18:26:32 -0400 Subject: Concept of Side-effect Message-ID: Hi, Can someone explain what does side-effect means in Erlang? What is side-effect free function, and why we need to write side-effect free function? A concrete example is preferred. Thanks, kaiduan From ok@REDACTED Thu Sep 17 01:48:20 2009 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 17 Sep 2009 11:48:20 +1200 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <9C09417A-DAE4-4B12-B791-447B01472D8B@souja.net> References: <4AB0A478.4070007@erlang-consulting.com> <9C09417A-DAE4-4B12-B791-447B01472D8B@souja.net> Message-ID: <96443209-C47C-4208-8898-CCF237D62ACB@cs.otago.ac.nz> On Sep 17, 2009, at 4:43 AM, Jayson Vantuyl wrote: > I don't really think that it's useful to classify data sharing as > data compression. It's very much in the spirit of dictionary-based compression. > Bottom line, there's an optimization (and a clearly important one) > that Erlang isn't doing. And can't reasonably be *expected* to do. It is reasonable to expect Erlang to *preserve* sharing, as when sending a term to another process, because failing to do so can make space use blow up in a rather sickening way which it's hard for a programmer to detect. I sometimes think that for every use case there is an equal and opposite use case. In the case of memory, for example, we've got *space* issues and *cache* issues. Looking for existing copies of stuff can save you space, but it can do terrible things to your cache (bringing in stuff that it turns out you don't want). The tradeoffs depend on how much space you may save, how likely the saving is, and how well you can avoid looking at irrelevant stuff while looking for an existing copy. The programmer is in a better position to know these things than the Erlang compiler or runtime system. One thing I didn't quite understand was why the original data source is emitting stuff with lots of duplication in the first place. Fixing the duplication problem at the source has the added benefit of reducing the cost of getting the data into an Erlang process to start with. From kagato@REDACTED Thu Sep 17 03:02:53 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Wed, 16 Sep 2009 18:02:53 -0700 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <96443209-C47C-4208-8898-CCF237D62ACB@cs.otago.ac.nz> References: <4AB0A478.4070007@erlang-consulting.com> <9C09417A-DAE4-4B12-B791-447B01472D8B@souja.net> <96443209-C47C-4208-8898-CCF237D62ACB@cs.otago.ac.nz> Message-ID: <8CE9EFA4-9851-495E-ACCC-C54CF9EC9AAF@souja.net> > And can't reasonably be *expected* to do. It is reasonable > to expect Erlang to *preserve* sharing, as when sending a term > to another process, because failing to do so can make space use > blow up in a rather sickening way which it's hard for a > programmer to detect. I wasn't suggesting Erlang create sharing where there is none, just that it preserves sharing unless requested not to. > I sometimes think that for every use case there is an equal > and opposite use case. In the case of memory, for example, > we've got *space* issues and *cache* issues. Looking for > existing copies of stuff can save you space, but it can > do terrible things to your cache (bringing in stuff that it > turns out you don't want). The tradeoffs depend on how much > space you may save, how likely the saving is, and how well you > can avoid looking at irrelevant stuff while looking for an > existing copy. The programmer is in a better position to know > these things than the Erlang compiler or runtime system. I'm not suggesting that we do it for every, single piece of data. We already sort of do it for atoms, most numbers are small enough that it's not a big win, so the only real question is for lists / tuples / binaries. The win for lists and binaries is pretty huge. Binaries are rarely small. Lists get huge too. I'm not suggesting a full blown hash-cons solution, but some way to prevent invisible expansion is pretty critical. > One thing I didn't quite understand was why the original data > source is emitting stuff with lots of duplication in the first > place. Fixing the duplication problem at the source has the > added benefit of reducing the cost of getting the data into an > Erlang process to start with. I've run into this when working with a simple graph algorithm. Representing edges as {source,dest} was great for atoms and horrible for strings. All of my tests used atoms, but at runtime, the strings were being duplicated (because I was messaging them around). It was noticeable. Another problem I had was with a backend for the Linux Network Block Device. I was tossing around disk blocks (4k binaries) and had pathological memory usage really quickly. Real development has real problems with unnecessary data duplication. This is not a matter of optimization. Someone needs to finish one of the alternate heap implementations. Really. -- Jayson Vantuyl kagato@REDACTED From baryluk@REDACTED Thu Sep 17 04:46:01 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Thu, 17 Sep 2009 04:46:01 +0200 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <96443209-C47C-4208-8898-CCF237D62ACB@cs.otago.ac.nz> References: <4AB0A478.4070007@erlang-consulting.com> <9C09417A-DAE4-4B12-B791-447B01472D8B@souja.net> <96443209-C47C-4208-8898-CCF237D62ACB@cs.otago.ac.nz> Message-ID: <1253155562.24240.3778.camel@sredniczarny> Dnia 2009-09-17, czw o godzinie 11:48 +1200, Richard O'Keefe pisze: > On Sep 17, 2009, at 4:43 AM, Jayson Vantuyl wrote: > > I don't really think that it's useful to classify data sharing as > > data compression. > > It's very much in the spirit of dictionary-based compression. > > > Bottom line, there's an optimization (and a clearly important one) > > that Erlang isn't doing. > > And can't reasonably be *expected* to do. It is reasonable > to expect Erlang to *preserve* sharing, as when sending a term > to another process, because failing to do so can make space use > blow up in a rather sickening way which it's hard for a > programmer to detect. > > I sometimes think that for every use case there is an equal > and opposite use case. In the case of memory, for example, > we've got *space* issues and *cache* issues. Looking for > existing copies of stuff can save you space, but it can > do terrible things to your cache (bringing in stuff that it > turns out you don't want). The tradeoffs depend on how much > space you may save, how likely the saving is, and how well you > can avoid looking at irrelevant stuff while looking for an > existing copy. The programmer is in a better position to know > these things than the Erlang compiler or runtime system. > > One thing I didn't quite understand was why the original data > source is emitting stuff with lots of duplication in the first > place. Fixing the duplication problem at the source has the > added benefit of reducing the cost of getting the data into an > Erlang process to start with. > There is also another good aspect of preserving sharing. If we have X = Y = something(), then testing for equality X == Y is just simple testing (in VM) for pointer first, if they match return true, if not then test it deeper, if needed, also using this rule, and eventually in the end use plain value. This can speed up many things. It is quite strange to anybody, that for example time complexity (and memory also as sharing is lost) of algorithm are different in situations: 1) we built data structure D, and execute algorithm a(D) 2) we build data structure D, send it to P, which then executes a(D). Difference between them can be very very big factor, and can depend on the size of D. In pathological situation your algorithm can change for example from O(n^2) to O(n^4). And i can imagine even bigger problems than that. PS. Beyond the fact that message passing and term_to_binary should be aware of sharing and preserve as much as possible from it (eventualy with some optional flag), i am also interested with some experimental hybrid heap approach in erlang. -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From baryluk@REDACTED Thu Sep 17 04:46:09 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Thu, 17 Sep 2009 04:46:09 +0200 Subject: Fwd: [erlang-questions] Re: data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <73E79269-24E9-43AF-A537-4C31BB0F79F8@rogvall.se> References: <962980AB-2FFA-4F6A-B26E-0BBB45178217@rogvall.se> <73E79269-24E9-43AF-A537-4C31BB0F79F8@rogvall.se> Message-ID: <1253155569.24240.3781.camel@sredniczarny> Dnia 2009-09-15, wto o godzinie 22:38 +0200, Tony Rogvall pisze: > > Here is nice term encoder/decoder (below) that will solve the size > > problem. > > The algorithm is not efficient (in Erlang) but if/when implemented > > in C it could be used as a bif. > > Still it will produce decent results on (all) shared data. > > > > Wow, really nice work. Quickly and simple. te module is really interesting. but still needs some optimazations in encode function like finding variables which are used only once and merging / propagating them into original structure. te:encode([ala,ewa,ala,ewa,{ala,bzium,"xasdddd","asdddd"},{foo,bar}]) is {ala,ewa,97,115,100,[], [5|6], [5|7], [5|8], [5|9], [4|10], [3|11], 120, [13|12], bzium, {1,15,14,12}, bar,foo, {18,17}, [19|6], [16|20], [2|21], [1|22], [2|23], [1|24], 25} it should be something more like: {ala,ewa,{l,"asdddd"},[], 120, [5|3], bzium. {1,6,7,3}, {l,{bar,foo}}, [2,1,2,1,8,9]} where {l, X} means literal value of X. or maybe even: {ala,ewa,{l,"asdddd"},[], 120, [2,1,2,1,{1,{l,bzium},[5|3],3},{l,{foo,bar}}]} or even {ala,ewa,{l,"asdddd"},[], [2,1,2,1,{1,{l,bzium},[{l,120}|3],3},{l,{foo,bar}}]} depending what is "smaller" I think that "optimization" of the output of the encode/1 function can be done as separate pass after encode/1. -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From ok@REDACTED Thu Sep 17 07:52:48 2009 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 17 Sep 2009 17:52:48 +1200 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <8CE9EFA4-9851-495E-ACCC-C54CF9EC9AAF@souja.net> References: <4AB0A478.4070007@erlang-consulting.com> <9C09417A-DAE4-4B12-B791-447B01472D8B@souja.net> <96443209-C47C-4208-8898-CCF237D62ACB@cs.otago.ac.nz> <8CE9EFA4-9851-495E-ACCC-C54CF9EC9AAF@souja.net> Message-ID: <5C939DED-2164-4B9F-B395-057D4E903F6F@cs.otago.ac.nz> On Sep 17, 2009, at 1:02 PM, Jayson Vantuyl wrote: >I've run into this when working with a simple graph algorithm. >Representing edges as {source,dest} was great for atoms and >horrible for strings. All of my tests used atoms, but at runtime, >the strings were being duplicated (because I was messaging them >around). It was noticeable. This sounds to me like a perfect example where duplication should be avoided at the source. Graphs should be sent as {graph,{NodeNames},[{F1,T1},...,{Fn,Tn}]} where the Fi and Ti are indices into the {NodeNames} tuple. > > Another problem I had was with a backend for the Linux Network Block > Device. I was tossing around disk blocks (4k binaries) and had > pathological memory usage really quickly. > > Real development has real problems with unnecessary data > duplication. This is not a matter of optimization. Someone needs > to finish one of the alternate heap implementations. Really. There seem to be two issues confused here. One of them is the fact that when you send a message, all sharing within the message is removed (except that large binaries are not supposed to be copied). We *agree* that this is a bad thing. My message was explicit that Erlang should preserve sharing. But it didn't sound as though that's what the original poster was talking about. I may well have misunderstood; it would not be the first time. It's claimed that preserving sharing would raise the cost of message sending too high. There's an answer to that. Set a modest threshold, say 100 cells or so, and try the existing way of sending. But if that threshold is crossed, give up, and start over with a method that preserves within-message sharing. From ok@REDACTED Thu Sep 17 08:00:34 2009 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 17 Sep 2009 18:00:34 +1200 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <1253155562.24240.3778.camel@sredniczarny> References: <4AB0A478.4070007@erlang-consulting.com> <9C09417A-DAE4-4B12-B791-447B01472D8B@souja.net> <96443209-C47C-4208-8898-CCF237D62ACB@cs.otago.ac.nz> <1253155562.24240.3778.camel@sredniczarny> Message-ID: On Sep 17, 2009, at 2:46 PM, Witold Baryluk wrote: > It is quite strange to anybody, that for example time complexity > (and memory also as sharing is lost) of algorithm are different in > situations: > > 1) we built data structure D, and execute algorithm a(D) > 2) we build data structure D, send it to P, which then executes a(D). My message explicitly said that I thought that Erlang should *PRESERVE* sharing, including in message passing. You didn't need to persuade me of something I already argued for. The point at issue, as I understood it, was an "OPTIMIZATION", namely INTRODUCING sharing. For example, if memory serves me correctly, whenever the Logix implementation of Flat Concurrent Prolog successfully executed a test X == Y, it smashed one of them to refer to the other. That's run-time introduction of sharing. The reason why Logix did that was that they didn't _have_ atoms, only strings, in order to avoid having to lock a centralised symbol table. But they wanted strings to _act_ rather like atoms. I recommend within a during message process passing preserve sharing YES[1] YES[2] introduce sharing NO NO [1] This is what Erlang does now. [2] This is a change, and it's going to be tricky to do it without slowing "normal" cases down. And I'm counting term_to_binary/binary_to_term here, because of their use when messages are passed over a wire. From rapsey@REDACTED Thu Sep 17 08:28:32 2009 From: rapsey@REDACTED (Rapsey) Date: Thu, 17 Sep 2009 08:28:32 +0200 Subject: [erlang-questions] Concept of Side-effect In-Reply-To: References: Message-ID: <97619b170909162328g39f8d0dah21a1158c586157c@mail.gmail.com> It means the same thing in Erlang as it does anywhere else. A function has a side effect, if it changes some state. Doing something with files, sockets, databases, processes etc. are side effects. A side effect free function is dependent only on its input values. When it has finished, it produces an output from those input values and nothing else (list_to_binary/1 for instance). You should strive to have as many side effect free functions as possible. They are easy to test, easy to understand and easy to work with. Functions that change state is where your bugs will generally be. The more you change state, the more complex your program becomes. If you change state all over the place, you are creating something very difficult to understand and debug. Sergej On Thu, Sep 17, 2009 at 12:26 AM, Kaiduan Xie wrote: > Hi, > > Can someone explain what does side-effect means in Erlang? What is > side-effect free function, and why we need to write side-effect free > function? A concrete example is preferred. > > Thanks, > > kaiduan > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From leap@REDACTED Thu Sep 17 08:54:17 2009 From: leap@REDACTED (Michael Turner) Date: Thu, 17 Sep 2009 06:54:17 +0000 Subject: Can people please stop misinterpreting what I wrote? In-Reply-To: <96443209-C47C-4208-8898-CCF237D62ACB@cs.otago.ac.nz> Message-ID: <3ZEHKAhY.1253170457.9451140.leap@gol.com> On 9/16/2009, "Richard O'Keefe" wrote: > >On Sep 17, 2009, at 4:43 AM, Jayson Vantuyl wrote: >> I don't really think that it's useful to classify data sharing as >> data compression. > >It's very much in the spirit of dictionary-based compression. And just so that everybody's clear: I *didn't* "classify data sharing as data compression." I *asked* what, exactly, in the problem that James' code is intended to solve, couldn't be solved in classic Erlang style with data compression? Did I misunderstand the problem? Oh, and just to be doubly clear: I've used shared data a lot in data structure design, from the early 1980s onward, and in some shared-memory multi-processor applications back as far as the late 1980s. I'm not religiously opposed to data structure sharing. I actually think it can be kind of cool, in circumstances where it's justified. I just have a question here, one I'd like see an answer for. I *don't* like seeing that question re-written into some questions I never asked, much less into some statements I never made. -michael turner >> Bottom line, there's an optimization (and a clearly important one) >> that Erlang isn't doing. > >And can't reasonably be *expected* to do. It is reasonable >to expect Erlang to *preserve* sharing, as when sending a term >to another process, because failing to do so can make space use >blow up in a rather sickening way which it's hard for a >programmer to detect. > >I sometimes think that for every use case there is an equal >and opposite use case. In the case of memory, for example, >we've got *space* issues and *cache* issues. Looking for >existing copies of stuff can save you space, but it can >do terrible things to your cache (bringing in stuff that it >turns out you don't want). The tradeoffs depend on how much >space you may save, how likely the saving is, and how well you >can avoid looking at irrelevant stuff while looking for an >existing copy. The programmer is in a better position to know >these things than the Erlang compiler or runtime system. > >One thing I didn't quite understand was why the original data >source is emitting stuff with lots of duplication in the first >place. Fixing the duplication problem at the source has the >added benefit of reducing the cost of getting the data into an >Erlang process to start with. > > >________________________________________________________________ >erlang-questions mailing list. See http://www.erlang.org/faq.html >erlang-questions (at) erlang.org > > From ulf.wiger@REDACTED Thu Sep 17 10:21:12 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 17 Sep 2009 10:21:12 +0200 Subject: [erlang-questions] Can people please stop misinterpreting what I wrote? In-Reply-To: <3ZEHKAhY.1253170457.9451140.leap@gol.com> References: <3ZEHKAhY.1253170457.9451140.leap@gol.com> Message-ID: <4AB1F178.80909@erlang-consulting.com> Michael Turner wrote: > > I *asked* what, exactly, in the problem that > James' code is intended to solve, couldn't be solved in > classic Erlang style with data compression? Going back to the OP's actual question is a novel concept, but not nearly as fun as redefining the question into what suits your own ambition, and going from there. :) Having re-read what James initially asked for, it seemed to me as a pretty clever poor-man's version for answering the question "are these two objects not just equal, but the /same/ object?". This cannot be done today in Erlang. If it could, it would be possible to write your own sharing-preserving term_to_binary(). The suggestion from James was that an access structure library could fake that by telling the caller "this is the object I have", and the caller could use that instead, and thereby achieving a higher degree of sharing. (To address your question then, compression seems to not address this problem, as the idea of having clever helper functions to help make use of sharing implies extremely lightweight semantics. Often, making use of sharing not only reduces memory consumption, but also improves performance, and James' suggesion really doesn't carry any overhead in terms of execution time - but compression definitely does.) The same effect could be achieved by calling gb_trees:find() first, and then perhaps calling insert, but I assume James would prefer to achieve this effect without having to access the tree twice? But given that an API change is asked for in a set of core libraries (since the APIs of the different access structures are harmonized, it wouldn't be just gb_trees), it begs the question - do we want to promote the use of sharing (inside terms)? If yes, what does that imply? The big, ugly problem that needs to be solved then is the one where you can in fact kill the entire VM if a term that aggressively uses sharing happens to get passed in a message (which can happen e.g. if the process crashes and there are linked processes). BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From bgustavsson@REDACTED Thu Sep 17 12:28:10 2009 From: bgustavsson@REDACTED (Bjorn Gustavsson) Date: Thu, 17 Sep 2009 12:28:10 +0200 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: References: Message-ID: <6672d0160909170328i1d223958ucccd486abb0afea7@mail.gmail.com> On Mon, Sep 14, 2009 at 6:22 PM, James Hague wrote: > This is easy to do in Erlang, because the compiler is very literal: > > fix_tuple({H, H}) -> {H, H}; > ... > > That ensures that identical looking elements in the tuple are sharing > memory locations. But there is absolutely no reason the compiler has > to do this. It would be perfectly valid to optimize away the entire > function, just returning the original value. We have no intention of introducing "optimizations" that would break that code. > Would any existing standard library functions make this nicer? What I > really want is to have a gb_trees:insert function that returns > {NewTree, InsertedValue} where InsertedValue references existing data > (unless it wasn't already in the tree; in that case, InsertedValue is > exactly what I passed in). Then I can happily use InsertedValue, > knowing data is being shared. In Wings3D (which is an application that depends heavily on sharing), I use a gb_tree to keep track of values that I want to share. I enter each term as both the key and the value and then I just do gb_trees:get/2 to retrieve the shared value. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From leap@REDACTED Thu Sep 17 13:05:14 2009 From: leap@REDACTED (Michael Turner) Date: Thu, 17 Sep 2009 11:05:14 +0000 Subject: No, really, please stop misinterpreting what I wrote In-Reply-To: <4AB1F178.80909@erlang-consulting.com> Message-ID: On 9/17/2009, "Ulf Wiger" wrote: >Michael Turner wrote: >> >> I *asked* what, exactly, in the problem that >> James' code is intended to solve, couldn't be solved in > > classic Erlang style with data compression? > >Going back to the OP's actual question is a novel concept, >but not nearly as fun as redefining the question into what >suits your own ambition, and going from there. :) You're saying I have some ambition to use data compression in Erlang for reducing the amount of space taken by a long list of strings with many repeated strings? Really? Why am I not aware of having any such ambition then? Oh, I see: you misinterpreted what I wrote. (Unless *I'm* misinterpreting what *you* wrote. Am I?) >Having re-read what James initially asked for, .... I just re-read what James initially asked for. Then he changed it when somebody pointed out a mistake. THEN it became the concrete problem that I asked about: a long list of strings, with many of them repeated -- how to save space? > .... it seemed >to me as a pretty clever poor-man's version for answering >the question "are these two objects not just equal, but >the /same/ object?". > >This cannot be done today in Erlang. So what? If you have unique identifiers for objects stored in some index structure for efficiently determining whether you've seen one of them before, why isn't that enough? If that doesn't give you the lightest of "lightweight semantics", so what? Just by using the right algorithm for the problem, you get most of the optimization benefits you're going to get. So what's really important is (as usual) is understanding the real problem, and what sort of optimizations are really necessary. Richard O'Keefe pulls back and asks: Why are you getting a long stream of strings with many repeated in the first place? Because that IS a problem, and data-sharing might be just a bandaid over it. >If it could, it would be possible to write your own >sharing-preserving term_to_binary(). Yes, and if there was some way embed self-modifying assembly language code in Erlang, you could .... Look, there are always lots of possibilities in software, because that's its defining characteritic. Relatively few of those possibilities are wise choices. What's the cost of this choice? (Possible answer: yet another way to crash Erlang, if I understand your reservation expressed below.) >(To address your question then, compression seems to not >address this problem, as the idea of having clever helper >functions to help make use of sharing implies extremely >lightweight semantics. Often, making use of sharing not >only reduces memory consumption, but also improves >performance, and James' suggesion really doesn't carry >any overhead in terms of execution time - but compression >definitely does.) As Richard pointed out, there is the memory hierachy to be considered. (And with RAM access time ratios of around 200-to-1 between off-chip main memory and L1 cache, these days, you're not being realistic about optimization unless you're thinking about that fact ALL THE TIME, at least to be sure that it doesn't matter for your problem.) If the access pattern is usually that you need decompress a portion of a list, then you read from that portion extensively before moving on, you might be better off using compression than anything else, simply because it saves much more memory. That is, it *would*, in the case of a long list, with many repeated strings, which was the first concrete example James brought up. But what IS the access pattern? And why is it like that? Not enough to go on, here. >The same effect could be achieved by calling gb_trees:find() >first, and then perhaps calling insert, but I assume James >would prefer to achieve this effect without having to access >the tree twice? I assume we'd all prefer everything to be fast, all the time. That seems to be a preference at Ericsson. In the telephony switch at Ericsson that has the most Erlang code in it (AFAIK), there's a lot of Erlang code. But as I understand it, also a lot of C code in that same switch. They don't use Erlang in that system for speed. They use it for robustness and expressive power for the concurrency-oriented aspect of the system. >But given that an API change is asked for in a set of core >libraries (since the APIs of the different access structures >are harmonized, it wouldn't be just gb_trees), it begs the >question - do we want to promote the use of sharing (inside >terms)? If yes, what does that imply? The big, ugly problem >that needs to be solved then is the one where you can in fact >kill the entire VM if a term that aggressively uses sharing >happens to get passed in a message (which can happen e.g. >if the process crashes and there are linked processes). I'm looking at Erlang for implementing a linguist's model of cognition. The data structures required to do this involve not just a lot of sharing, but a lot of circularity. I've decided (I hope wisely) that the best way to represent these data structures is to have nodes represented by processes, and links between the nodes represented by process IDs, and have the amount of data at each node, and in each message, bounded by a small constant. I might use the digraph library to maintain a map that mirrors the process structure. But if it turns out I don't have to do that, I might not. I'm not looking at doing it this way to save memory. Nor am I doing it to make the thing faster (except perhaps in some very long term view, in which going concurrency-oriented is the ultimate speedup). I'm taking this approach because it looks like the best way to make the thing actually work. Which, interestingly, is also very important in telephony. In a paper I can't immediately identify right now, the authors remarked that Erlang programmers often spend a fair amount of time trying to measure what's fast in Erlang, then writing stuff using what they discover is fast. The authors were disturbed, saying that they'd prefer that Erlang programmers implement things so as to be *clear* in Erlang, so that maintainers of the Erlang interpreter and compiler would know what to target for optimization. And now we have James Hague pointing out that an ambiguity (?) in Erlang semantics means that he's now got a great optimization, but one that might go away if Erlang is allowed to optimize it out. Well, C is really great for optimizing stuff, but I've had my "lightweight semantics" optimizations in that language optimized out by a subsequent release of the compiler, and I deserved what I got. One of C's co-designers contributed the now-famous saying: "Premature optimization is the root of all evil in software." Probably because, with that language being what it is, he'd seen a lot more evil than most of us. It's hard habit to acquire (because optimization can be so much fun), but it's a good one: the habit of asking yourself, in contemplating an optimization, "Would this solve the real problem? Or just compensate for it?" Sometimes (especially in legacy computing environments) compensation is the best you can do -- there's a black box you can't get into and fix. But if that's the reason, you should know it. And if there's a better language for those compensating optimizations, you should use it. -michael turner From leap@REDACTED Thu Sep 17 13:18:06 2009 From: leap@REDACTED (Michael Turner) Date: Thu, 17 Sep 2009 11:18:06 +0000 Subject: No subject In-Reply-To: <6672d0160909170328i1d223958ucccd486abb0afea7@mail.gmail.com> Message-ID: On 9/17/2009, "Bjorn Gustavsson" wrote: >On Mon, Sep 14, 2009 at 6:22 PM, James Hague wrote: >> This is easy to do in Erlang, because the compiler is very literal: >> >> fix_tuple({H, H}) -> {H, H}; >> ... >> >> That ensures that identical looking elements in the tuple are sharing >> memory locations. But there is absolutely no reason the compiler has >> to do this. It would be perfectly valid to optimize away the entire >> function, just returning the original value. > >We have no intention of introducing "optimizations" that would break >that code. But is James correct in implying that the semantics of Erlang admit of such interpretations? -michael turner From bgustavsson@REDACTED Thu Sep 17 14:08:45 2009 From: bgustavsson@REDACTED (Bjorn Gustavsson) Date: Thu, 17 Sep 2009 14:08:45 +0200 Subject: [erlang-questions] Re: In-Reply-To: References: <6672d0160909170328i1d223958ucccd486abb0afea7@mail.gmail.com> Message-ID: <6672d0160909170508obb13ed4hdfb85389802e1cf6@mail.gmail.com> On Thu, Sep 17, 2009 at 1:18 PM, Michael Turner wrote: >>> >>> That ensures that identical looking elements in the tuple are sharing >>> memory locations. But there is absolutely no reason the compiler has >>> to do this. It would be perfectly valid to optimize away the entire >>> function, just returning the original value. >> >>We have no intention of introducing "optimizations" that would break >>that code. > > But is James correct in implying that the semantics of Erlang admit of > such interpretations? Probably. As far as I know, there is no specification or documentation that explicitly says that an Erlang implementation must preserve sharing within a process. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From ulf.wiger@REDACTED Thu Sep 17 15:53:29 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 17 Sep 2009 15:53:29 +0200 Subject: No, really, please stop misinterpreting what I wrote In-Reply-To: References: Message-ID: <4AB23F59.7040700@erlang-consulting.com> Michael Turner wrote: > > On 9/17/2009, "Ulf Wiger" wrote: > >> Michael Turner wrote: >>> I *asked* what, exactly, in the problem that >>> James' code is intended to solve, couldn't be solved in >>> classic Erlang style with data compression? >> Going back to the OP's actual question is a novel concept, >> but not nearly as fun as redefining the question into what >> suits your own ambition, and going from there. :) > > You're saying I have some ambition to use data compression in Erlang for > reducing the amount of space taken by a long list of strings with many > repeated strings? No, I'm not saying that. But if you do, I have no problem with it. :) > Oh, I see: you misinterpreted what I wrote. That's what I meant, and pls note the smiley. I was just jokingly alluding to the fact that discussions often meander away from what the original poster was asking for. >> Having re-read what James initially asked for, .... > > I just re-read what James initially asked for. Then he changed it when > somebody pointed out a mistake. THEN it became the concrete problem > that I asked about: a long list of strings, with many of them repeated > -- how to save space? It basically split into different sub-threads, one being about the basic issues of implicit sharing within terms (in which some posters confused this sort of sharing with the no-shared semantics of concurrent Erlang.) James also wrote that he wanted a BIF that would find possibilities of sharing in a term and exploit them. Some posts refered to the "built-in compression" available in Erlang in the form of term_to_binary(Term, [compressed]). I think one could say that compressing using term_to_binary/2 is at the far end of the spectrum in this matter. If you compress to a binary, you lose every possiblity of pattern matching (except check for equality), where as "compression" using implicit sharing is completely transparent to pattern matching. That is, through fairly simple means, you can make use of sharing and still keep to ideomatic Erlang. Depending on the problem, sharing can give amazing yield with minimal work. >> .... it seemed >> to me as a pretty clever poor-man's version for answering >> the question "are these two objects not just equal, but >> the /same/ object?". >> >> This cannot be done today in Erlang. > > So what? [...] > >> If it could, it would be possible to write your own >> sharing-preserving term_to_binary(). > > Yes, and if there was some way embed self-modifying assembly > language code in Erlang, you could .... > > Look, there are always lots of possibilities in software, > because that's its defining characteritic. Relatively few > of those possibilities are wise choices. What's the cost > of this choice? (Possible answer: yet another way to crash > Erlang, if I understand your reservation expressed > below.) There are several problems where smart data structures can be used /only/ if one is allowed to rely on implicit sharing. I'm pretty sure that QuickCheck, for example, relies heavily on it, and also has a serious problem with the fact that the 'rule base' cannot be passed to another process. This has to do with another aspect of Erlang - if the process controlling a test run receives an untrappaple exit, the shrinking process won't work, since all information about the run will be gone. The simple remedy would be to spawn a process that executes one run and then reports back, but this can't be done, since the data structure that needs to be passed along 'explodes'. This particular case in itself answers the 'so what'. If it had been feasible to do it in Erlang, the authors of QuickCheck surely would have done it by now. Also, I would not dream to suggest that they should simply choose another data structure in order to solve the problem. Their knowledge in that area far exceeds mine. I've also had discussions with experienced Haskell and OCaml programmers who felt that the loss of sharing when sending data from one node to another was sufficient reason in itself not to use Erlang - since the use of sharing in functional programming is such a powerful tool. Obviously, I don't share the view that this disqualifies Erlang entirely, but at least I'm not alone in thinking that (a) sharing is a good thing, and (b) the occasional loss of sharing, partly beyond the programmer's control, is a bad thing. > I assume we'd all prefer everything to be fast, all the time. That > seems to be a preference at Ericsson. In the telephony switch at > Ericsson that has the most Erlang code in it (AFAIK), there's a lot of > Erlang code. But as I understand it, also a lot of C code in that same > switch. They don't use Erlang in that system for speed. They use it > for robustness and expressive power for the concurrency-oriented aspect > of the system. OT, but this is not a correct description. I assume you refer to the AXD 301? Most of the C code in the AXD 301 is low-level device processor code, written in C mainly for historical reasons. In the first versions, those device processors did not have the CPU or memory capacity to run Erlang, and instead ran VRTX - later OSE Delta. The programming style in that environment led to a lot of code duplication (there were more than 100 different device boards produced in the AXD 301 over the years). While it was correct then that it wouldn't have been possible to use Erlang at all, much less get sufficient characteristics with it, newer generations of ARM processors and the cost of RAM changed that. It would have been possible to use Erlang in the modern device boards of the AXD 301, and in many ways it would be preferable, but the cost of re-writing the core device board software made it a fairly uninteresting alternative at the end - the move to a more homogeneous network architecture and using IP across the board also changed the balance between control processor development and device processor development. There were other blocks of C code, in the form of 3rd party applications that were integrated rather than writing everything from scratch. This was not done for performance, usually, but more because of market considerations, and sometimes credibility (it's not necessarily a good idea to start writing your own BGP stack, for example, as a buggy BGP stack can cause endless trouble on the Internet). Indeed, it has been the experience at Ericsson that for signaling applications, Erlang application often have outstanding performance, especially when aspects such as load tolerance and portability to new and more powerful architectures are taken into account. This was also shown by the Herriott-Watt studies together with Motorola. Having said this, it is certainly true that the main reason for using Erlang is that it offers a very productive way of reaching a robust and well-working system. As Joe often says, if the product is /fast enough/, this is much more important than raw speed. > In a paper I can't immediately identify right now, the authors remarked > that Erlang programmers often spend a fair amount of time trying to > measure what's fast in Erlang, then writing stuff using what they > discover is fast. The authors were disturbed, saying that they'd > prefer that Erlang programmers implement things so as to be *clear* in > Erlang, so that maintainers of the Erlang interpreter and compiler would > know what to target for optimization. I don't know which paper you are refering to, but this is an argument that I have personally put forth several times in various contexts, on this list and elsewhere. The problem is not just that people write 'optimized' code at the expense of clarity. They often end up optimizing things that don't matter, and miss things, like algorithm optimization that really does. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From gene.tani@REDACTED Thu Sep 17 17:07:38 2009 From: gene.tani@REDACTED (Gene Tani) Date: Thu, 17 Sep 2009 08:07:38 -0700 (PDT) Subject: Concept of Side-effect In-Reply-To: References: Message-ID: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> On Sep 16, 3:26?pm, Kaiduan Xie wrote: > Hi, > > Can someone explain what does side-effect means in Erlang? What is > side-effect free function, and why we need to write side-effect free > function? A concrete example is preferred. > > Thanks, > > kaiduan > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org http://blog.tornkvist.org/blog.yaws?id=1239107937892262 http://www.cs.chalmers.se/Cs/Grundutb/Kurser/ppxt/HT2007/general/languages/armstrong-erlang_history.pdf (PDF page 14) and a FP vs. OO debate: http://news.ycombinator.com/item?id=493963 From james.hague@REDACTED Thu Sep 17 17:43:58 2009 From: james.hague@REDACTED (James Hague) Date: Thu, 17 Sep 2009 10:43:58 -0500 Subject: [erlang-questions] Re: No, really, please stop misinterpreting what I wrote In-Reply-To: <4AB23F59.7040700@erlang-consulting.com> References: <4AB23F59.7040700@erlang-consulting.com> Message-ID: Wow, this discussion has blown up :) Originally, what I wanted was a way to manually make sure that terms are shared. This is easy to do by having a data structure function that works like this: {New_Tree, Item_in_Tree} = some_tree:insert(Tree, Item) None of the library functions have this, as far as I can tell. I pass in a some data item, like a string, and it gets searched for in a tree (or similar data structure). If it exists, the same tree and a pointer to the item already in the tree are passed back. I can use the returned item rather than the original, knowing that sharing is occurring if it's possible. If the item doesn't exist, then I get back the new tree and exactly the same item I passed in. The reason I brought up Erlang semantics here, is that so that the check for if something is shared or not could be handled at the runtime level. A tangential topic is about enforcing sharing across all data structures--with hash consing, for example--but that's a major change that doesn't make sense. Then there's the issue of terms with shared data blowing up when converted to a binary and back, or sent to another process. This is a potentially critical flaw in some cases, but it's more easily dealt with than the previous paragraph. And it's not what I was originally talking about :) From kamiseq@REDACTED Thu Sep 17 17:48:07 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Thu, 17 Sep 2009 17:48:07 +0200 Subject: initializing process Message-ID: can somebody corrects me (and hopefully help with my problem). so the problem is like that 1)Im starting a server in erlang and among others I create one process that needs to decide if another local computer is up before whole server's initialization process is done. 2)the process is simple gen_server (AND HERE IS A PROBLEM) 3)as long as I dont return from init/1 fun, my process is not seen (I cant really use self() - is that correct??) by others so I CANT RECEIVE ANY MESSAGES. so I though I can spawn another processes while in init/1 that will loop and ping another side if it is ready. but I am afraid that there might be a chance that the gen_server will not yet return from init and spawned worker will try to send some notification, so or I will miss it or there will be some error. is this really a problem or Im just missing something here that is so obvious. All I really need is to init state of the gen_server, then start dynamically initializing the server so I cant hard code all conditions in init fun and I need to communicate with other processes. hope it is more or less clear take care pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From gleber.p@REDACTED Thu Sep 17 17:58:28 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 17 Sep 2009 17:58:28 +0200 Subject: [erlang-questions] initializing process In-Reply-To: References: Message-ID: <14f0e3620909170858p2850227fm5e6a8b3aaf6b42c9@mail.gmail.com> 2009/9/17 pawe? kami?ski : > (I cant really use self() - is that correct??) by others so I CANT RECEIVE ANY > MESSAGES. Why do you think so? You can use self() in init/1 and you can receive messages. No problem here. But you have to use message primitives (! and receive), not the gen_server's counterparts. > so I though I can spawn another processes while in init/1 that > will loop and ping another side if it is ready. but I am afraid that there > might be a chance that the gen_server will not yet return from init and > spawned worker will try to send some notification, so or I will miss it or > there will be some error. There is no problem with synchronizing two such processes. And you don't have to spawn any process at all - you can do anything you want in init/1. Either way this process will be stuck in init/1, so no reason to spawn another process (unless there is something else important in your situation). If the initialization can take some extended time I'd use gen_fsm instead, which will have two states (e.g. 'initializing' and 'ready'). > is this really a problem or Im just missing something here that is so > obvious. > All I really need is to init state of the gen_server, then start dynamically > initializing the server so I cant hard code all conditions in init fun and I > need to communicate with other processes. > > hope it is more or less clear > > take care > > pozdrawiam > Pawe? Kami?ski > > kamiseq@REDACTED > pkaminski.prv@REDACTED Pozdrawiam serdecznie, Gleb Peregud From ulf.wiger@REDACTED Thu Sep 17 18:06:04 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 17 Sep 2009 18:06:04 +0200 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: References: Message-ID: <4AB25E6C.2050205@erlang-consulting.com> James Hague wrote: > I've run into several cases where enforcing the sharing of data > resulted in a significant memory savings. BTW, here is an example from OTP, written by Robert Virding, no less, so it has to be an example of the type of code that for which Erlang was originally intended. :) %% expand_segs(Segs, EmptySeg) -> NewSegs. %% contract_segs(Segs) -> NewSegs. %% Expand/contract the segment tuple by doubling/halving the number %% of segments. We special case the powers of 2 upto 32, this should %% catch most case. N.B. the last element in the segments tuple is %% an extra element containing a default empty segment. expand_segs({B1}, Empty) -> {B1,Empty}; expand_segs({B1,B2}, Empty) -> {B1,B2,Empty,Empty}; expand_segs({B1,B2,B3,B4}, Empty) -> {B1,B2,B3,B4,Empty,Empty,Empty,Empty}; expand_segs({B1,B2,B3,B4,B5,B6,B7,B8}, Empty) -> {B1,B2,B3,B4,B5,B6,B7,B8, Empty,Empty,Empty,Empty,Empty,Empty,Empty,Empty}; expand_segs({B1,B2,B3,B4,B5,B6,B7,B8,B9,B10,B11,B12,B13,B14,B15,B16}, Empty) -> {B1,B2,B3,B4,B5,B6,B7,B8,B9,B10,B11,B12,B13,B14,B15,B16, Empty,Empty,Empty,Empty,Empty,Empty,Empty,Empty, Empty,Empty,Empty,Empty,Empty,Empty,Empty,Empty}; expand_segs(Segs, Empty) -> list_to_tuple(tuple_to_list(Segs) ++ lists:duplicate(tuple_size(Segs), Empty)). It is from dict.erl, and used to expand the hash table. Very deliberate use of sharing. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From paul-trapexit@REDACTED Thu Sep 17 18:18:57 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Thu, 17 Sep 2009 09:18:57 -0700 (PDT) Subject: [erlang-questions] Re: No, really, please stop misinterpreting what I wrote In-Reply-To: References: <4AB23F59.7040700@erlang-consulting.com> Message-ID: On Thu, 17 Sep 2009, James Hague wrote: > Wow, this discussion has blown up :) > > Originally, what I wanted was a way to manually make sure that terms > are shared. This is easy to do by having a data structure function > that works like this: > > {New_Tree, Item_in_Tree} = some_tree:insert(Tree, Item) > > None of the library functions have this, as far as I can tell. I pass > in a some data item, like a string, and it gets searched for in a tree > (or similar data structure). If it exists, the same tree and a pointer > to the item already in the tree are passed back. I can use the > returned item rather than the original, knowing that sharing is > occurring if it's possible. If the item doesn't exist, then I get back > the new tree and exactly the same item I passed in. I did post a solution to this (hashcons:share/1). Basically, (as I understand it) any of the standard associative maps you can insert the same term as key and value and then use the value returned, and that value will be shared with the associative map. (I'm not sure if all of the standard *set* implementations have this property). -- p From raould@REDACTED Thu Sep 17 19:46:23 2009 From: raould@REDACTED (Raoul Duke) Date: Thu, 17 Sep 2009 10:46:23 -0700 Subject: [erlang-questions] Re: No, really, please stop misinterpreting what I wrote In-Reply-To: <4AB23F59.7040700@erlang-consulting.com> References: <4AB23F59.7040700@erlang-consulting.com> Message-ID: <91a2ba3e0909171046w481d2261yf0ae10c051258c09@mail.gmail.com> >> discover is fast. The authors were disturbed, saying that they'd >> prefer that Erlang programmers implement things so as to be *clear* in >> Erlang, so that maintainers of the Erlang interpreter and compiler would >> know what to target for optimization. > contexts, on this list and elsewhere. The problem is not > just that people write 'optimized' code at the expense of clarity. > They often end up optimizing things that don't matter, and miss > things, like algorithm optimization that really does. yup, and it ain't just in Erlang. (as already alluded to re: c.) oh the humanity of all the wasted effort and cpu cycles. :( From joelr1@REDACTED Thu Sep 17 20:40:15 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 17 Sep 2009 19:40:15 +0100 Subject: [erlang-questions] large scale deployments and netsplits In-Reply-To: <95993.1253131334@snookles.snookles.com> References: <95993.1253131334@snookles.snookles.com> Message-ID: <9C4B44DD-0F3E-46A1-8950-4DE10CB0F4D2@gmail.com> On Sep 16, 2009, at 9:02 PM, Scott Lystig Fritchie wrote: > ... are the sizes of the Erlang VM processes big enough to run the > risk > of paging virtual memory pages to disk? If yes, then all bets are off > for timely responses to network distribution communication, including > liveness checks. I took a look at one of the nodes that just split and beam was using 30% of the memory and just 10% of the CPU. I don't think swapping is a problem. It still dumbfounds me that nodes split when my own latency-measuring process reports round-trips of a few milliseconds to just a few seconds. I shudder at the thought of using Mnesia in this scenario! It used to be that I thought of net splits as a remote possibility but at least with EC2 they are a solid reality. --- fastest mac firefox! http://wagerlabs.com From james.hague@REDACTED Thu Sep 17 20:56:49 2009 From: james.hague@REDACTED (James Hague) Date: Thu, 17 Sep 2009 13:56:49 -0500 Subject: [erlang-questions] Re: No, really, please stop misinterpreting what I wrote In-Reply-To: References: <4AB23F59.7040700@erlang-consulting.com> Message-ID: > I did post a solution to this (hashcons:share/1). Yup, and it was nice to see. My code is actually similar to that. James From james.hague@REDACTED Thu Sep 17 22:07:38 2009 From: james.hague@REDACTED (James Hague) Date: Thu, 17 Sep 2009 15:07:38 -0500 Subject: [erlang-questions] data sharing is outside the semantics of Erlang, but it sure is useful In-Reply-To: <6672d0160909170328i1d223958ucccd486abb0afea7@mail.gmail.com> References: <6672d0160909170328i1d223958ucccd486abb0afea7@mail.gmail.com> Message-ID: > In Wings3D (which is an application that depends heavily on sharing), > I use a gb_tree to keep track of values that I want to share. > I enter each term as both the key and the value and then I just do > gb_trees:get/2 to retrieve the shared value. That is an excellent idea! I was trying to hard to use gb_sets here :) From kagato@REDACTED Fri Sep 18 02:26:10 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 17 Sep 2009 17:26:10 -0700 Subject: [erlang-questions] initializing process In-Reply-To: References: Message-ID: <6926854D-B620-4874-9E3E-99589FE95A4C@souja.net> What exactly are you trying to do? It's really easy to create complex race conditions and strange startup ordering issues doing something like this. If you can have the node signal you (much less work). Just start out in an "uninitialized" state set a short timeout and just go idle. When you receive the "other_node_started" event, go into "running" mode. I'd recommend using sync_send_event from the server you're waiting on. That guarantees to close the loop in case they start up in a weird order. Timeouts are handy and highly underutilized. Other great uses include "forcibly garbage collecting during inactivity" and "delaying before hibernating". If you must ping the node from the waiting side, you can simulate a delaying loop with a gen_server (and a gen_fsm) by using the timeout feature. Just send the ping and set a timeout. Have the timeout function send the ping again and set a timeout again. Eventually, you'll get the response and can go about your merry way. Both of these methods prevent you from sitting in init for an undefined amount of time. All of this gets complicated in the face of restarts. It may not really be workable. It would help to know more about what you're doing. I think the "correct" way (i.e. overkill in the style of Ericsson) is to do this at an application level. You can set up a distributed Erlang application to start in "phases". Depending on your deployment, that might be the ultimate answer, since it also provides hooks for takeover / failover, which may be necessary depending on your application. On Sep 17, 2009, at 8:48 AM, pawe? kami?ski wrote: > can somebody corrects me (and hopefully help with my problem). > > so the problem is like that > 1)Im starting a server in erlang and among others I create one > process that > needs to decide if another local computer is up before whole server's > initialization process is done. > 2)the process is simple gen_server > (AND HERE IS A PROBLEM) > 3)as long as I dont return from init/1 fun, my process is not seen > (I cant > really use self() - is that correct??) by others so I CANT RECEIVE ANY > MESSAGES. so I though I can spawn another processes while in init/1 > that > will loop and ping another side if it is ready. but I am afraid that > there > might be a chance that the gen_server will not yet return from init > and > spawned worker will try to send some notification, so or I will miss > it or > there will be some error. > > is this really a problem or Im just missing something here that is so > obvious. > All I really need is to init state of the gen_server, then start > dynamically > initializing the server so I cant hard code all conditions in init > fun and I > need to communicate with other processes. > > hope it is more or less clear > > take care > > pozdrawiam > Pawe? Kami?ski > > kamiseq@REDACTED > pkaminski.prv@REDACTED > ______________________ -- Jayson Vantuyl kagato@REDACTED From kaiduanx@REDACTED Fri Sep 18 03:17:39 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Thu, 17 Sep 2009 21:17:39 -0400 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> Message-ID: Thanks all for the reply. Can someone provide an example in code? For example, a side-effect free function, and its counterpart function with side-effect. Thanks, kaiduan On Thu, Sep 17, 2009 at 11:07 AM, Gene Tani wrote: > > > On Sep 16, 3:26?pm, Kaiduan Xie wrote: >> Hi, >> >> Can someone explain what does side-effect means in Erlang? What is >> side-effect free function, and why we need to write side-effect free >> function? A concrete example is preferred. >> >> Thanks, >> >> kaiduan >> >> ________________________________________________________________ >> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html >> erlang-questions (at) erlang.org > > http://blog.tornkvist.org/blog.yaws?id=1239107937892262 > > http://www.cs.chalmers.se/Cs/Grundutb/Kurser/ppxt/HT2007/general/languages/armstrong-erlang_history.pdf > (PDF page 14) > > and a FP vs. OO debate: > > http://news.ycombinator.com/item?id=493963 > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From kagato@REDACTED Fri Sep 18 03:35:01 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 17 Sep 2009 18:35:01 -0700 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> Message-ID: <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> I'll do better. Here's an example that actually is necessary. Assume you have two processes. One updates a record in Mnesia, the other receives an acknowledgement that it did it. Mnesia can replay a transaction in times of extreme contention. Here's a buggy version of a function that charges some account. Look at the fun() inside. It has the side-effect that it sends a message to another process. > charge_account(Requestor,UserId,Amount) -> > {atomic,ok} = mnesia:transaction( > fun() -> > [ Current ] = mnesia:read(account,UserId), > OldAmount = Current#account.amount, > New = Current#account{ amount = OldAmount - Amount }, > ok = mnesia:write(New), > Requestor ! {updated,New}, > ok > end > ). This function will potentially send the message to Requestor multiple times if the transaction has to be retried. If you want to do this correctly, you can simply modify the function like this: > charge_account(Requestor,UserId,Amount) -> > {atomic,Result} = mnesia:transaction( > fun() -> > [ Current ] = mnesia:read(account,UserId), > OldAmount = Current#account.amount, > New = Current#account{ amount = OldAmount - Amount }, > ok = mnesia:write(New), > New > end, > Requestor ! {updated,Result}, > ). The inner fun() has no side-effects and can now be safely retried by Mnesia. This is only one instance where side-effects matter. The important thing to remember is that state data in Erlang takes the form of recursion, processes and messages. If you don't create recurse with a modified value, create a process, or send a message, then no state is changed. Even when you recurse, your state only affects a single process. This property makes it very easy to create very stable programs, because you can contain the places that state can get corrupted or lost. Hopefully this helps. On Sep 17, 2009, at 6:17 PM, Kaiduan Xie wrote: > Thanks all for the reply. Can someone provide an example in code? For > example, a side-effect free function, and its counterpart function > with side-effect. > > Thanks, > > kaiduan > > On Thu, Sep 17, 2009 at 11:07 AM, Gene Tani > wrote: >> >> >> On Sep 16, 3:26 pm, Kaiduan Xie wrote: >>> Hi, >>> >>> Can someone explain what does side-effect means in Erlang? What is >>> side-effect free function, and why we need to write side-effect free >>> function? A concrete example is preferred. >>> >>> Thanks, >>> >>> kaiduan >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >> >> http://blog.tornkvist.org/blog.yaws?id=1239107937892262 >> >> http://www.cs.chalmers.se/Cs/Grundutb/Kurser/ppxt/HT2007/general/languages/armstrong-erlang_history.pdf >> (PDF page 14) >> >> and a FP vs. OO debate: >> >> http://news.ycombinator.com/item?id=493963 >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From kagato@REDACTED Fri Sep 18 03:47:37 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 17 Sep 2009 18:47:37 -0700 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> Message-ID: A small addendum. Note that calling unsafe functions can create side-effects. Things like gen_server:call, for example, send messages. So you need to be careful with library functions. Thinking a bit, other times that you can modify state: * Changing the process dictionary (most people don't really do this) * ETS tables (messages in disguise) * DETS tables (messages in disguise) * Network Connections (messages in disguise) * digraphs (messages in disguise) On Sep 17, 2009, at 6:35 PM, Jayson Vantuyl wrote: > I'll do better. Here's an example that actually is necessary. > > Assume you have two processes. One updates a record in Mnesia, the > other receives an acknowledgement that it did it. > > Mnesia can replay a transaction in times of extreme contention. > Here's a buggy version of a function that charges some account. > Look at the fun() inside. It has the side-effect that it sends a > message to another process. > >> charge_account(Requestor,UserId,Amount) -> >> {atomic,ok} = mnesia:transaction( >> fun() -> >> [ Current ] = mnesia:read(account,UserId), >> OldAmount = Current#account.amount, >> New = Current#account{ amount = OldAmount - Amount }, >> ok = mnesia:write(New), >> Requestor ! {updated,New}, >> ok >> end >> ). > > This function will potentially send the message to Requestor > multiple times if the transaction has to be retried. > > If you want to do this correctly, you can simply modify the function > like this: > >> charge_account(Requestor,UserId,Amount) -> >> {atomic,Result} = mnesia:transaction( >> fun() -> >> [ Current ] = mnesia:read(account,UserId), >> OldAmount = Current#account.amount, >> New = Current#account{ amount = OldAmount - Amount }, >> ok = mnesia:write(New), >> New >> end, >> Requestor ! {updated,Result}, >> ). > > The inner fun() has no side-effects and can now be safely retried by > Mnesia. > > This is only one instance where side-effects matter. The important > thing to remember is that state data in Erlang takes the form of > recursion, processes and messages. If you don't create recurse with > a modified value, create a process, or send a message, then no state > is changed. Even when you recurse, your state only affects a single > process. This property makes it very easy to create very stable > programs, because you can contain the places that state can get > corrupted or lost. > > Hopefully this helps. > > On Sep 17, 2009, at 6:17 PM, Kaiduan Xie wrote: > >> Thanks all for the reply. Can someone provide an example in code? For >> example, a side-effect free function, and its counterpart function >> with side-effect. >> >> Thanks, >> >> kaiduan >> >> On Thu, Sep 17, 2009 at 11:07 AM, Gene Tani >> wrote: >>> >>> >>> On Sep 16, 3:26 pm, Kaiduan Xie wrote: >>>> Hi, >>>> >>>> Can someone explain what does side-effect means in Erlang? What is >>>> side-effect free function, and why we need to write side-effect >>>> free >>>> function? A concrete example is preferred. >>>> >>>> Thanks, >>>> >>>> kaiduan >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>> >>> http://blog.tornkvist.org/blog.yaws?id=1239107937892262 >>> >>> http://www.cs.chalmers.se/Cs/Grundutb/Kurser/ppxt/HT2007/general/languages/armstrong-erlang_history.pdf >>> (PDF page 14) >>> >>> and a FP vs. OO debate: >>> >>> http://news.ycombinator.com/item?id=493963 >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > > > -- > Jayson Vantuyl > kagato@REDACTED > > > -- Jayson Vantuyl kagato@REDACTED From bflatmaj7th@REDACTED Fri Sep 18 04:04:20 2009 From: bflatmaj7th@REDACTED (Richard Andrews) Date: Fri, 18 Sep 2009 12:04:20 +1000 Subject: [erlang-questions] initializing process In-Reply-To: <6926854D-B620-4874-9E3E-99589FE95A4C@souja.net> References: <6926854D-B620-4874-9E3E-99589FE95A4C@souja.net> Message-ID: <7702c0610909171904r80a3f63o7ad92699e4e38f66@mail.gmail.com> 2009/9/18 Jayson Vantuyl : > What exactly are you trying to do? ?It's really easy to create complex race > conditions and strange startup ordering issues doing something like this. > > If you can have the node signal you (much less work). ?Just start out in an > "uninitialized" state set a short timeout and just go idle. ?When you > receive the "other_node_started" event, go into "running" mode. ?I'd > recommend using sync_send_event from the server you're waiting on. ?That > guarantees to close the loop in case they start up in a weird order. > > Timeouts are handy and highly underutilized. ?Other great uses include > "forcibly garbage collecting during inactivity" and "delaying before > hibernating". > > If you must ping the node from the waiting side, you can simulate a delaying > loop with a gen_server (and a gen_fsm) by using the timeout feature. ?Just > send the ping and set a timeout. ?Have the timeout function send the ping > again and set a timeout again. ?Eventually, you'll get the response and can > go about your merry way. > > Both of these methods prevent you from sitting in init for an undefined > amount of time. Yep. I use a hello message for this kind of thing. Start in an unconnected state and start sending hello to a known target. When responses start getting returned (with a valid ref() of course) then go into fully running mode. If the target process is the last thing to start in the peer system then you know the whole peer is up and running. > All of this gets complicated in the face of restarts. ?It may not really be > workable. ?It would help to know more about what you're doing. > > I think the "correct" way (i.e. overkill in the style of Ericsson) is to do > this at an application level. ?You can set up a distributed Erlang > application to start in "phases". ?Depending on your deployment, that might > be the ultimate answer, since it also provides hooks for takeover / > failover, which may be necessary depending on your application. You can do it at the supervisor level rather than application level. Since children are started in order, you can have one of the children not return from init() until it confirms the peer is up and running. This will block the supervisor from starting any subsequent children until the prerequisites are in place. From kagato@REDACTED Fri Sep 18 04:11:43 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 17 Sep 2009 19:11:43 -0700 Subject: [erlang-questions] initializing process In-Reply-To: <7702c0610909171904r80a3f63o7ad92699e4e38f66@mail.gmail.com> References: <6926854D-B620-4874-9E3E-99589FE95A4C@souja.net> <7702c0610909171904r80a3f63o7ad92699e4e38f66@mail.gmail.com> Message-ID: <76D25611-0F7E-4AFB-A749-F9C0469A5691@souja.net> I think that synchronizing the supervisor this way is a bad thing, as it doesn't allow you to specify a limited "start time". In other words, I'd consider an uninitialized server as running, otherwise the supervisory machinery hangs. I'm also not sure that this behavior is guaranteed in the future. Also, since this is a cross-node thing, it's going to make "restart" a painful process that requires manually coordinating multiple machines. Even when we've been scripting these things on EC2, it's just not feasible to serialize all of this for large numbers of hosts (especially with EC2's, understandably long, startup delay). I really think this is better done with a series of gen_fsms that can bounce into and out of a "waiting" state, rather than requiring all manner of coordination when badness occurs. In other worse, work with Erlang's supervision framework and synchronization primitives, not against them. On Sep 17, 2009, at 7:04 PM, Richard Andrews wrote: > 2009/9/18 Jayson Vantuyl : >> What exactly are you trying to do? It's really easy to create >> complex race >> conditions and strange startup ordering issues doing something like >> this. >> >> If you can have the node signal you (much less work). Just start >> out in an >> "uninitialized" state set a short timeout and just go idle. When you >> receive the "other_node_started" event, go into "running" mode. I'd >> recommend using sync_send_event from the server you're waiting on. >> That >> guarantees to close the loop in case they start up in a weird order. >> >> Timeouts are handy and highly underutilized. Other great uses >> include >> "forcibly garbage collecting during inactivity" and "delaying before >> hibernating". >> >> If you must ping the node from the waiting side, you can simulate a >> delaying >> loop with a gen_server (and a gen_fsm) by using the timeout >> feature. Just >> send the ping and set a timeout. Have the timeout function send >> the ping >> again and set a timeout again. Eventually, you'll get the response >> and can >> go about your merry way. >> >> Both of these methods prevent you from sitting in init for an >> undefined >> amount of time. > > Yep. I use a hello message for this kind of thing. Start in an > unconnected state and start sending hello to a known target. When > responses start getting returned (with a valid ref() of course) then > go into fully running mode. If the target process is the last thing to > start in the peer system then you know the whole peer is up and > running. > >> All of this gets complicated in the face of restarts. It may not >> really be >> workable. It would help to know more about what you're doing. >> >> I think the "correct" way (i.e. overkill in the style of Ericsson) >> is to do >> this at an application level. You can set up a distributed Erlang >> application to start in "phases". Depending on your deployment, >> that might >> be the ultimate answer, since it also provides hooks for takeover / >> failover, which may be necessary depending on your application. > > You can do it at the supervisor level rather than application level. > Since children are started in order, you can have one of the children > not return from init() until it confirms the peer is up and running. > This will block the supervisor from starting any subsequent children > until the prerequisites are in place. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From kaiduanx@REDACTED Fri Sep 18 04:20:41 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Thu, 17 Sep 2009 22:20:41 -0400 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> Message-ID: Thanks a lot for the great examples. But I can not understand the following one, add(X,Y) -> X+Y. add_effects(X,Y) -> io:format("~p~n",[X+Y]). What side-effect add_effects generates? Why we say io:format() generates side-effect? kaiduan On Thu, Sep 17, 2009 at 9:47 PM, Jayson Vantuyl wrote: > A small addendum. > > Note that calling unsafe functions can create side-effects. ?Things like > gen_server:call, for example, send messages. ?So you need to be careful with > library functions. > > Thinking a bit, other times that you can modify state: > > * Changing the process dictionary (most people don't really do this) > * ETS tables (messages in disguise) > * DETS tables (messages in disguise) > * Network Connections (messages in disguise) > * digraphs (messages in disguise) > > On Sep 17, 2009, at 6:35 PM, Jayson Vantuyl wrote: > >> I'll do better. ?Here's an example that actually is necessary. >> >> Assume you have two processes. ?One updates a record in Mnesia, the other >> receives an acknowledgement that it did it. >> >> Mnesia can replay a transaction in times of extreme contention. ?Here's a >> buggy version of a function that charges some account. ?Look at the fun() >> inside. ?It has the side-effect that it sends a message to another process. >> >>> charge_account(Requestor,UserId,Amount) -> >>> ?{atomic,ok} = mnesia:transaction( >>> ? fun() -> >>> ? ? [ Current ] = mnesia:read(account,UserId), >>> ? ? OldAmount = Current#account.amount, >>> ? ? New = Current#account{ amount = OldAmount - Amount }, >>> ? ? ok = mnesia:write(New), >>> ? ? Requestor ! {updated,New}, >>> ? ? ok >>> ? end >>> ?). >> >> This function will potentially send the message to Requestor multiple >> times if the transaction has to be retried. >> >> If you want to do this correctly, you can simply modify the function like >> this: >> >>> charge_account(Requestor,UserId,Amount) -> >>> ?{atomic,Result} = mnesia:transaction( >>> ? fun() -> >>> ? ? [ Current ] = mnesia:read(account,UserId), >>> ? ? OldAmount = Current#account.amount, >>> ? ? New = Current#account{ amount = OldAmount - Amount }, >>> ? ? ok = mnesia:write(New), >>> ? ? New >>> ? end, >>> ? Requestor ! {updated,Result}, >>> ?). >> >> The inner fun() has no side-effects and can now be safely retried by >> Mnesia. >> >> This is only one instance where side-effects matter. ?The important thing >> to remember is that state data in Erlang takes the form of recursion, >> processes and messages. ?If you don't create recurse with a modified value, >> create a process, or send a message, then no state is changed. ?Even when >> you recurse, your state only affects a single process. ?This property makes >> it very easy to create very stable programs, because you can contain the >> places that state can get corrupted or lost. >> >> Hopefully this helps. >> >> On Sep 17, 2009, at 6:17 PM, Kaiduan Xie wrote: >> >>> Thanks all for the reply. Can someone provide an example in code? For >>> example, a side-effect free function, and its counterpart function >>> with side-effect. >>> >>> Thanks, >>> >>> kaiduan >>> >>> On Thu, Sep 17, 2009 at 11:07 AM, Gene Tani wrote: >>>> >>>> >>>> On Sep 16, 3:26 pm, Kaiduan Xie wrote: >>>>> >>>>> Hi, >>>>> >>>>> Can someone explain what does side-effect means in Erlang? What is >>>>> side-effect free function, and why we need to write side-effect free >>>>> function? A concrete example is preferred. >>>>> >>>>> Thanks, >>>>> >>>>> kaiduan >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html >>>>> erlang-questions (at) erlang.org >>>> >>>> http://blog.tornkvist.org/blog.yaws?id=1239107937892262 >>>> >>>> >>>> http://www.cs.chalmers.se/Cs/Grundutb/Kurser/ppxt/HT2007/general/languages/armstrong-erlang_history.pdf >>>> (PDF page 14) >>>> >>>> and a FP vs. OO debate: >>>> >>>> http://news.ycombinator.com/item?id=493963 >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>>> >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> >> >> >> -- >> Jayson Vantuyl >> kagato@REDACTED >> >> >> > > > > -- > Jayson Vantuyl > kagato@REDACTED > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From kaiduanx@REDACTED Fri Sep 18 04:34:35 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Thu, 17 Sep 2009 22:34:35 -0400 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: <4AB2EDF5.2030900@bigpond.net.au> References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> <4AB2EDF5.2030900@bigpond.net.au> Message-ID: Thanks Benjamin, but sometimes printing out to the console is really what you want. Maybe this is not a good example :) kaiduan On Thu, Sep 17, 2009 at 10:18 PM, Benjamin Tolputt wrote: > Kaiduan Xie wrote: >> Thanks a lot for the great examples. But I can not understand the following one, >> >> add(X,Y) -> >> ? ? X+Y. >> >> add_effects(X,Y) -> >> ? ? io:format("~p~n",[X+Y]). >> >> What side-effect add_effects generates? Why we say io:format() >> generates side-effect? >> > > The side-effect is that there is something printed to the console (or > stdout). If there were another function (say 'print_screen') that could > look at the screen, it is possible/likely a call to the add_effects > function would change the output of the function. > > -- > Regards, > > Benjamin Tolputt > Analyst Programmer > > From mjtruog@REDACTED Fri Sep 18 05:15:40 2009 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 17 Sep 2009 20:15:40 -0700 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> <4AB2EDF5.2030900@bigpond.net.au> Message-ID: <4AB2FB5C.6030204@gmail.com> It is the idea that it is modifying state outside the function. So printing to the console is a side-effect, it is externalized state. Kaiduan Xie wrote: > Thanks Benjamin, but sometimes printing out to the console is really > what you want. Maybe this is not a good example :) > > kaiduan > > On Thu, Sep 17, 2009 at 10:18 PM, Benjamin Tolputt > wrote: > >> Kaiduan Xie wrote: >> >>> Thanks a lot for the great examples. But I can not understand the following one, >>> >>> add(X,Y) -> >>> X+Y. >>> >>> add_effects(X,Y) -> >>> io:format("~p~n",[X+Y]). >>> >>> What side-effect add_effects generates? Why we say io:format() >>> generates side-effect? >>> >>> >> The side-effect is that there is something printed to the console (or >> stdout). If there were another function (say 'print_screen') that could >> look at the screen, it is possible/likely a call to the add_effects >> function would change the output of the function. >> >> -- >> Regards, >> >> Benjamin Tolputt >> Analyst Programmer >> >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From jeffm@REDACTED Fri Sep 18 05:00:29 2009 From: jeffm@REDACTED (jm) Date: Fri, 18 Sep 2009 13:00:29 +1000 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> <4AB2EDF5.2030900@bigpond.net.au> Message-ID: <4AB2F7CD.9050606@ghostgun.com> Kaiduan Xie wrote: > Thanks Benjamin, but sometimes printing out to the console is really > what you want. Maybe this is not a good example :) > Even when you do want the program to print to the screen it is a side-effect. Lets try breaking it down: >>> add(X,Y) -> >>> X+Y. >>> call add/2 with X and Y calculate X+Y return result result now available in calling function >>> add_effects(X,Y) -> >>> io:format("~p~n",[X+Y]). >>> >>> call add/2 with X and Y calculate X+Y pass message to io to print to screen (** side effect **) return result result now available in calling function >>> What side-effect add_effects generates? Why we say io:format() >>> generates side-effect? >>> >>> >>> note how in the second example the calling function doesn't know anything about the effects of the io:format/2 function call. It is an effect outside the, or aside from, main excution path and hence a side-effect. Jeff. From leap@REDACTED Fri Sep 18 05:49:41 2009 From: leap@REDACTED (Michael Turner) Date: Fri, 18 Sep 2009 03:49:41 +0000 Subject: [erlang-questions] Re: No, really, please stop misinterpreting what I wrote In-Reply-To: <4AB23F59.7040700@erlang-consulting.com> Message-ID: <6CJB5BrL.1253245781.4787000.leap@gol.com> On 9/17/2009, "Ulf Wiger" wrote: >Some posts refered to the "built-in compression" available >in Erlang in the form of term_to_binary(Term, [compressed]). > >I think one could say that compressing using term_to_binary/2 >is at the far end of the spectrum in this matter. If you >compress to a binary, you lose every possiblity of pattern >matching (except check for equality), ... That's funny. I just unzipped a file, looked at it in an editor, and did a search. What you apparently mean is "you lose every possibility of pattern matching DIRECTLY on the compressed representation". But if you selectively decompress parts of the long list, with some global dictionary for the more repeated parts, clearly one can do pattern matching on it. Will performance necessarily be poorer? It depends on the data and its access patterns. If you can't characterize those, any optimization attempt will be a shot in the dark anyway. >This [case of QuickCheck] case in itself answers the 'so what'. >If it had been feasible to do it in Erlang, the authors >of QuickCheck surely would have done it by now. Also, >I would not dream to suggest that they should simply >choose another data structure in order to solve the >problem. Their knowledge in that area far exceeds mine. Well, I guess I wouldn't dream of asking them to choose another data structure either. But maybe I would dream of asking them to choose a different *process* structure. If passing a rules-base around causes explosions, how about ... well, NOT passing a rules-base around? How about moderating access to the rules-base via processes and messages? After all, even if that incurs a performance penalty, it's still a testing framework, and very likely the code under test will be soaking up most of the cycles. Maybe they didn't do this because that's not how they originally designed the whole thing, and they just didn't want to bother redesigning it to be more loosely coupled in that respect? >I've also had discussions with experienced Haskell and >OCaml programmers who felt that the loss of sharing when >sending data from one node to another was sufficient >reason in itself not to use Erlang - since the use of >sharing in functional programming is such a powerful tool. >Obviously, I don't share the view that this disqualifies >Erlang entirely, but at least I'm not alone in thinking >that (a) sharing is a good thing, and (b) the occasional >loss of sharing, partly beyond the programmer's control, >is a bad thing. Again, I'm new to this, but my understanding of Erlang is that it's not a functional language per se -- it's a concurrency-oriented language in which processes are specified in functional terms. If you're not used to that emphasis, you won't consider concurrency at every turn, and you might have accumulated design and coding habits that don't match the semantic emphasis of Erlang, even if you come from a functional programming background. In any case, Erlang's problem as a small minority language isn't to please the users of other small minority languages so much that it gains followers from those camps. It's how to make itself understood -- and increasingly used for its strengths -- in a much larger software community, while also becoming more palatable in the process, where it can do it without compromise of its principle. If I had to choose between better syntax and semantics for records and better support for structure sharing, I'd *definitely* ask for better support for records, at this point. >> I assume we'd all prefer everything to be fast, all the time. That >> seems to be a preference at Ericsson. In the telephony switch at >> Ericsson that has the most Erlang code in it (AFAIK), there's a lot of >> Erlang code. But as I understand it, also a lot of C code in that same >> switch. They don't use Erlang in that system for speed. They use it >> for robustness and expressive power for the concurrency-oriented aspect >> of the system. > >OT, but this is not a correct description. > >I assume you refer to the AXD 301? Most of the C code in the AXD 301 >is low-level device processor code, written in C mainly for historical >reasons. In the first versions, those device processors did not have >the CPU or memory capacity to run Erlang, and instead ran VRTX - later >OSE Delta. [snip] What it seems to come down to is: Erlang would have overloaded much of the original hardware, if it would fit at all, so C/real-time-OS was the better solution at one point. Later, these components came to closely match my description of "black boxes you can't get into and fix", for a variety of reasons including the prohibitive cost of doing that. Except for cases like these ... >There were other blocks of C code, in the form of 3rd party >applications that were integrated rather than writing everything >from scratch..... which were designed in for other reasons that make them virtual black boxes that you can't get inside of and fix. Believe me, I spent some time in the trenches in telecom, I know first-hand how the spaghetti pile grows. All of which seems to be drifting away from a point: sure, it's nice to have things arranged so that you can express things naturally in a language and it will be guaranteed to be fast, or save memory, or have some other nice feature. But there's often a hidden or delayed cost. Remote procedure calls, for example, seemed like the natural "idiomatic" way to do interprocess communication. *Seemed*. For a long time. And to some very smart people, too. -michael turner From btolputt@REDACTED Fri Sep 18 04:18:29 2009 From: btolputt@REDACTED (Benjamin Tolputt) Date: Fri, 18 Sep 2009 12:18:29 +1000 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> Message-ID: <4AB2EDF5.2030900@bigpond.net.au> Kaiduan Xie wrote: > Thanks a lot for the great examples. But I can not understand the following one, > > add(X,Y) -> > X+Y. > > add_effects(X,Y) -> > io:format("~p~n",[X+Y]). > > What side-effect add_effects generates? Why we say io:format() > generates side-effect? > The side-effect is that there is something printed to the console (or stdout). If there were another function (say 'print_screen') that could look at the screen, it is possible/likely a call to the add_effects function would change the output of the function. -- Regards, Benjamin Tolputt Analyst Programmer From ulf.wiger@REDACTED Fri Sep 18 09:34:56 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 18 Sep 2009 09:34:56 +0200 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: <4AB2F7CD.9050606@ghostgun.com> References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> <4AB2EDF5.2030900@bigpond.net.au> <4AB2F7CD.9050606@ghostgun.com> Message-ID: <4AB33820.4090905@erlang-consulting.com> jm wrote: > > call add/2 with X and Y > calculate X+Y > pass message to io to print to screen (** side effect **) > return result > result now available in calling function Actually, the return result is 'ok', which basically only tells the caller that the function didn't crash. The main difference then becomes: - add(X, Y) returns "This is the result of X+Y" - add_effects(X,Y) returns "ok, I've done something" In the case of add_effects/2, you have to go elsewhere to find out exactly /what/ was done. This is a fairly common pattern when you look at code written by Erlang newbies. Some of my early sins where cast in stone in the AXD 301 code base and its successors. One was the initialization framework for creating mnesia tables, which I designed 11 years ago. I came up with a callback module, DataInit.erl, which was found automatically by the install program. It exported functions to 1. create tables 2. add data 3. optionally do something afterwards These phases were called across all applications, so first, all tables were created, then they were populated, and then further work could be done on the data. This worked very well, except for one thing: the API semantics were such that functions either returned 'ok' (in which case a smiley was output for that function) or {error, Reason} (a sad face printed, together with the error). While this made it very easy to see on the screen that everything went well, what the functions did - if they did anything at all - was completely opaque. When we wanted to describe the full data model, it became very clear that this wasn't a very good idea. The problem was that the API reflected an imperative mindset: "Go and do something" -> "ok, I've done something" It is a bit more difficult to craft an API that is more functional in nature: "What is your data model" -> "This is my data model" but in the long run, it is much more powerful. It would not only have allowed for easy extraction of the data model (perhaps even as part of static analysis), but also for coping with new situations. Much later, we discussed the option of upgrading the system by taking some nodes off-line, converting them to the newer version, bringing them on-line, synching, and then basically bootstrapping the other half to the new version. Converting the database off-line could have been automated as a well-isolated project if the data initialization modules had been more declarative in nature, but with the imperative model, a new callback - "Go and upgrade your data off-line" -> "ok, I've done that" (at least one such callback, perhaps more) would be needed in all applications, which made it a project that affected every single design team. There are other examples that are similar in nature. I've complained now and then about the way the supervision hierarchy is built in OTP. - First, you name an application callback module in the .app file. - The application controller calls Mod:start(Type, Args) in order to start the application. It is supposed to return {ok, Pid} ("ok, something happened. Here's your handle.") - What the start function normally does is call supervisor:start_link(...) with another callback module. This function returns {ok, Pid} (again, imperative) - The init function in the supervisor callback module is /usually/ declarative by nature. It is supposed to return a specification, which leads to the starting of workers. This part, we actually /did/ get right in the AXD 301, IMHO. I felt that it was tedious to chase this down every time, and guessed that >90% of all applications could get away with just specifying the supervisor declaration as an environment variable in the .app file, and call a generic start function. This worked like a charm, and made it much easier to see which processes were actually being started, what supervision strategies where being used, and likely removed a lot of unnecessarily imperative code (since it was easier to just declare the structure statically). BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From masklinn@REDACTED Fri Sep 18 10:18:55 2009 From: masklinn@REDACTED (Masklinn) Date: Fri, 18 Sep 2009 10:18:55 +0200 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> Message-ID: On 18 Sep 2009, at 03:35 , Jayson Vantuyl wrote: >> charge_account(Requestor,UserId,Amount) -> >> {atomic,Result} = mnesia:transaction( >> fun() -> >> [ Current ] = mnesia:read(account,UserId), >> OldAmount = Current#account.amount, >> New = Current#account{ amount = OldAmount - Amount }, >> ok = mnesia:write(New), >> New >> end, >> Requestor ! {updated,Result}, >> ). > > The inner fun() has no side-effects Not so. The inner fun() executes within a transactional context (that of mnesia) and only uses functions relative to that context, which means the context manager can handle errors & retries gracefully. However, mnesia:read and mnesia:write are still side-effecting functions, not pure ones: they read and alter external state which means the function isn't necessarily repeatable (if you call the fun() twice in a row, the record might have been deleted in-between, yielding different outcomes). A pure version of it would be the following: fun(Db) -> [ Current ] = mnesia:read(Db, account, UserId), OldAmmount = Current#account.amount, New = Current#account{amount = OldAmount - Amount }, NewDb = mnesia:write(Db, New), NewDb end here, a database is provided as argument to the function and instead of mutating that database we create a new one. This means if we repeatedly call this function and always provide it the same Db input, it'll always return the same NewDb. It doesn't rely on any external state, managed or not. It's pure. From masklinn@REDACTED Fri Sep 18 10:22:30 2009 From: masklinn@REDACTED (Masklinn) Date: Fri, 18 Sep 2009 10:22:30 +0200 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> <4AB2EDF5.2030900@bigpond.net.au> Message-ID: <29220F29-06C2-46C4-959A-4B381F82F052@masklinn.net> On 18 Sep 2009, at 04:34 , Kaiduan Xie wrote: > Thanks Benjamin, but sometimes printing out to the console is really > what you want. Maybe this is not a good example :) > The qualification of a side-effect as side-effect doesn't depend on "what you want", sometimes you want side-effects (indeed, programming would be quite uninteresting if we were only allowed pure constructs). A side-effect is simply a contact with a state, either reading from or writing to a state holder. In the case of your io:format, you're writing to the console and altering its state (by adding a new line with your printed string each time). From btolputt@REDACTED Fri Sep 18 04:48:52 2009 From: btolputt@REDACTED (Benjamin Tolputt) Date: Fri, 18 Sep 2009 12:48:52 +1000 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> <4AB2EDF5.2030900@bigpond.net.au> Message-ID: <4AB2F514.9050406@bigpond.net.au> Kaiduan Xie wrote: > Benjamin Tolputt wrote: > >> Kaiduan Xie wrote: >> >>> Thanks a lot for the great examples. But I can not understand the following one, >>> >>> add(X,Y) -> >>> X+Y. >>> >>> add_effects(X,Y) -> >>> io:format("~p~n",[X+Y]). >>> >>> What side-effect add_effects generates? Why we say io:format() >>> generates side-effect? >>> >> The side-effect is that there is something printed to the console (or >> stdout). If there were another function (say 'print_screen') that could >> look at the screen, it is possible/likely a call to the add_effects >> function would change the output of the function. >> > Thanks Benjamin, but sometimes printing out to the console is really > what you want. Maybe this is not a good example :) > Oh there is no debating that you may want that behaviour just, that in having the side-effect of printing to the screen, you then need to be aware of other functions that use that state (i.e. the example 'print_screen'). Side-effects are not necessarily bad things - they just add to the things you need to keep a track of. The less side-effects there are, the more "local" you can limit your thoughts when coding new functionality. Whilst I use Erlang for the things I can, I do a majority of my coding in C/C++. A majority of the times I have found bugs, it has been state-changing functionality (i.e. side-effects) in code I was unaware of. Erlang makes it harder to create these side-effects by limiting the ways in which it can be done :) P.S. Hope you don't mind my re-arrangement of your email - the "reply at top" made it harder to follow the flow of the email. -- Regards, Benjamin Tolputt Analyst Programmer From kagato@REDACTED Fri Sep 18 10:38:14 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 18 Sep 2009 01:38:14 -0700 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> Message-ID: <27AF33C5-8A7A-46E0-AC23-85841969201E@souja.net> > Not so. The inner fun() executes within a transactional context > (that of mnesia) and only uses functions relative to that context, > which means the context manager can handle errors & retries > gracefully. Okay, the inner fun() has intended side-effects. The point was, if the inner function had any OTHER side effects, it couldn't be retried. So, -1 for actually having intended but subtle side effects, but +1000 for showing a real case of when and why they're important--which was the poster's question. -- Jayson Vantuyl kagato@REDACTED From ulf.wiger@REDACTED Fri Sep 18 11:48:39 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 18 Sep 2009 11:48:39 +0200 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> Message-ID: <4AB35777.7050805@erlang-consulting.com> Masklinn wrote: > > Not so. The inner fun() executes within a transactional context (that of > mnesia) and only uses functions relative to that context, which means > the context manager can handle errors & retries gracefully. Associating freely and spinning the whole thing into a much more abstract sphere, mnesia transactions are in some ways similar to Transaction Monads in Haskell, except in Haskell, the compiler can actually enforce these things for you. Haskell is special in that it identifies side effects as a very special operation, with its own type. It handles side effects using a special construct, called the monad. This scares the life out of most people, but allows Haskell programmers to apply a whole set of sound laws on expressions that contain side effects, and determine, e.g. if different expressions containing side effects are equivalent. This presentation from the ICFP Haskell workshop in Edinburgh, given by "Oscar winner" Dan Piponi, is a wonderful attempt at describing these monad laws in terms that mere mortals can understand. http://www.vimeo.com/6590617 Simon Peyton-Jones once said that they should have called monads "warm fuzzy things" instead. Perhaps not, but at least, many of the important aspects of monads can be illustrated using warm fuzzy things. Tying back to the initial question (and apologies for raising it into the stratosphere), you could say that if you _do_ clearly separate pure logic from side effects, and you do it right, infinite power lies within your reach. :) BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ovidiudeac@REDACTED Fri Sep 18 13:52:01 2009 From: ovidiudeac@REDACTED (Ovidiu Deac) Date: Fri, 18 Sep 2009 14:52:01 +0300 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> Message-ID: <804cd4020909180452p7312e36dj385bf68fe8240a10@mail.gmail.com> Exceptions too? On Fri, Sep 18, 2009 at 4:47 AM, Jayson Vantuyl wrote: > A small addendum. > > Note that calling unsafe functions can create side-effects. ?Things like > gen_server:call, for example, send messages. ?So you need to be careful with > library functions. > > Thinking a bit, other times that you can modify state: > > * Changing the process dictionary (most people don't really do this) > * ETS tables (messages in disguise) > * DETS tables (messages in disguise) > * Network Connections (messages in disguise) > * digraphs (messages in disguise) > > On Sep 17, 2009, at 6:35 PM, Jayson Vantuyl wrote: > >> I'll do better. ?Here's an example that actually is necessary. >> >> Assume you have two processes. ?One updates a record in Mnesia, the other >> receives an acknowledgement that it did it. >> >> Mnesia can replay a transaction in times of extreme contention. ?Here's a >> buggy version of a function that charges some account. ?Look at the fun() >> inside. ?It has the side-effect that it sends a message to another process. >> >>> charge_account(Requestor,UserId,Amount) -> >>> ?{atomic,ok} = mnesia:transaction( >>> ? fun() -> >>> ? ? [ Current ] = mnesia:read(account,UserId), >>> ? ? OldAmount = Current#account.amount, >>> ? ? New = Current#account{ amount = OldAmount - Amount }, >>> ? ? ok = mnesia:write(New), >>> ? ? Requestor ! {updated,New}, >>> ? ? ok >>> ? end >>> ?). >> >> This function will potentially send the message to Requestor multiple >> times if the transaction has to be retried. >> >> If you want to do this correctly, you can simply modify the function like >> this: >> >>> charge_account(Requestor,UserId,Amount) -> >>> ?{atomic,Result} = mnesia:transaction( >>> ? fun() -> >>> ? ? [ Current ] = mnesia:read(account,UserId), >>> ? ? OldAmount = Current#account.amount, >>> ? ? New = Current#account{ amount = OldAmount - Amount }, >>> ? ? ok = mnesia:write(New), >>> ? ? New >>> ? end, >>> ? Requestor ! {updated,Result}, >>> ?). >> >> The inner fun() has no side-effects and can now be safely retried by >> Mnesia. >> >> This is only one instance where side-effects matter. ?The important thing >> to remember is that state data in Erlang takes the form of recursion, >> processes and messages. ?If you don't create recurse with a modified value, >> create a process, or send a message, then no state is changed. ?Even when >> you recurse, your state only affects a single process. ?This property makes >> it very easy to create very stable programs, because you can contain the >> places that state can get corrupted or lost. >> >> Hopefully this helps. >> >> On Sep 17, 2009, at 6:17 PM, Kaiduan Xie wrote: >> >>> Thanks all for the reply. Can someone provide an example in code? For >>> example, a side-effect free function, and its counterpart function >>> with side-effect. >>> >>> Thanks, >>> >>> kaiduan >>> >>> On Thu, Sep 17, 2009 at 11:07 AM, Gene Tani wrote: >>>> >>>> >>>> On Sep 16, 3:26 pm, Kaiduan Xie wrote: >>>>> >>>>> Hi, >>>>> >>>>> Can someone explain what does side-effect means in Erlang? What is >>>>> side-effect free function, and why we need to write side-effect free >>>>> function? A concrete example is preferred. >>>>> >>>>> Thanks, >>>>> >>>>> kaiduan >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html >>>>> erlang-questions (at) erlang.org >>>> >>>> http://blog.tornkvist.org/blog.yaws?id=1239107937892262 >>>> >>>> >>>> http://www.cs.chalmers.se/Cs/Grundutb/Kurser/ppxt/HT2007/general/languages/armstrong-erlang_history.pdf >>>> (PDF page 14) >>>> >>>> and a FP vs. OO debate: >>>> >>>> http://news.ycombinator.com/item?id=493963 >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>>> >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> >> >> >> -- >> Jayson Vantuyl >> kagato@REDACTED >> >> >> > > > > -- > Jayson Vantuyl > kagato@REDACTED > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From kagato@REDACTED Fri Sep 18 14:40:53 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 18 Sep 2009 05:40:53 -0700 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: <804cd4020909180452p7312e36dj385bf68fe8240a10@mail.gmail.com> References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> <804cd4020909180452p7312e36dj385bf68fe8240a10@mail.gmail.com> Message-ID: <452A3500-3B3A-44C6-B165-77446D37FD8A@souja.net> Actually, I'm not entirely sure that an exception would be a side effect. It doesn't get "returned" like a normal value, but I don't see that it changes state any more than a return value does. Put another way, a function has "no side-effects" if all the data goes in through the parameters and a result comes out the end--without anything else being touched. This means that we can safely optimize that call away, or run with a thousand different inputs in parallel (on a bunch of processors), or do any number of things without worry about two parallel executions of it stepping on each others toes. So, taking another stab at Kaiduan's question, maybe it's best illustrated by contrast. Having "no side-effects" is most important when you have concurrent code. In Erlang, no two processes share state that can be changed. If you change state in one process, it never changes in another one. In this way, a process has no "side effects" unless you choose to make some. Contrast this with the normal way of using shared memory with threads. In these languages, you have multiple threads trying to lock access to shared resources (or using complex lockless methods). The complications of avoiding deadlocks or generating correct behavior in threaded systems are very difficult compared to Erlang's isolation. Instead of having a bunch of processes fighting over shared resources, we let resources be owned by individual processes only, but give a powerful way to communicate among them. This isolation is embodied in the concept that no simple Erlang function can touch another process's state, thus there are no "side-effects" that the other process has to worry about. All side effects are contained in a few operations that are easily handled. Understanding the impact of delivering messages to mailboxes and spawning processes (which is 99% of side-effects in Erlang code) is much easier to debug and much easier to understand than finding race conditions with complex locking regimes, implementing even the simplest of lockless algorithms portably, or using system IPC to manage concurrency. With the possible exception of Software Transactional Memory, there doesn't seem to be much of anything that comes closer to simplifying concurrent programming. On Sep 18, 2009, at 4:52 AM, Ovidiu Deac wrote: > Exceptions too? > > On Fri, Sep 18, 2009 at 4:47 AM, Jayson Vantuyl > wrote: >> A small addendum. >> >> Note that calling unsafe functions can create side-effects. Things >> like >> gen_server:call, for example, send messages. So you need to be >> careful with >> library functions. >> >> Thinking a bit, other times that you can modify state: >> >> * Changing the process dictionary (most people don't really do this) >> * ETS tables (messages in disguise) >> * DETS tables (messages in disguise) >> * Network Connections (messages in disguise) >> * digraphs (messages in disguise) >> >> On Sep 17, 2009, at 6:35 PM, Jayson Vantuyl wrote: >> >>> I'll do better. Here's an example that actually is necessary. >>> >>> Assume you have two processes. One updates a record in Mnesia, >>> the other >>> receives an acknowledgement that it did it. >>> >>> Mnesia can replay a transaction in times of extreme contention. >>> Here's a >>> buggy version of a function that charges some account. Look at >>> the fun() >>> inside. It has the side-effect that it sends a message to another >>> process. >>> >>>> charge_account(Requestor,UserId,Amount) -> >>>> {atomic,ok} = mnesia:transaction( >>>> fun() -> >>>> [ Current ] = mnesia:read(account,UserId), >>>> OldAmount = Current#account.amount, >>>> New = Current#account{ amount = OldAmount - Amount }, >>>> ok = mnesia:write(New), >>>> Requestor ! {updated,New}, >>>> ok >>>> end >>>> ). >>> >>> This function will potentially send the message to Requestor >>> multiple >>> times if the transaction has to be retried. >>> >>> If you want to do this correctly, you can simply modify the >>> function like >>> this: >>> >>>> charge_account(Requestor,UserId,Amount) -> >>>> {atomic,Result} = mnesia:transaction( >>>> fun() -> >>>> [ Current ] = mnesia:read(account,UserId), >>>> OldAmount = Current#account.amount, >>>> New = Current#account{ amount = OldAmount - Amount }, >>>> ok = mnesia:write(New), >>>> New >>>> end, >>>> Requestor ! {updated,Result}, >>>> ). >>> >>> The inner fun() has no side-effects and can now be safely retried by >>> Mnesia. >>> >>> This is only one instance where side-effects matter. The >>> important thing >>> to remember is that state data in Erlang takes the form of >>> recursion, >>> processes and messages. If you don't create recurse with a >>> modified value, >>> create a process, or send a message, then no state is changed. >>> Even when >>> you recurse, your state only affects a single process. This >>> property makes >>> it very easy to create very stable programs, because you can >>> contain the >>> places that state can get corrupted or lost. >>> >>> Hopefully this helps. >>> >>> On Sep 17, 2009, at 6:17 PM, Kaiduan Xie wrote: >>> >>>> Thanks all for the reply. Can someone provide an example in code? >>>> For >>>> example, a side-effect free function, and its counterpart function >>>> with side-effect. >>>> >>>> Thanks, >>>> >>>> kaiduan >>>> >>>> On Thu, Sep 17, 2009 at 11:07 AM, Gene Tani >>>> wrote: >>>>> >>>>> >>>>> On Sep 16, 3:26 pm, Kaiduan Xie wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> Can someone explain what does side-effect means in Erlang? What >>>>>> is >>>>>> side-effect free function, and why we need to write side-effect >>>>>> free >>>>>> function? A concrete example is preferred. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> kaiduan >>>>>> >>>>>> ________________________________________________________________ >>>>>> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html >>>>>> erlang-questions (at) erlang.org >>>>> >>>>> http://blog.tornkvist.org/blog.yaws?id=1239107937892262 >>>>> >>>>> >>>>> http://www.cs.chalmers.se/Cs/Grundutb/Kurser/ppxt/HT2007/general/languages/armstrong-erlang_history.pdf >>>>> (PDF page 14) >>>>> >>>>> and a FP vs. OO debate: >>>>> >>>>> http://news.ycombinator.com/item?id=493963 >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>>> erlang-questions (at) erlang.org >>>>> >>>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>> >>> >>> >>> -- >>> Jayson Vantuyl >>> kagato@REDACTED >>> >>> >>> >> >> >> >> -- >> Jayson Vantuyl >> kagato@REDACTED >> >> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From ulf.wiger@REDACTED Fri Sep 18 15:03:12 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 18 Sep 2009 15:03:12 +0200 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: <452A3500-3B3A-44C6-B165-77446D37FD8A@souja.net> References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> <804cd4020909180452p7312e36dj385bf68fe8240a10@mail.gmail.com> <452A3500-3B3A-44C6-B165-77446D37FD8A@souja.net> Message-ID: <4AB38510.4050009@erlang-consulting.com> Jayson Vantuyl wrote: > Actually, I'm not entirely sure that an exception would be a side > effect. It doesn't get "returned" like a normal value, but I don't see > that it changes state any more than a return value does. It straddles the fence in some ways. Functions can be pure for all legal input and raise exceptions for illegal input. This is fine, but there are also functions that raise exceptions even for legal input. Consider the following code: pmap(F, L) -> Pids = [spawn_monitor(fun() -> exit({ok, F(X)}) end) || X <- L], [receive {'DOWN',Ref,_,_,R} -> unwrap(R) end || {_,Ref} <- Pids]. (A simplified version of slide 7 in this talk: http://www.erlang-factory.com/upload/presentations/56/UlfWiger_ErlangMulticoreEF.pdf) While it might seem contrived, the shell evaluator does roughly the same thing: evaluates the expression inside an exit() call in order to deliver the result and die in one and the same action. Such functions are only meaningful when considering their side-effects - the exceptions. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From masklinn@REDACTED Fri Sep 18 15:01:57 2009 From: masklinn@REDACTED (Masklinn) Date: Fri, 18 Sep 2009 15:01:57 +0200 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: <4AB35777.7050805@erlang-consulting.com> References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> <4AB35777.7050805@erlang-consulting.com> Message-ID: <79C430A6-BFEE-4AC5-9A52-D6B6ADCF5F13@masklinn.net> On 18 Sep 2009, at 11:48 , Ulf Wiger wrote: > Masklinn wrote: >> Not so. The inner fun() executes within a transactional context >> (that of mnesia) and only uses functions relative to that context, >> which means the context manager can handle errors & retries >> gracefully. > > Associating freely and spinning the whole thing into > a much more abstract sphere, mnesia transactions are > in some ways similar to Transaction Monads in Haskell, > except in Haskell, the compiler can actually enforce > these things for you. > Very much so, but I didn't want to delve too deep into it as I don't have a very good knowledge of Haskell's STM. > Haskell is special in that it identifies side effects > as a very special operation, with its own type. > It handles side effects using a special construct, > called the monad. It handles many things going far beyond mere side-effects using monads (Haskell's lists are monads for example, and so is its option type Maybe, which has pretty nice properties), but yes Haskell does use monads quite a lot to segregate side-effecting code from "pure" code (and has various monad types for various kinds of side-effects to ensure "incompatible" side-effects -- such as transactions & IO -- can't be interleaved) From esobchenko@REDACTED Fri Sep 18 15:07:02 2009 From: esobchenko@REDACTED (Eugen Sobchenko) Date: Fri, 18 Sep 2009 06:07:02 -0700 (PDT) Subject: mnesia question: create table in a transaction Message-ID: <6c830f58-c070-48fc-b2e1-7f2dc913ce77@g1g2000vbr.googlegroups.com> create_table/2 inside a regular mnesia transaction is prohibited. There is an error on nested transaction: add_table1() -> F = fun() -> % ... mnesia:create_table( test1, [ {ram_copies, [node()]}, {attributes, record_info(fields, test)}, {record_name, test} ] ) end, mnesia:transaction(F). 4> test:add_table1(). {atomic,{aborted,nested_transaction}} After a brief search I found a solution using undocumented features of mnesia_schema.erl: add_table2() -> F = fun() -> % ... Props = [ {ram_copies, [node()]}, {attributes, record_info(fields, test)}, {record_name, test} ], Cs = mnesia_schema:list2cs([{name, test2}|Props]), mnesia_schema:do_create_table(Cs) end, mnesia_schema:schema_transaction(F). 5> test:add_table2(). {atomic,ok} However, using of mnesia_schema:do_create_table with mnesia:transaction works too: add_table3() -> F = fun() -> % ... Props = [ {ram_copies, [node()]}, {attributes, record_info(fields, test)}, {record_name, test} ], Cs = mnesia_schema:list2cs([{name, test3}|Props]), mnesia_schema:do_create_table(Cs) end, mnesia:transaction(F). 6> test:add_table3(). {atomic,ok} Is there a practical difference between the second and third version? Can I use second or third version in my production code? From ulf.wiger@REDACTED Fri Sep 18 15:26:19 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 18 Sep 2009 15:26:19 +0200 Subject: [erlang-questions] mnesia question: create table in a transaction In-Reply-To: <6c830f58-c070-48fc-b2e1-7f2dc913ce77@g1g2000vbr.googlegroups.com> References: <6c830f58-c070-48fc-b2e1-7f2dc913ce77@g1g2000vbr.googlegroups.com> Message-ID: <4AB38A7B.8020109@erlang-consulting.com> Schema transactions are more conservative. To the extent that use of these functions is at all recommended, you should not use them within normal transactions. An additional problem is that you cannot write to the table just created within the same transaction, as the table itself doesn't materialize until the transaction is committed. I made a little hack to allow that in the rdbms project. If memory serves, it was done in an experimental version of (the already fairly experimental) rdbms, which I never committed to any outside store. This particular feature seemed to work well, though. As mnesia hacks go, this one was... well, medium - 4 monkey wrenches, simplest mnesia hacks starting at 3. BR, Ulf W Eugen Sobchenko wrote: > create_table/2 inside a regular mnesia transaction is prohibited. > There is an error on nested transaction: > > add_table1() -> > F = fun() -> > % ... > mnesia:create_table( test1, > [ > {ram_copies, [node()]}, > {attributes, record_info(fields, test)}, > {record_name, test} > ] > ) > end, > mnesia:transaction(F). > > 4> test:add_table1(). > {atomic,{aborted,nested_transaction}} > > After a brief search I found a solution using undocumented features of > mnesia_schema.erl: > > add_table2() -> > F = fun() -> > % ... > Props = [ > {ram_copies, [node()]}, > {attributes, record_info(fields, test)}, > {record_name, test} > ], > Cs = mnesia_schema:list2cs([{name, test2}|Props]), > mnesia_schema:do_create_table(Cs) > end, > mnesia_schema:schema_transaction(F). > > 5> test:add_table2(). > {atomic,ok} > > However, using of mnesia_schema:do_create_table with > mnesia:transaction works too: > > add_table3() -> > F = fun() -> > % ... > Props = [ > {ram_copies, [node()]}, > {attributes, record_info(fields, test)}, > {record_name, test} > ], > Cs = mnesia_schema:list2cs([{name, test3}|Props]), > mnesia_schema:do_create_table(Cs) > end, > mnesia:transaction(F). > > 6> test:add_table3(). > {atomic,ok} > > Is there a practical difference between the second and third version? > Can I use second or third version in my production code? > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From mononcqc@REDACTED Fri Sep 18 15:44:16 2009 From: mononcqc@REDACTED (Fred Hebert (MononcQc)) Date: Fri, 18 Sep 2009 09:44:16 -0400 Subject: [erlang-questions] Re: Concept of Side-effect In-Reply-To: <4AB38510.4050009@erlang-consulting.com> References: <4062c48b-0f20-412a-b899-0e137d5a9ae7@o35g2000vbi.googlegroups.com> <51E937A5-91BB-409D-8B2F-3345C80B8923@souja.net> <804cd4020909180452p7312e36dj385bf68fe8240a10@mail.gmail.com> <452A3500-3B3A-44C6-B165-77446D37FD8A@souja.net> <4AB38510.4050009@erlang-consulting.com> Message-ID: <8b9ee55b0909180644p2861ff6fg2f73f3cf50e8ff10@mail.gmail.com> On Fri, Sep 18, 2009 at 9:03 AM, Ulf Wiger wrote: > Jayson Vantuyl wrote: > >> Actually, I'm not entirely sure that an exception would be a side effect. >> It doesn't get "returned" like a normal value, but I don't see that it >> changes state any more than a return value does. >> > > It straddles the fence in some ways. Functions can be pure for all > legal input and raise exceptions for illegal input. This is fine, > but there are also functions that raise exceptions even for legal > input. > > Consider the following code: > > pmap(F, L) -> > Pids = [spawn_monitor(fun() -> > exit({ok, F(X)}) > end) || X <- L], > [receive {'DOWN',Ref,_,_,R} -> unwrap(R) end > || {_,Ref} <- Pids]. > > (A simplified version of slide 7 in this talk: > > http://www.erlang-factory.com/upload/presentations/56/UlfWiger_ErlangMulticoreEF.pdf > ) > > While it might seem contrived, the shell evaluator > does roughly the same thing: evaluates the expression > inside an exit() call in order to deliver the result and > die in one and the same action. > > Such functions are only meaningful when considering their > side-effects - the exceptions. > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > Wouldn't the ability to just drop in and either catch errors and exit signal or trap them in a monitor mean exceptions and errors are not necessarily side effects, no matter the purpose of the function? Tthe caller's side doing: case catch Mod:Fun(Args) of {ok, X} -> X; {'EXIT',Reason} -> ... end. Pretty much ends up transforming an error into a returnable value. There is no visible difference between the function called having an error or just returning {'EXIT',Reason} as a normal value, no matter the intent of the programmer. (I guess what I'm saying is a bit similar to what you are saying, in the end) From esobchenko@REDACTED Fri Sep 18 16:32:02 2009 From: esobchenko@REDACTED (Eugen Sobchenko) Date: Fri, 18 Sep 2009 07:32:02 -0700 (PDT) Subject: mnesia question: create table in a transaction In-Reply-To: <4AB38A7B.8020109@erlang-consulting.com> References: <6c830f58-c070-48fc-b2e1-7f2dc913ce77@g1g2000vbr.googlegroups.com> <4AB38A7B.8020109@erlang-consulting.com> Message-ID: On Sep 18, 4:26?pm, Ulf Wiger wrote: > Schema transactions are more conservative. To the extent > that use of these functions is at all recommended, you > should not use them within normal transactions. Does this mean that create_table2 better than create_table_3 in the context of my problem? Or have I misunderstood something? From paul-trapexit@REDACTED Fri Sep 18 17:15:26 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Fri, 18 Sep 2009 08:15:26 -0700 (PDT) Subject: [erlang-questions] Re: mnesia question: create table in a transaction In-Reply-To: References: <6c830f58-c070-48fc-b2e1-7f2dc913ce77@g1g2000vbr.googlegroups.com> <4AB38A7B.8020109@erlang-consulting.com> Message-ID: On Fri, 18 Sep 2009, Eugen Sobchenko wrote: > On Sep 18, 4:26?pm, Ulf Wiger wrote: > > Schema transactions are more conservative. To the extent > > that use of these functions is at all recommended, you > > should not use them within normal transactions. > > Does this mean that create_table2 better than create_table_3 in the > context of my problem? > Or have I misunderstood something? an ordinary invocation of mnesia:create_table/2 already creates a schema transaction for you; the table either appears everywhere or nowhere. what ulf was getting at was, if you want the table creation to be atomic with other normal data operations, this will be very tricky. however it is unclear from your example if that was the goal, or if you merely didn't recognize that create_table uses schema_transactions internally. -- p From kamiseq@REDACTED Fri Sep 18 18:16:58 2009 From: kamiseq@REDACTED (=?UTF-8?B?cGF3ZcWCIGthbWnFhHNraQ==?=) Date: Fri, 18 Sep 2009 18:16:58 +0200 Subject: [erlang-questions] initializing process In-Reply-To: <6926854D-B620-4874-9E3E-99589FE95A4C@souja.net> References: <6926854D-B620-4874-9E3E-99589FE95A4C@souja.net> Message-ID: 2009/9/18 Jayson Vantuyl > What exactly are you trying to do? exactly this is a application server that is getting requests from its clients via http and then it is doing some computation. but the key feature of the server is talking to other peer in local network and gathering information about other devices in the area, process that information and send it to clients. the problem is that the other computer is a third-party hardware and software black-box that I connect via tcp or rss and can switches off itself. my server must be 100% sure that the other side is alive before starting everything up. if my server is not responding clients will notify local stuff that something is going on and someone will go check. the server may loose connection with black-box and again will disconnect client. it is maybe weird but help troubleshooting and I know that my server is ok but it is THEIR :) fault. > It's really easy to create complex race conditions and strange startup > ordering issues doing something like this. > > that what I was worried about > If you can have the node signal you (much less work). Just start out in an > "uninitialized" state set a short timeout and just go idle. .. > If you must ping the node from the waiting side, you can simulate a delaying > loop with a gen_server (and a gen_fsm) by using the timeout feature. Just > send the ping and set a timeout. Have the timeout function send the ping > again and set a timeout again. Eventually, you'll get the response and can > go about your merry way. > yep that is the case I need to force the communication. ok lets say my device controller is gen_fsm I initialize it and set running_disconnected fsm state, so now my fsm is waiting for some event. who will ping the other side? I can't do it in init/1 as it may lead to race conditions as you said earlier. If I loose connection device controller should switch back to running_disconnected fsm state and fold all processes that relay on connection and start pinging again. gleb's advice was to do everything in init using PeerPid!{ping} and receive, it is some way. other is to skip behaviours and implement device controller as a simple process that will do its job but both are less elegant. > I think the "correct" way (i.e. overkill in the style of Ericsson) is to do > this at an application level. You can set up a distributed Erlang > application to start in "phases". Depending on your deployment, that might > be the ultimate answer, since it also provides hooks for takeover / > failover, which may be necessary depending on your application. > > what do you mean by application level and "You can set up a distributed Erlang application to start in "phases"" how you imagine the architecture for that? thanks for replay pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From esobchenko@REDACTED Fri Sep 18 18:39:14 2009 From: esobchenko@REDACTED (Eugen Sobchenko) Date: Fri, 18 Sep 2009 09:39:14 -0700 (PDT) Subject: mnesia question: create table in a transaction In-Reply-To: References: <6c830f58-c070-48fc-b2e1-7f2dc913ce77@g1g2000vbr.googlegroups.com> <4AB38A7B.8020109@erlang-consulting.com> Message-ID: <86bffc5b-88c9-4d61-871b-ef38effb6824@q14g2000vbi.googlegroups.com> Sorry my example was not clear. I understand that the table become materialized after transaction is commited and mnesia:create_table runs schema transaction internally. I do not want to use the newly created table within a transaction. My aim is to perform certain actions on existing tables (they were omited in my example) and then create a new table within single transaction. I just want to make sure that my solutions are applicable. From lenartlad@REDACTED Fri Sep 18 18:42:38 2009 From: lenartlad@REDACTED (Ladislav Lenart) Date: Fri, 18 Sep 2009 18:42:38 +0200 Subject: [erlang-questions] initializing process In-Reply-To: References: <6926854D-B620-4874-9E3E-99589FE95A4C@souja.net> Message-ID: <4AB3B87E.3040808@volny.cz> Hello. pawe? kami?ski wrote: > 2009/9/18 Jayson Vantuyl > >> What exactly are you trying to do? > > > exactly this is a application server that is getting requests from its > clients via http and then it is doing some computation. but the key feature > of the server is talking to other peer in local network and gathering > information about other devices in the area, process that information and > send it to clients. > > the problem is that the other computer is a third-party hardware and > software black-box that I connect via tcp or rss and can switches off > itself. my server must be 100% sure that the other side is alive before > starting everything up. if my server is not responding clients will notify > local stuff that something is going on and someone will go check. the server > may loose connection with black-box and again will disconnect client. it is > maybe weird but help troubleshooting and I know that my server is ok but it > is THEIR :) fault. > > >> It's really easy to create complex race conditions and strange startup >> ordering issues doing something like this. >> >> that what I was worried about > > >> If you can have the node signal you (much less work). Just start out in an >> "uninitialized" state set a short timeout and just go idle. .. >> > If you must ping the node from the waiting side, you can simulate a delaying >> loop with a gen_server (and a gen_fsm) by using the timeout feature. Just >> send the ping and set a timeout. Have the timeout function send the ping >> again and set a timeout again. Eventually, you'll get the response and can >> go about your merry way. >> > > yep that is the case I need to force the communication. > ok lets say my device controller is gen_fsm I initialize it and set > running_disconnected fsm state, so now my fsm is waiting for some event. who > will ping the other side? I can't do it in init/1 as it may lead to race > conditions as you said earlier. You could send yourself a proper event message (that will trigger the first ping) directly in the init function. Since you do this in init, your message will be the first in the process's mailbox (noone else can know the new pid). HTH, Ladislav Lenart From kiszl@REDACTED Fri Sep 18 20:59:57 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Fri, 18 Sep 2009 20:59:57 +0200 Subject: Ets match on a record vs Dialyzer Message-ID: <4AB3D8AD.1000706@tmit.bme.hu> Hello, I would like to match entries in an ets table based on some fields of records, as in the example module below. When using the _='_' form for defining the "don't care" record fields, Dialyzer emits the following (reasonable) warnings: rec.erl:6: Function p/0 has no local return rec.erl:9: Record construction #rec{x::1,y::'_',z::'_'} violates the declared type for #rec{} With multiple functions and modules, the no local return causes an an avalanche of further no local return's and never called's. Of course I could redefine the field types as {x::integer()|'_', y::integer()|'_', z::integer()|'_'} but I would prefer another match syntax or Dialyzer directive if available. So what choices do I have? Best Regards, Zoltan. ---------------------- -module(rec). -export([p/0]). -record(rec, {x::integer(), y::integer(), z::integer()}). p() -> ets:new(table, [named_table, {keypos, #rec.x}]), ets:insert(table, [#rec{x=1,y=2,z=3}, #rec{x=4,y=5,z=6}]), ets:match_object(table, #rec{x=1, _='_'}). From paul-trapexit@REDACTED Fri Sep 18 21:09:33 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Fri, 18 Sep 2009 12:09:33 -0700 (PDT) Subject: [erlang-questions] Ets match on a record vs Dialyzer In-Reply-To: <4AB3D8AD.1000706@tmit.bme.hu> References: <4AB3D8AD.1000706@tmit.bme.hu> Message-ID: On Fri, 18 Sep 2009, Zoltan Lajos Kis wrote: > Hello, > > I would like to match entries in an ets table based on some fields of > records, as in the example module below. > When using the _='_' form for defining the "don't care" record fields, > Dialyzer emits the following (reasonable) warnings: > > rec.erl:6: Function p/0 has no local return > rec.erl:9: Record construction #rec{x::1,y::'_',z::'_'} violates the > declared type for #rec{} > > With multiple functions and modules, the no local return causes an an > avalanche of further no local return's and never called's. > > Of course I could redefine the field types as {x::integer()|'_', > y::integer()|'_', z::integer()|'_'} but I would prefer another match > syntax or Dialyzer directive if available. So what choices do I have? I've had this problem constantly. I've moved away from relaxing the type specification in the record, and towards using setelement/3 to hide from the dialyzer. It helps that when using mnesia you get mnesia:table_info (Table, wild_pattern) to initialize the record. Cheers, -- p From kostis@REDACTED Fri Sep 18 21:15:32 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 18 Sep 2009 22:15:32 +0300 Subject: [erlang-questions] Ets match on a record vs Dialyzer In-Reply-To: <4AB3D8AD.1000706@tmit.bme.hu> References: <4AB3D8AD.1000706@tmit.bme.hu> Message-ID: <4AB3DC54.2050306@cs.ntua.gr> Zoltan Lajos Kis wrote: > Hello, > > I would like to match entries in an ets table based on some fields of > records, as in the example module below. > When using the _='_' form for defining the "don't care" record fields, > Dialyzer emits the following (reasonable) warnings: > > rec.erl:6: Function p/0 has no local return > rec.erl:9: Record construction #rec{x::1,y::'_',z::'_'} violates the > declared type for #rec{} > > With multiple functions and modules, the no local return causes an an > avalanche of further no local return's and never called's. > > Of course I could redefine the field types as {x::integer()|'_', > y::integer()|'_', z::integer()|'_'} but I would prefer another match > syntax or Dialyzer directive if available. So what choices do I have? From the options you mention, the best is to use a more relaxed type declaration in the record fields of interest. IMO, it was a design mistake to use a perfectly valid Erlang term (namely the atom '_') as a symbol with a different meaning in match specifications. Kostis > ---------------------- > > -module(rec). > -export([p/0]). > > -record(rec, {x::integer(), y::integer(), z::integer()}). > > p() -> > ets:new(table, [named_table, {keypos, #rec.x}]), > ets:insert(table, [#rec{x=1,y=2,z=3}, #rec{x=4,y=5,z=6}]), > ets:match_object(table, #rec{x=1, _='_'}). > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org From kiszl@REDACTED Fri Sep 18 21:25:28 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Fri, 18 Sep 2009 21:25:28 +0200 Subject: [erlang-questions] Ets match on a record vs Dialyzer In-Reply-To: <4AB3DC54.2050306@cs.ntua.gr> References: <4AB3D8AD.1000706@tmit.bme.hu> <4AB3DC54.2050306@cs.ntua.gr> Message-ID: <4AB3DEA8.4020801@tmit.bme.hu> Kostis Sagonas wrote: > Zoltan Lajos Kis wrote: >> Hello, >> >> I would like to match entries in an ets table based on some fields of >> records, as in the example module below. >> When using the _='_' form for defining the "don't care" record >> fields, Dialyzer emits the following (reasonable) warnings: >> >> rec.erl:6: Function p/0 has no local return >> rec.erl:9: Record construction #rec{x::1,y::'_',z::'_'} violates the >> declared type for #rec{} >> >> With multiple functions and modules, the no local return causes an an >> avalanche of further no local return's and never called's. >> >> Of course I could redefine the field types as {x::integer()|'_', >> y::integer()|'_', z::integer()|'_'} but I would prefer another match >> syntax or Dialyzer directive if available. So what choices do I have? > > From the options you mention, the best is to use a more relaxed type > declaration in the record fields of interest. > > IMO, it was a design mistake to use a perfectly valid Erlang term > (namely the atom '_') as a symbol with a different meaning in match > specifications. > > Kostis I have to agree. For some reason I believed '_' and '$1' are merely syntactic sugar, and they are magically transformed into some sort of match specifications. I had some fun trying to figure out how to make the compiler accept expressions like #rec{x=1, _='_'} = '$1' until I realized these really are just atoms :) > >> ---------------------- >> >> -module(rec). >> -export([p/0]). >> >> -record(rec, {x::integer(), y::integer(), z::integer()}). >> >> p() -> >> ets:new(table, [named_table, {keypos, #rec.x}]), >> ets:insert(table, [#rec{x=1,y=2,z=3}, #rec{x=4,y=5,z=6}]), >> ets:match_object(table, #rec{x=1, _='_'}). >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org > From kagato@REDACTED Sat Sep 19 00:57:01 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 18 Sep 2009 15:57:01 -0700 Subject: [erlang-questions] initializing process In-Reply-To: References: <6926854D-B620-4874-9E3E-99589FE95A4C@souja.net> Message-ID: <89B4853C-CEAC-4DEC-A7DC-9ACBAD23EDB1@souja.net> > exactly this is a application server that is getting requests from > its clients via http and then it is doing some computation. but the > key feature of the server is talking to other peer in local network > and gathering information about other devices in the area, process > that information and send it to clients. > > the problem is that the other computer is a third-party hardware and > software black-box that I connect via tcp or rss and can switches > off itself. my server must be 100% sure that the other side is alive > before starting everything up. if my server is not responding > clients will notify local stuff that something is going on and > someone will go check. the server may loose connection with black- > box and again will disconnect client. it is maybe weird but help > troubleshooting and I know that my server is ok but it is THEIR :) > fault. This is dysfunctional, but helpful to know. I had assumed you were writing the client and the server. Since you connect to the "black box" using TCP and RSS, I wouldn't think you could do ping an Erlang node on it (since you couldn't install one). Even if you can, the Erlang node running isn't the same as their TCP connections not working, so I'd recommend a pure-TCP solution. Here's what I'm thinking. At the top, put a single supervisor, running a one_for_all strategy. It should have one child, and be registered under a name. Have the child be a gen_fsm with initialized and uninitialized modes. In the init function, start in an 'uninitialized' state and specify a very short timeout (maybe 500 milliseconds). When you receive a timeout in that state, try to use the gen_tcp:connect (or an http client to a harmless URL, if it's an HTTP device). If there is a good connection, tell the supervisor to add a temporary child (the supervisor for your real workers). If it's a bad connection, set a timeout again. In the running state, have the timeout check, but perhaps with a longer loop time. If it gets a good connection, set the timeout. If it gets a bad connection, terminate the gen_fsm by returning {stop,"Couldn't make a connection",State}. If you want to be a bit more forgiving, keep track of successive failures to connect and only terminate after maybe five consecutive failures (or somesuch). I think that this has the correct behavior. Assuming that your actual supervisor has a good supervision tree it does the following: 1) When uninitialized, tries to connect every 500 ms (or perhaps longer if packets to the remote server go into a black hole). 2) When running, tries to connect every so often. 3) When running, has added a supervisor for the worker 4) When running and connections fail, exits, causing the supervisor to kill the actual supervisor (and I believe the temporaryness of it causes it not to restart, if not, try transient). 5) Everything is managed by the supervision tree, as it should be. 6) The gen_fsm provides good logging of when and why everything died. 7) The ping function can be made arbitrarily complex but is still isolated from the actual workers. 8) No modification of the black box is required. 9) It's probably 30-50 lines of code for the gen_fsm, and one, static supervisor. This is a good problem. I teach an Erlang class. If I decide to use this as an example, I'll send you a link to the code. > what do you mean by application level and "You can set up a > distributed Erlang application to start in "phases"" how you imagine > the architecture for that? In Erlang, you are supposed to package things as an application. Applications are defined by a .app file in the ebin directory of their providing module. For example, on my system, /opt/local/lib/erlang/ lib/mnesia-4.4.9/ebin/mnesia.app contains the application definition for Mnesia. This is used by the release system to create a boot script that starts Mnesia, if it's requested. When you do application:start(mnesia), this is where it finds the information to start it up. See here: http://www.erlang.org/doc/design_principles/applications.html#7 Once you have an application, look at the start_phases stuff in the application documentation. This shows pretty well how to make an application that has multiple phases, and synchronizes them across nodes (including doing exciting things like failover). It's worth understanding, although you can probably avoid it for now. In theory, you should develop your module in some sort of OTP-like root, under lib/module-vsn. When the time comes, you can roll a boot- script that will start your system (using systools:make_script/2), automatic upgrade instructions (using systools:make_relup/4) even a whole OTP install (using systools:make_tar/2). Very few people go through the trouble, but I'm working on stuff to make it easier (as is Ericsson, see reltool). Note that this can be a complicated process, since making the upgrade scripts (i.e. relup) requires having two copies of the application in the code_path, which Erlang doesn't like to do, by default. If you want to see roughly what the structure should look like, I have a git repository on GitHub: http://github.com/jvantuyl/erl-skel It's not complete in terms of automation, but the scripts/make_release file gives an idea as to how it's done. The directory structure is right, and the Rakefile will handle building the code for most simple cases (GNU make was a bit of a problem, and rake is generally everywhere now). Actually making automated releases is still on the TODO list, and it doesn't try to build any C extensions at all. With proper automation, you can eventually have it build a tarfile. With the tarfile, you can do an initial install just by uncompressing it. When you run "erl -heart" in the uncompressed directory, it will automagically start all of your applications with the parameters you specified in your release (.rel) files, handle crashes of the entire system, and easily run as a daemon. It also makes managing multiple deployments as easy as versioning a bunch of .rel files. If you go this route, you can even use the automatic code updating stuff (i.e. code_change/3 in gen_*, update instructions in .appup files, etc.) to update a running system. With proper preparation, this deploy process can even handle downgrades or updating Erlang itself, live! Like most of Erlang, it's powerful, but the learning curve is steep. -- Jayson Vantuyl kagato@REDACTED From vlm@REDACTED Sat Sep 19 09:54:13 2009 From: vlm@REDACTED (Lev Walkin) Date: Sat, 19 Sep 2009 00:54:13 -0700 Subject: Erlang, Yaws, and the deadly Tornado Message-ID: <4AB48E25.7030301@lionet.info> ====================== Since Facebook acquisition of FriendFeed, a bunch of technologies were released to the wild, including, most notably, a Tornado web server written in Python. The Tornado is touted as a ?a scalable, non-blocking web server and web framework?. See Wikipedia article http://en.wikipedia.org/wiki/Tornado_HTTP_Server on some details on the performance of that server, as well as some comparison with other web servers. The numbers looked interesting, so I decided to benchmark Tornado myself to check out how it fares against some Erlang tools. ====================== Since the data is highly graphical, I can only give the link to the rest of the benchmark. http://lionet.livejournal.com/42016.html -- vlm From roberto@REDACTED Sat Sep 19 12:38:12 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 19 Sep 2009 12:38:12 +0200 Subject: [erlang-questions] Erlang, Yaws, and the deadly Tornado In-Reply-To: <4AB48E25.7030301@lionet.info> References: <4AB48E25.7030301@lionet.info> Message-ID: <28D58204-6C60-4318-9E2D-B8B9F96DC124@widetag.com> glad to see that yucan and misultin seem to outperform all ;) would be interesting to see also how these perform re cpu, memory usage, ... but i do get these tests need loads of time. thank you lev for sharing the results of these tests with the rest of us, cheers, r. On 19/set/09, at 09:54, Lev Walkin wrote: > > ====================== > Since Facebook acquisition of FriendFeed, a bunch of technologies > were released to the wild, including, most notably, a Tornado web > server written in Python. The Tornado is touted as a ?a scalable, > non-blocking web server and web framework?. See Wikipedia article http://en.wikipedia.org/wiki/Tornado_HTTP_Server > on some details on the performance of that server, as well as some > comparison with other web servers. > > The numbers looked interesting, so I decided to benchmark Tornado > myself to check out how it fares against some Erlang tools. > ====================== > > Since the data is highly graphical, I can only give the link to the > rest > of the benchmark. > > http://lionet.livejournal.com/42016.html > > > > -- > vlm > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From mevans@REDACTED Sat Sep 19 17:55:06 2009 From: mevans@REDACTED (Evans, Matthew) Date: Sat, 19 Sep 2009 11:55:06 -0400 Subject: [erlang-questions] Erlang, Yaws, and the deadly Tornado In-Reply-To: <4AB48E25.7030301@lionet.info> References: <4AB48E25.7030301@lionet.info> Message-ID: Thanks for sharing. I can't see in your tests what version of Erlang you are using? If R12, would R13 be a better performer? ________________________________________ From: erlang-questions@REDACTED [erlang-questions@REDACTED] On Behalf Of Lev Walkin [vlm@REDACTED] Sent: Saturday, September 19, 2009 3:54 AM To: Erlang Questions Subject: [erlang-questions] Erlang, Yaws, and the deadly Tornado ====================== Since Facebook acquisition of FriendFeed, a bunch of technologies were released to the wild, including, most notably, a Tornado web server written in Python. The Tornado is touted as a ?a scalable, non-blocking web server and web framework?. See Wikipedia article http://en.wikipedia.org/wiki/Tornado_HTTP_Server on some details on the performance of that server, as well as some comparison with other web servers. The numbers looked interesting, so I decided to benchmark Tornado myself to check out how it fares against some Erlang tools. ====================== Since the data is highly graphical, I can only give the link to the rest of the benchmark. http://lionet.livejournal.com/42016.html -- vlm ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From vlm@REDACTED Sat Sep 19 19:30:07 2009 From: vlm@REDACTED (Lev Walkin) Date: Sat, 19 Sep 2009 10:30:07 -0700 Subject: [erlang-questions] Erlang, Yaws, and the deadly Tornado In-Reply-To: References: <4AB48E25.7030301@lionet.info> Message-ID: <4AB5151F.90005@lionet.info> Evans, Matthew wrote: > Thanks for sharing. > > I can't see in your tests what version of Erlang you are using? If R12, would R13 be a better performer? R13B01 > ________________________________________ > From: erlang-questions@REDACTED [erlang-questions@REDACTED] On Behalf Of Lev Walkin [vlm@REDACTED] > Sent: Saturday, September 19, 2009 3:54 AM > To: Erlang Questions > Subject: [erlang-questions] Erlang, Yaws, and the deadly Tornado > > ====================== > Since Facebook acquisition of FriendFeed, a bunch of technologies were > released to the wild, including, most notably, a Tornado web server > written in Python. The Tornado is touted as a ?a scalable, non-blocking > web server and web framework?. See Wikipedia article > http://en.wikipedia.org/wiki/Tornado_HTTP_Server on some details on the > performance of that server, as well as some comparison with other web > servers. > > The numbers looked interesting, so I decided to benchmark Tornado myself > to check out how it fares against some Erlang tools. > ====================== > > Since the data is highly graphical, I can only give the link to the rest > of the benchmark. > > http://lionet.livejournal.com/42016.html > > > > -- > vlm > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From v@REDACTED Sat Sep 19 21:37:58 2009 From: v@REDACTED (Valentin Micic) Date: Sat, 19 Sep 2009 21:37:58 +0200 Subject: [erlang-questions] Erlang, Yaws, and the deadly Tornado In-Reply-To: <4AB5151F.90005@lionet.info> Message-ID: <200909191938.n8JJcOSw030926@mail.pharos-avantgard.com> Do you think that HiPE may make any difference here? Yes, Yucan! V. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Lev Walkin Sent: 19 September 2009 07:30 PM To: Evans, Matthew Cc: Erlang Questions Subject: Re: [erlang-questions] Erlang, Yaws, and the deadly Tornado Evans, Matthew wrote: > Thanks for sharing. > > I can't see in your tests what version of Erlang you are using? If R12, would R13 be a better performer? R13B01 > ________________________________________ > From: erlang-questions@REDACTED [erlang-questions@REDACTED] On Behalf Of Lev Walkin [vlm@REDACTED] > Sent: Saturday, September 19, 2009 3:54 AM > To: Erlang Questions > Subject: [erlang-questions] Erlang, Yaws, and the deadly Tornado > > ====================== > Since Facebook acquisition of FriendFeed, a bunch of technologies were > released to the wild, including, most notably, a Tornado web server > written in Python. The Tornado is touted as a web server and web framework>. See Wikipedia article > http://en.wikipedia.org/wiki/Tornado_HTTP_Server on some details on the > performance of that server, as well as some comparison with other web > servers. > > The numbers looked interesting, so I decided to benchmark Tornado myself > to check out how it fares against some Erlang tools. > ====================== > > Since the data is highly graphical, I can only give the link to the rest > of the benchmark. > > http://lionet.livejournal.com/42016.html > > > > -- > vlm > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From vlm@REDACTED Sat Sep 19 23:13:53 2009 From: vlm@REDACTED (Lev Walkin) Date: Sat, 19 Sep 2009 14:13:53 -0700 Subject: [erlang-questions] Erlang, Yaws, and the deadly Tornado In-Reply-To: <200909191938.n8JJcOSw030926@mail.pharos-avantgard.com> References: <200909191938.n8JJcOSw030926@mail.pharos-avantgard.com> Message-ID: <4AB54991.7050109@lionet.info> Valentin Micic wrote: > Do you think that HiPE may make any difference here? I do not think so, since it uses extra-modular calls, such as gen_tcp and lists. But let's try it anyway... SMP-128-nohipe 7000 rps,7000,0,7000,0,7000,7000,7000,0,0,0 8000 rps,7982,0,8000,0,7998,7954,7994,0,0,0 9000 rps,8755,(533),9000,5,8785,8736,8744,5,6,5 10000 rps,8969,(1500),10000,15,9012,8953,8942,14,13,18 SMP-128-HIPE 7000 rps,6706,0,7000,0,6119,7000,7000,0,0,0 8000 rps,8001,0,8000,0,8002,8000,8002,0,0,0 9000 rps,8937,(33),9000,0,8923,8892,8998,0,1,0 10000 rps,9159,(1166),10000,11,9234,8661,9582,15,10,10 As you see, HIPE scales marginally better at 0.3% errors @ 9kRPS, whereas nohipe has 5% errors @9kRPS. Overall, this picture tells that it is not worth it. Those intra-modular calls between HIPE and non-hipe code can kill the whole idea. Also, HIPE assembled code has proven unstable under real workload in my project, which is far more important factor. > Yes, Yucan! > > V. > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Lev Walkin > Sent: 19 September 2009 07:30 PM > To: Evans, Matthew > Cc: Erlang Questions > Subject: Re: [erlang-questions] Erlang, Yaws, and the deadly Tornado > > Evans, Matthew wrote: >> Thanks for sharing. >> >> I can't see in your tests what version of Erlang you are using? If R12, > would R13 be a better performer? > > R13B01 > >> ________________________________________ >> From: erlang-questions@REDACTED [erlang-questions@REDACTED] On Behalf > Of Lev Walkin [vlm@REDACTED] >> Sent: Saturday, September 19, 2009 3:54 AM >> To: Erlang Questions >> Subject: [erlang-questions] Erlang, Yaws, and the deadly Tornado >> >> ====================== >> Since Facebook acquisition of FriendFeed, a bunch of technologies were >> released to the wild, including, most notably, a Tornado web server >> written in Python. The Tornado is touted as a > web server and web framework>. See Wikipedia article >> http://en.wikipedia.org/wiki/Tornado_HTTP_Server on some details on the >> performance of that server, as well as some comparison with other web >> servers. >> >> The numbers looked interesting, so I decided to benchmark Tornado myself >> to check out how it fares against some Erlang tools. >> ====================== >> >> Since the data is highly graphical, I can only give the link to the rest >> of the benchmark. >> >> http://lionet.livejournal.com/42016.html >> >> >> >> -- >> vlm >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From vlm@REDACTED Sun Sep 20 02:01:30 2009 From: vlm@REDACTED (Lev Walkin) Date: Sat, 19 Sep 2009 17:01:30 -0700 Subject: [erlang-questions] Erlang, Yaws, and the deadly Tornado In-Reply-To: <4AB54991.7050109@lionet.info> References: <200909191938.n8JJcOSw030926@mail.pharos-avantgard.com> <4AB54991.7050109@lionet.info> Message-ID: <4AB570DA.9010709@lionet.info> Lev Walkin wrote: > Valentin Micic wrote: >> Do you think that HiPE may make any difference here? > > I do not think so, since it uses extra-modular calls, such > as gen_tcp and lists. But let's try it anyway... Kostis Sagonas has enarmed me with a hipe:c(Module) hint with which you can recompile existing (standard) modules into native ones. Therefore I redid the hipe benchmark. Here's what I did: hipe:c(lists). hipe:c(proplists). hipe:c(gen_tcp). hipe:c(inet). % Test: 18> code:is_module_native(gen_tcp). true 19> and, of course, I compiled Yucan as native, using c(yucan, [native]). Here's an updated stats: Rate,Received reply rate,Normalized error rate (1/100%),"Generated request rate (also, expected reply rate)",Error rate,Attempt 1, Attempt 2, Attempt 3, Error 1, Error 2, Error 3 1000 rps,1000,0,1000,0,1000,1000,1000,0,0,0 2000 rps,1999,0,2000,0,1999,2000,2000,0,0,0 3000 rps,2999,0,3000,0,2999,2999,2999,0,0,0 4000 rps,3997,0,4000,0,3997,3997,3997,0,0,0 5000 rps,4997,0,5000,0,4998,4998,4997,0,0,0 6000 rps,5999,0,6000,0,5999,5999,5999,0,0,0 7000 rps,6999,0,7000,0,7000,6999,7000,0,0,0 8000 rps,7998,0,8000,0,7996,7996,8003,0,0,0 9000 rps,8422,66,9000,0,8972,8988,7308,0,1,1 10000 rps,8523,900,10000,9,7773,8234,9563,9,7,11 Compared to the earlier ones: > SMP-128-nohipe > > 7000 rps,7000,0,7000,0,7000,7000,7000,0,0,0 > 8000 rps,7982,0,8000,0,7998,7954,7994,0,0,0 > 9000 rps,8755,(533),9000,5,8785,8736,8744,5,6,5 > 10000 rps,8969,(1500),10000,15,9012,8953,8942,14,13,18 > > SMP-128-HIPE > > 7000 rps,6706,0,7000,0,6119,7000,7000,0,0,0 > 8000 rps,8001,0,8000,0,8002,8000,8002,0,0,0 > 9000 rps,8937,(33),9000,0,8923,8892,8998,0,1,0 > 10000 rps,9159,(1166),10000,11,9234,8661,9582,15,10,10 Here's a summary: 8000 RPS err % 9k RPS err% 10kRPS err% Non-HIPE 0% 5% 15% HiPE(yucan) 0% 0.3% 11% HiPE([...]) 0% 0.6% 9% Interpretation: contrary to my earlier assumption, compiling the yucan.erl module using HiPE was getting me the most benefit. I now attribute it to using {packet, http} filter, which is already implemented in C as part of Erlang VM. No significant amount of extra-modular calls are employed in Yucan, thus compiling the rest of the system modules (gen_tcp, gen_server, inet) into HiPE helped too much. Overall, the effect from HiPE on the numer of error-free connections per second seems to be about 10%, which, in my particular case, does not justify any additional deployment complexities. > As you see, HIPE scales marginally better at 0.3% errors @ 9kRPS, > whereas nohipe has 5% errors @9kRPS. > > Overall, this picture tells that it is not worth it. Those intra-modular > calls between HIPE and non-hipe code can kill the whole idea. Also, HIPE > assembled code has proven unstable under real workload in my project, > which is far more important factor. > >> Yes, Yucan! >> >> V. >> >> -----Original Message----- >> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On >> Behalf Of Lev Walkin >> Sent: 19 September 2009 07:30 PM >> To: Evans, Matthew >> Cc: Erlang Questions >> Subject: Re: [erlang-questions] Erlang, Yaws, and the deadly Tornado >> >> Evans, Matthew wrote: >>> Thanks for sharing. >>> I can't see in your tests what version of Erlang you are using? If R12, >> would R13 be a better performer? >> >> R13B01 >> >>> ________________________________________ >>> From: erlang-questions@REDACTED [erlang-questions@REDACTED] On >>> Behalf >> Of Lev Walkin [vlm@REDACTED] >>> Sent: Saturday, September 19, 2009 3:54 AM >>> To: Erlang Questions >>> Subject: [erlang-questions] Erlang, Yaws, and the deadly Tornado >>> >>> ====================== >>> Since Facebook acquisition of FriendFeed, a bunch of technologies were >>> released to the wild, including, most notably, a Tornado web server >>> written in Python. The Tornado is touted as a >> web server and web framework>. See Wikipedia article >>> http://en.wikipedia.org/wiki/Tornado_HTTP_Server on some details on the >>> performance of that server, as well as some comparison with other web >>> servers. >>> >>> The numbers looked interesting, so I decided to benchmark Tornado myself >>> to check out how it fares against some Erlang tools. >>> ====================== >>> >>> Since the data is highly graphical, I can only give the link to the rest >>> of the benchmark. >>> >>> http://lionet.livejournal.com/42016.html >>> >>> >>> >>> -- >>> vlm >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From jlprasantha@REDACTED Sun Sep 20 12:29:44 2009 From: jlprasantha@REDACTED (prasantha kumara) Date: Sun, 20 Sep 2009 15:59:44 +0530 Subject: how to improve the http:request send messages per second Message-ID: hi all i used http:request to send http request to the my java appliaction. but when i increse the the no of messages per second send tothe application.it fails that is it does not deliver some messages to the application can anybody give infomation to resole this thanks prasantha From corticalcomputer@REDACTED Sun Sep 20 13:35:41 2009 From: corticalcomputer@REDACTED (G.S.) Date: Sun, 20 Sep 2009 04:35:41 -0700 Subject: Any way to correct the round off errors? Message-ID: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> Hello everyone, When subtracting in Erlang: 0.92915-0.92945 we should get -0.0003 but Erlang gives: -2.9999999999996696e-4 (when doing 92915-92945 Erlang gives -30 so that's ok). Anyway to make make it give -0.0003 ?, and in general make it give more accurate answers? Regards, -Gene From vlm@REDACTED Sun Sep 20 13:44:36 2009 From: vlm@REDACTED (Lev Walkin) Date: Sun, 20 Sep 2009 04:44:36 -0700 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> Message-ID: <4AB615A4.5070908@lionet.info> G.S. wrote: > Hello everyone, > > When subtracting in Erlang: 0.92915-0.92945 we should get -0.0003 but Erlang > gives: -2.9999999999996696e-4 (when doing 92915-92945 Erlang gives -30 so > that's ok). > > Anyway to make make it give -0.0003 ?, and in general make it give more > accurate answers? Your computer hardware cannot represent 0.0003. [vlm@REDACTED:~]> cc -o c c.c && ./c -0.000300 -0.00029999999999999997 [vlm@REDACTED:~]> cat c.c #include int main() { double d = -0.0003; printf("%f\n", d); printf("%.20f\n", d); } [vlm@REDACTED:~]> So, Erlang does not give you less accurate numbers. Erlang just gives you a differently formatted decimal representation of a binary number which can't be 0.0003. -- vlm From kagato@REDACTED Sun Sep 20 13:43:19 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Sun, 20 Sep 2009 04:43:19 -0700 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> Message-ID: For the record, I just tested the same math in Python and Ruby. Same result. Java probably would as well. They all represent real numbers as IEEE 754 floating point numbers. The error you're seeing is because the fractions are represented as binary fractions. The only way you're going to get "precise" numbers is to use fixed-point arithmetic. Keep in mind that accurate is in the eye of the beholder. While you want to represent 3/10000 (which would be more accurate with base-10 fixed point arithmetic), if you wanted something more like 1/65536, floating point would be dead-on accurate, but fixed point base-10 would have error. Here's a partial discussion that may be of use. I probably should put implementing a decimal module on my list of things to do, though, because this is hardly a solved problem (and would be extra useful for handling money values). http://www.trapexit.org/forum/viewtopic.php?p=44093 On Sep 20, 2009, at 4:35 AM, G.S. wrote: > Hello everyone, > > When subtracting in Erlang: 0.92915-0.92945 we should get -0.0003 > but Erlang > gives: -2.9999999999996696e-4 (when doing 92915-92945 Erlang gives > -30 so > that's ok). > > Anyway to make make it give -0.0003 ?, and in general make it give > more > accurate answers? > > Regards, > -Gene -- Jayson Vantuyl kagato@REDACTED From kiszl@REDACTED Sun Sep 20 14:03:45 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sun, 20 Sep 2009 14:03:45 +0200 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <4AB615A4.5070908@lionet.info> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB615A4.5070908@lionet.info> Message-ID: <4AB61A21.7030406@tmit.bme.hu> Lev Walkin wrote: > G.S. wrote: >> Hello everyone, >> >> When subtracting in Erlang: 0.92915-0.92945 we should get -0.0003 but >> Erlang >> gives: -2.9999999999996696e-4 (when doing 92915-92945 Erlang gives >> -30 so >> that's ok). >> >> Anyway to make make it give -0.0003 ?, and in general make it give more >> accurate answers? > > Your computer hardware cannot represent 0.0003. > > [vlm@REDACTED:~]> cc -o c c.c && ./c > -0.000300 > -0.00029999999999999997 > [vlm@REDACTED:~]> cat c.c > #include > > int main() { > double d = -0.0003; > printf("%f\n", d); > printf("%.20f\n", d); > } > [vlm@REDACTED:~]> > > > So, Erlang does not give you less accurate numbers. > Erlang just gives you a differently formatted decimal > representation of a binary number which can't be 0.0003. > It's not the computer hardware, but the representation used for floating point numbers... You can come up with your own library of arithmetic functions where you represent decimals as you wish. Of course you will have to trade performance for the desired accuracy. Java for example has BigDecimals for this cause: >java Zed -0.00030 ------------- import java.math.BigDecimal; class Zed { public static void main(String[] args) { BigDecimal x = new BigDecimal("0.92915"); BigDecimal y = new BigDecimal("0.92945"); System.out.println( x.subtract(y) ); } } From r.kelsall@REDACTED Sun Sep 20 16:02:18 2009 From: r.kelsall@REDACTED (Richard Kelsall) Date: Sun, 20 Sep 2009 15:02:18 +0100 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> Message-ID: <4AB635EA.8070203@millstream.com> > When subtracting in Erlang: 0.92915-0.92945 we should get -0.0003 but Erlang > gives: -2.9999999999996696e-4 (when doing 92915-92945 Erlang gives -30 so > that's ok). > > Anyway to make make it give -0.0003 ?, and in general make it give more > accurate answers? That's an interesting question. The other replies tell us it's a 64 bit Double floating point representation : http://en.wikipedia.org/wiki/Double_precision_floating-point_format which has, according to Wikipedia, "about 16 decimal digits" of precision from the 52 bits. Their example is correct up to 16 digits : 0.33333333333333331482 * 1234567890123456 but yours is only correct up to 13 digits : 2.9999999999996696e-4 * 1 234567890123456 so we probably shouldn't trust more than 12 digits of precision because each calculation will lose some precision from the end of the number. Different calculations will lose different amounts of precision and very long sequences of calculations could reduce the number of correct digits below 12. You might think it would be nice to have every calculation keep a record in the double of how many digits we can trust. If we did this the output routine could give the exact answer to your calculation. But this would be very difficult to calculate and would consume processor power at every step. So instead floating point routines rely on the programmer to know about floating point accuracy and you always have to round the output appropriately. If you round that answer to 12 digits it is exactly correct. In general always use a number representation that can cope with numbers much bigger and with much more accuracy than you are expecting because bugs in this area are nasty. Richard. From baryluk@REDACTED Sun Sep 20 17:22:11 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Sun, 20 Sep 2009 17:22:11 +0200 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <4AB635EA.8070203@millstream.com> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB635EA.8070203@millstream.com> Message-ID: <1253460131.22238.51.camel@sredniczarny> Dnia 2009-09-20, nie o godzinie 15:02 +0100, Richard Kelsall pisze: > > When subtracting in Erlang: 0.92915-0.92945 we should get -0.0003 but Erlang > > gives: -2.9999999999996696e-4 (when doing 92915-92945 Erlang gives -30 so > > that's ok). > > > > Anyway to make make it give -0.0003 ?, and in general make it give more > > accurate answers? > > That's an interesting question. The other replies tell us it's a 64 bit > Double floating point representation : > > http://en.wikipedia.org/wiki/Double_precision_floating-point_format > > which has, according to Wikipedia, "about 16 decimal digits" of > precision from the 52 bits. Their example is correct up to 16 digits : > > 0.33333333333333331482 > * > 1234567890123456 > > but yours is only correct up to 13 digits : > > 2.9999999999996696e-4 > * > 1 234567890123456 Wikipedia says that it have about 16 decimal digits (52 or 53 binary digits), but it says nothing about how many of them are correct. Information about 16 digits is only providad because ie. printing more than 16 decimal digits doesn't provide any more information (rest of digits after 53 is just zeros, and after converting them to decimal they just some "random" digits, completly insignificant [after converting back to binary they are smaller than 2^-53 anyway, so efectively 0.0). > > so we probably shouldn't trust more than 12 digits of precision because > each calculation will lose some precision from the end of the number. Substraction (and addition) can lose any number of digits you wish. > Different calculations will lose different amounts of precision and > very long sequences of calculations could reduce the number of correct > digits below 12. Yep. > You might think it would be nice to have every calculation keep a record > in the double of how many digits we can trust. It is called interval methods. They are very robust. Can be connected with other techniques for fast and accurate (with 100% certainity) calculations. > If we did this the output > routine could give the exact answer to your calculation. But this would > be very difficult to calculate and would consume processor power at > every step. So instead floating point routines rely on the programmer > to know about floating point accuracy and you always have to round the > output appropriately. If you round that answer to 12 digits it is > exactly correct. > > In general always use a number representation that can cope with numbers > much bigger and with much more accuracy than you are expecting because > bugs in this area are nasty. It all depends what you want. Floating point number are just tricky. both 0.92915 and 0.92945 are already not representable as binary floating point numbers: > io:format("~.30f~n", [0.92915]). 0.929150000000000031442000000000 > 5> io:format("~.30f~n", [0.92945]). 0.929449999999999998401000000000 ok If you want something which calculates correctly in decimal number, use integers multiplied with some 10^k power. It is called fixed point arithmetic. -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From masklinn@REDACTED Sun Sep 20 18:06:48 2009 From: masklinn@REDACTED (Masklinn) Date: Sun, 20 Sep 2009 18:06:48 +0200 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <1253460131.22238.51.camel@sredniczarny> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB635EA.8070203@millstream.com> <1253460131.22238.51.camel@sredniczarny> Message-ID: On 20 Sep 2009, at 17:22 , Witold Baryluk wrote: > > If you want something which calculates correctly in decimal number, > use > integers multiplied with some 10^k power. It is called fixed point > arithmetic. Or use a decimal floating-point library such as gmp. From r.kelsall@REDACTED Sun Sep 20 18:08:40 2009 From: r.kelsall@REDACTED (Richard Kelsall) Date: Sun, 20 Sep 2009 17:08:40 +0100 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <1253460131.22238.51.camel@sredniczarny> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB635EA.8070203@millstream.com> <1253460131.22238.51.camel@sredniczarny> Message-ID: <4AB65388.8050601@millstream.com> Witold Baryluk wrote: >> so we probably shouldn't trust more than 12 digits of precision because >> each calculation will lose some precision from the end of the number. > Substraction (and addition) can lose any number of digits you wish. > I would be horrified if I added two doubles 0.111111111111 + 0.111111111111 and got 0.225745048327 I have no idea what the IEEE standard specifies, but I can't imagine anybody ever implementing or using a version that gave this answer. I would expect at least twelve significant digits of precision. 0.2222222222228765767 But if I added these two doubles 0.111111111111 + 0.000000000000111111111111 I wouldn't be surprised to get 0.111111111111343786587698 which gives the right answer to 12 significant digits, but loses all of the significant digits in the second number.The same for subtraction. Richard. From baryluk@REDACTED Sun Sep 20 18:12:31 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Sun, 20 Sep 2009 18:12:31 +0200 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB635EA.8070203@millstream.com> <1253460131.22238.51.camel@sredniczarny> Message-ID: <1253463151.22238.154.camel@sredniczarny> Dnia 2009-09-20, nie o godzinie 18:06 +0200, Masklinn pisze: > On 20 Sep 2009, at 17:22 , Witold Baryluk wrote: > > > > If you want something which calculates correctly in decimal number, > > use > > integers multiplied with some 10^k power. It is called fixed point > > arithmetic. > Or use a decimal floating-point library such as gmp. Sure. But considering that Erlang already have arbitrary precision integers, writing few functions for add/sub/mul/div isn't very hard :) It will be more portable than linking to gmp, :) Still operations will need to be called explicitly, because Erlang doesn't have operator overloading. ;) -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From baryluk@REDACTED Sun Sep 20 18:21:03 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Sun, 20 Sep 2009 18:21:03 +0200 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <4AB65388.8050601@millstream.com> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB635EA.8070203@millstream.com> <1253460131.22238.51.camel@sredniczarny> <4AB65388.8050601@millstream.com> Message-ID: <1253463663.22238.172.camel@sredniczarny> Dnia 2009-09-20, nie o godzinie 17:08 +0100, Richard Kelsall pisze: > Witold Baryluk wrote: > >> so we probably shouldn't trust more than 12 digits of precision because > >> each calculation will lose some precision from the end of the number. > > Substraction (and addition) can lose any number of digits you wish. > > > I would be horrified if I added two doubles > > 0.111111111111 + > 0.111111111111 > > and got > > 0.225745048327 > > I have no idea what the IEEE standard specifies, but I can't imagine > anybody ever implementing or using a version that gave this answer. > I would expect at least twelve significant digits of precision. > > 0.2222222222228765767 > > But if I added these two doubles > > 0.111111111111 + > 0.000000000000111111111111 > > I wouldn't be surprised to get > > 0.111111111111343786587698 > > which gives the right answer to 12 significant digits, but loses all > of the significant digits in the second number.The same for subtraction. > > > Richard. Maybe i written not clearly. I was thinking more about such situation: > 0.111111111111 - 0.111111111113. -1.9999973899231804e-12 -0.000000000001999997389923180435 > It have only one correct significant decimal digit. -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From r.kelsall@REDACTED Sun Sep 20 18:27:12 2009 From: r.kelsall@REDACTED (Richard Kelsall) Date: Sun, 20 Sep 2009 17:27:12 +0100 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <1253463663.22238.172.camel@sredniczarny> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB635EA.8070203@millstream.com> <1253460131.22238.51.camel@sredniczarny> <4AB65388.8050601@millstream.com> <1253463663.22238.172.camel@sredniczarny> Message-ID: <4AB657E0.7080708@millstream.com> Witold Baryluk wrote: > Maybe i written not clearly. I was thinking more about such situation: > >> 0.111111111111 - 0.111111111113. > -1.9999973899231804e-12 > -0.000000000001999997389923180435 > > It have only one correct significant decimal digit. > You are quite right. I hadn't thought of that one. Richard. From masklinn@REDACTED Sun Sep 20 19:00:07 2009 From: masklinn@REDACTED (Masklinn) Date: Sun, 20 Sep 2009 19:00:07 +0200 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <1253463151.22238.154.camel@sredniczarny> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB635EA.8070203@millstream.com> <1253460131.22238.51.camel@sredniczarny> <1253463151.22238.154.camel@sredniczarny> Message-ID: <36DDE5AC-AED6-4584-ACC7-651C91567FB1@masklinn.net> On 20 Sep 2009, at 18:12 , Witold Baryluk wrote: > Dnia 2009-09-20, nie o godzinie 18:06 +0200, Masklinn pisze: >> On 20 Sep 2009, at 17:22 , Witold Baryluk wrote: >>> If you want something which calculates correctly in decimal number, >>> use >>> integers multiplied with some 10^k power. It is called fixed point >>> arithmetic. >> Or use a decimal floating-point library such as gmp. > > Sure. But considering that Erlang already have arbitrary precision > integers, writing few functions for add/sub/mul/div isn't very hard :) Arbitrary precision integers are quite different beasts (and easier to handle) than arbitrary precision floats. Most languages with native arbitrary precision integers (python, ruby, Haskell's Integer type, ?) tend to relegate arbitrary precision floats to libraries, either stdlib or third-party, and pretty much systematically rely on IEEE754 floating-points natively (though they might have fraction types between integer and reals). > It will be more portable than linking to gmp, :) Not necessarily (gmp is quite damn portable), and it would be much harder to handle correctly. FWIW, Python's decimal module is 2500 lines of pure python. From baryluk@REDACTED Sun Sep 20 19:20:15 2009 From: baryluk@REDACTED (Witold Baryluk) Date: Sun, 20 Sep 2009 19:20:15 +0200 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <36DDE5AC-AED6-4584-ACC7-651C91567FB1@masklinn.net> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB635EA.8070203@millstream.com> <1253460131.22238.51.camel@sredniczarny> <1253463151.22238.154.camel@sredniczarny> <36DDE5AC-AED6-4584-ACC7-651C91567FB1@masklinn.net> Message-ID: <1253467215.22238.302.camel@sredniczarny> Dnia 2009-09-20, nie o godzinie 19:00 +0200, Masklinn pisze: > On 20 Sep 2009, at 18:12 , Witold Baryluk wrote: > > Dnia 2009-09-20, nie o godzinie 18:06 +0200, Masklinn pisze: > >> On 20 Sep 2009, at 17:22 , Witold Baryluk wrote: > >>> If you want something which calculates correctly in decimal number, > >>> use > >>> integers multiplied with some 10^k power. It is called fixed point > >>> arithmetic. > >> Or use a decimal floating-point library such as gmp. > > > > Sure. But considering that Erlang already have arbitrary precision > > integers, writing few functions for add/sub/mul/div isn't very hard :) > > Arbitrary precision integers are quite different beasts (and easier to > handle) than arbitrary precision floats. I was talking about arbitrary precision integer arithmetic (or fixed point arithmetic). Linking to gmp is just additional requirement, which is unnecessary given the fact that Erlang already have integer arithmetic. Something like this will suffice: -module(fpa). add({X,F},{Y,F}) -> {X+Y,F}. sub({X,F},{Y,F}) -> {X-Y,F}. neg({X,F}) -> {-X,F}. mul({X,F},{Y,F}) -> {(X*Y) div F, F}. div({X,F},{Y,F}) -> {(X*F) div Y, F}. fpa_to_float({X,F}) -> X / F. float_to_fpa6(X) -> {trunc(1000000*X), 1000000}. "Accurate" binary floating point arithmetic is completely different story, and few order of magnitud harder, and still quite active research area in numerical analysis. -- Witold Baryluk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: To jest cz??? wiadomo?ci podpisana cyfrowo URL: From r.kelsall@REDACTED Sun Sep 20 19:36:07 2009 From: r.kelsall@REDACTED (Richard Kelsall) Date: Sun, 20 Sep 2009 18:36:07 +0100 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <4d08db370909200949t268b7125ld65f7d11d11ed2ad@mail.gmail.com> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB635EA.8070203@millstream.com> <1253460131.22238.51.camel@sredniczarny> <4AB65388.8050601@millstream.com> <4d08db370909200949t268b7125ld65f7d11d11ed2ad@mail.gmail.com> Message-ID: <4AB66807.3080104@millstream.com> Hynek Vychodil wrote: > Interesting, but seems well for me. > > $ erl > Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.7.2 (abort with ^G) > 1> 0.111111111111 + 0.111111111111. > 0.222222222222 > 2> 0.111111111111 - 0.111111111111. > 0.0 > 3> 0.111111111111 + 0.11111111111. > 0.222222222221 > 4> 0.111111111111 - 0.11111111111. > 1.0000056338554941e-12 Yes, as far as I know Erlang is working perfectly. I was only speaking hypothetically. Sorry for the noise. Richard. From steven.charles.davis@REDACTED Sun Sep 20 20:29:56 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 20 Sep 2009 11:29:56 -0700 (PDT) Subject: SCTP accept? In-Reply-To: References: Message-ID: ...prim_inet doesn't even appear to have accept support for anything but TCP. Does this mean the Erlang SCTP implementation is worthless for any real use? I'm beginning to wonder if the "UDP-like" nature of SCTP has been misunderstood in the actual Erlang implementation. SCTP, like TCP, is connection-oriented and therefore you do need the accept step to allow multiple clients. Other api implementations eg python/java that I checked over do require this step, so it seems to me that perhaps the offering in Erlang isn't complete?? /sd From toby@REDACTED Sun Sep 20 23:11:23 2009 From: toby@REDACTED (Toby Thain) Date: Sun, 20 Sep 2009 17:11:23 -0400 Subject: [erlang-questions] No, really, please stop misinterpreting what I wrote In-Reply-To: References: Message-ID: <47E5B5A5-FC1B-4E6F-A91B-5F140DEBA5A5@telegraphics.com.au> On 17-Sep-09, at 7:05 AM, Michael Turner wrote: > ... > In a paper I can't immediately identify right now, the authors > remarked > that Erlang programmers often spend a fair amount of time trying to > measure what's fast in Erlang, then writing stuff using what they > discover is fast. The authors were disturbed, saying that they'd > prefer that Erlang programmers implement things so as to be *clear* in > Erlang, so that maintainers of the Erlang interpreter and compiler > would > know what to target for optimization. This is true of any language, I think, and particularly true of declarative languages (I am thinking of SQL, for example). > > And now we have James Hague pointing out that an ambiguity (?) in > Erlang > semantics means that he's now got a great optimization, but one that > might go away if Erlang is allowed to optimize it out. Well, C is > really great for optimizing stuff, but I've had my "lightweight > semantics" optimizations in that language optimized out by a > subsequent > release of the compiler, and I deserved what I got. One of C's > co-designers contributed the now-famous saying: "Premature > optimization > is the root of all evil in software." To be pedantic, I think that was actually Professor Knuth*. --T > Probably because, with that > language being what it is, he'd seen a lot more evil than most of us. > > It's hard habit to acquire (because optimization can be so much fun), > but it's a good one: the habit of asking yourself, in contemplating an > optimization, "Would this solve the real problem? Or just compensate > for it?" ... > > -michael turner > * http://shreevatsa.wordpress.com/2008/05/16/premature-optimization- is-the-root-of-all-evil/ + http://www-cs-staff.stanford.edu/~uno/ > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From kagato@REDACTED Mon Sep 21 01:37:41 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Sun, 20 Sep 2009 16:37:41 -0700 Subject: [erlang-questions] Re: SCTP accept? In-Reply-To: References: Message-ID: Short Version: Strictly speaking, SCTP doesn't really have an accept at the protocol level. It only has associations. While the protocol has a specification (RFC 2960), the socket API is only a draft. The draft does specify an accept function, but it's only for convenience of people who write imperative, heavily state-laden code. Since the protocol doesn't really have an "accept" at the low-level, it's all really mental acrobatics on the part of the Socket API. The Erlang implementation appears to use the API that fits SCTP's primary use-case. Associations happen automatically if you're listening, and you can kill them if you don't like them. I don't have a working SCTP deployment to play with right now, but I'm pretty sure that new associations come in as assoc_change events when you're receiving. Long Version: I've looked at two implementations; one in Python (pySCTP) and one in Java (javaSCTP). The Java one does not have an accept function. The Python one does. The Socket API draft goes so far as to have two types of SCTP-sockets, a one-to-many-style socket, and a one-to-one- style socket. At a protocol level, SCTP just forms associations. You can ask it to do so, and the behavior of the server isn't exactly specified. In the Socket API draft, they suggest that one-to-many sockets should automatically accept associations (like UDP does), and one-to-one sockets are either accepted explicitly (like TCP) or peeled off of a many-to-many socket using sctp_peeloff (unlike anything else at all). I personally think that this draft does a good job at enabling people to use SCTP in completely the wrong way. I think it's what happens when people who think in terms of threaded, stateful code work with a protocol that's good for stateless, event-driven coding. Basically, horror ensues. This is all largely because the use-case for SCTP is very different from the one for UDP or TCP. TCP is largely made for individual connections where a single server and single client have a meaningful relationship in a stream of bytes that ends when the connection closes. UDP is largely made for extremely unstructured blasting of stateless records of data between arbitrary combinations of hosts. SCTP is made for blended situations where a mesh of clients and servers blast stateless records to each other, interleaved with ordered streams of records. As an example of *the* SCTP use-case, consider tunneling phone-switch data over the internet. You may have a dozen phone switches, each blasting each other with streams of events that must be processed in a certain order. You may have multiple streams, e.g. control information (allocate this line, this line is ringing), load information (I'm loaded, route around me), messaging information (here are some SMS messages). On the other hand, there may be diagnostic information that is sent completely outside of any stream (I'm shutting down, I had a hardware fault, etc). Each of these might be of relative importance. Each of these typically will take the form of records of a specific size. Some of these should be ordered with respect to each other, but not all of them. This is where TCP falls down and SCTP shines. For TCP, each ordered stream would be a different connection between the machines. Each connection could be lost and would have to be re- established. If you were to multiplex multiple connections in a single TCP connection, you get head-of-line blocking problems. When you have record oriented data, you have to frame it with data structures to tell when one record ends and the other begins. If you get out of sync, the only way to resynchronize reliably may be to kill the connection. It's just not optimal. With SCTP, you simply initiate associations and watch for data / events. Data shows up in neat packets, marked with their source. Streams are neatly independent. There is no need to do framing. There are no head-of-line problems that you don't create for yourself. There is no management of associations, they just form and break as is possible. On Sep 20, 2009, at 11:29 AM, Steve Davis wrote: > ...prim_inet doesn't even appear to have accept support for anything > but TCP. Does this mean the Erlang SCTP implementation is worthless > for any real use? > > I'm beginning to wonder if the "UDP-like" nature of SCTP has been > misunderstood in the actual Erlang implementation. SCTP, like TCP, is > connection-oriented and therefore you do need the accept step to allow > multiple clients. Other api implementations eg python/java that I > checked over do require this step, so it seems to me that perhaps the > offering in Erlang isn't complete?? > > /sd > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From pablo.platt@REDACTED Mon Sep 21 02:12:56 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Sun, 20 Sep 2009 17:12:56 -0700 (PDT) Subject: packaging erlang application for ubuntu Message-ID: <792758.38980.qm@web31806.mail.mud.yahoo.com> Hi, I have an erlang application that I want to distribute to clients using ubuntu. I don't want to include the source code, only the compiled beam files. The installation needs to be automatic, install all the dependencies and create a boot script that will run the application as daemon on startup. Is reltools fits this task? Does an application release includes the erlang runtime so copying this folder will be sufficient to run the application? Installing the erlang ubuntu package separately might be a problem because the client might have a different erlang version installed or the defaultubuntu package might be different then the version the application was compiled against. What ubuntu packages are required to run a minimal erlang installation? Can someone provide an example of a simple erlang application packaged for ubuntu as a reference? Thanks From ok@REDACTED Mon Sep 21 03:31:13 2009 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 21 Sep 2009 13:31:13 +1200 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> Message-ID: On Sep 20, 2009, at 11:35 PM, G.S. wrote: > Hello everyone, > > When subtracting in Erlang: 0.92915-0.92945 we should get -0.0003 > but Erlang > gives: -2.9999999999996696e-4 (when doing 92915-92945 Erlang gives > -30 so > that's ok). > > Anyway to make make it give -0.0003 ?, and in general make it give > more > accurate answers? Erlang *DID* give you the most accurate answer possible. Here is a little C program. m% cat zoo.c #include int main(void) { double const x = 0.92915; double const y = 0.92945; printf("0.92915 = %.20f\n", x); printf(" 0.92945 = %.20f\n", y); printf("0.92915 - 0.92945 = %.20f\n", x - y); return 0; } m% cc zoo.c m% a.out 0.92915 = 0.92915000000000003144 0.92945 = 0.92944999999999999840 0.92915 - 0.92945 = -0.00029999999999996696 The problem is that the numbers 0.92915 an 0.92945 CANNOT BE REPRESENTED EXACTLY IN BINARY FLOATING-POINT. There's a new standard for decimal floating-point, which I believe is supported in shipping versions of the z/Series and POWER machines, and it will take a lot of the nasty surprise out of things like this. The answer you got involved four steps where round-off error can occur: decimal -> binary decimal -> binary again subtraction binary -> decimal Giving any answer than what Erlang gave would be giving WRONG answers. If you want to *print* the result to a lower precision, so that the effects of round-off error are (sometimes) (partially) hidden (if you are lucky), that's easy. The interactive top level will already print this number as -3.00000e-4. You can use vaguely C-like formats such as io:fwrite("~.6g", [-0.00029999999999996696]). Note that this doesn't change what the answer *is* (it's already as good as you have any right to expect), it just changes *how it is displayed*. From ok@REDACTED Mon Sep 21 03:50:23 2009 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 21 Sep 2009 13:50:23 +1200 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <4AB65388.8050601@millstream.com> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB635EA.8070203@millstream.com> <1253460131.22238.51.camel@sredniczarny> <4AB65388.8050601@millstream.com> Message-ID: <603F5CF3-AD4B-4262-B4CD-5F096FC33EAE@cs.otago.ac.nz> On Sep 21, 2009, at 4:08 AM, Richard Kelsall wrote: > Witold Baryluk wrote: >>> so we probably shouldn't trust more than 12 digits of precision >>> because >>> each calculation will lose some precision from the end of the >>> number. >> Substraction (and addition) can lose any number of digits you wish. > I would be horrified if I added two doubles > > 0.111111111111 + > 0.111111111111 > > and got > > 0.225745048327 Yes, but you missed the point. Those two numbers have the same sign. Witold Baryluk was talking about subtraction. Adding two numbers of opposite signs does subtraction. If you have x+y+e and x+z+f, where x is the "common" part of two similar numbers, y and z are the true differences, and e and f are errors, then (x+y) - (x+z) = y-z but (x+y+e) - (x+z+f) = y-z + e-f and the errors e, f that were small compared with x may be extremely large compared with y-z. Read for example http://www.cs.princeton.edu/introcs/91float/ specifically the section beginning "Catastrophic cancellation. Devastating loss of precision when small numbers are computed from large numbers by addition or subtraction." > I have no idea what the IEEE standard specifies, Well, shouldn't you _find out_? I mean, before using something as weird (but widespread) as floating point arithmetic, shouldn't you take the trouble to find out what it is *supposed* to do? The IEEE 754 standard is small and tolerably clear; there are drafts and summaries and review articles about it all over the web. > but I can't imagine > anybody ever implementing or using a version that gave this answer. They don't. Nobody ever suggested it would. The original example subtracts two numbers with similar values. Any fixed-width floating point system ever built is going to have trouble with that. IEEE floating-point arithmetic was designed with exceptional care and a demand for good behaviour even when that conflicted with speed. From brady.mccary@REDACTED Mon Sep 21 04:11:42 2009 From: brady.mccary@REDACTED (Brady McCary) Date: Sun, 20 Sep 2009 21:11:42 -0500 Subject: odbc:param_query with null? Message-ID: Erlangers, Is it possible to pass a null value as a parameter in odbc:param_query? Something like: T1 = {sql_varchar, 20}, V1 = null, P1 = {T1, [V1]}, odbc:param_query(Ref, "exec my_sp @arg1 = ?", [P1]). T2 = {sql_varchar, 20}, V2 = "blah", P2 = {T2, [V2]}, odbc:param_query(Ref, "exec my_sp @arg1 = ?", [P2]). The second query works. The first query does not work, rather it errors with the following: {badarg,odbc,param_query,'Params'} It seems reasonable that there is a convention to pass an sql NULL. I searched the odbc source for these to try and see if there was any reasoning to convert something to an sql NULL, but I couldn't find what I was looking for. Any help would be much appreciated. Brady From bopolissimus.lists@REDACTED Mon Sep 21 07:53:52 2009 From: bopolissimus.lists@REDACTED (Bopolissimus Platypus Jr) Date: Mon, 21 Sep 2009 17:53:52 +1200 Subject: shell question - import commands from external file Message-ID: Hi all, I have a newbie question. In the shell, is there a way to import *shell* commands from an external file? I'm often experimenting in the shell, e.g., writing a 5 or 8 line fun, trying out this or that function call, etc. Since there are often mistakes, I need to rewrite the fun a few times. Instead of having to retype (or look up in history) the several lines, I'd like to write the fun in a temporary file (e.g., ./tmp.erl) and then just load it into the shell as if i'd typed it in (e.g., in the postgresql shell I would do, "\i [filename"). I realize that I could write the experimental code into a temporary module, compile that and run it. But I'd like to know if there's a simpler way to do it. It's not just about avoiding the compile step, etc. I'd also like to be able to look at the values of variables that I've set (in the "imported" shell file), perform additional operations on those variables, etc. Thanks a lot for any pointers. tiger -- Gerald Timothy Quimpo http://bopolissimus.blogspot.com gquimpo@REDACTED bopolissimus.lists@REDACTED Public Key: "gpg --keyserver pgp.mit.edu --recv-keys 672F4C78" Usually, a very large amount of work goes into the parsing step (Perl and C++ are probably the best examples of this-both are basically unparseable.) From rtrlists@REDACTED Mon Sep 21 10:38:16 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 21 Sep 2009 09:38:16 +0100 Subject: [erlang-questions] shell question - import commands from external file In-Reply-To: References: Message-ID: <6a3ae47e0909210138n59615019td99d8b5168f630ee@mail.gmail.com> On Mon, Sep 21, 2009 at 6:53 AM, Bopolissimus Platypus Jr < bopolissimus.lists@REDACTED> wrote: > Hi all, > > I have a newbie question. In the shell, is there a way to import > *shell* commands from > an external file? I'm often experimenting in the shell, e.g., writing > a 5 or 8 line fun, > trying out this or that function call, etc. > > Since there are often mistakes, I need to rewrite the fun a few times. > Instead of > having to retype (or look up in history) the several lines, I'd like > to write the fun > in a temporary file (e.g., ./tmp.erl) and then just load it into the > shell as if i'd > typed it in (e.g., in the postgresql shell I would do, "\i [filename"). > > > I don't think there is any mechanism that would allow you to evaluate text in a file that contains shell commands like f(), c(), etc. But maybe file:consult/1, file:eval/1 or file:eval/2 might be close enough to what you're looking for? Robby From kagato@REDACTED Mon Sep 21 11:01:26 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Mon, 21 Sep 2009 02:01:26 -0700 Subject: [erlang-questions] shell question - import commands from external file In-Reply-To: References: Message-ID: <7502B115-7431-4684-B6A5-39A569016842@souja.net> The ultra-lightweight solution is to use the c(module) command to compile the file (in this case, named module.erl in the current directory). One useful side-effect of using c() to compile via the shell is that it automatically loads the new code if compilation is successful. No muss, no fuss. To easily play with code, simply slap said code into a function in a test module (perhaps named test) and compile it this way. Then run the code from the shell with test:my_function(). This is also handy because working with records and parse_transforms (like QLC) is easy. If you find that you need to work with multiple modules this way (for example, developing server processes), you can also use make:all([load]). Note that without the load option, it will build but not reload the code. The only downside of these techniques is that you can accidentally kill running processes if you're not careful. Erlang will only keep two copies of the code for a module resident. There's the "current" version and the "old" version. If you update code and a process is still using the "old" version, then another update will forcibly kill that process. Caveat emptor, and all that. On Sep 20, 2009, at 10:53 PM, Bopolissimus Platypus Jr wrote: > Hi all, > > I have a newbie question. In the shell, is there a way to import > *shell* commands from > an external file? I'm often experimenting in the shell, e.g., writing > a 5 or 8 line fun, > trying out this or that function call, etc. > > Since there are often mistakes, I need to rewrite the fun a few times. > Instead of > having to retype (or look up in history) the several lines, I'd like > to write the fun > in a temporary file (e.g., ./tmp.erl) and then just load it into the > shell as if i'd > typed it in (e.g., in the postgresql shell I would do, "\i > [filename"). > > I realize that I could write the experimental code into a temporary > module, > compile that and run it. But I'd like to know if there's a simpler > way to > do it. It's not just about avoiding the compile step, etc. I'd also > like to be > able to look at the values of variables that I've set (in the > "imported" shell > file), perform additional operations on those variables, etc. > > Thanks a lot for any pointers. > > tiger > > -- > Gerald Timothy Quimpo http://bopolissimus.blogspot.com > gquimpo@REDACTED bopolissimus.lists@REDACTED > Public Key: "gpg --keyserver pgp.mit.edu --recv-keys 672F4C78" > > Usually, a very large amount of work goes into the > parsing step (Perl and C++ are probably the best > examples of this-both are basically unparseable.) > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From masklinn@REDACTED Mon Sep 21 11:02:59 2009 From: masklinn@REDACTED (Masklinn) Date: Mon, 21 Sep 2009 11:02:59 +0200 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: <603F5CF3-AD4B-4262-B4CD-5F096FC33EAE@cs.otago.ac.nz> References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB635EA.8070203@millstream.com> <1253460131.22238.51.camel@sredniczarny> <4AB65388.8050601@millstream.com> <603F5CF3-AD4B-4262-B4CD-5F096FC33EAE@cs.otago.ac.nz> Message-ID: On 21 Sep 2009, at 03:50 , Richard O'Keefe wrote: > > The IEEE 754 standard is small and tolerably clear; > there are drafts and summaries and review articles about it > all over the web. First and foremost being Goldberg's "What Every Computer Scientist Should Know About Floating-Point Arithmetic" (http://docs.sun.com/source/806-3568/ncg_goldberg.html ) which is often considered the most basic knowledge requirement before any discussion of IEEE-754 floats. From john.hughes@REDACTED Mon Sep 21 11:29:48 2009 From: john.hughes@REDACTED (John Hughes) Date: Mon, 21 Sep 2009 11:29:48 +0200 Subject: [erlang-questions] shell question - import commands from external file In-Reply-To: <6a3ae47e0909210138n59615019td99d8b5168f630ee@mail.gmail.com> References: <6a3ae47e0909210138n59615019td99d8b5168f630ee@mail.gmail.com> Message-ID: >> I have a newbie question. In the shell, is there a way to import >> *shell* commands from >> an external file? I'm often experimenting in the shell, e.g., writing >> a 5 or 8 line fun, >> trying out this or that function call, etc. >> >> Since there are often mistakes, I need to rewrite the fun a few times. >> Instead of >> having to retype (or look up in history) the several lines, I'd like >> to write the fun >> in a temporary file (e.g., ./tmp.erl) and then just load it into the >> shell as if i'd >> typed it in (e.g., in the postgresql shell I would do, "\i [filename"). >> >> >> > I don't think there is any mechanism that would allow you to evaluate text > in a file that contains shell commands like f(), c(), etc. > > But maybe file:consult/1, file:eval/1 or file:eval/2 might be close enough > to what you're looking for? > > Robby > Cut-and-paste? I generally prefer to put even temporary definitions in a file and compile them though, not least so I don't have to use the full-blown module:function syntax everywhere. John From ulf.wiger@REDACTED Mon Sep 21 12:38:42 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 21 Sep 2009 12:38:42 +0200 Subject: [erlang-questions] packaging erlang application for ubuntu In-Reply-To: <792758.38980.qm@web31806.mail.mud.yahoo.com> References: <792758.38980.qm@web31806.mail.mud.yahoo.com> Message-ID: <4AB757B2.30900@erlang-consulting.com> This is not an answer to your question, but just a thought. One problem with the OTP way of building boot scripts is that it gets tied to a specific version of Erlang (the erts version, the application versions, etc.). How about if one could specify a chain of boot scripts on the command line? erl -boot start_sasl -boot my_app The main complication I can see is the presence of the erlanrc instruction at the end of the 'basic' boot script. One could make sure that that can be delayed until all the boot scripts have been executed (e.g. having it in all, but ignoring it in all but the last script). A fun prototype. I sadly don't have the time... BR, Ulf W Pablo Platt wrote: > Hi, > > I have an erlang application that I want to distribute to clients using ubuntu. > I don't want to include the source code, only the compiled beam files. > The installation needs to be automatic, install all the dependencies and > create a boot script that will run the application as daemon on startup. > > Is reltools fits this task? > Does an application release includes the erlang runtime so copying this folder will be sufficient to run the application? > Installing the erlang ubuntu package separately might be a problem because the client might have > a different erlang version installed or the defaultubuntu package might be different then the version the application was compiled against. > > What ubuntu packages are required to run a minimal erlang installation? > > Can someone provide an example of a simple erlang application packaged for ubuntu as a reference? > > Thanks > > > -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From christophe.romain@REDACTED Mon Sep 21 13:03:35 2009 From: christophe.romain@REDACTED (Christophe Romain) Date: Mon, 21 Sep 2009 13:03:35 +0200 Subject: [erlang-questions] packaging erlang application for ubuntu In-Reply-To: <792758.38980.qm@web31806.mail.mud.yahoo.com> References: <792758.38980.qm@web31806.mail.mud.yahoo.com> Message-ID: <20090921110335.GJ4155@localhost> >run a minimal erlang installation? did you looked at CEAN ? http://cean.process-one.net minimal erlang installation is about 4Mb tarball From pablo.platt@REDACTED Mon Sep 21 15:44:43 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 21 Sep 2009 06:44:43 -0700 (PDT) Subject: [erlang-questions] packaging erlang application for ubuntu In-Reply-To: <20090921110335.GJ4155@localhost> References: <792758.38980.qm@web31806.mail.mud.yahoo.com> <20090921110335.GJ4155@localhost> Message-ID: <889393.41725.qm@web31809.mail.mud.yahoo.com> ________________________________ From: Christophe Romain To: Pablo Platt Cc: Erlang Questions Sent: Monday, September 21, 2009 2:03:35 PM Subject: Re: [erlang-questions] packaging erlang application for ubuntu >run a minimal erlang installation? did you looked at CEAN ? http://cean.process-one.net minimal erlang installation is about 4Mb tarball Can I use it to package an erlang application that will be portable? Is there a folder I can copy that is self consistent? Will it run on other ubuntu release? (Intrepid, Jautnty...) Do I have to release the source code of the application? From christophe.romain@REDACTED Mon Sep 21 17:24:21 2009 From: christophe.romain@REDACTED (Christophe Romain) Date: Mon, 21 Sep 2009 17:24:21 +0200 Subject: [erlang-questions] packaging erlang application for ubuntu In-Reply-To: <889393.41725.qm@web31809.mail.mud.yahoo.com> References: <792758.38980.qm@web31806.mail.mud.yahoo.com> <20090921110335.GJ4155@localhost> <889393.41725.qm@web31809.mail.mud.yahoo.com> Message-ID: <20090921152421.GQ4155@localhost> >Can I use it to package an erlang application that will be portable? that all depends what you call portable. using 'repos' distribution you get a minimal erlang you can copy on usb key that works on linux, mac and windows. if you need your package to be copied on linux cluster with, let say amd64, i386, and powerpc; then you can concate the cean base for those 3 systems and it will work everywhere can handle only one tarball/package that run on all systems you need. by now, 19 platforms are supported. >Is there a folder I can copy that is self consistent? yes, just download the cean base archive(s) you need you can even move the installer directory without breaking things. >Will it run on other ubuntu release? (Intrepid, Jautnty...) yes, it should at least it runs on debian sarge, etch, lenny. >Do I have to release the source code of the application? CEAN is standard erlang, with some minor patchs to allow portability as we stand in the 'repos' distribution. licence is EPL. if you need custom packaging, contact me privately. From christophe.romain@REDACTED Mon Sep 21 18:20:11 2009 From: christophe.romain@REDACTED (Christophe Romain) Date: Mon, 21 Sep 2009 18:20:11 +0200 Subject: [erlang-questions] packaging erlang application for ubuntu In-Reply-To: <4AB757B2.30900@erlang-consulting.com> References: <792758.38980.qm@web31806.mail.mud.yahoo.com> <4AB757B2.30900@erlang-consulting.com> Message-ID: <20090921162011.GU4155@localhost> > How about if one could specify a chain of boot scripts on > the command line? > erl -boot start_sasl -boot my_app +1 may this be simpler: erl -boot start_sasl,my_app From fritchie@REDACTED Mon Sep 21 19:30:58 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 21 Sep 2009 12:30:58 -0500 Subject: [erlang-questions] shell question - import commands from external file In-Reply-To: Message of "Mon, 21 Sep 2009 11:29:48 +0200." Message-ID: <44446.1253554258@snookles.snookles.com> John Hughes wrote: jh> Cut-and-paste? Actually, I do that quite a bit. It isn't a tremendously pleasing thing to do, but it does have a number of benefits that can add up to something worthwhile. Not always, but enough that I keep doing it. During initial development, I have 1-line or 2-line test cases that I need use in the shell. I don't always follow test-driven-design doctrine exactly, especially when the API of the function(s)/module that I'm working on is still in flux. I keep yet another "xterm" window open with either: * cat > /dev/null * a text editor (for more permanent notes). If I have a command at the Erlang CLI that I believe I'll need again, I cut-and-paste from the CLI to the cat/editor window. The text editor flavor ends up being useful because I can save the stuff to a file. (Power failures have a tendency to screw up "xterm" and "cat", go figure.) I have such scribble notes from over 12 months ago on one project. It forms extra history and documentation of how I was using the functions back then, which has been a big help for writing extra documentation now.(*) On another method, a colleague of mine uses "rlwrap -a erl [args...]" and swears that it's the best thing since the ball point pen.(**) -Scott (*) Never mind that I should have written the documentation at the same time I wrote the code. (**) Or sliced bread, I forget exactly. See http://freshmeat.net/projects/rlwrap/ From prikrutil@REDACTED Mon Sep 21 19:42:44 2009 From: prikrutil@REDACTED (Sergey Samokhin) Date: Mon, 21 Sep 2009 21:42:44 +0400 Subject: Is it safe to unlink from gen_fsm on the fly? Message-ID: Hello. I'm trying to make my wrapper around epgsql more tolerant of network outages. When my gen_server detects that connection has been lost, a special process is created to make a connection in an asynchronous manner (I want to handle calls to the server, so I can't block in reconnect function). When connection is established, pid is sent back to the gen_server and 'connector' process is unlinked from the pid. Are there any pitfalls of unlinking from working gen_fsm? Does gen_fsm store parent id anywhere into the state? I haven't found using of the parent process's id in the epgsql sources. P.S. I see another way to implement smoothly reconnects without blocking gen_server: by using erlang:start_timer/3 in the gen_server. I haven't decided yet which way is better =) Thanks. -- Sergey Samokhin From kagato@REDACTED Mon Sep 21 21:43:38 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Mon, 21 Sep 2009 12:43:38 -0700 Subject: [erlang-questions] Is it safe to unlink from gen_fsm on the fly? In-Reply-To: References: Message-ID: <2F152643-CA6E-4C08-8FD0-6A7D41EE6E1C@souja.net> Unlinking is bad mostly because you can leak processes. One of the advantages of the supervision tree is that you can guarantee that all children get killed when restarts happen. I would recommend having the reconnection process always running, and have the gen_server send it an async message every time the connection is down and there is a pending request, maybe on a timeout. On Sep 21, 2009, at 10:42 AM, Sergey Samokhin wrote: > Hello. > > I'm trying to make my wrapper around epgsql more tolerant of network > outages. When my gen_server detects that connection has been lost, a > special process is created to make a connection in an asynchronous > manner (I want to handle calls to the server, so I can't block in > reconnect function). When connection is established, pid is sent back > to the gen_server and 'connector' process is unlinked from the pid. > > Are there any pitfalls of unlinking from working gen_fsm? Does gen_fsm > store parent id anywhere into the state? > > I haven't found using of the parent process's id in the epgsql > sources. > > P.S. I see another way to implement smoothly reconnects without > blocking gen_server: by using erlang:start_timer/3 in the gen_server. > I haven't decided yet which way is better =) > > Thanks. > > -- > Sergey Samokhin > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From bopolissimus.lists@REDACTED Mon Sep 21 23:49:36 2009 From: bopolissimus.lists@REDACTED (Bopolissimus Platypus Jr) Date: Tue, 22 Sep 2009 09:49:36 +1200 Subject: [erlang-questions] shell question - import commands from external file In-Reply-To: <7502B115-7431-4684-B6A5-39A569016842@souja.net> References: <7502B115-7431-4684-B6A5-39A569016842@souja.net> Message-ID: On Mon, Sep 21, 2009 at 9:01 PM, Jayson Vantuyl wrote: > This is also handy because working with records and parse_transforms (like QLC) is > easy. Thanks Jason, Robert, John, Scott. I'll test all those suggestions out and choose whichever is most convenient for the task at hand (or some combination of them). tiger -- Gerald Timothy Quimpo http://bopolissimus.blogspot.com gquimpo@REDACTED bopolissimus.lists@REDACTED Public Key: "gpg --keyserver pgp.mit.edu --recv-keys 672F4C78" Usually, a very large amount of work goes into the parsing step (Perl and C++ are probably the best examples of this-both are basically unparseable.) From pablo.platt@REDACTED Mon Sep 21 23:58:48 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 21 Sep 2009 14:58:48 -0700 (PDT) Subject: using the target_system module Message-ID: <892026.55458.qm@web31813.mail.mud.yahoo.com> Hi, I'm using the target_system module from http://erlang.org/doc/system_principles/create_target.html The OTP application I'm using for tests is simpletest from http://eaglestreet.wordpress.com/2008/08/22/releasing_erlang_otp_code/ The target_system module is really good. Is it officially supported and will be included in a future release or just an example? I have several remarks: 1. It doesn't work with the Ubuntu Jaunty erlang package. The doc should indicate that it only works with R13B and above. 2. I had to manually change the permissions of some of the scripts under the bin directory to executable. Is it possible that the module will take care of that? 3. It took me some time to understand that bin/start can't run without sys.config file under the releases dir. Can bin/start throw an error if it can't find the sys.config file? 4. The module should create a log dir. 5. In my case the create method built relative path. I'm not sure if this is intended or not but it might cause issues. 6. It will be helpful to have a tar file online with a test application that works with this module for easy learning. Thanks From bgustavsson@REDACTED Tue Sep 22 16:07:09 2009 From: bgustavsson@REDACTED (Bjorn Gustavsson) Date: Tue, 22 Sep 2009 16:07:09 +0200 Subject: Erlang/OTP R13B02 has been released Message-ID: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> Bug fix release : otp_src_R13B02 Build date : 2009-09-21 This is R13B02, the second maintenance release for the R13B major release. You can find the README file for the release at http://www.erlang.org/download/otp_src_R13B02.readme The source distribution and binary distribution for Windows can be downloaded from http://www.erlang.org/download/otp_src_R13B02.tar.gz http://www.erlang.org/download/otp_win32_R13B02.exe The distribution can also be downloaded using the BitTorrent protocol. Use the following torrent files to download the source distribution and binary distribution for Windows: http://www.erlang.org/download/otp_src_R13B02.tar.gz.torrent http://www.erlang.org/download/otp_win32_R13B02.exe.torrent Note: To unpack the TAR archive you need a GNU TAR compatible program. For installation instructions please read the README file that is part of the distribution. The on-line documentation can be found at: http://www.erlang.org/doc/ You can also download the complete HTML documentation or the Unix manual files http://www.erlang.org/download/otp_doc_html_R13B02.tar.gz http://www.erlang.org/download/otp_doc_man_R13B02.tar.gz We also want to thank those that sent us patches, suggestions and bug reports, The OTP Team -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From rapsey@REDACTED Tue Sep 22 16:33:44 2009 From: rapsey@REDACTED (Rapsey) Date: Tue, 22 Sep 2009 16:33:44 +0200 Subject: [erlang-questions] Erlang/OTP R13B02 has been released In-Reply-To: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> Message-ID: <97619b170909220733m460862c7l5d71e36ea85c47f5@mail.gmail.com> Build fails on OS X server v10.5.8: /usr/include/wx-2.8/wx/fontdlg.h: In constructor ?wxFontDialogBase::wxFontDialogBase(wxWindow*, const wxFontData*)?: /usr/include/wx-2.8/wx/fontdlg.h:67: warning: ?__base_ctor ? is deprecated (declared at /usr/include/wx-2.8/wx/fontdlg.h:66) gen/wxe_events.cpp: In function ?void initEventTable()?: gen/wxe_events.cpp:277: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_DOWN? was not declared in this scope gen/wxe_events.cpp:278: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_UP? was not declared in this scope gen/wxe_events.cpp:279: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_DOWN? was not declared in this scope gen/wxe_events.cpp:280: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_UP? was not declared in this scope gen/wxe_events.cpp:281: error: ?wxEVT_COMMAND_AUINOTEBOOK_PAGE_CLOSED? was not declared in this scope gen/wxe_events.cpp:282: error: ?wxEVT_COMMAND_AUINOTEBOOK_DRAG_DONE? was not declared in this scope gen/wxe_events.cpp:283: error: ?wxEVT_COMMAND_AUINOTEBOOK_BG_DCLICK? was not declared in this scope make[3]: *** [i386-apple-darwin9.8.0/wxe_events.o] Error 1 Sergej On Tue, Sep 22, 2009 at 4:07 PM, Bjorn Gustavsson wrote: > Bug fix release : otp_src_R13B02 > Build date : 2009-09-21 > > This is R13B02, the second maintenance release for the R13B major release. > > You can find the README file for the release at > > http://www.erlang.org/download/otp_src_R13B02.readme > > The source distribution and binary distribution for Windows can be > downloaded from > > http://www.erlang.org/download/otp_src_R13B02.tar.gz > http://www.erlang.org/download/otp_win32_R13B02.exe > > The distribution can also be downloaded using the BitTorrent > protocol. Use the following torrent files to download the source > distribution and binary distribution for Windows: > > http://www.erlang.org/download/otp_src_R13B02.tar.gz.torrent > http://www.erlang.org/download/otp_win32_R13B02.exe.torrent > > Note: To unpack the TAR archive you need a GNU TAR compatible program. > > For installation instructions please read the README file that is part > of the distribution. > > The on-line documentation can be found at: http://www.erlang.org/doc/ > You can also download the complete HTML documentation or the Unix manual > files > > http://www.erlang.org/download/otp_doc_html_R13B02.tar.gz > http://www.erlang.org/download/otp_doc_man_R13B02.tar.gz > > We also want to thank those that sent us patches, suggestions and bug > reports, > > The OTP Team > > > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From MARTIN.LOGAN@REDACTED Tue Sep 22 16:53:04 2009 From: MARTIN.LOGAN@REDACTED (Logan, Martin) Date: Tue, 22 Sep 2009 09:53:04 -0500 Subject: [erlang-questions] using the target_system module In-Reply-To: <892026.55458.qm@web31813.mail.mud.yahoo.com> References: <892026.55458.qm@web31813.mail.mud.yahoo.com> Message-ID: <77C1226CCC2F7C41B022662F4775DABC56FCE2C968@EGEXCMB01.oww.root.lcl> I don't actually like the target system library. It prescribes names and such that are not exactly natural in my opinion. We wrote the Sinan build system which when combined with the faxien package manager can accomplish this same functionality in a cleaner way I think. It will also work with lower versions of Erlang though I would recommend using faxien to just install a greater version of Erlang on your system. To get the erl shell and erlc installed once you have faxien installed just run faxien install-release erl You can find them at erlware.org. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Pablo Platt Sent: Monday, September 21, 2009 4:59 PM To: Erlang Questions Subject: [erlang-questions] using the target_system module Hi, I'm using the target_system module from http://erlang.org/doc/system_principles/create_target.html The OTP application I'm using for tests is simpletest from http://eaglestreet.wordpress.com/2008/08/22/releasing_erlang_otp_code/ The target_system module is really good. Is it officially supported and will be included in a future release or just an example? I have several remarks: 1. It doesn't work with the Ubuntu Jaunty erlang package. The doc should indicate that it only works with R13B and above. 2. I had to manually change the permissions of some of the scripts under the bin directory to executable. Is it possible that the module will take care of that? 3. It took me some time to understand that bin/start can't run without sys.config file under the releases dir. Can bin/start throw an error if it can't find the sys.config file? 4. The module should create a log dir. 5. In my case the create method built relative path. I'm not sure if this is intended or not but it might cause issues. 6. It will be helpful to have a tar file online with a test application that works with this module for easy learning. Thanks From james.hague@REDACTED Tue Sep 22 17:29:52 2009 From: james.hague@REDACTED (James Hague) Date: Tue, 22 Sep 2009 10:29:52 -0500 Subject: [erlang-questions] Erlang/OTP R13B02 has been released In-Reply-To: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> Message-ID: Maybe I missed it in the readme, but the "double install" problem under Windows, which happened if you selected "don't create shortcuts," appears to be fixed. Yay! From MARTIN.LOGAN@REDACTED Tue Sep 22 17:27:21 2009 From: MARTIN.LOGAN@REDACTED (Logan, Martin) Date: Tue, 22 Sep 2009 10:27:21 -0500 Subject: [erlang-questions] packaging erlang application for ubuntu In-Reply-To: <792758.38980.qm@web31806.mail.mud.yahoo.com> References: <792758.38980.qm@web31806.mail.mud.yahoo.com> Message-ID: <77C1226CCC2F7C41B022662F4775DABC56FCE2C96A@EGEXCMB01.oww.root.lcl> Faxien and Sinan really fit the bill. You can create a package and distribute it. If they run faxien also they can install it that way and faxien will take care of putting the correctly compiled version of erts on their system whether they have it previously or not. If they don't run faxien you can create a package that they can simply untar on their local machine and work with. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Pablo Platt Sent: Sunday, September 20, 2009 7:13 PM To: Erlang Questions Subject: [erlang-questions] packaging erlang application for ubuntu Hi, I have an erlang application that I want to distribute to clients using ubuntu. I don't want to include the source code, only the compiled beam files. The installation needs to be automatic, install all the dependencies and create a boot script that will run the application as daemon on startup. Is reltools fits this task? Does an application release includes the erlang runtime so copying this folder will be sufficient to run the application? Installing the erlang ubuntu package separately might be a problem because the client might have a different erlang version installed or the defaultubuntu package might be different then the version the application was compiled against. What ubuntu packages are required to run a minimal erlang installation? Can someone provide an example of a simple erlang application packaged for ubuntu as a reference? Thanks From mihai@REDACTED Tue Sep 22 17:41:12 2009 From: mihai@REDACTED (Mihai Balea) Date: Tue, 22 Sep 2009 11:41:12 -0400 Subject: [erlang-questions] Erlang/OTP R13B02 has been released In-Reply-To: <97619b170909220733m460862c7l5d71e36ea85c47f5@mail.gmail.com> References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> <97619b170909220733m460862c7l5d71e36ea85c47f5@mail.gmail.com> Message-ID: On Sep 22, 2009, at 10:33 AM, Rapsey wrote: > Build fails on OS X server v10.5.8: > > /usr/include/wx-2.8/wx/fontdlg.h: In constructor > ?wxFontDialogBase::wxFontDialogBase(wxWindow*, const wxFontData*)?: > /usr/include/wx-2.8/wx/fontdlg.h:67: warning: ?__base_ctor ? is > deprecated > (declared at /usr/include/wx-2.8/wx/fontdlg.h:66) > gen/wxe_events.cpp: In function ?void initEventTable()?: > gen/wxe_events.cpp:277: error: > ?wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_DOWN? > was not declared in this scope > gen/wxe_events.cpp:278: error: > ?wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_UP? was > not declared in this scope > gen/wxe_events.cpp:279: error: > ?wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_DOWN? > was not declared in this scope > gen/wxe_events.cpp:280: error: > ?wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_UP? was > not declared in this scope > gen/wxe_events.cpp:281: error: > ?wxEVT_COMMAND_AUINOTEBOOK_PAGE_CLOSED? was > not declared in this scope > gen/wxe_events.cpp:282: error: ?wxEVT_COMMAND_AUINOTEBOOK_DRAG_DONE? > was not > declared in this scope > gen/wxe_events.cpp:283: error: ?wxEVT_COMMAND_AUINOTEBOOK_BG_DCLICK? > was not > declared in this scope > make[3]: *** [i386-apple-darwin9.8.0/wxe_events.o] Error 1 Same here, OSX desktop 10.5.8. Mihai From hakan@REDACTED Tue Sep 22 18:39:11 2009 From: hakan@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Tue, 22 Sep 2009 18:39:11 +0200 Subject: Apply this patch before using Reltool in R13B02 Message-ID: <922d05850909220939xa06dcaerc420b183cf0c760a@mail.gmail.com> Oops!!! In the new release of Reltool (0.5) there is a severe bug that may cause your source files to be deleted. Please, apply the following patch before you use this version of Reltool: http://www.erlang.org/download/patches/otp_src_R13B02_OTP-8199.patch The bug is triggered when you generate a target system with archive files and your library directories are given with absolute paths. (Library directories are given with the ERL_LIBS environment variable or with the lib_dirs configuration parameter.) /H?kan --- H?kan Mattsson (uabhams) Erlang/OTP, Ericsson AB On Tue, Sep 22, 2009 at 4:07 PM, Bjorn Gustavsson wrote: > Bug fix release : otp_src_R13B02 > Build date ? ? ?: 2009-09-21 > > This is R13B02, the second maintenance release for the R13B major release. > > You can find the README file for the release at > > ?http://www.erlang.org/download/otp_src_R13B02.readme > > The source distribution and binary distribution for Windows can be > downloaded from > > ?http://www.erlang.org/download/otp_src_R13B02.tar.gz > ?http://www.erlang.org/download/otp_win32_R13B02.exe > > The distribution can also be downloaded using the BitTorrent > protocol. Use the following torrent files to download the source > distribution and binary distribution for Windows: > > ?http://www.erlang.org/download/otp_src_R13B02.tar.gz.torrent > ?http://www.erlang.org/download/otp_win32_R13B02.exe.torrent > > Note: To unpack the TAR archive you need a GNU TAR compatible program. > > For installation instructions please read the README file that is part > of the distribution. > > The on-line documentation can be found at: http://www.erlang.org/doc/ > You can also download the complete HTML documentation or the Unix manual files > > ?http://www.erlang.org/download/otp_doc_html_R13B02.tar.gz > ?http://www.erlang.org/download/otp_doc_man_R13B02.tar.gz > > We also want to thank those that sent us patches, suggestions and bug > reports, > > The OTP Team > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From twoggle@REDACTED Tue Sep 22 19:01:24 2009 From: twoggle@REDACTED (Tim Fletcher) Date: Tue, 22 Sep 2009 10:01:24 -0700 (PDT) Subject: Erlang/OTP R13B02 has been released In-Reply-To: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> Message-ID: > The distribution can also be downloaded using the BitTorrent > protocol. Use the following torrent files to download the source > distribution and binary distribution for Windows: > > ?http://www.erlang.org/download/otp_src_R13B02.tar.gz.torrent > ?http://www.erlang.org/download/otp_win32_R13B02.exe.torrent These don't appear to exist, and the torrent links at the top of http://www.erlang.org/download.html also don't work. From kevin@REDACTED Tue Sep 22 18:47:18 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Tue, 22 Sep 2009 12:47:18 -0400 Subject: [erlang-questions] Apply this patch before using Reltool in R13B02 In-Reply-To: <922d05850909220939xa06dcaerc420b183cf0c760a@mail.gmail.com> References: <922d05850909220939xa06dcaerc420b183cf0c760a@mail.gmail.com> Message-ID: <2E9FDD18-B1C6-4EA2-A600-E972E584D76C@hypotheticalabs.com> Any chance we can get a re-spin of the R13B02 release with this patch included? --Kevin On Sep 22, 2009, at 12:39 PM, H?kan Mattsson wrote: > Oops!!! > > In the new release of Reltool (0.5) there is a severe bug that may > cause your source files to be deleted. Please, apply the following > patch before you use this version of Reltool: > > http://www.erlang.org/download/patches/otp_src_R13B02_OTP-8199.patch > > The bug is triggered when you generate a target system with archive > files and your library directories are given with absolute paths. > (Library > directories are given with the ERL_LIBS environment variable or with > the > lib_dirs configuration parameter.) > > /H?kan > --- > H?kan Mattsson (uabhams) > Erlang/OTP, Ericsson AB > > On Tue, Sep 22, 2009 at 4:07 PM, Bjorn Gustavsson > wrote: >> Bug fix release : otp_src_R13B02 >> Build date : 2009-09-21 >> >> This is R13B02, the second maintenance release for the R13B major >> release. >> >> You can find the README file for the release at >> >> http://www.erlang.org/download/otp_src_R13B02.readme >> >> The source distribution and binary distribution for Windows can be >> downloaded from >> >> http://www.erlang.org/download/otp_src_R13B02.tar.gz >> http://www.erlang.org/download/otp_win32_R13B02.exe >> >> The distribution can also be downloaded using the BitTorrent >> protocol. Use the following torrent files to download the source >> distribution and binary distribution for Windows: >> >> http://www.erlang.org/download/otp_src_R13B02.tar.gz.torrent >> http://www.erlang.org/download/otp_win32_R13B02.exe.torrent >> >> Note: To unpack the TAR archive you need a GNU TAR compatible >> program. >> >> For installation instructions please read the README file that is >> part >> of the distribution. >> >> The on-line documentation can be found at: http://www.erlang.org/doc/ >> You can also download the complete HTML documentation or the Unix >> manual files >> >> http://www.erlang.org/download/otp_doc_html_R13B02.tar.gz >> http://www.erlang.org/download/otp_doc_man_R13B02.tar.gz >> >> We also want to thank those that sent us patches, suggestions and bug >> reports, >> >> The OTP Team >> >> -- >> Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From joe@REDACTED Tue Sep 22 19:16:21 2009 From: joe@REDACTED (Joe Williams) Date: Tue, 22 Sep 2009 10:16:21 -0700 Subject: [erlang-questions] Erlang/OTP R13B02 has been released In-Reply-To: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> Message-ID: <20090922101621.10bab2fc@der-dieb> I seem to be having an issue building on linux (ubuntu 9.04 x86_64). erlc -W +debug_info +warn_obsolete_guard -I/home/zeusfaber/downloads/otp_src_R13B02/lib/stdlib/include -o../ebin leex.erl ./leex.erl:none: internal error in core_module; crash reason: {{case_clause,{{var,1273,'C'},[]}}, [{v3_core,'-constant_bin_1/1-anonymous-0-',2}, {eval_bits,eval_field,3}, {eval_bits,expr_grp,4}, {v3_core,constant_bin_1,1}, {v3_core,constant_bin,1}, {v3_core,expr_bin,3}, {v3_core,expr,2}, {v3_core,novars,2}]} Anyone else getting this? Thanks. -Joe On Tue, 22 Sep 2009 16:07:09 +0200 Bjorn Gustavsson wrote: > Bug fix release : otp_src_R13B02 > Build date : 2009-09-21 > > This is R13B02, the second maintenance release for the R13B major > release. > > You can find the README file for the release at > > http://www.erlang.org/download/otp_src_R13B02.readme > > The source distribution and binary distribution for Windows can be > downloaded from > > http://www.erlang.org/download/otp_src_R13B02.tar.gz > http://www.erlang.org/download/otp_win32_R13B02.exe > > The distribution can also be downloaded using the BitTorrent > protocol. Use the following torrent files to download the source > distribution and binary distribution for Windows: > > http://www.erlang.org/download/otp_src_R13B02.tar.gz.torrent > http://www.erlang.org/download/otp_win32_R13B02.exe.torrent > > Note: To unpack the TAR archive you need a GNU TAR compatible program. > > For installation instructions please read the README file that is part > of the distribution. > > The on-line documentation can be found at: http://www.erlang.org/doc/ > You can also download the complete HTML documentation or the Unix > manual files > > http://www.erlang.org/download/otp_doc_html_R13B02.tar.gz > http://www.erlang.org/download/otp_doc_man_R13B02.tar.gz > > We also want to thank those that sent us patches, suggestions and bug > reports, > > The OTP Team > > > -- Name: Joseph A. Williams Email: joe@REDACTED Blog: http://www.joeandmotorboat.com/ From francesco@REDACTED Tue Sep 22 19:35:55 2009 From: francesco@REDACTED (Francesco Cesarini (Erlang Training and Consulting)) Date: Tue, 22 Sep 2009 18:35:55 +0100 Subject: Registration for the 2009 Erlang User Conference is Now Open! Message-ID: <4AB90AFB.2030305@erlang-consulting.com> Hi All, A note to say that we have now opened the registration for the 2009 Erlang User Conference. It sold out in 2007 and 2008... We hope the new facilities can accommodate everyone who wants to come this year. Register by visiting the conference site here: http://www.erlang-factory.com/conference/ErlangUserConference2009 We are adding new speakers on a daily basis, do come by often or follow @ErlangFactory on twitter or subscribe to the RSS feed on the site. A bit of history on the conference. Until recently, Ericsson used to host it in their conference facilities. With the help of sponsors, we were able to keep the event free of charge. As a result, the conference sold out in both 2007 and 2008, weeks before the program was published. To avoid a sold out event and cater for future expansion, we have rented the Astoria cinema in central Stockholm doubling the capacity. As a result, we have to charge a small fee for the conference. Considering that speakers still pay for their own travel and accommodation costs, together with a strong backing of generous sponsors, we've been able to keep the early registration costs under 999SEK (including VAT). This includes proceedings, breakfast, lunch and an evening reception / dinner. We are pulling every string we can to keep the costs low while providing a top event. If anyone wants to help and sponsor the conference and support Erlang, please drop me a line. Tutorials on the Friday are open to everyone on a first come first serve basis, irrespective of if you attend the conference or not. Hope to see you there! Francesco -- http://www.erlang-consulting.com From mark.peleus@REDACTED Tue Sep 22 19:46:11 2009 From: mark.peleus@REDACTED (mark peleus) Date: Tue, 22 Sep 2009 20:46:11 +0300 Subject: [erlang-questions] using the target_system module In-Reply-To: <77C1226CCC2F7C41B022662F4775DABC56FCE2C968@EGEXCMB01.oww.root.lcl> References: <892026.55458.qm@web31813.mail.mud.yahoo.com> <77C1226CCC2F7C41B022662F4775DABC56FCE2C968@EGEXCMB01.oww.root.lcl> Message-ID: <599dd3e0909221046x4639d536g39e562d99ea0506d@mail.gmail.com> I'll have a deeper look at Sinan but I actually like the way target_system work. With a single call it generates everything needed to make my erlang application portable. That's why I wanted to try and improve target_system. I like the way Sinan build an application skeleton but I rather have that in the erlang release instead of learning to work with another tool. Thanks On Tue, Sep 22, 2009 at 5:53 PM, Logan, Martin wrote: > I don't actually like the target system library. It prescribes names and > such that are not exactly natural in my opinion. We wrote the Sinan build > system which when combined with the faxien package manager can accomplish > this same functionality in a cleaner way I think. It will also work with > lower versions of Erlang though I would recommend using faxien to just > install a greater version of Erlang on your system. To get the erl shell and > erlc installed once you have faxien installed just run > > faxien install-release erl > > You can find them at erlware.org. > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Pablo Platt > Sent: Monday, September 21, 2009 4:59 PM > To: Erlang Questions > Subject: [erlang-questions] using the target_system module > > Hi, > > I'm using the target_system module from > http://erlang.org/doc/system_principles/create_target.html > The OTP application I'm using for tests is simpletest from > http://eaglestreet.wordpress.com/2008/08/22/releasing_erlang_otp_code/ > > The target_system module is really good. > Is it officially supported and will be included in a future release or just > an example? > > I have several remarks: > > 1. It doesn't work with the Ubuntu Jaunty erlang package. The doc should > indicate that it only works with R13B and above. > 2. I had to manually change the permissions of some of the scripts under > the bin directory to executable. > Is it possible that the module will take care of that? > 3. It took me some time to understand that bin/start can't run without > sys.config file under the releases dir. > Can bin/start throw an error if it can't find the sys.config file? > 4. The module should create a log dir. > 5. In my case the create method built relative path. I'm not sure if this > is intended or not but it might cause issues. > 6. It will be helpful to have a tar file online with a test application > that works with this module for easy learning. > > Thanks > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From prikrutil@REDACTED Tue Sep 22 19:52:51 2009 From: prikrutil@REDACTED (Sergey Samokhin) Date: Tue, 22 Sep 2009 21:52:51 +0400 Subject: [erlang-questions] Is it safe to unlink from gen_fsm on the fly? In-Reply-To: <2F152643-CA6E-4C08-8FD0-6A7D41EE6E1C@souja.net> References: <2F152643-CA6E-4C08-8FD0-6A7D41EE6E1C@souja.net> Message-ID: Hello, Jayson! Thanks for your letter! It was very helpful. On Mon, Sep 21, 2009 at 11:43 PM, Jayson Vantuyl wrote: > Unlinking is bad mostly because you can leak processes. ?One of the > advantages of the supervision tree is that you can guarantee that all > children get killed when restarts happen. I think it's possible to pass the connection in an (almost) safe manner having at any time at least one process from the supervision tree linked to the driver. There are three functional process: gen_server which is where SQL-requests are sent to, connector and epgsql driver. Here is (working but ugly) algorithm showing how to transfer the connection from the connector to the gen_server in such a way: 1. spawn_link connector process from gen_server 2. Do epgsql:connect() in connector 3. Send the connection from connector to gen_server 4. Link gen_server to the connection 5. Let connector know that gen_server is succesfully linked to the driver 6. Unlink connector from the driver and stop it The only thing I don't like about it except for complexity is that there may be some problems if PostgreSQL will crash between say 4 and 5 steps. > I would recommend having the reconnection process always running, and have > the gen_server send it an async message every time the connection is down > and there is a pending request, maybe on a timeout. Having the reconnection process always running is a brilliant idea that I missed while thinking about the problem! Thanks a lot for pointing it out! There is no need in sending async messages 'reconnect' from gen_server, because connector is always able to detect problems by itself (by catching exit signals coming from the epgsql driver) and send the connection to the gen_server asynchronously. -- Sergey Samokhin From kagato@REDACTED Tue Sep 22 20:34:53 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Tue, 22 Sep 2009 11:34:53 -0700 Subject: [erlang-questions] Is it safe to unlink from gen_fsm on the fly? In-Reply-To: References: <2F152643-CA6E-4C08-8FD0-6A7D41EE6E1C@souja.net> Message-ID: > I think it's possible to pass the connection in an (almost) safe > manner having at any time at least one process from the supervision > tree linked to the driver. If you have the reconnection process always running, do you need to spawn it and pass it around? > There are three functional process: gen_server which is where > SQL-requests are sent to, connector and epgsql driver. Here is > (working but ugly) algorithm showing how to transfer the connection > from the connector to the gen_server in such a way: > > 1. spawn_link connector process from gen_server > 2. Do epgsql:connect() in connector > 3. Send the connection from connector to gen_server > 4. Link gen_server to the connection > 5. Let connector know that gen_server is succesfully linked to the > driver > 6. Unlink connector from the driver and stop it > > The only thing I don't like about it except for complexity is that > there may be some problems if PostgreSQL will crash between say 4 and > 5 steps. The real danger isn't between 4 and 5 (i.e. what if we're waiting for the unlink request and it never comes). The real danger is everywhere from 4 to 6, since a crash in the remote server will kill the linked process, which will kill our gen_server. Since they're all linked, the whole thing can go off like a bomb. > Having the reconnection process always running is a brilliant idea > that I missed while thinking about the problem! Thanks a lot for > pointing it out! > > There is no need in sending async messages 'reconnect' from > gen_server, because connector is always able to detect problems by > itself (by catching exit signals coming from the epgsql driver) and > send the connection to the gen_server asynchronously. That said, you might be able to invert the scenario. What if it was a gen_event? They you could just throw off a synchronous event when you have a disconnection, and it could catch it to reconnect. This also has the advantage that it neatly packages into the application, as it's just a simple addition to the top_level supervisor. -- Jayson Vantuyl kagato@REDACTED From paul-trapexit@REDACTED Tue Sep 22 20:38:36 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Tue, 22 Sep 2009 11:38:36 -0700 (PDT) Subject: os:putenv/2 not helping with erl_ddll:load_driver/2 Message-ID: I'm encountering a difference between os:putenv/2 and setting the environment variable prior to invoking Erlang. I'm trying to pick up a library that the os puts in a funny place, which my linked-in driver depends upon. Manipulating the environment variable inside Erlang doesn't have the desired effect. Does anybody have a workaround? I don't want to have to restart the emulator to install this software. Thanks, -- p ----------- % erl -pa ../src Erlang (BEAM) emulator version 5.6.5 [source] [async-threads:0] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> os:getenv ("LD_LIBRARY_PATH"). "/home/pmineiro/lib:/usr/local/lib" 2> os:putenv ("LD_LIBRARY_PATH", "/usr/lib/lp_solve:/home/pmineiro/lib:/usr/local/lib"). true 3> os:getenv ("LD_LIBRARY_PATH"). "/usr/lib/lp_solve:/home/pmineiro/lib:/usr/local/lib" 4> erl_ddll:load_driver ("/usr/lib/", "liblpsolveerldrv"). {error,{open_error,-10}} 5> erl_ddll:format_error ({ open_error, -10 }). "liblpsolve55.so: cannot open shared object file: No such file or directory" 6> User switch command --> q % LD_LIBRARY_PATH="/usr/lib/lp_solve:/home/pmineiro/lib:/usr/local/lib" erl -pa ../src Erlang (BEAM) emulator version 5.6.5 [source] [async-threads:0] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> os:getenv ("LD_LIBRARY_PATH"). "/usr/lib/lp_solve:/home/pmineiro/lib:/usr/local/lib" 2> erl_ddll:load_driver ("/usr/lib/", "liblpsolveerldrv"). ok 3> User switch command --> q ----------- From klas.johansson@REDACTED Tue Sep 22 21:14:33 2009 From: klas.johansson@REDACTED (Klas Johansson) Date: Tue, 22 Sep 2009 21:14:33 +0200 Subject: [erlang-questions] shell question - import commands from external file In-Reply-To: References: Message-ID: Hi, > I have a newbie question. In the shell, is there a way to > import *shell* commands from an external file? It's possible to add user-defined functions to the shell (which can be called without the module name prefix). Just create a module called user_default.erl and make sure it's compiled and in in the code path. Here's a description from the shell [1] man page: If a command (local function call) is not recognized by the shell, an attempt is first made to find the function in the module user_default, where customized local commands can be placed. If found, then the function is evaluated. Otherwise, an attempt is made to evaluate the function in the module shell_default. The module user_default must be explicitly loaded. Here's a really short example: 1. Create user_default.erl: -module(user_default). -compile(export_all). foo() -> foo. 2. Compile it. 3. erl -pa 4. 1> foo(). foo You will have to recompile and reload the file if you change it though. You can make sure the module is automatically loaded by adding it to your ~/.erlang file as described in the man page for shell_default [2] (then you wouldn't need the "-pa" option to erl): To add your own commands to the shell, create a module called user_default and add the commands you want. Then add the following line as the first line in your .erlang file in your home directory. code:load_abs("$PATH/user_default"). Note: You should not change the shell_default module, since this implements the existing shell functions, but instead add your own functions to user_default. I don't know if this will help you or not, since it doesn't solve all of the problems you listed, but perhaps it can prove useful? [1] http://erlang.org/doc/man/shell.html [2] http://www.erlang.org/doc/man/shell_default.html Best Regards, Klas From vladdu55@REDACTED Tue Sep 22 22:14:14 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 22 Sep 2009 22:14:14 +0200 Subject: [erlang-questions] Erlang/OTP R13B02 has been released In-Reply-To: <20090922101621.10bab2fc@der-dieb> References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> <20090922101621.10bab2fc@der-dieb> Message-ID: <95be1d3b0909221314o2a0ab8bdte0ac0dcd17cb45ac@mail.gmail.com> On Tue, Sep 22, 2009 at 19:16, Joe Williams wrote: > I seem to be having an issue building on linux (ubuntu 9.04 x86_64). Works fine for me on 32 bits, so it may be a 64 bit problem. regards, Vlad From wallentin.dahlberg@REDACTED Tue Sep 22 22:30:01 2009 From: wallentin.dahlberg@REDACTED (Wallentin Dahlberg) Date: Tue, 22 Sep 2009 22:30:01 +0200 Subject: [erlang-questions] Erlang/OTP R13B02 has been released In-Reply-To: <95be1d3b0909221314o2a0ab8bdte0ac0dcd17cb45ac@mail.gmail.com> References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> <20090922101621.10bab2fc@der-dieb> <95be1d3b0909221314o2a0ab8bdte0ac0dcd17cb45ac@mail.gmail.com> Message-ID: Building R13B02 on, Description: Ubuntu 9.04 2.6.28-15-generic #52-Ubuntu SMP Wed Sep 9 10:48:52 UTC 2009 x86_64 works fine for me. // Bj?rn-Egil 2009/9/22 Vlad Dumitrescu > On Tue, Sep 22, 2009 at 19:16, Joe Williams wrote: > > I seem to be having an issue building on linux (ubuntu 9.04 x86_64). > > Works fine for me on 32 bits, so it may be a 64 bit problem. > > regards, > Vlad > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From hokan.stenholm@REDACTED Tue Sep 22 22:51:44 2009 From: hokan.stenholm@REDACTED (=?windows-1252?Q?H=E5kan_Stenholm?=) Date: Tue, 22 Sep 2009 22:51:44 +0200 Subject: [erlang-questions] Erlang/OTP R13B02 has been released In-Reply-To: References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> <97619b170909220733m460862c7l5d71e36ea85c47f5@mail.gmail.com> Message-ID: <4AB938E0.4050802@bredband.net> Mihai Balea wrote: > > On Sep 22, 2009, at 10:33 AM, Rapsey wrote: > >> Build fails on OS X server v10.5.8: >> >> /usr/include/wx-2.8/wx/fontdlg.h: In constructor >> ?wxFontDialogBase::wxFontDialogBase(wxWindow*, const wxFontData*)?: >> /usr/include/wx-2.8/wx/fontdlg.h:67: warning: ?__base_ctor ? is >> deprecated >> (declared at /usr/include/wx-2.8/wx/fontdlg.h:66) >> gen/wxe_events.cpp: In function ?void initEventTable()?: >> gen/wxe_events.cpp:277: error: >> ?wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_DOWN? >> was not declared in this scope >> gen/wxe_events.cpp:278: error: >> ?wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_UP? was >> not declared in this scope >> gen/wxe_events.cpp:279: error: >> ?wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_DOWN? >> was not declared in this scope >> gen/wxe_events.cpp:280: error: >> ?wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_UP? was >> not declared in this scope >> gen/wxe_events.cpp:281: error: >> ?wxEVT_COMMAND_AUINOTEBOOK_PAGE_CLOSED? was >> not declared in this scope >> gen/wxe_events.cpp:282: error: ?wxEVT_COMMAND_AUINOTEBOOK_DRAG_DONE? >> was not >> declared in this scope >> gen/wxe_events.cpp:283: error: ?wxEVT_COMMAND_AUINOTEBOOK_BG_DCLICK? >> was not >> declared in this scope >> make[3]: *** [i386-apple-darwin9.8.0/wxe_events.o] Error 1 > > Same here, OSX desktop 10.5.8. > > Mihai I also get the same error on a MacPro (tower) running OSX - 10.5.8 when doing ./configure (without flags) make R13B01 on the other hand compiled without issues, on the same system, although it should be noted that wxErlang didn't work - I found this fix http://groups.google.com/group/erlang-programming/browse_thread/thread/54da30686f88033a but haven't tried it - so no custom built wxWidgets exists on my machine. > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From vinoski@REDACTED Wed Sep 23 01:48:33 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 22 Sep 2009 19:48:33 -0400 Subject: [erlang-questions] os:putenv/2 not helping with erl_ddll:load_driver/2 In-Reply-To: References: Message-ID: <65b2728e0909221648x760e19c7l11c4557a98f23d33@mail.gmail.com> On Tue, Sep 22, 2009 at 2:38 PM, Paul Mineiro wrote: > I'm encountering a difference between os:putenv/2 and setting the > environment variable prior to invoking Erlang. > > I'm trying to pick up a library that the os puts in a funny place, > which my linked-in driver depends upon. Manipulating the environment > variable inside Erlang doesn't have the desired effect. > > Does anybody have a workaround? I don't want to have to restart the > emulator to install this software. > Assuming you're on Linux, you can't set the LD_LIBRARY_PATH after the process is already running and have it take effect. It's only loaded at process startup and can't be changed thereafter within the process for it to affect that process. You might consider instead adding a runtime search path to your driver at link-time using the -rpath ld option (or -Wl,-rpath,dir from gcc). --steve From paul-trapexit@REDACTED Wed Sep 23 02:27:22 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Tue, 22 Sep 2009 17:27:22 -0700 (PDT) Subject: [erlang-questions] os:putenv/2 not helping with erl_ddll:load_driver/2 In-Reply-To: <65b2728e0909221648x760e19c7l11c4557a98f23d33@mail.gmail.com> References: <65b2728e0909221648x760e19c7l11c4557a98f23d33@mail.gmail.com> Message-ID: On Tue, 22 Sep 2009, Steve Vinoski wrote: > On Tue, Sep 22, 2009 at 2:38 PM, Paul Mineiro wrote: > > > I'm encountering a difference between os:putenv/2 and setting the > > environment variable prior to invoking Erlang. > > > > I'm trying to pick up a library that the os puts in a funny place, > > which my linked-in driver depends upon. Manipulating the environment > > variable inside Erlang doesn't have the desired effect. > > > > Does anybody have a workaround? I don't want to have to restart the > > emulator to install this software. > > > > Assuming you're on Linux, you can't set the LD_LIBRARY_PATH after the > process is already running and have it take effect. It's only loaded at > process startup and can't be changed thereafter within the process for it to > affect that process. You might consider instead adding a runtime search path > to your driver at link-time using the -rpath ld option (or -Wl,-rpath,dir > from gcc). Wow, you saved me a ton of time! Hopefully also the next person searching on this as well. For the record, using libtool, the key option was -R which ultimately sets rpath, e.g., liblpsolveerldrv_la_LDFLAGS = -module -avoid-version -R /usr/lib/lp_solve/ -- p From pablo.platt@REDACTED Wed Sep 23 03:17:15 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 22 Sep 2009 18:17:15 -0700 (PDT) Subject: calling erlang functions from php Message-ID: <749284.69627.qm@web31814.mail.mud.yahoo.com> Hi, I have a website built with php. I have an erlang application running as a daemon on the same server. I need to call erlang functions from php and get back the result. I've found php/erlang and other php modules but I can't install a php module on this server just use php scripts. I can install and use any erlang app. The only way I know to solve it is to run an erlang webserver locally that the php will be able to talk to. Is there a better way to solve it? If using httpd server is the best way, what erlang server to use? It should be as light as possible, simple and obviously doesn't need features like ssl and doesn't need to handle large load. Thanks From rapsey@REDACTED Wed Sep 23 06:51:30 2009 From: rapsey@REDACTED (Rapsey) Date: Wed, 23 Sep 2009 06:51:30 +0200 Subject: [erlang-questions] calling erlang functions from php In-Reply-To: <749284.69627.qm@web31814.mail.mud.yahoo.com> References: <749284.69627.qm@web31814.mail.mud.yahoo.com> Message-ID: <97619b170909222151p640b38e6t24db9b6cf281ef32@mail.gmail.com> Write your own then. Erlang makes that pretty simple. Sergej On Wed, Sep 23, 2009 at 3:17 AM, Pablo Platt wrote: > Hi, > > I have a website built with php. > > I have an erlang application running as a daemon on the same server. > > I need to call erlang functions from php and get back the result. > > I've found php/erlang and other php modules but I can't install a php > module on this server > just use php scripts. > > I can install and use any erlang app. > > The only way I know to solve it is to run an erlang webserver locally that > the php will be able to talk to. > Is there a better way to solve it? > > If using httpd server is the best way, what erlang server to use? > > It should be as light as possible, simple and obviously doesn't need > features like ssl and doesn't need to handle large load. > > Thanks > > > From rapsey@REDACTED Wed Sep 23 09:00:23 2009 From: rapsey@REDACTED (Rapsey) Date: Wed, 23 Sep 2009 09:00:23 +0200 Subject: [erlang-questions] Erlang/OTP R13B02 has been released In-Reply-To: <4AB938E0.4050802@bredband.net> References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> <97619b170909220733m460862c7l5d71e36ea85c47f5@mail.gmail.com> <4AB938E0.4050802@bredband.net> Message-ID: <97619b170909230000g6e37c360xf7046ef6c540c7ca@mail.gmail.com> On Tue, Sep 22, 2009 at 10:51 PM, H?kan Stenholm < hokan.stenholm@REDACTED> wrote: > Mihai Balea wrote: > >> >> On Sep 22, 2009, at 10:33 AM, Rapsey wrote: >> >> Build fails on OS X server v10.5.8: >>> >>> /usr/include/wx-2.8/wx/fontdlg.h: In constructor >>> ?wxFontDialogBase::wxFontDialogBase(wxWindow*, const wxFontData*)?: >>> /usr/include/wx-2.8/wx/fontdlg.h:67: warning: ?__base_ctor ? is >>> deprecated >>> (declared at /usr/include/wx-2.8/wx/fontdlg.h:66) >>> gen/wxe_events.cpp: In function ?void initEventTable()?: >>> gen/wxe_events.cpp:277: error: >>> ?wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_DOWN? >>> was not declared in this scope >>> gen/wxe_events.cpp:278: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_UP? >>> was >>> not declared in this scope >>> gen/wxe_events.cpp:279: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_DOWN? >>> was not declared in this scope >>> gen/wxe_events.cpp:280: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_UP? >>> was >>> not declared in this scope >>> gen/wxe_events.cpp:281: error: ?wxEVT_COMMAND_AUINOTEBOOK_PAGE_CLOSED? >>> was >>> not declared in this scope >>> gen/wxe_events.cpp:282: error: ?wxEVT_COMMAND_AUINOTEBOOK_DRAG_DONE? was >>> not >>> declared in this scope >>> gen/wxe_events.cpp:283: error: ?wxEVT_COMMAND_AUINOTEBOOK_BG_DCLICK? was >>> not >>> declared in this scope >>> make[3]: *** [i386-apple-darwin9.8.0/wxe_events.o] Error 1 >>> >> >> Same here, OSX desktop 10.5.8. >> >> Mihai >> > I also get the same error on a MacPro (tower) running OSX - 10.5.8 when > doing > > ./configure (without flags) > make > > R13B01 on the other hand compiled without issues, on the same system, > although it should be noted that wxErlang didn't work - I found this fix > http://groups.google.com/group/erlang-programming/browse_thread/thread/54da30686f88033abut haven't tried it - so no custom built wxWidgets exists on my machine. > > Yes that works. Sergej From roques@REDACTED Wed Sep 23 11:02:49 2009 From: roques@REDACTED (Christian von Roques) Date: Wed, 23 Sep 2009 11:02:49 +0200 Subject: [erlang-questions] Re: Erlang/OTP R13B02 has been released In-Reply-To: (Tim Fletcher's message of "Tue\, 22 Sep 2009 10\:01\:24 -0700 \(PDT\)") References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> Message-ID: <87ljk62fuu.fsf@sloppy.mti.ag> Tim Fletcher writes: >> The distribution can also be downloaded using the BitTorrent >> protocol. Use the following torrent files to download the source >> distribution and binary distribution for Windows: >> >> ?http://www.erlang.org/download/otp_src_R13B02.tar.gz.torrent >> ?http://www.erlang.org/download/otp_win32_R13B02.exe.torrent > > These don't appear to exist, and the torrent links at the top of > http://www.erlang.org/download.html also don't work. The tracker http://www.erlang.org/stats/torrent.html shows the torrents to be available here: http://www.erlang.org/download/torrents/otp_src_R13B02.tar.gz.torrent http://www.erlang.org/download/torrents/otp_win32_R13B02.exe.torrent Christian. From fritchie@REDACTED Wed Sep 23 11:14:38 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 23 Sep 2009 04:14:38 -0500 Subject: Anyone tried using "mutrace" on beam.smp? Message-ID: <200909230914.n8N9Ecds068258@snookles.snookles.com> Hi, all. What does a person do when attacked by insomnia? Mutex profiling of the Erlang virtual machine, of course. After taking the advice of mutrace's author and compiling everything(*) with the "-rdynamic" linker flag(**), I tried it out ... but the stack trace info didn't get much better than with a 100% untouched-by-me R13B01 + Fedora Core 10 Linux build. After running a sometime CPU-, sometimes disk-I/O-limited workload for a few minutes, I've attached the report that I got at the end of this message. (Running on R13B01 on a dual core i386 laptop.) Of the top 10 contended mutexes, 77.5 sec / 456.2 sec wallclock runtime shows says 17% time spent in contention. Actual wallclock time spent waiting was probably less than that, but how much less I dunno. I assume that the OTP team has been using some other tool for its profiling & optimization of the VM's SMP innards. If "mutrace" isn't used but could be helpful, wonderful. Otherwise, it may be of interest to other gentle readers? http://0pointer.de/blog/projects/mutrace.html -Scott (*) beam.smp and then three shared libraries: trace_ip_drv.so, trace_file_drv.so, and crypto_drv.so. (**) I cheated. I took the "make" output, hunted for where each of the four above were linked, then cut-and-pasted a small shell script to re-do the linking step with "-rdynamic" added to each. mutrace: Showing statistics for process beam.smp (pid 2954). mutrace: 197412 mutexes used. Mutex #191163 (0x0xb7ae0b08) first referenced by: /tmp/libmutrace.so(pthread_mutex_init+0x124) [0xb7e24274] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp(ethr_mutex_init+0x2c) [0x81b7e1c] Mutex #191120 (0x0xb7ae0988) first referenced by: /tmp/libmutrace.so(pthread_mutex_init+0x124) [0xb7e24274] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp(ethr_mutex_init+0x2c) [0x81b7e1c] Mutex #135006 (0x0x820fdc0) first referenced by: /tmp/libmutrace.so(pthread_mutex_init+0x124) [0xb7e24274] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp(ethr_mutex_init+0x2c) [0x81b7e1c] /lib/libc.so.6(__libc_start_main+0xe5) [0x3b06e5] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp [0x806e801] Mutex #135005 (0x0x820fd80) first referenced by: /tmp/libmutrace.so(pthread_rwlock_init+0xfb) [0xb7e2406b] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp(ethr_rwmutex_init+0x17) [0x81b79c7] /lib/libc.so.6(__libc_start_main+0xe5) [0x3b06e5] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp [0x806e801] Mutex #197409 (0x0x883d85c) first referenced by: /tmp/libmutrace.so(pthread_mutex_init+0x124) [0xb7e24274] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp(ethr_mutex_init+0x2c) [0x81b7e1c] Mutex #197408 (0x0x883d25c) first referenced by: /tmp/libmutrace.so(pthread_mutex_init+0x124) [0xb7e24274] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp(ethr_mutex_init+0x2c) [0x81b7e1c] Mutex #184333 (0x0xb7b2c9e8) first referenced by: /tmp/libmutrace.so(pthread_mutex_init+0x124) [0xb7e24274] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp(ethr_gate_init+0x3b) [0x81b856b] /lib/libc.so.6(clone+0x5e) [0x47e04e] Mutex #135004 (0x0x820fca0) first referenced by: /tmp/libmutrace.so(pthread_mutex_init+0x124) [0xb7e24274] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp(ethr_mutex_init+0x2c) [0x81b7e1c] Mutex #135026 (0x0xb7b9bd28) first referenced by: /tmp/libmutrace.so(pthread_mutex_init+0x124) [0xb7e24274] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp(ethr_gate_init+0x3b) [0x81b856b] /lib/libc.so.6(clone+0x5e) [0x47e04e] Mutex #135000 (0x0x820f7c0) first referenced by: /tmp/libmutrace.so(pthread_mutex_init+0x124) [0xb7e24274] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp(ethr_mutex_init+0x2c) [0x81b7e1c] /lib/libc.so.6(__libc_start_main+0xe5) [0x3b06e5] /tmp/R13B01/lib/erlang/erts-5.7.2/bin/beam.smp [0x806e801] mutrace: Showing 10 most contended mutexes: Mutex # Locked Changed Cont. tot.Time[ms] avg.Time[ms] max.Time[ms] Flags 191163 4244283 306763 89886 11937.279 0.003 15.324 M-.--. 191120 8810450 308827 57879 20226.626 0.002 11.541 M-.--. 135006 5279274 330673 26464 20335.058 0.004 7.666 M-.--. 135005 3544998 267925 3208 6814.369 0.002 3.729 W-...r 197409 1360828 36825 523 3135.264 0.002 7.318 M-.--. 197408 3003752 36382 490 6441.524 0.002 7.694 M-.--. 184333 279995 140004 275 799.137 0.003 6.036 M-.--. 135004 668695 15303 264 5276.240 0.008 9.349 M-.--. 135026 155037 77522 260 455.899 0.003 4.352 M-.--. 135000 1061733 15905 222 2081.179 0.002 6.419 M-.--. ... ... ... ... ... ... ... |||||| /||||| Object: M = Mutex, W = RWLock /|||| State: x = dead, ! = inconsistent /||| Use: R = used in realtime thread /|| Mutex Type: r = RECURSIVE, e = ERRRORCHECK, a = ADAPTIVE /| Mutex Protocol: i = INHERIT, p = PROTECT / RWLock Kind: r = PREFER_READER, w = PREFER_WRITER, W = PREFER_WRITER_NONREC mutrace: Note that the flags column R is only valid in --track-rt mode! mutrace: Total runtime is 456184.437 ms. mutrace: Results for SMP with 2 processors. mutrace: WARNING: 9 internal hash collisions detected. Results might not be as reliable as they could be. mutrace: Try to increase --hash-size=, which is currently at 39997. mutrace: WARNING: 532 internal mutex contention detected. Results might not be reliable as they could be. mutrace: Try to increase --hash-size=, which is currently at 39997. From zerthurd@REDACTED Wed Sep 23 11:17:45 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Wed, 23 Sep 2009 16:17:45 +0700 Subject: [erlang-questions] Re: Erlang/OTP R13B02 has been released In-Reply-To: <87ljk62fuu.fsf@sloppy.mti.ag> References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> <87ljk62fuu.fsf@sloppy.mti.ag> Message-ID: >From http://erlang.org/download.html : There is a serious bug in the reltool application included in R13B02. See otp_src_R13B02_OTP-8199.readme. There is a source code patch and a corrected version of the reltool appplication. It more appropriated to include correct reltool version in otp_src_R13B02-tarball and distribute it instead of tarball with this bug. Can you fix it? Thank you. -- Maxim Treskin From rtrlists@REDACTED Wed Sep 23 11:28:39 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 23 Sep 2009 10:28:39 +0100 Subject: [erlang-questions] using the target_system module In-Reply-To: <599dd3e0909221046x4639d536g39e562d99ea0506d@mail.gmail.com> References: <892026.55458.qm@web31813.mail.mud.yahoo.com> <77C1226CCC2F7C41B022662F4775DABC56FCE2C968@EGEXCMB01.oww.root.lcl> <599dd3e0909221046x4639d536g39e562d99ea0506d@mail.gmail.com> Message-ID: <6a3ae47e0909230228o494c47d1h58b5cdfc4fd2cb52@mail.gmail.com> On Tue, Sep 22, 2009 at 6:46 PM, mark peleus wrote: > I'll have a deeper look at Sinan but I actually like the way target_system > work. > With a single call it generates everything needed to make my erlang > application portable. > That's why I wanted to try and improve target_system. > > > I always just interpreted the target_system code to be an example of how to automate building a release. It's a starting point, from which you can jump off and make something that fits your needs. It's definitely geared towards creating a full system, ready for installation. Systems like CEAN (or Rocks or CPAN or JSAN, etc.) appear to me to be more suited for pulling in the libs you need while developing, rather than for distributing and installing a system at a customer site. But maybe I've not spent enough time learning these kinds of technologies. Robby From nesrait@REDACTED Wed Sep 23 12:58:08 2009 From: nesrait@REDACTED (=?ISO-8859-1?Q?Davide_Marqu=EAs?=) Date: Wed, 23 Sep 2009 11:58:08 +0100 Subject: [erlang-questions] calling erlang functions from php In-Reply-To: <749284.69627.qm@web31814.mail.mud.yahoo.com> References: <749284.69627.qm@web31814.mail.mud.yahoo.com> Message-ID: <523869a70909230358tb2007c4u8d73cbfb84e7e2d0@mail.gmail.com> Hi, Thrift seems to be exactly what you need: http://wiki.apache.org/thrift/ThriftErlSkel Cheers, Davide From kenneth.lundin@REDACTED Wed Sep 23 15:44:30 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 23 Sep 2009 15:44:30 +0200 Subject: [erlang-questions] using the target_system module In-Reply-To: <892026.55458.qm@web31813.mail.mud.yahoo.com> References: <892026.55458.qm@web31813.mail.mud.yahoo.com> Message-ID: Hi, On Mon, Sep 21, 2009 at 11:58 PM, Pablo Platt wrote: > Hi, > > I'm using the target_system module from http://erlang.org/doc/system_principles/create_target.html > The OTP application I'm using for tests is simpletest from http://eaglestreet.wordpress.com/2008/08/22/releasing_erlang_otp_code/ > > The target_system module is really good. > Is it officially supported and will be included in a future release or just an example? It is just an example in the documentation. But in the new R13B02 release you have the reltool application which does the same thing as "target-system" but in a more flexible way intended for use both when you want to build a traditional embedded system and when you want to build a self contained package for easy distribution and intallation to end users. /Kenneth Erlang/OTP Ericsson > > I have several remarks: > > 1. It doesn't work with the Ubuntu Jaunty erlang package. The doc should indicate that it only works with R13B and above. > 2. I had to manually change the permissions of some of the scripts under the bin directory to executable. > Is it possible that the module will take care of that? > 3. It took me some time to understand that bin/start can't run without sys.config file under the releases dir. > Can bin/start throw an error if it can't find the sys.config file? > 4. The module should create a log dir. > 5. In my case the create method built relative path. I'm not sure if this is intended or not but it might cause issues. > 6. It will be helpful to have a tar file online with a test application that works with this module for easy learning. > > Thanks > > > > From dizzyd@REDACTED Wed Sep 23 15:51:23 2009 From: dizzyd@REDACTED (Dave Smith) Date: Wed, 23 Sep 2009 07:51:23 -0600 Subject: [erlang-questions] Re: Erlang/OTP R13B02 has been released In-Reply-To: References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> <87ljk62fuu.fsf@sloppy.mti.ag> Message-ID: I'd like to add my voice to this request. Having a reltool with such destructive behaviour in the official tarball is scary to say the least. D. On Wed, Sep 23, 2009 at 3:17 AM, Maxim Treskin wrote: > From http://erlang.org/download.html : > > There is a serious bug in the reltool application included in R13B02. > See otp_src_R13B02_OTP-8199.readme. There is a source code patch and a > corrected version of the reltool appplication. > > It more appropriated to include correct reltool version in > otp_src_R13B02-tarball and distribute it instead of tarball with this > bug. Can you fix it? > > Thank you. > > -- > Maxim Treskin > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From tonyg@REDACTED Wed Sep 23 15:34:00 2009 From: tonyg@REDACTED (Tony Garnock-Jones) Date: Wed, 23 Sep 2009 14:34:00 +0100 Subject: xmerl_scan for XMPP -- too much lookahead? Message-ID: <4ABA23C8.3020206@lshift.net> Hi all, I'm trying to use xmerl_scan to handle XMPP stanzas, and I think I've found a problem with it. I don't think it's intended for streaming XML in XMPP-like interleaved-request-and-response-document situations. The attached program feeds the string "" to xmerl_scan. I would have expected to receive the open-tag event before blocking for more data, but instead it requires at least one more character of input data before it will emit the open-tag event! If I instead pass " ", with a space after the close-bracket, it emits the open-tag event as expected, plus the start of an xmlText, before blocking for more data. My question, then, is: Is this a bug? Should xmerl_scan supply the open-tag event before blocking, when it is fed ""? (The specific context of this problem is dealing with the sent by the server at the handshake stage of XEP-114.) Regards, Tony P.S.: to run the attached program, $ erlc lookaheadbug.erl && erl -run lookaheadbug go -- [][][] Tony Garnock-Jones | Mob: +44 (0)7905 974 211 [][] LShift Ltd | Tel: +44 (0)20 7729 7060 [] [] http://www.lshift.net/ | Email: tonyg@REDACTED -------------- next part -------------- A non-text attachment was scrubbed... Name: lookaheadbug.erl Type: text/x-erlang Size: 832 bytes Desc: not available URL: From gleber.p@REDACTED Wed Sep 23 16:07:37 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 23 Sep 2009 16:07:37 +0200 Subject: [erlang-questions] xmerl_scan for XMPP -- too much lookahead? In-Reply-To: <4ABA23C8.3020206@lshift.net> References: <4ABA23C8.3020206@lshift.net> Message-ID: <14f0e3620909230707x5558fa0pb218a2e2bfa68e04@mail.gmail.com> Hi You could have seen it already :) But nevertheless there is exmpp [1] library from Process One. It supports such things: 1: http://support.process-one.net/doc/display/EXMPP/ HTH. Gleb Peregud On Wed, Sep 23, 2009 at 15:34, Tony Garnock-Jones wrote: > Hi all, > > I'm trying to use xmerl_scan to handle XMPP stanzas, and I think I've > found a problem with it. I don't think it's intended for streaming XML > in XMPP-like interleaved-request-and-response-document situations. > > The attached program feeds the string "" to xmerl_scan. I would > have expected to receive the open-tag event before blocking for more > data, but instead it requires at least one more character of input data > before it will emit the open-tag event! If I instead pass " ", > with a space after the close-bracket, it emits the open-tag event as > expected, plus the start of an xmlText, before blocking for more data. > > My question, then, is: > > ?Is this a bug? Should xmerl_scan supply the open-tag event before > ?blocking, when it is fed ""? > > (The specific context of this problem is dealing with the > sent by the server at the handshake stage of XEP-114.) > > Regards, > ?Tony > > P.S.: to run the attached program, > ?$ erlc lookaheadbug.erl && erl -run lookaheadbug go > -- > ?[][][] Tony Garnock-Jones ? ? | Mob: +44 (0)7905 974 211 > ? [][] LShift Ltd ? ? ? ? ? ? | Tel: +44 (0)20 7729 7060 > ?[] ?[] http://www.lshift.net/ | Email: tonyg@REDACTED > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From hakan@REDACTED Wed Sep 23 16:19:00 2009 From: hakan@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Wed, 23 Sep 2009 16:19:00 +0200 Subject: Erlang/OTP R13B02-1 has been released Message-ID: <922d05850909230719k5592e211ye51db32072a00b86@mail.gmail.com> Bug fix release : otp_src_R13B02-1 Build date : 2009-09-21 Reltool build date: 2009-09-23 This is R13B02-1, a patched version of R13B02 which was the second maintenance release for the R13B major release. R13B02-1 has the same contents as R13B02 with the exception of Reltool. In the new version of Reltool (0.5.1) this nasty bug has been fixed: OTP-8199 In the new release of Reltool (0.5) there is a severe bug that may cause your source files to be deleted. The bug is triggered when you generate a target system WITH archive files AND your library directories are given with absolute paths. (Library directories are given with the ERL_LIBS environment variable or with the lib_dirs configuration parameter.) You can find the README file for the R13B02-1 release at http://www.erlang.org/download/otp_src_R13B02-1.readme The source distribution and binary distribution for Windows can be downloaded from http://www.erlang.org/download/otp_src_R13B02-1.tar.gz http://www.erlang.org/download/otp_win32_R13B02-1.exe The distribution can also be downloaded using the BitTorrent protocol. Use the following torrent files to download the source distribution and binary distribution for Windows: http://www.erlang.org/download/otp_src_R13B02-1.tar.gz.torrent http://www.erlang.org/download/otp_win32_R13B02-1.exe.torrent Note: To unpack the TAR archive you need a GNU TAR compatible program. For installation instructions please read the README file that is part of the distribution. The on-line documentation can be found at: http://www.erlang.org/doc/ You can also download the complete HTML documentation or the Unix manual files http://www.erlang.org/download/otp_doc_html_R13B02-1.tar.gz http://www.erlang.org/download/otp_doc_man_R13B02-1.tar.gz We also want to thank those that sent us patches, suggestions and bug reports, The OTP Team --- H?kan Mattsson (uabhams) Erlang/OTP, Ericsson AB From tonyg@REDACTED Wed Sep 23 16:19:10 2009 From: tonyg@REDACTED (Tony Garnock-Jones) Date: Wed, 23 Sep 2009 15:19:10 +0100 Subject: [erlang-questions] xmerl_scan for XMPP -- too much lookahead? In-Reply-To: <14f0e3620909230707x5558fa0pb218a2e2bfa68e04@mail.gmail.com> References: <4ABA23C8.3020206@lshift.net> <14f0e3620909230707x5558fa0pb218a2e2bfa68e04@mail.gmail.com> Message-ID: <4ABA2E5E.80503@lshift.net> Hi Gleb, Yes, thanks, I've seen that already. We're already using it, in fact. If exmpp didn't exist, though, the bug (?) in xmerl_scan would make it difficult (impossible?) to write a pure-Erlang XEP-114 implementation using xmerl. Regards, Tony Gleb Peregud wrote: > Hi > > You could have seen it already :) But nevertheless there is exmpp [1] > library from Process One. It supports such things: > > 1: http://support.process-one.net/doc/display/EXMPP/ > > HTH. > Gleb Peregud -- [][][] Tony Garnock-Jones | Mob: +44 (0)7905 974 211 [][] LShift Ltd | Tel: +44 (0)20 7729 7060 [] [] http://www.lshift.net/ | Email: tonyg@REDACTED From mickael.remond@REDACTED Wed Sep 23 16:24:14 2009 From: mickael.remond@REDACTED (=?iso-8859-1?Q?Micka=EBl_R=E9mond?=) Date: Wed, 23 Sep 2009 16:24:14 +0200 Subject: [erlang-questions] xmerl_scan for XMPP -- too much lookahead? In-Reply-To: <14f0e3620909230707x5558fa0pb218a2e2bfa68e04@mail.gmail.com> References: <4ABA23C8.3020206@lshift.net> <14f0e3620909230707x5558fa0pb218a2e2bfa68e04@mail.gmail.com> Message-ID: <7F6EF0C0-9094-4B59-969C-F544B3871848@process-one.net> Hello, Le 23 sept. 2009 ? 16:07, Gleb Peregud a ?crit : > Hi > > You could have seen it already :) But nevertheless there is exmpp [1] > library from Process One. It supports such things: > > 1: http://support.process-one.net/doc/display/EXMPP/ Yes, come on Tony, use a real tool dedicated to the task ;) Our test and all reports we have shows it is very efficient. Let us know if you have questions, reports or improvements, -- Micka?l R?mond http://www.process-one.net/ From ulf.wiger@REDACTED Wed Sep 23 16:27:31 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 23 Sep 2009 16:27:31 +0200 Subject: [erlang-questions] xmerl_scan for XMPP -- too much lookahead? In-Reply-To: <4ABA23C8.3020206@lshift.net> References: <4ABA23C8.3020206@lshift.net> Message-ID: <4ABA3053.3050006@erlang-consulting.com> Tony, The support for streaming in xmerl_scan is quite hackish, and is bound to be broken one way or another. I doubt that it is feasible to correctly handle streams given how xmerl_scan works. As it is, you could reasonably ask of it to be a little bit more discerning - right now, xmerl_eventp only breaks at whitespace, which is very conservative. The main problem is that xmerl_scan is undisciplined when it comes to ensuring that it has enough characters to pattern-match in the current function head. E.g. %% [75] ExternalID ::= 'SYSTEM' S SystemLiteral %% | 'PUBLIC' S PubidLiteral S SystemLiteral scan_doctype1([], S=#xmerl_scanner{continuation_fun = F}) -> F(fun(MoreBytes, S1) -> scan_doctype1(MoreBytes, S1) end, fun(S1) -> ?fatal(unexpected_end, S1) end, S); scan_doctype1("PUBLIC" ++ T, S0) -> ... If given a stream fragment, like "PUBL", the matching above will fail, and xmerl_scan will derail. THIS is a serious bug - and I'm originally at fault. ;-) Have you tried using xmerl_sax_parser instead? The plan is to replace xmerl_scan completely for stream parsing. And given this, xmerl_eventp is unlikely to see any major improvements. BR, Ulf W Tony Garnock-Jones wrote: > Hi all, > > I'm trying to use xmerl_scan to handle XMPP stanzas, and I think I've > found a problem with it. I don't think it's intended for streaming XML > in XMPP-like interleaved-request-and-response-document situations. > > The attached program feeds the string "" to xmerl_scan. I would > have expected to receive the open-tag event before blocking for more > data, but instead it requires at least one more character of input data > before it will emit the open-tag event! If I instead pass " ", > with a space after the close-bracket, it emits the open-tag event as > expected, plus the start of an xmlText, before blocking for more data. > > My question, then, is: > > Is this a bug? Should xmerl_scan supply the open-tag event before > blocking, when it is fed ""? > > (The specific context of this problem is dealing with the > sent by the server at the handshake stage of XEP-114.) > > Regards, > Tony > > P.S.: to run the attached program, > $ erlc lookaheadbug.erl && erl -run lookaheadbug go > > > ------------------------------------------------------------------------ > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From tonyg@REDACTED Wed Sep 23 15:46:45 2009 From: tonyg@REDACTED (Tony Garnock-Jones) Date: Wed, 23 Sep 2009 14:46:45 +0100 Subject: xmerl_scan for XMPP -- too much lookahead? Message-ID: <4ABA26C5.6090506@lshift.net> Hi all, I'm trying to use xmerl_scan to handle XMPP stanzas, and I think I've found a problem with it. I don't think it's intended for streaming XML in XMPP-like interleaved-request-and-response-document situations. The attached program feeds the string "" to xmerl_scan. I would have expected to receive the open-tag event before blocking for more data, but instead it requires at least one more character of input data before it will emit the open-tag event! If I instead pass " ", with a space after the close-bracket, it emits the open-tag event as expected, plus the start of an xmlText, before blocking for more data. My question, then, is: Is this a bug? Should xmerl_scan supply the open-tag event before blocking, when it is fed ""? (The specific context of this problem is dealing with the sent by the server at the handshake stage of XEP-114.) Regards, Tony P.S.: to run the attached program, $ erlc lookaheadbug.erl && erl -run lookaheadbug go -- [][][] Tony Garnock-Jones | Mob: +44 (0)7905 974 211 [][] LShift Ltd | Tel: +44 (0)20 7729 7060 [] [] http://www.lshift.net/ | Email: tonyg@REDACTED -------------- next part -------------- A non-text attachment was scrubbed... Name: lookaheadbug.erl Type: text/x-erlang Size: 833 bytes Desc: not available URL: From info@REDACTED Wed Sep 23 16:31:35 2009 From: info@REDACTED (info) Date: Wed, 23 Sep 2009 16:31:35 +0200 Subject: {error,timeout} with gethostbyname Message-ID: <200909231631349711641@its3.ch> Hi all, On a windows 2003 server sbs this command returns {ok,"my_host"} : {ok,N}=inet:gethostname(). But the following command returns {error,timeout} : inet:gethostbyname(N). On a Vista machine, the previous command returns a correct answer. I know that it is not an erlang problem but if you could give me support, I will appreciate ;-) From tonyg@REDACTED Wed Sep 23 16:33:28 2009 From: tonyg@REDACTED (Tony Garnock-Jones) Date: Wed, 23 Sep 2009 15:33:28 +0100 Subject: [erlang-questions] xmerl_scan for XMPP -- too much lookahead? In-Reply-To: <4ABA3053.3050006@erlang-consulting.com> References: <4ABA23C8.3020206@lshift.net> <4ABA3053.3050006@erlang-consulting.com> Message-ID: <4ABA31B8.9060906@lshift.net> Ulf Wiger wrote: > Have you tried using xmerl_sax_parser instead? > The plan is to replace xmerl_scan completely for stream > parsing. And given this, xmerl_eventp is unlikely to see > any major improvements. Aha! Thank you, Ulf, I will try that. Regards, Tony -- [][][] Tony Garnock-Jones | Mob: +44 (0)7905 974 211 [][] LShift Ltd | Tel: +44 (0)20 7729 7060 [] [] http://www.lshift.net/ | Email: tonyg@REDACTED From hawk.mattsson@REDACTED Wed Sep 23 16:46:07 2009 From: hawk.mattsson@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Wed, 23 Sep 2009 16:46:07 +0200 Subject: Erlang/OTP R13B02-1 has been released In-Reply-To: <922d05850909230719k5592e211ye51db32072a00b86@mail.gmail.com> References: <922d05850909230719k5592e211ye51db32072a00b86@mail.gmail.com> Message-ID: <922d05850909230746k332c97c0s137aeb3eb11d577a@mail.gmail.com> The Windows exe is not yet available, but it will be released soon. /H?kan --- H?kan Mattsson (uabhams) Erlang/OTP, Ericsson AB 2009/9/23 H?kan Mattsson : > Bug fix release ? : otp_src_R13B02-1 > Build date ? ? ? ? ?: 2009-09-21 > Reltool build date: 2009-09-23 > > This is R13B02-1, a patched version of R13B02 which was the second > maintenance release for the R13B major release. R13B02-1 has the > same contents as R13B02 with the exception of Reltool. In the new > version of Reltool (0.5.1) this nasty bug has been fixed: > > ? OTP-8199 In the new release of Reltool (0.5) there is a severe bug > ? ? ? ? ? ? ? ? ? that may cause your source files to be deleted. The bug is > ? ? ? ? ? ? ? ? ? triggered when you generate a target system WITH archive > ? ? ? ? ? ? ? ? ? files AND your library directories are given with absolute > ? ? ? ? ? ? ? ? ? paths. (Library directories are given with the ERL_LIBS > ? ? ? ? ? ? ? ? ? environment variable or with the lib_dirs configuration > ? ? ? ? ? ? ? ? ? parameter.) > > You can find the README file for the R13B02-1 release at > > ?http://www.erlang.org/download/otp_src_R13B02-1.readme > > The source distribution and binary distribution for Windows can be > downloaded from > > ?http://www.erlang.org/download/otp_src_R13B02-1.tar.gz > ?http://www.erlang.org/download/otp_win32_R13B02-1.exe > > The distribution can also be downloaded using the BitTorrent > protocol. Use the following torrent files to download the source > distribution and binary distribution for Windows: > > ?http://www.erlang.org/download/otp_src_R13B02-1.tar.gz.torrent > ?http://www.erlang.org/download/otp_win32_R13B02-1.exe.torrent > > Note: To unpack the TAR archive you need a GNU TAR compatible program. > > For installation instructions please read the README file that is part > of the distribution. > > The on-line documentation can be found at: http://www.erlang.org/doc/ > You can also download the complete HTML documentation or the Unix manual files > > ?http://www.erlang.org/download/otp_doc_html_R13B02-1.tar.gz > ?http://www.erlang.org/download/otp_doc_man_R13B02-1.tar.gz > > We also want to thank those that sent us patches, suggestions and bug > reports, > > The OTP Team > > --- > H?kan Mattsson (uabhams) > Erlang/OTP, Ericsson AB From dizzyd@REDACTED Wed Sep 23 17:08:40 2009 From: dizzyd@REDACTED (Dave Smith) Date: Wed, 23 Sep 2009 09:08:40 -0600 Subject: [erlang-questions] Erlang/OTP R13B02-1 has been released In-Reply-To: <922d05850909230719k5592e211ye51db32072a00b86@mail.gmail.com> References: <922d05850909230719k5592e211ye51db32072a00b86@mail.gmail.com> Message-ID: Thanks! D. 2009/9/23 H?kan Mattsson : > Bug fix release ? : otp_src_R13B02-1 > Build date ? ? ? ? ?: 2009-09-21 > Reltool build date: 2009-09-23 > > This is R13B02-1, a patched version of R13B02 which was the second > maintenance release for the R13B major release. R13B02-1 has the > same contents as R13B02 with the exception of Reltool. In the new > version of Reltool (0.5.1) this nasty bug has been fixed: > > ? OTP-8199 In the new release of Reltool (0.5) there is a severe bug > ? ? ? ? ? ? ? ? ? that may cause your source files to be deleted. The bug is > ? ? ? ? ? ? ? ? ? triggered when you generate a target system WITH archive > ? ? ? ? ? ? ? ? ? files AND your library directories are given with absolute > ? ? ? ? ? ? ? ? ? paths. (Library directories are given with the ERL_LIBS > ? ? ? ? ? ? ? ? ? environment variable or with the lib_dirs configuration > ? ? ? ? ? ? ? ? ? parameter.) > > You can find the README file for the R13B02-1 release at > > ?http://www.erlang.org/download/otp_src_R13B02-1.readme > > The source distribution and binary distribution for Windows can be > downloaded from > > ?http://www.erlang.org/download/otp_src_R13B02-1.tar.gz > ?http://www.erlang.org/download/otp_win32_R13B02-1.exe > > The distribution can also be downloaded using the BitTorrent > protocol. Use the following torrent files to download the source > distribution and binary distribution for Windows: > > ?http://www.erlang.org/download/otp_src_R13B02-1.tar.gz.torrent > ?http://www.erlang.org/download/otp_win32_R13B02-1.exe.torrent > > Note: To unpack the TAR archive you need a GNU TAR compatible program. > > For installation instructions please read the README file that is part > of the distribution. > > The on-line documentation can be found at: http://www.erlang.org/doc/ > You can also download the complete HTML documentation or the Unix manual files > > ?http://www.erlang.org/download/otp_doc_html_R13B02-1.tar.gz > ?http://www.erlang.org/download/otp_doc_man_R13B02-1.tar.gz > > We also want to thank those that sent us patches, suggestions and bug > reports, > > The OTP Team > > --- > H?kan Mattsson (uabhams) > Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From dave.pawson@REDACTED Wed Sep 23 17:15:48 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Wed, 23 Sep 2009 16:15:48 +0100 Subject: [erlang-questions] Re: Erlang/OTP R13B02-1 has been released In-Reply-To: <922d05850909230746k332c97c0s137aeb3eb11d577a@mail.gmail.com> References: <922d05850909230719k5592e211ye51db32072a00b86@mail.gmail.com> <922d05850909230746k332c97c0s137aeb3eb11d577a@mail.gmail.com> Message-ID: <711a73df0909230815m1c2b31cdx54c35601c3d450ea@mail.gmail.com> 2009/9/23 H?kan Mattsson : > The Windows exe is not yet available, but it will be released soon. # Where it says 'Windows source', is that 'source' more generally, I.e. should build on Linux? regards -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From Zhiquan.Yuan.0193@REDACTED Wed Sep 23 17:20:32 2009 From: Zhiquan.Yuan.0193@REDACTED (Zhiqian Yuan) Date: Wed, 23 Sep 2009 17:20:32 +0200 Subject: TCP libraries in Erlang/OTP? Message-ID: <20090923172032.6aglmm0p6o8o0wcg@webmail6.uu.se> Hi everyone, we're now doing a project on IMS which is to implemented by erlang. And now we're on the HSS part, our teammates are trying to write the protocol stack of Diameter, but we don't know if we need to implement the TCP protocol as well. So we wonder if there're any libraries avaible in the OTP framework, so that we can encapsulate the Diameter package. Thanks. Regards From rapsey@REDACTED Wed Sep 23 17:37:01 2009 From: rapsey@REDACTED (Rapsey) Date: Wed, 23 Sep 2009 17:37:01 +0200 Subject: [erlang-questions] TCP libraries in Erlang/OTP? In-Reply-To: <20090923172032.6aglmm0p6o8o0wcg@webmail6.uu.se> References: <20090923172032.6aglmm0p6o8o0wcg@webmail6.uu.se> Message-ID: <97619b170909230837w36c1508t4f21c6fad5ba4703@mail.gmail.com> A lot of acronyms in your message I have never heard of. But I guess your looking for gen_tcp? Sergej On Wed, Sep 23, 2009 at 5:20 PM, Zhiqian Yuan < Zhiquan.Yuan.0193@REDACTED> wrote: > Hi everyone, we're now doing a project on IMS which is to implemented by > erlang. And now we're on the HSS part, our teammates are trying to write the > protocol stack of Diameter, but we don't know if we need to implement the > TCP protocol as well. So we wonder if there're any libraries avaible in the > OTP framework, so that we can encapsulate the Diameter package. Thanks. > > Regards > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ulf.wiger@REDACTED Wed Sep 23 17:43:21 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 23 Sep 2009 17:43:21 +0200 Subject: [erlang-questions] TCP libraries in Erlang/OTP? In-Reply-To: <20090923172032.6aglmm0p6o8o0wcg@webmail6.uu.se> References: <20090923172032.6aglmm0p6o8o0wcg@webmail6.uu.se> Message-ID: <4ABA4219.8090608@erlang-consulting.com> Zhiqian Yuan wrote: > Hi everyone, we're now doing a project on IMS which is to implemented by > erlang. And now we're on the HSS part, our teammates are trying to write > the protocol stack of Diameter, but we don't know if we need to > implement the TCP protocol as well. So we wonder if there're any > libraries avaible in the OTP framework, so that we can encapsulate the > Diameter package. Thanks. Huh? That is a pretty big task for a student project, at least if you are aiming at a working HSS - or is the goal to write an _entire_ IMS system? Fortunately, you don't also have to write a TCP stack. :) The gen_tcp module will do quite nicely. However, if you want to be true to the IMS specification, it is really SCTP you should be looking at. The SCTP support in OTP is not quite ready for this sort of thing, unless I've missed some important development, but given that you're in a student project, it might be good enough for your purposes. You may want to take a sneak peek at this: http://code.google.com/p/hss/ It doesn't have Diameter support. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From hakan@REDACTED Wed Sep 23 17:49:18 2009 From: hakan@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Wed, 23 Sep 2009 17:49:18 +0200 Subject: [erlang-questions] Re: Erlang/OTP R13B02-1 has been released In-Reply-To: <711a73df0909230815m1c2b31cdx54c35601c3d450ea@mail.gmail.com> References: <922d05850909230719k5592e211ye51db32072a00b86@mail.gmail.com> <922d05850909230746k332c97c0s137aeb3eb11d577a@mail.gmail.com> <711a73df0909230815m1c2b31cdx54c35601c3d450ea@mail.gmail.com> Message-ID: <922d05850909230849y9cd3ed3wb60ce0c17ab0e501@mail.gmail.com> 2009/9/23 Dave Pawson : > 2009/9/23 H?kan Mattsson : >> The Windows exe is not yet available, but it will be released soon. > # > > Where it says 'Windows source', is that 'source' more generally, > I.e. should build on Linux? The source distribution can be used to build Erlang/OTP on all supported platforms, including Windows. On WIndows you also have the option to use the binary distribution, which contains a prebuilt system. The binary distribution (Windows only) for R13B02-1 is not yet available, but it will be released soon. /H?kan --- H?kan Mattsson (uabhams) Erlang/OTP, Ericsson AB From tonyg@REDACTED Wed Sep 23 18:09:48 2009 From: tonyg@REDACTED (Tony Garnock-Jones) Date: Wed, 23 Sep 2009 17:09:48 +0100 Subject: Bugs in xmerl_sax_parser? Message-ID: <4ABA484C.8080105@lshift.net> Hi all, I've just been playing with xmerl_sax_parser and I think I've found two bugs: 1. xmerl_sax_parser:file/2 and :stream/2 behave differently. Create a file, "t.xml", containing and then run the two expressions xmerl_sax_parser:file("t.xml", [{event_fun, fun (E,_,_) -> io:format("~p~n", [E]), ok end}]). xmerl_sax_parser:stream("", [{event_fun, fun (E,_,_) -> io:format("~p~n", [E]), ok end}]). Note that they print the same event sequences, but the first results in {ok,ok,<<>>}, while the second results in {fatal_error,...}! 2. The namespace URI supplied on unprefixed attributes in startElement tuples is the same as the URI for the default namespace. According to http://www.w3.org/TR/xml-names/#defaulting, "The namespace name for an unprefixed attribute name always has no value." For example, running one of the expressions above causes the following to be printed: {startElement,"http://lshift.net/d","elem", {[],"elem"}, [{"http://lshift.net/d",[],"attr","123"}, {"http://lshift.net/x","x","attr","234"}]} While applications can work around this by ignoring the attribute's namespace name in cases where the prefix == "", I wonder if it wouldn't be better to supply "" for the namespace name for unprefixed attributes, like this: {startElement,"http://lshift.net/d","elem", {[],"elem"}, [{[],[],"attr","123"}, {"http://lshift.net/x","x","attr","234"}]} Regards, Tony -- [][][] Tony Garnock-Jones | Mob: +44 (0)7905 974 211 [][] LShift Ltd | Tel: +44 (0)20 7729 7060 [] [] http://www.lshift.net/ | Email: tonyg@REDACTED From info@REDACTED Wed Sep 23 18:48:32 2009 From: info@REDACTED (info) Date: Wed, 23 Sep 2009 18:48:32 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname References: <200909231631349711641@its3.ch>, <6a3ae47e0909230806oa09b389v380297ec7d074706@mail.gmail.com> Message-ID: <200909231848315462351@its3.ch> No, the microsoft firewall is disabled. I am using another one. On Wed, Sep 23, 2009 at 3:31 PM, info wrote: Hi all, On a windows 2003 server sbs this command returns {ok,"my_host"} : {ok,N}=inet:gethostname(). But the following command returns {error,timeout} : inet:gethostbyname(N). On a Vista machine, the previous command returns a correct answer. I know that it is not an erlang problem but if you could give me support, I will appreciate ;-) Is there a Microsoft Firewall Client installed on that server? Just asking, because I've had an issue with gethostbyname() hanging if the Firewall Client for ISA was installed on Server 2003. Robby From prikrutil@REDACTED Wed Sep 23 19:43:26 2009 From: prikrutil@REDACTED (Sergey Samokhin) Date: Wed, 23 Sep 2009 21:43:26 +0400 Subject: [erlang-questions] Is it safe to unlink from gen_fsm on the fly? In-Reply-To: References: <2F152643-CA6E-4C08-8FD0-6A7D41EE6E1C@souja.net> Message-ID: Hello, Jayson! > If you have the reconnection process always running, do you need to spawn it > and pass it around? No, I don't =) > That said, you might be able to invert the scenario. ?What if it was a > gen_event? ?They you could just throw off a synchronous event when you have > a disconnection, and it could catch it to reconnect. Which of the previously mentioned processes (gen_server, connector) are you suggesting to be implemented as a gen_event? Could you explain the picture with gen_event being used anyway in a bit more details? Assuming the connector is implemented as a gen_event, supervision three should have one of the two following forms (hope, spaces won't be skipped by ezmlm): a) [iface] and [connector] are both linked to the supervisor: [supervisor] | | [iface gen_server] [connector gen_event] b) connector is linked to the iface which in turn is linked to the supervisor: [supervisor] | [iface gen_server] | [connector gen_event] Is any of those the solution you had in mind while writing about gen_event? P.S. I have one more solution to note that hasn't been touched yet: there is only one process - gen_server - trying to reconnect every ?TIME ms by handling messages sent using erlang:send_after(?TIME, self(), reconnect). This way to reconnect can be considered as "partially" blocking, because the gen_server is still able to react on incoming message between 'reconnect' events. It's only useful when connect() is relative fast. Thanks. -- Sergey Samokhin From dizzyd@REDACTED Wed Sep 23 19:52:23 2009 From: dizzyd@REDACTED (Dave Smith) Date: Wed, 23 Sep 2009 11:52:23 -0600 Subject: Bug in inets HTTP Client w/ Chunked Encoding In-Reply-To: References: Message-ID: Could someone on the OTP team confirm that this is actually a bug or no? I hate seeing so many crashes in my error logs and would prefer not to have to depend on HTTP clients other than inets (from a packaging standpoint). Thanks. :) D. On Tue, Sep 8, 2009 at 8:56 AM, Dave Smith wrote: > Argh...I failed basic training of bug reporting apparently... > > O/S Platform: OS X 10.6 and 10.5 > Erlang Version: R13B01 (64 and 32-bit versions) > > D. > > On Tue, Sep 8, 2009 at 8:53 AM, Dave Smith wrote: >> Greetings, >> >> I stumbled across what I believe to be a bug in the inets HTTP client, >> when used with HTTP chunked encoding and having the client save >> directly to a file. The distilled test case is attached. >> >> From my reading of the client source code, it looks like there is a >> rather severe impedance mismatch between the httpc_handler and >> http_chunk modules. The http_chunk modules correctly decodes the >> chunks and calls back into httpc_manager:stream, but then fails to >> pass the modified HTTP request back out on completion. It's not clear >> to me exactly how one would fix this without a fair bit of rework. >> >> Of note, it's possible to work around the failure by having the >> library stream the chunks to the calling process. However, it's kinda >> of silly to have to do that when the HTTP client provides a feature >> for this very use case. >> >> Please let me know if I can do anything to further elucidate this bug. >> >> D. >> > From mjtruog@REDACTED Wed Sep 23 21:31:37 2009 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 23 Sep 2009 12:31:37 -0700 Subject: malloc is not thread-safe in glibc Message-ID: <4ABA7799.7060807@gmail.com> Wouldn't this malloc glibc bug affect Erlang? http://sourceware.org/bugzilla/show_bug.cgi?id=6952 The glibc maintainer works for RedHat and they have an update for RHEL. The ptmalloc3 in the bug report is here: http://www.malloc.de/en/ Since it is architecture dependent, it wouldn't be a problem for everyone. Just was surprised to see it. - Michael From larry@REDACTED Wed Sep 23 21:33:29 2009 From: larry@REDACTED (Larry Ogrodnek) Date: Wed, 23 Sep 2009 12:33:29 -0700 Subject: blowfish ECB implementation in erlang? Message-ID: <38ff4e6b0909231233i4435a436v170a8b1953b22c4e@mail.gmail.com> Hi, I'm wondering if anyone has put together an implementation of blowfish ECB? thanks. From mjtruog@REDACTED Wed Sep 23 21:44:13 2009 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 23 Sep 2009 12:44:13 -0700 Subject: [erlang-questions] malloc is not thread-safe in glibc In-Reply-To: <4ABA7799.7060807@gmail.com> References: <4ABA7799.7060807@gmail.com> Message-ID: <4ABA7A8D.8090703@gmail.com> In case you were wondering, this should relate to the RHEL release here: http://dcs.nac.uci.edu/support/sysadmin/security/archive/msg04994.html (so the release is relatively new) Michael Truog wrote: > Wouldn't this malloc glibc bug affect Erlang? > http://sourceware.org/bugzilla/show_bug.cgi?id=6952 > > The glibc maintainer works for RedHat and they have an update for RHEL. > The ptmalloc3 in the bug report is here: > http://www.malloc.de/en/ > > Since it is architecture dependent, it wouldn't be a problem for > everyone. Just was surprised to see it. > > - Michael > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From fritchie@REDACTED Wed Sep 23 21:53:12 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 23 Sep 2009 14:53:12 -0500 Subject: [erlang-questions] calling erlang functions from php In-Reply-To: Message of "Wed, 23 Sep 2009 11:58:08 BST." <523869a70909230358tb2007c4u8d73cbfb84e7e2d0@mail.gmail.com> Message-ID: <99940.1253735592@snookles.snookles.com> Does anybody know of a PHP native implementation (or C bindings) for Joe Armstrong's UBF? As sous chef for the UBF protocol work that has been slowly simmering(*) at http://github.com/norton/ubf/, I would of course heartily recommend taking a look at UBF. Except for the PHP client part, maybe. :-) -Scott From twoggle@REDACTED Wed Sep 23 22:05:11 2009 From: twoggle@REDACTED (Tim Fletcher) Date: Wed, 23 Sep 2009 13:05:11 -0700 (PDT) Subject: Erlang/OTP R13B02 has been released In-Reply-To: <87ljk62fuu.fsf@sloppy.mti.ag> References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> <87ljk62fuu.fsf@sloppy.mti.ag> Message-ID: > The tracker http://www.erlang.org/stats/torrent.html shows the > torrents to be available here: I didn't know about that page, but it also has faulty links; some forward slashes missing. From tonyg@REDACTED Wed Sep 23 22:19:31 2009 From: tonyg@REDACTED (Tony Garnock-Jones) Date: Wed, 23 Sep 2009 21:19:31 +0100 Subject: UBF-A in various languages (was Re: [erlang-questions] calling erlang functions from php) In-Reply-To: <99940.1253735592@snookles.snookles.com> References: <99940.1253735592@snookles.snookles.com> Message-ID: <4ABA82D3.7090005@lshift.net> Hi, There is a C implementation of UBF-A as part of http://www.eighty-twenty.org/~tonyg/Darcs/ubf/. That's currently a darcs repo, but I could bring forward my planned upgrade to git if there's interest. Regards, Tony Scott Lystig Fritchie wrote: > Does anybody know of a PHP native implementation (or C bindings) for Joe > Armstrong's UBF? As sous chef for the UBF protocol work that has been > slowly simmering(*) at http://github.com/norton/ubf/, I would of course > heartily recommend taking a look at UBF. Except for the PHP client > part, maybe. :-) > > -Scott > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From john.erickson@REDACTED Wed Sep 23 22:40:23 2009 From: john.erickson@REDACTED (Erickson, John) Date: Wed, 23 Sep 2009 14:40:23 -0600 Subject: net_kernel garbage collection Message-ID: <3C44D46958109E408F5848593B05A1D8C5B1B483@rrsmsx505.amr.corp.intel.com> Hello, I have a program that often has several gb of garbage under a process which was started from net_kernel:spawn_func and usually runs erlang:bif_return_trap. If I manually run garbage collection on this process, it will free this memory, but I was wondering if there is anyway to reduce the amount of garbage that is generated or periodically run garbage collection automatically. John From mjtruog@REDACTED Wed Sep 23 22:55:30 2009 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 23 Sep 2009 13:55:30 -0700 Subject: [erlang-questions] net_kernel garbage collection In-Reply-To: <3C44D46958109E408F5848593B05A1D8C5B1B483@rrsmsx505.amr.corp.intel.com> References: <3C44D46958109E408F5848593B05A1D8C5B1B483@rrsmsx505.amr.corp.intel.com> Message-ID: <4ABA8B42.2000806@gmail.com> You need to spawn a short-lived process since the process death will trigger garbage collection. If you want to be forceful about it, you can do something like the code I have attached below. Erickson, John wrote: > Hello, I have a program that often has several gb of garbage under a process which was started from net_kernel:spawn_func and usually runs erlang:bif_return_trap. If I manually run garbage collection on this process, it will free this memory, but I was wondering if there is anyway to reduce the amount of garbage that is generated or periodically run garbage collection automatically. > > John > %%% -*- coding: utf-8; Mode: erlang; tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- %%% ex: set softtabstop=4 tabstop=4 shiftwidth=4 expandtab fileencoding=utf-8: %%% %%%------------------------------------------------------------------------ %%% @doc %%% Enforce immediate garbage collection on a function %%% @end %%% %%% BSD LICENSE %%% %%% Copyright (c) 2009, Michael Truog %%% All rights reserved. %%% %%% Redistribution and use in source and binary forms, with or without %%% modification, are permitted provided that the following conditions are met: %%% %%% * Redistributions of source code must retain the above copyright %%% notice, this list of conditions and the following disclaimer. %%% * Redistributions in binary form must reproduce the above copyright %%% notice, this list of conditions and the following disclaimer in %%% the documentation and/or other materials provided with the %%% distribution. %%% * All advertising materials mentioning features or use of this %%% software must display the following acknowledgment: %%% This product includes software developed by Michael Truog %%% * The name of the author may not be used to endorse or promote %%% products derived from this software without specific prior %%% written permission %%% %%% THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND %%% CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, %%% INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES %%% OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE %%% DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR %%% CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, %%% SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, %%% BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR %%% SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS %%% INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, %%% WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING %%% NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE %%% OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH %%% DAMAGE. %%% %%% @author Michael Truog %%% @copyright 2009 Michael Truog %%% @version 0.0.1 %%%------------------------------------------------------------------------ -module(immediate_gc). -author('mjtruog [at] gmail (dot) com'). %% external interface -export([sync_fun/2, sync_fun/3, async_fun/2, async_fun/3]). %%%------------------------------------------------------------------------ %%% External interface functions %%%------------------------------------------------------------------------ %%------------------------------------------------------------------------- %% @doc %% Make a synchronous call to a function that will be %% garbage collected when the function returns. %% @end %%------------------------------------------------------------------------- -spec sync_fun(M :: atom(), F :: atom(), A :: list()) -> any(). sync_fun(M, F, A) when is_atom(M), is_atom(F), is_list(A) -> Parent = self(), Child = erlang:spawn_opt(fun() -> Parent ! {self(), erlang:apply(M, F, A)}, erlang:garbage_collect() end, [link, {fullsweep_after, 0}]), receive {Child, Result} -> Result end. %%------------------------------------------------------------------------- %% @doc %% Make a synchronous call to an anonymous function that will be %% garbage collected when the function returns. %% @end %%------------------------------------------------------------------------- -spec sync_fun(F :: fun(), A :: list()) -> any(). sync_fun(F, A) when is_function(F), is_list(A) -> Parent = self(), Child = erlang:spawn_opt(fun() -> Parent ! {self(), erlang:apply(F, A)}, erlang:garbage_collect() end, [link, {fullsweep_after, 0}]), receive {Child, Result} -> Result end. %%------------------------------------------------------------------------- %% @doc %% Make a asynchronous call to a function that will be %% garbage collected when the function returns. %% @end %%------------------------------------------------------------------------- -spec async_fun(M :: atom(), F :: atom(), A :: list()) -> pid(). async_fun(M, F, A) when is_atom(M), is_atom(F), is_list(A) -> erlang:spawn_opt(fun() -> erlang:apply(M, F, A), erlang:garbage_collect() end, [link, {fullsweep_after, 0}]). %%------------------------------------------------------------------------- %% @doc %% Make a asynchronous call to an anonymous function that will be %% garbage collected when the function returns. %% @end %%------------------------------------------------------------------------- -spec async_fun(F :: fun(), A :: list()) -> pid(). async_fun(F, A) when is_function(F), is_list(A) -> erlang:spawn_opt(fun() -> erlang:apply(F, A), erlang:garbage_collect() end, [link, {fullsweep_after, 0}]). From john.erickson@REDACTED Wed Sep 23 22:59:09 2009 From: john.erickson@REDACTED (Erickson, John) Date: Wed, 23 Sep 2009 14:59:09 -0600 Subject: [erlang-questions] net_kernel garbage collection In-Reply-To: <4ABA8B42.2000806@gmail.com> References: <3C44D46958109E408F5848593B05A1D8C5B1B483@rrsmsx505.amr.corp.intel.com> <4ABA8B42.2000806@gmail.com> Message-ID: <3C44D46958109E408F5848593B05A1D8C5B1B4DD@rrsmsx505.amr.corp.intel.com> I was under the impression the garbage collection was per-process. If so, will the process death of some unrelated process somehow affect the net_kernel (erlang system) process? John -----Original Message----- From: Michael Truog [mailto:mjtruog@REDACTED] Sent: Wednesday, September 23, 2009 3:56 PM To: Erickson, John Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] net_kernel garbage collection You need to spawn a short-lived process since the process death will trigger garbage collection. If you want to be forceful about it, you can do something like the code I have attached below. Erickson, John wrote: > Hello, I have a program that often has several gb of garbage under a process which was started from net_kernel:spawn_func and usually runs erlang:bif_return_trap. If I manually run garbage collection on this process, it will free this memory, but I was wondering if there is anyway to reduce the amount of garbage that is generated or periodically run garbage collection automatically. > > John > %%% -*- coding: utf-8; Mode: erlang; tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- %%% ex: set softtabstop=4 tabstop=4 shiftwidth=4 expandtab fileencoding=utf-8: %%% %%%------------------------------------------------------------------------ %%% @doc %%% Enforce immediate garbage collection on a function %%% @end %%% %%% BSD LICENSE %%% %%% Copyright (c) 2009, Michael Truog %%% All rights reserved. %%% %%% Redistribution and use in source and binary forms, with or without %%% modification, are permitted provided that the following conditions are met: %%% %%% * Redistributions of source code must retain the above copyright %%% notice, this list of conditions and the following disclaimer. %%% * Redistributions in binary form must reproduce the above copyright %%% notice, this list of conditions and the following disclaimer in %%% the documentation and/or other materials provided with the %%% distribution. %%% * All advertising materials mentioning features or use of this %%% software must display the following acknowledgment: %%% This product includes software developed by Michael Truog %%% * The name of the author may not be used to endorse or promote %%% products derived from this software without specific prior %%% written permission %%% %%% THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND %%% CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, %%% INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES %%% OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE %%% DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR %%% CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, %%% SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, %%% BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR %%% SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS %%% INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, %%% WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING %%% NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE %%% OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH %%% DAMAGE. %%% %%% @author Michael Truog %%% @copyright 2009 Michael Truog %%% @version 0.0.1 %%%------------------------------------------------------------------------ -module(immediate_gc). -author('mjtruog [at] gmail (dot) com'). %% external interface -export([sync_fun/2, sync_fun/3, async_fun/2, async_fun/3]). %%%------------------------------------------------------------------------ %%% External interface functions %%%------------------------------------------------------------------------ %%------------------------------------------------------------------------- %% @doc %% Make a synchronous call to a function that will be %% garbage collected when the function returns. %% @end %%------------------------------------------------------------------------- -spec sync_fun(M :: atom(), F :: atom(), A :: list()) -> any(). sync_fun(M, F, A) when is_atom(M), is_atom(F), is_list(A) -> Parent = self(), Child = erlang:spawn_opt(fun() -> Parent ! {self(), erlang:apply(M, F, A)}, erlang:garbage_collect() end, [link, {fullsweep_after, 0}]), receive {Child, Result} -> Result end. %%------------------------------------------------------------------------- %% @doc %% Make a synchronous call to an anonymous function that will be %% garbage collected when the function returns. %% @end %%------------------------------------------------------------------------- -spec sync_fun(F :: fun(), A :: list()) -> any(). sync_fun(F, A) when is_function(F), is_list(A) -> Parent = self(), Child = erlang:spawn_opt(fun() -> Parent ! {self(), erlang:apply(F, A)}, erlang:garbage_collect() end, [link, {fullsweep_after, 0}]), receive {Child, Result} -> Result end. %%------------------------------------------------------------------------- %% @doc %% Make a asynchronous call to a function that will be %% garbage collected when the function returns. %% @end %%------------------------------------------------------------------------- -spec async_fun(M :: atom(), F :: atom(), A :: list()) -> pid(). async_fun(M, F, A) when is_atom(M), is_atom(F), is_list(A) -> erlang:spawn_opt(fun() -> erlang:apply(M, F, A), erlang:garbage_collect() end, [link, {fullsweep_after, 0}]). %%------------------------------------------------------------------------- %% @doc %% Make a asynchronous call to an anonymous function that will be %% garbage collected when the function returns. %% @end %%------------------------------------------------------------------------- -spec async_fun(F :: fun(), A :: list()) -> pid(). async_fun(F, A) when is_function(F), is_list(A) -> erlang:spawn_opt(fun() -> erlang:apply(F, A), erlang:garbage_collect() end, [link, {fullsweep_after, 0}]). From mjtruog@REDACTED Wed Sep 23 23:08:46 2009 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 23 Sep 2009 14:08:46 -0700 Subject: [erlang-questions] net_kernel garbage collection In-Reply-To: <3C44D46958109E408F5848593B05A1D8C5B1B4DD@rrsmsx505.amr.corp.intel.com> References: <3C44D46958109E408F5848593B05A1D8C5B1B483@rrsmsx505.amr.corp.intel.com> <4ABA8B42.2000806@gmail.com> <3C44D46958109E408F5848593B05A1D8C5B1B4DD@rrsmsx505.amr.corp.intel.com> Message-ID: <4ABA8E5E.4070508@gmail.com> I don't think so. You should be able to determine where the problem lies with the instrument module here: http://erlang.org/doc/man/instrument.html You just need to feed the options "+Mim true +Mis true" to the VM before you use it (in documentation). Erickson, John wrote: > I was under the impression the garbage collection was per-process. If so, will the process death of some unrelated process somehow affect the net_kernel (erlang system) process? > > John > > -----Original Message----- > From: Michael Truog [mailto:mjtruog@REDACTED] > Sent: Wednesday, September 23, 2009 3:56 PM > To: Erickson, John > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] net_kernel garbage collection > > You need to spawn a short-lived process since the process death will > trigger garbage collection. If you want to be forceful about it, you > can do something like the code I have attached below. > > Erickson, John wrote: > >> Hello, I have a program that often has several gb of garbage under a process which was started from net_kernel:spawn_func and usually runs erlang:bif_return_trap. If I manually run garbage collection on this process, it will free this memory, but I was wondering if there is anyway to reduce the amount of garbage that is generated or periodically run garbage collection automatically. >> >> John >> >> > %%% -*- coding: utf-8; Mode: erlang; tab-width: 4; c-basic-offset: 4; > indent-tabs-mode: nil -*- > %%% ex: set softtabstop=4 tabstop=4 shiftwidth=4 expandtab > fileencoding=utf-8: > %%% > %%%------------------------------------------------------------------------ > %%% @doc > %%% Enforce immediate garbage collection on a function > %%% @end > %%% > %%% BSD LICENSE > %%% > %%% Copyright (c) 2009, Michael Truog > %%% All rights reserved. > %%% > %%% Redistribution and use in source and binary forms, with or without > %%% modification, are permitted provided that the following conditions > are met: > %%% > %%% * Redistributions of source code must retain the above copyright > %%% notice, this list of conditions and the following disclaimer. > %%% * Redistributions in binary form must reproduce the above copyright > %%% notice, this list of conditions and the following disclaimer in > %%% the documentation and/or other materials provided with the > %%% distribution. > %%% * All advertising materials mentioning features or use of this > %%% software must display the following acknowledgment: > %%% This product includes software developed by Michael Truog > %%% * The name of the author may not be used to endorse or promote > %%% products derived from this software without specific prior > %%% written permission > %%% > %%% THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > %%% CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, > %%% INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES > %%% OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE > %%% DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR > %%% CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > %%% SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, > %%% BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR > %%% SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS > %%% INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, > %%% WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING > %%% NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > %%% OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > %%% DAMAGE. > %%% > %%% @author Michael Truog > %%% @copyright 2009 Michael Truog > %%% @version 0.0.1 > %%%------------------------------------------------------------------------ > > -module(immediate_gc). > -author('mjtruog [at] gmail (dot) com'). > > %% external interface > -export([sync_fun/2, sync_fun/3, > async_fun/2, async_fun/3]). > > %%%------------------------------------------------------------------------ > %%% External interface functions > %%%------------------------------------------------------------------------ > > %%------------------------------------------------------------------------- > %% @doc > %% Make a synchronous call to a function that will be > %% garbage collected when the function returns. > %% @end > %%------------------------------------------------------------------------- > > -spec sync_fun(M :: atom(), F :: atom(), A :: list()) -> any(). > > sync_fun(M, F, A) when is_atom(M), is_atom(F), is_list(A) -> > Parent = self(), > Child = erlang:spawn_opt(fun() -> > Parent ! {self(), erlang:apply(M, F, A)}, > erlang:garbage_collect() > end, [link, {fullsweep_after, 0}]), > receive > {Child, Result} -> Result > end. > > %%------------------------------------------------------------------------- > %% @doc > %% Make a synchronous call to an anonymous function that will be > %% garbage collected when the function returns. > %% @end > %%------------------------------------------------------------------------- > > -spec sync_fun(F :: fun(), A :: list()) -> any(). > > sync_fun(F, A) when is_function(F), is_list(A) -> > Parent = self(), > Child = erlang:spawn_opt(fun() -> > Parent ! {self(), erlang:apply(F, A)}, > erlang:garbage_collect() > end, [link, {fullsweep_after, 0}]), > receive > {Child, Result} -> Result > end. > > %%------------------------------------------------------------------------- > %% @doc > %% Make a asynchronous call to a function that will be > %% garbage collected when the function returns. > %% @end > %%------------------------------------------------------------------------- > > -spec async_fun(M :: atom(), F :: atom(), A :: list()) -> pid(). > > async_fun(M, F, A) when is_atom(M), is_atom(F), is_list(A) -> > erlang:spawn_opt(fun() -> > erlang:apply(M, F, A), > erlang:garbage_collect() > end, [link, {fullsweep_after, 0}]). > > %%------------------------------------------------------------------------- > %% @doc > %% Make a asynchronous call to an anonymous function that will be > %% garbage collected when the function returns. > %% @end > %%------------------------------------------------------------------------- > > -spec async_fun(F :: fun(), A :: list()) -> pid(). > > async_fun(F, A) when is_function(F), is_list(A) -> > erlang:spawn_opt(fun() -> > erlang:apply(F, A), > erlang:garbage_collect() > end, [link, {fullsweep_after, 0}]). > > > > From andrew@REDACTED Thu Sep 24 04:21:43 2009 From: andrew@REDACTED (Andrew Thompson) Date: Wed, 23 Sep 2009 22:21:43 -0400 Subject: EEP 9 and binary strings Message-ID: <20090924022143.GK7677@hijacked.us> I'm wondering about the status of EEP 9 and the concept of binary strings. Did this EEP just die, does everyone just use the re module for dealing with them now? It seems pretty easy to do the basic string stuff with binary strings (I spent a few minutes and implemented strpos, strrpos, strchr, strrchr and substr) but am I reinventing the wheel needlessly? Most of the stuff I want to do doesn't need regular expressions but should I just use them anyway? Andrew From jwebb@REDACTED Thu Sep 24 06:38:49 2009 From: jwebb@REDACTED (John Webb) Date: Thu, 24 Sep 2009 13:38:49 +0900 Subject: [erlang-questions] Re: wxErlang, Mac OS X and Input Methods (was Re: [erlang-questions] Does Wx need smp??) In-Reply-To: <4A261EF2.1090304@erix.ericsson.se> References: <200905260026.24902.clist@uah.es> <73FE16B0EF85C2478ABCF13D16D7608BDDD5E6@X-EVS-02.uah.es> <4A2368ED.8070109@erix.ericsson.se> <200906021830.34474.clist@uah.es> <3E7F5F40-D938-4A48-814B-85ADCB5A1209@gol.com> <4A261EF2.1090304@erix.ericsson.se> Message-ID: <5B891F8F-2A45-4363-9037-2436A66656EA@gol.com> I have just compiled and installed Erlang/OTP R13B02 on Mac OS 10.6.1 and am happy to report that the issue described below is resolved. (Both wxWidgets 2.8.8 included in the OS distribution and a hand built 2.8.10 were fine.) As I no longer have Mac OS 10.5 installed I can't tell whether the new wxErlang init code or the OS upgrade (or both) fixed the problem but it is nice to have everything working. Many Thanks! John Webb On Jun 3, 2009, at 3:57 PM, Dan Gudmundsson wrote: > > Hi and thanks for the bug report. > > Getting the driver to connect to the window system on the mac > without actually having a real gui application is a real hack, > that someone (name escapes me) contributed to esdl. > > The solution might be a werl for the mac. > > But my mac skills (and my time) is rather limited so don't hold > your breath. > > /Dan > > John Webb wrote: >> Hi Dan, >> I have run into a problem which I think is related to this thread. >> I have compiled and installed Erlang/OTP 13B linked against >> wxWidgets 2.8.10 on Mac OS X 10.5.7. I also have the OS's Japanese >> input method enabled. >> The wxErlang examples work fine when direct keyboard input >> (English) is selected. However when a Japanese input method mode is >> selected (eg in order to type Japanese characters into a text >> widget) the mac's spinning rainbow cursor appears and the GUI app >> grinds to an unresponsive halt. >> The console repeats a number of error messages until the emulator >> is interrupted (see below). >> It seems that the application (client) tries to connect to the >> input method (server) and fails. The second error then indicates >> some consternation that this didn't happen in the main thread! >> Apple documentation is sparse in this area but I think that, from >> the client side, interaction with an input method via text services >> should be transparent. The wxWidget examples do not exhibit this >> problem which suggests an implementation problem (on Mac OS X at >> least). >> Perhaps there is a work around for the driver implementation (I >> would be interested to know if anyone else has experienced this >> problem on other platforms or other input methods). On the other >> hand I would put this as one case for implementing the GUI as a >> separate os process (a build option perhaps?) communicating with >> erlang via sockets. >> I like wxWidgets as a GUI for erlang and a lot of work has gone >> into wxErlang. If this does prove to be a problem not just for my >> machine then it would be a great shame if it were not available on >> each of the main platforms for each locale. >> Best Regards, >> John Webb >> Console Output: >> +++++++++++++++++ >> 2> menu:start(). >> 2009-05-27 12:39:03.286 beam.smp[7096:3507] IMKClient: exception >> caught trying to get the rootProxy for connection ((** NSConnection >> 0x3fb420 receivePort sendPort > 0x3fb120> refCount 7 **)): NSPortTimeoutException : connection >> timeout: did not receive reply >> 2009-05-27 12:39:03.288 beam.smp[7096:3507] This exception >> was thrown inside thread: {name = (null), num = >> 2}. This thread is NOT the main thread. >> 2009-05-27 12:39:08.700 beam.smp[7096:3507] IMKClient: exception >> caught trying to get the rootProxy for connection ((** NSConnection >> 0x17003980 receivePort sendPort > 0x3fb120> refCount 7 **)): NSPortTimeoutException : connection >> timeout: did not receive reply >> 2009-05-27 12:39:08.701 beam.smp[7096:3507] This exception >> was thrown inside thread: {name = (null), num = >> 2}. This thread is NOT the main thread. >> 2009-05-27 12:39:08.702 beam.smp[7096:3507] ****** Returning nil >> _server ********** >> : >> : >> (repeats 'ad infinitum', well 'ad ^G' ) >> +++++++++++++++++ >> On Jun 3, 2009, at 1:30 AM, Angel wrote: >>> El Lunes, 1 de Junio de 2009 07:36:45 Dan Gudmundsson escribi?: >>>> The reason was speed, as it is an object oriented api I thought >>>> there >>>> would be a lot of queries to the driver/port program. >>>> >>>> A GS was bashed a lot previously so I thought that we make it as >>>> fast as possible >>>> from the beginning. >>>> >>>> Maybe that wasn't the best decision? >>>> >>>> /Dan >>> >>> Well, from the novice point of view, I have seen many times in >>> this list that raw speed is a very questionable concept >>> because being fast doesnt mean scale better. Erlang itself is very >>> devoted to scale causing confusi?n and horror >>> to people on the first time when you see so much copying to allow >>> nthe non destructive model and mesage passing. >>> >>> For one scenario (a single nodel) a built-in driver maybe optimal, >>> maybe not, Windows seems to be the case here for >>> speed (most stuff in kernel) whereas X in the oposite shows you >>> can make GUI over a wire protocol and let nasty things >>> (as masive opengl and the like transfers) to pluging or extensions >>> (XVIDEO...) where tricky things can be done (IPC via >>> shared segments and other things). >>> >>> >>> The whole system often is so much complex to be achieved using >>> only one paradigm (funtional, process, threads, oop ...) >>> >>> Despite those things wxerlang seems pretty interesting and a very >>> good stating point for erlang going into the desktop.. >>> >>> /Angel >>>> >>>> clist wrote: >>>>> well smp is nearly mainsteam today i know. >>>>> >>>>> yet i still dont know what was the key on becoming a driver from >>>>> a port program as early >>>>> wxerlang (thesis code) seemed to be standalone >>>>> >>>>> /angel >>>>> >>>>> ________________________________ >>>>> >>>>> De: Dan Gudmundsson [mailto:dgud@REDACTED] >>>>> Enviado el: vie 29/05/2009 10:52 >>>>> Para: clist >>>>> CC: erlang-questions@REDACTED >>>>> Asunto: Re: [erlang-questions] Does Wx need smp?? >>>>> >>>>> >>>>> >>>>> wxWidgets must run in it's own thread, and the functions used in >>>>> erlang driver interface >>>>> to communicate with the emulator is only thread safe in an smp >>>>> emulator. >>>>> >>>>> With some work you could wrap that functionality, send it to the >>>>> erlang thread first and >>>>> call the functions from the emulator thread on an non smp >>>>> enabled erlang. >>>>> >>>>> But that is work that I don't have time with and nowadays smp >>>>> machines are everywhere, >>>>> e.g. my 6 year old PC is hyperthreaded and seen as an smp machine. >>>>> >>>>> /Dan >>>>> >>>>> Angel Alvarez wrote: >>>>>> El Martes, 26 de Mayo de 2009 Steve Davis escribi?: >>>>>>> Hello Angel, >>>>>>> >>>>>>> Yep, wx does require SMP, the reason being that the underlying >>>>>>> (cpp) >>>>>>> library code is required to run in a separate OS thread. >>>>>>> >>>>>>> The erl flag you want is -smp enable. If running under windows >>>>>>> you >>>>>>> must use werl not erl. >>>>>>> >>>>>>> To be fair, after taking a quick look at the official docs >>>>>>> (both user >>>>>>> guide and ref manual), this requirement is far from clear. >>>>>>> >>>>>>> /s >>>>>> Sorry, my fault! >>>>>> >>>>>> Niko (the packager whose rpm i picked up ) told me that, i >>>>>> wanted to just to probe new features >>>>>> so basically i ignored the docs....(like a good newbie :-) ) >>>>>> >>>>>> IMHO this is not very elegant as the purity of process >>>>>> isolation is gone. >>>>>> >>>>>> Was performance the key on introducing such a complex driver? >>>>>> im very new >>>>>> to just make this judgement but for the whole thing (minus >>>>>> opengl and the like) >>>>>> it seems enough a stdin/pipe interface than making a big driver. >>>>>> >>>>>> Perhaps memory conservation is another issue (well is erlang VM >>>>>> better at managing >>>>>> memory than a full blow C++ aplication??? ). >>>>>> >>>>>> Just checking >>>>>> http://apps.sourceforge.net/mediawiki/wxerlang/index.php?title=Memory_management >>>>>> >>>>>> shows a bif efort to catch deleted objects and try to be in >>>>>> sync with the pure erlang side... >>>>>> >>>>>> >>>>>> its not criticism, but mere curiosity, design questions are >>>>>> very educative. >>>>>> >>>>>> Regards, Angel >>>>>> >>>>>> PS: Ive just found http://wxerlang.sourceforge.net/docs/Report.pdf >>>>>> seems very promising for insigths..! >>>>> >>>>> >>>> >>> >>> >>> >>> -- >>> Este correo no tiene dibujos. Las formas extra?as en la pantalla >>> son letras. >>> __________________________________________ >>> >>> Clist UAH a.k.a Angel >>> __________________________________________ >>> Te has metido en un hipoteca de 70M y encima te roban una panda de >>> Ninjas... Crisis Ninja para Dummies. >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From akhil.kodali@REDACTED Thu Sep 24 07:36:15 2009 From: akhil.kodali@REDACTED (Akhil Kodali) Date: Thu, 24 Sep 2009 11:06:15 +0530 Subject: erlang web / rest services Message-ID: Hi, Is there any way or framework by which i can expose erlang services as web or rest services. The only reference i could find was by Steve Vinoski at InfoQ. Akhil From kamiseq@REDACTED Thu Sep 24 08:35:59 2009 From: kamiseq@REDACTED (kamiseq) Date: Thu, 24 Sep 2009 08:35:59 +0200 Subject: [erlang-questions] erlang web / rest services In-Reply-To: References: Message-ID: you mean something like this http://www.erlang-web.org/ ? pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From taavi@REDACTED Thu Sep 24 08:05:11 2009 From: taavi@REDACTED (Taavi Talvik) Date: Thu, 24 Sep 2009 09:05:11 +0300 Subject: [erlang-questions] erlang web / rest services In-Reply-To: References: Message-ID: Hello! I can reccomend webmachine: http://bitbucket.org/justin/webmachine/wiki/Home best regards, taavi On Sep 24, 2009, at 8:36 AM, Akhil Kodali wrote: > Hi, > > Is there any way or framework by which i can expose erlang services > as web > or rest services. > The only reference i could find was by Steve Vinoski at InfoQ. > > Akhil From raimo+erlang-questions@REDACTED Thu Sep 24 09:16:07 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 24 Sep 2009 09:16:07 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname In-Reply-To: <200909231631349711641@its3.ch> References: <200909231631349711641@its3.ch> Message-ID: <20090924071607.GA27757@erix.ericsson.se> On Wed, Sep 23, 2009 at 04:31:35PM +0200, info wrote: > Hi all, > On a windows 2003 server sbs this command returns {ok,"my_host"} : > > {ok,N}=inet:gethostname(). > > But the following command returns {error,timeout} : > > inet:gethostbyname(N). > > On a Vista machine, the previous command returns a correct answer. > > I know that it is not an erlang problem but if you could give me support, I will appreciate ;-) Erlang may have problems configuring name lookups. Give me the output from ets:tab2list(inet_db). Which erlang release are you running, it sounds like pre-R13B02? Try starting with erl -name foo and do the same, ets:tab2list/1 too. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Thu Sep 24 09:19:51 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 24 Sep 2009 09:19:51 +0200 Subject: [erlang-questions] Re: Erlang/OTP R13B02 has been released In-Reply-To: References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> <87ljk62fuu.fsf@sloppy.mti.ag> Message-ID: <20090924071951.GB27757@erix.ericsson.se> On Wed, Sep 23, 2009 at 01:05:11PM -0700, Tim Fletcher wrote: > > The tracker http://www.erlang.org/stats/torrent.html shows the > > torrents to be available here: > > I didn't know about that page, but it also has faulty links; some > forward slashes missing. Give an example, so I can fix it. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From twoggle@REDACTED Thu Sep 24 10:27:46 2009 From: twoggle@REDACTED (Tim Fletcher) Date: Thu, 24 Sep 2009 09:27:46 +0100 Subject: [erlang-questions] Re: Erlang/OTP R13B02 has been released In-Reply-To: <20090924071951.GB27757@erix.ericsson.se> References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> <87ljk62fuu.fsf@sloppy.mti.ag> <20090924071951.GB27757@erix.ericsson.se> Message-ID: 2009/9/24 Raimo Niskanen : > On Wed, Sep 23, 2009 at 01:05:11PM -0700, Tim Fletcher wrote: >> >> I didn't know about that page, but it also has faulty links; some >> forward slashes missing. > > Give an example, so I can fix it. The href of every link on http://www.erlang.org/stats/torrent.html begins with "http:/download/torrents", which i don't think is correct syntax. Whilst Firefox corrects for it (and the links work), Safari (and i presume more browsers) don't. Either "/download/torrents" or "http://erlang.org/download/torrents" should fix it. From raimo+erlang-questions@REDACTED Thu Sep 24 10:46:12 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 24 Sep 2009 10:46:12 +0200 Subject: [erlang-questions] Re: Erlang/OTP R13B02 has been released In-Reply-To: References: <6672d0160909220707h2823ebc4s95d8868bcb51e39d@mail.gmail.com> <87ljk62fuu.fsf@sloppy.mti.ag> <20090924071951.GB27757@erix.ericsson.se> Message-ID: <20090924084612.GA30396@erix.ericsson.se> On Thu, Sep 24, 2009 at 09:27:46AM +0100, Tim Fletcher wrote: > 2009/9/24 Raimo Niskanen : > > On Wed, Sep 23, 2009 at 01:05:11PM -0700, Tim Fletcher wrote: > >> > >> I didn't know about that page, but it also has faulty links; some > >> forward slashes missing. > > > > Give an example, so I can fix it. > > The href of every link on http://www.erlang.org/stats/torrent.html > begins with "http:/download/torrents", which i don't think is correct > syntax. Whilst Firefox corrects for it (and the links work), Safari > (and i presume more browsers) don't. Either "/download/torrents" or > "http://erlang.org/download/torrents" should fix it. Well well, what do you know... I just looked at the links by hovering the mouse pointer in Firefox, and it kindly adjusted the faulty link to look fine... You learn something new every day. Thank you for pointing this out! It will soon be fixed. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From bflatmaj7th@REDACTED Thu Sep 24 11:47:54 2009 From: bflatmaj7th@REDACTED (Richard Andrews) Date: Thu, 24 Sep 2009 19:47:54 +1000 Subject: [erlang-questions] blowfish ECB implementation in erlang? In-Reply-To: <38ff4e6b0909231233i4435a436v170a8b1953b22c4e@mail.gmail.com> References: <38ff4e6b0909231233i4435a436v170a8b1953b22c4e@mail.gmail.com> Message-ID: <7702c0610909240247q40be3ea6n602d99969731239a@mail.gmail.com> On Thu, Sep 24, 2009 at 5:33 AM, Larry Ogrodnek wrote: > Hi, I'm wondering if anyone has put together an implementation of blowfish ECB? No. But I did one for AES256 not long ago. The pattern in the crypto driver is pretty simple to extend. From info@REDACTED Thu Sep 24 11:56:48 2009 From: info@REDACTED (info) Date: Thu, 24 Sep 2009 11:56:48 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname References: <200909231631349711641@its3.ch>, <20090924071607.GA27757@erix.ericsson.se> Message-ID: <200909241156480367442@its3.ch> Hi ! Thank you for your help ! Here is what you want : [{socks5_noproxy,[]}, {res_recurse,1}, {res_id,0}, {hostname,"my_host"}, {res_alt_ns,[]}, {cache_refresh_interval,3600000}, {res_inet6,false}, {socks5_port,1080}, {socks5_methods,[none]}, {res_timeout,2000}, {cache_size,100}, {res_domain,[]}, {res_retry,3}, {res_ns,[]}, {res_lookup,[native,file]}, {sctp_module,inet_sctp}, {udp_module,inet_udp}, {tcp_module,inet_tcp}, {socks5_server,[]}, {res_search,[]}, {res_usevc,false}] I am running R13B01 On Wed, Sep 23, 2009 at 04:31:35PM +0200, info wrote: > Hi all, > On a windows 2003 server sbs this command returns {ok,"my_host"} : > > {ok,N}=inet:gethostname(). > > But the following command returns {error,timeout} : > > inet:gethostbyname(N). > > On a Vista machine, the previous command returns a correct answer. > > I know that it is not an erlang problem but if you could give me support, I will appreciate ;-) Erlang may have problems configuring name lookups. Give me the output from ets:tab2list(inet_db). Which erlang release are you running, it sounds like pre-R13B02? Try starting with erl -name foo and do the same, ets:tab2list/1 too. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From kevin@REDACTED Thu Sep 24 12:21:45 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Thu, 24 Sep 2009 06:21:45 -0400 Subject: [erlang-questions] erlang web / rest services In-Reply-To: References: Message-ID: +1000 to webmachine. I've used on several projects and in some of my training classes. It's a great framework. --Kevin On Sep 24, 2009, at 2:05 AM, Taavi Talvik wrote: > Hello! > > I can reccomend webmachine: > > http://bitbucket.org/justin/webmachine/wiki/Home > > best regards, > taavi > > On Sep 24, 2009, at 8:36 AM, Akhil Kodali wrote: > >> Hi, >> >> Is there any way or framework by which i can expose erlang services >> as web >> or rest services. >> The only reference i could find was by Steve Vinoski at InfoQ. >> >> Akhil > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From michal.ptaszek@REDACTED Thu Sep 24 12:32:23 2009 From: michal.ptaszek@REDACTED (Michal Ptaszek) Date: Thu, 24 Sep 2009 11:32:23 +0100 (BST) Subject: [erlang-questions] erlang web / rest services In-Reply-To: Message-ID: <22835509.159291253788343206.JavaMail.root@zimbra> Hi Akhil, I would recommend you Erlang Web: it should everything you will need for accessing the backend services. See Erlang Web's wiki page for details: http://wiki.erlang-web.org ----- "Kevin A. Smith" wrote: > +1000 to webmachine. I've used on several projects and in some of my > > training classes. It's a great framework. > > --Kevin > On Sep 24, 2009, at 2:05 AM, Taavi Talvik wrote: > > > Hello! > > > > I can reccomend webmachine: > > > > http://bitbucket.org/justin/webmachine/wiki/Home > > > > best regards, > > taavi > > > > On Sep 24, 2009, at 8:36 AM, Akhil Kodali wrote: > > > >> Hi, > >> > >> Is there any way or framework by which i can expose erlang services > > >> as web > >> or rest services. > >> The only reference i could find was by Steve Vinoski at InfoQ. > >> > >> Akhil > > > > -- Michal Ptaszek www.erlang-consulting.com From akhil.kodali@REDACTED Thu Sep 24 13:23:45 2009 From: akhil.kodali@REDACTED (Akhil Kodali) Date: Thu, 24 Sep 2009 16:53:45 +0530 Subject: [erlang-questions] erlang web / rest services In-Reply-To: <22835509.159291253788343206.JavaMail.root@zimbra> References: <22835509.159291253788343206.JavaMail.root@zimbra> Message-ID: Hi, Thank you for the input...will try both webmachine and erlang-web out. ---- Regards, Akhil Kodali On Thu, Sep 24, 2009 at 4:02 PM, Michal Ptaszek < michal.ptaszek@REDACTED> wrote: > Hi Akhil, > > I would recommend you Erlang Web: it should everything > you will need for accessing the backend services. > > See Erlang Web's wiki page for details: > http://wiki.erlang-web.org > > ----- "Kevin A. Smith" wrote: > > > +1000 to webmachine. I've used on several projects and in some of my > > > > training classes. It's a great framework. > > > > --Kevin > > On Sep 24, 2009, at 2:05 AM, Taavi Talvik wrote: > > > > > Hello! > > > > > > I can reccomend webmachine: > > > > > > http://bitbucket.org/justin/webmachine/wiki/Home > > > > > > best regards, > > > taavi > > > > > > On Sep 24, 2009, at 8:36 AM, Akhil Kodali wrote: > > > > > >> Hi, > > >> > > >> Is there any way or framework by which i can expose erlang services > > > > >> as web > > >> or rest services. > > >> The only reference i could find was by Steve Vinoski at InfoQ. > > >> > > >> Akhil > > > > > > > > -- > Michal Ptaszek > www.erlang-consulting.com > From hawk.mattsson@REDACTED Thu Sep 24 13:39:10 2009 From: hawk.mattsson@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Thu, 24 Sep 2009 13:39:10 +0200 Subject: Erlang/OTP R13B02-1 has been released In-Reply-To: <922d05850909230746k332c97c0s137aeb3eb11d577a@mail.gmail.com> References: <922d05850909230719k5592e211ye51db32072a00b86@mail.gmail.com> <922d05850909230746k332c97c0s137aeb3eb11d577a@mail.gmail.com> Message-ID: <922d05850909240439j7ecb9f4bleec8e3ac368a59c6@mail.gmail.com> Now it is available for Windows. /H?kan 2009/9/23 H?kan Mattsson : > The Windows exe is not yet available, but it will be released soon. > > /H?kan > --- > H?kan Mattsson (uabhams) > Erlang/OTP, Ericsson AB > > 2009/9/23 H?kan Mattsson : >> Bug fix release ? : otp_src_R13B02-1 >> Build date ? ? ? ? ?: 2009-09-21 >> Reltool build date: 2009-09-23 >> >> This is R13B02-1, a patched version of R13B02 which was the second >> maintenance release for the R13B major release. R13B02-1 has the >> same contents as R13B02 with the exception of Reltool. In the new >> version of Reltool (0.5.1) this nasty bug has been fixed: >> >> ? OTP-8199 In the new release of Reltool (0.5) there is a severe bug >> ? ? ? ? ? ? ? ? ? that may cause your source files to be deleted. The bug is >> ? ? ? ? ? ? ? ? ? triggered when you generate a target system WITH archive >> ? ? ? ? ? ? ? ? ? files AND your library directories are given with absolute >> ? ? ? ? ? ? ? ? ? paths. (Library directories are given with the ERL_LIBS >> ? ? ? ? ? ? ? ? ? environment variable or with the lib_dirs configuration >> ? ? ? ? ? ? ? ? ? parameter.) >> >> You can find the README file for the R13B02-1 release at >> >> ?http://www.erlang.org/download/otp_src_R13B02-1.readme >> >> The source distribution and binary distribution for Windows can be >> downloaded from >> >> ?http://www.erlang.org/download/otp_src_R13B02-1.tar.gz >> ?http://www.erlang.org/download/otp_win32_R13B02-1.exe >> >> The distribution can also be downloaded using the BitTorrent >> protocol. Use the following torrent files to download the source >> distribution and binary distribution for Windows: >> >> ?http://www.erlang.org/download/otp_src_R13B02-1.tar.gz.torrent >> ?http://www.erlang.org/download/otp_win32_R13B02-1.exe.torrent >> >> Note: To unpack the TAR archive you need a GNU TAR compatible program. >> >> For installation instructions please read the README file that is part >> of the distribution. >> >> The on-line documentation can be found at: http://www.erlang.org/doc/ >> You can also download the complete HTML documentation or the Unix manual files >> >> ?http://www.erlang.org/download/otp_doc_html_R13B02-1.tar.gz >> ?http://www.erlang.org/download/otp_doc_man_R13B02-1.tar.gz >> >> We also want to thank those that sent us patches, suggestions and bug >> reports, >> >> The OTP Team >> >> --- >> H?kan Mattsson (uabhams) >> Erlang/OTP, Ericsson AB From w.a.de.jong@REDACTED Thu Sep 24 13:46:01 2009 From: w.a.de.jong@REDACTED (Willem de Jong) Date: Thu, 24 Sep 2009 13:46:01 +0200 Subject: [erlang-questions] erlang web / rest services In-Reply-To: References: <22835509.159291253788343206.JavaMail.root@zimbra> Message-ID: <407d9ef80909240446k6d5ebecv3f95626307a1878@mail.gmail.com> Hi Akhil, One more suggestion: you might try to use the SOAP module that comes with Yaws. See http://yaws.hyber.org/soap_intro.yaws. Regards, Willem On Thu, Sep 24, 2009 at 1:23 PM, Akhil Kodali wrote: > Hi, > > Thank you for the input...will try both webmachine and erlang-web out. > > ---- > Regards, > Akhil Kodali > > > > On Thu, Sep 24, 2009 at 4:02 PM, Michal Ptaszek < > michal.ptaszek@REDACTED> wrote: > > > Hi Akhil, > > > > I would recommend you Erlang Web: it should everything > > you will need for accessing the backend services. > > > > See Erlang Web's wiki page for details: > > http://wiki.erlang-web.org > > > > ----- "Kevin A. Smith" wrote: > > > > > +1000 to webmachine. I've used on several projects and in some of my > > > > > > training classes. It's a great framework. > > > > > > --Kevin > > > On Sep 24, 2009, at 2:05 AM, Taavi Talvik wrote: > > > > > > > Hello! > > > > > > > > I can reccomend webmachine: > > > > > > > > http://bitbucket.org/justin/webmachine/wiki/Home > > > > > > > > best regards, > > > > taavi > > > > > > > > On Sep 24, 2009, at 8:36 AM, Akhil Kodali wrote: > > > > > > > >> Hi, > > > >> > > > >> Is there any way or framework by which i can expose erlang services > > > > > > >> as web > > > >> or rest services. > > > >> The only reference i could find was by Steve Vinoski at InfoQ. > > > >> > > > >> Akhil > > > > > > > > > > > > -- > > Michal Ptaszek > > www.erlang-consulting.com > > > From gleber.p@REDACTED Thu Sep 24 13:59:54 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 24 Sep 2009 13:59:54 +0200 Subject: [erlang-questions] erlang web / rest services In-Reply-To: References: <22835509.159291253788343206.JavaMail.root@zimbra> Message-ID: <14f0e3620909240459t4dc5910bq8a87ff322c025738@mail.gmail.com> Hi If you want a bit lower level approach you could try using Mochiweb [1]. It is a almost bare-bone HTTP web server. It is very easy to use and contains no unnecessary fuss. 1: http://code.google.com/p/mochiweb/ Best, Gleb Peregud On Thu, Sep 24, 2009 at 13:23, Akhil Kodali wrote: > Hi, > > Thank you for the input...will try both webmachine and erlang-web out. > > ---- > Regards, > Akhil Kodali > > > > On Thu, Sep 24, 2009 at 4:02 PM, Michal Ptaszek < > michal.ptaszek@REDACTED> wrote: > >> Hi Akhil, >> >> I would recommend you Erlang Web: it should everything >> you will need for accessing the backend services. >> >> See Erlang Web's wiki page for details: >> http://wiki.erlang-web.org >> >> ----- "Kevin A. Smith" wrote: >> >> > +1000 to webmachine. I've used on several projects and in some of my >> > >> > training classes. It's a great framework. >> > >> > --Kevin >> > On Sep 24, 2009, at 2:05 AM, Taavi Talvik wrote: >> > >> > > Hello! >> > > >> > > I can reccomend webmachine: >> > > >> > > http://bitbucket.org/justin/webmachine/wiki/Home >> > > >> > > best regards, >> > > taavi >> > > >> > > On Sep 24, 2009, at 8:36 AM, Akhil Kodali wrote: >> > > >> > >> Hi, >> > >> >> > >> Is there any way or framework by which i can expose erlang services >> > >> > >> as web >> > >> or rest services. >> > >> The only reference i could find was by Steve Vinoski at InfoQ. >> > >> >> > >> Akhil >> > > >> > > >> >> -- >> Michal Ptaszek >> www.erlang-consulting.com >> > From dave.pawson@REDACTED Thu Sep 24 14:43:46 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Thu, 24 Sep 2009 13:43:46 +0100 Subject: [erlang-questions] erlang web / rest services In-Reply-To: <407d9ef80909240446k6d5ebecv3f95626307a1878@mail.gmail.com> References: <22835509.159291253788343206.JavaMail.root@zimbra> <407d9ef80909240446k6d5ebecv3f95626307a1878@mail.gmail.com> Message-ID: <711a73df0909240543y2c5a2fbr424682c08b310b20@mail.gmail.com> 2009/9/24 Willem de Jong : > Hi Akhil, > > One more suggestion: you might try to use the SOAP module that comes with > Yaws. See http://yaws.hyber.org/soap_intro.yaws. > > Regards, > Willem I thought the OP wanted RESTful? Soap is anything but? regards -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From ulf.wiger@REDACTED Thu Sep 24 14:46:14 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 24 Sep 2009 14:46:14 +0200 Subject: how to document experimental status (Re: [erlang-questions] Erlang/OTP R13B02-1 has been released) In-Reply-To: <922d05850909230719k5592e211ye51db32072a00b86@mail.gmail.com> References: <922d05850909230719k5592e211ye51db32072a00b86@mail.gmail.com> Message-ID: <4ABB6A16.1080003@erlang-consulting.com> H?kan Mattsson wrote: > Bug fix release : otp_src_R13B02-1 > Build date : 2009-09-21 > Reltool build date: 2009-09-23 > > This is R13B02-1, a patched version of R13B02 which was the second > maintenance release for the R13B major release. R13B02-1 has the > same contents as R13B02 with the exception of Reltool. In the new > version of Reltool (0.5.1) this nasty bug has been fixed: This problem has stirred some discussion about whether the quality of the releases would be improved by a more open process. I don't want to stir that pot even more - but... Reltool is experimental. While the version number does give that away, and it says so in the release notes, shouldn't this also be written in highlight, in both the user guide and the reference manual? I really had to search for it in the docs, and even started doubting my memory for a while. I think anyone who missed it may be excused for being somewhat annoyed. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From hakan@REDACTED Thu Sep 24 14:54:00 2009 From: hakan@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Thu, 24 Sep 2009 14:54:00 +0200 Subject: [erlang-questions] packaging erlang application for ubuntu In-Reply-To: <792758.38980.qm@web31806.mail.mud.yahoo.com> References: <792758.38980.qm@web31806.mail.mud.yahoo.com> Message-ID: <922d05850909240553p4c3d131ag35171551eeda26c1@mail.gmail.com> On Mon, Sep 21, 2009 at 2:12 AM, Pablo Platt wrote: > Hi, > > I have an erlang application that I want to distribute to clients using ubuntu. > I don't want to include the source code, only the compiled beam files. > The installation needs to be automatic, install all the dependencies and > create a boot script that will run the application as daemon on startup. > > Is reltools fits this task? > Does an application release includes the erlang runtime so copying this folder > will be sufficient to run the application? Yes, you can use reltool for this. As you can see below, the standalone target system is self contained with an escript and a complete runtime system. The standalone system is relocatable and can be moved to a new location without the need to run any installation command. If you use the 'rel' parameter you can specify which applications you want to start without bothering about their version numbers. If you want to have a target system that a system with a couple of applications you can use the 'rel' parameter like this to generate a customized .rel file: 12 > reltool:get_rel([{sys, [{rel, "start_clean", "1.0", [kernel, stdlib, sasl, mnesia]}]}], "start_clean"). {ok,{release,{"start_clean","1.0"}, {erts,"5.7.3"}, [{kernel,"2.13.3"},{stdlib,"1.16.3"},{sasl,"2.1.7"},{mnesia,"4.4.11"}]}} Here I have an example of how you can generate a small standalone target system with reltool. Eflex is a small graphical tool which is started with an escript that runs in the background with the -detached flag (see the attached escript). The escript is invoked with ".../eflex2/bin/eflex". To start with I configure a simple standalone target system (commands 1..4). In the printouts from the shell you can see which files that the target system will consist of. Then I add a few configuration parameters to strip away stuff that I do not use. Such as a few applications (hipe, erts, crypto, syntax_tools) and some executables (epmd, heart, beam). In the generated target system the applications are packaged in archive files. But as wx has a driver, its priv directory resides outside the archive. /H?kan Erlang R13B02 (erts-5.7.3) [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-p oll:false] Eshell V5.7.3 (abort with ^G) 1> Sys = {sys, [{lib_dirs, ["/home/hakan/erl_libs"]}, {escript, "/home/hakan/bin/eflex", [{incl_cond, include}]}, {profile, standalone}]}, ok. ok 2> file:make_dir("eflex"). ok 3> reltool:create_target([Sys], "eflex"). ok. 4> io:format("~s\n", [os:cmd("find eflex")]). eflex eflex/bin eflex/bin/erl eflex/bin/epmd eflex/bin/eflex.escript eflex/bin/eflex eflex/bin/start.boot eflex/bin/start_clean.boot eflex/lib eflex/lib/hipe-3.7.3.ez eflex/lib/kernel-2.13.3.ez eflex/lib/crypto-1.6.1.ez eflex/lib/xmerl-1.2.1.ez eflex/lib/eflex-0.1.ez eflex/lib/erts-5.7.3.ez eflex/lib/stdlib-1.16.3.ez eflex/lib/compiler-4.6.3.ez eflex/lib/crypto-1.6.1 eflex/lib/crypto-1.6.1/priv eflex/lib/crypto-1.6.1/priv/lib eflex/lib/crypto-1.6.1/priv/lib/crypto_drv.so eflex/lib/crypto-1.6.1/priv/obj eflex/lib/crypto-1.6.1/priv/obj/Makefile eflex/lib/crypto-1.6.1/priv/obj/crypto_drv.o eflex/lib/wx-0.98.3.ez eflex/lib/syntax_tools-1.6.3.ez eflex/lib/wx-0.98.3 eflex/lib/wx-0.98.3/priv eflex/lib/wx-0.98.3/priv/x86_64-unknown-linux-gnu eflex/lib/wx-0.98.3/priv/x86_64-unknown-linux-gnu/wxe_driver.so eflex/lib/wx-0.98.3/priv/erlang-logo32.png eflex/lib/wx-0.98.3/priv/erlang-logo64.png eflex/erts-5.7.3 eflex/erts-5.7.3/bin eflex/erts-5.7.3/bin/erl eflex/erts-5.7.3/bin/beam eflex/erts-5.7.3/bin/epmd eflex/erts-5.7.3/bin/heart eflex/erts-5.7.3/bin/beam.smp eflex/erts-5.7.3/bin/erlexec eflex/erts-5.7.3/bin/inet_gethost eflex/erts-5.7.3/bin/child_setup ok 5> Sys2 = {sys, [{lib_dirs, ["/home/hakan/erl_libs"]}, {escript, "/home/hakan/bin/eflex", [{incl_cond, include}]}, {app, hipe, [{incl_cond, exclude}]}, {app, erts, [{incl_cond, exclude}]}, {app, crypto, [{incl_cond, exclude}]}, {app, syntax_tools, [{incl_cond, exclude}]}, {profile, standalone}, {excl_sys_filters,{add, ["^bin/(epmd|start.boot)(|\\.exe)$", "^erts.*/bin/(beam|epmd|heart)(|\\.exe)$"]}}]}, ok. ok 6> file:make_dir("eflex2"). ok 7> reltool:create_target([Sys2], "eflex2"). ok 8> io:format("~s\n", [os:cmd("find eflex2")]). eflex2 eflex2/bin eflex2/bin/erl eflex2/bin/eflex.escript eflex2/bin/eflex eflex2/bin/start_clean.boot eflex2/lib eflex2/lib/kernel-2.13.3.ez eflex2/lib/xmerl-1.2.1.ez eflex2/lib/eflex-0.1.ez eflex2/lib/stdlib-1.16.3.ez eflex2/lib/compiler-4.6.3.ez eflex2/lib/wx-0.98.3.ez eflex2/lib/wx-0.98.3 eflex2/lib/wx-0.98.3/priv eflex2/lib/wx-0.98.3/priv/x86_64-unknown-linux-gnu eflex2/lib/wx-0.98.3/priv/x86_64-unknown-linux-gnu/wxe_driver.so eflex2/lib/wx-0.98.3/priv/erlang-logo32.png eflex2/lib/wx-0.98.3/priv/erlang-logo64.png eflex2/erts-5.7.3 eflex2/erts-5.7.3/bin eflex2/erts-5.7.3/bin/erl eflex2/erts-5.7.3/bin/beam.smp eflex2/erts-5.7.3/bin/erlexec eflex2/erts-5.7.3/bin/inet_gethost eflex2/erts-5.7.3/bin/child_setup ok 9> reltool:get_target_spec([Sys2]). {ok,[{create_dir,"bin", [{copy_file,"eflex.escript","/home/hakan/bin/eflex"}, {copy_file,"eflex","erts-5.7.3/bin/escript"}, {copy_file,"erl","erts-5.7.3/bin/dyn_erl"}, {write_file,"start_clean.boot", <<131,104,3,100,0,6,115,99,114,105,112,116,104,2,107,0, ...>>}]}, {create_dir,"erts-5.7.3", [{create_dir,"bin", [{copy_file,"child_setup"}, {copy_file,"inet_gethost"}, {copy_file,"erlexec"}, {copy_file,"beam.smp"}, {copy_file,"erl","erts-5.7.3/bin/dyn_erl"}]}]}, {create_dir,"lib", [{archive,"compiler-4.6.3.ez",[], [{create_dir,"compiler-4.6.3", [{create_dir,"ebin", [{copy_file,"compiler.app"}, {copy_file,"beam_asm.beam"}, {copy_file,"beam_block.beam"}, {copy_file,"beam_bool.beam"}, {copy_file,"beam_bsm.beam"}, {copy_file,[...]}, {copy_file,...}, {...}|...]}]}]}, {archive,"eflex-0.1.ez",[], [{create_dir,"eflex-0.1","/home/hakan/erl_libs/eflex", [{create_dir,"ebin", [{copy_file,"eflex.app"}, {copy_file,"eflex.beam"}, {copy_file,"eflex_break.beam"}, {copy_file,[...]}, {copy_file,...}, {...}|...]}]}]}, {archive,"kernel-2.13.3.ez",[], [{create_dir,"kernel-2.13.3", [{create_dir,"ebin", [{copy_file,"kernel.app"}, {copy_file,"application.beam"}, {copy_file,"application_controller.beam"}, {copy_file,[...]}, {copy_file,...}, {...}|...]}]}]}, {archive,"stdlib-1.16.3.ez",[], [{create_dir,"stdlib-1.16.3", [{create_dir,"ebin", [{copy_file,"stdlib.app"}, {copy_file,"array.beam"}, {copy_file,[...]}, {copy_file,...}, {...}|...]}]}]}, {archive,"wx-0.98.3.ez",[], [{create_dir,"wx-0.98.3", [{create_dir,"ebin", [{copy_file,"gl.beam"}, {copy_file,[...]}, {copy_file,...}, {...}|...]}]}]}, {create_dir,"wx-0.98.3", [{create_dir,"priv", [{copy_file,"erlang-logo64.png"}, {copy_file,"erlang-logo32.png"}, {create_dir,"x86_64-unknown-linux-gnu", [{copy_file,...}]}]}]}, {archive,"xmerl-1.2.1.ez",[], [{create_dir,"xmerl-1.2.1", [{create_dir,"ebin",[{copy_file,...},{...}|...]}]}]}]}]} 10> -------------- next part -------------- A non-text attachment was scrubbed... Name: eflex Type: application/octet-stream Size: 792 bytes Desc: not available URL: From attila.r.nohl@REDACTED Thu Sep 24 15:04:53 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 24 Sep 2009 15:04:53 +0200 Subject: [erlang-questions] how to document experimental status (Re: [erlang-questions] Erlang/OTP R13B02-1 has been released) In-Reply-To: <401d3ba30909240604q1efad37epa3d83d396a73abaa@mail.gmail.com> References: <922d05850909230719k5592e211ye51db32072a00b86@mail.gmail.com> <4ABB6A16.1080003@erlang-consulting.com> <401d3ba30909240604q1efad37epa3d83d396a73abaa@mail.gmail.com> Message-ID: <401d3ba30909240604n67dabacbwde348aa9440f599b@mail.gmail.com> 2009/9/24, Ulf Wiger : [...] > Reltool is experimental. While the version number does give > that away, I don't know - I've used to use mplayer for years before it got to version 1.0, so maybe version numbers are not that meaningful nowadays... > and it says so in the release notes, In my experience that's a piece of documentation that is never read. > shouldn't > this also be written in highlight, in both the user guide > and the reference manual? Those look to better places. However, I have an other question - is it really wise to bundle "enterprise-grade" software together with experimental in the very same .tar archive? People might think that the whole stuff is rock solid, not just parts with X.Y version number where X>=1... From kevin@REDACTED Thu Sep 24 15:34:17 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Thu, 24 Sep 2009 09:34:17 -0400 Subject: [erlang-questions] erlang web / rest services In-Reply-To: <711a73df0909240543y2c5a2fbr424682c08b310b20@mail.gmail.com> References: <22835509.159291253788343206.JavaMail.root@zimbra> <407d9ef80909240446k6d5ebecv3f95626307a1878@mail.gmail.com> <711a73df0909240543y2c5a2fbr424682c08b310b20@mail.gmail.com> Message-ID: <10F35B50-93D9-4E07-B5F5-6DD1C9B9AF95@hypotheticalabs.com> That's what I thought, too. I know Justin, Andy, and Bryan have spent a lot of time trying to make webmachine follow Roy's thesis as closely as possible hence my recommendation. If you're just looking to hook up HTTP to Erlang mochiweb is a good choice, too. --Kevin On Sep 24, 2009, at 8:43 AM, Dave Pawson wrote: > 2009/9/24 Willem de Jong : >> Hi Akhil, >> >> One more suggestion: you might try to use the SOAP module that >> comes with >> Yaws. See http://yaws.hyber.org/soap_intro.yaws. >> >> Regards, >> Willem > > I thought the OP wanted RESTful? > > Soap is anything but? > > > > regards > > > > > -- > Dave Pawson > XSLT XSL-FO FAQ. > Docbook FAQ. > http://www.dpawson.co.uk > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From bob@REDACTED Thu Sep 24 15:48:04 2009 From: bob@REDACTED (Bob Ippolito) Date: Thu, 24 Sep 2009 06:48:04 -0700 Subject: [erlang-questions] erlang web / rest services In-Reply-To: <10F35B50-93D9-4E07-B5F5-6DD1C9B9AF95@hypotheticalabs.com> References: <22835509.159291253788343206.JavaMail.root@zimbra> <407d9ef80909240446k6d5ebecv3f95626307a1878@mail.gmail.com> <711a73df0909240543y2c5a2fbr424682c08b310b20@mail.gmail.com> <10F35B50-93D9-4E07-B5F5-6DD1C9B9AF95@hypotheticalabs.com> Message-ID: <6a36e7290909240648t3d0b5229o55fc402d52b73866@mail.gmail.com> Note also that webmachine is built on top of mochiweb. On Thu, Sep 24, 2009 at 6:34 AM, Kevin A. Smith wrote: > That's what I thought, too. I know Justin, Andy, and Bryan have spent a lot > of time trying to make webmachine follow Roy's thesis as closely as possible > hence my recommendation. > > If you're just looking to hook up HTTP to Erlang mochiweb is a good choice, > too. > > --Kevin > On Sep 24, 2009, at 8:43 AM, Dave Pawson wrote: > >> 2009/9/24 Willem de Jong : >>> >>> Hi Akhil, >>> >>> One more suggestion: you might try to use the SOAP module that comes with >>> Yaws. See http://yaws.hyber.org/soap_intro.yaws. >>> >>> Regards, >>> Willem >> >> I thought the OP wanted RESTful? >> >> Soap is anything but? >> >> >> >> regards >> >> >> >> >> -- >> Dave Pawson >> XSLT XSL-FO FAQ. >> Docbook FAQ. >> http://www.dpawson.co.uk >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From info@REDACTED Thu Sep 24 16:55:57 2009 From: info@REDACTED (info) Date: Thu, 24 Sep 2009 16:55:57 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname References: <200909231631349711641@its3.ch>, <20090924071607.GA27757@erix.ericsson.se> Message-ID: <200909241655568439572@its3.ch> I also tried to add the address of my dns server. inet_db:add_ns({ip}). But no effect. inet_getrc(). gives now: [{host,{127,0,0,1},["my_host"]}, {nameserver,{192,168,1,33}}] n Wed, Sep 23, 2009 at 04:31:35PM +0200, info wrote: > Hi all, > On a windows 2003 server sbs this command returns {ok,"my_host"} : > > {ok,N}=inet:gethostname(). > > But the following command returns {error,timeout} : > > inet:gethostbyname(N). > > On a Vista machine, the previous command returns a correct answer. > > I know that it is not an erlang problem but if you could give me support, I will appreciate ;-) Erlang may have problems configuring name lookups. Give me the output from ets:tab2list(inet_db). Which erlang release are you running, it sounds like pre-R13B02? Try starting with erl -name foo and do the same, ets:tab2list/1 too. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From thijsterlouw@REDACTED Thu Sep 24 17:25:09 2009 From: thijsterlouw@REDACTED (Thijs) Date: Thu, 24 Sep 2009 08:25:09 -0700 (PDT) Subject: net_kernel garbage collection In-Reply-To: <4ABA8E5E.4070508@gmail.com> References: <3C44D46958109E408F5848593B05A1D8C5B1B483@rrsmsx505.amr.corp.intel.com> <4ABA8B42.2000806@gmail.com> <3C44D46958109E408F5848593B05A1D8C5B1B4DD@rrsmsx505.amr.corp.intel.com> <4ABA8E5E.4070508@gmail.com> Message-ID: Is Erlang's hibernate not an option for you? You could use erlang:hibernate() or if you decide to use a gen_server process, return hibernate instead of a timeout. http://erlang.org/doc/man/gen_server.html : "If hibernate is specified instead of a timeout value, the process will go into hibernation when waiting for the next message to arrive (by calling proc_lib:hibernate/3). " From joelr1@REDACTED Thu Sep 24 18:07:35 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 24 Sep 2009 17:07:35 +0100 Subject: memsup:get_memory_usage() on leopard Message-ID: <4EE7B92A-09E9-4797-8D7C-0BDC999EA254@gmail.com> Does not work and prevents the os_mon app from starting. Here's a patch... diff --git a/memsup.erl b/memsup.erl index 4e4eff0..f9ad44e 100644 --- a/memsup.erl +++ b/memsup.erl @@ -728,8 +728,10 @@ get_memory_usage({unix,darwin}) -> io_lib:fread("Pages active:~d.", skip_to_eol(Str2)), {ok, [Inactive],Str4} = io_lib:fread("Pages inactive:~d.", skip_to_eol(Str3)), + {ok, _,Str5} = + io_lib:fread("Pages speculative:~d.", skip_to_eol(Str4)), {ok, [Wired],_} = - io_lib:fread("Pages wired down:~d.", skip_to_eol(Str4)), + io_lib:fread("Pages wired down:~d.", skip_to_eol(Str5)), NMemUsed = (Wired + Active + Inactive) * 4000, NMemTotal = NMemUsed + Free * 4000, {NMemUsed,NMemTotal}; --- fastest mac firefox! http://wagerlabs.com From mjtruog@REDACTED Thu Sep 24 19:56:35 2009 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 24 Sep 2009 10:56:35 -0700 Subject: [erlang-questions] Re: net_kernel garbage collection In-Reply-To: References: <3C44D46958109E408F5848593B05A1D8C5B1B483@rrsmsx505.amr.corp.intel.com> <4ABA8B42.2000806@gmail.com> <3C44D46958109E408F5848593B05A1D8C5B1B4DD@rrsmsx505.amr.corp.intel.com> <4ABA8E5E.4070508@gmail.com> Message-ID: <4ABBB2D3.6090403@gmail.com> hibernate will help a little, to help remove references to some of the data. However, if you are processing lots of binary data you would need a short lived process to free the data when the data is no longer needed. Thijs wrote: > Is Erlang's hibernate not an option for you? You could use > erlang:hibernate() or if you decide to use a gen_server process, > return hibernate instead of a timeout. > > http://erlang.org/doc/man/gen_server.html : > "If hibernate is specified instead of a timeout value, the process > will go into hibernation when waiting for the next message to arrive > (by calling proc_lib:hibernate/3). " > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From roberto@REDACTED Thu Sep 24 20:29:57 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 24 Sep 2009 20:29:57 +0200 Subject: Misultin v0.1 released In-Reply-To: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> References: <544758AC-B45D-4D0C-9BF4-7BABA0C091C7@widetag.com> Message-ID: Dear all, Misultin v0.3 has just been released. Speed has been improved by a switch in the listener process, which now uses active instead of passive mode in request parsing, except for BODY where passive is still used. This change has been inspired by Lev Walkin and his recents benchmark tests. Benchmarks have been updated, including a comparison with Mochiweb and Apache running PHP: http://code.google.com/p/misultin/wiki/Benchmarks Cheers, >-|-|-(?> !r From vinoski@REDACTED Thu Sep 24 21:57:41 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Thu, 24 Sep 2009 15:57:41 -0400 Subject: [erlang-questions] erlang web / rest services In-Reply-To: References: Message-ID: <65b2728e0909241257n4e3cdcaevea040165fa74a81e@mail.gmail.com> On Thu, Sep 24, 2009 at 1:36 AM, Akhil Kodali wrote: > Hi, > > Is there any way or framework by which i can expose erlang services as web > or rest services. > The only reference i could find was by Steve Vinoski at InfoQ. A recent article that Dave Bryson and I wrote addresses this very topic: < http://steve.vinoski.net/pdf/IC-Build_Your_Next_Web_Application_with_Erlang.pdf > --steve From raould@REDACTED Fri Sep 25 01:50:30 2009 From: raould@REDACTED (Raoul Duke) Date: Thu, 24 Sep 2009 16:50:30 -0700 Subject: how goes the iphone port? Message-ID: <91a2ba3e0909241650gc7b45c9gd07a5dd5144a6f94@mail.gmail.com> hi, so i see some folks got a form of Erlang ported to iPhone. any reports on performance? thanks. From harmlessbystandard@REDACTED Fri Sep 25 03:41:56 2009 From: harmlessbystandard@REDACTED (Davichi pavel) Date: Thu, 24 Sep 2009 18:41:56 -0700 (PDT) Subject: [erlang-questions] how goes the iphone port? In-Reply-To: <91a2ba3e0909241650gc7b45c9gd07a5dd5144a6f94@mail.gmail.com> References: <91a2ba3e0909241650gc7b45c9gd07a5dd5144a6f94@mail.gmail.com> Message-ID: <13321.76216.qm@web31404.mail.mud.yahoo.com> It's going okay. We're not sure how well it works at the moment, and I and the main programmer, Tony Rogvall, will be too busy to work on it for a while, so We're looking for someone to take over as Project Admins. Anyone interested? Sometimes you have to stand in the mud to get anything done -Loki- ________________________________ From: Raoul Duke To: erlang-questions Sent: Thursday, September 24, 2009 5:50:30 PM Subject: [erlang-questions] how goes the iphone port? hi, so i see some folks got a form of Erlang ported to iPhone. any reports on performance? thanks. ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From ttmrichter@REDACTED Fri Sep 25 04:52:50 2009 From: ttmrichter@REDACTED (Michael Richter) Date: Fri, 25 Sep 2009 10:52:50 +0800 Subject: Escript doc mismatch? Message-ID: Quoting from http://www.erlang.org/doc/man/escript.html: The header of an Erlang script differs from a normal Erlang module. The > first line is intended to be the interpreter line, which invokes escript. > However if you invoke escript like this > > $ *escript factorial 5* > > *the contents of the first line does not matter, but it cannot contain > Erlang code as it will be ignored. * > (*Emphasis* mine. *Bold* in original.) The reality: michael@REDACTED:~/junk$ cat factorial > This line should be ignored, no? > %% -*- erlang -*- > %%! -smp enable -sname factorial -mnesia debug verbose > main([String]) -> > try > N = list_to_integer(String), > F = fac(N), > io:format("factorial ~w = ~w\n", [N,F]) > catch > _:_ -> > usage() > end; > main(_) -> > usage(). > > usage() -> > io:format("usage: factorial integer\n"), > halt(1). > > fac(0) -> 1; > fac(N) -> N * fac(N-1). > michael@REDACTED:~/junk$ escript factorial 10 > factorial:4: undefined macro 'main' > escript: There were compilation errors. > A further test: michael@REDACTED:~/junk$ cat factorial > main([String]) -> > try > N = list_to_integer(String), > F = fac(N), > io:format("factorial ~w = ~w\n", [N,F]) > catch > _:_ -> > usage() > end; > main(_) -> > usage(). > > usage() -> > io:format("usage: factorial integer\n"), > halt(1). > > fac(0) -> 1; > fac(N) -> N * fac(N-1). > michael@REDACTED:~/junk$ escript factorial 10 > factorial 10 = 3628800 > By my reading of the manual that first script should have worked (first line ignored thus passing over invalid Erlang) and the second should have failed (first line ignored, taking with it an important chunk of Erlang code). The reason I want this to work properly is because I'd like to have escript working under Windows. For that to happen the first line has to be something like "@escript %0 %* & exit/b" which is obviously not valid Erlang. I've already got escript (mostly) working under Windows using some hacks. The first hack works under the TCC/LE CMD.EXE drop-in. For this I create an alias called "#!" that is quietly replaced with "cmd/c". I also make use of a special variable in that shell--"%_batchname"--that gives me the full name of the batch file being executed. Together these give me a batch header that works for escript: "#! escript.exe %_batchname %* & exit/b". Not everybody uses TCC/LE, however (although they really should if they're stuck in the Windows ghetto!), so that is only a useful tool for my machines. An attempt to get this working for CMD.EXE itself involving a batch file called "#!.bat" was successful (sorta) except for the fact that to stop the command from being echoed to the screen I have to put an "@" character in front of things which goes back to breaking as per the above tests. Is there any chance that we could get the documented behaviour instead of trying to make workarounds for it? From ulf.wiger@REDACTED Fri Sep 25 09:12:46 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 25 Sep 2009 09:12:46 +0200 Subject: running without net tick Message-ID: <4ABC6D6E.6020608@erlang-consulting.com> The problem of netsplits in Erlang comes up now and again. I've mentioned that we used to have a more robust supervision algorithm for device processor monitoring in AXD 301... I read the following comment in kernel/src/dist_util.erl %% Send a TICK to the other side. %% %% This will happen every 15 seconds (by default) %% The idea here is that every 15 secs, we write a little %% something on the connection if we haven't written anything for %% the last 15 secs. %% This will ensure that nodes that are not responding due to %% hardware errors (Or being suspended by means of ^Z) will %% be considered to be down. If we do not want to have this %% we must start the net_kernel (in erlang) without its %% ticker process, In that case this code will never run ...and thought: promising - it is then possible to experiment with other tick algorithms? However, looking at net_kernel.erl: init({Name, LongOrShortNames, TickT}) -> process_flag(trap_exit,true), case init_node(Name, LongOrShortNames) of {ok, Node, Listeners} -> process_flag(priority, max), Ticktime = to_integer(TickT), Ticker = spawn_link(net_kernel, ticker, [self(), Ticktime]), In other words, you can't set net_ticktime to anything other than an integer (and it has to be a smallint, since it's used in a receive ... after expression. (To do justice to the comment above, couldn't a net_ticktime of, say, 0 turn off net ticking altogether?) What one can do then, is to set net_ticktime to a very large number, and then run a user-level heartbeat. If netsplits are still experienced without visible problems in the user-level monitoring, or perhaps even serviced traffic during this interval, then something is definitely wrong with the tick algorithm. :) BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ulf.wiger@REDACTED Fri Sep 25 09:17:20 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 25 Sep 2009 09:17:20 +0200 Subject: documentation of incremental backups in mnesia Message-ID: <4ABC6E80.2090205@erlang-consulting.com> From the old telecom days, I recall that Mnesia supports incremental backups (has been for years, and proven in battle too). I was mildly surprised that it didn't seem well known until I began searching for a reference in the documentation. It is documented, sort of, but only in passing. You have to read pretty carefully to catch it at all, and you probably need to experiment a bit before understanding how it works. I think it's a great feature that deserves a sub-chapter of its own in the User Guide, and some more detail in the reference guide as well. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ulf.wiger@REDACTED Fri Sep 25 09:48:19 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 25 Sep 2009 09:48:19 +0200 Subject: Erlang Stockholm User Group meeting Oct 01 Message-ID: <4ABC75C3.5080803@erlang-consulting.com> Here's a reminder that the next Stockholm Erlang User Group meeting will be *next Thursday*, 1 Oct, at Kreditor HQ (with Erlounge afterwards). http://www.erlang-consulting.com/etc/usergroup/stockholm (Advance registration required.) Speakers: Kostis Sagonas and John Hughes Sorry for spamming the erlang-questions list, but I first forgot to advertise this event on the erlangstockholm@REDACTED list, and now that I have, I haven't received any email from it myself. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From raimo+erlang-questions@REDACTED Fri Sep 25 09:54:37 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 25 Sep 2009 09:54:37 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname In-Reply-To: <200909241655568439572@its3.ch> References: <20090924071607.GA27757@erix.ericsson.se> <200909241655568439572@its3.ch> Message-ID: <20090925075437.GA27366@erix.ericsson.se> On Thu, Sep 24, 2009 at 04:55:57PM +0200, info wrote: > I also tried to add the address of my dns server. > inet_db:add_ns({ip}). > But no effect. > > inet_getrc(). gives now: > > [{host,{127,0,0,1},["my_host"]}, > {nameserver,{192,168,1,33}}] > You have {res_lookup,[native,file]} meaning first is the OS gethostbyname equivalent function used. During the startup of Erlang it discovered that "my_host" could not be resolved so it entered "my_host" as {127,0,0,1} into the internal hosts table to ensure that inet:gethostbyname("my_host") would succeed. But the native lookup method times out, probably due to misconfigured name resolving in the OS, so the internal hosts table is never used. How long does the call inet:gethostbyname("my_host") take to return {error,timeout}? Can the native lookup method look up any name at all? Try: inet_gethost_native:gethostbyname("localhost"). inet_gethost_native:gethostbyname("my_host"). inet_gethost_native:gethostbyname("www.google.com"). >From a Windows command shell, try: nslookup my_host nslookup localhost Pay special attention to which nameserver that answers. If you can not fix the network configuration, which will solve your problem, you can configure erlang to not use the native lookup method. The erlang DNS resolver client was much improved to R13B02, but it worked well in R13B01 too. Set the lookup method to [file,dns] and add your nameserver as you tried: inet_db:add_ns({192,168,1,33}). inet_db:set_lookup([file,dns]). Then try inet:gethostbyname("my_host"), or even inet_res:gethostbyname("my_host). But I hope you can fix your network configuration instead... > > > n Wed, Sep 23, 2009 at 04:31:35PM +0200, info wrote: > > Hi all, > > On a windows 2003 server sbs this command returns {ok,"my_host"} : > > > > {ok,N}=inet:gethostname(). > > > > But the following command returns {error,timeout} : > > > > inet:gethostbyname(N). > > > > On a Vista machine, the previous command returns a correct answer. > > > > I know that it is not an erlang problem but if you could give me support, I will appreciate ;-) > > Erlang may have problems configuring name lookups. > Give me the output from ets:tab2list(inet_db). > > Which erlang release are you running, it sounds like pre-R13B02? > > Try starting with erl -name foo > and do the same, ets:tab2list/1 too. > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From joelr1@REDACTED Fri Sep 25 10:31:24 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 25 Sep 2009 09:31:24 +0100 Subject: [erlang-questions] running without net tick In-Reply-To: <4ABC6D6E.6020608@erlang-consulting.com> References: <4ABC6D6E.6020608@erlang-consulting.com> Message-ID: <2000A179-B4E8-474A-91DA-83A77AA270EF@gmail.com> On Sep 25, 2009, at 8:12 AM, Ulf Wiger wrote: > What one can do then, is to set net_ticktime to a very large > number, and then run a user-level heartbeat. I will try this today. I ran my own heartbeat before and saw netsplits with my own roundtrip of just a few seconds. I don't think I was able to ssh to the node that split which could make it a EC2 problem. Still, how come my round trip was within 10s when the node split and not a large number? --- fastest mac firefox! http://wagerlabs.com From dmitriid@REDACTED Fri Sep 25 10:53:09 2009 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Fri, 25 Sep 2009 11:53:09 +0300 Subject: Temp directory/file from Erlang Message-ID: <034AD643-8E7E-4EC7-9A07-5137E560F163@gmail.com> Hi all Is there a way to find out the path to the system's temp directory and/ or create a temporary file? Thank you From hakan@REDACTED Fri Sep 25 11:41:29 2009 From: hakan@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Fri, 25 Sep 2009 11:41:29 +0200 Subject: [erlang-questions] Escript doc mismatch? In-Reply-To: References: Message-ID: <922d05850909250241m4ef67490rf8b3aaf74ed88116@mail.gmail.com> On Fri, Sep 25, 2009 at 4:52 AM, Michael Richter wrote: > Quoting from http://www.erlang.org/doc/man/escript.html: > > The header of an Erlang script differs from a normal Erlang module. The >> first line is intended to be the interpreter line, which invokes escript. >> However if you invoke escript like this >> >> $ *escript factorial 5* >> >> *the contents of the first line does not matter, but it cannot contain >> Erlang code as it will be ignored. * The documentation needs to be updated. If the escript has a header, the first line in the file must start with #!. If you run the escript by explicitly invoking the escript executable, e.g. like "escript factorial 5", the characters after #! on the first line are ignored. As the header is optional you can in fact "execute" an .erl, .beam or .zip file as long as escript can find the main/1 function. # cat factorial.erl -module(factorial). -export([main/1, fac/1]). main([String]) -> try N = list_to_integer(String), F = fac(N), io:format("factorial ~w = ~w\n", [N,F]) catch _:_ -> usage() end; main(_) -> usage(). usage() -> io:format("usage: factorial integer\n"), halt(1). fac(0) -> 1; fac(N) -> N * fac(N-1). # escript factorial.erl 5 factorial 5 = 120 # erlc factorial.erl # escript factorial.beam 4 factorial 4 = 24 # zip factorial.zip factorial.erl factorial.beam adding: factorial.erl (deflated 44%) adding: factorial.beam (deflated 29%) # rm factorial.erl # rm factorial.beam # escript factorial.zip 7 factorial 7 = 5040 /H?kan From olopierpa@REDACTED Fri Sep 25 11:56:37 2009 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Fri, 25 Sep 2009 11:56:37 +0200 Subject: [erlang-questions] Temp directory/file from Erlang In-Reply-To: <034AD643-8E7E-4EC7-9A07-5137E560F163@gmail.com> References: <034AD643-8E7E-4EC7-9A07-5137E560F163@gmail.com> Message-ID: <7352e43a0909250256lba64019i55463550bc2351a9@mail.gmail.com> On Fri, Sep 25, 2009 at 10:53, Dmitrii Dimandt wrote: > Hi all > > Is there a way to find out the path to the system's temp directory and/or > create a temporary file? The usual way is to check the TEMP and/or TMP environment variables. P. From info@REDACTED Fri Sep 25 12:03:38 2009 From: info@REDACTED (info) Date: Fri, 25 Sep 2009 12:03:38 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname References: <20090924071607.GA27757@erix.ericsson.se>, <200909241655568439572@its3.ch>, <20090925075437.GA27366@erix.ericsson.se> Message-ID: <200909251203380496915@its3.ch> Hi Raimo, Note that I appreciate your help because it seems that the problem is not in erlang. On Thu, Sep 24, 2009 at 04:55:57PM +0200, info wrote: > I also tried to add the address of my dns server. > inet_db:add_ns({ip}). > But no effect. > > inet_getrc(). gives now: > > [{host,{127,0,0,1},["my_host"]}, > {nameserver,{192,168,1,33}}] > You have {res_lookup,[native,file]} meaning first is the OS gethostbyname equivalent function used. During the startup of Erlang it discovered that "my_host" could not be resolved so it entered "my_host" as {127,0,0,1} into the internal hosts table to ensure that inet:gethostbyname("my_host") would succeed. But the native lookup method times out, probably due to misconfigured name resolving in the OS, so the internal hosts table is never used. How long does the call inet:gethostbyname("my_host") take to return {error,timeout}? >>>>>>>>>>>> 6-7 seconds Can the native lookup method look up any name at all? Try: inet_gethost_native:gethostbyname("localhost"). >>>>>>>>>>>> {error,timeout} inet_gethost_native:gethostbyname("my_host"). >>>>>>>>>>>> {error,timeout} inet_gethost_native:gethostbyname("www.google.com"). >>>>>>>>>>>> {error,timeout} >From a Windows command shell, try: nslookup my_host >>>>>>>>>>>>server: my_host.my_domain.local >>>>>>>>>>>>address: 192.168.1.33 >>>>>>>>>>>>name: my_host.my_domain.local >>>>>>>>>>>>address: 192.168.1.33 nslookup localhost >>>>>>>>>>>>server: my_host.my_domain.local >>>>>>>>>>>>address: 192.168.1.33 >>>>>>>>>>>>my_host.my_domain.local doesn't find localhost: no existing domain (*** I translated from french ! ) Pay special attention to which nameserver that answers. If you can not fix the network configuration, which will solve your problem, you can configure erlang to not use the native lookup method. The erlang DNS resolver client was much improved to R13B02, but it worked well in R13B01 too. >>>>>>>>>>>>My problem is I don't see what is wrong in my network configuration or in my dns server configuration ! Set the lookup method to [file,dns] and add your nameserver as you tried: inet_db:add_ns({192,168,1,33}). >>>>>>>>>>>>ok inet_db:set_lookup([file,dns]). >>>>>>>>>>>>ok Then try inet:gethostbyname("my_host"), >>>>>>>>>>>>{ok,{hostent,"my-host",[],inet,4,[{127,0,0,1}]}} BETTER !! or even inet_res:gethostbyname("my_host). >>>>>>>>>>>>{error,timeout} WHY ? But I hope you can fix your network configuration instead... > > > n Wed, Sep 23, 2009 at 04:31:35PM +0200, info wrote: > > Hi all, > > On a windows 2003 server sbs this command returns {ok,"my_host"} : > > > > {ok,N}=inet:gethostname(). > > > > But the following command returns {error,timeout} : > > > > inet:gethostbyname(N). > > > > On a Vista machine, the previous command returns a correct answer. > > > > I know that it is not an erlang problem but if you could give me support, I will appreciate ;-) > > Erlang may have problems configuring name lookups. > Give me the output from ets:tab2list(inet_db). > > Which erlang release are you running, it sounds like pre-R13B02? > > Try starting with erl -name foo > and do the same, ets:tab2list/1 too. > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From v@REDACTED Fri Sep 25 12:04:20 2009 From: v@REDACTED (Valentin Micic) Date: Fri, 25 Sep 2009 12:04:20 +0200 Subject: [erlang-questions] running without net tick In-Reply-To: <4ABC6D6E.6020608@erlang-consulting.com> Message-ID: <200909251005.n8PA54Sw006936@mail.pharos-avantgard.com> You may change TICK value all day long, but if the underlying infrastructure s in some kind of trouble, that alone is not going to solve the problem. The following is just a speculation, but quite plausible in my mind: AFAIK, ERTS is multiplexing inter-nodal traffic over a single socket. Thus, if the socket is heavily utilized, the sending buffer may get congested due to dynamically reduced TCP window size (because remote side is not flushing its buffer fast enough -- if the same process is reading and writing the socket, this may cause a deadlock under a heavy load). As much as I am not certain about particular implementation here, I know that sender will not wait for ever -- it will eventually timeout and this (exception?) has to be handled somehow by the sender. The reasonable course of action would be to reset the connection. If and when that happens, node can be declared unreachable; therefore the "net-split" may occur. In other words, net-split may occur with or without "ticker" process running and regardless of the real network availability (*). I think the net-tick method is good on its own, however, it is utilizing a *wrong* transport! IMO, tick should be handled as out-of-band data, and this cannot be done using TCP/IP (well, at least not at the user level). My suggestion would be to use UDP for net-kernel communication (including TICK messages). This way one would be able to find out about peer health more reliably (yes, a small protocol may be required, but that's relatively easy). To make things simpler regarding the distribution, one may use the same port number as advertised in EPMD for a particular node, hence bind UDP socket to that number. V/ (*) I've seen "net-splits" between nodes collocated on the same machine -- therefore indicating TCP buffer/load related issue. Maybe situation may be improved by creation of more than one connection between two nodes, but that may come with a bag of problems on its own. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Ulf Wiger Sent: 25 September 2009 09:13 AM To: erlang-questions Questions Subject: [erlang-questions] running without net tick The problem of netsplits in Erlang comes up now and again. I've mentioned that we used to have a more robust supervision algorithm for device processor monitoring in AXD 301... I read the following comment in kernel/src/dist_util.erl %% Send a TICK to the other side. %% %% This will happen every 15 seconds (by default) %% The idea here is that every 15 secs, we write a little %% something on the connection if we haven't written anything for %% the last 15 secs. %% This will ensure that nodes that are not responding due to %% hardware errors (Or being suspended by means of ^Z) will %% be considered to be down. If we do not want to have this %% we must start the net_kernel (in erlang) without its %% ticker process, In that case this code will never run ...and thought: promising - it is then possible to experiment with other tick algorithms? However, looking at net_kernel.erl: init({Name, LongOrShortNames, TickT}) -> process_flag(trap_exit,true), case init_node(Name, LongOrShortNames) of {ok, Node, Listeners} -> process_flag(priority, max), Ticktime = to_integer(TickT), Ticker = spawn_link(net_kernel, ticker, [self(), Ticktime]), In other words, you can't set net_ticktime to anything other than an integer (and it has to be a smallint, since it's used in a receive ... after expression. (To do justice to the comment above, couldn't a net_ticktime of, say, 0 turn off net ticking altogether?) What one can do then, is to set net_ticktime to a very large number, and then run a user-level heartbeat. If netsplits are still experienced without visible problems in the user-level monitoring, or perhaps even serviced traffic during this interval, then something is definitely wrong with the tick algorithm. :) BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From chandrashekhar.mullaparthi@REDACTED Fri Sep 25 12:07:24 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 25 Sep 2009 11:07:24 +0100 Subject: [erlang-questions] documentation of incremental backups in mnesia In-Reply-To: <4ABC6E80.2090205@erlang-consulting.com> References: <4ABC6E80.2090205@erlang-consulting.com> Message-ID: Hi Ulf, 2009/9/25 Ulf Wiger > From the old telecom days, I recall that Mnesia supports > incremental backups (has been for years, and proven in > battle too). I was mildly surprised that it didn't seem > well known until I began searching for a reference in the > documentation. > > It is documented, sort of, but only in passing. You have to > read pretty carefully to catch it at all, and you probably > need to experiment a bit before understanding how it works. > > Could you please point to where it is mentioned in passing. It seems to be a very useful feature. regards, Chandru From kagato@REDACTED Fri Sep 25 12:25:29 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 25 Sep 2009 03:25:29 -0700 Subject: [erlang-questions] running without net tick In-Reply-To: <200909251005.n8PA54Sw006936@mail.pharos-avantgard.com> References: <200909251005.n8PA54Sw006936@mail.pharos-avantgard.com> Message-ID: Short Version: Why not open a special "tick" TCP port? UDP would require a reliable delivery implementation. TCP saves quite a bit of work in that regard (and gets a lot of important but subtle things right). Long Version: Also, never say never. Actually, you CAN send out-of-band data (also called urgent data) using TCP. The original "WinNuke" (i.e. ping-of-death for Windows 95) was due to having a corrupt OOB header in a TCP packet. In classic Microsoft / Internet style, the issue was further confused because it was an Out-of-Bounds bug, so a generation of networking consultants have minor deviations in their interpretations of the meaning of the letters OOB. As for TCP Urgent Data / OOB, it seems to be specified well enough at the protocol level, but iit doesn't appear to be handled uniformly in different socket implementations. Under Linux, you use send/recv with the MSG_OOB option (or set the SO_OOBINLINE socket option to just inline the data). It appears to try to keep it at a certain point in the data stream (i.e. to preserve some of the ordering) and certain conditions can cause it to become part of the "normal" stream of data. It also can cause some odd signals to be delivered to the process. Still, TCP *does* have OOB data support, just maybe it isn't easily usable everywhere. On Sep 25, 2009, at 3:04 AM, Valentin Micic wrote: > You may change TICK value all day long, but if the underlying > infrastructure > s in some kind of trouble, that alone is not going to solve the > problem. > > The following is just a speculation, but quite plausible in my mind: > > AFAIK, ERTS is multiplexing inter-nodal traffic over a single > socket. Thus, > if the socket is heavily utilized, the sending buffer may get > congested due > to dynamically reduced TCP window size (because remote side is not > flushing > its buffer fast enough -- if the same process is reading and writing > the > socket, this may cause a deadlock under a heavy load). As much as I > am not > certain about particular implementation here, I know that sender > will not > wait for ever -- it will eventually timeout and this (exception?) > has to be > handled somehow by the sender. The reasonable course of action would > be to > reset the connection. If and when that happens, node can be declared > unreachable; therefore the "net-split" may occur. In other words, > net-split > may occur with or without "ticker" process running and regardless of > the > real network availability (*). > > > I think the net-tick method is good on its own, however, it is > utilizing a > *wrong* transport! IMO, tick should be handled as out-of-band data, > and this > cannot be done using TCP/IP (well, at least not at the user level). My > suggestion would be to use UDP for net-kernel communication > (including TICK > messages). This way one would be able to find out about peer health > more > reliably (yes, a small protocol may be required, but that's relatively > easy). > > To make things simpler regarding the distribution, one may use the > same port > number as advertised in EPMD for a particular node, hence bind UDP > socket to > that number. > > V/ > > (*) I've seen "net-splits" between nodes collocated on the same > machine -- > therefore indicating TCP buffer/load related issue. Maybe situation > may be > improved by creation of more than one connection between two nodes, > but that > may come with a bag of problems on its own. > > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang- > questions@REDACTED] On > Behalf Of Ulf Wiger > Sent: 25 September 2009 09:13 AM > To: erlang-questions Questions > Subject: [erlang-questions] running without net tick > > > The problem of netsplits in Erlang comes up now and again. > I've mentioned that we used to have a more robust > supervision algorithm for device processor monitoring in > AXD 301... > > I read the following comment in kernel/src/dist_util.erl > > %% Send a TICK to the other side. > %% > %% This will happen every 15 seconds (by default) > %% The idea here is that every 15 secs, we write a little > %% something on the connection if we haven't written anything for > %% the last 15 secs. > %% This will ensure that nodes that are not responding due to > %% hardware errors (Or being suspended by means of ^Z) will > %% be considered to be down. If we do not want to have this > %% we must start the net_kernel (in erlang) without its > %% ticker process, In that case this code will never run > > > ...and thought: promising - it is then possible to experiment > with other tick algorithms? > > However, looking at net_kernel.erl: > > init({Name, LongOrShortNames, TickT}) -> > process_flag(trap_exit,true), > case init_node(Name, LongOrShortNames) of > {ok, Node, Listeners} -> > process_flag(priority, max), > Ticktime = to_integer(TickT), > Ticker = spawn_link(net_kernel, ticker, [self(), > Ticktime]), > > In other words, you can't set net_ticktime to anything other > than an integer (and it has to be a smallint, since it's used > in a receive ... after expression. > > (To do justice to the comment above, couldn't a net_ticktime > of, say, 0 turn off net ticking altogether?) > > What one can do then, is to set net_ticktime to a very large > number, and then run a user-level heartbeat. If netsplits are > still experienced without visible problems in the user-level > monitoring, or perhaps even serviced traffic during this > interval, then something is definitely wrong with the tick > algorithm. :) > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From federico@REDACTED Fri Sep 25 12:10:14 2009 From: federico@REDACTED (Federico Fissore) Date: Fri, 25 Sep 2009 12:10:14 +0200 Subject: [erlang-questions] Temp directory/file from Erlang In-Reply-To: <7352e43a0909250256lba64019i55463550bc2351a9@mail.gmail.com> References: <034AD643-8E7E-4EC7-9A07-5137E560F163@gmail.com> <7352e43a0909250256lba64019i55463550bc2351a9@mail.gmail.com> Message-ID: <4ABC9706.8060101@fissore.org> Pierpaolo Bernardi, il 25/09/2009 11:56, ha scritto: > On Fri, Sep 25, 2009 at 10:53, Dmitrii Dimandt wrote: >> Hi all >> >> Is there a way to find out the path to the system's temp directory and/or >> create a temporary file? > > The usual way is to check the TEMP and/or TMP environment variables. > > P. > > uhm, on linux I have no TEMP nor TMP env vars fede From v@REDACTED Fri Sep 25 13:26:22 2009 From: v@REDACTED (Valentin Micic) Date: Fri, 25 Sep 2009 13:26:22 +0200 Subject: [erlang-questions] running without net tick In-Reply-To: Message-ID: <200909251127.n8PBR6Sw007322@mail.pharos-avantgard.com> I beg to differ -- my take is that TCP reliability is a part of the problem in this case. Whilst buffering and flow control is important for, say, file transfer, it is completely irrelevant for TCIK and health-checks (So what if it doesn't get there, I can send it again without any consequence!). Argument about UDP unreliability sounds more like a mantra than a proper argument (if only I got a penny every time I've heard it (-:). There are only two fundamental differences (*) between TCP and UDP... actually only one, because the second is conditioned by the first: TCP supports stream, whilst UDP message-bound communication; thus, as a consequence, TCP requires some form of flow control to support stream processing. In this particular case: what possible benefit one can derive from sending a message over the stream as opposed to sending just a message? If the message is short enough to fit in a datagram -- none! As for ability to send urgent data (OOB) over TCP socket -- data streams and OOB data are mixing like oil and water. I am yet to see a successful utilization of OOB (issued by a user) that hasn't resulted in connection reset (or system shutdown (-;). Lastly, if TICK is implemented via separate TCP socket, that would double networking resources required -- you'd need a new socket for every node you're connected to. With UDP, all you need is one socket, and a very basic protocol: 1) Ask when you have to; 2) Answer when asked. Mind you, net-kernel is already doing this. V/ (*) If one disregards things that UDP can which TCP cannot do, such as a multi-drop, multicasting, etc. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Jayson Vantuyl Sent: 25 September 2009 12:25 PM To: Erlang-Questions Questions Subject: Re: [erlang-questions] running without net tick Short Version: Why not open a special "tick" TCP port? UDP would require a reliable delivery implementation. TCP saves quite a bit of work in that regard (and gets a lot of important but subtle things right). Long Version: Also, never say never. Actually, you CAN send out-of-band data (also called urgent data) using TCP. The original "WinNuke" (i.e. ping-of-death for Windows 95) was due to having a corrupt OOB header in a TCP packet. In classic Microsoft / Internet style, the issue was further confused because it was an Out-of-Bounds bug, so a generation of networking consultants have minor deviations in their interpretations of the meaning of the letters OOB. As for TCP Urgent Data / OOB, it seems to be specified well enough at the protocol level, but iit doesn't appear to be handled uniformly in different socket implementations. Under Linux, you use send/recv with the MSG_OOB option (or set the SO_OOBINLINE socket option to just inline the data). It appears to try to keep it at a certain point in the data stream (i.e. to preserve some of the ordering) and certain conditions can cause it to become part of the "normal" stream of data. It also can cause some odd signals to be delivered to the process. Still, TCP *does* have OOB data support, just maybe it isn't easily usable everywhere. On Sep 25, 2009, at 3:04 AM, Valentin Micic wrote: > You may change TICK value all day long, but if the underlying > infrastructure > s in some kind of trouble, that alone is not going to solve the > problem. > > The following is just a speculation, but quite plausible in my mind: > > AFAIK, ERTS is multiplexing inter-nodal traffic over a single > socket. Thus, > if the socket is heavily utilized, the sending buffer may get > congested due > to dynamically reduced TCP window size (because remote side is not > flushing > its buffer fast enough -- if the same process is reading and writing > the > socket, this may cause a deadlock under a heavy load). As much as I > am not > certain about particular implementation here, I know that sender > will not > wait for ever -- it will eventually timeout and this (exception?) > has to be > handled somehow by the sender. The reasonable course of action would > be to > reset the connection. If and when that happens, node can be declared > unreachable; therefore the "net-split" may occur. In other words, > net-split > may occur with or without "ticker" process running and regardless of > the > real network availability (*). > > > I think the net-tick method is good on its own, however, it is > utilizing a > *wrong* transport! IMO, tick should be handled as out-of-band data, > and this > cannot be done using TCP/IP (well, at least not at the user level). My > suggestion would be to use UDP for net-kernel communication > (including TICK > messages). This way one would be able to find out about peer health > more > reliably (yes, a small protocol may be required, but that's relatively > easy). > > To make things simpler regarding the distribution, one may use the > same port > number as advertised in EPMD for a particular node, hence bind UDP > socket to > that number. > > V/ > > (*) I've seen "net-splits" between nodes collocated on the same > machine -- > therefore indicating TCP buffer/load related issue. Maybe situation > may be > improved by creation of more than one connection between two nodes, > but that > may come with a bag of problems on its own. > > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang- > questions@REDACTED] On > Behalf Of Ulf Wiger > Sent: 25 September 2009 09:13 AM > To: erlang-questions Questions > Subject: [erlang-questions] running without net tick > > > The problem of netsplits in Erlang comes up now and again. > I've mentioned that we used to have a more robust > supervision algorithm for device processor monitoring in > AXD 301... > > I read the following comment in kernel/src/dist_util.erl > > %% Send a TICK to the other side. > %% > %% This will happen every 15 seconds (by default) > %% The idea here is that every 15 secs, we write a little > %% something on the connection if we haven't written anything for > %% the last 15 secs. > %% This will ensure that nodes that are not responding due to > %% hardware errors (Or being suspended by means of ^Z) will > %% be considered to be down. If we do not want to have this > %% we must start the net_kernel (in erlang) without its > %% ticker process, In that case this code will never run > > > ...and thought: promising - it is then possible to experiment > with other tick algorithms? > > However, looking at net_kernel.erl: > > init({Name, LongOrShortNames, TickT}) -> > process_flag(trap_exit,true), > case init_node(Name, LongOrShortNames) of > {ok, Node, Listeners} -> > process_flag(priority, max), > Ticktime = to_integer(TickT), > Ticker = spawn_link(net_kernel, ticker, [self(), > Ticktime]), > > In other words, you can't set net_ticktime to anything other > than an integer (and it has to be a smallint, since it's used > in a receive ... after expression. > > (To do justice to the comment above, couldn't a net_ticktime > of, say, 0 turn off net ticking altogether?) > > What one can do then, is to set net_ticktime to a very large > number, and then run a user-level heartbeat. If netsplits are > still experienced without visible problems in the user-level > monitoring, or perhaps even serviced traffic during this > interval, then something is definitely wrong with the tick > algorithm. :) > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From ulf.wiger@REDACTED Fri Sep 25 13:29:08 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 25 Sep 2009 13:29:08 +0200 Subject: [erlang-questions] running without net tick In-Reply-To: <200909251005.n8PA54Sw006936@mail.pharos-avantgard.com> References: <200909251005.n8PA54Sw006936@mail.pharos-avantgard.com> Message-ID: <4ABCA984.1080106@erlang-consulting.com> Valentin Micic wrote: > You may change TICK value all day long, but if the underlying > infrastructure s in some kind of trouble, that alone is not going to > solve the problem. Indeed. My suggestion was about getting the built-in net tick out of the way in order to try other methods. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ulf.wiger@REDACTED Fri Sep 25 13:43:25 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 25 Sep 2009 13:43:25 +0200 Subject: [erlang-questions] documentation of incremental backups in mnesia In-Reply-To: References: <4ABC6E80.2090205@erlang-consulting.com> Message-ID: <4ABCACDD.9080701@erlang-consulting.com> Chandru wrote: > Hi Ulf, > > 2009/9/25 Ulf Wiger > > > It is documented, sort of, but only in passing. You have to > read pretty carefully to catch it at all, and you probably > need to experiment a bit before understanding how it works. > > > Could you please point to where it is mentioned in passing. It seems to > be a very useful feature. Good point. It's actually not documented at all. :) Long ago, it was mentioned in the release notes: http://erlang.org/documentation/doc-5.0.1/lib/mnesia-3.9.2/doc/html/notes.html (That particular page is from the year 2000. I'm guessing it appeared first in the OTP R5B release.) BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From puzza007@REDACTED Fri Sep 25 14:19:57 2009 From: puzza007@REDACTED (Paul Oliver) Date: Fri, 25 Sep 2009 13:19:57 +0100 Subject: [erlang-questions] blowfish ECB implementation in erlang? In-Reply-To: <38ff4e6b0909231233i4435a436v170a8b1953b22c4e@mail.gmail.com> References: <38ff4e6b0909231233i4435a436v170a8b1953b22c4e@mail.gmail.com> Message-ID: On Wed, Sep 23, 2009 at 8:33 PM, Larry Ogrodnek wrote: > Hi, I'm wondering if anyone has put together an implementation of blowfish > ECB? > > thanks. > Hi Larry, Check the following page for the patch that added cfb64. http://www.nabble.com/Patch-to-add-Blowfish-cfb64-to-crypto-app-td24232164.html You should almost be able to copy this verbatim to add ecb. Cheers, Paul. From chandrashekhar.mullaparthi@REDACTED Fri Sep 25 14:39:03 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 25 Sep 2009 13:39:03 +0100 Subject: [erlang-questions] documentation of incremental backups in mnesia In-Reply-To: <4ABCACDD.9080701@erlang-consulting.com> References: <4ABC6E80.2090205@erlang-consulting.com> <4ABCACDD.9080701@erlang-consulting.com> Message-ID: 2009/9/25 Ulf Wiger > Chandru wrote: > >> Hi Ulf, >> >> 2009/9/25 Ulf Wiger > ulf.wiger@REDACTED>> >> >> It is documented, sort of, but only in passing. You have to >> read pretty carefully to catch it at all, and you probably >> need to experiment a bit before understanding how it works. >> >> >> Could you please point to where it is mentioned in passing. It seems to >> be a very useful feature. >> > > Good point. It's actually not documented at all. :) > > Long ago, it was mentioned in the release notes: > > > http://erlang.org/documentation/doc-5.0.1/lib/mnesia-3.9.2/doc/html/notes.html > > (That particular page is from the year 2000. I'm guessing > it appeared first in the OTP R5B release.) > > Thank you. Your memory is very good! cheers Chandru From raimo+erlang-questions@REDACTED Fri Sep 25 14:46:13 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 25 Sep 2009 14:46:13 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname In-Reply-To: <200909251203380496915@its3.ch> References: <20090925075437.GA27366@erix.ericsson.se> <200909251203380496915@its3.ch> Message-ID: <20090925124613.GA32148@erix.ericsson.se> On Fri, Sep 25, 2009 at 12:03:38PM +0200, info wrote: > Hi Raimo, > Note that I appreciate your help because it seems that the problem is not in erlang. > > On Thu, Sep 24, 2009 at 04:55:57PM +0200, info wrote: > > I also tried to add the address of my dns server. > > inet_db:add_ns({ip}). > > But no effect. > > > > inet_getrc(). gives now: > > > > [{host,{127,0,0,1},["my_host"]}, > > {nameserver,{192,168,1,33}}] > > > You have {res_lookup,[native,file]} meaning first is the OS > gethostbyname equivalent function used. During the startup > of Erlang it discovered that "my_host" could not be > resolved so it entered "my_host" as {127,0,0,1} into the > internal hosts table to ensure that inet:gethostbyname("my_host") > would succeed. > > But the native lookup method times out, probably due > to misconfigured name resolving in the OS, so the > internal hosts table is never used. > > How long does the call inet:gethostbyname("my_host") > take to return {error,timeout}? > >>>>>>>>>>>> 6-7 seconds Ok... > > Can the native lookup method look up any name at all? > Try: inet_gethost_native:gethostbyname("localhost"). > >>>>>>>>>>>> {error,timeout} > inet_gethost_native:gethostbyname("my_host"). > >>>>>>>>>>>> {error,timeout} > inet_gethost_native:gethostbyname("www.google.com"). > >>>>>>>>>>>> {error,timeout} Nothing worked. > >From a Windows command shell, try: > nslookup my_host > >>>>>>>>>>>>server: my_host.my_domain.local > >>>>>>>>>>>>address: 192.168.1.33 > >>>>>>>>>>>>name: my_host.my_domain.local > >>>>>>>>>>>>address: 192.168.1.33 > nslookup localhost > >>>>>>>>>>>>server: my_host.my_domain.local > >>>>>>>>>>>>address: 192.168.1.33 > >>>>>>>>>>>>my_host.my_domain.local doesn't find localhost: no existing domain (*** I translated from french ! ) Well well, of course that worked... This means your host is its own DNS server (?). > > Pay special attention to which nameserver that answers. > > If you can not fix the network configuration, which will > solve your problem, you can configure erlang to not > use the native lookup method. The erlang DNS resolver > client was much improved to R13B02, but it worked > well in R13B01 too. > >>>>>>>>>>>>My problem is I don't see what is wrong in my network configuration or in my dns server configuration ! > Set the lookup method to [file,dns] and add your > nameserver as you tried: > inet_db:add_ns({192,168,1,33}). > >>>>>>>>>>>>ok > inet_db:set_lookup([file,dns]). > >>>>>>>>>>>>ok > Then try inet:gethostbyname("my_host"), > >>>>>>>>>>>>{ok,{hostent,"my-host",[],inet,4,[{127,0,0,1}]}} BETTER !! That is the patched 'file' lookup method that found the name. This is often enough to run a local non-distributed node. > or even inet_res:gethostbyname("my_host). > >>>>>>>>>>>>{error,timeout} WHY ? Maybe your firewall does not allow DNS requests from erlang. Try: rr(inet_res). inet_res:nslookup("my_host.my_domain.local", in, a, [{{192,168,1,33},53}]). And, maybe using the external IP address is what is not allowed. inet_res:nslookup("my_host.my_domain.local", in, a, [{{127,0,0,1},53}]). > But I hope you can fix your network configuration instead... > > > > > > > n Wed, Sep 23, 2009 at 04:31:35PM +0200, info wrote: > > > Hi all, > > > On a windows 2003 server sbs this command returns {ok,"my_host"} : > > > > > > {ok,N}=inet:gethostname(). > > > > > > But the following command returns {error,timeout} : > > > > > > inet:gethostbyname(N). > > > > > > On a Vista machine, the previous command returns a correct answer. > > > > > > I know that it is not an erlang problem but if you could give me support, I will appreciate ;-) > > > > Erlang may have problems configuring name lookups. > > Give me the output from ets:tab2list(inet_db). > > > > Which erlang release are you running, it sounds like pre-R13B02? > > > > Try starting with erl -name foo > > and do the same, ets:tab2list/1 too. > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jeraymond@REDACTED Fri Sep 25 14:48:34 2009 From: jeraymond@REDACTED (Jeremy Raymond) Date: Fri, 25 Sep 2009 08:48:34 -0400 Subject: When to use receive/call with timeouts? Message-ID: <59da11980909250548vd73f609kaebbe4e748f922bc@mail.gmail.com> As I'm learning Erlang I was wondering if there were any best practices or general guidelines as to appropriate use of timeouts when using the receive primitive or gen_server:call. For example is it good to use them for remote processes, but try to avoid them for local processes? Are there any general scenarios as to when you would want to use them or not want to use them. Thanks, Jeremy From peter@REDACTED Fri Sep 25 14:55:09 2009 From: peter@REDACTED (Peter Donner) Date: Fri, 25 Sep 2009 14:55:09 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname In-Reply-To: <200909251203380496915@its3.ch> References: <20090924071607.GA27757@erix.ericsson.se>, <200909241655568439572@its3.ch>, <20090925075437.GA27366@erix.ericsson.se> <200909251203380496915@its3.ch> Message-ID: <4ABCBDAD.7000907@markup.at> info wrote: > nslookup localhost >>>>>>>>>>>>> server: my_host.my_domain.local >>>>>>>>>>>>> address: 192.168.1.33 >>>>>>>>>>>>> my_host.my_domain.local doesn't find localhost: no existing domain (*** I translated from french ! ) You should get the address 127.0.0.1 for localhost. Did you remove the entry "127.0.0.1 localhost" in your hosts file? -- Peter Donner From olopierpa@REDACTED Fri Sep 25 15:03:00 2009 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Fri, 25 Sep 2009 15:03:00 +0200 Subject: [erlang-questions] Temp directory/file from Erlang In-Reply-To: <4ABC9706.8060101@fissore.org> References: <034AD643-8E7E-4EC7-9A07-5137E560F163@gmail.com> <7352e43a0909250256lba64019i55463550bc2351a9@mail.gmail.com> <4ABC9706.8060101@fissore.org> Message-ID: <7352e43a0909250603t79a3ce17na52ad3b1136e1cf8@mail.gmail.com> On Fri, Sep 25, 2009 at 12:10, Federico Fissore wrote: > Pierpaolo Bernardi, il 25/09/2009 11:56, ha scritto: >> On Fri, Sep 25, 2009 at 10:53, Dmitrii Dimandt wrote: >>> Hi all >>> >>> Is there a way to find out the path to the system's temp directory and/or >>> create a temporary file? >> >> The usual way is to check the TEMP and/or TMP environment variables. > > uhm, on linux I have no TEMP nor TMP env vars oops. You are right. I misremembered the unix convention. On unix, /tmp and /var/tmp are assumed to exist. On windows is used the env var TEMP. (More info here: http://en.wikipedia.org/wiki/Temporary_folder ) Ciao P. From info@REDACTED Fri Sep 25 15:16:41 2009 From: info@REDACTED (info) Date: Fri, 25 Sep 2009 15:16:41 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname References: <20090925075437.GA27366@erix.ericsson.se>, <200909251203380496915@its3.ch>, <20090925124613.GA32148@erix.ericsson.se> Message-ID: <200909251516379712491@its3.ch> On Fri, Sep 25, 2009 at 12:03:38PM +0200, info wrote: > Hi Raimo, > Note that I appreciate your help because it seems that the problem is not in erlang. > > On Thu, Sep 24, 2009 at 04:55:57PM +0200, info wrote: > > I also tried to add the address of my dns server. > > inet_db:add_ns({ip}). > > But no effect. > > > > inet_getrc(). gives now: > > > > [{host,{127,0,0,1},["my_host"]}, > > {nameserver,{192,168,1,33}}] > > > You have {res_lookup,[native,file]} meaning first is the OS > gethostbyname equivalent function used. During the startup > of Erlang it discovered that "my_host" could not be > resolved so it entered "my_host" as {127,0,0,1} into the > internal hosts table to ensure that inet:gethostbyname("my_host") > would succeed. > > But the native lookup method times out, probably due > to misconfigured name resolving in the OS, so the > internal hosts table is never used. > > How long does the call inet:gethostbyname("my_host") > take to return {error,timeout}? > > > > > > > > > > > > > 6-7 seconds Ok... > > Can the native lookup method look up any name at all? > Try: inet_gethost_native:gethostbyname("localhost"). > > > > > > > > > > > > > {error,timeout} > inet_gethost_native:gethostbyname("my_host"). > > > > > > > > > > > > > {error,timeout} > inet_gethost_native:gethostbyname("www.google.com"). > > > > > > > > > > > > > {error,timeout} Nothing worked. > >From a Windows command shell, try: > nslookup my_host > > > > > > > > > > > > >server: my_host.my_domain.local > > > > > > > > > > > > >address: 192.168.1.33 > > > > > > > > > > > > >name: my_host.my_domain.local > > > > > > > > > > > > >address: 192.168.1.33 > nslookup localhost > > > > > > > > > > > > >server: my_host.my_domain.local > > > > > > > > > > > > >address: 192.168.1.33 > > > > > > > > > > > > >my_host.my_domain.local doesn't find localhost: no existing domain (*** I translated from french ! ) Well well, of course that worked... This means your host is its own DNS server (?). > > Pay special attention to which nameserver that answers. > > If you can not fix the network configuration, which will > solve your problem, you can configure erlang to not > use the native lookup method. The erlang DNS resolver > client was much improved to R13B02, but it worked > well in R13B01 too. > > > > > > > > > > > > >My problem is I don't see what is wrong in my network configuration or in my dns server configuration ! > Set the lookup method to [file,dns] and add your > nameserver as you tried: > inet_db:add_ns({192,168,1,33}). > > > > > > > > > > > > >ok > inet_db:set_lookup([file,dns]). > > > > > > > > > > > > >ok > Then try inet:gethostbyname("my_host"), > > > > > > > > > > > > >{ok,{hostent,"my-host",[],inet,4,[{127,0,0,1}]}} BETTER !! That is the patched 'file' lookup method that found the name. This is often enough to run a local non-distributed node. > or even inet_res:gethostbyname("my_host). > > > > > > > > > > > > >{error,timeout} WHY ? Maybe your firewall does not allow DNS requests from erlang. Try: rr(inet_res). +++++++ [connect_opts,dns_header,dns_query,dns_rec,dns_rr,hostent, listen_opts,sctp_opts,udp_opts] inet_res:nslookup("my_host.my_domain.local", in, a, [{{192,168,1,33},53}]). +++++++ {ok,#dns_rec{header = #dns_header{id = 2,qr = 1,opcode = 0, aa = 1,tc = 0,rd = 1,ra = 1,pr = 0, rcode = 0}, qdlist = [#dns_query{domain = "my_host.my_domain.local", type = a,class = in}], anlist = [#dns_rr{domain = "my_host.my_domain.local",class = in, type = a,cnt = 0,tm = undefined,ttl = 3600, bm = [], data = {192,168,1,33}, func = false}], nslist = [],arlist = []}} And, maybe using the external IP address is what is not allowed. inet_res:nslookup("my_host.my_domain.local", in, a, [{{127,0,0,1},53}]). +++++++++ {ok,#dns_rec{header = #dns_header{id = 3,qr = 1,opcode = 0, aa = 1,tc = 0,rd = 1,ra = 1,pr = 0, rcode = 0}, qdlist = [#dns_query{domain = "my_host.my_domain.local", type = a,class = in}], anlist = [#dns_rr{domain = "my_host.my_domain.local",class = in, type = a,cnt = 0,tm = undefined,ttl = 3600, bm = [], data = {192,168,1,33}, func = false}], nslist = [],arlist = []}} +++++++++++++ inet_gethost_native:gethostbyname("www.google.com"). {error,timeout} inet_gethost_native:gethostbyname("localhost"). {error,timeout} inet_gethost_native:gethostbyname("my_host"). {error,timeout} DO YOU SEE A DIFFERENCE ? > But I hope you can fix your network configuration instead... > > > > > > > n Wed, Sep 23, 2009 at 04:31:35PM +0200, info wrote: > > > Hi all, > > > On a windows 2003 server sbs this command returns {ok,"my_host"} : > > > > > > {ok,N}=inet:gethostname(). > > > > > > But the following command returns {error,timeout} : > > > > > > inet:gethostbyname(N). > > > > > > On a Vista machine, the previous command returns a correct answer. > > > > > > I know that it is not an erlang problem but if you could give me support, I will appreciate ;-) > > > > Erlang may have problems configuring name lookups. > > Give me the output from ets:tab2list(inet_db). > > > > Which erlang release are you running, it sounds like pre-R13B02? > > > > Try starting with erl -name foo > > and do the same, ets:tab2list/1 too. > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From info@REDACTED Fri Sep 25 15:19:10 2009 From: info@REDACTED (info) Date: Fri, 25 Sep 2009 15:19:10 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname References: <20090924071607.GA27757@erix.ericsson.se>, <200909241655568439572@its3.ch>, <20090925075437.GA27366@erix.ericsson.se>, <200909251203380496915@its3.ch>, <4ABCBDAD.7000907@markup.at> Message-ID: <200909251519104404560@its3.ch> No. This entry is in my hosts file. info wrote: > nslookup localhost > > > > > > > > > > > > > server: my_host.my_domain.local > > > > > > > > > > > > > address: 192.168.1.33 > > > > > > > > > > > > > my_host.my_domain.local doesn't find localhost: no existing domain (*** I translated from french ! ) You should get the address 127.0.0.1 for localhost. Did you remove the entry "127.0.0.1 localhost" in your hosts file? -- Peter Donner ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From hakan@REDACTED Fri Sep 25 15:24:59 2009 From: hakan@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Fri, 25 Sep 2009 15:24:59 +0200 Subject: [erlang-questions] documentation of incremental backups in mnesia In-Reply-To: <4ABCACDD.9080701@erlang-consulting.com> References: <4ABC6E80.2090205@erlang-consulting.com> <4ABCACDD.9080701@erlang-consulting.com> Message-ID: <922d05850909250624i6acbe7a1p2bf51c125170f765@mail.gmail.com> On Fri, Sep 25, 2009 at 1:43 PM, Ulf Wiger wrote: > Chandru wrote: >> >> Hi Ulf, >> >> 2009/9/25 Ulf Wiger > > >> >> ? ?It is documented, sort of, but only in passing. You have to >> ? ?read pretty carefully to catch it at all, and you probably >> ? ?need to experiment a bit before understanding how it works. >> >> >> ?Could you please point to where it is mentioned in passing. It seems to >> be a very useful feature. > > Good point. It's actually not documented at all. :) > > Long ago, it was mentioned in the release notes: > > http://erlang.org/documentation/doc-5.0.1/lib/mnesia-3.9.2/doc/html/notes.html > > (That particular page is from the year 2000. I'm guessing > it appeared first in the OTP R5B release.) Yes, some new nice backup features (incremental backup, selective backup and local backup) was introduced in september 1999 for OTP R5B. I was quite surprised when you pointed out that they not were documented. Perhaps it is time to document them, but I think that we ought to add more test cases for these features before doing that. As they have been undocumented for almost exactly a decade I suspect that very few (if any) users have been using them. /H?kan From raimo+erlang-questions@REDACTED Fri Sep 25 15:33:35 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 25 Sep 2009 15:33:35 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname In-Reply-To: <200909251516379712491@its3.ch> References: <20090925124613.GA32148@erix.ericsson.se> <200909251516379712491@its3.ch> Message-ID: <20090925133335.GB383@erix.ericsson.se> On Fri, Sep 25, 2009 at 03:16:41PM +0200, info wrote: > On Fri, Sep 25, 2009 at 12:03:38PM +0200, info wrote: > > Hi Raimo, > > Note that I appreciate your help because it seems that the problem is not in erlang. > > > > On Thu, Sep 24, 2009 at 04:55:57PM +0200, info wrote: > > > I also tried to add the address of my dns server. > > > inet_db:add_ns({ip}). > > > But no effect. > > > > > > inet_getrc(). gives now: > > > > > > [{host,{127,0,0,1},["my_host"]}, > > > {nameserver,{192,168,1,33}}] > > > > > You have {res_lookup,[native,file]} meaning first is the OS > > gethostbyname equivalent function used. During the startup > > of Erlang it discovered that "my_host" could not be > > resolved so it entered "my_host" as {127,0,0,1} into the > > internal hosts table to ensure that inet:gethostbyname("my_host") > > would succeed. > > > > But the native lookup method times out, probably due > > to misconfigured name resolving in the OS, so the > > internal hosts table is never used. > > > > How long does the call inet:gethostbyname("my_host") > > take to return {error,timeout}? > > > > > > > > > > > > > > 6-7 seconds > > Ok... > > > > > Can the native lookup method look up any name at all? > > Try: inet_gethost_native:gethostbyname("localhost"). > > > > > > > > > > > > > > {error,timeout} > > inet_gethost_native:gethostbyname("my_host"). > > > > > > > > > > > > > > {error,timeout} > > inet_gethost_native:gethostbyname("www.google.com"). > > > > > > > > > > > > > > {error,timeout} > > Nothing worked. > > > >From a Windows command shell, try: > > nslookup my_host > > > > > > > > > > > > > >server: my_host.my_domain.local > > > > > > > > > > > > > >address: 192.168.1.33 > > > > > > > > > > > > > >name: my_host.my_domain.local > > > > > > > > > > > > > >address: 192.168.1.33 > > nslookup localhost > > > > > > > > > > > > > >server: my_host.my_domain.local > > > > > > > > > > > > > >address: 192.168.1.33 > > > > > > > > > > > > > >my_host.my_domain.local doesn't find localhost: no existing domain (*** I translated from french ! ) > > Well well, of course that worked... > This means your host is its own DNS server (?). > > > > > Pay special attention to which nameserver that answers. > > > > If you can not fix the network configuration, which will > > solve your problem, you can configure erlang to not > > use the native lookup method. The erlang DNS resolver > > client was much improved to R13B02, but it worked > > well in R13B01 too. > > > > > > > > > > > > > >My problem is I don't see what is wrong in my network configuration or in my dns server configuration ! > > Set the lookup method to [file,dns] and add your > > nameserver as you tried: > > inet_db:add_ns({192,168,1,33}). > > > > > > > > > > > > > >ok > > inet_db:set_lookup([file,dns]). > > > > > > > > > > > > > >ok > > Then try inet:gethostbyname("my_host"), > > > > > > > > > > > > > >{ok,{hostent,"my-host",[],inet,4,[{127,0,0,1}]}} BETTER !! > > That is the patched 'file' lookup method that found the name. > This is often enough to run a local non-distributed node. > > > or even inet_res:gethostbyname("my_host). > > > > > > > > > > > > > >{error,timeout} WHY ? > > Maybe your firewall does not allow DNS requests from erlang. > Try: > rr(inet_res). > +++++++ > [connect_opts,dns_header,dns_query,dns_rec,dns_rr,hostent, > listen_opts,sctp_opts,udp_opts] > > inet_res:nslookup("my_host.my_domain.local", in, a, [{{192,168,1,33},53}]). > +++++++ > {ok,#dns_rec{header = #dns_header{id = 2,qr = 1,opcode = 0, > aa = 1,tc = 0,rd = 1,ra = 1,pr = 0, > rcode = 0}, > qdlist = [#dns_query{domain = "my_host.my_domain.local", > type = a,class = in}], > anlist = [#dns_rr{domain = "my_host.my_domain.local",class = in, > type = a,cnt = 0,tm = undefined,ttl = 3600, > bm = [], > data = {192,168,1,33}, > func = false}], > nslist = [],arlist = []}} > > > And, maybe using the external IP address is what is not allowed. > inet_res:nslookup("my_host.my_domain.local", in, a, [{{127,0,0,1},53}]). > +++++++++ > {ok,#dns_rec{header = #dns_header{id = 3,qr = 1,opcode = 0, > aa = 1,tc = 0,rd = 1,ra = 1,pr = 0, > rcode = 0}, > qdlist = [#dns_query{domain = "my_host.my_domain.local", > type = a,class = in}], > anlist = [#dns_rr{domain = "my_host.my_domain.local",class = in, > type = a,cnt = 0,tm = undefined,ttl = 3600, > bm = [], > data = {192,168,1,33}, > func = false}], > nslist = [],arlist = []}} > +++++++++++++ > inet_gethost_native:gethostbyname("www.google.com"). > {error,timeout} > inet_gethost_native:gethostbyname("localhost"). > {error,timeout} > inet_gethost_native:gethostbyname("my_host"). > {error,timeout} > > DO YOU SEE A DIFFERENCE ? Nope. This means that: inet_db:add_ns({127,0,0,1}). inet_db:set_domain("my_domain.local"). inet_db:add_search("my_domain.local"). inet_db:set_lookup([file,dns]). should make inet:gethostbyname("my_host") work. But that is a workaround. The native lookup should work. Windows nslookup also probably bypasses the normal name resolving functions and does direct DNS queries. Is it maybe an issue with your DHCP server that i does not state a DNS server, since you use your local machine as DNS server. Google for name resolving for Windows 2003... > > > But I hope you can fix your network configuration instead... > > > > > > > > > > > n Wed, Sep 23, 2009 at 04:31:35PM +0200, info wrote: > > > > Hi all, > > > > On a windows 2003 server sbs this command returns {ok,"my_host"} : > > > > > > > > {ok,N}=inet:gethostname(). > > > > > > > > But the following command returns {error,timeout} : > > > > > > > > inet:gethostbyname(N). > > > > > > > > On a Vista machine, the previous command returns a correct answer. > > > > > > > > I know that it is not an erlang problem but if you could give me support, I will appreciate ;-) > > > > > > Erlang may have problems configuring name lookups. > > > Give me the output from ets:tab2list(inet_db). > > > > > > Which erlang release are you running, it sounds like pre-R13B02? > > > > > > Try starting with erl -name foo > > > and do the same, ets:tab2list/1 too. > > > > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From info@REDACTED Fri Sep 25 15:46:01 2009 From: info@REDACTED (info) Date: Fri, 25 Sep 2009 15:46:01 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname References: <20090925124613.GA32148@erix.ericsson.se>, <200909251516379712491@its3.ch>, <20090925133335.GB383@erix.ericsson.se> Message-ID: <200909251546014289830@its3.ch> On Fri, Sep 25, 2009 at 03:16:41PM +0200, info wrote: > On Fri, Sep 25, 2009 at 12:03:38PM +0200, info wrote: > > Hi Raimo, > > Note that I appreciate your help because it seems that the problem is not in erlang. > > > > On Thu, Sep 24, 2009 at 04:55:57PM +0200, info wrote: > > > I also tried to add the address of my dns server. > > > inet_db:add_ns({ip}). > > > But no effect. > > > > > > inet_getrc(). gives now: > > > > > > [{host,{127,0,0,1},["my_host"]}, > > > {nameserver,{192,168,1,33}}] > > > > > You have {res_lookup,[native,file]} meaning first is the OS > > gethostbyname equivalent function used. During the startup > > of Erlang it discovered that "my_host" could not be > > resolved so it entered "my_host" as {127,0,0,1} into the > > internal hosts table to ensure that inet:gethostbyname("my_host") > > would succeed. > > > > But the native lookup method times out, probably due > > to misconfigured name resolving in the OS, so the > > internal hosts table is never used. > > > > How long does the call inet:gethostbyname("my_host") > > take to return {error,timeout}? > > > > > > > > > > > > > > 6-7 seconds > > Ok... > > > > > Can the native lookup method look up any name at all? > > Try: inet_gethost_native:gethostbyname("localhost"). > > > > > > > > > > > > > > {error,timeout} > > inet_gethost_native:gethostbyname("my_host"). > > > > > > > > > > > > > > {error,timeout} > > inet_gethost_native:gethostbyname("www.google.com"). > > > > > > > > > > > > > > {error,timeout} > > Nothing worked. > > > >From a Windows command shell, try: > > nslookup my_host > > > > > > > > > > > > > >server: my_host.my_domain.local > > > > > > > > > > > > > >address: 192.168.1.33 > > > > > > > > > > > > > >name: my_host.my_domain.local > > > > > > > > > > > > > >address: 192.168.1.33 > > nslookup localhost > > > > > > > > > > > > > >server: my_host.my_domain.local > > > > > > > > > > > > > >address: 192.168.1.33 > > > > > > > > > > > > > >my_host.my_domain.local doesn't find localhost: no existing domain (*** I translated from french ! ) > > Well well, of course that worked... > This means your host is its own DNS server (?). > > > > > Pay special attention to which nameserver that answers. > > > > If you can not fix the network configuration, which will > > solve your problem, you can configure erlang to not > > use the native lookup method. The erlang DNS resolver > > client was much improved to R13B02, but it worked > > well in R13B01 too. > > > > > > > > > > > > > >My problem is I don't see what is wrong in my network configuration or in my dns server configuration ! > > Set the lookup method to [file,dns] and add your > > nameserver as you tried: > > inet_db:add_ns({192,168,1,33}). > > > > > > > > > > > > > >ok > > inet_db:set_lookup([file,dns]). > > > > > > > > > > > > > >ok > > Then try inet:gethostbyname("my_host"), > > > > > > > > > > > > > >{ok,{hostent,"my-host",[],inet,4,[{127,0,0,1}]}} BETTER !! > > That is the patched 'file' lookup method that found the name. > This is often enough to run a local non-distributed node. > > > or even inet_res:gethostbyname("my_host). > > > > > > > > > > > > > >{error,timeout} WHY ? > > Maybe your firewall does not allow DNS requests from erlang. > Try: > rr(inet_res). > +++++++ > [connect_opts,dns_header,dns_query,dns_rec,dns_rr,hostent, > listen_opts,sctp_opts,udp_opts] > > inet_res:nslookup("my_host.my_domain.local", in, a, [{{192,168,1,33},53}]). > +++++++ > {ok,#dns_rec{header = #dns_header{id = 2,qr = 1,opcode = 0, > aa = 1,tc = 0,rd = 1,ra = 1,pr = 0, > rcode = 0}, > qdlist = [#dns_query{domain = "my_host.my_domain.local", > type = a,class = in}], > anlist = [#dns_rr{domain = "my_host.my_domain.local",class = in, > type = a,cnt = 0,tm = undefined,ttl = 3600, > bm = [], > data = {192,168,1,33}, > func = false}], > nslist = [],arlist = []}} > > > And, maybe using the external IP address is what is not allowed. > inet_res:nslookup("my_host.my_domain.local", in, a, [{{127,0,0,1},53}]). > +++++++++ > {ok,#dns_rec{header = #dns_header{id = 3,qr = 1,opcode = 0, > aa = 1,tc = 0,rd = 1,ra = 1,pr = 0, > rcode = 0}, > qdlist = [#dns_query{domain = "my_host.my_domain.local", > type = a,class = in}], > anlist = [#dns_rr{domain = "my_host.my_domain.local",class = in, > type = a,cnt = 0,tm = undefined,ttl = 3600, > bm = [], > data = {192,168,1,33}, > func = false}], > nslist = [],arlist = []}} > +++++++++++++ > inet_gethost_native:gethostbyname("www.google.com"). > {error,timeout} > inet_gethost_native:gethostbyname("localhost"). > {error,timeout} > inet_gethost_native:gethostbyname("my_host"). > {error,timeout} > > DO YOU SEE A DIFFERENCE ? Nope. This means that: inet_db:add_ns({127,0,0,1}). inet_db:set_domain("my_domain.local"). inet_db:add_search("my_domain.local"). inet_db:set_lookup([file,dns]). should make inet:gethostbyname("my_host") work. ******* Correct: inet:gethostbyname("my_host"). {ok,{hostent,"my_host",[],inet,4,[{127,0,0,1}]}} But that is a workaround. The native lookup should work. Windows nslookup also probably bypasses the normal name resolving functions and does direct DNS queries. Is it maybe an issue with your DHCP server that i does not state a DNS server, ****** Client DNS service and Client DHCP service are running. since you use your local machine as DNS server. ****** Exactly: but it is not forbidden ? Normally I should not use a server to access to the web with a browser ... but it is another story. Google for name resolving for Windows 2003... ****** I already past long times on Google ... > > > But I hope you can fix your network configuration instead... > > > > > > > > > > > n Wed, Sep 23, 2009 at 04:31:35PM +0200, info wrote: > > > > Hi all, > > > > On a windows 2003 server sbs this command returns {ok,"my_host"} : > > > > > > > > {ok,N}=inet:gethostname(). > > > > > > > > But the following command returns {error,timeout} : > > > > > > > > inet:gethostbyname(N). > > > > > > > > On a Vista machine, the previous command returns a correct answer. > > > > > > > > I know that it is not an erlang problem but if you could give me support, I will appreciate ;-) > > > > > > Erlang may have problems configuring name lookups. > > > Give me the output from ets:tab2list(inet_db). > > > > > > Which erlang release are you running, it sounds like pre-R13B02? > > > > > > Try starting with erl -name foo > > > and do the same, ets:tab2list/1 too. > > > > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From guilherme.silveira@REDACTED Fri Sep 25 16:59:30 2009 From: guilherme.silveira@REDACTED (Guilherme Silveira) Date: Fri, 25 Sep 2009 11:59:30 -0300 Subject: changing the internal message transfer protocol Message-ID: <581bf6160909250759o1b634f01m7c1810143c47d0f1@mail.gmail.com> Hello there, I would like to change the internal protocol used by erlang when transfering information to/from differente nodes as in: Pid = spawn(NodeName, M, F, A). Pid!{values} I am not looking into changing the values->binary serialization process, but the spawn/send/receive message internals... I believe I will have to work out on the erlang implementation code, is that right? Anyone can help me out where to find the code or docs to change/read? Regards Guilherme Silveira Caelum | Ensino e Inova??o SP (11) 5571-2751 RJ (21) 2220-4156 Celular (11) 83358084 From freza@REDACTED Fri Sep 25 17:21:45 2009 From: freza@REDACTED (Jachym Holecek) Date: Fri, 25 Sep 2009 17:21:45 +0200 Subject: [erlang-questions] changing the internal message transfer protocol In-Reply-To: <581bf6160909250759o1b634f01m7c1810143c47d0f1@mail.gmail.com> References: <581bf6160909250759o1b634f01m7c1810143c47d0f1@mail.gmail.com> Message-ID: <20090925152145.GA5947@hanele> Hello, # Guilherme Silveira 2009-09-25: > I would like to change the internal protocol used by erlang when > transfering information to/from differente nodes as in: > > Pid = spawn(NodeName, M, F, A). See lib/kernel/src/erlang.erl:spawn/4. > Pid!{values} See erts/emulator/beam/bif.c:ebif_bang_2(). Just a result of quick code inspection, I don't posses actual wisdom regarding how they work ;-). Have fun, -- Jachym From guilherme.silveira@REDACTED Fri Sep 25 18:08:05 2009 From: guilherme.silveira@REDACTED (Guilherme Silveira) Date: Fri, 25 Sep 2009 13:08:05 -0300 Subject: [erlang-questions] changing the internal message transfer protocol In-Reply-To: <20090925152145.GA5947@hanele> References: <581bf6160909250759o1b634f01m7c1810143c47d0f1@mail.gmail.com> <20090925152145.GA5947@hanele> Message-ID: <581bf6160909250908v50f58c80kbbd7d9f9a66fdf94@mail.gmail.com> Thanks Jachym, >> Pid = spawn(NodeName, M, F, A). > See lib/kernel/src/erlang.erl:spawn/4. spawn invokes gen_server when NodeName =/= node(), which invokes gen:call, which invokes erlang:send. But I was unable to find who exports erlang:send... the erlang module (erlang.erl) only exports dsend, which connnects+invokes send. Reading the net_kernel and do_connect implementations, it seems as connections are treated with their own local pids and kept alive. But still... Any suggestions where to find the send implementation? Regards > >> Pid!{values} > > See erts/emulator/beam/bif.c:ebif_bang_2(). > > Just a result of quick code inspection, I don't posses actual wisdom > regarding how they work ;-). > > Have fun, > ? ? ? ?-- Jachym > From kostis@REDACTED Fri Sep 25 18:11:55 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 25 Sep 2009 19:11:55 +0300 Subject: [erlang-questions] changing the internal message transfer protocol In-Reply-To: <581bf6160909250908v50f58c80kbbd7d9f9a66fdf94@mail.gmail.com> References: <581bf6160909250759o1b634f01m7c1810143c47d0f1@mail.gmail.com> <20090925152145.GA5947@hanele> <581bf6160909250908v50f58c80kbbd7d9f9a66fdf94@mail.gmail.com> Message-ID: <4ABCEBCB.5070701@cs.ntua.gr> Guilherme Silveira wrote: > Thanks Jachym, > >>> Pid = spawn(NodeName, M, F, A). >> See lib/kernel/src/erlang.erl:spawn/4. > spawn invokes gen_server when NodeName =/= node(), which invokes > gen:call, which invokes erlang:send. > > But I was unable to find who exports erlang:send... the erlang module > (erlang.erl) only exports dsend, which connnects+invokes send. > > Any suggestions where to find the send implementation? The send operation is a built-in function (BIF) and BEAM instruction implemented in C. Kostis From clist@REDACTED Fri Sep 25 18:13:42 2009 From: clist@REDACTED (Angel) Date: Fri, 25 Sep 2009 18:13:42 +0200 Subject: reinventing the wheel? Message-ID: <200909251813.42522.clist@uah.es> The multikernel... http://www.barrelfish.org/barrelfish_sosp09.pdf Seems pretty interesting and IMHO is so erlangishh... Its time for a microkernel... What do you think about? -- Este correo no tiene dibujos. Las formas extra?as en la pantalla son letras. __________________________________________ Clist UAH a.k.a Angel __________________________________________ HONEY BUNNY - Any of you *uckin' pricks move and I'll execute every mother*ucking last one of you. From freza@REDACTED Fri Sep 25 18:54:23 2009 From: freza@REDACTED (Jachym Holecek) Date: Fri, 25 Sep 2009 18:54:23 +0200 Subject: [erlang-questions] changing the internal message transfer protocol In-Reply-To: <581bf6160909250908v50f58c80kbbd7d9f9a66fdf94@mail.gmail.com> References: <581bf6160909250759o1b634f01m7c1810143c47d0f1@mail.gmail.com> <20090925152145.GA5947@hanele> <581bf6160909250908v50f58c80kbbd7d9f9a66fdf94@mail.gmail.com> Message-ID: <20090925165423.GA6174@hanele> # Guilherme Silveira 2009-09-25: > >> Pid = spawn(NodeName, M, F, A). > > See lib/kernel/src/erlang.erl:spawn/4. > spawn invokes gen_server when NodeName =/= node(), which invokes > gen:call, which invokes erlang:send. > > But I was unable to find who exports erlang:send... the erlang module > (erlang.erl) only exports dsend, which connnects+invokes send. Yep, some of the functions in various "core" modules are implemented directly in C instead of the corresponding Erlang module. Such functions are called "builtin functions" and live in bif.c/erl_bif_${module}.c and possibly other places I missed, see below on how to lookup their home. > Reading the net_kernel and do_connect implementations, it seems as > connections are treated with their own local pids and kept alive. But > still... > > Any suggestions where to find the send implementation? Builtin functions are described in erts/emulator/beam/bif.tab, this is used to generate BIF dispatch table at compile time using the erts/emulator/utils/make_tables script. For send/2 we have: bif erlang:send/2 which according to make_tables seems to mean (and my Perl-fu is very weak) that it's implemented by the send_2() function -- this lives in bif.c as "grep -rs '^send_2[(]' erts/" reveals. This has been a fair amount of guesswork; I'm sure someone will correct me if I'm wrong. HTH, -- Jachym From chris.newcombe@REDACTED Fri Sep 25 19:32:30 2009 From: chris.newcombe@REDACTED (Chris Newcombe) Date: Fri, 25 Sep 2009 10:32:30 -0700 Subject: Is driver_mk_atom() thread safe? Message-ID: <781dd98c0909251032m3512a4cwf0166ef2bbfd7d11@mail.gmail.com> Hi BEAM / OTP team, I have a driver that uses private thread-pools. I know there are heavy restrictions on calling most erl_driver functions from private threads, and for some functions the documentation now helpfully says whether they are thread-safe, or only thread-safe when using the SMP emulator etc. The documentation for driver_mk_atom() doesn't mention thread-safety (see below). Please could it be documented one way or the other? (If it's not thread-safe then that's fine, I'll just have to work around it. But it would be great to know.) Many thanks, Chris >From the erl_driver man-page: ErlDrvTermData driver_mk_atom(char* string) This function returns an atom given a name string. The atom is created and won't change, so the return value may be saved and reused, which is faster than looking up the atom several times. From dizzyd@REDACTED Fri Sep 25 20:16:15 2009 From: dizzyd@REDACTED (Dave Smith) Date: Fri, 25 Sep 2009 12:16:15 -0600 Subject: [erlang-questions] Is driver_mk_atom() thread safe? In-Reply-To: <781dd98c0909251032m3512a4cwf0166ef2bbfd7d11@mail.gmail.com> References: <781dd98c0909251032m3512a4cwf0166ef2bbfd7d11@mail.gmail.com> Message-ID: I believe it is NOT thread-safe unless explicitly said to be. From the docs: "Most functions in this API are not thread-safe, i.e., they may not be called from an arbitrary thread. Function that are not documented as thread-safe may only be called from driver call-backs or function calls descending from a driver call-back call. Note that driver call-backs may be called from different threads. This, however, is not a problem for any functions in this API, since the emulator have control over these threads." HTH. D. On Fri, Sep 25, 2009 at 11:32 AM, Chris Newcombe wrote: > Hi BEAM / OTP team, > > I have a driver that uses private thread-pools. ? I know there are > heavy restrictions on calling most erl_driver functions from private > threads, and for some functions the documentation now helpfully says > whether they are thread-safe, or only thread-safe when using the SMP > emulator etc. > > The documentation for driver_mk_atom() doesn't mention thread-safety > (see below). > Please could it be documented one way or the other? ? (If it's not > thread-safe then that's fine, I'll just have to work around it. ?But > it would be great to know.) > > Many thanks, > > Chris > > > From the erl_driver man-page: > > ? ErlDrvTermData driver_mk_atom(char* string) > > ? This function returns an atom given a name string. The atom is > created and won't change, so the return value may be saved and reused, > which is faster than looking up the atom several times. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From kagato@REDACTED Fri Sep 25 20:52:41 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 25 Sep 2009 11:52:41 -0700 Subject: [erlang-questions] running without net tick In-Reply-To: <200909251127.n8PBR6Sw007322@mail.pharos-avantgard.com> References: <200909251127.n8PBR6Sw007322@mail.pharos-avantgard.com> Message-ID: <168420A4-67DA-430B-A65B-5068848F1248@souja.net> I completely agree on the oil-and-water statement. That said, TCP supports OOB, it's just a bad idea to use it. The theory of "why it's not working" that was mentioned was that the other data multiplexed over the stream was choking out the ticks. A dedicated connection makes that a non-issue. It's the classic head- of-line blocking problem. Let's say that ticks were reduced to a 4-byte timestamp (to give some reference point if the connection is broken and re-established). Let's say you send them every 10 seconds. TCP has a 40-byte overhead. Ethernet usually has a 1500-byte MTU. That makes room for an hour's worth of ticks in a single TCP packet over average ethernet (and probably at least 20-minutes worth over any useable MTU). If we are limited to IP, want to have any chance to make it through a firewall, want timely retries, and want a tick to generate one-packet (or less aggregate), we either have UDP or TCP. In the above case, TCP generates about as much packet traffic as UDP and is reasonably close to timely. The packets are larger and the retries are wasted, but they also back off exponentially. A dedicated connection does not have the "head-of-line" problem due to multiplexing (which is, admittedly, unproven as the problem). The point of this exercise is that, unless you're going over a very small or very latent pipe, UDP doesn't really give us anything other than more work. Why more work, you say? Because UDP doesn't retry. Sending a tick every 10 seconds over TCP is not the same thing as sending a tick every 10 seconds over UDP. Why? Assume 75%, random packet loss. That means you're likely to get a single UDP tick every 40 seconds. With TCP, the automatic retry will turn that into approximately one tick every 10.X seconds, where X is entirely dependent on latency (and probably very small). Does TCP do this with more traffic? Yes. However, it does it with exponential backoff, a window to limit the outstanding number of packets, PMTU support so that the packets don't get fragmented, an RST mechanism to break connections if the remote host has rebooted, the option to use SSL to encrypt the session, etc. There are almost no cases that actually demand UDP that a single TCP connection doesn't do very well. I'd strongly recommend not ignoring its benefits and realizing that real network conditions almost never favor UDP and that UDP does not favor a simple implementation. On Sep 25, 2009, at 4:26 AM, Valentin Micic wrote: > I beg to differ -- my take is that TCP reliability is a part of the > problem > in this case. Whilst buffering and flow control is important for, > say, file > transfer, it is completely irrelevant for TCIK and health-checks (So > what if > it doesn't get there, I can send it again without any consequence!). > > Argument about UDP unreliability sounds more like a mantra than a > proper > argument (if only I got a penny every time I've heard it (-:). There > are > only two fundamental differences (*) between TCP and UDP... actually > only > one, because the second is conditioned by the first: TCP supports > stream, > whilst UDP message-bound communication; thus, as a consequence, TCP > requires > some form of flow control to support stream processing. > > In this particular case: what possible benefit one can derive from > sending a > message over the stream as opposed to sending just a message? If the > message > is short enough to fit in a datagram -- none! > > As for ability to send urgent data (OOB) over TCP socket -- data > streams and > OOB data are mixing like oil and water. I am yet to see a successful > utilization of OOB (issued by a user) that hasn't resulted in > connection > reset (or system shutdown (-;). > > Lastly, if TICK is implemented via separate TCP socket, that would > double > networking resources required -- you'd need a new socket for every > node > you're connected to. With UDP, all you need is one socket, and a > very basic > protocol: > > 1) Ask when you have to; > 2) Answer when asked. > > Mind you, net-kernel is already doing this. > > V/ > > (*) If one disregards things that UDP can which TCP cannot do, such > as a > multi-drop, multicasting, etc. > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang- > questions@REDACTED] On > Behalf Of Jayson Vantuyl > Sent: 25 September 2009 12:25 PM > To: Erlang-Questions Questions > Subject: Re: [erlang-questions] running without net tick > > Short Version: > > Why not open a special "tick" TCP port? UDP would require a reliable > delivery implementation. TCP saves quite a bit of work in that regard > (and gets a lot of important but subtle things right). > > Long Version: > > Also, never say never. > > Actually, you CAN send out-of-band data (also called urgent data) > using TCP. The original "WinNuke" (i.e. ping-of-death for Windows 95) > was due to having a corrupt OOB header in a TCP packet. In classic > Microsoft / Internet style, the issue was further confused because it > was an Out-of-Bounds bug, so a generation of networking consultants > have minor deviations in their interpretations of the meaning of the > letters OOB. > > As for TCP Urgent Data / OOB, it seems to be specified well enough at > the protocol level, but iit doesn't appear to be handled uniformly in > different socket implementations. > > Under Linux, you use send/recv with the MSG_OOB option (or set the > SO_OOBINLINE socket option to just inline the data). It appears to > try to keep it at a certain point in the data stream (i.e. to preserve > some of the ordering) and certain conditions can cause it to become > part of the "normal" stream of data. It also can cause some odd > signals to be delivered to the process. Still, TCP *does* have OOB > data support, just maybe it isn't easily usable everywhere. > > On Sep 25, 2009, at 3:04 AM, Valentin Micic wrote: > >> You may change TICK value all day long, but if the underlying >> infrastructure >> s in some kind of trouble, that alone is not going to solve the >> problem. >> >> The following is just a speculation, but quite plausible in my mind: >> >> AFAIK, ERTS is multiplexing inter-nodal traffic over a single >> socket. Thus, >> if the socket is heavily utilized, the sending buffer may get >> congested due >> to dynamically reduced TCP window size (because remote side is not >> flushing >> its buffer fast enough -- if the same process is reading and writing >> the >> socket, this may cause a deadlock under a heavy load). As much as I >> am not >> certain about particular implementation here, I know that sender >> will not >> wait for ever -- it will eventually timeout and this (exception?) >> has to be >> handled somehow by the sender. The reasonable course of action would >> be to >> reset the connection. If and when that happens, node can be declared >> unreachable; therefore the "net-split" may occur. In other words, >> net-split >> may occur with or without "ticker" process running and regardless of >> the >> real network availability (*). >> >> >> I think the net-tick method is good on its own, however, it is >> utilizing a >> *wrong* transport! IMO, tick should be handled as out-of-band data, >> and this >> cannot be done using TCP/IP (well, at least not at the user level). >> My >> suggestion would be to use UDP for net-kernel communication >> (including TICK >> messages). This way one would be able to find out about peer health >> more >> reliably (yes, a small protocol may be required, but that's >> relatively >> easy). >> >> To make things simpler regarding the distribution, one may use the >> same port >> number as advertised in EPMD for a particular node, hence bind UDP >> socket to >> that number. >> >> V/ >> >> (*) I've seen "net-splits" between nodes collocated on the same >> machine -- >> therefore indicating TCP buffer/load related issue. Maybe situation >> may be >> improved by creation of more than one connection between two nodes, >> but that >> may come with a bag of problems on its own. >> >> >> -----Original Message----- >> From: erlang-questions@REDACTED [mailto:erlang- >> questions@REDACTED] On >> Behalf Of Ulf Wiger >> Sent: 25 September 2009 09:13 AM >> To: erlang-questions Questions >> Subject: [erlang-questions] running without net tick >> >> >> The problem of netsplits in Erlang comes up now and again. >> I've mentioned that we used to have a more robust >> supervision algorithm for device processor monitoring in >> AXD 301... >> >> I read the following comment in kernel/src/dist_util.erl >> >> %% Send a TICK to the other side. >> %% >> %% This will happen every 15 seconds (by default) >> %% The idea here is that every 15 secs, we write a little >> %% something on the connection if we haven't written anything for >> %% the last 15 secs. >> %% This will ensure that nodes that are not responding due to >> %% hardware errors (Or being suspended by means of ^Z) will >> %% be considered to be down. If we do not want to have this >> %% we must start the net_kernel (in erlang) without its >> %% ticker process, In that case this code will never run >> >> >> ...and thought: promising - it is then possible to experiment >> with other tick algorithms? >> >> However, looking at net_kernel.erl: >> >> init({Name, LongOrShortNames, TickT}) -> >> process_flag(trap_exit,true), >> case init_node(Name, LongOrShortNames) of >> {ok, Node, Listeners} -> >> process_flag(priority, max), >> Ticktime = to_integer(TickT), >> Ticker = spawn_link(net_kernel, ticker, [self(), >> Ticktime]), >> >> In other words, you can't set net_ticktime to anything other >> than an integer (and it has to be a smallint, since it's used >> in a receive ... after expression. >> >> (To do justice to the comment above, couldn't a net_ticktime >> of, say, 0 turn off net ticking altogether?) >> >> What one can do then, is to set net_ticktime to a very large >> number, and then run a user-level heartbeat. If netsplits are >> still experienced without visible problems in the user-level >> monitoring, or perhaps even serviced traffic during this >> interval, then something is definitely wrong with the tick >> algorithm. :) >> >> BR, >> Ulf W >> -- >> Ulf Wiger >> CTO, Erlang Training & Consulting Ltd >> http://www.erlang-consulting.com >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > > > -- > Jayson Vantuyl > kagato@REDACTED > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From jeraymond@REDACTED Fri Sep 25 22:11:14 2009 From: jeraymond@REDACTED (Jeremy Raymond) Date: Fri, 25 Sep 2009 16:11:14 -0400 Subject: How to test for missing tail recursive calls in service loops? Message-ID: <59da11980909251311l4fb46dd8t8313a78b485f4281@mail.gmail.com> Hello, If a service loop is missing a tail recursive call in a receive clause then the first time a message is sent to the process matching the bad clause the call the receive succeeds. However subsequent calls will fail as the process is no longer waiting on a receive due to the missing tail recursive call. Is there a good way to test for this error besides just make multiple calls to the service matching the same clause? Is there some way to interrogate the process to see if it's currently waiting on a receive? Thanks, Jeremy From v@REDACTED Fri Sep 25 22:25:45 2009 From: v@REDACTED (Valentin Micic) Date: Fri, 25 Sep 2009 22:25:45 +0200 Subject: [erlang-questions] running without net tick In-Reply-To: <168420A4-67DA-430B-A65B-5068848F1248@souja.net> Message-ID: <200909252026.n8PKQLSw009706@mail.pharos-avantgard.com> Let see... say, you send a request over TCP for which you're expecting a reply -- and nothing happens. The fact that TCP has a retransmission mechanism (to ensure a reliable delivery) does not help you make any meaningful decision at this point, does it? So, what you're going to do when this happens? Give up? Retry? Whichever way you slice it, you cannot get away form implementing some kind of application level protocol to handle such a condition. Considering this, UDP makes us work more... how exactly??? (Actually, sometimes TCP may make it worse. If your request times out and you do not tear down the connection over which request has been made, your request will be delivered although there will be no one interested in a reply. Worse yet, you may be issuing another request, etc. I say, sometimes you're better off just reliably losing the whole thing. <-:) Anyway, let's agree to disagree on this one. V/ -----Original Message----- From: Jayson Vantuyl [mailto:kagato@REDACTED] Sent: 25 September 2009 08:53 PM To: Valentin Micic Cc: 'Erlang-Questions Questions' Subject: Re: [erlang-questions] running without net tick I completely agree on the oil-and-water statement. That said, TCP supports OOB, it's just a bad idea to use it. The theory of "why it's not working" that was mentioned was that the other data multiplexed over the stream was choking out the ticks. A dedicated connection makes that a non-issue. It's the classic head- of-line blocking problem. Let's say that ticks were reduced to a 4-byte timestamp (to give some reference point if the connection is broken and re-established). Let's say you send them every 10 seconds. TCP has a 40-byte overhead. Ethernet usually has a 1500-byte MTU. That makes room for an hour's worth of ticks in a single TCP packet over average ethernet (and probably at least 20-minutes worth over any useable MTU). If we are limited to IP, want to have any chance to make it through a firewall, want timely retries, and want a tick to generate one-packet (or less aggregate), we either have UDP or TCP. In the above case, TCP generates about as much packet traffic as UDP and is reasonably close to timely. The packets are larger and the retries are wasted, but they also back off exponentially. A dedicated connection does not have the "head-of-line" problem due to multiplexing (which is, admittedly, unproven as the problem). The point of this exercise is that, unless you're going over a very small or very latent pipe, UDP doesn't really give us anything other than more work. Why more work, you say? Because UDP doesn't retry. Sending a tick every 10 seconds over TCP is not the same thing as sending a tick every 10 seconds over UDP. Why? Assume 75%, random packet loss. That means you're likely to get a single UDP tick every 40 seconds. With TCP, the automatic retry will turn that into approximately one tick every 10.X seconds, where X is entirely dependent on latency (and probably very small). Does TCP do this with more traffic? Yes. However, it does it with exponential backoff, a window to limit the outstanding number of packets, PMTU support so that the packets don't get fragmented, an RST mechanism to break connections if the remote host has rebooted, the option to use SSL to encrypt the session, etc. There are almost no cases that actually demand UDP that a single TCP connection doesn't do very well. I'd strongly recommend not ignoring its benefits and realizing that real network conditions almost never favor UDP and that UDP does not favor a simple implementation. On Sep 25, 2009, at 4:26 AM, Valentin Micic wrote: > I beg to differ -- my take is that TCP reliability is a part of the > problem > in this case. Whilst buffering and flow control is important for, > say, file > transfer, it is completely irrelevant for TCIK and health-checks (So > what if > it doesn't get there, I can send it again without any consequence!). > > Argument about UDP unreliability sounds more like a mantra than a > proper > argument (if only I got a penny every time I've heard it (-:). There > are > only two fundamental differences (*) between TCP and UDP... actually > only > one, because the second is conditioned by the first: TCP supports > stream, > whilst UDP message-bound communication; thus, as a consequence, TCP > requires > some form of flow control to support stream processing. > > In this particular case: what possible benefit one can derive from > sending a > message over the stream as opposed to sending just a message? If the > message > is short enough to fit in a datagram -- none! > > As for ability to send urgent data (OOB) over TCP socket -- data > streams and > OOB data are mixing like oil and water. I am yet to see a successful > utilization of OOB (issued by a user) that hasn't resulted in > connection > reset (or system shutdown (-;). > > Lastly, if TICK is implemented via separate TCP socket, that would > double > networking resources required -- you'd need a new socket for every > node > you're connected to. With UDP, all you need is one socket, and a > very basic > protocol: > > 1) Ask when you have to; > 2) Answer when asked. > > Mind you, net-kernel is already doing this. > > V/ > > (*) If one disregards things that UDP can which TCP cannot do, such > as a > multi-drop, multicasting, etc. > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang- > questions@REDACTED] On > Behalf Of Jayson Vantuyl > Sent: 25 September 2009 12:25 PM > To: Erlang-Questions Questions > Subject: Re: [erlang-questions] running without net tick > > Short Version: > > Why not open a special "tick" TCP port? UDP would require a reliable > delivery implementation. TCP saves quite a bit of work in that regard > (and gets a lot of important but subtle things right). > > Long Version: > > Also, never say never. > > Actually, you CAN send out-of-band data (also called urgent data) > using TCP. The original "WinNuke" (i.e. ping-of-death for Windows 95) > was due to having a corrupt OOB header in a TCP packet. In classic > Microsoft / Internet style, the issue was further confused because it > was an Out-of-Bounds bug, so a generation of networking consultants > have minor deviations in their interpretations of the meaning of the > letters OOB. > > As for TCP Urgent Data / OOB, it seems to be specified well enough at > the protocol level, but iit doesn't appear to be handled uniformly in > different socket implementations. > > Under Linux, you use send/recv with the MSG_OOB option (or set the > SO_OOBINLINE socket option to just inline the data). It appears to > try to keep it at a certain point in the data stream (i.e. to preserve > some of the ordering) and certain conditions can cause it to become > part of the "normal" stream of data. It also can cause some odd > signals to be delivered to the process. Still, TCP *does* have OOB > data support, just maybe it isn't easily usable everywhere. > > On Sep 25, 2009, at 3:04 AM, Valentin Micic wrote: > >> You may change TICK value all day long, but if the underlying >> infrastructure >> s in some kind of trouble, that alone is not going to solve the >> problem. >> >> The following is just a speculation, but quite plausible in my mind: >> >> AFAIK, ERTS is multiplexing inter-nodal traffic over a single >> socket. Thus, >> if the socket is heavily utilized, the sending buffer may get >> congested due >> to dynamically reduced TCP window size (because remote side is not >> flushing >> its buffer fast enough -- if the same process is reading and writing >> the >> socket, this may cause a deadlock under a heavy load). As much as I >> am not >> certain about particular implementation here, I know that sender >> will not >> wait for ever -- it will eventually timeout and this (exception?) >> has to be >> handled somehow by the sender. The reasonable course of action would >> be to >> reset the connection. If and when that happens, node can be declared >> unreachable; therefore the "net-split" may occur. In other words, >> net-split >> may occur with or without "ticker" process running and regardless of >> the >> real network availability (*). >> >> >> I think the net-tick method is good on its own, however, it is >> utilizing a >> *wrong* transport! IMO, tick should be handled as out-of-band data, >> and this >> cannot be done using TCP/IP (well, at least not at the user level). >> My >> suggestion would be to use UDP for net-kernel communication >> (including TICK >> messages). This way one would be able to find out about peer health >> more >> reliably (yes, a small protocol may be required, but that's >> relatively >> easy). >> >> To make things simpler regarding the distribution, one may use the >> same port >> number as advertised in EPMD for a particular node, hence bind UDP >> socket to >> that number. >> >> V/ >> >> (*) I've seen "net-splits" between nodes collocated on the same >> machine -- >> therefore indicating TCP buffer/load related issue. Maybe situation >> may be >> improved by creation of more than one connection between two nodes, >> but that >> may come with a bag of problems on its own. >> >> >> -----Original Message----- >> From: erlang-questions@REDACTED [mailto:erlang- >> questions@REDACTED] On >> Behalf Of Ulf Wiger >> Sent: 25 September 2009 09:13 AM >> To: erlang-questions Questions >> Subject: [erlang-questions] running without net tick >> >> >> The problem of netsplits in Erlang comes up now and again. >> I've mentioned that we used to have a more robust >> supervision algorithm for device processor monitoring in >> AXD 301... >> >> I read the following comment in kernel/src/dist_util.erl >> >> %% Send a TICK to the other side. >> %% >> %% This will happen every 15 seconds (by default) >> %% The idea here is that every 15 secs, we write a little >> %% something on the connection if we haven't written anything for >> %% the last 15 secs. >> %% This will ensure that nodes that are not responding due to >> %% hardware errors (Or being suspended by means of ^Z) will >> %% be considered to be down. If we do not want to have this >> %% we must start the net_kernel (in erlang) without its >> %% ticker process, In that case this code will never run >> >> >> ...and thought: promising - it is then possible to experiment >> with other tick algorithms? >> >> However, looking at net_kernel.erl: >> >> init({Name, LongOrShortNames, TickT}) -> >> process_flag(trap_exit,true), >> case init_node(Name, LongOrShortNames) of >> {ok, Node, Listeners} -> >> process_flag(priority, max), >> Ticktime = to_integer(TickT), >> Ticker = spawn_link(net_kernel, ticker, [self(), >> Ticktime]), >> >> In other words, you can't set net_ticktime to anything other >> than an integer (and it has to be a smallint, since it's used >> in a receive ... after expression. >> >> (To do justice to the comment above, couldn't a net_ticktime >> of, say, 0 turn off net ticking altogether?) >> >> What one can do then, is to set net_ticktime to a very large >> number, and then run a user-level heartbeat. If netsplits are >> still experienced without visible problems in the user-level >> monitoring, or perhaps even serviced traffic during this >> interval, then something is definitely wrong with the tick >> algorithm. :) >> >> BR, >> Ulf W >> -- >> Ulf Wiger >> CTO, Erlang Training & Consulting Ltd >> http://www.erlang-consulting.com >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > > > -- > Jayson Vantuyl > kagato@REDACTED > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From magnus@REDACTED Fri Sep 25 22:26:25 2009 From: magnus@REDACTED (Magnus Henoch) Date: Fri, 25 Sep 2009 21:26:25 +0100 Subject: How to test for missing tail recursive calls in service loops? In-Reply-To: <59da11980909251311l4fb46dd8t8313a78b485f4281@mail.gmail.com> (Jeremy Raymond's message of "Fri, 25 Sep 2009 16:11:14 -0400") References: <59da11980909251311l4fb46dd8t8313a78b485f4281@mail.gmail.com> Message-ID: <84y6o24vpq.fsf@linux-b2a3.site> Jeremy Raymond writes: > If a service loop is missing a tail recursive call in a receive clause then > the first time a message is sent to the process matching the bad clause the > call the receive succeeds. However subsequent calls will fail as the process > is no longer waiting on a receive due to the missing tail recursive call. Is > there a good way to test for this error besides just make multiple calls to > the service matching the same clause? Is there some way to interrogate the > process to see if it's currently waiting on a receive? typer comes to mind. This is my foo.erl. Note that foo/0 does a proper tail call, but I forgot to make bar/0 do the same. -module(foo). -compile(export_all). foo() -> receive foo -> erlang:display(foo), foo(); stop -> {ok, done} end. bar() -> receive bar -> erlang:display(bar); stop -> {ok, done} end. Let's run typer on it: $ typer foo.erl %% File: "foo.erl" %% --------------- -spec foo() -> {'ok','done'}. -spec bar() -> 'true' | {'ok','done'}. Here we see that foo/0 always returns {ok, done}, but bar/0 sometimes returns true - that's our warning signal. -- Magnus Henoch, magnus@REDACTED Erlang Training and Consulting http://www.erlang-consulting.com/ From kagato@REDACTED Fri Sep 25 22:31:19 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 25 Sep 2009 13:31:19 -0700 Subject: [erlang-questions] How to test for missing tail recursive calls in service loops? In-Reply-To: <59da11980909251311l4fb46dd8t8313a78b485f4281@mail.gmail.com> References: <59da11980909251311l4fb46dd8t8313a78b485f4281@mail.gmail.com> Message-ID: <2DAACBD4-92B2-477A-9D92-ECA5F039E009@souja.net> You can implement the "work" part of the loop in a function, then wrap the whole thing in a process that always tail-recurses (and use exit() to exit the process). That way you just write your function like it's going to loop, and it does. Or, you can just use gen_server, which does something effectively like this (it calls you and loops for you). In general, though, there's not really a way to detect this, as there are plenty of normal "termination conditions" where a process should legitimately exit instead of recursing. Unfortunately, it's a compiler, not a mind-reader. :( On Sep 25, 2009, at 1:11 PM, Jeremy Raymond wrote: > Hello, > > If a service loop is missing a tail recursive call in a receive > clause then > the first time a message is sent to the process matching the bad > clause the > call the receive succeeds. However subsequent calls will fail as the > process > is no longer waiting on a receive due to the missing tail recursive > call. Is > there a good way to test for this error besides just make multiple > calls to > the service matching the same clause? Is there some way to > interrogate the > process to see if it's currently waiting on a receive? > > > Thanks, > > Jeremy -- Jayson Vantuyl kagato@REDACTED From kagato@REDACTED Fri Sep 25 22:52:00 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 25 Sep 2009 13:52:00 -0700 Subject: [erlang-questions] running without net tick In-Reply-To: <200909252026.n8PKQLSw009706@mail.pharos-avantgard.com> References: <200909252026.n8PKQLSw009706@mail.pharos-avantgard.com> Message-ID: Packet loss is not as simple as you think. With UDP, you either need to roll a whole backoff implementation or accept that you're going to have horrible behavior under even modest amounts of packet loss. Fragmentation of packets is also particularly mean. It can amplify latency spectacularly when enough packets are involved. TCP avoids this. With UDP, you have to roll it for yourself (and you'll need an ICMP socket to do so, which usually means running as root on Unix). Ticks might be small enough that this isn't necessary, so you might get out of this one for free. Congestion control is problematic, and TCP implementations have a number of answers for this that can be tweaked at the OS. UDP does not. I'm not suggesting that the old data is useful. Just throw it away and have a way to detect that it's old. I am suggesting that having TCP handle the retries (and letting windowing / retransmission handle the trouble for you) is less developer work and behaves better. TCP having retransmission does not help you make a single decision, no. Instead, it prevents purely random losses (which are VERY common in environments like EC2, when network links get saturated) from causing all problems to be amplified to a multiple of your tick time, in a ratio that gets exponentially compared to the fraction of lost packets. Implement it using UDP. Implement one using TCP. Compare the difference in behavior. Notice how the TCP one is simpler, behaves better, and has similar bandwidth consumption. You're obviously going to have to learn it through experimentation. That's fine, as that's how I learned it. I encourage you to keep an open mind, take measurements, and consider that every line of code may someday give a bug a happy home. Good luck. On Sep 25, 2009, at 1:25 PM, Valentin Micic wrote: > Let see... say, you send a request over TCP for which you're > expecting a > reply -- and nothing happens. The fact that TCP has a retransmission > mechanism (to ensure a reliable delivery) does not help you make any > meaningful decision at this point, does it? So, what you're going to > do when > this happens? Give up? Retry? Whichever way you slice it, you cannot > get > away form implementing some kind of application level protocol to > handle > such a condition. > > Considering this, UDP makes us work more... how exactly??? > > (Actually, sometimes TCP may make it worse. If your request times > out and > you do not tear down the connection over which request has been > made, your > request will be delivered although there will be no one interested > in a > reply. Worse yet, you may be issuing another request, etc. I say, > sometimes > you're better off just reliably losing the whole thing. <-:) > > Anyway, let's agree to disagree on this one. > > V/ > > -----Original Message----- > From: Jayson Vantuyl [mailto:kagato@REDACTED] > Sent: 25 September 2009 08:53 PM > To: Valentin Micic > Cc: 'Erlang-Questions Questions' > Subject: Re: [erlang-questions] running without net tick > > I completely agree on the oil-and-water statement. That said, TCP > supports OOB, it's just a bad idea to use it. > > The theory of "why it's not working" that was mentioned was that the > other data multiplexed over the stream was choking out the ticks. > > A dedicated connection makes that a non-issue. It's the classic head- > of-line blocking problem. > > Let's say that ticks were reduced to a 4-byte timestamp (to give some > reference point if the connection is broken and re-established). > Let's say you send them every 10 seconds. TCP has a 40-byte > overhead. Ethernet usually has a 1500-byte MTU. That makes room for > an hour's worth of ticks in a single TCP packet over average ethernet > (and probably at least 20-minutes worth over any useable MTU). > > If we are limited to IP, want to have any chance to make it through a > firewall, want timely retries, and want a tick to generate one-packet > (or less aggregate), we either have UDP or TCP. In the above case, > TCP generates about as much packet traffic as UDP and is reasonably > close to timely. The packets are larger and the retries are wasted, > but they also back off exponentially. A dedicated connection does not > have the "head-of-line" problem due to multiplexing (which is, > admittedly, unproven as the problem). > > The point of this exercise is that, unless you're going over a very > small or very latent pipe, UDP doesn't really give us anything other > than more work. Why more work, you say? Because UDP doesn't retry. > Sending a tick every 10 seconds over TCP is not the same thing as > sending a tick every 10 seconds over UDP. Why? Assume 75%, random > packet loss. That means you're likely to get a single UDP tick every > 40 seconds. With TCP, the automatic retry will turn that into > approximately one tick every 10.X seconds, where X is entirely > dependent on latency (and probably very small). Does TCP do this with > more traffic? Yes. However, it does it with exponential backoff, a > window to limit the outstanding number of packets, PMTU support so > that the packets don't get fragmented, an RST mechanism to break > connections if the remote host has rebooted, the option to use SSL to > encrypt the session, etc. > > There are almost no cases that actually demand UDP that a single TCP > connection doesn't do very well. I'd strongly recommend not ignoring > its benefits and realizing that real network conditions almost never > favor UDP and that UDP does not favor a simple implementation. > > On Sep 25, 2009, at 4:26 AM, Valentin Micic wrote: > >> I beg to differ -- my take is that TCP reliability is a part of the >> problem >> in this case. Whilst buffering and flow control is important for, >> say, file >> transfer, it is completely irrelevant for TCIK and health-checks (So >> what if >> it doesn't get there, I can send it again without any consequence!). >> >> Argument about UDP unreliability sounds more like a mantra than a >> proper >> argument (if only I got a penny every time I've heard it (-:). There >> are >> only two fundamental differences (*) between TCP and UDP... actually >> only >> one, because the second is conditioned by the first: TCP supports >> stream, >> whilst UDP message-bound communication; thus, as a consequence, TCP >> requires >> some form of flow control to support stream processing. >> >> In this particular case: what possible benefit one can derive from >> sending a >> message over the stream as opposed to sending just a message? If the >> message >> is short enough to fit in a datagram -- none! >> >> As for ability to send urgent data (OOB) over TCP socket -- data >> streams and >> OOB data are mixing like oil and water. I am yet to see a successful >> utilization of OOB (issued by a user) that hasn't resulted in >> connection >> reset (or system shutdown (-;). >> >> Lastly, if TICK is implemented via separate TCP socket, that would >> double >> networking resources required -- you'd need a new socket for every >> node >> you're connected to. With UDP, all you need is one socket, and a >> very basic >> protocol: >> >> 1) Ask when you have to; >> 2) Answer when asked. >> >> Mind you, net-kernel is already doing this. >> >> V/ >> >> (*) If one disregards things that UDP can which TCP cannot do, such >> as a >> multi-drop, multicasting, etc. >> >> -----Original Message----- >> From: erlang-questions@REDACTED [mailto:erlang- >> questions@REDACTED] On >> Behalf Of Jayson Vantuyl >> Sent: 25 September 2009 12:25 PM >> To: Erlang-Questions Questions >> Subject: Re: [erlang-questions] running without net tick >> >> Short Version: >> >> Why not open a special "tick" TCP port? UDP would require a reliable >> delivery implementation. TCP saves quite a bit of work in that >> regard >> (and gets a lot of important but subtle things right). >> >> Long Version: >> >> Also, never say never. >> >> Actually, you CAN send out-of-band data (also called urgent data) >> using TCP. The original "WinNuke" (i.e. ping-of-death for Windows >> 95) >> was due to having a corrupt OOB header in a TCP packet. In classic >> Microsoft / Internet style, the issue was further confused because it >> was an Out-of-Bounds bug, so a generation of networking consultants >> have minor deviations in their interpretations of the meaning of the >> letters OOB. >> >> As for TCP Urgent Data / OOB, it seems to be specified well enough at >> the protocol level, but iit doesn't appear to be handled uniformly in >> different socket implementations. >> >> Under Linux, you use send/recv with the MSG_OOB option (or set the >> SO_OOBINLINE socket option to just inline the data). It appears to >> try to keep it at a certain point in the data stream (i.e. to >> preserve >> some of the ordering) and certain conditions can cause it to become >> part of the "normal" stream of data. It also can cause some odd >> signals to be delivered to the process. Still, TCP *does* have OOB >> data support, just maybe it isn't easily usable everywhere. >> >> On Sep 25, 2009, at 3:04 AM, Valentin Micic wrote: >> >>> You may change TICK value all day long, but if the underlying >>> infrastructure >>> s in some kind of trouble, that alone is not going to solve the >>> problem. >>> >>> The following is just a speculation, but quite plausible in my mind: >>> >>> AFAIK, ERTS is multiplexing inter-nodal traffic over a single >>> socket. Thus, >>> if the socket is heavily utilized, the sending buffer may get >>> congested due >>> to dynamically reduced TCP window size (because remote side is not >>> flushing >>> its buffer fast enough -- if the same process is reading and writing >>> the >>> socket, this may cause a deadlock under a heavy load). As much as I >>> am not >>> certain about particular implementation here, I know that sender >>> will not >>> wait for ever -- it will eventually timeout and this (exception?) >>> has to be >>> handled somehow by the sender. The reasonable course of action would >>> be to >>> reset the connection. If and when that happens, node can be declared >>> unreachable; therefore the "net-split" may occur. In other words, >>> net-split >>> may occur with or without "ticker" process running and regardless of >>> the >>> real network availability (*). >>> >>> >>> I think the net-tick method is good on its own, however, it is >>> utilizing a >>> *wrong* transport! IMO, tick should be handled as out-of-band data, >>> and this >>> cannot be done using TCP/IP (well, at least not at the user level). >>> My >>> suggestion would be to use UDP for net-kernel communication >>> (including TICK >>> messages). This way one would be able to find out about peer health >>> more >>> reliably (yes, a small protocol may be required, but that's >>> relatively >>> easy). >>> >>> To make things simpler regarding the distribution, one may use the >>> same port >>> number as advertised in EPMD for a particular node, hence bind UDP >>> socket to >>> that number. >>> >>> V/ >>> >>> (*) I've seen "net-splits" between nodes collocated on the same >>> machine -- >>> therefore indicating TCP buffer/load related issue. Maybe situation >>> may be >>> improved by creation of more than one connection between two nodes, >>> but that >>> may come with a bag of problems on its own. >>> >>> >>> -----Original Message----- >>> From: erlang-questions@REDACTED [mailto:erlang- >>> questions@REDACTED] On >>> Behalf Of Ulf Wiger >>> Sent: 25 September 2009 09:13 AM >>> To: erlang-questions Questions >>> Subject: [erlang-questions] running without net tick >>> >>> >>> The problem of netsplits in Erlang comes up now and again. >>> I've mentioned that we used to have a more robust >>> supervision algorithm for device processor monitoring in >>> AXD 301... >>> >>> I read the following comment in kernel/src/dist_util.erl >>> >>> %% Send a TICK to the other side. >>> %% >>> %% This will happen every 15 seconds (by default) >>> %% The idea here is that every 15 secs, we write a little >>> %% something on the connection if we haven't written anything for >>> %% the last 15 secs. >>> %% This will ensure that nodes that are not responding due to >>> %% hardware errors (Or being suspended by means of ^Z) will >>> %% be considered to be down. If we do not want to have this >>> %% we must start the net_kernel (in erlang) without its >>> %% ticker process, In that case this code will never run >>> >>> >>> ...and thought: promising - it is then possible to experiment >>> with other tick algorithms? >>> >>> However, looking at net_kernel.erl: >>> >>> init({Name, LongOrShortNames, TickT}) -> >>> process_flag(trap_exit,true), >>> case init_node(Name, LongOrShortNames) of >>> {ok, Node, Listeners} -> >>> process_flag(priority, max), >>> Ticktime = to_integer(TickT), >>> Ticker = spawn_link(net_kernel, ticker, [self(), >>> Ticktime]), >>> >>> In other words, you can't set net_ticktime to anything other >>> than an integer (and it has to be a smallint, since it's used >>> in a receive ... after expression. >>> >>> (To do justice to the comment above, couldn't a net_ticktime >>> of, say, 0 turn off net ticking altogether?) >>> >>> What one can do then, is to set net_ticktime to a very large >>> number, and then run a user-level heartbeat. If netsplits are >>> still experienced without visible problems in the user-level >>> monitoring, or perhaps even serviced traffic during this >>> interval, then something is definitely wrong with the tick >>> algorithm. :) >>> >>> BR, >>> Ulf W >>> -- >>> Ulf Wiger >>> CTO, Erlang Training & Consulting Ltd >>> http://www.erlang-consulting.com >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> >> >> >> -- >> Jayson Vantuyl >> kagato@REDACTED >> >> >> >> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > > > -- > Jayson Vantuyl > kagato@REDACTED > > > > From v@REDACTED Sat Sep 26 00:53:04 2009 From: v@REDACTED (Valentin Micic) Date: Sat, 26 Sep 2009 00:53:04 +0200 Subject: [erlang-questions] running without net tick In-Reply-To: Message-ID: <200909252253.n8PMrfSw010130@mail.pharos-avantgard.com> There is an urban legend saying that NASA invested a good few million dollars in order to develop a ballpoint pen that can work in zero gravity. Russians used pencil. As much as I don't think that Russian pencil should be used for signing a nuclear disarmament treaty, it will work just fine for a logbook keeping. Why would I need to implement a backoff? What horrible behavior you're talking about? For goodness sake, we're just talking about loosing packets that we can afford to lose! Packet loss *is* simple if a problem we're trying to solve is simple. OTOH, if we apply a complex solution to a simple problem, we are elevating the problem to the same level of complexity as given by the solution. By indiscriminately applying experience related to a complex problem in a context of a simple problem, you are implying that simple solutions, hence, problems do not exist. Clearly not the case -- I would choose UDP over TCP for implementation of management protocol any day of the week and twice on Sunday. Not because I am stubborn, but because it worked for me in the past. And still does. V/ -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Jayson Vantuyl Sent: 25 September 2009 10:52 PM To: Erlang-Questions Questions Subject: Re: [erlang-questions] running without net tick Packet loss is not as simple as you think. With UDP, you either need to roll a whole backoff implementation or accept that you're going to have horrible behavior under even modest amounts of packet loss. Fragmentation of packets is also particularly mean. It can amplify latency spectacularly when enough packets are involved. TCP avoids this. With UDP, you have to roll it for yourself (and you'll need an ICMP socket to do so, which usually means running as root on Unix). Ticks might be small enough that this isn't necessary, so you might get out of this one for free. Congestion control is problematic, and TCP implementations have a number of answers for this that can be tweaked at the OS. UDP does not. I'm not suggesting that the old data is useful. Just throw it away and have a way to detect that it's old. I am suggesting that having TCP handle the retries (and letting windowing / retransmission handle the trouble for you) is less developer work and behaves better. TCP having retransmission does not help you make a single decision, no. Instead, it prevents purely random losses (which are VERY common in environments like EC2, when network links get saturated) from causing all problems to be amplified to a multiple of your tick time, in a ratio that gets exponentially compared to the fraction of lost packets. Implement it using UDP. Implement one using TCP. Compare the difference in behavior. Notice how the TCP one is simpler, behaves better, and has similar bandwidth consumption. You're obviously going to have to learn it through experimentation. That's fine, as that's how I learned it. I encourage you to keep an open mind, take measurements, and consider that every line of code may someday give a bug a happy home. Good luck. On Sep 25, 2009, at 1:25 PM, Valentin Micic wrote: > Let see... say, you send a request over TCP for which you're > expecting a > reply -- and nothing happens. The fact that TCP has a retransmission > mechanism (to ensure a reliable delivery) does not help you make any > meaningful decision at this point, does it? So, what you're going to > do when > this happens? Give up? Retry? Whichever way you slice it, you cannot > get > away form implementing some kind of application level protocol to > handle > such a condition. > > Considering this, UDP makes us work more... how exactly??? > > (Actually, sometimes TCP may make it worse. If your request times > out and > you do not tear down the connection over which request has been > made, your > request will be delivered although there will be no one interested > in a > reply. Worse yet, you may be issuing another request, etc. I say, > sometimes > you're better off just reliably losing the whole thing. <-:) > > Anyway, let's agree to disagree on this one. > > V/ > > -----Original Message----- > From: Jayson Vantuyl [mailto:kagato@REDACTED] > Sent: 25 September 2009 08:53 PM > To: Valentin Micic > Cc: 'Erlang-Questions Questions' > Subject: Re: [erlang-questions] running without net tick > > I completely agree on the oil-and-water statement. That said, TCP > supports OOB, it's just a bad idea to use it. > > The theory of "why it's not working" that was mentioned was that the > other data multiplexed over the stream was choking out the ticks. > > A dedicated connection makes that a non-issue. It's the classic head- > of-line blocking problem. > > Let's say that ticks were reduced to a 4-byte timestamp (to give some > reference point if the connection is broken and re-established). > Let's say you send them every 10 seconds. TCP has a 40-byte > overhead. Ethernet usually has a 1500-byte MTU. That makes room for > an hour's worth of ticks in a single TCP packet over average ethernet > (and probably at least 20-minutes worth over any useable MTU). > > If we are limited to IP, want to have any chance to make it through a > firewall, want timely retries, and want a tick to generate one-packet > (or less aggregate), we either have UDP or TCP. In the above case, > TCP generates about as much packet traffic as UDP and is reasonably > close to timely. The packets are larger and the retries are wasted, > but they also back off exponentially. A dedicated connection does not > have the "head-of-line" problem due to multiplexing (which is, > admittedly, unproven as the problem). > > The point of this exercise is that, unless you're going over a very > small or very latent pipe, UDP doesn't really give us anything other > than more work. Why more work, you say? Because UDP doesn't retry. > Sending a tick every 10 seconds over TCP is not the same thing as > sending a tick every 10 seconds over UDP. Why? Assume 75%, random > packet loss. That means you're likely to get a single UDP tick every > 40 seconds. With TCP, the automatic retry will turn that into > approximately one tick every 10.X seconds, where X is entirely > dependent on latency (and probably very small). Does TCP do this with > more traffic? Yes. However, it does it with exponential backoff, a > window to limit the outstanding number of packets, PMTU support so > that the packets don't get fragmented, an RST mechanism to break > connections if the remote host has rebooted, the option to use SSL to > encrypt the session, etc. > > There are almost no cases that actually demand UDP that a single TCP > connection doesn't do very well. I'd strongly recommend not ignoring > its benefits and realizing that real network conditions almost never > favor UDP and that UDP does not favor a simple implementation. > > On Sep 25, 2009, at 4:26 AM, Valentin Micic wrote: > >> I beg to differ -- my take is that TCP reliability is a part of the >> problem >> in this case. Whilst buffering and flow control is important for, >> say, file >> transfer, it is completely irrelevant for TCIK and health-checks (So >> what if >> it doesn't get there, I can send it again without any consequence!). >> >> Argument about UDP unreliability sounds more like a mantra than a >> proper >> argument (if only I got a penny every time I've heard it (-:). There >> are >> only two fundamental differences (*) between TCP and UDP... actually >> only >> one, because the second is conditioned by the first: TCP supports >> stream, >> whilst UDP message-bound communication; thus, as a consequence, TCP >> requires >> some form of flow control to support stream processing. >> >> In this particular case: what possible benefit one can derive from >> sending a >> message over the stream as opposed to sending just a message? If the >> message >> is short enough to fit in a datagram -- none! >> >> As for ability to send urgent data (OOB) over TCP socket -- data >> streams and >> OOB data are mixing like oil and water. I am yet to see a successful >> utilization of OOB (issued by a user) that hasn't resulted in >> connection >> reset (or system shutdown (-;). >> >> Lastly, if TICK is implemented via separate TCP socket, that would >> double >> networking resources required -- you'd need a new socket for every >> node >> you're connected to. With UDP, all you need is one socket, and a >> very basic >> protocol: >> >> 1) Ask when you have to; >> 2) Answer when asked. >> >> Mind you, net-kernel is already doing this. >> >> V/ >> >> (*) If one disregards things that UDP can which TCP cannot do, such >> as a >> multi-drop, multicasting, etc. >> >> -----Original Message----- >> From: erlang-questions@REDACTED [mailto:erlang- >> questions@REDACTED] On >> Behalf Of Jayson Vantuyl >> Sent: 25 September 2009 12:25 PM >> To: Erlang-Questions Questions >> Subject: Re: [erlang-questions] running without net tick >> >> Short Version: >> >> Why not open a special "tick" TCP port? UDP would require a reliable >> delivery implementation. TCP saves quite a bit of work in that >> regard >> (and gets a lot of important but subtle things right). >> >> Long Version: >> >> Also, never say never. >> >> Actually, you CAN send out-of-band data (also called urgent data) >> using TCP. The original "WinNuke" (i.e. ping-of-death for Windows >> 95) >> was due to having a corrupt OOB header in a TCP packet. In classic >> Microsoft / Internet style, the issue was further confused because it >> was an Out-of-Bounds bug, so a generation of networking consultants >> have minor deviations in their interpretations of the meaning of the >> letters OOB. >> >> As for TCP Urgent Data / OOB, it seems to be specified well enough at >> the protocol level, but iit doesn't appear to be handled uniformly in >> different socket implementations. >> >> Under Linux, you use send/recv with the MSG_OOB option (or set the >> SO_OOBINLINE socket option to just inline the data). It appears to >> try to keep it at a certain point in the data stream (i.e. to >> preserve >> some of the ordering) and certain conditions can cause it to become >> part of the "normal" stream of data. It also can cause some odd >> signals to be delivered to the process. Still, TCP *does* have OOB >> data support, just maybe it isn't easily usable everywhere. >> >> On Sep 25, 2009, at 3:04 AM, Valentin Micic wrote: >> >>> You may change TICK value all day long, but if the underlying >>> infrastructure >>> s in some kind of trouble, that alone is not going to solve the >>> problem. >>> >>> The following is just a speculation, but quite plausible in my mind: >>> >>> AFAIK, ERTS is multiplexing inter-nodal traffic over a single >>> socket. Thus, >>> if the socket is heavily utilized, the sending buffer may get >>> congested due >>> to dynamically reduced TCP window size (because remote side is not >>> flushing >>> its buffer fast enough -- if the same process is reading and writing >>> the >>> socket, this may cause a deadlock under a heavy load). As much as I >>> am not >>> certain about particular implementation here, I know that sender >>> will not >>> wait for ever -- it will eventually timeout and this (exception?) >>> has to be >>> handled somehow by the sender. The reasonable course of action would >>> be to >>> reset the connection. If and when that happens, node can be declared >>> unreachable; therefore the "net-split" may occur. In other words, >>> net-split >>> may occur with or without "ticker" process running and regardless of >>> the >>> real network availability (*). >>> >>> >>> I think the net-tick method is good on its own, however, it is >>> utilizing a >>> *wrong* transport! IMO, tick should be handled as out-of-band data, >>> and this >>> cannot be done using TCP/IP (well, at least not at the user level). >>> My >>> suggestion would be to use UDP for net-kernel communication >>> (including TICK >>> messages). This way one would be able to find out about peer health >>> more >>> reliably (yes, a small protocol may be required, but that's >>> relatively >>> easy). >>> >>> To make things simpler regarding the distribution, one may use the >>> same port >>> number as advertised in EPMD for a particular node, hence bind UDP >>> socket to >>> that number. >>> >>> V/ >>> >>> (*) I've seen "net-splits" between nodes collocated on the same >>> machine -- >>> therefore indicating TCP buffer/load related issue. Maybe situation >>> may be >>> improved by creation of more than one connection between two nodes, >>> but that >>> may come with a bag of problems on its own. >>> >>> >>> -----Original Message----- >>> From: erlang-questions@REDACTED [mailto:erlang- >>> questions@REDACTED] On >>> Behalf Of Ulf Wiger >>> Sent: 25 September 2009 09:13 AM >>> To: erlang-questions Questions >>> Subject: [erlang-questions] running without net tick >>> >>> >>> The problem of netsplits in Erlang comes up now and again. >>> I've mentioned that we used to have a more robust >>> supervision algorithm for device processor monitoring in >>> AXD 301... >>> >>> I read the following comment in kernel/src/dist_util.erl >>> >>> %% Send a TICK to the other side. >>> %% >>> %% This will happen every 15 seconds (by default) >>> %% The idea here is that every 15 secs, we write a little >>> %% something on the connection if we haven't written anything for >>> %% the last 15 secs. >>> %% This will ensure that nodes that are not responding due to >>> %% hardware errors (Or being suspended by means of ^Z) will >>> %% be considered to be down. If we do not want to have this >>> %% we must start the net_kernel (in erlang) without its >>> %% ticker process, In that case this code will never run >>> >>> >>> ...and thought: promising - it is then possible to experiment >>> with other tick algorithms? >>> >>> However, looking at net_kernel.erl: >>> >>> init({Name, LongOrShortNames, TickT}) -> >>> process_flag(trap_exit,true), >>> case init_node(Name, LongOrShortNames) of >>> {ok, Node, Listeners} -> >>> process_flag(priority, max), >>> Ticktime = to_integer(TickT), >>> Ticker = spawn_link(net_kernel, ticker, [self(), >>> Ticktime]), >>> >>> In other words, you can't set net_ticktime to anything other >>> than an integer (and it has to be a smallint, since it's used >>> in a receive ... after expression. >>> >>> (To do justice to the comment above, couldn't a net_ticktime >>> of, say, 0 turn off net ticking altogether?) >>> >>> What one can do then, is to set net_ticktime to a very large >>> number, and then run a user-level heartbeat. If netsplits are >>> still experienced without visible problems in the user-level >>> monitoring, or perhaps even serviced traffic during this >>> interval, then something is definitely wrong with the tick >>> algorithm. :) >>> >>> BR, >>> Ulf W >>> -- >>> Ulf Wiger >>> CTO, Erlang Training & Consulting Ltd >>> http://www.erlang-consulting.com >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> >> >> >> -- >> Jayson Vantuyl >> kagato@REDACTED >> >> >> >> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > > > -- > Jayson Vantuyl > kagato@REDACTED > > > > ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From kagato@REDACTED Sat Sep 26 01:23:04 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 25 Sep 2009 16:23:04 -0700 Subject: [erlang-questions] running without net tick In-Reply-To: <200909252253.n8PMrfSw010130@mail.pharos-avantgard.com> References: <200909252253.n8PMrfSw010130@mail.pharos-avantgard.com> Message-ID: <32F30C3C-F7E2-45DE-8D3B-858275FF0EC9@souja.net> What do you think that the ticks do? If you lose one, then a node is considered lost. Without a retry strategy that is drastically less than the tick time (i.e. the one currently handled by TCP), you will see splits. If you just use UDP, then ANY packet loss will cause a split. If you are using UDP, then you have to compensate for this, i.e. engineer a retry scheme. Minimally, you have to at least have a tick interval that might (for example) be shorter than the split interval. As we are talking about quite a few nodes, you will have retry synchronization issues. This problem is as old as the Internet. UDP saves bytes and avoids head-of-line blocking, that's about it. With a dedicated TCP connection, there is no head-of-line blocking. And, unlike a retry model with UDP, you don't have to maintain TCP across a few different OSes, and I strongly suggest that it solves this problem in a way that is less wasteful than you seem to think. As for the ball-point pen example, it's a really good example of what's going on. When all they had were pencils, NASA and the Russians both just used pencils. They developed a pen because the pencils were a safety hazard, and then the Americans and Russians used them, because they solved a real problem. Just like the people behind that particular (false, http://www.snopes.com/business/genius/spacepen.asp) urban legend, you don't seem to understand the problem-space. Unless I'm missing something about the net_kernel implementation, the only upside of UDP is saving a few bytes, at the expense of complexity that must be maintained at the core of a fault-tolerant system. On Sep 25, 2009, at 3:53 PM, Valentin Micic wrote: > There is an urban legend saying that NASA invested a good few million > dollars in order to develop a ballpoint pen that can work in zero > gravity. > Russians used pencil. As much as I don't think that Russian pencil > should be > used for signing a nuclear disarmament treaty, it will work just > fine for a > logbook keeping. > > Why would I need to implement a backoff? What horrible behavior you're > talking about? For goodness sake, we're just talking about loosing > packets > that we can afford to lose! Packet loss *is* simple if a problem we're > trying to solve is simple. OTOH, if we apply a complex solution to a > simple > problem, we are elevating the problem to the same level of > complexity as > given by the solution. By indiscriminately applying experience > related to a > complex problem in a context of a simple problem, you are implying > that > simple solutions, hence, problems do not exist. > > Clearly not the case -- I would choose UDP over TCP for > implementation of > management protocol any day of the week and twice on Sunday. Not > because I > am stubborn, but because it worked for me in the past. And still does. > > > V/ > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang- > questions@REDACTED] On > Behalf Of Jayson Vantuyl > Sent: 25 September 2009 10:52 PM > To: Erlang-Questions Questions > Subject: Re: [erlang-questions] running without net tick > > Packet loss is not as simple as you think. With UDP, you either need > to roll a whole backoff implementation or accept that you're going to > have horrible behavior under even modest amounts of packet loss. > > Fragmentation of packets is also particularly mean. It can amplify > latency spectacularly when enough packets are involved. TCP avoids > this. With UDP, you have to roll it for yourself (and you'll need an > ICMP socket to do so, which usually means running as root on Unix). > Ticks might be small enough that this isn't necessary, so you might > get out of this one for free. > > Congestion control is problematic, and TCP implementations have a > number of answers for this that can be tweaked at the OS. UDP does > not. > > I'm not suggesting that the old data is useful. Just throw it away > and have a way to detect that it's old. I am suggesting that having > TCP handle the retries (and letting windowing / retransmission handle > the trouble for you) is less developer work and behaves better. > > TCP having retransmission does not help you make a single decision, > no. Instead, it prevents purely random losses (which are VERY common > in environments like EC2, when network links get saturated) from > causing all problems to be amplified to a multiple of your tick time, > in a ratio that gets exponentially compared to the fraction of lost > packets. > > Implement it using UDP. Implement one using TCP. Compare the > difference in behavior. Notice how the TCP one is simpler, behaves > better, and has similar bandwidth consumption. You're obviously going > to have to learn it through experimentation. That's fine, as that's > how I learned it. I encourage you to keep an open mind, take > measurements, and consider that every line of code may someday give a > bug a happy home. > > Good luck. > > On Sep 25, 2009, at 1:25 PM, Valentin Micic wrote: > >> Let see... say, you send a request over TCP for which you're >> expecting a >> reply -- and nothing happens. The fact that TCP has a retransmission >> mechanism (to ensure a reliable delivery) does not help you make any >> meaningful decision at this point, does it? So, what you're going to >> do when >> this happens? Give up? Retry? Whichever way you slice it, you cannot >> get >> away form implementing some kind of application level protocol to >> handle >> such a condition. >> >> Considering this, UDP makes us work more... how exactly??? >> >> (Actually, sometimes TCP may make it worse. If your request times >> out and >> you do not tear down the connection over which request has been >> made, your >> request will be delivered although there will be no one interested >> in a >> reply. Worse yet, you may be issuing another request, etc. I say, >> sometimes >> you're better off just reliably losing the whole thing. <-:) >> >> Anyway, let's agree to disagree on this one. >> >> V/ >> >> -----Original Message----- >> From: Jayson Vantuyl [mailto:kagato@REDACTED] >> Sent: 25 September 2009 08:53 PM >> To: Valentin Micic >> Cc: 'Erlang-Questions Questions' >> Subject: Re: [erlang-questions] running without net tick >> >> I completely agree on the oil-and-water statement. That said, TCP >> supports OOB, it's just a bad idea to use it. >> >> The theory of "why it's not working" that was mentioned was that the >> other data multiplexed over the stream was choking out the ticks. >> >> A dedicated connection makes that a non-issue. It's the classic >> head- >> of-line blocking problem. >> >> Let's say that ticks were reduced to a 4-byte timestamp (to give some >> reference point if the connection is broken and re-established). >> Let's say you send them every 10 seconds. TCP has a 40-byte >> overhead. Ethernet usually has a 1500-byte MTU. That makes room for >> an hour's worth of ticks in a single TCP packet over average ethernet >> (and probably at least 20-minutes worth over any useable MTU). >> >> If we are limited to IP, want to have any chance to make it through a >> firewall, want timely retries, and want a tick to generate one-packet >> (or less aggregate), we either have UDP or TCP. In the above case, >> TCP generates about as much packet traffic as UDP and is reasonably >> close to timely. The packets are larger and the retries are wasted, >> but they also back off exponentially. A dedicated connection does >> not >> have the "head-of-line" problem due to multiplexing (which is, >> admittedly, unproven as the problem). >> >> The point of this exercise is that, unless you're going over a very >> small or very latent pipe, UDP doesn't really give us anything other >> than more work. Why more work, you say? Because UDP doesn't retry. >> Sending a tick every 10 seconds over TCP is not the same thing as >> sending a tick every 10 seconds over UDP. Why? Assume 75%, random >> packet loss. That means you're likely to get a single UDP tick every >> 40 seconds. With TCP, the automatic retry will turn that into >> approximately one tick every 10.X seconds, where X is entirely >> dependent on latency (and probably very small). Does TCP do this >> with >> more traffic? Yes. However, it does it with exponential backoff, a >> window to limit the outstanding number of packets, PMTU support so >> that the packets don't get fragmented, an RST mechanism to break >> connections if the remote host has rebooted, the option to use SSL to >> encrypt the session, etc. >> >> There are almost no cases that actually demand UDP that a single TCP >> connection doesn't do very well. I'd strongly recommend not ignoring >> its benefits and realizing that real network conditions almost never >> favor UDP and that UDP does not favor a simple implementation. >> >> On Sep 25, 2009, at 4:26 AM, Valentin Micic wrote: >> >>> I beg to differ -- my take is that TCP reliability is a part of the >>> problem >>> in this case. Whilst buffering and flow control is important for, >>> say, file >>> transfer, it is completely irrelevant for TCIK and health-checks (So >>> what if >>> it doesn't get there, I can send it again without any consequence!). >>> >>> Argument about UDP unreliability sounds more like a mantra than a >>> proper >>> argument (if only I got a penny every time I've heard it (-:). There >>> are >>> only two fundamental differences (*) between TCP and UDP... actually >>> only >>> one, because the second is conditioned by the first: TCP supports >>> stream, >>> whilst UDP message-bound communication; thus, as a consequence, TCP >>> requires >>> some form of flow control to support stream processing. >>> >>> In this particular case: what possible benefit one can derive from >>> sending a >>> message over the stream as opposed to sending just a message? If the >>> message >>> is short enough to fit in a datagram -- none! >>> >>> As for ability to send urgent data (OOB) over TCP socket -- data >>> streams and >>> OOB data are mixing like oil and water. I am yet to see a successful >>> utilization of OOB (issued by a user) that hasn't resulted in >>> connection >>> reset (or system shutdown (-;). >>> >>> Lastly, if TICK is implemented via separate TCP socket, that would >>> double >>> networking resources required -- you'd need a new socket for every >>> node >>> you're connected to. With UDP, all you need is one socket, and a >>> very basic >>> protocol: >>> >>> 1) Ask when you have to; >>> 2) Answer when asked. >>> >>> Mind you, net-kernel is already doing this. >>> >>> V/ >>> >>> (*) If one disregards things that UDP can which TCP cannot do, such >>> as a >>> multi-drop, multicasting, etc. >>> >>> -----Original Message----- >>> From: erlang-questions@REDACTED [mailto:erlang- >>> questions@REDACTED] On >>> Behalf Of Jayson Vantuyl >>> Sent: 25 September 2009 12:25 PM >>> To: Erlang-Questions Questions >>> Subject: Re: [erlang-questions] running without net tick >>> >>> Short Version: >>> >>> Why not open a special "tick" TCP port? UDP would require a >>> reliable >>> delivery implementation. TCP saves quite a bit of work in that >>> regard >>> (and gets a lot of important but subtle things right). >>> >>> Long Version: >>> >>> Also, never say never. >>> >>> Actually, you CAN send out-of-band data (also called urgent data) >>> using TCP. The original "WinNuke" (i.e. ping-of-death for Windows >>> 95) >>> was due to having a corrupt OOB header in a TCP packet. In classic >>> Microsoft / Internet style, the issue was further confused because >>> it >>> was an Out-of-Bounds bug, so a generation of networking consultants >>> have minor deviations in their interpretations of the meaning of the >>> letters OOB. >>> >>> As for TCP Urgent Data / OOB, it seems to be specified well enough >>> at >>> the protocol level, but iit doesn't appear to be handled uniformly >>> in >>> different socket implementations. >>> >>> Under Linux, you use send/recv with the MSG_OOB option (or set the >>> SO_OOBINLINE socket option to just inline the data). It appears to >>> try to keep it at a certain point in the data stream (i.e. to >>> preserve >>> some of the ordering) and certain conditions can cause it to become >>> part of the "normal" stream of data. It also can cause some odd >>> signals to be delivered to the process. Still, TCP *does* have OOB >>> data support, just maybe it isn't easily usable everywhere. >>> >>> On Sep 25, 2009, at 3:04 AM, Valentin Micic wrote: >>> >>>> You may change TICK value all day long, but if the underlying >>>> infrastructure >>>> s in some kind of trouble, that alone is not going to solve the >>>> problem. >>>> >>>> The following is just a speculation, but quite plausible in my >>>> mind: >>>> >>>> AFAIK, ERTS is multiplexing inter-nodal traffic over a single >>>> socket. Thus, >>>> if the socket is heavily utilized, the sending buffer may get >>>> congested due >>>> to dynamically reduced TCP window size (because remote side is not >>>> flushing >>>> its buffer fast enough -- if the same process is reading and >>>> writing >>>> the >>>> socket, this may cause a deadlock under a heavy load). As much as I >>>> am not >>>> certain about particular implementation here, I know that sender >>>> will not >>>> wait for ever -- it will eventually timeout and this (exception?) >>>> has to be >>>> handled somehow by the sender. The reasonable course of action >>>> would >>>> be to >>>> reset the connection. If and when that happens, node can be >>>> declared >>>> unreachable; therefore the "net-split" may occur. In other words, >>>> net-split >>>> may occur with or without "ticker" process running and regardless >>>> of >>>> the >>>> real network availability (*). >>>> >>>> >>>> I think the net-tick method is good on its own, however, it is >>>> utilizing a >>>> *wrong* transport! IMO, tick should be handled as out-of-band data, >>>> and this >>>> cannot be done using TCP/IP (well, at least not at the user level). >>>> My >>>> suggestion would be to use UDP for net-kernel communication >>>> (including TICK >>>> messages). This way one would be able to find out about peer health >>>> more >>>> reliably (yes, a small protocol may be required, but that's >>>> relatively >>>> easy). >>>> >>>> To make things simpler regarding the distribution, one may use the >>>> same port >>>> number as advertised in EPMD for a particular node, hence bind UDP >>>> socket to >>>> that number. >>>> >>>> V/ >>>> >>>> (*) I've seen "net-splits" between nodes collocated on the same >>>> machine -- >>>> therefore indicating TCP buffer/load related issue. Maybe situation >>>> may be >>>> improved by creation of more than one connection between two nodes, >>>> but that >>>> may come with a bag of problems on its own. >>>> >>>> >>>> -----Original Message----- >>>> From: erlang-questions@REDACTED [mailto:erlang- >>>> questions@REDACTED] On >>>> Behalf Of Ulf Wiger >>>> Sent: 25 September 2009 09:13 AM >>>> To: erlang-questions Questions >>>> Subject: [erlang-questions] running without net tick >>>> >>>> >>>> The problem of netsplits in Erlang comes up now and again. >>>> I've mentioned that we used to have a more robust >>>> supervision algorithm for device processor monitoring in >>>> AXD 301... >>>> >>>> I read the following comment in kernel/src/dist_util.erl >>>> >>>> %% Send a TICK to the other side. >>>> %% >>>> %% This will happen every 15 seconds (by default) >>>> %% The idea here is that every 15 secs, we write a little >>>> %% something on the connection if we haven't written anything for >>>> %% the last 15 secs. >>>> %% This will ensure that nodes that are not responding due to >>>> %% hardware errors (Or being suspended by means of ^Z) will >>>> %% be considered to be down. If we do not want to have this >>>> %% we must start the net_kernel (in erlang) without its >>>> %% ticker process, In that case this code will never run >>>> >>>> >>>> ...and thought: promising - it is then possible to experiment >>>> with other tick algorithms? >>>> >>>> However, looking at net_kernel.erl: >>>> >>>> init({Name, LongOrShortNames, TickT}) -> >>>> process_flag(trap_exit,true), >>>> case init_node(Name, LongOrShortNames) of >>>> {ok, Node, Listeners} -> >>>> process_flag(priority, max), >>>> Ticktime = to_integer(TickT), >>>> Ticker = spawn_link(net_kernel, ticker, [self(), >>>> Ticktime]), >>>> >>>> In other words, you can't set net_ticktime to anything other >>>> than an integer (and it has to be a smallint, since it's used >>>> in a receive ... after expression. >>>> >>>> (To do justice to the comment above, couldn't a net_ticktime >>>> of, say, 0 turn off net ticking altogether?) >>>> >>>> What one can do then, is to set net_ticktime to a very large >>>> number, and then run a user-level heartbeat. If netsplits are >>>> still experienced without visible problems in the user-level >>>> monitoring, or perhaps even serviced traffic during this >>>> interval, then something is definitely wrong with the tick >>>> algorithm. :) >>>> >>>> BR, >>>> Ulf W >>>> -- >>>> Ulf Wiger >>>> CTO, Erlang Training & Consulting Ltd >>>> http://www.erlang-consulting.com >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>> >>> >>> >>> -- >>> Jayson Vantuyl >>> kagato@REDACTED >>> >>> >>> >>> >>> >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> >> >> >> -- >> Jayson Vantuyl >> kagato@REDACTED >> >> >> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From v@REDACTED Sat Sep 26 02:53:17 2009 From: v@REDACTED (Valentin Micic) Date: Sat, 26 Sep 2009 02:53:17 +0200 Subject: [erlang-questions] running without net tick In-Reply-To: <32F30C3C-F7E2-45DE-8D3B-858275FF0EC9@souja.net> Message-ID: <200909260053.n8Q0rsSw010462@mail.pharos-avantgard.com> Ticks do ticking :-). The following is from Erlang's documentation: "Once every TickTime/4 second, all connected nodes are ticked (if anything else has been written to a node) and if nothing has been received from another node within the last four (4) tick times that node is considered to be down. This ensures that nodes which are not responding, for reasons such as hardware errors, are considered to be down." Reading this, I would never conclude that losing one tick would cause a node to be considered unreachable -- primarily because tick cannot be lost, as it uses reliable connection (TCP) to talk to its peer. What I was saying in reply to Ulf's original post was that I've seen nodes that were collocated on the same machine ending-up net-splitting (?!?), and that cannot be explained by network problem or packet loss to that matter nor any change to tick values may influence. You keep mentioning head-of-line blocking, well, double it and you'll see how severe it can be -- I cannot receive anything because process responsible for receiving is busy blocking on send operation (*); because peer cannot receive anything as it is busy blocking on its send operation. It pretty much resembles this conversation :-). And I've just timed-out! V/ (*) Can anyone confirm that a single process is handling both sending and receiving functions? If this is, indeed, the case -- it would be a good idea to split sending and receiving functions into separate processes, as the asynchronous nature of Erlang distribution mechanism lend itself to a deadlock of this kind. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Jayson Vantuyl Sent: 26 September 2009 01:23 AM To: Valentin Micic Cc: 'Erlang-Questions Questions' Subject: Re: [erlang-questions] running without net tick What do you think that the ticks do? If you lose one, then a node is considered lost. Without a retry strategy that is drastically less than the tick time (i.e. the one currently handled by TCP), you will see splits. If you just use UDP, then ANY packet loss will cause a split. If you are using UDP, then you have to compensate for this, i.e. engineer a retry scheme. Minimally, you have to at least have a tick interval that might (for example) be shorter than the split interval. As we are talking about quite a few nodes, you will have retry synchronization issues. This problem is as old as the Internet. UDP saves bytes and avoids head-of-line blocking, that's about it. With a dedicated TCP connection, there is no head-of-line blocking. And, unlike a retry model with UDP, you don't have to maintain TCP across a few different OSes, and I strongly suggest that it solves this problem in a way that is less wasteful than you seem to think. As for the ball-point pen example, it's a really good example of what's going on. When all they had were pencils, NASA and the Russians both just used pencils. They developed a pen because the pencils were a safety hazard, and then the Americans and Russians used them, because they solved a real problem. Just like the people behind that particular (false, http://www.snopes.com/business/genius/spacepen.asp) urban legend, you don't seem to understand the problem-space. Unless I'm missing something about the net_kernel implementation, the only upside of UDP is saving a few bytes, at the expense of complexity that must be maintained at the core of a fault-tolerant system. On Sep 25, 2009, at 3:53 PM, Valentin Micic wrote: > There is an urban legend saying that NASA invested a good few million > dollars in order to develop a ballpoint pen that can work in zero > gravity. > Russians used pencil. As much as I don't think that Russian pencil > should be > used for signing a nuclear disarmament treaty, it will work just > fine for a > logbook keeping. > > Why would I need to implement a backoff? What horrible behavior you're > talking about? For goodness sake, we're just talking about loosing > packets > that we can afford to lose! Packet loss *is* simple if a problem we're > trying to solve is simple. OTOH, if we apply a complex solution to a > simple > problem, we are elevating the problem to the same level of > complexity as > given by the solution. By indiscriminately applying experience > related to a > complex problem in a context of a simple problem, you are implying > that > simple solutions, hence, problems do not exist. > > Clearly not the case -- I would choose UDP over TCP for > implementation of > management protocol any day of the week and twice on Sunday. Not > because I > am stubborn, but because it worked for me in the past. And still does. > > > V/ > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang- > questions@REDACTED] On > Behalf Of Jayson Vantuyl > Sent: 25 September 2009 10:52 PM > To: Erlang-Questions Questions > Subject: Re: [erlang-questions] running without net tick > > Packet loss is not as simple as you think. With UDP, you either need > to roll a whole backoff implementation or accept that you're going to > have horrible behavior under even modest amounts of packet loss. > > Fragmentation of packets is also particularly mean. It can amplify > latency spectacularly when enough packets are involved. TCP avoids > this. With UDP, you have to roll it for yourself (and you'll need an > ICMP socket to do so, which usually means running as root on Unix). > Ticks might be small enough that this isn't necessary, so you might > get out of this one for free. > > Congestion control is problematic, and TCP implementations have a > number of answers for this that can be tweaked at the OS. UDP does > not. > > I'm not suggesting that the old data is useful. Just throw it away > and have a way to detect that it's old. I am suggesting that having > TCP handle the retries (and letting windowing / retransmission handle > the trouble for you) is less developer work and behaves better. > > TCP having retransmission does not help you make a single decision, > no. Instead, it prevents purely random losses (which are VERY common > in environments like EC2, when network links get saturated) from > causing all problems to be amplified to a multiple of your tick time, > in a ratio that gets exponentially compared to the fraction of lost > packets. > > Implement it using UDP. Implement one using TCP. Compare the > difference in behavior. Notice how the TCP one is simpler, behaves > better, and has similar bandwidth consumption. You're obviously going > to have to learn it through experimentation. That's fine, as that's > how I learned it. I encourage you to keep an open mind, take > measurements, and consider that every line of code may someday give a > bug a happy home. > > Good luck. > > On Sep 25, 2009, at 1:25 PM, Valentin Micic wrote: > >> Let see... say, you send a request over TCP for which you're >> expecting a >> reply -- and nothing happens. The fact that TCP has a retransmission >> mechanism (to ensure a reliable delivery) does not help you make any >> meaningful decision at this point, does it? So, what you're going to >> do when >> this happens? Give up? Retry? Whichever way you slice it, you cannot >> get >> away form implementing some kind of application level protocol to >> handle >> such a condition. >> >> Considering this, UDP makes us work more... how exactly??? >> >> (Actually, sometimes TCP may make it worse. If your request times >> out and >> you do not tear down the connection over which request has been >> made, your >> request will be delivered although there will be no one interested >> in a >> reply. Worse yet, you may be issuing another request, etc. I say, >> sometimes >> you're better off just reliably losing the whole thing. <-:) >> >> Anyway, let's agree to disagree on this one. >> >> V/ >> >> -----Original Message----- >> From: Jayson Vantuyl [mailto:kagato@REDACTED] >> Sent: 25 September 2009 08:53 PM >> To: Valentin Micic >> Cc: 'Erlang-Questions Questions' >> Subject: Re: [erlang-questions] running without net tick >> >> I completely agree on the oil-and-water statement. That said, TCP >> supports OOB, it's just a bad idea to use it. >> >> The theory of "why it's not working" that was mentioned was that the >> other data multiplexed over the stream was choking out the ticks. >> >> A dedicated connection makes that a non-issue. It's the classic >> head- >> of-line blocking problem. >> >> Let's say that ticks were reduced to a 4-byte timestamp (to give some >> reference point if the connection is broken and re-established). >> Let's say you send them every 10 seconds. TCP has a 40-byte >> overhead. Ethernet usually has a 1500-byte MTU. That makes room for >> an hour's worth of ticks in a single TCP packet over average ethernet >> (and probably at least 20-minutes worth over any useable MTU). >> >> If we are limited to IP, want to have any chance to make it through a >> firewall, want timely retries, and want a tick to generate one-packet >> (or less aggregate), we either have UDP or TCP. In the above case, >> TCP generates about as much packet traffic as UDP and is reasonably >> close to timely. The packets are larger and the retries are wasted, >> but they also back off exponentially. A dedicated connection does >> not >> have the "head-of-line" problem due to multiplexing (which is, >> admittedly, unproven as the problem). >> >> The point of this exercise is that, unless you're going over a very >> small or very latent pipe, UDP doesn't really give us anything other >> than more work. Why more work, you say? Because UDP doesn't retry. >> Sending a tick every 10 seconds over TCP is not the same thing as >> sending a tick every 10 seconds over UDP. Why? Assume 75%, random >> packet loss. That means you're likely to get a single UDP tick every >> 40 seconds. With TCP, the automatic retry will turn that into >> approximately one tick every 10.X seconds, where X is entirely >> dependent on latency (and probably very small). Does TCP do this >> with >> more traffic? Yes. However, it does it with exponential backoff, a >> window to limit the outstanding number of packets, PMTU support so >> that the packets don't get fragmented, an RST mechanism to break >> connections if the remote host has rebooted, the option to use SSL to >> encrypt the session, etc. >> >> There are almost no cases that actually demand UDP that a single TCP >> connection doesn't do very well. I'd strongly recommend not ignoring >> its benefits and realizing that real network conditions almost never >> favor UDP and that UDP does not favor a simple implementation. >> >> On Sep 25, 2009, at 4:26 AM, Valentin Micic wrote: >> >>> I beg to differ -- my take is that TCP reliability is a part of the >>> problem >>> in this case. Whilst buffering and flow control is important for, >>> say, file >>> transfer, it is completely irrelevant for TCIK and health-checks (So >>> what if >>> it doesn't get there, I can send it again without any consequence!). >>> >>> Argument about UDP unreliability sounds more like a mantra than a >>> proper >>> argument (if only I got a penny every time I've heard it (-:). There >>> are >>> only two fundamental differences (*) between TCP and UDP... actually >>> only >>> one, because the second is conditioned by the first: TCP supports >>> stream, >>> whilst UDP message-bound communication; thus, as a consequence, TCP >>> requires >>> some form of flow control to support stream processing. >>> >>> In this particular case: what possible benefit one can derive from >>> sending a >>> message over the stream as opposed to sending just a message? If the >>> message >>> is short enough to fit in a datagram -- none! >>> >>> As for ability to send urgent data (OOB) over TCP socket -- data >>> streams and >>> OOB data are mixing like oil and water. I am yet to see a successful >>> utilization of OOB (issued by a user) that hasn't resulted in >>> connection >>> reset (or system shutdown (-;). >>> >>> Lastly, if TICK is implemented via separate TCP socket, that would >>> double >>> networking resources required -- you'd need a new socket for every >>> node >>> you're connected to. With UDP, all you need is one socket, and a >>> very basic >>> protocol: >>> >>> 1) Ask when you have to; >>> 2) Answer when asked. >>> >>> Mind you, net-kernel is already doing this. >>> >>> V/ >>> >>> (*) If one disregards things that UDP can which TCP cannot do, such >>> as a >>> multi-drop, multicasting, etc. >>> >>> -----Original Message----- >>> From: erlang-questions@REDACTED [mailto:erlang- >>> questions@REDACTED] On >>> Behalf Of Jayson Vantuyl >>> Sent: 25 September 2009 12:25 PM >>> To: Erlang-Questions Questions >>> Subject: Re: [erlang-questions] running without net tick >>> >>> Short Version: >>> >>> Why not open a special "tick" TCP port? UDP would require a >>> reliable >>> delivery implementation. TCP saves quite a bit of work in that >>> regard >>> (and gets a lot of important but subtle things right). >>> >>> Long Version: >>> >>> Also, never say never. >>> >>> Actually, you CAN send out-of-band data (also called urgent data) >>> using TCP. The original "WinNuke" (i.e. ping-of-death for Windows >>> 95) >>> was due to having a corrupt OOB header in a TCP packet. In classic >>> Microsoft / Internet style, the issue was further confused because >>> it >>> was an Out-of-Bounds bug, so a generation of networking consultants >>> have minor deviations in their interpretations of the meaning of the >>> letters OOB. >>> >>> As for TCP Urgent Data / OOB, it seems to be specified well enough >>> at >>> the protocol level, but iit doesn't appear to be handled uniformly >>> in >>> different socket implementations. >>> >>> Under Linux, you use send/recv with the MSG_OOB option (or set the >>> SO_OOBINLINE socket option to just inline the data). It appears to >>> try to keep it at a certain point in the data stream (i.e. to >>> preserve >>> some of the ordering) and certain conditions can cause it to become >>> part of the "normal" stream of data. It also can cause some odd >>> signals to be delivered to the process. Still, TCP *does* have OOB >>> data support, just maybe it isn't easily usable everywhere. >>> >>> On Sep 25, 2009, at 3:04 AM, Valentin Micic wrote: >>> >>>> You may change TICK value all day long, but if the underlying >>>> infrastructure >>>> s in some kind of trouble, that alone is not going to solve the >>>> problem. >>>> >>>> The following is just a speculation, but quite plausible in my >>>> mind: >>>> >>>> AFAIK, ERTS is multiplexing inter-nodal traffic over a single >>>> socket. Thus, >>>> if the socket is heavily utilized, the sending buffer may get >>>> congested due >>>> to dynamically reduced TCP window size (because remote side is not >>>> flushing >>>> its buffer fast enough -- if the same process is reading and >>>> writing >>>> the >>>> socket, this may cause a deadlock under a heavy load). As much as I >>>> am not >>>> certain about particular implementation here, I know that sender >>>> will not >>>> wait for ever -- it will eventually timeout and this (exception?) >>>> has to be >>>> handled somehow by the sender. The reasonable course of action >>>> would >>>> be to >>>> reset the connection. If and when that happens, node can be >>>> declared >>>> unreachable; therefore the "net-split" may occur. In other words, >>>> net-split >>>> may occur with or without "ticker" process running and regardless >>>> of >>>> the >>>> real network availability (*). >>>> >>>> >>>> I think the net-tick method is good on its own, however, it is >>>> utilizing a >>>> *wrong* transport! IMO, tick should be handled as out-of-band data, >>>> and this >>>> cannot be done using TCP/IP (well, at least not at the user level). >>>> My >>>> suggestion would be to use UDP for net-kernel communication >>>> (including TICK >>>> messages). This way one would be able to find out about peer health >>>> more >>>> reliably (yes, a small protocol may be required, but that's >>>> relatively >>>> easy). >>>> >>>> To make things simpler regarding the distribution, one may use the >>>> same port >>>> number as advertised in EPMD for a particular node, hence bind UDP >>>> socket to >>>> that number. >>>> >>>> V/ >>>> >>>> (*) I've seen "net-splits" between nodes collocated on the same >>>> machine -- >>>> therefore indicating TCP buffer/load related issue. Maybe situation >>>> may be >>>> improved by creation of more than one connection between two nodes, >>>> but that >>>> may come with a bag of problems on its own. >>>> >>>> >>>> -----Original Message----- >>>> From: erlang-questions@REDACTED [mailto:erlang- >>>> questions@REDACTED] On >>>> Behalf Of Ulf Wiger >>>> Sent: 25 September 2009 09:13 AM >>>> To: erlang-questions Questions >>>> Subject: [erlang-questions] running without net tick >>>> >>>> >>>> The problem of netsplits in Erlang comes up now and again. >>>> I've mentioned that we used to have a more robust >>>> supervision algorithm for device processor monitoring in >>>> AXD 301... >>>> >>>> I read the following comment in kernel/src/dist_util.erl >>>> >>>> %% Send a TICK to the other side. >>>> %% >>>> %% This will happen every 15 seconds (by default) >>>> %% The idea here is that every 15 secs, we write a little >>>> %% something on the connection if we haven't written anything for >>>> %% the last 15 secs. >>>> %% This will ensure that nodes that are not responding due to >>>> %% hardware errors (Or being suspended by means of ^Z) will >>>> %% be considered to be down. If we do not want to have this >>>> %% we must start the net_kernel (in erlang) without its >>>> %% ticker process, In that case this code will never run >>>> >>>> >>>> ...and thought: promising - it is then possible to experiment >>>> with other tick algorithms? >>>> >>>> However, looking at net_kernel.erl: >>>> >>>> init({Name, LongOrShortNames, TickT}) -> >>>> process_flag(trap_exit,true), >>>> case init_node(Name, LongOrShortNames) of >>>> {ok, Node, Listeners} -> >>>> process_flag(priority, max), >>>> Ticktime = to_integer(TickT), >>>> Ticker = spawn_link(net_kernel, ticker, [self(), >>>> Ticktime]), >>>> >>>> In other words, you can't set net_ticktime to anything other >>>> than an integer (and it has to be a smallint, since it's used >>>> in a receive ... after expression. >>>> >>>> (To do justice to the comment above, couldn't a net_ticktime >>>> of, say, 0 turn off net ticking altogether?) >>>> >>>> What one can do then, is to set net_ticktime to a very large >>>> number, and then run a user-level heartbeat. If netsplits are >>>> still experienced without visible problems in the user-level >>>> monitoring, or perhaps even serviced traffic during this >>>> interval, then something is definitely wrong with the tick >>>> algorithm. :) >>>> >>>> BR, >>>> Ulf W >>>> -- >>>> Ulf Wiger >>>> CTO, Erlang Training & Consulting Ltd >>>> http://www.erlang-consulting.com >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>> >>> >>> >>> -- >>> Jayson Vantuyl >>> kagato@REDACTED >>> >>> >>> >>> >>> >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> >> >> >> -- >> Jayson Vantuyl >> kagato@REDACTED >> >> >> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From kagato@REDACTED Sat Sep 26 03:24:57 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 25 Sep 2009 18:24:57 -0700 Subject: [erlang-questions] running without net tick In-Reply-To: <200909260053.n8Q0rsSw010462@mail.pharos-avantgard.com> References: <200909260053.n8Q0rsSw010462@mail.pharos-avantgard.com> Message-ID: <036C957B-57F9-4369-9322-91144A818568@souja.net> Now we're getting somewhere. First, concerning what I've been trying to communicate. With 4 ticks to a timeout and 10 second timeouts, you will have a timeout approximately every three minutes with UDP. I did a few simulations, and with that level of packet loss, I showed 5 events in 20 minutes that lost 4 consecutive ticks. With TCP, highly, highly, highly unlikely. And, as I keep saying, a dedicated connection *lacks* head- of-line blocking, so that's not going to be an issue. Use of UDP would vastly increasesthe loss of ticks in congested networks, while not really saying much about the liveness of a host (which is the point of the ticks). Second, concerning what you've just brought up (as you've never mentioned netsplits on a single machine before), that certainly sounds like some sort of starvation. It's a bug. If you were to use a second TCP socket (as I've suggested), you're likely to have to same behavior. That is, unless it really is due to the ticks getting caught in the backlog, as has been suggested. Again, in that case a second connection fixes it as well. UDP doesn't change any starvation behavior. If emulator is blocked reading from a TCP connection, it can't do a recv. If it's handled non-blocking, another connection is all that's needed, TCP is a lot simpler to implement (and perfectly performant / safe) than a UDP, and it does not amplify the failures due to packet loss in real networks. -- Jayson Vantuyl kagato@REDACTED From igorrs@REDACTED Sat Sep 26 05:54:29 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Sat, 26 Sep 2009 00:54:29 -0300 Subject: Basic question about Erlang binaries Message-ID: Hello. Can I retrieve the value of a specific bit position of an arbitrary Erlang binary in guaranteed constant time? >From some tests I've made, it seems pattern matching to pick a specific bit near the middle of a binary takes linear time with respect to the size of the binary. Is that correct? Thanks. Igor. -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From seancribbs@REDACTED Sat Sep 26 06:29:51 2009 From: seancribbs@REDACTED (Sean Cribbs) Date: Sat, 26 Sep 2009 00:29:51 -0400 Subject: [erlang-questions] Basic question about Erlang binaries In-Reply-To: References: Message-ID: <4ABD98BF.5080605@gmail.com> If the position of this bit is constant, I would think so, e.g. for the 33rd bit: <<_:32, Bit:1/integer, _Rest/binary>> = Binary. I am not certain of the implementation details of binaries, but I believe it's simple pointer arithmetic. If that's the case, then it would be proportional to the offset into the binary that you wanted. How are you doing the pattern matching? Sean Igor Ribeiro Sucupira wrote: > Hello. > > Can I retrieve the value of a specific bit position of an arbitrary > Erlang binary in guaranteed constant time? > > >From some tests I've made, it seems pattern matching to pick a > specific bit near the middle of a binary takes linear time with > respect to the size of the binary. Is that correct? > > Thanks. > Igor. > > From mjtruog@REDACTED Sat Sep 26 07:34:52 2009 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 25 Sep 2009 22:34:52 -0700 Subject: monitoring an Erlang port? Message-ID: <4ABDA7FC.4040408@gmail.com> There currently isn't a way (even if undocumented) to monitor an Erlang port directly to determine if it is responding to tick messages, right? (not an Erlang port driver, since those can be monitored as drivers in erl_ddll) Thanks, Michael From mjtruog@REDACTED Sat Sep 26 07:42:09 2009 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 25 Sep 2009 22:42:09 -0700 Subject: [erlang-questions] monitoring an Erlang port? In-Reply-To: <4ABDA7FC.4040408@gmail.com> References: <4ABDA7FC.4040408@gmail.com> Message-ID: <4ABDA9B1.3080607@gmail.com> Sorry, was confusing this with cnodes. Ignore the question please. Michael Truog wrote: > There currently isn't a way (even if undocumented) to monitor an Erlang > port directly to determine if it is responding to tick messages, right? > (not an Erlang port driver, since those can be monitored as drivers in > erl_ddll) > > Thanks, > Michael > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From v@REDACTED Sat Sep 26 08:18:32 2009 From: v@REDACTED (Valentin Micic) Date: Sat, 26 Sep 2009 08:18:32 +0200 Subject: [erlang-questions] running without net tick In-Reply-To: <036C957B-57F9-4369-9322-91144A818568@souja.net> Message-ID: <200909260619.n8Q6J9Sw011631@mail.pharos-avantgard.com> > And, as I keep saying, a dedicated connection *lacks* head- > of-line blocking, so that's not going to be an issue. This is a very brave statement that has no foundation in reality. > Second, concerning what you've just brought up (as you've never > mentioned netsplits on a single machine before) Another statement that has no foundation in reality. Listen, I have better things to do with my time. V/ From igorrs@REDACTED Sat Sep 26 08:25:13 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Sat, 26 Sep 2009 03:25:13 -0300 Subject: [erlang-questions] Basic question about Erlang binaries In-Reply-To: <4ABD98BF.5080605@gmail.com> References: <4ABD98BF.5080605@gmail.com> Message-ID: On Sat, Sep 26, 2009 at 1:29 AM, Sean Cribbs wrote: > If the position of this bit is constant, I would think so, e.g. for the 33rd > bit: Well... formally, if the bit position is constant, you can access that position in constant time, even if the access time of an arbitrary position in an arbitrary Erlang binary is linear. ;-) Just to state the question more clearly: Is there a way to retrieve the value of an arbitrary bit position of an arbitrary Erlang binary in guaranteed constant time? > <<_:32, Bit:1/integer, _Rest/binary>> = Binary. That pattern matching never works, as the number of bits in _Rest will never be a multiple of 8 (so it can never be an Erlang binary). > I am not certain of the implementation details of binaries, but I believe > it's simple pointer arithmetic. >From my tests, it doesn't seem to be. I was doing the pattern matching like this: <<_:X, Bit:1, _:Y>> = Binary. Where X + 1 + Y is, of course, the number of bits in Binary (known at that point, obviously) and X + 1 is the position I want to access. I also tried something similar to what you did (only making sure I only picked positions which were at the end of some octet), but in both cases the pattern matching seems to take linear time with respect to the desired position. > ?If that's the case, then it would be > proportional to the offset into the binary that you wanted. If it was just pointer arithmetic, it would be constant (not proportional to the offset). But, as I said, it doesn't seem to be. Try this in the shell: Binary = term_to_binary(lists:seq(1, 1000000)). lists:foreach(fun(_) -> <<_:7, Bit:1, _/binary>> = Binary end, lists:seq(1, 1000000)). lists:foreach(fun(_) -> <<_:1048575, Bit:1, _/binary>> = Binary end, lists:seq(1, 1000000)). The second statement takes some seconds to run, while the third one will take forever (of course I haven't waited :-)). Thank you for replying. Igor. > ?How are you > doing the pattern matching? > > Sean > > Igor Ribeiro Sucupira wrote: >> >> Hello. >> >> Can I retrieve the value of a specific bit position of an arbitrary >> Erlang binary in guaranteed constant time? >> >> >From some tests I've made, it seems pattern matching to pick a >> specific bit near the middle of a binary takes linear time with >> respect to the size of the binary. Is that correct? >> >> Thanks. >> Igor. >> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From igorrs@REDACTED Sat Sep 26 09:04:27 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Sat, 26 Sep 2009 04:04:27 -0300 Subject: [erlang-questions] Basic question about Erlang binaries In-Reply-To: References: <4ABD98BF.5080605@gmail.com> Message-ID: On Sat, Sep 26, 2009 at 3:25 AM, Igor Ribeiro Sucupira wrote: > On Sat, Sep 26, 2009 at 1:29 AM, Sean Cribbs wrote: > > If it was just pointer arithmetic, it would be constant (not > proportional to the offset). But, as I said, it doesn't seem to be. > > Try this in the shell: > > Binary = term_to_binary(lists:seq(1, 1000000)). > lists:foreach(fun(_) -> <<_:7, Bit:1, _/binary>> = Binary end, > lists:seq(1, 1000000)). > lists:foreach(fun(_) -> <<_:1048575, Bit:1, _/binary>> = Binary end, > lists:seq(1, 1000000)). > > The second statement takes some seconds to run, while the third one > will take forever (of course I haven't waited ?:-)). Hum... this one below finishes fast. :-) lists:foreach(fun(_) -> <<_:1048568/binary, Bit:8, _/binary>> = Binary end, lists:seq(1, 1000000)). What happened in the other example (with _:1048575)? Is some validation taking place there to make sure an integer matches the initial portion of Binary (?)? Just curious. :-) Thanks. Igor. From rvirding@REDACTED Sat Sep 26 14:56:37 2009 From: rvirding@REDACTED (Robert Virding) Date: Sat, 26 Sep 2009 14:56:37 +0200 Subject: [erlang-questions] Basic question about Erlang binaries In-Reply-To: References: <4ABD98BF.5080605@gmail.com> Message-ID: <3dbc6d1c0909260556r464a439dqe94d889988df82fb@mail.gmail.com> 2009/9/26 Igor Ribeiro Sucupira > On Sat, Sep 26, 2009 at 3:25 AM, Igor Ribeiro Sucupira > wrote: > > On Sat, Sep 26, 2009 at 1:29 AM, Sean Cribbs > wrote: > > > > If it was just pointer arithmetic, it would be constant (not > > proportional to the offset). But, as I said, it doesn't seem to be. > > > > Try this in the shell: > > > > Binary = term_to_binary(lists:seq(1, 1000000)). > > lists:foreach(fun(_) -> <<_:7, Bit:1, _/binary>> = Binary end, > > lists:seq(1, 1000000)). > > lists:foreach(fun(_) -> <<_:1048575, Bit:1, _/binary>> = Binary end, > > lists:seq(1, 1000000)). > > > > The second statement takes some seconds to run, while the third one > > will take forever (of course I haven't waited :-)). > > > Hum... this one below finishes fast. :-) > > lists:foreach(fun(_) -> <<_:1048568/binary, Bit:8, _/binary>> = Binary > end, lists:seq(1, 1000000)). > > What happened in the other example (with _:1048575)? Is some > validation taking place there to make sure an integer matches the > initial portion of Binary (?)? > > Just curious. :-) > This is not really strange if you consider what you are asking it to do. You are telling it to build a 1048575 bit integer, which you then discard, with bits from the binary. It will actually build the integer here before it throws it away. When you tell it to build a 1048568 bit binary it can do that without copying the data so it is much faster. I haven't tested this in the compiler if it actually builds the integer. Instead of /binary you can use /bits or /bitstring (synonyms) to build binaries without the restriction of being full bytes. Robert From igorrs@REDACTED Sat Sep 26 18:06:24 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Sat, 26 Sep 2009 13:06:24 -0300 Subject: [erlang-questions] Basic question about Erlang binaries In-Reply-To: <3dbc6d1c0909260556r464a439dqe94d889988df82fb@mail.gmail.com> References: <4ABD98BF.5080605@gmail.com> <3dbc6d1c0909260556r464a439dqe94d889988df82fb@mail.gmail.com> Message-ID: On Sat, Sep 26, 2009 at 9:56 AM, Robert Virding wrote: > 2009/9/26 Igor Ribeiro Sucupira >> >> On Sat, Sep 26, 2009 at 3:25 AM, Igor Ribeiro Sucupira >> wrote: >> > On Sat, Sep 26, 2009 at 1:29 AM, Sean Cribbs >> > wrote: >> > >> > If it was just pointer arithmetic, it would be constant (not >> > proportional to the offset). But, as I said, it doesn't seem to be. >> > >> > Try this in the shell: >> > >> > Binary = term_to_binary(lists:seq(1, 1000000)). >> > lists:foreach(fun(_) -> <<_:7, Bit:1, _/binary>> = Binary end, >> > lists:seq(1, 1000000)). >> > lists:foreach(fun(_) -> <<_:1048575, Bit:1, _/binary>> = Binary end, >> > lists:seq(1, 1000000)). >> > >> > The second statement takes some seconds to run, while the third one >> > will take forever (of course I haven't waited ?:-)). >> >> >> Hum... this one below finishes fast. ?:-) >> >> lists:foreach(fun(_) -> <<_:1048568/binary, Bit:8, _/binary>> = Binary >> end, lists:seq(1, 1000000)). >> >> What happened in the other example (with _:1048575)? Is some >> validation taking place there to make sure an integer matches the >> initial portion of Binary (?)? >> >> Just curious. ?:-) > > This is not really strange if you consider what you are asking it to do. You > are telling it to build a 1048575 bit integer, Am I? Maybe I'm missing some concept, but to me it seems I'm asking it to build Bit and to make sure the pattern matching is valid. So, what is necessary is to check that the integer can be built and is unique. Can't this be done in constant time? In other words, I believe the pattern matching above means something like: "Can you please give me the 1048576th bit, as long as the first 1048575 bits represent a unique (signed, little-endian, blablabla) integer and and the rest represent an Erlang binary?" I'm being pretty informal, as it's Saturday (:-)), but I believe you understand my point. > which you then discard, with > bits from the binary. It will actually build the integer here before it > throws it away. When you tell it to build a 1048568 bit binary it can do > that without copying the data I known. I just didn't think it would actually build it and then discard it. > so it is much faster. I haven't tested this in > the compiler if it actually builds the integer. > > Instead of /binary you can use /bits or /bitstring (synonyms) to build > binaries without the restriction of being full bytes. Hum... thanks. I believe I heard that before, but had forgotten it. Sean's pattern would work with a small change, then: <<_:32, Bit:1/integer, _Rest/bits>> = Binary. > Robert Thank you. Igor. -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From ulf.wiger@REDACTED Sat Sep 26 20:38:58 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 26 Sep 2009 20:38:58 +0200 Subject: [erlang-questions] running without net tick In-Reply-To: <200909251005.n8PA54Sw006936@mail.pharos-avantgard.com> References: <200909251005.n8PA54Sw006936@mail.pharos-avantgard.com> Message-ID: <4ABE5FC2.8070608@erlang-consulting.com> Valentin Micic wrote: > You may change TICK value all day long, but if the underlying infrastructure > s in some kind of trouble, that alone is not going to solve the problem. FWIW, we once experienced a particularly confusing kind of false positive with the net ticks. This happened during /very low/ network load. The problem was a bug in the internal communication paths of the AXD 301, which caused some packets to be lost, and the max_rexmit_interval setting happened to be higher than the net_ticktime. So basically, the TCP stack could end up setting the retransmit time to longer than Erlang's detection time (which we had set to 10 seconds.) BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From guilherme.silveira@REDACTED Sat Sep 26 21:21:27 2009 From: guilherme.silveira@REDACTED (Guilherme Silveira) Date: Sat, 26 Sep 2009 16:21:27 -0300 Subject: [erlang-questions] changing the internal message transfer protocol In-Reply-To: <20090925165423.GA6174@hanele> References: <581bf6160909250759o1b634f01m7c1810143c47d0f1@mail.gmail.com> <20090925152145.GA5947@hanele> <581bf6160909250908v50f58c80kbbd7d9f9a66fdf94@mail.gmail.com> <20090925165423.GA6174@hanele> Message-ID: <581bf6160909261221lce7b31j228a9b4b64d1422f@mail.gmail.com> Thanks... found the source code I was looking for Regards Guilherme Silveira Caelum | Ensino e Inova??o SP (11) 5571-2751 RJ (21) 2220-4156 Celular (11) 83358084 2009/9/25 Jachym Holecek : > # Guilherme Silveira 2009-09-25: >> >> Pid = spawn(NodeName, M, F, A). >> > See lib/kernel/src/erlang.erl:spawn/4. >> spawn invokes gen_server when NodeName =/= node(), which invokes >> gen:call, which invokes erlang:send. >> >> But I was unable to find who exports erlang:send... the erlang module >> (erlang.erl) only exports dsend, which connnects+invokes send. > > Yep, some of the functions in various "core" modules are implemented > directly in C instead of the corresponding Erlang module. Such functions > are called "builtin functions" and live in bif.c/erl_bif_${module}.c and > possibly other places I missed, see below on how to lookup their home. > >> Reading the net_kernel and do_connect implementations, it seems as >> connections are treated with their own local pids and kept alive. But >> still... >> >> Any suggestions where to find the send implementation? > > Builtin functions are described in erts/emulator/beam/bif.tab, this > is used to generate BIF dispatch table at compile time using the > erts/emulator/utils/make_tables script. For send/2 we have: > > ?bif erlang:send/2 > > which according to make_tables seems to mean (and my Perl-fu is very > weak) that it's implemented by the send_2() function -- this lives > in bif.c as "grep -rs '^send_2[(]' erts/" reveals. > > This has been a fair amount of guesswork; I'm sure someone will correct > me if I'm wrong. > > HTH, > ? ? ? ?-- Jachym > From rvirding@REDACTED Sat Sep 26 22:04:58 2009 From: rvirding@REDACTED (Robert Virding) Date: Sat, 26 Sep 2009 22:04:58 +0200 Subject: [erlang-questions] Basic question about Erlang binaries In-Reply-To: References: <4ABD98BF.5080605@gmail.com> <3dbc6d1c0909260556r464a439dqe94d889988df82fb@mail.gmail.com> Message-ID: <3dbc6d1c0909261304m6a8c74b6q7a772fdfa13e8edd@mail.gmail.com> 2009/9/26 Igor Ribeiro Sucupira > > > > This is not really strange if you consider what you are asking it to do. > You > > are telling it to build a 1048575 bit integer, > > Am I? > Maybe I'm missing some concept, but to me it seems I'm asking it to > build Bit and to make sure the pattern matching is valid. So, what is > necessary is to check that the integer can be built and is unique. > Can't this be done in constant time? > > In other words, I believe the pattern matching above means something > like: "Can you please give me the 1048576th bit, as long as the first > 1048575 bits represent a unique (signed, little-endian, blablabla) > integer and and the rest represent an Erlang binary?" > I'm being pretty informal, as it's Saturday (:-)), but I believe you > understand my point. > Yes I understand your point. :-) The problem is that what you MEAN is give me the 1048576th bit, but what you WRITE in your code is: take the first 1048575 bits and make an integer, then throw it away; take 1 bit and return it; put the remaining bits into a binary and then throw it away. This may be a bit on the philosophical side but pattern matching is really pulling a structure apart into its components and not really accessing a small part of a structure, although it is often used that way. As in this case. That is why you have to give a pattern for the whole binary. Or so I have always considered it. Now the interpreter is pretty naive and does exactly what you tell it, but the compiler should be better, though I haven't tried it. Robert From igorrs@REDACTED Sat Sep 26 22:43:31 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Sat, 26 Sep 2009 17:43:31 -0300 Subject: [erlang-questions] Basic question about Erlang binaries In-Reply-To: <3dbc6d1c0909261304m6a8c74b6q7a772fdfa13e8edd@mail.gmail.com> References: <4ABD98BF.5080605@gmail.com> <3dbc6d1c0909260556r464a439dqe94d889988df82fb@mail.gmail.com> <3dbc6d1c0909261304m6a8c74b6q7a772fdfa13e8edd@mail.gmail.com> Message-ID: On Sat, Sep 26, 2009 at 5:04 PM, Robert Virding wrote: > 2009/9/26 Igor Ribeiro Sucupira >> >> > This is not really strange if you consider what you are asking it to do. >> > You >> > are telling it to build a 1048575 bit integer, >> >> Am I? >> Maybe I'm missing some concept, but to me it seems I'm asking it to >> build Bit and to make sure the pattern matching is valid. So, what is >> necessary is to check that the integer can be built and is unique. >> Can't this be done in constant time? >> >> In other words, I believe the pattern matching above means something >> like: "Can you please give me the 1048576th bit, as long as the first >> 1048575 bits represent a unique (signed, little-endian, blablabla) >> integer and and the rest represent an Erlang binary?" >> I'm being pretty informal, as it's Saturday (:-)), but I believe you >> understand my point. > > Yes I understand your point. :-) > > The problem is that what you MEAN is give me the 1048576th bit, but what you > WRITE in your code is: take the first 1048575 bits and make an integer, then > throw it away; take 1 bit and return it; put the remaining bits into a > binary and then throw it away. > > This may be a bit on the philosophical side but pattern matching is really > pulling a structure apart into its components and not really accessing a > small part of a structure, although it is often used that way. As in this > case. That is why you have to give a pattern for the whole binary. Or so I > have always considered it. > > Now the interpreter is pretty naive and does exactly what you tell it, but > the compiler should be better, You are right. I experimented with some compiled code and it behaves like a smart compiler should. :-) Thanks. Igor. -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From jeraymond@REDACTED Sun Sep 27 00:12:42 2009 From: jeraymond@REDACTED (Jeremy Raymond) Date: Sat, 26 Sep 2009 18:12:42 -0400 Subject: [erlang-questions] How to test for missing tail recursive calls in service loops? In-Reply-To: <2DAACBD4-92B2-477A-9D92-ECA5F039E009@souja.net> References: <59da11980909251311l4fb46dd8t8313a78b485f4281@mail.gmail.com> <2DAACBD4-92B2-477A-9D92-ECA5F039E009@souja.net> Message-ID: <59da11980909261512y7a8c74ffr185f3728ddce2917@mail.gmail.com> Oh man, I was hoping for a good mind-reading solution. Thanks both for your suggestions. I'll try the function idea and give typer a look. On Fri, Sep 25, 2009 at 4:31 PM, Jayson Vantuyl wrote: > You can implement the "work" part of the loop in a function, then wrap the > whole thing in a process that always tail-recurses (and use exit() to exit > the process). That way you just write your function like it's going to > loop, and it does. > > Or, you can just use gen_server, which does something effectively like this > (it calls you and loops for you). > > In general, though, there's not really a way to detect this, as there are > plenty of normal "termination conditions" where a process should > legitimately exit instead of recursing. Unfortunately, it's a compiler, not > a mind-reader. :( > > > On Sep 25, 2009, at 1:11 PM, Jeremy Raymond wrote: > > Hello, >> >> If a service loop is missing a tail recursive call in a receive clause >> then >> the first time a message is sent to the process matching the bad clause >> the >> call the receive succeeds. However subsequent calls will fail as the >> process >> is no longer waiting on a receive due to the missing tail recursive call. >> Is >> there a good way to test for this error besides just make multiple calls >> to >> the service matching the same clause? Is there some way to interrogate the >> process to see if it's currently waiting on a receive? >> >> >> Thanks, >> >> Jeremy >> > > > -- > Jayson Vantuyl > kagato@REDACTED > > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ofer.affias@REDACTED Sun Sep 27 08:40:48 2009 From: ofer.affias@REDACTED (vim) Date: Sat, 26 Sep 2009 23:40:48 -0700 (PDT) Subject: mneisa table fragmentation vs manually splitting Message-ID: <05840888-1633-4702-a3ce-839b038fe9a4@m20g2000vbp.googlegroups.com> hi, i am working on a erlang based distributed architecture, at the end i intend to use it on EC2. my application included a database, and since i want to spread the load on the database, i thought of splitting the table. the question is should i choose mnesia built-in feature 'fragmented table' over manually splitting the database into clusters? i.e. suppose i have a simple key-value sets which i need to store in my database. and suppose i have 1000 keys, and i did some measurements and i want only 10 keys on each machine (for better performance) so i need 100 machines. thus, if i understand correctly the documentation, i have at least two options: 1. manual split: i can set up clusters of 10 machines each. on each machine i will have a node with a running mnesia, and on a such cluster all the mnesia nodes will be connected. furthermore, i can use master-slave duplicated database scheme, if my application will know how to use a specific node to write to a specific key. i assume here that *all* the nodes in my world (100) are erlang connected, and on each 10 machine cluster there is a connected (shared) mnesia database. 2. table fragmentation: i can connect all my 100 machines with single mnesia schema, and fragment the table into 10 pieces, where each cluster of 10 machines will share each piece. again, i can use the same master-slave scheme i described above to access my database. am i correct with those assumptions? should i prefer one method over the other? thanks for your help, vim. From mononcqc@REDACTED Sun Sep 27 18:04:09 2009 From: mononcqc@REDACTED (Fred Hebert (MononcQc)) Date: Sun, 27 Sep 2009 12:04:09 -0400 Subject: EUnit and Servers Message-ID: <8b9ee55b0909270904g18964525j8e6f6284c598c9b6@mail.gmail.com> I have used EUnit before, but rarely did server testing with it. I decided to give it a hand with some toy code of mine, but I'm getting errors I'm not able to understand the cause of. I'm not willing to blame the tools (eunit) before the tool (me), so I'm asking for help here: Here's a paste of the test module: http://paste.lisp.org/display/87803 I don't think it's really important to understand the intricacies of the functions from box or air, but you can get a quick description at the bottom of this post if you want. When running the tests, I get the result: 20> c("tests/box_tests"). {ok,box_tests} 21> eunit:test(box). box_tests: status_test_...*failed* ::exit:{noproc,{gen_server,call,[box,get_status]}} in function gen_server:call/2 in call from box_tests:'-status_test_/0-fun-2-'/0 in call from box_tests:'-status_test_/0-fun-4-'/0 ======================================================= Failed: 1. Skipped: 0. Passed: 5. So it seems like the box:start_link/0 function for this specific test failed somehow. What happens when I just add a second version of add_test_/0 and name it add2_test_/0 is that now, add_test_/0 *AND* status_test_/0 now both fail, but add2_test_/0 won't fail. I'm not quite sure why this happens; all test setups are pretty much the same, but some work and others won't. I have obviously done something wrong but can't find what. Anyone got an idea? (I'm guessing race conditions on start_link/1s and stop/0s, but I'm not quite sure). Thanks in advance. --------------------------------------- description of the functions: air defines an #air{} record with quantities of oxygen, co2, n2 and argon in it. air provides functions to work on them: total of gases, the ratio of each of them, a map and mergewith function (applying a function on each element of the record, returning a new one), etc. box is a server holding a quantity of gas. get_air/1 will subtract a quantity of air from the box. add_air/1 will add air (as the name says it) and get_status/0 shows the quantity of each gas in the box. init/1 is called all the time because it will create a new #air{} record with proportions similar to what we've got in the atmosphere. From kevin@REDACTED Sun Sep 27 18:19:06 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Sun, 27 Sep 2009 12:19:06 -0400 Subject: [erlang-questions] EUnit and Servers In-Reply-To: <8b9ee55b0909270904g18964525j8e6f6284c598c9b6@mail.gmail.com> References: <8b9ee55b0909270904g18964525j8e6f6284c598c9b6@mail.gmail.com> Message-ID: <682A5E7C-3C11-4AC8-A69A-DB9DCD850B26@hypotheticalabs.com> Whenever I get errors like that it means the tests are running before setup. I haven't used the inorder directive at all but you could try changing the macros to their underscore equivalent like this: ?assert - > ?_assert, etc. --Kevin On Sep 27, 2009, at 12:04 PM, Fred Hebert (MononcQc) wrote: > I have used EUnit before, but rarely did server testing with it. I > decided > to give it a hand with some toy code of mine, but I'm getting errors > I'm not > able to understand the cause of. I'm not willing to blame the tools > (eunit) > before the tool (me), so I'm asking for help here: > > Here's a paste of the test module: > http://paste.lisp.org/display/87803 > > I don't think it's really important to understand the intricacies of > the > functions from box or air, but you can get a quick description at > the bottom > of this post if you want. > > When running the tests, I get the result: > > 20> c("tests/box_tests"). > {ok,box_tests} > 21> eunit:test(box). > box_tests: status_test_...*failed* > ::exit:{noproc,{gen_server,call,[box,get_status]}} > in function gen_server:call/2 > in call from box_tests:'-status_test_/0-fun-2-'/0 > in call from box_tests:'-status_test_/0-fun-4-'/0 > > > ======================================================= > Failed: 1. Skipped: 0. Passed: 5. > > > So it seems like the box:start_link/0 function for this specific test > failed somehow. > What happens when I just add a second version of add_test_/0 and > name it > add2_test_/0 is that now, add_test_/0 *AND* status_test_/0 now both > fail, > but add2_test_/0 won't fail. > > I'm not quite sure why this happens; all test setups are pretty much > the > same, but some work and others won't. I have obviously done > something wrong > but can't find what. Anyone got an idea? (I'm guessing race > conditions on > start_link/1s and stop/0s, but I'm not quite sure). > > Thanks in advance. > > --------------------------------------- > description of the functions: > air defines an #air{} record with quantities of oxygen, co2, n2 and > argon in > it. air provides functions to work on them: total of gases, the > ratio of > each of them, a map and mergewith function (applying a function on > each > element of the record, returning a new one), etc. > > box is a server holding a quantity of gas. get_air/1 will subtract a > quantity of air from the box. add_air/1 will add air (as the name > says it) > and get_status/0 shows the quantity of each gas in the box. init/1 > is called > all the time because it will create a new #air{} record with > proportions > similar to what we've got in the atmosphere. From jeraymond@REDACTED Sun Sep 27 18:53:48 2009 From: jeraymond@REDACTED (Jeremy Raymond) Date: Sun, 27 Sep 2009 12:53:48 -0400 Subject: eunit or common_test? Message-ID: <59da11980909270953y48aa79a5q9d3d43ec2fcf62d6@mail.gmail.com> Hello, I've been taking a look a both eunit and common test. Are there any advantages one over the other, or does it really just come down to preference on which to use? Thanks, Jeremy From peter@REDACTED Sun Sep 27 19:12:05 2009 From: peter@REDACTED (Peter Sabaini) Date: Sun, 27 Sep 2009 19:12:05 +0200 Subject: Erlang on smart phones Message-ID: <1254071525.8966.44.camel@gram.local> Hi, there seems to be an effort to make Erlang/OTP run on iPhones -- are there ports for any other smart phones, eg. Android-based ones? Thx, peter. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From rory@REDACTED Sun Sep 27 19:12:37 2009 From: rory@REDACTED (Rory Byrne) Date: Sun, 27 Sep 2009 19:12:37 +0200 Subject: Erlang used to build www.edgynet.com Message-ID: <20090927171237.GA9847@almeida.jinsky.com> Hi Everyone, We've just released a web-application where the server-side is written entirely in Erlang and I thought it might be of interest to other Erlang developers working on web-related projects. It's an online network simulation environment called Edgynet: http://www.edgynet.com We're using the Bayeux protocol [1] for two-way asynchronous messaging between the browser and the server. MochiWeb handles all the HTTP. If you're outside of Europe, you'll probably find the shells on the site are quite sluggish. This is due to latency [2]. The demo server is in Germany, and ideally you want to have a ping round-trip time to it (www.edgynet.com) of 60ms or less. The screencasts posted on the site should give an indication of how responsive shell interaction will be on a low latency connection. Many thanks to the Erlang/OTP Team. Erlang is incredible! Best regards, Rory [1] http://cometd.org/documentation/bayeux [2] Well it's actually the effect of latency combined with the limitation that a web browser will only allow 2 outstanding HTTP requests to a server at any one time (as per the HTTP 1.1 spec). From mononcqc@REDACTED Sun Sep 27 19:14:55 2009 From: mononcqc@REDACTED (Fred Hebert) Date: Sun, 27 Sep 2009 13:14:55 -0400 Subject: [erlang-questions] EUnit and Servers In-Reply-To: <682A5E7C-3C11-4AC8-A69A-DB9DCD850B26@hypotheticalabs.com> References: <8b9ee55b0909270904g18964525j8e6f6284c598c9b6@mail.gmail.com> <682A5E7C-3C11-4AC8-A69A-DB9DCD850B26@hypotheticalabs.com> Message-ID: <8b9ee55b0909271014n52e4535dn88d614fed6784959@mail.gmail.com> On Sun, Sep 27, 2009 at 12:19 PM, Kevin A. Smith wrote: > Whenever I get errors like that it means the tests are running before > setup. I haven't used the inorder directive at all but you could try > changing the macros to their underscore equivalent like this: ?assert -> > ?_assert, etc. > > --Kevin > On Sep 27, 2009, at 12:04 PM, Fred Hebert (MononcQc) wrote: > > I have used EUnit before, but rarely did server testing with it. I decided >> to give it a hand with some toy code of mine, but I'm getting errors I'm >> not >> able to understand the cause of. I'm not willing to blame the tools >> (eunit) >> before the tool (me), so I'm asking for help here: >> >> Here's a paste of the test module: >> http://paste.lisp.org/display/87803 >> >> I don't think it's really important to understand the intricacies of the >> functions from box or air, but you can get a quick description at the >> bottom >> of this post if you want. >> >> When running the tests, I get the result: >> >> 20> c("tests/box_tests"). >> {ok,box_tests} >> 21> eunit:test(box). >> box_tests: status_test_...*failed* >> ::exit:{noproc,{gen_server,call,[box,get_status]}} >> in function gen_server:call/2 >> in call from box_tests:'-status_test_/0-fun-2-'/0 >> in call from box_tests:'-status_test_/0-fun-4-'/0 >> >> >> ======================================================= >> Failed: 1. Skipped: 0. Passed: 5. >> >> >> So it seems like the box:start_link/0 function for this specific test >> failed somehow. >> What happens when I just add a second version of add_test_/0 and name it >> add2_test_/0 is that now, add_test_/0 *AND* status_test_/0 now both fail, >> but add2_test_/0 won't fail. >> >> I'm not quite sure why this happens; all test setups are pretty much the >> same, but some work and others won't. I have obviously done something >> wrong >> but can't find what. Anyone got an idea? (I'm guessing race conditions on >> start_link/1s and stop/0s, but I'm not quite sure). >> >> Thanks in advance. >> >> --------------------------------------- >> description of the functions: >> air defines an #air{} record with quantities of oxygen, co2, n2 and argon >> in >> it. air provides functions to work on them: total of gases, the ratio of >> each of them, a map and mergewith function (applying a function on each >> element of the record, returning a new one), etc. >> >> box is a server holding a quantity of gas. get_air/1 will subtract a >> quantity of air from the box. add_air/1 will add air (as the name says it) >> and get_status/0 shows the quantity of each gas in the box. init/1 is >> called >> all the time because it will create a new #air{} record with proportions >> similar to what we've got in the atmosphere. >> > > I have tried this before and here's the problem: http://paste.lisp.org/display/87803#2 The test still passes, even though there's a very obvious ?_assert(a == b) that should fail. I'M guessing I just don't understand how EUnit is supposed to work in that case. I'M a bit lost here. From richardc@REDACTED Sun Sep 27 19:16:11 2009 From: richardc@REDACTED (Richard Carlsson) Date: Sun, 27 Sep 2009 19:16:11 +0200 Subject: [erlang-questions] EUnit and Servers In-Reply-To: <8b9ee55b0909270904g18964525j8e6f6284c598c9b6@mail.gmail.com> References: <8b9ee55b0909270904g18964525j8e6f6284c598c9b6@mail.gmail.com> Message-ID: <4ABF9DDB.9060000@it.uu.se> The first of your test generators creates test objects as it should: init_test_() -> [?_assert(close_enough(air:new(21, 0.04, 78, 0.9), get_val(box:init(100)))), ?_assert(close_enough(air:new(0.21, 0.0004, 0.78, 0.009), get_val(box:init(1)))), ?_assertError(function_clause, box:init(-1))]. (i.e., using macro calls beginning with ?_...). However, the rest of the generators are using the macro forms that will execute immediately, when the generator itself runs: add_test_() -> {setup, fun() -> box:start_link(0) end, fun(_) -> box:stop() end, fun() -> {inorder, [?assertEqual(ok, box:add_air(get_val(box:init(100)))), ?assert(close_enough(box:get_air(200), get_val(box:init(100))))]} end}. Just change those ?assert... to ?_assert... and try again. This is also why the total count in the report was weird (just 6?): ======================================================= Failed: 1. Skipped: 0. Passed: 5. I know that EUnit is currently not good at telling you when your generators are doing things that you probably didn't intend. I'll try to improve that. /Richard From gordon@REDACTED Sun Sep 27 19:17:12 2009 From: gordon@REDACTED (Gordon Guthrie) Date: Sun, 27 Sep 2009 18:17:12 +0100 Subject: [erlang-questions] eunit or common_test? In-Reply-To: <59da11980909270953y48aa79a5q9d3d43ec2fcf62d6@mail.gmail.com> References: <59da11980909270953y48aa79a5q9d3d43ec2fcf62d6@mail.gmail.com> Message-ID: Jeremy First approximation: * for functions without side-effects write EUnit unit tests as they will be definitive. * for functions/sub-systems/systems that have side effects use common test Gordon 2009/9/27 Jeremy Raymond : > Hello, > > I've been taking a look a both eunit and common test. Are there any > advantages one over the other, or does it really just come down to > preference on which to use? > > > Thanks, > > Jeremy > From freza@REDACTED Sun Sep 27 19:28:26 2009 From: freza@REDACTED (Jachym Holecek) Date: Sun, 27 Sep 2009 19:28:26 +0200 Subject: [erlang-questions] EUnit and Servers In-Reply-To: <8b9ee55b0909270904g18964525j8e6f6284c598c9b6@mail.gmail.com> References: <8b9ee55b0909270904g18964525j8e6f6284c598c9b6@mail.gmail.com> Message-ID: <20090927172826.GA24739@hanele> # Fred Hebert (MononcQc) 2009-09-27: > I have used EUnit before, but rarely did server testing with it. I decided > to give it a hand with some toy code of mine, but I'm getting errors I'm not > able to understand the cause of. I'm not willing to blame the tools (eunit) > before the tool (me), so I'm asking for help here: > > Here's a paste of the test module: > http://paste.lisp.org/display/87803 status_test_() -> {setup, fun() -> box:start_link(200) end, fun(_) -> box:stop() end, fun() -> {inorder, [?assert(close_enough(box:get_status(), get_val(box:init(200)))), ?assert(close_enough(begin box:get_air(100), box:get_status() end, get_val(box:init(100))))]} end}. I think that you need one of the following (in all the tests you pasted): (1) Make the test function have arity one (not zero) and use ?_assert() as Kevin points out. This is probably the right thing to do. (2) Keep the test function of arity zero, but change the body to simply the two ?assert()s executed in a row. This is probably less right. Now, (2) is equivalent to what you have now, execution-wise, which makes me suspect there's something wrong with either box:start_link/1 or box:stop/0. Wild speculation: does box:stop/0 guarantee the server is stone dead by the time it returns? Because if not, the box server could live a little while longer, making box:start_link/1 in the next test silently fail, and dying right afterwards, making box:get_status/0 explode. And one perhaps obvious note: making stop() into a synchronous call is not good enough -- the thing isn't dead unless you've seen corresponding 'EXIT' or 'DOWN' message. HTH, -- Jachym From mononcqc@REDACTED Sun Sep 27 19:46:01 2009 From: mononcqc@REDACTED (Fred Hebert) Date: Sun, 27 Sep 2009 13:46:01 -0400 Subject: [erlang-questions] EUnit and Servers In-Reply-To: <20090927172826.GA24739@hanele> References: <8b9ee55b0909270904g18964525j8e6f6284c598c9b6@mail.gmail.com> <20090927172826.GA24739@hanele> Message-ID: <8b9ee55b0909271046v6ac1ce99t9b62f7ffa046eae4@mail.gmail.com> On Sun, Sep 27, 2009 at 1:28 PM, Jachym Holecek wrote: > # Fred Hebert (MononcQc) 2009-09-27: > > I have used EUnit before, but rarely did server testing with it. I > decided > > to give it a hand with some toy code of mine, but I'm getting errors I'm > not > > able to understand the cause of. I'm not willing to blame the tools > (eunit) > > before the tool (me), so I'm asking for help here: > > > > Here's a paste of the test module: > > http://paste.lisp.org/display/87803 > > status_test_() -> > {setup, fun() -> box:start_link(200) end, fun(_) -> box:stop() end, > fun() -> > {inorder, > [?assert(close_enough(box:get_status(), > get_val(box:init(200)))), > ?assert(close_enough(begin > box:get_air(100), > box:get_status() > end, > get_val(box:init(100))))]} > end}. > > I think that you need one of the following (in all the tests you pasted): > > (1) Make the test function have arity one (not zero) and use ?_assert() > as Kevin points out. This is probably the right thing to do. > > (2) Keep the test function of arity zero, but change the body to simply > the two ?assert()s executed in a row. This is probably less right. > > Now, (2) is equivalent to what you have now, execution-wise, which makes > me suspect there's something wrong with either box:start_link/1 or > box:stop/0. > > Wild speculation: does box:stop/0 guarantee the server is stone dead by > the time it returns? Because if not, the box server could live a little > while longer, making box:start_link/1 in the next test silently fail, > and dying right afterwards, making box:get_status/0 explode. > > And one perhaps obvious note: making stop() into a synchronous call is > not good enough -- the thing isn't dead unless you've seen corresponding > 'EXIT' or 'DOWN' message. > > HTH, > -- Jachym > Ah, thank you Richard and Jachym. I switched to the ?_assert() macros, added an arity of one to the test function (fun(_) -> {inorder, [...]} end). I then revised the box gen_server: I changed stop/0 from doing a cast to a call and everything started working fine. Thanks for the help, much appreciated. From fritchie@REDACTED Sun Sep 27 22:40:01 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Sun, 27 Sep 2009 15:40:01 -0500 Subject: Big RAM Erlang stories/wisdom/anecdotes? Message-ID: <84869.1254084001@snookles.snookles.com> Greetings, all. If all goes according to plan, I'll soon be getting my hands on some 64GB RAM, 64-bit boxes to work with. Their Erlang apps will be using a large fraction of that RAM to store stuff in ETS ordered_set tables. Erlang process heap growth is much jumpier. As noted by Ulf Wiger and Erik Stenman on this list(*), an Erlang process's heap grows in steps based on the Fibonacci sequence. Looking at erl_gc.c, the sequence starts at 34(***). Actually, it stops being Fibonacci-based at index 22, 1346269 words. >From then, it's based on 20% more than the previous index. My Erlang apps will rarely have individual binaries larger than 4MB, and we try very hard to create really big process heaps or really big terms, e.g. list-of-ASCII-byte error messages created by the default error_logger, which we don't use. Does anyone have war stories about big RAM machines (more than 16GB, for instance) + Erlang where you wish that the heap steps were a bit smaller? Once you're dealing with 1GB or more, "merely 20% more" ends up being a pretty big number when talking about RAM allocation. Or wisdom of past development experiences? Or anecdotes? I'm pondering playing with heap sizing in a couple ways: 1. What if growth beyond index 46 (?) were smaller than 20%? 2. What if a erts_gc.c:minor_collection() didn't automatically bump up to the next heap size? Any idea of the comment below is still relevant today? /* * Allocate an old heap if we don't have one and if we'll need one. */ if (OLD_HEAP(p) == NULL && mature != 0) { Eterm* n_old; /* Note: We choose a larger heap size than strictly needed, * which seems to reduce the number of fullsweeps. * This improved Estone by more than 1200 estones on my computer * (Ultra Sparc 10). */ Hrm, I've never tried calculating an Estone. Maybe I should try. :-) -Scott (*) And perhaps by others, sorry if my grep'ing didn't find your postings. (**) When putting millions of things into an ETS table via something simple like: [ets:insert(Tab, {X, B1, B2, B3, B4}) || X <- lists:seq(1, 10*1024*1024)]. ... I see the VM's OS process size grow is small steps. Yay. B1, B2, B3, and B4 are 63 byte binaries, if anyone cares. Yes, I've considered avoiding ETS and using a C driver, like Berkeley DB or Tokyo Cabinet. I haven't seriously thought about it yet, but if they're more RAM-efficient by a factor of only 2, then that's only ... a factor of 2. {shrug} (***): Max heap size for 32-bit environments actually stops at index 53 or index 54, I forget which. Megabyte calculations are truncated, probably. Based on erl_gc.c:erts_init_gc(). index 0, heap size (words): 34 (0 MB @ 32bit, 0 MB @ 64bit) index 1, heap size (words): 55 (0 MB @ 32bit, 0 MB @ 64bit) index 2, heap size (words): 89 (0 MB @ 32bit, 0 MB @ 64bit) index 3, heap size (words): 144 (0 MB @ 32bit, 0 MB @ 64bit) index 4, heap size (words): 233 (0 MB @ 32bit, 0 MB @ 64bit) index 5, heap size (words): 377 (0 MB @ 32bit, 0 MB @ 64bit) index 6, heap size (words): 610 (0 MB @ 32bit, 0 MB @ 64bit) index 7, heap size (words): 987 (0 MB @ 32bit, 0 MB @ 64bit) index 8, heap size (words): 1597 (0 MB @ 32bit, 0 MB @ 64bit) index 9, heap size (words): 2584 (0 MB @ 32bit, 0 MB @ 64bit) index 10, heap size (words): 4181 (0 MB @ 32bit, 0 MB @ 64bit) index 11, heap size (words): 6765 (0 MB @ 32bit, 0 MB @ 64bit) index 12, heap size (words): 10946 (0 MB @ 32bit, 0 MB @ 64bit) index 13, heap size (words): 17711 (0 MB @ 32bit, 0 MB @ 64bit) index 14, heap size (words): 28657 (0 MB @ 32bit, 0 MB @ 64bit) index 15, heap size (words): 46368 (0 MB @ 32bit, 0 MB @ 64bit) index 16, heap size (words): 75025 (0 MB @ 32bit, 0 MB @ 64bit) index 17, heap size (words): 121393 (0 MB @ 32bit, 0 MB @ 64bit) index 18, heap size (words): 196418 (0 MB @ 32bit, 1 MB @ 64bit) index 19, heap size (words): 317811 (1 MB @ 32bit, 2 MB @ 64bit) index 20, heap size (words): 514229 (1 MB @ 32bit, 3 MB @ 64bit) index 21, heap size (words): 832040 (3 MB @ 32bit, 6 MB @ 64bit) index 22, heap size (words): 1346269 (5 MB @ 32bit, 10 MB @ 64bit) index 23, heap size (words): 1682835 (6 MB @ 32bit, 12 MB @ 64bit) index 24, heap size (words): 2103540 (8 MB @ 32bit, 16 MB @ 64bit) index 25, heap size (words): 2629425 (10 MB @ 32bit, 20 MB @ 64bit) index 26, heap size (words): 3286780 (12 MB @ 32bit, 25 MB @ 64bit) index 27, heap size (words): 4108475 (15 MB @ 32bit, 31 MB @ 64bit) index 28, heap size (words): 5135590 (19 MB @ 32bit, 39 MB @ 64bit) index 29, heap size (words): 6419485 (24 MB @ 32bit, 48 MB @ 64bit) index 30, heap size (words): 8024355 (30 MB @ 32bit, 61 MB @ 64bit) index 31, heap size (words): 10030440 (38 MB @ 32bit, 76 MB @ 64bit) index 32, heap size (words): 12538050 (47 MB @ 32bit, 95 MB @ 64bit) index 33, heap size (words): 15672560 (59 MB @ 32bit, 119 MB @ 64bit) index 34, heap size (words): 19590700 (74 MB @ 32bit, 149 MB @ 64bit) index 35, heap size (words): 24488375 (93 MB @ 32bit, 186 MB @ 64bit) index 36, heap size (words): 30610465 (116 MB @ 32bit, 233 MB @ 64bit) index 37, heap size (words): 38263080 (145 MB @ 32bit, 291 MB @ 64bit) index 38, heap size (words): 47828850 (182 MB @ 32bit, 364 MB @ 64bit) index 39, heap size (words): 59786060 (228 MB @ 32bit, 456 MB @ 64bit) index 40, heap size (words): 74732575 (285 MB @ 32bit, 570 MB @ 64bit) index 41, heap size (words): 93415715 (356 MB @ 32bit, 712 MB @ 64bit) index 42, heap size (words): 116769640 (445 MB @ 32bit, 890 MB @ 64bit) index 43, heap size (words): 145962050 (556 MB @ 32bit, 1113 MB @ 64bit) index 44, heap size (words): 182452560 (696 MB @ 32bit, 1392 MB @ 64bit) index 45, heap size (words): 228065700 (870 MB @ 32bit, 1740 MB @ 64bit) index 46, heap size (words): 285082125 (1087 MB @ 32bit, 2175 MB @ 64bit) index 47, heap size (words): 356352655 (1359 MB @ 32bit, 2718 MB @ 64bit) index 48, heap size (words): 445440815 (1699 MB @ 32bit, 3398 MB @ 64bit) index 49, heap size (words): 556801015 (2124 MB @ 32bit, 4248 MB @ 64bit) index 50, heap size (words): 696001265 (2655 MB @ 32bit, 5310 MB @ 64bit) index 51, heap size (words): 870001580 (3318 MB @ 32bit, 6637 MB @ 64bit) index 52, heap size (words): 1087501975 (4148 MB @ 32bit, 8296 MB @ 64bit) index 53, heap size (words): 1359377465 (5185 MB @ 32bit, 10371 MB @ 64bit) index 54, heap size (words): 1699221830 (6482 MB @ 32bit, 12964 MB @ 64bit) index 55, heap size (words): 2124027285 (8102 MB @ 32bit, 16205 MB @ 64bit) index 56, heap size (words): 2655034105 (10128 MB @ 32bit, 20256 MB @ 64bit) index 57, heap size (words): 3318792630 (12660 MB @ 32bit, 25320 MB @ 64bit) index 58, heap size (words): 4148490785 (15825 MB @ 32bit, 31650 MB @ 64bit) index 59, heap size (words): 5185613480 (19781 MB @ 32bit, 39563 MB @ 64bit) index 60, heap size (words): 6482016850 (24726 MB @ 32bit, 49453 MB @ 64bit) index 61, heap size (words): 8102521060 (30908 MB @ 32bit, 61817 MB @ 64bit) index 62, heap size (words): 10128151325 (38635 MB @ 32bit, 77271 MB @ 64bit) index 63, heap size (words): 12660189155 (48294 MB @ 32bit, 96589 MB @ 64bit) index 64, heap size (words): 15825236440 (60368 MB @ 32bit, 120736 MB @ 64bit) index 65, heap size (words): 19781545550 (75460 MB @ 32bit, 150921 MB @ 64bit) index 66, heap size (words): 24726931935 (94325 MB @ 32bit, 188651 MB @ 64bit) index 67, heap size (words): 30908664915 (117907 MB @ 32bit, 235814 MB @ 64bit) index 68, heap size (words): 38635831140 (147383 MB @ 32bit, 294767 MB @ 64bit) index 69, heap size (words): 48294788925 (184229 MB @ 32bit, 368459 MB @ 64bit) index 70, heap size (words): 60368486155 (230287 MB @ 32bit, 460574 MB @ 64bit) index 71, heap size (words): 75460607690 (287859 MB @ 32bit, 575718 MB @ 64bit) index 72, heap size (words): 94325759610 (359824 MB @ 32bit, 719648 MB @ 64bit) index 73, heap size (words): 117907199510 (449780 MB @ 32bit, 899560 MB @ 64bit) index 74, heap size (words): 147383999385 (562225 MB @ 32bit, 1124450 MB @ 64bit) index 75, heap size (words): 184229999230 (702781 MB @ 32bit, 1405563 MB @ 64bit) index 76, heap size (words): 230287499035 (878477 MB @ 32bit, 1756954 MB @ 64bit) index 77, heap size (words): 287859373790 (1098096 MB @ 32bit, 2196192 MB @ 64bit) index 78, heap size (words): 359824217235 (1372620 MB @ 32bit, 2745240 MB @ 64bit) index 79, heap size (words): 449780271540 (1715775 MB @ 32bit, 3431551 MB @ 64bit) index 80, heap size (words): 562225339425 (2144719 MB @ 32bit, 4289438 MB @ 64bit) index 81, heap size (words): 702781674280 (2680899 MB @ 32bit, 5361798 MB @ 64bit) index 82, heap size (words): 878477092850 (3351124 MB @ 32bit, 6702248 MB @ 64bit) index 83, heap size (words): 1098096366060 (4188905 MB @ 32bit, 8377810 MB @ 64bit) index 84, heap size (words): 1372620457575 (5236131 MB @ 32bit, 10472263 MB @ 64bit) index 85, heap size (words): 1715775571965 (6545164 MB @ 32bit, 13090328 MB @ 64bit) index 86, heap size (words): 2144719464955 (8181455 MB @ 32bit, 16362910 MB @ 64bit) index 87, heap size (words): 2680899331190 (10226819 MB @ 32bit, 20453638 MB @ 64bit) index 88, heap size (words): 3351124163985 (12783524 MB @ 32bit, 25567048 MB @ 64bit) index 89, heap size (words): 4188905204980 (15979405 MB @ 32bit, 31958810 MB @ 64bit) index 90, heap size (words): 5236131506225 (19974256 MB @ 32bit, 39948513 MB @ 64bit) index 91, heap size (words): 6545164382780 (24967820 MB @ 32bit, 49935641 MB @ 64bit) index 92, heap size (words): 8181455478475 (31209775 MB @ 32bit, 62419551 MB @ 64bit) index 93, heap size (words): 10226819348090 (39012219 MB @ 32bit, 78024439 MB @ 64bit) index 94, heap size (words): 12783524185110 (48765274 MB @ 32bit, 97530549 MB @ 64bit) index 95, heap size (words): 15979405231385 (60956593 MB @ 32bit, 121913186 MB @ 64bit) index 96, heap size (words): 19974256539230 (76195741 MB @ 32bit, 152391483 MB @ 64bit) index 97, heap size (words): 24967820674035 (95244677 MB @ 32bit, 190489354 MB @ 64bit) index 98, heap size (words): 31209775842540 (119055846 MB @ 32bit, 238111693 MB @ 64bit) index 99, heap size (words): 39012219803175 (148819808 MB @ 32bit, 297639616 MB @ 64bit) index 100, heap size (words): 48765274753965 (186024760 MB @ 32bit, 372049520 MB @ 64bit) index 101, heap size (words): 60956593442455 (232530950 MB @ 32bit, 465061900 MB @ 64bit) index 102, heap size (words): 76195741803065 (290663687 MB @ 32bit, 581327375 MB @ 64bit) index 103, heap size (words): 95244677253830 (363329609 MB @ 32bit, 726659219 MB @ 64bit) index 104, heap size (words): 119055846567285 (454162012 MB @ 32bit, 908324024 MB @ 64bit) index 105, heap size (words): 148819808209105 (567702515 MB @ 32bit, 1135405030 MB @ 64bit) index 106, heap size (words): 186024760261380 (709628144 MB @ 32bit, 1419256288 MB @ 64bit) index 107, heap size (words): 232530950326725 (887035180 MB @ 32bit, 1774070360 MB @ 64bit) index 108, heap size (words): 290663687908405 (1108793975 MB @ 32bit, 2217587950 MB @ 64bit) index 109, heap size (words): 363329609885505 (1385992469 MB @ 32bit, 2771984938 MB @ 64bit) index 110, heap size (words): 454162012356880 (1732490586 MB @ 32bit, 3464981173 MB @ 64bit) index 111, heap size (words): 567702515446100 (2165613233 MB @ 32bit, 4331226466 MB @ 64bit) index 112, heap size (words): 709628144307625 (2707016541 MB @ 32bit, 5414033083 MB @ 64bit) index 113, heap size (words): 887035180384530 (3383770677 MB @ 32bit, 6767541354 MB @ 64bit) index 114, heap size (words): 1108793975480660 (4229713346 MB @ 32bit, 8459426692 MB @ 64bit) index 115, heap size (words): 1385992469350825 (5287141683 MB @ 32bit, 10574283366 MB @ 64bit) index 116, heap size (words): 1732490586688530 (6608927103 MB @ 32bit, 13217854207 MB @ 64bit) index 117, heap size (words): 2165613233360660 (8261158879 MB @ 32bit, 16522317759 MB @ 64bit) index 118, heap size (words): 2707016541700825 (10326448599 MB @ 32bit, 20652897199 MB @ 64bit) index 119, heap size (words): 3383770677126030 (12908060749 MB @ 32bit, 25816121499 MB @ 64bit) index 120, heap size (words): 4229713346407535 (16135075936 MB @ 32bit, 32270151873 MB @ 64bit) index 121, heap size (words): 5287141683009415 (20168844921 MB @ 32bit, 40337689842 MB @ 64bit) index 122, heap size (words): 6608927103761765 (25211056151 MB @ 32bit, 50422112302 MB @ 64bit) index 123, heap size (words): 8261158879702205 (31513820189 MB @ 32bit, 63027640378 MB @ 64bit) index 124, heap size (words): 10326448599627755 (39392275236 MB @ 32bit, 78784550473 MB @ 64bit) index 125, heap size (words): 12908060749534690 (49240344045 MB @ 32bit, 98480688091 MB @ 64bit) index 126, heap size (words): 16135075936918360 (61550430057 MB @ 32bit, 123100860114 MB @ 64bit) index 127, heap size (words): 20168844921147950 (76938037571 MB @ 32bit, 153876075143 MB @ 64bit) index 128, heap size (words): 25211056151434935 (96172546964 MB @ 32bit, 192345093928 MB @ 64bit) index 129, heap size (words): 31513820189293665 (120215683705 MB @ 32bit, 240431367410 MB @ 64bit) index 130, heap size (words): 39392275236617080 (150269604631 MB @ 32bit, 300539209263 MB @ 64bit) index 131, heap size (words): 49240344045771350 (187837005789 MB @ 32bit, 375674011579 MB @ 64bit) index 132, heap size (words): 61550430057214185 (234796257237 MB @ 32bit, 469592514474 MB @ 64bit) index 133, heap size (words): 76938037571517730 (293495321546 MB @ 32bit, 586990643093 MB @ 64bit) index 134, heap size (words): 96172546964397160 (366869151933 MB @ 32bit, 733738303866 MB @ 64bit) index 135, heap size (words): 120215683705496450 (458586439916 MB @ 32bit, 917172879833 MB @ 64bit) index 136, heap size (words): 150269604631870560 (573233049895 MB @ 32bit, 1146466099791 MB @ 64bit) index 137, heap size (words): 187837005789838200 (716541312369 MB @ 32bit, 1433082624739 MB @ 64bit) index 138, heap size (words): 234796257237297750 (895676640462 MB @ 32bit, 1791353280924 MB @ 64bit) index 139, heap size (words): 293495321546622185 (1119595800577 MB @ 32bit, 2239191601155 MB @ 64bit) index 140, heap size (words): 366869151933277730 (1399494750722 MB @ 32bit, 2798989501444 MB @ 64bit) index 141, heap size (words): 458586439916597160 (1749368438402 MB @ 32bit, 3498736876805 MB @ 64bit) index 142, heap size (words): 573233049895746450 (2186710548003 MB @ 32bit, 4373421096006 MB @ 64bit) index 143, heap size (words): 716541312369683060 (2733388185003 MB @ 32bit, 5466776370007 MB @ 64bit) index 144, heap size (words): 895676640462103825 (3416735231254 MB @ 32bit, 6833470462509 MB @ 64bit) index 145, heap size (words): 1119595800577629780 (4270919039068 MB @ 32bit, 8541838078137 MB @ 64bit) index 146, heap size (words): 1399494750722037225 (5338648798835 MB @ 32bit, -6914888446744 MB @ 64bit) index 147, heap size (words): 1749368438402546530 (6673310998544 MB @ 32bit, -4245564047326 MB @ 64bit) index 148, heap size (words): 2186710548003183160 (8341638748181 MB @ 32bit, -908908548053 MB @ 64bit) index 149, heap size (words): 2733388185003978950 (-7165137609189 MB @ 32bit, 3261910826036 MB @ 64bit) index 150, heap size (words): 3416735231254973685 (-4558375500383 MB @ 32bit, 8475435043649 MB @ 64bit) index 151, heap size (words): 4270919039068717105 (-1299922864374 MB @ 32bit, -2599845728749 MB @ 64bit) index 152, heap size (words): 5338648798835896380 (2773142930635 MB @ 32bit, 5546285861270 MB @ 64bit) index 153, heap size (words): 6673310998544870475 (7864475174398 MB @ 32bit, -1863235695619 MB @ 64bit) From v@REDACTED Sun Sep 27 23:25:48 2009 From: v@REDACTED (Valentin Micic) Date: Sun, 27 Sep 2009 23:25:48 +0200 Subject: [erlang-questions] Big RAM Erlang stories/wisdom/anecdotes? In-Reply-To: <84869.1254084001@snookles.snookles.com> Message-ID: <200909272126.n8RLQVSw025251@mail.pharos-avantgard.com> > Does anyone have war stories about big RAM machines (more than 16GB, > for instance) + Erlang where you wish that the heap steps were a bit > smaller? One of our systems is using a few machines with 128 GB RAM each, and quite uneventfully so. In order to maximize memory used by ETS, we've used binaries that are up to 64 octets long, or rather, we're storing a simple two-tuple binaries, one is 20 octet key, and another holds everything else within 64 octets (also contains some room for growth). We had to do a few specialized term-to-binary-to-term conversions to support this, but CPU cost for this turned to be minimal. Why 64-octets? Well, during the testing we picked up that if one exceeds 64 octets, additional 64+change octets would have been allocated from heap without releasing 64 octets that have been already allocated for that entry by ETS. We've never had time to get to the bottom of this, so we just decided to use 64 bytes instead. Additional programming proved to be trivial. V/ From puzza007@REDACTED Mon Sep 28 00:35:33 2009 From: puzza007@REDACTED (Paul Oliver) Date: Sun, 27 Sep 2009 23:35:33 +0100 Subject: [erlang-questions] blowfish ECB implementation in erlang? In-Reply-To: References: <38ff4e6b0909231233i4435a436v170a8b1953b22c4e@mail.gmail.com> Message-ID: On Fri, Sep 25, 2009 at 1:19 PM, Paul Oliver wrote: > On Wed, Sep 23, 2009 at 8:33 PM, Larry Ogrodnek wrote: > >> Hi, I'm wondering if anyone has put together an implementation of blowfish >> ECB? >> >> thanks. >> > > Hi Larry, > > Check the following page for the patch that added cfb64. > > > http://www.nabble.com/Patch-to-add-Blowfish-cfb64-to-crypto-app-td24232164.html > > You should almost be able to copy this verbatim to add ecb. > > Cheers, > Paul. > Hi Larry, The attached patch should do ECB for you. It works for me on the test data I obtained from http://www.schneier.com/code/vectors.txt. puzza@REDACTED:~/Downloads/otp_src_R13B02-1$ /tmp/bin/erl Erlang R13B02 (erts-5.7.3) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.7.3 (abort with ^G) 1> %% FEDCBA9876543210 == Key = <<254,220,186,152,118,84,50,16>> 1> %% 0123456789ABCDEF == Clear = <<1,35,69,103,137,171,205,239>> 1> %% 0ACEAB0FC6A0A28D == Enc = <<10,206,171,15,198,160,162,141>> 1> 1> crypto:start(). ok 2> Key = <<254,220,186,152,118,84,50,16>>. <<254,220,186,152,118,84,50,16>> 3> Clear = <<1,35,69,103,137,171,205,239>>. <<1,35,69,103,137,171,205,239>> 4> Enc = <<10,206,171,15,198,160,162,141>>. <<10,206,171,15,198,160,162,141>> 5> Enc == crypto:blowfish_ecb_encrypt(Key, Clear). true 6> Clear == crypto:blowfish_ecb_decrypt(Key, Enc). true 7> I'm off on holiday tomorrow so if it doesn't work as it should I'll take another look in a couple of weeks. ;} If it works OK please reply and perhaps it can be picked up in a future OTP release. Cheers, Paul. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Blowfish-ECB.patch Type: application/octet-stream Size: 4318 bytes Desc: not available URL: From john@REDACTED Mon Sep 28 09:47:46 2009 From: john@REDACTED (john@REDACTED) Date: Mon, 28 Sep 2009 01:47:46 -0600 Subject: [erlang-questions] Erlang on smart phones In-Reply-To: <1254071525.8966.44.camel@gram.local> References: <1254071525.8966.44.camel@gram.local> Message-ID: <20090928014746.acfp1pgpwww0wsgo@box320.bluehost.com> Quoting Peter Sabaini : > Hi, > > there seems to be an effort to make Erlang/OTP run on iPhones -- are > there ports for any other smart phones, eg. Android-based ones? Several years ago, I ported Erlang onto a Nokia 770 Internet tablet, which meant simply (IIRC) cross-compiling for an ARM chip and making a Debian package. I didn't get much interest at the time, so I haven't bothered to do it for later models (like the N900) but I would guess it wouldn't be too hard to do such a thing again. Cheers, - johnk From francesco@REDACTED Mon Sep 28 11:27:32 2009 From: francesco@REDACTED (Francesco Cesarini (Erlang Training and Consulting)) Date: Mon, 28 Sep 2009 10:27:32 +0100 Subject: Interview with Joe Armstrong and Simon Peyton Jones Message-ID: <4AC08184.2020702@erlang-consulting.com> For those of you who do not follow InfoQ, I warmly recommend Sadek Drobi's interview with Joe Armstrong and Simon Peyton Jones recorded at the Erlang Factory in London this June. They have a candid fireside chat on concurrency, error handling, fault tolerance, the Universe and everything. It was recorded soon after Simon's keynote on Erlang and Haskell, two childhood friends. Entertaining and interesting at the same time. http://www.infoq.com/interviews/armstrong-peyton-jones-erlang-haskell Francesco -- http://www.erlang-consulting.com From mayamatakeshi@REDACTED Mon Sep 28 12:23:22 2009 From: mayamatakeshi@REDACTED (mayamatakeshi) Date: Mon, 28 Sep 2009 19:23:22 +0900 Subject: Avoiding case nesting Message-ID: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> Hello, newbie question: Suppose that I need to check if a certain string starts with one of several prefixes. So I write a checkprefix function that return the tail after the prefix if it is found. If the prefixes were known at compile time, I could use function pattern matching and have a very clean code for this, however, the prefixes are loaded from configuration file and I ended up writing something like this: case checkprefix(FirstPrefix, String) of {ok, Tail} -> do_this(Tail); _ -> case checkprefix(SecondPrefix, String) of {ok, Tail} -> ` do_that(Tail); _ -> case checkprefix(ThirdPrefix, String) of and so on... So, is there a way to avoid this cascading of case expressions? regards, takeshi From gleber.p@REDACTED Mon Sep 28 12:38:30 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 28 Sep 2009 12:38:30 +0200 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> Message-ID: <14f0e3620909280338j5f13aac6j7cd658f193d9d5e@mail.gmail.com> On Mon, Sep 28, 2009 at 12:23, mayamatakeshi wrote: > Hello, > newbie question: > Suppose that I need to check if a certain string starts with one of several > prefixes. > So I write a checkprefix function that return the tail after the prefix if > it is found. > If the prefixes were known at compile time, I could use function pattern > matching and have a very clean code for this, however, the prefixes are > loaded from configuration file and I ended up writing something like this: > > case checkprefix(FirstPrefix, String) of > {ok, Tail} -> > do_this(Tail); > _ -> > case checkprefix(SecondPrefix, String) of > {ok, Tail} -> > ` do_that(Tail); > _ -> > case checkprefix(ThirdPrefix, String) of > and so on... > > So, is there a way to avoid this cascading of case expressions? One could do this: Prefixes = [{"prefix1", fun do_this/1}, {"prefix2", fun do_that/1}, {"prefix3", fun do_something/1}], catch lists:foldl(fun({Prefix, Fun}, Tail) -> case checkprefix(Prefix, Tail) of {ok, NewTail} -> NewTail; _ -> throw(done) end end, String, Prefixes). P.S. Sorry for HTML formatting From kagato@REDACTED Mon Sep 28 12:38:59 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Mon, 28 Sep 2009 03:38:59 -0700 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> Message-ID: I believe that you can still use pattern matching. Something like: > case String of > [ FirstPrefix | Tail ] -> do_this(Tail); > [ SecondPrefix | Tail ] -> do_that(Tail); > ... > _ -> didnt_match > end. Although, if you have a variable number of prefixes, it may be better do this with recursion and a list of {Prefix,HandlerFunc}. Also, if you do have to work with nested cases, this article might help you make them easier to debug: > http://www.trapexit.org/Nested_Cases Good luck! On Sep 28, 2009, at 3:23 AM, mayamatakeshi wrote: > Hello, > newbie question: > Suppose that I need to check if a certain string starts with one of > several > prefixes. > So I write a checkprefix function that return the tail after the > prefix if > it is found. > If the prefixes were known at compile time, I could use function > pattern > matching and have a very clean code for this, however, the prefixes > are > loaded from configuration file and I ended up writing something like > this: > > case checkprefix(FirstPrefix, String) of > {ok, Tail} -> > do_this(Tail); > _ -> > case checkprefix(SecondPrefix, String) of > {ok, Tail} -> > ` do_that(Tail); > _ -> > case checkprefix(ThirdPrefix, String) of > and so on... > > So, is there a way to avoid this cascading of case expressions? > > regards, > takeshi -- Jayson Vantuyl kagato@REDACTED From cthulahoops@REDACTED Mon Sep 28 12:43:57 2009 From: cthulahoops@REDACTED (Adam Kelly) Date: Mon, 28 Sep 2009 11:43:57 +0100 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> Message-ID: <8d1798e90909280343p6c4454d7u5a3ff62be98842c1@mail.gmail.com> 2009/9/28 mayamatakeshi : > case checkprefix(FirstPrefix, String) of > ? ?{ok, Tail} -> > ? ? ? ?do_this(Tail); > ? ?_ -> > ? ? ? ?case checkprefix(SecondPrefix, String) of > ? ? ? ? ? ?{ok, Tail} -> > ` ? ? ? ? ? ? ? ?do_that(Tail); > ? ? ? ? ? ?_ -> > ? ? ? ? ? ? ? ?case checkprefix(ThirdPrefix, String) of > ? ? ? ? ? ? ? ? ? ?and so on... > > So, is there a way to avoid this cascading of case expressions? Something like: case checkprefixes(Prefixes) of {ok, Tail} -> do_this(Tail); {error, not_found} -> ... end. checkprefixes([], String) -> {error, not_found}; checkprefixes([H|T], String) -> case check_prefix(H, String) of {ok, Tail} -> {ok, Tail}; _ -> % I'd be specific in what I'm matching here. check_prefixes(T, String) end. Adam. From mayamatakeshi@REDACTED Mon Sep 28 13:50:21 2009 From: mayamatakeshi@REDACTED (mayamatakeshi) Date: Mon, 28 Sep 2009 20:50:21 +0900 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: <14f0e3620909280338j5f13aac6j7cd658f193d9d5e@mail.gmail.com> References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> <14f0e3620909280338j5f13aac6j7cd658f193d9d5e@mail.gmail.com> Message-ID: <15b9404e0909280450k49eedae4qd22a0bc5a86fbc50@mail.gmail.com> On Mon, Sep 28, 2009 at 7:38 PM, Gleb Peregud wrote: > On Mon, Sep 28, 2009 at 12:23, mayamatakeshi > wrote: > > So, is there a way to avoid this cascading of case expressions? > > One could do this: > > Prefixes = [{"prefix1", fun do_this/1}, > {"prefix2", fun do_that/1}, > {"prefix3", fun do_something/1}], > catch lists:foldl(fun({Prefix, Fun}, Tail) -> > case checkprefix(Prefix, Tail) of > {ok, NewTail} -> > NewTail; > _ -> > throw(done) > end > end, String, Prefixes). > This works. I'm a little concerned about the run time cost of throwing exceptions to force early processing termination (this will be called a lot) and this really seems somewhat abusing, but I will not discard this without checking performance. Thanks. From mayamatakeshi@REDACTED Mon Sep 28 14:08:42 2009 From: mayamatakeshi@REDACTED (mayamatakeshi) Date: Mon, 28 Sep 2009 21:08:42 +0900 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> Message-ID: <15b9404e0909280508j3d1de76ao2d36cf9f546072e9@mail.gmail.com> On Mon, Sep 28, 2009 at 7:38 PM, Jayson Vantuyl wrote: > I believe that you can still use pattern matching. Something like: > > case String of >> [ FirstPrefix | Tail ] -> do_this(Tail); >> [ SecondPrefix | Tail ] -> do_that(Tail); >> ... >> _ -> didnt_match >> end. >> > Hello, I tried like the above, but it will only work if the prefixes are single characters like $a. If the prefix is a string, then it will not match because it will become a list within a list. > Although, if you have a variable number of prefixes, it may be better do > this with recursion and a list of {Prefix,HandlerFunc}. > Yes, I can see this now, as all replies talk about it. > > Also, if you do have to work with nested cases, this article might help you > make them easier to debug: > > http://www.trapexit.org/Nested_Cases >> > > Thanks From ulf.wiger@REDACTED Mon Sep 28 14:10:03 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 28 Sep 2009 14:10:03 +0200 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> Message-ID: <4AC0A79B.2020707@erlang-consulting.com> I would probably use lists:dropwhile/2 in this situation, at least if you do not expect to take action in the case where it isn't a prefix. List = prefixes(), case lists:dropwhile(fun(P) -> not_prefix(P, String) end, List) of [] -> % no prefix ... [Prefix|_] -> Tail = lists:nthtail(length(Prefix), String), do_that(Tail) end. BR, Ulf W mayamatakeshi wrote: > Hello, > newbie question: > Suppose that I need to check if a certain string starts with one of several > prefixes. > So I write a checkprefix function that return the tail after the prefix if > it is found. > If the prefixes were known at compile time, I could use function pattern > matching and have a very clean code for this, however, the prefixes are > loaded from configuration file and I ended up writing something like this: > > case checkprefix(FirstPrefix, String) of > {ok, Tail} -> > do_this(Tail); > _ -> > case checkprefix(SecondPrefix, String) of > {ok, Tail} -> > ` do_that(Tail); > _ -> > case checkprefix(ThirdPrefix, String) of > and so on... > > So, is there a way to avoid this cascading of case expressions? > > regards, > takeshi > -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From mayamatakeshi@REDACTED Mon Sep 28 14:13:39 2009 From: mayamatakeshi@REDACTED (mayamatakeshi) Date: Mon, 28 Sep 2009 21:13:39 +0900 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: <8d1798e90909280343p6c4454d7u5a3ff62be98842c1@mail.gmail.com> References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> <8d1798e90909280343p6c4454d7u5a3ff62be98842c1@mail.gmail.com> Message-ID: <15b9404e0909280513mfe3ea4fj2919beb1b282624@mail.gmail.com> On Mon, Sep 28, 2009 at 7:43 PM, Adam Kelly wrote: > 2009/9/28 mayamatakeshi : > > case checkprefix(FirstPrefix, String) of > > {ok, Tail} -> > > do_this(Tail); > > _ -> > > case checkprefix(SecondPrefix, String) of > > {ok, Tail} -> > > ` do_that(Tail); > > _ -> > > case checkprefix(ThirdPrefix, String) of > > and so on... > > > > So, is there a way to avoid this cascading of case expressions? > > Something like: > > case checkprefixes(Prefixes) of > {ok, Tail} -> > do_this(Tail); > {error, not_found} -> > ... > end. > > checkprefixes([], String) -> > {error, not_found}; > checkprefixes([H|T], String) -> > case check_prefix(H, String) of > {ok, Tail} -> > {ok, Tail}; > _ -> % I'd be specific in what I'm matching here. > check_prefixes(T, String) > end. Thanks. This worked fine. From raimo+erlang-questions@REDACTED Mon Sep 28 14:46:12 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 28 Sep 2009 14:46:12 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname In-Reply-To: <200909271926489300445@its3.ch> References: <20090925133335.GB383@erix.ericsson.se> <200909271926489300445@its3.ch> Message-ID: <20090928124612.GA9042@erix.ericsson.se> First some words about the history of these dark corners of name resolution in Erlang, especially for Windows. In the beginning Erlang tried to mimic how Unix OS:es does name resolution by parsing files in the same way as the OS. This turned out to be very hard to get correct for all dialects, and impossible for Windows, so the native resolver was introduced and is now default for Erlang. It uses the native OS calls gethosbyname(), gethostbyaddr(), getipnodebyname() getipnodebyaddr(), getaddrinfo(), getnameinfo() et.al on different OS:es so Erlang will behave as any other application on the OS. The building blocks used in the old way are still there, inet_hosts, inet_res+inet_dns, and inet_db plus inet_config and more does a lot of strange things to both be backwards compatible and still useful... OK, then, let's move on. On Sun, Sep 27, 2009 at 07:26:49PM +0200, info wrote: > Please find a lot of questions in order to undertand the process for understanding how erlang is working with the DNS data. > > When erl is starting, the inet_db is built: correct ? Yes. > When erl stops, the inet_db is destroyed: correct ? Well of course; it is in memory only, and it is not written anywhere. > Q: from which information does erl build this db ? from the DNS data ? if yes, which file or database or other source ? For short name distribution mode and no distribution mode, almost nothing is initialized. inet:gethostname() is called, which calls the OS function gethostname(). If it returns a FQDN, the domain is taken from that. The lookup method is set to 'native'. Then a check is made; if the hostname can not be looked up through the 'native' lookup method: Add 'file' as a lookup method to use after 'native'. If the domain is not set; add 127.0.0.1 as an address of the hostname to the 'file' method. If the domain is set; try to look up the FQDN through the 'native' lookup method, if that succeedes; add the resolved IP address and aliases to the 'file' method. If the FQDN lookup fails, add 127.0.0.1 as above. This fixup of the 'host' method is done to ensure that the own hostname can be looked up. For long name distribution mode (windows), load configuration from windows registry: \hklm\system\CurrentControlSet\Services\TcpIp\Parameters\ varibles: DataBasePath, Domain, DhcpDomain, EnableDNS, NameServer and SearchList. The DataBasePath is used to read a Unix style "hosts" file. There are more variables that should be read, I guess, for example DhcpNameServer. After this the same check as for short name mode is done, to maybe patch in the 'file' lookup. > Q:which module in erlang does that ? inet_config through inet_db. > Q:is it possible to "follow" it ? Er,... what do you mean "follow". > Q:In my case inet_db doesn't contain the domain name: normal ? Since gethostname() on Windows apparently returns the short hostname, it is normal. > Q:With the native option, which default information I should find in this db ? As above. With the -sname or no name option, you should only get 'hostname' initialized. With the -name option you should get more, probably res_domain maybe res_search, if this data is in the registry. > You suggested this workaround: > inet_db:add_ns({127,0,0,1}). > inet_db:set_domain("my_domain.local"). > inet_db:add_search("my_domain.local"). > inet_db:set_lookup([file,dns]). > Q:Does it mean that this information are missing in my inet_db ? > ******* > Correct: > inet:gethostbyname("my_host"). > {ok,{hostent,"my_host",[],inet,4,[{127,0,0,1}]}} Well, since 'native' lookup is supposed to work, not really, but yes... for 'dns' lookup it is missing. In the best of worlds this could be read from the registry, but on neither of my (Windows 2003, Windows XP) machines is the nameserver in the registry... Oh, and you got the wrong answer there, the lookup order should probably be [dns,file] (my mistake). You got the loopback address from the 'file' method instead of the external IP address from the 'dns' method. Does inet:gethostbyname("www.google.com") work with this configuration? > > In fact I try to use erlang in order to find what is wrong in my DNS data. There are too much info on google about name resolution ... It appears DNS is working for you as both nslookup and a sufficiently configured inet_res can look up names through DNS. It is some other part of the name server in your OS configuration that fails. Have you looked at this: http://support.microsoft.com/kb/323380 esp. "How to Configure Forwarders". ...or... I just looked at my 2003 server, anid under Start -> My Computer (right click) -> Properties [Computer Name] there is a "Full computer name:" and a "Domain:" configured. Under Start -> Control Panel -> Network Connections -> Local Area Connection (right click) -> Status, Properties, Internet Protocol (TCP/IP), Properties, I have set which DNS servers to use. This might be what makes native lookup work for me. : -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From mayamatakeshi@REDACTED Mon Sep 28 15:03:49 2009 From: mayamatakeshi@REDACTED (mayamatakeshi) Date: Mon, 28 Sep 2009 22:03:49 +0900 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: <4AC0A79B.2020707@erlang-consulting.com> References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> <4AC0A79B.2020707@erlang-consulting.com> Message-ID: <15b9404e0909280603jc79ce89ob898763cc2ffbeeb@mail.gmail.com> On Mon, Sep 28, 2009 at 9:10 PM, Ulf Wiger wrote: > > I would probably use lists:dropwhile/2 in this > situation, at least if you do not expect to take action > in the case where it isn't a prefix. > > List = prefixes(), > case lists:dropwhile(fun(P) -> not_prefix(P, String) end, List) of > [] -> > % no prefix > ... > [Prefix|_] -> > Tail = lists:nthtail(length(Prefix), String), > do_that(Tail) > end. Yes, tested. This one works nice too. Thanks. From mayamatakeshi@REDACTED Mon Sep 28 15:11:07 2009 From: mayamatakeshi@REDACTED (mayamatakeshi) Date: Mon, 28 Sep 2009 22:11:07 +0900 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> <15b9404e0909280508j3d1de76ao2d36cf9f546072e9@mail.gmail.com> Message-ID: <15b9404e0909280611k24fbcabep156b7b3df66ff232@mail.gmail.com> On Mon, Sep 28, 2009 at 9:23 PM, Dale Harvey wrote: > > > 2009/9/28 mayamatakeshi > >> On Mon, Sep 28, 2009 at 7:38 PM, Jayson Vantuyl wrote: >> >> > I believe that you can still use pattern matching. Something like: >> > >> > case String of >> >> [ FirstPrefix | Tail ] -> do_this(Tail); >> >> [ SecondPrefix | Tail ] -> do_that(Tail); >> >> ... >> >> _ -> didnt_match >> >> end. >> >> >> > >> Hello, >> I tried like the above, but it will only work if the prefixes are single >> characters like $a. If the prefix is a string, then it will not match >> because it will become a list within a list. >> >> > you can use > > case String of > FirstPrefix ++ Tail -> do_this(Tail); > SecondPrefix ++ Tail -> do_that(Tail); > "a" ++ Tail -> do_that(Tail); > > to match lists > Thanks, however, in my case, the prefixes are variables set to strings and not literal strings, so when I try the above it will not even compile: erlc will complain with "illegal pattern". Only the last pattern ( "a" ++ Tail ) will pass compilation. From lars@REDACTED Mon Sep 28 15:18:58 2009 From: lars@REDACTED (Lars Thorsen) Date: Mon, 28 Sep 2009 15:18:58 +0200 Subject: [erlang-questions] Bugs in xmerl_sax_parser? In-Reply-To: <4ABA484C.8080105@lshift.net> References: <4ABA484C.8080105@lshift.net> Message-ID: <4AC0B7C2.9070601@erix.ericsson.se> Hi Tony, thank you for your information about the errors. 1. xmerl_sax_parse:stream/2 failed with {fatal_error,_, "Continuation function undefined, and more data needed",_,_} when no continuation function was defined even though it was a complete document as input. 2. The namespace URI supplied on unprefixed attributes in startElement tuples is the same as the URI for the default namespace. According to the standard the namespace for an unprefixed attribute should always has no value. I've fixed the errors and a patch can be fetched at "http://www.erlang.org/download/patches/". The files are named: otp_src_R13B02_OTP-8213_OTP-8214.patch otp_src_R13B02_OTP-8213_OTP-8214.readme Regards Lars Thorsen OTP Development Tony Garnock-Jones wrote: > Hi all, > > I've just been playing with xmerl_sax_parser and I think I've found two > bugs: > > 1. xmerl_sax_parser:file/2 and :stream/2 behave differently. Create a > file, "t.xml", containing > > xmlns:x='http://lshift.net/x' /> > > and then run the two expressions > > xmerl_sax_parser:file("t.xml", [{event_fun, fun (E,_,_) -> > io:format("~p~n", [E]), ok end}]). > xmerl_sax_parser:stream(" xmlns='http://lshift.net/d' xmlns:x='http://lshift.net/x' />", > [{event_fun, fun (E,_,_) -> io:format("~p~n", [E]), ok end}]). > > Note that they print the same event sequences, but the first results > in {ok,ok,<<>>}, while the second results in {fatal_error,...}! > > 2. The namespace URI supplied on unprefixed attributes in startElement > tuples is the same as the URI for the default namespace. According > to http://www.w3.org/TR/xml-names/#defaulting, "The namespace name > for an unprefixed attribute name always has no value." > > For example, running one of the expressions above causes the > following to be printed: > > {startElement,"http://lshift.net/d","elem", > {[],"elem"}, > [{"http://lshift.net/d",[],"attr","123"}, > {"http://lshift.net/x","x","attr","234"}]} > > While applications can work around this by ignoring the attribute's > namespace name in cases where the prefix == "", I wonder if it > wouldn't be better to supply "" for the namespace name for > unprefixed attributes, like this: > > {startElement,"http://lshift.net/d","elem", > {[],"elem"}, > [{[],[],"attr","123"}, > {"http://lshift.net/x","x","attr","234"}]} > > Regards, > Tony From mayamatakeshi@REDACTED Mon Sep 28 15:56:12 2009 From: mayamatakeshi@REDACTED (mayamatakeshi) Date: Mon, 28 Sep 2009 22:56:12 +0900 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: <4d08db370909280631u7984886dk2a9bc1d642aba318@mail.gmail.com> References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> <14f0e3620909280338j5f13aac6j7cd658f193d9d5e@mail.gmail.com> <15b9404e0909280450k49eedae4qd22a0bc5a86fbc50@mail.gmail.com> <4d08db370909280631u7984886dk2a9bc1d642aba318@mail.gmail.com> Message-ID: <15b9404e0909280656g2627538dw5964be34cae1a079@mail.gmail.com> On Mon, Sep 28, 2009 at 10:31 PM, Hynek Vychodil wrote: > > If you are concerned about the runtime cost you should think about some > sort of compiling. You can use parse transform to made it in compile time if > it is reasonable for you. It means you have fixed set of prefixes but you > want easy configuration. But it seems for me almost same as just one module > as some kind of "configuration" file. Another choice is on fly compilation > when you need change it programmatically in run time. When you realize how > big advantage hot code swapping in Erlang is, you can found it most > promising. > Oh. Even having read about hot code swapping and on the fly compilation, as a newbie I was unable to connect them with a simple thing like streamlining configuration. Yes, now I can see I can compile a module on the fly and the problem with function pattern matching goes away as what before was a variable (loaded using application:get_env) now becomes a literal value. Thanks a lot. From info@REDACTED Mon Sep 28 16:08:51 2009 From: info@REDACTED (info) Date: Mon, 28 Sep 2009 16:08:51 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname References: <20090925133335.GB383@erix.ericsson.se>, <200909271926489300445@its3.ch>, <20090928124612.GA9042@erix.ericsson.se> Message-ID: <200909281608509276841@its3.ch> Hi, A lot of data for me. Thank you. I must digest them, understand them and I will contact you again. Find also my answers. regards, john First some words about the history of these dark corners of name resolution in Erlang, especially for Windows. In the beginning Erlang tried to mimic how Unix OS:es does name resolution by parsing files in the same way as the OS. This turned out to be very hard to get correct for all dialects, and impossible for Windows, so the native resolver was introduced and is now default for Erlang. It uses the native OS calls gethosbyname(), gethostbyaddr(), getipnodebyname() getipnodebyaddr(), getaddrinfo(), getnameinfo() et.al on different OS:es so Erlang will behave as any other application on the OS. The building blocks used in the old way are still there, inet_hosts, inet_res+inet_dns, and inet_db plus inet_config and more does a lot of strange things to both be backwards compatible and still useful... OK, then, let's move on. On Sun, Sep 27, 2009 at 07:26:49PM +0200, info wrote: > Please find a lot of questions in order to undertand the process for understanding how erlang is working with the DNS data. > > When erl is starting, the inet_db is built: correct ? Yes. > When erl stops, the inet_db is destroyed: correct ? Well of course; it is in memory only, and it is not written anywhere. > Q: from which information does erl build this db ? from the DNS data ? if yes, which file or database or other source ? For short name distribution mode and no distribution mode, almost nothing is initialized. inet:gethostname() is called, which calls the OS function gethostname(). If it returns a FQDN, the domain is taken from that. The lookup method is set to 'native'. Then a check is made; if the hostname can not be looked up through the 'native' lookup method: Add 'file' as a lookup method to use after 'native'. If the domain is not set; add 127.0.0.1 as an address of the hostname to the 'file' method. If the domain is set; try to look up the FQDN through the 'native' lookup method, if that succeedes; add the resolved IP address and aliases to the 'file' method. If the FQDN lookup fails, add 127.0.0.1 as above. This fixup of the 'host' method is done to ensure that the own hostname can be looked up. For long name distribution mode (windows), load configuration from windows registry: \hklm\system\CurrentControlSet\Services\TcpIp\Parameters\ varibles: DataBasePath, Domain, DhcpDomain, EnableDNS, NameServer and SearchList. >>>>>> in my case, domain, nameserver and searchlist are empty. I tried to fill them but no change ! The DataBasePath is used to read a Unix style "hosts" file. There are more variables that should be read, I guess, for example DhcpNameServer. After this the same check as for short name mode is done, to maybe patch in the 'file' lookup. > Q:which module in erlang does that ? inet_config through inet_db. > Q:is it possible to "follow" it ? Er,... what do you mean "follow". >>>>> to trace. > Q:In my case inet_db doesn't contain the domain name: normal ? Since gethostname() on Windows apparently returns the short hostname, it is normal. > Q:With the native option, which default information I should find in this db ? As above. With the -sname or no name option, you should only get 'hostname' initialized. With the -name option you should get more, probably res_domain maybe res_search, if this data is in the registry. > You suggested this workaround: > inet_db:add_ns({127,0,0,1}). > inet_db:set_domain("my_domain.local"). > inet_db:add_search("my_domain.local"). > inet_db:set_lookup([file,dns]). > Q:Does it mean that this information are missing in my inet_db ? > ******* > Correct: > inet:gethostbyname("my_host"). > {ok,{hostent,"my_host",[],inet,4,[{127,0,0,1}]}} Well, since 'native' lookup is supposed to work, not really, but yes... for 'dns' lookup it is missing. In the best of worlds this could be read from the registry, but on neither of my (Windows 2003, Windows XP) machines is the nameserver in the registry... Oh, and you got the wrong answer there, the lookup order should probably be [dns,file] (my mistake). You got the loopback address from the 'file' method instead of the external IP address from the 'dns' method. >>>>> in my inet_db I have [native,file] by default Does inet:gethostbyname("www.google.com") work with this configuration? >>>>> yes > > In fact I try to use erlang in order to find what is wrong in my DNS data. There are too much info on google about name resolution ... It appears DNS is working for you as both nslookup and a sufficiently configured inet_res can look up names through DNS. It is some other part of the name server in your OS configuration that fails. Have you looked at this: http://support.microsoft.com/kb/323380 esp. "How to Configure Forwarders". ...or... I just looked at my 2003 server, anid under Start - > My Computer (right click) - > Properties [Computer Name] there is a "Full computer name:" and a "Domain:" configured. >>>>> Also for me: full computer name = my_host.my_domain.local Domain = my_domain.local Under Start - > Control Panel - > Network Connections - > Local Area Connection (right click) - > Status, Properties, Internet Protocol (TCP/IP), Properties, I have set which DNS servers to use. This might be what makes native lookup work for me. >>>>> I set 192.168.1.33 which is the ip address of my dns server and also the ip address of my host. Then it is not suffisant for working in native lookup ! : -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From hakan@REDACTED Mon Sep 28 16:17:02 2009 From: hakan@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Mon, 28 Sep 2009 16:17:02 +0200 Subject: [erlang-questions] Escript doc mismatch? In-Reply-To: <922d05850909250241m4ef67490rf8b3aaf74ed88116@mail.gmail.com> References: <922d05850909250241m4ef67490rf8b3aaf74ed88116@mail.gmail.com> Message-ID: <922d05850909280717n3d881a0dod2daacb1afa30687@mail.gmail.com> 2009/9/25 H?kan Mattsson : > On Fri, Sep 25, 2009 at 4:52 AM, Michael Richter wrote: >> Quoting from http://www.erlang.org/doc/man/escript.html: >> >> The header of an Erlang script differs from a normal Erlang module. The >>> first line is intended to be the interpreter line, which invokes escript. >>> However if you invoke escript like this >>> >>> $ *escript factorial 5* >>> >>> *the contents of the first line does not matter, but it cannot contain >>> Erlang code as it will be ignored. * The #! will not be mandatory on the first line in R13B03. > The documentation needs to be updated. The documentation will be updated in R13B03 to clarify pieces of the header syntax as well as documenting some features that have been experimental up to now, like the "-d" option. Try "escript -d factorial 5" to see what I am getting at. /H?kan > > If the escript has a header, the first line in the file must start with #!. > If you run the escript by explicitly invoking the escript executable, > e.g. like "escript factorial 5", the characters after #! on the first line > are ignored. > > As the header is optional you can in fact ?"execute" an .erl, .beam or > .zip file as long as escript can find the main/1 function. > > # cat factorial.erl > -module(factorial). > -export([main/1, fac/1]). > > main([String]) -> > ? ?try > ? ? ? ?N = list_to_integer(String), > ? ? ? ?F = fac(N), > ? ? ? ?io:format("factorial ~w = ~w\n", [N,F]) > ? ?catch > ? ? ? ?_:_ -> > ? ? ? ? ? ?usage() > ? ?end; > main(_) -> > ? ?usage(). > > usage() -> > ? ?io:format("usage: factorial integer\n"), > ? ?halt(1). > > fac(0) -> 1; > fac(N) -> N * fac(N-1). > > # escript factorial.erl 5 > factorial 5 = 120 > > # erlc factorial.erl > # escript factorial.beam 4 > factorial 4 = 24 > > # zip factorial.zip factorial.erl factorial.beam > ?adding: factorial.erl (deflated 44%) > ?adding: factorial.beam (deflated 29%) > # rm factorial.erl > # rm factorial.beam > # escript factorial.zip 7 > factorial 7 = 5040 > > /H?kan From roberto.ostinelli@REDACTED Mon Sep 28 16:39:34 2009 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Mon, 28 Sep 2009 16:39:34 +0200 Subject: replicated mnesia write operations Message-ID: dear all, let's say that i have a mnesia database on node 1, which has a disc_copy on node 2. +--------------+??? +--------------+ | db disc_copy |____| db disc_copy | | node 1?????? |??? | node 2?????? | +--------------+??? +--------------+ i would like the following: * WRITE operations on node 1 get propagated to (pretty standard behaviour): ? . disc and ram on node 1 ? . disc and ram on node 2 * WRITE operations on node 2 get ONLY propagated to: ? . ram on node 2 i.e. i would like that write operations on node 2 do not get propagated to node 1, nor to the disc of node 2. is this possible? r. From hakan@REDACTED Mon Sep 28 17:11:20 2009 From: hakan@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Mon, 28 Sep 2009 17:11:20 +0200 Subject: [erlang-questions] replicated mnesia write operations In-Reply-To: References: Message-ID: <922d05850909280811n2dbfe0c7j5781c93bb360fdd6@mail.gmail.com> On Mon, Sep 28, 2009 at 4:39 PM, Roberto Ostinelli wrote: > dear all, > > let's say that i have a mnesia database on node 1, which has a > disc_copy on node 2. > > +--------------+??? +--------------+ > | db disc_copy |____| db disc_copy | > | node 1?????? |??? | node 2?????? | > +--------------+??? +--------------+ > > i would like the following: > > * WRITE operations on node 1 get propagated to (pretty standard behaviour): > ? . disc and ram on node 1 > ? . disc and ram on node 2 > > * WRITE operations on node 2 get ONLY propagated to: > ? . ram on node 2 > > i.e. i would like that write operations on node 2 do not get > propagated to node 1, nor to the disc of node 2. Why do you want such a setup? > is this possible? Not with full transaction support on both nodes. You can however write directly in the ets tables on node 2. See mnesia:activity/4. /H?kan --- H?kan Mattsson (uabhams) Erlang/OTP, Ericsson AB From roberto@REDACTED Mon Sep 28 17:28:52 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 28 Sep 2009 17:28:52 +0200 Subject: [erlang-questions] replicated mnesia write operations In-Reply-To: <922d05850909280811n2dbfe0c7j5781c93bb360fdd6@mail.gmail.com> References: <922d05850909280811n2dbfe0c7j5781c93bb360fdd6@mail.gmail.com> Message-ID: > Why do you want such a setup? i'm working in a cloud environment where i need to keep configuration data centralized, and portions of it distributed evenly across computing machines. this data gets modified temporarily, and the output is sent otherwise [i.e not written in the db]. >> is this possible? > > Not with full transaction support on both nodes. > You can however write directly in the ets tables on node 2. > See mnesia:activity/4. thank you h?kan, will try this. r. From ok@REDACTED Tue Sep 29 04:25:11 2009 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 29 Sep 2009 15:25:11 +1300 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> Message-ID: Let's generalise slightly: We are given a sequence of strings [S1,...,Sn]. From this we want to generate a function F such that F(S) = {i,R} if i is the smallest integer for which S == Si ++ R, or F(S) = no_match otherwise. If we knew these strings at program construction time, we could just write f(S1 ++ R) -> {1,R}; ... f(Sn ++ R) -> {n,R}; f(_) -> no_match. This suggests one way to proceed: just write this text out to a file (with a suitable module header), compile it, load it, and use it. Thanks to hot loading, you can repeat the process whenever you want to change the sequence of strings. At the other extreme, seek(Target, [Prefix|Prefixes], I) -> case lists:prefix(Prefix, Target) of true -> {I, lists:nthtail(length(Prefix), Target)} ; false-> seek(Target, Prefixes, I+1) end; seek(_, [], _) -> no_match. searcher(Prefixes) -> fun (Target) -> seek(Target, Prefixes, 1) end. The first trades high preprocessing time for low run time; the second trades high run time for low preprocessing time. Somewhere in the middle you could construct a ternary search tree. In a language without run-time code generation, a ternary search tree would probably be the best way to go. From mayamatakeshi@REDACTED Tue Sep 29 05:37:04 2009 From: mayamatakeshi@REDACTED (mayamatakeshi) Date: Tue, 29 Sep 2009 12:37:04 +0900 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> Message-ID: <15b9404e0909282037pd59f437n8a4a0528147e8cb2@mail.gmail.com> On Tue, Sep 29, 2009 at 11:25 AM, Richard O'Keefe wrote: > Let's generalise slightly: > > We are given a sequence of strings [S1,...,Sn]. > From this we want to generate a function F > such that F(S) = {i,R} if i is the smallest integer > for which S == Si ++ R, or F(S) = no_match otherwise. > > If we knew these strings at program construction time, > we could just write > > f(S1 ++ R) -> {1,R}; > ... > f(Sn ++ R) -> {n,R}; > f(_) -> no_match. > > This suggests one way to proceed: just write this text > out to a file (with a suitable module header), compile > it, load it, and use it. Thanks to hot loading, you can > repeat the process whenever you want to change the sequence > of strings. > Yes, I realized that after reading Hynek's reply. > > At the other extreme, > > seek(Target, [Prefix|Prefixes], I) -> > case lists:prefix(Prefix, Target) > of true -> {I, lists:nthtail(length(Prefix), Target)} > ; false-> seek(Target, Prefixes, I+1) > end; > seek(_, [], _) -> > no_match. > > searcher(Prefixes) -> > fun (Target) -> seek(Target, Prefixes, 1) end. > > > The first trades high preprocessing time for low run time; > the second trades high run time for low preprocessing time. > > Somewhere in the middle you could construct a ternary search > tree. In a language without run-time code generation, a > ternary search tree would probably be the best way to go. > I will probably use on the fly compilation for a fixed set of prefixes. But I will also need to test against dynamic sets of prefixes fetched from a DB. So I will end up using the other end too. Thanks for pointing the ternary search tree. I read about It and it is a little over my head right now but I think I will eventually go with it if an approach like searcher/seek doesn't show acceptable performance. From volodymyrgl@REDACTED Tue Sep 29 05:41:18 2009 From: volodymyrgl@REDACTED (Volodymyr G. Lukiianyk) Date: Tue, 29 Sep 2009 06:41:18 +0300 Subject: Generating data of Erlang data-types in C Message-ID: <4AC181DE.7030002@gmail.com> Hi. Sorry if this question appears to be stupid. Well, my knowledge of Erlang is next to zero. Here is the problem description. There is some network application (E_app) written in Erlang which expects data received from network to be in form of an Erlang tuple with possible elements of atom, tuple and list types. This data needs to be formed by a program written in C (C_app). This problem, of course, could be solved with a help of a separate "translator" application written in Erlang, which would receive data from C_app in more C-friendly format and send it to E_app in Erlang-specific format. But full solution in C is much more preferable. Is there any way (some library?) to create Erlang-specific data from C application? Thanks in advance. From kagato@REDACTED Tue Sep 29 05:46:41 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Mon, 28 Sep 2009 20:46:41 -0700 Subject: [erlang-questions] Generating data of Erlang data-types in C In-Reply-To: <4AC181DE.7030002@gmail.com> References: <4AC181DE.7030002@gmail.com> Message-ID: This is actually pretty challenging. It is in no way a stupid question. You might try ei (to marshall the external term binaries) or ei_connect (to be a full-blown Erlang node). http://www.erlang.org/doc/apps/erl_interface/index.html On Sep 28, 2009, at 8:41 PM, Volodymyr G. Lukiianyk wrote: > Hi. > > Sorry if this question appears to be stupid. Well, my knowledge of > Erlang is next > to zero. > > Here is the problem description. There is some network application > (E_app) > written in Erlang which expects data received from network to be in > form of an > Erlang tuple with possible elements of atom, tuple and list types. > This data > needs to be formed by a program written in C (C_app). This problem, > of course, > could be solved with a help of a separate "translator" application > written in > Erlang, which would receive data from C_app in more C-friendly > format and send it > to E_app in Erlang-specific format. But full solution in C is much > more preferable. > > Is there any way (some library?) to create Erlang-specific data from > C application? > > Thanks in advance. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- Jayson Vantuyl kagato@REDACTED From andrew@REDACTED Tue Sep 29 06:22:13 2009 From: andrew@REDACTED (Andrew Thompson) Date: Tue, 29 Sep 2009 00:22:13 -0400 Subject: [erlang-questions] Generating data of Erlang data-types in C In-Reply-To: References: <4AC181DE.7030002@gmail.com> Message-ID: <20090929042212.GA22515@hijacked.us> On Mon, Sep 28, 2009 at 08:46:41PM -0700, Jayson Vantuyl wrote: > This is actually pretty challenging. It is in no way a stupid question. > > You might try ei (to marshall the external term binaries) or > ei_connect (to be a full-blown Erlang node). > > http://www.erlang.org/doc/apps/erl_interface/index.html > ei is certainly the way I do this, the documentation is a little scant and the library has some odd omissions but with a little creativity you can get a lot done. I suspect my mod_erlang_event for FreeSWITCH is one of the most complex ei applications that has the source available: http://wiki.freeswitch.org/wiki/Mod_erlang_event Andrew From max.lapshin@REDACTED Tue Sep 29 07:08:58 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 29 Sep 2009 09:08:58 +0400 Subject: [erlang-questions] Generating data of Erlang data-types in C In-Reply-To: <20090929042212.GA22515@hijacked.us> References: <4AC181DE.7030002@gmail.com> <20090929042212.GA22515@hijacked.us> Message-ID: erl_interface was unuseful for us because our application used epoll for work and I reimplemented it from scratch in ObjectiveC. It took about one day: erlang protocol is very, very simple. From raimo+erlang-questions@REDACTED Tue Sep 29 10:06:47 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 29 Sep 2009 10:06:47 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname In-Reply-To: <200909281821302082131@its3.ch> References: <20090928124612.GA9042@erix.ericsson.se> <200909281821302082131@its3.ch> Message-ID: <20090929080647.GA11676@erix.ericsson.se> On Mon, Sep 28, 2009 at 06:21:30PM +0200, info wrote: > Ok. > According to this information, I followed by hand the code (not easy ...) until: > inet_gethost_native:gethostbyname("my_host",inet) which returns {error,timeout} > > This function calls getit(1,1,"my_host") but I don't know how to simulate it : it calls Pid ! {{self(),Ref}, Req} > After I am lost ... there is do_open_port(get_poolsize(),get_extra_args()) ? Yep. You would probably not want to go further... inet_gethost_native is a server that keeps a port program written in C, erts-5.7.1\bin\inet_gethost.exe, source in erts/etc/common/inet_gethost.c for doing the actual calls to gethostbyname(). inet_gethost_native will compose a query, send it to the port program, the port program will dispatch it to one of its worker threads, wait for the answer, reply, and inet_gethost_native presents an answer. You can, if you run werl not from cygwin, activate a debug console. 1> inet_gethost_native:control({debug_level,99}). ok 2> inet_gethost_native:gethostbyname("foo"). {ok,{hostent,"foo.wherever.ericsson.com",[],inet,4, [{127,128,129,130}]}} Command 1> pops up a debug console, here is my content after 2>: Hej C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):debug_level = 99 C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Got data on index 0. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Got data from erlang. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):OPeration == 1. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Saved domainname . C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Created worker[2232] with fd 3822312 C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Saved domainname . C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[2232] (DEBUG):Worker got data on message que. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[2232] (DEBUG):Worker got request, op = 1, proto = 1, data = foo. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[2232] (DEBUG):Starting gethostbyname(foo) C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[2232] (DEBUG):gethostbyname OK C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Got data on index 1. Number 3 and 2 from the end says gethostbyname(foo) was called and returned OK. The data is then passed up to the waiting inet_gethost_native and converted to hostent tuple. Do you have DNS or other name resolving configured on some other interface in your machine that may cause lookup failure and timeout? You can also increase the timeout that inet_gethost_native uses to wait for an answer, to see if there is a very late answer after all. inet_db:set_timeout(15000) will give you a timeout of one minute instead of 8 seconds default. inet_gethost_native uses 4 times that value in milliseconds as timeout. > > > > > > First some words about the history of these dark corners > of name resolution in Erlang, especially for Windows. > > In the beginning Erlang tried to mimic how Unix OS:es > does name resolution by parsing files in the same way > as the OS. This turned out to be very hard to get > correct for all dialects, and impossible for Windows, > so the native resolver was introduced and is now > default for Erlang. It uses the native OS calls > gethosbyname(), gethostbyaddr(), getipnodebyname() > getipnodebyaddr(), getaddrinfo(), getnameinfo() et.al > on different OS:es so Erlang will behave as any other > application on the OS. > > The building blocks used in the old way are still there, > inet_hosts, inet_res+inet_dns, and inet_db plus inet_config > and more does a lot of strange things to both be backwards > compatible and still useful... > > OK, then, let's move on. > > > On Sun, Sep 27, 2009 at 07:26:49PM +0200, info wrote: > > Please find a lot of questions in order to undertand the process for understanding how erlang is working with the DNS data. > > > > When erl is starting, the inet_db is built: correct ? > > Yes. > > > When erl stops, the inet_db is destroyed: correct ? > > Well of course; it is in memory only, and it is not written anywhere. > > > Q: from which information does erl build this db ? from the DNS data ? if yes, which file or database or other source ? > > For short name distribution mode and no distribution mode, > almost nothing is initialized. inet:gethostname() is > called, which calls the OS function gethostname(). > If it returns a FQDN, the domain is taken from that. > The lookup method is set to 'native'. > Then a check is made; if the hostname can not be > looked up through the 'native' lookup method: > Add 'file' as a lookup method to use after 'native'. > If the domain is not set; add 127.0.0.1 as an > address of the hostname to the 'file' method. > If the domain is set; try to look up the FQDN > through the 'native' lookup method, if that succeedes; > add the resolved IP address and aliases to the 'file' method. > If the FQDN lookup fails, add 127.0.0.1 as above. > This fixup of the 'host' method is done to ensure > that the own hostname can be looked up. > > For long name distribution mode (windows), load configuration from > windows registry: > \hklm\system\CurrentControlSet\Services\TcpIp\Parameters\ > varibles: > DataBasePath, Domain, DhcpDomain, EnableDNS, NameServer and SearchList. > The DataBasePath is used to read a Unix style "hosts" file. > There are more variables that should be read, I guess, > for example DhcpNameServer. > After this the same check as for short name mode is done, > to maybe patch in the 'file' lookup. > > > Q:which module in erlang does that ? > > inet_config through inet_db. > > > Q:is it possible to "follow" it ? > > Er,... what do you mean "follow". > > > Q:In my case inet_db doesn't contain the domain name: normal ? > > Since gethostname() on Windows apparently returns the short > hostname, it is normal. > > > Q:With the native option, which default information I should find in this db ? > > As above. With the -sname or no name option, you should only > get 'hostname' initialized. With the -name option you should > get more, probably res_domain maybe res_search, if this data > is in the registry. > > > You suggested this workaround: > > inet_db:add_ns({127,0,0,1}). > > inet_db:set_domain("my_domain.local"). > > inet_db:add_search("my_domain.local"). > > inet_db:set_lookup([file,dns]). > > Q:Does it mean that this information are missing in my inet_db ? > > ******* > > Correct: > > inet:gethostbyname("my_host"). > > {ok,{hostent,"my_host",[],inet,4,[{127,0,0,1}]}} > > Well, since 'native' lookup is supposed to work, not really, > but yes... for 'dns' lookup it is missing. In the best of > worlds this could be read from the registry, but on neither > of my (Windows 2003, Windows XP) machines is the nameserver > in the registry... > > Oh, and you got the wrong answer there, the lookup order > should probably be [dns,file] (my mistake). You got the > loopback address from the 'file' method instead of > the external IP address from the 'dns' method. > > Does inet:gethostbyname("www.google.com") work with > this configuration? > > > > > In fact I try to use erlang in order to find what is wrong in my DNS data. There are too much info on google about name resolution ... > > It appears DNS is working for you as both nslookup and > a sufficiently configured inet_res can look up names > through DNS. It is some other part of the name server > in your OS configuration that fails. > > Have you looked at this: > http://support.microsoft.com/kb/323380 > esp. "How to Configure Forwarders". > > ...or... > > I just looked at my 2003 server, anid under Start - > My Computer > (right click) - > Properties [Computer Name] there is a > "Full computer name:" and a "Domain:" configured. > > Under Start - > Control Panel - > Network Connections - > > Local Area Connection (right click) - > Status, > Properties, Internet Protocol (TCP/IP), Properties, > I have set which DNS servers to use. This might > be what makes native lookup work for me. > > : > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From info@REDACTED Tue Sep 29 11:35:58 2009 From: info@REDACTED (info) Date: Tue, 29 Sep 2009 11:35:58 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname References: <20090928124612.GA9042@erix.ericsson.se>, <200909281821302082131@its3.ch>, <20090929080647.GA11676@erix.ericsson.se> Message-ID: <200909291135579948752@its3.ch> On Mon, Sep 28, 2009 at 06:21:30PM +0200, info wrote: > Ok. > According to this information, I followed by hand the code (not easy ...) until: > inet_gethost_native:gethostbyname("my_host",inet) which returns {error,timeout} > > This function calls getit(1,1,"my_host") but I don't know how to simulate it : it calls Pid ! {{self(),Ref}, Req} > After I am lost ... there is do_open_port(get_poolsize(),get_extra_args()) ? Yep. You would probably not want to go further... inet_gethost_native is a server that keeps a port program written in C, erts-5.7.1\bin\inet_gethost.exe, source in erts/etc/common/inet_gethost.c for doing the actual calls to gethostbyname(). inet_gethost_native will compose a query, send it to the port program, the port program will dispatch it to one of its worker threads, wait for the answer, reply, and inet_gethost_native presents an answer. You can, if you run werl not from cygwin, activate a debug console. 1 > inet_gethost_native:control({debug_level,99}). ok 2 > inet_gethost_native:gethostbyname("foo"). {ok,{hostent,"foo.wherever.ericsson.com",[],inet,4, [{127,128,129,130}]}} Command 1 > pops up a debug console, here is my content after 2 >: Hej C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):debug_level = 99 C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Got data on index 0. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Got data from erlang. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):OPeration == 1. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Saved domainname . C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Created worker[2232] with fd 3822312 C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Saved domainname . C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[2232] (DEBUG):Worker got data on message que. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[2232] (DEBUG):Worker got request, op = 1, proto = 1, data = foo. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[2232] (DEBUG):Starting gethostbyname(foo) C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[2232] (DEBUG):gethostbyname OK C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Got data on index 1. Number 3 and 2 from the end says gethostbyname(foo) was called and returned OK. The data is then passed up to the waiting inet_gethost_native and converted to hostent tuple. The last lines for me are: - got on index 0 - got data from erlang - OPeration == 3 - Stalled worker Do you have DNS or other name resolving configured on some other interface in your machine that may cause lookup failure and timeout? - My Hosts file has only one line: 127.0.0.1 localhost - Wins is disabled - Only one DNS server, one Domain controller (no replication) = same machine - AD could be damaged ? - I have another network connection but it is desactivated: could this one disturbed ? - What else ? You can also increase the timeout that inet_gethost_native uses to wait for an answer, to see if there is a very late answer after all. inet_db:set_timeout(15000) will give you a timeout of one minute instead of 8 seconds default. inet_gethost_native uses 4 times that value in milliseconds as timeout. - I tried it but {error,timeout} comes before the end ! > > > > > > First some words about the history of these dark corners > of name resolution in Erlang, especially for Windows. > > In the beginning Erlang tried to mimic how Unix OS:es > does name resolution by parsing files in the same way > as the OS. This turned out to be very hard to get > correct for all dialects, and impossible for Windows, > so the native resolver was introduced and is now > default for Erlang. It uses the native OS calls > gethosbyname(), gethostbyaddr(), getipnodebyname() > getipnodebyaddr(), getaddrinfo(), getnameinfo() et.al > on different OS:es so Erlang will behave as any other > application on the OS. > > The building blocks used in the old way are still there, > inet_hosts, inet_res+inet_dns, and inet_db plus inet_config > and more does a lot of strange things to both be backwards > compatible and still useful... > > OK, then, let's move on. > > > On Sun, Sep 27, 2009 at 07:26:49PM +0200, info wrote: > > Please find a lot of questions in order to undertand the process for understanding how erlang is working with the DNS data. > > > > When erl is starting, the inet_db is built: correct ? > > Yes. > > > When erl stops, the inet_db is destroyed: correct ? > > Well of course; it is in memory only, and it is not written anywhere. > > > Q: from which information does erl build this db ? from the DNS data ? if yes, which file or database or other source ? > > For short name distribution mode and no distribution mode, > almost nothing is initialized. inet:gethostname() is > called, which calls the OS function gethostname(). > If it returns a FQDN, the domain is taken from that. > The lookup method is set to 'native'. > Then a check is made; if the hostname can not be > looked up through the 'native' lookup method: > Add 'file' as a lookup method to use after 'native'. > If the domain is not set; add 127.0.0.1 as an > address of the hostname to the 'file' method. > If the domain is set; try to look up the FQDN > through the 'native' lookup method, if that succeedes; > add the resolved IP address and aliases to the 'file' method. > If the FQDN lookup fails, add 127.0.0.1 as above. > This fixup of the 'host' method is done to ensure > that the own hostname can be looked up. > > For long name distribution mode (windows), load configuration from > windows registry: > \hklm\system\CurrentControlSet\Services\TcpIp\Parameters\ > varibles: > DataBasePath, Domain, DhcpDomain, EnableDNS, NameServer and SearchList. > The DataBasePath is used to read a Unix style "hosts" file. > There are more variables that should be read, I guess, > for example DhcpNameServer. > After this the same check as for short name mode is done, > to maybe patch in the 'file' lookup. > > > Q:which module in erlang does that ? > > inet_config through inet_db. > > > Q:is it possible to "follow" it ? > > Er,... what do you mean "follow". > > > Q:In my case inet_db doesn't contain the domain name: normal ? > > Since gethostname() on Windows apparently returns the short > hostname, it is normal. > > > Q:With the native option, which default information I should find in this db ? > > As above. With the -sname or no name option, you should only > get 'hostname' initialized. With the -name option you should > get more, probably res_domain maybe res_search, if this data > is in the registry. > > > You suggested this workaround: > > inet_db:add_ns({127,0,0,1}). > > inet_db:set_domain("my_domain.local"). > > inet_db:add_search("my_domain.local"). > > inet_db:set_lookup([file,dns]). > > Q:Does it mean that this information are missing in my inet_db ? > > ******* > > Correct: > > inet:gethostbyname("my_host"). > > {ok,{hostent,"my_host",[],inet,4,[{127,0,0,1}]}} > > Well, since 'native' lookup is supposed to work, not really, > but yes... for 'dns' lookup it is missing. In the best of > worlds this could be read from the registry, but on neither > of my (Windows 2003, Windows XP) machines is the nameserver > in the registry... > > Oh, and you got the wrong answer there, the lookup order > should probably be [dns,file] (my mistake). You got the > loopback address from the 'file' method instead of > the external IP address from the 'dns' method. > > Does inet:gethostbyname("www.google.com") work with > this configuration? > > > > > In fact I try to use erlang in order to find what is wrong in my DNS data. There are too much info on google about name resolution ... > > It appears DNS is working for you as both nslookup and > a sufficiently configured inet_res can look up names > through DNS. It is some other part of the name server > in your OS configuration that fails. > > Have you looked at this: > http://support.microsoft.com/kb/323380 > esp. "How to Configure Forwarders". > > ...or... > > I just looked at my 2003 server, anid under Start - > My Computer > (right click) - > Properties [Computer Name] there is a > "Full computer name:" and a "Domain:" configured. > > Under Start - > Control Panel - > Network Connections - > > Local Area Connection (right click) - > Status, > Properties, Internet Protocol (TCP/IP), Properties, > I have set which DNS servers to use. This might > be what makes native lookup work for me. > > : > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From dmercer@REDACTED Tue Sep 29 15:14:35 2009 From: dmercer@REDACTED (David Mercer) Date: Tue, 29 Sep 2009 08:14:35 -0500 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> Message-ID: Richard O'Keefe wrote: > This suggests one way to proceed: just write this text > out to a file (with a suitable module header), compile > it, load it, and use it. Thanks to hot loading, you can > repeat the process whenever you want to change the sequence > of strings. Correct me if I am wrong, but can you not create "virtual" modules without actually writing the module file. I believe you can pass some Erlang terms to the compiler and have the new module compiled and loaded without the disk overhead. Never done this, but I thought it was possible... Cheers, David From john.erickson@REDACTED Tue Sep 29 18:11:57 2009 From: john.erickson@REDACTED (Erickson, John) Date: Tue, 29 Sep 2009 10:11:57 -0600 Subject: Memory Usage Message-ID: <3C44D46958109E408F5848593B05A1D8C5DAF8F5@rrsmsx505.amr.corp.intel.com> Hi, I am trying to store a large list of items and am a little surprised how much memory erlang is using. As a test, I tried creating a list of 10M integers and measuring memory usage. It takes around 600M. If I convert this list to a binary using term_to_binary, it is only about 50M, which is more like what I would expect. So I am wondering why there is 10x memory usage in the usual form? One other small oddity, it seems after calling garbage_collect I am using more memory. John plxc7436> erl erl Eshell V5.7.2 (abort with ^G) 1> memory(). memory(). [{total,6841320}, {processes,678056}, {processes_used,664224}, {system,6163264}, {atom,397393}, {atom_used,395331}, {binary,14192}, {code,3517670}, {ets,249376}] 2> L=lists:seq(1,10000000). L=lists:seq(1,10000000). [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22, 23,24,25,26,27,28,29|...] 3> memory(). memory(). [{total,611799440}, {processes,605491288}, {processes_used,605477456}, {system,6308152}, {atom,444017}, {atom_used,410654}, {binary,22912}, {code,3604955}, {ets,251952}] 4> garbage_collect(). garbage_collect(). true 5> memory(). memory(). [{total,950170384}, {processes,943869400}, {processes_used,943855568}, {system,6300984}, {atom,444017}, {atom_used,410654}, {binary,15744}, {code,3604955}, {ets,251952}] 6> size(term_to_binary(lists:seq(1,10000000))). size(term_to_binary(lists:seq(1,10000000))). 49999242 7> From andrew@REDACTED Tue Sep 29 18:48:32 2009 From: andrew@REDACTED (Andrew Thompson) Date: Tue, 29 Sep 2009 12:48:32 -0400 Subject: [erlang-questions] Memory Usage In-Reply-To: <3C44D46958109E408F5848593B05A1D8C5DAF8F5@rrsmsx505.amr.corp.intel.com> References: <3C44D46958109E408F5848593B05A1D8C5DAF8F5@rrsmsx505.amr.corp.intel.com> Message-ID: <20090929164832.GF22515@hijacked.us> On Tue, Sep 29, 2009 at 10:11:57AM -0600, Erickson, John wrote: > Hi, I am trying to store a large list of items and am a little surprised how much memory erlang is using. As a test, I tried creating a list of 10M integers and measuring memory usage. It takes around 600M. If I convert this list to a binary using term_to_binary, it is only about 50M, which is more like what I would expect. So I am wondering why there is 10x memory usage in the usual form? One other small oddity, it seems after calling garbage_collect I am using more memory. > Erlang lists incur about an 8x expansion in memory usage since each element is a pair of 32 bit integers. I just dealt with this myself in my gen_smtp erlang email server. I was having a terrible time reading in large emails (3mb or larger) because of the ballooning memory usage. I rewrote everything to use binaries wherever possible (had to write some functions for working with binary strings) but I've seen a huge reduction in the time and memory needed to handle large emails. So yeah, binaries are a much better solution for any larger data, they're just a little tricker to work with because the bitstring library proposed by EEP9 doesn't seem to exist. Andrew From ulf.wiger@REDACTED Tue Sep 29 19:09:35 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 29 Sep 2009 19:09:35 +0200 Subject: [erlang-questions] Memory Usage In-Reply-To: <20090929164832.GF22515@hijacked.us> References: <3C44D46958109E408F5848593B05A1D8C5DAF8F5@rrsmsx505.amr.corp.intel.com> <20090929164832.GF22515@hijacked.us> Message-ID: <4AC23F4F.6080608@erlang-consulting.com> Andrew Thompson wrote: > On Tue, Sep 29, 2009 at 10:11:57AM -0600, Erickson, John wrote: >> Hi, I am trying to store a large list of items and am a little >> surprised how much memory erlang is using. As a test, I tried >> creating a list of 10M integers and measuring memory usage. It >> takes around 600M. If I convert this list to a binary using >> term_to_binary, it is only about 50M, which is more like what I >> would expect. So I am wondering why there is 10x memory usage in >> the usual form? One other small oddity, it seems after calling >> garbage_collect I am using more memory. >> > > Erlang lists incur about an 8x expansion in memory usage since each > element is a pair of 32 bit integers. There is also a general problem of building large terms on the process heap. Garbage collection will kick in each time the heap fills up, and it will sweep for garbage to reclaim. Since there is no garbage, it will fail, and try a 'full sweep', which also checks long-lived data. Since that doesn't find any garbage either, the VM will resize the heap instead. Since the VM cannot foresee that the heap will continue to grow, it will repeat this pattern for each GC, ending up making several copies of the heap each time. For this reason, one should try to favor algorithms that execute in roughly constant space - e.g. fetching one or a few objects at a time from ETS, rather than pulling the entire set at once. One can also try to size the heap at spawn-time (using spawn_opt) to avoid GC whenever possible. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From rvirding@REDACTED Tue Sep 29 20:22:21 2009 From: rvirding@REDACTED (Robert Virding) Date: Tue, 29 Sep 2009 20:22:21 +0200 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> Message-ID: <3dbc6d1c0909291122y522a0857l8d5c27da6413baa2@mail.gmail.com> 2009/9/29 David Mercer > Richard O'Keefe wrote: > > > This suggests one way to proceed: just write this text > > out to a file (with a suitable module header), compile > > it, load it, and use it. Thanks to hot loading, you can > > repeat the process whenever you want to change the sequence > > of strings. > > Correct me if I am wrong, but can you not create "virtual" modules without > actually writing the module file. I believe you can pass some Erlang terms > to the compiler and have the new module compiled and loaded without the > disk > overhead. Never done this, but I thought it was possible... > Oh yes, it is definitely possible. You create the same datastructures as you would get reading in the file and then pass them to the compiler. Whether you do it this way or create a file really depends on what you want/need. Macros don't work if you create the terms and pass them directly to the compiler. Robert From mjtruog@REDACTED Tue Sep 29 20:48:46 2009 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 29 Sep 2009 11:48:46 -0700 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: <3dbc6d1c0909291122y522a0857l8d5c27da6413baa2@mail.gmail.com> References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> <3dbc6d1c0909291122y522a0857l8d5c27da6413baa2@mail.gmail.com> Message-ID: <4AC2568E.3090305@gmail.com> Robert Virding wrote: > 2009/9/29 David Mercer > > >> Richard O'Keefe wrote: >> >> >>> This suggests one way to proceed: just write this text >>> out to a file (with a suitable module header), compile >>> it, load it, and use it. Thanks to hot loading, you can >>> repeat the process whenever you want to change the sequence >>> of strings. >>> >> Correct me if I am wrong, but can you not create "virtual" modules without >> actually writing the module file. I believe you can pass some Erlang terms >> to the compiler and have the new module compiled and loaded without the >> disk >> overhead. Never done this, but I thought it was possible... >> >> > > Oh yes, it is definitely possible. You create the same datastructures as you > would get reading in the file and then pass them to the compiler. Whether > you do it this way or create a file really depends on what you want/need. > > Macros don't work if you create the terms and pass them directly to the > compiler. > > Robert > An example of this is here: http://github.com/JacobVorreuter/dynamic_compile/blob/master/src/dynamic_compile.erl - Michael From ok@REDACTED Wed Sep 30 06:18:59 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 30 Sep 2009 17:18:59 +1300 Subject: [erlang-questions] Avoiding case nesting In-Reply-To: References: <15b9404e0909280323x7ac6a423h1dad22073a28d535@mail.gmail.com> Message-ID: <3ADD2760-00A9-46BD-8883-EC1F267BA583@cs.otago.ac.nz> On Sep 30, 2009, at 2:14 AM, David Mercer wrote: > Richard O'Keefe wrote: > >> This suggests one way to proceed: just write this text >> out to a file (with a suitable module header), compile >> it, load it, and use it. Thanks to hot loading, you can >> repeat the process whenever you want to change the sequence >> of strings. > > Correct me if I am wrong, but can you not create "virtual" modules > without > actually writing the module file. My message was not intended to be exhaustive. The approach of writing an Erlang source file is not confined to Erlang: a software component in some other language could provide the strings to Erlang in _this_ form rather than some other form. From ngocdaothanh@REDACTED Wed Sep 30 10:02:29 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Wed, 30 Sep 2009 17:02:29 +0900 Subject: UTF8 and EDoc Message-ID: <5c493e530909300102x2d929ad5l9bb468d6eb390c55@mail.gmail.com> Greetings, 1. When I use EDoc library in Erlang R13B02-1 to create document with Japanese characters in the doc comments, there is error: edoc: error in doclet 'edoc_doclet': {'EXIT',{no_translation,[{io,put_chars,[<0.54.0>,unicode,<<60,33,68,79,67,84,89,80,...>>]},{edoc_lib,write_file,4},{edoc_doclet,source,9},{lists,foldl,3},{edoc_doclet,sources,5},{edoc_doclet,gen,6},{edoc_lib,run_plugin,5},{lists,foreach,2}]}}. ** exception exit: error in function edoc_lib:run_plugin/5 in call from lists:foreach/2 And the doc is not generated. 2. The cause of the problem is at io:put_chars of write_file in edoc_lib.erl. My dirty hack: write_file(Text, Dir, Name, Package) -> Dir1 = filename:join([Dir | packages:split(Package)]), File = filename:join(Dir1, Name), ok = filelib:ensure_dir(File), case file:open(File, [write]) of {ok, FD} -> %io:put_chars(FD, Text), <-- ERROR ok = file:close(FD), file:write_file(File, unicode:characters_to_binary(Text)); <-- HACK {error, R} -> R1 = file:format_error(R), report("could not write file '~s': ~s.", [File, R1]), exit(error) end. Could someone who takes care of EDoc look into the problem? Thank you, Ngoc From bgustavsson@REDACTED Wed Sep 30 15:49:10 2009 From: bgustavsson@REDACTED (Bjorn Gustavsson) Date: Wed, 30 Sep 2009 15:49:10 +0200 Subject: [erlang-questions] Memory Usage In-Reply-To: <3C44D46958109E408F5848593B05A1D8C5DAF8F5@rrsmsx505.amr.corp.intel.com> References: <3C44D46958109E408F5848593B05A1D8C5DAF8F5@rrsmsx505.amr.corp.intel.com> Message-ID: <6672d0160909300649v4871c253t37e6432cd429edad@mail.gmail.com> On Tue, Sep 29, 2009 at 6:11 PM, Erickson, John wrote: > Hi, I am trying to store a large list of items and am a little surprised how much memory erlang is using. As a test, I tried creating a list of 10M integers and measuring memory usage. It takes around 600M. If I convert this list to a binary using term_to_binary, it is only about 50M, which is more like what I would expect. So I am wondering why there is 10x memory usage in the usual form? One other small oddity, it seems after calling garbage_collect I am using more memory. It is not meaningful to use memory/0 to draw any conclusions about memory usage for an expression you have evaluated in the shell. The reason is that the shell internally uses multiple processes that communicate with each other, and those processes can be garbaged-collected at different times. To find out how many heap words a data structure requires, use one of (the undocumented) erts_debug:flat_size(Term) or erts_debug:size(Term) functions. (Unless the Term contains shared sub terms, they return the same result, but erts_debug:flat_size/1 is faster.) If you run the 32-bit emulator, multiply the value with 4 to obtain the memory consumption in bytes (otherwise, multiply with 8 for the 64-bit emulator). /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From info@REDACTED Wed Sep 30 16:38:47 2009 From: info@REDACTED (info) Date: Wed, 30 Sep 2009 16:38:47 +0200 Subject: [erlang-questions] {error,timeout} with gethostbyname References: <20090928124612.GA9042@erix.ericsson.se>, <200909281821302082131@its3.ch>, <20090929080647.GA11676@erix.ericsson.se> Message-ID: <200909301638470965513@its3.ch> On Mon, Sep 28, 2009 at 06:21:30PM +0200, info wrote: > Ok. > According to this information, I followed by hand the code (not easy ...) until: > inet_gethost_native:gethostbyname("my_host",inet) which returns {error,timeout} > > This function calls getit(1,1,"my_host") but I don't know how to simulate it : it calls Pid ! {{self(),Ref}, Req} > After I am lost ... there is do_open_port(get_poolsize(),get_extra_args()) ? Yep. You would probably not want to go further... inet_gethost_native is a server that keeps a port program written in C, erts-5.7.1\bin\inet_gethost.exe, source in erts/etc/common/inet_gethost.c for doing the actual calls to gethostbyname(). inet_gethost_native will compose a query, send it to the port program, the port program will dispatch it to one of its worker threads, wait for the answer, reply, and inet_gethost_native presents an answer. You can, if you run werl not from cygwin, activate a debug console. 1 > inet_gethost_native:control({debug_level,99}). ok 2 > inet_gethost_native:gethostbyname("foo"). {ok,{hostent,"foo.wherever.ericsson.com",[],inet,4, [{127,128,129,130}]}} Command 1 > pops up a debug console, here is my content after 2 >: Hej C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):debug_level = 99 C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Got data on index 0. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Got data from erlang. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):OPeration == 1. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Saved domainname . C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Created worker[2232] with fd 3822312 C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Saved domainname . C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[2232] (DEBUG):Worker got data on message que. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[2232] (DEBUG):Worker got request, op = 1, proto = 1, data = foo. C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[2232] (DEBUG):Starting gethostbyname(foo) C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[2232] (DEBUG):gethostbyname OK C:\PROGRA~1\ERL57~1.1\ERTS-5~1.1\bin\inet_gethost.exe[452] (DEBUG):Got data on index 1. Number 3 and 2 from the end says gethostbyname(foo) was called and returned OK. The data is then passed up to the waiting inet_gethost_native and converted to hostent tuple. The last lines for me are: - got on index 0 - got data from erlang - OPeration == 3 - Stalled worker Do you have DNS or other name resolving configured on some other interface in your machine that may cause lookup failure and timeout? - My Hosts file has only one line: 127.0.0.1 localhost - Wins is disabled - Only one DNS server, one Domain controller (no replication) = same machine - AD could be damaged ? - I have another network connection but it is desactivated: could this one disturbed ? - What else ? You can also increase the timeout that inet_gethost_native uses to wait for an answer, to see if there is a very late answer after all. inet_db:set_timeout(15000) will give you a timeout of one minute instead of 8 seconds default. inet_gethost_native uses 4 times that value in milliseconds as timeout. - I tried it but {error,timeout} comes before the end ! > > > > > > First some words about the history of these dark corners > of name resolution in Erlang, especially for Windows. > > In the beginning Erlang tried to mimic how Unix OS:es > does name resolution by parsing files in the same way > as the OS. This turned out to be very hard to get > correct for all dialects, and impossible for Windows, > so the native resolver was introduced and is now > default for Erlang. It uses the native OS calls > gethosbyname(), gethostbyaddr(), getipnodebyname() > getipnodebyaddr(), getaddrinfo(), getnameinfo() et.al > on different OS:es so Erlang will behave as any other > application on the OS. > > The building blocks used in the old way are still there, > inet_hosts, inet_res+inet_dns, and inet_db plus inet_config > and more does a lot of strange things to both be backwards > compatible and still useful... > > OK, then, let's move on. > > > On Sun, Sep 27, 2009 at 07:26:49PM +0200, info wrote: > > Please find a lot of questions in order to undertand the process for understanding how erlang is working with the DNS data. > > > > When erl is starting, the inet_db is built: correct ? > > Yes. > > > When erl stops, the inet_db is destroyed: correct ? > > Well of course; it is in memory only, and it is not written anywhere. > > > Q: from which information does erl build this db ? from the DNS data ? if yes, which file or database or other source ? > > For short name distribution mode and no distribution mode, > almost nothing is initialized. inet:gethostname() is > called, which calls the OS function gethostname(). > If it returns a FQDN, the domain is taken from that. > The lookup method is set to 'native'. > Then a check is made; if the hostname can not be > looked up through the 'native' lookup method: > Add 'file' as a lookup method to use after 'native'. > If the domain is not set; add 127.0.0.1 as an > address of the hostname to the 'file' method. > If the domain is set; try to look up the FQDN > through the 'native' lookup method, if that succeedes; > add the resolved IP address and aliases to the 'file' method. > If the FQDN lookup fails, add 127.0.0.1 as above. > This fixup of the 'host' method is done to ensure > that the own hostname can be looked up. > > For long name distribution mode (windows), load configuration from > windows registry: > \hklm\system\CurrentControlSet\Services\TcpIp\Parameters\ > varibles: > DataBasePath, Domain, DhcpDomain, EnableDNS, NameServer and SearchList. > The DataBasePath is used to read a Unix style "hosts" file. > There are more variables that should be read, I guess, > for example DhcpNameServer. > After this the same check as for short name mode is done, > to maybe patch in the 'file' lookup. > > > Q:which module in erlang does that ? > > inet_config through inet_db. > > > Q:is it possible to "follow" it ? > > Er,... what do you mean "follow". > > > Q:In my case inet_db doesn't contain the domain name: normal ? > > Since gethostname() on Windows apparently returns the short > hostname, it is normal. > > > Q:With the native option, which default information I should find in this db ? > > As above. With the -sname or no name option, you should only > get 'hostname' initialized. With the -name option you should > get more, probably res_domain maybe res_search, if this data > is in the registry. > > > You suggested this workaround: > > inet_db:add_ns({127,0,0,1}). > > inet_db:set_domain("my_domain.local"). > > inet_db:add_search("my_domain.local"). > > inet_db:set_lookup([file,dns]). > > Q:Does it mean that this information are missing in my inet_db ? > > ******* > > Correct: > > inet:gethostbyname("my_host"). > > {ok,{hostent,"my_host",[],inet,4,[{127,0,0,1}]}} > > Well, since 'native' lookup is supposed to work, not really, > but yes... for 'dns' lookup it is missing. In the best of > worlds this could be read from the registry, but on neither > of my (Windows 2003, Windows XP) machines is the nameserver > in the registry... > > Oh, and you got the wrong answer there, the lookup order > should probably be [dns,file] (my mistake). You got the > loopback address from the 'file' method instead of > the external IP address from the 'dns' method. > > Does inet:gethostbyname("www.google.com") work with > this configuration? > > > > > In fact I try to use erlang in order to find what is wrong in my DNS data. There are too much info on google about name resolution ... > > It appears DNS is working for you as both nslookup and > a sufficiently configured inet_res can look up names > through DNS. It is some other part of the name server > in your OS configuration that fails. > > Have you looked at this: > http://support.microsoft.com/kb/323380 > esp. "How to Configure Forwarders". > > ...or... > > I just looked at my 2003 server, anid under Start - > My Computer > (right click) - > Properties [Computer Name] there is a > "Full computer name:" and a "Domain:" configured. > > Under Start - > Control Panel - > Network Connections - > > Local Area Connection (right click) - > Status, > Properties, Internet Protocol (TCP/IP), Properties, > I have set which DNS servers to use. This might > be what makes native lookup work for me. > > : > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From daveb@REDACTED Wed Sep 30 21:18:26 2009 From: daveb@REDACTED (Dave Bryson) Date: Wed, 30 Sep 2009 14:18:26 -0500 Subject: OS X 10.5 build without WxWidgets Message-ID: <18569D8E-0920-40F8-A005-A0005B566B05@miceda.org> I apologize in advanced if this has already been discussed... I'm trying to build otp_13B02-1 on OS X 10.5.8 and the build is failing on a few wxwidget errors: /usr/include/wx-2.8/wx/fontdlg.h: In constructor ?wxFontDialogBase::wxFontDialogBase(wxWindow*, const wxFontData*)?: /usr/include/wx-2.8/wx/fontdlg.h:67: warning: ?__base_ctor ? is deprecated (declared at /usr/include/wx-2.8/wx/fontdlg.h:66) gen/wxe_events.cpp: In function ?void initEventTable()?: gen/wxe_events.cpp:277: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_DOWN? was not declared in this scope gen/wxe_events.cpp:278: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_UP? was not declared in this scope gen/wxe_events.cpp:279: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_DOWN? was not declared in this scope gen/wxe_events.cpp:280: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_UP? was not declared in this scope gen/wxe_events.cpp:281: error: ?wxEVT_COMMAND_AUINOTEBOOK_PAGE_CLOSED? was not declared in this scope gen/wxe_events.cpp:282: error: ?wxEVT_COMMAND_AUINOTEBOOK_DRAG_DONE? was not declared in this scope gen/wxe_events.cpp:283: error: ?wxEVT_COMMAND_AUINOTEBOOK_BG_DCLICK? was not declared in this scope make[4]: *** [i386-apple-darwin9.8.0/wxe_events.o] Error 1 make[3]: *** [release] Error 2 make[2]: *** [release] Error 2 make[1]: *** [release] Error 2 make: *** [install.libs] Error 2 Is it possible to build WITHOUT wxwidget support? Thanks! Dave From rapsey@REDACTED Wed Sep 30 21:39:13 2009 From: rapsey@REDACTED (Rapsey) Date: Wed, 30 Sep 2009 21:39:13 +0200 Subject: [erlang-questions] OS X 10.5 build without WxWidgets In-Reply-To: <18569D8E-0920-40F8-A005-A0005B566B05@miceda.org> References: <18569D8E-0920-40F8-A005-A0005B566B05@miceda.org> Message-ID: <97619b170909301239v1490e713g4c1aa73b39b7564@mail.gmail.com> build erlang like this: ./configure ...your params touch lib/wx/SKIP make Sergej On Wed, Sep 30, 2009 at 9:18 PM, Dave Bryson wrote: > I apologize in advanced if this has already been discussed... > > I'm trying to build otp_13B02-1 on OS X 10.5.8 and the build is failing on > a few wxwidget errors: > > /usr/include/wx-2.8/wx/fontdlg.h: In constructor > ?wxFontDialogBase::wxFontDialogBase(wxWindow*, const wxFontData*)?: > /usr/include/wx-2.8/wx/fontdlg.h:67: warning: ?__base_ctor ? is deprecated > (declared at /usr/include/wx-2.8/wx/fontdlg.h:66) > gen/wxe_events.cpp: In function ?void initEventTable()?: > gen/wxe_events.cpp:277: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_DOWN? > was not declared in this scope > gen/wxe_events.cpp:278: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_UP? > was not declared in this scope > gen/wxe_events.cpp:279: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_DOWN? > was not declared in this scope > gen/wxe_events.cpp:280: error: ?wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_UP? was > not declared in this scope > gen/wxe_events.cpp:281: error: ?wxEVT_COMMAND_AUINOTEBOOK_PAGE_CLOSED? was > not declared in this scope > gen/wxe_events.cpp:282: error: ?wxEVT_COMMAND_AUINOTEBOOK_DRAG_DONE? was > not declared in this scope > gen/wxe_events.cpp:283: error: ?wxEVT_COMMAND_AUINOTEBOOK_BG_DCLICK? was > not declared in this scope > make[4]: *** [i386-apple-darwin9.8.0/wxe_events.o] Error 1 > make[3]: *** [release] Error 2 > make[2]: *** [release] Error 2 > make[1]: *** [release] Error 2 > make: *** [install.libs] Error 2 > > > Is it possible to build WITHOUT wxwidget support? > > > Thanks! > Dave > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > >