From tty@REDACTED Fri Jul 1 05:48:04 2005 From: tty@REDACTED (tty@REDACTED) Date: Thu, 30 Jun 2005 23:48:04 -0400 Subject: simple_one_for_one worker Message-ID: Hello, I have a simple_one_for_one supervisor. Its workers are meant to perform one task/transaction then terminate. How should this worker be defined ? Using gen_server seem to be the easiest but I'm confuse about terminating the worker. Thanks tee From eudesh@REDACTED Fri Jul 1 06:51:01 2005 From: eudesh@REDACTED (Eranga Udesh) Date: Fri, 1 Jul 2005 10:51:01 +0600 Subject: Distributed Erlang embeded startup problem In-Reply-To: References: Message-ID: Hi, I have created a target system. It's a distributed system, having 2 optional nodes, if one failes application failover to the other node. If the high priority node comes alive it takes over. Attached files are, sys.config - configuration file in one node. The other node's file just have the name of this node in "sync_nodes_optional" esme.system - Linux init file The init file is located at /etc/rc.d/init.d and in linked to /etc/rc.d/rc3.d/S91esme.system When I ran init files manually, the system starts well. When a node fails, it fall backs and everything works smoothly. But when I restart both systems, once they come up, I see both are running the same application. When I ran the nodes() in one node, I can see that it has connected to other node too. What puzzeling me is, why it runs correctly when I ran manually and why it doesn't in a system reboot. Please help. Thanks in advance, - Eranga -------------- next part -------------- %% This file is identified via the erl command line option -config File. %% Note that File should have no extension, e.g. %% erl -config .../sys (if this file is called sys.config) %% %% In this file, you can redefine application environment variables. %% This way, you don't have to modify the .app files of e.g. OTP applications. %% %% For this example, we set the kernel environment variables for %% the distributed application controller. In order to run this %% example on your machine, you must change the node names below. %% See erl -man application for a thorough description of what is %% possible to configure. [{kernel, [{distributed, [{omni_esme, 5000, [omni_esme@REDACTED, omni_esme@REDACTED]}]}, {start_dist_ac, true}, {sync_nodes_mandatory, []}, {sync_nodes_optional, [omni_esme@REDACTED]}, {sync_nodes_timeout, 5000}]}]. -------------- next part -------------- #!/bin/sh # # File name: S75otp.system # Purpose: Automatically starts Erlang and applications when the # system starts # Author: # Resides in: /etc/rc3.d/init.d # MYNAME=`basename $0` ALLARGS=$* ROOTDIR=/usr2/omni_esme NODE_NAME=omni_esme MNESIA_DIR=$ROOTDIR/db PIPEDIR=$ROOTDIR/pipe/ LOGDIR=$ROOTDIR/logs RELEASEDIR=$ROOTDIR/system COOKIE=wavenetlk ERL=$RELEASEDIR/bin/erl ERL_TOP=$RELEASEDIR RUN_USER=root NUM_THREADS=4 RUN_ERL_LOG_GENERATIONS=5 RUN_ERL_LOG_MAXSIZE=10000000 HEART_COMMAND="/etc/rc.d/init.d/$MYNAME $ALLARGS" HEART_BEAT_TIMEOUT=30 HEART_BEAT_BOOT_DELAY=10 TERM=sun export RUN_ERL_LOG_GENERATIONS RUN_ERL_LOG_MAXSIZE HEART_COMMAND HEART_BEAT_TIMEOUT HEART_BEAT_BOOT_DELAY TERM ERL_TOP if [ ! -d /usr/bin ] then # /usr not mounted exit fi if test ! -d $RELEASEDIR then echo "Release directory does not exist ($RELEASEDIR)" exit fi if test ! -d $PIPEDIR then echo "Pipe directory does not exist." mkdir $PIPEDIR else echo "Pipe directory already exists." fi if test ! -d $LOGDIR then echo "Log directory does not exist. Create a new one." mkdir $LOGDIR else echo "Log directory already exists." fi if [ -z "$RELDIR" ] then RELDIR=$RELEASEDIR/releases fi START_ERL_DATA=$RELDIR/start_erl.data killproc() { # kill the named process(es) echo "Killing $1." pid=`pidof $1` [ "$pid" != "" ] && kill $pid } # Start/stop processes required for Erlang case $1 in start) echo "Starting ISUP STP" echo $RELEASEDIR/bin/run_erl -daemon $PIPEDIR $LOGDIR "$RELEASEDIR/bin/start_erl $RELEASEDIR $RELDIR $START_ERL_DATA \ -heart -sname $NODE_NAME -mnesia dir '\"$MNESIA_DIR\"' +A$NUM_THREADS -setcookie $COOKIE". $RELEASEDIR/bin/run_erl -daemon $PIPEDIR $LOGDIR "$RELEASEDIR/bin/start_erl $RELEASEDIR $RELDIR $START_ERL_DATA \ -heart -sname $NODE_NAME -mnesia dir '\"$MNESIA_DIR\"' +A$NUM_THREADS -setcookie $COOKIE" ;; debug) echo "Starting ISUP STP in debug mode" $RELEASEDIR/bin/start_erl $RELEASEDIR $RELDIR $START_ERL_DATA \ -sname $NODE_NAME -mnesia dir '"$MNESIA_DIR"' +A$NUM_THREADS -setcookie $COOKIE ;; stop) echo "Stopping ISUP STP" #$ERL -noshell -sname isupstp_stopper -pa $PA \ # -s web_server stop webserver001@$HOSTNAME killproc heart ;; *) echo "Usage: $0 {start|stop|debug}" exit 1 esac exit 0 From casper2000a@REDACTED Fri Jul 1 06:54:13 2005 From: casper2000a@REDACTED (casper2000a@REDACTED) Date: Fri, 1 Jul 2005 10:54:13 +0600 Subject: Distributed Erlang embeded startup problem In-Reply-To: <17092.25538.528942.681751@antilipe.corelatus.se> References: <944da41d050630013574bf6e15@mail.gmail.com> <20050630114930.6373.qmail@web41904.mail.yahoo.com> <944da41d050630110617a7d5c8@mail.gmail.com> <17092.25538.528942.681751@antilipe.corelatus.se> Message-ID: <1120193653.42c4cc755edc7@www.omnibis.com> Hi, I have created a target system. It's a distributed system, having 2 optional nodes, if one failes application failover to the other node. If the high priority node comes alive it takes over. Attached files are, sys.config - configuration file in one node. The other node's file just have the name of this node in "sync_nodes_optional" esme.system - Linux init file The init file is located at /etc/rc.d/init.d and in linked to /etc/rc.d/rc3.d/S91esme.system When I ran init files manually, the system starts well. When a node fails, it fall backs and everything works smoothly. But when I restart both systems, once they come up, I see both are running the same application. Looks like it doesn't take the applcation as a distributed application. Is there anything missin in my sys.config file? That file is located in the releases/1.0 of that release directory. When I execute the command, nodes() in one node, I can see that it has connected to other node too. What puzzeling me is, why it runs correctly when I ran manually and why it doesn't in a system reboot. Is it an environment settins or commnd line parameter problem? Please help. Thanks in advance, - Eranga --------------This mail sent through OmniBIS.com-------------- -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: sys.config URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: esme.system URL: From bengt.kleberg@REDACTED Fri Jul 1 08:21:36 2005 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 01 Jul 2005 08:21:36 +0200 Subject: simple_one_for_one worker In-Reply-To: References: Message-ID: <42C4E0F0.6080201@ericsson.com> On 2005-07-01 05:48, tty@REDACTED wrote: > Hello, > > I have a simple_one_for_one supervisor. Its workers are meant to > perform one task/transaction then terminate. How should this worker > be defined ? Using gen_server seem to be the easiest but I'm confuse > about terminating the worker. terminating a gen_server is handled like this: any one of the callback modules 3 handle_* functions (handle_call/3, handle_cast/2, handle_info/2) can return a tuple with the first member equal to stop (eg {stop,Reason,Reply,NewState} for handle_call/3). The gen_server will then call Module:terminate(Reason,NewState) and terminate. does this answer your question? bengt From eudesh@REDACTED Fri Jul 1 10:06:56 2005 From: eudesh@REDACTED (Eranga Udesh) Date: Fri, 1 Jul 2005 14:06:56 +0600 Subject: Distributed Erlang embeded startup problem In-Reply-To: <42C4F466.5080206@ericsson.com> References: <42C4F466.5080206@ericsson.com> Message-ID: Hi Bengt, Thanks for your response. The 2 nodes run in 2 seperate computers. As you suggested I started the 2 nodes manually, connected to 2 nodes using to_erl to make sure they are in sync and only one node is running the application, quite Erlang in the inactive node and restarted that computer. After restarted I connected back to that node, but still that node has started the application. I ran nodes() command and it showed it is connected to the other node. Please advice. Thanks, - Eranga On 7/1/05, Bengt Kleberg wrote: > On 2005-06-30 17:42, Eranga Udesh wrote: > > Hi, > > > > I have created a target system. It's a distributed system, having 2 > > optional nodes, if one failes application failover to the other node. > > If the high priority node comes alive it takes over. > > is that a physical computer for each node, or just one computer? > > > > But when I restart both systems, once they come up, I see both are > > running the same application. > > do you restart them both at the same time? what happens if you restart > them one by one? > > > bedngt > From thomasl_erlang@REDACTED Fri Jul 1 14:57:42 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Fri, 1 Jul 2005 05:57:42 -0700 (PDT) Subject: Erlang Micro Edition. In-Reply-To: <944da41d050630110617a7d5c8@mail.gmail.com> Message-ID: <20050701125742.82780.qmail@web34411.mail.mud.yahoo.com> --- Alex Arnon wrote: > I'd like to envision an Erlang VM which would > perform all (or most) > high-level services: > - It would have access to sockets (including RAW > ones) and pipes. > - It would be able to open files and read/write. > binary<=>eterm would > be useful in any case. > - I would like to be able to easily build and link > in port drivers for > controlling special devices. > - I would like to be able to constrain resource > usage per application, e.g.: > - Process priorities. > - Memory usage constraints (per-process or > process group). At present, there is no support for memory constraints inside of Erlang. Also, there is no raw socket support. (Though there seems to be a libpcap driver somewhere out there.) > Now, how cut-down would the functionality of such a > VM and standard library be? Mildly extended, considering the above; though you could in principle get rid of some stuff too. > Would a statically-linked one take less than a MB? > More? About 500KB, maybe? Difficult to say, though Matt Lang's experience in that other mail indicates <5MB in toto for something reasonable. >From the process statistics, it looked like the current emulator was about 1.4 MB + data areas. You could pare it down in three ways (at least). All of them will need a bit of non-standard digging in the code. 1. Get rid of unused Erlang code -- edit stdlib.app and kernel.app and pray. 2. Get rid of unused BIFs -- edit or remove the C files and edit the appropriate config tables so that those primitives are not provided to the compiler. Pray some more :-) 3. Pulling out deeply integrated functionality. Well, hack it until it works, I guess. I'd do them in that order, as needed, basically. > How about the runtime usage? Could we write a DHCP > server that would > need only 30KB of heap space? A web management > interface - 200K? A fresh erlang process is normally pretty small -- about 1 KB of "overhead". The process stack is shrunk and grown automatically as needed. Here is some more data: http://erlang.se/doc/doc-5.4/doc/efficiency_guide/advanced.html A simple protocol FSM doesn't need a lot of data on top of that. For DHCP, I guess you will also use timer.erl to keep track of leases and an ets table for keeping track of assigned addresses. It doesn't sound too demanding, spacewise. Considering web servers, there are a couple already (inets, yaws). Those are fairly full-featured, but serving a static page, say, still shouldn't be too costly. I'm not sure about the current space costs, but my intuition is that they are modest. There are three costs to consider: - the extra erlang code for a feature - the persistent space (e.g., ets tables and server processes) to provide the service - the transient space (e.g., the allocation done to service a single request) Hmmm ... In the end I guess there is no substitute for a bit of experimentation :-) Best, Thomas ____________________________________________________ Yahoo! Sports Rekindle the Rivalries. Sign up for Fantasy Football http://football.fantasysports.yahoo.com From erlang-list@REDACTED Fri Jul 1 15:39:29 2005 From: erlang-list@REDACTED (Dominic Williams) Date: Fri, 01 Jul 2005 15:39:29 +0200 Subject: Erlang Thesis projects finally online! In-Reply-To: <200506301219.j5UCJSQ19573@server3.web-mania.com> References: <200506301219.j5UCJSQ19573@server3.web-mania.com> Message-ID: <42C54791.8010608@dominicwilliams.net> Erlang Training and Consulting wrote: > Test Driven Development In Erlang When I started using Erlang a couple of years ago, I had been using Extreme Programming and TDD for 4 years, so I naturally kept using TDD with Erlang. I actually found Erlang to be ideally suited to TDD. So, I was very interested to read that paper, and the conclusion is interesting (and satisfying for someone with my bias!) I have a question, and a couple of remarks. 1) I don't understand how you measured the number of faults found in the test-first code. The article says that you define a fault to be a failure of the predefined test cases, but the whole point of TDD is to write code that passes the tests. 2) The test provided in Appendix A is not typical of TDD tests, because it doesn't test anything. It just calls some functions, and asserts that they return ok. If I were working in TDD, I would make that test pass in 30 seconds and 3 lines of code: new_company(_,_,_,_) -> ok. new_site(_,_,_,_,_,_) -> ok. new_certificate(_,_,_,_,_,_,_) -> ok. Now, I would have to improve the tests to force me to write code that is more useful. 3) Test driven development is not (only) about testing, it is about design. The article incorrectly states that in TDD, one creates a design then writes tests. The technique is actually to let the need to write tests influence the design, and also not to aim at a definitive design immediately, but on the contrary to write tests gradually, code the simplest thing that will pass the tests so far, then add tests and evolve the design. So, another angle on the question "is TDD appropriate for Erlang" is "does TDD encourage good Erlang design". My experience is that it does: - it encourages writing side-effect free code, because it's the easiest thing to test. - when side-effects are needed, it encourages writing higher-order side-effect-free functions that do most of the complex stuff, and take the side-effect producing code as arguments. - there is a very natural fit between pattern matching and adding increasingly complex test cases. Be that as it may, thanks for the interesting research, and long live the Erlang/TDD combination! Regards, Dominic Williams http://www.dominicwilliams.net ---- From tty@REDACTED Fri Jul 1 16:38:23 2005 From: tty@REDACTED (tty@REDACTED) Date: Fri, 1 Jul 2005 10:38:23 -0400 Subject: simple_one_for_one worker Message-ID: Bengt Kleberg wrote: > any one of the callback modules 3 handle_* functions (handle_call/3, > handle_cast/2, handle_info/2) can return a tuple with the first member > equal to stop (eg {stop,Reason,Reply,NewState} for handle_call/3). > Unfortunately it does not answer my question. I am aware of those callback however who should call them ? Should the Supervisor do that after calling supervisor:terminate_child ? Or does supervisor:terminate_child call one of the handle_* functions. Furthur more since my gen_server process should terminate after it is done processing does this imply it should call supervisor:terminate_child on itself ? Basically the architecture I have in mind looks like: supervisor / | \ worker1 worker2 . . . workerN where worker1 ... workerN are all from the same module. In a non-OTP world I would just do a spawn_link to create the workers and the running process would be *non* recursive i.e. it terminate on the final statement. Thanks tee From francesco@REDACTED Fri Jul 1 17:54:07 2005 From: francesco@REDACTED (Francesco Cesarini (Erlang Training & Consulting)) Date: Fri, 01 Jul 2005 16:54:07 +0100 Subject: Erlang Thesis projects finally online! In-Reply-To: <42C54791.8010608@dominicwilliams.net> References: <200506301219.j5UCJSQ19573@server3.web-mania.com> <42C54791.8010608@dominicwilliams.net> Message-ID: <42C5671F.60708@erlang-consulting.com> The papers Dominic refers to were announced on our mailing list, not the erlang-questions one. As it might be of interest to more, here is the original email: For those of you who missed the presentations last week, we have finally managed to add two interesting thesis projects online at http://www.erlang-consulting.com/erlangarticles_fs.html Remote Controlling Devices Using Instant Messaging -- Building an Intelligent Gateway In Erlang/OTP Abstract: This paper shows how instant messaging technology can be used for remote controlling of devices, and outlines some of the issues involved, of which the most important is security. The concept of controlling and monitoring devices using instant messaging dialogue, presence and buddy list features is applied to a home automation context, and the idea and implementation of a prototype system is described. The paper describes how the excellent robustness and prototyping qualities of Erlang/OTP were exploited to quickly build a prototype system. It also shows how a gateway capable of speaking multiple device protocols can provide a single access point to different kinds of devices and services, and how the concept of agents can be used to add a layer of intelligence to the set of devices being controlled or monitored. PDF file is available at http://www.erlang-consulting.com/thesis/remote_controlling_instant_messaging.pdf Test Driven Development In Erlang Abstract: Test Driven Development has been around for quite some time, but it has recently re-emerged through the introduction of Agile methods such as Extreme Programming and Internet Speed Programming. Test Driven Development has been proven to reduce faults in software written in Java and C++, but there are no reasons to believe these results are language specific. A case study was conducted during the development of an Erlang/OTP prototype comparing Test Last and Test Driven Development. It was concluded that Erlang/OTP based Test Driven Development improved the quality of the code by reducing faults and increasing overall productivity. http://www.erlang-consulting.com/thesis/test_driven_erlang.pdf Enjoy! Martin, Simon and everyone else at Erlang Training & Consulting From michael206@REDACTED Fri Jul 1 18:09:21 2005 From: michael206@REDACTED (Michael Leonhard) Date: Sat, 2 Jul 2005 01:09:21 +0900 Subject: simple_one_for_one worker In-Reply-To: References: Message-ID: <21f8b6e205070109091dad8f2a@mail.gmail.com> Tee, Perhaps you are confused about how to implement simple_one_for_one supervisor and its children? You can see a real implementation in: http://tamale.net/erlchat/chat-0.1/src/chat_liason_sup.erl http://tamale.net/erlchat/chat-0.1/src/chat_liason.erl I am just a beginner with Erlang, but I will do my best to explain. You should make a start_link/0 method to start your supervisor. This method should call supervisor:start_link/3 to say "I implement the supervisor behavior. Please start me!": chat_liason_sup:start_link() -> supervisor:start_link({local, chat_liason_sup}, chat_liason_sup, []). What happens then is that the Erlang supervisor module makes a new process for your supervisor to live inside and then calls your module's init method. The init method should return a "specification" tuple that says what kind of supervisor it is. In our case, it is a simple_one_for_one supervisor, so this structure defines the module and parameters that are used to start the children: chat_liason_sup:init(_Args) -> LiasonSpec = {chat_liason, {chat_liason, start_link, []} , temporary, brutal_kill, worker, [chat_liason]}, StartSpecs = {{simple_one_for_one, 0, 1}, [LiasonSpec]}, {ok, StartSpecs}. No children are started yet! The supervisor only prepares to manage children. When a child needs to be started, your program should call the child module's start/1 method. That start method should call the supervisor:start_child/2 method to request a new child process: chat_liason:start(DataForChild) -> supervisor:start_child(chat_liason_sup, DataForChild). Notice that this call includes only the supervisor's name and some data for the new child. The child's module, start method, and basic data have all been defined previously, when the supervisor first started. The supervisor:start_child/2 method sends a message to the running supervisor process to ask for a new child. This message is handled automatically by the Erlang supervisor module running the process. It calls the child's start method like this: chat_liason:start_link(DataForChild). In this example, the children are all gen_servers. The child starts itself as a gen_server: chat_liason:start_link(Args) -> Options = [], gen_server:start_link(chat_liason, Args, Options). >From then on, the child process runs a loop in the Erlang gen_server module. The process calls methods in the child module when certain things happen. It calls init/1 to let the child module get ready and make its initial state. When a message comes in, the gen_server module processes it and calls the child's corresponding method: handle_call(Request, From, State) -> Result handle_cast(Request, State) -> Result handle_info(Info, State) -> Result If the returned Result is {stop,Reason,NewState} then the gen_server calls the child's terminate/2 method and then the gen_server dies. You cannot use terminate_child/2 on a simple_one_for_one supervisor. A child dies when it gets a message and then returns stop in its Result tuple. All of the methods in your module that implement the supervisor behavior are called automatically by the Erlang supervisor module. Similarly, the child's methods to support gen_server behavior are called automatically by the Erlang gen_server module. You should never call these methods yourself. The details of the supervisor and gen_server modules and behaviors can be found in the stdlib documentation. Click on supervisor or gen_server in the frame on the left: http://erlang.se/doc/doc-5.4.8/lib/stdlib-1.13.8/doc/html/index.html I hope this helps. -Michael Leonhard michael206@REDACTED On 7/1/05, tty@REDACTED wrote: > I am aware of those callback however who should call them ? Should the Supervisor > do that after calling supervisor:terminate_child ? Or does supervisor:terminate_child > call one of the handle_* functions. > > Furthur more since my gen_server process should terminate after it is done > processing does this imply it should call supervisor:terminate_child on itself ? From tty@REDACTED Fri Jul 1 18:34:47 2005 From: tty@REDACTED (tty@REDACTED) Date: Fri, 1 Jul 2005 12:34:47 -0400 Subject: simple_one_for_one worker Message-ID: Hello Michael, I am familiar with gen_server and their call backs. You did hit it on the head when you said I was confused about implementing simple_one_for_one supervisors. Thanks for your reply. I believe I grok it now. tee From alex.arnon@REDACTED Fri Jul 1 23:16:36 2005 From: alex.arnon@REDACTED (Alex Arnon) Date: Fri, 1 Jul 2005 23:16:36 +0200 Subject: Erlang Micro Edition. In-Reply-To: <944da41d05070114153cfc35e8@mail.gmail.com> References: <944da41d050630110617a7d5c8@mail.gmail.com> <20050701125742.82780.qmail@web34411.mail.mud.yahoo.com> <944da41d05070114153cfc35e8@mail.gmail.com> Message-ID: <944da41d0507011416104db062@mail.gmail.com> On 7/1/05, Thomas Lindgren wrote: [snip] > > Hmmm ... In the end I guess there is no substitute for > a bit of experimentation :-) > Indeed :) Thomas and Matthias - thank you for your replies. I will take a deeper look at the current VM code. It seems that it most definitely should be the starting point. Cheers, Alex. From eudesh@REDACTED Sat Jul 2 08:40:06 2005 From: eudesh@REDACTED (Eranga Udesh) Date: Sat, 2 Jul 2005 12:40:06 +0600 Subject: Distributed Erlang embeded startup problem In-Reply-To: <42C54023.6060703@ericsson.com> References: <42C4F466.5080206@ericsson.com> <42C54023.6060703@ericsson.com> Message-ID: Oh.... too bad. I need to find a solution soon. Anyway, that and wish u a wonderful vacation. - Eranga On 7/1/05, Bengt Kleberg wrote: > On 2005-07-01 10:06, Eranga Udesh wrote: > ...deleted > > Please advice. > > unfortunatly i am going on my summer vacation today. i hope somebody > else will be able to help you. > > > bengt > From mikael.karlsson@REDACTED Sat Jul 2 19:25:23 2005 From: mikael.karlsson@REDACTED (Mikael Karlsson) Date: Sat, 2 Jul 2005 19:25:23 +0200 Subject: Web interface framework for Erlang In-Reply-To: <000f01c57c8d$d4be4740$c800a8c0@Simon> References: <000f01c57c8d$d4be4740$c800a8c0@Simon> Message-ID: <200507021925.24322.mikael.karlsson@creado.com> I think this is a great idea. I would say that Yaws, and maybe J-EAI, would be a good foundation to start with. Some work with PostgreSQL and MySQL connectors, SOAP and also Mozilla/XUL have been mentioned on the list. Apart from that, a lot of work would remain I guess. Concerning a lightweight version of WebObjects i think that Ruby on Rails could be a good source for inspiration. http://www.rubyonrails.org/ Cheers Mikael Wed 29 June 2005 11:35 Simon Aurell wrote: > Hello, > > We are currently looking into the idea of building a web framework for > Erlang. I.e. a framework for good separation of presentation and > domain/business logic, aimed at web interfaces. It would have features > such as easy mapping of data entities to html objects, a nice api for > common tasks such as handling form data and probably some handy > utilities. A lightweight version of frameworks like WebObjects, I guess. > > Can anyone tell us what have been done in this area? I've been looking > around in the Erlang repositories I know of but have found very little. > I know of a couple of template engines, but that seems to be it. Are > there any components or libraries that could be useful? Any help would > be greatly appreciated. > > > Cheers, > > Simon > > > ---- > Erlang Training and Consulting > http://www.erlang-consulting.com From b88zhou@REDACTED Sat Jul 2 19:30:34 2005 From: b88zhou@REDACTED (Brian Zhou) Date: Sat, 2 Jul 2005 10:30:34 -0700 Subject: erlang cross compilation Message-ID: I'd like to share what I did to cross compile erlang for NSLU2 (http://www.nslu2-linux.org). The compilation build host machine: $ uname -a Linux hostname 2.6.10-co-0.6.2 #5 Sat Feb 5 10:19:16 IST 2005 i686 GNU/Linux The target machine: $ uname -a Linux slug 2.4.22-xfs #1 Sun Jun 12 21:17:17 PDT 2005 armv5b unknown unknown GNU/Linux The general steps should work for other cross compilation settings as well, it has been used to package R10B-5 and R10B-6. The Makefile and patches are checked in http://cvs.sourceforge.net/viewcvs.py/nslu/unslung/make/erlang.mk?view=markup http://cvs.sourceforge.net/viewcvs.py/nslu/unslung/sources/erlang/ The steps are fully automated in the Makefile, basically 1. Make sure there's an erlang version on the build host with matching version. To be absolutely sure and minimize build dependencies, a host version is built first in erlang-host/ subdirectory. 2. Patch erts/emulator/Makefile.in to use host gcc for the mkliteral tool. See sources/erlang/erts-emulator-Makefile.in.patch for detail. 3. Configure. It's always a good idea to run configure natively so we have a reference. The goal is for the cross configure to generate the exact same erts/$arch/config.h as a native configure does. Lots of the configuration settings can be overriden using ac_cv_ environment variables. Examples here are ac_cv_c_bigendian=yes ac_cv_func_mmap_fixed_mapped=yes In some other cases you have to patch erts/configure.in and run "autoconf configure.in > configure" in erts/, see sources/erlang/erts-configure.in.patch for example. 4. Build the target erlang, this can take a little while (much better than the 10+ hours native build anyway). Start the build by PATH="fullpath_of_erlang-host/bin:$PATH" make -C erlang noboot 5. Building SAE will involve some more patches. The basic idea is to build host version of SAE first, then use the host version of ecc and elink to build the native SAE, see erts-boot-src-Makefile.patch. 6. Packaging. The only tricky part is start_sasl.boot. This can be accomplished by patching erts/etc/unix/Install.src to use the erlang-host/ executables. For NSLU2, we also need to use /opt as prefix for everything. Some simple tests I did: # ipkg install erlang $ erl Erlang (BEAM) emulator version 5.4.6 [source] Eshell V5.4.6 (abort with ^G) 1> User switch command --> q $ cat fib2 #!/usr/bin/env escript -export([main/1]). -mode(compile). main([X]) -> J = list_to_integer(X), N = fib(J), io:format("fib ~w = ~w~n",[J, N]). fib(0) -> 0; fib(1) -> 1; fib(N) -> fib(N-1) + fib(N-2). $ ./fib2 20 fib 20 = 6765 Thanks Matthias Lang for the wonderful web page about his previous erlang cross compilation experience http://www.corelatus.com/~matthias/erlang_on_860.html Cheers, -Brian Zhou From eudesh@REDACTED Sat Jul 2 19:40:21 2005 From: eudesh@REDACTED (Eranga Udesh) Date: Sat, 2 Jul 2005 23:40:21 +0600 Subject: Distributed Erlang embeded startup problem In-Reply-To: References: <42C4F466.5080206@ericsson.com> <42C54023.6060703@ericsson.com> Message-ID: After trying so many combinations, I could fix the problem by using below settings in the sys.config [{kernel, [{distributed, [{omni_esme, 5000, [omni_esme@REDACTED, omni_esme@REDACTED]}]}, {net_setuptime, 15}, {start_dist_ac, true}, {sync_nodes_mandatory, []}, {sync_nodes_optional, [omni_esme@REDACTED]}, {sync_nodes_timeout, 30000}]}]. As I can understand it must have been due to delay in getting the connection to the other node. Since the sync_nodes_timeout was 5000 and there wasn't a net_setuptime parameter, the starting node application should have been started, before connecting to the other node. Please correct me if I am wrong. - Eranga On 7/2/05, Eranga Udesh wrote: > Oh.... too bad. I need to find a solution soon. Anyway, that and wish > u a wonderful vacation. > > - Eranga > > > On 7/1/05, Bengt Kleberg wrote: > > On 2005-07-01 10:06, Eranga Udesh wrote: > > ...deleted > > > Please advice. > > > > unfortunatly i am going on my summer vacation today. i hope somebody > > else will be able to help you. > > > > > > bengt > > > From tty@REDACTED Sun Jul 3 16:56:08 2005 From: tty@REDACTED (tty@REDACTED) Date: Sun, 3 Jul 2005 10:56:08 -0400 Subject: Web interface framework for Erlang Message-ID: If looking at other languages you might also consider struts and tapestry from the Apache project. Both are Java centric. Regards tee -------- Original Message -------- From: Mikael Karlsson Apparently from: owner-erlang-questions@REDACTED To: "Simon Aurell" , Cc: Subject: Re: Web interface framework for Erlang Date: Sat, 2 Jul 2005 19:25:23 +0200 > I think this is a great idea. > I would say that Yaws, and maybe J-EAI, would be a good > foundation to start with. Some work with PostgreSQL and MySQL > connectors, SOAP and also Mozilla/XUL have been mentioned on > the list. > > Apart from that, a lot of work would remain I guess. > > Concerning a lightweight version of WebObjects i think > that Ruby on Rails could be a good source for inspiration. > http://www.rubyonrails.org/ > > Cheers > Mikael > > Wed 29 June 2005 11:35 Simon Aurell wrote: > > Hello, > > > > We are currently looking into the idea of building a web framework for > > Erlang. I.e. a framework for good separation of presentation and > > domain/business logic, aimed at web interfaces. It would have features > > such as easy mapping of data entities to html objects, a nice api for > > common tasks such as handling form data and probably some handy > > utilities. A lightweight version of frameworks like WebObjects, I guess. > > > > Can anyone tell us what have been done in this area? I've been looking > > around in the Erlang repositories I know of but have found very little. > > I know of a couple of template engines, but that seems to be it. Are > > there any components or libraries that could be useful? Any help would > > be greatly appreciated. > > > > > > Cheers, > > > > Simon > > > > > > ---- > > Erlang Training and Consulting > > http://www.erlang-consulting.com From simon@REDACTED Mon Jul 4 10:38:07 2005 From: simon@REDACTED (Simon Aurell) Date: Mon, 04 Jul 2005 09:38:07 +0100 Subject: Web interface framework for Erlang In-Reply-To: References: Message-ID: <42C8F56F.7030500@erlang-consulting.com> Tee: Struts is definitely one of the frameworks that I've come in contact with before which could serve as a model. However, the whole idea is to create something for Erlang. As Mikael mentioned, Yaws is a good idea and is probably what we will build it on. I've only got a vague idea of what J-EAI is. How would you say it could be used in building in this framework? I've heard lots about Ruby on Rails. Will definitely look into it. What would you say would be the most important features in a first prototype? Simon tty@REDACTED wrote: >If looking at other languages you might also consider struts and tapestry from the Apache project. Both are Java centric. > >Regards > >tee > >-------- Original Message -------- >From: Mikael Karlsson >Apparently from: owner-erlang-questions@REDACTED >To: "Simon Aurell" , >Cc: >Subject: Re: Web interface framework for Erlang >Date: Sat, 2 Jul 2005 19:25:23 +0200 > > > >>I think this is a great idea. >>I would say that Yaws, and maybe J-EAI, would be a good >>foundation to start with. Some work with PostgreSQL and MySQL >>connectors, SOAP and also Mozilla/XUL have been mentioned on >>the list. >> >>Apart from that, a lot of work would remain I guess. >> >>Concerning a lightweight version of WebObjects i think >>that Ruby on Rails could be a good source for inspiration. >>http://www.rubyonrails.org/ >> >>Cheers >>Mikael >> >> -- http://www.erlang-consulting.com From luvishsatija@REDACTED Mon Jul 4 10:36:49 2005 From: luvishsatija@REDACTED (luvish satija) Date: Mon, 4 Jul 2005 09:36:49 +0100 (BST) Subject: SSL Setup Problem Message-ID: <20050704083649.35241.qmail@web8504.mail.in.yahoo.com> Hello all, I am trying to set up ssl for erlang on my system but facing some problems. I am using the following .rel file for making boot script. {release, {"OTP APN 181 01","R10B"}, {erts, "5.4.6"}, [{kernel,"2.10.7"}, {stdlib,"1.13.7"}, {ssl, "3.0.5"}]}. Now, when I try to call make_script on this, I get the following warnings: *WARNING* ssl: Source code not found: 'SSL-PKIX'.erl *WARNING* ssl: Source code not found: 'PKIX1Algorithms88'.erl *WARNING* ssl: Source code not found: 'PKIX1Explicit88'.erl *WARNING* ssl: Source code not found: 'PKIX1Implicit88'.erl *WARNING* ssl: Source code not found: 'PKIXAttributeCertificate'.erl *WARNING* ssl: Source code not found: ssl_pkix_oid.erl But to my surprise, these files are actually present on my system ( in directories /usr/local/lib/erlang/lib/ssl-3.0.5/pkix/ and /usr/lib/erlang/lib/ssl-3.0.5/pkix/). I have tried giving all kind of path options like -pa, -pz and in make_script also, but to no use. If I igonore these warnings and start the ssl system using the produced boot script, then I am not able to listen on any port using ssl:socket() and following error is thrown: =ERROR REPORT==== 4-Jul-2005::13:34:53 === ** Generic server <0.40.0> terminating ** Last message in was {listen,<0.38.0>, 5555, [binary, {packet,0}, {active,false}, {nodelay,true}]} ** When Server state == {st,listener, <0.31.0>, <0.38.0>, <0.38.0>, nil, true, [], nil, nil, nil, false} ** Reason for termination == ** {normal,{gen_server,call, [ssl_server, {listen,<0.40.0>,{0,0,0,0},5555,[],5}, infinity]}} This error message is beyong my understandings. If anybody can help me in this regard, I would be obliged. Thanks in advance, /luvish _______________________________________________________ Too much spam in your inbox? Yahoo! Mail gives you the best spam protection for FREE! http://in.mail.yahoo.com From chandrashekhar.mullaparthi@REDACTED Mon Jul 4 11:02:26 2005 From: chandrashekhar.mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Mon, 4 Jul 2005 10:02:26 +0100 Subject: SSL Setup Problem In-Reply-To: <20050704083649.35241.qmail@web8504.mail.in.yahoo.com> References: <20050704083649.35241.qmail@web8504.mail.in.yahoo.com> Message-ID: <5c8698de42c55411c3d774d6864540cf@t-mobile.co.uk> Hi, On 4 Jul 2005, at 09:36, luvish satija wrote: > Hello all, > > I am trying to set up ssl for erlang on my system > but facing some problems. I am using the following > .rel file for making boot script. > > {release, {"OTP APN 181 01","R10B"}, {erts, "5.4.6"}, > [{kernel,"2.10.7"}, > {stdlib,"1.13.7"}, > {ssl, "3.0.5"}]}. > > Now, when I try to call make_script on this, I get > the following warnings: > > *WARNING* ssl: Source code not found: 'SSL-PKIX'.erl This is ok - the systools:make_script looks for source files under App/src directory. In this case, the source files are in a non-standard directory which is why you see these error messages. > > If I igonore these warnings and start the ssl > system using the produced boot script, then I am not > able to listen on any port using ssl:socket() and > following error is thrown: You should be listening using ssl:listen/2 - ssl:socket isn't even an exported function. cheers Chandru From luvishsatija@REDACTED Mon Jul 4 11:17:30 2005 From: luvishsatija@REDACTED (luvish satija) Date: Mon, 4 Jul 2005 10:17:30 +0100 (BST) Subject: SSL Setup Problem In-Reply-To: <5c8698de42c55411c3d774d6864540cf@t-mobile.co.uk> Message-ID: <20050704091730.33936.qmail@web8506.mail.in.yahoo.com> hi chandru, thanks for the reply. ssl:socket() was a mistake, I'm actually using ssl:listen() but still I've not been able to get it working properly. I don't know where i'm missing out. As soon as i start my server, the error is thrown, which i included in my last mail, showing that gen_server is terminating. if you can suggest some other solution, please help me. thanks /luvish --- Chandrashekhar Mullaparthi wrote: > Hi, > > On 4 Jul 2005, at 09:36, luvish satija wrote: > > > Hello all, > > > > I am trying to set up ssl for erlang on my > system > > but facing some problems. I am using the following > > .rel file for making boot script. > > > > {release, {"OTP APN 181 01","R10B"}, {erts, > "5.4.6"}, > > [{kernel,"2.10.7"}, > > {stdlib,"1.13.7"}, > > {ssl, "3.0.5"}]}. > > > > Now, when I try to call make_script on this, I > get > > the following warnings: > > > > *WARNING* ssl: Source code not found: > 'SSL-PKIX'.erl > > > This is ok - the systools:make_script looks for > source files under > App/src directory. In this case, the source files > are in a non-standard > directory which is why you see these error messages. > > > > > If I igonore these warnings and start the ssl > > system using the produced boot script, then I am > not > > able to listen on any port using ssl:socket() and > > following error is thrown: > > You should be listening using ssl:listen/2 - > ssl:socket isn't even an > exported function. > > cheers > Chandru > > _______________________________________________________ Too much spam in your inbox? Yahoo! Mail gives you the best spam protection for FREE! http://in.mail.yahoo.com From erlang@REDACTED Mon Jul 4 12:07:03 2005 From: erlang@REDACTED (Michael McDaniel) Date: Mon, 4 Jul 2005 03:07:03 -0700 Subject: SSL Setup Problem In-Reply-To: <20050704083649.35241.qmail@web8504.mail.in.yahoo.com> References: <20050704083649.35241.qmail@web8504.mail.in.yahoo.com> Message-ID: <20050704100703.GS11207@fangora.autosys.us> On Mon, Jul 04, 2005 at 09:36:49AM +0100, luvish satija wrote: > Hello all, > > I am trying to set up ssl for erlang on my system > but facing some problems. I am using the following > .rel file for making boot script. > > {release, {"OTP APN 181 01","R10B"}, {erts, "5.4.6"}, > [{kernel,"2.10.7"}, > {stdlib,"1.13.7"}, > {ssl, "3.0.5"}]}. > > Now, when I try to call make_script on this, I get > the following warnings: > > *WARNING* ssl: Source code not found: 'SSL-PKIX'.erl > *WARNING* ssl: Source code not found: > 'PKIX1Algorithms88'.erl > *WARNING* ssl: Source code not found: > 'PKIX1Explicit88'.erl > *WARNING* ssl: Source code not found: > 'PKIX1Implicit88'.erl > *WARNING* ssl: Source code not found: > 'PKIXAttributeCertificate'.erl > *WARNING* ssl: Source code not found: ssl_pkix_oid.erl > > But to my surprise, these files are > actually present on my system ( in directories > /usr/local/lib/erlang/lib/ssl-3.0.5/pkix/ and > /usr/lib/erlang/lib/ssl-3.0.5/pkix/). I have tried > giving all kind of path options like -pa, -pz and in > make_script also, but to no use. > If anybody can help me in this regard, I would be > obliged. > Thanks in advance, > /luvish _______________________________________________________ I make symbolic links, for example, in /usr/local/lib/erlang/lib/ssl-3.0.5/src lrwxrwxrwx 1 60 2005-03-27 23:03 mk_ssl_pkix_oid.erl -> /usr/local/lib/erlang/lib/ssl-3.0.5/pkix/mk_ssl_pkix_oid.erl lrwxrwxrwx 1 62 2005-03-27 23:03 PKIX1Algorithms88.erl -> /usr/local/lib/erlang/lib/ssl-3.0.5/pkix/PKIX1Algorithms88.erl lrwxrwxrwx 1 60 2005-03-27 23:03 PKIX1Explicit88.erl -> /usr/local/lib/erlang/lib/ssl-3.0.5/pkix/PKIX1Explicit88.erl lrwxrwxrwx 1 60 2005-03-27 23:03 PKIX1Implicit88.erl -> /usr/local/lib/erlang/lib/ssl-3.0.5/pkix/PKIX1Implicit88.erl lrwxrwxrwx 1 69 2005-03-27 23:03 PKIXAttributeCertificate.erl -> /usr/local/lib/erlang/lib/ssl-3.0.5/pkix/PKIXAttributeCertificate.erl lrwxrwxrwx 1 53 2005-03-27 23:03 SSL-PKIX.erl -> /usr/local/lib/erlang/lib/ssl-3.0.5/pkix/SSL-PKIX.erl lrwxrwxrwx 1 57 2005-03-27 23:03 ssl_pkix_oid.erl -> /usr/local/lib/erlang/lib/ssl-3.0.5/pkix/ssl_pkix_oid.erl ~Michael From tim@REDACTED Mon Jul 4 13:31:10 2005 From: tim@REDACTED (Tim Bates) Date: Mon, 04 Jul 2005 21:01:10 +0930 Subject: Web interface framework for Erlang In-Reply-To: <42C8F56F.7030500@erlang-consulting.com> References: <42C8F56F.7030500@erlang-consulting.com> Message-ID: <42C91DFE.9080600@bates.id.au> Simon Aurell wrote: > I've heard lots about Ruby on Rails. Will definitely look into it. I'm very familiar with Ruby on Rails (more so than with Erlang - currently developing with it and have submitted sizable patches) so if you have questions about it feel free to email me. I would be interested to see how some of its concepts could be applied to Erlang. Tim. -- Tim Bates tim@REDACTED From joelr1@REDACTED Mon Jul 4 15:26:32 2005 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 4 Jul 2005 15:26:32 +0200 Subject: TCP stack throughput Message-ID: Folks, After having problems with the test harness for my poker server I decided to set up a simple test. Could someone please explain the results to me? I'm on Mac OSX 10.4.1 and the number of descriptors per process (set with ulimit -n) is 10240. I'm also wondering what the results would be when running this test on your platforms. The files are at http://wagerlabs.com/fac.erl and http:// wagerlabs.com/tcp_server.erl. The server is Joe's code with slightly different TCP settings. I'm first launching the tcp server to handle factorial requests. I then launch, say, 100 clients very rapidly to calculate the factorial. Here, without further ado are the results of my test. Try increasing the last parameter to 50 or 100 to see the number of refused connections go much higher. c-fd82e055:~/work/code-dump joelr$ erl Erlang (BEAM) emulator version 5.4.8 [source] [hipe] [threads:0] Eshell V5.4.8 (abort with ^G) 1> fac:start(5000). Starting a port server on 5000... {ok,<0.33.0>} 2> fac:fac(localhost, 5000, 20, 20). ok #1: Calculating factorial(20) #2: Calculating factorial(20) #3: Calculating factorial(20) #4: Calculating factorial(20) #5: Calculating factorial(20) #6: Calculating factorial(20) #1: fac(20) = 2432902008176640000 #2: fac(20) = 2432902008176640000 #3: fac(20) = 2432902008176640000 #4: fac(20) = 2432902008176640000 #7: Calculating factorial(20) #8: Calculating factorial(20) #5: fac(20) = 2432902008176640000 #6: fac(20) = 2432902008176640000 #7: fac(20) = 2432902008176640000 #8: fac(20) = 2432902008176640000 #13: Calculating factorial(20) #13: fac(20) = 2432902008176640000 #12: start_client: {error,econnreset} #14: start_client: {error,econnreset} #15: start_client: {error,econnreset} #16: start_client: {error,econnreset} #17: start_client: {error,econnreset} #18: start_client: {error,econnreset} #19: start_client: {error,econnreset} #20: start_client: {error,econnreset} #11: Calculating factorial(20) #10: Calculating factorial(20) #9: Calculating factorial(20) #11: fac(20) = 2432902008176640000 #9: fac(20) = 2432902008176640000 #10: fac(20) = 2432902008176640000 -- http://wagerlabs.com/tech From joelr1@REDACTED Mon Jul 4 15:37:49 2005 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 4 Jul 2005 15:37:49 +0200 Subject: TCP stack throughput In-Reply-To: References: Message-ID: <1FDAF1B3-57A4-425C-96DB-DDCC41093D7A@gmail.com> The files are not available for some reason so I'm pasting them here just in case... -module(fac). -behaviour(gen_server). -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). -export([start/1, stop/1]). -export([fac/4]). -record(server, { self, port, players, pending }). new(Port) -> #server { self = self(), port = Port }. start(Port) -> gen_server:start(fac, [Port], []). init([Port]) -> process_flag(trap_exit, true), Server = new(Port), tcp_server:start_raw_server(Port, fun(Sock) -> handler(Sock, Server) end, 50000, 2048), {ok, Server}. stop(Server) -> gen_server:cast(Server, stop). terminate(normal, Server) -> tcp_server:stop(Server#server.port), ok. handle_cast(stop, Server) -> {stop, normal, Server}; handle_cast(Event, Server) -> error_logger:info_report([{module, ?MODULE}, {line, ?LINE}, {self, self()}, {message, Event}]), {noreply, Server}. handle_call(Event, From, Server) -> error_logger:info_report([{module, ?MODULE}, {line, ?LINE}, {self, self()}, {message, Event}, {from, From}]), {noreply, Server}. handle_info({'EXIT', _Pid, _Reason}, Server) -> %% child exit? {noreply, Server}; handle_info(Info, Server) -> error_logger:info_report([{module, ?MODULE}, {line, ?LINE}, {self, self()}, {message, Info}]), {noreply, Server}. code_change(_OldVsn, Server, _Extra) -> {ok, Server}. %%% %%% Implementation %%% fac(N) when N > 0 -> N * fac(N - 1); fac(0) -> 1. fac(_Host, _Port, _N, 0) -> ok; fac(Host, Port, N, Max) -> Fun = fun() -> case tcp_server:start_client(Host, Port, 2048) of {ok, Sock} -> case gen_tcp:send(Sock, <>) of ok -> receive {tcp, Sock, <>} -> io:format("#~w: fac(~w) = ~w~n", [Max, N, Fac]); {error, closed} -> io:format("#~w: connection closed~n", [Max]); Any -> io:format("#~w: receive: ~w~n", [Max, Any]) end, gen_tcp:close(Sock); Any -> io:format("#~w: gen_tcp: ~w~n", [Max, Any]) end; Any -> io:format("#~w: start_client: ~w~n", [Max, Any]) end end, spawn(Fun), fac(Host, Port, N, Max - 1). handler(Socket, Parent) -> receive {tcp, Socket, <>} -> io:format("#~w: Calculating factorial(~w)~n", [Iter, N]), Fac = fac(N), gen_tcp:send(Socket, <>), handler(Socket, Parent); {tcp_closed, Socket} -> %%io:format("Socket ~w closed~n", [Socket]); ok; Any -> io:format("Got ~w in the handler~n", [Any]), gen_tcp:send(Socket, <<0:2048>>) end. ----------------------- %% Copyright (C) 2002, Joe Armstrong %% File : tcp_server.erl %% Author : Joe Armstrong (joe@REDACTED) %% Purpose : Keeps track of a number of TCP sessions %% Last modified: 2002-11-17 -module(tcp_server). -export([start_raw_server/4, start_client/3, stop/1, children/1, send/2]). -define(KILL_DELAY, 1000). %% -export([start_child/3]). %% start_raw_server(Port, Fun, Max) %% This server accepts up to Max connections on Port %% The *first* time a connection is made to Port %% Then Fun(Socket) is called. %% Thereafter messages to the socket result in messsages to the handler. %% a typical server is usually written like this: %% To setup a lister %% start_server(Port) -> %% S = self(), %% process_flag(trap_exit, true), %% tcp_server:start_raw_server(Port, %% fun(Socket) -> input_handler(Socket, S) end, %% 15, %% 0) %% loop(). %% The loop() process is a central controller that all %% processes can use to synchronize amongst themselfves if necessary %% It ends up as the variable "Controller" in the input_handler %% A typical server is written like this: %% input_handler(Socket, Controller) -> %% receive %% {tcp, Socket, Bin} -> %% ... %% gen_tcp:send(Socket, ...) %% %% {tcp_closed, Socket} -> %% %% %% Any -> %% ... %% %% end. start_client(Host, Port, Length) -> gen_tcp:connect(Host, Port, [binary, {active, true}, {packet, 2}, {packet_size, Length}], 10000). %% Note when start_raw_server returns it should be ready to %% Immediately accept connections start_raw_server(Port, Fun, Max, Length) -> Name = port_name(Port), case whereis(Name) of undefined -> Self = self(), Pid = spawn_link(fun() -> cold_start(Self, Port, Fun, Max, Length) end), receive {Pid, ok} -> register(Name, Pid), {ok, Pid}; {Pid, Error} -> Error end; _Pid -> {error, already_started} end. stop(Port) when integer(Port) -> Name = port_name(Port), case whereis(Name) of undefined -> not_started; Pid -> exit(Pid, kill), (catch unregister(Name)), stopped end. children(Port) when integer(Port) -> port_name(Port) ! {children, self()}, receive {session_server, Reply} -> Reply end. port_name(Port) when integer(Port) -> list_to_atom("portServer" ++ integer_to_list(Port)). cold_start(Master, Port, Fun, Max, Length) -> process_flag(trap_exit, true), io:format("Starting a port server on ~p...~n",[Port]), case gen_tcp:listen(Port, [binary, %% {dontroute, true}, {nodelay,true}, {packet_size, Length}, {packet, 2}, {reuseaddr, true}, {active, false}]) of {ok, Listen} -> %% io:format("Listening on:~p~n",[Listen]), Master ! {self(), ok}, New = start_accept(Listen, Fun), %% Now we're ready to run socket_loop(Listen, New, [], Fun, Max); Error -> Master ! {self(), Error} end. %% Don't mess with the following code uless you really know what you're %% doing (and Thanks to Magnus for heping me get it right) socket_loop(Listen, New, Active, Fun, Max) -> receive {istarted, New} -> Active1 = [New|Active], possibly_start_another(false, Listen, Active1, Fun, Max); {'EXIT', New, _Why} -> %%io:format("Child exit=~p~n",[Why]), possibly_start_another(false, Listen, Active, Fun, Max); {'EXIT', Pid, _Why} -> %%io:format("Child exit=~p~n",[Why]), Active1 = lists:delete(Pid, Active), possibly_start_another(New, Listen, Active1, Fun, Max); {children, From} -> From ! {session_server, Active}, socket_loop(Listen, New, Active, Fun, Max); Other -> io:format("Here in loop:~p~n",[Other]) end. possibly_start_another(New, Listen, Active, Fun, Max) when pid(New) -> socket_loop(Listen, New, Active, Fun, Max); possibly_start_another(false, Listen, Active, Fun, Max) -> case length(Active) of N when N < Max -> New = start_accept(Listen, Fun), socket_loop(Listen, New, Active, Fun, Max); _ -> error_logger:warning_report( [{module, ?MODULE}, {line, ?LINE}, {message, "Connections maxed out"}, {maximum, Max}, {connected, length(Active)}, {now, now()}]), socket_loop(Listen, false, Active, Fun, Max) end. start_accept(Listen, Fun) -> S = self(), spawn_link(fun() -> start_child(S, Listen, Fun) end). start_child(Parent, Listen, Fun) -> case gen_tcp:accept(Listen) of {ok, Socket} -> Parent ! {istarted,self()}, % tell the controller inet:setopts(Socket, [{nodelay,true}, {packet, 2}, {active, true}]), % before we activate socket %% send an event to kill the connection %% after a certain time unless %% a login packet is received. %%erlang:send_after(?KILL_DELAY, self(), 'KILL'), Fun(Socket); _Other -> exit(oops) end. send(Socket, Data) -> case gen_tcp:send(Socket, Data) of ok -> ok; Error -> error_logger:error_report([{message, "gen_tcp:send error"}, {module, ?MODULE}, {line, ?LINE}, {socket, Socket}, {port_info, erlang:port_info(Socket, connected)}, {data, Data}, {error, Error}]) end. From ulf.wiger@REDACTED Mon Jul 4 16:08:08 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Mon, 4 Jul 2005 16:08:08 +0200 Subject: TCP stack throughput Message-ID: I ran the tests on Solaris and Windows 2000. On Solaris, 20 clients went well. 50 triggered the 10 second timeout in the client's connect() call. Increasing that timeout to 30 secs, 50 clients went well, but the timeouts reappeared when trying 100 clients. No connections refused, though. On Win 2000, 20 clients went well, but with 50 clients, lots of 'econnrefused'. /Uffe > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Joel Reymont > Sent: den 4 juli 2005 15:27 > To: Erlang Users' List > Subject: TCP stack throughput > > > Folks, > > After having problems with the test harness for my poker server I > decided to set up a simple test. Could someone please explain the > results to me? > > I'm on Mac OSX 10.4.1 and the number of descriptors per process (set > with ulimit -n) is 10240. I'm also wondering what the results would > be when running this test on your platforms. > > The files are at http://wagerlabs.com/fac.erl and http:// > wagerlabs.com/tcp_server.erl. The server is Joe's code with slightly > different TCP settings. > > I'm first launching the tcp server to handle factorial requests. I > then launch, say, 100 clients very rapidly to calculate the > factorial. Here, without further ado are the results of my test. > > Try increasing the last parameter to 50 or 100 to see the number of > refused connections go much higher. > > c-fd82e055:~/work/code-dump joelr$ erl > Erlang (BEAM) emulator version 5.4.8 [source] [hipe] [threads:0] > > Eshell V5.4.8 (abort with ^G) > 1> fac:start(5000). > Starting a port server on 5000... > {ok,<0.33.0>} > 2> fac:fac(localhost, 5000, 20, 20). > ok > #1: Calculating factorial(20) > #2: Calculating factorial(20) > #3: Calculating factorial(20) > #4: Calculating factorial(20) > #5: Calculating factorial(20) > #6: Calculating factorial(20) > #1: fac(20) = 2432902008176640000 > #2: fac(20) = 2432902008176640000 > #3: fac(20) = 2432902008176640000 > #4: fac(20) = 2432902008176640000 > #7: Calculating factorial(20) > #8: Calculating factorial(20) > #5: fac(20) = 2432902008176640000 > #6: fac(20) = 2432902008176640000 > #7: fac(20) = 2432902008176640000 > #8: fac(20) = 2432902008176640000 > #13: Calculating factorial(20) > #13: fac(20) = 2432902008176640000 > #12: start_client: {error,econnreset} > #14: start_client: {error,econnreset} > #15: start_client: {error,econnreset} > #16: start_client: {error,econnreset} > #17: start_client: {error,econnreset} > #18: start_client: {error,econnreset} > #19: start_client: {error,econnreset} > #20: start_client: {error,econnreset} > #11: Calculating factorial(20) > #10: Calculating factorial(20) > #9: Calculating factorial(20) > #11: fac(20) = 2432902008176640000 > #9: fac(20) = 2432902008176640000 > #10: fac(20) = 2432902008176640000 > > > -- > http://wagerlabs.com/tech > > > From jabba@REDACTED Mon Jul 4 16:15:53 2005 From: jabba@REDACTED (Jani Launonen) Date: Mon, 4 Jul 2005 17:15:53 +0300 (EEST) Subject: TCP stack throughput In-Reply-To: References: Message-ID: Have you tried to increase the backlog of the listening socket? It defaults to 5 and you could increase it by using {backlog, 100} option (could be other than 100) in gen_tcp:listen/2 -function? I haven't tested it, but just a quick idea. See listen/2 function at http://erlang.se/doc/doc-5.4.8/lib/kernel-2.10.9/doc/html/index.html Hopefully works. -+-+-+- Jani Launonen From joelr1@REDACTED Mon Jul 4 16:29:46 2005 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 4 Jul 2005 16:29:46 +0200 Subject: TCP stack throughput In-Reply-To: References: Message-ID: Jani, Mea culpa, I forgot about the backlog. Still, setting backlog to 1024 and trying the test with 500 clients about 200 kick out with a reset connection. I tried increasing the backlog to 2048 but I get the same number of errors as with 1024. Thanks, Joel On Jul 4, 2005, at 4:15 PM, Jani Launonen wrote: > Have you tried to increase the backlog of the listening socket? It > defaults to 5 and you could increase it by using {backlog, 100} > option (could be other than 100) in gen_tcp:listen/2 -function? > > I haven't tested it, but just a quick idea. -- http://wagerlabs.com/tech From ulf.wiger@REDACTED Mon Jul 4 16:39:25 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Mon, 4 Jul 2005 16:39:25 +0200 Subject: FW: TCP stack throughput Message-ID: With {backlog, 1024}, I could do 500 clients on Solaris. Increasing the connect() timeout to 60 seconds, I could just barely manage 900. No connections refused or reset. On Windows 2000, the large backlog enabled 50 clients, but 100 was too much. Lots of 'econnrefused'. /Uffe -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED]On Behalf Of Jani Launonen Sent: den 4 juli 2005 16:16 To: Joel Reymont Cc: Erlang Users' List Subject: Re: TCP stack throughput Have you tried to increase the backlog of the listening socket? It defaults to 5 and you could increase it by using {backlog, 100} option (could be other than 100) in gen_tcp:listen/2 -function? I haven't tested it, but just a quick idea. See listen/2 function at http://erlang.se/doc/doc-5.4.8/lib/kernel-2.10.9/doc/html/index.html Hopefully works. -+-+-+- Jani Launonen From jabba@REDACTED Mon Jul 4 16:44:02 2005 From: jabba@REDACTED (Jani Launonen) Date: Mon, 4 Jul 2005 17:44:02 +0300 (EEST) Subject: TCP stack throughput In-Reply-To: References: Message-ID: Ok, I've tested the original version with SuSE 9.1 (Linux kernel 2.6.5-7.155.29-default) with R10-2 on AMD Athlon 2100+. > Try increasing the last parameter to 50 or 100 to see the number of refused > connections go much higher. After parametre value of 100 I start getting {error, econnreset} errors. 20 and 50 were just ok. After I added the backlog parametre to listen socket and set it's value to 100, everything goes just fine and rather more quickly. I'd say, problem found and corrected. Hopefully it does the trick for you too. -+-+-+- Jani Launonen From joelr1@REDACTED Mon Jul 4 17:17:54 2005 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 4 Jul 2005 17:17:54 +0200 Subject: TCP stack throughput In-Reply-To: References: Message-ID: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> So this means 500 simultanous connection requests on Solaris and about 300 on Mac OSX (FreeBSD). My question, I suppose, is whether this is high enough. Is there a limitation in Erlang or is this maxing out the platform TCP stack? What do you folks recon? Thanks, Joel On Jul 4, 2005, at 4:39 PM, Ulf Wiger (AL/EAB) wrote: > > With {backlog, 1024}, I could do 500 clients on Solaris. > Increasing the connect() timeout to 60 seconds, I could > just barely manage 900. No connections refused or reset. > > On Windows 2000, the large backlog enabled 50 clients, but > 100 was too much. Lots of 'econnrefused'. -- http://wagerlabs.com/tech From mickael.remond@REDACTED Mon Jul 4 19:16:43 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Mon, 04 Jul 2005 19:16:43 +0200 Subject: TCP stack throughput In-Reply-To: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> Message-ID: <42C96EFB.6080301@erlang-fr.org> Joel Reymont wrote: > So this means 500 simultanous connection requests on Solaris and about > 300 on Mac OSX (FreeBSD). My question, I suppose, is whether this is > high enough. > > Is there a limitation in Erlang or is this maxing out the platform TCP > stack? > > What do you folks recon? Hello, I am not sure I understand your test quite well. Basically you are connecting and deconnecting very quickly. Is that the case ? If this is the case, what are you trying to benchmark ? Most of the time mesuring the TCP/IP throughput implies trying to know how much data can be processed through a TCP socket, but usually a long running one. My experience in Erlang, shows that it can handle very well simultaneous process, so you can have many processs and thus connexion simultaneously. I did not try to benchmark quick connect/reconnect. From experience, there might be some performance problem. It looks like a denial of service for the application. Does it answer your questions ? -- Micka?l R?mond From jabba@REDACTED Mon Jul 4 19:25:56 2005 From: jabba@REDACTED (Jani Launonen) Date: Mon, 4 Jul 2005 20:25:56 +0300 (EEST) Subject: TCP stack throughput In-Reply-To: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> Message-ID: > So this means 500 simultanous connection requests on Solaris and about 300 on > Mac OSX (FreeBSD). My question, I suppose, is whether this is high enough. My Linux did also 500 without problem, but I didn't push it further. > Is there a limitation in Erlang or is this maxing out the platform TCP stack? Have you tried to trace or any other tool? Unfortunately I'm not able to help you with those. > What do you folks recon? Are sure that this test does measure the poker game behaviour? I do have my doubts, but on the other hand, it's just a hunch. Have you checked Klacke's Yaws? Apparently it uses a pool of processes waiting for jobs. Perhaps you might check this out: http://www.trapexit.org/docs/howto/simple_non_blocking_pattern.html By the way --- I tried to change the factorial to tail recursive version in anticipation that constant memory space might help take pressure off EVM's memory management, but it didn't do anything obvious to help situation. > Thanks, Joel > > On Jul 4, 2005, at 4:39 PM, Ulf Wiger (AL/EAB) wrote: >> >> With {backlog, 1024}, I could do 500 clients on Solaris. >> Increasing the connect() timeout to 60 seconds, I could >> just barely manage 900. No connections refused or reset. >> >> On Windows 2000, the large backlog enabled 50 clients, but >> 100 was too much. Lots of 'econnrefused'. > > -- > http://wagerlabs.com/tech > > -+-+-+- Jani Launonen From joelr1@REDACTED Mon Jul 4 19:31:00 2005 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 4 Jul 2005 19:31:00 +0200 Subject: TCP stack throughput In-Reply-To: <42C96EFB.6080301@erlang-fr.org> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> Message-ID: I'm probably using the wrong term here. I'm not measuring throughput but the number of connections that can be accepted by Erlang in a very short period of time. On Jul 4, 2005, at 7:16 PM, Mickael Remond wrote: > I am not sure I understand your test quite well. Basically you are > connecting and deconnecting very quickly. Is that the case ? > If this is the case, what are you trying to benchmark ? > Most of the time mesuring the TCP/IP throughput implies trying to > know how much data can be processed through a TCP socket, but > usually a long running one. -- http://wagerlabs.com/tech From matthias@REDACTED Mon Jul 4 23:34:11 2005 From: matthias@REDACTED (Matthias Lang) Date: Mon, 4 Jul 2005 23:34:11 +0200 Subject: TCP stack throughput In-Reply-To: <42C96EFB.6080301@erlang-fr.org> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> Message-ID: <17097.43859.831827.297323@antilipe.corelatus.se> Mickael Remond writes: > Joel Reymont wrote: > > So this means 500 simultanous connection requests on Solaris and about > > 300 on Mac OSX (FreeBSD). My question, I suppose, is whether this is > > high enough. > > > > Is there a limitation in Erlang or is this maxing out the platform TCP > > stack? > > > > What do you folks recon? > > Hello, > > I am not sure I understand your test quite well. Basically you are > connecting and deconnecting very quickly. Is that the case ? > If this is the case, what are you trying to benchmark ? > Most of the time mesuring the TCP/IP throughput implies trying to know > how much data can be processed through a TCP socket, but usually a long > running one. I'm with Mickael. Your results could be interpreted as showing one _or more_ of the following: a) That your test client is the bottleneck b) That your test server is the bottleneck c) That the OS/tcp stack isn't really designed for the sort of use you're testing. My gut feeling says (c), mainly because different people are reporting fairly different behaviour from one OS to another. I've written a quick and dirty C server which does something similar enough to the erlang server to work with your client. You could experiment with it to try and eliminate hypothesis (b). The program works for me on linux 2.6.x. Matthias -------------- next part -------------- A non-text attachment was scrubbed... Name: server.c Type: application/octet-stream Size: 1548 bytes Desc: not available URL: From sean.hinde@REDACTED Tue Jul 5 01:05:51 2005 From: sean.hinde@REDACTED (Sean Hinde) Date: Tue, 5 Jul 2005 00:05:51 +0100 Subject: TCP stack throughput In-Reply-To: <17097.43859.831827.297323@antilipe.corelatus.se> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> Message-ID: <2cb9022a3b07263b827f1d3732fb9b40@mac.com> I had some discussions once with some folks who were deploying ad server type systems - lots of very short lived connections. The game was all about winding down time wait etc until the tcp stack stopped running out of port numbers. They had a funky in house designed C web server with OS threads, which did work mostly. OTOH if you are the one controlling the client behaviour then you can ensure they use persistent connections where possible. Sean On 4 Jul 2005, at 22:34, Matthias Lang wrote: > Mickael Remond writes: >> Joel Reymont wrote: >>> So this means 500 simultanous connection requests on Solaris and >>> about >>> 300 on Mac OSX (FreeBSD). My question, I suppose, is whether this is >>> high enough. >>> >>> Is there a limitation in Erlang or is this maxing out the platform >>> TCP >>> stack? >>> >>> What do you folks recon? >> >> Hello, >> >> I am not sure I understand your test quite well. Basically you are >> connecting and deconnecting very quickly. Is that the case ? >> If this is the case, what are you trying to benchmark ? >> Most of the time mesuring the TCP/IP throughput implies trying to know >> how much data can be processed through a TCP socket, but usually a >> long >> running one. > > I'm with Mickael. > > Your results could be interpreted as showing one _or more_ of the > following: > > a) That your test client is the bottleneck > > b) That your test server is the bottleneck > > c) That the OS/tcp stack isn't really designed for the sort of > use you're testing. > > My gut feeling says (c), mainly because different people are reporting > fairly different behaviour from one OS to another. I've written a > quick and dirty C server which does something similar enough to the > erlang server to work with your client. You could experiment with it > to try and eliminate hypothesis (b). > > The program works for me on linux 2.6.x. > > Matthias > > From joelr1@REDACTED Tue Jul 5 00:17:52 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 5 Jul 2005 00:17:52 +0200 Subject: TCP stack throughput In-Reply-To: <17097.43859.831827.297323@antilipe.corelatus.se> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> Message-ID: <1C3C8872-57CC-425E-97D4-70438EAA9A4E@gmail.com> Matthias, I ran your C server and I should say that I'm very unhappy with the results. Unhappy for Erlang that is. The test runs about twice as fast and I don't get any connection resets running 1000 simultaneous connections on Mac OSX 10.4.1. Granted, I get 0 as the factorial result and the C calculation is probably way more optimized than the Erlang one but you can still connect at least 2 times as many clients as we managed to connect with Erlang on Solaris and 3 times the number I managed to connect myself. I invite everyone to run the test and experience the difference. Joel On Jul 4, 2005, at 11:34 PM, Matthias Lang wrote: > I'm with Mickael. > > Your results could be interpreted as showing one _or more_ of the > following: > > a) That your test client is the bottleneck > > b) That your test server is the bottleneck > > c) That the OS/tcp stack isn't really designed for the sort of > use you're testing. -- http://wagerlabs.com/tech From joelr1@REDACTED Tue Jul 5 01:43:20 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 5 Jul 2005 01:43:20 +0200 Subject: TCP stack throughput In-Reply-To: <2cb9022a3b07263b827f1d3732fb9b40@mac.com> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> <2cb9022a3b07263b827f1d3732fb9b40@mac.com> Message-ID: <8E882CC4-A17D-423D-A847-9FA99ECC51EE@gmail.com> I suppose I could try rewriting the gateway/load balancer in C... I think I'll look at the Erlang TCP stack code first. It's not possible for me to use persistent connections in this case as all the gateway is doing is handing out host/port pairs for clients to reconnect to. On Jul 5, 2005, at 1:05 AM, Sean Hinde wrote: > OTOH if you are the one controlling the client behaviour then you > can ensure they use persistent connections where possible. -- http://wagerlabs.com/tech From kramer@REDACTED Tue Jul 5 05:01:16 2005 From: kramer@REDACTED (Reto Kramer) Date: Mon, 4 Jul 2005 20:01:16 -0700 Subject: Getting locks and sharing: was RE: Getting concurrency In-Reply-To: References: Message-ID: On Jun 18, 2005, at 4:13 PM, Ulf Wiger wrote: > Den 2005-06-17 09:38:37 skrev Joe Armstrong (AL/EAB) > : > >>> The most complex way is to make the mulator multi-CPU aware, >>> with all the hassle involved. Just the distributed garbage >>> collection could probably support a couple of Ph.D. theses. >>> On the plus side, load balancing is automatic. >>> >> >> We shouldn't do this :-) > > I disagree. dito. I find it very unnatural to have to conceive of multiple nodes per machine "just" because there's no VM technology in Erlang that takes advantage of multi-CPU machines _transparently_. Where can I learn about what's been done in the past and get a sense of the Erlang specific challenges and the complexity of such a multi CPU aware VM? Did the HiPE group attempt this in the past? Thanks, - Reto From luvishsatija@REDACTED Tue Jul 5 07:36:03 2005 From: luvishsatija@REDACTED (luvish satija) Date: Tue, 5 Jul 2005 06:36:03 +0100 (BST) Subject: ssl setup problem (contd.) Message-ID: <20050705053603.95532.qmail@web8502.mail.in.yahoo.com> hello all, I explained in my last few mails about my endeavour to set up ssl for erlang ( R10B and ssl ver. 3.0.5) on linux. But still I have reached nowhere. So, I have tried to set up ssl on an old version of erlang ( R8B and ssl ver 2.3.3). When i start erl shell with suitable flags then the following is the scenario: (the flags are -proto_dist, -boot, -ssl_dist_opt client_certfile, -ssl_dist_opt server_certfile, with proper arguements) Erlang (BEAM) emulator version 5.1 [source] Eshell V5.1 (abort with ^G) 1> whereis(ssl_server). <0.32.0> >>>>> this shows that ssl server is started. 2> {ok, Socket}=ssl:listen(4000,[binary]). {ok,{sslsocket,4,<0.37.0>}} >>>>>> this shows that ssl socket is created at port 4000 3> ssl:accept(Socket). {error,ebadf} >>>>>> but why this error is coming when i'm using sslsocket type. Do i need to do something else other than these obvious steps ( i'm going on the same lines as that of gen_tcp)? I apologize for such frequent posts about my doubts, but I am trying to work up this ssl thing from last week without any good results. thanks in advance /luvish _______________________________________________________ Too much spam in your inbox? Yahoo! Mail gives you the best spam protection for FREE! http://in.mail.yahoo.com From matthias@REDACTED Tue Jul 5 08:12:28 2005 From: matthias@REDACTED (Matthias Lang) Date: Tue, 5 Jul 2005 08:12:28 +0200 Subject: Getting locks and sharing: was RE: Getting concurrency In-Reply-To: References: Message-ID: <17098.9420.766553.530941@antilipe.corelatus.se> Reto Kramer writes: > Where can I learn about what's been done in the past and get a sense of > the Erlang specific challenges and the complexity of such a multi CPU > aware VM? Did the HiPE group attempt this in the past? Pekka Hedqvist and Tony Rogvall (not the HIPE group) implemented exactly such a beast for the (now defunct) JAM VM. I can't find the report online, but here's a summary from memory: - We implemented a multithreaded erlang VM by hacking the JAM VM. - On a four-CPU machine, it got an almost linear speedup on code which wasn't entirely contrived - Our VM modifications got to the "99% complete" stage, but then the BEAM VM came out and we felt ill at the thought of having to do it all again. Suddenly, other problems looked much more interesting. Matthias From ke.han@REDACTED Tue Jul 5 08:26:48 2005 From: ke.han@REDACTED (ke.han) Date: Tue, 05 Jul 2005 14:26:48 +0800 Subject: TCP stack throughput In-Reply-To: <8E882CC4-A17D-423D-A847-9FA99ECC51EE@gmail.com> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> <2cb9022a3b07263b827f1d3732fb9b40@mac.com> <8E882CC4-A17D-423D-A847-9FA99ECC51EE@gmail.com> Message-ID: <42CA2828.6090403@redstarling.com> Joel Reymont wrote: > It's not possible for me to use persistent connections in this case as > all the gateway is doing is handing out host/port pairs for clients to > reconnect to. So why are you using TCP for this problem? UDP / RUDP seems more appropriate. You can even encrypt the UDP payload if that's necessary. ke han From joelr1@REDACTED Tue Jul 5 08:31:12 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 5 Jul 2005 08:31:12 +0200 Subject: Erlang in financial modelling Message-ID: <06167DDA-3B65-4643-AF7B-65E798369134@gmail.com> Folks, How feasible it is to use Erlang for computationally-heavy financial modeling? Neural networks, genetic, algorithms, etc. Has anyone done testing against C++, C#, Java, etc.? Thanks, Joel -- http://wagerlabs.com/tech From erlang@REDACTED Tue Jul 5 08:34:49 2005 From: erlang@REDACTED (Michael McDaniel) Date: Mon, 4 Jul 2005 23:34:49 -0700 Subject: SSL Setup Problem In-Reply-To: <20050704111354.75272.qmail@web8502.mail.in.yahoo.com> References: <20050704100703.GS11207@fangora.autosys.us> <20050704111354.75272.qmail@web8502.mail.in.yahoo.com> Message-ID: <20050705063448.GU11207@fangora.autosys.us> On Mon, Jul 04, 2005 at 12:13:54PM +0100, luvish satija wrote: > Hi Michael, > The symbolic link did the trick. Now i'm able to get > the boot script without any warnings. > But still no ssl application is working. To test it > actually, i simply gave ssl:listen() in the command > prompt and was welcomed by the following errors: > > > root@REDACTED:~/lvs# erl -boot > /home/maniyan/lvs/start_ssl -proto_dist inet_ssl > -ssl_dist_opt client_certfile > "/home/maniyan/lvs/ssl_server.pem" -ssl_dist_opt > server_certfile "/home/maniyan/lvs/ssl_server.pem" > -ssl_dist_opt verify 1 depth 1 > Erlang (BEAM) emulator version 5.4.6 [source] [hipe] > > Eshell V5.4.6 (abort with ^G) > 1> ssl:listen(4000,[binary]). > ** exited: {normal,{gen_server,call, > [ssl_server, > > {listen,<0.37.0>,{0,0,0,0},4000,[],5}, > infinity]}} ** > > =ERROR REPORT==== 4-Jul-2005::16:42:38 === > ** Generic server <0.37.0> terminating > ** Last message in was {listen,<0.35.0>,4000,[binary]} > ** When Server state == {st,listener, > <0.31.0>, > <0.35.0>, > <0.35.0>, > nil, > true, > [], > nil, > nil, > nil, > false} > ** Reason for termination == > ** {normal,{gen_server,call, > [ssl_server, > > {listen,<0.37.0>,{0,0,0,0},4000,[],5}, > infinity]}} > > Now I don't know what does it mean and why it is > comping because i'm doing all the things according to > the manual. If you have any clue about this, or if you > can suggest some other way, then it would really help > me. > Thanks, > /luvish _______________________________________________________ I get same error on my version 5.4.4 system which runs ssl. ~Michael From joelr1@REDACTED Tue Jul 5 08:37:22 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 5 Jul 2005 08:37:22 +0200 Subject: TCP stack throughput In-Reply-To: <42CA2828.6090403@redstarling.com> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> <2cb9022a3b07263b827f1d3732fb9b40@mac.com> <8E882CC4-A17D-423D-A847-9FA99ECC51EE@gmail.com> <42CA2828.6090403@redstarling.com> Message-ID: <5BAC66D4-C8F4-4AE8-AF23-828777CB0B40@gmail.com> I would take UDP over TCP any day, except for the need to make it reliable. This would also require me to implement something like RUDP on the client side and I don't look forward to doing it in Basic. Maybe I'll have to someday but I'm stalling as much as possible :-). On Jul 5, 2005, at 8:26 AM, ke.han wrote: > So why are you using TCP for this problem? UDP / RUDP seems more > appropriate. You can even encrypt the UDP payload if that's > necessary. -- http://wagerlabs.com/tech From chandrashekhar.mullaparthi@REDACTED Tue Jul 5 10:07:15 2005 From: chandrashekhar.mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Tue, 5 Jul 2005 09:07:15 +0100 Subject: ssl setup problem (contd.) In-Reply-To: <20050705053603.95532.qmail@web8502.mail.in.yahoo.com> References: <20050705053603.95532.qmail@web8502.mail.in.yahoo.com> Message-ID: Hi, I don't have a solution for your problem but I can suggest an alternative. Unless you can really think of a reason to have SSL handling within the erlang node (like getting access to client certificates), consider using stunnel which is available from http://stunnel.org You get several advantages doing it this way. - SSL handling is done in separate UNIX process so you can take advantage of a multi-CPU machine if you have one. - stunnel is quite mature and works pretty well. I haven't used it under a heavy load but there are other systems in our network which use this for pretty heavy duty stuff. One down side is that you then can't fit it into the OTP release framework. A way around it is to write a wrapper application which launches stunnel and monitors it. cheers Chandru On 5 Jul 2005, at 06:36, luvish satija wrote: > hello all, > I explained in my last few mails about my endeavour > to set up ssl for erlang ( R10B and ssl ver. 3.0.5) > on linux. But still I have reached nowhere. > So, I have tried to set up ssl on an old version of > erlang ( R8B and ssl ver 2.3.3). When i start erl > shell with suitable flags then the following is the > scenario: > (the flags are -proto_dist, -boot, -ssl_dist_opt > client_certfile, -ssl_dist_opt server_certfile, with > proper arguements) > > > Erlang (BEAM) emulator version 5.1 [source] > > Eshell V5.1 (abort with ^G) > 1> whereis(ssl_server). > <0.32.0> >>>>>> this shows that ssl server is started. > > 2> {ok, Socket}=ssl:listen(4000,[binary]). > {ok,{sslsocket,4,<0.37.0>}} > >>>>>>> this shows that ssl socket is created at port > 4000 > 3> ssl:accept(Socket). > {error,ebadf} > >>>>>>> but why this error is coming when i'm using > sslsocket type. > > Do i need to do something else other than these > obvious steps ( i'm going on the same lines as that of > gen_tcp)? I apologize for such frequent posts about my > doubts, but I am trying to work up this ssl thing from > last week without any good results. > thanks in advance > /luvish > > > > > > _______________________________________________________ > Too much spam in your inbox? Yahoo! Mail gives you the best spam > protection for FREE! http://in.mail.yahoo.com > From ulf.wiger@REDACTED Tue Jul 5 10:28:00 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Tue, 5 Jul 2005 10:28:00 +0200 Subject: Getting locks and sharing: was RE: Getting concurrency Message-ID: Matthias Lang wrote: > > Reto Kramer writes: > > > Where can I learn about what's been done in the past and > get a sense of > > the Erlang specific challenges and the complexity of such > a multi CPU > > aware VM? Did the HiPE group attempt this in the past? > > Pekka Hedqvist and Tony Rogvall (not the HIPE group) implemented > exactly such a beast for the (now defunct) JAM VM. I can't find the > report online, but here's a summary from memory: http://www.erlang.se/publications/xjobb/0089-hedqvist.pdf /Uffe From tony@REDACTED Tue Jul 5 11:02:21 2005 From: tony@REDACTED (Tony Rogvall) Date: Tue, 5 Jul 2005 11:02:21 +0200 Subject: TCP stack throughput In-Reply-To: <1C3C8872-57CC-425E-97D4-70438EAA9A4E@gmail.com> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> <1C3C8872-57CC-425E-97D4-70438EAA9A4E@gmail.com> Message-ID: 5 jul 2005 kl. 00.17 skrev Joel Reymont: > Matthias, > > I ran your C server and I should say that I'm very unhappy with the > results. Unhappy for Erlang that is. The test runs about twice as > fast and I don't get any connection resets running 1000 > simultaneous connections on Mac OSX 10.4.1. > > Granted, I get 0 as the factorial result and the C calculation is > probably way more optimized than the Erlang one but you can still > connect at least 2 times as many clients as we managed to connect > with Erlang on Solaris and 3 times the number I managed to connect > myself. > > I invite everyone to run the test and experience the difference. > There is a difference between the C-server and the Erlang-server and that is that the C-server is closing the socket after the calculation while the Erlang- server is waiting for the client to close. This will probably tie up resources for longer time than needed. Regards /Tony From carsten@REDACTED Tue Jul 5 11:04:55 2005 From: carsten@REDACTED (Carsten Schultz) Date: Tue, 05 Jul 2005 11:04:55 +0200 Subject: Getting locks and sharing In-Reply-To: <17098.9420.766553.530941@antilipe.corelatus.se> References: <17098.9420.766553.530941@antilipe.corelatus.se> Message-ID: <42CA4D37.8060409@codimi.de> Matthias Lang schrieb: > Reto Kramer writes: > > > Where can I learn about what's been done in the past and get a sense of > > the Erlang specific challenges and the complexity of such a multi CPU > > aware VM? Did the HiPE group attempt this in the past? > > Pekka Hedqvist and Tony Rogvall (not the HIPE group) implemented > exactly such a beast for the (now defunct) JAM VM. I can't find the > report online, but here's a summary from memory: BTW, I have recently read that GHC people are working on a multi CPU version. Just imagine it being easier to write an application that takes advantage of multiple CPUs in Haskell than in Erlang. Does not this motivate someone? :-) Take care, Carsten -- Carsten Schultz (2:38, 33:47) http://carsten.codimi.de/ PGP/GPG key on the pgp.net key servers, fingerprint on my home page. From mikael.karlsson@REDACTED Tue Jul 5 11:18:29 2005 From: mikael.karlsson@REDACTED (Mikael Karlsson) Date: Tue, 5 Jul 2005 11:18:29 +0200 Subject: Web interface framework for Erlang In-Reply-To: <42C8F56F.7030500@erlang-consulting.com> References: <42C8F56F.7030500@erlang-consulting.com> Message-ID: <200507051118.30227.mikael.karlsson@creado.com> I looked around a bit more on the Erlang list and found the attachement ws.tar.gz in posting: http://www.erlang.org/ml-archive/erlang-questions/200502/msg00289.html Extract from file ws/stl/doc/manual.tex: -- WSTL - A framework for web application development} author: Micka?l R?mond - mickael.remond@REDACTED Introduction WSTL is a framework developped by Vladimir Sekissov. It is based on a component approach and provide all the needed features to develop consistent and maintenable Web Application: - Component based development model - Templating system: STL - Integrated database: Mnesia - Connection pooling for relational database Access (Through ODBC)} WSTL has been developed with the Erlang programming language and uses Yaws as its web server. .... ------- Does anybody now the status of this? Mikael m?ndag 04 juli 2005 10:38 skrev Simon Aurell: > Tee: Struts is definitely one of the frameworks that I've come in > contact with before which could serve as a model. However, the whole > idea is to create something for Erlang. > > As Mikael mentioned, Yaws is a good idea and is probably what we will > build it on. I've only got a vague idea of what J-EAI is. How would you > say it could be used in building in this framework? I've heard lots > about Ruby on Rails. Will definitely look into it. > > What would you say would be the most important features in a first > prototype? > > Simon > From joelr1@REDACTED Tue Jul 5 11:21:49 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 5 Jul 2005 11:21:49 +0200 Subject: TCP stack throughput In-Reply-To: References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> <1C3C8872-57CC-425E-97D4-70438EAA9A4E@gmail.com> Message-ID: <94E4F547-C0BF-45C3-95A2-05238FF2CB98@gmail.com> Tony, I made the server close the connection right after sending but it did not make much of a difference. On Mac OSX 200 out of 500 connection requests are refused. With the C server 1,000 connections are accepted easily and within a second or two. With Erlang the most we got is 500 connections on Solaris and that's with a large connection timeout. Joel On Jul 5, 2005, at 11:02 AM, Tony Rogvall wrote: > > There is a difference between the C-server and the Erlang-server > and that is that the > C-server is closing the socket after the calculation while the > Erlang-server is waiting > for the client to close. This will probably tie up resources for > longer time than needed. -- http://wagerlabs.com/tech From mickael.remond@REDACTED Tue Jul 5 11:44:52 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Tue, 05 Jul 2005 11:44:52 +0200 Subject: Web interface framework for Erlang In-Reply-To: <200507051118.30227.mikael.karlsson@creado.com> References: <42C8F56F.7030500@erlang-consulting.com> <200507051118.30227.mikael.karlsson@creado.com> Message-ID: <42CA5694.8000608@erlang-fr.org> Mikael Karlsson wrote: > Does anybody now the status of this? Hello, I am not sure Vladimir is still developping it. I played with it a lot when it was under development. The framework has a very good potential. It is still not documented enough but, I think it might be a good basis for further works. -- Micka?l R?mond From mickael.remond@REDACTED Tue Jul 5 11:50:21 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Tue, 05 Jul 2005 11:50:21 +0200 Subject: Web interface framework for Erlang and J-EAI In-Reply-To: <42C8F56F.7030500@erlang-consulting.com> References: <42C8F56F.7030500@erlang-consulting.com> Message-ID: <42CA57DD.30609@erlang-fr.org> Simon Aurell wrote: > As Mikael mentioned, Yaws is a good idea and is probably what we will > build it on. I've only got a vague idea of what J-EAI is. How would you > say it could be used in building in this framework? Very simple: The web is slowly moving to some kind of "connected" protocol to replace HTTP. This is what has been called AJAX, but we are only at the beginning of the movement. We have "web pages" that basically are client that refresh themselves while being connected to a back-end server. J-EAI (and more generally XMPP) can handled this connected part very nicely. This is a bus that can distribute XML stream to connected client (or put some info in message queue if needed). As a transition, xe can use standard HTTP to distribute the web pages application (Javascript). They can then connect to a J-EAI bus to get and send realtime event from the back-end server and refresh its view. This is very roughly the idea and how you can use J-EAI buses to serve Web based applications. I hope this helps, -- Micka?l R?mond From ulf.wiger@REDACTED Tue Jul 5 11:50:53 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Tue, 5 Jul 2005 11:50:53 +0200 Subject: TCP stack throughput Message-ID: Well, actually 900 connections on Solaris, with a 60 sec timeout. That's still not very impressive. I think one big difference between the erlang implementation and the c program is that the c program does blocking i/o. I tried rewriting the tcp_server so that it handles all connections in one process, one at a time. That didn't help much. In fact, it got slightly worse. Perhaps not too surprising... One could also try to use the basic socket interface, rather than gen_tcp. I tried simulating that by removing one of the io:formats (;-)), but that made little difference. /Uffe > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Joel Reymont > Sent: den 5 juli 2005 11:22 > To: Erlang Users' List > Subject: Re: TCP stack throughput > > > Tony, > > I made the server close the connection right after sending > but it did > not make much of a difference. On Mac OSX 200 out of 500 connection > requests are refused. > > With the C server 1,000 connections are accepted easily and within a > second or two. With Erlang the most we got is 500 connections on > Solaris and that's with a large connection timeout. > > Joel > > On Jul 5, 2005, at 11:02 AM, Tony Rogvall wrote: > > > > There is a difference between the C-server and the Erlang-server > > and that is that the > > C-server is closing the socket after the calculation while the > > Erlang-server is waiting > > for the client to close. This will probably tie up resources for > > longer time than needed. > > -- > http://wagerlabs.com/tech > > > From Marc.Vanwoerkom@REDACTED Tue Jul 5 12:12:18 2005 From: Marc.Vanwoerkom@REDACTED (Marc van Woerkom) Date: Tue, 05 Jul 2005 12:12:18 +0200 Subject: Fwd: Web interface framework for Erlang and J-EAI Message-ID: --- die weitergeleitete Nachricht folgt --- -------------- next part -------------- An embedded message was scrubbed... From: "Marc van Woerkom" Subject: Re: Web interface framework for Erlang and J-EAI Date: Tue, 05 Jul 2005 12:11:30 +0200 Size: 2239 URL: From svg@REDACTED Tue Jul 5 12:14:29 2005 From: svg@REDACTED (Vladimir Sekissov) Date: Tue, 05 Jul 2005 16:14:29 +0600 (YEKST) Subject: Web interface framework for Erlang In-Reply-To: <200507051118.30227.mikael.karlsson@creado.com> References: <42C8F56F.7030500@erlang-consulting.com> <200507051118.30227.mikael.karlsson@creado.com> Message-ID: <20050705.161429.73661648.svg@surnet.ru> Good day, mikael.karlsson> I looked around a bit more on the Erlang list and found mikael.karlsson> the attachement ws.tar.gz in posting: mikael.karlsson> mikael.karlsson> http://www.erlang.org/ml-archive/erlang-questions/200502/msg00289.html mikael.karlsson> mikael.karlsson> Extract from file ws/stl/doc/manual.tex: ... mikael.karlsson> Does anybody now the status of this? I haven't time for a year to bring the code to the state when it could be usable by others except me. But if somebody is interesting in it I try to package all stuff with examples at weekends. Best Regards, Vladimir Sekissov From mickael.remond@REDACTED Tue Jul 5 12:48:31 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Tue, 05 Jul 2005 12:48:31 +0200 Subject: Fwd: Web interface framework for Erlang and J-EAI In-Reply-To: References: Message-ID: <42CA657F.5070109@erlang-fr.org> Marc van Woerkom wrote: > The central idea of those AJAX applications IMHO is not > getting rid of HTTP. I fully agrees. They tends toward a connected protocol, but for now there is no standard for this connected protocol. I think that XMPP would be perfect for this task and J-EAI (backed with ejabberd), being very scalable is a very strong brick in this movement. -- Micka?l R?mond http://www.erlang-projects.org/ http://www.3pblog.net/ From thomasl_erlang@REDACTED Tue Jul 5 12:51:24 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 5 Jul 2005 03:51:24 -0700 (PDT) Subject: Getting locks and sharing: was RE: Getting concurrency In-Reply-To: Message-ID: <20050705105124.62994.qmail@web34408.mail.mud.yahoo.com> --- Reto Kramer wrote: > I find it very unnatural to have to conceive of > multiple nodes per > machine "just" because there's no VM technology in > Erlang that takes > advantage of multi-CPU machines _transparently_. Note that an approach of using multiple sequential nodes instead of a multithreaded node could be fairly transparent to the programmer, with the appropriate high-level support. Also note that it's not clear how transparent the issues of parallelism _should_ be, in either approach. > Where can I learn about what's been done in the past > and get a sense of > the Erlang specific challenges and the complexity of > such a multi CPU > aware VM? Have a look at Pekka's MSc, or dig your way through the endless ranks of papers on implicit parallelization from the 80s to the early 90s :-) I'd say the first issue for a multithreaded VM is to make the runtime system thread safe (and robust). The second issue is to make it scalable. The third issue is to make it competitive with a sequential VM. (And somewhere in that list, we can begin worrying about how well our whole erlang systems perform :-) Best, Thomas __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From joelr1@REDACTED Tue Jul 5 13:18:02 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 5 Jul 2005 13:18:02 +0200 Subject: Strange Erlang crash Message-ID: I'm on Mac OSX 10.4.1. My erl crashes periodically on startup. In this particular case it crashed twice before starting up on the third time. {error_logger,{{2005,7,5},{13,16,9}},supervisor_report,[{supervisor, {local,kernel_sup}},{errorContext,start_error},{reason, {'DOWN',#Ref<0.0.0.12>,process,<0.15.0>,normal}},{offender, [{pid,undefined},{name,code_server},{mfa,{code,start_link,[]}}, {restart_type,permanent},{shutdown,2000},{child_type,worker}]}]} {error_logger,{{2005,7,5},{13,16,9}},crash_report,[[{pid,<0.7.0>}, {registered_name,[]},{error_info,{shutdown,{kernel,start,[normal, []]}}},{initial_call,{application_master,init,[<0.5.0>,<0.6.0>, {appl_data,kernel, [application_controller,erl_reply,auth,boot_server,code_server,disk_log_ server,disk_log_sup,erl_prim_loader,error_logger,file_server,file_server _2,fixtable_server,global_group,global_name_server,heart,init,kernel_con fig,kernel_sup,net_kernel,net_sup,rex,user,os_server,ddll_server,erl_epm d,inet_db,pg2],undefined,{kernel,[]}, [application,application_controller,application_master,application_start er,auth,code,code_aux,packages,code_server,dist_util,erl_boot_server,erl _distribution,erl_prim_loader,erl_reply,erlang,error_handler,error_logge r,file,file_server,old_file_server,file_io_server,prim_file,global,globa l_group,global_search,group,heart,hipe_unified_loader,hipe_sparc_loader, hipe_ppc_loader,hipe_ppc64_loader,hipe_x86_loader,hipe_amd64_loader,inet 6_tcp,inet6_tcp_dist,inet6_udp,inet_config,inet_hosts,inet_gethost_nativ e,inet_tcp_dist,init,kernel,kernel_config,net,net_adm,net_kernel,os,ram_ file,rpc,user,user_drv,user_sup,disk_log,disk_log_1,disk_log_server,disk _log_sup,dist_ac,erl_ddll,erl_epmd,erts_debug,gen_tcp,gen_udp,prim_inet, inet,inet_db,inet_dns,inet_parse,inet_res,inet_tcp,inet_udp,pg2,seq_trac e,wrap_log_reader,zlib,otp_ring0],[],infinity,infinity},normal]}}, {ancestors,[<0.6.0>]},{messages,[{'EXIT',<0.8.0>,normal}]},{links, [<0.6.0>,<0.5.0>]},{dictionary,[]},{trap_exit,true},{status,running}, {heap_size,987},{stack_size,21},{reductions,1041}],[]]} {error_logger,{{2005,7,5},{13,16,9}},std_info,[{application,kernel}, {exited,{shutdown,{kernel,start,[normal,[]]}}},{type,permanent}]} {"Kernel pid terminated",application_controller,shutdown} Crash dump was written to: erl_crash.dump Kernel pid terminated (application_controller) (shutdown) c-fd82e055:~/work/code-dump joelr$ erl Erlang (BEAM) emulator version 5.4.8 [source] [hipe] [threads:0] Eshell V5.4.8 (abort with ^G) 1> -- http://wagerlabs.com/tech From thomasl_erlang@REDACTED Tue Jul 5 13:30:59 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 5 Jul 2005 04:30:59 -0700 (PDT) Subject: Getting locks and sharing In-Reply-To: <42CA4D37.8060409@codimi.de> Message-ID: <20050705113059.94874.qmail@web34403.mail.mud.yahoo.com> --- Carsten Schultz wrote: > Just imagine it being easier to write an > application that > takes advantage of multiple CPUs in Haskell than in > Erlang. Does not > this motivate someone? :-) The winner of the ICFP contest a couple of years ago exploited 16 workstations and was written in ... C++ :-) Best, Thomas ____________________________________________________ Yahoo! Sports Rekindle the Rivalries. Sign up for Fantasy Football http://football.fantasysports.yahoo.com From thomasl_erlang@REDACTED Tue Jul 5 13:31:06 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 5 Jul 2005 04:31:06 -0700 (PDT) Subject: Getting locks and sharing In-Reply-To: <42CA4D37.8060409@codimi.de> Message-ID: <20050705113106.94914.qmail@web34403.mail.mud.yahoo.com> --- Carsten Schultz wrote: > Just imagine it being easier to write an > application that > takes advantage of multiple CPUs in Haskell than in > Erlang. Does not > this motivate someone? :-) The winner of the ICFP contest a couple of years ago exploited 16 workstations and was written in ... C++ :-) Best, Thomas ____________________________________________________ Yahoo! Sports Rekindle the Rivalries. Sign up for Fantasy Football http://football.fantasysports.yahoo.com From ulf.wiger@REDACTED Tue Jul 5 14:34:43 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Tue, 5 Jul 2005 14:34:43 +0200 Subject: TCP stack throughput Message-ID: I rewrote the tcp_server module slightly (attached), and ran the server and client in different shells to better observe what's happening. I tried setting the acceptor process to high priority until the accept() call returned, then setting it to normal prio, yielding, and then calling the handler. That didn't help. The delay is not due to scheduling delay caused by processing backlog in erlang. I then wrapped the inet:accept() call with a timer:tc() and logged the result in an ets table. After trying 1000 clients, I checked the ets table, and found that the longest accept() call took 43 seconds. That's running one acceptor at a time. See below. I also noticed that the accept calls seem to go quickly in batches of 100-200, then followed by an increasingly long delay. A possibility is perhaps that there are retransmissions at the TCP level. But previous runs with tcpdump gave no such indications. /Uffe start_acceptor(S, Fun, T) -> Parent = self(), spawn_opt( fun() -> case timer:tc(inet_tcp, accept, [S]) of {Time, {ok, Sock}} -> ets:insert(stats, {self(), Time}), Parent ! accepted, process_flag(priority, normal), erlang:yield(), receive {tcp, Sock, Msg} -> Prev = ets:update_counter(T, total, 1), io:format("<~p> ", [Prev]), Fun(Msg, Sock), inet_tcp:close(Sock); Msg -> io:format("Msg = ~p~n", [Msg]) end; Error -> erlang:fault(Error) end end, [link, {priority, high}]), receive accepted -> start_acceptor(S, Fun, T) end. 4> io:format("~p~n", [v(2)]). [{<0.40.0>,43316529}, {<0.723.0>,26158659}, {<0.627.0>,12659283}, {<0.506.0>,5538478}, {<0.396.0>,1964537}, {<0.567.0>,419001}, {<0.667.0>,418777}, {<0.436.0>,397587}, {<0.511.0>,369515}, {<0.404.0>,368789}, {<0.691.0>,188574}, {<0.595.0>,186015}, {<0.468.0>,178119}, {<0.683.0>,127997}, {<0.452.0>,119223}, {<0.583.0>,118169}, {<0.150.0>,117279}, {<0.140.0>,103751}, {<0.356.0>,76253}, {<0.252.0>,58598}, {<0.364.0>,52827}, {<0.428.0>,50812}, {<0.302.0>,45687}, {<0.199.0>,39077}, {<0.77.0>,35216}, {<0.244.0>,34073}, {<0.148.0>,31787}, {<0.388.0>,26835}, {<0.643.0>,20185}, {<0.41.0>,19363}, {<0.420.0>,18621}, {<0.294.0>,18435}, {<0.88.0>,17394}, {<0.635.0>,17230}, {<0.49.0>,14304}, {<0.587.0>,12749}, {<0.50.0>,11834}, {<0.651.0>,10356}, {<0.551.0>,9884}, {<0.699.0>,9562}, {<0.474.0>,9471}, {<0.87.0>,9411}, {<0.707.0>,9321}, {<0.675.0>,9027}, {<0.743.0>,9001}, {<0.751.0>,8918}, {<0.365.0>,8869}, {<0.535.0>,8674}, {<0.490.0>,8666}, {<0.767.0>,8644}, {<0.93.0>,8605}, {<0.611.0>,8598}, {<0.775.0>,8582}, {<0.498.0>,8577}, {<0.619.0>,8553}, {<0.543.0>,8533}, {<0.527.0>,8479}, {<0.482.0>,8372}, {<0.519.0>,8369}, {<0.366.0>,8359}, {<0.412.0>,8350}, {<0.57.0>,8309}, {<0.92.0>,8292}, {<0.575.0>,8191}, {<0.309.0>,8184}, {<0.759.0>,8147}, {<0.603.0>,8135}, {<0.368.0>,8120}, {<0.85.0>,8089}, {<0.460.0>,8014}, {<0.151.0>,8003}, {<0.735.0>,8001}, {<0.52.0>,7991}, {<0.56.0>,7980}, {<0.86.0>,7974}, {<0.303.0>,7971}, {<0.94.0>,7960}, {<0.367.0>,7922}, {<0.51.0>,7920}, {<0.444.0>,7904}, {<0.78.0>,7900}, {<0.369.0>,7876}, {<0.152.0>,7872}, {<0.58.0>,7870}, {<0.89.0>,7869}, {<0.311.0>,7862}, {<0.83.0>,7854}, {<0.153.0>,7844}, {<0.304.0>,7842}, {<0.156.0>,7832}, {<0.310.0>,7831}, {<0.155.0>,7830}, {<0.305.0>,7829}, {<0.308.0>,7821}, {<0.81.0>,7818}, {<0.90.0>,7817}, {<0.149.0>,7810}, {<0.91.0>,7804}, {<0.55.0>,7799}, {<0.53.0>,7798}, {<0.80.0>,7790}, {<0.306.0>,7783}, {<0.82.0>,7770}, {<0.312.0>,7760}, {<0.157.0>,7750}, {<0.154.0>,7748}, {<0.307.0>,7744}, {<0.159.0>,7742}, {<0.95.0>,7740}, {<0.59.0>,7733}, {<0.158.0>,7724}, {<0.161.0>,7715}, {<0.79.0>,7700}, {<0.160.0>,7678}, {<0.54.0>,7677}, {<0.84.0>,7676}, {<0.559.0>,7370}, {<0.715.0>,6912}, {<0.659.0>,5917}, {<0.370.0>,5507}, {<0.162.0>,4666}, {<0.313.0>,2693}, {<0.176.0>,2637}, {<0.69.0>,2596}, {<0.124.0>,2576}, {<0.321.0>,2560}, {<0.200.0>,2302}, {<0.270.0>,2296}, {<0.96.0>,2240}, {<0.380.0>,2234}, {<0.392.0>,2198}, {<0.60.0>,2190}, {<0.211.0>,2128}, {<0.100.0>,2048}, {<0.333.0>,2019}, {<0.282.0>,1993}, {<0.112.0>,1988}, {<0.223.0>,1983}, {<0.345.0>,1959}, {<0.163.0>,1944}, {<0.235.0>,1939}, {<0.371.0>,1930}, {<0.136.0>,1923}, {<0.201.0>,1809}, {<0.389.0>,1777}, {<0.314.0>,1767}, {<0.187.0>,1748}, {<0.61.0>,1734}, {<0.97.0>,1731}, {<0.167.0>,1616}, {<0.209.0>,1486}, {<0.258.0>,1277}, {<0.253.0>,1270}, {<0.114.0>,1152}, {<0.179.0>,1149}, {<0.390.0>,1140}, {<0.113.0>,1088}, {<0.170.0>,1050}, {<0.236.0>,1035}, {<0.378.0>,1034}, {<0.281.0>,1030}, {<0.70.0>,1029}, {<0.164.0>,1023}, {<0.72.0>,1017}, {<0.177.0>,1016}, {<0.217.0>,1014}, {<0.125.0>,1009}, {<0.212.0>,1007}, {<0.322.0>,1006}, {<0.101.0>,999}, {<0.188.0>,999}, {<0.268.0>,998}, {<0.73.0>,997}, {<0.271.0>,997}, {<0.62.0>,992}, {<0.254.0>,991}, {<0.393.0>,989}, {<0.372.0>,986}, {<0.168.0>,985}, {<0.381.0>,983}, {<0.331.0>,982}, {<0.210.0>,981}, {<0.315.0>,980}, {<0.74.0>,979}, {<0.283.0>,976}, {<0.224.0>,975}, {<0.137.0>,974}, {<0.98.0>,974}, {<0.132.0>,971}, {<0.261.0>,970}, {<0.259.0>,970}, {<0.273.0>,963}, {<0.391.0>,961}, {<0.99.0>,960}, {<0.63.0>,959}, {<0.202.0>,959}, {<0.76.0>,958}, {<0.334.0>,958}, {<0.65.0>,956}, {<0.166.0>,956}, {<0.71.0>,954}, {<0.67.0>,952}, {<0.165.0>,951}, {<0.255.0>,951}, {<0.64.0>,950}, {<0.104.0>,949}, {<0.110.0>,949}, {<0.115.0>,949}, {<0.257.0>,948}, {<0.178.0>,948}, {<0.341.0>,947}, {<0.66.0>,947}, {<0.387.0>,947}, {<0.75.0>,946}, {<0.227.0>,945}, {<0.174.0>,945}, {<0.256.0>,945}, {<0.204.0>,945}, {<0.171.0>,944}, {<0.316.0>,943}, {<0.68.0>,941}, {<0.172.0>,940}, {<0.346.0>,939}, {<0.260.0>,938}, {<0.373.0>,937}, {<0.105.0>,937}, {<0.323.0>,936}, {<0.218.0>,936}, {<0.280.0>,936}, {<0.317.0>,936}, {<0.169.0>,935}, {<0.126.0>,935}, {<0.133.0>,935}, {<0.102.0>,934}, {<0.103.0>,934}, {<0.213.0>,934}, {<0.111.0>,934}, {<0.214.0>,934}, {<0.266.0>,933}, {<0.109.0>,933}, {<0.375.0>,933}, {<0.265.0>,933}, {<0.377.0>,932}, {<0.262.0>,932}, {<0.263.0>,932}, {<0.206.0>,932}, {<0.264.0>,932}, {<0.379.0>,932}, {<0.269.0>,931}, {<0.237.0>,931}, {<0.173.0>,930}, {<0.290.0>,930}, {<0.203.0>,930}, {<0.189.0>,929}, {<0.180.0>,929}, {<0.208.0>,929}, {<0.216.0>,927}, {<0.318.0>,926}, {<0.123.0>,925}, {<0.107.0>,925}, {<0.319.0>,925}, {<0.175.0>,924}, {<0.267.0>,924}, {<0.374.0>,923}, {<0.119.0>,923}, {<0.274.0>,923}, {<0.116.0>,923}, {<0.207.0>,922}, {<0.350.0>,922}, {<0.293.0>,922}, {<0.376.0>,922}, {<0.106.0>,921}, {<0.205.0>,921}, {<0.320.0>,921}, {<0.198.0>,921}, {<0.272.0>,920}, {<0.186.0>,920}, {<0.182.0>,920}, {<0.120.0>,919}, {<0.215.0>,919}, {<0.222.0>,919}, {<0.121.0>,919}, {<0.194.0>,919}, {<0.117.0>,919}, {<0.135.0>,919}, {<0.108.0>,919}, {<0.190.0>,918}, {<0.230.0>,918}, {<0.228.0>,918}, {<0.291.0>,917}, {<0.278.0>,917}, {<0.192.0>,917}, {<0.284.0>,917}, {<0.127.0>,916}, {<0.240.0>,916}, {<0.122.0>,916}, {<0.184.0>,916}, {<0.279.0>,916}, {<0.226.0>,916}, {<0.332.0>,915}, {<0.385.0>,915}, {<0.220.0>,915}, {<0.138.0>,914}, {<0.219.0>,914}, {<0.234.0>,914}, {<0.275.0>,913}, {<0.118.0>,913}, {<0.292.0>,912}, {<0.325.0>,912}, {<0.394.0>,912}, {<0.383.0>,912}, {<0.196.0>,912}, {<0.134.0>,911}, {<0.328.0>,911}, {<0.181.0>,911}, {<0.129.0>,911}, {<0.276.0>,911}, {<0.395.0>,911}, {<0.139.0>,911}, {<0.335.0>,910}, {<0.131.0>,910}, {<0.351.0>,910}, {<0.225.0>,909}, {<0.238.0>,908}, {<0.324.0>,908}, {<0.329.0>,908}, {<0.183.0>,908}, {<0.221.0>,907}, {<0.197.0>,907}, {<0.193.0>,907}, {<0.242.0>,907}, {<0.277.0>,906}, {<0.195.0>,906}, {<0.327.0>,905}, {<0.326.0>,904}, {<0.353.0>,904}, {<0.342.0>,904}, {<0.337.0>,903}, {<0.232.0>,903}, {<0.191.0>,902}, {<0.343.0>,902}, {<0.229.0>,902}, {<0.288.0>,902}, {<0.239.0>,902}, {<0.285.0>,902}, {<0.289.0>,902}, {<0.347.0>,902}, {<0.128.0>,901}, {<0.352.0>,901}, {<0.338.0>,901}, {<0.386.0>,901}, {<0.384.0>,900}, {<0.243.0>,900}, {<0.382.0>,900}, {<0.185.0>,900}, {<0.339.0>,899}, {<0.336.0>,899}, {<0.286.0>,899}, {<0.340.0>,899}, {<0.348.0>,899}, {<0.354.0>,899}, {<0.349.0>,898}, {<0.231.0>,898}, {<0.355.0>,898}, {<0.233.0>,897}, {<0.330.0>,897}, {<0.241.0>,895}, {<0.287.0>,892}, {<0.130.0>,892}, {<0.344.0>,891}, {<0.45.0>,261}, {<0.628.0>,224}, {<0.728.0>,223}, {<0.507.0>,223}, {<0.453.0>,223}, {<0.397.0>,222}, {<0.596.0>,222}, {<0.636.0>,221}, {<0.437.0>,221}, {<0.588.0>,221}, {<0.692.0>,220}, {<0.584.0>,220}, {<0.141.0>,220}, {<0.684.0>,219}, {<0.512.0>,219}, {<0.469.0>,218}, {<0.568.0>,218}, {<0.245.0>,218}, {<0.644.0>,217}, {<0.668.0>,217}, {<0.429.0>,215}, {<0.357.0>,214}, {<0.405.0>,214}, {<0.716.0>,210}, {<0.421.0>,209}, {<0.660.0>,207}, {<0.475.0>,205}, {<0.42.0>,203}, {<0.413.0>,202}, {<0.652.0>,202}, {<0.604.0>,202}, {<0.520.0>,201}, {<0.445.0>,201}, {<0.536.0>,200}, {<0.461.0>,199}, {<0.676.0>,199}, {<0.576.0>,198}, {<0.295.0>,198}, {<0.552.0>,198}, {<0.483.0>,197}, {<0.700.0>,197}, {<0.752.0>,196}, {<0.462.0>,196}, {<0.499.0>,195}, {<0.528.0>,195}, {<0.491.0>,195}, {<0.620.0>,195}, {<0.560.0>,194}, {<0.612.0>,194}, {<0.544.0>,193}, {<0.744.0>,192}, {<0.736.0>,192}, {<0.46.0>,192}, {<0.760.0>,192}, {<0.708.0>,191}, {<0.776.0>,191}, {<0.358.0>,191}, {<0.597.0>,191}, {<0.508.0>,191}, {<0.142.0>,190}, {<0.768.0>,190}, {<0.631.0>,189}, {<0.556.0>,189}, {<0.569.0>,188}, {<0.246.0>,188}, {<0.693.0>,187}, {<0.513.0>,187}, {<0.669.0>,187}, {<0.438.0>,187}, {<0.296.0>,187}, {<0.629.0>,187}, {<0.589.0>,187}, {<0.717.0>,187}, {<0.43.0>,186}, {<0.585.0>,186}, {<0.470.0>,185}, {<0.685.0>,185}, {<0.406.0>,184}, {<0.637.0>,183}, {<0.430.0>,183}, {<0.454.0>,183}, {<0.476.0>,183}, {<0.645.0>,182}, {<0.398.0>,182}, {<0.422.0>,181}, {<0.661.0>,181}, {<0.605.0>,180}, {<0.729.0>,180}, {<0.146.0>,180}, {<0.359.0>,179}, {<0.143.0>,179}, {<0.247.0>,179}, {<0.545.0>,179}, {<0.529.0>,179}, {<0.521.0>,179}, {<0.577.0>,179}, {<0.509.0>,179}, {<0.630.0>,178}, {<0.250.0>,178}, {<0.360.0>,178}, {<0.248.0>,178}, {<0.48.0>,178}, {<0.446.0>,178}, {<0.570.0>,178}, {<0.553.0>,178}, {<0.730.0>,178}, {<0.537.0>,178}, {<0.47.0>,178}, {<0.361.0>,178}, {<0.144.0>,178}, {<0.363.0>,178}, {<0.670.0>,177}, {<0.414.0>,177}, {<0.492.0>,177}, {<0.471.0>,177}, {<0.561.0>,177}, {<0.455.0>,177}, {<0.653.0>,177}, {<0.251.0>,177}, {<0.613.0>,177}, {<0.145.0>,177}, {<0.147.0>,177}, {<0.297.0>,177}, {<0.249.0>,177}, {<0.709.0>,176}, {<0.44.0>,176}, {<0.777.0>,176}, {<0.591.0>,176}, {<0.638.0>,176}, {<0.472.0>,176}, {<0.594.0>,176}, {<0.586.0>,176}, {<0.362.0>,176}, {<0.298.0>,176}, {<0.514.0>,176}, {<0.441.0>,176}, {<0.761.0>,176}, {<0.745.0>,176}, {<0.593.0>,176}, {<0.677.0>,176}, {<0.694.0>,176}, {<0.599.0>,176}, {<0.632.0>,176}, {<0.592.0>,176}, {<0.301.0>,175}, {<0.517.0>,175}, {<0.687.0>,175}, {<0.686.0>,175}, {<0.300.0>,175}, {<0.431.0>,175}, {<0.598.0>,175}, {<0.600.0>,175}, {<0.484.0>,175}, {<0.571.0>,175}, {<0.478.0>,175}, {<0.500.0>,175}, {<0.439.0>,175}, {<0.457.0>,175}, {<0.456.0>,175}, {<0.672.0>,174}, {<0.732.0>,174}, {<0.510.0>,174}, {<0.572.0>,174}, {<0.695.0>,174}, {<0.769.0>,174}, {<0.701.0>,174}, {<0.477.0>,174}, {<0.590.0>,174}, {<0.621.0>,174}, {<0.671.0>,174}, {<0.554.0>,174}, {<0.688.0>,174}, {<0.515.0>,174}, {<0.440.0>,174}, {<0.473.0>,174}, {<0.458.0>,174}, {<0.690.0>,174}, {<0.647.0>,174}, {<0.633.0>,174}, {<0.299.0>,174}, {<0.442.0>,174}, {<0.402.0>,173}, {<0.698.0>,173}, {<0.663.0>,173}, {<0.696.0>,173}, {<0.689.0>,173}, {<0.399.0>,173}, {<0.737.0>,173}, {<0.718.0>,173}, {<0.601.0>,173}, {<0.459.0>,173}, {<0.639.0>,173}, {<0.719.0>,173}, {<0.426.0>,173}, {<0.516.0>,173}, {<0.662.0>,173}, {<0.602.0>,173}, {<0.731.0>,173}, {<0.753.0>,173}, {<0.409.0>,172}, {<0.424.0>,172}, {<0.407.0>,172}, {<0.733.0>,172}, {<0.634.0>,172}, {<0.640.0>,172}, {<0.558.0>,172}, {<0.518.0>,172}, {<0.608.0>,172}, {<0.463.0>,172}, {<0.573.0>,172}, {<0.646.0>,172}, {<0.674.0>,172}, {<0.654.0>,172}, {<0.734.0>,172}, {<0.427.0>,172}, {<0.673.0>,172}, {<0.423.0>,171}, {<0.425.0>,171}, {<0.524.0>,171}, {<0.650.0>,171}, {<0.443.0>,171}, {<0.546.0>,171}, {<0.666.0>,171}, {<0.607.0>,171}, {<0.408.0>,171}, {<0.481.0>,171}, {<0.433.0>,171}, {<0.664.0>,171}, {<0.574.0>,171}, {<0.641.0>,171}, {<0.720.0>,171}, {<0.432.0>,171}, {<0.757.0>,171}, {<0.410.0>,171}, {<0.522.0>,171}, {<0.578.0>,171}, {<0.434.0>,171}, {<0.401.0>,171}, {<0.400.0>,171}, {<0.467.0>,170}, {<0.501.0>,170}, {<0.606.0>,170}, {<0.403.0>,170}, {<0.755.0>,170}, {<0.655.0>,170}, {<0.754.0>,170}, {<0.411.0>,170}, {<0.480.0>,170}, {<0.697.0>,170}, {<0.615.0>,170}, {<0.495.0>,170}, {<0.582.0>,170}, {<0.642.0>,170}, {<0.448.0>,170}, {<0.756.0>,170}, {<0.648.0>,170}, {<0.464.0>,170}, {<0.562.0>,170}, {<0.526.0>,170}, {<0.532.0>,170}, {<0.623.0>,170}, {<0.494.0>,170}, {<0.649.0>,170}, {<0.487.0>,170}, {<0.721.0>,170}, {<0.555.0>,170}, {<0.614.0>,170}, {<0.557.0>,169}, {<0.502.0>,169}, {<0.746.0>,169}, {<0.711.0>,169}, {<0.493.0>,169}, {<0.665.0>,169}, {<0.505.0>,169}, {<0.435.0>,169}, {<0.710.0>,169}, {<0.722.0>,169}, {<0.763.0>,169}, {<0.579.0>,169}, {<0.762.0>,169}, {<0.530.0>,169}, {<0.523.0>,169}, {<0.486.0>,169}, {<0.488.0>,169}, {<0.616.0>,169}, {<0.771.0>,168}, {<0.542.0>,168}, {<0.622.0>,168}, {<0.765.0>,168}, {<0.739.0>,168}, {<0.779.0>,168}, {<0.703.0>,168}, {<0.465.0>,168}, {<0.563.0>,168}, {<0.658.0>,168}, {<0.702.0>,168}, {<0.450.0>,168}, {<0.773.0>,168}, {<0.738.0>,168}, {<0.479.0>,168}, {<0.451.0>,168}, {<0.503.0>,168}, {<0.497.0>,168}, {<0.525.0>,168}, {<0.466.0>,168}, {<0.531.0>,168}, {<0.547.0>,168}, {<0.580.0>,168}, {<0.447.0>,168}, {<0.656.0>,168}, {<0.539.0>,167}, {<0.533.0>,167}, {<0.713.0>,167}, {<0.449.0>,167}, {<0.766.0>,167}, {<0.624.0>,167}, {<0.609.0>,167}, {<0.485.0>,167}, {<0.740.0>,167}, {<0.496.0>,167}, {<0.772.0>,167}, {<0.489.0>,167}, {<0.538.0>,167}, {<0.617.0>,167}, {<0.778.0>,167}, {<0.548.0>,167}, {<0.541.0>,167}, {<0.748.0>,167}, {<0.678.0>,167}, {<0.770.0>,167}, {<0.750.0>,167}, {<0.504.0>,167}, {<0.742.0>,167}, {<0.774.0>,167}, {<0.540.0>,167}, {<0.626.0>,167}, {<0.680.0>,166}, {<0.704.0>,166}, {<0.741.0>,166}, {<0.625.0>,166}, {<0.565.0>,166}, {<0.764.0>,166}, {<0.419.0>,166}, {<0.705.0>,166}, {<0.780.0>,166}, {<0.712.0>,166}, {<0.747.0>,166}, {<0.681.0>,166}, {<0.418.0>,166}, {<0.758.0>,166}, {<0.679.0>,166}, {<0.534.0>,166}, {<0.682.0>,166}, {<0.782.0>,166}, {<0.564.0>,165}, {<0.415.0>,165}, {<0.581.0>,165}, {<0.550.0>,165}, {<0.714.0>,165}, {<0.610.0>,165}, {<0.549.0>,165}, {<0.566.0>,165}, {<0.618.0>,165}, {<0.706.0>,165}, {<0.416.0>,165}, {<0.781.0>,165}, {<0.417.0>,165}, {<0.657.0>,165}, {<0.749.0>,164}] ok -------------- next part -------------- A non-text attachment was scrubbed... Name: fac.tar.gz Type: application/x-gzip Size: 1660 bytes Desc: fac.tar.gz URL: From tim@REDACTED Tue Jul 5 14:42:02 2005 From: tim@REDACTED (Tim Bates) Date: Tue, 05 Jul 2005 22:12:02 +0930 Subject: ODBC application doesn't compile under Linux Message-ID: <42CA801A.3060605@bates.id.au> Hi folks, The file lib/Makefile in the Erlang distribution includes the following lines: ifeq ($(findstring win32,$(TARGET)),win32) OTHER_SUB_DIRECTORIES = \ ... odbc ... else ... ifeq ($(findstring sparc-sun-solaris,$(TARGET)),sparc-sun-solaris) OTHER_SUB_DIRECTORIES += odbc endif ... endif Why does ODBC only get compiled under win32 and solaris? How do I get it to compile under Gentoo Linux? Why doesn't adding "odbc" to OTHER_SUB_DIRECTORIES work? I have unixodbc installed. Thanks, Tim. -- Tim Bates tim@REDACTED From csanto@REDACTED Tue Jul 5 15:17:00 2005 From: csanto@REDACTED (Corrado Santoro) Date: Tue, 05 Jul 2005 15:17:00 +0200 Subject: TCP stack throughput In-Reply-To: References: Message-ID: <42CA884C.8050104@diit.unict.it> Ulf Wiger (AL/EAB) wrote: > I also noticed that the accept calls seem to go quickly in > batches of 100-200, then followed by an increasingly long > delay. Have you tried "netstat"? You should note many sockets in the FIN_WAIT state. So probably the delay you experience is due to the fact that there are no more sockets available (in the kernel socket table), so you must wait until sockets in FIN_WAIT are finally flushed. --C. -- ====================================================== Eng. Corrado Santoro, Ph.D. University of Catania - Engineering Faculty Department of Computer Science and Telecommunications Engineering Viale A. Doria, 6 - 95125 CATANIA (ITALY) Tel: +39 095 7382144 Int. (5) 8035 +39 095 7382380 +39 095 7382365 +39 095 7382364 VoIP: sip:8035@REDACTED Fax: +39 095 7382397 EMail: csanto@REDACTED Personal Home Page: http://www.diit.unict.it/users/csanto NUXI Home Page: http://nuxi.iit.unict.it ====================================================== From jabba@REDACTED Tue Jul 5 15:32:36 2005 From: jabba@REDACTED (Jani Launonen) Date: Tue, 5 Jul 2005 16:32:36 +0300 (EEST) Subject: TCP stack throughput In-Reply-To: <94E4F547-C0BF-45C3-95A2-05238FF2CB98@gmail.com> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> <1C3C8872-57CC-425E-97D4-70438EAA9A4E@gmail.com> <94E4F547-C0BF-45C3-95A2-05238FF2CB98@gmail.com> Message-ID: > I made the server close the connection right after sending but it did not > make much of a difference. On Mac OSX 200 out of 500 connection requests > are refused. It appears that Linux kernels are by default limited to 128 backlog entries (http://bcr2.uwaterloo.ca/~brecht/papers/getpaper.php?file=usenix-2004.pdf, ch 3, 2. paragraph) Perhaps configuring kernel differently one could increase the performance dramatically as going from backlog length from 5 to 100 demonstrated. I don't know if this applies to other operating systems as well or how one could make the change (by writing /proc/sys/... or by configuring and recompiling kernel). >[rest cut] -+-+-+- Jani Launonen From jabba@REDACTED Tue Jul 5 15:55:00 2005 From: jabba@REDACTED (Jani Launonen) Date: Tue, 5 Jul 2005 16:55:00 +0300 (EEST) Subject: TCP stack throughput In-Reply-To: <94E4F547-C0BF-45C3-95A2-05238FF2CB98@gmail.com> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> <1C3C8872-57CC-425E-97D4-70438EAA9A4E@gmail.com> <94E4F547-C0BF-45C3-95A2-05238FF2CB98@gmail.com> Message-ID: Well, I found a page that show how to increase backlog in several operating systems: http://www.douzhe.com/linuxtips/1111.html In Linux one could say: # sysctl -w net.ipv4.tcp_max_syn_backlog="2048" I tested with 800 connections and it worked although not blindingly fast as I hoped, but without errors. 1000 connections gave timeouts (with default 10 second timeout) but otherwise ok. -+-+-+- Jani Launonen From joelr1@REDACTED Tue Jul 5 15:58:00 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 5 Jul 2005 15:58:00 +0200 Subject: TCP stack throughput In-Reply-To: References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> <1C3C8872-57CC-425E-97D4-70438EAA9A4E@gmail.com> <94E4F547-C0BF-45C3-95A2-05238FF2CB98@gmail.com> Message-ID: <9922C6ED-45CB-4D3C-9A8A-06189A2F4F4F@gmail.com> Jani, It does not explain why I was able to get with the C server 3x the connections of an Erlang server. All on the same platform! I think this is the key point! Thanks, Joel On Jul 5, 2005, at 3:55 PM, Jani Launonen wrote: > Well, I found a page that show how to increase backlog in several > operating systems: > http://www.douzhe.com/linuxtips/1111.html -- http://wagerlabs.com/uptick From jabba@REDACTED Tue Jul 5 16:46:23 2005 From: jabba@REDACTED (Jani Launonen) Date: Tue, 5 Jul 2005 17:46:23 +0300 (EEST) Subject: TCP stack throughput In-Reply-To: <9922C6ED-45CB-4D3C-9A8A-06189A2F4F4F@gmail.com> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> <1C3C8872-57CC-425E-97D4-70438EAA9A4E@gmail.com> <94E4F547-C0BF-45C3-95A2-05238FF2CB98@gmail.com> <9922C6ED-45CB-4D3C-9A8A-06189A2F4F4F@gmail.com> Message-ID: On Tue, 5 Jul 2005, Joel Reymont wrote: > Jani, > >It does not explain why I was able to get with the C server 3x the connections >of an Erlang server. All on the same platform! I think this is the key point! Yes, I'm aware of the difference between C and Erlang implementations, but I just hoped to find an easy solution to get performance to acceptable level with Erlang too. Of course it might increase the performance of C implementation as well but I thought that performance level high enough (although you probably want all you can get) might make it unnecessary to make exotic out of Erlang solutions. Just shooting from the hip. > Thanks, Joel No problem. I'm glad if I could contribute something as I've enjoyed reading about your poker game server. > On Jul 5, 2005, at 3:55 PM, Jani Launonen wrote: > >> Well, I found a page that show how to increase backlog in several operating >> systems: >> http://www.douzhe.com/linuxtips/1111.html > > -- > http://wagerlabs.com/uptick > > > -+-+-+- Jani Launonen From feeley@REDACTED Tue Jul 5 17:10:29 2005 From: feeley@REDACTED (Marc Feeley) Date: Tue, 5 Jul 2005 11:10:29 -0400 Subject: TCP stack throughput In-Reply-To: <9922C6ED-45CB-4D3C-9A8A-06189A2F4F4F@gmail.com> References: <2A2ACC48-59AF-4DC1-980A-41A405BD6A6F@gmail.com> <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> <1C3C8872-57CC-425E-97D4-70438EAA9A4E@gmail.com> <94E4F547-C0BF-45C3-95A2-05238FF2CB98@gmail.com> <9922C6ED-45CB-4D3C-9A8A-06189A2F4F4F@gmail.com> Message-ID: On 5-Jul-05, at 9:58 AM, Joel Reymont wrote: > Jani, > > It does not explain why I was able to get with the C server 3x the > connections of an Erlang server. All on the same platform! I think > this is the key point! > > Thanks, Joel > For kicks I rewrote the C version of the server (i.e. server.c) in Scheme, but with threads (so that each request is handled in its own thread), and compiled it with Gambit-C 4.0 beta 14. The client sends 1000 concurrent requests to the server. The results attached below show that the Scheme version achieves almost the same speed as C (less than 10% difference of real time) and that no ECONNREFUSED errors were generated. I suspect that Erlang's scheduler is starving the main process (which accepts the connections). Gambit's scheduler uses a "priority-boost" to give a slight priority to processes whose I/O has just become possible. Perhaps something like this would help improve Erlang's performance and avoid the ECONNREFUSED errors. Marc ; Platform: 400 MHz PowerBook G4, 384 MB RAM ; ; Program is a client opening 1000 concurrent connections to the server ; which reads a 10 octet request and replies with a 256 octet response. ; The client and server are running on the same machine. ; ; Server written in C: ; ; real 0m1.271s ; user 0m0.277s ; sys 0m0.333s ; ; Server written in Scheme and compiled with Gambit-C 4.0 beta 14: ; ; real 0m1.348s ; user 0m0.199s ; sys 0m0.386s ; ; There were no ECONNREFUSED errors in both cases. From erlang@REDACTED Tue Jul 5 18:02:37 2005 From: erlang@REDACTED (Michael McDaniel) Date: Tue, 5 Jul 2005 09:02:37 -0700 Subject: SSL Setup Problem In-Reply-To: <20050705063448.GU11207@fangora.autosys.us> References: <20050704100703.GS11207@fangora.autosys.us> <20050704111354.75272.qmail@web8502.mail.in.yahoo.com> <20050705063448.GU11207@fangora.autosys.us> Message-ID: <20050705160237.GV11207@fangora.autosys.us> On Mon, Jul 04, 2005 at 11:34:49PM -0700, Michael McDaniel wrote: > On Mon, Jul 04, 2005 at 12:13:54PM +0100, luvish satija wrote: > > Hi Michael, > > The symbolic link did the trick. Now i'm able to get > > the boot script without any warnings. > > But still no ssl application is working. To test it > > actually, i simply gave ssl:listen() in the command > > prompt and was welcomed by the following errors: > > > > > > root@REDACTED:~/lvs# erl -boot > > /home/maniyan/lvs/start_ssl -proto_dist inet_ssl > > -ssl_dist_opt client_certfile > > "/home/maniyan/lvs/ssl_server.pem" -ssl_dist_opt > > server_certfile "/home/maniyan/lvs/ssl_server.pem" > > -ssl_dist_opt verify 1 depth 1 > > Erlang (BEAM) emulator version 5.4.6 [source] [hipe] > > > > Eshell V5.4.6 (abort with ^G) > > 1> ssl:listen(4000,[binary]). > > ** exited: {normal,{gen_server,call, > > [ssl_server, > > > > {listen,<0.37.0>,{0,0,0,0},4000,[],5}, > > infinity]}} ** > > > > =ERROR REPORT==== 4-Jul-2005::16:42:38 === > > ** Generic server <0.37.0> terminating > > ** Last message in was {listen,<0.35.0>,4000,[binary]} > > ** When Server state == {st,listener, > > <0.31.0>, > > <0.35.0>, > > <0.35.0>, > > nil, > > true, > > [], > > nil, > > nil, > > nil, > > false} > > ** Reason for termination == > > ** {normal,{gen_server,call, > > [ssl_server, > > > > {listen,<0.37.0>,{0,0,0,0},4000,[],5}, > > infinity]}} > > > > Now I don't know what does it mean and why it is > > comping because i'm doing all the things according to > > the manual. If you have any clue about this, or if you > > can suggest some other way, then it would really help > > me. > > Thanks, > > /luvish > _______________________________________________________ > > I get same error on my version 5.4.4 system which runs ssl. > > ~Michael _______________________________________________________ And my version 5.4.8 (R10B-6) which I just set up for ssl. ~Michael > From mickael.remond@REDACTED Tue Jul 5 18:21:38 2005 From: mickael.remond@REDACTED (=?ISO-8859-1?Q?Micka=EBl_R=E9mond?=) Date: Tue, 5 Jul 2005 18:21:38 +0200 Subject: ODBC application doesn't compile under Linux In-Reply-To: <42CA801A.3060605@bates.id.au> References: <42CA801A.3060605@bates.id.au> Message-ID: <3308A837-DEB2-4D2A-BB3B-022236538532@erlang-fr.org> Le 5 juil. 05 ? 14:42, Tim Bates a ?crit : > Why does ODBC only get compiled under win32 and solaris? How do I > get it > to compile under Gentoo Linux? Why doesn't adding "odbc" to > OTHER_SUB_DIRECTORIES work? I have unixodbc installed. Hello, What version of Erlang/OTP are you using ? Regarding ODBC under Linux I tried with Erlang R10B-6 and the compilation support is now fine and very easy. Modification of makefiles are not needed anymore. -- Micka?l R?mond From roger.larsson@REDACTED Tue Jul 5 18:27:41 2005 From: roger.larsson@REDACTED (Roger Larsson) Date: Tue, 5 Jul 2005 18:27:41 +0200 Subject: TCP stack throughput In-Reply-To: <17097.43859.831827.297323@antilipe.corelatus.se> References: <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> Message-ID: <200507051827.42171.roger.larsson@norran.net> On Monday 04 July 2005 23.34, Matthias Lang wrote: > Mickael Remond writes: > > Joel Reymont wrote: > > > So this means 500 simultanous connection requests on Solaris and > > > about 300 on Mac OSX (FreeBSD). My question, I suppose, is whether > > > this is high enough. > > > > > > Is there a limitation in Erlang or is this maxing out the platform > > > TCP stack? > > > > > > What do you folks recon? > > > > Hello, > > > > I am not sure I understand your test quite well. Basically you are > > connecting and deconnecting very quickly. Is that the case ? > > If this is the case, what are you trying to benchmark ? > > Most of the time mesuring the TCP/IP throughput implies trying to know > > how much data can be processed through a TCP socket, but usually a long > > running one. > > I'm with Mickael. > > Your results could be interpreted as showing one _or more_ of the > following: > > a) That your test client is the bottleneck > > b) That your test server is the bottleneck > > c) That the OS/tcp stack isn't really designed for the sort of > use you're testing. > > My gut feeling says (c), mainly because different people are reporting > fairly different behaviour from one OS to another. I've written a > quick and dirty C server which does something similar enough to the > erlang server to work with your client. You could experiment with it > to try and eliminate hypothesis (b). > > The program works for me on linux 2.6.x. > > Matthias What about the factorial - C version does not do it. Isn't it possible that it creates a bigger CPU load than expected? (due to using bignums) /RogerL From serge@REDACTED Tue Jul 5 18:29:45 2005 From: serge@REDACTED (Serge Aleynikov) Date: Tue, 05 Jul 2005 12:29:45 -0400 Subject: inets crashing in R10B-6 In-Reply-To: <42BC8B1D.9040007@hq.idt.net> References: <42BC843E.5070405@hq.idt.net> <42BC8B1D.9040007@hq.idt.net> Message-ID: <42CAB579.7060408@hq.idt.net> All, In case you experienced the same issue with inets crashing under R10B-6, I got the attached patch from Peter @ OTP team that fixes the issue. You will need to replace the http_transport.[erl|beam] of inets with the one attached. Regards, Serge -- Serge Aleynikov R&D Telecom, IDT Corp. Tel: (973) 438-3436 Fax: (973) 438-1464 serge@REDACTED -------------- next part -------------- An embedded message was scrubbed... From: UAB L/K Peter Andersson Subject: Re: inets crashing in R10B-6 Date: Fri, 01 Jul 2005 16:54:35 +0200 Size: 19081 URL: From erlang@REDACTED Tue Jul 5 18:41:35 2005 From: erlang@REDACTED (Michael McDaniel) Date: Tue, 5 Jul 2005 09:41:35 -0700 Subject: ODBC application doesn't compile under Linux In-Reply-To: <42CA801A.3060605@bates.id.au> References: <42CA801A.3060605@bates.id.au> Message-ID: <20050705164129.GY11207@fangora.autosys.us> On Tue, Jul 05, 2005 at 10:12:02PM +0930, Tim Bates wrote: > Hi folks, > The file lib/Makefile in the Erlang distribution includes the following > lines: > > ifeq ($(findstring win32,$(TARGET)),win32) > OTHER_SUB_DIRECTORIES = \ > ... odbc ... > else > ... > ifeq ($(findstring sparc-sun-solaris,$(TARGET)),sparc-sun-solaris) > OTHER_SUB_DIRECTORIES += odbc > endif > ... > endif > > > Why does ODBC only get compiled under win32 and solaris? How do I get it > to compile under Gentoo Linux? Why doesn't adding "odbc" to > OTHER_SUB_DIRECTORIES work? I have unixodbc installed. > > Thanks, > Tim. > > -- > Tim Bates > tim@REDACTED ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May be useful... http://www.trapexit.org/docs/howto/odbc-howto.html (R10B-5 and R10B-6 were much simpler) ~Michael From feeley@REDACTED Tue Jul 5 20:11:41 2005 From: feeley@REDACTED (Marc Feeley) Date: Tue, 5 Jul 2005 14:11:41 -0400 Subject: TCP stack throughput In-Reply-To: <200507051827.42171.roger.larsson@norran.net> References: <42C96EFB.6080301@erlang-fr.org> <17097.43859.831827.297323@antilipe.corelatus.se> <200507051827.42171.roger.larsson@norran.net> Message-ID: On 5-Jul-05, at 12:27 PM, Roger Larsson wrote: > What about the factorial - C version does not do it. > Isn't it possible that it creates a bigger CPU load than expected? > (due to using bignums) > Good point! When I added a call to factorial in my Scheme version of the server (N=100), ECONNREFUSED errors are generated on the client side. These errors occur because the client is generating connection requests faster than the server can complete them (by pulling them out of the backlog queue). The problem is that the client and server are OS processes running on the same machine, so when the client is active (generating connection requests) the server is inactive and can't empty the backlog queue. This problem will become more severe when the multitasking quantum of the OS is large and when it is faster to generate a connection request than to pull one out of the backlog queue (which depends on the implementation of the TCP stack). Anyway, I "solved" the problem by decreasing the OS process priority of the client (with a "nice -20"). This eliminates the ECONNREFUSED errors. Marc From ulf.wiger@REDACTED Tue Jul 5 21:53:34 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Tue, 5 Jul 2005 21:53:34 +0200 Subject: TCP stack throughput Message-ID: Corrado Santoro wrote: > > Have you tried "netstat"? You should note many sockets in the > FIN_WAIT > state. So probably the delay you experience is due to the fact that > there are no more sockets available (in the kernel socket > table), so you > must wait until sockets in FIN_WAIT are finally flushed. You're absolutely right, except my netstat lists it as TIME_WAIT. This is described in the man page as: TIME_WAIT Wait after close for remote shutdown retransmission. I found this page explaining the problem of sockets lingering in TIME_WAIT, with examples of how to reduce the time on different operating systems: http://www.octetstring.com/support/issues/Tuning-TIMEWAIT.php Unfortunately, I can't do this on my system, as I lack root privs. At least, I can inspect it: > /usr/sbin/ndd -get /dev/tcp tcp_time_wait_interval 240000 /Uffe From ulf.wiger@REDACTED Tue Jul 5 22:03:41 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Tue, 5 Jul 2005 22:03:41 +0200 Subject: TCP stack throughput Message-ID: Waiting until all the sockets in TIME_WAIT had closed, I re-ran the test with (optimistically) 1200 clients. The server fairly quickly racked up about 900 connections in 15 seconds, before it started getting into trouble. (actually, the first 600 came in under 10 seconds, then the first hints of slowdown appeared.) Some 170 clients timed out after 1 minute. /Uffe From joe.armstrong@REDACTED Wed Jul 6 09:29:45 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Wed, 6 Jul 2005 09:29:45 +0200 Subject: TCP stack throughput Message-ID: Matthias Lang wrote: > Mickael Remond writes: > > Joel Reymont wrote: > > > So this means 500 simultanous connection requests on > Solaris and about > Matthias Lang said: > Your results could be interpreted as showing one _or more_ of the > following: > > a) That your test client is the bottleneck > > b) That your test server is the bottleneck > > c) That the OS/tcp stack isn't really designed for the sort of > use you're testing. > > My gut feeling says (c), mainly because different people are reporting > fairly different behaviour from one OS to another. I've written a > quick and dirty C server which does something similar enough to the > erlang server to work with your client. You could experiment with it > to try and eliminate hypothesis (b). > I think you might run into problems distinguishing between a) and b) here, if you run the client and server on the *same* machine then both the client and server start failing at the same time when the machine is loaded. When I tested yaws against apache. I ran tests on a 16 node cluster. I put yaws (or apache) on the server, then ran 15 clients on each of the other nodes in the cluster. Now if the client and server stack have roughly equivalent breaking characteristics then at the point when the server breaks the clients should have be running at say 1/15'th of their max capacity - so rate measurements on the clients should be valid. As regards c) I have recently reviewed a paper on an implementation of TCP/IP in Erlang (this paper has been accepted for the next SIGPLAN Erlang workshop) - the results were interesting - they showed that 1) A pure Erlang stack was much more "even" than conventional stacks. Conventional stacks seemed to favour a few connections and starve out the rest, So a few connections got very good bandwidth, the rest were very poor. The Erlang stack was much fairer. 2) The Erlang stack was c. 25% of the efficiency of the native stack The Erlang stack also could synchronize the state of each connection with a second machine, so that the second machine could take over a live TCP connection. All of this seems to be saying that if you want to handle very large numbers of very short-lived connections use a pure erlang stack - but if you want to handle a small number of long-lived high bandwidth connections use a conventional stack. If this is true, it would not surprise me, since this behaviour mirrors how processes are handled in Erlang and in the OS - since Erlang favours losts of small light-weight processes and the OS a few heavy-weight processes. /Joe > The program works for me on linux 2.6.x. > > Matthias > > From paris@REDACTED Wed Jul 6 20:03:18 2005 From: paris@REDACTED (Javier =?iso-8859-1?Q?Par=EDs_Fern=E1ndez?=) Date: Wed, 6 Jul 2005 20:03:18 +0200 Subject: TCP stack throughput In-Reply-To: References: Message-ID: <20050706180318.GA18071@dc.fi.udc.es> On Wed, Jul 06, 2005 at 09:29:45AM +0200, Joe Armstrong (AL/EAB) wrote: > Matthias Lang wrote: > > Mickael Remond writes: > > > Joel Reymont wrote: > > > > So this means 500 simultanous connection requests on > > Solaris and about > > Matthias Lang said: > > Your results could be interpreted as showing one _or more_ of the > > following: > > > > a) That your test client is the bottleneck > > > > b) That your test server is the bottleneck > > > > c) That the OS/tcp stack isn't really designed for the sort of > > use you're testing. > > > As regards c) I have recently reviewed a paper on an implementation of TCP/IP > in Erlang (this paper has been accepted for the next SIGPLAN Erlang workshop) - the > results were interesting - they showed that > > 1) A pure Erlang stack was much more "even" than conventional stacks. > Conventional stacks seemed to favour a few connections and starve out the rest, > So a few connections got very good bandwidth, the rest were very poor. > The Erlang stack was much fairer. > > 2) The Erlang stack was c. 25% of the efficiency of the native stack > > The Erlang stack also could synchronize the state of each connection with a second machine, so that the second machine could take over a live TCP connection. > > All of this seems to be saying that if you want to handle very large numbers > of very short-lived connections use a pure erlang stack - but if you want to handle > a small number of long-lived high bandwidth connections use a conventional stack. > > If this is true, it would not surprise me, since this behaviour mirrors how processes > are handled in Erlang and in the OS - since Erlang favours losts of small light-weight processes and the OS a few heavy-weight processes. I'll try this particular case with the Erlang Stack as soon as posible, as it seems an interesting test for Erlang vs Native OS. (However, as this is the course end in Spain, it may take a while to have some time to try the example). Regards. Javier. From kruegger@REDACTED Thu Jul 7 01:59:47 2005 From: kruegger@REDACTED (Stephen Han) Date: Wed, 6 Jul 2005 16:59:47 -0700 Subject: construct binary Message-ID: <86f1f535050706165928844a90@mail.gmail.com> Hello I am trying to write a IS-683 PRL encode/decode tool for our testing. One of the parameter format is ---------------------------------- NUM_CHANS | 5 BITS ---------------------------------- *NUM_CHANS occurrences of the following field: ----------------------------------- CHAN | 11 BITS ------------------------------------ * finally pad some bits to fit into the byte ----------------------------------- RESERVED | As Required ------------------------------------ So if the number of channel is 2 then 0 31 +-----------+--------------+--------------+-------------+ 5bit 11bit 11bit 5 bit(for padding) +-----------+--------------+--------------+-------------+ Trying to use the bit syntax in Erlang but gives me hard time... Can anybody demonstrate Erlang way to write it? regards, From vances@REDACTED Thu Jul 7 03:26:12 2005 From: vances@REDACTED (Vance Shipley) Date: Wed, 6 Jul 2005 21:26:12 -0400 Subject: construct binary In-Reply-To: <86f1f535050706165928844a90@mail.gmail.com> References: <86f1f535050706165928844a90@mail.gmail.com> Message-ID: <20050707012611.GE68834@feeble.motivity.ca> Stephen, I think this does it. -Vance 1> prl:decode(<<2:5, 1:11, 2:11, 0:5>>). [1,2] -module(prl). -export([decode/1]). decode(<> = Bin) -> decode(Bin, NumChans, NumChans, []). decode(_, _, 0, Acc) -> Acc; decode(Bin, NumChans, N, Acc) -> SkipBefore = (N - 1) * 11, SkipAfter = (NumChans - N) * 11, Pad = 8 - ((NumChans * 11 + 5) rem 8), <<_:5, _:SkipBefore, Chan:11, _:SkipAfter, _:Pad>> = Bin, decode(Bin, NumChans, N - 1, [Chan | Acc]). On Wed, Jul 06, 2005 at 04:59:47PM -0700, Stephen Han wrote: } } I am trying to write a IS-683 PRL encode/decode tool for our testing. From yannnn@REDACTED Thu Jul 7 08:51:11 2005 From: yannnn@REDACTED (Ki-Youn Sung) Date: Thu, 7 Jul 2005 15:51:11 +0900 Subject: why?? {error, enotsock} Message-ID: <357f1a790507062351fa2cd01@mail.gmail.com> Hello. My System is WindowsXP SP2. Recently, I installed version R10B-6, and runned "erl -sname name". but I don't run erl, erl program error is below, {error_logger,{{2005,7,7},{15,47,2}},'Protocol: ~p: register/listen error: ~p~n' ,[inet_tcp,enotsock]} {error_logger,{{2005,7,7},{15,47,2}},crash_report,[[{pid,<0.17.0>},{registered_n ame,net_kernel},{error_info,{error,badarg}},{initial_call,{gen,init_it,[gen_serv er,<0.14.0>,<0.14.0>,{local,net_kernel},net_kernel,{aa,shortnames,15000},[]]}},{ ancestors,[net_sup,kernel_sup,<0.8.0>]},{messages,[]},{links,[<0.14.0>]},{dictio nary,[{longnames,false}]},{trap_exit,true},{status,running},{heap_size,233},{sta ck_size,23},{reductions,251}],[]]} {error_logger,{{2005,7,7},{15,47,2}},supervisor_report,[{supervisor,{local,net_s up}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid ,undefined},{name,net_kernel},{mfa,{net_kernel,start_link,[[aa,shortnames]]}},{r estart_type,permanent},{shutdown,2000},{child_type,worker}]}]} {error_logger,{{2005,7,7},{15,47,2}},supervisor_report,[{supervisor,{local,kerne l_sup}},{errorContext,start_error},{reason,shutdown},{offender,[{pid,undefined}, {name,net_sup},{mfa,{erl_distribution,start_link,[]}},{restart_type,permanent},{ shutdown,infinity},{child_type,supervisor}]}]} {error_logger,{{2005,7,7},{15,47,2}},crash_report,[[{pid,<0.7.0>},{registered_na me,[]},{error_info,{shutdown,{kernel,start,[normal,[]]}}},{initial_call,{applica tion_master,init,[<0.5.0>,<0.6.0>,{appl_data,kernel,[application_controller,erl_ reply,auth,boot_server,code_server,disk_log_server,disk_log_sup,erl_prim_loader, error_logger,file_server,file_server_2,fixtable_server,global_group,global_name_ server,heart,init,kernel_config,kernel_sup,net_kernel,net_sup,rex,user,os_server ,ddll_server,erl_epmd,inet_db,pg2],undefined,{kernel,[]},[application,applicatio n_controller,application_master,application_starter,auth,code,code_aux,packages, code_server,dist_util,erl_boot_server,erl_distribution,erl_open_port,erl_prim_lo ader,erl_reply,erlang,error_handler,error_logger,file,file_server,old_file_serve r,file_io_server,prim_file,global,global_group,global_search,group,heart,inet6_t cp,inet6_tcp_dist,inet6_udp,inet_config,inet_hosts,inet_gethost_native,inet_tcp_ dist,init,kernel,kernel_config,net,net_adm,net_kernel,os,ram_file,rpc,user,user_ drv,user_sup,disk_log,disk_log_1,disk_log_server,disk_log_sup,dist_ac,erl_ddll,e rl_epmd,erts_debug,gen_tcp,gen_udp,prim_inet,inet,inet_db,inet_dns,inet_parse,in et_res,inet_tcp,inet_udp,pg2,seq_trace,socks5,socks5_auth,socks5_tcp,socks5_udp, wrap_log_reader,otp_ring0],[],infinity,infinity},normal]}},{ancestors,[<0.6.0>]} ,{messages,[{'EXIT',<0.8.0>,normal}]},{links,[<0.6.0>,<0.5.0>]},{dictionary,[]}, {trap_exit,true},{status,running},{heap_size,610},{stack_size,23},{reductions,10 80}],[]]} {error_logger,{{2005,7,7},{15,47,2}},std_info,[{application,kernel},{exited,{shu tdown,{kernel,start,[normal,[]]}}},{type,permanent}]} {"Kernel pid terminated",application_controller,shutdown} Kernel pid terminated (application_controller) (shutdown) and, tried again this, erl> gen_tcp:listen(0, []) erl> {error, enotsock} Do you know Why.?? Thank You.. From matthias@REDACTED Thu Jul 7 08:54:43 2005 From: matthias@REDACTED (Matthias Lang) Date: Thu, 7 Jul 2005 08:54:43 +0200 Subject: construct binary In-Reply-To: <20050707012611.GE68834@feeble.motivity.ca> References: <86f1f535050706165928844a90@mail.gmail.com> <20050707012611.GE68834@feeble.motivity.ca> Message-ID: <17100.53683.365347.802380@antilipe.corelatus.se> Here's my try: decode(<>) -> decode2(N_channels, <>). decode2(0, _) -> []; decode2(N, <>) -> [C|decode2(N-1, <>)]. encode(Channels) when length(Channels) < 32 -> encode2(<<(length(Channels)):5, 0:3>>, 5, 3, Channels). encode2(Bin, _Used, _Pad, []) -> Bin; encode2(Bin, Bits, Pad, [C|Cs]) when C >= 0, C < 2048 -> <> = Bin, New_pad = (Pad - 11) band 7, encode2(<>, Bits + 11, New_pad, Cs). Matthias -------------------- Vance Shipley writes: > Stephen, > > I think this does it. > > -Vance > > > 1> prl:decode(<<2:5, 1:11, 2:11, 0:5>>). > [1,2] > > > -module(prl). > -export([decode/1]). > > decode(<> = Bin) -> > decode(Bin, NumChans, NumChans, []). > decode(_, _, 0, Acc) -> > Acc; > decode(Bin, NumChans, N, Acc) -> > SkipBefore = (N - 1) * 11, > SkipAfter = (NumChans - N) * 11, > Pad = 8 - ((NumChans * 11 + 5) rem 8), > <<_:5, _:SkipBefore, Chan:11, _:SkipAfter, _:Pad>> = Bin, > decode(Bin, NumChans, N - 1, [Chan | Acc]). > > > On Wed, Jul 06, 2005 at 04:59:47PM -0700, Stephen Han wrote: > } > } I am trying to write a IS-683 PRL encode/decode tool for our testing. From matthias@REDACTED Thu Jul 7 10:35:59 2005 From: matthias@REDACTED (Matthias Lang) Date: Thu, 7 Jul 2005 10:35:59 +0200 Subject: why?? {error, enotsock} In-Reply-To: <357f1a790507062351fa2cd01@mail.gmail.com> References: <357f1a790507062351fa2cd01@mail.gmail.com> Message-ID: <17100.59759.229026.57467@antilipe.corelatus.se> Ki-Youn Sung writes: > Hello. > > My System is WindowsXP SP2. > > Recently, I installed version R10B-6, and runned "erl -sname name". > but I don't run erl, erl program error is below, > > {error_logger,{{2005,7,7},{15,47,2}},'Protocol: ~p: register/listen error: ~p~n' > ,[inet_tcp,enotsock]} [...] > and, tried again this, > > erl> gen_tcp:listen(0, []) > erl> {error, enotsock} > Do you know Why.?? It looks like you already found the previous time this was discussed: http://comments.gmane.org/gmane.comp.lang.erlang.general/8410%3Cbr I don't think anyone came up with a satisfactory answer then, beyond the usual Microsoft troubleshooting procedure, i.e. - randomly install and uninstall various service packs, hoping to find the magic one which only breaks features you don't need - re-install windows There must be a better way. Matthias From vances@REDACTED Thu Jul 7 18:56:28 2005 From: vances@REDACTED (Vance Shipley) Date: Thu, 7 Jul 2005 12:56:28 -0400 Subject: construct binary In-Reply-To: <17100.53683.365347.802380@antilipe.corelatus.se> References: <86f1f535050706165928844a90@mail.gmail.com> <20050707012611.GE68834@feeble.motivity.ca> <17100.53683.365347.802380@antilipe.corelatus.se> Message-ID: <20050707165628.GF68834@feeble.motivity.ca> Mathias, I'd have to say that your version is much better as it's shorter and simpler. Not to mention the fact that, as you pointed out, my pad calculation was flawed. The one difference though is that my version didn't create a new binary so would presumably be more efficient. -Vance On Thu, Jul 07, 2005 at 08:54:43AM +0200, Matthias Lang wrote: } } decode(<>) -> } decode2(N_channels, <>). } decode2(0, _) -> } []; } decode2(N, <>) -> } [C|decode2(N-1, <>)]. From joelr1@REDACTED Thu Jul 7 20:09:28 2005 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 7 Jul 2005 20:09:28 +0200 Subject: New trading systems platform Message-ID: <880595AF-DEDA-4244-BC69-5001BFDF86F1@gmail.com> Folks, Poker is not flying off of my shelf but I gained invaluable experience in building that software. I'm building a new ATS development platform on top of Erlang. I will focus on real-time trading systems and tick data. The core of the system will be open source but I'll make money on data feed handlers and add-ons. This is a bit of a research project so any suggestions and critique is welcome. I will also eat my own dogfood and use the platform for trading. There are several features in Erlang that make this the ideal platform for processing and analyzing high volumes of real-time data. Please take a look at the following links. Wall St. is apparently being swamped by real-time data and companies like http:// www.vhayu.com/ have sprung to fill the niche. http://www.wallstreetandtech.com/showArticle.jhtml?articleID=164903661 http://db.riskwaters.com/public/showPage.html? page=printer_friendly&print=195098 http://www.dbta.com/in-depth/mar05/rugg-palmer.html This is just the type of work that can be cheaply done with Erlang and I'm surprised that no one has thought of it before. Data feed adapters can be easily coded with Erlang and it comes with a high-performance database that makes storing tick and instrument data easy. I will initially focus on tick data as this should make my job easier. I won't have to deal with futures rollovers, etc. Building grids and clusters is also simple as Erlang is all about large numbers of very lightweight distributed processes running on a network of "nodes". Processes don't need to be aware of where on the network other processes are running, they just message each other. You should be able to build yourself a server farm with real-time data feeds from various sources all going into the same database. I will enable this. And of course there's that 99.999% uptime. I'm concerned that Erlang might not be good enough performance-wise for numerical math on large lists of tuples. Still, I think coding the financial models in Erlang could be an advantage since the data will be in Mnesia, on the same node. What do you think? Thanks, Joel -- http://wagerlabs.com/uptick From kruegger@REDACTED Thu Jul 7 21:58:47 2005 From: kruegger@REDACTED (Stephen Han) Date: Thu, 7 Jul 2005 12:58:47 -0700 Subject: construct binary In-Reply-To: <20050707165628.GF68834@feeble.motivity.ca> References: <86f1f535050706165928844a90@mail.gmail.com> <20050707012611.GE68834@feeble.motivity.ca> <17100.53683.365347.802380@antilipe.corelatus.se> <20050707165628.GF68834@feeble.motivity.ca> Message-ID: <86f1f53505070712583cdc2d35@mail.gmail.com> Hi Thanks guys... This is great example how to use bit syntax elegantly not just for my problem for any bit related problems... thank you so much... If you look at my code, you will know why I was so frustrated... I always feel it is not right but I used it. I will attach mine although it is embarrassing. I am brave enough to post mine to show which is elegant and which is not for the sake of Erlang community. :-( -module(eprl). -compile (export_all). pack( Channels ) -> Num = length(Channels), pack_num(Num, pack_channel(Num, Channels)). reserve(1) -> 0; reserve(2) -> 5; reserve(3) -> 2; reserve(4) -> 7; reserve(5) -> 4; reserve(6) -> 1; reserve(7) -> 6; reserve(8) -> 3; reserve(N) when N > 0 -> reserve( N - 8 ). pack_num(N, B) -> R1 = reserve(N), R2 = reserve(N+1), L = ((size(B)-1) * 8 - R2), << B1:8, B2:L, _:R2 >> = B, << N:5, B1:8, B2:L, 0:R1 >>. pack_channel(1, [A1]) -> << A1:11, 0:5 >>; pack_channel(2, [A1, A2]) -> << A1:11, A2:11, 0:2 >>; pack_channel(3, [A1, A2, A3]) -> << A1:11, A2:11, A3:11, 0:7 >>; pack_channel(4, [A1, A2, A3, A4]) -> << A1:11, A2:11, A3:11, A4:11, 0:4 >>; pack_channel(5, [A1, A2, A3, A4, A5]) -> << A1:11, A2:11, A3:11, A4:11, A5:11, 0:1 >>; pack_channel(6, [A1, A2, A3, A4, A5, A6]) -> << A1:11, A2:11, A3:11, A4:11, A5:11, A6:11, 0:6 >>; pack_channel(7, [A1, A2, A3, A4, A5, A6, A7]) -> << A1:11, A2:11, A3:11, A4:11, A5:11, A6:11, A7:11, 0:3 >>; pack_channel(8, [A1, A2, A3, A4, A5, A6, A7, A8]) -> << A1:11, A2:11, A3:11, A4:11, A5:11, A6:11, A7:11, A8:11 >>; pack_channel(N, [A1, A2, A3, A4, A5, A6, A7, A8|T]) when N > 0 -> B1 = << A1:11, A2:11, A3:11, A4:11, A5:11, A6:11, A7:11, A8:11 >>, B2 = pack_channel(N-8, T), list_to_binary([B1, B2]). regards, On 7/7/05, Vance Shipley wrote: > Mathias, > > I'd have to say that your version is much better as it's shorter > and simpler. Not to mention the fact that, as you pointed out, > my pad calculation was flawed. The one difference though is that > my version didn't create a new binary so would presumably be more > efficient. > > -Vance > > On Thu, Jul 07, 2005 at 08:54:43AM +0200, Matthias Lang wrote: > } > } decode(<>) -> > } decode2(N_channels, <>). > } decode2(0, _) -> > } []; > } decode2(N, <>) -> > } [C|decode2(N-1, <>)]. > From james.hague@REDACTED Thu Jul 7 22:00:23 2005 From: james.hague@REDACTED (James Hague) Date: Thu, 7 Jul 2005 15:00:23 -0500 Subject: New trading systems platform In-Reply-To: <880595AF-DEDA-4244-BC69-5001BFDF86F1@gmail.com> References: <880595AF-DEDA-4244-BC69-5001BFDF86F1@gmail.com> Message-ID: >I'm concerned that Erlang might not be good enough performance-wise >for numerical math on large lists of tuples. Still, I think coding >the financial models in Erlang could be an advantage since the data >will be in Mnesia, on the same node. Much financial work like this is done in array languages, like K (http://www.kx.com) and J (http://www.jsoftware.com). They're designed to churn through large data sets quickly. J is free, so one conceivable option is to communicate with a J process from Erlang, but that may be too roundabout. J will make your head explode. Honestly, I'd like to see some basic tuple math added to Erlang, but this clashes with the sensibilities of some people. For example: 1 + {1,2,3} -> {3,4,5} {1,2,3} + 1 -> {3,4,5} {1,2,3} + {10,20,30} -> {11,22,33} 2 * {1,2,3} -> {2,4,6} I don't think this would be that hard to hack into the emulator. From hp@REDACTED Thu Jul 7 22:40:16 2005 From: hp@REDACTED (HP Wei) Date: Thu, 7 Jul 2005 16:40:16 -0400 (EDT) Subject: New trading systems platform In-Reply-To: <880595AF-DEDA-4244-BC69-5001BFDF86F1@gmail.com> References: <880595AF-DEDA-4244-BC69-5001BFDF86F1@gmail.com> Message-ID: On Thu, 7 Jul 2005, Joel Reymont wrote: > I'm concerned that Erlang might not be good enough performance-wise > for numerical math on large lists of tuples. Still, I think coding > the financial models in Erlang could be an advantage since the data > will be in Mnesia, on the same node. > yeah, I share the same concern over the speed of data analysis. A complicated trading model (involving CPU intensive calculations such as regression, optimization etc) coded in pure Erlang may not be fast enough. I believe those big shots on Wall Street use C/C++ for their model calculations. [ So, your trading system may need to be a hybrid system in order to compete. ] On the other hand, for simple technical analysis on the level of "Stocks and Commodities magazine", pure Erlang may be good enough. ----------------- I do not use Erlang often enough to make comments on data storage using Mnesia. I just want to point out that to store tick data of many years needs a LOT of disk space plus the capability of compression (when writing) (and decompression when reading) in your database design. Happy coding, --HP From joelr1@REDACTED Thu Jul 7 22:44:35 2005 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 7 Jul 2005 22:44:35 +0200 Subject: New trading systems platform In-Reply-To: References: <880595AF-DEDA-4244-BC69-5001BFDF86F1@gmail.com> Message-ID: <96622D6B-3010-4186-9B83-7E0E5112E1EA@gmail.com> HP, On Jul 7, 2005, at 10:40 PM, HP Wei wrote: > A complicated trading model (involving CPU intensive calculations > such as regression, optimization etc) coded in pure Erlang > may not be fast enough. > I believe those big shots on Wall Street use C/C++ for their > model calculations. I'm going to hook up Common Lisp, either as a linked-in driver or as a port. But only if speed proves to be lacking. > I just want to point out that to store tick data > of many years needs a LOT of disk space plus > the capability of compression (when writing) > (and decompression when reading) in your database design. Do you have a model for tick data storage? Fields, etc. Why would I bother to compress/decompress? Wouldn't keeping 1-2-3Gb in-memory be enough? Should I really care about disk storage? Joel -- http://wagerlabs.com/uptick From hp@REDACTED Thu Jul 7 23:18:21 2005 From: hp@REDACTED (HP Wei) Date: Thu, 7 Jul 2005 17:18:21 -0400 (EDT) Subject: New trading systems platform In-Reply-To: <96622D6B-3010-4186-9B83-7E0E5112E1EA@gmail.com> References: <880595AF-DEDA-4244-BC69-5001BFDF86F1@gmail.com> <96622D6B-3010-4186-9B83-7E0E5112E1EA@gmail.com> Message-ID: hi Joel, > > I just want to point out that to store tick data > > of many years needs a LOT of disk space plus > > the capability of compression (when writing) > > (and decompression when reading) in your database design. > > Do you have a model for tick data storage? Fields, etc. All I am allowed to say is that we have a proprietary model. > Why would I bother to compress/decompress? Wouldn't keeping 1-2-3Gb > in-memory be enough? Should I really care about disk storage? hmmm, I think this depends on the operation and the scale of your trading system. Usually, for real-time trading, you only need data as back as only a few months to get your initial parameters. If you don't access them very often during the trading, then IO is not an issue. And putting all today's data in memory is certainly ok with nowaday's computers. However, when you do backtesting (or model studying), I believe you will need several years of tick data, then you are likely gonna hit the issues of disk space and the efficiency of database IO. --HP From ok@REDACTED Fri Jul 8 03:10:22 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 8 Jul 2005 13:10:22 +1200 (NZST) Subject: New trading systems platform Message-ID: <200507080110.j681AM3M080714@atlas.otago.ac.nz> James Hague wrote: Much financial work like this is done in array languages, like K (http://www.kx.com) and J (http://www.jsoftware.com). They're designed to churn through large data sets quickly. J is free, so one conceivable option is to communicate with a J process from Erlang, but that may be too roundabout. J will make your head explode. For an open source array language which is pretty good at number crunching and superb at graphical presentation of results, has an extremely active user community, lots of books, *and* use in econometrics amongst other disciplines, you can't go past R (http://www.r-project.org). Honestly, I'd like to see some basic tuple math added to Erlang, but this clashes with the sensibilities of some people. For example: 1 + {1,2,3} -> {3,4,5} {1,2,3} + 1 -> {3,4,5} {1,2,3} + {10,20,30} -> {11,22,33} 2 * {1,2,3} -> {2,4,6} What should {1,2} + {10,20,30} do, and why? I don't think this would be that hard to hack into the emulator. Erlang is a language with run-time types, not compile-time types. In such a language, *in*compatibilities between built-in types and operators are to be cherished, not discarded, lest errors go unreported. The difficulty of hacking a feature into the emulator is not the real issue, but conceptual integrity and the effect on other uses of the language. It might, for example, be better to introduce a whole new "array" data type; that would be much more work, but it could yield better performance (using long-known techniques from APL) without sacrificing any of the run-time type checking we now have. From joelr1@REDACTED Fri Jul 8 07:49:41 2005 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 8 Jul 2005 07:49:41 +0200 Subject: New trading systems platform In-Reply-To: <200507080110.j681AM3M080714@atlas.otago.ac.nz> References: <200507080110.j681AM3M080714@atlas.otago.ac.nz> Message-ID: <6509694C-202B-4F73-9A3E-94720551DCFA@gmail.com> What is wrong with using lists for this purpose? You do process your time series sequentially, front to back, and I think you can do it recursively as well. Erlang has tail recursion and you can easily drop the first element of the list with [H|T]. I create http://groups-beta.google.com/group/uptick to disucss points specific to the platform as opposed to Erlang. Thanks, Joel > James Hague wrote: > Much financial work like this is done in array languages, like K > (http://www.kx.com) and J (http://www.jsoftware.com). They're > designed to churn through large data sets quickly. J is free, > so one > conceivable option is to communicate with a J process from > Erlang, but > that may be too roundabout. J will make your head explode. -- http://wagerlabs.com/uptick From erlang@REDACTED Fri Jul 8 08:55:19 2005 From: erlang@REDACTED (Peter-Henry Mander) Date: Fri, 08 Jul 2005 07:55:19 +0100 Subject: Metafrog is down. Message-ID: <1120805719.6543.9.camel@hymir.newport-networks.com> Hi Gurus, Sorry to bring bad news. This is the second high profile Yaws crash within a month. Is there a way of avoiding these embarrassing outages? http://metafrog.erlang-projects.org/ ERROR erlang code crashed: File: yaws/index.yaws:2 Reason: {{badmatch,{aborted,{node_not_running,nonode@REDACTED}}}, [{metafrog_log,get_latest,2}, {m1,wiki,0}, {m1,out,1}, {yaws_server,deliver_dyn_part,8}, {yaws_server,aloop,3}, {yaws_server,acceptor0,2}, {proc_lib,init_p,5}]} Req: {http_request,'GET',{abs_path,"/"},{1,1}} -- "London can take it". From ulf.wiger@REDACTED Fri Jul 8 10:25:44 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Fri, 8 Jul 2005 10:25:44 +0200 Subject: New trading systems platform Message-ID: HP Wei wrote: > I do not use Erlang often enough to make comments on > data storage using Mnesia. > I just want to point out that to store tick data > of many years needs a LOT of disk space plus > the capability of compression (when writing) > (and decompression when reading) in your database design. Some form of archiving mechanism seems to be needed. Re. compression, there is of course a very convenient form available to erlang programmers: Bin = term_to_binary(Data, [compressed]). Decompression is transparently done by binary_to_term(Bin). This uses LZW compression on the binary. Perhaps this is not good enough? Perhaps one could come up with a caching scheme in Mnesia, where table segments can be converted from ram_copies to disc_only_copies after a certain time? One can use a custom fragmentation scheme that divides objects into time periods. How many fragments could one have in one table? /Uffe From sanjaya@REDACTED Fri Jul 8 10:32:51 2005 From: sanjaya@REDACTED (Sanjaya Vitharana) Date: Fri, 8 Jul 2005 14:32:51 +0600 Subject: output format of term_to_binary & the logic inside it ?? Message-ID: <008601c58397$a0eeebc0$6900a8c0@wavenet.lk> Hi...!! Can, anybody explane me the output format of term_to_binary & the logic inside it ?? What is the Erlang binary data object & Erlang external term format ? NOTE: If this is already described in someware in your documentation kindly help me to find .... Ex1: 70> term_to_binary(255). <<131,97,255>> --- What is <<131,97...?? 71> term_to_binary(256). <<131,98,0,0,1,0>> --- Why this change from 97 to 98 <<131,98 72> term_to_binary(257). <<131,98,0,0,1,1>> --- . Also <<131,98,0,0, what means from 0.0 73> term_to_binary(258). <<131,98,0,0,1,2>> Ex2: 58> term_to_binary(2). <<131,97,2>> --- <<131,97 -- may be 2 places to discribe the integer type 59> term_to_binary(a). <<131,100,0,1,97>> --- <<131,100,0 -- anyhow why this becoms 3 places to discribe a atom. 60> term_to_binary(aa). <<131,100,0,2,97,97>> ----- What actually means <<131,100,0 ??? Ex3: Also see the difference in List & Tuple 61> term_to_binary([1,2,3]). <<131,107,0,3,1,2,3>> --- 107 is for Lists ??? 62> term_to_binary({1,2,3}). <<131,104,3,97,1,97,2,97,3>> --- 104 is for Tuples ??? 63> term_to_binary({{1,2,3},1}). <<131,104,2,104,3,97,1,97,2,97,3,97,1>> --- <<131,104,2, - 2 means number of eliments in a tuple is it ??? so 131 is for ??? Where is the actual location discribe this all Erlang binary types ? Regards, Need some help !!! Sanjaya -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasl_erlang@REDACTED Fri Jul 8 11:53:38 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Fri, 8 Jul 2005 02:53:38 -0700 (PDT) Subject: New trading systems platform In-Reply-To: Message-ID: <20050708095338.9517.qmail@web34413.mail.mud.yahoo.com> --- "Ulf Wiger (AL/EAB)" wrote: > [big datasets in mnesia] Handling relatively huge datasets in Erlang seems to be part of the zeitgeist, a view I get both from discussions on this list and elsewhere. Best, Thomas ____________________________________________________ Sell on Yahoo! Auctions ? no fees. Bid on great items. http://auctions.yahoo.com/ From chandrashekhar.mullaparthi@REDACTED Fri Jul 8 12:00:14 2005 From: chandrashekhar.mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Fri, 8 Jul 2005 11:00:14 +0100 Subject: New trading systems platform In-Reply-To: References: Message-ID: On 8 Jul 2005, at 09:25, Ulf Wiger (AL/EAB) wrote: > One can use a > custom fragmentation scheme that divides objects into time > periods. How many fragments could one have in one table? > I guess the number of fragments is limited by the number of file descriptors available to the node (unless there is some limitation in mnesia). Chandru From joelr1@REDACTED Fri Jul 8 12:18:00 2005 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 8 Jul 2005 12:18:00 +0200 Subject: New trading systems platform In-Reply-To: <20050708095338.9517.qmail@web34413.mail.mud.yahoo.com> References: <20050708095338.9517.qmail@web34413.mail.mud.yahoo.com> Message-ID: <5409F163-8307-4933-AF14-E7D12D325DB9@gmail.com> And why not? I see this as running a tail-recursive function on each element of a huge list. With the data all cached in memory. It seems like it has not been done before otherwise we would have had some data. I should have a proof of concept sometime soon and I really don't want to push lists of of Erlang for processing somewhere else. This would have do be done over sockets most likely and would only add latency. On Jul 8, 2005, at 11:53 AM, Thomas Lindgren wrote: > Handling relatively huge datasets in Erlang seems to > be part of the zeitgeist, a view I get both from > discussions on this list and elsewhere. -- http://wagerlabs.com/uptick From vlad@REDACTED Fri Jul 8 14:10:45 2005 From: vlad@REDACTED (Vlad Balin) Date: Fri, 8 Jul 2005 16:10:45 +0400 Subject: New trading systems platform In-Reply-To: Message-ID: > HP Wei wrote: > > I do not use Erlang often enough to make comments on > > data storage using Mnesia. > > I just want to point out that to store tick data > > of many years needs a LOT of disk space plus > > the capability of compression (when writing) > > (and decompression when reading) in your database design. > > Some form of archiving mechanism seems to be needed. > Re. compression, there is of course a very convenient > form available to erlang programmers: > > Bin = term_to_binary(Data, [compressed]). > > Decompression is transparently done by binary_to_term(Bin). > > This uses LZW compression on the binary. Perhaps this is > not good enough? It depends. If you represent tick sequence as list of relative price differences, it _probably_ will work. I would say, this is the thing one should start from. But usually custom huffman-alike compression scheme more effective in such applications in terms of combination of compression speed and ratio. > Perhaps one could come up with a caching scheme in Mnesia, > where table segments can be converted from ram_copies > to disc_only_copies after a certain time? One can use a > custom fragmentation scheme that divides objects into time > periods. How many fragments could one have in one table? In case of ATS we need disk tick storage for the backtesting purpose only. It implies very simple access scenario (rare look-ups, sequential [forward in time] read of large data volumes, "joins" of several tables by time field). It make sense to develop file-based storage for tick storage (one file per instrument, data stored sequentially by chunks of compressed ticks), and avoid mnesia. Vlad From james.hague@REDACTED Fri Jul 8 16:03:13 2005 From: james.hague@REDACTED (James Hague) Date: Fri, 8 Jul 2005 09:03:13 -0500 Subject: New trading systems platform In-Reply-To: <6509694C-202B-4F73-9A3E-94720551DCFA@gmail.com> References: <200507080110.j681AM3M080714@atlas.otago.ac.nz> <6509694C-202B-4F73-9A3E-94720551DCFA@gmail.com> Message-ID: > What is wrong with using lists for this purpose? > > You do process your time series sequentially, front to back, and I > think you can do it recursively as well. Erlang has tail recursion > and you can easily drop the first element of the list with [H|T]. You were worrying about Erlang performance for general number crunching relative to C. Most of the time I'd say using lists is fine, but not if you're potentially talking about gigabytes of data and the kind of processing where a factor of 10 speed increase is very significant. Erlang lists take twice as much space as tuples, and, much as I hate to admit it, if you just need lean and mean data processing on large and structurally simple data sets, then you'll always win in C (and by quite a lot, too). That said, I'd still try the raw Erlang solution and see how it goes. From james.hague@REDACTED Fri Jul 8 16:19:06 2005 From: james.hague@REDACTED (James Hague) Date: Fri, 8 Jul 2005 09:19:06 -0500 Subject: New trading systems platform In-Reply-To: <200507080110.j681AM3M080714@atlas.otago.ac.nz> References: <200507080110.j681AM3M080714@atlas.otago.ac.nz> Message-ID: Richard O'Keefe wrote: > What should {1,2} + {10,20,30} do, and why? It should exit with a badarith code. Why? There's no clear meaning to adding vectors of different lengths. (I tried it in J and got back "length error."). Likewise, {1,2} + {1,two} should also result in badarith. > Erlang is a language with run-time types, not compile-time types. > In such a language, *in*compatibilities between built-in types and > operators are to be cherished, not discarded, lest errors go unreported. > The difficulty of hacking a feature into the emulator is not the real > issue, but conceptual integrity and the effect on other uses of the language. True, I can understand that point. But at the same time, with test-driven development, I don't see it as any different than other issues caused by dynamic typing. > It might, for example, be better to introduce a whole new "array" data type; > that would be much more work, but it could yield better performance (using > long-known techniques from APL) without sacrificing any of the run-time > type checking we now have. Strictly from a selfish point of view based on the kind of applications I work on, I'd like to see "array of float" as a fundamental type. Floats are individually heap allocated, so there's a big win to putting them in a homogeneous array (OCaml has taken a similar route). This would also be a big space/time win for applications like Wings3D. But from a conceptual point of view, arrays and tuples are the same thing, so why split them up? From hp@REDACTED Fri Jul 8 16:41:27 2005 From: hp@REDACTED (HP Wei) Date: Fri, 8 Jul 2005 10:41:27 -0400 (EDT) Subject: New trading systems platform In-Reply-To: References: <200507080110.j681AM3M080714@atlas.otago.ac.nz> <6509694C-202B-4F73-9A3E-94720551DCFA@gmail.com> Message-ID: On Fri, 8 Jul 2005, James Hague wrote: > > What is wrong with using lists for this purpose? > > > > You do process your time series sequentially, front to back, and I > > think you can do it recursively as well. Erlang has tail recursion > > and you can easily drop the first element of the list with [H|T]. > > You were worrying about Erlang performance for general number > crunching relative to C. Most of the time I'd say using lists is > fine, but not if you're potentially talking about gigabytes of data > and the kind of processing where a factor of 10 speed increase is very > significant. Exactly!! Especially when it is time and $$ on the line, those big shots on Wall Street won't move away from C/C++ even when they are ugly and hard to maintain... > Erlang lists take twice as much space as tuples, and, > much as I hate to admit it, if you just need lean and mean data > processing on large and structurally simple data sets, then you'll > always win in C (and by quite a lot, too). In addition to financial world where data set is huge (but with simple structures), another example is physics world where an experiment at an accerlerator generates gigantic data sets. The TTree class in ROOT (C++) package (from CERN) might give an example as to how to organize (write with compression and read with decompression) large data set. > That said, I'd still try the raw Erlang solution and see how it goes. Please do. For a small scale trading system, Erlang should be nicer than the 'popular' TradeStation (with its ugly (macro) Easy Language). --HP From mickael.remond@REDACTED Fri Jul 8 15:22:51 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Fri, 08 Jul 2005 15:22:51 +0200 Subject: Metafrog is down. In-Reply-To: <1120805719.6543.9.camel@hymir.newport-networks.com> References: <1120805719.6543.9.camel@hymir.newport-networks.com> Message-ID: <42CE7E2B.8010808@erlang-fr.org> Peter-Henry Mander wrote: > Hi Gurus, > > Sorry to bring bad news. This is the second high profile Yaws crash > within a month. Is there a way of avoiding these embarrassing outages? It is up and running again. The partition where this application is running had no more disk space and this cause Mnesia to crash. I made more space available and relaunched the system. To avoid this kind of problem, I need to add monitoring feature to the server, to send me email when so particular indicators are worrying. I will set this up as soon as possible. Thank you for letting me know. -- Micka?l R?mond http://www.erlang-projects.org/ From ulf.wiger@REDACTED Fri Jul 8 18:53:41 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Fri, 8 Jul 2005 18:53:41 +0200 Subject: New trading systems platform Message-ID: Vlad Balin wrote: > > In case of ATS we need disk tick storage for the backtesting > purpose only. It implies very simple access scenario (rare > look-ups, sequential [forward in time] read of large data > volumes, "joins" of several tables by time field). It make > sense to develop file-based storage for tick storage (one > file per instrument, data stored sequentially by chunks of > compressed ticks), and avoid mnesia. Perhaps mnesia should be avoided... I'd still like to explore ways to keep it in the game though. (: The best way to dump large sets of tuples to disk in mnesia, if sequential storage is needed, is disk_log. disk_log:chunk() allows you to fetch one object at a time. It's hardly going to compete performance-wise with a custom solution written in C/C++. ...or could it? We've seen several times that total system performance in Erlang often compares very well with that of similar systems written in C. This gets more obvious the more complex the application is, and perhaps especially the more concurrency that needs to be handled. One potential problem here is if there is some frequently executed "tight loop" that is difficult to make fast in erlang. Storing data in RAM, in erlang format, helps reduce the semantic gap. Mnesia and ets have some nice fold functions. Does the data always get accessed sequentially, or are queries also needed? How about disc_copies (which are stored using disk_log on disk) that are cleared out of RAM when they get old enough? When accessed, they'd get loaded again and this fully transparent to the programmer. I foresee some mind-expanding modifications of the core mnesia table loading code -- good fun! A final thought. Performance in disk-based accesses can be improved significantly if one can nail down the data format. This is something I think should be an option in mnesia. /Uffe From mikael.karlsson@REDACTED Fri Jul 8 19:22:04 2005 From: mikael.karlsson@REDACTED (Mikael Karlsson) Date: Fri, 8 Jul 2005 19:22:04 +0200 Subject: Web interface framework for Erlang In-Reply-To: <20050705.161429.73661648.svg@surnet.ru> References: <200507051118.30227.mikael.karlsson@creado.com> <20050705.161429.73661648.svg@surnet.ru> Message-ID: <200507081922.04854.mikael.karlsson@creado.com> Looking at what is in there it seems to be quite some work put into it and since the description fits what was asked for earlier I think it would be a good idea to package it. I sure would not mind to try it out for some applications. Best Regards Mikael - Introduction - WSTL is a framework developped by Vladimir Sekissov. It is based on a - component approach and provide all the needed features to develop - consistent and maintenable Web Application: Tue 05 July 2005 12:14 Vladimir Sekissov wrote: > Good day, > > mikael.karlsson> I looked around a bit more on the Erlang list and found > mikael.karlsson> the attachement ws.tar.gz in posting: > mikael.karlsson> > mikael.karlsson> > http://www.erlang.org/ml-archive/erlang-questions/200502/msg00289.html > mikael.karlsson> > mikael.karlsson> Extract from file ws/stl/doc/manual.tex: > ... > mikael.karlsson> Does anybody now the status of this? > > I haven't time for a year to bring the code to the state when it could > be usable by others except me. But if somebody is interesting in it I > try to package all stuff with examples at weekends. > > Best Regards, > Vladimir Sekissov From nm@REDACTED Fri Jul 8 20:01:28 2005 From: nm@REDACTED (Gaspar Chilingarov) Date: Fri, 08 Jul 2005 23:01:28 +0500 Subject: precise debugging :) Message-ID: <42CEBF78.1040007@web.am> Hi all! While my erlang program is running it sometimes gives me messages like - Error in process <0.443.0> with exit value: {badarg,[{udp_driver,input_handler,2},{udp_driver,input_handler_msg,1}]} which is totally useless, because input_handler have a lot of clauses. are there any ways to get precise point, where I got failures or I should enable tracing and look into trace log? thanks in advance -- Gaspar Chilingarov System Administrator t +37491 419763 w www.web.am e nm@REDACTED From joelr1@REDACTED Fri Jul 8 20:51:31 2005 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 8 Jul 2005 20:51:31 +0200 Subject: Storing time series data Message-ID: <93176FC8-54F8-4BBD-AD82-89A45A97B8BB@gmail.com> Here's how I think time series can be stored in Mnesia or on disk... A tick is tuple with a fixed size, indexed on a time stamp. I think that rather than storing a regular list of tick in Mnesia I should store compressed term binaries. A single binary could be a list or tuple of 1 week's worth of tick data, maybe a month's worth. A separate index will need to be kept to map time stamp to binary chunk and offset within the binary. The storage has to be optimized for fast retrieval and ticks are normally appended as opposed to inserted. Inserting a new tick somewhere in the middle would require rehashing the whole database but I think this will be quite rare. Normally, ticks will be appended to binary chunks and chunks appended to the database. A trading strategy will need a sliding window of N ticks where the first tick is always dropped and the window is advanced by one tick. I think it will be quite easy to slide through binary chunks and if I understand it correctly binaries will not be copied but references will be created. Unless of course the sliding window spans a few binary chunks. What do you think? Thanks, Joel -- http://wagerlabs.com/uptick From joelr1@REDACTED Fri Jul 8 21:09:39 2005 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 8 Jul 2005 21:09:39 +0200 Subject: New trading systems platform In-Reply-To: References: <200507080110.j681AM3M080714@atlas.otago.ac.nz> <6509694C-202B-4F73-9A3E-94720551DCFA@gmail.com> Message-ID: On Jul 8, 2005, at 4:03 PM, James Hague wrote: > Erlang lists take twice as much space as tuples, and, > much as I hate to admit it, Wow, I didn't know this! Is there any particular reason for that? > if you just need lean and mean data > processing on large and structurally simple data sets, then you'll > always win in C (and by quite a lot, too). I would like to avoid that for as long as possible. > That said, I'd still try the raw Erlang solution and see how it goes. I set up http://groups-beta.google.com/group/uptick to discuss the particulars of my platform, as opposed to particulars of Erlang. Please feel free to join everyone! Joel -- http://wagerlabs.com/uptick From joelr1@REDACTED Fri Jul 8 21:16:03 2005 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 8 Jul 2005 21:16:03 +0200 Subject: New trading systems platform In-Reply-To: References: <200507080110.j681AM3M080714@atlas.otago.ac.nz> <6509694C-202B-4F73-9A3E-94720551DCFA@gmail.com> Message-ID: <50DAC947-64FF-4DBB-9D9D-8C6C779E1FA5@gmail.com> QuantStudio/QuantDeveloper is a .NET trading systems toolkit that I sold for a year. It takes its roots in R. It's quite unmanageable for a regular programmer wanabee and even for a seasoned developer. I would like to avoid the mistakes of QS/QD and make the interface as user-friendly as possible. Maybe even use a custom language to code systems and totally hide Erlang. Joel On Jul 8, 2005, at 4:41 PM, HP Wei wrote: > The TTree class in ROOT (C++) package (from CERN) might > give an example as to how to organize (write with compression > and read with decompression) large data set. -- http://wagerlabs.com/uptick From joelr1@REDACTED Sat Jul 9 10:03:12 2005 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 9 Jul 2005 10:03:12 +0200 Subject: New trading systems platform In-Reply-To: References: <200507080110.j681AM3M080714@atlas.otago.ac.nz> <6509694C-202B-4F73-9A3E-94720551DCFA@gmail.com> Message-ID: <7D763D6F-FBC5-403A-A30E-8A7673D71B6E@gmail.com> This is an interesting information that might be of use for Mnesia: http://kx.com/papers/kdb-tickdemo.php "Kdb achieves this kind of efficiency because of its design. It is fully vertically partitioned and replaces tables by ordered arrays. For this reason, moving averages, correlations and other such queries can be done directly in the database." Joel -- http://wagerlabs.com/uptick From robert.balogh@REDACTED Sat Jul 9 15:15:13 2005 From: robert.balogh@REDACTED (=?iso-8859-1?Q?R=F3bert_Balogh_=28IJ/ETH=29?=) Date: Sat, 9 Jul 2005 15:15:13 +0200 Subject: unix pid Message-ID: Hej erlang experts, I'd like to ask You if I'm on UNIX/Linux system and I do eq: os:cmd(cmdtool). in erlang, how possible to know the pid of that process what is created by this? thanks a lot for Your help. br, /Robi From meena_selvam@REDACTED Sat Jul 9 17:22:32 2005 From: meena_selvam@REDACTED (MEENA SELVAM) Date: Sat, 9 Jul 2005 08:22:32 -0700 (PDT) Subject: clarification needed in this code snippet Message-ID: <20050709152232.89023.qmail@web30402.mail.mud.yahoo.com> Hi, I have not understood how the following code functions: To me, it appears that in lines 8-9, pattern matching occurs based on the output of pc_lookup function, and the unbound variable Cookie gets bound to either undefined or any other value. Different actions are taken depending on Cookie value. But the output of pc_lookup should be a port_cache record for this to happen. But according to definition of pc_lookup, the output is a record of type sac_port_cache. This is a working code, and I don't follow the discrepancy in the record type of the output. Can anyone clarify for me? 1 %% xsp = {xid(), sid(), int()} 2 -record(port_cache, {xsp, type, dcookie, pcookie}). 3 -define(PC_TAB, sac_port_cache). ... ... 4 init() -> 5 ets:new(?PC_TAB, [set, protected, named_table, {keypos, #port_cache.xsp}]), ... 6 existing_dyn_sessionp(Switch, Port, Mac) -> 7 XSP = mk_xsp(Switch, Port), 8 case pc_lookup(XSP) of 9 #port_cache{ dcookie = Cookie } when Cookie /= undefined -> .... 10 #port_cache{ dcookie = Cookie } when Cookie == undefined -> ..... 11 end. ... ... 12 pc_lookup(XSP) -> 13 case ets:lookup(?PC_TAB, XSP) of 14 [Val] -> 15 Val; 16 [] -> 17 not_found 18 end. ____________________________________________________ Sell on Yahoo! Auctions ? no fees. Bid on great items. http://auctions.yahoo.com/ From erlang@REDACTED Sat Jul 9 18:05:25 2005 From: erlang@REDACTED (Michael McDaniel) Date: Sat, 9 Jul 2005 09:05:25 -0700 Subject: unix pid In-Reply-To: References: Message-ID: <20050709160524.GH11207@fangora.autosys.us> On Sat, Jul 09, 2005 at 03:15:13PM +0200, R?bert Balogh (IJ/ETH) wrote: > Hej erlang experts, > > I'd like to ask You if I'm on UNIX/Linux system and I do eq: os:cmd(cmdtool). in erlang, how possible to know the pid of that process what is created by this? > > thanks a lot for Your help. > > br, > /Robi ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ I am not certain I understand your question. The following may be of assistance. The first example (1>, 2>) shows Erlang system information about the PID returned from the Erlang process. Note I ran spawn/3 otherwise the os:cmd/1 would not return until after "sleep 7" finished. The second example (3>, 4>) shows host OS PID of the background 'find /' process. Note I ran "find / &" as background process. $ erl Erlang (BEAM) emulator version 5.4.8 [hipe] Eshell V5.4.8 (abort with ^G) 1> spawn(os, cmd, ["sleep 7"]). <0.35.0> 2> i(0,35,0). [{current_function,{os,unix_cmd,1}}, {initial_call,{os,cmd,1}}, {status,waiting}, {message_queue_len,0}, {messages,[]}, {links,[<0.36.0>]}, {dictionary,[]}, {trap_exit,false}, {error_handler,error_handler}, {priority,normal}, {group_leader,<0.24.0>}, {heap_size,233}, {stack_size,3}, {reductions,14}, {garbage_collection,[{fullsweep_after,65535}]}] 3> 3> os:cmd("find / &"). [] 4> os:cmd("ps xauw | egrep find | egrep -v grep"). "mmcdanie 10468 0.0 0.1 2552 700 ? D 08:55 0:03 find /\n" 5> ~Michael From rprice@REDACTED Sat Jul 9 19:03:09 2005 From: rprice@REDACTED (Roger Price) Date: Sat, 9 Jul 2005 19:03:09 +0200 (CEST) Subject: Comment on comment Message-ID: I would like to suggest a clarification of the specification of a comment in the Erlang Reference Manual section 1.4.3. The current specification is: ? Comments are allowed anywhere within a module. Comments start with the character % and stop at end of line. ? I think it would be more precise to say: Comments may be placed anywhere in a module except within strings and atoms. The comment begins with the character "%", continues up to, but does not include the next end-of-line, and has the value of white space. Example 1 - a comment "Hello World" % Well known string Example 2 - comment introduces white space 42> 12% 42> 34 . ** 2: syntax error before: 34 ** Example 3 - string and atom, not comments 46> {"% percent",'%%%%%'} . {"% percent",'%%%%%'} Roger From joelr1@REDACTED Sat Jul 9 20:31:55 2005 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 9 Jul 2005 20:31:55 +0200 Subject: I must be stupid Message-ID: Could you please tell me what's wrong with my code below? I'm using 10RB6 on Mac OSX 10.4.1. -module(tuple). -compile([export_all]). test1(N) -> populate1([], N). populate1(Acc, 0) -> Acc; populate1(Acc, Size) -> Acc1 = [Size|Acc], populate1(Acc1, Size - 1). What I get: 2> hipe:c(tuple). {ok,tuple} 3> timer:tc(tuple, test1, [1000000]). {58,{'EXIT',{badarg,[]}}} 9> tuple:populate1([], 0). [] 10> tuple:populate1([], 1). [1] 11> tuple:populate1([], 2). =ERROR REPORT==== 9-Jul-2005::20:27:33 === Error in process <0.42.0> with exit value: {badarg,[]} ** exited: {badarg,[]} ** Thanks, Joel -- http://wagerlabs.com/uptick From joelr1@REDACTED Sat Jul 9 20:34:12 2005 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 9 Jul 2005 20:34:12 +0200 Subject: I must be stupid In-Reply-To: References: Message-ID: <8E1BCA8B-8034-44E3-B0E0-47BEDD920659@gmail.com> Actually, this looks like a HiPE issue. 1> c(tuple). {ok,tuple} 2> tuple:populate1([], 2). [1,2] 3> timer:tc(tuple, test1, [1000000]). {1281653, [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27 |...]} The above works fine. Another interesting issue is that creating a list of 1,000,000 items in this fashion populate1(Acc, 0) -> Acc; populate1(Acc, Size) -> Acc1 = [Size|Acc], populate1(Acc1, Size - 1). creates just a second without HiPE whereas doing it like this takes I don't know how much time (stopped after half an hour or so) populate(T, 0) -> T; populate(T, Size) -> populate(setelement(Size, T, Size), Size - 1). Is there a simple explanation? Thanks, Joel -- http://wagerlabs.com/uptick From hakan.stenholm@REDACTED Sat Jul 9 21:11:18 2005 From: hakan.stenholm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Stenholm?=) Date: Sat, 09 Jul 2005 21:11:18 +0200 Subject: I must be stupid In-Reply-To: <8E1BCA8B-8034-44E3-B0E0-47BEDD920659@gmail.com> References: <8E1BCA8B-8034-44E3-B0E0-47BEDD920659@gmail.com> Message-ID: <42D02156.8090104@mbox304.swipnet.se> Joel Reymont wrote: > .... > > creates just a second without HiPE whereas doing it like this takes I > don't know how much time (stopped after half an hour or so) > > populate(T, 0) -> > T; > > populate(T, Size) -> > populate(setelement(Size, T, Size), Size - 1). > > Is there a simple explanation? Yes. Updating a tuple _allways_ creates a new copy of the updated tuple [1] - as there is no destructive update. Attaching a new element to the head of a list, only allocates memory for that element. In other words a size N tuple in populate/2, will allocate memory for N*N elements [2], rather than N elements, as in the populate1/2 call, so things will obviously go much slower. [1]: a new copy of the internal pointer array used - the data pointed to will remain unchanged, only one or more tuple pointers will differ in the new tuple. [2]: where N*(N-1) elements of the N*N elements, will later be discarded by the garbage collector. > > Thanks, Joel > > -- > http://wagerlabs.com/uptick > > > > From ulf.wiger@REDACTED Sat Jul 9 21:21:16 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Sat, 9 Jul 2005 21:21:16 +0200 Subject: I must be stupid Message-ID: This is true. The 'lines' contrib at http://jungerl.sourceforge.net was an attempt to be able to handle very large data structures with tuple semantics. It uses a tree of tuples, so it has O(logN) complexity. It will never copy large tuples on update. On the operation you're trying, nothing in Erlang will beat a list though. Iterating over a tuple is not going to be faster than iterating over a list, either(*). It's only when you require random access that the tuple starts to shine. If the ratio of random access over updates is high enough, large tuples might be a viable option. /Uffe (*) At least not in BEAM. Don't know about HiPE. > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of H?kan Stenholm > Sent: den 9 juli 2005 21:11 > To: Joel Reymont > Cc: Erlang Users' List > Subject: Re: I must be stupid > > > Joel Reymont wrote: > > > .... > > > > creates just a second without HiPE whereas doing it like > this takes I > > don't know how much time (stopped after half an hour or so) > > > > populate(T, 0) -> > > T; > > > > populate(T, Size) -> > > populate(setelement(Size, T, Size), Size - 1). > > > > Is there a simple explanation? > > Yes. > > Updating a tuple _allways_ creates a new copy of the updated > tuple [1] - > as there is no destructive update. > > Attaching a new element to the head of a list, only allocates > memory for > that element. > > In other words a size N tuple in populate/2, will allocate memory for > N*N elements [2], rather than N elements, as in the > populate1/2 call, so > things will obviously go much slower. > > [1]: a new copy of the internal pointer array used - the data > pointed to > will remain unchanged, only one or more tuple pointers will differ in > the new tuple. > [2]: where N*(N-1) elements of the N*N elements, will later > be discarded > by the garbage collector. > > > > > Thanks, Joel > > > > -- > > http://wagerlabs.com/uptick > > > > > > > > > > From joelr1@REDACTED Sat Jul 9 21:36:53 2005 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 9 Jul 2005 21:36:53 +0200 Subject: When are binaries copied as opposed to referenced? Message-ID: <5021A269-8B28-45B1-8A3B-4A746060D540@gmail.com> Folks, Would you comment on my assumptions regarding Erlang in this post: http://wagerlabs.com/uptick/2005/07/efficient-backtesting-and-data- storage.html Suppose I have a few million ticks to test on. I would need a "sliding window" onto this data where I always add to the front of the window and drop the last tick as I move forward in time. Lets say I need to keep at least 500 ticks around for my strategy to work. If I keep the orinal few million ticks in memory as a large binary or a list of binaries then the sliding window would most likely be created as a reference into the large binary as opposed to a copy. If that is the case then there should be no garbage collection. Multiple processes (strategies) running in the same VM would obtain as many references to the large binary as they need. Is this correct? Thanks, Joel -- http://wagerlabs.com/uptick From richardc@REDACTED Sun Jul 10 00:34:05 2005 From: richardc@REDACTED (Richard Carlsson) Date: Sun, 10 Jul 2005 00:34:05 +0200 Subject: I must be stupid In-Reply-To: References: Message-ID: <42D050DD.8010207@csd.uu.se> Joel Reymont wrote: > Could you please tell me what's wrong with my code below? I'm using > 10RB6 on Mac OSX 10.4.1. > > ... > > What I get: > > 2> hipe:c(tuple). > {ok,tuple} > 3> timer:tc(tuple, test1, [1000000]). > {58,{'EXIT',{badarg,[]}}} If this is a HiPE problem, could you first check that you get the same behaviour with the following compiler invokation: > c(tuple, [native]). (rather than using the hipe module directly)? /Richard From erlang@REDACTED Sun Jul 10 03:24:31 2005 From: erlang@REDACTED (Michael McDaniel) Date: Sat, 9 Jul 2005 18:24:31 -0700 Subject: unix pid In-Reply-To: References: Message-ID: <20050710012430.GO11207@fangora.autosys.us> I think I understand your question. Perhaps I am misunderstanding the Erlang documentation. os:cmd/1 says... --------------------------------------------- cmd(Command) -> string() Types: Command = string() | atom() Executes Command in a command shell of the target OS and returns the result as a string. This function is a replacement of the previous unix:cmd/1; on a Unix platform they are equivalent. Examples: LsOut = os:cmd("ls"), % on unix platform DirOut = os:cmd("dir"), % on Win32 platform --------------------------------------------- If I do the following from my (Linux) system prompt $ sleep 97 & [3] 11702 $ ps xauw | egrep sleep | egrep -v mmcdanie 11702 0.0 0.1 2732 672 pts/49 S 18:23 0:00 sleep 97 $ Since the Pid is returned as a string from the command prompt, I would expect the following from within Erlang to return the Pid also, though it does not. $ erl Erlang (BEAM) emulator version 5.4.8 [hipe] Eshell V5.4.8 (abort with ^G) 1> os:cmd("sleep 97 &"). [] 2> q(). ok $ I do not know how to associate a specific Unix PID with the specific Erlang instance/Pid that created the Unix PID. ~Michael On Sat, Jul 09, 2005 at 06:15:52PM +0200, R?bert Balogh (IJ/ETH) wrote: > Hej Michael, > > Sorry for that the not clear question. So I'd like to know the pid of the UNIX process what is exits by erlang os:cmd(..). I know what You're sent to me. The second example is closer to my question, but it's not perfect, because: > - I start several erlang nodes in the same unix > - and in every erlang node I'm starting the same unix process (!!) and I have to know the unix process what I created at the start phase, because when this erlang process is terminate must be terminate the UNIX process to what is belongs to this erlang process. If I follow the second example it's not possible to separate the UNIX process which belongs to which erlang if the several erlang processes in the erlang nodes are starting in the same time. > > Is this clearer? Sorry again for this... > > br, > /Robi > > -----Original Message----- > From: Michael McDaniel [mailto:erlang@REDACTED] > Sent: Saturday, July 09, 2005 6:05 PM > To: R?bert Balogh (IJ/ETH) > Cc: Erlang question (E-mail) > Subject: Re: unix pid > > > On Sat, Jul 09, 2005 at 03:15:13PM +0200, R?bert Balogh (IJ/ETH) wrote: > > Hej erlang experts, > > > > I'd like to ask You if I'm on UNIX/Linux system and I do eq: os:cmd(cmdtool). in erlang, how possible to know the pid of that process what is created by this? > > > > thanks a lot for Your help. > > > > br, > > /Robi > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > I am not certain I understand your question. The following may be of assistance. > > The first example (1>, 2>) shows Erlang system information about the PID returned from the > Erlang process. Note I ran spawn/3 otherwise the os:cmd/1 would not return until after > "sleep 7" finished. > > The second example (3>, 4>) shows host OS PID of the background 'find /' process. > Note I ran "find / &" as background process. > > > $ erl > Erlang (BEAM) emulator version 5.4.8 [hipe] > > Eshell V5.4.8 (abort with ^G) > 1> spawn(os, cmd, ["sleep 7"]). > <0.35.0> > 2> i(0,35,0). > [{current_function,{os,unix_cmd,1}}, > {initial_call,{os,cmd,1}}, > {status,waiting}, > {message_queue_len,0}, > {messages,[]}, > {links,[<0.36.0>]}, > {dictionary,[]}, > {trap_exit,false}, > {error_handler,error_handler}, > {priority,normal}, > {group_leader,<0.24.0>}, > {heap_size,233}, > {stack_size,3}, > {reductions,14}, > {garbage_collection,[{fullsweep_after,65535}]}] > 3> > 3> os:cmd("find / &"). > [] > 4> os:cmd("ps xauw | egrep find | egrep -v grep"). > "mmcdanie 10468 0.0 0.1 2552 700 ? D 08:55 0:03 find /\n" > 5> > > ~Michael From robert.balogh@REDACTED Sun Jul 10 09:01:13 2005 From: robert.balogh@REDACTED (=?iso-8859-1?Q?R=F3bert_Balogh_=28IJ/ETH=29?=) Date: Sun, 10 Jul 2005 09:01:13 +0200 Subject: unix pid Message-ID: Hej Michael, Thanks for Your answer. So this is what I don't know, how it possible to associate the UNIX pid to erlang pid :-( Maybe some other erlang experts will be knows. thanks again, br, /Robi -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED] Sent: Sunday, July 10, 2005 3:25 AM To: erlang-questions@REDACTED Subject: Re: unix pid I think I understand your question. Perhaps I am misunderstanding the Erlang documentation. os:cmd/1 says... --------------------------------------------- cmd(Command) -> string() Types: Command = string() | atom() Executes Command in a command shell of the target OS and returns the result as a string. This function is a replacement of the previous unix:cmd/1; on a Unix platform they are equivalent. Examples: LsOut = os:cmd("ls"), % on unix platform DirOut = os:cmd("dir"), % on Win32 platform --------------------------------------------- If I do the following from my (Linux) system prompt $ sleep 97 & [3] 11702 $ ps xauw | egrep sleep | egrep -v mmcdanie 11702 0.0 0.1 2732 672 pts/49 S 18:23 0:00 sleep 97 $ Since the Pid is returned as a string from the command prompt, I would expect the following from within Erlang to return the Pid also, though it does not. $ erl Erlang (BEAM) emulator version 5.4.8 [hipe] Eshell V5.4.8 (abort with ^G) 1> os:cmd("sleep 97 &"). [] 2> q(). ok $ I do not know how to associate a specific Unix PID with the specific Erlang instance/Pid that created the Unix PID. ~Michael On Sat, Jul 09, 2005 at 06:15:52PM +0200, R?bert Balogh (IJ/ETH) wrote: > Hej Michael, > > Sorry for that the not clear question. So I'd like to know the pid of the UNIX process what is exits by erlang os:cmd(..). I know what You're sent to me. The second example is closer to my question, but it's not perfect, because: > - I start several erlang nodes in the same unix > - and in every erlang node I'm starting the same unix process (!!) and I have to know the unix process what I created at the start phase, because when this erlang process is terminate must be terminate the UNIX process to what is belongs to this erlang process. If I follow the second example it's not possible to separate the UNIX process which belongs to which erlang if the several erlang processes in the erlang nodes are starting in the same time. > > Is this clearer? Sorry again for this... > > br, > /Robi > > -----Original Message----- > From: Michael McDaniel [mailto:erlang@REDACTED] > Sent: Saturday, July 09, 2005 6:05 PM > To: R?bert Balogh (IJ/ETH) > Cc: Erlang question (E-mail) > Subject: Re: unix pid > > > On Sat, Jul 09, 2005 at 03:15:13PM +0200, R?bert Balogh (IJ/ETH) wrote: > > Hej erlang experts, > > > > I'd like to ask You if I'm on UNIX/Linux system and I do eq: os:cmd(cmdtool). in erlang, how possible to know the pid of that process what is created by this? > > > > thanks a lot for Your help. > > > > br, > > /Robi > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > I am not certain I understand your question. The following may be of assistance. > > The first example (1>, 2>) shows Erlang system information about the PID returned from the > Erlang process. Note I ran spawn/3 otherwise the os:cmd/1 would not return until after > "sleep 7" finished. > > The second example (3>, 4>) shows host OS PID of the background 'find /' process. > Note I ran "find / &" as background process. > > > $ erl > Erlang (BEAM) emulator version 5.4.8 [hipe] > > Eshell V5.4.8 (abort with ^G) > 1> spawn(os, cmd, ["sleep 7"]). > <0.35.0> > 2> i(0,35,0). > [{current_function,{os,unix_cmd,1}}, > {initial_call,{os,cmd,1}}, > {status,waiting}, > {message_queue_len,0}, > {messages,[]}, > {links,[<0.36.0>]}, > {dictionary,[]}, > {trap_exit,false}, > {error_handler,error_handler}, > {priority,normal}, > {group_leader,<0.24.0>}, > {heap_size,233}, > {stack_size,3}, > {reductions,14}, > {garbage_collection,[{fullsweep_after,65535}]}] > 3> > 3> os:cmd("find / &"). > [] > 4> os:cmd("ps xauw | egrep find | egrep -v grep"). > "mmcdanie 10468 0.0 0.1 2552 700 ? D 08:55 0:03 find /\n" > 5> > > ~Michael From joelr1@REDACTED Sun Jul 10 09:03:19 2005 From: joelr1@REDACTED (Joel Reymont) Date: Sun, 10 Jul 2005 09:03:19 +0200 Subject: I must be stupid In-Reply-To: <42D050DD.8010207@csd.uu.se> References: <42D050DD.8010207@csd.uu.se> Message-ID: I do not get the error after c(tuple, [native]). Strangely enough, I do not get the error after hipe:c(tuple) either. Wierd. On Jul 10, 2005, at 12:34 AM, Richard Carlsson wrote: > If this is a HiPE problem, could you first check that you get > the same behaviour with the following compiler invokation: > > > c(tuple, [native]). > > (rather than using the hipe module directly)? -- http://wagerlabs.com/uptick From robert.balogh@REDACTED Sun Jul 10 15:31:10 2005 From: robert.balogh@REDACTED (=?iso-8859-1?Q?R=F3bert_Balogh_=28IJ/ETH=29?=) Date: Sun, 10 Jul 2005 15:31:10 +0200 Subject: mnesia db dir Message-ID: Hej mnesia experts, I'm a beginner in mnesia and I have some questions about mnesia. Sorry but I couldn't find the answer for these in the mnesia doc :-( The first question is it possible to set the path of the mnesia db from "code". I mean is there other way than erl -sname nodename -mnesia dir '"dirpath"'? I wrote a small code for learning mnesia and I take a big "discovery". It's not possible to insert any record in the table, if the table name is different that the name of his record. Here is my example: -module(ex_mnesia). -export([start/0]). -export([insert/1]). -record(rPerson,{name = [], city = []}). start()-> mnesia:create_schema([node()]), mnesia:start(), mnesia:create_table(table,[{attributes,record_info(fields,rPerson)}, {disc_copies,[node()]}]). insert(Value)-> Fun = fun()-> mnesia:write(table,#rPerson{name = Value},write) end, mnesia:transaction(Fun). After calling insert I got this in the erlang shell: (a@REDACTED)5> ex_mnesia:insert(aa). {aborted,{bad_type,{rPerson,aa,[]}}} If I change the name of table to rPerson it's possible to insert records in the table. So is that true that no way to set different name the table as his record name or I did something wrong? Here is what I got after change table name: (a@REDACTED)3> ex_mnesia:insert(aa). {atomic,ok} thanks a lot the answers, br, /Robi From ulf.wiger@REDACTED Sun Jul 10 17:45:58 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Sun, 10 Jul 2005 17:45:58 +0200 Subject: mnesia db dir Message-ID: R?bert Balogh wrote: > > The first question is it possible to set the path of the > mnesia db from "code". I mean is there other way than erl > -sname nodename -mnesia dir '"dirpath"'? You can use application:set_env(mnesia, dir, "..."). You should do this after loading the mnesia application, but before starting it. This is not something you should make a habit out of. application:set_env/3 is mainly there for software upgrade. A more structured way to do it (if you don't want to pass it as a command-line parameter) is to use a sys.config file. This is when you use the sasl framework to start your system. > I wrote a small code for learning mnesia and I take a big > "discovery". It's not possible to insert any record in the > table, if the table name is different that the name of his record. > Here is my example: You can use the option {record_name, rPerson} when creating the table. Mnesia will then verify that all objects inserted in 'table' are of the type #rPerson{} (as defined by the combination of the 'record_name' and 'attributes' options. /Uffe > > -module(ex_mnesia). > > -export([start/0]). > > -export([insert/1]). > > -record(rPerson,{name = [], > city = []}). > > > start()-> > mnesia:create_schema([node()]), > mnesia:start(), > > mnesia:create_table(table,[{attributes,record_info(fields,rPerson)}, > {disc_copies,[node()]}]). > > > insert(Value)-> > Fun = fun()-> > mnesia:write(table,#rPerson{name = Value},write) > end, > mnesia:transaction(Fun). > > After calling insert I got this in the erlang shell: > (a@REDACTED)5> ex_mnesia:insert(aa). > {aborted,{bad_type,{rPerson,aa,[]}}} > > If I change the name of table to rPerson it's possible to > insert records in the table. So is that true that no way to > set different name the table as his record name or I did > something wrong? > > Here is what I got after change table name: > (a@REDACTED)3> ex_mnesia:insert(aa). > {atomic,ok} > > > thanks a lot the answers, > > br, > /Robi > > > > From ulf.wiger@REDACTED Sun Jul 10 17:49:53 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Sun, 10 Jul 2005 17:49:53 +0200 Subject: mnesia db dir Message-ID: R?bert Balogh > The first question is it possible to set the path of the > mnesia db from "code". I mean is there other way than erl > -sname nodename -mnesia dir '"dirpath"'? Sorry, I was a little trigger-happy. Using mnesia:start(ExtraEnv) should do the trick, e.g. mnesia:start([{dir, "..."}]). This effectively does the same thing as calling application:set_env/3 after loading, but before starting mnesia, but is obviously much more convenient. Regards, Uffe From ok@REDACTED Mon Jul 11 00:11:54 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 11 Jul 2005 10:11:54 +1200 (NZST) Subject: New trading systems platform Message-ID: <200507102211.j6AMBsSS102825@atlas.otago.ac.nz> I asked a trick question: > What should {1,2} + {10,20,30} do, and why? and James Hague fell RIGHT into the trap. It should exit with a badarith code. Why? There's no clear meaning to adding vectors of different lengths. (I tried it in J and got back "length error."). Likewise, {1,2} + {1,two} should also result in badarith. But there is another, considerably more popular, array language in which applying a binary operation to vectors of different lengths IS defined, and for excellent reasons: > c(1,2) + c(10,20,30) [1] 11 22 31 I am not going to say "this one is right, that one is wrong", my point is that there is no *OBVIOUS* answer and it takes a great deal of hard thinking to produce a real design. It's NOT just a matter of hacking on the VM. Oh yes, and {1,2} + {1,two} tells us that there is a question about what, precisely, the type error should complain about. Should it complain about (2, two) or should it complain about ({1,2}, {1,two})? You can argue this one either way. We need a coherent principle (or small set of such principles) which will let us decide such questions consistently. True, I can understand that point. But at the same time, with test-driven development, I don't see it as any different than other issues caused by dynamic typing. My point here is that it smashes a powerful new debugging tool for Erlang, the type inference program we've been hearing about recently. Adding this feature is NOT just a matter of hacking on the VM, it would require serious work on the type inference program and other high-powered tools. I didn't say that it was any different from other issues (although I would be prepared to argue that). What I said meant that it ADDS to other issues. There's a famous paradox in philosophy: one grain of sand is not a heap. adding a grain of sand to a bunch of sand is obviously too small a change to convert a non-heap into a heap. yet if you keep on adding grains of sand, eventually you DO have a heap. An addition could perfectly well be similar in kind to other things in a non-heap, and you might think that making one more change won't convert a non-heap language into a useless-heap language, but if you KEEP making such changes, a useless-heap is what you will get. I still have unpleasant memories of PL/I, where you could add just about anything to just about anything, whether it made sense or not. Take for example (1+'2'). There obviously *is* a number inside that quoted atom, so why *not* let it be extracted? (Would you get 3 or '3'?) If a binary happens to be the term_to_binary() representation of a number, or of a tuple with numbers inside it, &c, why *not* hack on the VM so that it automatically tried binary_to_term() any time that a binary as such made no sense? We have to draw the line *somewhere*, and what we have now is tolerably coherent. > It might, for example, be better to introduce a whole new "array" data type; > that would be much more work, but it could yield better performance (using > long-known techniques from APL) without sacrificing any of the run-time > type checking we now have. Strictly from a selfish point of view based on the kind of applications I work on, I'd like to see "array of float" as a fundamental type. Floats are individually heap allocated, so there's a big win to putting them in a homogeneous array (OCaml has taken a similar route). It's also what Squeak Smalltalk has done, and if I've understood correctly the GHC Haskell compiler supports this in a rather clever way. (And of course Clean has supported for *ages* without having to be clever about it.) But from a conceptual point of view, arrays and tuples are the same thing, so why split them up? Because from a conceptual point of view arrays and tuples *AREN'T* the same thing. Valid analogues are array:list and record:tuple. (As you may have noticed, Erlang -records *are* tuples.) Sure, some tuples may *happen* to have fields that are all the same type, but that's not usual, just as C structs may happen to have fields all of the same type, but usually don't. There are oodles of array operations that make no sense at all on tuples; it could be nice having a data type which supported those operations. From ok@REDACTED Mon Jul 11 01:01:33 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 11 Jul 2005 11:01:33 +1200 (NZST) Subject: unix pid Message-ID: <200507102301.j6AN1Xah046776@atlas.otago.ac.nz> Michael McDaniel wrote: Perhaps I am misunderstanding the Erlang documentation. Actually, no. It's the UNIX sheels you're misunderstanding. os:cmd/1 says... --------------------------------------------- cmd(Command) -> string() Types: Command = string() | atom() Executes Command in a command shell of the target OS and returns the result as a string. That is, it captures the STANDARD OUTPUT of the command, just as backquote does in the shell. So what's written to the standard output? If I do the following from my (Linux) system prompt $ sleep 97 & [3] 11702 Since the Pid is returned as a string from the command prompt, I would expect the following from within Erlang to return the Pid also, though it does not. Much depends on what shell you run and how it is invoked. For example, you get the [3] job number from the C shell (csh) or jsh, but not from the Bourne shell (sh). More to the point, look at this: f% sh $ sleep 1 & 18399 $ (sleep 1 &)>/tmp/foo.out $ cat /tmp/foo.out 18407 $ sh -c "sleep 10 &" $ ps PID TTY TIME CMD 4008 pts/6 0:06 csh 18416 pts/6 0:00 ps 18415 pts/6 0:00 sleep 18398 pts/6 0:00 sh Did you notice where 'sh -c ""' produced NO output? Now let's try it from C: f% cat foo.c extern int system(char const *); int main(void) { (void) system("sleep 10 &"); return 0; } f% cc foo.c f% a.out f% ps f% ps PID TTY TIME CMD 4008 pts/6 0:06 csh 18445 pts/6 0:00 ps 18444 pts/6 0:00 sleep Again, NO OUTPUT. The point here is that when a shell creates another process in response to an "&" in the command, it writes the PID (plus, in the case of csh, the job number) if and only if it thinks there is a human being watching the output; if it thinks a program is watching the output, the PID is *not* written. What's the answer? You will have to explicitly write out the process number using $! . $ sh -c "sleep 10 & > echo $!" 18509 $ Or in Erlang: 1> os:cmd("sleep 10 &\necho $!"). "18525\n" From erlang@REDACTED Mon Jul 11 01:38:20 2005 From: erlang@REDACTED (Michael McDaniel) Date: Sun, 10 Jul 2005 16:38:20 -0700 Subject: unix pid In-Reply-To: <200507102301.j6AN1Xah046776@atlas.otago.ac.nz> References: <200507102301.j6AN1Xah046776@atlas.otago.ac.nz> Message-ID: <20050710233820.GH11878@fangora.autosys.us> On Mon, Jul 11, 2005 at 11:01:33AM +1200, Richard A. O'Keefe wrote: > Michael McDaniel wrote: > Perhaps I am misunderstanding the Erlang documentation. > > Actually, no. It's the UNIX sheels you're misunderstanding. > > os:cmd/1 says... > --------------------------------------------- > cmd(Command) -> string() > > Types: > Command = string() | atom() > > Executes Command in a command shell of the target OS and returns > the result as a string. > > That is, it captures the STANDARD OUTPUT of the command, just as > backquote does in the shell. So what's written to the standard output? > > If I do the following from my (Linux) system prompt > > $ sleep 97 & > [3] 11702 > > Since the Pid is returned as a string from the command prompt, I would expect the following > from within Erlang to return the Pid also, though it does not. > > Much depends on what shell you run and how it is invoked. > For example, you get the [3] job number from the C shell (csh) or jsh, > but not from the Bourne shell (sh). More to the point, look at this: > > f% sh > $ sleep 1 & > 18399 > $ (sleep 1 &)>/tmp/foo.out > $ cat /tmp/foo.out > 18407 > $ sh -c "sleep 10 &" > $ ps > PID TTY TIME CMD > 4008 pts/6 0:06 csh > 18416 pts/6 0:00 ps > 18415 pts/6 0:00 sleep > 18398 pts/6 0:00 sh > > Did you notice where 'sh -c ""' produced NO output? > Now let's try it from C: > > f% cat foo.c > extern int system(char const *); > > int main(void) { > (void) system("sleep 10 &"); > return 0; > } > f% cc foo.c > f% a.out > f% ps > f% ps > PID TTY TIME CMD > 4008 pts/6 0:06 csh > 18445 pts/6 0:00 ps > 18444 pts/6 0:00 sleep > > Again, NO OUTPUT. > > The point here is that when a shell creates another process in response > to an "&" in the command, it writes the PID (plus, in the case of csh, > the job number) if and only if it thinks there is a human being watching > the output; if it thinks a program is watching the output, the PID is > *not* written. > > What's the answer? You will have to explicitly write out the process > number using $! . > > $ sh -c "sleep 10 & > > echo $!" > 18509 > $ > > Or in Erlang: > 1> os:cmd("sleep 10 &\necho $!"). > "18525\n" > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Wonderful! Thank you for the explanation and the solution. ~Michael From peter.fulop@REDACTED Mon Jul 11 08:52:36 2005 From: peter.fulop@REDACTED (=?iso-8859-1?Q?P=E9ter_F=FCl=F6p_=28IJ/ETH=29?=) Date: Mon, 11 Jul 2005 08:52:36 +0200 Subject: TCP socket problem Message-ID: Hi All I have a problem. I am writing an erlang module, which communicate with a TCL modul using TCP socket. The problem is, that the all the messages are sent from erlang module to TCL modul , if the erlang TCP socket is closed, none the less I use the gen_tcp:send command. Have you got any idea? Thanks, Peter From robert.balogh@REDACTED Mon Jul 11 09:02:54 2005 From: robert.balogh@REDACTED (=?iso-8859-1?Q?R=F3bert_Balogh_=28IJ/ETH=29?=) Date: Mon, 11 Jul 2005 09:02:54 +0200 Subject: unix pid Message-ID: Hej Richard, Thanks this detailed info about it. br, /Robi -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED] Sent: Monday, July 11, 2005 1:38 AM To: erlang-questions@REDACTED Subject: Re: unix pid On Mon, Jul 11, 2005 at 11:01:33AM +1200, Richard A. O'Keefe wrote: > Michael McDaniel wrote: > Perhaps I am misunderstanding the Erlang documentation. > > Actually, no. It's the UNIX sheels you're misunderstanding. > > os:cmd/1 says... > --------------------------------------------- > cmd(Command) -> string() > > Types: > Command = string() | atom() > > Executes Command in a command shell of the target OS and returns > the result as a string. > > That is, it captures the STANDARD OUTPUT of the command, just as > backquote does in the shell. So what's written to the standard output? > > If I do the following from my (Linux) system prompt > > $ sleep 97 & > [3] 11702 > > Since the Pid is returned as a string from the command prompt, I would expect the following > from within Erlang to return the Pid also, though it does not. > > Much depends on what shell you run and how it is invoked. > For example, you get the [3] job number from the C shell (csh) or jsh, > but not from the Bourne shell (sh). More to the point, look at this: > > f% sh > $ sleep 1 & > 18399 > $ (sleep 1 &)>/tmp/foo.out > $ cat /tmp/foo.out > 18407 > $ sh -c "sleep 10 &" > $ ps > PID TTY TIME CMD > 4008 pts/6 0:06 csh > 18416 pts/6 0:00 ps > 18415 pts/6 0:00 sleep > 18398 pts/6 0:00 sh > > Did you notice where 'sh -c ""' produced NO output? > Now let's try it from C: > > f% cat foo.c > extern int system(char const *); > > int main(void) { > (void) system("sleep 10 &"); > return 0; > } > f% cc foo.c > f% a.out > f% ps > f% ps > PID TTY TIME CMD > 4008 pts/6 0:06 csh > 18445 pts/6 0:00 ps > 18444 pts/6 0:00 sleep > > Again, NO OUTPUT. > > The point here is that when a shell creates another process in response > to an "&" in the command, it writes the PID (plus, in the case of csh, > the job number) if and only if it thinks there is a human being watching > the output; if it thinks a program is watching the output, the PID is > *not* written. > > What's the answer? You will have to explicitly write out the process > number using $! . > > $ sh -c "sleep 10 & > > echo $!" > 18509 > $ > > Or in Erlang: > 1> os:cmd("sleep 10 &\necho $!"). > "18525\n" > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Wonderful! Thank you for the explanation and the solution. ~Michael From joe.armstrong@REDACTED Mon Jul 11 09:22:03 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Mon, 11 Jul 2005 09:22:03 +0200 Subject: New trading systems platform Message-ID: Can I add a few comments? - two problems have been discussed 1) Storing *lots* of data 2) Processing *lot* of data 1)The implicit assumption seems to be that the data set is small enough that with some kind of trickery it will fit onto a small number of machines. Why not ask the question "How can we store all tick data from all markets forever" Answering this question (or even making progress) seems much more interesting than fitting a "small" data set onto a system. However you look at it compression will in practice give you a constant store factor gain. Suppose you can store 10 times as much data using compression than without, and doing so allows you to store data for one year from one market. How do you store data from 500 markets for 1000 years? IMHO If you make a platform that "stores all data forever" then you have something worth selling :-) - even partial success is interesting. Now how do you store all data forever? - I don't know - but we can guess that distributed hash tables (chord, pastry, DKS, ...) etc will be involved. Now essentially all these (chord, pastry, ...) are the same - (ie all involve a "distance" metric, and to find a key you try to move "closer" to the key in each step) all of these result in log N lookup likes. Log N will eventually beat all compression schememes :-) So as regards storage - make a storage scheme that can store "everything" (or at least a lot of data) 2) Speed. " go parallel young man " Yes, of course C beats Erlang - but in Erlang you can *easily* write parallel programs which you can't in C. If C is 10 times faster than Erlang then use 11 machines - or a hundred - or a thousand. Making distributed parallel algorithms is how to get speed - not chasing speed on one machine. ---- So what is the platform/architecture for 1 + 2? My best guess is: A peer 2 peer system - running on a cluster (why not and open system? - because of "insoluble" security problems) with: 1) Redundant distributed hash tables 2) Parallised computations This is "slightly beyond the state of the art" - various simulations of (say) chord rings show that they can break after a very long time (parameters like the rate at which you can repaired a broken data set, and the rate of machine arrivals and failures are important) /Joe > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of HP Wei > Sent: den 7 juli 2005 23:18 > To: Joel Reymont > Cc: Erlang Users' List > Subject: Re: New trading systems platform > > > > hi Joel, > > > > I just want to point out that to store tick data > > > of many years needs a LOT of disk space plus > > > the capability of compression (when writing) > > > (and decompression when reading) in your database design. > > > > Do you have a model for tick data storage? Fields, etc. > > All I am allowed to say is that we have a proprietary model. > > > Why would I bother to compress/decompress? Wouldn't keeping 1-2-3Gb > > in-memory be enough? Should I really care about disk storage? > > hmmm, I think this depends on the operation > and the scale of your trading system. > > Usually, for real-time trading, you only need data as back as > only a few months to get your initial parameters. If you don't > access them very often during the trading, then IO is not > an issue. > And putting all today's data in memory is certainly ok > with nowaday's computers. > > However, when you do backtesting (or model studying), I believe > you will need several years of > tick data, then you are likely gonna hit the issues of > disk space and the efficiency of database IO. > > --HP > > From thomasl_erlang@REDACTED Mon Jul 11 11:45:15 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 11 Jul 2005 02:45:15 -0700 (PDT) Subject: New trading systems platform In-Reply-To: Message-ID: <20050711094515.89031.qmail@web34411.mail.mud.yahoo.com> --- Joel Reymont wrote: > > On Jul 8, 2005, at 4:03 PM, James Hague wrote: > > > Erlang lists take twice as much space as tuples, > and, > > much as I hate to admit it, > > Wow, I didn't know this! Is there any particular > reason for that? A list is a singly-linked list (each cell 2 words, so 2N words), while a tuple is a fixed-size vector (of N+1 words). Best, Thomas ____________________________________________________ Sell on Yahoo! Auctions ? no fees. Bid on great items. http://auctions.yahoo.com/ From vlad@REDACTED Mon Jul 11 15:53:34 2005 From: vlad@REDACTED (Vlad Balin) Date: Mon, 11 Jul 2005 17:53:34 +0400 Subject: New trading systems platform In-Reply-To: Message-ID: > > In case of ATS we need disk tick storage for the backtesting > > purpose only. It implies very simple access scenario (rare > > look-ups, sequential [forward in time] read of large data > > volumes, "joins" of several tables by time field). It make > > sense to develop file-based storage for tick storage (one > > file per instrument, data stored sequentially by chunks of > > compressed ticks), and avoid mnesia. > > Perhaps mnesia should be avoided... I'd still like to > explore ways to keep it in the game though. (: Well, i would say mnesia should have BTree-based storage option then :), and it should support clustered index. It should be able to work with huge databases without tricks like partitioning single table into several pieces. Easiest way to achieve that would be to use Berkley DB http://www.sleepycat.com/ as a storage ;) > A final thought. Performance in disk-based accesses can > be improved significantly if one can nail down the > data format. This is something I think should be an option > in mnesia. Here's a bit different option. User might want to store data by chunks in custom [compressed] format. Each chunk is being compressed separately, and contains range of values. It would be helpful to do something to allow this stuff with chunks to be customized and accessed transparently, as if it would be plain table with no chunks. From mickael.remond@REDACTED Mon Jul 11 16:11:39 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Mon, 11 Jul 2005 16:11:39 +0200 Subject: New trading systems platform In-Reply-To: References: Message-ID: <42D27E1B.6030109@erlang-fr.org> Vlad Balin wrote: > Well, i would say mnesia should have BTree-based storage option then :), > and it should support clustered index. It should be able to work with > huge databases without tricks like partitioning single table into several > pieces. > > Easiest way to achieve that would be to use Berkley DB > http://www.sleepycat.com/ as a storage ;) Yes. See the paper presented during EUC 2004: Bets is a Berkeley DB based storage backend for Mnesia. http://www.erlang.se/euc/04/Synapse.pdf -- Micka?l R?mond From joelr1@REDACTED Mon Jul 11 17:20:56 2005 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 11 Jul 2005 17:20:56 +0200 Subject: New trading systems platform In-Reply-To: <42D27E1B.6030109@erlang-fr.org> References: <42D27E1B.6030109@erlang-fr.org> Message-ID: <253313A4-843C-4455-B9FC-8EE99CE86B28@gmail.com> Doesn't the Berkley DB cost a lot of money for commercial usage? On Jul 11, 2005, at 4:11 PM, Mickael Remond wrote: > Yes. See the paper presented during EUC 2004: Bets is a Berkeley DB > based storage backend for Mnesia. > > http://www.erlang.se/euc/04/Synapse.pdf -- http://wagerlabs.com/uptick From mickael.remond@REDACTED Mon Jul 11 17:42:47 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Mon, 11 Jul 2005 17:42:47 +0200 Subject: New trading systems platform In-Reply-To: <253313A4-843C-4455-B9FC-8EE99CE86B28@gmail.com> References: <42D27E1B.6030109@erlang-fr.org> <253313A4-843C-4455-B9FC-8EE99CE86B28@gmail.com> Message-ID: <42D29377.3050605@erlang-fr.org> Joel Reymont wrote: > Doesn't the Berkley DB cost a lot of money for commercial usage? I suppose, if you want to use in in closed source version. However, what is interesting in the Synapse approach, is that you keep Mnesia as is but change the backend. Depending on your needs you can use different backends. This is interesting because you will not use the same backend for real-time in-ram database or for huge on disk storage. You could also decide to choose your backend depending on your licensing requirement. This architecture looks like MySQL one: http://www.tm-research.com/manuals/databases/mysql/architecture.png Note that on this schema, the RAM backend for MySQL is Network DataBase (NDB), which use to be an Ericsson product. -- Micka?l R?mond From thomasl_erlang@REDACTED Mon Jul 11 17:46:03 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 11 Jul 2005 08:46:03 -0700 (PDT) Subject: Meyer, OO and concurrency Message-ID: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> I found this on a blog "Bertrand Meyer had eloquently proven at ICSOC 2003 that OO cannot deal with concurrency very well, in other words the granularity and programming model of an object don't let you create (autonomous) "peers"." http://www.ebpml.org/site_updates.htm Does anyone know more about this? I haven't been able to find what Meyer's actual argument was, but it sounds like what Some People have been preaching for a while :-) Best, Thomas ____________________________________________________ Sell on Yahoo! Auctions ? no fees. Bid on great items. http://auctions.yahoo.com/ From todd@REDACTED Mon Jul 11 18:12:47 2005 From: todd@REDACTED (todd) Date: Mon, 11 Jul 2005 09:12:47 -0700 Subject: Meyer, OO and concurrency In-Reply-To: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> Message-ID: <42D29A7F.1030306@possibility.com> Thomas Lindgren wrote: >I found this on a blog > >"Bertrand Meyer had eloquently proven at ICSOC 2003 >that OO cannot deal with concurrency very well, in >other words the granularity and programming model of >an object don't let you create (autonomous) "peers"." > >http://www.ebpml.org/site_updates.htm > >Does anyone know more about this? I haven't been able >to find what Meyer's actual argument was, but it >sounds like what Some People have been preaching for a >while :-) > > That's absurd. Send a message to an active object that runs in its own thread. Done. From alex.arnon@REDACTED Mon Jul 11 18:48:17 2005 From: alex.arnon@REDACTED (Alex Arnon) Date: Mon, 11 Jul 2005 19:48:17 +0300 Subject: Meyer, OO and concurrency In-Reply-To: <42D29A7F.1030306@possibility.com> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> Message-ID: <944da41d050711094853a73e11@mail.gmail.com> On 7/11/05, todd wrote: > Thomas Lindgren wrote: > > >I found this on a blog > > > >"Bertrand Meyer had eloquently proven at ICSOC 2003 > >that OO cannot deal with concurrency very well, in > >other words the granularity and programming model of > >an object don't let you create (autonomous) "peers"." > > > >http://www.ebpml.org/site_updates.htm > > > >Does anyone know more about this? I haven't been able > >to find what Meyer's actual argument was, but it > >sounds like what Some People have been preaching for a > >while :-) > > > > > That's absurd. Send a message to an active object that runs in its own > thread. Done. > > I think they're speaking of OO modelling. From thomasl_erlang@REDACTED Mon Jul 11 19:00:19 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 11 Jul 2005 10:00:19 -0700 (PDT) Subject: Sutter & message passing Message-ID: <20050711170019.37497.qmail@web34405.mail.mud.yahoo.com> If you remember the very good article "The Free Lunch is Over" by Herb Sutter, it may be interesting to hear what MS (or at least Sutter) is up to: http://pluralsight.com/blogs/hsutter/archive/2005/03/19/6804.aspx "Iain McInnes wrote in a comment that he sees the solution to concurrency as: Don't share data between threads. Just don't Easy to say: People didn't like giving up GOTO, either. What's the alternative ? Asynchronous message exchange. Bingo. That's the core lesson, and it's the cornerstone of a design I've been working on since late last year: Shared state is evil (for concurrency, but also for so many other reasons including plain old single-threaded coupling). Async messages are the way to communicate." Alas, he later goes on to say that functional languages are "too different" :-) "... we need something a lot smoother, cleaner, and applicable to current imperative programming languages. I and other people are actively working on that." Best, Thomas ____________________________________________________ Sell on Yahoo! Auctions ? no fees. Bid on great items. http://auctions.yahoo.com/ From david.nospam.hopwood@REDACTED Mon Jul 11 20:33:58 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Mon, 11 Jul 2005 19:33:58 +0100 Subject: Meyer, OO and concurrency In-Reply-To: <42D29A7F.1030306@possibility.com> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> Message-ID: <42D2BB96.50703@blueyonder.co.uk> todd wrote: > Thomas Lindgren wrote: > >> I found this on a blog >> >> "Bertrand Meyer had eloquently proven at ICSOC 2003 >> that OO cannot deal with concurrency very well, in >> other words the granularity and programming model of >> an object don't let you create (autonomous) "peers"." >> >> http://www.ebpml.org/site_updates.htm >> >> Does anyone know more about this? I haven't been able >> to find what Meyer's actual argument was, but it >> sounds like what Some People have been preaching for a >> while :-) > > That's absurd. Send a message to an active object that runs in its own > thread. Done. There's a bit more to it than that: -- David Hopwood From todd@REDACTED Mon Jul 11 21:15:59 2005 From: todd@REDACTED (todd) Date: Mon, 11 Jul 2005 12:15:59 -0700 Subject: Meyer, OO and concurrency In-Reply-To: <42D2BB96.50703@blueyonder.co.uk> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> Message-ID: <42D2C56F.80502@possibility.com> David Hopwood wrote: > todd wrote: > >> Thomas Lindgren wrote: >> >>> I found this on a blog >>> >>> "Bertrand Meyer had eloquently proven at ICSOC 2003 >>> that OO cannot deal with concurrency very well, in >>> other words the granularity and programming model of >>> an object don't let you create (autonomous) "peers"." >>> >>> http://www.ebpml.org/site_updates.htm >>> >>> Does anyone know more about this? I haven't been able >>> to find what Meyer's actual argument was, but it >>> sounds like what Some People have been preaching for a >>> while :-) >> >> >> That's absurd. Send a message to an active object that runs in its >> own thread. Done. > > > There's a bit more to it than that: > > There's always more and there's always less. I have created many large scale distributed OO systems. Would it be simpler if a shared nothing system was enforceable in C++ or Java? Certainly. I think Java really screwed up. C++ is, well, C++. Saying OO doesn't let you create autonomous peers simply isn't so. Some of my own thoughts are at http://www.possibility.com/epowiki/Wiki.jsp?page=ArchitectureDiscussion From lennart.ohman@REDACTED Mon Jul 11 21:45:12 2005 From: lennart.ohman@REDACTED (=?iso-8859-1?Q?Lennart_=D6hman?=) Date: Mon, 11 Jul 2005 21:45:12 +0200 Subject: precise debugging :) In-Reply-To: <42CEBF78.1040007@web.am> Message-ID: <20050711193440.6777599B2@mailrelay.t-mobile.com> Hi, yes many people complains about error messages. However much can most often be told immediately from the message. You are experiencing a badarg, you shall search for the cause among the arguments to a BIF. Good luck, Lennart ------------------------------------------------------------- Lennart Ohman office : +46-8-587 623 27 Sj?land & Thyselius Telecom AB cellular: +46-70-552 6735 Sehlstedtsgatan 6 fax : +46-8-667 8230 SE-11528, STOCKHOLM, SWEDEN email : lennart.ohman@REDACTED -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED] On Behalf Of Gaspar Chilingarov Sent: den 8 juli 2005 20:01 To: erlang-questions@REDACTED Subject: precise debugging :) Hi all! While my erlang program is running it sometimes gives me messages like - Error in process <0.443.0> with exit value: {badarg,[{udp_driver,input_handler,2},{udp_driver,input_handler_msg,1}]} which is totally useless, because input_handler have a lot of clauses. are there any ways to get precise point, where I got failures or I should enable tracing and look into trace log? thanks in advance -- Gaspar Chilingarov System Administrator t +37491 419763 w www.web.am e nm@REDACTED From ulf.wiger@REDACTED Mon Jul 11 22:12:05 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Mon, 11 Jul 2005 22:12:05 +0200 Subject: New trading systems platform Message-ID: Vlad Balin wrote: > > Well, i would say mnesia should have BTree-based storage > option then :), Disk-based BTree-based storage, you mean? Otherwise, Mnesia already has them, in ordered_set tables. Ram_copies and disc_copies can be ordered_set, but not disc_only_copies. To make efficient ordered_set disc_only_copies for Mnesia, I think some restrictions on at least the key are in order. It's very difficult to implement BTrees efficiently if you can't make any assumptions about type and size of the keys, or even that they are of the _same_ type. While perhaps "un-erlangish", it's usually not a problem to to this in database applications. I think this is one of the best ways to boost disk performance in Mnesia. > and it should support clustered index. Off-the-shelf mnesia doesn't do this, but I have a hacked version that does. Even my version doesn't support globally ordered clustered indexes, though. I haven't figured out a good algorithm for that (the trick being knowing where to put the partition splits -- again, without being able to make any assumptions about the types of data used as index values.) > It should be able to work with huge databases without > tricks like partitioning single table into several > pieces. Why? If your alternative is to circumvent mnesia, inventing various tricks to do this, or lots of extra programming to interface with some SQL database -- why impose this purity restriction on mnesia? Partitioning tables in Mnesia is not that difficult. I don't know enough about trading systems to know whether it creates especially severe problems in this case, but for the most part, the user of a mnesia table is not aware of whether or not it is partitioned. > > Easiest way to achieve that would be to use Berkley DB > http://www.sleepycat.com/ as a storage ;) > > > A final thought. Performance in disk-based accesses can > > be improved significantly if one can nail down the > > data format. This is something I think should be an option > > in mnesia. > Here's a bit different option. User might want to store data by > chunks in custom [compressed] format. Each chunk is being > compressed separately, and contains range of values. It > would be helpful to do something to allow this stuff > with chunks to be customized and accessed transparently, as > if it would be plain table with no chunks. This is quite possible to do today, based on what's in mnesia, or ets, dets or disk_log, for that matter. Storing binary payload in either of these formats is a piece of cake. In order to offer a nice interface to the application programmer, some (in some cases quite a bit of) coding is probably needed. /Uffe From david.nospam.hopwood@REDACTED Mon Jul 11 23:42:41 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Mon, 11 Jul 2005 22:42:41 +0100 Subject: Meyer, OO and concurrency In-Reply-To: <42D2C56F.80502@possibility.com> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> Message-ID: <42D2E7D1.4010103@blueyonder.co.uk> todd wrote: > David Hopwood wrote: >> todd wrote: >>> Thomas Lindgren wrote: >>> >>>> I found this on a blog >>>> >>>> "Bertrand Meyer had eloquently proven at ICSOC 2003 >>>> that OO cannot deal with concurrency very well, in >>>> other words the granularity and programming model of >>>> an object don't let you create (autonomous) "peers"." >>>> >>>> http://www.ebpml.org/site_updates.htm >>>> >>>> Does anyone know more about this? I haven't been able >>>> to find what Meyer's actual argument was, but it >>>> sounds like what Some People have been preaching for a >>>> while :-) >>> >>> That's absurd. Send a message to an active object that runs in its >>> own thread. Done. >> >> There's a bit more to it than that: >> >> > There's always more and there's always less. I have created many large > scale distributed OO systems. But were they *just* OO systems, or did they impose some additional constraints, and use additional abstractions, not present in typical OO languages? The abstract and slides for Meyer's presentation are at . It's clear from the slides that he is referring *only* to typical OO languages with either shared-state (threads and locks/monitors), or no concurrency support. From what I know of SCOOP[LI] (see ), I'm skeptical that it solves the problem, but the argument that there is a problem with conventional OO support for concurrency is not controversial. The blog that was mentioned above seems to be at : # OO has gone in about 15 years from a mind opener to a mind bottleneck. # Why is that? # # If we consider that the paradigm shift that OO introduced was not a new # programming model, but rather the introduction of modeling concepts into # traditional programming language we might understand why today we are # hitting a wall. It might also help us break that wall. Actually, this # wall is a double wall. Today we need to evolve both the modeling # capabilities and the programming foundation. Cw from Microsoft Research # is the best proof of this double evolution. I have written many times # about making the message a first class citizen of the programming language # and enabling peer-to-peer computing (aka connected systems). Bertrand Meyer # had eloquently proven at ICSOC 2003 that OO cannot deal with concurrency # very well, in other words the granularity and programming model of an # object don't let you create (autonomous) "peers". So I won't write much # more on that, I would like rather to focus on "modeling". This argument suffers from a lack of knowledge of the history of OO. The changes that the author suggests wouldn't be an evolution; they would be a return to the roots of OO in actor systems. "Peer-to-peer computing (aka connected systems)" are what Hewitt wrote of as "Open Systems". Actor systems supported and emphasised the use of first-class messages from the start. The Join calculus concurrency model of Comega is essentially a reinvention of the actor model (search for JoinCalculus in , and ). > Would it be simpler if a shared nothing > system was enforceable in C++ or Java? Certainly. I think Java really > screwed up. C++ is, well, C++. Saying OO doesn't let you create > autonomous peers simply isn't so. But that is not what Meyer actually says; it is the blog author's (Jean-Jacques Dubray?) fairly inaccurate paraphrase. > Some of my own thoughts are at > http://www.possibility.com/epowiki/Wiki.jsp?page=ArchitectureDiscussion No public editing? -- David Hopwood From todd@REDACTED Tue Jul 12 04:31:56 2005 From: todd@REDACTED (todd) Date: Mon, 11 Jul 2005 19:31:56 -0700 Subject: Meyer, OO and concurrency In-Reply-To: <42D2E7D1.4010103@blueyonder.co.uk> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> Message-ID: <42D32B9C.2000108@possibility.com> David Hopwood wrote: > But were they *just* OO systems, or did they impose some additional > constraints, and use additional abstractions, not present in typical OO > languages? Well, as I build abstractions in OO I don't feel making new abstractions is unreasonable :-) You could make a usable system with an Actor and some sort of TCP/UDP abstraction. I have made that portable across many different operating systems with little difficulty. For generalized messaging use XML. Of course in a real system you need timers, semaphores, ISRs, TSS, etc. The problem I found is programmers can't help but access data structures across threads, so I would like language support to prevent it or to at least have it turned on by command. There are some cases in critical algorithms that I wouldn't mind the ability to share data structures, but certainly not by default. > The abstract and slides for Meyer's presentation are at > . It's > clear from the slides that he is referring *only* to typical OO languages > with either shared-state (threads and locks/monitors), or no concurrency > support. From what I know of SCOOP[LI] (see > ), I'm skeptical that it > solves > the problem, but the argument that there is a problem with > conventional OO > support for concurrency is not controversial. As long as the underlying language support libraries and features are thread safe there is nothing stopping you from building what you need. > > No public editing? After much public defacing I stopped allowing public editing. From erlang@REDACTED Tue Jul 12 10:42:27 2005 From: erlang@REDACTED (Peter-Henry Mander) Date: Tue, 12 Jul 2005 09:42:27 +0100 Subject: Meyer, OO and concurrency In-Reply-To: <42D32B9C.2000108@possibility.com> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> Message-ID: <1121157747.27930.33.camel@hymir.newport-networks.com> Hi Gurus, Sorry but I've got to stick my oar in. I've found that modelling a OO system in Erlang recovers the original concept of objects as concurrent actors. A long time ago (circa early 1990's) I was taking baby steps in OO programming, and I naively couldn't understand at the time why the 'method invocations' or 'messages' to objects were not asynchronous, and that objects couldn't perform background tasks implicitly. I then found that the 'message' concept was modeled using function calls, and was disappointed that I was shackled to a single thread of execution in my programs due to this lobotomised implementation. I even embarked on writing a basic preemptive RTOS for the Atari ST (remember them?) to enable me to create an environment that enabled me to use *real* Objects with asynchronous message passing! I'm proud to say it worked, but it sadly didn't get very far, at the time I wanted to be a cartoonist!... *cough*. (I never claimed to be wise). Fast forward to Erlang, and suddenly the shackles are gone. Server processes with asynchronous messages are to me the distillation of what Objects should have always been. Hurrah! I ask myself *why* has it taken soooo long to recognise the need for concurrency? But then again, the OO paradigm took a *very* long time to become mainstream. I don't believe that OO ever meant the same thing to all people, it's become a desperately twisted (polymorphic? overloaded?) concept. To me the original Object concept _is_ concurrent. And as for imperative vs. functional, if the shoes fit and you can wear more than one at a time... :-) Oh well. Rant over. Pete. On Mon, 2005-07-11 at 19:31 -0700, todd wrote: > David Hopwood wrote: > > > But were they *just* OO systems, or did they impose some additional > > constraints, and use additional abstractions, not present in typical OO > > languages? > > Well, as I build abstractions in OO I don't feel making new abstractions > is unreasonable :-) > You could make a usable system with an Actor and some sort of TCP/UDP > abstraction. > I have made that portable across many different operating systems with > little difficulty. > For generalized messaging use XML. Of course in a real system you need > timers, semaphores, > ISRs, TSS, etc. > > The problem I found is programmers can't help but access data structures > across > threads, so I would like language support to prevent it or to at least > have it > turned on by command. There are some cases in critical algorithms that I > wouldn't > mind the ability to share data structures, but certainly not by default. > > > The abstract and slides for Meyer's presentation are at > > . It's > > clear from the slides that he is referring *only* to typical OO languages > > with either shared-state (threads and locks/monitors), or no concurrency > > support. From what I know of SCOOP[LI] (see > > ), I'm skeptical that it > > solves > > the problem, but the argument that there is a problem with > > conventional OO > > support for concurrency is not controversial. > > As long as the underlying language support libraries and features are > thread safe there > is nothing stopping you from building what you need. > > > > > No public editing? > > After much public defacing I stopped allowing public editing. > > > -- "The Tao of Programming flows far away and returns on the wind of morning." From joelr1@REDACTED Tue Jul 12 11:27:32 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 12 Jul 2005 11:27:32 +0200 Subject: Parsing formatted text Message-ID: Folks, How would you parse a line like this in Erlang? TRADE,X_._.,20040901,00:00:02,1105.25,,1 I would need to skip the first two fields and convert the rest from ascii. Thanks, Joel -- http://wagerlabs.com/uptick From joelr1@REDACTED Tue Jul 12 11:30:54 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 12 Jul 2005 11:30:54 +0200 Subject: Parsing formatted text In-Reply-To: References: Message-ID: <98FF4978-46DA-4975-A76A-2C9A4BF84E04@gmail.com> I will rephrase a bit... How would you parse the date, time and numbers in the string below. I understand that I can extract the fields using binary matching and I'm doing that already. I wish there was a version of io:format that read data from a "string". On Jul 12, 2005, at 11:27 AM, Joel Reymont wrote: > Folks, > > How would you parse a line like this in Erlang? > > TRADE,X_._.,20040901,00:00:02,1105.25,,1 -- http://wagerlabs.com/uptick From ola.a.andersson@REDACTED Tue Jul 12 11:44:22 2005 From: ola.a.andersson@REDACTED (Ola Andersson A (AL/EAB)) Date: Tue, 12 Jul 2005 11:44:22 +0200 Subject: Parsing formatted text Message-ID: <148408C0A2D44A41AB295D74E18399755DBA16@esealmw105.eemea.ericsson.se> string:tokens may be a good start. /OLA. > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Joel Reymont > Sent: den 12 juli 2005 11:31 > To: Erlang Users' List > Subject: Re: Parsing formatted text > > > I will rephrase a bit... How would you parse the date, time and > numbers in the string below. I understand that I can extract the > fields using binary matching and I'm doing that already. > > I wish there was a version of io:format that read data from a > "string". > > On Jul 12, 2005, at 11:27 AM, Joel Reymont wrote: > > > Folks, > > > > How would you parse a line like this in Erlang? > > > > TRADE,X_._.,20040901,00:00:02,1105.25,,1 > > -- > http://wagerlabs.com/uptick > > > > From joelr1@REDACTED Tue Jul 12 11:48:05 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 12 Jul 2005 11:48:05 +0200 Subject: Parsing formatted text In-Reply-To: <148408C0A2D44A41AB295D74E18399755DBA16@esealmw105.eemea.ericsson.se> References: <148408C0A2D44A41AB295D74E18399755DBA16@esealmw105.eemea.ericsson.se> Message-ID: Ola, Thanks for the pointer. I'm wondering about parsing the tokens themselves, though, that is the date, time and numbers in the following: TRADE,X_._.,20040901,00:00:02,1105.25,,1 Thanks, Joel On Jul 12, 2005, at 11:44 AM, Ola Andersson A (AL/EAB) wrote: > string:tokens may be a good start. -- http://wagerlabs.com/uptick From joelr1@REDACTED Tue Jul 12 11:58:38 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 12 Jul 2005 11:58:38 +0200 Subject: Parsing formatted text In-Reply-To: References: <148408C0A2D44A41AB295D74E18399755DBA16@esealmw105.eemea.ericsson.se> Message-ID: <9A634DF1-C6EB-480D-8ED9-FE276641738F@gmail.com> It seems like the Erlang cookbook at http://schemecookbook.org/Erlang supplies the answers. On Jul 12, 2005, at 11:48 AM, Joel Reymont wrote: > Ola, > > Thanks for the pointer. I'm wondering about parsing the tokens > themselves, though, that is the date, time and numbers in the > following: > > TRADE,X_._.,20040901,00:00:02,1105.25,,1 -- http://wagerlabs.com/uptick From csanto@REDACTED Tue Jul 12 12:05:59 2005 From: csanto@REDACTED (Corrado Santoro) Date: Tue, 12 Jul 2005 12:05:59 +0200 Subject: Parsing formatted text In-Reply-To: References: Message-ID: <42D39607.4000903@diit.unict.it> What about "yecc"? --C. Joel Reymont wrote: > Folks, > > How would you parse a line like this in Erlang? > > TRADE,X_._.,20040901,00:00:02,1105.25,,1 > > I would need to skip the first two fields and convert the rest from ascii. > > Thanks, Joel > > -- > http://wagerlabs.com/uptick > > > -- ====================================================== Eng. Corrado Santoro, Ph.D. University of Catania - Engineering Faculty Department of Computer Science and Telecommunications Engineering Viale A. Doria, 6 - 95125 CATANIA (ITALY) Tel: +39 095 7382144 Int. (5) 8035 +39 095 7382380 +39 095 7382365 +39 095 7382364 VoIP: sip:8035@REDACTED Fax: +39 095 7382397 EMail: csanto@REDACTED Personal Home Page: http://www.diit.unict.it/users/csanto NUXI Home Page: http://nuxi.iit.unict.it ====================================================== From dietmar@REDACTED Tue Jul 12 13:39:49 2005 From: dietmar@REDACTED (Dietmar Schaefer) Date: Tue, 12 Jul 2005 13:39:49 +0200 Subject: start order of SNMP agent Message-ID: <42D3AC05.7070209@ast.dfs.de> Hi ! I figured out a strange start behaviour of my distributed snmp agent. (based on mnesia) It runs on two nodes (cmmc@REDACTED,cmmc@REDACTED). Something from which we don not know what it might be makes node *a01 special. If we stop the agent on node *a01 the snmp agent of node b01 reports. END OF MIB. If we stop the agent on node b01 we can still poll agent on node a01. If node b01 comes up before a01 is up I get lots of snmp errors. Can please sombody give me a hint ? Dietmar From serge@REDACTED Tue Jul 12 15:08:39 2005 From: serge@REDACTED (Serge Aleynikov) Date: Tue, 12 Jul 2005 09:08:39 -0400 Subject: Parsing formatted text In-Reply-To: <98FF4978-46DA-4975-A76A-2C9A4BF84E04@gmail.com> References: <98FF4978-46DA-4975-A76A-2C9A4BF84E04@gmail.com> Message-ID: <42D3C0D7.7000709@hq.idt.net> Refer to io_lib:fread docs: > io_lib:fread("TRADE,~d,~d:~d:~d,~f", "TRADE,20040901,00:00:02,15.6"). {ok,[20040901,0,0,2,15.6000],[]} Joel Reymont wrote: > I will rephrase a bit... How would you parse the date, time and numbers > in the string below. I understand that I can extract the fields using > binary matching and I'm doing that already. > > I wish there was a version of io:format that read data from a "string". > > On Jul 12, 2005, at 11:27 AM, Joel Reymont wrote: > >> Folks, >> >> How would you parse a line like this in Erlang? >> >> TRADE,X_._.,20040901,00:00:02,1105.25,,1 > > > -- > http://wagerlabs.com/uptick From todd@REDACTED Tue Jul 12 16:35:38 2005 From: todd@REDACTED (todd) Date: Tue, 12 Jul 2005 07:35:38 -0700 Subject: Meyer, OO and concurrency In-Reply-To: <1121157747.27930.33.camel@hymir.newport-networks.com> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> Message-ID: <42D3D53A.2040400@possibility.com> Peter-Henry Mander wrote: >I ask myself *why* has it taken soooo long to recognise the need for >concurrency? > You've always been able to do concurrency, so maybe it wasn't seen as a dramatic need. I never did. I could always add it. I was happy that Java had concurrency until I saw how broken it was. And don't forget you still have a lot of people, especially in the agile keep it simple camp, that say concurrency is an unecessary complexity. The exact opposite of seeing concurrency as a fundamental building block. They have, of course, thrown the baby out with the bath water, but the attitude is common. From dietmar@REDACTED Tue Jul 12 16:52:11 2005 From: dietmar@REDACTED (Dietmar Schaefer) Date: Tue, 12 Jul 2005 16:52:11 +0200 Subject: (Master) SNMP Agent more basic question Message-ID: <42D3D91B.9010202@ast.dfs.de> HI ! A more basic question concerning SNMP master agents just rised ! It's all about "Using the SNMP Agent in a Distributed Environment" Is it possible to habe two nodes running two MASTER agents which share a mnesia database? The database is designed so that each node has it's own entries. The index is the name (or a computed has value) of the node. For example: Setting a MIB variable: setMibActive(State) when is_integer(State),State < 3 -> {ok,Key} = cmmc_util:getIndex(), %% Hostname is index ! F = fun() -> [E] = mnesia:dirty_read({scalarVariables,Key}), % {NewState,_Rest} = string:to_integer(State), New = E#scalarVariables{fourDMibActive = State}, mnesia:write(New) end, Any advice ??? Regards Dietmar From mickael.remond@REDACTED Tue Jul 12 17:00:35 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Tue, 12 Jul 2005 17:00:35 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D3D53A.2040400@possibility.com> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D3D53A.2040400@possibility.com> Message-ID: <42D3DB13.9080103@erlang-fr.org> todd wrote: > And don't forget you still have a lot of people, especially in the agile > keep it simple camp, that say concurrency is an unecessary complexity. That's probably because they have used tools where concurrency is so hard to model. Erlang is amazing as it let you write concurrent program, from the very first day you learn it. -- Micka?l R?mond From erlang@REDACTED Tue Jul 12 17:09:35 2005 From: erlang@REDACTED (Peter-Henry Mander) Date: Tue, 12 Jul 2005 16:09:35 +0100 Subject: Meyer, OO and concurrency In-Reply-To: <42D3D53A.2040400@possibility.com> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D3D53A.2040400@possibility.com> Message-ID: <1121180975.27930.92.camel@hymir.newport-networks.com> Hi Todd, True, concurrency isn't in itself a new idea. It's the preconception that concurrent programming is itself intrinsically hard that gets my goat (*). The fact that is *is* hard in C++/Java/FlavourOfTheMonth language is an indictment. I agree that the KISS bunch tend to avoid concurrency unless they're compelled to use it, for the same reasons above that the tools just don't hack it. There is also a managerial school of thought which encourages the use of the lowest possible common denominator when choosing programming tools and methods... I do apologise, I seem to drifted off into another rant! Pete. (*) ... however, concurrency problems like race conditions can be quite tricky to handle sometimes, but that may be blamed on inadequate design. Mea culpa. On Tue, 2005-07-12 at 07:35 -0700, todd wrote: > Peter-Henry Mander wrote: > > >I ask myself *why* has it taken soooo long to recognise the need for > >concurrency? > > > You've always been able to do concurrency, so maybe it wasn't seen as a > dramatic need. I never did. I could always add it. I was happy that Java > had concurrency until I saw how broken it was. > > And don't forget you still have a lot of people, especially in the agile > keep it simple camp, that say concurrency is an unecessary complexity. > The exact opposite of seeing concurrency as a fundamental building > block. They have, of course, thrown the baby out with the bath water, > but the attitude is common. > > > From joelr1@REDACTED Tue Jul 12 17:53:03 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 12 Jul 2005 17:53:03 +0200 Subject: Mnesia and uniqueness on two keys Message-ID: I have a record defined like this: -record(tick_data, { symbol, date, ticks % tuple }). I create the table like this: mnesia:create_table(tick_data, [ {disc_copies, Nodes}, {index, [date]}, {type, set}, {attributes, record_info(fields, tick_data)} ]) My intent is to have a table where multiple records with the same symbol but different dates can coexist. Will the above work or do I need to create a bag table? In other words, does Mnesia expect uniqueness on each keys or do both keys work as a composite? Thanks, Joel -- http://wagerlabs.com/uptick From sean.hinde@REDACTED Wed Jul 13 01:38:00 2005 From: sean.hinde@REDACTED (Sean Hinde) Date: Wed, 13 Jul 2005 00:38:00 +0100 Subject: Mnesia and uniqueness on two keys In-Reply-To: References: Message-ID: Nowhere does the documentation claim that an index is part of a key. Sean On 12 Jul 2005, at 16:53, Joel Reymont wrote: > I have a record defined like this: > > -record(tick_data, { > symbol, > date, > ticks % tuple > }). > > I create the table like this: > > mnesia:create_table(tick_data, > [ > {disc_copies, Nodes}, > {index, [date]}, > {type, set}, > {attributes, record_info(fields, tick_data)} > ]) > > My intent is to have a table where multiple records with the same > symbol but different dates can coexist. Will the above work or do I > need to create a bag table? > > In other words, does Mnesia expect uniqueness on each keys or do both > keys work as a composite? > > Thanks, Joel > > -- > http://wagerlabs.com/uptick > > > From joelr1@REDACTED Wed Jul 13 04:00:08 2005 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 13 Jul 2005 04:00:08 +0200 Subject: Mnesia and uniqueness on two keys In-Reply-To: References: Message-ID: I'm obviously missing something. What is a key then? Thanks, Joel On Jul 13, 2005, at 1:38 AM, Sean Hinde wrote: > Nowhere does the documentation claim that an index is part of a key. -- http://wagerlabs.com/uptick From dietmar@REDACTED Wed Jul 13 06:38:43 2005 From: dietmar@REDACTED (Dietmar Schaefer) Date: Wed, 13 Jul 2005 06:38:43 +0200 Subject: SNMP Message-ID: <42D49AD3.6030807@ast.dfs.de> Hi ! (good morning ) Does somebody know what SNMP A-MIB-SERVER MDATA INFO find_next -> could not find table_entry ME with means ? Regards Dietmar From sanjaya@REDACTED Wed Jul 13 07:30:56 2005 From: sanjaya@REDACTED (Sanjaya Vitharana) Date: Wed, 13 Jul 2005 11:30:56 +0600 Subject: DTD / xmerl_scan:file help !!!!! {invalid_nmtoken,"*) #RE" Message-ID: <00bf01c5876c$0c3bf1c0$6900a8c0@wavenet.lk> Hi....!!! I get the below error when adding * (astric) to attribute list of my "key" element. As below ---Note the * (astric in the end) Error I got is. 1> {ParsResult2,Misc2}=xmerl_scan:file('ivr.xml',[{validation,true }]). 2798- fatal: {invalid_nmtoken,"*) #RE"} ** exited: {fatal,{{invalid_nmtoken,"*) #RE"}, {file,"./ivr.dtd"}, {line,21}, {col,47}}} ** Seems to be * (astric) is invalid in DTD (or may be xmerl_scan validation problem). But i want to use * (astric) in my XML file as below. So there must be a way to define it in DTD ..... can any DTD expert help me in this case. --------------- XML File I use ------------------------- ......... http://localhost/greeting_lite/select_song_greeting.php ------------------- Note the attribute * ......... -------- This DTD is working without any problem ---------- ------------------------------------------------------------------------------------ NOTE: ......... (doted line) means there are some other fields Waiting quick reply. Regards. Sanjaya Vitharana -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvg@REDACTED Wed Jul 13 08:26:06 2005 From: rvg@REDACTED (Rudolph van Graan) Date: Wed, 13 Jul 2005 08:26:06 +0200 Subject: Parsing formatted text In-Reply-To: References: Message-ID: Hey, string:tokens will work, however, it will not return the second last field (empty one between two commas) and silently discard it: (rvg@REDACTED)2> string:tokens("TRADE,X_._., 20040901,00:00:02,1105.25,,1",","). ["TRADE","X_._.","20040901","00:00:02","1105.25","1"] Below a version that is strict (Note the empty list in 2nd last position) (rvg@REDACTED)3> customlists:tokens("TRADE,X_._., 20040901,00:00:02,1105.25,,1",","). ["TRADE","X_._.","20040901","00:00:02","1105.25",[],"1"] %%% ======================================================================== ================== tokens(S, Tokens) -> tokens(S, Tokens, [], []). %%% tokens(Input,Tokens,Current,Result) tokens([],Tokens,[],Result) -> lists:reverse(Result); tokens([],Tokens,Current,Result) -> tokens([],Tokens,[],[lists:reverse(Current)|Result]); tokens([C|S],Tokens,Current,Result) -> case lists:member(C, Tokens) of true -> case S of [] -> tokens(S,Tokens,[],[""|[lists:reverse(Current)|Result]]); _ -> tokens(S,Tokens,[],[lists:reverse(Current)|Result]) end; false -> tokens(S,Tokens,[C|Current],Result) end. Rudolph On 12 Jul 2005, at 11:27 AM, Joel Reymont wrote: > Folks, > > How would you parse a line like this in Erlang? > > TRADE,X_._.,20040901,00:00:02,1105.25,,1 > > I would need to skip the first two fields and convert the rest from > ascii. > > Thanks, Joel > > -- > http://wagerlabs.com/uptick > > > > > From samuel@REDACTED Wed Jul 13 08:25:57 2005 From: samuel@REDACTED (Samuel Rivas) Date: Wed, 13 Jul 2005 08:25:57 +0200 Subject: Mnesia and uniqueness on two keys In-Reply-To: References: Message-ID: <20050713062557.GA19948@crusher.lfcia.pri> Joel Reymont wrote: > I'm obviously missing something. What is a key then? The key is the first element of the record. The OID is the tuple with the record name and the key. Regards -- Samuel From sanjaya@REDACTED Wed Jul 13 08:46:25 2005 From: sanjaya@REDACTED (Sanjaya Vitharana) Date: Wed, 13 Jul 2005 12:46:25 +0600 Subject: DTD / xmerl_scan:file help !!!!! {invalid_nmtoken,"*) #RE" Message-ID: <00e601c58776$96f49830$6900a8c0@wavenet.lk> Reverting to my below question, is working well, but it allows all the rage of CDATA to appear with the attribute "no". But I want something like below to strict the valid keys to some specific. ---Note the * (astric in the end) --- and of course NOT WORKING. Hope you can understand. Are there any way to do this ...???? Regards, Sanjaya Vitharana ----- Original Message ----- From: Sanjaya Vitharana To: erlang-questions@REDACTED Sent: Wednesday, July 13, 2005 11:30 AM Subject: DTD / xmerl_scan:file help !!!!! {invalid_nmtoken,"*) #RE" Hi....!!! I get the below error when adding * (astric) to attribute list of my "key" element. As below ---Note the * (astric in the end) Error I got is. 1> {ParsResult2,Misc2}=xmerl_scan:file('ivr.xml',[{validation,true }]). 2798- fatal: {invalid_nmtoken,"*) #RE"} ** exited: {fatal,{{invalid_nmtoken,"*) #RE"}, {file,"./ivr.dtd"}, {line,21}, {col,47}}} ** Seems to be * (astric) is invalid in DTD (or may be xmerl_scan validation problem). But i want to use * (astric) in my XML file as below. So there must be a way to define it in DTD ..... can any DTD expert help me in this case. --------------- XML File I use ------------------------- ......... http://localhost/greeting_lite/select_song_greeting.php ------------------- Note the attribute * ......... -------- This DTD is working without any problem ---------- ------------------------------------------------------------------------------------ NOTE: ......... (doted line) means there are some other fields Waiting quick reply. Regards. Sanjaya Vitharana -------------- next part -------------- An HTML attachment was scrubbed... URL: From joe.armstrong@REDACTED Wed Jul 13 09:24:33 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Wed, 13 Jul 2005 09:24:33 +0200 Subject: Meyer, OO and concurrency Message-ID: Todd wrote > Peter-Henry Mander wrote: > > >I ask myself *why* has it taken soooo long to recognise the need for > >concurrency? > > > You've always been able to do concurrency, so maybe it wasn't > seen as a > dramatic need. I never did. I could always add it. I was > happy that Java > had concurrency until I saw how broken it was. Why has it taken so long? - answer: because people things that concurrent programming is difficult - and because in the languages to which they have been exposed concurrent programming is extremely difficult. Anyway, you haven't "always been able to do concurrency" - Let me try the following argument on you. Suppose there was an OO language called K. K "does" objects. In K you can use objects freely - but subject to the following restrictions: - you should try to use less than say 2000 objects - if you use more than 20000 objects your program will crash - instead of destroying objects and creating new ones you should try to keep a pool of old objects and reuse them as often as possible - you should try and keep the total number of objects down because programs with lots of objects are difficult to write and understand and debug Now of course, "K does objects" - but not in any *useful* sense - and even if it does objects - the programming style is so warped that nobody wants to *use* objects as a method for structuring their programs. Now replace K by {Java, C++, C#, ...} and the word "object" by "process" :-) *Very few languages do concurrency* - in the sense that it is natural and easy to use and that there are not arbitrary restrictions on how you use processes etc. Concurrent algorithms often provide a "natural" way to describe things (especially for real-world programs - since the real world IS concurrent) - a sub-set of concurrent programming is sequential programming (but the converse is not true) - so in this sense concurrent programming beats sequential programming hands down. The reason why concurrent programming has got a bad name and a reputation of being difficult has to do with the threads, locks, mutexes, critical regions, shared memory school of programming. The non-shared memory, pure copying asynchronous message passing, paradigm is far easier to program (witness all the web applications, virtually all use non-shared memory (ie separated processors) and pure message passing (defined by RFC's)) and very easy to understand. /Joe > > And don't forget you still have a lot of people, especially > in the agile > keep it simple camp, that say concurrency is an unecessary > complexity. > The exact opposite of seeing concurrency as a fundamental building > block. They have, of course, thrown the baby out with the bath water, > but the attitude is common. > > > From matthias@REDACTED Wed Jul 13 09:42:00 2005 From: matthias@REDACTED (Matthias Lang) Date: Wed, 13 Jul 2005 09:42:00 +0200 Subject: Meyer, OO and concurrency In-Reply-To: References: Message-ID: <17108.50632.101556.719755@antilipe.corelatus.se> Joe Armstrong (AL/EAB) writes: > Why has it taken so long? - answer: because people think that > concurrent programming is difficult - and because in the > languages to which they have been exposed > concurrent programming is extremely difficult. I would add to that: because there is a widespread tendency to think of concurrency as a burden you put up with in order to get better performance. The idea that language-level concurrency might help you structure a system to make it less complex than it would otherwise be is surprising to most people in the industry---for reasons you've discussed earlier on in this thread. Matthias From her@REDACTED Wed Jul 13 10:25:22 2005 From: her@REDACTED (Helmut Enck-Radana) Date: Wed, 13 Jul 2005 10:25:22 +0200 Subject: DTD / xmerl_scan:file help !!!!! {invalid_nmtoken,"*) #RE" In-Reply-To: <00e601c58776$96f49830$6900a8c0@wavenet.lk> References: <00e601c58776$96f49830$6900a8c0@wavenet.lk> Message-ID: <6.2.1.2.0.20050713101231.03784b18@paradigma-software.de> Hi, in XML DTDs enumerated values have to be NMTOKENs, i.e. you can only use strings which could be used as names. You have to use a different schema language, if you need to to express the constraint you described. May be a better solution is not to use the asterisc, but a symbol which conforms to the NMTOKEN production instead. -- Helmut At 08:46 2005-07-13, you wrote: >Reverting to my below question, > > no CDATA #REQUIRED> > >is working well, but it allows all the rage of CDATA to appear with the >attribute "no". > >But I want something like below to strict the valid keys to some specific. > > no (1|2|3|4|5|6|7|8|9|z|r|c|*) #REQUIRED> ---Note the * (astric > in the end) --- and of course NOT WORKING. > >Hope you can understand. > >Are there any way to do this ...???? > >Regards, > >Sanjaya Vitharana From joe.armstrong@REDACTED Wed Jul 13 10:39:46 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Wed, 13 Jul 2005 10:39:46 +0200 Subject: Meyer, OO and concurrency Message-ID: Yes - the crazy thing is that this method of organising systems is really old. The Unix pipe and socket mechanisms provide precisely the type of concurrency that I'm after. Now the pipe mechanism was apparently (and I don't know if this was true) first suggest by C.A.R.Hoare - The CSP work is probably the best ever way describing systems. (( the problem with CSP is that synchronous protocols cause dependencies between components - so ther are no longer isolated) I stumbled upon some recent lectures at That Tony Hoare had given: http://research.microsoft.com/~thoare/StructuredConcurrent%20programming_files/v3_document.htm The Title of this lecture is "Structured Concurrent Programming" - and it's all about concurrency without shared memory. To quote from slide one: " In this lecture, I will explain the idea of concurrency as a simple and attractive program structuring principle. My hope is that this understanding will help developers plan and design systems that make effective use of concurrency in the environment of the global network. The end result will be a reduction of the well-known risks involved in concurrent programs, and an increase in their benefits. Among the benefits is responsiveness to the users' needs, as I described in my last lecture on concurrent programs waiting faster; that lecture actually introduced all the structuring principles that I need today." Hoare's structuring principles correspond closely to Erlangs' mechanisms - with one significant difference - In Erlang messages are asynchronous, in Hoare's system synchronous. (Actually not a problem since synchronous message passing can easily be layered on top of an asynchronous system) In an earlier lecture "Concurrent programs wait faster" http://research.microsoft.com/~thoare/Concurrent%20programs%20wait%20faster.final_files/v3_document.htm Hoare argued that not only did concurrency provide an attractive program structuring principle - but also that it resulted in the theoretically fastest possible program. /Joe > -----Original Message----- > From: Matthias Lang [mailto:matthias@REDACTED] > Sent: den 13 juli 2005 09:42 > To: Joe Armstrong (AL/EAB) > Cc: todd; erlang-questions@REDACTED > Subject: RE: Meyer, OO and concurrency > > > Joe Armstrong (AL/EAB) writes: > > > Why has it taken so long? - answer: because people think that > > concurrent programming is difficult - and because in the > > languages to which they have been exposed > > concurrent programming is extremely difficult. > > I would add to that: because there is a widespread tendency to think > of concurrency as a burden you put up with in order to get better > performance. The idea that language-level concurrency might help you > structure a system to make it less complex than it would otherwise be > is surprising to most people in the industry---for reasons you've > discussed earlier on in this thread. > > Matthias > From joelr1@REDACTED Wed Jul 13 10:52:25 2005 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 13 Jul 2005 10:52:25 +0200 Subject: Mnesia and uniqueness on two keys In-Reply-To: <20050713062557.GA19948@crusher.lfcia.pri> References: <20050713062557.GA19948@crusher.lfcia.pri> Message-ID: <50964686-F76C-44D1-9838-1C337388F72B@gmail.com> How is uniqueness ensured then? Based on? On Jul 13, 2005, at 8:25 AM, Samuel Rivas wrote: > The key is the first element of the record. The OID is the tuple > with > the record name and the key. -- http://wagerlabs.com/uptick From joelr1@REDACTED Wed Jul 13 11:22:25 2005 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 13 Jul 2005 11:22:25 +0200 Subject: Batch-loading Mnesia Message-ID: <0B5AF9E2-D468-4ED3-A59C-EB2C12CD44B1@gmail.com> Folks, It's taking me way over an hour (maybe two hours or three hours) to load 119,275 records into Mnesia. Parsing the same file without adding record takes about 6 seconds. I'm doing everything on the same node, adding record sequentially. Is there a way to speed things up? The table has the following format -record(tick, { symbol, datetime, timestamp, price, size }). Symbol is a string (list) of 4 characters, datetime is a bignum formed from a date and time 20040901,09:39:38 by stripping the comma and the colons. Timestamp is now(), price and size are float and integer respectively. Table is created like this: mnesia:create_table(tick, [ {disc_copies, Nodes}, {index, [datetime]}, {type, bag}, {attributes, record_info(fields, tick)}]) The function that adds a record to the database looks like this: add_tick(Symbol, Date, Time, Price, Size) -> Tick = #tick { symbol = Symbol, datetime = Date * 1000000 + Time, price = Price, size = Size, timestamp = now() }, %%F = fun() -> mnesia:write(Tick) end, %%mnesia:transaction(F). mnesia:dirty_write(Tick). -- http://wagerlabs.com/uptick From sanjaya@REDACTED Wed Jul 13 11:42:07 2005 From: sanjaya@REDACTED (Sanjaya Vitharana) Date: Wed, 13 Jul 2005 15:42:07 +0600 Subject: DTD / xmerl_scan:file help !!!!! {invalid_nmtoken,"*) #RE" References: <00e601c58776$96f49830$6900a8c0@wavenet.lk> <6.2.1.2.0.20050713101231.03784b18@paradigma-software.de> Message-ID: <013501c5878f$22814f20$6900a8c0@wavenet.lk> got your point. changing it to will sort the problem. Since the mobile phone has the * (astric) key, it is interesting to know how to express the constraint as we like. Such as, Could you please guide me to some reference materials or some better examples to pick this quickly. ---------------------------------------------------------------------------- ----------------------- Further more, is it not possile to define a datatype like below ??? MyType::= #x9 | #xA | #xD or some other way ..... ( this is NOT Working too ) Or using Hope it will possible. As I never did it before, I don't know how this can be done. Can anybody help me ....!!!! ---------------------------------------------------------------------------- ----------------------- Regards, Sanjaya Vitharana ----- Original Message ----- From: "Helmut Enck-Radana" To: "Sanjaya Vitharana" Cc: Sent: Wednesday, July 13, 2005 2:25 PM Subject: Re: DTD / xmerl_scan:file help !!!!! {invalid_nmtoken,"*) #RE" > Hi, > > in XML DTDs enumerated values have to be NMTOKENs, i.e. you can only use > strings which could be used as names. You have to use a different schema > language, if you need to to express the constraint you described. May be a > better solution is not to use the asterisc, but a symbol which conforms to > the NMTOKEN production instead. > > -- Helmut > > At 08:46 2005-07-13, you wrote: > >Reverting to my below question, > > > > > no CDATA #REQUIRED> > > > >is working well, but it allows all the rage of CDATA to appear with the > >attribute "no". > > > >But I want something like below to strict the valid keys to some specific. > > > > > no (1|2|3|4|5|6|7|8|9|z|r|c|*) #REQUIRED> ---Note the * (astric > > in the end) --- and of course NOT WORKING. > > > >Hope you can understand. > > > >Are there any way to do this ...???? > > > >Regards, > > > >Sanjaya Vitharana > > From sanjaya@REDACTED Wed Jul 13 12:16:24 2005 From: sanjaya@REDACTED (Sanjaya Vitharana) Date: Wed, 13 Jul 2005 16:16:24 +0600 Subject: How to Use xmerl_xpath:string ??? Message-ID: <014801c58793$ec3f2270$6900a8c0@wavenet.lk> Hi...!!! Can anybody give me a example of using xmerl_xpath:string function. what is the format of xPathString() ??? Since my ParsResult2=xmlElement() I think my input format of xPathString() is invalid. No docs found. So ... Need Help from expert ..... !!!!! ------ below is my try, but it gives an error ------------------------------------------ 54> {ParsResult2,Misc2}=xmerl_scan:file('ivr.xml',[{validation,true]). 55> xmerl_xpath:string("///",ParsResult2). =ERROR REPORT==== 13-Jul-2005::15:49:58 === Error in process <0.103.0> on node 'ivr@REDACTED' with exit value: {function_clau se,[{xmerl_xpath,match_expr,[{comp,'>',{comp,'<',{comp,'>',{comp,'<',{path,abs,' /'},{path,rel,{step,{child,{name,{'IVR',[],"IVR"}},[]}}}},{path,abs,'/'}},{path, rel,{step,{child,{name,{content,[],"content"}},[]}}}},{path,abs,'/'}},{... ** exited: {function_clause,[{xmerl_xpath, match_expr, [{comp, '>', {comp, '<', {comp, '>', {comp, '<', {path,abs,'/'}, {path,rel,{step|...}}}, {path,abs,'/'}}, {path, rel, {step, {child, {name,{...}}, []}}}}, {path,abs,'/'}}, {state, {xmlContext, forward, {xmlNode, element, {xmlElement, 'IVR', 'IVR', [], {xmlNamespace,[],[]}, [], 1, []|...}, [], 1}, 1, [], [], [], [], {xmlNode, root_node, {xmlDocument,{xmlElement|...}}, [], 1}}, []}]}, {xmerl_xpath,string,5}, {erl_eval,do_apply,5}, {shell,exprs,6}, {shell,eval_loop,3}]} ** regards, Sanjaya Vitharana. -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.nospam.hopwood@REDACTED Wed Jul 13 12:32:03 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Wed, 13 Jul 2005 11:32:03 +0100 Subject: Meyer, OO and concurrency In-Reply-To: References: Message-ID: <42D4EDA3.6000600@blueyonder.co.uk> Joe Armstrong (AL/EAB) wrote: > Now the pipe mechanism was apparently (and I don't know if this was true) > first suggest by C.A.R.Hoare says that it was suggested by Doug McIlroy in 1972. : MSM: Where did the idea [for Unix pipes] come from? McIlroy: Goes way back. There was ... in the early sixties, Conway wrote an article about co-routines. '63, perhaps, in the CACM. I had been doing macros, starting back in '59 or '60. And, if you think about macros, they mainly involve switching data streams. You're taking in your ... you're taking input, you suddenly come to a macro call, and that says, "Stop taking input from here, go take it from the definition." In the middle of the definition, you'll find another macro call. So, macros... even as early as '64, ah, somewhere I talked of a macro processor as a switchyard for data streams. Also, in '64, there's a paper that's hanging on Brian's wall, still. He dredged out somewhere, where I talked about screwing together streams like garden hoses. [...] Robin Milner's CCS, from which CSP was mainly derived, was developed from about 1973 on (see ). Hoare's first paper on CSP was published as a tech report in 1976, and in CACM in 1978 (http://portal.acm.org/citation.cfm?id=359585). So these were too late to have been an influence on the design of pipes in Unix. > Hoare's structuring principles correspond closely to Erlangs' mechanisms - > with one significant difference - In Erlang messages are asynchronous, in > Hoare's system synchronous. (Actually not a problem since synchronous > message passing can easily be layered on top of an asynchronous system) In a non-distributed system yes. In a distributed system it's much more difficult (impossible with exactly the CSP semantics) to implement synchronous message passing given the possibility of network partition and process failure. -- David Hopwood From joelr1@REDACTED Wed Jul 13 13:10:37 2005 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 13 Jul 2005 13:10:37 +0200 Subject: Mnesia not suitable for time series storage? Message-ID: <956B8936-A7A2-46AB-893B-09434539C025@gmail.com> Folks, I'm trying to create a tick database on top of Mnesia. Ticks are price quotes (trades in a stock) sent by the exchange a few times a second for active securities. My two requirements for tick storage are very high-speed writes and high-speed sequential retrieval. I basically issue one type of query and that is to give me all the ticks in a particular security that happened between this and that time. An issue that presents a problem is that exchanges might send many quotes per second but they do not give time in milliseconds, so I need to keep track of the order in which the quotes came in. My unoptimized tick record looks like this: -record(tick, { symbol, datetime, timestamp, price, size }). Symbol is a string (list) of 4 characters, datetime is a bignum formed from a date and time 20040901,09:39:38 by stripping the comma and the colons. Timestamp is now(), price and size are float and integer respectively. The timestamp field is what keeps track of the order in which price quotes come in. I do not know yet how to tell Mnesia to return the records sorted by timestamp, at least not how to do it efficiently. Table is created like this: mnesia:create_table(tick, [ {disc_copies, Nodes}, {index, [datetime]}, {type, bag}, {attributes, record_info(fields, tick)}]) The table is a bag because ticks are unique only on the combination of symbol, datetime AND timestamp. I tried making the table a set but only got one record in it. Now, here's the kicker... I parse through a file of historical quotes and insert ticks into Mnesia one by one and time the insert. Inserts start at around 1ms and quickly grow to 5, 7, 8, etc. I'm at 24ms per insert right now and I killed my previous attempts to load about 120,000 after an hour or two. I would like to be able to store at least 500 ticks per second in Mnesia and have good lookup speed as well. Is this possible? Thanks, Joel -- http://wagerlabs.com/uptick From ke.han@REDACTED Wed Jul 13 13:20:48 2005 From: ke.han@REDACTED (ke.han) Date: Wed, 13 Jul 2005 19:20:48 +0800 Subject: Mnesia and uniqueness on two keys In-Reply-To: References: Message-ID: <42D4F910.9050508@redstarling.com> Joel Reymont wrote: > I have a record defined like this: > > -record(tick_data, { > symbol, > date, > ticks % tuple > }). > > I create the table like this: > > mnesia:create_table(tick_data, > [ > {disc_copies, Nodes}, > {index, [date]}, > {type, set}, > {attributes, record_info(fields, tick_data)} > ]) > > My intent is to have a table where multiple records with the same > symbol but different dates can coexist. Will the above work or do I > need to create a bag table? > > In other words, does Mnesia expect uniqueness on each keys or do both > keys work as a composite? > > Thanks, Joel Joel, I'm a novice, but my understanding is that the first field to your record is always the key. So if you want to key by symbol and date, you have the first field as a tuple {symbol, date} and the second field is ticks. This means you get the keying and querying as you desire. You simply have to unpack the tuple in order to get at the symbol or date values. I would appreciate if someone more experienced can confirm/deny my understanding. thanks, ke han > > -- > http://wagerlabs.com/uptick > > > > From samuel@REDACTED Wed Jul 13 13:22:40 2005 From: samuel@REDACTED (Samuel Rivas) Date: Wed, 13 Jul 2005 13:22:40 +0200 Subject: Mnesia and uniqueness on two keys In-Reply-To: <50964686-F76C-44D1-9838-1C337388F72B@gmail.com> References: <20050713062557.GA19948@crusher.lfcia.pri> <50964686-F76C-44D1-9838-1C337388F72B@gmail.com> Message-ID: <20050713112240.GA22292@crusher.lfcia.pri> Joel Reymont wrote: > How is uniqueness ensured then? > > Based on? > > On Jul 13, 2005, at 8:25 AM, Samuel Rivas wrote: > > > The key is the first element of the record. The OID is the tuple > >with > >the record name and the key. If the table is not a bag, when a tuple with an existing key is inserted it replaces the old tuple. If the table is a bag then duplicated keys are allowed. -- Samuel From micael.karlberg@REDACTED Wed Jul 13 13:31:56 2005 From: micael.karlberg@REDACTED (Micael Karlberg) Date: Wed, 13 Jul 2005 13:31:56 +0200 Subject: SNMP In-Reply-To: <42D49AD3.6030807@ast.dfs.de> References: <42D49AD3.6030807@ast.dfs.de> Message-ID: <17108.64428.157612.473548@gargle.gargle.HOWL> Hi, This is written by the agent mib-server. It keeps a data structure that basically represents the mib-structure (of all the loaded mib's). It found the table-entry it was looking for, but not the corresponding data (which is stored separetly). Regards, /BMK Dietmar Schaefer writes: > Hi ! (good morning ) > > > Does somebody know what > > > SNMP A-MIB-SERVER MDATA INFO > > find_next -> could not find table_entry ME with > > > > > means ? > > > > > Regards > > > > Dietmar From joelr1@REDACTED Wed Jul 13 13:53:34 2005 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 13 Jul 2005 13:53:34 +0200 Subject: Mnesia not suitable for time series storage? In-Reply-To: <1121255291.20249.12.camel@localhost.localdomain> References: <956B8936-A7A2-46AB-893B-09434539C025@gmail.com> <1121255291.20249.12.camel@localhost.localdomain> Message-ID: Laurent, I _am_ using dirty_write. I believe the insert time increases linearly because it's a bag table. Luke reported similar findings 2-3 years ago. Joel On Jul 13, 2005, at 1:48 PM, Laurent Picouleau wrote: > > Have you considered using mnesia:dirty_write instead of mnesia:write? -- http://wagerlabs.com/uptick From thomasl_erlang@REDACTED Wed Jul 13 14:22:21 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 13 Jul 2005 05:22:21 -0700 (PDT) Subject: Mnesia not suitable for time series storage? In-Reply-To: <956B8936-A7A2-46AB-893B-09434539C025@gmail.com> Message-ID: <20050713122222.1573.qmail@web34413.mail.mud.yahoo.com> --- Joel Reymont wrote: > My two requirements for tick storage are very > high-speed writes and > high-speed sequential retrieval. Assuming that you write as the ticks arrive, this sounds like a good fit for a plain old file per stream. So maybe mnesia is overkill? OK, this doesn't answer your question, I know :-) Best, Thomas __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From Jouni.Ryno@REDACTED Wed Jul 13 14:52:37 2005 From: Jouni.Ryno@REDACTED (Jouni =?ISO-8859-1?Q?Ryn=F6?=) Date: Wed, 13 Jul 2005 15:52:37 +0300 Subject: Mnesia not suitable for time series storage? In-Reply-To: <956B8936-A7A2-46AB-893B-09434539C025@gmail.com> References: <956B8936-A7A2-46AB-893B-09434539C025@gmail.com> Message-ID: <1121259157.6743.17.camel@adic> Dear Joel I had an similar issue in my housekeeping database, also a bag, disc_only db. The details are now escaping my mind, but the increased time comes, when the mnesia has to search the whole DB before insertion. Don't know, if mnesia can optimise the insert, when the DB is in memory (and is set), but in disc only case it cannot, as items are appended to the file as they come, not necessarily in sorted order. The trick was to use fragmented tables, with a special key calculation. In my case, the fragment was simply the day. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % the key is in 1/65536 second units. % key_to_frag_number(State, Key) when record(State, hash_state) -> %% io:fwrite("From ~p to ~w~n", [Key, trunc(Key)]), IntKey = (Key - ?UNIX_20000101) div 5662310400. % 24*60*60*65536 This made insertion and searches to work in reasonable time scales (20-60 seconds), instead of minutes even on small data set. regards Jouni On Wed, 2005-07-13 at 13:10 +0200, Joel Reymont wrote: > An issue that presents a problem is that exchanges might send many > quotes per second but they do not give time in milliseconds, so I > need to keep track of the order in which the quotes came in. > > My unoptimized tick record looks like this: > > -record(tick, { > symbol, > datetime, > timestamp, > price, > size > }). > > Symbol is a string (list) of 4 characters, datetime is a bignum > formed from a date and time 20040901,09:39:38 by stripping the comma > and the colons. Timestamp is now(), price and size are float and > integer respectively. > > The timestamp field is what keeps track of the order in which price > quotes come in. I do not know yet how to tell Mnesia to return the > records sorted by timestamp, at least not how to do it efficiently. > > Table is created like this: > > mnesia:create_table(tick, > [ > {disc_copies, Nodes}, > {index, [datetime]}, > {type, bag}, > {attributes, record_info(fields, tick)}]) > > The table is a bag because ticks are unique only on the combination > of symbol, datetime AND timestamp. I tried making the table a set but > only got one record in it. > > Now, here's the kicker... I parse through a file of historical quotes > and insert ticks into Mnesia one by one and time the insert. Inserts > start at around 1ms and quickly grow to 5, 7, 8, etc. I'm at 24ms per > insert right now and I killed my previous attempts to load about > 120,000 after an hour or two. > > I would like to be able to store at least 500 ticks per second in > Mnesia and have good lookup speed as well. Is this possible? > > Thanks, Joel > > -- > http://wagerlabs.com/uptick > > > -- Jouni Ryn? mailto://Jouni.Ryno@REDACTED/ http://www.geo.fmi.fi/~ryno/ Finnish Meteorological Institute http://www.fmi.fi/ Space Research http://www.geo.fmi.fi/ P.O.BOX 503 Tel (+358)-9-19294656 FIN-00101 Helsinki FAX (+358)-9-19294603 Finland priv-GSM (+358)-50-5302903 "It's just zeros and ones, it cannot be hard" From ulf@REDACTED Wed Jul 13 15:10:16 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 15:10:16 +0200 Subject: Mnesia and uniqueness on two keys In-Reply-To: References: Message-ID: Den 2005-07-13 04:00:08 skrev Joel Reymont : > I'm obviously missing something. What is a key then? > > Thanks, Joel > > On Jul 13, 2005, at 1:38 AM, Sean Hinde wrote: > >> Nowhere does the documentation claim that an index is part of a key. In database parlance, a 'key attribute' uniquely identifies a record. An 'index' is a structure that, given an attribute value points to one or several actual objects. The attribute value used by the index is called an 'index field'. A 'primary index' uses the primary key as its index field, thus producing a 'dense index' (where each index value always points to exactly one object). An index that uses another key attribute in the object is called a 'secondary index', and the index field in question a 'secondary key'. Thus, you can define an index on an attribute, but you cannot tell mnesia that the attribute is a 'key' attribute (i.e. unique). An ets 'set' or 'ordered_set' table can be seen as a primary index. Mnesia also supports indexes on non-key attributes, but has no notion of 'secondary keys'. In Mnesia, the 2nd element of the record is always the primary key (if 'set' or 'ordered_set'). I know of no good RDBMS terminology for 'bag' tables, since such tables are not part of the relational database model. Furthermore, all attributes in mnesia are considered atomic -- there are no composite attributes. Thus, you cannot index a combination of attributes, or part of an attribute. /Uffe From serge@REDACTED Wed Jul 13 15:10:39 2005 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 13 Jul 2005 09:10:39 -0400 Subject: Mnesia not suitable for time series storage? In-Reply-To: <956B8936-A7A2-46AB-893B-09434539C025@gmail.com> References: <956B8936-A7A2-46AB-893B-09434539C025@gmail.com> Message-ID: <42D512CF.8010606@hq.idt.net> I believe the main problem with the speed is that you organized the tick table to be a bag and also have another BTree index on the date. The search time related to inserting into this table will grow logarithmically. Here is an idea to try. Alter table layout in order to improve speed by the following: 1. Use a separate set table for each symbol (this will partition data better). 2. Use a surrogate key, like date, or {date, hour}, and store all ticks related to that key in a form of a list: -record(tick_data, { key = {date, hour}, data = [ {minute,second,timestamp,price,size} ] }). This is not as convenient as your original implementation, as it will require some additional logic in retrieving the data, but I believe it'll improve the speed of inserts. Since for each key data is inserted in a sorted order, you can use a binary search on the data list for a given key. Serge Joel Reymont wrote: > Folks, > > I'm trying to create a tick database on top of Mnesia. Ticks are price > quotes (trades in a stock) sent by the exchange a few times a second > for active securities. > > My two requirements for tick storage are very high-speed writes and > high-speed sequential retrieval. > > I basically issue one type of query and that is to give me all the > ticks in a particular security that happened between this and that time. > > An issue that presents a problem is that exchanges might send many > quotes per second but they do not give time in milliseconds, so I need > to keep track of the order in which the quotes came in. > > My unoptimized tick record looks like this: > > -record(tick, { > symbol, > datetime, > timestamp, > price, > size > }). > > Symbol is a string (list) of 4 characters, datetime is a bignum formed > from a date and time 20040901,09:39:38 by stripping the comma and the > colons. Timestamp is now(), price and size are float and integer > respectively. > > The timestamp field is what keeps track of the order in which price > quotes come in. I do not know yet how to tell Mnesia to return the > records sorted by timestamp, at least not how to do it efficiently. > > Table is created like this: > > mnesia:create_table(tick, > [ > {disc_copies, Nodes}, > {index, [datetime]}, > {type, bag}, > {attributes, record_info(fields, tick)}]) > > The table is a bag because ticks are unique only on the combination of > symbol, datetime AND timestamp. I tried making the table a set but only > got one record in it. > > Now, here's the kicker... I parse through a file of historical quotes > and insert ticks into Mnesia one by one and time the insert. Inserts > start at around 1ms and quickly grow to 5, 7, 8, etc. I'm at 24ms per > insert right now and I killed my previous attempts to load about > 120,000 after an hour or two. > > I would like to be able to store at least 500 ticks per second in > Mnesia and have good lookup speed as well. Is this possible? > > Thanks, Joel > > -- > http://wagerlabs.com/uptick From her@REDACTED Wed Jul 13 15:19:33 2005 From: her@REDACTED (Helmut Enck-Radana) Date: Wed, 13 Jul 2005 15:19:33 +0200 Subject: How to Use xmerl_xpath:string ??? In-Reply-To: <014801c58793$ec3f2270$6900a8c0@wavenet.lk> References: <014801c58793$ec3f2270$6900a8c0@wavenet.lk> Message-ID: <6.2.1.2.0.20050713151025.03a93070@paradigma-software.de> At 12:16 2005-07-13, you wrote: >what is the format of xPathString() ??? [...] > No docs found. So ... Need Help from expert ..... !!!!! xmerl docs: http://erlang.se/doc/doc-5.4/lib/xmerl-1.0/doc/html/ XPath spec: http://www.w3.org/TR/xpath > 55> xmerl_xpath:string("///",ParsResult2). /// is not an XPath expression. May be you mean /IVR/content or /IVR/content/* or /IVR/* -- Helmut From ulf@REDACTED Wed Jul 13 15:22:24 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 15:22:24 +0200 Subject: Meyer, OO and concurrency In-Reply-To: References: Message-ID: Den 2005-07-13 10:39:46 skrev Joe Armstrong (AL/EAB) : > Now the pipe mechanism was apparently (and I don't know if this was > true) first suggest by C.A.R.Hoare - The CSP work is probably the best > ever way describing systems. > (( the problem with CSP is that synchronous protocols cause dependencies > between components - so ther are no longer isolated) From Hoare's book "Communicating Sequential Processes", pp 241-2 (http://www.usingcsp.com/): "The idea was first propounded by Conway who illustrated it by examples similar to 4.4 X2 and X3, except that all components of a pipe were expected to terminate successfully instead of running forever. He proposed that the pipe structure should be used for writing a multiple-pass compiler for a programming language." Hoare refers to M.E. Conway's "Design of a Separable Transition-Diagram Compiler," Comm. ACM 6 (7), 8?15 (1963). /Uffe From ulf@REDACTED Wed Jul 13 15:33:30 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 15:33:30 +0200 Subject: Mnesia not suitable for time series storage? In-Reply-To: <20050713122222.1573.qmail@web34413.mail.mud.yahoo.com> References: <20050713122222.1573.qmail@web34413.mail.mud.yahoo.com> Message-ID: Den 2005-07-13 14:22:21 skrev Thomas Lindgren : > > > --- Joel Reymont wrote: >> My two requirements for tick storage are very >> high-speed writes and >> high-speed sequential retrieval. > > Assuming that you write as the ticks arrive, this > sounds like a good fit for a plain old file per > stream. So maybe mnesia is overkill? > > OK, this doesn't answer your question, I know :-) I was also thinking along the lines of having a load-balancing front-end that uses disk_log to store ticks persistently. One could switch logs in order to get a reasonable chunk in each. The log could be stored on a RAID disk, providing redundancy. The load balancer could then forward ticks to a diskless cluster, where ticks are possibly stored in non-replicated, ram-only fragmented tables. A problem would be to 'garbage-collect' the cluster, removing ticks that are deemed as 'too old'. Or perhaps this gets a lot easier if one uses ets tables instead of mnesia. Keeping a list of ets tables, and the age of objects in each, one can simply drop tables that are too old. Problem here is that we end up writing our own distributed database. /Uffe From ulf@REDACTED Wed Jul 13 15:36:54 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 15:36:54 +0200 Subject: Mnesia and uniqueness on two keys In-Reply-To: <50964686-F76C-44D1-9838-1C337388F72B@gmail.com> References: <20050713062557.GA19948@crusher.lfcia.pri> <50964686-F76C-44D1-9838-1C337388F72B@gmail.com> Message-ID: Den 2005-07-13 10:52:25 skrev Joel Reymont : > How is uniqueness ensured then? > > Based on? > > On Jul 13, 2005, at 8:25 AM, Samuel Rivas wrote: > >> The key is the first element of the record. The OID is the tuple with >> the record name and the key. For indexes, uniqueness is not checked at all. Ets and dets tables (the underpinnings of mnesia) enforce uniqueness if you use the 'set' or 'ordered_set' semantics. An existing object with the same key is simply replaced. In ets, you can use the rather new function ets:insert_new/2, which will insert the object, or list of objects, only if there isn't already a conflicting key in the table; otherwise it returns 'false'. With mnesia, you have to check for yourself, if you don't want to overwrite objects. /Uffe From ulf@REDACTED Wed Jul 13 15:45:42 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 15:45:42 +0200 Subject: Mnesia not suitable for time series storage? In-Reply-To: <956B8936-A7A2-46AB-893B-09434539C025@gmail.com> References: <956B8936-A7A2-46AB-893B-09434539C025@gmail.com> Message-ID: Den 2005-07-13 13:10:37 skrev Joel Reymont : > The timestamp field is what keeps track of the order in which price > quotes come in. I do not know yet how to tell Mnesia to return the > records sorted by timestamp, at least not how to do it efficiently. I don't know either. My hacked version of mnesia supports ordered indexes, for this very reason. /Uffe From ulf@REDACTED Wed Jul 13 15:47:49 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 15:47:49 +0200 Subject: Mnesia not suitable for time series storage? In-Reply-To: <42D512CF.8010606@hq.idt.net> References: <956B8936-A7A2-46AB-893B-09434539C025@gmail.com> <42D512CF.8010606@hq.idt.net> Message-ID: Den 2005-07-13 15:10:39 skrev Serge Aleynikov : > I believe the main problem with the speed is that you organized the tick > table to be a bag and also have another BTree index on the date. The > search time related to inserting into this table will grow > logarithmically. For the record, mnesia indexes are bags, not BTrees. I think the problem is that mnesia's bag tables store only one copy of each object -- i.e. it doesn't allow duplicate objects (even though duplicate keys are OK). In ets, there is 'duplicate_bag', which doesn't check for duplicates, but mnesia will basically have to do a linear search through all objects with the same key. I can imagine that duplicate_bag would mess up the mnesia transaction support pretty badly. /Uffe From todd@REDACTED Wed Jul 13 16:31:55 2005 From: todd@REDACTED (todd) Date: Wed, 13 Jul 2005 07:31:55 -0700 Subject: Meyer, OO and concurrency In-Reply-To: References: Message-ID: <42D525DB.9030806@possibility.com> Joe Armstrong (AL/EAB) wrote: >Suppose there was an OO language called K. K "does" objects. > >In K you can use objects freely - but subject to the following restrictions: > > - you should try to use less than say 2000 objects > - if you use more than 20000 objects your program will crash > - instead of destroying objects and creating new ones you should try > to keep a pool of old objects and reuse them as often as possible > - you should try and keep the total number of objects down because > programs with lots of objects are difficult to write and understand and debug > >Now of course, "K does objects" - but not in any *useful* sense - and even if it does >objects - the programming style is so warped that nobody wants to *use* objects as a method for structuring their programs. > >Now replace K by {Java, C++, C#, ...} and the word "object" by "process" :-) > > I won't argue that "good" native support is not better, but the analogy doesn't apply. How many threads do I really need? I need as many as I have actual concurrent activities. Even then it's OK for activities to block. So I don't need thousands of threads for good effect. I can activate passivate objects or I can have enough RAM. >Concurrent algorithms often provide a "natural" way to describe things (especially for >real-world programs - since the real world IS concurrent) - a sub-set of concurrent programming is sequential programming (but the converse is not true) - so in this sense >concurrent programming beats sequential programming hands down. > > Having had these conversations many times with embedded programmers, they just think a shared piece of data protected by a lock makes sense, even if they become mystefied when it all comes crashing down around them. Sending someone a paper on pi calculus is next to worthless. >The reason why concurrent programming has got a bad name and a reputation >of being difficult has to do with the threads, locks, mutexes, critical regions, >shared memory school of programming. > > I'd also point out that ericsson doesn't program their entire telecom devices in erlang. That shows there is a weakness and it turns people off. >The non-shared memory, pure copying asynchronous message passing, paradigm is >far easier to program (witness all the web applications, virtually all use >non-shared memory (ie separated processors) and pure message passing (defined by RFC's)) >and very easy to understand. > > And for scalability simply add clusters, replicated web tiers, replicated datababase backends, and L4-L7 redirectors :-) From joe.armstrong@REDACTED Wed Jul 13 16:50:41 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Wed, 13 Jul 2005 16:50:41 +0200 Subject: Meyer, OO and concurrency Message-ID: > > Joe Armstrong (AL/EAB) wrote: > > >Suppose there was an OO language called K. K "does" objects. > > > >In K you can use objects freely - but subject to the > following restrictions: > > > > - you should try to use less than say 2000 objects > > - if you use more than 20000 objects your program will crash > > - instead of destroying objects and creating new ones > you should try > > to keep a pool of old objects and reuse them as often > as possible > > - you should try and keep the total number of objects > down because > > programs with lots of objects are difficult to write > and understand and debug > > > >Now of course, "K does objects" - but not in any *useful* > sense - and even if it does > >objects - the programming style is so warped that nobody > wants to *use* objects as a method for structuring their programs. > > > >Now replace K by {Java, C++, C#, ...} and the word "object" > by "process" :-) > > > > > I won't argue that "good" native support is not better, but > the analogy > doesn't apply. How many threads do I really need? I need as many as I > have actual concurrent activities. Even then it's OK for > activities to > block. So I don't need thousands of threads for good effect. I can > activate passivate objects or I can have enough RAM. But our telcoms apps do need lots of processes - say 100,000 subscribers with a call model that needs 6 process/call = 600,000 processes ... > > >Concurrent algorithms often provide a "natural" way to > describe things (especially for > >real-world programs - since the real world IS concurrent) - > a sub-set of concurrent programming is sequential programming > (but the converse is not true) - so in this sense > >concurrent programming beats sequential programming hands down. > > > > > Having had these conversations many times with embedded programmers, > they just think a shared piece of data protected by a lock > makes sense, > even if they become mystefied when it all comes crashing down around > them. Sending someone a paper on pi calculus is next to worthless. > Erlang is not the Pi calculus - it's a programming langage that's easy to use. Don't show them the theory - give them the code that does the job :-) > >The reason why concurrent programming has got a bad name and > a reputation > >of being difficult has to do with the threads, locks, > mutexes, critical regions, > >shared memory school of programming. > > > > > I'd also point out that ericsson doesn't program their entire telecom > devices in erlang. That shows there is > a weakness and it turns people off. I don't think so. My banks uses Cobol (a lot) and C (a little) applying your argument would mean that C was weak ... The reason that Ericsson does not use Erlang for "their entire telecom devices" has nothing to do with the the merits of Erlang and everything do do with the failure of competing technologies. The big Erlang success story (AXD) came about when the "conventional" way of doing things failed. At the time Erlang was chosen for the AXD because all the alternative (conventional) methods had failed - it was not chosen becausen it was the only thing that worked. It's pretty clear that the best software technologies are rarely used in industry - old crap technology are extremely "sticky" and not easily replaced. > >The non-shared memory, pure copying asynchronous message > passing, paradigm is > >far easier to program (witness all the web applications, > virtually all use > >non-shared memory (ie separated processors) and pure message > passing (defined by RFC's)) > >and very easy to understand. > > > > > And for scalability simply add clusters, replicated web tiers, > replicated datababase backends, and L4-L7 redirectors :-) > Absolutly - "go parallel young man" /Joe > From vances@REDACTED Wed Jul 13 17:21:39 2005 From: vances@REDACTED (Vance Shipley) Date: Wed, 13 Jul 2005 11:21:39 -0400 Subject: Mnesia and uniqueness on two keys In-Reply-To: References: Message-ID: <20050713152139.GA86423@feeble.motivity.ca> On Wed, Jul 13, 2005 at 03:10:16PM +0200, Ulf Wiger wrote: } } In Mnesia, the 2nd element of the record is always the } primary key (if 'set' or 'ordered_set'). I know of no I had not noticed that before. In ets you can specify {keypos,Pos} so that any field in the record can be the key. I wonder why this support isn't in mnesia? -Vance From todd@REDACTED Wed Jul 13 17:23:38 2005 From: todd@REDACTED (todd) Date: Wed, 13 Jul 2005 08:23:38 -0700 Subject: Meyer, OO and concurrency In-Reply-To: References: Message-ID: <42D531FA.3090608@possibility.com> Joe Armstrong (AL/EAB) wrote: > >But our telcoms apps do need lots of processes - say 100,000 subscribers >with a call model that needs 6 process/call = 600,000 processes ... > > Need vs want. I've dealt with larger numbers mapped onto a set of far fewer threads with reasonable latency. >I don't think so. My banks uses Cobol (a lot) and C (a little) applying your argument would mean that C was weak ... > It's not really an argument. I am just telling you what the perception is from a real live person who has had these conversations within a real live telecom company. The fact that things like rerouting algorithms were not implemented in erlang was seen as a weakness. The fact that C++ could be used every level was seen as a strength. From ulf@REDACTED Wed Jul 13 17:42:12 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 17:42:12 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D525DB.9030806@possibility.com> References: <42D525DB.9030806@possibility.com> Message-ID: Den 2005-07-13 16:31:55 skrev todd : > Joe Armstrong (AL/EAB) wrote: >> Now replace K by {Java, C++, C#, ...} and the word "object" by >> "process" :-) >> > I won't argue that "good" native support is not better, butthe analogy > doesn't apply. How many threads do I really need? I think the analogy applies. BTW, the answer to that question doesn't start to dawn on people until they've spent a sufficiently long time with a language that lifts the arbitrary restrictions. Being used to languages with crappy concurrency support and UNIX-style heavyweight processes, it's too difficult to make the mental shift required to answer that question. I know lots of people with over 5 years of experience programming Erlang, who still can have difficulties breaking out of the sequential programming mindset. > I need as many as I have actual concurrent activities. Oh, but can you see which ones they are? It's about as difficult as doing object modeling, you need to get into the mode of thinking before you realise what is a good candidate for a process. > Even then it's OK for activities to block. So I don't need > thousands of threads for good effect. I can activate passivate > objects or I can have enough RAM. This is of course easier to do in Erlang than in most other languages, but erlang programmers seldom do it -- why? Because in more cases than not, you end up with messy, unintuitive code with weird bugs lurking around every corner. BTW, if Erlang processes were another order of magnitude or two lighter, we could start using processes in ways that we don't care to consider today. Doing an OO framework on top of Erlang would be a breeze. Neural net programming might be a real possibility. The limitations of our programming languages colour our thinking tremendously. > I'd also point out that ericsson doesn't program their > entire telecom devices in erlang. That shows there is > a weakness and it turns people off. And since you are still not convinced, that also shows that there is a weakness. ;-) Only a few days ago, I listened to a complaint about someone who still maintains that faxes are superior to computers and email. That says fairly little about email, but much more about the person in question. /Uffe From erlang@REDACTED Wed Jul 13 17:44:51 2005 From: erlang@REDACTED (Peter-Henry Mander) Date: Wed, 13 Jul 2005 16:44:51 +0100 Subject: Meyer, OO and concurrency In-Reply-To: <42D531FA.3090608@possibility.com> References: <42D531FA.3090608@possibility.com> Message-ID: <1121269491.17001.47.camel@hymir.newport-networks.com> On Wed, 2005-07-13 at 08:23 -0700, todd wrote: > The fact that C++ > could be used every level was seen as a strength. Like when you have a hammer everything is a nail? It could also be argued that C++ has warts for every possible purpose. Erlang isn't suitable for every situation, but it certainly has some real sweet spots when developing software not so close to the bare metal. Pete. -- "The Tao of Programming flows far away and returns on the wind of morning." From ulf@REDACTED Wed Jul 13 17:46:33 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 17:46:33 +0200 Subject: Mnesia and uniqueness on two keys In-Reply-To: <20050713152139.GA86423@feeble.motivity.ca> References: <20050713152139.GA86423@feeble.motivity.ca> Message-ID: Den 2005-07-13 17:21:39 skrev Vance Shipley : > On Wed, Jul 13, 2005 at 03:10:16PM +0200, Ulf Wiger wrote: > } > } In Mnesia, the 2nd element of the record is always the > } primary key (if 'set' or 'ordered_set'). I know of no > > > I had not noticed that before. In ets you can specify > {keypos,Pos} so that any field in the record can be the > key. I wonder why this support isn't in mnesia? Practical and performance reasons. Mnesia needs to extract the oid of records in many places, esp. in transaction handling. Even though one could abstract away the extra lookup to figure out the key position, it would be fairly messy to carry the key position around everywhere in order to avoid lots of extra meta-data lookups. I imagine (and vaguely recall) that it was seen as a reasonable sacrifice, but I don't recall the specific events or problems that led to the decision to drop keypos -- because it _was_ in there from the start. Perhaps Klacke or H?kan remembers? /Uffe From todd@REDACTED Wed Jul 13 18:19:51 2005 From: todd@REDACTED (todd) Date: Wed, 13 Jul 2005 09:19:51 -0700 Subject: Meyer, OO and concurrency In-Reply-To: References: <42D525DB.9030806@possibility.com> Message-ID: <42D53F27.2020804@possibility.com> Ulf Wiger wrote: > Den 2005-07-13 16:31:55 skrev todd : > >> Joe Armstrong (AL/EAB) wrote: >> >>> Now replace K by {Java, C++, C#, ...} and the word "object" by >>> "process" :-) >>> >> I won't argue that "good" native support is not better, butthe >> analogy doesn't apply. How many threads do I really need? > > > Oh, but can you see which ones they are? > It's about as difficult as doing object modeling, you > need to get into the mode of thinking before you realise > what is a good candidate for a process. It's somewhat pointless to argue about what's the best way. My response was that it is very doable in OO and it can work very well. That doesn't imply that I wouldn't want the world to be different. I would also want erlang to be more capable for performance related tasks as well. > > And since you are still not convinced, that also shows that > there is a weakness. ;-) When we have people litteraly searching out nanoseconds in their algorithms and doing everything possible to guarantee task switch latency, then I do remain unconvinced. > > Only a few days ago, I listened to a complaint about someone > who still maintains that faxes are superior to computers and > email. That says fairly little about email, but much more > about the person in question. So sorry I have yet to give up my chisel and stone. From ulf@REDACTED Wed Jul 13 18:24:56 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 18:24:56 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D531FA.3090608@possibility.com> References: <42D531FA.3090608@possibility.com> Message-ID: Den 2005-07-13 17:23:38 skrev todd : > Joe Armstrong (AL/EAB) wrote: > >> >> But our telcoms apps do need lots of processes - say 100,000 subscribers >> with a call model that needs 6 process/call = 600,000 processes ... > Need vs want. I've dealt with larger numbers mapped onto a set of far > fewer threads with reasonable latency. Sure, we have too. We've supported 70-80,000 active state machines using Erlang versions that supported only 32,000 processes. We've also gone through some iterations of finding workarounds to the problem of not being able to use the most intuitive model. Now, with that limit removed, we're more or less revisiting models that we started with 9 years ago, in the earliest prototypes. An enormous amount of work could have been saved if we could have used the most intuitive models from the start (This is not to be viewed as criticism. No other technology that we knew of then, or now, could have given us better support.) >> I don't think so. My banks uses Cobol (a lot) and C (a little) applying >> your argument would mean that C was weak ... >> > It's not really an argument. I am just telling you what the perception > is from a real live person who has had these conversations within a real > live telecom company. The fact that things like rerouting algorithms > were not implemented in erlang was seen as a weakness. The fact that C++ > could be used every level was seen as a strength. I'm sure Joe is willing to listen to what other people have experienced when talking to others about Erlang. Hopefully _you_ are aware of whom it is you're trying to educate? I believe that Joe has talked to more people, given more courses, and fought more battles over Erlang than most (Joe can most likely name all those who could claim to have done more.) /Uffe From jerome@REDACTED Wed Jul 13 18:51:35 2005 From: jerome@REDACTED (=?ISO-8859-1?Q?J=E9r=F4me_Desquilbet?=) Date: Wed, 13 Jul 2005 18:51:35 +0200 Subject: Same Mnesia database from different Unix processes Message-ID: <42D54697.8040901@desquilbet.org> Hello, Is it possible to access the same Mnesia database from different Unix processes? Like two erl shells or Yaws and an erl shell? In my particular case, I would like to use tv on a database that is used by a running Yaws. I guess this a very basic question, sorry, J?r?me. From todd@REDACTED Wed Jul 13 18:53:29 2005 From: todd@REDACTED (todd) Date: Wed, 13 Jul 2005 09:53:29 -0700 Subject: Meyer, OO and concurrency In-Reply-To: References: <42D531FA.3090608@possibility.com> Message-ID: <42D54709.7050601@possibility.com> Ulf Wiger wrote: > > I'm sure Joe is willing to listen to what other people > have experienced when talking to others about Erlang. > Hopefully _you_ are aware of whom it is you're trying to > educate? I am not trying to educate anyone, that's way too hard.. I am just responding to a post. You are free to consider me a relic from the age of fax if you wish. From thomasl_erlang@REDACTED Wed Jul 13 19:13:18 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 13 Jul 2005 10:13:18 -0700 (PDT) Subject: Meyer, OO and concurrency In-Reply-To: <42D53F27.2020804@possibility.com> Message-ID: <20050713171318.28673.qmail@web34412.mail.mud.yahoo.com> --- todd wrote: > When we have people litteraly searching out > nanoseconds in their > algorithms and doing everything possible to > guarantee task switch > latency, then I do remain unconvinced. The sensible approach to this argument is probably to concede that sort of problem to the low-level programmers (for now :-) and use Erlang for more suitable work. My impression is that Erlang even so produces quite respectable performance in many real applications. For instance, my most recent effort, a mobile network node, exceeded the performance requirements by 2x (at least -- that's when the testers stopped). For this sort of application, there's really no need to wring out nanoseconds or fret about task switching times. In fact, doing so is just wasting time and money. Best, Thomas __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From chas@REDACTED Wed Jul 13 19:35:18 2005 From: chas@REDACTED (Charles Blair) Date: Wed, 13 Jul 2005 12:35:18 -0500 Subject: catching SIGTERM Message-ID: <20050713173518.GC11715@jump.lib.uchicago.edu> i'd like to be able to trap a SIGTERM from an application written in erlang+yaws so i can dump some state to disk when that happens. is there a way to do this? searching the net brought up two similar questions with no responses. thanks. From ulf@REDACTED Wed Jul 13 20:16:35 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 20:16:35 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D54709.7050601@possibility.com> References: <42D531FA.3090608@possibility.com> <42D54709.7050601@possibility.com> Message-ID: Den 2005-07-13 18:53:29 skrev todd : > Ulf Wiger wrote: > >> I'm sure Joe is willing to listen to what other people >> have experienced when talking to others about Erlang. >> Hopefully _you_ are aware of whom it is you're trying to >> educate? > > I am not trying to educate anyone, that's way too hard..I am just > responding to a post. You are free to consider > me a relic from the age of fax if you wish. You accounted for the views of others that C++ was a safer and more all-round choice than Erlang. I suggested that their views might be colored by lack of knowledge and experience of the alternative that they were snubbing. You indicated several times that you didn't necessarily agree with them. Thus, the "fax anecdote" was not directed at you. There may be fax fanatics lurking on erlang-questions -- I don't know. I've assumed that most of them have little experience with erlang. Your presence on this list led me to believe that you see some redeeming qualities in Erlang after all. ;) /Uffe From ulf@REDACTED Wed Jul 13 20:18:27 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 20:18:27 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D53F27.2020804@possibility.com> References: <42D525DB.9030806@possibility.com> <42D53F27.2020804@possibility.com> Message-ID: Den 2005-07-13 18:19:51 skrev todd : > When we have people litteraly searching out nanoseconds in their > algorithms and doing everything possible to guarantee task switch > latency, then I do remain unconvinced. This is a group of users that has never been targeted by Erlang. /Uffe From ulf@REDACTED Wed Jul 13 20:20:55 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 20:20:55 +0200 Subject: Same Mnesia database from different Unix processes In-Reply-To: <42D54697.8040901@desquilbet.org> References: <42D54697.8040901@desquilbet.org> Message-ID: Den 2005-07-13 18:51:35 skrev J?r?me Desquilbet : > Hello, > Is it possible to access the same Mnesia database from different Unix > processes? Like two erl shells or Yaws and an erl shell? > In my particular case, I would like to use tv on a database that is used > by a running Yaws. > I guess this a very basic question, sorry, > J?r?me. You can start the other erlang node with the option -mnesia extra_db_nodes \[...\], telling mnesia to fetch a ram-based data dictionary from the node running yaws. Then, you can access all tables from the other node. Search the archive for extra_db_nodes to see issues on quoting etc. in different operating systems if you don't figure it out on your own. /Uffe From todd@REDACTED Wed Jul 13 20:59:00 2005 From: todd@REDACTED (todd) Date: Wed, 13 Jul 2005 11:59:00 -0700 Subject: Meyer, OO and concurrency In-Reply-To: References: <42D531FA.3090608@possibility.com> <42D54709.7050601@possibility.com> Message-ID: <42D56474.40201@possibility.com> Ulf Wiger wrote: >> I am not trying to educate anyone, that's way too hard..I am just >> responding to a post. You are free to consider >> me a relic from the age of fax if you wish. > > You accounted for the views of others that C++ was a safer > and more all-round choice than Erlang. You've got to be kidding. I said just the opposite several times. The contention that you can't do concurrency on an OO langauge like C++ is just plain wrong. That you can do it better in other languages like erlang is unquestioned, assuming a suitable definition of better. If better involves low latency then erlang is not better. If better is related to the concurrency model then erlang is better. Partitioning a large embedded system into two different language domains is very unnatractive to many many people. Having been the one to try and sell this, I know. Call that the myopic application of the hammer-nail pattern, if it pleases you. Can I do concurrency in C++ well enough to keep a single language for the lowest part of the system to the highest? Yes. > This is a group of users that has never been targeted by Erlang. That's the embedded world. It has the ultimate mix of high performance code, mixed with scaling issues, mixed with low performance code. To ask people to partition the system using a completely new language they have no experience with is not attractive enough. Everything you need to do in erlang you also have to be able to do on the C++ side as well. If you could make it so it could all be done in one language that would be attractive. So, I know I am wasting my time saying all this, but my original intent was to counter the original assertion, not comment on erlang. From joelr1@REDACTED Wed Jul 13 21:09:05 2005 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 13 Jul 2005 21:09:05 +0200 Subject: Mnesia not suitable for time series storage? In-Reply-To: References: <20050713122222.1573.qmail@web34413.mail.mud.yahoo.com> Message-ID: <90C24DC4-A1CF-4422-9DB7-B9CD4E5B8A6C@gmail.com> Writing a distributed database is something that I would like to avoid at all cost. Even if it requires my switching to another platform. On Jul 13, 2005, at 3:33 PM, Ulf Wiger wrote: > [...] > Or perhaps this gets a lot easier if one uses ets > tables instead of mnesia. Keeping a list of ets > tables, and the age of objects in each, one can > simply drop tables that are too old. Problem here > is that we end up writing our own distributed database. -- http://wagerlabs.com/uptick From jerome@REDACTED Wed Jul 13 21:26:27 2005 From: jerome@REDACTED (=?ISO-8859-1?Q?J=E9r=F4me_Desquilbet?=) Date: Wed, 13 Jul 2005 21:26:27 +0200 Subject: Example of a language mixing OO and concurrency (was: Meyer, OO and concurrency) In-Reply-To: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> Message-ID: <42D56AE3.8060002@desquilbet.org> As an example for people interested in various programming languages, Ada 0Y (3rd revision of the ISO standard, sometimes in 200Y), includes new features for mixing OO and concurrency. See a presentation here: http://www.cs.kuleuven.ac.be/%7Edirk/ada-belgium/events/04/040616-aec-ada2005.pdf J?r?me. From jerome@REDACTED Wed Jul 13 22:59:52 2005 From: jerome@REDACTED (=?ISO-8859-15?Q?J=E9r=F4me_Desquilbet?=) Date: Wed, 13 Jul 2005 22:59:52 +0200 Subject: Same Mnesia database from different Unix processes (was also: Bonjour + Mnesia) In-Reply-To: References: <42D54697.8040901@desquilbet.org> Message-ID: <42D580C8.3020905@desquilbet.org> Thanks a lot Ulf. I didn't manage to make it work on the command line, but in the shell, yes, like this: $ erl -sname newnode > .mnesia:start(). > .mnesia:change_config(extra_db_nodes, ['othernode']). > .tv:start(). > .mnesia:stop(). > .init:stop(). $ And yes, it is explained in the reference manual ;-) J?r?me. Ulf Wiger wrote: > Den 2005-07-13 18:51:35 skrev J?r?me Desquilbet : > >> Hello, >> Is it possible to access the same Mnesia database from different Unix >> processes? Like two erl shells or Yaws and an erl shell? >> In my particular case, I would like to use tv on a database that is >> used by a running Yaws. >> I guess this a very basic question, sorry, >> J?r?me. > > > You can start the other erlang node with the option > -mnesia extra_db_nodes \[...\], telling mnesia to > fetch a ram-based data dictionary from the node running > yaws. Then, you can access all tables from the other node. > > Search the archive for extra_db_nodes to see issues on > quoting etc. in different operating systems if you don't > figure it out on your own. > > /Uffe > > From ulf@REDACTED Wed Jul 13 23:36:23 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 23:36:23 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D56474.40201@possibility.com> References: <42D531FA.3090608@possibility.com> <42D54709.7050601@possibility.com> <42D56474.40201@possibility.com> Message-ID: Den 2005-07-13 20:59:00 skrev todd : > Ulf Wiger wrote: > >> You accounted for the views of others that C++ was a safer >> and more all-round choice than Erlang. > > You've got to be kidding. I said just the opposite several times. ??? On 2005-07-13 17:23:38 you wrote: > I am just telling you what the perception is from a real live person who > has had these conversations within a real live telecom company. The fact > that things like rerouting algorithms were not implemented in erlang was > seen as a weakness. The fact that C++ could be used every level was seen > as a strength. I don't think my description of what you wrote is terribly far-fetched. > The contention that you can't do concurrency on an OO langauge like > C++ is just plain wrong. No one has contended that. It's been contended, however, that threads programming with shared state is very difficult and error-prone. > That you can do it better in other languages like erlang is unquestioned, > assuming a suitable definition of better. If better involves low latency > then erlang is not better. A very ambitious benchmark of database management systems for embedded applications was once done at Ericsson. For various reasons, I doubt that it will be published outside. Mnesia was one of the DBMSes tested, and it performed as well as the best third party product (Clustra) in terms of TPS and scalability (on which it was only beaten by two DBMSes developed in C++ in-house.) As an afterthought, latency was derived from the test results. By this measure, mnesia was the winner by a wide margin. It was contended by some that they would have done better if they'd known that latency was to be measured, but just about everyone was quite surprised that mnesia did as well as it did on latency. > Partitioning a large embedded system into two different language domains > is very unnatractive to many many people. Having been the one to try and > sell this, I know. Call that the myopic application of the hammer-nail > pattern, if it pleases you. I have plenty of experience building systems that use several different languages. There are several problems with this, I agree. My personal opinion is that the difficulties are often exaggerated, and that the single-language approach is often highly over-estimated. > Can I do concurrency in C++ well enough to keep a single language for > the lowest part of the system to the highest? Yes. Good for you. I've seen many examples of this approach failing miserably. My own opinion of C++ is that it's a disaster in large projects, but can be successful in a small project with very knowledgeable C++ programmers, and at least a few brilliant ones. Even then, I've often thought (especially looking back) that the problems with taking the C++ approach have been radically under- estimated, and that using Erlang would have been a much better option -- even if it meant having to teach it to all developers from scratch. > > This is a group of users that has never been targeted by Erlang. > > That's the embedded world. That's _one type_ of embedded systems. I'm pretty familiar with it, even though I've not programmed in such an environment myself. Telco systems tend to have subsystems with these characteristics, but the telco systems themselves are also embedded systems. There are also embedded systems that require hard real-time characteristics, and these are expressly not targeted by Erlang. But accepting the requirement for liveliness guarantees can buy endless difficulties if the system doesn't require it, and the concurrency patterns are tricky enough. > It has the ultimate mix of high performance code, mixed with scaling > issues, mixed with low performance code. To ask people to partition > the system using a completely new language they have no experience > with is not attractive enough. > Everything you need to do in erlang you also have to be able to do on > the C++ side as well. Obviously, I can't comment on the systems you've worked on, but having worked with multi-language systems since... well, 1989, I'd have to say that with a good partitioning, this is absolutely not the case. > So, I know I am wasting my time saying all this, It's not a waste of time, as long as the discussion stays professional. Eventually, one may have to agree to disagree and wrap things up, though. (: /Uffe From ulf@REDACTED Wed Jul 13 23:39:36 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 13 Jul 2005 23:39:36 +0200 Subject: Same Mnesia database from different Unix processes (was also: Bonjour + Mnesia) In-Reply-To: <42D580C8.3020905@desquilbet.org> References: <42D54697.8040901@desquilbet.org> <42D580C8.3020905@desquilbet.org> Message-ID: Den 2005-07-13 22:59:52 skrev J?r?me Desquilbet : > Thanks a lot Ulf. I didn't manage to make it work on the command line, > but in the shell, yes, like this: > > $ erl -sname newnode > > .mnesia:start(). > > .mnesia:change_config(extra_db_nodes, ['othernode']). > > .tv:start(). > > .mnesia:stop(). > > .init:stop(). > $ > > And yes, it is explained in the reference manual ;-) > J?r?me. You can also put it in a sys.config file, e.g. in the sys.config: ------------------ [{mnesia, [ {extra_db_nodes, [...]} ]} ]. Then: erl -sys ..../sys /Uffe From sean.hinde@REDACTED Wed Jul 13 23:59:59 2005 From: sean.hinde@REDACTED (Sean Hinde) Date: Wed, 13 Jul 2005 22:59:59 +0100 Subject: Batch-loading Mnesia In-Reply-To: <0B5AF9E2-D468-4ED3-A59C-EB2C12CD44B1@gmail.com> References: <0B5AF9E2-D468-4ED3-A59C-EB2C12CD44B1@gmail.com> Message-ID: If you are using the same value of datetime for all rows in your test routine then this could lead to that behaviour. The reason is that all the primary keys must be added to a single entry in the index lookup table, which gets slow after a while. Even so, that is spectacularly slow compared with my experience of such small tables Sean On 13 Jul 2005, at 10:22, Joel Reymont wrote: > Folks, > > It's taking me way over an hour (maybe two hours or three hours) to > load 119,275 records into Mnesia. Parsing the same file without > adding record takes about 6 seconds. > > I'm doing everything on the same node, adding record sequentially. > Is there a way to speed things up? > > The table has the following format > > -record(tick, { > symbol, > datetime, > timestamp, > price, > size > }). > > Symbol is a string (list) of 4 characters, datetime is a bignum > formed from a date and time 20040901,09:39:38 by stripping the > comma and the colons. Timestamp is now(), price and size are float > and integer respectively. > > Table is created like this: > > mnesia:create_table(tick, > [ > {disc_copies, Nodes}, > {index, [datetime]}, > {type, bag}, > {attributes, record_info(fields, tick)}]) > > The function that adds a record to the database looks like this: > > add_tick(Symbol, Date, Time, Price, Size) -> > Tick = #tick { > symbol = Symbol, > datetime = Date * 1000000 + Time, > price = Price, > size = Size, > timestamp = now() > }, > %%F = fun() -> mnesia:write(Tick) end, > %%mnesia:transaction(F). > mnesia:dirty_write(Tick). > > -- > http://wagerlabs.com/uptick > > > > From ok@REDACTED Thu Jul 14 00:24:19 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 14 Jul 2005 10:24:19 +1200 (NZST) Subject: DTD / xmerl_scan:file help !!!!! {invalid_nmtoken,"*) #RE" Message-ID: <200507132224.j6DMOJqF149209@atlas.otago.ac.nz> "Sanjaya Vitharana" wrote: ---Note the * (astric in the end) The asterisk (NOT "astric") is *NOT ALLOWED* in an enumerated attribute. Here are the relevant rules from the XML specification (one does expect people programming with XML to read the XML specification...): [59] Enumeration ::= '(' S? Nmtoken (S? '|' S? Nmtoken)* S? ')' [7] Nmtoken ::= (Namechar)+ [4] Namechar ::= Letter | Digit | '.' | '-' | '_' | ':' | CombiningChar | Extender Roughly speaking, an Nmtoken is any non-empty sequence of characters that may occur in an identifier, not necessarily starting with a letter. So 1 is an Nmtoken and c is an Nmtoken but * is *NOT* an Nmtoken. Error I got is. 1> {ParsResult2,Misc2}=3Dxmerl_scan:file('ivr.xml',[{validation,true }]). 2798- fatal: {invalid_nmtoken,"*) #RE"} ** exited: {fatal,{{invalid_nmtoken,"*) #RE"}, {file,"./ivr.dtd"}, {line,21}, {col,47}}} ** Seems to be * (astric) is invalid in DTD (or may be xmerl_scan = validation problem). Any validating XML parser *MUST* report this error somehow. This particular error message is, for a wonder, just right. It tells you something true ("*" is not an Nmtoken) and relevant (by implication, an Nmtoken is required there) and gives you the right location. But i want to use * (astric) in my XML file as below. So there must be a way to define it in DTD ..... can any DTD expert help me in this case. My first suggestion is TRY WANTING SOMETHING ELSE. Why not try wanting '-' or '.' instead? They *are* Nmtokens. My second suggestion, if there is someone pointing a gun at your head saying "use the un-XML-like asterisk here or DIE", is to give up the two things that enumerated attributes provide: - a validating parser checks that the value of an enumeration attribute is in fact one of the listed values - a validating parser removes white space around the Nmtoken in such an attribute value, so that 'r', 'r ', ' r ' all give the same 'r' value. If you are willing to give those up, then all you have to write is and any old string will be accepted by the XML parser. This means that it is then your problem to check the values given a parsed document, rather than the parser's problem. From vances@REDACTED Thu Jul 14 00:26:30 2005 From: vances@REDACTED (Vance Shipley) Date: Wed, 13 Jul 2005 18:26:30 -0400 Subject: Same Mnesia database from different Unix processes (was also: Bonjour + Mnesia) In-Reply-To: References: <42D54697.8040901@desquilbet.org> <42D580C8.3020905@desquilbet.org> Message-ID: <20050713222629.GB89016@feeble.motivity.ca> On Wed, Jul 13, 2005 at 11:39:36PM +0200, Ulf Wiger wrote: } } You can also put it in a sys.config file, e.g. ... } Then: erl -sys ..../sys I think Uffe meant -config and not -sys. -Vance From ok@REDACTED Thu Jul 14 00:32:01 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 14 Jul 2005 10:32:01 +1200 (NZST) Subject: DTD / xmerl_scan:file help !!!!! {invalid_nmtoken,"*) #RE" Message-ID: <200507132232.j6DMW1u2139534@atlas.otago.ac.nz> Helmut Enck-Radana wrote: in XML DTDs enumerated values have to be NMTOKENs, i.e. you can only use strings which could be used as names. That's not quite right. All Names are Nmtokens, but Names have to start with a letter, and Nmtokens don't. 1 is an Nmtoken but not a Name. An extremely horrible design glitch in SGML is that SGML uses "--" to begin a comment, but "--" is also an Nmtoken. There are thus a number of places where any reasonable person would expect to be able to write a comment, but you can't. You have to use a different schema language, One of the features in the full version of SGML is that an attribute can be specified as "DATA " where specifies one of the standardised regular expression notations and is a pattern in that notation saying what good values look like. I've always thought it a pity that XML didn't retain that feature. From ok@REDACTED Thu Jul 14 00:47:11 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 14 Jul 2005 10:47:11 +1200 (NZST) Subject: Meyer, OO and concurrency Message-ID: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> Peter-Henry Mander wrote: Why are we so stuck in the muddy quagmire of reinventing workarounds to avoid explicit concurrency in our software? My question is more about psychology than about technology. And it applies equally to functional programming as it does to concurrency. Is it down to education? I suspect that it is primarily due to one thing: assignment statements. Assignment statements have an AMAZING grip on programmers' minds. To my atonishment, horror, and disgust, there is even a serious attempt to get assignment statements into the next version of the Prolog standard. You may not believe that, so I shall repeat it: there is a serious attempt to get ASSIGNMENT STATEMENTS into the next version of the Prolog standard, on the alleged grounds that assignments are absolutely necessary for efficiency and that all existing systems support them anyway. (Worse still, the proposal has two different kinds of assignment statements, or is it three.) And that proposal is being seriously advanced at a time when lots of Prolog systems now support constraint handling, which means that bits of code may run in orders and at times which are, to even a really good programmer, quite unpredictable. Prolog needs assignment statements like birds need rails. Even Erlang has the shameful process dictionary, but at least process dictionaries are private and cannot be modified by any other process. I believe that there is a vicious circle operating between education and assignment statements. I've been reading about parallel programming for years. Apart from Erlang, the only language I've ever really felt comfortable writing anything concurrent in is Ada95, because you DON'T explicitly manipulate locks in Ada, that's the compiler's job. And the one place I know of that taught Ada junked it in favour of Java because Java had standardised graphics. I note that Dijkstra had a series of papers on the topic "Building elephants out of mosquitoes humming in harmony", not entirely unlike systolic systems. Oddly enough, the notation Dijkstra invented for "A Discipline of Programming" is a functional language in imperative guise: each block must list ALL its inputs and ALL its outputs and there is no sharing between mutable arrays, so (rather like Euclid) it is possible to mechanically transliterate any program in Dijkstra's notation into, say, a program in the functional language Clean. From ok@REDACTED Thu Jul 14 00:55:04 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 14 Jul 2005 10:55:04 +1200 (NZST) Subject: DTD / xmerl_scan:file help !!!!! {invalid_nmtoken,"*) #RE" Message-ID: <200507132255.j6DMt453151447@atlas.otago.ac.nz> changing it to will sort the problem. Except that the word is "asterisk". Since the mobile phone has the * (astric) key, What has this to do with your problem? Neither my current mobile phone nor my previous one has any key labelled "z" or "r" or "c", and while Unicode _does_ have a phone symbol, there doesn't seem to be any way to represent "red phone handset" or "green phone handset" in Unicode. it is interesting to know how to express the constraint as we like. You can't. Could you please guide me to some reference materials or some better examples to pick this quickly. Read the XML specification. It's all on-line, it's all free. http://www.w3.org look at the alphabetic list of topics down the left edge (it's a long one). is it not possile to define a datatype like below ??? MyType::= #x9 | #xA | #xD Not in a DTD, no. You _can_ define data types in XML Schemas. (The Schema specification is available from the W3 web site also.) Or using Entities are simple parameterless unconditional string replacement macros. As I never did it before, I don't know how this can be done. All that is involved is shifting the checking from the XML parser to your program that walks the parse tree. Wrox Press and O'Reilly have plenty of good books about XML. From ulf@REDACTED Thu Jul 14 02:02:07 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 14 Jul 2005 02:02:07 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> References: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> Message-ID: Den 2005-07-14 00:47:11 skrev Richard A. O'Keefe : > Even Erlang has the shameful process dictionary, but at least process > dictionaries are private and cannot be modified by any other process. (Mainly for the lurkers and beginners on the list) I've argued before that I think process dictionaries have received a bad reputation unjustly. They have the same basic semantics as private ets tables, but have some nice advantages besides the obvious advantage of convenience. Attached is an incomplete (and inefficient) implementation of a process dictionary, written in plain erlang. The inefficiency comes mainly from the very naiive choice of access structures in the program, not from the message passing (unless the key-value pairs are big and we're not using shared or hybrid heap.) There's nothing in the process dictionary that can't be implemented with same semantics in plain erlang, using a registered process and some message passing. 9> c(procdict). {ok,procdict} 10> procdict:start(). <0.67.0> 11> procdict:put(a,17). undefined 12> procdict:get(a). 17 13> procdict:get(). [{a,17}] 14> procdict:get(b). undefined 15> procdict:put(a,147). 17 One might argue that it's wrong to make it so convenient and efficient to use something like a process dictionary. Of course, if the process communication can be made arbitrarily fast, you can easily implement destructive assignment of global variables in Erlang -- process dictionary or no process dictionary. The culprit is the ! operator. ;-) /Uffe -------------- next part -------------- A non-text attachment was scrubbed... Name: procdict.erl Type: application/octet-stream Size: 2078 bytes Desc: not available URL: From vances@REDACTED Thu Jul 14 03:24:21 2005 From: vances@REDACTED (Vance Shipley) Date: Wed, 13 Jul 2005 21:24:21 -0400 Subject: Meyer, OO and concurrency In-Reply-To: References: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> Message-ID: <20050714012420.GC89016@feeble.motivity.ca> On Thu, Jul 14, 2005 at 02:02:07AM +0200, Ulf Wiger wrote: } } I've argued before that I think process dictionaries have received } a bad reputation unjustly. They have the same basic semantics as } private ets tables, but have some nice advantages besides the obvious } advantage of convenience. Sure, it's just an optimization. We could use erlang servers instead of ets as well but ets uses bifs and performs better in most cases. I used the process dictionary when I wrote my first application. It would be interesting to see what that looked like now but it's been lost to protect the ignorant. The answer to global variables seemed to be to use the process dictionary so that's what we did. The light came on soon afterwards and I haven't used it again. Nor have I used "private ets tables". At times I probably should have as the error reports can be unwieldy when you have too much state data. But this is not to say that there is no place for the process dictionary or ets. It's just that in most cases folks will use them to program imperatively. -Vance From ulf@REDACTED Thu Jul 14 09:54:09 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 14 Jul 2005 09:54:09 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <20050714012420.GC89016@feeble.motivity.ca> References: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> <20050714012420.GC89016@feeble.motivity.ca> Message-ID: Den 2005-07-14 03:24:21 skrev Vance Shipley : > I used the process dictionary when I wrote my first application. > It would be interesting to see what that looked like now but it's > been lost to protect the ignorant. The answer to global variables > seemed to be to use the process dictionary so that's what we did. > > The light came on soon afterwards and I haven't used it again. Nor > have I used "private ets tables". At times I probably should have > as the error reports can be unwieldy when you have too much state > data. But this is not to say that there is no place for the process > dictionary or ets. It's just that in most cases folks will use them > to program imperatively. Granted. The part I've objected to in the past was the "process dictionary bad, ets good" mentality, where both have similar semantics, and both can be emulated by processes and message passing. The underlying notion is _side effects_. They are absolutely necessary for any interesting program, but how often we use them, and in how many places in our code, has a dramatic effect on code quality. BTW, as Klacke (who wrote the first ets implementation) says: The purpose of ets was to provide efficient off-heap storage for large data sets. Thus, one of the key features of ets is that it's not garbage-collected. /Uffe From erlang@REDACTED Thu Jul 14 10:07:17 2005 From: erlang@REDACTED (Peter-Henry Mander) Date: Thu, 14 Jul 2005 09:07:17 +0100 Subject: Meyer, OO and concurrency In-Reply-To: References: <42D531FA.3090608@possibility.com> <42D54709.7050601@possibility.com> <42D56474.40201@possibility.com> Message-ID: <1121328438.31308.37.camel@hymir.newport-networks.com> On Wed, 2005-07-13 at 23:36 +0200, Ulf Wiger wrote: > Den 2005-07-13 20:59:00 skrev todd : > > > Ulf Wiger wrote: > > > >> You accounted for the views of others that C++ was a safer > >> and more all-round choice than Erlang. > > > > You've got to be kidding. I said just the opposite several times. > > ??? Hi Uffe, hi Todd, I think yet another language is causing unintended discord: English. You're a good sport Todd. Having re-read the postings the whole discussion could well have descended into a flamefest, but like Uffe said, it's stayed pretty civil nonetheless. May I just add a drop more oil(*) to the topic? Please don't take what follows personally. I have found that when learning Erlang it's helpful to un-learn all one's assumptions and practices, and not carry over any experience of imperative programming; It simply doesn't help when trying to grasp some of the fundamental concepts that apply in functional/logic programming. I have also found that Erlang scares people, in my case experienced C++ programmers, because they're stripped of all acquired knowledge and tools; they must start to re-learn from scratch something they've grown confident and capable in. It's _not_ like switching from C++ to Java, for example, and involves dismissing a lot of invested time and study in imperative languages. As a result only a few people I know have happily got their hands dirty, and they're usually the ones who remember Prolog et al. from university. To appreciate the Erlang Way you need to climb a steep, but short learning curve, and immerse oneself completely until the pennies drop. Erlang, for all it's academic inspirations, is a pragmatic language. There are many level by which it connects with external program modules, ranging from IP/UDP to tightly coupled link-in drivers written on the bare metal in C. There are several levels of term storage such as process dictionaries, ets and mnesia. The OTP toolkit is so valuable in it's own right that Erlang almost becomes a simple enablement technology. This is why I became interested in Erlang, it has an excellent Megaco stack which allowed me to provide a protocol tester in record time, even though I had to learn a completely new language. Pete. (*) which may lubricate or add fuel to the fire, lets see... -- "The Tao of Programming flows far away and returns on the wind of morning." From mike@REDACTED Thu Jul 14 11:31:33 2005 From: mike@REDACTED (Michael Williams) Date: 14 Jul 2005 09:31:33 GMT Subject: Concepts that people find hard (was: Re: Meyer, OO and concurrency) References: <42D56474.40201@possibility.com>, , <1121328438.31308.37.camel@hymir.newport-networks.com> Message-ID: In article <1121328438.31308.37.camel@REDACTED>, erlang@REDACTED (Peter-Henry Mander) writes: |> I have also found that Erlang scares people, in my case experienced C++ |> programmers, because they're stripped of all acquired knowledge and |> tools; they must start to re-learn from scratch something they've grown |> confident and capable in. It's _not_ like switching from C++ to Java, Many years ago (:-) I held a lot of basic Erlang courses. Many people had trouble with atoms, single assignment, scope of variables, recursion etc, but maybe as many as one student in ten never grasped the idea of processes and concurrency. It was as if they had some wierd mental block and I could do very little to help them. It is a bit like riding a bicycle or learning to swim. Once you know how to do it, it's very hard to comprehend that one once found it difficult. Many people first experience concurrency in an extremely hard context, e.g. UNIX sockets or pthreads. When you already have a concept which people initially find it hard to grasp, obfuscating it with a bewildering barrage of system calls (socket, bind, listen, accept, select etc) with obscure semantics rapidly turns it into an area where people just use "cookbook" solutions to avoid having to grasp the details of how it all hangs together (if is does so at all). So it's not altogether suprising that Erlang wins ground only slowly. Not only are people subjected to the monster of concurrency, they also have to contend with a functional language at the same time. But once one has understood it, its hard to see how anyone can tolerate the mind boggling difficulties of trying to do concurrent programming in C++ combined with a wierd OS or middleware. /mike From irmatov@REDACTED Thu Jul 14 13:20:22 2005 From: irmatov@REDACTED (Timur Irmatov) Date: Thu, 14 Jul 2005 16:20:22 +0500 Subject: Concepts that people find hard (was: Re: Meyer, OO and concurrency) In-Reply-To: References: <42D56474.40201@possibility.com> <1121328438.31308.37.camel@hymir.newport-networks.com> Message-ID: <241d382f0507140420ad9e78f@mail.gmail.com> On 14 Jul 2005 09:31:33 GMT, Michael Williams wrote: > So it's not altogether suprising that Erlang wins ground only slowly. Not only > are people subjected to the monster of concurrency, they also have to contend > with a functional language at the same time. But once one has understood it, its > hard to see how anyone can tolerate the mind boggling difficulties of trying to > do concurrent programming in C++ combined with a wierd OS or middleware. I came to Erlang less than a month ago. I work as system administrator at small ISP and do some programming with python/perl. Recently I've searched the internet for open-source SIP proxies. First I looked at Partysip, then SER. And then I found YXA, written in Erlang. It had an overview of it's design in terms of workers and supervisors, it seemed interesting to me. Then I came to http://www.erlang.org/white_paper.html. That thing *really* blew my mind. I started to learn Erlang. The language really made me excited. I had a little experience with C/C++/Java. It is really hard to read and understand other people's code. That was completely different with Erlang. I browsed the code of YXA and surprisingly noticed that it is really possible to understand how everything works! "A language that doesn't affect the way you think about programming, is not worth knowing." (c) unknown. May be the language is not suitable for some of embedded development, but it is great for many other applications. -- Timur Irmatov, JID: thor@REDACTED From jerome@REDACTED Thu Jul 14 15:29:59 2005 From: jerome@REDACTED (=?ISO-8859-1?Q?J=E9r=F4me_Desquilbet?=) Date: Thu, 14 Jul 2005 15:29:59 +0200 Subject: Whorfian mind-lock? (Re: Concepts that people find hard) In-Reply-To: References: <42D56474.40201@possibility.com>, , <1121328438.31308.37.camel@hymir.newport-networks.com> Message-ID: <42D668D7.9040108@desquilbet.org> Isn't it what is called a Whorfian mind-lock? ( http://catb.org/%7Eesr/jargon/html/W/Whorfian-mind-lock.html ) Some times ago, I tried to explore how the SWH can apply to software engineering (using UML as an example, http://jerome.desquilbet.org/lojban/rattle/#be ). J?r?me. Michael Williams wrote: > It was as if they had some wierd mental block and I could do very little to > help them. It is a bit like riding a bicycle or learning to swim. Once you know > how to do it, it's very hard to comprehend that one once found it difficult. From sebastian@REDACTED Thu Jul 14 15:49:53 2005 From: sebastian@REDACTED (Sebastian Bello) Date: Thu, 14 Jul 2005 10:49:53 -0300 Subject: mnesia query Message-ID: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> Hi all, imagine I have a query which returns a very big number of records, but am interested in just a fixed small number of them. Is there a way to retrieve, let's say, just the first N or just N of them? Not doing, ie getting the whole result, implies a performance penalty other than memory consumption? Thanks, Sebastian- Prepaid Expertise - Programmable Switches Powered by Ericsson Licensed Technology Sebasti?n Bello - Engineer - Development Center - IN Switch Solutions Inc. Headquarters - Miami-U.S.A. Tel: 1305-3578076 Fax: 1305-7686260 Development Center - Montevideo - Uruguay Tel/Fax: 5982-7104457 IN SWITCH EMEA Phone: +33 0 6 0335 9427 - Fax: +33 0 4 93655773 / emea@REDACTED IN SWITCH ASIA Phone: +92 51 2800397/8- Fax: +92 51 2800399/ inswasia@REDACTED e-mail: sebastian@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1429 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: logoparafondo.gif Type: image/gif Size: 16886 bytes Desc: not available URL: From james.hague@REDACTED Thu Jul 14 16:07:12 2005 From: james.hague@REDACTED (James Hague) Date: Thu, 14 Jul 2005 09:07:12 -0500 Subject: Decoupling the Windows GUI from werl.exe Message-ID: The Windows front-end to Erlang is built into the werl.exe executable. I'm talking about the GUI that lets you enter expressions, see results, change the font, etc. (For non-Windows users, this necessary because the standard command-line terminal window is fundamentally crippled in several ways.) My question is "Why can't this GUI be decoupled from the BEAM emulator?" werl.exe could just as easily open a socket to communicate with an external application. That application could be written in C# or Delphi or even potentially Erlang itself using a GUI toolkit. The advantages are that all of the GUI code can be removed from werl.exe, plus it opens the door for GUI front-ends to be much nicer (different colors for input and output, for example). Additionally, the GUI could be modified without having to install a full cygwin suite. Is there an obvious reason why things can be structured this way? From ulf@REDACTED Thu Jul 14 17:14:59 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 14 Jul 2005 17:14:59 +0200 Subject: mnesia query In-Reply-To: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> References: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> Message-ID: Den 2005-07-14 15:49:53 skrev Sebastian Bello : > Hi all, > > imagine I have a query which returns a very big number of records, but > am interested in just a fixed small number of them. Is there a way to > retrieve, let's say, just the first N or just N of them? Not doing, ie > getting the whole result, implies a performance penalty other than > memory consumption? What type of query is it? Mnemosyne? QLC? or is it just select()? With select(), you can add an argument, Limit, that makes the function break out, and return the objects found so far, together with a continuation. This works for ets and mnesia, even though with mnesia, I think you also have to specify a bunch of other arguments. With QLC and Mnemosyne, you can use cursors. Read the appropriate manual to figure out how they work. In the Mnemosyne user guide, you can find the following example: "Use the cursor with a query evaluation to produce a few solutions only. With a handle we create a cursor by calling mnemosyne:cursor/1. With the cursor we can repeatedly call mnemosyne:next_answers to get more solutions. When an empty list is returned there are no more possible solutions. Delete the cursor with mnemosyne:delete_cursor/1. Handle = query [ S.snb || S <- table(subscriber), S.li = none] end, AFewAnswers = mnesia:transaction( fun() -> Cursor = mnemosyne:cursor(Handle), % now get at least two but not % more than five solutions: L = mnemosyne:next_answers(Cursor,2,5), mnemosyne:delete_cursor(Cursor), L end)" /Uffe From joe.armstrong@REDACTED Thu Jul 14 17:54:15 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Thu, 14 Jul 2005 17:54:15 +0200 Subject: Meyer, OO and concurrency Message-ID: todd wrote > You've got to be kidding. I said just the opposite several times. > > The contention that you can't do concurrency on an OO langauge like > C++ is just plain wrong. Oh - and there I was thinking that C++ didn't do concurrency at all - What most people perceive as "concurrency in C++" is a class library that provides a wrapper on top of the OS that the C++ system runs on. Threads/process in C++ are usually not C++ Threads or processes but thinly disguised OS threads or processes. The problem with this is that the semantics of the program is *amazingly* UNDEFINED. If the OS has non-premptive scheduling you get non-premptive scheduling, if the OS has priorities you get priorities, and guess what, you don't know. TWO parallel programs in C++ run on two different machines with two different OS's will interleave (at an instruction level) in a completely different way, which make debugging and reproducibility a nightmare. The same is true in java. Java threads are a thin interface to the OS's threads. << history - In the early Java days there were native and emulated threads - the native threads mapped to OS threads. Unfortunately the two threads had different semantics - native were faster and behaved differently on different machines. Which is why they chose them, believing performance to be more important than correctness ... >> In Erlang/Oz .. Processes are part of the language and NOT the OS. This means all parallel programs should behave the same way on all machines. (Actually if they run the same VM and have No external I/O they *should* behave in exactly the same way) This is a desirable property of a language. > > That you can do it better in other languages like Erlang is > unquestioned, > assuming a suitable definition of better. If better involves > low latency > then > erlang is not better. What does this mean? - what is the "it" in "do it better". What does "If better involves low latency then Erlang is not better." mean? I can think of a lot of definitions of latency - in some an Erlang implementation will be faster than a C implementation, in others it will be the other way around. > If better is related to the concurrency > model then > erlang is better. > > Partitioning a large embedded system into two different > language domains > is very unnatractive to many many people. Having been the one > to try and > sell this, I know. Call that the myopic application of the > hammer-nail > pattern, > if it pleases you. Partioning a large system (be it embedded or not) into different languages make a great deal of sense. In the embedded systems we build we choose the most suitable tools for the job. So we use Erlang for high level control, C for device drivers, Assembler for embedded devices, VHDL for chip designs, shell scripts for configuration, makefiles, yacc, bison, lex, ..... > Can I do concurrency in C++ well enough to keep a single language for > the lowest part of the system to the highest? Yes. In C++ - that's very difficult. The reason we invented Erlang was to try and solve some of the problems that crop up in making embedded systems with a long operation life - these problems seemed at the time to be intractable in C++. How do you make a system which in principle should run "forever"? - how do you change code in a running system? How do you run multiple versions of code in the same system (old and new code)? - how do you stop faults from propagating from one part of the system to another? How do you keep distributed systems consistent as you upgrade them? Erlang was an attempt to answer these questions - not perfect by any means, but a step in the right direction. As I pointed out earlier, in C++ concurrency is not *defined in the language* but is "inherited from the OS" - this means that the most basic of building blocks, the very mechanisms for running two programs in parallel on the same machine is not defined in the language and can in principle behave differently on two different OS's - > > > This is a group of users that has never been targeted by Erlang. > > That's the embedded world. It has the ultimate mix of high performance > code, mixed with scaling issues, mixed with low performance code. Embedded does not mean fast. It means put in a box and sold in a box with little possibility to change the functionality. > To > ask people to partition the system using a completely new > language they have > no experience with is not attractive enough. Everything you > need to do in > erlang you also have to be able to do on the C++ side as > well. If you could > make it so it could all be done in one language that would be > attractive. It would be attractive - but I think it is unrealistic. I use yacc/lex to write a parser/tokeniser, I write makefiles, shell scripts, These are all specialist languages which do specialist and well-defined jobs. Prolog is *wonderful* for data base queries and visual basic for a quick GUI. My biggest grumble is not "that I can't do everything in one language" but more that "All the nice tools don't work the same way on all OSs" and "interoperability between tools is non-existant". Windows visual basic etc. is great for GUIs - but only GUIs - windows applications are usually non-scriptable, Unix ones are, mixing them is impossible. I'd love to develop on one platform and deploy my code on all known OSs - "with zero changes" - this is just not possible, so I develop my code on the "least bad" OS I can find - and usually don't bother to even try and port it to the "even worse" OS. <> The great hope is a "component technology" but given the progress of the XML/WSDL/SOAP/XML-RPC bandwagon don't hold onto your hats. << flame on This is not directed at you :-) - It seems to me that peoples dislike of using multiple languages in one application is rooted in the horrendous problems of interfacing things written in different languages together. This is a crazy situation. If language X is best for solving problem X then I should use X and no other language. Same for Y. If I need an X and Y in the same system then it often turns out that it is painfully difficult to connect X to Y. So difficult that, that which should have been written in X is re-written in Y, or vice. versa, whichever is the easier. Scandalous. Language designers must think about interoperability. flame off >> /Joe From ulf@REDACTED Thu Jul 14 17:53:12 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 14 Jul 2005 17:53:12 +0200 Subject: Whorfian mind-lock? (Re: Concepts that people find hard) In-Reply-To: <42D668D7.9040108@desquilbet.org> References: <42D56474.40201@possibility.com> <1121328438.31308.37.camel@hymir.newport-networks.com> <42D668D7.9040108@desquilbet.org> Message-ID: Den 2005-07-14 15:29:59 skrev J?r?me Desquilbet : > Isn't it what is called a Whorfian mind-lock? ( > http://catb.org/%7Eesr/jargon/html/W/Whorfian-mind-lock.html ) > Some times ago, I tried to explore how the SWH can apply to software > engineering (using UML as an example, > http://jerome.desquilbet.org/lojban/rattle/#be ). > J?r?me. Bringing the topic back to concurrency, it's been argued by me and others several times that UML, as regards concurrency modeling, is broken, esp. for complex state machinery. There's a Whorfian Mind Lock for you. ;) /Uffe From sebastian@REDACTED Thu Jul 14 18:05:58 2005 From: sebastian@REDACTED (Sebastian Bello) Date: Thu, 14 Jul 2005 13:05:58 -0300 Subject: mnesia query References: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> Message-ID: <035701c5888d$ed3cacb0$3000a8c0@INSWITCH244> We are using select. Ok, I'll check for that. Three more questions (please tell me if they have already been aswered in the list): 1- what are the options to get the result ordered by the key (in this particular case)? 2- is it possible to perform a query using two indexes? 3- is there a guide from where to read about performance issues in mnesia? Thanks you for your complete response. Sebastian- ----- Original Message ----- From: "Ulf Wiger" To: Sent: Thursday, July 14, 2005 12:14 PM Subject: Re: mnesia query > Den 2005-07-14 15:49:53 skrev Sebastian Bello : > > > Hi all, > > > > imagine I have a query which returns a very big number of records, but > > am interested in just a fixed small number of them. Is there a way to > > retrieve, let's say, just the first N or just N of them? Not doing, ie > > getting the whole result, implies a performance penalty other than > > memory consumption? > > What type of query is it? Mnemosyne? QLC? or is it just select()? > With select(), you can add an argument, Limit, that makes the function > break out, and return the objects found so far, together with a > continuation. This works for ets and mnesia, even though with mnesia, > I think you also have to specify a bunch of other arguments. > > With QLC and Mnemosyne, you can use cursors. Read the appropriate > manual to figure out how they work. > > In the Mnemosyne user guide, you can find the following example: > > "Use the cursor with a query evaluation to produce a few solutions only. > With a handle we create a cursor by calling mnemosyne:cursor/1. With the > cursor we can repeatedly call mnemosyne:next_answers to get more solutions. > When an empty list is returned there are no more possible solutions. Delete > the cursor with mnemosyne:delete_cursor/1. > > Handle = > query > [ S.snb || S <- table(subscriber), > S.li = none] > end, > > AFewAnswers = > mnesia:transaction( > fun() -> > Cursor = mnemosyne:cursor(Handle), > % now get at least two but not > % more than five solutions: > L = mnemosyne:next_answers(Cursor,2,5), > mnemosyne:delete_cursor(Cursor), > L > end)" > > > > /Uffe > > From david.nospam.hopwood@REDACTED Thu Jul 14 18:23:42 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Thu, 14 Jul 2005 17:23:42 +0100 Subject: Meyer, OO and concurrency In-Reply-To: <1121157747.27930.33.camel@hymir.newport-networks.com> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> Message-ID: <42D6918E.8000705@blueyonder.co.uk> Peter-Henry Mander wrote: > Hi Gurus, > > Sorry but I've got to stick my oar in. > > I've found that modelling a OO system in Erlang recovers the original > concept of objects as concurrent actors. Almost, but: - Erlang does not garbage-collect processes when they are waiting unconditionally for a message that can never be sent. - Erlang Pids are forgeable, and it is possible to enumerate the Pids of all processes (which was prohibited in actor systems). - processes in Erlang implementations, although they are more lightweight than threads, are still a bit too heavyweight to be used in the same way as objects. These are mainly implementation rather than language design issues, although minor language/API changes would be needed to make Pids unforgeable. > I ask myself *why* has it taken soooo long to recognise the need for > concurrency? But then again, the OO paradigm took a *very* long time to > become mainstream. Perhaps it is only because computer science is such a young field, that we expect changes to be adopted quickly. In other fields, fundamental changes in approach do take decades to become adopted, and it isn't considered surprising. See . -- David Hopwood From ulf@REDACTED Thu Jul 14 18:47:18 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 14 Jul 2005 18:47:18 +0200 Subject: mnesia query In-Reply-To: <035701c5888d$ed3cacb0$3000a8c0@INSWITCH244> References: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> <035701c5888d$ed3cacb0$3000a8c0@INSWITCH244> Message-ID: Den 2005-07-14 18:05:58 skrev Sebastian Bello : > We are using select. Ok, I'll check for that. Three more questions > (please > tell me if they have already been aswered in the list): > > 1- what are the options to get the result ordered by the key (in this > particular case)? The result is going to be in whatever order that the table is in. If it's an ordered set, you will have a defined order, otherwise not. > 2- is it possible to perform a query using two indexes? I think so, if you use mnemosyne. > 3- is there a guide from where to read about performance issues in > mnesia? Don't think so. There may be some words in the user guide, but it's been a while since I read it through. In the reference manual, some of the functions are described with performance issues highlighted. /Uffe From Jouni.Ryno@REDACTED Thu Jul 14 18:55:30 2005 From: Jouni.Ryno@REDACTED (Jouni =?ISO-8859-1?Q?Ryn=F6?=) Date: Thu, 14 Jul 2005 19:55:30 +0300 Subject: Meyer, OO and concurrency In-Reply-To: References: Message-ID: <1121360130.6710.24.camel@adic> On Thu, 2005-07-14 at 17:54 +0200, Joe Armstrong (AL/EAB) wrote: > > To > > ask people to partition the system using a completely new > > language they have > > no experience with is not attractive enough. Everything you > > need to do in > > erlang you also have to be able to do on the C++ side as > > well. If you could > > make it so it could all be done in one language that would be > > attractive. > > It would be attractive - but I think it is unrealistic. > > I use yacc/lex to write a parser/tokeniser, I write makefiles, shell scripts, > > These are all specialist languages which do specialist and well-defined jobs. > > Prolog is *wonderful* for data base queries and visual basic for a quick GUI. > > My biggest grumble is not "that I can't do everything in one language" but > more that "All the nice tools don't work the same way on all OSs" and > "interoperability between tools is non-existant". > > It seems to me that peoples dislike of using multiple languages in one application > is rooted in the horrendous problems of interfacing things written in different languages together. > > This is a crazy situation. If language X is best for solving problem X then > I should use X and no other language. Same for Y. If I need an X and Y in the same system then it often turns out that it is painfully difficult to connect X to Y. > > So difficult that, that which should have been written in X is re-written in Y, > or vice. versa, whichever is the easier. Scandalous. > > Language designers must think about interoperability. Somewhere there has been an opinion, that managers hate multiple languages, as it's difficult to find programmes fluent in just the right combination. Although a good programmer should be able to do reasonable job in any language, I personally would hate to start using C++. And last spring I had to program a little with Ada. I was constantly fighting against the STRONG typing, trying to make shortcuts in C-style, void pointers, unions and such. I just could not make my mind up to it. Which is funny, as in OCCAM I found the need to make explicit number casting in arithmetic a good think. But that's the one side of embedded world. But I'm still all to the right language to right tool philosophy. Somehow it just has been come to only C for low level, Tcl/Tk for graphics and format manipulation. And Erlang for the rest. Interoperability via ASCII files. Still to find good enough reason to go for XML ... So maybe the managers are somewhat right. Anyway, although there are nice languages and libraries to make graphics (Tk, gtk+glade, python etc), I have even heard of people trying to do graphics with Erlang and X :) And someone even spending good summer days by making line plotting widget on top of ex11 instead of reading a good book ... it still needs a bit of work :( regards Jouni -- Jouni Ryn? mailto://Jouni.Ryno@REDACTED/ http://www.geo.fmi.fi/~ryno/ Finnish Meteorological Institute http://www.fmi.fi/ Space Research http://www.geo.fmi.fi/ P.O.BOX 503 Tel (+358)-9-19294656 FIN-00101 Helsinki FAX (+358)-9-19294603 Finland priv-GSM (+358)-50-5302903 "It's just zeros and ones, it cannot be hard" From sebastian@REDACTED Thu Jul 14 19:01:25 2005 From: sebastian@REDACTED (Sebastian Bello) Date: Thu, 14 Jul 2005 14:01:25 -0300 Subject: mnesia query References: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> <035701c5888d$ed3cacb0$3000a8c0@INSWITCH244> Message-ID: <036501c58895$ac331260$3000a8c0@INSWITCH244> Regarding 2: - and using select? - is there any drawback to using different query languages in an application to improve performance? Sebastian- ----- Original Message ----- From: "Ulf Wiger" To: Sent: Thursday, July 14, 2005 1:47 PM Subject: Re: mnesia query > Den 2005-07-14 18:05:58 skrev Sebastian Bello : > > > We are using select. Ok, I'll check for that. Three more questions > > (please > > tell me if they have already been aswered in the list): > > > > 1- what are the options to get the result ordered by the key (in this > > particular case)? > > The result is going to be in whatever order that the table > is in. If it's an ordered set, you will have a defined order, > otherwise not. > > > > 2- is it possible to perform a query using two indexes? > > I think so, if you use mnemosyne. > > > > 3- is there a guide from where to read about performance issues in > > mnesia? > > Don't think so. There may be some words in the user guide, > but it's been a while since I read it through. > In the reference manual, some of the functions are described > with performance issues highlighted. > > /Uffe > > From todd@REDACTED Thu Jul 14 19:10:58 2005 From: todd@REDACTED (todd) Date: Thu, 14 Jul 2005 10:10:58 -0700 Subject: Meyer, OO and concurrency In-Reply-To: References: Message-ID: <42D69CA2.6040905@possibility.com> Joe Armstrong (AL/EAB) wrote: >Oh - and there I was thinking that C++ didn't do concurrency at all - > >What most people perceive as "concurrency in C++" is a class library that provides >a wrapper on top of the OS that the C++ system runs on. > > Yep. And it fits in well with OO, which was the orignal point. >Threads/process in C++ are usually not C++ Threads or processes but thinly disguised OS >threads or processes. > > Indeed, unless you create a run to completion type event system that is multiplexed across different threads. >The problem with this is that the semantics of the program is *amazingly* UNDEFINED. > >If the OS has non-premptive scheduling you get non-premptive scheduling, if the OS has >priorities you get priorities, and guess what, you don't know. > > There are always issues. The plus is if I have multiple cores, they are also available. >The same is true in java. Java threads are a thin interface to the OS's threads. > > Depends on which version at which time. In the olden days we use to make "thread" packages using setjmp and longjmp too. They didn't always map to OS threads. >Unfortunately the two threads had different semantics - native were faster and >behaved differently on different machines. Which is why they chose them, believing >performance to be more important than correctness ... > > Native wasn't always faster either. Not by a long shot. > >In Erlang/Oz .. Processes are part of the language and NOT the OS. > >This means all parallel programs should behave the same way on all machines. >(Actually if they run the same VM and have No external I/O they *should* behave >in exactly the same way) > >This is a desirable property of a language. > > I am a pragmatic satisficer at heart. I try to make things work and keep them working while dreaming of a better world. Yet I manage not to disparage those who live with me, above me, or below me. > > >>That you can do it better in other languages like Erlang is >>unquestioned, >>assuming a suitable definition of better. If better involves >>low latency >>then >>erlang is not better. >> >> > >What does this mean? - what is the "it" in "do it better". > >What does "If better involves low latency then Erlang is not better." >mean? > >I can think of a lot of definitions of latency - in some an Erlang >implementation will be faster than a C implementation, in others it will be the other >way around. > > If you think latency is in the language then I am confused. It's a property of the OS when using OS services. My latency under VxWorks is most excellent. Run the same program on microshit and it stinks out loud. > >Partioning a large system (be it embedded or not) into different languages make a great deal of sense. > > It certainly can for your project. It may not for another project. >In the embedded systems we build we choose the most suitable tools for the job. > > For a suitable definition of suitable. >In C++ - that's very difficult. The reason we invented Erlang was to try and >solve some of the problems that crop up in making embedded systems with a >long operation life - these problems seemed at the time to be intractable in >C++. > Intractable, no. Hard and sucky, most certainly. > How do you make a system which in principle should run "forever"? - how >do you change code in a running system? > Use a system pair architecture. It also handles failover. For non redundant nodes provide a quick reboot with application level failover. >How do you run multiple versions of code >in the same system (old and new code)? > Don't. Upgrade the system as a whole. Between systems application protocols need to handle versions, as they need to anyway. > - how do you stop faults from propagating >from one part of the system to another? > Fail fast. Use a process architecture if you can. If you can't then protection patterns around memory blocks, use task specific memory allocators, use an asyc message passing architecture, use TDD, use extensive system tests, build in loads of pain in the ass diagnostics, etc, etc. > How do you keep distributed systems consistent >as you upgrade them? > > Use a system pair and conversion. Here's some stuff on the subject: http://www.possibility.com/epowiki/Wiki.jsp?page=EmbeddedUpgradeProposal >Erlang was an attempt to answer these questions - not perfect by any means, but a step >in the right direction. > > I have answered them imperfectly as well, though using stone knives and bear skins. >Embedded does not mean fast. It means put in a box and sold in a box with little possibility to change the functionality. > > It means a lot of things depending on your project. >It seems to me that peoples dislike of using multiple languages in one application >is rooted in the horrendous problems of interfacing things written in different languages together. > > No. It's about managing risk. You are asking people to use a language they know nothing about, that uses a completely different paradigm, for which they have no people experienced in the language, for which they have no idea what will happen if things go wrong, and provides facilities that they know can be done in C++ and does not require duplicate infrastructures. And you are asking them to do all this on their most important and critical of all projects. So it's not just about interfacing languages. It's serious business. It's the fate of a startup for example. The fate of a career at least. Don't trivialize what is involved. From ulf@REDACTED Thu Jul 14 19:23:47 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 14 Jul 2005 19:23:47 +0200 Subject: Meyer, OO and concurrency In-Reply-To: References: Message-ID: Den 2005-07-14 17:54:15 skrev Joe Armstrong (AL/EAB) : > > todd wrote > >> You've got to be kidding. I said just the opposite several times. >> >> The contention that you can't do concurrency on an OO langauge like >> C++ is just plain wrong. > > Oh - and there I was thinking that C++ didn't do concurrency at all - Well, in defense of Todd, he wrote that you can do concurrency in C++, not that C++ does concurrency. The projects I've seen that have tackled reasonably interesting concurrency problems in C++ have ended up writing their own scheduler, their own message-passing libraries, and their own error handling mechanisms. That's a lot of work, and quite often leads to "unexpected" delays in the project, but there's no question that you can do it. I've also had the nagging suspicion more than once that any objections that rolling your own concurrency support was going to be more difficult than they thought didn't work as a deterrant -- rather the opposite: a few die-hard C++ programmers sought the challenge, and really wanted to try writing their own concurrency middleware. /Uffe From csanto@REDACTED Thu Jul 14 19:51:01 2005 From: csanto@REDACTED (Corrado Santoro) Date: Thu, 14 Jul 2005 19:51:01 +0200 Subject: Meyer, OO and concurrency In-Reply-To: References: Message-ID: <42D6A605.3040005@diit.unict.it> The statements by Joe reported below suggest me to tell you my experience. Five years ago I had to realize a distributed process control system for a water distribution plant. I decided to use, as controllers, some embedded PCs with a small-sized Linux. Such controllers had to run periodic (and concurrent) control tasks that acquired some signals from the field, performed some computations and produced outputs for actuators. I started to write the control tasks in C++. A mess. Maybe I'm not a good C++ programmer, but I had to deal not only with task's aim but also with memory allocation problems, consistency in accessing shared structures, scheduling control, concurrency control, dangling pointers, etc. So I decided to change phylosophy and designed a new language, C-like, but specifically thought for distributed control systems. It was: - Task-oriented: the program were a collection of periodic and independent tasks modeled as finite-state machines; - Message-oriented: communication among tasks could be performed via message passing; - Distributed: message passing worked also among different machines; - Concurrent: tasks were executed concurrently and according to their periodicity, there were also the possibility of "shared variables" with atomic access (no need for explicit locks). I wrote the interpreter in C++. Once done, I realized that writing process control software was fast, bugs free, effective and let me to think only to the real problem I had to solve: to write some tasks that sample a field, perform computation and drive actuators. Nothing more. Last but not least, this system allowed me to operate directly on the target system, in order to patch the control software (the interpreter occupied about 200K): given that the target system had a solid-state hard disk of 32 MBytes, surely I couldn't install a GCC and do the same thing with C++. The system is still working now. Unfortunately, I didn't know Erlang, otherwise I didn't re-invent the wheel :-))) Ciao, --Corrado Joe Armstrong (AL/EAB) wrote: > todd wrote > > >>You've got to be kidding. I said just the opposite several times. >> >>The contention that you can't do concurrency on an OO langauge like >>C++ is just plain wrong. > > > Oh - and there I was thinking that C++ didn't do concurrency at all - > > What most people perceive as "concurrency in C++" is a class library that provides > a wrapper on top of the OS that the C++ system runs on. > > Threads/process in C++ are usually not C++ Threads or processes but thinly disguised OS > threads or processes. > > The problem with this is that the semantics of the program is *amazingly* UNDEFINED. > > If the OS has non-premptive scheduling you get non-premptive scheduling, if the OS has > priorities you get priorities, and guess what, you don't know. > > TWO parallel programs in C++ run on two different machines with two different OS's will interleave (at an instruction level) in a completely different way, which make debugging and reproducibility a nightmare. > > The same is true in java. Java threads are a thin interface to the OS's threads. > > > << history - In the early Java days there were native and emulated threads - the native threads mapped to OS threads. > > Unfortunately the two threads had different semantics - native were faster and > behaved differently on different machines. Which is why they chose them, believing > performance to be more important than correctness ... > > > > In Erlang/Oz .. Processes are part of the language and NOT the OS. > > This means all parallel programs should behave the same way on all machines. > (Actually if they run the same VM and have No external I/O they *should* behave > in exactly the same way) > > This is a desirable property of a language. > > >>That you can do it better in other languages like Erlang is >>unquestioned, >>assuming a suitable definition of better. If better involves >>low latency >>then >>erlang is not better. > > > What does this mean? - what is the "it" in "do it better". > > What does "If better involves low latency then Erlang is not better." > mean? > > I can think of a lot of definitions of latency - in some an Erlang > implementation will be faster than a C implementation, in others it will be the other > way around. > > >>If better is related to the concurrency >>model then >>erlang is better. >> >>Partitioning a large embedded system into two different >>language domains >>is very unnatractive to many many people. Having been the one >>to try and >>sell this, I know. Call that the myopic application of the >>hammer-nail >>pattern, >>if it pleases you. > > > Partioning a large system (be it embedded or not) into different languages make a great deal of sense. > > In the embedded systems we build we choose the most suitable tools for the job. > > So we use Erlang for high level control, C for device drivers, Assembler for > embedded devices, VHDL for chip designs, shell scripts for configuration, makefiles, > yacc, bison, lex, ..... > > >>Can I do concurrency in C++ well enough to keep a single language for >>the lowest part of the system to the highest? Yes. > > > In C++ - that's very difficult. The reason we invented Erlang was to try and > solve some of the problems that crop up in making embedded systems with a > long operation life - these problems seemed at the time to be intractable in > C++. How do you make a system which in principle should run "forever"? - how > do you change code in a running system? How do you run multiple versions of code > in the same system (old and new code)? - how do you stop faults from propagating > from one part of the system to another? How do you keep distributed systems consistent > as you upgrade them? > > Erlang was an attempt to answer these questions - not perfect by any means, but a step > in the right direction. > > As I pointed out earlier, in C++ concurrency is not *defined in the language* > but is "inherited from the OS" - this means that the most basic of building blocks, > the very mechanisms for running two programs in parallel on the same machine > is not defined in the language and can in principle behave differently on two different > OS's - > > >> > This is a group of users that has never been targeted by Erlang. >> >>That's the embedded world. It has the ultimate mix of high performance >>code, mixed with scaling issues, mixed with low performance code. > > > Embedded does not mean fast. It means put in a box and sold in a box with little possibility to change the functionality. > > >>To >>ask people to partition the system using a completely new >>language they have >>no experience with is not attractive enough. Everything you >>need to do in >>erlang you also have to be able to do on the C++ side as >>well. If you could >>make it so it could all be done in one language that would be >>attractive. > > > It would be attractive - but I think it is unrealistic. > > I use yacc/lex to write a parser/tokeniser, I write makefiles, shell scripts, > > These are all specialist languages which do specialist and well-defined jobs. > > Prolog is *wonderful* for data base queries and visual basic for a quick GUI. > > My biggest grumble is not "that I can't do everything in one language" but > more that "All the nice tools don't work the same way on all OSs" and > "interoperability between tools is non-existant". > > Windows visual basic etc. is great for GUIs - but only GUIs - windows applications > are usually non-scriptable, Unix ones are, mixing them is impossible. > > I'd love to develop on one platform and deploy my code on all known OSs - "with zero changes" - this is just not possible, so I develop my code on the "least bad" OS I can find - and usually don't bother to even try and port it to the "even worse" OS. > > <> > > The great hope is a "component technology" but given the progress of the > XML/WSDL/SOAP/XML-RPC bandwagon don't hold onto your hats. > > << flame on > > This is not directed at you :-) - > > It seems to me that peoples dislike of using multiple languages in one application > is rooted in the horrendous problems of interfacing things written in different languages together. > > This is a crazy situation. If language X is best for solving problem X then > I should use X and no other language. Same for Y. If I need an X and Y in the same system then it often turns out that it is painfully difficult to connect X to Y. > > So difficult that, that which should have been written in X is re-written in Y, > or vice. versa, whichever is the easier. Scandalous. > > Language designers must think about interoperability. > > flame off >> > > /Joe > > -- ====================================================== Eng. Corrado Santoro, Ph.D. University of Catania - Engineering Faculty Department of Computer Science and Telecommunications Engineering Viale A. Doria, 6 - 95125 CATANIA (ITALY) Tel: +39 095 7382144 Int. (5) 8035 +39 095 7382380 +39 095 7382365 +39 095 7382364 VoIP: sip:8035@REDACTED Fax: +39 095 7382397 EMail: csanto@REDACTED Personal Home Page: http://www.diit.unict.it/users/csanto NUXI Home Page: http://nuxi.iit.unict.it ====================================================== From david.nospam.hopwood@REDACTED Thu Jul 14 20:39:49 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Thu, 14 Jul 2005 19:39:49 +0100 Subject: Meyer, OO and concurrency In-Reply-To: <42D69CA2.6040905@possibility.com> References: <42D69CA2.6040905@possibility.com> Message-ID: <42D6B175.8040005@blueyonder.co.uk> todd wrote: > Joe Armstrong (AL/EAB) wrote: > >> What does "If better involves low latency then Erlang is not better." >> mean? >> >> I can think of a lot of definitions of latency - in some an Erlang >> implementation will be faster than a C implementation, in others it >> will be the other way around. >> > If you think latency is in the language then I am confused. It's a > property of the OS when using OS services. No, it's a property of any software component when using the services of that component. At the language level, the relevant questions are "What obstacles does the language put in the way of writing components with guaranteed low latency? Or with low expected latency?" Similar questions also apply to language implementations. -- David Hopwood From ulf@REDACTED Thu Jul 14 21:03:55 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 14 Jul 2005 21:03:55 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D6918E.8000705@blueyonder.co.uk> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> Message-ID: Den 2005-07-14 18:23:42 skrev David Hopwood : > Peter-Henry Mander wrote: > Hi Gurus, > Sorry but I've got to stick my oar in. > I've found that modelling a OO system in Erlang recovers the original > concept of objects as concurrent actors. > > Almost, but: > - Erlang does not garbage-collect processes when they are waiting > unconditionally for a message that can never be sent. This was discussed recently in comp.lang.functional and some other newsgroups. I think what came out of the discussion was that a process never becomes unreachable in Erlang, and thus, there is no way to determine that a process is waiting in vain. Besides, I believe that it's quite beneficial for debugging that a hanging process isn't automatically removed before anyone has a chance to inspect it. > - Erlang Pids are forgeable, and it is possible to enumerate > the Pids of all processes (which was prohibited in actor > systems). ... part of the reason why processes are always reachable. Also a great feature during debugging. > - processes in Erlang implementations, although they are more > lightweight than threads, are still a bit too heavyweight to be > used in the same way as objects. Agreed. This is one reason why mapping OOD to Erlang is not such a great idea. For different types of object, you need to choose different implementations in Erlang. Every type of object can't reasonably be mapped onto a process. /Uffe From todd@REDACTED Thu Jul 14 21:10:51 2005 From: todd@REDACTED (todd) Date: Thu, 14 Jul 2005 12:10:51 -0700 Subject: Meyer, OO and concurrency In-Reply-To: <42D6B175.8040005@blueyonder.co.uk> References: <42D69CA2.6040905@possibility.com> <42D6B175.8040005@blueyonder.co.uk> Message-ID: <42D6B8BB.8040801@possibility.com> David Hopwood wrote: > todd wrote: > >> Joe Armstrong (AL/EAB) wrote: >> >>> What does "If better involves low latency then Erlang is not better." >>> mean? >>> >>> I can think of a lot of definitions of latency - in some an Erlang >>> implementation will be faster than a C implementation, in others it >>> will be the other way around. >>> >> If you think latency is in the language then I am confused. It's a >> property of the OS when using OS services. > > > No, it's a property of any software component when using the services > of that component. At the language level, the relevant questions are > "What obstacles does the language put in the way of writing components > with guaranteed low latency? Or with low expected latency?" Similar > questions also apply to language implementations. I am blocking on an a semaphore which is an OS construct. Sombody unblocks me through an OS sempahore call. That schedules my task to run, in the OS. The OS runs my task. What's the language have to do with any of that? From david.nospam.hopwood@REDACTED Thu Jul 14 21:12:55 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Thu, 14 Jul 2005 20:12:55 +0100 Subject: Meyer, OO and concurrency In-Reply-To: <42D6B8BB.8040801@possibility.com> References: <42D69CA2.6040905@possibility.com> <42D6B175.8040005@blueyonder.co.uk> <42D6B8BB.8040801@possibility.com> Message-ID: <42D6B937.7020802@blueyonder.co.uk> todd wrote: > David Hopwood wrote: >> todd wrote: >>> Joe Armstrong (AL/EAB) wrote: >>> >>>> What does "If better involves low latency then Erlang is not better." >>>> mean? >>>> >>>> I can think of a lot of definitions of latency - in some an Erlang >>>> implementation will be faster than a C implementation, in others it >>>> will be the other way around. >>>> >>> If you think latency is in the language then I am confused. It's a >>> property of the OS when using OS services. >> >> No, it's a property of any software component when using the services >> of that component. At the language level, the relevant questions are >> "What obstacles does the language put in the way of writing components >> with guaranteed low latency? Or with low expected latency?" Similar >> questions also apply to language implementations. > > I am blocking on an a semaphore which is an OS construct. Sombody > unblocks me through an OS sempahore call. That schedules my task to run, > in the OS. The OS runs my task. What's the language have to do with any > of that? Nothing, *for this example*. But if the semaphore (or any other kind of concurrency construct) were provided by a language, then the language would have a lot to do with it. -- David Hopwood From todd@REDACTED Thu Jul 14 21:29:58 2005 From: todd@REDACTED (todd) Date: Thu, 14 Jul 2005 12:29:58 -0700 Subject: Meyer, OO and concurrency In-Reply-To: <42D6B937.7020802@blueyonder.co.uk> References: <42D69CA2.6040905@possibility.com> <42D6B175.8040005@blueyonder.co.uk> <42D6B8BB.8040801@possibility.com> <42D6B937.7020802@blueyonder.co.uk> Message-ID: <42D6BD36.1080807@possibility.com> David Hopwood wrote: > Nothing, *for this example*. But if the semaphore (or any other kind of > concurrency construct) were provided by a language, then the language > would > have a lot to do with it. If it were provided by the language how would a latency guarantee be possible given the language doesn't control the CPU? This one weak spot in erlang's process approach. BTW, anyone know if ruby will adopt erlang's approach? That would be interesting. From ulf@REDACTED Thu Jul 14 21:40:33 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 14 Jul 2005 21:40:33 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D69CA2.6040905@possibility.com> References: <42D69CA2.6040905@possibility.com> Message-ID: Den 2005-07-14 19:10:58 skrev todd : > No. It's about managing risk. You are asking people to use a language > they know nothing about, that uses a completely different paradigm, for > which they have no people experienced in the language, for which they > have no idea what will happen if things go wrong, and provides > facilities that they know can be done in C++ and does not require > duplicate infrastructures. I don't think that's what they're asked to do. Most of the time, what we ask people is to at least _study_ the issue. Quite often, Erlang is dismissed based on prejudice and, at most, a very cursory glance. If you're investing millions annually into developing complex software, why is it unreasonable to put some money into actually _investigating_ if new implementation techniques might shave off a portion of that, perhaps getting you to market faster, and allowing you to save money on maintenance as well? If people keep claiming that you can actually do this, don't you think it would be prudent to assign someone to actually look into it, perhaps with some prototypes? > And you are asking them to do all this on their most important > and critical of all projects. Mike Williams used to say: "if you don't conduct experiments at the beginning of your project, your whole project will become an experiment". Obviously, you shouldn't jump head-first into a technique that you haven't studied and experimented with, putting your whole company on the line. You experiment, you build prototypes, you live with the technology for a while and learn about its pros and cons. THEN, you decide to which extent you are willing to use it. > So it's not just about interfacing languages. It's serious business. So much more reason to be vigilant about researching new ways to become more efficient! > It's the fate of a startup for example. Startups usually have a hard time doing too much experimenting. Several startups use Erlang, though, because they learned it at the University or elsewhere, and use it as a competitive advantage against those who are too set in their ways to try new techniques. Other startups may use Lisp, Haskell or what have you. They don't have to live with big company politics and turf battles -- IMO the most important factor hindering the spread of niche technologies. > The fate of a career at least. No doubt. Many people pushing Erlang or similar technologies find out the hard way that it can be career-limiting to do so. > Don't trivialize what is involved. At some point, many of us have had to choose: keep pushing or take a step back and quit trying to make people see the light. This is not trivialising the issue (well, to some extent it is, but only relatively) -- it's more about protecting your own mental health. Fighting windmills is no fun, and tends to wear you down after a while. You pick something you believe in and run with it. I'd like to think that I can still migrate to something more exciting than Erlang, when that day comes, but it's not easy to know when you've become stuck in your ways. /Uffe From david.nospam.hopwood@REDACTED Thu Jul 14 21:46:37 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Thu, 14 Jul 2005 20:46:37 +0100 Subject: Meyer, OO and concurrency In-Reply-To: References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> Message-ID: <42D6C11D.6000007@blueyonder.co.uk> Ulf Wiger wrote: > Den 2005-07-14 18:23:42 skrev David Hopwood > : > >> Peter-Henry Mander wrote: >> Hi Gurus, >> Sorry but I've got to stick my oar in. >> I've found that modelling a OO system in Erlang recovers the original >> concept of objects as concurrent actors. >> >> Almost, but: >> - Erlang does not garbage-collect processes when they are waiting >> unconditionally for a message that can never be sent. > > This was discussed recently in comp.lang.functional and > some other newsgroups. Yes, starting at . > I think what came out of the discussion > was that a process never becomes unreachable in Erlang, and > thus, there is no way to determine that a process is waiting > in vain. Besides, I believe that it's quite beneficial for > debugging that a hanging process isn't automatically removed > before anyone has a chance to inspect it. > >> - Erlang Pids are forgeable, and it is possible to enumerate >> the Pids of all processes (which was prohibited in actor >> systems). > > ... part of the reason why processes are always reachable. That's why the two issues above would have to be addressed at the same time. > Also a great feature during debugging. By default, debugging should not cause objects/processes to remain alive if they would not otherwise do so. The debugger should hold only weak references, unless the user specifically asks for some objects/ processes to be strongly referenced. The fact that normal code cannot obtain a list of all processes doesn't prevent a debugger from being able to do so; conceptually, the debugger is part of the language implementation. Note that a debugger necessarily has to be able to handle cases where a value being watched becomes inaccessible, so doing so for processes as well as other values does not add much complexity to the debugger or its user interface. >> - processes in Erlang implementations, although they are more >> lightweight than threads, are still a bit too heavyweight to be >> used in the same way as objects. > > Agreed. This is one reason why mapping OOD to Erlang is not > such a great idea. For different types of object, you need to > choose different implementations in Erlang. Every type of > object can't reasonably be mapped onto a process. I believe that it is quite feasible to implement Erlang (and similar asynchronous message passing languages) in such a way that every type of object can be reasonably mapped onto a process. I'll post a pointer to more details about that soon. -- David Hopwood From todd@REDACTED Thu Jul 14 23:00:14 2005 From: todd@REDACTED (todd) Date: Thu, 14 Jul 2005 14:00:14 -0700 Subject: Meyer, OO and concurrency In-Reply-To: References: <42D69CA2.6040905@possibility.com> Message-ID: <42D6D25E.7020703@possibility.com> Ulf Wiger wrote: > Den 2005-07-14 19:10:58 skrev todd : > >> No. It's about managing risk. You are asking people to use a >> language they know nothing about, that uses a completely different >> paradigm, for which they have no people experienced in the language, >> for which they have no idea what will happen if things go wrong, >> and provides facilities that they know can be done in C++ and does >> not require duplicate infrastructures. > > > I don't think that's what they're asked to do. > Most of the time, what we ask people is to at > least _study_ the issue. Quite often, Erlang is dismissed > based on prejudice and, at most, a very cursory glance. At a telecom company I worked at we studied the issue. I was the erlang proponent. It may be that people do study, but don't come to the conclusion you think obvious because they value something different than you do. > > If you're investing millions annually into developing complex > software, why is it unreasonable to put some money into actually > _investigating_ if new implementation techniques might shave off > a portion of that, perhaps getting you to market faster, and > allowing you to save money on maintenance as well? Perhaps is the big word. Unless it's a clear win people want to stick with what they know will work. > > Mike Williams used to say: "if you don't conduct experiments > at the beginning of your project, your whole project will > become an experiment". Every project is an experiment. Which is why people want to stick with what has worked in the past. Conducting a little experiment won't change the weight of experience. The interesting book, paradox of choice, talks about how people are risk adverse and would rather take a smaller win now than a potentially bigger but less sure win later. >> Don't trivialize what is involved. > > > At some point, many of us have had to choose: keep pushing > or take a step back and quit trying to make people see the > light. This is not trivialising the issue (well, to some extent > it is, but only relatively) When you say the reason people don't want to use erlang is simply because they don't want to use multiple languages, that is trivializing the issue. From ulf@REDACTED Thu Jul 14 23:03:03 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 14 Jul 2005 23:03:03 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D6C11D.6000007@blueyonder.co.uk> References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> <42D6C11D.6000007@blueyonder.co.uk> Message-ID: Den 2005-07-14 21:46:37 skrev David Hopwood : > By default, debugging should not cause objects/processes to remain > alive if they would not otherwise do so. The debugger should hold only > weak references, unless the user specifically asks for some objects/ > processes to be strongly referenced. The fact that normal code cannot > obtain a list of all processes doesn't prevent a debugger from being > able to do so; conceptually, the debugger is part of the language > implementation. But in this case, hanging processes can be debugged (even though it's usually not easy to figure out what they're waiting for), even if you didn't expect them to hang, and didn't have the debugger active. I really see this as a non-problem. Erlang processes _are_ automatically GC'd when they terminate. If they haven't terminated, they remain in memory. One could imagine a specification stating that a process waiting indefinitely should be automatically removed, but I have never considered it a problem that Erlang doesn't do so. In Erlang, a process waiting indefinitely for something that will never occur is probably (but not certainly) in a faulty state. It is very difficult to determine that a message can never be sent. Another process may be at fault, and the problem may be fixed by changing and reloading a module in some other part of the system. /Uffe From ulf@REDACTED Thu Jul 14 23:36:07 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 14 Jul 2005 23:36:07 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D6D25E.7020703@possibility.com> References: <42D69CA2.6040905@possibility.com> <42D6D25E.7020703@possibility.com> Message-ID: Den 2005-07-14 23:00:14 skrev todd : > Ulf Wiger wrote: > >> I don't think that's what they're asked to do. >> Most of the time, what we ask people is to at >> least _study_ the issue. Quite often, Erlang is dismissed >> based on prejudice and, at most, a very cursory glance. > > At a telecom company I worked at we studied the issue. > I was the erlang proponent. > It may be that people do study, but don't come to the conclusion > you think obvious because they value something different than you do. It may be. Certainly some people will have studied the issue, and not all will reach the same conclusion. If your OS is VxWorks, for example, and you feel that using semaphonres is a good way to solve your problem, then there are loads of reasons why Erlang might not be a good choice. We have actively discouraged people from using Erlang on VxWorks in the past. I guess VxWorks supports memory protection now, so combining C and Erlang on VxWorks is perhaps no longer the disaster it used to be. >> If you're investing millions annually into developing complex >> software, why is it unreasonable to put some money into actually >> _investigating_ if new implementation techniques might shave off >> a portion of that, perhaps getting you to market faster, and >> allowing you to save money on maintenance as well? > > Perhaps is the big word. Unless it's a clear win people want to stick > with what they know will work. Except some 7 out of 10 large software projects fail (or even more, depending on which study you want to believe in most) > >> >> Mike Williams used to say: "if you don't conduct experiments >> at the beginning of your project, your whole project will >> become an experiment". > > Every project is an experiment. Which is why people want to > stick with what has worked in the past. Conducting a little > experiment won't change the weight of experience. The interesting book, > paradox of > choice, talks about how people are risk adverse and would rather > take a smaller win now than a potentially bigger but less sure win > later. > >>> Don't trivialize what is involved. >> >> >> At some point, many of us have had to choose: keep pushing >> or take a step back and quit trying to make people see the >> light. This is not trivialising the issue (well, to some extent >> it is, but only relatively) > > When you say the reason people don't want to use erlang is simply > because they don't want to use multiple languages, > that is trivializing the issue. But no one suggested that. Joe suggested that one important reason why people dislike mixing languages is that many language environments don't mix well, and it's very difficult to get them to interoperate. There are several domains where Erlang isn't a contender at all. There are some domains where Erlang might be a good alternative, but it's not obviously better than other, more established approaches. Then there are domains where Erlang is so much better than e.g. C++ that practically all who give it a chance find it hard to believe the grudgery they used to put themselves through. Even in these domains, many people will refuse to give it a chance. Some will claim that they do this on good grounds, and talk portentously about business cases and risk reduction. Some will fight it for political gain, or perhaps because they happen to dislike a person who is much in favour of it. Then again, some people will simply prioritize differently and/or disagree that Erlang would be so much better. (If I had to guess, I'd place you in the middle category. Many of the arguments in this thread have been about the latter one.) Still, with very few exceptions, people are not malicious and honestly try to make the best choices in each situation. The fact that people given the same data can still draw completely opposite conclusions is both stimulating and good for our evolution, I think. (: /Uffe From vances@REDACTED Thu Jul 14 23:41:05 2005 From: vances@REDACTED (Vance Shipley) Date: Thu, 14 Jul 2005 17:41:05 -0400 Subject: Meyer, OO and concurrency In-Reply-To: References: <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> <42D6C11D.6000007@blueyonder.co.uk> Message-ID: <20050714214105.GJ644@feeble.motivity.ca> One can easily imagine an error handler which is simply never called into duty. Distinguishing that from a process which can't ever receive what it is waiting for would be impossible if you're allowed dynamic code loading. -Vance On Thu, Jul 14, 2005 at 11:03:03PM +0200, Ulf Wiger wrote: } } In Erlang, a process waiting indefinitely for something that } will never occur is probably (but not certainly) in a faulty } state. It is very difficult to determine that a message can } never be sent. Another process may be at fault, and the problem } may be fixed by changing and reloading a module in some other } part of the system. From ulf@REDACTED Fri Jul 15 00:19:08 2005 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 15 Jul 2005 00:19:08 +0200 Subject: mnesia query In-Reply-To: <036501c58895$ac331260$3000a8c0@INSWITCH244> References: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> <035701c5888d$ed3cacb0$3000a8c0@INSWITCH244> <036501c58895$ac331260$3000a8c0@INSWITCH244> Message-ID: Den 2005-07-14 19:01:25 skrev Sebastian Bello : > Regarding 2: > > - and using select? Select doesn't use indexes. mnesia:index_match_object() does. > - is there any drawback to using different query languages in an > application to improve performance? Not really, I think. /Uffe From david.nospam.hopwood@REDACTED Fri Jul 15 01:03:24 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Fri, 15 Jul 2005 00:03:24 +0100 Subject: Meyer, OO and concurrency In-Reply-To: References: <20050711154603.65257.qmail@web34411.mail.mud.yahoo.com> <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> <42D6C11D.6000007@blueyonder.co.uk> Message-ID: <42D6EF3C.7020202@blueyonder.co.uk> Ulf Wiger wrote: > David Hopwood wrote: > >> By default, debugging should not cause objects/processes to remain >> alive if they would not otherwise do so. The debugger should hold only >> weak references, unless the user specifically asks for some objects/ >> processes to be strongly referenced. The fact that normal code cannot >> obtain a list of all processes doesn't prevent a debugger from being >> able to do so; conceptually, the debugger is part of the language >> implementation. > > But in this case, hanging processes can be debugged (even though > it's usually not easy to figure out what they're waiting for), > even if you didn't expect them to hang, and didn't have the > debugger active. In actors-style OO, a "hanging" process is not an error; it is quite expected to have dead actors that can no longer receive any messages -- just as common as it is to have dead objects in any other OO language. Note that processes that are intended to be continuously active until explicitly terminated will typically have timeouts on message reception, so they will not be GC'd. > I really see this as a non-problem. Erlang processes _are_ > automatically GC'd when they terminate. If they haven't > terminated, they remain in memory. One could imagine a > specification stating that a process waiting indefinitely > should be automatically removed, but I have never considered > it a problem that Erlang doesn't do so. It is not a problem if you don't attempt to do actors-style OO in Erlang. But the context was exactly that: Peter-Henry Mander wrote: # I've found that modelling a OO system in Erlang recovers the original # concept of objects as concurrent actors. I pointed out that one of the important differences is in the semantics of GC. In the comp.lang.functional thread, the context was whether Erlang's processes were equivalent in expressive power to SML references; they are not for the same reason. What is rather frustrating is that Erlang is *almost* an actor language, but doesn't quite make the grade because of issues like this one. There is a community of potential users (including the capability-based security community) who would love to use Erlang as an actor language, but without these changes, that just isn't going to work. It would be a bit like trying to do OO in a non-GC'd language, or trying to use OS threads as if they were lightweight processes. > In Erlang, a process waiting indefinitely for something that > will never occur is probably (but not certainly) in a faulty > state. Well, this is simply not true of actor systems. So it can only be true of Erlang systems that are not designed like actor systems. > It is very difficult to determine that a message can > never be sent. Another process may be at fault, and the problem > may be fixed by changing and reloading a module in some other > part of the system. If an object is not reachable and is in an infinite wait, then it can be collected completely independently of any other code, and regardless of whether other code can be reloaded. -- David Hopwood From david.nospam.hopwood@REDACTED Fri Jul 15 01:44:14 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Fri, 15 Jul 2005 00:44:14 +0100 Subject: Meyer, OO and concurrency In-Reply-To: <20050714214105.GJ644@feeble.motivity.ca> References: <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> <42D6C11D.6000007@blueyonder.co.uk> <20050714214105.GJ644@feeble.motivity.ca> Message-ID: <42D6F8CE.40701@blueyonder.co.uk> Vance Shipley wrote: > On Thu, Jul 14, 2005 at 11:03:03PM +0200, Ulf Wiger wrote: > } > } In Erlang, a process waiting indefinitely for something that > } will never occur is probably (but not certainly) in a faulty > } state. It is very difficult to determine that a message can > } never be sent. Another process may be at fault, and the problem > } may be fixed by changing and reloading a module in some other > } part of the system. > > One can easily imagine an error handler which is simply never > called into duty. Distinguishing that from a process which > can't ever receive what it is waiting for would be impossible > if you're allowed dynamic code loading. I think there may be some misunderstandings in this thread about what process GC in actor systems is intended to achieve. As for any kind of value in any GC algorithm, processes are only collected based on a conservative approximation of whether they can affect the future behaviour of the program. It is quite possible to have processes that cannot do anything useful but that are still reachable; these are not collected. But this is really no different from any other kind of memory leak that can occur despite the use of precise GC, and is in practice easily avoidable. The point of the definition of GC in actor systems is that it allows you to use actors/processes as objects, in much the same way as you would do in a conventional OO language, but in a message passing concurrency model. In fact, an actor can represent any point on a continuum of passivity -- reactivity -- autonomy, from completely passive, stateless values, to thread-like processes that run continuously and that communicate and cause effects autonomously. The conceptual coherence of actor systems lies to a great extent in removing *artificial* distinctions between these kinds of object. Dynamic code loading makes essentially no difference to the design of GC (any more than it does for conventional OO languages with dynamic code loading, such as Java). -- David Hopwood From david.nospam.hopwood@REDACTED Fri Jul 15 02:14:16 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Fri, 15 Jul 2005 01:14:16 +0100 Subject: Meyer, OO and concurrency In-Reply-To: <42D6BD36.1080807@possibility.com> References: <42D69CA2.6040905@possibility.com> <42D6B175.8040005@blueyonder.co.uk> <42D6B8BB.8040801@possibility.com> <42D6B937.7020802@blueyonder.co.uk> <42D6BD36.1080807@possibility.com> Message-ID: <42D6FFD8.4020701@blueyonder.co.uk> todd wrote: > David Hopwood wrote: > >> Nothing, *for this example*. But if the semaphore (or any other kind of >> concurrency construct) were provided by a language, then the language >> would have a lot to do with it. > > If it were provided by the language how would a latency guarantee be > possible given the language doesn't control the CPU? By implementing the semaphore (in this example) entirely at user level. In general, problems in supporting real-time guarantees (latency, fairness, etc.) can be due to the OS definition, OS implementation, language definition, or language implementation. But some of these problems can be worked around, or the problematic features can be avoided. If the language works around or avoids problems with the OS, then a programmer does not need to be concerned with those problems at all. For example, suppose that an OS provides adequate support for real-time scheduling and for pinning memory pages (and therefore is able to run selected user-level code with low latency), but all of its synchronization abstractions are broken in respect of latency guarantees. Never mind; it is possible to implement synchronization at user level, by making use of atomic operations and/or memory barriers supported by the processor architecture, bypassing the OS. Or, the language design and implementation may make it unnecessary to use explicit synchronization at all. Of course, if a language implementation is running on an OS that is hopelessly broken in respect of real-time issues, and does not allow running *any* code with low latency, then the implementation will also be hopelessly broken in that respect. This is just as true of C++ as it is of Erlang. -- David Hopwood From ulf@REDACTED Fri Jul 15 03:21:48 2005 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 15 Jul 2005 03:21:48 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D6F8CE.40701@blueyonder.co.uk> References: <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> <42D6C11D.6000007@blueyonder.co.uk> <20050714214105.GJ644@feeble.motivity.ca> <42D6F8CE.40701@blueyonder.co.uk> Message-ID: Den 2005-07-15 01:44:14 skrev David Hopwood : > I think there may be some misunderstandings in this thread about what > process GC in actor systems is intended to achieve. As for any kind > of value in any GC algorithm, processes are only collected based on > a conservative approximation of whether they can affect the future > behaviour of the program. It is quite possible to have processes that > cannot do anything useful but that are still reachable; these are not > collected. But this is really no different from any other kind of > memory leak that can occur despite the use of precise GC, and is in > practice easily avoidable. Since you're talking about the actor model specifically, and state that some issues are only problems if you try to implement OO in the actor model, I thought I'd try to read about the actor model. I found this page: http://www.agent.ai/main.php?folderID=221 I read "Towards a Theory of Actor Computation" by Agha et al. The hour is late, so perhaps my reading was too sloppy. I didn't really see how the message reception semantics were defined. The paper talks about data channels (which are not present in Erlang), and specifically disallows passing lambda expressions in a message to an actor. This is allowed in Erlang. From my first read, I assume that the reception semantics are FIFO, event-based, much like in CSP, and not selective receive a la Erlang. Also, it appears as if computation paths triggered by messages are concurrent, which would imply spawning another process for each message (or simply ignoring this, and serializing everything within the actor/process.) To me, it sounds like you would need a control layer in order to do actor programming in Erlang. Perhaps you'd also then want to use explicit channels? Essentially, you'd write a framework for actor programming. If so, messages can be forced to traverse a controlled channel, and are accepted only if they have the proper format and arrive through the proper channels. This would also open up possibilities for reachability analysis. True, all this could be subverted. This is the problem of adding a layer to a langugage that only almost conforms to the wanted semantics. /Uffe From ulf@REDACTED Fri Jul 15 03:40:25 2005 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 15 Jul 2005 03:40:25 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D6B8BB.8040801@possibility.com> References: <42D69CA2.6040905@possibility.com> <42D6B175.8040005@blueyonder.co.uk> <42D6B8BB.8040801@possibility.com> Message-ID: Den 2005-07-14 21:10:51 skrev todd : > I am blocking on an a semaphore which is an OS construct. Sombody > unblocks me through an OS sempahore call. That schedules my task to run, > in the OS. The OS runs my task. What's the language have to do with any > of that? Just so we understand each other: at our project we do this kind of stuff in device processors, using C in OSE Delta. It could be that we'll eventually take advantage of multi-core architectures, and move the whole thing up into Erlang, using simple drivers written in C to control the hardware. If we can have e.g. 8 CPU cores at 1-2 GHz each, I think latency is going to be OK for our purposes, and the main reason to make this move would be that the Erlang-based stuff is simply so much easier to write and debug. We're not there yet (partly because the hardware necessary is not there yet), but it's certainly something we're considering. The semaphore programming you describe is also something we've had experience with, but it's not the way you do things in OSE Delta. OSE has processes and asynchronous message passing, similar to Erlang in fact. For the stuff we do in Erlang, using semaphores is out of the question, and so is an event-based architecture, since it would make our state machines much more difficult to program. /Uffe From todd@REDACTED Fri Jul 15 05:28:14 2005 From: todd@REDACTED (todd) Date: Thu, 14 Jul 2005 20:28:14 -0700 Subject: Meyer, OO and concurrency In-Reply-To: <42D6FFD8.4020701@blueyonder.co.uk> References: <42D69CA2.6040905@possibility.com> <42D6B175.8040005@blueyonder.co.uk> <42D6B8BB.8040801@possibility.com> <42D6B937.7020802@blueyonder.co.uk> <42D6BD36.1080807@possibility.com> <42D6FFD8.4020701@blueyonder.co.uk> Message-ID: <42D72D4E.4060807@possibility.com> David Hopwood wrote: > Of course, if a language implementation is running on an OS that is > hopelessly broken in respect of real-time issues, and does not allow > running *any* code with low latency, then the implementation will also > be hopelessly broken in that respect. This is just as true of C++ as it > is of Erlang. It's not difficult. Interrupt. How long till the task runs? The OS can guarantee that. A language can't. From valentin@REDACTED Fri Jul 15 08:44:42 2005 From: valentin@REDACTED (Valentin Micic) Date: Fri, 15 Jul 2005 08:44:42 +0200 Subject: Meyer, OO and concurrency References: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> Message-ID: <00de01c58908$b01b9af0$0100a8c0@MONEYMAKER2> From: "Ulf Wiger" I've argued before that I think process dictionaries have received a bad reputation unjustly. They have the same basic semantics as private ets tables, but have some nice advantages besides the obvious advantage of convenience. * * * Now that you've mention it: because of this *reputation*, I've been avoiding dictionary (BIFs put & get), for a long time. Recently, I did some ad-hoc measurement, and noticed that dictionary migt be considerably faster in accessing data than ets -- possibly because data from dictionary are passed by reference, wheres ets has to reply with message. What are the limitations of dictionary, and *why* it received a bad reputation? Valentin. From ulf@REDACTED Fri Jul 15 10:01:56 2005 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 15 Jul 2005 10:01:56 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <00de01c58908$b01b9af0$0100a8c0@MONEYMAKER2> References: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> <00de01c58908$b01b9af0$0100a8c0@MONEYMAKER2> Message-ID: Den 2005-07-15 08:44:42 skrev Valentin Micic: > because of this *reputation*, I've been avoiding dictionary > (BIFs put & get), for a long time. Recently, I did some > ad-hoc measurement, and noticed that dictionary migt be > considerably faster in accessing data than ets -- possibly > because data from dictionary are passed by reference, wheres ets > has to reply with message. What are the limitations of dictionary, > and *why* it received a bad reputation? I think it was frowned upon as early as in the Erlang book (chapter 8.6, where its use is discouraged), since it breaks referential transparency (it introduces side effects in your functions.) This is a fair warning, since it's tempting to use process dictionaries as a means to get global variables. I guess what's missing is a really good treatment of side effects, and how to manage them properly. As side effects, you should at least count message passing, spawn, link & monitor, the register BIFs, the port BIFs, the process dictionary, ets, dets file, disk_log, mnesia. All of them can be viewed as based on the basic process concept and message passing. And just about everything that's said to be bad about the process dictionary can be applied to message passing in general. The Erlang book never warns the user to take care with messages in the same fashion, and certainly doesn't discourage the use of them. (: /Uffe From matthias@REDACTED Fri Jul 15 11:09:36 2005 From: matthias@REDACTED (Matthias Lang) Date: Fri, 15 Jul 2005 11:09:36 +0200 Subject: why does the process dictionary have a bad rep? In-Reply-To: <00de01c58908$b01b9af0$0100a8c0@MONEYMAKER2> References: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> <00de01c58908$b01b9af0$0100a8c0@MONEYMAKER2> Message-ID: <17111.32080.293077.861066@antilipe.corelatus.se> Valentin Micic writes: [about the process dictionary] > has to reply with message. What are the limitations of dictionary, and *why* > it received a bad reputation? In addition to the points already made, another reason is that a significant proportion of those who haven't seen single-assignment before will seize upon anything which "finally lets me assign to a damn variable like I want to", and the first 'anything' they usually encounter is the process dictionary. (ab)using processes and message passing to achieve the same effect at least has the side effect of teaching them about message passing. Matthias From ulf@REDACTED Fri Jul 15 12:14:04 2005 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 15 Jul 2005 12:14:04 +0200 Subject: why does the process dictionary have a bad rep? In-Reply-To: <17111.32080.293077.861066@antilipe.corelatus.se> References: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> <00de01c58908$b01b9af0$0100a8c0@MONEYMAKER2> <17111.32080.293077.861066@antilipe.corelatus.se> Message-ID: Den 2005-07-15 11:09:36 skrev Matthias Lang : > Valentin Micic writes: > > [about the process dictionary] > > has to reply with message. What are the limitations of dictionary, > and *why* > > it received a bad reputation? > > In addition to the points already made, another reason is that a > significant proportion of those who haven't seen single-assignment > before will seize upon anything which "finally lets me assign to a > damn variable like I want to", and the first 'anything' they usually > encounter is the process dictionary. > > (ab)using processes and message passing to achieve the same effect at > least has the side effect of teaching them about message passing. Agreed. The problem isn't that we shouldn't caution about use of the process dictionary. We should do so in a way that people get a chance to learn why it should be used with care, and what else should be used with care. I have yet to find the chapter that explains this. Erlang has a pragmatic approach to message passing (no monads, etc.) This is wonderful in many ways, but it also introduces some juicy opportunities to write terrible code. /Uffe From nm@REDACTED Fri Jul 15 16:25:30 2005 From: nm@REDACTED (Gaspar Chilingarov) Date: Fri, 15 Jul 2005 19:25:30 +0500 Subject: jungerl/zlib changes Message-ID: <42D7C75A.9020208@web.am> Hi all! I suggest the following changes to zlib module distributed in jungerl - --cut-- Index: zlib.erl =================================================================== RCS file: /cvsroot/jungerl/jungerl/lib/zlib/src/zlib.erl,v retrieving revision 1.2 diff -c -r1.2 zlib.erl *** zlib.erl 18 Feb 2004 11:35:29 -0000 1.2 --- zlib.erl 15 Jul 2005 13:26:57 -0000 *************** *** 105,116 **** %% open a z_stream open() -> ! ok = load(), open_port({spawn, zlib_drv}, [binary]). %% close and release z_stream close(Z) -> ! port_close(Z). deflateInit(Z, Level) -> call(Z, ?DEFLATE_INIT, <<(arg_level(Level)):32>>). --- 105,121 ---- %% open a z_stream open() -> ! % ok = load(), %% it is loaded automagically open_port({spawn, zlib_drv}, [binary]). %% close and release z_stream close(Z) -> ! port_close(Z), ! receive ! { 'EXIT', Z, normal } -> ok ! after 0 -> ! ok ! end. deflateInit(Z, Level) -> call(Z, ?DEFLATE_INIT, <<(arg_level(Level)):32>>). --cut-- first of all - imho open_port automatically loads external driver, and if I leave it load() call, it will fail for subsequent calls to zlib:compress or zlib:uncompress - because module is already loaded. seconds change - is to have receive command , which will cleanup port exit message from process queue, if such message exists. -- Gaspar Chilingarov System Administrator t +37491 419763 (mob) t +37410 240399 (office) w www.web.am i 63174784 e nm@REDACTED From ke.han@REDACTED Fri Jul 15 17:18:22 2005 From: ke.han@REDACTED (ke.han) Date: Fri, 15 Jul 2005 23:18:22 +0800 Subject: why does the process dictionary have a bad rep? In-Reply-To: References: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> <00de01c58908$b01b9af0$0100a8c0@MONEYMAKER2> <17111.32080.293077.861066@antilipe.corelatus.se> Message-ID: <42D7D3BE.5080603@redstarling.com> This thread is talking about two things: abuse of the proc dict and abuse of message passing. I understand the example of using the proc dict to get around single assignment. You guys have lost me on the second part. Can someone give an example of message passing abuse? What does message passing have to do with variable assignment? thanks, ke han Ulf Wiger wrote: > Den 2005-07-15 11:09:36 skrev Matthias Lang : > >> Valentin Micic writes: >> >> [about the process dictionary] >> > has to reply with message. What are the limitations of dictionary, >> and *why* >> > it received a bad reputation? >> >> In addition to the points already made, another reason is that a >> significant proportion of those who haven't seen single-assignment >> before will seize upon anything which "finally lets me assign to a >> damn variable like I want to", and the first 'anything' they usually >> encounter is the process dictionary. >> >> (ab)using processes and message passing to achieve the same effect at >> least has the side effect of teaching them about message passing. > > > > Agreed. The problem isn't that we shouldn't caution about use > of the process dictionary. We should do so in a way that people > get a chance to learn why it should be used with care, and what > else should be used with care. > > I have yet to find the chapter that explains this. > Erlang has a pragmatic approach to message passing (no monads, > etc.) This is wonderful in many ways, but it also introduces > some juicy opportunities to write terrible code. > > /Uffe > > From ulf@REDACTED Fri Jul 15 17:43:33 2005 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 15 Jul 2005 17:43:33 +0200 Subject: why does the process dictionary have a bad rep? In-Reply-To: <42D7D3BE.5080603@redstarling.com> References: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> <00de01c58908$b01b9af0$0100a8c0@MONEYMAKER2> <17111.32080.293077.861066@antilipe.corelatus.se> <42D7D3BE.5080603@redstarling.com> Message-ID: Den 2005-07-15 17:18:22 skrev ke.han : > This thread is talking about two things: abuse of the proc dict and > abuse of message passing. > I understand the example of using the proc dict to get around single > assignment. You guys have lost me on the second part. > Can someone give an example of message passing abuse? What does message > passing have to do with variable assignment? > thanks, ke han The things listed in the Erlang book about the process dictionary apply equally to message passing. - if a computation is rolled back by a catch, messages sent during that computation cannot be rolled back - if functions send messages (have side effects), the order in which functions are called can become very important. - understanding a single function is much easier if it only depends on the input arguments, and not on message passing inside the function. Without side-effects, you know that the function will always return the same result each time it's called with a given set of arguments - regardless of timing issues. - verifying code that doesn't have side-effects is also much easier, as the state space is radically reduced. This is related to the previous point. In general, it's a very good idea to isolate all side-effects as much as possible, and to keep them clearly separated from the code that can be side-effect free. /Uffe From sean.hinde@REDACTED Fri Jul 15 17:52:14 2005 From: sean.hinde@REDACTED (Sean Hinde) Date: Fri, 15 Jul 2005 16:52:14 +0100 Subject: why does the process dictionary have a bad rep? In-Reply-To: <42D7D3BE.5080603@redstarling.com> References: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> <00de01c58908$b01b9af0$0100a8c0@MONEYMAKER2> <17111.32080.293077.861066@antilipe.corelatus.se> <42D7D3BE.5080603@redstarling.com> Message-ID: It is possible to model a global variable using a named process. A set and get synchronous call (i.e. message passing) API would update the state of the named process. Sean On 15 Jul 2005, at 16:18, ke.han wrote: > This thread is talking about two things: abuse of the proc dict and > abuse of message passing. > I understand the example of using the proc dict to get around > single assignment. You guys have lost me on the second part. > Can someone give an example of message passing abuse? What does > message passing have to do with variable assignment? > thanks, ke han > > > Ulf Wiger wrote: > >> Den 2005-07-15 11:09:36 skrev Matthias Lang : >> >>> Valentin Micic writes: >>> >>> [about the process dictionary] >>> > has to reply with message. What are the limitations of >>> dictionary, and *why* >>> > it received a bad reputation? >>> >>> In addition to the points already made, another reason is that a >>> significant proportion of those who haven't seen single-assignment >>> before will seize upon anything which "finally lets me assign to a >>> damn variable like I want to", and the first 'anything' they usually >>> encounter is the process dictionary. >>> >>> (ab)using processes and message passing to achieve the same >>> effect at >>> least has the side effect of teaching them about message passing. >>> >> Agreed. The problem isn't that we shouldn't caution about use >> of the process dictionary. We should do so in a way that people >> get a chance to learn why it should be used with care, and what >> else should be used with care. >> I have yet to find the chapter that explains this. >> Erlang has a pragmatic approach to message passing (no monads, >> etc.) This is wonderful in many ways, but it also introduces >> some juicy opportunities to write terrible code. >> /Uffe >> > > From Marc.Vanwoerkom@REDACTED Thu Jul 14 15:13:16 2005 From: Marc.Vanwoerkom@REDACTED (Marc van Woerkom) Date: Thu, 14 Jul 2005 15:13:16 +0200 Subject: Meyer, OO and concurrency In-Reply-To: Message-ID: >The non-shared memory, pure copying asynchronous message >passing, paradigm is >far easier to program (witness all the web applications, >virtually all use >non-shared memory (ie separated processors) and pure >message passing (defined by RFC's)) >and very easy to understand. More evidence that shared memory is evil: --- 2. We recently read that the fix for the Hyper-Threading vulnerability is considered non-trivial. Why is that? John Baldwin: The issue found with HT is that the two logical CPUs on a single core share the same caches and as a result there are ways for one logical CPU to spy on the activities of the other CPU in the same core. The proposed fixes involve ways of guaranteeing that all of threads on a single core are all allowed to spy on each other. For example, one policy is that only threads with the same user ID should be allowed to run together no the same core. The problem is that right now FreeBSD treats logical CPUs as separate CPUs and schedules available threads on the first CPU that becomes available. It would be a bit of work to make the scheduler more aware of logical CPUs and to schedule threads with respect to UIDs, etc. Robert N M Watson: It's worth observing that this is a serious vulnerability across a range of operating systems, not just FreeBSD. If you allow untrusted users on the same system as an SSH daemon, you're at risk, which affects everyone from desktop users, to ISPs, to military end-users. It's also a very hard problem to solve -- we're looking at it from the perspective of improving the scheduler, bringing in OpenSSL updates to limit timing attacks, and obviously we're hoping that CPU vendors take this opportunity to explore how to harden CPU architectures against this sort of attack. Because this vulnerability isn't just about scheduling, crypto, or hyper-threading, a lot of hard work will have to into a long term solutions. --- From http://www.osnews.com/story.php?news_id=10951 Regards, Marc From Thomas.Fischbacher@REDACTED Fri Jul 15 18:36:14 2005 From: Thomas.Fischbacher@REDACTED (Thomas Fischbacher) Date: Fri, 15 Jul 2005 18:36:14 +0200 (CEST) Subject: Installing Eddie under Debian GNU/Linux (Sarge/Woody)? Message-ID: Hello together, (1) I am in the situation that I have to set up a DNS server for a cluster of machines which may well be small and simple, but should do its job well. (2) For some not too strange reasons, I am quite reluctant to use either of BIND9 or djbdns. (3) I like the idea of having a reliable DNS server written in a safe language, and if it's functional, the better. (4) There are a few other lightweight implementations out there, like oak (written in python) or the Net::DNS::Server perl library. These, unfortunately, seem to be quite broken. (5) Hence, I would like to give the Erlang DNS server from http://eddie.sourceforge.net a try. It can do much more than what I need, but that does not matter to me. (Somehow, I have the impression that Erlang is just the tool one should use for building an application like a DNS server, so why not have a look. I consider myself quite proficient with functional programming, but never had any hands-on exposure to erlang whatsoever so far. However, I know about the underlying concepts and find them quite nice and appealing.) My problem is: I cannot get the thing to compile. I downloaded the sources from http://eddie.sourceforge.net/1p50.html (eddie-1.5.1.tar.gz) and use Debian 3.1 on x86/32 bit (Sarge - but as there are no Erlang packages for Sarge, I resorted to installing those from woody). The log transcript at the end of this email contains pretty much all the information I can provide about this in a distilled form. Does anybody have an idea what is going wrong and how to fix it? But before I delve deeper into this, I'd like to ask two further questions: (a) Does anyone know how Eddie's DNS server chooses source ports for recursive resolver queries? (I have IP source address randomization on my machine from the grsecurity.net kernel patch, but it's better not to have to rely on that.) Is the IP ID random number generation in the DNS reasonable? (b) I am assuming that with an Erlang-implemented DNS, there should not be any possibility for low-level system attacks that result in the execution of foreign code (i.e. stack or heap overflows and their relatives) as a principal matter of the design. Is that assumption justified? === SHELL LOG === tf@REDACTED:/tmp$ apt-cache show erlang Package: erlang Priority: optional Section: non-US Installed-Size: 27304 Maintainer: Samuel Tardieu Architecture: i386 Version: 1:8.0-4 Replaces: erlang-base (<< 1:8.0-2) Depends: libc6 (>= 2.2.4-4), libncurses5 (>= 5.2.20010310-1), libssl0.9.6, xlibs (>> 4.1.0), erlang-base (=1:8.0-4) Pre-Depends: dpkg (>= 1.4.1.17) Recommends: erlang-manpages, erlang-doc-html Conflicts: erlang-jams, erlang-jams-erl, erlang-dev Filename: pool/non-US/main/e/erlang/erlang_8.0-4_i386.deb Size: 13555706 MD5sum: ca5410df8facb7d2a80b75c406cebda1 Description: A real-time, concurrent and distributed functional language Open Source Erlang is a functional programming language designed at the Ericsson Computer Science Laboratory. . Some of Erlang's main features are: Clear declarative syntax and is largely free from side-effects; Builtin support for real-time, concurrent and distributed programming; Designed for development of robust and continuously operated programs; Dynamic code replacement at runtime. . This package contains architecture-specific files of the runtime system. tf@REDACTED:/tmp$ tar xzf eddie-1.5.1.tar.gz tf@REDACTED:/tmp$ cd eddie-1.5.1 tf@REDACTED:/tmp/eddie-1.5.1$ sh configure creating cache ./config.cache checking host system type... i686-pc-linux-gnu checking target system type... i686-pc-linux-gnu checking build system type... i686-pc-linux-gnu checking for gmake... no configure: warning: gmake not found, will try to use: make checking for ssh... /usr/bin/ssh checking for eddie_rootdir.erl... /tmp/eddie-1.5.1/eddie_rootdir.erl checking for eddie root directory... /tmp/eddie-1.5.1 checking for erl... /usr/bin/erl checking for erlc... /usr/bin/erlc checking for whoami... /usr/bin/whoami checking for addgroup... /usr/sbin/addgroup checking for groupadd... /usr/sbin/groupadd checking checking for Erlang machine type... {"init terminating in do_boot",{undef,[{compile,file,[eddie_rootdir,[report_errors,report_warnings]]},{lists,map,2},{c,lc_batch,1},{init,start_it,1},{init,start_em,1}]}} init terminating in do_boot () no configure: error: Unable to determine Erlang machine type: jam/beam tf@REDACTED:/tmp/eddie-1.5.1$ ls -la erl_crash.dump -rw-r----- 1 tf tf 75389 Jul 15 17:55 erl_crash.dump tf@REDACTED:/tmp/eddie-1.5.1$ head -n 20 erl_crash.dump Fri Jul 15 17:55:42 2005 Slogan: init terminating in do_boot () Erlang (BEAM) emulator version 5.1 Compiled on Tue Dec 18 21:07:58 2001 Process Information -------------------------------------------------- <0.0.0> Running. Registered as: init Spawned as: otp_ring0:start/2 Message buffer data: 45 words Link list: [<0.5.0>,<0.4.0>,<0.2.0>] Reductions 3611 stack+heap 377 old_heap_sz=233 Heap unused=95 OldHeap unused=233 Stack dump: program counter = 0x81c66cc (init:boot_loop/2 + 32) cp = 0x81c654c (init:things_to_string/1 + 68) tf@REDACTED:/tmp/eddie-1.5.1$ tail -n 10 erl_crash.dump '-i/1-fun-1-' '-i/1-fun-0-' '-alive_processes/0-fun-0-' '-lc_batch/1-fun-0-' '-lc/1-fun-0-' io_request_loop get_until_loop '-do_start/4-fun-0-' tf@REDACTED:/tmp/eddie-1.5.1$ # This is where things fail: tf@REDACTED:/tmp/eddie-1.5.1$ head -n 1028 configure | tail -n 10 echo $ac_n "checking checking for Erlang machine type""... $ac_c" 1>&6 echo "configure:1023: checking checking for Erlang machine type" >&5 rm -f $EDDIE_ROOTDIR.{jam,beam} $ac_cv_erl -compile $EDDIE_ROOTDIR EDDIE_ROOTDIR_FILES=`echo $EDDIE_ROOTDIR.*` case "$EDDIE_ROOTDIR_FILES" in *jam*) EMULATOR=jam tf@REDACTED:/tmp/eddie-1.5.1$ # ...and this is what I get when I try to compile this manually: tf@REDACTED:/tmp/eddie-1.5.1$ erlc eddie_rootdir.erl Compiler function compile:compile/3 failed: {undef,[{compile,compile, ["/tmp/eddie-1.5.1/eddie_rootdir", "/tmp/eddie-1.5.1/eddie_rootdir", {options,[], "/tmp/eddie-1.5.1", undefined, [], 0, false, 999, [], [], '/tmp/eddie-1.5.1', undefined}]}, {erl_compile,compile_file,4}, {erl_compile,compile3,3}, {erl_compile,compiler_runner,1}]} tf@REDACTED:/tmp/eddie-1.5.1$ === END SHELL LOG === -- regards, tf@REDACTED (o_ Thomas Fischbacher - http://www.cip.physik.uni-muenchen.de/~tf //\ (lambda (n) ((lambda (p q r) (p p q r)) (lambda (g x y) V_/_ (if (= x 0) y (g g (- x 1) (* x y)))) n 1)) (Debian GNU) From will@REDACTED Fri Jul 15 19:16:09 2005 From: will@REDACTED (Will Newton) Date: Fri, 15 Jul 2005 18:16:09 +0100 Subject: Installing Eddie under Debian GNU/Linux (Sarge/Woody)? In-Reply-To: References: Message-ID: <200507151816.09531.will@misconception.org.uk> On Friday 15 July 2005 17:36, Thomas Fischbacher wrote: > I downloaded the sources from http://eddie.sourceforge.net/1p50.html > (eddie-1.5.1.tar.gz) and use Debian 3.1 on x86/32 bit (Sarge - but as > there are no Erlang packages for Sarge, I resorted to installing those > from woody). The ones from Woody are quite old now IIRC (R8). The erlang packages from sid should install, and if they don't rebuilding should be trivial. From Thomas.Fischbacher@REDACTED Fri Jul 15 19:46:55 2005 From: Thomas.Fischbacher@REDACTED (Thomas Fischbacher) Date: Fri, 15 Jul 2005 19:46:55 +0200 (CEST) Subject: Installing Eddie under Debian GNU/Linux (Sarge/Woody)? In-Reply-To: <200507151816.09531.will@misconception.org.uk> References: <200507151816.09531.will@misconception.org.uk> Message-ID: On Fri, 15 Jul 2005, Will Newton wrote: > > there are no Erlang packages for Sarge, I resorted to installing those > > from woody). > > The ones from Woody are quite old now IIRC (R8). The erlang packages from sid > should install, and if they don't rebuilding should be trivial. Well, so is eddie. But I'll give it a try. -- regards, tf@REDACTED (o_ Thomas Fischbacher - http://www.cip.physik.uni-muenchen.de/~tf //\ (lambda (n) ((lambda (p q r) (p p q r)) (lambda (g x y) V_/_ (if (= x 0) y (g g (- x 1) (* x y)))) n 1)) (Debian GNU) From serge@REDACTED Fri Jul 15 20:09:40 2005 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 15 Jul 2005 14:09:40 -0400 Subject: New trading systems platform In-Reply-To: <42D27E1B.6030109@erlang-fr.org> References: <42D27E1B.6030109@erlang-fr.org> Message-ID: <42D7FBE4.4040009@hq.idt.net> Do you know if the Bets source is available for download? Mickael Remond wrote: > Vlad Balin wrote: > >> Well, i would say mnesia should have BTree-based storage option then :), >> and it should support clustered index. It should be able to work with >> huge databases without tricks like partitioning single table into several >> pieces. >> >> Easiest way to achieve that would be to use Berkley DB >> http://www.sleepycat.com/ as a storage ;) > > > Yes. See the paper presented during EUC 2004: Bets is a Berkeley DB > based storage backend for Mnesia. > > http://www.erlang.se/euc/04/Synapse.pdf -- Serge Aleynikov R&D Telecom, IDT Corp. Tel: (973) 438-3436 Fax: (973) 438-1464 serge@REDACTED From Thomas.Fischbacher@REDACTED Fri Jul 15 20:23:00 2005 From: Thomas.Fischbacher@REDACTED (Thomas Fischbacher) Date: Fri, 15 Jul 2005 20:23:00 +0200 (CEST) Subject: Installing Eddie under Debian GNU/Linux (Sarge/Woody)? In-Reply-To: <200507151816.09531.will@misconception.org.uk> References: <200507151816.09531.will@misconception.org.uk> Message-ID: On Fri, 15 Jul 2005, Will Newton wrote: > > I downloaded the sources from http://eddie.sourceforge.net/1p50.html > > (eddie-1.5.1.tar.gz) and use Debian 3.1 on x86/32 bit (Sarge - but as > > there are no Erlang packages for Sarge, I resorted to installing those > > from woody). > > The ones from Woody are quite old now IIRC (R8). The erlang packages from sid > should install, and if they don't rebuilding should be trivial. Indeed, that seemed to do the job. Thanks. I had to upgrade libc, libc6-dev, openssl to the packages from etch, but now it seems to compile. (No, I'm actually not that insane to mess up my Debian installation for playing around with a DNS implementation I might or might not use in the end. I'm doing this from within user mode linux with copy-on-write persistent disks.) Now for the next problem: fdsrv_drv.c wants to #include "driver.h" which comes from the erlang header files. There is such a file in include/obsolete/ - this seems to have changed to erl_driver.h in the version of Erlang I have now. (I cannot just replace driver.h by erl_driver.h, but am I permitted to use obsolete/driver.h?) All that is certainly not that nice, but on the other hand, I am maybe not that interested in the fdsrv component at all. Let's see. At least, "make dns_server" seems to work well now. -- regards, tf@REDACTED (o_ Thomas Fischbacher - http://www.cip.physik.uni-muenchen.de/~tf //\ (lambda (n) ((lambda (p q r) (p p q r)) (lambda (g x y) V_/_ (if (= x 0) y (g g (- x 1) (* x y)))) n 1)) (Debian GNU) From Thomas.Fischbacher@REDACTED Fri Jul 15 21:10:51 2005 From: Thomas.Fischbacher@REDACTED (Thomas Fischbacher) Date: Fri, 15 Jul 2005 21:10:51 +0200 (CEST) Subject: Installing Eddie under Debian GNU/Linux (Sarge/Woody)? In-Reply-To: References: <200507151816.09531.will@misconception.org.uk> Message-ID: On Fri, 15 Jul 2005, Thomas Fischbacher wrote: > At least, "make dns_server" seems to work well now. With some extra bad treatment of the code in a few places, I am a step further towards getting a working eddie. Now, make stops here: ====> /usr/bin/erlc -bbeam -W -I/usr/lib/erlang/lib/mnemosyne-1.2.5/ebin -I/tmp/eddie-1.5.1/misc/include -I/tmp/eddie-1.5.1/servant/include -I/tmp/eddie-1.5.1/inet_server/src -I/tmp/eddie-1.5.1/load_info/src -pa /tmp/eddie-1.5.1/misc -pa /tmp/eddie-1.5.1/servant -o./ make.erl /usr/bin/erl -pa ../load_info/ebin -pa ../inet_server/ebin -pa ../sync_nodes/ebin -pa ../oam/ebin -pa ../misc/ebin -pa ../master/ebin -pa ../servant/ebin -pa ../lonelb/ebin -pa . -noshell +B -s make script eddie beam .. mnesia: No valid version ("3.9.3") of .app file found. Found file "/usr/lib/erlang/lib/mnesia-4.2.2/ebin/mnesia.app" with version "4.2.2" os_mon: No valid version ("1.3.9") of .app file found. Found file "/usr/lib/erlang/lib/os_mon-1.7.3/ebin/os_mon.app" with version "1.7.3" sasl: No valid version ("1.9.1") of .app file found. Found file "/usr/lib/erlang/lib/sasl-2.0.1/ebin/sasl.app" with version "2.0.1" stdlib: No valid version ("1.9.2") of .app file found. Found file "/usr/lib/erlang/lib/stdlib-1.13.7/ebin/stdlib.app" with version "1.13.7" kernel: No valid version ("2.6.1.5") of .app file found. Found file "/usr/lib/erlang/lib/kernel-2.10.7/ebin/kernel.app" with version "2.10.7" error make[1]: *** [eddie.boot] Error 1 make[1]: Leaving directory `/tmp/eddie-1.5.1/system' make: *** [system] Error 2 <==== Any clue what this is supposed to mean? Somehow, I get the impression that I am almost done. -- regards, tf@REDACTED (o_ Thomas Fischbacher - http://www.cip.physik.uni-muenchen.de/~tf //\ (lambda (n) ((lambda (p q r) (p p q r)) (lambda (g x y) V_/_ (if (= x 0) y (g g (- x 1) (* x y)))) n 1)) (Debian GNU) From serge@REDACTED Fri Jul 15 21:12:44 2005 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 15 Jul 2005 15:12:44 -0400 Subject: catching SIGTERM In-Reply-To: <20050713173518.GC11715@jump.lib.uchicago.edu> References: <20050713173518.GC11715@jump.lib.uchicago.edu> Message-ID: <42D80AAC.10402@hq.idt.net> I can think of two ways of doing this: 1. Write a port driver in C (or another language) that would handle SIGTERM events and dispatch them to the linked Erlang process using message passing. (If interested I can provide you with such code, though I believe the second approach is easier). 2. Call some function that dumps state from another node. Here is an example of calling server:dump_state() on the 'nodename' node from another node: echo "rpc:call('nodename@`hostname -s`', server, dump_state, [])." | \ erl -sname restarter -hidden Serge Charles Blair wrote: > i'd like to be able to trap a SIGTERM from an application written in > erlang+yaws so i can dump some state to disk when that happens. is > there a way to do this? searching the net brought up two similar > questions with no responses. thanks. From yerl@REDACTED Fri Jul 15 21:34:18 2005 From: yerl@REDACTED (yerl) Date: Fri, 15 Jul 2005 21:34:18 +0200 Subject: Installing Eddie under Debian GNU/Linux (Sarge/Woody)? In-Reply-To: References: <200507151816.09531.will@misconception.org.uk> Message-ID: <42D80FBA.9030701@club-internet.fr> Hi Thomas! This is because you're using a new version of OTP. Set the correct valu in the system.mk file. Yerl Thomas Fischbacher a ?crit : >On Fri, 15 Jul 2005, Will Newton wrote: > > > >>>I downloaded the sources from http://eddie.sourceforge.net/1p50.html >>>(eddie-1.5.1.tar.gz) and use Debian 3.1 on x86/32 bit (Sarge - but as >>>there are no Erlang packages for Sarge, I resorted to installing those >>>from woody). >>> >>> >>The ones from Woody are quite old now IIRC (R8). The erlang packages from sid >>should install, and if they don't rebuilding should be trivial. >> >> > >Indeed, that seemed to do the job. Thanks. > >I had to upgrade libc, libc6-dev, openssl to the packages from etch, but >now it seems to compile. (No, I'm actually not that insane to mess up my >Debian installation for playing around with a DNS implementation I might >or might not use in the end. I'm doing this from within user mode linux >with copy-on-write persistent disks.) > >Now for the next problem: fdsrv_drv.c wants to > >#include "driver.h" > >which comes from the erlang header files. There is such a file in >include/obsolete/ - this seems to have changed to erl_driver.h in the >version of Erlang I have now. (I cannot just replace driver.h by >erl_driver.h, but am I permitted to use obsolete/driver.h?) > > >All that is certainly not that nice, but on the other hand, I am maybe >not that interested in the fdsrv component at all. Let's see. > >At least, "make dns_server" seems to work well now. > > > > From Thomas.Fischbacher@REDACTED Fri Jul 15 21:42:00 2005 From: Thomas.Fischbacher@REDACTED (Thomas Fischbacher) Date: Fri, 15 Jul 2005 21:42:00 +0200 (CEST) Subject: Installing Eddie under Debian GNU/Linux (Sarge/Woody)? In-Reply-To: <42D80FBA.9030701@club-internet.fr> References: <200507151816.09531.will@misconception.org.uk> <42D80FBA.9030701@club-internet.fr> Message-ID: On Fri, 15 Jul 2005, yerl wrote: > Hi Thomas! > This is because you're using a new version of OTP. > Set the correct valu in the system.mk file. > Yerl Thanks. I also noticed by now, and indeed: ===> system/mk_dot_erlang.sh /usr/local/lib/eddie > /usr/local/lib/eddie/bin/.eddie_path ./install-sh -c -o root -m 4755 misc/priv/fdsrv /usr/local/lib/eddie/lib/misc-1.5.1/priv/fdsrv ./install-sh -c -m 0644 dns_server/system/examples/lb_dns.boot /usr/local/etc/lb_dns.boot.dist ./install-sh -c -m 0644 dns_server/system/examples/lb_dns.boot /usr/local/etc/lb_dns.boot.dist # Make a .eddie_path file system/mk_dot_erlang.sh /usr/local/lib/eddie > /usr/local/lib/eddie/bin/.eddie_path Eddie installation complete <=== Hard to believe I got it that far. Could anyone please explain to me what this "pinning down to versions" thing is, and what the rationale behind that is / what it is good for? -- regards, tf@REDACTED (o_ Thomas Fischbacher - http://www.cip.physik.uni-muenchen.de/~tf //\ (lambda (n) ((lambda (p q r) (p p q r)) (lambda (g x y) V_/_ (if (= x 0) y (g g (- x 1) (* x y)))) n 1)) (Debian GNU) From jahakala@REDACTED Fri Jul 15 21:42:59 2005 From: jahakala@REDACTED (Jani Hakala) Date: Fri, 15 Jul 2005 22:42:59 +0300 Subject: Installing Eddie under Debian GNU/Linux (Sarge/Woody)? In-Reply-To: (Thomas Fischbacher's message of "Fri, 15 Jul 2005 21:10:51 +0200 (CEST)") References: <200507151816.09531.will@misconception.org.uk> Message-ID: <87br53yhx8.fsf@pingviini.kortex.jyu.fi> Thomas Fischbacher writes: > On Fri, 15 Jul 2005, Thomas Fischbacher wrote: > >> At least, "make dns_server" seems to work well now. > > With some extra bad treatment of the code in a few places, I am a step > further towards getting a working eddie. Now, make stops here: > Try applying the following patch -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: eddie.diff URL: -------------- next part -------------- Or edit those system.mk files and run find . -name '*.src'|xargs touch before make if necessary. Jani Hakala From Thomas.Fischbacher@REDACTED Fri Jul 15 22:03:09 2005 From: Thomas.Fischbacher@REDACTED (Thomas Fischbacher) Date: Fri, 15 Jul 2005 22:03:09 +0200 (CEST) Subject: Installing Eddie under Debian GNU/Linux (Sarge/Woody)? In-Reply-To: <87br53yhx8.fsf@pingviini.kortex.jyu.fi> References: <200507151816.09531.will@misconception.org.uk> <87br53yhx8.fsf@pingviini.kortex.jyu.fi> Message-ID: On Fri, 15 Jul 2005, Jani Hakala wrote: > > > Try applying the following patch > Thanks for the tip. I already figured this out of my own by now. By the way, one also has to patch the system.mk in dns_server. Right now, I am quite undecided about eddie, considering that it feels to a newbie as if over the years, bit rot had done quite some damage. Nevertheless, I think the erlang community still might have something quite interesting to offer to the world here, if a few points can be resolved (including licensing - maybe). After all, there are a lot of people around who just do not want to run BIND9, chrooted or not, and who would not consider installing software by a person who really considers himself the only authority in the world that has the right to define what a filesystem hierarchy should look like. There are some further less known alternatives, like MaraDNS, usually also done in C (or C++), but having a reasonably complete DNS server implemented in a safe language is certainly a very big plus. (I hope I'm not over-estimating erlang in terms of security here - as I said, I'm just a beginner.) Well, I'll play around a bit more with it tomorrow, and also have a close look at the code. Thanks all, you're great. -- regards, tf@REDACTED (o_ Thomas Fischbacher - http://www.cip.physik.uni-muenchen.de/~tf //\ (lambda (n) ((lambda (p q r) (p p q r)) (lambda (g x y) V_/_ (if (= x 0) y (g g (- x 1) (* x y)))) n 1)) (Debian GNU) From vances@REDACTED Fri Jul 15 23:35:28 2005 From: vances@REDACTED (Vance Shipley) Date: Fri, 15 Jul 2005 17:35:28 -0400 Subject: Meyer, OO and concurrency In-Reply-To: References: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> <20050714012420.GC89016@feeble.motivity.ca> Message-ID: <20050715213527.GD35959@feeble.motivity.ca> Admonishing message passing seems a strange concept however I would add a warning about registered processes. I rarely use them anymore as I am a purist and in the pursuit of functional programming I avoid needing them. There are many reasons to do so, much more than using the process dictionary, but again doing so is often due to laziness. On the other hand debugging an application with hundreds of interacting anonymous processes is a bit more difficult. -Vance From kramer@REDACTED Sat Jul 16 06:39:15 2005 From: kramer@REDACTED (Reto Kramer) Date: Fri, 15 Jul 2005 21:39:15 -0700 Subject: looking for eperf and dtop Message-ID: <7b57b4f303fafb276dddba184fbbd240@acm.org> I'm looking for eperf and dtop, the two tools Mats Conqvist names in his 2004 presentation on debugging "large scale Erlang applications". I've tried OTP, jungerl, trapexit and google, but cannot find them. Can eperf and dtop be made available to the open source Erlang users (a la pan in jungerl perhaps)? Thanks, - Reto From tony@REDACTED Sat Jul 16 10:35:23 2005 From: tony@REDACTED (Tony Rogvall) Date: Sat, 16 Jul 2005 10:35:23 +0200 Subject: jungerl/zlib changes In-Reply-To: <42D7C75A.9020208@web.am> References: <42D7C75A.9020208@web.am> Message-ID: <6B25DCA8-2BFA-4784-AE97-EBF11018FA44@rogvall.com> 15 jul 2005 kl. 16.25 skrev Gaspar Chilingarov: > Hi all! > > > I suggest the following changes to zlib module distributed in > jungerl - > > --cut-- > --cut-- > > first of all - imho open_port automatically loads external driver, > and if I leave it load() call, it will fail for subsequent calls to > zlib:compress or zlib:uncompress - because module is already loaded. > > seconds change - is to have receive command , which will cleanup > port exit message from process queue, if such message exists. You are right, but the zlib library exists already in the otp (since R9C-0?). This means that we should stop using the jungerl version (delete from jungerl?). Check the otp source code. Regards /Tony > > > > -- > Gaspar Chilingarov > System Administrator > > t +37491 419763 (mob) > t +37410 240399 (office) > w www.web.am > i 63174784 > e nm@REDACTED > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chas@REDACTED Sat Jul 16 21:52:29 2005 From: chas@REDACTED (Charles Blair) Date: Sat, 16 Jul 2005 14:52:29 -0500 Subject: catching SIGTERM In-Reply-To: Your message of "Fri, 15 Jul 2005 15:12:44 EDT." <42D80AAC.10402@hq.idt.net> Message-ID: <20050716195229.74F072E129C@jump.lib.uchicago.edu> > 2. Call some function that dumps state from another node. Here is an > example of calling server:dump_state() on the 'nodename' node from > another node: > > echo "rpc:call('nodename@`hostname -s`', server, dump_state, [])." | \ > erl -sname restarter -hidden thanks. i whipped up a proof of concept using this approach, and it worked like a charm. thanks again. From vances@REDACTED Sat Jul 16 22:50:05 2005 From: vances@REDACTED (Vance Shipley) Date: Sat, 16 Jul 2005 16:50:05 -0400 Subject: Meyer, OO and concurrency In-Reply-To: References: <200507132247.j6DMlBRK145672@atlas.otago.ac.nz> <20050714012420.GC89016@feeble.motivity.ca> <20050715213527.GD35959@feeble.motivity.ca> Message-ID: <20050716205004.GE35959@feeble.motivity.ca> On Sat, Jul 16, 2005 at 01:55:34PM +0200, Nigel Head wrote: } } How do you find, for example, a particular server in a distributed } system, in particular when it might be moving from node to node if it } fails and gets restarted by the distributed application stuff ? With functional programming you supply all the data required to a function, instead of global variables you use arguments. The same applies here, start the clients with the pid of the server. I generally use registered processes if the client and server are in seperate applications. Within the same application it is usually easy enough to initialize the client properly so that it knows where it's server is. Another trick I use is to inspect the supervisors. -module(client). -export([init/1, init_1/2, idle/2]). -behaviour(gen_fsm). -record(state, {sup, server}). init(Sup) when is_pid(Sup) -> {ok, init_1, #state{sup = Sup}}. init_1(timeout, State) -> MyGeneration = supervisor:which_children(State#state.sup), {value, {sister, Server, _, _}} = lists:keysearch(sister, 1, MyGeneration), {next_state, idle, State#state{server = Server}}. idle(Event, State) -> ... gen_server:call(State#state.server, Request), ... If processes are part of the same application you will have them all under one supervision tree. You can inspect the current children of a supervisor to determine the pid of another process. In the above we find the process which was started with the Id sister: -module(sup). -export(init/1). -behaviour(supervisor). init(_) -> ... ChildSpec = {sister,StartFunc,Restart,Shutdown,Type,Modules} ... {ok,{{RestartStrategy,MaxR,MaxT},[ChildSpec]}} I found that most of my uses of registered processes were unnecessary. The point was that it isn't functional style and as such should be discouraged. That isn't to say using them is wrong just that you should be hesitant to do so. -Vance From david.nospam.hopwood@REDACTED Sun Jul 17 00:20:13 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Sat, 16 Jul 2005 23:20:13 +0100 Subject: Meyer, OO and concurrency In-Reply-To: <42D72D4E.4060807@possibility.com> References: <42D69CA2.6040905@possibility.com> <42D6B175.8040005@blueyonder.co.uk> <42D6B8BB.8040801@possibility.com> <42D6B937.7020802@blueyonder.co.uk> <42D6BD36.1080807@possibility.com> <42D6FFD8.4020701@blueyonder.co.uk> <42D72D4E.4060807@possibility.com> Message-ID: <42D9881D.5030702@blueyonder.co.uk> todd wrote: > David Hopwood wrote: > >> Of course, if a language implementation is running on an OS that is >> hopelessly broken in respect of real-time issues, and does not allow >> running *any* code with low latency, then the implementation will also >> be hopelessly broken in that respect. This is just as true of C++ as it >> is of Erlang. > > It's not difficult. I agree that it isn't difficult: if you want a given set of real-time guarantees, use a platform that supports those guarantees. But a platform is (hardware, OS implementation, language implementation), not just an OS. > Interrupt. How long till the task runs? The OS > can guarantee that. A language can't. What I said in my previous post applies just as much to interrupts as to any other feature: sometimes it is possible for the language implementation to work around problems with latency of interrupt handlers in the OS, and sometimes it isn't. -- David Hopwood From david.nospam.hopwood@REDACTED Sun Jul 17 01:26:25 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Sun, 17 Jul 2005 00:26:25 +0100 Subject: Meyer, OO and concurrency In-Reply-To: References: <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> <42D6C11D.6000007@blueyonder.co.uk> <20050714214105.GJ644@feeble.motivity.ca> <42D6F8CE.40701@blueyonder.co.uk> Message-ID: <42D997A1.9090406@blueyonder.co.uk> Ulf Wiger wrote: > Den 2005-07-15 01:44:14 skrev David Hopwood > : > >> I think there may be some misunderstandings in this thread about what >> process GC in actor systems is intended to achieve. As for any kind >> of value in any GC algorithm, processes are only collected based on >> a conservative approximation of whether they can affect the future >> behaviour of the program. It is quite possible to have processes that >> cannot do anything useful but that are still reachable; these are not >> collected. But this is really no different from any other kind of >> memory leak that can occur despite the use of precise GC, and is in >> practice easily avoidable. > > Since you're talking about the actor model specifically, > and state that some issues are only problems if you try > to implement OO in the actor model, I thought I'd try to > read about the actor model. I found this page: > > http://www.agent.ai/main.php?folderID=221 > > I read "Towards a Theory of Actor Computation" by Agha et al. That's probably not the best paper to start with. Try sections I--IX of /Actors and Continuous Functionals/, or if you have time, read Agha's book (http://c2.com/cgi/wiki?AghaActorsBook). (All papers I mention in this post are available from .) > The hour is late, so perhaps my reading was too sloppy. I > didn't really see how the message reception semantics were > defined. The paper talks about data channels (which are not > present in Erlang), and specifically disallows passing > lambda expressions in a message to an actor. That restriction is specific to that paper, and highly atypical of actor languages in general. There is an equivalence between actors and lambda abstractions, and so no need to introduce lambda abstractions as a separate concept in actor languages. I think the paper did it that way in order for the kernel language it is defining to be a strict extension of the lambda calculus. > This is allowed > in Erlang. From my first read, I assume that the reception > semantics are FIFO, event-based, much like in CSP, and not > selective receive a la Erlang. Various actor languages did have selective receive, for example the "communication handlers" of Act2 and Act3. In any case, selective receive does not affect the expressive power of a message passing model: it can be simulated, albeit inefficiently, by receiving unconditionally and then re-sending the message to self if necessary. CSP uses synchronous message passing; actors and Erlang are asynchronous. For a comparison of actors with CSP, see . > Also, it appears as if > computation paths triggered by messages are concurrent, > which would imply spawning another process for each message > (or simply ignoring this, and serializing everything within > the actor/process.) You say "spawning another process" as if it's expensive or difficult. The pure actor model uses continuation passing style, and compilation of sequential code into this model does involve creating an actor for every continuation. That doesn't mean that you would write code in a high-level actor language in continuation passing style. The reason that *kernel* actor languages take concurrent behaviours as primitive is that concurrent behaviours and message passing are sufficient to express sequential control flow (see /Viewing Control Structures as Patterns of Passing Messages/), whereas sequential behaviours would not be sufficient to express concurrency. All high-level actor languages provide both. > To me, it sounds like you would need a control layer in order > to do actor programming in Erlang. Perhaps you'd also then > want to use explicit channels? Essentially, you'd write a > framework for actor programming. No, Erlang would directly be an actor language, but for the issues I pointed out earlier in the thread. -- David Hopwood From chas@REDACTED Sun Jul 17 03:19:43 2005 From: chas@REDACTED (Charles Blair) Date: Sat, 16 Jul 2005 20:19:43 -0500 Subject: two style questions Message-ID: <20050717011943.D93212E1574@jump.lib.uchicago.edu> as a relative newcomer to erlang, i read the thread Meyer, OO and concurrency with some interest, and wanted to isolate two comments for further discussion here, as i have some questions about them: i want to be sure i'm on the right track for the work i'm doing. uffe writes: One could imagine a specification stating that a process waiting indefinitely should be automatically removed, but I have never considered it a problem that Erlang doesn't do so. In Erlang, a process waiting indefinitely for something that will never occur is probably (but not certainly) in a faulty state. here's my first question: the only way i know of to keep an ets table open in memory is to spawn a process that opens it, does whatever else it has to, then recurs on itself waiting for a message that never comes: is there another way to keep an ets table open indefinitely? on a related note, does it make sense to call erlang:hibernate right before the recursive step? vance writes: ... I would add a warning about registered processes. I rarely use them anymore as I am a purist and in the pursuit of functional programming I avoid needing them. There are many reasons to do so, much more than using the process dictionary, but again doing so is often due to laziness. here's my second question. i've implemented the server part of a client/server protocol where data are delivered in chunks of a certain size. in addition to the data, the server sends the client a resumption token which, when sent back to the server by the client, lets the server send along the next chunk of data. there are several ways of implementing this, but the way i've chosen to do it is to use a registered process to keep state, passing along the name of the process in the resumption token. when i get the token back from the client, i look up the relevant process, and pick up where i left off. the question is, is this a reasonable implementation from the erlang perspective? (i might add that the clients can be written in any language; token passing is done via http. i might also add that i switched to erlang from common lisp to force myself to write in a purely functional style.) From joelr1@REDACTED Sun Jul 17 13:00:14 2005 From: joelr1@REDACTED (Joel Reymont) Date: Sun, 17 Jul 2005 13:00:14 +0200 Subject: Time series db: please help me optimize it Message-ID: Folks, This is my attempt at creating a time-series database in Erlang. I specifically want to store a day's worth of data in a separate disk log and keep the columns separate. You can read more on the whys here: http://groups-beta.google.com/group/uptick/ You can get the data from http://wagerlabs.com/ES4U.csv.gz. Run it like this: tickdb:load("ES4U", "ES4U.csv"). I had a version that did not cache open disk logs but used tickdb:add directly, without going through a gen_server. It ran almost 3x faster! Any suggestions on how to optimize this are appreciated. There are 119,275 trade record in the CSV file. I divide the total run time by this number to get the average per insert. I would like to do at least 100 inserts per second. The code is part of the open source trading platform that I'm creating. Thanks, Joel -- http://wagerlabs.com/uptick -------------- next part -------------- A non-text attachment was scrubbed... Name: text.erl Type: application/octet-stream Size: 1624 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tickdb.erl Type: application/octet-stream Size: 4736 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tickplus.erl Type: application/octet-stream Size: 1168 bytes Desc: not available URL: -------------- next part -------------- From sebastian@REDACTED Sun Jul 17 15:26:43 2005 From: sebastian@REDACTED (Sebastian Bello) Date: Sun, 17 Jul 2005 10:26:43 -0300 Subject: mnesia query References: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> <035701c5888d$ed3cacb0$3000a8c0@INSWITCH244> <036501c58895$ac331260$3000a8c0@INSWITCH244> Message-ID: <002901c58ad3$2d2fff60$3000a8c0@INSWITCH244> But is it possible to use mnesia:index_match_object() with two indexes? Sebastian- ----- Original Message ----- From: "Ulf Wiger" To: "Sebastian Bello" ; Sent: Thursday, July 14, 2005 7:19 PM Subject: Re: mnesia query > Den 2005-07-14 19:01:25 skrev Sebastian Bello : > > > Regarding 2: > > > > - and using select? > > Select doesn't use indexes. mnesia:index_match_object() > does. > > > > - is there any drawback to using different query languages in an > > application to improve performance? > > Not really, I think. > > /Uffe > > From ulf@REDACTED Sun Jul 17 16:19:00 2005 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 17 Jul 2005 16:19:00 +0200 Subject: mnesia query In-Reply-To: <002901c58ad3$2d2fff60$3000a8c0@INSWITCH244> References: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> <035701c5888d$ed3cacb0$3000a8c0@INSWITCH244> <036501c58895$ac331260$3000a8c0@INSWITCH244> <002901c58ad3$2d2fff60$3000a8c0@INSWITCH244> Message-ID: Den 2005-07-17 15:26:43 skrev Sebastian Bello : > But is it possible to use mnesia:index_match_object() with two indexes? > Sebastian- No. You specify exactly _one_ index that index_match_object() is to use. /Uffe From sebastian@REDACTED Sun Jul 17 16:33:34 2005 From: sebastian@REDACTED (Sebastian Bello) Date: Sun, 17 Jul 2005 11:33:34 -0300 Subject: mnesia query References: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> <035701c5888d$ed3cacb0$3000a8c0@INSWITCH244> <036501c58895$ac331260$3000a8c0@INSWITCH244> <002901c58ad3$2d2fff60$3000a8c0@INSWITCH244> Message-ID: <007101c58adc$838050f0$3000a8c0@INSWITCH244> Thanks! Sebastian- ----- Original Message ----- From: "Ulf Wiger" To: "Sebastian Bello" ; Sent: Sunday, July 17, 2005 11:19 AM Subject: Re: mnesia query > Den 2005-07-17 15:26:43 skrev Sebastian Bello : > > > But is it possible to use mnesia:index_match_object() with two indexes? > > Sebastian- > > No. You specify exactly _one_ index that index_match_object() is to use. > > /Uffe > > From ulf@REDACTED Sun Jul 17 16:41:00 2005 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 17 Jul 2005 16:41:00 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42D997A1.9090406@blueyonder.co.uk> References: <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> <42D6C11D.6000007@blueyonder.co.uk> <20050714214105.GJ644@feeble.motivity.ca> <42D6F8CE.40701@blueyonder.co.uk> <42D997A1.9090406@blueyonder.co.uk> Message-ID: Den 2005-07-17 01:26:25 skrev David Hopwood : > Various actor languages did have selective receive, for example the > "communication handlers" of Act2 and Act3. In any case, selective > receive does not affect the expressive power of a message passing > model: it can be simulated, albeit inefficiently, by receiving > unconditionally and then re-sending the message to self if necessary. Actually, you can't do it like that. That could for example cause your process to go into a tight loop consuming and re-sending the same unwanted message while waiting for some other message, which would have a rather nasty effect on cpu load and timing characteristics. One reason for using selective receive is to achieve scoping of messages :a subroutine can wait for a given message, ignoring unknown messages (c.f. gen_server:call()). But ignoring and re-sending are not equivalent. Re-sending unknown messages from a subroutine could have the unwanted side-effect that two messages from some other process arrive out of order, e.g. because one came in before the message of interest to the subroutine, and the next one came in right after it; re-sending the first would cause it to arrive after the second message from the same process, thus messing up a behaviour that is otherwise guaranteed by Erlang: that two messages sent from one process to another will arrive in the same order as they were sent. Hoare suggested using buffer processes to achieve selective receive. That would work, albeit inefficiently. /Uffe From ulf@REDACTED Sun Jul 17 17:36:13 2005 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 17 Jul 2005 17:36:13 +0200 Subject: two style questions In-Reply-To: <20050717011943.D93212E1574@jump.lib.uchicago.edu> References: <20050717011943.D93212E1574@jump.lib.uchicago.edu> Message-ID: Den 2005-07-17 03:19:43 skrev Charles Blair : > > as a relative newcomer to erlang, i read the thread Meyer, OO and > concurrency with some interest, and wanted to isolate two comments for > further discussion here, as i have some questions about them: i want > to be sure i'm on the right track for the work i'm doing. > > uffe writes: > > One could imagine a specification stating that a process waiting > indefinitely should be automatically removed, but I have never > considered it a problem that Erlang doesn't do so. > > In Erlang, a process waiting indefinitely for something that will > never occur is probably (but not certainly) in a faulty state. > > here's my first question: the only way i know of to keep an ets table > open in memory is to spawn a process that opens it, does whatever else > it has to, then recurs on itself waiting for a message that never > comes: is there another way to keep an ets table open indefinitely? on > a related note, does it make sense to call erlang:hibernate right > before the recursive step? No, there's basically no other way. A process has to own the ets table, and when the process dies, the ets table is automatically removed. What one often does, if it's desireable that the ets table outlive the process using it (e.g. if the table has a very large amount of data that you don't want to lose even if the process restarts), is to let another process create the ets table. One convenient way to achieve this is to use mnesia (you can, for example create a ram_copy, local_content table in mnesia, and then use mnesia:activity(ets, fun() ... end) to access it.) A quick and dirty approach for a supervised process is to create the ets table in the start function, before spawning the process. This makes the supervisor the owner of the ets table (which then has to be a public named_table). This is quite convenient, but can cause some problems during soft upgrade, since there is no straightforward way to e.g. rename or delete the ets table without restarting the supervisor.) erlang:hibernate/3 is essentially an optimization with a curious semantic twist(*), but the main point of calling hibernate/3 is to save some memory. Otherwise, it doesn't really make sense. (*) The twist is that the {M,F,A} passed to hibernate will be called when a message has arrived. This is the only reasonable way to have code run _before_ consuming a message. One possible use of this could be to detect that new code has been loaded, and perhaps transform state before jumping back into action code. > vance writes: > > ... I would add a warning about registered processes. I rarely > use them anymore as I am a purist and in the pursuit of > functional programming I avoid needing them. There are many > reasons to do so, much more than using the process dictionary, > but again doing so is often due to laziness. > > here's my second question. i've implemented the server part of a > client/server protocol where data are delivered in chunks of a certain > size. in addition to the data, the server sends the client a > resumption token which, when sent back to the server by the client, > lets the server send along the next chunk of data. there are several > ways of implementing this, but the way i've chosen to do it is to use > a registered process to keep state, passing along the name of the > process in the resumption token. when i get the token back from the > client, i look up the relevant process, and pick up where i left > off. the question is, is this a reasonable implementation from the > erlang perspective? (i might add that the clients can be written in > any language; token passing is done via http. i might also add that i > switched to erlang from common lisp to force myself to write in a > purely functional style.) If the state can outlive the process holding it (which is possible for example if the state is stored in ets (and another process owns the ets table, as explained above), then it can make sense to use the registered name of a process. Otherwise, using the pid seems quite sensible, except for the issue of having to serialize the pid and present it in the external protocol. Using a symbolic name migh look a bit better there. You might even want to use another kind of registry than the built-in one, since you'd otherwise have to call list_to_atom/1 based on information in the http message. This is a potential memory leak, as atoms are never garbage collected. There is a registry in jungerl, called proc_reg. I've done some work on it that hasn't yet made it into jungerl. The version at Jungerl actually failed a QuickCheck session, so I've re-written it completely. There is a race condition in the code that could theoretically bite someone who uses the library in a very unintuitive way. I heartily recommend the new version, which has yet to be released. ;-) /Uffe From vlad_dumitrescu@REDACTED Sun Jul 17 18:19:08 2005 From: vlad_dumitrescu@REDACTED (vladdu) Date: Sun, 17 Jul 2005 18:19:08 +0200 Subject: Erlang scanner returning column numbers References: Message-ID: <20050717161908.84574469EC@bang.trapexit.org> Hi, Since I am about to need enhanced functionality for the scanner and parser, I can't use the regular ones so I moved on to syntax_tools. Now I can get comments and handle source code with macros (via epp_dodger). I still need one feature, though: column numbers. Ulf said earlier he has done such an improvement, and I think it would be useful to add such a feature to syntax_tools too (the latter should be trivial if the scanner returns the information). Any thoughts, ideas, suggestions? Ulf, is your code available? Thanks in advance, Vlad _________________________________________________________ Sent using Mail2Forum (http://m2f.sourceforge.net) From ulf@REDACTED Sun Jul 17 18:39:56 2005 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 17 Jul 2005 18:39:56 +0200 Subject: Erlang scanner returning column numbers In-Reply-To: <20050717161908.84574469EC@bang.trapexit.org> References: <20050717161908.84574469EC@bang.trapexit.org> Message-ID: Den 2005-07-17 18:19:08 skrev vladdu : > I still need one feature, though: column numbers. Ulf said earlier he > has done such an improvement, and I think it would be useful to add such > a feature to syntax_tools too (the latter should be trivial if the > scanner returns the information). I once wrote a small hack in erl_scan.erl that maintained column numbers. I got as far as verifying that this small hack crashed the compiler. I don't recall where it crashed, and didn't really have the time or energy to investigate. > Any thoughts, ideas, suggestions? > > Ulf, is your code available? No, my working version was in the CCviewer parser, which was specialized for hypertext-linked html output. It was not a correct parser, but FWIW, CCviewer is by all means available. /Uffe From ulf@REDACTED Sun Jul 17 19:21:15 2005 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 17 Jul 2005 19:21:15 +0200 Subject: Time series db: please help me optimize it In-Reply-To: References: Message-ID: I changed one line of code in tickplus.erl and added three to tickdb.erl, and now the program seems to be flying along: What I did: changed the gen_server:cast(DB, {'ADD'...}) to a gen_server:call(). What was happening was that the process reading the tick data sucked up the 119,275 TRADE lines in ES4U.csv, and then asynchronously told the disk_log server to store the parsed data to disk. The poor logging server's message queue became extremely large, and garbage collection ate up all the cpu time. Monitoring the werl.exe process in Win XP (sigh), it kept growing, and the message queue length of the DB process grew at least as large as 56,000 messages, before it started shrinking. My result from the modified code: 2> tickplus:load("ES4U","ES4U.csv"). New log: tick/price/20040901 New log: tick/size/20040901 New log: tick/price/20040902 New log: tick/size/20040902 New log: tick/price/20040903 New log: tick/size/20040903 done: 8641.000000ms ok Same thing with the original code: 1> tickplus:load("ES4U","ES4U.csv"). New log: tick/price/20040901 New log: tick/size/20040901 New log: tick/price/20040902 New log: tick/size/20040902 done: 263094.000000ms ok New log: tick/price/20040903 New log: tick/size/20040903 Obviously, "done" here only means that the reader has sent its last message to the DB process, but the DB process still has a backlog to take care of. Adding a 'sync' call to make sure that the DB process is done: 3> tickplus:load("ES4U","ES4U.csv"). New log: tick/price/20040901 New log: tick/size/20040901 New log: tick/price/20040902 New log: tick/size/20040902 New log: tick/price/20040903 New log: tick/size/20040903 synced: 410234.000000ms ok BTW: $ grep TRADE ES4U.csv > trade_lines.csv $ ls -l *.csv *.LOG -rwx------+ 1 ?garen Ingen 682113 Jul 17 18:28 1826599643.LOG -rwx------+ 1 ?garen Ingen 664686 Jul 17 18:25 2449200400.LOG -rwx------+ 1 ?garen Ingen 1495347 Jul 17 18:28 3804582371.LOG -rwx------+ 1 ?garen Ingen 1847757 Jul 17 18:28 4022035944.LOG -rwx------+ 1 ?garen Ingen 1552230 Jul 17 18:25 676106329.LOG -rwx------+ 1 ?garen Ingen 802032 Jul 17 18:25 92519506.LOG -rwx------+ 1 ?garen Ingen 18386970 Jul 13 00:55 ES4U.csv -rw-r--r-- 1 ?garen Ingen 4920505 Jul 17 18:30 trade_lines.csv 5647435 bytes for the disk_logs vs 4920505 for the original TRADE data. That's 15% storage overhead. Doesn't seem excessive to me. Did all the ticks get logged properly? 26> timer:tc(count,logs,[[4022035944,1826599643,3804582371]]). {219000,119278} using this program to count the objects: ============================================================ -module(count). -export([log/1, logs/1]). log(L) -> {ok, _} = disk_log:open([{name, L}]), Count = count(L, disk_log:chunk(L, start), 0), disk_log:close(L), Count. logs(Ls) -> lists:foldl( fun(L, Sum) -> Sum + log(L) end, 0, Ls). count(L, {Cont, Terms}, C) -> count(L, disk_log:chunk(L, Cont), C+length(Terms)); count(L, eof, C) -> C. ============================================================= /Uffe Den 2005-07-17 13:00:14 skrev Joel Reymont : > Folks, > > This is my attempt at creating a time-series database in Erlang. I > specifically want to store a day's worth of data in a separate disk > log and keep the columns separate. You can read more on the whys > here: http://groups-beta.google.com/group/uptick/ > > You can get the data from http://wagerlabs.com/ES4U.csv.gz. Run it > like this: > > tickdb:load("ES4U", "ES4U.csv"). > > I had a version that did not cache open disk logs but used tickdb:add > directly, without going through a gen_server. It ran almost 3x faster! > > Any suggestions on how to optimize this are appreciated. There are > 119,275 trade record in the CSV file. I divide the total run time by > this number to get the average per insert. I would like to do at > least 100 inserts per second. > > The code is part of the open source trading platform that I'm creating. > > Thanks, Joel > > -- > http://wagerlabs.com/uptick > > From chas@REDACTED Sun Jul 17 20:23:03 2005 From: chas@REDACTED (Charles Blair) Date: Sun, 17 Jul 2005 13:23:03 -0500 Subject: two style questions In-Reply-To: Your message of "Sun, 17 Jul 2005 17:36:13 +0200." Message-ID: <20050717182303.515A02E2430@jump.lib.uchicago.edu> > If the state can outlive the process holding it (which is possible > for example if the state is stored in ets (and another process > owns the ets table, as explained above), then it can make sense > to use the registered name of a process. Otherwise, using the > pid seems quite sensible, except for the issue of having to > serialize the pid and present it in the external protocol. > Using a symbolic name migh look a bit better there. initially i had thought to use pid_to_list, but was discouraged from using it because of the warning in the documentation: "This BIF is intended for debugging and for use in the Erlang operating system. It should not be used in application programs." > You might even want to use another kind of registry than the > built-in one, since you'd otherwise have to call list_to_atom/1 > based on information in the http message. This is a potential > memory leak, as atoms are never garbage collected. thanks for the reminder: i'd forgotten that, or rather, remembered it elsewhere but not here. it seems best, then, to create a table that stores symbolic name / PID associations, returning the former to the client. thanks for your response. From vlad_dumitrescu@REDACTED Sun Jul 17 20:25:44 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Sun, 17 Jul 2005 20:25:44 +0200 Subject: Erlang scanner returning column numbers References: <20050717161908.84574469EC@bang.trapexit.org> Message-ID: From: "Ulf Wiger" > I once wrote a small hack in erl_scan.erl that maintained > column numbers. I got as far as verifying that this small > hack crashed the compiler. I don't recall where it crashed, > and didn't really have the time or energy to investigate. Okay, then it's the hard way for me! :-) Thanks anyway! /Vlad From david.nospam.hopwood@REDACTED Sun Jul 17 23:31:20 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Sun, 17 Jul 2005 22:31:20 +0100 Subject: Meyer, OO and concurrency In-Reply-To: References: <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> <42D6C11D.6000007@blueyonder.co.uk> <20050714214105.GJ644@feeble.motivity.ca> <42D6F8CE.40701@blueyonder.co.uk> <42D997A1.9090406@blueyonder.co.uk> Message-ID: <42DACE28.7060302@blueyonder.co.uk> Ulf Wiger wrote: > Den 2005-07-17 01:26:25 skrev David Hopwood > : > >> Various actor languages did have selective receive, for example the >> "communication handlers" of Act2 and Act3. In any case, selective >> receive does not affect the expressive power of a message passing >> model: it can be simulated, albeit inefficiently, by receiving >> unconditionally and then re-sending the message to self if necessary. > > Actually, you can't do it like that. That could for example > cause your process to go into a tight loop The loop doesn't have to be tight. > consuming and re-sending the same unwanted message while waiting for > some other message, which would have a rather nasty effect on > cpu load and timing characteristics. Right, but that's an efficiency issue, not an expressiveness one, and therefore it couldn't be a fundamental difference between models (i.e. it could not be the case that the actor model forbids selective receive). -- David Hopwood From mickael.remond@REDACTED Mon Jul 18 00:21:43 2005 From: mickael.remond@REDACTED (=?ISO-8859-1?Q?Micka=EBl_R=E9mond?=) Date: Mon, 18 Jul 2005 00:21:43 +0200 Subject: New trading systems platform In-Reply-To: <42D7FBE4.4040009@hq.idt.net> References: <42D27E1B.6030109@erlang-fr.org> <42D7FBE4.4040009@hq.idt.net> Message-ID: <9A7E7593-1316-4F75-BE47-B2DB9BE265C0@erlang-fr.org> Le 15 juil. 05 ? 20:09, Serge Aleynikov a ?crit : > Do you know if the Bets source is available for download? No. It has not been released yet. The idea however proves that it is possible to plug several storage backend to Mnesia to benefit from different characteristics. Very interesting. -- Micka?l R?mond From ulf@REDACTED Mon Jul 18 00:46:26 2005 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 18 Jul 2005 00:46:26 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42DACE28.7060302@blueyonder.co.uk> References: <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> <42D6C11D.6000007@blueyonder.co.uk> <20050714214105.GJ644@feeble.motivity.ca> <42D6F8CE.40701@blueyonder.co.uk> <42D997A1.9090406@blueyonder.co.uk> <42DACE28.7060302@blueyonder.co.uk> Message-ID: Den 2005-07-17 23:31:20 skrev David Hopwood : > Ulf Wiger wrote: >> Den 2005-07-17 01:26:25 skrev David Hopwood >> : >> >>> Various actor languages did have selective receive, for example the >>> "communication handlers" of Act2 and Act3. In any case, selective >>> receive does not affect the expressive power of a message passing >>> model: it can be simulated, albeit inefficiently, by receiving >>> unconditionally and then re-sending the message to self if necessary. >>> >> Actually, you can't do it like that. That could for example >> cause your process to go into a tight loop > > The loop doesn't have to be tight. Doesn't have to be, but could be. A good programmer would hopefully think it through and write code that doesn't risk wrecking the characteristics of the system. But how do you make sure that this doesn't happen? Would you enforce a 'sleep' every time someone resends to self? How would you enforce that? >> consuming and re-sending the same unwanted message while waiting for >> some other message, which would have a rather nasty effect on >> cpu load and timing characteristics. > > Right, but that's an efficiency issue, not an expressiveness one, and > therefore it couldn't be a fundamental difference between models (i.e. > it could not be the case that the actor model forbids selective receive). It's not just an efficiency issue. It can potentially cause the system to misbehave so badly that it's no longer usable for its primary mission. Even worse, this behaviour can be timing-dependent, such that it doesn't show up even in fairly rigorous testing, but then happens in the field, due to unforeseen interworking between components. I will not say that it's impossible to write safe code using an event-based programming model, but there are some really nasty pitfalls, and mortal programmers fall into them all the time. I would go as far as to say that an event-based programming model is outright dangerous for many complex concurrency patterns. Forgive me if I come across as cranky. This actor programming stuff is intriguing. I have yet to reach the point where the benefits as compared to the 'old erlang way' are revealed to me, convincing me that it's actually a better way to program. Being on vacation, I confess I've not been able to follow all references given. One thing I often find lacking in other paradigm is proper support for handling really tricky state machine programming, for example. The problem with these programs is that if the underlying support is deficient, complexity explodes in your face, leaving the program nearly impossible to write. Concurrent Haskell has some really weird and wonderful approaches to concurrency -- such as the 'serializable transaction monad' (http://homepages.inf.ed.ac.uk/wadler/linksetaps/slides/peyton-jones.ppt) There are as yet some efficiency worries surrounding it, but at least here it's fairly easy to see the pot of gold at the end of the rainbow! (: /Uffe /Uffe From david.nospam.hopwood@REDACTED Mon Jul 18 01:03:49 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Mon, 18 Jul 2005 00:03:49 +0100 Subject: Meyer, OO and concurrency In-Reply-To: References: <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> <42D6C11D.6000007@blueyonder.co.uk> <20050714214105.GJ644@feeble.motivity.ca> <42D6F8CE.40701@blueyonder.co.uk> <42D997A1.9090406@blueyonder.co.uk> <42DACE28.7060302@blueyonder.co.uk> Message-ID: <42DAE3D5.9090001@blueyonder.co.uk> Ulf Wiger wrote: > Den 2005-07-17 23:31:20 skrev David Hopwood > : >> Ulf Wiger wrote: >>> Den 2005-07-17 01:26:25 skrev David Hopwood >>> : >>> >>>> Various actor languages did have selective receive, for example the >>>> "communication handlers" of Act2 and Act3. In any case, selective >>>> receive does not affect the expressive power of a message passing >>>> model: it can be simulated, albeit inefficiently, by receiving >>>> unconditionally and then re-sending the message to self if necessary. >>>> >>> Actually, you can't do it like that. That could for example >>> cause your process to go into a tight loop >> >> The loop doesn't have to be tight. > > Doesn't have to be, but could be. A good programmer would > hopefully think it through and write code that doesn't risk > wrecking the characteristics of the system. But how do you > make sure that this doesn't happen? Would you enforce a > 'sleep' every time someone resends to self? How would you > enforce that? It can't be enforced. Anyway, I wasn't suggesting that it *should* be done this way; only that the fact that it *could* be done this way implies that the presence of selective receive is a matter of efficiency rather than expressive power. Of course efficiency is important. Knowing which features are equivalent in expressive power is important for security analysis, which is a big motivating factor in the design of the actor model. > It's not just an efficiency issue. It can potentially cause the > system to misbehave so badly that it's no longer usable for its > primary mission. Efficiency issues can sometimes do that. > Forgive me if I come across as cranky. This actor programming > stuff is intriguing. I have yet to reach the point where the > benefits as compared to the 'old erlang way' are revealed to > me, convincing me that it's actually a better way to program. Actors really isn't all that different from Erlang; the apparent differences are exaggerated by the fact that most of the papers on actors are talking about kernel languages or the pure actor model, which is designed to have an absolutely minimal set of concepts. I'll have a look at Concurrent Haskell's 'serializable transaction monad', thanks for the reference. -- David Hopwood From ok@REDACTED Mon Jul 18 03:43:34 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 18 Jul 2005 13:43:34 +1200 (NZST) Subject: Meyer, OO and concurrency Message-ID: <200507180143.j6I1hYxX184022@atlas.otago.ac.nz> "Valentin Micic" asked, quote reasonably: What are the limitations of [the process] dictionary, and *why* [has] it received a bad reputation? The Erlang book itself says "The process dictionary should be used with extreme care" and "their use leads to unclear programs and should be avoided wherever possible" and "we do not wish to encourage its use", which is hardly a recommendation. let's look at the interface, as described in that book. put(Key, Value) -> Result = get(Key), %process_dictionary.store(Key, Value), Result. get(Key) when %process_dictionary.includes_key(Key) -> %process_dictionary.fetch(Key); get(_) -> undefined. get() -> [{K1,V1}, ..., {Kn,Vn}]. /* returns a {Key,Value} pair for each key in the dictionary, the order of the results is not specified. In particular, it is nowhere guaranteed that get() == get(). */ get_keys(Value) -> [Key || {Key,Val} <- get(), Val == Value]. erase(Key) when %process_dictionary.includes_key(Key) -> Result = %process_dictionary.fetch(Key), %process_dictionary.remove_key(Key), Result; erase(_) -> undefined. erase() -> Result <- get(), [erase(Key) || {Key,_} <- Result], Result. To start with, there are some quirky aspects to this. There is (or at any rate was) no easy way to tell the difference between a key which has the value 'undefined' associated with it and a key which has NO value associated with it. There's a hard way, namely to call get() and look in the result, but that's not pleasant. Suppose someone wants to use 'fred' as a local variable in some calculation. Being a careful programmer, our friend realises that something else in the same process may also be trying to use 'fred', so our friend arranges to put the original value back afterwards: Old_Fred = put(fred, New_Value), ..., put(fred, Old_Fred) Now, if this codes is invoked when 'fred' *is* in use, all is well, but if it is invoked when 'fred' is *not* in use, 'fred' will be defined afterwards when it wasn't previously. So our friend tries to fix this: Old_Fred = put(fred, New_Value), ... if Old_Fred == undefined -> erase(fred) ; Old_Fred /= undefined -> put(fred, Old_Fred) end which is fine until the day when the outer code has defined 'fred' to be 'undefined'. A fairly simple test of any design like this is "Can you write a function with(Key, Value, Closure) which temporarily binds Key to Value then invokes Closure and on the way out (whether by normal return or by exception) restores Key to its original status in the dictionary?" Thanks to 'catch', you _can_ ensure that a clean-up action will be performed even if there is an exception. But you cannot do the rest of the job without having to pick up the *whole* process dictionary (which could be very large) using get(). Another version of with() would be with([{K1,V1},...,{Kn,Vn}], Closure), but it suffers the same problem: you have to pick up the *whole* process dictionary to discover which keys should be restored and which erased. Perhaps more interesting, the book provides NO PERFORMANCE GUARANTEES. Nor can I find any discussion of performance in the on-line documentation at www.erlang.se. For all we are told to the contrary, the process dictionary might use linear search or worse. For example, in Prolog, the 'recorded' data base only uses the principal functor of the key for indexing, and if Erlang did the same, then using keys a..z would be efficient but keys {a}..{z} would be inefficient. Nor are we advised how much copying gets done. So we have *NO* idea how to use this facility efficiently. As far as I can tell from a quick scan of erl_process_dict.[ch], process dictionaries are some kind of expanding hash table and hashing depends on the whole key. They should be pretty good, BUT the fact remains that this is not actually promised anywhere that I can find, and there are still issues about copying. Finally, there's a fairly lethal objection from a software engineering point of view. In effect, Erlang with the process dictionary offers us a language with two levels of variable scope: there is an outer global level of mutable variables each function has its own set of read-only variables Now when you are debugging, you may be very interested in finding out how a variable came to have a particular value. For ordinary Erlang variables, no worries. For conventional programming languages, relatively few worries. For example, cscope(1) can find references to global variables in C, and there is even a free slicing program for C that can track down where something might have been set. (And failing that, there are watchpoints in debuggers like dbx.) But in Erlang, the "names" of global variables are the *values* of run-time expressions, and while it *may* be easy to find assignments to a particular variable, in general you have to look at EVERY place where you call put/2. If there is anything in the debugger to set a watchpoint on a process dictionary entry, I've missed it. *With* a proper "effects" system (like FX-80), something very like the process dictionary could be quite useful. As it is, the process dictionary interface is an invitation to (inadvertent) "spaghetti data". There isn't even any run-time type distinction between the key and the value; swap the arguments in a call to put/2 and you may never notice. Note that this interacts with the lack of a simple way to tell whether a key is defined (and the corresponding fact that get/1 never fails): if you *meant* to write put(one, 2) ... get(one)+1 and by mistake wrote put(2, one) ... get(one)+1 instead, you are NOT told that 'one' isn't in the dictionary, you are told that you have 'badarith'. While I personally think that functional languages are better off without assignment statements (unless 'tamed' by an effects system), you don't have to agree with me about that to accept that the criticisms I have put forward in this message show that the process dictionary, as currently defined, is not as good an interface as it should have been, and is risky enough to avoid unless you have a REALLY good reason to use it. From vel.accel@REDACTED Mon Jul 18 08:46:32 2005 From: vel.accel@REDACTED (travlr) Date: Mon, 18 Jul 2005 02:46:32 -0400 Subject: New trading systems platform Message-ID: <1e52e08805071723466fede3e0@mail.gmail.com> Hi there, I've followed J.Reymont over here from the trading forum, so I want to thank him for introducing me to erlang. My first impression is that Erlang seems to be a dream for RT, DC-HPC, so I want to thank the developers for it's utility and it's OpenSource. The discussion here about efficiency for large data set retrieval and number crunching, led me to want to share my favorite libraries of choice. I'm a Python aficionado because of it's remarkable scripting ease, and it's near transparency over "C" libraries (amongst other reasons)... Here are some excellent packages for efficiency and easy porting: For DB, Pytables is a thin wrapper over the NCSA's HDF5 library: http://pytables.sourceforge.net/html/WelcomePage.html http://hdf.ncsa.uiuc.edu/HDF5/ For Number Crunching I use the STCI's Numarray package, as well as Numeric/Scipy. http://www.stsci.edu/resources/software_hardware/numarray Pytables is quite amazing and also built with Numarray/Numeric so is therefore nearly seamless for retrieval and then crunching: The desires of all the situations mentioned in this thread, regarding DB's and Number crunching, are contained very well in these packages. I'd be curious to hear comments. Regards Peter From vel.accel@REDACTED Mon Jul 18 08:46:55 2005 From: vel.accel@REDACTED (travlr) Date: Mon, 18 Jul 2005 02:46:55 -0400 Subject: New trading systems platform Message-ID: <1e52e088050717234653b123b7@mail.gmail.com> Hi there, I've followed J.Reymont over here from the trading forum, so I want to thank him for introducing me to erlang. My first impression is that Erlang seems to be a dream for RT, DC-HPC, so I want to thank the developers for it's utility and it's OpenSource. The discussion here about efficiency for large data set retrieval and number crunching, led me to want to share my favorite libraries of choice. I'm a Python aficionado because of it's remarkable scripting ease, and it's near transparency over "C" libraries (amongst other reasons)... Here are some excellent packages for efficiency and easy porting: For DB, Pytables is a thin wrapper over the NCSA's HDF5 library: http://pytables.sourceforge.net/html/WelcomePage.html http://hdf.ncsa.uiuc.edu/HDF5/ For Number Crunching I use the STCI's Numarray package, as well as Numeric/Scipy. http://www.stsci.edu/resources/software_hardware/numarray Pytables is quite amazing and also built with Numarray/Numeric so is therefore nearly seamless for retrieval and then crunching: The desires of all the situations mentioned in this thread, regarding DB's and Number crunching, are contained very well in these packages. I'd be curious to hear comments. Regards Peter From John-Olof.Bauner@REDACTED Mon Jul 18 10:03:25 2005 From: John-Olof.Bauner@REDACTED (John-Olof Bauner) Date: Mon, 18 Jul 2005 10:03:25 +0200 Subject: Meyer, OO and concurrency In-Reply-To: <42DACE28.7060302@blueyonder.co.uk> References: <42D29A7F.1030306@possibility.com> <42D2BB96.50703@blueyonder.co.uk> <42D2C56F.80502@possibility.com> <42D2E7D1.4010103@blueyonder.co.uk> <42D32B9C.2000108@possibility.com> <1121157747.27930.33.camel@hymir.newport-networks.com> <42D6918E.8000705@blueyonder.co.uk> <42D6C11D.6000007@blueyonder.co.uk> <20050714214105.GJ644@feeble.motivity.ca> <42D6F8CE.40701@blueyonder.co.uk> <42D997A1.9090406@blueyonder.co.uk> <42DACE28.7060302@blueyonder.co.uk> Message-ID: <42DB624D.4040106@ericsson.com> David Hopwood wrote: > Ulf Wiger wrote: > >> Den 2005-07-17 01:26:25 skrev David Hopwood >> : >> >>> Various actor languages did have selective receive, for example the >>> "communication handlers" of Act2 and Act3. In any case, selective >>> receive does not affect the expressive power of a message passing >>> model: it can be simulated, albeit inefficiently, by receiving >>> unconditionally and then re-sending the message to self if necessary. >> >> >> Actually, you can't do it like that. That could for example >> cause your process to go into a tight loop > > > The loop doesn't have to be tight. > For an actor in ObjectTime you can use defer for signals that comes in when you don't want them. You should not forget to put these signals back to the queue though. But as Ulf argues the paradigm is poor as compared to selective receive. >> consuming and re-sending the same unwanted message while waiting for >> some other message, which would have a rather nasty effect on >> cpu load and timing characteristics. > > > Right, but that's an efficiency issue, not an expressiveness one, and > therefore it couldn't be a fundamental difference between models (i.e. > it could not be the case that the actor model forbids selective receive). > There is an underlying system, RTS/OS, and if it allows selective receive and the model allows it then there should not be any problem. John-Olof From tpatro@REDACTED Mon Jul 18 10:51:39 2005 From: tpatro@REDACTED (Tamas Patrovics) Date: Mon, 18 Jul 2005 10:51:39 +0200 Subject: ESense 1.6 released In-Reply-To: References: Message-ID: Changes https://sourceforge.net/project/shownotes.php?group_id=139206&release_id=342958 Files https://sourceforge.net/projects/esense/ Homepage http://esense.sourceforge.net/ From taj.khattra@REDACTED Mon Jul 18 12:36:44 2005 From: taj.khattra@REDACTED (Taj Khattra) Date: Mon, 18 Jul 2005 03:36:44 -0700 Subject: Meyer, OO and concurrency In-Reply-To: References: Message-ID: <57a21f73050718033660447b1b@mail.gmail.com> On 7/14/05, Joe Armstrong (AL/EAB) wrote: > Threads/process in C++ are usually not C++ Threads or processes but thinly disguised OS > threads or processes. > > The problem with this is that the semantics of the program is *amazingly* UNDEFINED. from http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2004/n1680.pdf: ``Unfortunately, programming multithreaded applications of any kind in C++ remains a black art. Properties critical for reliable, efficient, and correct multithreaded execution are left unspecified. [...] many questions that programmers have about how to predict and control multithreaded code simply do not have answers. Superstition, myths, bad advice, and "hey, it worked for me" stories are rampant.'' this is due to the lack of a clearly specified shared memory model (a la java, until jsr-133) in the presence of multiple threads and instruction re-orderings due to compiler and processor "optimizations". follow the links at http://www.hpl.hp.com/personal/Hans_Boehm/c++mm/ for the gory details, if you dare. :) From rprice@REDACTED Mon Jul 18 14:44:32 2005 From: rprice@REDACTED (Roger Price) Date: Mon, 18 Jul 2005 14:44:32 +0200 (CEST) Subject: Three wishes Message-ID: Wish 1: No scope extrusion Erlang allows variable name scope extrusion from if, case or receive primitives, "Concurrent programming in Erlang", section 2.8.1. I would like an option which supresses this scope extrusion, and makes the variables introduced within an if, case or receive unknown outside. I would also like this option to be the default compiler behaviour. The current scope extrusion is counter-intuitive, and adds unwanted complexity. Is there any case in which it is a "must have"? Wish 2: Block comments Some programming languages provide line comments, e.g. Erlang, TeX, Scheme, whereas others provide block comments, e.g. ML, Haskell, SGML. Both forms of comment have their uses, and I would like to be able to use block comments as well as line comments in an Erlang program. Wish 3: ISO latin 9 If you are using the ISO latin 1 character set and can see the Euro symbol, then you are not using ISO latin 1, but rather ISO latin 9 as defined by ISO/IEC 8859-15 [1]. For example, on my Dell laptop the following function call displays a Euro: 3> io:format ("~c~n", [$\244]) . ? ok Confusion between the two is very common. I would like Erlang to be an ISO latin 9 language. None of these changes should affect the time or space efficiency of a Erlang compiler. Best Regards, Roger [1] "Draft available at http://www.evertype.com/standards/iso8859/8859-15-en.pdf. See also http://www.cs.tut.fi/~jkorpela/latin9.html for a comparison with ISO Latin 1. From michael206@REDACTED Mon Jul 18 16:12:50 2005 From: michael206@REDACTED (Michael Leonhard) Date: Mon, 18 Jul 2005 23:12:50 +0900 Subject: mnesia query In-Reply-To: <007101c58adc$838050f0$3000a8c0@INSWITCH244> References: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> <035701c5888d$ed3cacb0$3000a8c0@INSWITCH244> <036501c58895$ac331260$3000a8c0@INSWITCH244> <002901c58ad3$2d2fff60$3000a8c0@INSWITCH244> <007101c58adc$838050f0$3000a8c0@INSWITCH244> Message-ID: <21f8b6e2050718071244aaab50@mail.gmail.com> I was interested in learning more about what kind of product Mr. Bello might be developing. I went to his company website and clicked on the menu bar link titled "Our Technology", http://www.inswitch.us/english/technologies/. It is surprising that this page describes Erlang/OTP but makes no mention of the creators of the technology. Ericsson is mentioned as "largest user of Erlang". I always thought that Ericsson created Erlang and OTP and licenses it with an open source license. The Inswitch website gives me the impression that Erlang/OTP was created by Inswitch. Is this right? -Michael michael206@REDACTED On 7/17/05, Sebastian Bello wrote: > Thanks! > Sebastian- From jozsef.berces@REDACTED Mon Jul 18 16:18:51 2005 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces_=28LA/ETH=29?=) Date: Mon, 18 Jul 2005 16:18:51 +0200 Subject: NBAP Message-ID: In March, there was a short discussion about compiling the NBAP ASN.1 source. The problem was with some integer check. I just tried to compile 3GPP TS 25.433 V5.12.0 with OTP R10B-6 and got the same error. Is there any plan to remove the bug? Thanks in advance, Jozsef <> -------------- next part -------------- A non-text attachment was scrubbed... Name: asn1ct.log Type: application/octet-stream Size: 19268 bytes Desc: asn1ct.log URL: From thomasl_erlang@REDACTED Mon Jul 18 16:31:35 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 18 Jul 2005 07:31:35 -0700 (PDT) Subject: Three wishes In-Reply-To: Message-ID: <20050718143135.75938.qmail@web34402.mail.mud.yahoo.com> --- Roger Price wrote: > I would > like an option which supresses this scope extrusion, > and makes the > variables introduced within an if, case or receive > unknown outside. It's easy enough to provide, e.g., transform each SwitchExpr construct above into: (fun() -> SwitchExpr end)() Having this as an optional parse transform would be okay with me, even though I probably wouldn't bother to use it. (The warnings you get when you export variables out of case/if/receive is enough, thanks :-) > I > would also like this option to be the default > compiler behaviour. I suspect actually breaking code that uses the old convention would be too radical though ... > The > current scope extrusion is counter-intuitive, and > adds unwanted > complexity. Is there any case in which it is a > "must have"? That depends on your intuition, I guess. I'll agree that the current scoping rules add some complexity, at least for compilers and code processors. On the other hand, the underlying problem remains after you've made your restriction. You can still write: X = f(Y=g(A), Z=h(B)) and as a consequence have X, Y and Z bound after the call. (Basically, an Erlang expression returns a value and a new environment, not just a value.) Best, Thomas __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From ulf@REDACTED Mon Jul 18 17:16:54 2005 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 18 Jul 2005 17:16:54 +0200 Subject: mnesia query In-Reply-To: <21f8b6e2050718071244aaab50@mail.gmail.com> References: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> <035701c5888d$ed3cacb0$3000a8c0@INSWITCH244> <036501c58895$ac331260$3000a8c0@INSWITCH244> <002901c58ad3$2d2fff60$3000a8c0@INSWITCH244> <007101c58adc$838050f0$3000a8c0@INSWITCH244> <21f8b6e2050718071244aaab50@mail.gmail.com> Message-ID: Den 2005-07-18 16:12:50 skrev Michael Leonhard : > I was interested in learning more about what kind of product Mr. Bello > might be developing. I went to his company website and clicked on the > menu bar link titled "Our Technology", > http://www.inswitch.us/english/technologies/. It is surprising that > this page describes Erlang/OTP but makes no mention of the creators of > the technology. Ericsson is mentioned as "largest user of Erlang". I > always thought that Ericsson created Erlang and OTP and licenses it > with an open source license. The Inswitch website gives me the > impression that Erlang/OTP was created by Inswitch. Is this right? Hmm. It would be appropriate with at least a link to the www.erlang.org website. /Uffe From matthias@REDACTED Mon Jul 18 18:35:32 2005 From: matthias@REDACTED (Matthias Lang) Date: Mon, 18 Jul 2005 18:35:32 +0200 Subject: Inswitch being less than clear about where Erlang came from In-Reply-To: References: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> <035701c5888d$ed3cacb0$3000a8c0@INSWITCH244> <036501c58895$ac331260$3000a8c0@INSWITCH244> <002901c58ad3$2d2fff60$3000a8c0@INSWITCH244> <007101c58adc$838050f0$3000a8c0@INSWITCH244> <21f8b6e2050718071244aaab50@mail.gmail.com> Message-ID: <17115.55892.814252.703855@antilipe.corelatus.se> Ulf Wiger writes: > Den 2005-07-18 16:12:50 skrev Michael Leonhard : > > > I was interested in learning more about what kind of product Mr. Bello > > might be developing. I went to his company website and clicked on the > > menu bar link titled "Our Technology", > > http://www.inswitch.us/english/technologies/. It is surprising that > > this page describes Erlang/OTP but makes no mention of the creators of > > the technology. Ericsson is mentioned as "largest user of Erlang". I > > always thought that Ericsson created Erlang and OTP and licenses it > > with an open source license. The Inswitch website gives me the > > impression that Erlang/OTP was created by Inswitch. Is this right? > Hmm. It would be appropriate with at least a link to the > www.erlang.org website. Reading that Inswitch page gave me a strange feeling of deja vu: Inswitch say The Erlang FAQ says ---------------------------------------------------------------------- Who uses Erlang? Who uses Erlang for product development? The largest user of Erlang is The largest user of Erlang is Ericsson. Ericsson uses it to (surprise!) Ericsson. Ericsson use it to write software used in write software used in telecommunications systems. telecommunications systems. Many projects have used it, a Many (dozens) projects have used it, a particularly large one is the particularly large one is the extremely scaleable AXD301 ATM extremely scaleable AXD301 ATM switch. AXD301 has several switch. AXD301 has several hundred people working on it hundred people working on it and the code volume has reached and the code volume has reached about 850 kloc of Erlang (and about 850 kloc of Erlang (and 1 Mloc of C/C++). 1 Mloc of C/C++). Matthias (the FAQ maintainer) From nm@REDACTED Mon Jul 18 18:49:39 2005 From: nm@REDACTED (Gaspar Chilingarov) Date: Mon, 18 Jul 2005 21:49:39 +0500 Subject: jungerl/zlib changes In-Reply-To: <6B25DCA8-2BFA-4784-AE97-EBF11018FA44@rogvall.com> References: <42D7C75A.9020208@web.am> <6B25DCA8-2BFA-4784-AE97-EBF11018FA44@rogvall.com> Message-ID: <42DBDDA3.4070803@web.am> Tony Rogvall wrote: > > 15 jul 2005 kl. 16.25 skrev Gaspar Chilingarov: > >> Hi all! >> >> >> >> first of all - imho open_port automatically loads external driver, >> and if I leave it load() call, it will fail for subsequent calls to >> zlib:compress or zlib:uncompress - because module is already loaded. >> >> seconds change - is to have receive command , which will cleanup port >> exit message from process queue, if such message exists. > > > You are right, but the zlib library exists already in the otp (since > R9C-0?). This means that > we should stop using the jungerl version (delete from jungerl?). > Check the otp source code. > > Regards > > /Tony I've found it already. Just notices some zlib lines when building solaris version of erlang :) But anyway second question remains -- should zlib cleanup caller process message queue from port's 'EXIT' message? Are there any useful point of having calling process receive this message -- this can be somehow confusing, because if my process wants to receive 'EXIT' messages, lookup them in ets table and somehow restart died processes(or reopen ports) -- it will be confused receiving message from unregistred process, which may be error condition for the program. -- Gaspar Chilingarov System Administrator t +37491 419763 (mob) t +37410 240399 (office) w www.web.am i 63174784 e nm@REDACTED From mikael.karlsson@REDACTED Mon Jul 18 19:45:43 2005 From: mikael.karlsson@REDACTED (Mikael Karlsson) Date: Mon, 18 Jul 2005 19:45:43 +0200 Subject: NBAP In-Reply-To: References: Message-ID: <200507181945.43011.mikael.karlsson@creado.com> The OTP folks fixed our bug allright, and we compile the complete NBAP ASN.1 source now (quite cool actually). Maybe the fix did not make it into R10B-6 ( or maybe you found a new bug from a later version of the NBAP spec. ) Since you seem to be "Ericsson internal" I guess they could provide you with a descent patch if you ask them :-) Mikael Mon 18 July 2005 16:18 J?zsef B?rces (LA/ETH) wrote: > In March, there was a short discussion about compiling the NBAP ASN.1 > source. The problem was with some integer check. > > I just tried to compile 3GPP TS 25.433 V5.12.0 with OTP R10B-6 and got the > same error. > > Is there any plan to remove the bug? > > Thanks in advance, > Jozsef > > <> From david.nospam.hopwood@REDACTED Mon Jul 18 20:33:02 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Mon, 18 Jul 2005 19:33:02 +0100 Subject: Three wishes In-Reply-To: References: Message-ID: <42DBF5DE.1020608@blueyonder.co.uk> Roger Price wrote: > Wish 3: ISO latin 9 > > If you are using the ISO latin 1 character set and can see the Euro > symbol, then you are not using ISO latin 1, but rather ISO latin 9 as > defined by ISO/IEC 8859-15 [1]. For example, on my Dell laptop the > following function call displays a Euro: > > 3> io:format ("~c~n", [$\244]) . > ? > ok > > Confusion between the two is very common. I would like Erlang to be an ISO > latin 9 language. Please don't. If strings are to be in a single ASCII-compatible charset, it should be UTF-8. (The difficulties in using a charset with variable-length character encodings are typically overstated.) Better would be a thorough review of the language and libraries to provide full Unicode support. Specific reasons not to use ISO-Latin-9 are that it's not compatible with ISO-Latin-1, and that in practice hardly anyone uses it. A more common, mostly ISO-Latin-1-compatible charset that includes the ? character is Windows CP1252+euro (http://www.microsoft.com/globaldev/reference/sbcs/1252.mspx), not that I'm suggesting using that either. [Note that this email got encoded automatically in UTF-8 without me having to do anything special. Besides ?, here are a few other currency characters: ? ? ? ? ? ? ? ?. Try doing that in Latin-9.] -- David Hopwood From eudesh@REDACTED Mon Jul 18 21:40:22 2005 From: eudesh@REDACTED (Eranga Udesh) Date: Tue, 19 Jul 2005 01:40:22 +0600 Subject: Distributed Erlang embeded startup problem In-Reply-To: References: <42C4F466.5080206@ericsson.com> <42C54023.6060703@ericsson.com> Message-ID: Hi, I have to bring to subject up again, since my problem doesn't seems to be solved yet. I have 2 Log servers running in distributed environment. When the master log server goes down, a slave node is started in a seperate computer. When the master node starts again, first time it crashes. Same time the slave node application stop. Since I have activated HEART, the master node restarts and and that time it start correctly. Erlang runs in embedded mode What could be the problem. I have attached my 2 sys.config files. Also, I have given below a cleaned output I get in the erlang.log.1 file. Please help. - Eranga Erlang (BEAM) emulator version 5.4.6 [source] [hipe] [threads:5] . . . =INFO REPORT==== 19-Jul-2005::00:23:00 === "Application takeover." module: omnilog_app . . . =SUPERVISOR REPORT==== 19-Jul-2005::00:23:44 === Supervisor: {<0.160.0>,omnilog_sup} Context: start_error Reason: {already_started,<2420.1452.0>} Offender: [{pid,undefined}, {name,txalarm1_log}, {mfa,{gen_event,start_link,[{global,txalarm1_log}]}}, {restart_type,permanent}, {shutdown,5000}, {child_type,worker}] =CRASH REPORT==== 19-Jul-2005::00:23:44 === crasher: pid: <0.69.0> registered_name: [] error_info: "invalid return value from omnilog_app:start({takeover,omni_log@REDACTED},[]) -> {'EXIT',\n {{badmatch,\n {error,shutdown}},\n [{omnilog_app,\n start_system,\n 1},\n {application_master,\n start_it_old,\n 4}]}}" initial_call: {application_master,init, [<0.6.0>, <0.68.0>, {appl_data, omni_log, [omni_log], undefined, {omnilog_app,[]}, [omnilog_app, omnilog_sup, omnilog_disklog, omnilog_cfg, omnilog_tty_logger, omnilog, omnilog_fmt], [], infinity, infinity}, {takeover,omni_log@REDACTED}]} ancestors: [<0.68.0>] messages: [{'EXIT',<0.70.0>,normal}] links: [<0.68.0>,<0.6.0>] dictionary: [] trap_exit: true status: running heap_size: 1597 stack_size: 21 reductions: 143 neighbours: =INFO REPORT==== 19-Jul-2005::00:23:44 === application: omni_log exited: "invalid return value from omnilog_app:start({takeover,omni_log@REDACTED},[]) -> {'EXIT',\n {{badmatch,\n {error,shutdown}},\n [{omnilog_app,\n start_system,\n 1},\n {application_master,\n start_it_old,\n 4}]}}" type: permanent Eshell V5.4.6 (abort with ^G) (omni_log@REDACTED)1> {"Kernel pid terminated",application_controller,shutdown} heart: Tue Jul 19 00:23:46 2005: Erlang has closed. heart: Tue Jul 19 00:23:51 2005: Unable to kill old process, kill failed (tried multiple times). heart: Tue Jul 19 00:23:51 2005: Executed "/etc/rc.d/init.d/log.system start". Terminating. -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: sys.config URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: sys.config URL: From ok@REDACTED Tue Jul 19 00:30:04 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 19 Jul 2005 10:30:04 +1200 (NZST) Subject: Three wishes Message-ID: <200507182230.j6IMU4H1198690@atlas.otago.ac.nz> Roger Price wrote: Erlang allows variable name scope extrusion from if, case or receive primitives, "Concurrent programming in Erlang", section 2.8.1. "Scope extrusion" is a bad way to think about Erlang. A better way to think about it is to think of there being a *single* scope level for *all* variables in a clause; a variable which is defined in some but not all branches of a conditional form is unavailable rather than out of scope. I would like an option which supresses this scope extrusion, and makes the variables introduced within an if, case or receive unknown outside. I would also like this option to be the default compiler behaviour. This would massively break backwards compatibility, all in the name of adding a rather dubious feature (Erlang variables are not otherwise locally scoped, the proposed begin ... end feature not having been adopted). The current scope extrusion is counter-intuitive, and adds unwanted complexity. It is perfectly intuitive provided you first form your intuition. It is exactly what you get in Prolog and other related languages. And it does not *add* complexity, it *removes* it: there is no notion of nested scopes in the language, only that certain variables are known to have paths on which they are not defined. Is there any case in which it is a "must have"? Why yes, in the common case of things like if X > Y -> Sign = 1, Abs = X - Y ; X < Y -> Sign = -1, Abs = Y - X ; X ==Y -> Sign = 0, Abs = 0 end, ... use Sign and Abs ... where the *point* of the conditional is to bind several variables rather than to compute a value. Wish 2: Block comments Some programming languages provide line comments, e.g. Erlang, TeX, Scheme, whereas others provide block comments, e.g. ML, Haskell, SGML. Both forms of comment have their uses, and I would like to be able to use block comments as well as line comments in an Erlang program. And Erlang's ancestors had PL/I-style /*...*/ comments too. This is a "feature" that was *removed* in the history of the language. See the Ada Quality an Style Guidelines and the Ada Rationale for reasons why PL/I-style comments are not such a good idea as all that. (If you think that they are good for commenting out chunks of code, you are sadly mistaken; they are extremely error prone for that use.) Wish 3: ISO latin 9 Confusion between the two is very common. I would like Erlang to be an ISO latin 9 language. What does it mean to "be" an ISO Latin 9 language? What about people like me? On none of my machines is there a locale that supports *both* 8859-15 *and* my country. I really have no choice but to use Latin 1. (What would *really* suit my country would be Latin 4 or Latin 6, but there is no locale available to me which uses those either.) The intention was expressed many years ago that Erlang should become a Unicode language, and I for one have proposed an -encoding(<>). declaration, so that -encoding(iso_8859_15). should suit you. None of these changes should affect the time or space efficiency of a Erlang compiler. Support for Latin-0, or anything outside Latin-1, will require some fairly major changes to allow general Unicode characters in atoms and fairly major rework of the lexical analyser. There are also some rather serious design questions about the relationship between strings and binaries. (It's not just the compiler that's affected by "being an ISO Latin 9 language", it's the runtime and libraries as well.) These things HAVE to come, but it would be a pity to spend time on one specific 8-bit character set instead of going straight for the big one. (After all, the first 256 characters of Unicode are identical to Latin 1, not to Latin 0.) From klacke@REDACTED Tue Jul 19 10:56:20 2005 From: klacke@REDACTED (Claes Wikstom) Date: Tue, 19 Jul 2005 10:56:20 +0200 Subject: ibits.erl Message-ID: <42DCC034.6080806@hyber.org> Just wrote this code, I use it to keep the size down on records in large mnesia tables. Enjoy /klacke -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ibits.erl URL: From tim@REDACTED Tue Jul 19 13:25:35 2005 From: tim@REDACTED (Tim Bates) Date: Tue, 19 Jul 2005 20:55:35 +0930 Subject: Directory structure for non-OTP projects Message-ID: <42DCE32F.9090604@bates.id.au> Hi folks, I'm trying to set up the directory structure for development of my project. I'm trying to follow the recommendations in the OTP Design Principles but I'm unsure where to get or how to adapt the Makefiles etc for use with my own code. I'd also like to use the OTP test_server. Has anyone got recommendations (or better, examples) of how to set up a development environment? I'm using Linux and vim. Thanks, Tim. -- Tim Bates tim@REDACTED From klacke@REDACTED Tue Jul 19 14:18:26 2005 From: klacke@REDACTED (Claes Wikstom) Date: Tue, 19 Jul 2005 14:18:26 +0200 Subject: Directory structure for non-OTP projects In-Reply-To: <42DCE32F.9090604@bates.id.au> References: <42DCE32F.9090604@bates.id.au> Message-ID: <42DCEF92.7080408@hyber.org> Tim Bates wrote: > Hi folks, > I'm trying to set up the directory structure for development of my > project. I'm trying to follow the recommendations in the OTP Design > Principles but I'm unsure where to get or how to adapt the Makefiles etc > for use with my own code. I'd also like to use the OTP test_server. Has > anyone got recommendations (or better, examples) of how to set up a > development environment? I'm using Linux and vim. I don't like the OTP build environment one bit. The Makefiles are hard to read and debug. If your project is reasonably small you could take a look at the yaws build environment. It's easier/more naive than otp and doesn't use any bootscripts at all. /klacke From ola.a.andersson@REDACTED Tue Jul 19 14:45:36 2005 From: ola.a.andersson@REDACTED (Ola Andersson A (AL/EAB)) Date: Tue, 19 Jul 2005 14:45:36 +0200 Subject: Three wishes Message-ID: <148408C0A2D44A41AB295D74E18399755DBA30@esealmw105.eemea.ericsson.se> > Richard A. O'Keefe wrote:> > > Roger Price wrote: > Erlang allows variable name scope extrusion from if, > case or receive > primitives, "Concurrent programming in Erlang", section 2.8.1. > > "Scope extrusion" is a bad way to think about Erlang. > A better way to think about it is to think of there being a *single* > scope level for *all* variables in a clause; > a variable which is defined in some but not all branches of a > conditional > form is unavailable rather than out of scope. > > I would like an option which supresses this scope extrusion, and > makes the variables introduced within an if, case or receive > unknown outside. I would also like this option to be the > default compiler behaviour. > > This would massively break backwards compatibility, all in the name of > adding a rather dubious feature (Erlang variables are not otherwise > locally scoped, the proposed begin ... end feature not having been > adopted). > > The current scope extrusion is counter-intuitive, and adds > unwanted complexity. > > It is perfectly intuitive provided you first form your intuition. > It is exactly what you get in Prolog and other related languages. > And it does not *add* complexity, it *removes* it: there is no notion > of nested scopes in the language, only that certain variables > are known > to have paths on which they are not defined. > > Is there any case in which it is a "must have"? > > Why yes, in the common case of things like > > if X > Y -> Sign = 1, Abs = X - Y > ; X < Y -> Sign = -1, Abs = Y - X > ; X ==Y -> Sign = 0, Abs = 0 > end, > ... use Sign and Abs ... > > where the *point* of the conditional is to bind several variables > rather than to compute a value. > I believe the answer is no. There are no cases where you "must have" exports from case and so on. In the example above there are at least three different ways to solve the problem whitout having to resort to exporting from the if statement. I have _never_ used export from case, if or receive, nor have I ever felt a need for it. I would also like to see a compiler option that prevents such exports. Wasn't there serious plans to actually make these exports, from case, if and receive, compile error some 5-6 years ago? I guess the backwards compatibility issues prevented that from happening. Too bad. I have noticed that people with a background in other programming languages, like C, tend to use this export "feature" rather extensively. In fact, many of them seem unable to write code whitout using it. A kind of mindlock. They usually end up with functions spanning hundreds of lines of code with a multitude of nested case statements exporting variables all over the place. I have seen it in several projects and it is a nightmare to debug. I my view it just encourages bad coding habits, like programming C in Erlang, when exports from case are allowed. I'm aware that it would not prevent bad coding to disallow it, but I believe it would guide people in the right direction. (Away from thinking C that is..) I realize that it would be difficult enforce this, considering the vast existing erlang code base, but a compiler option would be nice though. /OLA. From sebastian@REDACTED Tue Jul 19 15:22:44 2005 From: sebastian@REDACTED (Sebastian Bello) Date: Tue, 19 Jul 2005 10:22:44 -0300 Subject: Fw: Inswitch being less than clear about where Erlang came from Message-ID: <004a01c58c64$f3b52380$3000a8c0@INSWITCH244> I forward Daniel Fernandez' mail (I guess it didn't reach the list). Sebastian- ----- Original Message ----- From: "Daniel Fernandez" To: ; ; Cc: "'Sebastian Bello'" ; Sent: Monday, July 18, 2005 5:43 PM Subject: RE: Inswitch being less than clear about where Erlang came from Hi Michael, Thanks for your comments. I believe you are right, the description on our website is not clear, and it could be interpreted wrongly. But it has never been our intention to show ourselves as Erlang creators. I apologize for any misunderstanding that it may have caused you, and I will have it fixed by tomorrow. And again, thanks for your comments. Regards, daniel IN Switch Solutions Inc. - Prepaid Expertise - Programmable Switches Daniel Fernandez - Director Mobile Phone: 5989 9667353 / daniel@REDACTED -----Original Message----- From: Sebastian Bello [mailto:sebastian@REDACTED] Sent: Monday, July 18, 2005 2:20 PM To: Daniel Fern?ndez Subject: Fw: Inswitch being less than clear about where Erlang came from ----- Original Message ----- From: "Matthias Lang" To: "Ulf Wiger" Cc: Sent: Monday, July 18, 2005 1:35 PM Subject: Re: Inswitch being less than clear about where Erlang came from > Ulf Wiger writes: > > Den 2005-07-18 16:12:50 skrev Michael Leonhard : > > > > > I was interested in learning more about what kind of product Mr. Bello > > > might be developing. I went to his company website and clicked on the > > > menu bar link titled "Our Technology", > > > http://www.inswitch.us/english/technologies/. It is surprising that > > > this page describes Erlang/OTP but makes no mention of the creators of > > > the technology. Ericsson is mentioned as "largest user of Erlang". I > > > always thought that Ericsson created Erlang and OTP and licenses it > > > with an open source license. The Inswitch website gives me the > > > impression that Erlang/OTP was created by Inswitch. Is this right? > > > Hmm. It would be appropriate with at least a link to the > > www.erlang.org website. > > Reading that Inswitch page gave me a strange feeling of deja vu: > > Inswitch say The Erlang FAQ says > ---------------------------------------------------------------------- > Who uses Erlang? Who uses Erlang for product development? > > The largest user of Erlang is The largest user of Erlang is > Ericsson. Ericsson uses it to (surprise!) Ericsson. Ericsson use it to > write software used in write software used in > telecommunications systems. telecommunications systems. > Many projects have used it, a Many (dozens) projects have used it, a > particularly large one is the particularly large one is the > extremely scaleable AXD301 ATM extremely scaleable AXD301 ATM > switch. AXD301 has several switch. AXD301 has several > hundred people working on it hundred people working on it > and the code volume has reached and the code volume has reached > about 850 kloc of Erlang (and about 850 kloc of Erlang (and > 1 Mloc of C/C++). 1 Mloc of C/C++). > > Matthias (the FAQ maintainer) > From joe.armstrong@REDACTED Tue Jul 19 15:37:36 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Tue, 19 Jul 2005 15:37:36 +0200 Subject: Inswitch being less than clear about where Erlang came from Message-ID: > > > I was interested in learning more about what kind of > product Mr. Bello > > > might be developing. I went to his company website and > clicked on the > > > menu bar link titled "Our Technology", > > > http://www.inswitch.us/english/technologies/. It is > surprising that > > > this page describes Erlang/OTP but makes no mention of ********* True - But this page makes no claim that they developed OTP - the link to the page is "Our technology" which I guess should be liberally interpreted as "The technology we use" > the creators of > > > the technology. Ericsson is mentioned as "largest user > of Erlang". I > > > always thought that Ericsson created Erlang and OTP and > licenses it > > > with an open source license. The Inswitch website gives me the ******* > > > impression that Erlang/OTP was created by Inswitch. Is > this right? I didn't get that impression. Bu then I'm biased - Page http://www.inswitch.us/english/aboutus/company/profile/profile.htm Says "With the foundation of IN Switch Solutions, a major software restructure was accomplished. IN Switch Licensed Erlang, (Ericsson-Language) a switch running environment developed and supported by Ericsson. " - pretty clear to me > > > Hmm. It would be appropriate with at least a link to the > > www.erlang.org website. - Yes. It's a shame the Erlang licence did not say - "any web site advertising an Erlang product must display the Erlang logo, suitable hidden so that nobody can find it :-) But my overall reaction was "I'm pleased to discover yet another company who is is using Erlang" rather than " here why haven't they referred to us " Now if anybody from Inswitch is reading, we would appreciate a link to www.erlang.org from your main pages - This will increase our rating in the google link weighting algorithm - oh and good luck with Erlang. /Joe From nm@REDACTED Tue Jul 19 16:24:15 2005 From: nm@REDACTED (Gaspar Chilingarov) Date: Tue, 19 Jul 2005 19:24:15 +0500 Subject: SYNC vs ASYNC calls In-Reply-To: References: Message-ID: <42DD0D0F.9040605@web.am> Ulf Wiger wrote: > > I changed one line of code in tickplus.erl and added three > to tickdb.erl, and now the program seems to be flying along: > > What I did: changed the gen_server:cast(DB, {'ADD'...}) to > a gen_server:call(). > I would like to suggest put this example somewhere into documentation or efficiency guide -- i'd also hit the same problem in my program - i've asked mailing list about poor performance of parses -- somthing like 400 comma separated lines/second. Later I've found that I hit erlang's limits of message queue creation speed. changing call from async to synchronous also bumped parse speed about 10 times. I guess this is a standard error, which novices(and not so novices ;))) can do, thinking that passing confirmation message back to caller for each parsed line it too much overhead -- thus writing programs which flood worker process queue with messages. -- Gaspar Chilingarov System Administrator t +37491 419763 (mob) t +37410 240399 (office) w www.web.am i 63174784 e nm@REDACTED From samuel@REDACTED Tue Jul 19 16:25:25 2005 From: samuel@REDACTED (Samuel Rivas) Date: Tue, 19 Jul 2005 16:25:25 +0200 Subject: Three wishes In-Reply-To: <200507182230.j6IMU4H1198690@atlas.otago.ac.nz> References: <200507182230.j6IMU4H1198690@atlas.otago.ac.nz> Message-ID: <20050719142525.GA22525@crusher.lfcia.pri> Richard A. O'Keefe wrote: > Is there any case in which it is a "must have"? > > Why yes, in the common case of things like > > if X > Y -> Sign = 1, Abs = X - Y > ; X < Y -> Sign = -1, Abs = Y - X > ; X ==Y -> Sign = 0, Abs = 0 > end, > ... use Sign and Abs ... > > where the *point* of the conditional is to bind several variables > rather than to compute a value. Well, it is not really a "must" but a "could". Personally, I feel matching the if result is more erlangish (arguing whether it is better or worse is more a matter of taste); test(X, Y) -> {Sign, Abs} = if X > Y -> {1, X - Y}; X < Y -> {-1, Y - X}; X == Y -> {0, 0} end, ... More sentences ... I see this more as a potential "best practice" than a reason for the compiler to print a warning. Having the warnings when an exported variable is part of another assignment outside the if/match block seems enough for me. Best regards. -- Samuel From klacke@REDACTED Tue Jul 19 17:00:17 2005 From: klacke@REDACTED (Claes Wikstom) Date: Tue, 19 Jul 2005 17:00:17 +0200 Subject: SYNC vs ASYNC calls In-Reply-To: <42DD0D0F.9040605@web.am> References: <42DD0D0F.9040605@web.am> Message-ID: <42DD1581.7020905@hyber.org> Gaspar Chilingarov wrote: > > I guess this is a standard error, which novices(and not so novices ;))) > can do, thinking that passing confirmation message back to caller for > each parsed line it too much overhead -- thus writing programs which > flood worker process queue with messages. > Well, just typing > i(). at the erlang prompt is always a good idea whenever you suspect something fishy. This would have revealed the above condition immediately /klacke From thomasl_erlang@REDACTED Tue Jul 19 17:35:37 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 19 Jul 2005 08:35:37 -0700 (PDT) Subject: Three wishes In-Reply-To: <20050719142525.GA22525@crusher.lfcia.pri> Message-ID: <20050719153537.75469.qmail@web34405.mail.mud.yahoo.com> --- Samuel Rivas wrote: > Personally, I feel > matching the if result is more erlangish (arguing > whether it is better > or worse is more a matter of taste); > > test(X, Y) -> > {Sign, Abs} = if X > Y -> {1, X - Y}; > X < Y -> {-1, Y - X}; > X == Y -> {0, 0} > end, > ... More sentences ... But doing it the other way REALLY IS, historically speaking, the Erlang way. :-) test(X,Y) -> if X > Y -> Sign = 1, Abs = X-Y; X < Y -> Sign = -1, Abs = Y-X; true -> Sign = 0, Abs = 0 end, %% use Sign, Abs (Though due to the warnings you get nowadays, I tend to pass around tuples.) Happily, the compiler nowadays gets rid of most intermediate tuples. In the days when it didn't, using them reduced performance, of course. Best, Thomas __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From Jouni.Ryno@REDACTED Tue Jul 19 11:33:28 2005 From: Jouni.Ryno@REDACTED (Jouni =?ISO-8859-1?Q?Ryn=F6?=) Date: Tue, 19 Jul 2005 12:33:28 +0300 Subject: ex11, plotting and Cairo ... In-Reply-To: <42DCC034.6080806@hyber.org> References: <42DCC034.6080806@hyber.org> Message-ID: <1121765608.5465.25.camel@adic> Dear All As a summer time hobby I started to program a plotting widget for Joe's ex11 (2004.09.09) version. Although there are better tools and programs :) than Erlang for graphics, sometimes a plot could be handy just from Erlang, like algorithm testing, timing checking etc. After coming to the "hello, world" -level, one gets a bit greedy. What else should it do and how? From Tcl/Tk there's the Blt-library, but aiming to that would be a bit too ambitious. Live zooming at least :) The other issue is the plot itself. It's easy to get to the screen, but hardcopy gets more complicated (xwd -> xwdtopnm -> pnmtojpeg -> ...). And no publications with these plots :), although that was not the intention at first either. The Gtk, Mozilla and OpenOffice seem to be converting to use Cairo library as 2D drawing library. It has backends to at least ps, gl and xlib. Drawing and printing can then be done with the same code. On the other hand, the whole point of ex11 is bypassing the slow drawing library in between. But maybe a special Cairo-canvas on top of ex11-canvas. So would there be any point of implementing Erlang binding to Cairo? Or would it then make more sense to use Gtk, which would then use Cairo indirectly (and provide themes and other fancy (and slow) stuff. still 2 weeks of holiday left (minus wife and the kids) Jouni ps. the plot is done by explot:plot("Hello, World"). -- Jouni Ryn? mailto://Jouni.Ryno@REDACTED/ http://www.geo.fmi.fi/~ryno/ Finnish Meteorological Institute http://www.fmi.fi/ Space Research http://www.geo.fmi.fi/ P.O.BOX 503 Tel (+358)-9-19294656 FIN-00101 Helsinki FAX (+358)-9-19294603 Finland priv-GSM (+358)-50-5302903 "It's just zeros and ones, it cannot be hard" -------------- next part -------------- A non-text attachment was scrubbed... Name: plot.jpg Type: image/jpeg Size: 19460 bytes Desc: not available URL: From will@REDACTED Tue Jul 19 18:59:04 2005 From: will@REDACTED (Will Newton) Date: Tue, 19 Jul 2005 17:59:04 +0100 Subject: ex11, plotting and Cairo ... In-Reply-To: <1121765608.5465.25.camel@adic> References: <42DCC034.6080806@hyber.org> <1121765608.5465.25.camel@adic> Message-ID: <200507191759.04589.will@misconception.org.uk> On Tuesday 19 July 2005 10:33, Jouni Ryn? wrote: > So would there be any point of implementing Erlang binding to Cairo? Or > would it then make more sense to use Gtk, which would then use Cairo > indirectly (and provide themes and other fancy (and slow) stuff. Some time ago I hacked the erlang GTK+ bindings to work with recent 2.x versions. Let me know if you would find the code useful. From Jouni.Ryno@REDACTED Tue Jul 19 12:10:32 2005 From: Jouni.Ryno@REDACTED (Jouni =?ISO-8859-1?Q?Ryn=F6?=) Date: Tue, 19 Jul 2005 13:10:32 +0300 Subject: ex11, plotting and Cairo ... In-Reply-To: <200507191759.04589.will@misconception.org.uk> References: <42DCC034.6080806@hyber.org> <1121765608.5465.25.camel@adic> <200507191759.04589.will@misconception.org.uk> Message-ID: <1121767832.5465.31.camel@adic> On Tue, 2005-07-19 at 17:59 +0100, Will Newton wrote: > On Tuesday 19 July 2005 10:33, Jouni Ryn? wrote: > > > So would there be any point of implementing Erlang binding to Cairo? Or > > would it then make more sense to use Gtk, which would then use Cairo > > indirectly (and provide themes and other fancy (and slow) stuff. > > Some time ago I hacked the erlang GTK+ bindings to work with recent 2.x > versions. Let me know if you would find the code useful. I got halfway about a year ago. Must have been the previous holiday :) So sure. Shouldn't the sourceforge site to also updated. Bjorng? Jouni -- Jouni Ryn? mailto://Jouni.Ryno@REDACTED/ http://www.geo.fmi.fi/~ryno/ Finnish Meteorological Institute http://www.fmi.fi/ Space Research http://www.geo.fmi.fi/ P.O.BOX 503 Tel (+358)-9-19294656 FIN-00101 Helsinki FAX (+358)-9-19294603 Finland priv-GSM (+358)-50-5302903 "It's just zeros and ones, it cannot be hard" From paris@REDACTED Tue Jul 19 19:39:12 2005 From: paris@REDACTED (Javier =?iso-8859-1?Q?Par=EDs_Fern=E1ndez?=) Date: Tue, 19 Jul 2005 19:39:12 +0200 Subject: TCP stack throughput In-Reply-To: <20050706180318.GA18071@dc.fi.udc.es> References: <20050706180318.GA18071@dc.fi.udc.es> Message-ID: <20050719173912.GA8577@dc.fi.udc.es> Hi, Ok, a bit late O:) I modified the files slightly, because out TCP/IP stack works with a passive model (the client must make the reads explicitly). The files can be downloaded from http://www.lfcia.org/~paris/fac.erl http://www.lfcia.org/~paris/tcp_server.erl To take away the cost of the factorial, I made all the test computing the factorial of 1. I observed several funny things. The time spent processing the connections seems to go up exponentially, but it is not limited by cpu, which was always under 50%. I don't know what is throttling it. When I trid with 5000 connections, the logs started to show retransmissions, so I think it is probably related to some buffering limit with the socket. This obviously skyrocketed the total processing time. The total processing time for the tests was: 500 in 0.6 seconds 1000 in 1.7 seconds 2000 in 6.2 seconds 5000 was still closing connections after 3 minutes There are tcpdump logs in http://www.lfcia.org/~paris/dump500.gz http://www.lfcia.org/~paris/dump1000.gz http://www.lfcia.org/~paris/dump2000.gz http://www.lfcia.org/~paris/dump5000.gz I think it would be interesting to find what is the limiting the stack here, and why there are dropped packets, as I think it should be possible to get better performance. However, I had not made a test similar to this, so connection opening and closing is one of the things that are not optimized in the stack. Regards, Javier. From erlang@REDACTED Tue Jul 19 19:45:08 2005 From: erlang@REDACTED (Peter-Henry Mander) Date: Tue, 19 Jul 2005 18:45:08 +0100 Subject: ex11, plotting and Cairo ... In-Reply-To: <1121765608.5465.25.camel@adic> References: <42DCC034.6080806@hyber.org> <1121765608.5465.25.camel@adic> Message-ID: <1121795108.28586.30.camel@hymir.newport-networks.com> Hi Jouni, I found that creating graphs in SVG is quite handy, since it's an XML type of thing, it's quite easy to use. Firefox is beginning to support SVG, and there's an Adobe plugin available, but support for all the really coo-o-ol features of the standard is still patchy. There's a few Open Source drawing applications that can be used too, for tweaking the graphs. Pete. On Tue, 2005-07-19 at 12:33 +0300, Jouni Ryn? wrote: > Dear All > > As a summer time hobby I started to program a plotting widget for Joe's > ex11 (2004.09.09) version. Although there are better tools and > programs :) than Erlang for graphics, sometimes a plot could be handy > just from Erlang, like algorithm testing, timing checking etc. -- "The Tao of Programming flows far away and returns on the wind of morning." From bjarne@REDACTED Tue Jul 19 20:18:25 2005 From: bjarne@REDACTED (=?iso-8859-1?Q?Bjarne_D=E4cker?=) Date: Tue, 19 Jul 2005 20:18:25 +0200 Subject: Inswitch being less than clear about where Erlang came from References: Message-ID: <000101c58c90$29e3c760$540569d4@segeltorp> ----- Original Message ----- From: "Joe Armstrong (AL/EAB)" To: ; "Ulf Wiger" Cc: Sent: Tuesday, July 19, 2005 3:37 PM Subject: RE: Inswitch being less than clear about where Erlang came from > - - - - - - > > Now if anybody from Inswitch is reading, we would appreciate a link to > www.erlang.org from your main pages - This will increase our rating in > the google link weighting algorithm - oh and good luck with Erlang. > > /Joe > > ... and I would like to invite Inswitch to make a presentation at the Erlang User Conference in Stockholm on November 10. Please see the invitation and call for papers at http://www.erlang.se/euc/05/ Welcome Bjarne From kostis@REDACTED Tue Jul 19 22:58:42 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 19 Jul 2005 22:58:42 +0200 (MEST) Subject: Erlang'05 call for participation Message-ID: <200507192058.j6JKwg8w016538@spikklubban.it.uu.se> Call for Participation: Erlang'05 You are cordially invited to register and attend the 4th ACM SIGPLAN Erlang Workshop which will take place in Tallinn, Estonia on Sept 25th. The program, list of accepted papers, and registration information appears at the workshop's homepage http://www.erlang.se/workshop/2005/ Please note that the early registration deadine is July 29th, which is also the date when the block bookings and special rates from most of the local hotels are valid. All relevant info is available from http://www.cs.ioc.ee/tfp-icfp-gpce05/ Hoping to see many of you in beautiful Tallinn. Kostis Sagonas Erlang'05 Workshop chair From david.nospam.hopwood@REDACTED Wed Jul 20 01:47:10 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Wed, 20 Jul 2005 00:47:10 +0100 Subject: Meyer, OO and concurrency In-Reply-To: <57a21f73050718033660447b1b@mail.gmail.com> References: <57a21f73050718033660447b1b@mail.gmail.com> Message-ID: <42DD90FE.9000101@blueyonder.co.uk> Taj Khattra wrote: > On 7/14/05, Joe Armstrong (AL/EAB) wrote: > >>Threads/process in C++ are usually not C++ Threads or processes but >>thinly disguised OS threads or processes. >> >>The problem with this is that the semantics of the program is *amazingly* >>UNDEFINED. > > from http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2004/n1680.pdf: > > ``Unfortunately, programming multithreaded applications of any > kind in C++ remains a black art. Properties critical for reliable, > efficient, and correct multithreaded execution are left unspecified. > [...] many questions that programmers have about how to predict and > control multithreaded code simply do not have answers. Superstition, > myths, bad advice, and "hey, it worked for me" stories are rampant.'' > > this is due to the lack of a clearly specified shared memory model (a > la java, until jsr-133) in the presence of multiple threads and > instruction re-orderings due to compiler and processor > "optimizations". Although lack of a clearly specified shared memory model doesn't help, I don't think that specifying such a model would solve the problems that people typically encounter with using shared memory concurrency in applications (especially large applications). With or without a clearly specified model, it is not too difficult to write applications that avoid low-level race hazards, and that therefore have sequentially consistent behaviour under the intended semantics of systems like POSIX or Java (even pre-JSR-133). But that isn't enough: higher-level hazards from unintended interleavings are still possible, and trying to use locks to avoid these presents too great a risk of deadlock (for reasons explained in ). Such a model would, however, have some use in languages and operating systems that need to implement other models of concurrency in terms of shared state. -- David Hopwood From ok@REDACTED Wed Jul 20 06:11:50 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 20 Jul 2005 16:11:50 +1200 (NZST) Subject: Three wishes Message-ID: <200507200411.j6K4Bo8W210171@atlas.otago.ac.nz> In answer to the question > Is there any case in which it is a "must have"? I wrote: > Why yes, in the common case of things like > > if X > Y -> Sign = 1, Abs = X - Y > ; X < Y -> Sign = -1, Abs = Y - X > ; X ==Y -> Sign = 0, Abs = 0 > end, > ... use Sign and Abs ... > > where the *point* of the conditional is to bind several variables > rather than to compute a value. My argument is that we "must have" this feature because it is USED. The person who hates it, hates it because it is USED. Samuel Rivas replied: Well, it is not really a "must" but a "could". One of us is misunderstanding "must have". I think that a feature which was designed for good reason (familiarity to Prolog and Strand programmers) and which is in active use by many people is a "must have" because I don't believe in kicking innocent users in the teeth. (The slogan I proposed for the ISO Prolog committee, which they explicitly rejected, was "Non calcitrandus in dentes artifex".) Personally, I feel matching the if result is more erlangish I would dispute that. Part of the point here is that a dumb compiler can generate good code WITHOUT having to slog through an analysis to determine that certain tuples don't *really* need to exist. test(X, Y) -> {Sign, Abs} = if X > Y -> {1, X - Y}; X < Y -> {-1, Y - X}; X == Y -> {0, 0} end, ... More sentences ... This - shoves a block of code way over to the right for no good reason - is more error-prone than the current practice (if you forget to bind a variable in one of the cases, the compiler will warn you downstream that you are trying to use a variable that you mayn't, while if you get the number of elements in one of these tuples wrong, you will get a run-time error instead, IF you test well) - forces you to write the expressions in the same order in each case, instead of the order which is most natural for each case - moves the names a long distance away from the values they are bound to, so that it is MUCH harder to see which variables are bound to which values - introduces a new family of mistakes where you get the number of expressions in each tuple right, but the order wrong in one of them None of this strikes me as particularly "Erlangish", and it seems to me one of the merits of the language that it offers you a choice here. I see this more as a potential "best practice" than a reason for the compiler to print a warning. It's odd to see something which separates names from values and makes more mistakes not only possible but likely described as "best practice". From bfulg@REDACTED Wed Jul 20 07:41:40 2005 From: bfulg@REDACTED (Brent Fulgham) Date: Tue, 19 Jul 2005 22:41:40 -0700 Subject: Compiler Error (Wings3d on Mac OS X) Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm getting an Internal Consistency Check error when attempting to build Wings3D on Mac OS X 10.4.2 (Erlang OTP R10B-6): erlc -pa /ebin +warn_unused_vars -I/include -I ../e3d -W +debug_info '-Dwings_version="0.98.31"' -pa ../ebin -o../ebin wings_color.erl wings_color: function internal_rgb_to_hsv/3+97: Internal consistency check failed - please report this bug. Instruction: {test,is_eq_exact,{f,80},[{x,0},{atom,error}]} Error: {unsafe_instruction,{float_error_state,cleared}}: make[2]: *** [../ebin/wings_color.beam] Error 1 What would people recommend as a first step toward debugging this problem? Is it arcana only in the realm of Ulf, Joe, or one of the other Erlang masters? The offending statements seems to be: internal_rgb_to_hsv(R, G, B) -> Max = lists:max([R,G,B]), Min = lists:min([R,G,B]), V = Max, {Hue,Sat} = try {if Min == B -> (G-Min)/(R+G-2.0*Min); Min == R -> (1.0+(B-Min)/(B+G-2.0*Min)); Min == G -> (2.0+(R-Min)/(B+R-2.0*Min)) end*120,(Max-Min)/Max} catch error:badarith -> {undefined,0.0} end, {Hue,Sat,V}. Thanks, - -Brent -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) iD8DBQFC3eQUzGDdrzfvUpURAo8IAJ9hfewubm2qWMRygrxkvmHyu6O9IwCfcxhd mRalH4N24feln9YcpMmg5cQ= =iRcZ -----END PGP SIGNATURE----- From joe.armstrong@REDACTED Wed Jul 20 09:11:32 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Wed, 20 Jul 2005 09:11:32 +0200 Subject: ex11, plotting and Cairo ... Message-ID: What fun. As regards graphics in Erlang the situation is still a mess. I see things the problem is more one of "programming models" than making an Erlang interface to X (replace X by your favourite package). Suppose you have a rock solid interface to Gtk - then you still have to learn the Gtk programming model - which is not an easy job. The problem with most graphics packages is the programming model which are often horribly difficult. Most "packers" are difficult to use, and often, for no apparent reason get layouts wrong. For simple graphics, just a window with a few boxes, most graphics packages are far too complex to use. Worse, the programs are so complex that the use of a "visual tool" to create the code is almost obligatory - that you need to use such a tool is a sure sign that something is wrong. What I want is VSG (very simple graphics) - I have an "acid test" for VSG, I want to write three applications: 1) A traditional GUI (say for an IRC chat widget) 2) A drag-and-drop interface (say for drawing state diagrams) 3) An image manipulation application (think GIMP) All of these should be easy to write using VSG, using a small number of intuitive primitives. The ex11 programming model was an attempt at this. I'm currently working on VSG with a TCL/TK backend. One thing I did find was that the only model I like is based on a canvas widget. To create buttons (and controls in general) I just draw rectangles and lines onto a canvas widget and animate them on mouse clicks. If we want to make a decent graphics system I think the way to do so is as follows: 1) Implement a solid set of cross-platform graphics primitives 2) Implement a canvas widget on top of 1) 3) Implement a GUI set on top of 2) ex11 just used the X11 primitives - which were *very* simple to use. But they were not portable. I had thought that one way to do this is to implement an interface to GDK and then reimplement something like GTK in Erlang. The latter step should be pretty easy, writing complex widgets with concurrency at your fingertips is pretty easy. So if you want to spend your 2 weeks of holiday fruitfully try to make a solid interface to GDK that runs on windows and linux. << aside - why can't we just define a low-level API and then layer graphics on top of this. The low level could be implemented using your favorite package. The following API would suffice mkWin(X, Y, Width, Ht) => Win draw(Win, rectangle, X, Y, Width, Ht, Color) => Tag draw(Win, line, X1, Y1, X2, Y2, Width, Color) => Tag draw(Win, text, X, Y, Font, Size, Text) => Tag draw(Win, image, X, Y, ImageFile) => Tag ... add_event_handler(Win, X, Y, Width, Ht, mouse_click, Fun) => Tag delete(Win, Tag) => true So you can create a window (mkWin) - draw things (draw) - add event handlers and deleted tagged things. Given this (or something like this) then pretty fancy applications can be layered on top of this. >> Cheers /Joe > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Jouni Ryn? > Sent: den 19 juli 2005 11:33 > To: erlang-questions@REDACTED > Subject: ex11, plotting and Cairo ... > > > Dear All > > As a summer time hobby I started to program a plotting widget > for Joe's > ex11 (2004.09.09) version. Although there are better tools and > programs :) than Erlang for graphics, sometimes a plot could be handy > just from Erlang, like algorithm testing, timing checking etc. > > After coming to the "hello, world" -level, one gets a bit greedy. What > else should it do and how? From Tcl/Tk there's the Blt-library, but > aiming to that would be a bit too ambitious. Live zooming at least :) > > The other issue is the plot itself. It's easy to get to the > screen, but > hardcopy gets more complicated (xwd -> xwdtopnm -> pnmtojpeg -> ...). > And no publications with these plots :), although that was not the > intention at first either. > > The Gtk, Mozilla and OpenOffice seem to be converting to use Cairo > library as 2D drawing library. It has backends to at least ps, gl and > xlib. Drawing and printing can then be done with the same code. On the > other hand, the whole point of ex11 is bypassing the slow drawing > library in between. But maybe a special Cairo-canvas on top of > ex11-canvas. > > So would there be any point of implementing Erlang binding to > Cairo? Or > would it then make more sense to use Gtk, which would then use Cairo > indirectly (and provide themes and other fancy (and slow) stuff. > > still 2 weeks of holiday left (minus wife and the kids) > Jouni > > ps. the plot is done by explot:plot("Hello, World"). > -- > > Jouni Ryn? mailto://Jouni.Ryno@REDACTED/ > http://www.geo.fmi.fi/~ryno/ > Finnish Meteorological Institute http://www.fmi.fi/ > Space Research http://www.geo.fmi.fi/ > P.O.BOX 503 Tel (+358)-9-19294656 > FIN-00101 Helsinki FAX (+358)-9-19294603 > Finland priv-GSM (+358)-50-5302903 > > "It's just zeros and ones, it cannot be hard" > > From kostis@REDACTED Wed Jul 20 10:06:26 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 20 Jul 2005 10:06:26 +0200 (MEST) Subject: Compiler Error (Wings3d on Mac OS X) In-Reply-To: Mail from 'Brent Fulgham ' dated: Tue, 19 Jul 2005 22:41:40 -0700 Message-ID: <200507200806.j6K86Qsj008984@spikklubban.it.uu.se> Brent Fulgham wrote: > I'm getting an Internal Consistency Check error when attempting to > build Wings3D on Mac OS X 10.4.2 (Erlang OTP R10B-6): > > erlc -pa /ebin +warn_unused_vars -I/include -I ../e3d -W +debug_info > '-Dwings_version="0.98.31"' -pa ../ebin -o../ebin wings_color.erl > wings_color: function internal_rgb_to_hsv/3+97: > Internal consistency check failed - please report this bug. > Instruction: {test,is_eq_exact,{f,80},[{x,0},{atom,error}]} > Error: {unsafe_instruction,{float_error_state,cleared}}: > > make[2]: *** [../ebin/wings_color.beam] Error 1 > > What would people recommend as a first step toward debugging this > problem? I would recommend doing what you did: Just report it to this list. > Is it arcana only in the realm of Ulf, Joe, ... Perhaps the names Bjorn and Raimo are more appropriate here. Until they are back from vacations, here is a solution to your problem. Make the code look as follows and it will pass the compiler: internal_rgb_to_hsv(R, G, B) -> Max = lists:max([R,G,B]), Min = lists:min([R,G,B]), V = Max, {Hue,Sat} = %% try {if Min == B -> (G-Min)/(R+G-2.0*Min); Min == R -> (1.0+(B-Min)/(B+G-2.0*Min)); Min == G -> (2.0+(R-Min)/(B+R-2.0*Min)) end*120,(Max-Min)/Max} %% catch %% error:badarith -> %% {undefined,0.0} %% end , {Hue,Sat,V}. Best, Kostis From thomasl_erlang@REDACTED Wed Jul 20 11:12:22 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 20 Jul 2005 02:12:22 -0700 (PDT) Subject: Erlang (mild) PL buzz Message-ID: <20050720091222.13666.qmail@web34415.mail.mud.yahoo.com> I see that Erlang is getting mentioned here and there in the academic lit, in this case PLDI 2005. For instance, the bit syntax is mentioned in related work in PADS: A domain-specific language for processing ad hoc data, Kathleen Fisher and Robert Gruber. Also, Erlang is mentioned in passing in: Threads Cannot Be Implemented As A Library, Hans-J. Boehm. Fame at last? :-) Best, Thomas ____________________________________________________ Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs From vlad_dumitrescu@REDACTED Wed Jul 20 11:29:01 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Wed, 20 Jul 2005 11:29:01 +0200 Subject: ex11, plotting and Cairo ... References: Message-ID: ----- Original Message ----- From: "Joe Armstrong (AL/EAB)" > I see things the problem is more one of "programming models" than making > an Erlang interface to X (replace X by your favourite package). Hi, As I've also been struggling with this, I'd like to throw in my two cents. About programming models: One important feature of an Ernalg graphics package would be (for me) that it is "erlangish", and doesn't force me to use C-like code. A good example of what I mean is ex11, but one reason why it works so easily is that X itself works just like we want it to. Making a regular graphics package work like that is much harder. > One thing I did find was that the only model I like is based on a canvas > widget. To create buttons (and controls in general) I just draw rectangles > and lines > onto a canvas widget and animate them on mouse clicks. That's what all toolkits do :-) If an "erlang programming model" is to be enforced, then this is probably the only way, to use external code for the low-level drawing. > << aside - why can't we just define a low-level API and then layer > graphics > on top of this. The low level could be implemented using your favorite > package. > > The following API would suffice > > mkWin(X, Y, Width, Ht) => Win > draw(Win, rectangle, X, Y, Width, Ht, Color) => Tag > draw(Win, line, X1, Y1, X2, Y2, Width, Color) => Tag > draw(Win, text, X, Y, Font, Size, Text) => Tag > draw(Win, image, X, Y, ImageFile) => Tag > ... > add_event_handler(Win, X, Y, Width, Ht, mouse_click, Fun) => Tag > delete(Win, Tag) => true > > So you can create a window (mkWin) - draw things (draw) - add event > handlers > and deleted tagged things. > > Given this (or something like this) then pretty fancy applications can be > layered on > top of this. > >> First of all, agreeing on an API isn't easy at all! ;-) That said, I think the problem is that once the basic functionality is in place, one will want to add more features. In this day and age, nobody will want to use a GUI that looks like Windows 1.0... So the API should be able to take in new functionality without breaking. A suggestion for such an API is the PS/PDF model. One backend that is reasonably stable, reasonably portable and lends itself to advanced graphics use is OpenGL. Esdl is a good start, just look at Wings. The only drawback is that SDL only supports one top-level window, but I know that Dan and the esdl team are working on something. best regards, Vlad From erlang@REDACTED Wed Jul 20 12:03:38 2005 From: erlang@REDACTED (Peter-Henry Mander) Date: Wed, 20 Jul 2005 11:03:38 +0100 Subject: ex11, plotting and Cairo ... Message-ID: <1121853819.16160.27.camel@hymir.newport-networks.com> Hi Joe, (Enters scene with proverbial oar) Although you are rightly concerned about portability, I really liked ex11 because it _removed_ many layers between application code and graphics, cutting out a lot of sloooow ugly crap between the two. Considering this, why not attack the MS-Windows GDI directly in Erlang, maybe with a k.i.s.s. C driver to ease the connection? As far as I'm aware there are essentially only two low-level graphics APIs that cover 99.997% of computers: MS-Windows GDI and MIT-X11. Furthermore, with Erlang distribution you gain the advantage of a graphics API for Windows that is network oriented, like X11. Believe me, I sorely miss the client-server architecture of X11 when working on MS-Windows. I install Cygwin-X11 within minutes! I'm not claiming it's easy, but ex11 supplemented with "eGDI" would be a solid foundation for concurrency-oriented graphics. Keep the middle and upper layers written in Erlang and have just two low-layer protocols to choose from, possibly switched automagically according to OS. Pete. On Wed, 2005-07-20 at 09:11 +0200, Joe Armstrong (AL/EAB) wrote: > What fun. As regards graphics in Erlang the situation is still a mess. > > I see things the problem is more one of "programming models" than making > an Erlang interface to X (replace X by your favourite package). > [snipped] -- "The Tao of Programming flows far away and returns on the wind of morning." From vlad_dumitrescu@REDACTED Wed Jul 20 13:48:25 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Wed, 20 Jul 2005 13:48:25 +0200 Subject: ex11, plotting and Cairo ... References: <1121853819.16160.27.camel@hymir.newport-networks.com> Message-ID: From: "Peter-Henry Mander" > Considering this, why not attack the MS-Windows GDI directly in Erlang, > maybe with a k.i.s.s. C driver to ease the connection? Easiest would be to write a graphics server and talk with it via a socket. I think someone said earlier he had been giving it a try. This would actually be a simplified X server, and could be made portable too. Question is: should the protocol be compatible with X? If yes, is there a danger[*] that this server will become an alternative X server? If no, could it be mapped onto X? [*] in the meaning that it would mean to reinvent a lot of wheels... regards, Vlad From joe.armstrong@REDACTED Wed Jul 20 13:47:45 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Wed, 20 Jul 2005 13:47:45 +0200 Subject: ex11, plotting and Cairo ... Message-ID: Vlad Dumitrescu wrote > From: "Joe Armstrong (AL/EAB)" > > I see things the problem is more one of "programming > models" than making > > an Erlang interface to X (replace X by your favourite package). > > Hi, > > As I've also been struggling with this, I'd like to throw in > my two cents. > > About programming models: One important feature of an Ernalg graphics > package would be (for me) that it is "erlangish", and doesn't > force me to > use C-like code. A good example of what I mean is ex11, but > one reason why > it works so easily is that X itself works just like we want > it to. Making a > regular graphics package work like that is much harder. > > > One thing I did find was that the only model I like is > based on a canvas > > widget. To create buttons (and controls in general) I just > draw rectangles > > and lines > > onto a canvas widget and animate them on mouse clicks. > > That's what all toolkits do :-) If an "erlang programming > model" is to be > enforced, then this is probably the only way, to use external > code for the > low-level drawing. > > > << aside - why can't we just define a low-level API and then layer > > graphics > > on top of this. The low level could be implemented using > your favorite > > package. > > > > The following API would suffice > > > > mkWin(X, Y, Width, Ht) => Win > > draw(Win, rectangle, X, Y, Width, Ht, Color) => Tag > > draw(Win, line, X1, Y1, X2, Y2, Width, Color) => Tag > > draw(Win, text, X, Y, Font, Size, Text) => Tag > > draw(Win, image, X, Y, ImageFile) => Tag > > ... > > add_event_handler(Win, X, Y, Width, Ht, mouse_click, Fun) => Tag > > delete(Win, Tag) => true > > > > So you can create a window (mkWin) - draw things (draw) - add event > > handlers > > and deleted tagged things. > > > > Given this (or something like this) then pretty fancy > applications can be > > layered on > > top of this. > > >> > > First of all, agreeing on an API isn't easy at all! ;-) Why not? - given the above I can program a wide range of apps. Much better to have a stable well defined low-level than nothing. Actually to make a "drop-dead-beautiful" interface all I need is anti-aliased fonts and alpha-blending of all display objects. If we could agree on a set of stable cross-platform low-level primitives then life would be much easier. > That said, I think the problem is that once the basic > functionality is in > place, one will want to add more features. In this day and > age, nobody will > want to use a GUI that looks like Windows 1.0... So the API > should be able > to take in new functionality without breaking. A suggestion > for such an API > is the PS/PDF model. > YES * 10000 - Then I could take my erlguten code and put it into the GUIs proper kerning and ligatures in the text in the menus!!! > One backend that is reasonably stable, reasonably portable > and lends itself > to advanced graphics use is OpenGL. Esdl is a good start, > just look at > Wings. The only drawback is that SDL only supports one > top-level window, but > I know that Dan and the esdl team are working on something. Go Dan, go ... /Joe > > best regards, > Vlad > > From Jouni.Ryno@REDACTED Wed Jul 20 07:28:43 2005 From: Jouni.Ryno@REDACTED (Jouni =?ISO-8859-1?Q?Ryn=F6?=) Date: Wed, 20 Jul 2005 08:28:43 +0300 Subject: ex11, plotting and Cairo ... In-Reply-To: References: Message-ID: <1121837323.5440.27.camel@adic> Good, the open questions I wanted to see :) The reason I prompted for Cairo was the moment I needed justified text on the plot labels in ex11. The xlib provides function to calculate the string width, it's done by taking the font metrics and calculating the total width from individual characters. That was easy in erlang :) text_width(Parent, Font, Str) -> Display = sw:rpc(Parent, display), F = ex11_lib:xQueryFont(Display, Font), CharList = F#fontInfo.char_infos, {Sizes, CharList} = lists:mapfoldl(fun(Char, CharList) -> CI = lists:nth(Char, CharList), {CI#charInfo.width, CharList} end, CharList, Str), lists:sum(Sizes). But adding new routines opens the pandora box, I want more ... Cairo has the PDF 1.4 programming model, and seems to handle the font and text related needs at reasonable level (alpha and such ...) And it would be the canvas only API ... Windows or whatever drawing backend is needed, it's selected before the drawing is done. About the GDK: from the GDK-Cairo interaction Cairo is a graphics library that supports vector graphics and image compositing that can be used with GDK. Since 2.8, GTK+ does most of its drawing using Cairo. GDK does not wrap the Cairo API, instead it allows to create Cairo contexts which can be used to draw on GDK drawables. Additional functions allow to convert GDK's rectangles and regions into Cairo paths and to use pixbufs as sources for drawing operations. Anyway, it would seem to me, that Cairo API for canvas would be sufficient, for windows and events, X/ex11 is enough :). But I take a look on the GDK ... Jouni On Wed, 2005-07-20 at 09:11 +0200, Joe Armstrong (AL/EAB) wrote: > What fun. As regards graphics in Erlang the situation is still a mess. > > I see things the problem is more one of "programming models" than making > an Erlang interface to X (replace X by your favourite package). > > Suppose you have a rock solid interface to Gtk - then you still have to > learn the Gtk programming model - which is not an easy job. > > The problem with most graphics packages is the programming model which are often > horribly difficult. Most "packers" are difficult to use, and often, for no apparent > reason get layouts wrong. For simple graphics, just a window with a few boxes, > most graphics packages are far too complex to use. > > Worse, the programs are so complex that the use of a "visual tool" to create the > code is almost obligatory - that you need to use such a tool is a sure sign that > something is wrong. > > What I want is VSG (very simple graphics) - I have an "acid test" for VSG, I want to > write three applications: > > 1) A traditional GUI (say for an IRC chat widget) > 2) A drag-and-drop interface (say for drawing state diagrams) > 3) An image manipulation application (think GIMP) > > All of these should be easy to write using VSG, using a small number of > intuitive primitives. > > The ex11 programming model was an attempt at this. > > I'm currently working on VSG with a TCL/TK backend. > > One thing I did find was that the only model I like is based on a canvas > widget. To create buttons (and controls in general) I just draw rectangles and lines > onto a canvas widget and animate them on mouse clicks. > > If we want to make a decent graphics system I think the way to do so is as follows: > > 1) Implement a solid set of cross-platform graphics primitives > 2) Implement a canvas widget on top of 1) > 3) Implement a GUI set on top of 2) > > ex11 just used the X11 primitives - which were *very* simple to use. But they were > not portable. > > I had thought that one way to do this is to implement an interface to GDK and then > reimplement something like GTK in Erlang. The latter step should be pretty easy, > writing complex widgets with concurrency at your fingertips is pretty easy. > > So if you want to spend your 2 weeks of holiday fruitfully try to make > a solid interface to GDK that runs on windows and linux. > > << aside - why can't we just define a low-level API and then layer graphics > on top of this. The low level could be implemented using your favorite package. > > The following API would suffice > > mkWin(X, Y, Width, Ht) => Win > draw(Win, rectangle, X, Y, Width, Ht, Color) => Tag > draw(Win, line, X1, Y1, X2, Y2, Width, Color) => Tag > draw(Win, text, X, Y, Font, Size, Text) => Tag > draw(Win, image, X, Y, ImageFile) => Tag > ... > add_event_handler(Win, X, Y, Width, Ht, mouse_click, Fun) => Tag > delete(Win, Tag) => true > > So you can create a window (mkWin) - draw things (draw) - add event handlers > and deleted tagged things. > > Given this (or something like this) then pretty fancy applications can be layered on > top of this. > >> > > > Cheers > > /Joe > > > > -----Original Message----- > > From: owner-erlang-questions@REDACTED > > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Jouni Ryn? > > Sent: den 19 juli 2005 11:33 > > To: erlang-questions@REDACTED > > Subject: ex11, plotting and Cairo ... > > > > > > Dear All > > > > As a summer time hobby I started to program a plotting widget > > for Joe's > > ex11 (2004.09.09) version. Although there are better tools and > > programs :) than Erlang for graphics, sometimes a plot could be handy > > just from Erlang, like algorithm testing, timing checking etc. > > > > After coming to the "hello, world" -level, one gets a bit greedy. What > > else should it do and how? From Tcl/Tk there's the Blt-library, but > > aiming to that would be a bit too ambitious. Live zooming at least :) > > > > The other issue is the plot itself. It's easy to get to the > > screen, but > > hardcopy gets more complicated (xwd -> xwdtopnm -> pnmtojpeg -> ...). > > And no publications with these plots :), although that was not the > > intention at first either. > > > > The Gtk, Mozilla and OpenOffice seem to be converting to use Cairo > > library as 2D drawing library. It has backends to at least ps, gl and > > xlib. Drawing and printing can then be done with the same code. On the > > other hand, the whole point of ex11 is bypassing the slow drawing > > library in between. But maybe a special Cairo-canvas on top of > > ex11-canvas. > > > > So would there be any point of implementing Erlang binding to > > Cairo? Or > > would it then make more sense to use Gtk, which would then use Cairo > > indirectly (and provide themes and other fancy (and slow) stuff. > > > > still 2 weeks of holiday left (minus wife and the kids) > > Jouni > > > > ps. the plot is done by explot:plot("Hello, World"). > > -- > > > > Jouni Ryn? mailto://Jouni.Ryno@REDACTED/ > > http://www.geo.fmi.fi/~ryno/ > > Finnish Meteorological Institute http://www.fmi.fi/ > > Space Research http://www.geo.fmi.fi/ > > P.O.BOX 503 Tel (+358)-9-19294656 > > FIN-00101 Helsinki FAX (+358)-9-19294603 > > Finland priv-GSM (+358)-50-5302903 > > > > "It's just zeros and ones, it cannot be hard" > > > > -- Jouni Ryn? mailto://Jouni.Ryno@REDACTED/ http://www.geo.fmi.fi/~ryno/ Finnish Meteorological Institute http://www.fmi.fi/ Space Research http://www.geo.fmi.fi/ P.O.BOX 503 Tel (+358)-9-19294656 FIN-00101 Helsinki FAX (+358)-9-19294603 Finland priv-GSM (+358)-50-5302903 "It's just zeros and ones, it cannot be hard" From mike@REDACTED Wed Jul 20 14:35:34 2005 From: mike@REDACTED (Michael Williams) Date: 20 Jul 2005 12:35:34 GMT Subject: Erlang (mild) PL buzz References: <20050720091222.13666.qmail@web34415.mail.mud.yahoo.com> Message-ID: In article <20050720091222.13666.qmail@REDACTED>, thomasl_erlang@REDACTED (Thomas Lindgren) writes: |> Threads Cannot Be Implemented As A Library, Hans-J. |> Boehm. |> Interesting paper, but somewaht far from reality. Anyone who programmed a concurrent application as shown in the examples in this paper would need their heads examined. Even in C... /mike From kostis@REDACTED Wed Jul 20 15:25:26 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 20 Jul 2005 15:25:26 +0200 (MEST) Subject: Erlang (mild) PL buzz In-Reply-To: Mail from 'mike@erix.ericsson.se (Michael Williams)' dated: 20 Jul 2005 12:35:34 GMT Message-ID: <200507201325.j6KDPQF2024031@spikklubban.it.uu.se> Michael Williams wrote: > > |> Threads Cannot Be Implemented As A Library, Hans-J. Boehm. > |> > > Interesting paper, but somewaht far from reality. Anyone who programmed > a concurrent application as shown in the examples in this paper would > need their heads examined. Even in C... The point of the paper is different, at least in my reading and from listening to the paper's presentation at PLDI. The point is that in a language that has not been designed with concurrency integrated in the language/compiler but where concurrency is merely an add-on in the form of a library, the compiler can, quite reasonably, create such nasty situations through transformations that do not take into account that the execution will be multi-threaded if a library such as Pthreads is linked in. Hence, the disclaimer that this does not apply to Erlang (which has a different model of concurrency) or CML. Kostis From erlang@REDACTED Wed Jul 20 15:53:29 2005 From: erlang@REDACTED (Peter-Henry Mander) Date: Wed, 20 Jul 2005 14:53:29 +0100 Subject: ex11, plotting and Cairo ... In-Reply-To: References: <1121853819.16160.27.camel@hymir.newport-networks.com> Message-ID: <1121867609.16160.48.camel@hymir.newport-networks.com> Hi Vlad, Simplified portable X server sounds good to me, it would be very useful to me as a stand-alone product in itself! If there's a Open Source one, even better! Cygwin X server has a bit too much baggage to be called simple. Xming (http://freedesktop.org/wiki/Xming) looks like it fits the bill, but I'm hitting a wall with it. I'm not sure I grasp the meaning of your question. The whole idea is to avoid reinvention by using X11. Joe is very keen to make it drop dead gorgeous, but he's invented ex11 which to my mind is "good enough" and it may become the enemy of his better PDF/PS GUI. Sorry Joe, I know what you'd like to see since I'm an ex-typography student and I appreciate your aesthetic inclination, but the concurrency model and thin layers is what I want above the kerning, antialiasing etc. Pete. On Wed, 2005-07-20 at 13:48 +0200, Vlad Dumitrescu wrote: > From: "Peter-Henry Mander" > > Considering this, why not attack the MS-Windows GDI directly in Erlang, > > maybe with a k.i.s.s. C driver to ease the connection? > > Easiest would be to write a graphics server and talk with it via a socket. I > think someone said earlier he had been giving it a try. This would actually > be a simplified X server, and could be made portable too. > > Question is: should the protocol be compatible with X? If yes, is there a > danger[*] that this server will become an alternative X server? If no, could > it be mapped onto X? > > [*] in the meaning that it would mean to reinvent a lot of wheels... > > regards, > Vlad > > > -- "The Tao of Programming flows far away and returns on the wind of morning." From tomas.langer@REDACTED Wed Jul 20 16:03:51 2005 From: tomas.langer@REDACTED (Tomas Langer (LN/EAB)) Date: Wed, 20 Jul 2005 16:03:51 +0200 Subject: seq trace and spawn Message-ID: Hi, I have a question about sequential trace and what happens if a new process is created in the traced seq. Is it possible to get the seq trace token to follow the newly spawn:ed process? (without update the spawn statment in the code with an extra parameter). The flow is something like this: seq trace is started | (a lot of signaling is generated) | spawn an new process-----------------------------| | | (the traced seq. is finishing) | | the signaling continues on this new proccess regards tomas From david.nospam.hopwood@REDACTED Wed Jul 20 16:11:16 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Wed, 20 Jul 2005 15:11:16 +0100 Subject: Erlang (mild) PL buzz In-Reply-To: References: <20050720091222.13666.qmail@web34415.mail.mud.yahoo.com> Message-ID: <42DE5B84.90608@blueyonder.co.uk> Michael Williams wrote: > In article <20050720091222.13666.qmail@REDACTED>, > thomasl_erlang@REDACTED (Thomas Lindgren) writes: > > |> Threads Cannot Be Implemented As A Library, Hans-J. Boehm. > > Interesting paper, but somewaht far from reality. Anyone who programmed > a concurrent application as shown in the examples in this paper would > need their heads examined. Even in C... There are essentially two parts to the paper: sections 1-4 which talk about problems with the specification of pthreads and interactions with compiler optimizations, and section 5 which argues for lock-free programming facilities in C/C++. I agree that anyone who tries to use lock-free primitives for general application programming needs their head examining. (This does not apply to using lock-free primitives to implement higher-level concurrency primitives in languages and operating systems.) However, the first part of the paper is completely on-target about the inadequacy of the pthreads specification. For example: # The current Pthreads specification explicitly allows [word tearing], # without any restriction on the field types. By our reading, it even # allows it for adjacent global variables outside of a struct declaration. # Since linkers may, and commonly do, reorder globals, this implies that # an update to any global variable may potentially read and rewrite any # other global variable. # We do not believe that any interesting Pthreads programs can be claimed # to be portable under these rules. -- David Hopwood From vlad_dumitrescu@REDACTED Wed Jul 20 16:22:01 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Wed, 20 Jul 2005 16:22:01 +0200 Subject: ex11, plotting and Cairo ... References: <1121837323.5440.27.camel@adic> Message-ID: ----- Original Message ----- From: "Jouni Ryn?" > Cairo has the PDF 1.4 programming model, and seems to handle the font > and text related needs at reasonable level (alpha and such ...) And it > would be the canvas only API ... Windows or whatever drawing backend is > needed, it's selected before the drawing is done. I would also like Cairo very much, but there is one catch so far (AFAIK): Windows support is still shaky. Compiling an application with cairo, freetype, libpng and the other libs isn't directly straightforward on Windows either. /Vlad From vlad_dumitrescu@REDACTED Wed Jul 20 16:55:30 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Wed, 20 Jul 2005 16:55:30 +0200 Subject: ex11, plotting and Cairo ... References: <1121853819.16160.27.camel@hymir.newport-networks.com> <1121867609.16160.48.camel@hymir.newport-networks.com> Message-ID: ----- Original Message ----- From: "Peter-Henry Mander" >Xming (http://freedesktop.org/wiki/Xming) looks like it fits the > bill, but I'm hitting a wall with it. What is the problem? > I'm not sure I grasp the meaning of your question. The whole idea is to > avoid reinvention by using X11. Joe is very keen to make it drop dead > gorgeous, but he's invented ex11 which to my mind is "good enough" and > it may become the enemy of his better PDF/PS GUI. What I meant was that IMHO some parts of the X11 protocol aren't needed by a simplified version. Also, some of the enhanced functionality is to be found in X11 extensions. Maybe the simplified X server could be just a protocol middle-man? That is, it could talk to the real X server and handle those things that the simplified version should be oblivious of. (It's been a while since working with X, so maybe there aren't many such things in reality). regards, Vlad From joe.armstrong@REDACTED Wed Jul 20 18:00:38 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Wed, 20 Jul 2005 18:00:38 +0200 Subject: ex11, plotting and Cairo ... Message-ID: > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Peter-Henry > Mander > Sent: den 20 juli 2005 15:53 > To: Vlad Dumitrescu > Cc: Erlang Questions > Subject: Re: ex11, plotting and Cairo ... > > > Hi Vlad, > > Simplified portable X server sounds good to me, it would be > very useful > to me as a stand-alone product in itself! If there's a Open > Source one, > even better! Cygwin X server has a bit too much baggage to be called > simple. Xming (http://freedesktop.org/wiki/Xming) looks like > it fits the > bill, but I'm hitting a wall with it. > > I'm not sure I grasp the meaning of your question. The whole > idea is to > avoid reinvention by using X11. Joe is very keen to make it drop dead > gorgeous, but he's invented ex11 which to my mind is "good enough" and > it may become the enemy of his better PDF/PS GUI. > > Sorry Joe, I know what you'd like to see since I'm an ex-typography > student and I appreciate your aesthetic inclination, but the > concurrency > model and thin layers is what I want above the kerning, antialiasing > etc. But the primitives are the same! - suppose I have a primitive draw(Win, character, X, Y, font, glyfNumber) Which draws a particular Glyf at a X, Y then I get kerning by a "better" computation of where X and Y are - a stupid algorithm just takes the bounding box width to compute where the next character should go. So given a the same set of primitives we can have quick and dirty or beautiful formatting, still with a decent concurrency model and decent layering /Joe > > Pete. > > On Wed, 2005-07-20 at 13:48 +0200, Vlad Dumitrescu wrote: > > From: "Peter-Henry Mander" > > > Considering this, why not attack the MS-Windows GDI > directly in Erlang, > > > maybe with a k.i.s.s. C driver to ease the connection? > > > > Easiest would be to write a graphics server and talk with > it via a socket. I > > think someone said earlier he had been giving it a try. > This would actually > > be a simplified X server, and could be made portable too. > > > > Question is: should the protocol be compatible with X? If > yes, is there a > > danger[*] that this server will become an alternative X > server? If no, could > > it be mapped onto X? > > > > [*] in the meaning that it would mean to reinvent a lot of wheels... > > > > regards, > > Vlad > > > > > > > -- > "The Tao of Programming > flows far away > and returns > on the wind of morning." > > > > From vlad_dumitrescu@REDACTED Wed Jul 20 18:21:37 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Wed, 20 Jul 2005 18:21:37 +0200 Subject: ex11, plotting and Cairo ... References: Message-ID: ----- Original Message ----- From: "Joe Armstrong (AL/EAB)" >But the primitives are the same! - suppose I have a primitive > >draw(Win, character, X, Y, font, glyfNumber) > >Which draws a particular Glyf at a X, Y then I get kerning by a "better" >computation >of where X and Y are - a stupid algorithm just takes the bounding box width >to compute >where the next character should go. > >So given a the same set of primitives we can have quick and dirty or >beautiful >formatting, still with a decent concurrency model and decent layering Just a thought: maybe sometimes one wants to output unkerned text (for speed, maybe, or because the font is fixed-size), or wants a better kerning algorithm. Likewise sometimes one wants antialiased, semi-transparent painting, sometimes good old aliased opaque is just fine (or even desireable). So we need an API for specifying that too. I tried for a while ago to extract a simple API from the PDF model, or from the Java2D APIs, but if such issues should be taken care of, then there's not much simplification to be done... regards, Vlad From Jouni.Ryno@REDACTED Wed Jul 20 18:26:16 2005 From: Jouni.Ryno@REDACTED (Jouni =?ISO-8859-1?Q?Ryn=F6?=) Date: Wed, 20 Jul 2005 19:26:16 +0300 Subject: ex11, plotting and Cairo ... In-Reply-To: References: Message-ID: <1121876776.5440.40.camel@adic> On Wed, 2005-07-20 at 09:11 +0200, Joe Armstrong (AL/EAB) wrote: snip > I'm currently working on VSG with a TCL/TK backend. > snip > One thing I did find was that the only model I like is based on a canvas > widget. To create buttons (and controls in general) I just draw rectangles and lines > onto a canvas widget and animate them on mouse clicks. Knowing the Tk canvas, I can understand how it would be a good to have model. As I have the tendency to start from details and low level, I could think something like this: We could have the cairo model on top of ex11 with some extensions. Like adding the item tags. And keeping a list of item enclosures. Then a click point to the canvas would be send through the item list. To see if any item under the mouse is interested. Something like that must already happen under the Tk-hood. The reason for pushing for Cairo like library is the possibility for different backends. Printing/picturing the nice computer generated images being always the issue. Cairo has recently been put into so many main stream open source projects, that it should have a bright future. On the other hand, the same projects use C++ or java. Does that ring any bells. regards Jouni -- Jouni Ryn? mailto://Jouni.Ryno@REDACTED/ http://www.geo.fmi.fi/~ryno/ Finnish Meteorological Institute http://www.fmi.fi/ Space Research http://www.geo.fmi.fi/ P.O.BOX 503 Tel (+358)-9-19294656 FIN-00101 Helsinki FAX (+358)-9-19294603 Finland priv-GSM (+358)-50-5302903 "It's just zeros and ones, it cannot be hard" From samuel@REDACTED Wed Jul 20 18:41:15 2005 From: samuel@REDACTED (Samuel Rivas) Date: Wed, 20 Jul 2005 18:41:15 +0200 Subject: Three wishes In-Reply-To: <200507200411.j6K4Bo8W210171@atlas.otago.ac.nz> References: <200507200411.j6K4Bo8W210171@atlas.otago.ac.nz> Message-ID: <20050720164115.GA2091@crusher.lfcia.pri> Richard A. O'Keefe wrote: > My argument is that we "must have" this feature because it is USED. > The person who hates it, hates it because it is USED. > > Samuel Rivas replied: > > Well, it is not really a "must" but a "could". > > One of us is misunderstanding "must have". I think that a feature which > was designed for good reason (familiarity to Prolog and Strand programmers) > and which is in active use by many people is a "must have" We had different interpretations, I understood "must have" as a thing that is needed to write code that couldn't be wrote otherwise (in a block with similar amount of sentences). I agree there is no point to disable this feature breaking loads of old code and I also agree that, in this matter, exported variables are a "must have". > Personally, I feel matching the if result is more erlangish > > I would dispute that. Part of the point here is that a dumb compiler > can generate good code WITHOUT having to slog through an analysis to > determine that certain tuples don't *really* need to exist. I think basing your coding style in the compiler shortcomings/limitations is not a good idea if it obscures the code. The point here is whether exported variables should be avoided or not basing in the clarity of the programs. > test(X, Y) -> > {Sign, Abs} = if X > Y -> {1, X - Y}; > X < Y -> {-1, Y - X}; > X == Y -> {0, 0} > end, > ... More sentences ... > > This > [ ... Several reasons against previous block ... ] > > None of this strikes me as particularly "Erlangish", and it seems to me > one of the merits of the language that it offers you a choice here. Of course there are points against both approaches. On the other hand, I like better considering an if/case block as a box that returns something rather than a block that *does* something. In my view, it is more functional: you can easily change the if for a function. > I see this more as a potential "best practice" than a reason > for the compiler to print a warning. > > It's odd to see something which separates names from values and makes > more mistakes not only possible but likely described as "best practice". I said _potential_ :). If hundreds of reasons encouraged one of the two coding styles, that would probably be a "best practice". Otherwise, it will remain, as I said, a matter of taste. -- Samuel From vlad_dumitrescu@REDACTED Wed Jul 20 18:42:08 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Wed, 20 Jul 2005 18:42:08 +0200 Subject: ex11, plotting and Cairo ... References: <1121876776.5440.40.camel@adic> Message-ID: ----- Original Message ----- From: "Jouni Ryn?" > We could have the cairo model on top of ex11 with some extensions. Like > adding the item tags. And keeping a list of item enclosures. Then a > click point to the canvas would be send through the item list. To see if > any item under the mouse is interested. Something like that must already > happen under the Tk-hood. I have the very basics of such an event handling system, modeled after the QNX Photon library. If there is interest, I can make it available for inspiration. One basic question is also: will these items or regions be active objects? There are pros and cons with both alternatives. /Vlad From ke.han@REDACTED Wed Jul 20 19:45:15 2005 From: ke.han@REDACTED (ke.han) Date: Thu, 21 Jul 2005 01:45:15 +0800 Subject: utf-8 or other unicode support Message-ID: <42DE8DAB.1050206@redstarling.com> Hello, Its my understanding that erlang strings are not utf-8 or otherwise unicode. I need to develop a web application to support Latin and Asian languages. For HTML pages, I"m assuming I can just code them in the proper language and yaws will handle things (This is a big assumption and I would like feedback on this). What about handling data in mnesia or dets? In years past when a DB did not support utf-8, but my Java strings were utf-8, I could encode them to binary for storage. Unfortunately, this diminishes query ability. In addition, since most of app data was in Latin characters, I had to encode/decode fields just in case which added lots of unnecessary overhead. What have others done? thanks, ke han From jerome@REDACTED Wed Jul 20 20:20:36 2005 From: jerome@REDACTED (=?ISO-8859-1?Q?J=E9r=F4me_Desquilbet?=) Date: Wed, 20 Jul 2005 20:20:36 +0200 Subject: utf-8 or other unicode support In-Reply-To: <42DE8DAB.1050206@redstarling.com> References: <42DE8DAB.1050206@redstarling.com> Message-ID: <42DE95F4.4030904@desquilbet.org> I have done this successfully: display utf-8 pages with Yaws, capture text in input fields, save them into files, load the files, same but with storage in mnesia tables, then display the text again on a page... I have done this with Yaws running on MacOSX, the web browser being either Safari on MacOSX, of Firefox on MacOSX or on Windows XP, it works. I will soon check the server side on Linux, but I guess it will work, too. The only small problem is with tv that don't show the accented letters properly from the mnesia fields (a Tk issue I guess). So, I think you can spend some time on a small mockup, at least it is not sure it will break :-) J?r?me. ke.han wrote: > Hello, > Its my understanding that erlang strings are not utf-8 or otherwise > unicode. > I need to develop a web application to support Latin and Asian languages. > > For HTML pages, I"m assuming I can just code them in the proper language > and yaws will handle things (This is a big assumption and I would like > feedback on this). > > What about handling data in mnesia or dets? > In years past when a DB did not support utf-8, but my Java strings were > utf-8, I could encode them to binary for storage. Unfortunately, this > diminishes query ability. In addition, since most of app data was in > Latin characters, I had to encode/decode fields just in case which added > lots of unnecessary overhead. > > What have others done? > thanks, ke han > From carsten@REDACTED Wed Jul 20 20:20:21 2005 From: carsten@REDACTED (Carsten Schultz) Date: Wed, 20 Jul 2005 20:20:21 +0200 Subject: utf-8 or other unicode support In-Reply-To: <42DE8DAB.1050206@redstarling.com> References: <42DE8DAB.1050206@redstarling.com> Message-ID: <42DE95E5.9080100@codimi.de> Hi! ke.han schrieb: > Its my understanding that erlang strings are not utf-8 or otherwise > unicode. > I need to develop a web application to support Latin and Asian languages. > > For HTML pages, I"m assuming I can just code them in the proper language > and yaws will handle things (This is a big assumption and I would like > feedback on this). Unfortunately not true, but not a big problem either. Here it csn be a good thing that Erlang has no string type anyway. Strings are just lists of ints, and if your input and output routines can deal with it, there is no reason to restrict yourself to ints <256. You can have larger ints and simply claim that they are unicode. Regarding output, I have hacked yaws_api.erl to have a changed definition of htmlize_char/1: htmlize_char($>) -> <<">">>; htmlize_char($<) -> <<"<">>; htmlize_char($&) -> <<"&">>; htmlize_char($") -> <<""">>; htmlize_char(C) when C>255 -> [$&,$#,integer_to_list(C),$;]; htmlize_char(X) -> X. (htmlize/1 also has to be modified slightly.) For my application with only a few non-iso-8859-1 characters that works nicely. You might want to do your own utf-8-encoding at that point. Regards, Carsten -- Carsten Schultz (2:38, 33:47) http://carsten.codimi.de/ PGP/GPG key on the pgp.net key servers, fingerprint on my home page. From Jouni.Ryno@REDACTED Wed Jul 20 21:06:39 2005 From: Jouni.Ryno@REDACTED (Jouni =?ISO-8859-1?Q?Ryn=F6?=) Date: Wed, 20 Jul 2005 22:06:39 +0300 Subject: ex11, plotting and Cairo ... In-Reply-To: References: Message-ID: <1121886399.5440.64.camel@adic> Finding a simple subset from a huge parameter space may not be possible. But how about different backends for canvas. All primitives could have options list, default values given upon canvas initialisation. Application could then mix different canvas types, fast for window user interface, slow and fancy, where it matters. So the it would still be simple draw(Win, character, X, Y, font, glyfNumber) AND draw(Win, character, X, Y, font, glyfNumber, [tag=fancy, render=xlib, kern=[], alpha=0.9, justification=right, ...]) Written that, I don't know, if it's a good idea to allow primitives so much. Changing the rendering backend, really not. But something like that. Anyway, it must be possible to make out something with the basic command. If it comes out bad looking, one can start twiddling with options. Usually, at first you don't care. (like "make it first to work, then make it fast") And about the canvas item tags. Those without tags would be like labels, no activity expected. With tags, user action to be done i.e. active objects. Or something like this. Jouni ----- Original Message ----- From: "Jouni Ryn?" > We could have the cairo model on top of ex11 with some extensions. Like > adding the item tags. And keeping a list of item enclosures. Then a > click point to the canvas would be send through the item list. To see if > any item under the mouse is interested. Something like that must already > happen under the Tk-hood. I have the very basics of such an event handling system, modeled after the QNX Photon library. If there is interest, I can make it available for inspiration. One basic question is also: will these items or regions be active objects? There are pros and cons with both alternatives. /Vlad -- Jouni Ryn? mailto://Jouni.Ryno@REDACTED/ http://www.geo.fmi.fi/~ryno/ Finnish Meteorological Institute http://www.fmi.fi/ Space Research http://www.geo.fmi.fi/ P.O.BOX 503 Tel (+358)-9-19294656 FIN-00101 Helsinki FAX (+358)-9-19294603 Finland priv-GSM (+358)-50-5302903 "It's just zeros and ones, it cannot be hard" From ke.han@REDACTED Wed Jul 20 23:22:13 2005 From: ke.han@REDACTED (ke.han) Date: Thu, 21 Jul 2005 05:22:13 +0800 Subject: utf-8 or other unicode support In-Reply-To: <20050720184221.C45BD46DCA@bang.trapexit.org> References: <20050720184221.C45BD46DCA@bang.trapexit.org> Message-ID: <42DEC085.9050201@redstarling.com> tobbe wrote: > I think you should check out the 'gettext' package which > just has been released. > > Cheers, Tobbe > _________________________________________________________ > Sent using Mail2Forum (http://m2f.sourceforge.net) > excellent...nothing like "just-in-time code" to save the day!!! thanks, ke han From meena_selvam@REDACTED Wed Jul 20 23:41:22 2005 From: meena_selvam@REDACTED (MEENA SELVAM) Date: Wed, 20 Jul 2005 14:41:22 -0700 (PDT) Subject: how to check, if a function has been called Message-ID: <20050720214122.41965.qmail@web30405.mail.mud.yahoo.com> Hi, When I run a erlang program , how can i check a method has been called.. apart from putting prints just typing a function name or similar..? meena __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From vances@REDACTED Thu Jul 21 00:14:06 2005 From: vances@REDACTED (Vance Shipley) Date: Wed, 20 Jul 2005 18:14:06 -0400 Subject: Erlang (mild) PL buzz In-Reply-To: <20050720091222.13666.qmail@web34415.mail.mud.yahoo.com> References: <20050720091222.13666.qmail@web34415.mail.mud.yahoo.com> Message-ID: <20050720221406.GA623@feeble.motivity.ca> On Wed, Jul 20, 2005 at 02:12:22AM -0700, Thomas Lindgren wrote: } } Fame at last? :-) Imitation is the sincerest form of flattery: Termite: http://lambda-the-ultimate.org/node/view/841 Erlisp: http://www.dirkgerrits.com/programming/erlisp/ -Vance From ulf@REDACTED Thu Jul 21 00:47:13 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 21 Jul 2005 00:47:13 +0200 Subject: how to check, if a function has been called In-Reply-To: <20050720214122.41965.qmail@web30405.mail.mud.yahoo.com> References: <20050720214122.41965.qmail@web30405.mail.mud.yahoo.com> Message-ID: Den 2005-07-20 23:41:22 skrev MEENA SELVAM : > Hi, > When I run a erlang program , how can i check a method > has been called.. apart from putting prints > > just typing a function name or similar..? > meena You can use the dbg.erl module, which makes use of the built-in trace support in Erlang. Example: Eshell V5.4.8 (abort with ^G) 1> dbg:tracer(). {ok,<0.32.0>} 2> dbg:tp(lists,sort,'_',[]). {ok,[{matched,nonode@REDACTED,2}]} 3> dbg:p(all,[c]). {ok,[{matched,nonode@REDACTED,25}]} 4> lists:sort([4,3,1,5,2]). (<0.30.0>) call lists:sort([4,3,1,5,2]) <== Here! [1,2,3,4,5] If you want to see what the function returns: Eshell V5.4.8 (abort with ^G) 1> dbg:tracer(). {ok,<0.32.0>} 2> dbg:tp(lists,sort,'_',dbg:fun2ms(fun(_) -> return_trace() end)). {ok,[{matched,nonode@REDACTED,2},{saved,1}]} 3> dbg:p(all,[c]). {ok,[{matched,nonode@REDACTED,25}]} 4> lists:sort([4,3,1,5,2]). (<0.30.0>) call lists:sort([4,3,1,5,2]) [1,2,3,4,5](<0.30.0>) returned from lists:sort/1 -> [1,2,3,4,5] <=== ! dbg:tp/4 defines a trace pattern on exported functions. dbg:tpl/4 does the same for non-exported functions. Regards, Uffe From richardc@REDACTED Thu Jul 21 01:16:54 2005 From: richardc@REDACTED (Richard Carlsson) Date: Thu, 21 Jul 2005 01:16:54 +0200 Subject: Erlang (mild) PL buzz In-Reply-To: <20050720221406.GA623@feeble.motivity.ca> References: <20050720091222.13666.qmail@web34415.mail.mud.yahoo.com> <20050720221406.GA623@feeble.motivity.ca> Message-ID: <42DEDB66.30204@csd.uu.se> Vance Shipley wrote: > Imitation is the sincerest form of flattery: > > Termite: http://lambda-the-ultimate.org/node/view/841 > Erlisp: http://www.dirkgerrits.com/programming/erlisp/ And none of them referenced Alexander Bottema's masters thesis from way back in 1996, " An Asynchronous Message Passing and Concurrent Scheme Implementation": ftp://ftp.csd.uu.se/pub/papers/masters-theses/0090-bottema.ps.gz /Richard (I have mailed them the link; you don't have to.) From meena_selvam@REDACTED Thu Jul 21 01:44:06 2005 From: meena_selvam@REDACTED (MEENA SELVAM) Date: Wed, 20 Jul 2005 16:44:06 -0700 (PDT) Subject: what is wrong with this code setting a nested record? Message-ID: <20050720234406.71358.qmail@web30411.mail.mud.yahoo.com> Hi, the parameter A is of record type arg, which contains a subrecord of type, headers and which has a field host: The host field initially contains www.msn.com, and I am trying to overwrite with 47.80.18.95, but the following code doesnt work.. it still has www.msn.com itself. display_login2(A, ReplyMsgs, URL, IP, FromLogout) -> (A#arg.headers)#headers{host="47.80.18.95"}, am i doing anything wrong here? meena __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From meena_selvam@REDACTED Thu Jul 21 02:21:26 2005 From: meena_selvam@REDACTED (MEENA SELVAM) Date: Wed, 20 Jul 2005 17:21:26 -0700 (PDT) Subject: How can i access the other header of yaws arg structure, Message-ID: <20050721002126.34798.qmail@web30402.mail.mud.yahoo.com> Hi, I have a arg record A passed by the yaws webserver to the web application: How can i access the 3rd element of the other header, whereas I have the other header itself is available as (A#arg.headers)#headers.Other A= {arg,#Port<16338.534>, {headers,undefined, "image/gif, image/x-xbitmap, image/jpeg, image/ pjpeg, */*", "www.msn.com", undefined, undefined, undefined, undefined, undefined, undefined, undefined, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)", undefined, [], undefined, undefined, undefined, undefined, undefined, undefined, undefined, [{http_header, 0, "X-Yaws-Had-Clicert", undefined, "false"}, {http_header, 49, 'X-Forwarded-For', undefined, "11.10.1.107"}, {http_header, 7, 'Via', undefined, "1.1 www.msn.com (Alteon iSD-SSL/5.5)"}, {http_header, 0, "X-Forwarded-By", undefined, "47.80.18.95"}, {http_header, 10, 'Accept-Encoding', undefined, "gzip, deflate"}, {http_header, 11, 'Accept-Language', undefined, "en-us"}]}, {http_request,'GET',{abs_path,"/"},{1,1}}, undefined, "/", [], undefined, .... and much more The records arg and header are defined as below: -record(arg, { clisock, %% the socket leading to the peer client headers, %% headers req, %% request ..... some more }). -record(headers, { connection, accept, host, if_modified_since, if_match, if_none_match, if_range, if_unmodified_since, range, referer, user_agent, accept_ranges, cookie = [], keep_alive, location, content_length, content_type, content_encoding, authorization, transfer_encoding, other = [] %% misc other headers }). __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From hakan.stenholm@REDACTED Thu Jul 21 02:39:31 2005 From: hakan.stenholm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Stenholm?=) Date: Thu, 21 Jul 2005 02:39:31 +0200 Subject: what is wrong with this code setting a nested record? In-Reply-To: <20050720234406.71358.qmail@web30411.mail.mud.yahoo.com> References: <20050720234406.71358.qmail@web30411.mail.mud.yahoo.com> Message-ID: <42DEEEC3.8040801@mbox304.swipnet.se> MEENA SELVAM wrote: >Hi, > >the parameter A is of record type arg, which contains >a subrecord of type, headers and which has a field >host: > >The host field initially contains www.msn.com, and I >am trying to overwrite with 47.80.18.95, but the >following code doesnt work.. it still has www.msn.com >itself. > >display_login2(A, ReplyMsgs, URL, IP, FromLogout) -> > > (A#arg.headers)#headers{host="47.80.18.95"}, > >am i doing anything wrong here? > > It should work (see my test code below), but may have forgotten to do: NewHeader = (A#arg.headers)#headers{host="47.80.18.95"}, ... xxx(NewHeader, ...) %% use the new (updated) Header rather than the old one in A ... It would be helpful to see a bit more of your code, I get the feeling that your trying to do a destructive update on a record in a single assignment language, this won't work - data elements in Erlang can only be created but never modified[1] (with the exception for ets/mnesia/process dictionaries which can conceptually be viewed as processes storing data in some regular Erlang data type like a binary tree - which is updated by creating new nodes and leaves to replace the old ones - rather than overwriting old ones). [1] : deletion is done by the GC. ================== test.erl ===================== -module(test). -export([test/0]). -record(foo, { bar }). -record(bar, { a, b }). test() -> Bar = #bar{a = 1, b = 2}, Foo = #foo{bar = Bar}, io:format("Foo = ~p~n", [Foo]), %% create new bar record %% access foo record and create new bar record based on foo.bar Bar2 = (Foo#foo.bar)#bar{b = new_val}, io:format("Bar2 = ~p~n", [Bar2]), %% create new foo record %% create new foo record containing the new Bar2 bar record Foo2 = Foo#foo{bar = Bar2}, io:format("Foo2 = ~p~n", [Foo2]), ok. ================== test.erl ===================== 2> c(test). {ok,test} 3> test:test(). Foo = {foo,{bar,1,2}} Bar2 = {bar,1,new_val} Foo2 = {foo,{bar,1,new_val}} ok >meena > > >__________________________________________________ >Do You Yahoo!? >Tired of spam? Yahoo! Mail has the best spam protection around >http://mail.yahoo.com > > > From hakan.stenholm@REDACTED Thu Jul 21 02:47:53 2005 From: hakan.stenholm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Stenholm?=) Date: Thu, 21 Jul 2005 02:47:53 +0200 Subject: How can i access the other header of yaws arg structure, In-Reply-To: <20050721002126.34798.qmail@web30402.mail.mud.yahoo.com> References: <20050721002126.34798.qmail@web30402.mail.mud.yahoo.com> Message-ID: <42DEF0B9.90902@mbox304.swipnet.se> MEENA SELVAM wrote: >Hi, > >I have a arg record A passed by the yaws webserver to >the web application: > > >How can i access the 3rd element of the other header, >whereas I have the other header itself is available as >(A#arg.headers)#headers.Other > > #headers.other appears to be a list, the third element of a list can be accessed in two ways: %% using pattern matching [_,_,ThirdElement|_] = List %% or %% using nth/2 in the lists module ThirdElement = lists:nth(3, List) >A= {arg,#Port<16338.534>, > {headers,undefined, > "image/gif, >image/x-xbitmap, image/jpeg, image/ >pjpeg, */*", > "www.msn.com", > undefined, > undefined, > undefined, > undefined, > undefined, > undefined, > undefined, > "Mozilla/4.0 >(compatible; MSIE 6.0; Windows NT >5.1)", > undefined, > [], > undefined, > undefined, > undefined, > undefined, > undefined, > undefined, > undefined, > [{http_header, > 0, > >"X-Yaws-Had-Clicert", > undefined, > "false"}, > {http_header, > 49, > >'X-Forwarded-For', > undefined, > "11.10.1.107"}, > {http_header, > 7, > 'Via', > undefined, > "1.1 www.msn.com >(Alteon iSD-SSL/5.5)"}, > {http_header, > 0, > >"X-Forwarded-By", > undefined, > "47.80.18.95"}, > {http_header, > 10, > >'Accept-Encoding', > undefined, > "gzip, >deflate"}, > {http_header, > 11, > >'Accept-Language', > undefined, > "en-us"}]}, > >{http_request,'GET',{abs_path,"/"},{1,1}}, > undefined, > "/", > [], > undefined, > .... and much more > > >The records arg and header are defined as below: > >-record(arg, { > clisock, %% the socket leading to the >peer client > headers, %% headers > req, %% request > > ..... some more > }). > >-record(headers, { > connection, > accept, > host, > if_modified_since, > if_match, > if_none_match, > if_range, > if_unmodified_since, > range, > referer, > user_agent, > accept_ranges, > cookie = [], > keep_alive, > location, > content_length, > content_type, > content_encoding, > authorization, > transfer_encoding, > other = [] %% misc other headers > }). > > > >__________________________________________________ >Do You Yahoo!? >Tired of spam? Yahoo! Mail has the best spam protection around >http://mail.yahoo.com > > > From harveyd@REDACTED Thu Jul 21 03:17:31 2005 From: harveyd@REDACTED (Dale Harvey) Date: Thu, 21 Jul 2005 02:17:31 +0100 Subject: Fwd: Yaws / mod_rewrite alternative In-Reply-To: References: Message-ID: Hey Coming from a php/apache background, I was wondering if there was any simple way to emulate mod_rewrite in yaws, I would like all pages on a site server from a single function, where i can parse the url manually to get the input i need. appmod will allow me to internally redirect the request, but it does so regardless of extension, and I havent been able to find a way around this, from what i can gather I cant serve css / jpg files from out/1 as it needs a {html or {ehtml tuple returned, and even if it were possible it doesnt seem like the best solution basically what i want is RewriteRule !\.(gif|jpg|png|css|js)$ mymodule:out(A) Cheers Dale From harveyd@REDACTED Thu Jul 21 03:55:39 2005 From: harveyd@REDACTED (Dale Harvey) Date: Thu, 21 Jul 2005 02:55:39 +0100 Subject: Yaws / mod_rewrite alternative Message-ID: Hey Coming from a php/apache background, I was wondering if there was any simple way to emulate mod_rewrite in yaws, I would like all pages on a site server from a single function, where i can parse the url manually to get the input i need. appmod will allow me to internally redirect the request, but it does so regardless of extension, and I havent been able to find a way around this, from what i can gather I cant serve css / jpg files from out/1 as it needs a {html or {ehtml tuple returned, and even if it were possible it doesnt seem like the best solution basically what i want is RewriteRule !\.(gif|jpg|png|css|js)$ mymodule:out(A) Cheers Dale From bfulg@REDACTED Thu Jul 21 05:40:24 2005 From: bfulg@REDACTED (Brent Fulgham) Date: Wed, 20 Jul 2005 20:40:24 -0700 Subject: Compiler Error (Wings3d on Mac OS X) In-Reply-To: <200507200806.j6K86Qsj008984@spikklubban.it.uu.se> References: <200507200806.j6K86Qsj008984@spikklubban.it.uu.se> Message-ID: <53B75770-260C-4904-B973-3E88E254DEEB@pacbell.net> On Jul 20, 2005, at 1:06 AM, Kostis Sagonas wrote: > Make the code look as follows and it will pass the compiler: > > internal_rgb_to_hsv(R, G, B) -> > Max = lists:max([R,G,B]), > Min = lists:min([R,G,B]), > V = Max, > {Hue,Sat} = %% try > {if > Min == B -> (G-Min)/(R+G-2.0*Min); > Min == R -> (1.0+(B-Min)/(B+G-2.0*Min)); > Min == G -> (2.0+(R-Min)/(B+R-2.0*Min)) > end*120,(Max-Min)/Max} > %% catch > %% error:badarith -> > %% {undefined,0.0} > %% end > , > {Hue,Sat,V}. Well, this certainly allows the compile to continue, but I run across man more cases of the try/catch causing problems. They all seem to be related to error:badarith. Apparently, this is a bug in R10B-6, which will be corrected in R10B-7 (per http://sourceforge.net/tracker/index.php? func=detail&aid=1231771&group_id=33028&atid=406952). -Brent -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part URL: From meena_selvam@REDACTED Thu Jul 21 06:11:28 2005 From: meena_selvam@REDACTED (MEENA SELVAM) Date: Wed, 20 Jul 2005 21:11:28 -0700 (PDT) Subject: browsers behaviour towards yaws http redirect - unclear Message-ID: <20050721041128.52991.qmail@web30407.mail.mud.yahoo.com> The redirect below adds a "Location:" header to the responce headers to the Browser. The Browser then displays the new page as indicated in the "Location" header. out(_Arg) -> L="http://www.google.com/search?num=20&hl=en&lr=lang_en%7Clang_sv&q=yaws", {redirect, L}. What other things can replace L in the {redirect, L}. Is there any other list of values that can be passed along with redirect? Ie what are the other forms of redirect return values? In my situation, the code was written by someone else.. the redirection occurs as expected, that is the redirected page contents are displayed in the browser, but still the address field in the browser is same as the original uri. What is controlling the address field of the redirected page. Does the browser look into the http response headers to decide on this or it is designed to put the original uri itself ? Ideally it should put the location as the address field ? am I right? Or is it possible, because a proxy is involved, deliberately the redirected uri is deliberately hidden from the http client? In other words, can there be any motive for such a thing .. ie browser still sticking on to old uri? Possibility 2: Again, in my situation, the redirection is done at dns level also. It is quite possible that the browser gets back the redirected uri / ip as part of the dns response (custom dns server is designed to do this. That is any URI, in the browser is resolved to a fixed ip address). If this is the case, the browser will send a http-request to the new ip 47.80.18.95. a yaws webserver is running in this host and listening to http requests. So the index page is displayed in the browser correctly. But the address field is still the old, say www.msn.com So when is there a possibility that a http request (which is just after a dns manipulation) is serviced correctly to the extent, the proper page is displayed, but with a flaw namely the address field remains to be the old one before the dns lookup? In both possibilities, is this a browser issue? Or is it something to do with the redirect at the web server . In the web server code, redirects are involved. The code is as follows: we have a captive portal module which is dns based redirection of all uri to a fixed ip. %% This is for the WSS captive portal redirect server (listening on port 8888) captive_redirect(A) -> O = A#arg.opaque, RedirUrl = O#rdir_opaque.redir_url, OrigArg = A#arg{ req = O#rdir_opaque.orig_req }, OrigUrlTmp = yaws_api:request_url(OrigArg), OrigUrl = lists:flatten( yaws_api:format_url(OrigUrlTmp#url{port = undefined})), Vars = [{"url", OrigUrl}, {"host", (A#arg.headers)#headers.host}, {"path", begin {abs_path, P} = (O#rdir_opaque.orig_req)#http_request.path, P end}, {"method", atom_to_list((O#rdir_opaque.orig_req)#http_request.method)} ], [{redirect, expand_link_href_vb(RedirUrl, Vars, A)}]. expand_link_href_vb(Str, Vars, A) -> expand_link_href_vb(Str, Vars, A, true). expand_link_href_vb([], _, _,_) -> []; expand_link_href_vb([" Var = string:strip(VarX, right, $>), case lists:keysearch(Var, 1, Vars) of {value, {_, Val}} when UrlEncodeP == true -> [yaws_api:url_encode(Val) | expand_link_href_vb(T, Vars, A, UrlEncodeP)]; {value, {_, Val}} -> [Val | expand_link_href_vb(T, Vars, A, UrlEncodeP)]; _ -> [" [H|expand_link_href_vb(T, Vars, A, UrlEncodeP)]. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From kramer@REDACTED Thu Jul 21 08:06:46 2005 From: kramer@REDACTED (Reto Kramer) Date: Wed, 20 Jul 2005 23:06:46 -0700 Subject: erlmerge - The new way to install Erlang apps. In-Reply-To: <20050720184115.8373046D7C@bang.trapexit.org> References: <20050720184115.8373046D7C@bang.trapexit.org> Message-ID: Tobbe, Erlmerge is great! Quick question though - let's say I do install an app like the esmb package itself that declares it's package dependencies to be the specific version numbers you have in your example. {dependencies,[..., ..., {fd_server,"2.3.0"}, ...]} Some time later I upgrade fd_server to 2.4.0 (some app, other than esmb, may require 2.4 and pulls it in). The good citizen developer of fd_server of course made the 2.4 version backwards compatible with 2.3 (promise ;-). Will I need to create a new esmb.app file (which includes the fd_server 2.4 reference) and force all users to "upgrade" esmb, or is the dependency declaration interpreted by the esmb system as ... at least {fd_server, "2.3.0"} ... but if any higher number is already installed the dependency is satisfied? Or is the dependency declaration a string literal match with what's currently in the package database? The fd_server 2.3.0 version is still in the erl/lib directory. Will it remain in the package database also? What I'm looking for is really a way to say: "at least version 2.3 of fd_server". Is that the interpretation that is given to the declaration? Thanks, - Reto On Jul 20, 2005, at 11:41 AM, tobbe wrote: > > erlmerge is a tool that makes it extremely easy to install > > and maintain Erlang software. The tool keeps a package > > database which makes it possible to install, delete, update > > Erlang applications. It also handles dependencies between > > applications. > > > > A full description can be found among the HowTo's: > > > > http://www.trapexit.org/docs/howto/erlmerge.html > > > > It is still to be regarded as somewhat Beta, but I would > > like to get some feedback if possible. > > > > As a short teaser. To install the gettext package into your > > Erlang installation, do: > > > > erlmerge -i gettext > > > > erlmerge will fetch the tar ball via HTTP and install > > it into your Erlang installation. After the above you > > just start Erlang, and voila: > > > > # erl > > Erlang (BEAM) emulator version 5.4.6 [source] [hipe] > > Eshell V5.4.6 (abort with ^G) > > 1> m(gettext). > > Module gettext compiled: Date: July 20 2005, Time: 15.54 > > Compiler options: > [{cwd,"/usr/local/lib/erlang/lib/gettext-1.3.0/src"}, > > ....etc... > > > > Cheers, Tobbe > _________________________________________________________ > Sent using Mail2Forum (http://m2f.sourceforge.net) > From david.nospam.hopwood@REDACTED Thu Jul 21 10:14:15 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Thu, 21 Jul 2005 09:14:15 +0100 Subject: ex11, plotting and Cairo ... In-Reply-To: References: Message-ID: <42DF5957.6080704@blueyonder.co.uk> Joe Armstrong (AL/EAB) wrote: > On Behalf Of Peter-Henry Mander >> >>Sorry Joe, I know what you'd like to see since I'm an ex-typography >>student and I appreciate your aesthetic inclination, but the >>concurrency model and thin layers is what I want above the kerning, >>antialiasing, etc. > > But the primitives are the same! - suppose I have a primitive > > draw(Win, character, X, Y, font, glyfNumber) > > Which draws a particular Glyf at a X, Y then I get kerning by a "better" computation > of where X and Y are - a stupid algorithm just takes the bounding box width to compute > where the next character should go. APIs that only draw individual glyphs don't work adequately with "complex" scripts [*]. Although in principle you can have glyphs for presentation forms and have the API/protocol client do all positioning and glyph selection, that is not how OpenType fonts are designed to work, and if you don't support OpenType then you'd be reinventing a lot of wheels. > So given a the same set of primitives we can have quick and dirty or beautiful > formatting, still with a decent concurrency model and decent layering Yes, but it should be possible to send a Unicode string over the network protocol, not just a set of glyphs and positions. The server, which has direct access to all the font data, is in a better position to do kerning and glyph selection, especially for large fonts. [*] Arabic, and various Indic and SE Asian scripts. Unfortunately, complex scripts are too often interpreted by Western API designers as "I don't read any of those obscure scripts, so I don't need to think about that yet." -- David Hopwood From erlang@REDACTED Thu Jul 21 10:34:24 2005 From: erlang@REDACTED (Peter-Henry Mander) Date: Thu, 21 Jul 2005 09:34:24 +0100 Subject: ex11, plotting and Cairo ... In-Reply-To: References: <1121853819.16160.27.camel@hymir.newport-networks.com> <1121867609.16160.48.camel@hymir.newport-networks.com> Message-ID: <1121934864.16160.76.camel@hymir.newport-networks.com> Hi Vlad, Xming troubles can be solved with RTFM I think. Removing access control (-ac) allows me to send an xterm display to Xming so it does work in priciple, albeit forgoing a secure connection I think. I'd prefer SSH tunneling. The following gives usable results, DirectDraw mode (-engine 2) redraws cleanly compared to default GDI, and -internalwm uses MS-Windows decorations. Xming -ac -internalwm -engine 2 I agree that the X11 protocol is a superset of what is strictly necessary for simple graphics, and the extensions even more so. The middleman concept is something Joe may approve of, I remember he posted something about this idea a while back. It also could serve as a mechanism to introduce support for X11 extensions without cluttering the basic API, e.g. antialiasing, alpha-blending etc... I wonder if the kerning algorithms could be supported in there too? Pete. -- "The Tao of Programming flows far away and returns on the wind of morning." On Wed, 2005-07-20 at 16:55 +0200, Vlad Dumitrescu wrote: > ----- Original Message ----- > From: "Peter-Henry Mander" > >Xming (http://freedesktop.org/wiki/Xming) looks like it fits the > > bill, but I'm hitting a wall with it. > > What is the problem? > [snipped] > > Maybe the simplified X server could be just a protocol middle-man? > That is, it could talk to the real X server and handle those things that the > simplified version should be oblivious of. (It's been a while since working > with X, so maybe there aren't many such things in reality). > > regards, > Vlad > From tim@REDACTED Thu Jul 21 11:33:33 2005 From: tim@REDACTED (Tim Bates) Date: Thu, 21 Jul 2005 19:03:33 +0930 Subject: Directory structure for non-OTP projects In-Reply-To: <42DCEF92.7080408@hyber.org> References: <42DCE32F.9090604@bates.id.au> <42DCEF92.7080408@hyber.org> Message-ID: <42DF6BED.3000202@bates.id.au> Claes Wikstom wrote: > I don't like the OTP build environment one bit. The Makefiles are hard > to read and debug. > If your project is reasonably small you could take a look at the > yaws build environment. It's easier/more naive than otp and doesn't use > any bootscripts at all. Thanks klacke, but the yaws build environment does not appear to deal with multiple applications, nor have I any idea how the test_server would fit in with all of this. I've looked at a couple of other open source projects, including eddie and ejabberd. None of them uses quite the same build environment or directory structure as OTP. I'd like not to roll my own, so: does anyone use the directory structure recommended in the OTP Design Principles? If so, how have they set up their build environment? How does this build environment interact with the erl shell, the test_server and OTP release handling? Thanks, Tim. -- Tim Bates tim@REDACTED From chandrashekhar.mullaparthi@REDACTED Thu Jul 21 12:31:04 2005 From: chandrashekhar.mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Thu, 21 Jul 2005 11:31:04 +0100 Subject: Directory structure for non-OTP projects In-Reply-To: <42DF6BED.3000202@bates.id.au> References: <42DCE32F.9090604@bates.id.au> <42DCEF92.7080408@hyber.org> <42DF6BED.3000202@bates.id.au> Message-ID: <7a6679dcd096e0fd9a29686d8ad1b5cf@t-mobile.co.uk> We have our own homegrown build tool which exports apps from CVS, does some tag checks, builds the beam files and then the boot scripts. It is not ideal but it kind of does the job. You might want to look at the builder app in jungerl. I was a beta tester for it and it is quite good. The only reason we never adopted it for our work is inertia. cheers Chandru On 21 Jul 2005, at 10:33, Tim Bates wrote: > Claes Wikstom wrote: >> I don't like the OTP build environment one bit. The Makefiles are hard >> to read and debug. >> If your project is reasonably small you could take a look at the >> yaws build environment. It's easier/more naive than otp and doesn't >> use >> any bootscripts at all. > > Thanks klacke, but the yaws build environment does not appear to deal > with multiple applications, nor have I any idea how the test_server > would fit in with all of this. > > I've looked at a couple of other open source projects, including eddie > and ejabberd. None of them uses quite the same build environment or > directory structure as OTP. I'd like not to roll my own, so: does > anyone > use the directory structure recommended in the OTP Design Principles? > If > so, how have they set up their build environment? How does this build > environment interact with the erl shell, the test_server and OTP > release > handling? > > Thanks, > Tim. From Marc.Vanwoerkom@REDACTED Thu Jul 21 16:42:09 2005 From: Marc.Vanwoerkom@REDACTED (Marc van Woerkom) Date: Thu, 21 Jul 2005 16:42:09 +0200 Subject: CPU/Hardware optimized for Erlang Message-ID: Has there been any cpu/hardware design optimized for Erlang? Would one aim for a beam virtual machine in hardware or something more lowlevel, but still optimized somehow? Regards, Marc From klacke@REDACTED Thu Jul 21 16:40:12 2005 From: klacke@REDACTED (Claes Wikstom) Date: Thu, 21 Jul 2005 16:40:12 +0200 Subject: Fwd: Yaws / mod_rewrite alternative In-Reply-To: References: Message-ID: <42DFB3CC.4090704@hyber.org> Dale Harvey wrote: > Hey > > Coming from a php/apache background, I was wondering if there was any > simple way to emulate mod_rewrite in yaws, > > I would like all pages on a site server from a single function, where > i can parse the url manually to get the input i need. > > appmod will allow me to internally redirect the request, but it does > so regardless of extension, and I havent been able to find a way > around this, from what i can gather I cant serve css / jpg files from > out/1 as it needs a {html or {ehtml tuple returned, and even if it > were possible it doesnt seem like the best solution > Hmm, you want dynmically generated css/jpg files shipped by yaws. I'm not sure how to to do that since it's the .yaws extension which makes yaws invoke the out/1 function and everything. > basically what i want is > RewriteRule !\.(gif|jpg|png|css|js)$ mymodule:out(A) > Take a look at # man yaws.conf arg_rewrite_mod = Module It is possible to install a module that rewrites all the Arg #arg{} records at an early stage in the yaws server. This can be used to do various things such as checking a cookie, rewrit- ing paths etc. All in all, I think you need to tell us better what it is you want to do. /klacke It may solve your problems ..... /klacke From thomasl_erlang@REDACTED Thu Jul 21 17:44:13 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 21 Jul 2005 08:44:13 -0700 (PDT) Subject: CPU/Hardware optimized for Erlang In-Reply-To: Message-ID: <20050721154413.98192.qmail@web34410.mail.mud.yahoo.com> --- Marc van Woerkom wrote: > Has there been any cpu/hardware design optimized > for Erlang? > > Would one aim for a beam virtual machine in hardware > or something more lowlevel, but still optimized > somehow? The canonical example is probably ECOMP http://www.erlang.se/euc/00/processor.ppt The BEAM originates in the WAM, which was intended for hardware implementation (well, microcode on a VAX). The details depend on what you want to do :-) I'm normally skeptical to special-purpose hardware for specific programming languages -- such chips have always gotten steamrollered by the killer micros. Instead, my advice would be to try to target an application niche, a problem. Graphics cards, network processors, the APZ, ... are all intended for applications, not languages. And have been fairly successful to boot. Best, Thomas ____________________________________________________ Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs From harveyd@REDACTED Thu Jul 21 17:46:17 2005 From: harveyd@REDACTED (Dale Harvey) Date: Thu, 21 Jul 2005 16:46:17 +0100 Subject: Fwd: Yaws / mod_rewrite alternative In-Reply-To: <42DFB3CC.4090704@hyber.org> References: <42DFB3CC.4090704@hyber.org> Message-ID: > Hmm, you want dynmically generated css/jpg files > shipped by yaws. I'm not sure how to to do that since > it's the .yaws extension which makes yaws invoke > the out/1 function and everything. > Im invoking out/1 from appmod in yaws.conf, > Take a look at > > # man yaws.conf > > arg_rewrite_mod = Module > It is possible to install a module that rewrites all the Arg > #arg{} records at an early stage in the yaws server. This can > be used to do various things such as checking a cookie, rewrit- > ing paths etc. > Heh yeh i noticed that, but erlang / yaws docs + google doesnt make anymore mention of it, or at least that i can find > All in all, I think you need to tell us better what it is you > want to do. I want all "pages" on a site served by a single function www.domain.com/ - served by mymodule:out(A) www.domain.com/home/ - served by mymodule:out(A) www.domain.com/anything/else - served by mymodule:out(A) and all files, css, images, js just given the actual file www.domain.com/style.css - to just send style.css I got it working, but its not the nicest solution, a series of appmod = <"home",mymodule> appmod = <"contact",mymodule> ect ect for each top level section and a rewrite for root ( I cant find how to just to <"/",mymodule> then i can just check is _regular(docroot++appmoddata), and redirect_local if it is a file, else check the url and send back html arg_mod_rewrite does seem like it would have the solution I want though, I just cant find anymore information on it Cheers Dale From harrisbhatti@REDACTED Thu Jul 21 18:03:10 2005 From: harrisbhatti@REDACTED (M. Harris Bhatti) Date: Thu, 21 Jul 2005 17:03:10 +0100 Subject: Encryption/Hashing In-Reply-To: References: Message-ID: Hi, Im new to erlang and was wondering which would be the most adequate function to hash or encrypt passwords for my test application. I need something that can be flexible in the sense, that it allows me to change to some other encryption function, if i choose to do so at some later stage, without requiring me to reset the passwords for existing users. -- Regards, Harris M. Bhatti From ulf@REDACTED Thu Jul 21 18:14:14 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 21 Jul 2005 18:14:14 +0200 Subject: CPU/Hardware optimized for Erlang In-Reply-To: References: Message-ID: Den 2005-07-21 16:42:09 skrev Marc van Woerkom : > Has there been any cpu/hardware design optimized > for Erlang? > > Would one aim for a beam virtual machine in hardware > or something more lowlevel, but still optimized > somehow? Well, there is the old Erlang Processor (ECOMP) (http://www.erlang.se/euc/00/processor.ppt) I think that perhaps if it had materialized during 2001, and e.g. the robotics industry, or people building small embedded devices, had taken an interest in it and Erlang, then perhaps it could have been successful. Now, after 4 years of no progress, I think multi-core CPUs and multi-pro Erlang shows much more promise for the high-end, even if perhaps not for very small embedded devices. OTOH, I don't see people breaking down doors to be able to use Erlang in these systems. They seem content with Java, or even C. That was otherwise a very intriguing niche for ECOMP: a small cheap chip (50-100,000 gates), perhaps 2-300 MHz, really low power, and support for a high-level language that does concurrency and fault-tolerance well enough not to need an operating system. Also, latency and GC would have been quite easy to get a handle on. /Uffe From bjarne@REDACTED Thu Jul 21 18:28:16 2005 From: bjarne@REDACTED (=?iso-8859-1?Q?Bjarne_D=E4cker?=) Date: Thu, 21 Jul 2005 18:28:16 +0200 Subject: Hotel accomodation for the Erlang User Conference 2005 Message-ID: <002501c58e11$335f50c0$fd0e69d4@segeltorp> Hello Hotell ?lvsj? offers Ericsson rebate rates for people attending the EUC for the two nights November 9-10 and November 10-11. Please see their home page http://www.hotellalvsjo.nu/eng/index.htm When you contact them you have to mention the code word "Erlang" ! There happens to be other events going on at the same time so you should book rather soon. The Stockholm international fair is located in ?lvsj? too. ?lvsj? is a rather boring suburban place but it is only five minutes walk to the Ericsson conference centre where the EUC takes place. There is also a commuter train station in ?lvsj? and the trains only take seven minutes to Stockholm central. By the way, trains or busses to or from Arlanda airport also stop at Stockholm central. Best wishes Bjarne PS. Program planning is well under way for this year's EUC and a provisional program will likely be published already at the end of September. Still more fun presentations are welcome, especially dealing with new unexpected applications. http://www.erlang.se/euc/05/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Jul 21 18:50:00 2005 From: erlang@REDACTED (Michael McDaniel) Date: Thu, 21 Jul 2005 09:50:00 -0700 Subject: Encryption/Hashing In-Reply-To: References: Message-ID: <20050721164959.GR3913@fangora.autosys.us> On Thu, Jul 21, 2005 at 05:03:10PM +0100, M. Harris Bhatti wrote: > Hi, > > Im new to erlang and was wondering which would be the most adequate > function to hash or encrypt passwords for my test application. I need > something that can be flexible in the sense, that it allows me to > change to some other encryption function, if i choose to do so at some > later stage, without requiring me to reset the passwords for existing users. > > -- > Regards, > Harris M. Bhatti ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ See documentation for crypto module which contains a variety such as md5, sha, aes, des. ~Michael From meena_selvam@REDACTED Thu Jul 21 20:15:41 2005 From: meena_selvam@REDACTED (MEENA SELVAM) Date: Thu, 21 Jul 2005 11:15:41 -0700 (PDT) Subject: yaws webserver specific question Message-ID: <20050721181541.76263.qmail@web30410.mail.mud.yahoo.com> The following record definitions are in yaws_api.hrl Does any one know what is the type of the individual elements of the other header? I have included a typical arg record with typical values of the other record. But I am not sure how I can access the IP address (47.80.18.95 )in the fourth element of the other record can i do something like this. will it work? [_,_,_,FourthElement|_] = (A#arg.headers)#headers.Other [_,_,_,_,IP] = FourthElement -record(arg, { clisock, %% the socket leading to the peer client headers, %% headers req, %% request ..... ..... ..... }). -record(headers, { connection, accept, host, %% host header if_modified_since, ... ... ... other = [] %% misc other headers }). A= {arg,#Port<16338.534>, > {headers,undefined, > "image/gif, >image/x-xbitmap, image/jpeg, image/ >pjpeg, */*", > "www.msn.com", > undefined, > undefined, > undefined, > undefined, > undefined, > undefined, > undefined, > "Mozilla/4.0 >(compatible; MSIE 6.0; Windows NT >5.1)", > undefined, > [], > undefined, > undefined, > undefined, > undefined, > undefined, > undefined, > undefined, > [{http_header, > 0, > >"X-Yaws-Had-Clicert", > undefined, > "false"}, > {http_header, > 49, > >'X-Forwarded-For', > undefined, > "11.10.1.107"}, > {http_header, > 7, > 'Via', > undefined, > "1.1 www.msn.com >(Alteon iSD-SSL/5.5)"}, > {http_header, > 0, > >"X-Forwarded-By", > undefined, > "47.80.18.95"}, > {http_header, > 10, > >'Accept-Encoding', > undefined, > "gzip, >deflate"}, > {http_header, > 11, > >'Accept-Language', > undefined, > "en-us"}]}, > >{http_request,'GET',{abs_path,"/"},{1,1}}, > undefined, > "/", > [], > undefined, .... > .... and much more > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From james.hague@REDACTED Thu Jul 21 20:55:57 2005 From: james.hague@REDACTED (James Hague) Date: Thu, 21 Jul 2005 13:55:57 -0500 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <20050721154413.98192.qmail@web34410.mail.mud.yahoo.com> References: <20050721154413.98192.qmail@web34410.mail.mud.yahoo.com> Message-ID: On 7/21/05, Thomas Lindgren wrote: > > I'm normally skeptical to special-purpose hardware for > specific programming languages -- such chips have > always gotten steamrollered by the killer micros. In the past I would have agreed, but that was before the killer micro manufacturers started running into hard limits. Power consumption has been increasing disproportionally with performance. And now notebook computers are outselling desktops, and that's where power consumption is a big deal. If an Erlang processor gave performance similar to BEAM on a 3GHz Pentium 4, but used 1/10 the power, then that's a big deal. (Though I know it still wouldn't get used in notebooks, but maybe in some kind of Palm or Blackberry-like device?) James From zoltan.podlovics@REDACTED Thu Jul 21 21:30:00 2005 From: zoltan.podlovics@REDACTED (Zoltan Podlovics) Date: Thu, 21 Jul 2005 21:30:00 +0200 Subject: CPU/Hardware optimized for Erlang In-Reply-To: References: Message-ID: <80902d980507211230576ba321@mail.gmail.com> Is there any way to use hardware"plugins" for high performance computation-intensive applications (encryption, xml parsing, pattern matching with fpga) with ECOMP? Erlang (with a small tcp/ip stack) would be perfect choice for these appliances (especially when it comes with fault-tolerant hw reconfiguration and sw update). For example: reliable email services or IDS with hw (virus or intrusion) pattern matcher. Regards, Zoltan On 7/21/05, Ulf Wiger wrote: > Den 2005-07-21 16:42:09 skrev Marc van Woerkom > Well, there is the old Erlang Processor (ECOMP) > (http://www.erlang.se/euc/00/processor.ppt) > > I think that perhaps if it had materialized during 2001, > and e.g. the robotics industry, or people building small > embedded devices, had taken an interest in it and Erlang, > then perhaps it could have been successful. > > Now, after 4 years of no progress, I think multi-core > CPUs and multi-pro Erlang shows much more promise for > the high-end, even if perhaps not for very small embedded > devices. OTOH, I don't see people breaking down doors to > be able to use Erlang in these systems. They seem content > with Java, or even C. > > That was otherwise a very intriguing niche for ECOMP: > a small cheap chip (50-100,000 gates), perhaps 2-300 MHz, > really low power, and support for a high-level language > that does concurrency and fault-tolerance well enough > not to need an operating system. Also, latency and GC > would have been quite easy to get a handle on. > > /Uffe From ulf@REDACTED Thu Jul 21 22:41:08 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 21 Jul 2005 22:41:08 +0200 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <80902d980507211230576ba321@mail.gmail.com> References: <80902d980507211230576ba321@mail.gmail.com> Message-ID: Den 2005-07-21 21:30:00 skrev Zoltan Podlovics : > Is there any way to use hardware"plugins" for high performance > computation-intensive applications (encryption, xml parsing, pattern > matching with fpga) with ECOMP? Erlang (with a small tcp/ip stack) > would be perfect choice for these appliances (especially when it comes > with fault-tolerant hw reconfiguration and sw update). For example: > reliable email services or IDS with hw (virus or intrusion) pattern > matcher. > > Regards, > Zoltan The idea with ECOMP was partly that you'd be able to control hardware directly, through the use of ports. Task switching in ECOMP took about 20 cpu cycles, and one idea was to use multiple process queues, so that you could task switch in one cpu cycle by alternating between the queues. All message passing was by reference, since it was essentially a shared heap architecture. Also, garbage collection was reference counting, and the actual freeing of data was done in HW, in the background, without stealing cpu cycles from the program. All in all, ECOMP should have been fairly good for stream based programming -- maybe not wire-speed in the realm of gigabits, or hard real-time as in dsp functions, but still... /Uffe From hakan.stenholm@REDACTED Thu Jul 21 22:45:36 2005 From: hakan.stenholm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Stenholm?=) Date: Thu, 21 Jul 2005 22:45:36 +0200 Subject: How can i access the other header of yaws arg structure In-Reply-To: <20050721175944.28755.qmail@web30411.mail.mud.yahoo.com> References: <20050721175944.28755.qmail@web30411.mail.mud.yahoo.com> Message-ID: <42E00970.9060601@mbox304.swipnet.se> MEENA SELVAM wrote: >In this the fourth element of other header itself >looks like a record.. that means i should look for the >record definition of other, or is it possible to >pattern match and get the actual IP? > >[_,_,FourthElement|_] = (A#arg.headers)#headers.Other > The above will not work Other should be other - as Other is a variable rather than a atom which is required for a record field name - records are internaly stored as tuples and all acess calls are translated to (erlang:)element/2 calls, the use of atoms is presumably required to be able map record fieldnames to tuple positions at compile time. See "Programming Examples" (http://erlang.se/doc/doc-5.4.8/doc/programming_examples/part_frame.html) in the web documentation for a more indeapth explanation about records as well as chapter 8 in the "Erlang Reference Manual" (http://erlang.se/doc/doc-5.4.8/doc/reference_manual/part_frame.html) web page. >[_,_,_,_,IP] = FourthElement > > >Fourth Element of other header actually contains the >IP address which I am interested to access. > >Will the above work? > I'm somewhat unsure what kind or records / data objects are stored in headers.other but you will probably want to do something like: %% select appropriate misc.header [_,_,Element|_] = (A#arg.headers)#headers.other, %% get the IP address from Element [_,_,_,_,IP] = Element, %% In your example you asume that the entry Element, from the misc. headers, %% is a 5 element list containing a IP address as it's last element. %% A personal guess would be that Element is a record with a 'ip' field where %% the ip is probably stored as a list of integers (e.g. [127,123,100,5]) so %% IP would be retrived as: IP = Element#somerecord.ip %% calls element(#somerecord.ip, Element) %% or #somerecord{ip = IP} = Element %% pattern match syntax that allows retrieval of %% several record fields %% this could be combined into [_,_,#somerecord{ip = IP}|_] = (A#arg.headers)#headers.other, %% if you prefer to nest the pattern matching code - I usually find this somewhat %% cumbersome to read especially if there are several '='. %% You could even use: #headers{other = [_,_,#somerecord{ip = IP}|_]} = A#arg.headers, %% or #arg{header = #headers{other = [_,_,#somerecord{ip = IP}|_]}} = A, > >A= {arg,#Port<16338.534>, > {headers,undefined, > "image/gif, image/x-xbitmap, >image/jpeg, image/pjpeg, */*", > "www.msn.com", > undefined, > undefined, > undefined, > undefined, > undefined, > undefined, > undefined, > "Mozilla/4.0 (compatible; MSIE 6.0; >Windows NT 5.1)", > undefined, > [], > undefined, > undefined, > undefined, > undefined, > undefined, > undefined, > undefined, > >[{http_header,0,"X-Yaws-Had-Clicert",undefined,"false"}, > >{http_header,49,'X-Forwarded-For',undefined,"11.10.1.107"}, > undefined, > "false"}, > {http_header, > 49, > >'X-Forwarded-For', > undefined, > "11.10.1.107"}, > {http_header, > 7, > 'Via', > undefined, > "1.1 www.apr.com >(Alteon iSD-SSL/5.5)"}, > {http_header, > 0, > >"X-Forwarded-By", > undefined, > "47.80.18.95"}, > {http_header, > 10, > >'Accept-Encoding', > undefined, > "gzip, >deflate"}, > {http_header, > > >meena > >--- H?kan Stenholm >wrote: > > >>MEENA SELVAM wrote: >> >> >>>Hi, >>> >>>I have a arg record A passed by the yaws webserver >>> >>to >> >>>the web application: >>> >>> >>>How can i access the 3rd element of the other >>> >>header, >> >>>whereas I have the other header itself is available >>> >>as >> >>>(A#arg.headers)#headers.Other >>> >>> >>> >>#headers.other appears to be a list, the third >>element of a list can be accessed in two ways: >> >>%% using pattern matching >>[_,_,ThirdElement|_] = List >> >>%% or >> >>%% using nth/2 in the lists module >>ThirdElement = lists:nth(3, List) >> >> >> >>>A= {arg,#Port<16338.534>, >>> {headers,undefined, >>> "image/gif, >>>image/x-xbitmap, image/jpeg, image/ >>>pjpeg, */*", >>> "www.msn.com", >>> undefined, >>> undefined, >>> undefined, >>> undefined, >>> undefined, >>> undefined, >>> undefined, >>> "Mozilla/4.0 >>>(compatible; MSIE 6.0; Windows NT >>>5.1)", >>> undefined, >>> [], >>> undefined, >>> undefined, >>> undefined, >>> undefined, >>> undefined, >>> undefined, >>> undefined, >>> [{http_header, >>> 0, >>> >>>"X-Yaws-Had-Clicert", >>> undefined, >>> "false"}, >>> {http_header, >>> 49, >>> >>>'X-Forwarded-For', >>> undefined, >>> >>> >>"11.10.1.107"}, >> >>> {http_header, >>> 7, >>> 'Via', >>> undefined, >>> "1.1 >>> >>www.msn.com >> >>>(Alteon iSD-SSL/5.5)"}, >>> {http_header, >>> 0, >>> >>>"X-Forwarded-By", >>> undefined, >>> >>> >>"47.80.18.95"}, >> >>> {http_header, >>> 10, >>> >>>'Accept-Encoding', >>> undefined, >>> "gzip, >>>deflate"}, >>> {http_header, >>> 11, >>> >>>'Accept-Language', >>> undefined, >>> "en-us"}]}, >>> >>>{http_request,'GET',{abs_path,"/"},{1,1}}, >>> undefined, >>> "/", >>> [], >>> undefined, >>> .... and much more >>> >>> >>>The records arg and header are defined as below: >>> >>>-record(arg, { >>> clisock, %% the socket leading to >>> >>the >> >>>peer client >>> headers, %% headers >>> req, %% request >>> >>> ..... some more >>> }). >>> >>>-record(headers, { >>> connection, >>> accept, >>> host, >>> if_modified_since, >>> if_match, >>> if_none_match, >>> if_range, >>> if_unmodified_since, >>> range, >>> referer, >>> user_agent, >>> accept_ranges, >>> cookie = [], >>> keep_alive, >>> location, >>> content_length, >>> content_type, >>> content_encoding, >>> authorization, >>> transfer_encoding, >>> other = [] %% misc other headers >>> }). >>> >>> >>> >>>__________________________________________________ >>>Do You Yahoo!? >>>Tired of spam? Yahoo! Mail has the best spam >>> >>protection around >> >>>http://mail.yahoo.com >>> >>> >>> >>> >> > > > > >____________________________________________________ >Start your day with Yahoo! - make it your home page >http://www.yahoo.com/r/hs > > > From hakan.stenholm@REDACTED Thu Jul 21 23:23:39 2005 From: hakan.stenholm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Stenholm?=) Date: Thu, 21 Jul 2005 23:23:39 +0200 Subject: what is wrong with this code setting a nested record? FOLLOW UP In-Reply-To: <20050721173132.89388.qmail@web30412.mail.mud.yahoo.com> References: <20050721173132.89388.qmail@web30412.mail.mud.yahoo.com> Message-ID: <42E0125B.6010200@mbox304.swipnet.se> MEENA SELVAM wrote: >I have a function display_login2(A) where A is >follows: >(The arg and header record definitions are given at >the end.) > > > >A= {arg,#Port<16338.534>, > >> {headers,undefined, >> "image/gif, >>image/x-xbitmap, image/jpeg, image/ >>pjpeg, */*", >> "www.msn.com", >> undefined, >> undefined, >> undefined, >> undefined, >> undefined, >> undefined, >> undefined, >> "Mozilla/4.0 >>(compatible; MSIE 6.0; Windows NT >>5.1)", >> undefined, >> [], >> undefined, >> undefined, >> undefined, >> undefined, >> undefined, >> undefined, >> undefined, >> [{http_header, >> 0, >> >>"X-Yaws-Had-Clicert", >> undefined, >> "false"}, >> {http_header, >> 49, >> >>'X-Forwarded-For', >> undefined, >> "11.10.1.107"}, >> {http_header, >> 7, >> 'Via', >> undefined, >> "1.1 >> >www.msn.com > >>(Alteon iSD-SSL/5.5)"}, >> {http_header, >> 0, >> >>"X-Forwarded-By", >> undefined, >> "47.80.18.95"}, >> {http_header, >> 10, >> >>'Accept-Encoding', >> undefined, >> "gzip, >>deflate"}, >> {http_header, >> 11, >> >>'Accept-Language', >> undefined, >> "en-us"}]}, >> >>{http_request,'GET',{abs_path,"/"},{1,1}}, >> undefined, >> "/", >> [], >> undefined, >> > .... > >> .... and much more >> >> >-record(arg, { > clisock, %% the socket leading to the >peer client > headers, %% headers > req, %% request > ..... > ..... > ..... > }). > >-record(headers, { > connection, > accept, > host, %% host header > if_modified_since, > ... > ... > ... > other = [] %% misc other headers > }). > >I need to replace the host header (which is >www.msn.com) by the redirected ip 47.80.18.95(which is >part of the Other Header) > >Is this the only way to do this: >Create a new record of type headers, with www.msn.com >replaced by the 47.80.18.95 > Yes > > NewHeader = > (A#arg.headers)#headers{host="47.80.18.95"}, > >[ typically i will make host as extracted from 3rd >element of other header] > >Create 9 other new records of the 9 different types >of the sub records of arg record. (Totally I have 10 >including headers) > >Newclisock = A#arg.clisock >Newreq = A#arg.req >... >.. > > >Now create a new record of type Arg , by using the >newly created 10 records like this: >ANew = #arg{clisock = NewClisock, > headers = NewHeader, > req = NewReq > ... > ...}, > There is no need to extract all the field values (sub records), it is sufficient to only set the fields that need to be updated e.g.: %% possibly get some sub records from A ... %% do some code that creates new values for some fields (sub records) NewReq = ... NewVal = ... %% update record (or rather creat a new one based on the old one) %% with only the affected fields changed ANew = A#arg{clisock = NewVal, req = NewReq}, ... > >and use Anew instead of A from now on. > Use ANew when the updated version is needed and A when the old version is needed - both are available in the current function clause scoop, from the point that they are declared until the end of the clause. No variable in a function clause can change its content after it has been given a value in that scoop - updating values in each iteration step is achived by passing updated values as arguments to the next recursive call, so variables in the current scoop never change. %% a simple list length calculation function, a accumulator is used to %% make the function tail recursive (and to show how data is passed between %% each iteration / recursive call) length(List) -> length(List, 0). length([], Length) -> Length; %% base case - end of list found, %% return the length value that has been accumulated length([_|R], Length) -> length(R,Lenght+1). %% pass new length and list remainder %% to next call of length/2 > >Will this work? Also, is the syntax right while I am >creating ANew? > > > >--- H?kan Stenholm >wrote: > > >>MEENA SELVAM wrote: >> >> >>>Hi, >>> >>>the parameter A is of record type arg, which >>> >>contains >> >>>a subrecord of type, headers and which has a field >>>host: >>> >>>The host field initially contains www.msn.com, and >>> >>I >> >>>am trying to overwrite with 47.80.18.95, but the >>>following code doesnt work.. it still has >>> >>www.msn.com >> >>>itself. >>> >>>display_login2(A, ReplyMsgs, URL, IP, FromLogout) >>> >>-> >> >>> (A#arg.headers)#headers{host="47.80.18.95"}, >>> >>>am i doing anything wrong here? >>> >>> >>> >>It should work (see my test code below), but may >>have forgotten to do: >> >> NewHeader = >>(A#arg.headers)#headers{host="47.80.18.95"}, >> ... >> xxx(NewHeader, ...) %% use the new >>(updated) Header rather >>than the old one in A >> ... >> >>It would be helpful to see a bit more of your code, >>I get the feeling >>that your trying to do a destructive update on a >>record in a single >>assignment language, this won't work - data elements >>in Erlang can only >>be created but never modified[1] (with the exception >>for >>ets/mnesia/process dictionaries which can >>conceptually be viewed as >>processes storing data in some regular Erlang data >>type like a binary >>tree - which is updated by creating new nodes and >>leaves to replace the >>old ones - rather than overwriting old ones). >> >>[1] : deletion is done by the GC. >> >>================== test.erl ===================== >> >>-module(test). >> >>-export([test/0]). >> >>-record(foo, { >> bar >> }). >> >>-record(bar, { >> a, >> b >> }). >> >>test() -> >> Bar = #bar{a = 1, b = 2}, >> Foo = #foo{bar = Bar}, >> >> io:format("Foo = ~p~n", [Foo]), >> >> %% create new bar record >> %% access foo record and create new bar record >>based on foo.bar >> Bar2 = (Foo#foo.bar)#bar{b = new_val}, >> io:format("Bar2 = ~p~n", [Bar2]), >> >> %% create new foo record >> %% create new foo record containing the new Bar2 >>bar record >> Foo2 = Foo#foo{bar = Bar2}, >> io:format("Foo2 = ~p~n", [Foo2]), >> >> ok. >> >>================== test.erl ===================== >> >>2> c(test). >>{ok,test} >>3> test:test(). >>Foo = {foo,{bar,1,2}} >>Bar2 = {bar,1,new_val} >>Foo2 = {foo,{bar,1,new_val}} >>ok >> >> >>>meena >>> >>> >>>__________________________________________________ >>>Do You Yahoo!? >>>Tired of spam? Yahoo! Mail has the best spam >>> >>protection around >> >>>http://mail.yahoo.com >>> >>> >>> >>> >> > > > > >__________________________________ >Do you Yahoo!? >Yahoo! Mail - Find what you need with new enhanced search. >http://info.mail.yahoo.com/mail_250 > > From olivier@REDACTED Fri Jul 22 11:10:57 2005 From: olivier@REDACTED (olivier) Date: Fri, 22 Jul 2005 11:10:57 +0200 Subject: erl interface question (ERL_ATOM macros) Message-ID: <42E0B821.2010009@dolphian.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, I have a C++/C node that does an rpc call in a module. The function returns the atoms true or false. This piece of code: if (ERL_ATOM_SIZE(result) == 4) // true ... do things ... Does not work unless i add something that uses ERL_ATOM_PTR, like: cout << "Atom :" << ERL_ATOM_PTR(result) << endl; or unsigned char *foo = ERL_ATOM_PTR(result); Before the "if" test. Is it a bug (i'm using R10B-5, debian) or did i miss something ? Thanks in advance, - -- Olivier -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFC4LghpqVXaJzJYNIRAuFLAJ4lh8lPCzNUPx6RvEPHyOCncDUyqQCfaG3J Nw+FeIGcZK2hxG37p5lTCSY= =tq// -----END PGP SIGNATURE----- From vances@REDACTED Fri Jul 22 11:36:56 2005 From: vances@REDACTED (Vance Shipley) Date: Fri, 22 Jul 2005 05:36:56 -0400 Subject: towards an Erlang IDE Message-ID: <20050722093656.GC623@feeble.motivity.ca> I cam across DiaGen recently which is the "Diagram Editor Generator". It is used to develop diagram editors. You specify the diagram language and it creates a graphical diagram editor. Sounds like just the ticket for that SDL editor I'd like to have. The idea of specifying a "diagram language" intrigues me. This sounds like a Service Creation Environment in telecom parlance. Maybe an Erlang IDE should look a lot like an SCE; more about communicating processes and less about editing files. In thinking about graphical design tools I again looked at graphviz and uDraw(Graph). Several years ago I used graphviz in a tool to graph gen_fsm modules by parsing the abstract syntax of the module (the "reverse engineering" school of design). The graphviz and uDraw tools also allow editing the graphs. I'm thinking of bringing it full circle and editing the erlang modules using a GUI. For example if I change the name of a node on the graph the corresponding gen_fsm state handler gets renamed. uDraw and graphviz tools provide you an extensible diagram editor. With it you could create a "Supervision Heirarchy Tree" diagram editor or a "Finite State Machine" diagram editor. You can annotate nodes and edges using definable menu items and list boxes. Both of these tools are based on Tcl/Tk. This probably opens the door for a tight integration with gs? I'd like an appmon like application for display and editing of graphs obtained by parsing the modules. Create new graphs using the GUI which document the design. Template erlang modules are created for new projects. The diagram attributes are embedded as comments. Opening an existing project reparses the modules into a graph and includes the attributes to adorn the graph. Edit and output a new source module. Use a similiar scheme for inspecting running processes ala appmon or Erlang Verification Tool. The EVT looks very interesting. It uses uDraw to display graphs it creates. This leads me to source code control. I still have the idea that storing Abstract Erlang syntax trees in CVS would be a perfect answer to the issue of indentation style. Maybe a complete IDE would encompass source code management so that work is done on abstract forms directly bypassing the parse stage? -Vance uDraw(Graph): http://www.informatik.uni-bremen.de/uDrawGraph/en/home.html graphviz: http://www.graphviz.org Erlang Verification Tool: http://www.sics.se/fdt/vericode/evt.html DiaGen: http://www2-data.informatik.unibw-muenchen.de/DiaGen/ Tripping Hippy: http://www.erlang.org/ml-archive/erlang-questions/200110/msg00010.html From thomasl_erlang@REDACTED Fri Jul 22 14:25:49 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Fri, 22 Jul 2005 05:25:49 -0700 (PDT) Subject: CPU/Hardware optimized for Erlang In-Reply-To: Message-ID: <20050722122549.32771.qmail@web34413.mail.mud.yahoo.com> --- James Hague wrote: > On 7/21/05, Thomas Lindgren > wrote: > > > > I'm normally skeptical to special-purpose hardware > for > > specific programming languages -- such chips have > > always gotten steamrollered by the killer micros. > > In the past I would have agreed, but that was before > the killer micro > manufacturers started running into hard limits. > Power consumption has > been increasing disproportionally with performance. Even so, you will probably have to build an ASIC at best, while Intel puts 400 people to hand-optimizing the circuitry on the next x86 ... (they will thus likely have a big clock speed advantage too). (And why would the erlang chip have comparable performance at lower power?) I think the main drawback of an "erlang" chip is that the tradeoffs are (not identical but) fairly similar to a "C" or "Fortran" chip (aka workstation and supercomputer). It's thus hard to see that it could compete with a conventional server. Or why not compare JAM, ECOMP and HIPE? ECOMP was about 30x faster than JAM, while HIPE at the time was about 10-20x faster than JAM. Would ECOMP have been capable of evolving as quickly as JAM/HIPE on modern hardware? If not, it would go the way of the Lisp Machine. That's not to say there is no role for a specialist chip (perhaps one could spin a successor to APZ? :-) but I think one needs to do some thinking on its role. This, in my mind, leads to more concrete application-oriented features, rather than language-oriented ones. Also, who knows, I might be unduly pessimistic. I really wouldn't mind being proven wrong :-) Best, Thomas __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From s.blochberger@REDACTED Fri Jul 22 14:54:39 2005 From: s.blochberger@REDACTED (s.blochberger@REDACTED) Date: Fri, 22 Jul 2005 13:54:39 +0100 Subject: Erlang R10B6 not working on Windows2003 Server Message-ID: <0MKwh2-1Dvx322AXT-0003i9@mrelayeu.kundenserver.de> Two days ago I installed the latest release of Erlang - uninstalling R5 beforehand. It happens that Erlang doesn't want to work. A fw seconds after starting the console, the following error appears (one character is cut off because it wouldn't let me copy the whole width). {error_logger,{{2005,7,21},{9,54,16}},supervisor_report,[{supervisor,{local,ker el_sup}},{errorContext,start_error},{reason,{'DOWN',#Ref<0.0.0.10>,process,<0.1 .0>,normal}},{offender,[{pid,undefined},{name,code_server},{mfa,{code,start_lin ,[]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]} {error_logger,{{2005,7,21},{9,54,16}},crash_report,[[{pid,<0.7.0>},{registered_ ame,[]},{error_info,{shutdown,{kernel,start,[normal,[]]}}},{initial_call,{appli ation_master,init,[<0.5.0>,<0.6.0>,{appl_data,kernel,[application_controller,er _reply,auth,boot_server,code_server,disk_log_server,disk_log_sup,erl_prim_loade ,error_logger,file_server,file_server_2,fixtable_server,global_group,global_nam _server,heart,init,kernel_config,kernel_sup,net_kernel,net_sup,rex,user,os_serv r,ddll_server,erl_epmd,inet_db,pg2],undefined,{kernel,[]},[application,applicat on_controller,application_master,application_starter,auth,code,code_aux,package ,code_server,dist_util,erl_boot_server,erl_distribution,erl_prim_loader,erl_rep y,erlang,error_handler,error_logger,file,file_server,old_file_server,file_io_se ver,prim_file,global,global_group,global_search,group,heart,inet6_tcp,inet6_tcp dist,inet6_udp,inet_config,inet_hosts,inet_gethost_native,inet_tcp_dist,init,ke nel,kernel_config,net,net_adm,net_kernel,os,ram_file,rpc,user,user_drv,user_sup disk_log,disk_log_1,disk_log_server,disk_log_sup,dist_ac,erl_ddll,erl_epmd,erts debug,gen_tcp,gen_udp,prim_inet,inet,inet_db,inet_dns,inet_parse,inet_res,inet_ cp,inet_udp,pg2,seq_trace,wrap_log_reader,zlib,otp_ring0],[],infinity,infinity} normal]}},{ancestors,[<0.6.0>]},{messages,[{'EXIT',<0.8.0>,normal}]},{links,[<0 6.0>,<0.5.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,987 ,{stack_size,21},{reductions,1062}],[]]} {error_logger,{{2005,7,21},{9,54,16}},std_info,[{application,kernel},{exited,{s utdown,{kernel,start,[normal,[]]}}},{type,permanent}]} {"Kernel pid terminated",application_controller,shutdown} Abnormal termination I would really appreciate any assistance. -- Dipl.-Inf.(FH) Sabine Dinis Blochberger s.blochberger@REDACTED O p 3 r a c i o n a l http://www.op3racional.de P l u t ? o http://extranet.op3racional.info S a f e N e t http://safenet.op3racional.de L a n S c h o o l http://lanschool.op3racional.de P o n t o C e n t r a l http://www.ponto-central.op3racional.de From sebastian@REDACTED Fri Jul 22 15:29:36 2005 From: sebastian@REDACTED (Sebastian Bello) Date: Fri, 22 Jul 2005 10:29:36 -0300 Subject: mnesia: inserting a large number of records Message-ID: <003701c58ec1$68b8eff0$dc00a8c0@INSWITCH244> Hi all, a programm reads records from a text file and inserts them in a mnesia table. We are performing this insertions within a transaction so in case of an error the whole file can be reprocessed. The file holds approx. 5.000-10.000 records. It seems the transaction time is not linear; I'm wondering if there is a faster way to perform the insertions, maybe using a table lock, I don't know. Any suggestions? Thanks, Sebastian- Prepaid Expertise - Programmable Switches Powered by Ericsson Licensed Technology Sebasti?n Bello - Engineer - Development Center - IN Switch Solutions Inc. Headquarters - Miami-U.S.A. Tel: 1305-3578076 Fax: 1305-7686260 Development Center - Montevideo - Uruguay Tel/Fax: 5982-7104457 IN SWITCH EMEA Phone: +33 0 6 0335 9427 - Fax: +33 0 4 93655773 / emea@REDACTED IN SWITCH ASIA Phone: +92 51 2800397/8- Fax: +92 51 2800399/ inswasia@REDACTED e-mail: sebastian@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1429 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: logoparafondo.gif Type: image/gif Size: 16886 bytes Desc: not available URL: From vlad_dumitrescu@REDACTED Fri Jul 22 15:54:48 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Fri, 22 Jul 2005 15:54:48 +0200 Subject: Erlang R10B6 not working on Windows2003 Server References: <0MKwh2-1Dvx322AXT-0003i9@mrelayeu.kundenserver.de> Message-ID: ----- Original Message ----- From: > It happens that Erlang doesn't want to work. A fw seconds after starting > the console, the following error appears (one character is cut off because > it wouldn't let me copy the whole width). Hi, Does this happen every time? I also had the same error on WinXP, the first time I started Erlang. Afterwards, it worked just fine. regards, Vlad From joelr1@REDACTED Fri Jul 22 17:47:22 2005 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 22 Jul 2005 17:47:22 +0200 Subject: Erlang R10B6 not working on Windows2003 Server In-Reply-To: <0MKwh2-1Dvx322AXT-0003i9@mrelayeu.kundenserver.de> References: <0MKwh2-1Dvx322AXT-0003i9@mrelayeu.kundenserver.de> Message-ID: <6A36A753-F4C6-493C-A1DF-6CC1D6775AB2@gmail.com> I have this very error on Mac OSX, periodically. Restarting the Erlang VM once might bring the same error but restarting it more than twice usually helps. Go figure! Joel On Jul 22, 2005, at 2:54 PM, s.blochberger@REDACTED wrote: > Two days ago I installed the latest release of Erlang - > uninstalling R5 beforehand. > > It happens that Erlang doesn't want to work. A fw seconds after > starting the console, the following error appears (one character is > cut off because it wouldn't let me copy the whole width). > > {error_logger,{{2005,7,21},{9,54,16}},supervisor_report, > [{supervisor,{local,ker > el_sup}},{errorContext,start_error},{reason, > {'DOWN',#Ref<0.0.0.10>,process,<0.1 > .0>,normal}},{offender,[{pid,undefined},{name,code_server},{mfa, > {code,start_lin -- http://wagerlabs.com/uptick From sebastian@REDACTED Fri Jul 22 19:04:03 2005 From: sebastian@REDACTED (Sebastian Bello) Date: Fri, 22 Jul 2005 14:04:03 -0300 Subject: mnesia query References: <033501c5887a$ea9ecaf0$3000a8c0@INSWITCH244> <035701c5888d$ed3cacb0$3000a8c0@INSWITCH244> <036501c58895$ac331260$3000a8c0@INSWITCH244> <002901c58ad3$2d2fff60$3000a8c0@INSWITCH244> Message-ID: <022801c58edf$5da6f080$dc00a8c0@INSWITCH244> I've done some test with indexes and am a bit confused. I've ran the same query using select, index_read and QLC and performance seems to follow that same order(faster to slower). I noted that performance depedends on the number of records associated to the value of the index searched. There's surely something I'm missing here. My questions would be: 1. shouldn't performance be better for index_read and QLC than select (since an index is beeing used)? 2. when using QLC, should indexes be used explicitly (or just write a normal list comprehension query)? 3. depending on 2, how can I use two indexes with QLC? 4. just to be sure, should a search through two indexes be always faster than using just one? I saw some old questions regarding this point, but didn't get a clear idea about it. Thanks, Sebastian- ----- Original Message ----- From: "Ulf Wiger" To: "Sebastian Bello" ; Sent: Sunday, July 17, 2005 11:19 AM Subject: Re: mnesia query > Den 2005-07-17 15:26:43 skrev Sebastian Bello : > > > But is it possible to use mnesia:index_match_object() with two indexes? > > Sebastian- > > No. You specify exactly _one_ index that index_match_object() is to use. > > /Uffe > > From Jouni.Ryno@REDACTED Fri Jul 22 19:23:29 2005 From: Jouni.Ryno@REDACTED (joneksi) Date: Fri, 22 Jul 2005 19:23:29 +0200 Subject: ERL_DRV_FLOAT ? References: Message-ID: <20050722172329.9E1A446DD6@bang.trapexit.org> Which version of R10B? For some reason I started to make a cairo driver. And it does define the points as floats :( Jouni Ryn? bjorn at erix.ericsson.se wrote: Probably an oversight, or that the implementor thought that no one would ever want to pass back floating point numbers from a driver. We'll probably add ERL_DRV_FLOAT in the R10B release. /Bjorn Mickael Remond writes: > Hello, > > I have working with edtk and erl_driver recently. I have seen that there > is no ERL_DRV_FLOAT, to do float type transfer from C to Erlang, back > and forth. > > I was wondering if there were any particular reason for that ? > > -- > Micka?l R?mond > http://www.erlang-projects.org/ > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB Post generated using Mail2Forum (http://m2f.sourceforge.net) (end of quote) _________________________________________________________ Sent using Mail2Forum (http://m2f.sourceforge.net) From will@REDACTED Fri Jul 22 19:23:27 2005 From: will@REDACTED (Will Newton) Date: Fri, 22 Jul 2005 18:23:27 +0100 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <20050722122549.32771.qmail@web34413.mail.mud.yahoo.com> References: <20050722122549.32771.qmail@web34413.mail.mud.yahoo.com> Message-ID: <200507221823.27538.will@misconception.org.uk> On Friday 22 July 2005 13:25, Thomas Lindgren wrote: > Even so, you will probably have to build an ASIC at > best, while Intel puts 400 people to hand-optimizing > the circuitry on the next x86 ... (they will thus > likely have a big clock speed advantage too). > > (And why would the erlang chip have comparable > performance at lower power?) I'm currently working with a chip that has several hardware "threads". On a single piece of code it's performance is not great, but when you have 2 or more threads it's performance per watt can be quite impressive. Also bear in mind that Intel chips are constrained very much by their legacy. If you have a specific load (e.g. floating point, integer, I/O, DSP, highly concurrent) it is not that difficult to build a chip that is faster than a more general purpose processor. At present it doesn't run Erlang however. :-) From vlad_dumitrescu@REDACTED Fri Jul 22 21:51:31 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Fri, 22 Jul 2005 21:51:31 +0200 Subject: ex11, plotting and Cairo ... References: <1121853819.16160.27.camel@hymir.newport-networks.com> <1121867609.16160.48.camel@hymir.newport-networks.com> <1121934864.16160.76.camel@hymir.newport-networks.com> Message-ID: ----- Original Message ----- From: "Peter-Henry Mander" > Xming troubles can be solved with RTFM I think. Removing access control > (-ac) allows me to send an xterm display to Xming so it does work in > priciple, albeit forgoing a secure connection I think. I'd prefer SSH > tunneling. I only use X (as Cygwin/X, but it's the same thing as Xming) locally, so I didn't hit any access control problems, but at least using Cygwin makes all settings to be handled just like under a regular Unix, with xhost and xauth and all that. This includes using ssh with X forwarding. /Vlad From ulf@REDACTED Sun Jul 24 10:36:21 2005 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 24 Jul 2005 10:36:21 +0200 Subject: TCP stack throughput In-Reply-To: <20050719173912.GA8577@dc.fi.udc.es> References: <20050706180318.GA18071@dc.fi.udc.es> <20050719173912.GA8577@dc.fi.udc.es> Message-ID: Den 2005-07-19 19:39:12 skrev Javier Par?s Fern?ndez : > I observed several funny things. The time spent processing the > connections seems to go up exponentially, but it is not limited > by cpu, which was always under 50%. I don't know what is throttling it. > > When I trid with 5000 connections, the logs started to show > retransmissions, > so I think it is probably related to some buffering limit with > the socket. This obviously skyrocketed the total processing time. Have you tried playing with setsockopt()? This seems to be the way to set socket send and receive buffer sizes on most platforms. Parameters are SO_SNDBUF and SO_RCVBUF http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/setsockopt_2.asp http://docs.hp.com/en/36960-90058/ch04s05.html http://www.ussg.iu.edu/hypermail/linux/kernel/0102.0/0038.html On Linux, the buffer sizes "can be set globally with the net.ipv4.tcp_wmem and net.ipv4.tcp_rmem sysctl variables" (http://unixhelp.ed.ac.uk/CGI/man-cgi?tcp+7) The defaults seem to vary. On HP AIX, the default is 4096. On Windows CE, it's 32768 (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/setsockopt_2.asp) /Uffe > > The total processing time for the tests was: > 500 in 0.6 seconds > 1000 in 1.7 seconds > 2000 in 6.2 seconds > 5000 was still closing connections after 3 minutes > > There are tcpdump logs in > http://www.lfcia.org/~paris/dump500.gz > http://www.lfcia.org/~paris/dump1000.gz > http://www.lfcia.org/~paris/dump2000.gz > http://www.lfcia.org/~paris/dump5000.gz > > I think it would be interesting to find what is the limiting the stack > here, and why there are dropped packets, as I think it should be > possible to get better performance. However, I had not made a test > similar to this, so connection opening and closing is one of the things > that are not optimized in the stack. > > Regards, > Javier. > From vlad_dumitrescu@REDACTED Sun Jul 24 10:33:38 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Sun, 24 Jul 2005 10:33:38 +0200 Subject: towards an Erlang IDE References: <20050722093656.GC623@feeble.motivity.ca> Message-ID: ----- Original Message ----- From: "Vance Shipley" > I'm thinking of bringing it > full circle and editing the erlang modules using a GUI. >... > I'd like an appmon like application for display and > editing of graphs obtained by parsing the modules. > ... > Use a similiar scheme for inspecting running processes > ala appmon or Erlang Verification Tool. The EVT looks > very interesting. It uses uDraw to display graphs it creates. Hi, Eclipse has a nice graphical editor framework plugin, GEF, that could be used for something like that. Once the Erlide environment is in place, I was thinking about trying something similar (but I'd start with something simpler, like the application monitor). Regarding storing parse trees in the CVS, I think the disadvantage is just as big as the problem it solves: you won't be able to easily compare/merge different versions, unless using a special tool that will transform to text before comparing. regards, Vlad From s.blochberger@REDACTED Mon Jul 25 10:50:18 2005 From: s.blochberger@REDACTED (s.blochberger@REDACTED) Date: Mon, 25 Jul 2005 09:50:18 +0100 Subject: Erlang R10B6 not working on Windows2003 Server Message-ID: <0MKxQS-1DwyfD0dcI-0004Ja@mrelayeu.kundenserver.de> ----- Original Message ----- From: > It happens that Erlang doesn't want to work. A fw seconds after starting > the console, the following error appears (one character is cut off because > it wouldn't let me copy the whole width). Hi, Does this happen every time? I also had the same error on WinXP, the first time I started Erlang. Afterwards, it worked just fine. regards, Vlad Unfortunately, it does. I opened the console, waited for the abnormal termination, closed and opened it again several times. Every time it has the same error... I installed with the administrator account. -- Dipl.-Inf.(FH) Sabine Dinis Blochberger s.blochberger@REDACTED O p 3 r a c i o n a l http://www.op3racional.de P l u t ? o http://extranet.op3racional.info S a f e N e t http://safenet.op3racional.de L a n S c h o o l http://lanschool.op3racional.de P o n t o C e n t r a l http://www.ponto-central.op3racional.de From thomasl_erlang@REDACTED Mon Jul 25 11:20:24 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 25 Jul 2005 02:20:24 -0700 (PDT) Subject: CPU/Hardware optimized for Erlang In-Reply-To: <200507221823.27538.will@misconception.org.uk> Message-ID: <20050725092024.47205.qmail@web34412.mail.mud.yahoo.com> --- Will Newton wrote: > Also bear in mind that Intel chips are constrained > very much by their legacy. I don't think this is a big handicap. Yes, it leads to extra complexity, but high-performance hardware already is very complex (as I'm sure you know). The instruction set or legacy features of IA32 really doesn't seem to be the bottleneck when you look at the results. > If you have a specific load (e.g. floating point, > integer, I/O, DSP, highly > concurrent) it is not that difficult to build a chip > that is faster than a > more general purpose processor. I agree -- my recommendation thus remains: optimize the hardware for the application, not the language :-) In the case of Erlang-in-telecoms, perhaps the APZ processor could be a fine source of inspiration (it's certainly a very successful product line). Best, Thomas ____________________________________________________ Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs From mickael.remond@REDACTED Mon Jul 25 15:15:44 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Mon, 25 Jul 2005 15:15:44 +0200 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <20050725092024.47205.qmail@web34412.mail.mud.yahoo.com> References: <20050725092024.47205.qmail@web34412.mail.mud.yahoo.com> Message-ID: <42E4E600.5050808@erlang-fr.org> Thomas Lindgren wrote: > > --- Will Newton wrote: > >>Also bear in mind that Intel chips are constrained >>very much by their legacy. > > I don't think this is a big handicap. Yes, it leads to > extra complexity, but high-performance hardware > already is very complex (as I'm sure you know). The > instruction set or legacy features of IA32 really > doesn't seem to be the bottleneck when you look at the > results. On this topic, I have read a rumour that Intel would like to introduce Java VM optimisation to accelerate execution in future Pentium version (Source is here: First comment: http://lwn.net/Articles/144546/). Unfortunatly, I have not been able to found any related article on the net. -- Micka?l R?mond From sebastian@REDACTED Mon Jul 25 15:34:17 2005 From: sebastian@REDACTED (Sebastian Bello) Date: Mon, 25 Jul 2005 10:34:17 -0300 Subject: mnesia growth limits Message-ID: <02ff01c5911d$8f56d9c0$dc00a8c0@INSWITCH244> Hi all, some questions regarding a mnesia database growth: 1- is there a limit in the number of records a database running on a single node can hold? How is this limit related to the individual tables' limits? 2- in a disk_copies database, how does the amount of memory affect mnesia's performance? Thanks, Sebastian- Prepaid Expertise - Programmable Switches Powered by Ericsson Licensed Technology Sebasti?n Bello - Engineer - Development Center - IN Switch Solutions Inc. Headquarters - Miami-U.S.A. Tel: 1305-3578076 Fax: 1305-7686260 Development Center - Montevideo - Uruguay Tel/Fax: 5982-7104457 IN SWITCH EMEA Phone: +33 0 6 0335 9427 - Fax: +33 0 4 93655773 / emea@REDACTED IN SWITCH ASIA Phone: +92 51 2800397/8- Fax: +92 51 2800399/ inswasia@REDACTED e-mail: sebastian@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1429 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: logoparafondo.gif Type: image/gif Size: 16886 bytes Desc: not available URL: From bengt.kleberg@REDACTED Mon Jul 25 16:19:37 2005 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 25 Jul 2005 16:19:37 +0200 Subject: Parsing formatted text In-Reply-To: <98FF4978-46DA-4975-A76A-2C9A4BF84E04@gmail.com> References: <98FF4978-46DA-4975-A76A-2C9A4BF84E04@gmail.com> Message-ID: <42E4F4F9.8090000@ericsson.com> On 2005-07-12 11:30, Joel Reymont wrote: ...deleted > > I wish there was a version of io:format that read data from a "string". i have written such a thing. it uses io:fread() with a custom io process that uses a string instead of a file. it works, but uses the nice ''New Erlang Standard Libraries'' by richardc@REDACTED i have included my code below if you are interested in a starting point for a better way of doing it. bengt -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: string_io.erl URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: string_io_test.erl URL: From will@REDACTED Mon Jul 25 19:46:46 2005 From: will@REDACTED (Will Newton) Date: Mon, 25 Jul 2005 18:46:46 +0100 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <20050725092024.47205.qmail@web34412.mail.mud.yahoo.com> References: <20050725092024.47205.qmail@web34412.mail.mud.yahoo.com> Message-ID: <200507251846.46500.will@misconception.org.uk> On Monday 25 July 2005 10:20, Thomas Lindgren wrote: > I don't think this is a big handicap. Yes, it leads to > extra complexity, but high-performance hardware > already is very complex (as I'm sure you know). The > instruction set or legacy features of IA32 really > doesn't seem to be the bottleneck when you look at the > results. I don't have any hard numbers of course (this is the Internet after all), but I would imagine that the number of transistors required to implement the wide array of instructions in such a chip limits the possibilities of putting more cores on a die because eventually your power dissipation will become a problem. > > If you have a specific load (e.g. floating point, > > integer, I/O, DSP, highly > > concurrent) it is not that difficult to build a chip > > that is faster than a > > more general purpose processor. > > I agree -- my recommendation thus remains: optimize > the hardware for the application, not the language :-) The processor I am working on tries to take that approach: http://www.metagence.com/Products/index.asp > In the case of Erlang-in-telecoms, perhaps the APZ > processor could be a fine source of inspiration (it's > certainly a very successful product line). Are you aware of any overviews of this processor architecture? From ulf@REDACTED Mon Jul 25 23:20:34 2005 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 25 Jul 2005 23:20:34 +0200 Subject: mnesia growth limits In-Reply-To: <02ff01c5911d$8f56d9c0$dc00a8c0@INSWITCH244> References: <02ff01c5911d$8f56d9c0$dc00a8c0@INSWITCH244> Message-ID: Den 2005-07-25 15:34:17 skrev Sebastian Bello : > Hi all, > > some questions regarding a mnesia database growth: > 1- is there a limit in the number of records a database running on a > single node can hold? How is this limit related to the individual > tables' limits? There isn't really a hard limit, unless all data resides in RAM. Then, mnesia is limited by the 4 GB address space in the current Erlang VM. Of course, this is not a limit on the number of records, but rather the total memory occupied by the table. I've personally had 10-20 million small objects in one node without problems, and others may have managed more. There is no per-table limit. There is a configurable limit on the number of ETS tables allowed in the system. This is usually not a big problem for mnesia, as the default is ca 1400 tables. > 2- in a disk_copies database, how does the amount of memory affect > mnesia's performance? A disc_copies table keeps all objects both in RAM (ets tables) and on disk. Thus, for good performance, all objects should fit comfortably within physical memory. However, with swapping enabled, things can work out reasonably well even if you exceed the physical memory, if a rather large portion of the data is only accessed very seldomly. Whether this holds for your applicaation is something you should explore through experimentation. /Uffe From david.nospam.hopwood@REDACTED Mon Jul 25 23:32:52 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Mon, 25 Jul 2005 22:32:52 +0100 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <200507251846.46500.will@misconception.org.uk> References: <20050725092024.47205.qmail@web34412.mail.mud.yahoo.com> <200507251846.46500.will@misconception.org.uk> Message-ID: <42E55A84.8090104@blueyonder.co.uk> Will Newton wrote: > On Monday 25 July 2005 10:20, Thomas Lindgren wrote: > >>I don't think this is a big handicap. Yes, it leads to >>extra complexity, but high-performance hardware >>already is very complex (as I'm sure you know). The >>instruction set or legacy features of IA32 really >>doesn't seem to be the bottleneck when you look at the >>results. > > I don't have any hard numbers of course (this is the Internet after all), but > I would imagine that the number of transistors required to implement the wide > array of instructions in such a chip limits the possibilities of putting more > cores on a die because eventually your power dissipation will become a > problem. It occurs to me that it might be useful to have only one core that is x86-compatible, and the rest using an architecture that can be optimized for die area and power dissipation without being constrained by backward compatibility. (In fact, the Cell processor is not too dissimilar to this, with PPC instead of x86.) -- David Hopwood From sebastian@REDACTED Tue Jul 26 00:06:54 2005 From: sebastian@REDACTED (Sebastian Bello) Date: Mon, 25 Jul 2005 19:06:54 -0300 Subject: mnesia growth limits References: <02ff01c5911d$8f56d9c0$dc00a8c0@INSWITCH244> Message-ID: <035101c59165$2c54b340$dc00a8c0@INSWITCH244> Thank you very much! Sebastian- ----- Original Message ----- From: "Ulf Wiger" To: Sent: Monday, July 25, 2005 6:20 PM Subject: Re: mnesia growth limits > Den 2005-07-25 15:34:17 skrev Sebastian Bello : > > > Hi all, > > > > some questions regarding a mnesia database growth: > > 1- is there a limit in the number of records a database running on a > > single node can hold? How is this limit related to the individual > > tables' limits? > > There isn't really a hard limit, unless all data resides in > RAM. Then, mnesia is limited by the 4 GB address space in the > current Erlang VM. Of course, this is not a limit on the > number of records, but rather the total memory occupied by > the table. I've personally had 10-20 million small objects > in one node without problems, and others may have managed > more. > > There is no per-table limit. There is a configurable limit > on the number of ETS tables allowed in the system. This is > usually not a big problem for mnesia, as the default is > ca 1400 tables. > > > > 2- in a disk_copies database, how does the amount of memory affect > > mnesia's performance? > > A disc_copies table keeps all objects both in RAM (ets tables) > and on disk. Thus, for good performance, all objects should > fit comfortably within physical memory. > > However, with swapping enabled, things can work out reasonably > well even if you exceed the physical memory, if a rather large > portion of the data is only accessed very seldomly. Whether > this holds for your applicaation is something you should > explore through experimentation. > > /Uffe > > From ok@REDACTED Tue Jul 26 00:56:06 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 26 Jul 2005 10:56:06 +1200 (NZST) Subject: CPU/Hardware optimized for Erlang Message-ID: <200507252256.j6PMu6QR261259@atlas.otago.ac.nz> Will Newton wrote: > Also bear in mind that Intel chips are constrained > very much by their legacy. But Thomas Lindgren doesn't think it's a handicap. It is quite enough of a handicap for Transmeta to carve themselves a niche. The Pentiums do run-time compilation of IA32 instructions to a sort of RISC with the compilation done in hardware a little at a time; the Transmeta machines did run-time compilation in software in great big gulps and were able to achieve near-Pentium speed at a MUCH lower cost in electrical power. Complexity certainly has an effect on barriers to market entry. Intel machines are impressive, but basically, they are brick outhouses that fly only because they have extremely large jet engines strapped on. Intel chips spend a lot of time, area, and electricity supporting things that Erlang doesn't particularly need, while they DON'T support a bunch of things that Erlang DOES need. The ECOMP slides point repeatedly to (electrical) power as a concern. In the case of Erlang-in-telecoms, perhaps the APZ processor could be a fine source of inspiration (it's certainly a very successful product line). Reference? From ok@REDACTED Tue Jul 26 00:57:41 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 26 Jul 2005 10:57:41 +1200 (NZST) Subject: CPU/Hardware optimized for Erlang Message-ID: <200507252257.j6PMvf8Y220893@atlas.otago.ac.nz> Mickael Remond wrote: On this topic, I have read a rumour that Intel would like to introduce Java VM optimisation to accelerate execution in future Pentium version Hmm. Intel already *have* a machine that does that, the ARM. There are versions of the ARM with direct-in-hardware JVM support. From yvan.godin@REDACTED Tue Jul 26 08:48:41 2005 From: yvan.godin@REDACTED (yvan sur free) Date: Tue, 26 Jul 2005 08:48:41 +0200 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <200507252257.j6PMvf8Y220893@atlas.otago.ac.nz> References: <200507252257.j6PMvf8Y220893@atlas.otago.ac.nz> Message-ID: <200507260848.41298.yvan.godin@free.fr> Le mardi 26 Juillet 2005 00:57, Richard A. O'Keefe a ?crit?: > Mickael Remond wrote: > On this topic, I have read a rumour that Intel would like to > introduce Java VM optimisation to accelerate execution in future > Pentium version > > Hmm. Intel already *have* a machine that does that, the ARM. > There are versions of the ARM with direct-in-hardware JVM support. Hum are you sure ? I usually read about embed board for robotic and never see direct hardware JVM support. Most of the time there is an JVM code on external flash memory on an ARM based board. Do you have any link please ? Thank Yvan http://yvan.godin.free.fr NB: for CPU/Hardware optimized for Erlang why not. Probably it's time to get a breaking technologie. I am far to be skilled enougth but I had a dream on an hyper cube processor ;-) http://yvan.godin.free.fr/wordpress/index.php?p=11 From Marc.Vanwoerkom@REDACTED Tue Jul 26 08:42:25 2005 From: Marc.Vanwoerkom@REDACTED (Marc van Woerkom) Date: Tue, 26 Jul 2005 08:42:25 +0200 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <200507252256.j6PMu6QR261259@atlas.otago.ac.nz> Message-ID: >It is quite enough of a handicap for Transmeta to carve >themselves >a niche. The Pentiums do run-time compilation of IA32 >instructions >to a sort of RISC with the compilation done in hardware a >little at >a time; the Transmeta machines did run-time compilation >in software >in great big gulps and were able to achieve near-Pentium >speed at a >MUCH lower cost in electrical power. I contacted a german distributor for Transmeta dev kits, but he said that Transmeta stopped development and support was no longer available. :-( It was exactly that Code Morphing System (CMS) which, if changeable by developers, would have made the design of custom instruction possible. Anyone knows what is left of Transmeta? Are they still in Business? Regards, Marc From matthias@REDACTED Tue Jul 26 09:07:45 2005 From: matthias@REDACTED (Matthias Lang) Date: Tue, 26 Jul 2005 09:07:45 +0200 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <200507252256.j6PMu6QR261259@atlas.otago.ac.nz> References: <200507252256.j6PMu6QR261259@atlas.otago.ac.nz> Message-ID: <17125.57665.805511.254402@antilipe.corelatus.se> Richard A. O'Keefe writes: > It is quite enough of a handicap for Transmeta to carve themselves > a niche. On the stock market, their niche closely resembles a hole: http://investor.transmeta.com/stock.cfm?tab=2&Period=60&Bench1= Matthias From ulrika@REDACTED Tue Jul 26 09:14:58 2005 From: ulrika@REDACTED (ulrika@REDACTED) Date: Tue, 26 Jul 2005 09:14:58 +0200 (CEST) Subject: How to call a gen_server from another node? Message-ID: <4102.193.15.249.14.1122362098.squirrel@webmail1.b-one.nu> I have a gen_server question that I hope somebody could help me with. I'm developing a remote debugging feature to use in a test environment where we're running the same application on several computers. The idea is to be able to debug these machines locally by using a macro, which sends a message to the user/s who listens to that specific machine, via a server program on a specific "debug node". So there's the application running on several nodes, the debug node and a node on the local host. I have a fully functional program now, but I'm using a simple server - i.e. I have written a receive loop, which captures the messages sent via the macro and then distributes them to the correct local hosts. Is it possible to use a gen_server instead? How do I send a message from a macro on one node to a gen_server process on another/call a function in some other program on another node? I'm sending messages to the server like this (using the macro): {ProcessName, Node} ! Msg Thanks for help, Ulrika From vlad.xx.dumitrescu@REDACTED Tue Jul 26 09:21:38 2005 From: vlad.xx.dumitrescu@REDACTED (Vlad Dumitrescu XX (LN/EAB)) Date: Tue, 26 Jul 2005 09:21:38 +0200 Subject: How to call a gen_server from another node? Message-ID: <11498CB7D3FCB54897058DE63BE3897C4BE591@esealmw105.eemea.ericsson.se> -----Original Message----- From: [mailto:owner-erlang-questions@REDACTED]On Behalf Of ulrika@REDACTED > Is it possible to use a gen_server instead? How do I send a message from a > macro on one node to a gen_server process on another/call a function in > some other program on another node? Hi, Of course you can use a gen_server. The macro would then have to become gen_server:cast({ProcessName, Node}, Msg) and in the gen_server it's business as usual, handle it in handle_cast. It would also work with the macro you already have, and then the handling would have to be done in handle_info. regards, Vlad From mickael.remond@REDACTED Tue Jul 26 10:14:08 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Tue, 26 Jul 2005 10:14:08 +0200 Subject: CPU/Hardware optimized for Erlang In-Reply-To: References: Message-ID: <42E5F0D0.4030204@erlang-fr.org> Marc van Woerkom wrote: > Anyone knows what is left of Transmeta? > Are they still in Business? The company still exist, but only licencing its "Intellectual Property": http://www.handtops.com/show/news/78 Last time I heard of them, they were selling their factory. -- Micka?l R?mond From thomasl_erlang@REDACTED Tue Jul 26 11:21:54 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 26 Jul 2005 02:21:54 -0700 (PDT) Subject: CPU/Hardware optimized for Erlang In-Reply-To: <200507251846.46500.will@misconception.org.uk> Message-ID: <20050726092154.81780.qmail@web34401.mail.mud.yahoo.com> --- Will Newton wrote: > I don't have any hard numbers of course (this is the > Internet after all), but > I would imagine that the number of transistors > required to implement the wide > array of instructions in such a chip limits the > possibilities of putting more > cores on a die because eventually your power > dissipation will become a > problem. Note that after decoding, the CISC instructions have been translated to uops, traces of uops, or what have you. I believe the difference with a RISC is small after that point. I also think other architectural features are more costly at this time, e.g., big branch predictors, large multiported register files, large instruction windows, fast clock ... > http://www.metagence.com/Products/index.asp I'll have a peek :-) > Are you aware of any overviews of this processor > architecture? The APZ family run the Ericsson AXE switches, and are programmed in PLEX. Lots of features will be recognized by Erlang programmers. While they are getting a bit long in the tooth (even pre-2000, I heard of multiple efforts to write emulators and JITs to get rid of them) they have had at least a 20-year run. When I left Ericsson, pre-2K, there were tens of thousands of people working with AXE and PLEX. Here's a sample article: http://www.ericsson.com/about/publications/review/1999_03/85.shtml I think ericsson.com has a bit more info if you rummage around. (Presumably, there are also *plenty* of readers on this list who do :-) Best, Thomas __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From John-Olof.Bauner@REDACTED Tue Jul 26 12:18:31 2005 From: John-Olof.Bauner@REDACTED (John-Olof Bauner) Date: Tue, 26 Jul 2005 12:18:31 +0200 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <20050726092154.81780.qmail@web34401.mail.mud.yahoo.com> References: <20050726092154.81780.qmail@web34401.mail.mud.yahoo.com> Message-ID: <42E60DF7.40208@ericsson.com> Thomas Lindgren wrote: >>Are you aware of any overviews of this processor >>architecture? >> >> > >The APZ family run the Ericsson AXE switches, and are >programmed in PLEX. Lots of features will be >recognized by Erlang programmers. > At some point the volume did not defend an own processor. I think it is only feasable for high volume devices. http://www.ericsson.com/press/archive/2000Q4/20001010-0060.html John-Olof From thomasl_erlang@REDACTED Tue Jul 26 12:34:12 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 26 Jul 2005 03:34:12 -0700 (PDT) Subject: CPU/Hardware optimized for Erlang In-Reply-To: <200507252256.j6PMu6QR261259@atlas.otago.ac.nz> Message-ID: <20050726103412.73888.qmail@web34410.mail.mud.yahoo.com> --- "Richard A. O'Keefe" wrote: > Will Newton wrote: > > Also bear in mind that Intel chips are > constrained > > very much by their legacy. > > But Thomas Lindgren > doesn't think it's a handicap. > > It is quite enough of a handicap for Transmeta to > carve themselves > a niche. The Pentiums do run-time compilation of > IA32 instructions > to a sort of RISC with the compilation done in > hardware a little at > a time; the Transmeta machines did run-time > compilation in software > in great big gulps and were able to achieve > near-Pentium speed at a > MUCH lower cost in electrical power. One should recall that Transmeta first wanted to compete (in the x86 market!) on performance, then switched to 'power' when raw performance wasn't competitive. And ultimately seem to have failed once Intel/AMD caught up. Even IBM and PowerPC have abandoned the desktop now. (Though, again, they seem to be doing interesting application-oriented SOCs -- Cell, Blue Gene.) The low-power market for general purpose desktops, blades, etc. seems to be sewn up by the incumbents today. An "Erlang" processor would have to play a different game compared to Transmeta, since you no longer can sell to the same market (until MS ports Office). So, what would induce a telecoms company to switch from Sun/Solaris to Erlang processor/*? Or Google to switch from Intel/Linux? Or Cisco to switch from ... Or an embedded box maker to use Erlang processors instead of brand X? For scientific work, a business case is not necessary of course. But in this case, what unique "Erlang" features will provide the big edge in, say, power? Multithreading + slow clock? Something else? And would it beat an Erlang R10B port to a "similar but non-Erlang" architecture? How sustainable is this advantage over time? (Is the advantage inherent in "Erlang" or something else?) (I'm being the difficult skeptic here, but these questions would ultimately have to be answered.) It seems far more fruitful to consider application areas instead. > Complexity certainly has an effect on barriers to > market entry. Of course. However, in this case, I'd also say that keeping backwards compatibility since 1979, with all attendant complexity, has NOT proved to be a great disadvantage regarding performance or power. (While also being a big business advantage.) > Intel machines are impressive, but basically, they > are brick outhouses > that fly only because they have extremely large jet > engines strapped on. ... flying faster than the competition, using the same or less fuel. And at cheaper prices, too :-) >> [APZ] > > Reference? See my other mail, or google "APZ Ericsson Review". Best, Thomas PS. And I haven't even started on compatibility issues with "software Erlang" yet :-) ____________________________________________________ Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs From thomasl_erlang@REDACTED Tue Jul 26 12:47:33 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 26 Jul 2005 03:47:33 -0700 (PDT) Subject: CPU/Hardware optimized for Erlang In-Reply-To: <42E60DF7.40208@ericsson.com> Message-ID: <20050726104733.24068.qmail@web34413.mail.mud.yahoo.com> --- John-Olof Bauner wrote: >>[APZ family] > > At some point the volume did not defend an own > processor. I think it is > only feasable for high volume devices. > http://www.ericsson.com/press/archive/2000Q4/20001010-0060.html Nothing withstands the killer micros. (On the other hand, Compaq has been assimilated and Alpha is basically dead :-) That's the other big difficulty for a real processor family: you have to _remain_ competitive ... The APZ still had 20+ years, well done. Best, Thomas __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From John-Olof.Bauner@REDACTED Tue Jul 26 12:54:00 2005 From: John-Olof.Bauner@REDACTED (John-Olof Bauner) Date: Tue, 26 Jul 2005 12:54:00 +0200 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <20050726104733.24068.qmail@web34413.mail.mud.yahoo.com> References: <20050726104733.24068.qmail@web34413.mail.mud.yahoo.com> Message-ID: <42E61648.7070205@ericsson.com> Thomas Lindgren wrote: >--- John-Olof Bauner >wrote: > > > >>>[APZ family] >>> >>> >>At some point the volume did not defend an own >>processor. I think it is >>only feasable for high volume devices. >> >> >> >http://www.ericsson.com/press/archive/2000Q4/20001010-0060.html > >Nothing withstands the killer micros. (On the other >hand, Compaq has been assimilated and Alpha is >basically dead :-) > > Only with x86 you are safe or who can kill Wintel? J-O From babchand_78@REDACTED Tue Jul 26 15:11:56 2005 From: babchand_78@REDACTED (Babu Dhandapani) Date: Tue, 26 Jul 2005 14:11:56 +0100 (BST) Subject: having installation problems Message-ID: <20050726131156.55358.qmail@web26610.mail.ukl.yahoo.com> Hi, I'm new to this field. I'm trying to install erlang in BSD. I went to dir /usr/ports/lang/erlang executed command "pkg_add -r erlang" & then executed "make" but it's giving the following error. Any help is appereciated. Thanks in advance. ===> Vulnerability check disabled, database not found ===> Extracting for erlang-r10b6_1,1 => Checksum OK for erlang/otp_src_R10B-6.tar.gz. => Checksum OK for erlang/otp_doc_man_R10B-6.tar.gz. => Checksum OK for erlang/otp_doc_html_R10B-6.tar.gz. ===> erlang-r10b6_1,1 depends on file: /usr/local/bin/perl5.8.6 - found ===> Patching for erlang-r10b6_1,1 ===> erlang-r10b6_1,1 depends on file: /usr/local/bin/perl5.8.6 - found ===> Applying FreeBSD patches for erlang-r10b6_1,1 ===> erlang-r10b6_1,1 depends on executable: gmake - found ===> erlang-r10b6_1,1 depends on file: /usr/local/bin/perl5.8.6 - found ===> erlang-r10b6_1,1 depends on shared library: iodbc.3 - found ===> Configuring for erlang-r10b6_1,1 checking host system type... i386-unknown-freebsd5.4 checking for GNU make... yes (gmake) checking for a BSD compatible install... /usr/bin/install -c -o root -g wheel checking whether ln -s works... yes checking for ranlib... ranlib creating ./config.status creating Makefile configuring in lib running /bin/sh /usr/ports/lang/erlang/work/otp_src_R10B-6/lib/configure --enable-threads --enable-hipe --enable-kernel-poll --prefix=/usr/local --cache-file=.././config.cache --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/lib creating ./config.status configuring in erl_interface/. running /bin/sh /usr/ports/lang/erlang/work/otp_src_R10B-6/lib/erl_interface/./configure --enable-threads --enable-hipe --enable-kernel-poll --prefix=/usr/local --cache-file=../../.././config.cache --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/lib/erl_interface/. checking host system type... i386-unknown-freebsd5.4 checking for gcc... cc checking whether the C compiler (cc -O -pipe -I/usr/local/include -L/usr/local/lib) works... yes checking whether the C compiler (cc -O -pipe -I/usr/local/include -L/usr/local/lib) is a cross-compiler... no checking whether we are using GNU C... yes checking whether cc accepts -g... yes checking how to run the C preprocessor... cc -E checking for ranlib... ranlib checking size of short... 2 checking size of int... 4 checking size of long... 4 checking size of void *... 4 checking size of long long... 8 checking for ar... ar checking for a BSD compatible install... /usr/bin/install -c -o root -g wheel checking for gethostbyname in -lnsl... no checking for getpeername in -lsocket... no checking for ANSI C header files... yes checking for sys/wait.h that is POSIX.1 compatible... yes checking for arpa/inet.h... yes checking for fcntl.h... yes checking for limits.h... yes checking for malloc.h... no checking for netdb.h... yes checking for netinet/in.h... yes checking for stddef.h... yes checking for stdlib.h... yes checking for string.h... yes checking for sys/param.h... yes checking for sys/socket.h... yes checking for sys/select.h... yes checking for sys/time.h... yes checking for unistd.h... yes checking for uid_t in sys/types.h... yes checking for pid_t... yes checking for size_t... yes checking whether time.h and sys/time.h may both be included... yes checking for working alloca.h... no checking for alloca... yes checking whether cc needs -traditional... no checking for 8-bit clean memcmp... yes checking for dup2... yes checking for gethostbyaddr... yes checking for gethostbyname... yes checking for gethostbyaddr_r... no checking for gethostbyname_r... no checking for gethostname... yes checking for writev... yes checking for gethrtime... no checking for gettimeofday... yes checking for inet_ntoa... yes checking for memchr... yes checking for memmove... yes checking for memset... yes checking for select... yes checking for socket... yes checking for strchr... yes checking for strerror... yes checking for strrchr... yes checking for strstr... yes checking for uname... yes checking for res_gethostbyname... no checking for res_gethostbyname in -lresolv... no checking for clock_gettime... yes checking for native win32 threads... no checking for pthread_create in -lpthread... yes checking for pthread_atfork in -lpthread... yes checking for pthread.h... yes checking for pthread/mit/pthread.h... no creating ./config.status creating src/i386-unknown-freebsd5.4/Makefile creating src/i386-unknown-freebsd5.4/eidefs.mk creating src/i386-unknown-freebsd5.4/config.h configuring in gs/. running /bin/sh /usr/ports/lang/erlang/work/otp_src_R10B-6/lib/gs/./configure --enable-threads --enable-hipe --enable-kernel-poll --prefix=/usr/local --cache-file=../../.././config.cache --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/lib/gs/. checking host system type... i386-unknown-freebsd5.4 checking for prebuilt tcl/tk in tcl/binaries/i386_freebsd5.tar.gz... not found creating ./config.status creating tcl/i386-unknown-freebsd5.4/Makefile creating tcl/win32/Makefile configuring in megaco/. running /bin/sh /usr/ports/lang/erlang/work/otp_src_R10B-6/lib/megaco/./configure --enable-threads --enable-hipe --enable-kernel-poll --prefix=/usr/local --cache-file=../../.././config.cache --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/lib/megaco/. checking host system type... i386-unknown-freebsd5.4 checking for flex... flex checking for yywrap in -lfl... yes checking for gcc... cc checking whether the C compiler (cc -O -pipe -I/usr/local/include -L/usr/local/lib) works... yes checking whether the C compiler (cc -O -pipe -I/usr/local/include -L/usr/local/lib) is a cross-compiler... no checking whether we are using GNU C... yes checking whether cc accepts -g... yes checking for ld.sh... no checking for ld... ld checking for linker flags for loadable drivers... -shared checking for perl... perl creating ./config.status creating src/flex/i386-unknown-freebsd5.4/Makefile configuring in odbc/. running /bin/sh /usr/ports/lang/erlang/work/otp_src_R10B-6/lib/odbc/./configure --enable-threads --enable-hipe --enable-kernel-poll --prefix=/usr/local --cache-file=../../.././config.cache --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/lib/odbc/. checking host system type... i386-unknown-freebsd5.4 checking for gcc... cc checking whether the C compiler (cc -O -pipe -I/usr/local/include -L/usr/local/lib) works... yes checking whether the C compiler (cc -O -pipe -I/usr/local/include -L/usr/local/lib) is a cross-compiler... no checking whether we are using GNU C... yes checking whether cc accepts -g... yes checking for mixed cygwin and native VC++ environment... no checking whether gmake sets ${MAKE}... yes checking for ld.sh... no checking for ld... ld checking for SQLAllocHandle in -lodbc... yes checking for main in -lodbc32... no checking for pthread_create in -lpthread... yes checking for odbc in standard locations... -L/usr/local/lib checking for connect... yes checking for gethostbyname... yes checking how to run the C preprocessor... cc -E checking for ANSI C header files... yes checking for fcntl.h... yes checking for netdb.h... yes checking for stdlib.h... yes checking for string.h... yes checking for sys/socket.h... yes checking for working const... yes checking for size_t... yes checking for memset... yes checking for socket... yes creating ./config.status creating c_src/i386-unknown-freebsd5.4/Makefile configuring in snmp/. running /bin/sh /usr/ports/lang/erlang/work/otp_src_R10B-6/lib/snmp/./configure --enable-threads --enable-hipe --enable-kernel-poll --prefix=/usr/local --cache-file=../../.././config.cache --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/lib/snmp/. checking host system type... i386-unknown-freebsd5.4 checking for perl... perl creating ./config.status creating mibs/Makefile configuring in erts running /bin/sh /usr/ports/lang/erlang/work/otp_src_R10B-6/erts/configure --enable-threads --enable-hipe --enable-kernel-poll --prefix=/usr/local --cache-file=.././config.cache --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/erts creating cache .././config.cache checking host system type... i386-unknown-freebsd5.4 checking for gcc... cc checking whether the C compiler (cc -O -pipe -I/usr/local/include -L/usr/local/lib) works... yes checking whether the C compiler (cc -O -pipe -I/usr/local/include -L/usr/local/lib) is a cross-compiler... no checking whether we are using GNU C... yes checking whether cc accepts -g... yes checking for POSIXized ISC... no checking for gcc... (cached) cc checking whether the C compiler (cc -O -pipe -I/usr/local/include -L/usr/local/lib) works... yes checking whether the C compiler (cc -O -pipe -I/usr/local/include -L/usr/local/lib) is a cross-compiler... no checking whether we are using GNU C... (cached) yes checking whether cc accepts -g... (cached) yes checking for mixed cygwin and native VC++ environment... no checking how to run the C preprocessor... cc -E checking for ranlib... ranlib checking for bison... bison -y checking for perl5... /usr/local/bin/perl5 checking whether ln -s works... yes checking for ar... ar checking for rm... /bin/rm checking for mkdir... /bin/mkdir checking for a BSD compatible install... /usr/bin/install -c -o root -g wheel checking how to create a directory including parents... /usr/bin/install -c -o root -g wheel -d checking for Cygwin environment... no checking for mingw32 environment... no checking for executable suffix... no checking for object suffix... o checking for extra flags needed to export symbols... -Wl,-export-dynamic checking for sin in -lm... yes checking for dlopen in -ldl... no checking for main in -linet... no checking for native win32 threads... no checking for pthread_create in -lpthread... yes checking for pthread.h... yes checking for pthread/mit/pthread.h... no checking for sys/time.h... yes checking for pthread_atfork... yes checking for pthread_mutexattr_settype... yes checking for pthread_mutexattr_setkind_np... yes checking whether the emulator should use threads... yes; thread support requested checking for tgetent in -lncurses... yes checking for connect... yes checking for gethostbyname... yes checking for gethostbyname_r... no checking if netdb.h requires netinet/in.h to be previously included... no checking for socklen_t... yes checking for h_errno declaration in netdb.h... yes checking for dirent.h that defines DIR... yes checking for opendir in -ldir... no checking for ANSI C header files... yes checking for sys/wait.h that is POSIX.1 compatible... yes checking whether time.h and sys/time.h may both be included... yes checking for fcntl.h... yes checking for limits.h... yes checking for unistd.h... yes checking for syslog.h... yes checking for dlfcn.h... yes checking for ieeefp.h... yes checking for sys/stropts.h... no checking for sys/ioctl.h... yes checking for sys/time.h... (cached) yes checking for sys/uio.h... yes checking for sys/sockio.h... yes checking for sys/socketio.h... no checking for net/errno.h... no checking for malloc.h... no checking for mach-o/dyld.h... no checking for arpa/nameser.h... yes checking for sys/devpoll.h... no checking for linux/kpoll.h... no checking for sys/event.h... yes checking for SO_BSDCOMPAT declaration... no checking for INADDR_LOOPBACK in netinet/in.h... yes checking for sys_errlist declaration in stdio.h or errno.h... yes checking for working const... yes checking return type of signal handlers... void checking for off_t... yes checking for pid_t... yes checking for size_t... yes checking whether struct tm is in sys/time.h or time.h... time.h checking whether struct sockaddr has sa_len field... yes checking for struct exception (and matherr function)... yes checking size of char... 1 checking size of short... 2 checking size of int... 4 checking size of long... 4 checking size of void *... 4 checking size of long long... 8 checking size of size_t... 4 checking size of off_t... 8 checking int/long/void*/size_t sizes... ok checking whether byte ordering is bigendian... no checking for getipnodebyname... yes checking for getipnodebyaddr... yes checking for ieee_handler... no checking for fpsetmask... no checking for finite... yes checking for isnan... yes checking for isinf... yes checking for res_gethostbyname... no checking for dlopen... yes checking for pread... yes checking for pwrite... yes checking for writev... yes checking for memmove... yes checking for strerror... yes checking for strerror_r... yes checking for strncasecmp... yes checking for gethrtime... no checking for localtime_r... yes checking for gmtime_r... yes checking for mremap... no checking for memcpy... yes checking for mallopt... no checking whether setvbuf arguments are reversed... no checking if vfork is known to hang multithreaded applications... no checking for vfork.h... no checking for working vfork... yes checking for vprintf... yes checking for unistd.h... (cached) yes checking for getpagesize... yes checking for working mmap... yes checking for conflicting declaration of fread... yes checking for IP version 6 support... yes checking for multicast support... yes checking how to correct for time adjustments... none checking for unreliable floating point execptions... reliable checking if gethrvtime works and how to use it... not working checking if clock_gettime works... not working checking whether to redefine FD_SETSIZE... no checking for vperfctr_info in -lperfctr... no checking for working poll()... ok checking for compiler flags for loadable drivers... -D_THREAD_SAFE -D_REENTRANT -O -pipe -I/usr/local/include -I/usr/ports/lang/erlang/work/otp_src_R10B-6/erts/i386-unknown-freebsd5.4 -fPIC checking for linker for loadable drivers... ld checking for linker flags for loadable drivers... -shared checking for 'runtime library path' linker flag... -R checking for a compiler that handles jumptables... cc checking for OpenSSL >= 0.9.7 in standard locations... /usr checking for OpenSSL kerberos 5 support... no checking for kstat_open in -lkstat... no checking for javac.sh... no checking for javac... no checking for guavac... no checking for gcj... no checking for jikes... no checking for bock... no checking for c++... c++ updating cache .././config.cache creating ./config.status creating emulator/i386-unknown-freebsd5.4/Makefile creating emulator/zlib/i386-unknown-freebsd5.4/Makefile creating epmd/src/i386-unknown-freebsd5.4/Makefile creating etc/common/i386-unknown-freebsd5.4/Makefile creating include/internal/i386-unknown-freebsd5.4/ethread.mk creating lib_src/i386-unknown-freebsd5.4/Makefile creating ../make/i386-unknown-freebsd5.4/otp.mk creating ../lib/ic/c_src/i386-unknown-freebsd5.4/Makefile creating ../lib/os_mon/c_src/i386-unknown-freebsd5.4/Makefile creating ../lib/ssl/c_src/i386-unknown-freebsd5.4/Makefile creating ../lib/ssl/examples/certs/i386-unknown-freebsd5.4/Makefile creating ../lib/crypto/c_src/i386-unknown-freebsd5.4/Makefile creating ../lib/ssh/c_src/i386-unknown-freebsd5.4/Makefile creating ../lib/orber/c_src/i386-unknown-freebsd5.4/Makefile creating ../lib/runtime_tools/c_src/i386-unknown-freebsd5.4/Makefile creating ../lib/asn1/c_src/i386-unknown-freebsd5.4/Makefile creating i386-unknown-freebsd5.4/config.h creating include/internal/i386-unknown-freebsd5.4/ethread_header_config.h ********************************************************************* ********************** APPLICATIONS DISABLED ********************** ********************************************************************* jinterface : No Java compiler found ********************************************************************* ===> Building for erlang-r10b6_1,1 cd erts/emulator && ERL_TOP=/usr/ports/lang/erlang/work/otp_src_R10B-6 gmake generate depend gmake[1]: Entering directory `/usr/ports/lang/erlang/work/otp_src_R10B-6/erts/emulator' gmake -f i386-unknown-freebsd5.4/Makefile generate gmake[2]: Entering directory `/usr/ports/lang/erlang/work/otp_src_R10B-6/erts/emulator' Makefile.in has been updated, please re-run configure! gmake[2]: Leaving directory `/usr/ports/lang/erlang/work/otp_src_R10B-6/erts/emulator' gmake[1]: Leaving directory `/usr/ports/lang/erlang/work/otp_src_R10B-6/erts/emulator' *** Error code 2 Stop in /usr/ports/lang/erlang. Thanks, Babu ___________________________________________________________ Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com From nm@REDACTED Tue Jul 26 16:02:08 2005 From: nm@REDACTED (Gaspar Chilingarov) Date: Tue, 26 Jul 2005 19:02:08 +0500 Subject: having installation problems In-Reply-To: <20050726131156.55358.qmail@web26610.mail.ukl.yahoo.com> References: <20050726131156.55358.qmail@web26610.mail.ukl.yahoo.com> Message-ID: <42E64260.2000708@web.am> Hello! It's more freebsd-related question , than erlang :) Babu Dhandapani wrote: > Hi, > I'm new to this field. > I'm trying to install erlang in BSD. > I went to dir /usr/ports/lang/erlang > executed command "pkg_add -r erlang" pkg_add -r fetches binary package from remote server and installs it. you do not need to run make then. Choose one of the ways -- either you install from package or from port. Ok, let's see what if you install from port - probably you have no auto* ports - autoconf and automake. I got latest versions of then, and erlang updates automagically w/o any problems. > & then executed "make" > but it's giving the following error. > > Any help is appereciated. > > Thanks in advance. > > ===> Vulnerability check disabled, database not found > ===> Extracting for erlang-r10b6_1,1 > => Checksum OK for erlang/otp_src_R10B-6.tar.gz. > => Checksum OK for erlang/otp_doc_man_R10B-6.tar.gz. > => Checksum OK for erlang/otp_doc_html_R10B-6.tar.gz. > ===> erlang-r10b6_1,1 depends on file: > /usr/local/bin/perl5.8.6 - found > ===> Patching for erlang-r10b6_1,1 > ===> erlang-r10b6_1,1 depends on file: > /usr/local/bin/perl5.8.6 - found > ===> Applying FreeBSD patches for erlang-r10b6_1,1 > ===> erlang-r10b6_1,1 depends on executable: gmake - > found > ===> erlang-r10b6_1,1 depends on file: > /usr/local/bin/perl5.8.6 - found > ===> erlang-r10b6_1,1 depends on shared library: > iodbc.3 - found > ===> Configuring for erlang-r10b6_1,1 > checking host system type... i386-unknown-freebsd5.4 > checking for GNU make... yes (gmake) > checking for a BSD compatible install... > /usr/bin/install -c -o root -g wheel > checking whether ln -s works... yes > checking for ranlib... ranlib > creating ./config.status > creating Makefile > configuring in lib > running /bin/sh > /usr/ports/lang/erlang/work/otp_src_R10B-6/lib/configure > --enable-threads --enable-hipe --enable-kernel-poll > --prefix=/usr/local --cache-file=.././config.cache > --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/lib > creating ./config.status > configuring in erl_interface/. > running /bin/sh > /usr/ports/lang/erlang/work/otp_src_R10B-6/lib/erl_interface/./configure > --enable-threads --enable-hipe --enable-kernel-poll > --prefix=/usr/local > --cache-file=../../.././config.cache > --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/lib/erl_interface/. > checking host system type... i386-unknown-freebsd5.4 > checking for gcc... cc > checking whether the C compiler (cc -O -pipe > -I/usr/local/include -L/usr/local/lib) works... yes > checking whether the C compiler (cc -O -pipe > -I/usr/local/include -L/usr/local/lib) is a > cross-compiler... no > checking whether we are using GNU C... yes > checking whether cc accepts -g... yes > checking how to run the C preprocessor... cc -E > checking for ranlib... ranlib > checking size of short... 2 > checking size of int... 4 > checking size of long... 4 > checking size of void *... 4 > checking size of long long... 8 > checking for ar... ar > checking for a BSD compatible install... > /usr/bin/install -c -o root -g wheel > checking for gethostbyname in -lnsl... no > checking for getpeername in -lsocket... no > checking for ANSI C header files... yes > checking for sys/wait.h that is POSIX.1 compatible... > yes > checking for arpa/inet.h... yes > checking for fcntl.h... yes > checking for limits.h... yes > checking for malloc.h... no > checking for netdb.h... yes > checking for netinet/in.h... yes > checking for stddef.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking for sys/param.h... yes > checking for sys/socket.h... yes > checking for sys/select.h... yes > checking for sys/time.h... yes > checking for unistd.h... yes > checking for uid_t in sys/types.h... yes > checking for pid_t... yes > checking for size_t... yes > checking whether time.h and sys/time.h may both be > included... yes > checking for working alloca.h... no > checking for alloca... yes > checking whether cc needs -traditional... no > checking for 8-bit clean memcmp... yes > checking for dup2... yes > checking for gethostbyaddr... yes > checking for gethostbyname... yes > checking for gethostbyaddr_r... no > checking for gethostbyname_r... no > checking for gethostname... yes > checking for writev... yes > checking for gethrtime... no > checking for gettimeofday... yes > checking for inet_ntoa... yes > checking for memchr... yes > checking for memmove... yes > checking for memset... yes > checking for select... yes > checking for socket... yes > checking for strchr... yes > checking for strerror... yes > checking for strrchr... yes > checking for strstr... yes > checking for uname... yes > checking for res_gethostbyname... no > checking for res_gethostbyname in -lresolv... no > checking for clock_gettime... yes > checking for native win32 threads... no > checking for pthread_create in -lpthread... yes > checking for pthread_atfork in -lpthread... yes > checking for pthread.h... yes > checking for pthread/mit/pthread.h... no > creating ./config.status > creating src/i386-unknown-freebsd5.4/Makefile > creating src/i386-unknown-freebsd5.4/eidefs.mk > creating src/i386-unknown-freebsd5.4/config.h > configuring in gs/. > running /bin/sh > /usr/ports/lang/erlang/work/otp_src_R10B-6/lib/gs/./configure > --enable-threads --enable-hipe --enable-kernel-poll > --prefix=/usr/local > --cache-file=../../.././config.cache > --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/lib/gs/. > checking host system type... i386-unknown-freebsd5.4 > checking for prebuilt tcl/tk in > tcl/binaries/i386_freebsd5.tar.gz... not found > creating ./config.status > creating tcl/i386-unknown-freebsd5.4/Makefile > creating tcl/win32/Makefile > configuring in megaco/. > running /bin/sh > /usr/ports/lang/erlang/work/otp_src_R10B-6/lib/megaco/./configure > --enable-threads --enable-hipe --enable-kernel-poll > --prefix=/usr/local > --cache-file=../../.././config.cache > --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/lib/megaco/. > checking host system type... i386-unknown-freebsd5.4 > checking for flex... flex > checking for yywrap in -lfl... yes > checking for gcc... cc > checking whether the C compiler (cc -O -pipe > -I/usr/local/include -L/usr/local/lib) works... yes > checking whether the C compiler (cc -O -pipe > -I/usr/local/include -L/usr/local/lib) is a > cross-compiler... no > checking whether we are using GNU C... yes > checking whether cc accepts -g... yes > checking for ld.sh... no > checking for ld... ld > checking for linker flags for loadable drivers... > -shared > checking for perl... perl > creating ./config.status > creating src/flex/i386-unknown-freebsd5.4/Makefile > configuring in odbc/. > running /bin/sh > /usr/ports/lang/erlang/work/otp_src_R10B-6/lib/odbc/./configure > --enable-threads --enable-hipe --enable-kernel-poll > --prefix=/usr/local > --cache-file=../../.././config.cache > --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/lib/odbc/. > checking host system type... i386-unknown-freebsd5.4 > checking for gcc... cc > checking whether the C compiler (cc -O -pipe > -I/usr/local/include -L/usr/local/lib) works... yes > checking whether the C compiler (cc -O -pipe > -I/usr/local/include -L/usr/local/lib) is a > cross-compiler... no > checking whether we are using GNU C... yes > checking whether cc accepts -g... yes > checking for mixed cygwin and native VC++ > environment... no > checking whether gmake sets ${MAKE}... yes > checking for ld.sh... no > checking for ld... ld > checking for SQLAllocHandle in -lodbc... yes > checking for main in -lodbc32... no > checking for pthread_create in -lpthread... yes > checking for odbc in standard locations... > -L/usr/local/lib > checking for connect... yes > checking for gethostbyname... yes > checking how to run the C preprocessor... cc -E > checking for ANSI C header files... yes > checking for fcntl.h... yes > checking for netdb.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking for sys/socket.h... yes > checking for working const... yes > checking for size_t... yes > checking for memset... yes > checking for socket... yes > creating ./config.status > creating c_src/i386-unknown-freebsd5.4/Makefile > configuring in snmp/. > running /bin/sh > /usr/ports/lang/erlang/work/otp_src_R10B-6/lib/snmp/./configure > --enable-threads --enable-hipe --enable-kernel-poll > --prefix=/usr/local > --cache-file=../../.././config.cache > --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/lib/snmp/. > checking host system type... i386-unknown-freebsd5.4 > checking for perl... perl > creating ./config.status > creating mibs/Makefile > configuring in erts > running /bin/sh > /usr/ports/lang/erlang/work/otp_src_R10B-6/erts/configure > --enable-threads --enable-hipe --enable-kernel-poll > --prefix=/usr/local --cache-file=.././config.cache > --srcdir=/usr/ports/lang/erlang/work/otp_src_R10B-6/erts > creating cache .././config.cache > checking host system type... i386-unknown-freebsd5.4 > checking for gcc... cc > checking whether the C compiler (cc -O -pipe > -I/usr/local/include -L/usr/local/lib) works... yes > checking whether the C compiler (cc -O -pipe > -I/usr/local/include -L/usr/local/lib) is a > cross-compiler... no > checking whether we are using GNU C... yes > checking whether cc accepts -g... yes > checking for POSIXized ISC... no > checking for gcc... (cached) cc > checking whether the C compiler (cc -O -pipe > -I/usr/local/include -L/usr/local/lib) works... yes > checking whether the C compiler (cc -O -pipe > -I/usr/local/include -L/usr/local/lib) is a > cross-compiler... no > checking whether we are using GNU C... (cached) yes > checking whether cc accepts -g... (cached) yes > checking for mixed cygwin and native VC++ > environment... no > checking how to run the C preprocessor... cc -E > checking for ranlib... ranlib > checking for bison... bison -y > checking for perl5... /usr/local/bin/perl5 > checking whether ln -s works... yes > checking for ar... ar > checking for rm... /bin/rm > checking for mkdir... /bin/mkdir > checking for a BSD compatible install... > /usr/bin/install -c -o root -g wheel > checking how to create a directory including > parents... /usr/bin/install -c -o root -g wheel -d > checking for Cygwin environment... no > checking for mingw32 environment... no > checking for executable suffix... no > checking for object suffix... o > checking for extra flags needed to export symbols... > -Wl,-export-dynamic > checking for sin in -lm... yes > checking for dlopen in -ldl... no > checking for main in -linet... no > checking for native win32 threads... no > checking for pthread_create in -lpthread... yes > checking for pthread.h... yes > checking for pthread/mit/pthread.h... no > checking for sys/time.h... yes > checking for pthread_atfork... yes > checking for pthread_mutexattr_settype... yes > checking for pthread_mutexattr_setkind_np... yes > checking whether the emulator should use threads... > yes; thread support requested > checking for tgetent in -lncurses... yes > checking for connect... yes > checking for gethostbyname... yes > checking for gethostbyname_r... no > checking if netdb.h requires netinet/in.h to be > previously included... no > checking for socklen_t... yes > checking for h_errno declaration in netdb.h... yes > checking for dirent.h that defines DIR... yes > checking for opendir in -ldir... no > checking for ANSI C header files... yes > checking for sys/wait.h that is POSIX.1 compatible... > yes > checking whether time.h and sys/time.h may both be > included... yes > checking for fcntl.h... yes > checking for limits.h... yes > checking for unistd.h... yes > checking for syslog.h... yes > checking for dlfcn.h... yes > checking for ieeefp.h... yes > checking for sys/stropts.h... no > checking for sys/ioctl.h... yes > checking for sys/time.h... (cached) yes > checking for sys/uio.h... yes > checking for sys/sockio.h... yes > checking for sys/socketio.h... no > checking for net/errno.h... no > checking for malloc.h... no > checking for mach-o/dyld.h... no > checking for arpa/nameser.h... yes > checking for sys/devpoll.h... no > checking for linux/kpoll.h... no > checking for sys/event.h... yes > checking for SO_BSDCOMPAT declaration... no > checking for INADDR_LOOPBACK in netinet/in.h... yes > checking for sys_errlist declaration in stdio.h or > errno.h... yes > checking for working const... yes > checking return type of signal handlers... void > checking for off_t... yes > checking for pid_t... yes > checking for size_t... yes > checking whether struct tm is in sys/time.h or > time.h... time.h > checking whether struct sockaddr has sa_len field... > yes > checking for struct exception (and matherr > function)... yes > checking size of char... 1 > checking size of short... 2 > checking size of int... 4 > checking size of long... 4 > checking size of void *... 4 > checking size of long long... 8 > checking size of size_t... 4 > checking size of off_t... 8 > checking int/long/void*/size_t sizes... ok > checking whether byte ordering is bigendian... no > checking for getipnodebyname... yes > checking for getipnodebyaddr... yes > checking for ieee_handler... no > checking for fpsetmask... no > checking for finite... yes > checking for isnan... yes > checking for isinf... yes > checking for res_gethostbyname... no > checking for dlopen... yes > checking for pread... yes > checking for pwrite... yes > checking for writev... yes > checking for memmove... yes > checking for strerror... yes > checking for strerror_r... yes > checking for strncasecmp... yes > checking for gethrtime... no > checking for localtime_r... yes > checking for gmtime_r... yes > checking for mremap... no > checking for memcpy... yes > checking for mallopt... no > checking whether setvbuf arguments are reversed... no > checking if vfork is known to hang multithreaded > applications... no > checking for vfork.h... no > checking for working vfork... yes > checking for vprintf... yes > checking for unistd.h... (cached) yes > checking for getpagesize... yes > checking for working mmap... yes > checking for conflicting declaration of fread... yes > checking for IP version 6 support... yes > checking for multicast support... yes > checking how to correct for time adjustments... none > checking for unreliable floating point execptions... > reliable > checking if gethrvtime works and how to use it... not > working > checking if clock_gettime works... not working > checking whether to redefine FD_SETSIZE... no > checking for vperfctr_info in -lperfctr... no > checking for working poll()... ok > checking for compiler flags for loadable drivers... > -D_THREAD_SAFE -D_REENTRANT -O -pipe > -I/usr/local/include > -I/usr/ports/lang/erlang/work/otp_src_R10B-6/erts/i386-unknown-freebsd5.4 > -fPIC > checking for linker for loadable drivers... ld > checking for linker flags for loadable drivers... > -shared > checking for 'runtime library path' linker flag... -R > checking for a compiler that handles jumptables... cc > checking for OpenSSL >= 0.9.7 in standard locations... > /usr > checking for OpenSSL kerberos 5 support... no > checking for kstat_open in -lkstat... no > checking for javac.sh... no > checking for javac... no > checking for guavac... no > checking for gcj... no > checking for jikes... no > checking for bock... no > checking for c++... c++ > updating cache .././config.cache > creating ./config.status > creating emulator/i386-unknown-freebsd5.4/Makefile > creating > emulator/zlib/i386-unknown-freebsd5.4/Makefile > creating epmd/src/i386-unknown-freebsd5.4/Makefile > creating etc/common/i386-unknown-freebsd5.4/Makefile > creating > include/internal/i386-unknown-freebsd5.4/ethread.mk > creating lib_src/i386-unknown-freebsd5.4/Makefile > creating ../make/i386-unknown-freebsd5.4/otp.mk > creating > ../lib/ic/c_src/i386-unknown-freebsd5.4/Makefile > creating > ../lib/os_mon/c_src/i386-unknown-freebsd5.4/Makefile > creating > ../lib/ssl/c_src/i386-unknown-freebsd5.4/Makefile > creating > ../lib/ssl/examples/certs/i386-unknown-freebsd5.4/Makefile > creating > ../lib/crypto/c_src/i386-unknown-freebsd5.4/Makefile > creating > ../lib/ssh/c_src/i386-unknown-freebsd5.4/Makefile > creating > ../lib/orber/c_src/i386-unknown-freebsd5.4/Makefile > creating > ../lib/runtime_tools/c_src/i386-unknown-freebsd5.4/Makefile > creating > ../lib/asn1/c_src/i386-unknown-freebsd5.4/Makefile > creating i386-unknown-freebsd5.4/config.h > creating > include/internal/i386-unknown-freebsd5.4/ethread_header_config.h > ********************************************************************* > ********************** APPLICATIONS DISABLED > ********************** > ********************************************************************* > > jinterface : No Java compiler found > > ********************************************************************* > ===> Building for erlang-r10b6_1,1 > cd erts/emulator && > ERL_TOP=/usr/ports/lang/erlang/work/otp_src_R10B-6 > gmake generate depend > gmake[1]: Entering directory > `/usr/ports/lang/erlang/work/otp_src_R10B-6/erts/emulator' > gmake -f i386-unknown-freebsd5.4/Makefile generate > gmake[2]: Entering directory > `/usr/ports/lang/erlang/work/otp_src_R10B-6/erts/emulator' > > Makefile.in has been updated, please re-run > configure! > > gmake[2]: Leaving directory > `/usr/ports/lang/erlang/work/otp_src_R10B-6/erts/emulator' > gmake[1]: Leaving directory > `/usr/ports/lang/erlang/work/otp_src_R10B-6/erts/emulator' > *** Error code 2 > > Stop in /usr/ports/lang/erlang. > > > Thanks, > Babu > > > > > > ___________________________________________________________ > Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com > -- Gaspar Chilingarov System Administrator t +37491 419763 (mob) t +37410 240399 (office) w www.web.am i 63174784 e nm@REDACTED From erlang@REDACTED Tue Jul 26 16:25:17 2005 From: erlang@REDACTED (Inswitch Solutions) Date: Tue, 26 Jul 2005 11:25:17 -0300 Subject: String variable eval in binaries Message-ID: <020b01c591ed$dc181690$4a00a8c0@Inswitch251> Hi, <<"hello">>. <<104,101,108,108,111>> <<[$h, $e, $l, $l, $o]>>. ** exited: {badarg,[{erl_eval,expr,3}]} ** Str = "hello". <>. ** exited: {badarg,[{erl_eval,expr,3}]} ** Int =1. <>. <<1>> "hello" == [$h, $e, $l, $l, $o]. true Any ideas? thanks, Eduardo Prepaid Expertise - Programmable Switches Powered by Ericsson Licensed Technology Eng. Eduardo Figoli - Development Center - IN Switch Solutions Inc. Headquarters - Miami-U.S.A. Tel: 1305-3578076 Fax: 1305-7686260 Development Center - Montevideo - Uruguay Tel/Fax: 5982-7104457 e-mail: eduardo@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1429 bytes Desc: not available URL: From thomasl_erlang@REDACTED Tue Jul 26 16:57:26 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 26 Jul 2005 07:57:26 -0700 (PDT) Subject: String variable eval in binaries In-Reply-To: <020b01c591ed$dc181690$4a00a8c0@Inswitch251> Message-ID: <20050726145726.22818.qmail@web34407.mail.mud.yahoo.com> --- Inswitch Solutions wrote: > <<[$h, $e, $l, $l, $o]>>. > ** exited: {badarg,[{erl_eval,expr,3}]} ** You can't put a list directly into a binary. (At least not today.) > Str = "hello". > <>. > ** exited: {badarg,[{erl_eval,expr,3}]} ** Again, Str is a list. You can do Bin1 = list_to_binary(Str). <>. %% redundant in this case Note that you can't even write <> directly: the default type of Bin1 inside the binary expression is something else, and the value of Bin1 can't be converted to that. So, to summarize: 1. The notation <<"foo">> suggests a list/string put inside a binary, but this is not what it means. 2. You can't put a list directly into a binary. Convert the list into a binary first :-) (That's more helpful when you are mashing together several lists, I guess.) Best, Thomas __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From serge@REDACTED Tue Jul 26 16:59:13 2005 From: serge@REDACTED (Serge Aleynikov) Date: Tue, 26 Jul 2005 10:59:13 -0400 Subject: String variable eval in binaries In-Reply-To: <020b01c591ed$dc181690$4a00a8c0@Inswitch251> References: <020b01c591ed$dc181690$4a00a8c0@Inswitch251> Message-ID: <42E64FC1.2000208@hq.idt.net> I don't think it's a valid syntax. Use: <<$h, $e, $l, $l, $o>>. and list_to_binary(Str). Serge Inswitch Solutions wrote: > > Hi, > > <<"hello">>. > <<104,101,108,108,111>> > <<[$h, $e, $l, $l, $o]>>. > ** exited: {badarg,[{erl_eval,expr,3}]} ** > Str = "hello". > <>. > ** exited: {badarg,[{erl_eval,expr,3}]} ** > Int =1. > <>. > <<1>> > > "hello" == [$h, $e, $l, $l, $o]. > true > > > Any ideas? > > > thanks, > Eduardo > > > > *Prepaid Expertise - Programmable Switches > Powered by Ericsson Licensed Technology > Eng. Eduardo Figoli - Development Center - IN Switch Solutions Inc. > Headquarters - Miami-U.S.A. Tel: 1305-3578076 Fax: 1305-7686260 > Development Center - Montevideo - Uruguay Tel/Fax: 5982-7104457 > e-mail: eduardo@REDACTED From james.hague@REDACTED Tue Jul 26 20:03:46 2005 From: james.hague@REDACTED (James Hague) Date: Tue, 26 Jul 2005 13:03:46 -0500 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <20050722122549.32771.qmail@web34413.mail.mud.yahoo.com> References: <20050722122549.32771.qmail@web34413.mail.mud.yahoo.com> Message-ID: Thomas Lindgren wrote: >(And why would the erlang chip have comparable >performance at lower power?) I wasn't talking about a 3GHz Erlang processor using less power than, say, a 3GHz Pentium 4 (though I suspect it would). I was musing about an Erlang processor that gives performance roughly equivalent to the BEAM *emulator* running on a 3GHz Pentium 4. Using the old ECOMP numbers, I'm guessing that this could be done with an Erlang processor running in the ballpark of 100MHz. To a lot of programmers, a super micro CPU is simply a platform to write interpreted Python, Perl, REBOL, etc. Or Erlang. Cutting out the middleman and designing the hardware to execute the language directly is certainly going to be more efficient. This is different than the long ago debates on the subject, in which people (Wirth, et al) would compare writing a *compiler* for an off the shelf CPU vs. designing a custom processor. In this case we're talking about looser dynamic languages. On a related note, you could design an interesting cache architecture for a language without destructive updates. From ok@REDACTED Wed Jul 27 02:27:07 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 27 Jul 2005 12:27:07 +1200 (NZST) Subject: CPU/Hardware optimized for Erlang Message-ID: <200507270027.j6R0R73E247363@atlas.otago.ac.nz> I wrote: > Hmm. Intel already *have* a machine that does that, the ARM. > There are versions of the ARM with direct-in-hardware JVM support. yvan sur free challenged this. I usually read about embed board for robotic and never see direct hardware JVM support. Most of the time there is an JVM code on external flash memory on an ARM based board. Do you have any link please ? Well, a web search for "ARM Thumb Java" will get you there quick enough. Try this article: Mobile Devices: RISC-Java blend powers cores by Edward Nevill ... & Andrew Rose ... EE Times January 6, 2002 URL: htp://www.eetimes.com/story/OEG20010723S0049 [hmm, that link points to a 2001-07-23 version of the same text.] ... we came up with the final design--Jazelle, a way of integrating Java execution right into the ARM core. ... To create Jazelle technology, we added a third instruction set--Java byte code--to the processor, with instruction-set support for entering and exiting Java applications. ... Once in Java state, the processor is in every way a Java processor, ... Or, of course, you could just visit www.arm.com. Their "choosing cores" video mentions Jazelle in the "application cores" section. Type "Jazelle" into the search box and one of the results you get is a June 2004 press release: Sony Ericsson Licences (sic.) ARM JTEK Software for Jazelle Java Acceleration Technology so Ericsson are using this processor in Java phones. Of course this direct low-power execution of Java byte code gives Java an advantage in the mobile phone market that Erlang does not enjoy. From ok@REDACTED Wed Jul 27 03:38:05 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 27 Jul 2005 13:38:05 +1200 (NZST) Subject: CPU/Hardware optimized for Erlang Message-ID: <200507270138.j6R1c5uF276744@atlas.otago.ac.nz> By the way, I note that someone mentioned that Ericsson had jumped APZ ship for Alpha, but that Alpha was dead. But HP announced new improved Alphas in October 2003 and more new improved Alphas in August 2004. The "Alpha RetainTrust program" is named in the best tradition of NewSpeak. It's all about encouraging customers to *dis*trust Alpha and jump to "Integrity". Even so, Alpha isn't *quite* dead yet. From vances@REDACTED Wed Jul 27 04:43:13 2005 From: vances@REDACTED (Vance Shipley) Date: Tue, 26 Jul 2005 22:43:13 -0400 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <200507270138.j6R1c5uF276744@atlas.otago.ac.nz> References: <200507270138.j6R1c5uF276744@atlas.otago.ac.nz> Message-ID: <20050727024313.GF79370@feeble.motivity.ca> A little web research reveals that in June 2004 Ericsson announced it would be deploying AXE 810 switches in Macau using the APZ 212 33C processors. I assume the experiment with Alphas was a dead end, probably in no small part because they went away. Anyway it is nice to see that someone is still buying big iron. :) -Vance From fritchie@REDACTED Wed Jul 27 08:20:11 2005 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 27 Jul 2005 01:20:11 -0500 Subject: TCP stack throughput In-Reply-To: Message of "Sun, 24 Jul 2005 10:36:21 +0200." Message-ID: <200507270620.j6R6KBIm019207@snookles.snookles.com> I'm sorry, I don't have Javier's original message in my mailbox. >>>>> "uw" == Ulf Wiger writes: uw> Den 2005-07-19 19:39:12 skrev Javier Par?s Fern?ndez uw> : >> I observed several funny things. The time spent processing the >> connections seems to go up exponentially, but it is not limited by >> cpu, which was always under 50%. I don't know what is throttling >> it. Are you using Linux, and if yes, which Linux distribution are you using? I was very surprised to discover that the "top" program distributed with Red Hat Advanced Server 3.0, when running on an Intel CPU with Hyper-Threading enabled(*), will report that: main() { int i; for (;;) { i++; } } ... will only consume 50% CPU time. I don't know why "top"'s behavior is different for AS 3.0: I'm used to "top" reporting 100% CPU consumption by a program like that, regardless of how many CPUs the machine has. -Scott (*) I do not know if disabling HT makes a difference or not. From irmatov@REDACTED Wed Jul 27 08:51:08 2005 From: irmatov@REDACTED (Timur Irmatov) Date: Wed, 27 Jul 2005 11:51:08 +0500 Subject: TCP stack throughput In-Reply-To: <200507270620.j6R6KBIm019207@snookles.snookles.com> References: <200507270620.j6R6KBIm019207@snookles.snookles.com> Message-ID: <241d382f05072623514e81aca0@mail.gmail.com> On 7/27/05, Scott Lystig Fritchie wrote: > Are you using Linux, and if yes, which Linux distribution are you > using? I was very surprised to discover that the "top" program > distributed with Red Hat Advanced Server 3.0, when running on an Intel > CPU with Hyper-Threading enabled(*), will report that: > > main() > { > int i; > > for (;;) { > i++; > } > } > > ... will only consume 50% CPU time. > > I don't know why "top"'s behavior is different for AS 3.0: I'm used to > "top" reporting 100% CPU consumption by a program like that, > regardless of how many CPUs the machine has. I had similar experience with squid running on Xeon processor (which is detected as two processors). As squid is single-threaded app it uses only one CPU at a time and thus 'top' reports that average usage of both processors was only 50% maximum.. -- Timur Irmatov, JID: thor@REDACTED From valentin@REDACTED Wed Jul 27 09:10:31 2005 From: valentin@REDACTED (Valentin Micic) Date: Wed, 27 Jul 2005 09:10:31 +0200 Subject: TCP stack throughput References: <200507270620.j6R6KBIm019207@snookles.snookles.com> Message-ID: <009501c5927a$476c4330$0100a8c0@MONEYMAKER2> > (*) I do not know if disabling HT makes a difference or not. IMHO, it certainly should. Previous versions of RedHat had a very poor support for mutliple CPUs, i.e. any given process would be bound to single CPU, and hence, always executed on this particular processor. It is my understanding that task scheduling did improve in AS 3.0, spreading process execution accross all available processors. HT enabled CPU is seen as two independent CPUs, and under such circumstances, a running away process shall utilise 50% of each available processor. It makes sence, if you think about it... any given process, at any given point shall be executed on one CPU. Upon spending its slice, it will be rescheduled to another CPU, leaving the previous alone. Thus, each processor is spending 50% of its time running, and another 50% waiting... assuming that we have only one runaway. From vances@REDACTED Wed Jul 27 09:34:30 2005 From: vances@REDACTED (Vance Shipley) Date: Wed, 27 Jul 2005 03:34:30 -0400 Subject: towards an Erlang IDE In-Reply-To: References: <20050722093656.GC623@feeble.motivity.ca> Message-ID: <20050727073429.GA81477@feeble.motivity.ca> On Sun, Jul 24, 2005 at 10:33:38AM +0200, Vlad Dumitrescu wrote: } } Eclipse has a nice graphical editor framework plugin, GEF, that could be } used for something like that. Once the Erlide environment is in place, I } was thinking about trying something similar (but I'd start with something } simpler, like the application monitor). I look forward to trying Erlide when it is ready. If I can get over Java. I think what I proposed is quite straight forward. The weak point is that it will, as my last effort was, be geared toward my particular coding style. } Regarding storing parse trees in the CVS, I think the disadvantage is just } as big as the problem it solves: you won't be able to easily compare/merge } different versions, unless using a special tool that will transform to text } before comparing. The abstract Erlang syntax trees are not a binary format, it's just a more structured textual term format. A source code control system should have no problem tracking changes. What I did have in mind though was automating the conversion to and from normal .erl source files. Ultimately the IDE would include an editor which handled all of this transparently. It would check out a syntax tree representing a module, convert it to a source file and place it into the editing window. -Vance From vlad.xx.dumitrescu@REDACTED Wed Jul 27 10:02:51 2005 From: vlad.xx.dumitrescu@REDACTED (Vlad Dumitrescu XX (LN/EAB)) Date: Wed, 27 Jul 2005 10:02:51 +0200 Subject: towards an Erlang IDE Message-ID: <11498CB7D3FCB54897058DE63BE3897C4BE598@esealmw105.eemea.ericsson.se> > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Vance Shipley > The abstract Erlang syntax trees are not a binary format, it's just a > more structured textual term format. A source code control system > should have no problem tracking changes. What I did have in > mind though > was automating the conversion to and from normal .erl source files. > Ultimately the IDE would include an editor which handled all of this > transparently. It would check out a syntax tree representing > a module, > convert it to a source file and place it into the editing window. Yes, I know. What I meant was that one would be locked on using that specific IDE. Anyone who isn't will only see hard-to-read-by-humans abstract code... regards, Vlad From bengt.kleberg@REDACTED Wed Jul 27 10:18:14 2005 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 27 Jul 2005 10:18:14 +0200 Subject: towards an Erlang IDE In-Reply-To: <11498CB7D3FCB54897058DE63BE3897C4BE598@esealmw105.eemea.ericsson.se> References: <11498CB7D3FCB54897058DE63BE3897C4BE598@esealmw105.eemea.ericsson.se> Message-ID: <42E74346.9010106@ericsson.com> On 2005-07-27 10:02, Vlad Dumitrescu XX (LN/EAB) wrote: >>-----Original Message----- >>From: owner-erlang-questions@REDACTED >>[mailto:owner-erlang-questions@REDACTED]On Behalf Of Vance Shipley ...deleted >>Ultimately the IDE would include an editor which handled all of this >>transparently. It would check out a syntax tree representing >>a module, >>convert it to a source file and place it into the editing window. > > > Yes, I know. What I meant was that one would be locked on using that specific IDE. Anyone who isn't will only see hard-to-read-by-humans abstract code... an alternative could be to have a standalone (erlang) tool that convert to/from syntax tree representation. for the benefit of all editors (or other uses). bengt From vlad.xx.dumitrescu@REDACTED Wed Jul 27 10:38:50 2005 From: vlad.xx.dumitrescu@REDACTED (Vlad Dumitrescu XX (LN/EAB)) Date: Wed, 27 Jul 2005 10:38:50 +0200 Subject: towards an Erlang IDE Message-ID: <11498CB7D3FCB54897058DE63BE3897C4BE599@esealmw105.eemea.ericsson.se> > an alternative could be to have a standalone (erlang) tool > that convert > to/from syntax tree representation. for the benefit of all > editors (or > other uses). Yes, that's a good idea. The next problem that comes up is which versioning systems to support - ClearCase, CVS, maybe Subversion? Hopefully it might be possible to use the same scripts for all of them. /Vlad From vances@REDACTED Wed Jul 27 10:24:03 2005 From: vances@REDACTED (Vance Shipley) Date: Wed, 27 Jul 2005 04:24:03 -0400 Subject: mnesia: inserting a large number of records In-Reply-To: <003701c58ec1$68b8eff0$dc00a8c0@INSWITCH244> References: <003701c58ec1$68b8eff0$dc00a8c0@INSWITCH244> Message-ID: <20050727082402.GB81477@feeble.motivity.ca> On Fri, Jul 22, 2005 at 10:29:36AM -0300, Sebastian Bello wrote: } } a programm reads records from a text file and inserts them in } a mnesia table. We are performing this insertions within a } transaction so in case of an error the whole file can be } reprocessed. The file holds approx. 5.000-10.000 records. } It seems the transaction time is not linear; I'm wondering if } there is a faster way to perform the insertions, maybe using } a table lock, I don't know. Any suggestions? Sebastian, I had a similiar challenge where we wanted to import large text files into a distributed mnesia database while it was in production. In our case we mostly needed to replace the existing copy so I came up with the following scheme: - create a new ram based table (e.g. foo_import) - use a write lock transaction fun with mnesia:ets/1 to insert records - use mnesia:change_table_copy_type/3 to change it to a disc based table on the local node only - activate a check point on this table table - backup this checkpoint using a custom mnesia_backup behaviour callback module to change the records on the fly to use the real table name (e.g. #foo_import{} to #foo{}). The idea is that you create the table in an ets context without lock overheads so that it is a fast operation (i.e. the user doesn't wait long) and then write it out to a binary backup file on disk. Now the user may use mnesia:restore/2 to replace the working table with the backup. You can do this while the system is running and transactions will block while it replaces the table. In our experience a couple seconds at worst. As I said we just replace the current table but you could just as easily insert the records into the existing table using the keep_tables option. I haven't tried this scheme so I can't say how it performs. For our purposes we changed the time it took to perform the import from many minutes, if not hours, to maybe twenty seconds. Aftet that as I said the table can be replaced in a couple seconds. -Vance From john-olof.bauner@REDACTED Wed Jul 27 11:07:45 2005 From: john-olof.bauner@REDACTED (John-Olof Bauner (KI/EAB)) Date: Wed, 27 Jul 2005 11:07:45 +0200 Subject: CPU/Hardware optimized for Erlang Message-ID: What I heard we still sell low end machines with APZ and high end with Alpha. Totgesagte leben l?nger as they say in Germany. J-O -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED]On Behalf Of Vance Shipley Sent: den 27 juli 2005 04:43 To: Richard A. O'Keefe Cc: erlang-questions@REDACTED Subject: Re: CPU/Hardware optimized for Erlang A little web research reveals that in June 2004 Ericsson announced it would be deploying AXE 810 switches in Macau using the APZ 212 33C processors. I assume the experiment with Alphas was a dead end, probably in no small part because they went away. Anyway it is nice to see that someone is still buying big iron. :) -Vance From nm@REDACTED Wed Jul 27 12:44:24 2005 From: nm@REDACTED (Gaspar Chilingarov) Date: Wed, 27 Jul 2005 15:44:24 +0500 Subject: question about supervisors Message-ID: <42E76588.7030703@web.am> Hi all! It's supposed if I get PID of process and it's supervisor, I can make assumption that even the process dies, supervisor will survive in any case and do not exit - so even if worker PID is invalid I always can ask supervisor for PID of new worker process? Thanks in advance -- Gaspar Chilingarov System Administrator t +37491 419763 (mob) t +37410 240399 (office) w www.web.am i 63174784 e nm@REDACTED From francesco@REDACTED Wed Jul 27 13:01:10 2005 From: francesco@REDACTED (Francesco Cesarini (Erlang Training & Consulting)) Date: Wed, 27 Jul 2005 12:01:10 +0100 Subject: question about supervisors In-Reply-To: <42E76588.7030703@web.am> References: <42E76588.7030703@web.am> Message-ID: <42E76976.7030404@erlang-consulting.com> Yes, assuming the supervisor itself does not crash and is restarted by its supervisor. If you only have one instance of that worker / supervisor, it is common to register them with a static name, avoiding the need of the pid. Francesco -- http://www.erlang-consulting.com Gaspar Chilingarov wrote: > Hi all! > > It's supposed if I get PID of process and it's supervisor, > I can make assumption that even the process dies, supervisor will > survive in any case and do not exit - so even if worker PID is invalid I > always can ask supervisor for PID of new worker process? > > Thanks in advance > From mickael.remond@REDACTED Wed Jul 27 13:24:21 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Wed, 27 Jul 2005 13:24:21 +0200 Subject: question about supervisors In-Reply-To: <42E76588.7030703@web.am> References: <42E76588.7030703@web.am> Message-ID: <42E76EE5.10700@erlang-fr.org> Gaspar Chilingarov wrote: > Hi all! > > It's supposed if I get PID of process and it's supervisor, > I can make assumption that even the process dies, supervisor will > survive in any case and do not exit - so even if worker PID is invalid I > always can ask supervisor for PID of new worker process? The behaviour of the supervisor depends of its setting. For example, strategies can define what to do when a process crash (Should the supervisor only restart this process ? all its supervised process, etc). In this "strategy", you can define the maximum restart frequency. It means that if the supervised processed maximum restart level in a given time has been reached, the supervised process will exit. So basically, the supervisor survives but, in some case, it might decide to terminate. You should pay attention to your supervisors setting and the organisation of the processus in the supervision tree. This is important to define the overall behaviour of your application in case of errors. For more information: http://www.erlang.se/doc/doc-5.3/doc/design_principles/sup_princ.html#strategy -- Micka?l R?mond http://www.3pblog.net/ From paris@REDACTED Wed Jul 27 13:33:32 2005 From: paris@REDACTED (Javier =?iso-8859-1?Q?Par=EDs_Fern=E1ndez?=) Date: Wed, 27 Jul 2005 13:33:32 +0200 Subject: TCP stack throughput In-Reply-To: <200507270620.j6R6KBIm019207@snookles.snookles.com> References: <200507270620.j6R6KBIm019207@snookles.snookles.com> Message-ID: <20050727113332.GA30306@dc.fi.udc.es> On Wed, Jul 27, 2005 at 01:20:11AM -0500, Scott Lystig Fritchie wrote: > I'm sorry, I don't have Javier's original message in my mailbox. > > >>>>> "uw" == Ulf Wiger writes: > > uw> Den 2005-07-19 19:39:12 skrev Javier Par?s Fern?ndez > uw> : > >> I observed several funny things. The time spent processing the > >> connections seems to go up exponentially, but it is not limited by > >> cpu, which was always under 50%. I don't know what is throttling > >> it. > > Are you using Linux, and if yes, which Linux distribution are you > using? I was very surprised to discover that the "top" program > distributed with Red Hat Advanced Server 3.0, when running on an Intel > CPU with Hyper-Threading enabled(*), will report that: > > main() > { > int i; > > for (;;) { > i++; > } > } > > ... will only consume 50% CPU time. Well, in an HT CPU this is not true, but in a true SMP system it could be seen as correct. After all, half the CPU capacity of the system is not used. By the way, the system is Debian Linux, and the cpu has HT, but a loop similar to the one you put above shows 100% CPU usage in top. Regards, Javier. From nm@REDACTED Wed Jul 27 13:35:14 2005 From: nm@REDACTED (Gaspar Chilingarov) Date: Wed, 27 Jul 2005 16:35:14 +0500 Subject: question about supervisors In-Reply-To: <42E76976.7030404@erlang-consulting.com> References: <42E76588.7030703@web.am> <42E76976.7030404@erlang-consulting.com> Message-ID: <42E77172.8070709@web.am> Francesco Cesarini (Erlang Training & Consulting) wrote: > Yes, assuming the supervisor itself does not crash and is restarted by > its supervisor. > yeah documentation says the following: --------- ........If more than MaxR number of restarts occur in the last MaxT seconds, then the supervisor terminates all the child processes and then itself. When the supervisor terminates, then the next higher level supervisor takes some action. It either restarts the terminated supervisor, or terminates itself. ....... ---------- so , usually I can think that things gone very bad, if supervisor died and I can terminate another processes too. > If you only have one instance of that worker / supervisor, it is common > to register them with a static name, avoiding the need of the pid. > > Francesco It was discussed a few time ago - in a "Meyer, OO and concurrency" topic I'm repeating this question just to clarify this issue :) --quote-- With functional programming you supply all the data required to a function, instead of global variables you use arguments. The same applies here, start the clients with the pid of the server. I generally use registered processes if the client and server are in separate applications. Within the same application it is usually easy enough to initialize the client properly so that it knows where it's server is. Another trick I use is to inspect the supervisors. --/quote-- I can see one possible quite useful aspect of applying such technique -- writing test for such code should be easier, that registering bunch of processes which proper names. Another Question: how do supervisor handles large number of processes ? I have about 2^13 processes, which sometimes - on timeouts or on data arrival do processing and time to time save their state to dets. Also I have manager which maps FooID* to worker PID, but does not restart workers - it's ok for worker to die or lose some little amount of data, if no alive worker for some FooID found, worker is started, it reads state from dets and continues to process arriving data. Usually I have at least one message per minute for each worker process. In case if manager process dies (for some reason) -- which is the better way ? kill all worker processes and let them spawn again? or each worker process should understand, that manager died, find restarted manager and register as worker for FooID -- because it re FooID -- some key to identify data :) -- Gaspar Chilingarov System Administrator t +37491 419763 (mob) t +37410 240399 (office) w www.web.am i 63174784 e nm@REDACTED From thomasl_erlang@REDACTED Wed Jul 27 13:41:21 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 27 Jul 2005 04:41:21 -0700 (PDT) Subject: CPU/Hardware optimized for Erlang In-Reply-To: Message-ID: <20050727114121.82909.qmail@web34408.mail.mud.yahoo.com> --- James Hague wrote: > Thomas Lindgren wrote: > >(And why would the erlang chip have comparable > >performance at lower power?) > > I wasn't talking about a 3GHz Erlang processor using > less power than, > say, a 3GHz Pentium 4 (though I suspect it would). > I was musing about > an Erlang processor that gives performance roughly > equivalent to the > BEAM *emulator* running on a 3GHz Pentium 4. Sure, that's what I was thinking about too. > Using the old ECOMP > numbers, I'm guessing that this could be done with > an Erlang processor > running in the ballpark of 100MHz. Well, ECOMP had 30x speedup compared to JAM, not BEAM. On similar programs at that time, HIPE had a 10-20x speedup over JAM (when compiling JAM bytecodes to Sparc asm), if memory serves. So, assuming no further compiler improvements, ECOMP would have to remain within a factor of 3 or so of the desktop systems, to be competitive in speed. At that time, power wasn't really on the map, but presumably a similar comparison could be made by porting BEAM or HIPE to a suitable incumbent. > To a lot of programmers, a super micro CPU is simply > a platform to > write interpreted Python, Perl, REBOL, etc. Or > Erlang. (Even 'very dynamic' languages can be compiled efficiently on stock hardware; cf. Self or CMUCL. I have recently seen a python analyzer based on Self-like class analysis too.) > Cutting out > the middleman and designing the hardware to execute > the language > directly is certainly going to be more efficient. Maybe I'm a pessimist, but why would it be so? That is, from all I have seen of Lisp Machines, dataflow machines, 5th gen Prolog machines, Smalltalk machines (SOAR, or perhaps the "Recursiv", anyone remember that one?), Java chips -- none of them has been competitive. The Lisp Machines came closest, doing very well indeed for a while, but Lucid showed that Lisp on stock hardware could do equally well. (And maybe there is a niche for Java chips -- we will see.) As someone whose name I can't remember liked to ask in situations like this, "what has changed?" Why is it going to be different this time? Designing out the middleman sounds like "closing the semantic gap", which sets my hackles on end :-) Closing the semantic gap was about providing high-level machine instructions so that programmers could express their intentions more clearly in their asm programs. RISC trounced the "semantic gap" theory so hard it wasn't even funny. (Well, the funny part was hearing sonorous pronouncements in the class room one year about how we should overcome the semantic gap, and then seeing all that rapidly reduced to dust :-) The particular point made at that time was that an optimizing compiler could do at least as well as fancy high-level instructions. (And usually far better.) So, I seriously doubt the viability of rewriting some emulator or interpreter in silicon, which is the closest I can think of "executing the language directly". Every few years, someone comes up with a new BEAM but you will then be stuck with JAM. At each point, you would instead do well to ask whether the feature can be emulated with a "regular" instruction sequence, and what you gain compared to that option. (Those who like Prolog can compare the WAM to the BAM as an example.) > This is different > than the long ago debates on the subject, in which > people (Wirth, et > al) would compare writing a *compiler* for an off > the shelf CPU vs. > designing a custom processor. In this case we're > talking about looser > dynamic languages. I assume Wirth et al were comparing payoff for effort? E.g., is it more fruitful to just write a better compiler for the money you will spend on building the hardware, etc? An engineering trade off. When offering a new system, you would on the other hand have to compare your system with the best the competition can offer, which would at a guess be a suitable port of OTP. The hardware design would have to plan for this, of course. > On a related note, you could design an interesting > cache architecture > for a language without destructive updates. What would that be? Best, Thomas ____________________________________________________ Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs From serge@REDACTED Wed Jul 27 17:28:53 2005 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 27 Jul 2005 11:28:53 -0400 Subject: tuple comparison Message-ID: <42E7A835.6030101@hq.idt.net> I have a couple of questions that are not very obvious from documentation. 1. When we compare two tuples of equal sizes, T1 = {I1, I2, ..., In}, T2 = {J1, J2, ..., Jn}, if T1 > T2 then is there a guarantee that the following always holds true: (I1 >= J1) and (I2 >= J2) and ... (In >= Jn) 2. What's the meaning of such a result: 53> {1, 2, 3} > 10. true I would think that this comparison should throw an exception. Serge -- Serge Aleynikov R&D Telecom, IDT Corp. Tel: (973) 438-3436 Fax: (973) 438-1464 serge@REDACTED From thomasl_erlang@REDACTED Wed Jul 27 18:02:21 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 27 Jul 2005 09:02:21 -0700 (PDT) Subject: tuple comparison In-Reply-To: <42E7A835.6030101@hq.idt.net> Message-ID: <20050727160221.40458.qmail@web34408.mail.mud.yahoo.com> --- Serge Aleynikov wrote: > I have a couple of questions that are not very > obvious from documentation. > > 1. When we compare two tuples of equal sizes, > > T1 = {I1, I2, ..., In}, > T2 = {J1, J2, ..., Jn}, > > if T1 > T2 then > > is there a guarantee that the following always holds > true: > > (I1 >= J1) and (I2 >= J2) and ... (In >= Jn) No, for instance {2,1} > {1,2} and {1,1,2,1} > {1,1,1,2}. See below for more. > 2. What's the meaning of such a result: > > 53> {1, 2, 3} > 10. > true > > I would think that this comparison should throw an > exception. Comparisons operate over all terms. Section 2.5.6 in the Erlang book gives an informal ordering of the terms. It says "tuples are ordered first by their size, then by their elements". Okay. The Erlang 4.7 spec basically says comparisons reflect term order, and 'term order' is lexicographic over tuples (p.45, 4.11.2). Best, Thomas ____________________________________________________ Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs From james.hague@REDACTED Wed Jul 27 18:07:57 2005 From: james.hague@REDACTED (James Hague) Date: Wed, 27 Jul 2005 11:07:57 -0500 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <20050727114121.82909.qmail@web34408.mail.mud.yahoo.com> References: <20050727114121.82909.qmail@web34408.mail.mud.yahoo.com> Message-ID: Thomas Lindgren wrote: >Well, ECOMP had 30x speedup compared to JAM, not BEAM. >On similar programs at that time, HIPE had a 10-20x >speedup over JAM (when compiling JAM bytecodes to >Sparc asm), if memory serves. So, assuming no further >compiler improvements, ECOMP would have to remain >within a factor of 3 or so of the desktop systems, to >be competitive in speed. Good points. I had forgotten about the ECOMP numbers being relative to JAM. >As someone whose name I can't remember liked to ask in >situations like this, "what has changed?" Why is it >going to be different this time? Three things: First, CPU designers are running up against physical issues in terms of simply bumping up clock speed, most notably power dissipation. For a long while it was a given that each generation of a processor would be significantly faster than the last, but this has tapered off dramatically. High-end processors keep getting faster, yes, but now power consumption and heat are growing at prohibitive rates. So, optimistically, maybe one alternative is to get some instructions doing a bit more, rather than having to break things down into a stream of simpler instructions. I'm not talking about VAX-style CISC, however. Along the same lines, notebook computers have taken off, so lower power consumption is a big issue. Sure, you're not going to be running telecom apps and servers from a notebook, but it's an interesting trend. Third, FPGA technology has gotten advanced and commonplace. There are people on the web building working CPUs--often with very specific purposes in mind--as a hobby. This doesn't solve fundamental issues, but it sure makes prototyping and refining a design easier. James From ulf@REDACTED Thu Jul 28 01:20:44 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 28 Jul 2005 01:20:44 +0200 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <20050727114121.82909.qmail@web34408.mail.mud.yahoo.com> References: <20050727114121.82909.qmail@web34408.mail.mud.yahoo.com> Message-ID: Den 2005-07-27 13:41:21 skrev Thomas Lindgren : > Well, ECOMP had 30x speedup compared to JAM, not BEAM. > On similar programs at that time, HIPE had a 10-20x > speedup over JAM (when compiling JAM bytecodes to > Sparc asm), if memory serves. So, assuming no further > compiler improvements, ECOMP would have to remain > within a factor of 3 or so of the desktop systems, to > be competitive in speed. AFAIK, in 2000, AXD 301 was using BEAM (we never did use JAM in the final product), and the most relevant benchmark was compiling a call control prototype to ECOMP and comparing it to the same code running on the AXD 301 target platform with 450 MHz UltraSPARC 2i processors with Solaris and BEAM. > At that time, power wasn't really on the map, but > presumably a similar comparison could be made by > porting BEAM or HIPE to a suitable incumbent. For us, power was very much on the map. That was one reason why SPARC could be competitive to Intel at the time. We had a 20W power budget for the CPU board, if memory serves. > So, I seriously doubt the viability of rewriting some > emulator or interpreter in silicon, which is the > closest I can think of "executing the language > directly". Every few years, someone comes up with a > new BEAM but you will then be stuck with JAM. But this is not what ECOMP did. The ECOMP compiler prototype - written by Peter Lundell on his spare time, while working as our System Project Manager - hooked into the OTP compiler framework, about at the same level as beam_asm, I think. As such, it ought to have been fairly future-proof. ECOMP had an instruction set modeled after the Erlang language -- not any specific virtual machine. It was designed to handle garbage-collected languages (with sub-instructions for reference counting) without global variables (a deep register file). On average, one line of Erlang would correspond to 3 ECOMP instructions. Most of the BIFs needed for the prototype were actually implemented in plain Erlang, and not optimized at all. Some things, like ETS, were hand-written in ECOMP assembly, but ETS ended up being fairly simple tuple trees. /Uffe -- Ulf Wiger From ok@REDACTED Thu Jul 28 04:19:54 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 28 Jul 2005 14:19:54 +1200 (NZST) Subject: CPU/Hardware optimized for Erlang Message-ID: <200507280219.j6S2JsIW276916@atlas.otago.ac.nz> Thomas Lindgren wrote about ECOMP: At that time, power wasn't really on the map, but presumably a similar comparison could be made by porting BEAM or HIPE to a suitable incumbent. On the contrary, the ECOMP slides mention low power as an issue more than once. (And maybe there is a niche for Java chips -- we will see.) Note that the ARM/Jazelle stuff isn't *exactly* a Java chip. The ARM/Jazelle machines support *three* instruction sets: ARM a mildly unconventional architecture that's recognisably a RISC Thumb a compressed version of ARM (16-bit instructions instead of 32-bit instructions, but basically only instruction fetch and decode are different) JVM most of the JVM instructions are executed natively, holding up to 4 top of stack items in ARM registers. Some of the more complicated JVM instructions trap out to ARM code. So ARM/Jazelle machines can run Java pretty much directly, quite a bit faster than any JVM emulator for ARM, AND at lower CPU power and in less instruction memory than any possible native code translation. As someone whose name I can't remember liked to ask in situations like this, "what has changed?" Why is it going to be different this time? What has changed for ARM at any rate is "Mobile phones". When Lisp machines walked the earth (and I still have fond memories of the Xerox Lisp machines) there weren't any mobile phones. Now there are mobile phones with WAP and pxting and games and there are people wanting to run Java on them. So there is now a large market for machines that can - run Java - in a fixed amount of memory that isn't as large as one might like - with a fixed amount of battern power that is never large enough Now the ECOMP work was about designing a *core* so that you could easily drop an "Erlang machine" into any ASIC. That was a plausible niche for an Erlang processor: the area HAD to be small and the memory HAD to be small and the processor's share of the power budget couldn't be too large. What was missing was a large enough market for Erlang-controlled ASICs. Designing out the middleman sounds like "closing the semantic gap", which sets my hackles on end :-) Closing the semantic gap was about providing high-level machine instructions so that programmers could express their intentions more clearly in their asm programs. RISC trounced the "semantic gap" theory so hard it wasn't even funny. (Well, the funny part was hearing sonorous pronouncements in the class room one year about how we should overcome the semantic gap, and then seeing all that rapidly reduced to dust :-) But something important changed. Memory technology changed. In the era of "small semantic gap" machines like the B6700, memory was small and exceeding expensive. When large semicondutor memories came along, suddenly nobody *minded* all that much of the number of bytes needed for executable code tripled. Of course, these days, as we have been so eloquently reminded in this thread, RISC has been "trounced" by the well known BISC (Bizarre ...) the Pentium, and the Itanium 2 architecture that HP and Intel dreamed up is seriously weird. The particular point made at that time was that an optimizing compiler could do at least as well as fancy high-level instructions. (And usually far better.) But this too depended on the change from ferrite core memory to semiconductor memory and the general improvement in CPU speed: on slow machines with small memories you couldn't _afford_ fancy compilers. The old machines with instructions tuned to COBOL, for example. 15 kilowords was a large memory! A quick skim through beam_emu.c (is the BEAM instruction set actually documented anywhere?) suggests to me that it could be emulated at least twice as fast on stock hardware, or if not BEAM, then something very like it. I don't see any great need for an Erlang machine to replace the current Erlang system for what the current Erlang system does well. But technology does change, and there are new kinds of computing coming. I've recently been reading about electrochemical transistors. They are large and slow, but it looks to me as though in a couple of years you'd be able to print an ECOMP-like machine on a piece of paper the size of a paper-back book page for cents or possibly fractional cents. It would NOT be fast. On the other hand, EC stuff includes electrochomic (clear/dark blue) and light sensor stuff. Yep, we're about to enter the Age of Intelligent Wallpaper! From ok@REDACTED Thu Jul 28 04:26:49 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 28 Jul 2005 14:26:49 +1200 (NZST) Subject: tuple comparison Message-ID: <200507280226.j6S2QnuF263542@atlas.otago.ac.nz> 2. What's the meaning of such a result: 53> {1, 2, 3} > 10. true It's defined in the book. (Section 2.5.6.) For the purpose of comparison the following ordering is defined: number < atom < reference < port < pid < tuple < list Tuples are ordered first by their size then by their elements. Lists are ordered by comparing heads, then tails. I would think that this comparison should throw an exception. It is *AMAZINGLY* useful to have a total order on terms. It means you can sort any collection of terms. It means that ordering is well defined in ets/dets/mnesia. From nm@REDACTED Thu Jul 28 08:20:55 2005 From: nm@REDACTED (Gaspar Chilingarov) Date: Thu, 28 Jul 2005 11:20:55 +0500 Subject: large amount of processes - how to organize workers/manager processes ? Message-ID: <42E87947.9090607@web.am> Hi all! Another Question: how do supervisor handles large number of processes ? I have about 8000-20000 processes, which sometimes - on timeouts or on data arrival do processing and time to time save their state to dets. Also I have manager which maps FooID* to worker PID, but does not restart workers - it's ok for worker to die or lose some little amount of data, if no alive worker for some FooID found, worker is started by manager, it reads state from dets and continues to process arriving data. Usually I have at least one message per minute for each worker process. In case if manager process dies (for some reason) -- which is the better way ? kill all worker processes and let them spawn again? or each worker process should somehow understand, that manager died, find restarted manager and register as worker for FooID -- because worker is totally independent from manager, when it processes data. FooID -- some key to identify to whom data block should be sent. -- Gaspar Chilingarov System Administrator t +37491 419763 (mob) t +37410 240399 (office) w www.web.am i 63174784 e nm@REDACTED From thomasl_erlang@REDACTED Thu Jul 28 11:37:14 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 28 Jul 2005 02:37:14 -0700 (PDT) Subject: CPU/Hardware optimized for Erlang In-Reply-To: Message-ID: <20050728093714.42922.qmail@web34411.mail.mud.yahoo.com> --- Ulf Wiger wrote: > Den 2005-07-27 13:41:21 skrev Thomas Lindgren > : > > > Well, ECOMP had 30x speedup compared to JAM, not > BEAM. > > > On similar programs at that time, HIPE had a > 10-20x > > speedup over JAM (when compiling JAM bytecodes to > > Sparc asm), if memory serves. So, assuming no > further > > compiler improvements, ECOMP would have to remain > > within a factor of 3 or so of the desktop systems, > to > > be competitive in speed. > > AFAIK, in 2000, AXD 301 was using BEAM (we never did > use > JAM in the final product), and the most relevant > benchmark was compiling a call control prototype to > ECOMP and comparing it to the same code running on > the AXD 301 target platform with 450 MHz UltraSPARC > 2i > processors with Solaris and BEAM. I was thinking of a predecessor system that Robert Tjarnstrom reported about a couple of years before that I believe. At that point, the projected performance was 30x speedup over JAM unless I entirely misremember. Slide 14 in the ECOMP presentation is a bit unclear: did you get 30x speedup on the call control software vs the BEAM/UltraSparc setup above? Or was this on other benchmarks vs the BEAM/UltraSparc? Or vs some other baseline? Also, does the line "measured per use of clock cycles" on slide 14 mean that the 30x speedup was 1/30 the number of clock cycles were used running on ECOMP vs the UltraSparc for the benchmark(s)? That aside, and most importantly: did you decide to go ahead? > > So, I seriously doubt the viability of rewriting > some > > emulator or interpreter in silicon, which is the > > closest I can think of "executing the language > > directly". Every few years, someone comes up with > a > > new BEAM but you will then be stuck with JAM. > > But this is not what ECOMP did. The ECOMP compiler > prototype - written by Peter Lundell on his spare > time, while working as our System Project Manager Well done. > - hooked into the OTP compiler framework, about at > the same level as beam_asm, I think. As such, it > ought to have been fairly future-proof. Let me expand my comment a bit, then: if you model your instruction set too closely on the language (which was what I was arguing against), you will not be able to take advantage of future improvements in how things are represented and done when implementing the language. In effect, by dictating the policy in hardware, you are discounting the effects of improvements in compiler- and runtime system intelligence (to a greater or lesser extent, depending on your choices). Thus, as an analogy (only): if you implement your hardware based on, say, the JAM instruction set, subsequent radical innovations such as BEAM are more difficult (at the very least) to exploit. Hence my diatribe against too high-level hardware. Best, Thomas __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From thomasl_erlang@REDACTED Thu Jul 28 12:39:20 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 28 Jul 2005 03:39:20 -0700 (PDT) Subject: CPU/Hardware optimized for Erlang In-Reply-To: <200507280219.j6S2JsIW276916@atlas.otago.ac.nz> Message-ID: <20050728103920.55359.qmail@web34408.mail.mud.yahoo.com> --- "Richard A. O'Keefe" wrote: > Thomas Lindgren wrote > > On the contrary, the ECOMP slides mention low power > as an issue more than once. Well, I was thinking of a somewhat earlier event, and at that time, power wasn't a big argument yet. See my reply to Ulf. My apologies to all. > Note that the ARM/Jazelle stuff isn't *exactly* a > Java chip. A clever way of doing it, I'll say. Having competitive ARM/Thumb support in the first place is likely vital in this market, though. > >["what has changed?"] > > What has changed for ARM at any rate is "Mobile > phones". Yes. A new market where previous offerings didn't fit, followed by a snowball effect. > Now the ECOMP work was about designing a *core* so > that you could easily > drop an "Erlang machine" into any ASIC. That was a > plausible niche for > an Erlang processor: the area HAD to be small and > the memory HAD to be > small and the processor's share of the power budget > couldn't be too large. > What was missing was a large enough market for > Erlang-controlled ASICs. In principle you could build an SOC with one or more Erlang processors on it. Cell with ECOMPs instead of vector processors ... Wouldn't that be neat, hardware lovers? :-) [Previous caveats still applied. But IBM would build it if you paid them.] > Of course, these days, > as we have been > so eloquently reminded in this thread, RISC has been > "trounced" by > the well known BISC (Bizarre ...) the Pentium, I'd say the first shudder was when Yale Patt et al showed that one could translate VAX instructions to RISC on the fly. ("Runtime generation of HPS micro instructions from a VAX instruction stream", MICRO-19, 1986.) And then Robert Colwell and Intel's P6 showed that it also could be done well in practice ten years later or so. CISC was back in the game (reincarnated as a superscalar RISC with a somewhat complex frontend ...). > and the Itanium 2 > architecture that HP and Intel dreamed up is > seriously weird. I actually liked the EPIC principles, since they addressed many of the issues I wanted to get at when exploiting ILP in a compiler ... though alas, it seems Intel/HP also made the end product too complex. And to be honest it doesn't seem to be quite competitive with x86 either :-) The original intention was, I believe, to have a simple decoder and a simple in-order implementation, which would lead to fast clocking and high performance. The compiler would take care of the nasty stuff. In practice, things have been different. Partly due to people now being _really_good_ at building x86:s but not EPICs. In the same vein, people aren't as good at building EPIC compilers either (and some problems, such as memory latency, are difficult for a compiler in the first place). >> [compiler technology made the difference] > > But this too depended on the change from ferrite > core memory to semiconductor > memory and the general improvement in CPU speed: on > slow machines with > small memories you couldn't _afford_ fancy > compilers. The old machines with > instructions tuned to COBOL, for example. 15 > kilowords was a large memory! A natural but painful mistake to make. (Speaking from the safety of 20/20 hindsight, I haste to add.) The general theme here probably is underestimating Moore's law. (Or failing to dominate the world in the timespan given you by Moore's law.) > But technology > does change, and there are new kinds of computing > coming. Indeed. And I am, of course, fully willing to eat my words at a future date :-) Best, Thomas ____________________________________________________ Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs From ulf@REDACTED Thu Jul 28 12:46:58 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 28 Jul 2005 12:46:58 +0200 Subject: CPU/Hardware optimized for Erlang In-Reply-To: <20050728093714.42922.qmail@web34411.mail.mud.yahoo.com> References: <20050728093714.42922.qmail@web34411.mail.mud.yahoo.com> Message-ID: Den 2005-07-28 11:37:14 skrev Thomas Lindgren : > Slide 14 in the ECOMP presentation is a bit unclear: > did you get 30x speedup on the call control software > vs the BEAM/UltraSparc setup above? Or was this on > other benchmarks vs the BEAM/UltraSparc? Or vs some > other baseline? I believe that the result of the prototype was that we could match our 450 MHz Ultra with a <20 MHz FPGA, and that a 200 MHz ECOMP ASIC would: - fit nicely into the corner of our switch port chip - run circles around the UltraSPARC - use about 1/10 of the power > Also, does the line "measured per use of clock cycles" > on slide 14 mean that the 30x speedup was 1/30 the > number of clock cycles were used running on ECOMP vs > the UltraSparc for the benchmark(s)? Yes, the prototype was run in a VHDL simulator and an ECOMP emulator written in Erlang. To compare performance, clock cycles were counted. While the Ultra is capable of executing up to 4 instructions per clock cycle, we got slightly less than 1 instruction per cycle on our erlang code. I don't know how wait states in ECOMP were accounted for, but one point of using a threaded architecture was that ECOMP would be able to switch to another process while waiting for data (or whatever), thus reducing the number of actual wait states in the processor. > That aside, and most importantly: did you decide to go > ahead? No. The main reason was that it would have required a rather extensive redesign of our software at the time (not the move to ECOMP in itself, but moving call control farther out towards the interfaces, rather than having a central processor handling several interfaces. Nowadays, we do more or less this (one call control cpu per interface), but this transition took a long time, and other events (most importantly, Robert T. was reassigned) meanwhile caused the ECOMP project to go into hibernation. /Uffe -- Ulf Wiger From thomasl_erlang@REDACTED Thu Jul 28 12:55:55 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 28 Jul 2005 03:55:55 -0700 (PDT) Subject: CPU/Hardware optimized for Erlang In-Reply-To: Message-ID: <20050728105556.12437.qmail@web34406.mail.mud.yahoo.com> --- James Hague wrote: > Thomas Lindgren wrote: > Good points. I had forgotten about the ECOMP > numbers being relative to JAM. Hrrm, yes ... Well, er :-) > >As someone whose name I can't remember liked to ask > in > >situations like this, "what has changed?" Why is it > >going to be different this time? > > Three things: > [no clock speed increase => more work/instr good] > [power concerns] How should the complex instructions be implemented? Longer pipelines? Slower clock + more work per stage? More internal parallelism? I'd say the main use for a complex instruction is that it is a compact representation of "some work", in particular in memory/cache. While actually executing, resource requirements for any given instruction can vary widely. (Cue that infamous VAX instruction.) > [FPGAs] FPGAs are very interesting, definitely. I think the manufacturers now offer simple migration paths from FPGA to ASIC too, if you find you have a winner. Best, Thomas ____________________________________________________ Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs From tim@REDACTED Thu Jul 28 13:08:29 2005 From: tim@REDACTED (Tim Bates) Date: Thu, 28 Jul 2005 20:38:29 +0930 Subject: Directory structure for non-OTP projects In-Reply-To: <7a6679dcd096e0fd9a29686d8ad1b5cf@t-mobile.co.uk> References: <42DCE32F.9090604@bates.id.au> <42DCEF92.7080408@hyber.org> <42DF6BED.3000202@bates.id.au> <7a6679dcd096e0fd9a29686d8ad1b5cf@t-mobile.co.uk> Message-ID: <42E8BCAD.10803@bates.id.au> Chandrashekhar Mullaparthi wrote: > We have our own homegrown build tool which exports apps from CVS, does > some tag checks, builds the beam files and then the boot scripts. It is > not ideal but it kind of does the job. You might want to look at the > builder app in jungerl. I was a beta tester for it and it is quite good. > The only reason we never adopted it for our work is inertia. Thanks Chandru, builder looks promising but I'm having difficulty working out how to use it. I'm currently trying to roll my own primitive development environment, and I'm having difficulties with the OTP test_server. OTP seems to start a new Erlang node to run the tests in, and the new node doesn't inherit the environment of the old one (specifically, the -pz option I specified on the command line). I've also had to modify the test_server code to look in the right places for my test suites (Spec/test rather than Spec_test) because the test_server as shipped doesn't seem to be compatible with the sample test suites it ships with. I want to keep my system install of Erlang separate from my development environment, but maybe I'm approaching things the wrong way. Do people normally have a separate install of Erlang for each project, with only the modules they need? Do people normally go through some variant of the release process described in the OTP Design Principles every time they change a line of code? These seem a bit heavyweight, but I can't see how else to do it. Can someone enlighten me? Thanks, Tim. -- Tim Bates tim@REDACTED From chandrashekhar.mullaparthi@REDACTED Thu Jul 28 15:16:06 2005 From: chandrashekhar.mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Thu, 28 Jul 2005 14:16:06 +0100 Subject: Directory structure for non-OTP projects In-Reply-To: <42E8BCAD.10803@bates.id.au> References: <42DCE32F.9090604@bates.id.au> <42DCEF92.7080408@hyber.org> <42DF6BED.3000202@bates.id.au> <7a6679dcd096e0fd9a29686d8ad1b5cf@t-mobile.co.uk> <42E8BCAD.10803@bates.id.au> Message-ID: On 28 Jul 2005, at 12:08, Tim Bates wrote: > > I want to keep my system install of Erlang separate from my development > environment, but maybe I'm approaching things the wrong way. Do people > normally have a separate install of Erlang for each project, with only > the modules they need? Yes, that is usually how it is. Each erlang node we deploy has everything it needs to run under one top level directory. > Do people normally go through some variant of the > release process described in the OTP Design Principles every time they > change a line of code? No - not really. This is what we do. - If some modules are changed, we put them in the right paths and load the new modules into the node. - If we need to add new applications or the version number of an application has changed (for e.g because of backward incompatible changes or because we added new modules) we go through the release process. But even in this case, we don't write .appup and .relup files. We just build a new boot file, copy the boot file and the new stuff into appropriate places and either wait for the next node restart or we do a controlled restart. We maintain uptime by diverting traffic to standby nodes. - If we have to upgrade the ERTS and/or some OTP libraries, we build a whole new release and install it in place of the old one and restart the node. I know of quite a few people you do not use much of the OTP framework and instead have their own way of starting up nodes. But that is only really recommended once you have attained 'guru' status :-) I hope this helps. cheers Chandru From serge@REDACTED Thu Jul 28 18:47:55 2005 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 28 Jul 2005 12:47:55 -0400 Subject: SASL - duplicate alarms bug Message-ID: <42E90C3B.3060207@hq.idt.net> Is there a reason that the SASL's alarm handler doesn't check for duplicate alarms before adding new ones to the list? If you do the following on the system with low disk space / free memory: 1. Start SASL 2. Start OS_MON 3. Stop OS_MON 4. Start OS_MON you'll get duplicate alarms that are not easily clearled (you'd need to call alarm_handler:clear_alarm() multiple times for the same AlarmID... 17> alarm_handler:get_alarms(). [{system_memory_high_watermark,[]}, {{disk_almost_full,"/mnt/cdrom"},[]}, {{disk_almost_full,"/usr"},[]}, {system_memory_high_watermark,[]}, {{disk_almost_full,"/mnt/cdrom"},[]}, {{disk_almost_full,"/usr"},[]}] 18> alarm_handler:clear_alarm(system_memory_high_watermark). [{{disk_almost_full,"/mnt/cdrom"},[]}, {{disk_almost_full,"/usr"},[]}, {system_memory_high_watermark,[]}, {{disk_almost_full,"/mnt/cdrom"},[]}, {{disk_almost_full,"/usr"},[]}] This looks like a bug to me. Serge From chas@REDACTED Thu Jul 28 19:49:51 2005 From: chas@REDACTED (Charles Blair) Date: Thu, 28 Jul 2005 12:49:51 -0500 Subject: list comprehension question Message-ID: <20050728174951.GB24291@jump.lib.uchicago.edu> in something like this: [X || X <- lists:seq(1,25)] is the result guaranteed to reflect the order of the generator, or not? i'm working with a list of randomly generated results, and i want to print them in the order in which they were generated--is lists:foreach the way to go here? a list comprehension would be simpler to write and nicer to look at. thanks. From hakan.stenholm@REDACTED Thu Jul 28 21:51:33 2005 From: hakan.stenholm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Stenholm?=) Date: Thu, 28 Jul 2005 21:51:33 +0200 Subject: list comprehension question In-Reply-To: <20050728174951.GB24291@jump.lib.uchicago.edu> References: <20050728174951.GB24291@jump.lib.uchicago.edu> Message-ID: <42E93745.5050600@mbox304.swipnet.se> Charles Blair wrote: >in something like this: > >[X || X <- lists:seq(1,25)] > >is the result guaranteed to reflect the order of the generator > You'll get [1,2,3,4, ... , 25] as expected (I at least have never seen anything else), but I can't see that the documentation guarantees this in any formal way - except that all examples act in this way and that all listcomprehension code I have ever seen appears to assume this. >, or >not? > >i'm working with a list of randomly generated results, and i want to >print them in the order in which they were generated--is lists:foreach >the way to go here? a list comprehension would be simpler to write and >nicer to look at. > >thanks. > > > > > > From sebastian@REDACTED Thu Jul 28 22:51:29 2005 From: sebastian@REDACTED (Sebastian Bello) Date: Thu, 28 Jul 2005 17:51:29 -0300 Subject: mnesia: inserting a large number of records References: <003701c58ec1$68b8eff0$dc00a8c0@INSWITCH244> <20050727082402.GB81477@feeble.motivity.ca> Message-ID: <0e7901c593b6$23478db0$dc00a8c0@INSWITCH244> Vance, thank you very much for your response and your detailed description. The solution looks a bit complicated to me, but I'll try some tests. Thanks! Sebastian- ----- Original Message ----- From: "Vance Shipley" To: "Sebastian Bello" Cc: Sent: Wednesday, July 27, 2005 5:24 AM Subject: Re: mnesia: inserting a large number of records > On Fri, Jul 22, 2005 at 10:29:36AM -0300, Sebastian Bello wrote: > } > } a programm reads records from a text file and inserts them in > } a mnesia table. We are performing this insertions within a > } transaction so in case of an error the whole file can be > } reprocessed. The file holds approx. 5.000-10.000 records. > } It seems the transaction time is not linear; I'm wondering if > } there is a faster way to perform the insertions, maybe using > } a table lock, I don't know. Any suggestions? > > Sebastian, > > I had a similiar challenge where we wanted to import large text > files into a distributed mnesia database while it was in production. > In our case we mostly needed to replace the existing copy so I > came up with the following scheme: > > - create a new ram based table (e.g. foo_import) > - use a write lock transaction fun with mnesia:ets/1 to > insert records > - use mnesia:change_table_copy_type/3 to change it to a > disc based table on the local node only > - activate a check point on this table table > - backup this checkpoint using a custom mnesia_backup > behaviour callback module to change the records on > the fly to use the real table name (e.g. #foo_import{} > to #foo{}). > > The idea is that you create the table in an ets context without > lock overheads so that it is a fast operation (i.e. the user doesn't > wait long) and then write it out to a binary backup file on disk. > > Now the user may use mnesia:restore/2 to replace the working > table with the backup. You can do this while the system is running > and transactions will block while it replaces the table. In our > experience a couple seconds at worst. As I said we just replace > the current table but you could just as easily insert the records > into the existing table using the keep_tables option. I haven't > tried this scheme so I can't say how it performs. For our purposes > we changed the time it took to perform the import from many minutes, > if not hours, to maybe twenty seconds. Aftet that as I said the > table can be replaced in a couple seconds. > > -Vance > > From ulf@REDACTED Thu Jul 28 22:56:38 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 28 Jul 2005 22:56:38 +0200 Subject: list comprehension question In-Reply-To: <20050728174951.GB24291@jump.lib.uchicago.edu> References: <20050728174951.GB24291@jump.lib.uchicago.edu> Message-ID: Den 2005-07-28 19:49:51 skrev Charles Blair : > in something like this: > > [X || X <- lists:seq(1,25)] > > is the result guaranteed to reflect the order of the generator, or > not? > > i'm working with a list of randomly generated results, and i want to > print them in the order in which they were generated--is lists:foreach > the way to go here? a list comprehension would be simpler to write and > nicer to look at. According to the erlang spec 4.7, page 94, the order between elements must be preserved. It's difficult to imagine the semantics of list comprehensions without this guarantee. Besides, the programming examples section in the erlang documentation implicitly states this as well: (http://erlang.se/doc/doc-5.4.8/doc/programming_examples/list_comprehensions.html#3.5) "As an example, list comprehensions can be used to simplify some of the functions in lists.erl: append(L) -> [X || L1 <- L, X <- L1]. map(Fun, L) -> [Fun(X) || X <- L]. filter(Pred, L) -> [X || X <- L, Pred(X)]." /Uffe -- Ulf Wiger From chas@REDACTED Thu Jul 28 23:42:41 2005 From: chas@REDACTED (Charles Blair) Date: Thu, 28 Jul 2005 16:42:41 -0500 Subject: list comprehension question In-Reply-To: Your message of "Thu, 28 Jul 2005 22:56:38 +0200." Message-ID: <20050728214241.F1E4530DD85@jump.lib.uchicago.edu> > According to the erlang spec 4.7, page 94, the order between elements > must be preserved. thanks for pointing me to that. i'll remember to look at it in future. > It's difficult to imagine the semantics of list comprehensions > without this guarantee. Besides, the programming examples section > in the erlang documentation implicitly states this as well: > > (http://erlang.se/doc/doc-5.4.8/doc/programming_examples/list_comprehensions.html#3.5) > > "As an example, list comprehensions can be used to simplify > some of the functions in lists.erl: > > append(L) -> [X || L1 <- L, X <- L1]. > map(Fun, L) -> [Fun(X) || X <- L]. > filter(Pred, L) -> [X || X <- L, Pred(X)]." yes, but since list comprehensions seem to be inspired by set comprehensions (where order has no meaning for a set), and since Thompson, in Haskell: The Craft of Functional Programming, describes the program for computing permutations by means of list comprehensions in part as "If xs is not empty, a permutation is given by picking an element x from xs and putting x at the front of the permutation of the remainder ..."--he does not say "by picking the first element", and this algorithm corresponds exactly to the one given in Programming Examples 3.3, and since the documentation for lists:map says "The evaluation order is implementation dependent" (and it is lists:map which is implicated in 3.5), while that for lists:foreach says "This function is used for its side effects and the evaluation order is defined to be the same as the order of the elements in the list,"--yes, it would have been odd if it didn't work that way, but i felt i needed to make sure. thanks. From carsten@REDACTED Fri Jul 29 00:11:02 2005 From: carsten@REDACTED (Carsten Schultz) Date: Fri, 29 Jul 2005 00:11:02 +0200 Subject: list comprehension question In-Reply-To: <20050728214241.F1E4530DD85@jump.lib.uchicago.edu> References: <20050728214241.F1E4530DD85@jump.lib.uchicago.edu> Message-ID: <42E957F6.4050401@codimi.de> Hi! Charles Blair schrieb: > [...] and since the documentation for lists:map says "The > evaluation order is implementation dependent" [...] The evaluation order is not the same as the order of the elements of the result list. lists:reverse(lists:map(F, lists:reverse(Xs))) and lists:map(F, Xs) yield the same result (if F is without side effects!), ie the orders of the elements in the result lists are the same, but they do not evaluate the elements in the same order. hth Carsten -- Carsten Schultz (2:38, 33:47) http://carsten.codimi.de/ PGP/GPG key on the pgp.net key servers, fingerprint on my home page. From chas@REDACTED Fri Jul 29 00:34:56 2005 From: chas@REDACTED (Charles Blair) Date: Thu, 28 Jul 2005 17:34:56 -0500 Subject: list comprehension question In-Reply-To: Your message of "Fri, 29 Jul 2005 00:11:02 +0200." <42E957F6.4050401@codimi.de> Message-ID: <20050728223456.2DFF530DFBF@jump.lib.uchicago.edu> > hth yes, it does; thanks. i was conscious of the fact that "evaluation order" might not imply anything about "result order," but i felt i wanted to be sure. From ok@REDACTED Fri Jul 29 03:57:41 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 29 Jul 2005 13:57:41 +1200 (NZST) Subject: list comprehension question Message-ID: <200507290157.j6T1vfsX299925@atlas.otago.ac.nz> Charles Blair worried that: Thompson, in Haskell: The Craft of Functional Programming, describes the program for computing permutations by means of list comprehensions in part as "If xs is not empty, a permutation is given by picking an element x from xs and putting x at the front of the permutation of the remainder ..."--he does not say "by picking the first element", and this algorithm corresponds exactly to the one given in Programming Examples 3.3, and since the documentation for lists:map says "The evaluation order is implementation dependent" (and it is lists:map which is implicated in 3.5), Reassurance: (1) List comprehension is defined in the Haskell report by source-to-source transformation to normal function calls. Here are the rules: [e | True] -> [e] [e | x] -> [e | x, True] [e | test, &c] -> if test then [e | &c] else [] [e | pat <- exp, &c] -> concatMap (\pat -> [e | &c] _ -> []) exp [e | let decls, &c] -> (let decls in [e | &c]) where concatMap :: (a -> [b]) -> [a] -> [b] concatMap f [] = [] concatMap f (x:xs) = f x ++ concatMap f xs We can figure out from this that a simple [ expr | pat <- list, guard ] is equivalent to let f (pat : xs) | guard = expr : f xs f ( _ : xs) = f xs f [] = [] in f list This simplified case and the general case both MUST preserve the order of elements, according to the Haskell specification. (2) I think the comment about lists:map/2 has been misunderstood. map(F, [X1,...,Xn]) is equivalent to [F(X1), ..., F(Xn)] When the comment says that the *evaluation* order is undefined, it means that there is no promise about whether F(X1) or F(Xn) will be *evaluated* first. There *is* a promise about which of them will be (spatially) first in the result. To make this clearer, imagine the following two definitions of map: map2(F, [X|Xs]) -> map1(F, [X|Xs]) -> T = map2(F, Xs), H = F(X), H = F(X), T = map1(F, Xs), [H|T]; [H|T]; map2(_, []) -> map1(_, []) -> []. []. If F is free of side effects, map1/2 and map2/2 will always give the same order: the order of elements within the result is defined. But they *evaluate* those elements in opposite orders. From paf@REDACTED Fri Jul 29 12:53:40 2005 From: paf@REDACTED (Paulo Alexandre Ferreira) Date: Fri, 29 Jul 2005 11:53:40 +0100 Subject: CPU/Hardware optimized for Erlang (FPGA) Message-ID: <20050729093930.M89802@dei.isep.ipp.pt> Thomas Lindgren has wrote: > FPGAs are very interesting, definitely. I think the > manufacturers now offer simple migration paths from > FPGA to ASIC too, if you find you have a winner. Recycling an old (2003 ) post : Really really crazy idea: :-) :-) :-) Why not "compile Erlang directly into Hardware" ? We have usually lots of processes, that could be "mapped" into different areas/blocks of the FPGA. Each process will have its own hardware, enough to do its own tasks, and comunicates with other "processes/hardware blocks" by message passing. Pattern matching can be easily done in software also ( I presume ) if one chooses the data tags well. Things like this loop are simple and fast in Hardware. loop(Counter)-> receive {up,Pid} -> loop(Counter+1); {down,Pid} -> loop(Counter-1); {query,Pid} -> Pid ! {value,Counter}, loop(Counter) end. Variants: a) At startup, all "processes/hardware blocks" are created and start "processing" the messages they receive. Easier, more inefficient use of the hardware. b) The "hardware blocks/processes" are "created" at runtime reconfiguring dynamically the FPGA. Harder, but more efficient use of the hardware. So instead of making hardware that "parallelizes" serial software with things like "sliding registers", why not start with software that is already "parallel" like Erlang, and put it into Hardware ? Low power is a niche, but maybe the question that Erlang answers is not related to low power, but how to use to the maximum the 10^6 of transistors that can be placed today on a CPU. This is also true of FPGAs that have today more than 10^6 system gates. There are several answers on how to use tons of hardware to get performance: RISC/CISCs of today: big caches, pipeline, out of order execution, multiple execution units, The hardware takes care of "parallelizing" the software. It is the processor that at run time, decides (or tries to find) what can be done by the different execution units. EPIC: several independent execution units, whose control is "available" to the software. The software (the compiler) is in charge of the parallelism. The CPU has no initiative. The compiler receives a "high level language" source code, and on the translation to executable code, must discover the parallelism, and take care of it. Usually the compiler does a bad job, and the programmer has to do it instead. :-) :-) "Erlang in hardware" All the "control" is available to the programmer. When an Erlang programmer "splits" the work to be done between different processes, he will be already distributing that work among several pieces of hardware. Different processes will be mapped in different areas of the hardware. Questions/considerations: Versatility: Every time the "program" changes we must "change" the hardware. This is inflexible, and suitable only for embedded systems, but can be done fast with FPGAs. Size of the hardware FPGAs: One could easily use more than one FPGA, because (well almost) all comunication is by message passing. In our Erlang program we have the connections between the different pieces of hardware made explicit by the sending and receiving of messages. So our "hardware buses" will match the sending and receiving of messages. Restarting a process: This can be done "resetting" the hardware of that process. "(UBF) Contract checkers": can be placed between the processes. Formal verification: can go down to the hardware. This is obviously a "silly season, thinking out loud" post but, but one interesting idea to develop/try would be : Erlang as an Hardware Description Language. Greetings Paulo Ferreira From mickael.remond@REDACTED Fri Jul 29 13:27:58 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Fri, 29 Jul 2005 13:27:58 +0200 Subject: Erlang Foundation follow-up Message-ID: <42EA12BE.9070509@erlang-fr.org> Hello, Following my presentation during the Erlang User Conference in 2004, I thought that the best way to eventually make the Foundation happens was to prototype it. It works well for program development, why not for community organisation :-) So, I finally managed to find time to put a prototype together. The motivations and objectives are described here: http://www.3pblog.net/index.php?entry=entry050729-131619 You should start reading this page to understand why two new different websites were created (mainly for separation of concern and simplicity): http://www.erlang-foundation.org/ http://www.planeterlang.org/ Please, bear in mind that all this is a "prototype" and that having feedbacks, corrections, suggestions, etc is the only way to push this initiative toward a full, complete "implementation" of the Erlang Foundation. I hope this will help involving more people in the definition of a possible future community for Erlang. -- Micka?l R?mond From tim@REDACTED Sat Jul 30 05:16:26 2005 From: tim@REDACTED (Tim Bates) Date: Sat, 30 Jul 2005 12:46:26 +0930 Subject: Directory structure for non-OTP projects In-Reply-To: References: <42DCE32F.9090604@bates.id.au> <42DCEF92.7080408@hyber.org> <42DF6BED.3000202@bates.id.au> <7a6679dcd096e0fd9a29686d8ad1b5cf@t-mobile.co.uk> <42E8BCAD.10803@bates.id.au> Message-ID: <42EAF10A.3030006@bates.id.au> Chandrashekhar Mullaparthi wrote: >> Do people normally go through some variant of the release process >> described in the OTP Design Principles every time they change a >> line of code? > > No - not really. This is what we do. > > - If some modules are changed, we put them in the right paths and > load the new modules into the node. > > - If we need to add new applications or the version number of an > application has changed (for e.g because of backward incompatible > changes or because we added new modules) we go through the release > process. But even in this case, we don't write .appup and .relup > files. We just build a new boot file, copy the boot file and the new > stuff into appropriate places and either wait for the next node > restart or we do a controlled restart. We maintain uptime by > diverting traffic to standby nodes. > > - If we have to upgrade the ERTS and/or some OTP libraries, we build > a whole new release and install it in place of the old one and > restart the node. Okay, but what about during development? This is the bit I don't get. How do I set up my workstation to develop my code? What do I do when I change a line of code in a module and want to run the test_server again? It's slowly making more sense but it hasn't clicked yet where I'm supposed to put my code. My code must be in the default search path, otherwise the test_server won't find it, which appears to mean that I have to put it in the same source tree as OTP, but that means having to use the OTP build process, and that doesn't seem to be well adapted for adding my own applications to. Do people really go through the whole `make install` process or build a target system or something every time they change even one line of code during development? It just seems very heavy. There must be a better way but I don't see it. Can someone who uses the test_server please describe how they have their development environment set up on their workstation? Where have you installed erlang? Where do you put your own code in relation to that? What do you do each time you change your code during development? This seems like it ought to be a simple problem but I don't get it and it's frustrating that I haven't written a single line of Erlang yet. Tim. -- Tim Bates tim@REDACTED From sean.hinde@REDACTED Sat Jul 30 11:13:57 2005 From: sean.hinde@REDACTED (Sean Hinde) Date: Sat, 30 Jul 2005 10:13:57 +0100 Subject: Directory structure for non-OTP projects In-Reply-To: <42EAF10A.3030006@bates.id.au> References: <42DCE32F.9090604@bates.id.au> <42DCEF92.7080408@hyber.org> <42DF6BED.3000202@bates.id.au> <7a6679dcd096e0fd9a29686d8ad1b5cf@t-mobile.co.uk> <42E8BCAD.10803@bates.id.au> <42EAF10A.3030006@bates.id.au> Message-ID: <050C1486-2607-421B-9D6F-B82345261ADB@gmail.com> Hi, > Can someone who uses the test_server please describe how they have > their > development environment set up on their workstation? Where have you > installed erlang? Where do you put your own code in relation to that? > What do you do each time you change your code during development? This > seems like it ought to be a simple problem but I don't get it and it's > frustrating that I haven't written a single line of Erlang yet. I think the answer must be that no-one uses the test server in the way you describe. Most people I have seen writing Erlang code have an interactive shell open and use this to compile or load changed modules. Testing of modules is done directly in the shell, perhaps by calling test routines in the same or another dedicated module. The test server is a heavyweight system typically used for testing nightly builds. The typical "compile whole statically linked system" and then run the unit tests routine is not necessary with hot code loading and an interactive shell. Having said that, I think it would be quite interesting to see how a lightweight unit test framework would look for Erlang. It should be easy enough for someone well versed in the whole junit thing to put one together. If the test specs looked something like the ones used to drive the test server then even better. Regards, Sean From ke.han@REDACTED Sat Jul 30 11:51:33 2005 From: ke.han@REDACTED (ke.han) Date: Sat, 30 Jul 2005 17:51:33 +0800 Subject: Directory structure for non-OTP projects In-Reply-To: <42EAF10A.3030006@bates.id.au> References: <42DCE32F.9090604@bates.id.au> <42DCEF92.7080408@hyber.org> <42DF6BED.3000202@bates.id.au> <7a6679dcd096e0fd9a29686d8ad1b5cf@t-mobile.co.uk> <42E8BCAD.10803@bates.id.au> <42EAF10A.3030006@bates.id.au> Message-ID: <42EB4DA5.8000507@redstarling.com> Tim, From one newbie to another, here is what I do and I'm guessing its in line with how the erlang REPOS and erlmerge see people using their dev and deploy environment. I just install erlang as normal (either root or under a less privileged id) as normal. If you can use an older version of OTP, then starting with REPOS may be even cleaner. From there, I add my dev projects under the $ERL_HOME/lib directory. I also have this directory as the root to a subversion store so I can easily save and rollback my files. > Do people really go through the whole > `make install` process or build a target system or something every time > they change even one line of code during development? It just seems very > heavy. There must be a better way but I don't see it. No, I don't need to do any such thing. During development, I just use the full installed directory structure as is. For deployment, you just want to trim down the /lib tree and perhaps recompile the erlang install to only use those features you need. As erlmerge gets more used, you should be able to start with a smaller erlang install and use erlmerge to add in just the projects you require. Then, in the spirit of REPOS, you just run the entire system under one directory hierarchy. If you need another node, you can either copy the entire hierarchy (if node on a different machine) or reuse it with different script startup files to start the second node (on the same machine) > > Can someone who uses the test_server please describe how they have their > development environment set up on their workstation? I use Win XP as my dev desktop. I find that using TurtoiseSVN embedded in Windows Explorer along with a erlang syntax sensitive text editor (erlide, UltraEdit, emacs) gives me all I need to edit, test, and store my code. One thing nice about viewing your code under one tree structure with the rest of OTP, is its easy to access the OTP source files to read and understand the library better. Erlide is nice for this. > Where have you > installed erlang? Where do you put your own code in relation to that? > What do you do each time you change your code during development? This > seems like it ought to be a simple problem but I don't get it and it's > frustrating that I haven't written a single line of Erlang yet. I put my own code under the main erlang tree. Sure you can use a separate /lib tree from OTP and then have a script startup file to append the new path to the erlang vm.. But you don't have to. Anyway, that's the very simply way I see the world and its working so far. regards, ke han From mickael.remond@REDACTED Sun Jul 31 09:38:50 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Sun, 31 Jul 2005 09:38:50 +0200 Subject: Directory structure for non-OTP projects In-Reply-To: <42EB4DA5.8000507@redstarling.com> References: <42DCE32F.9090604@bates.id.au> <42DCEF92.7080408@hyber.org> <42DF6BED.3000202@bates.id.au> <7a6679dcd096e0fd9a29686d8ad1b5cf@t-mobile.co.uk> <42E8BCAD.10803@bates.id.au> <42EAF10A.3030006@bates.id.au> <42EB4DA5.8000507@redstarling.com> Message-ID: <42EC800A.4020103@erlang-fr.org> ke.han wrote: > Tim, > From one newbie to another, here is what I do and I'm guessing its in > line with how the erlang REPOS and erlmerge see people using their dev > and deploy environment. > I just install erlang as normal (either root or under a less privileged > id) as normal. If you can use an older version of OTP, then starting > with REPOS may be even cleaner. > From there, I add my dev projects under the $ERL_HOME/lib directory. > I also have this directory as the root to a subversion store so I can > easily save and rollback my files. I am happy to this that new people are using Erlang REPOS. We are progressing toward a new stable version. I have just released the version 1.3 beta 1, that contains the latest version of Erlang/OTP (R10B-6) and update version of various software. See: http://www.erlang-projects.org/Public/projects/workgroups/repos/erlang_repos_1.3-b1/view and for download: http://downloads.erlang-projects.org/repos/repos-1.3_b1.iso.tar.gz Please tell me if you need to add software or libraries that are not currently present in REPOS. We would definitly need more contributors. One of the idea that I would like to implement is for example the addition of preconfigured Emacs modules (Erlang mode, Distel and Esense). REPOS would no include Emacs, but having a a config file on the CDROM, you could launch your version with an Erlang development properly configured. Adding Eclipse and Erlang extension is another interesting option. -- Micka?l R?mond http://www.3pblog.net/ From mickael.remond@REDACTED Sun Jul 31 09:45:34 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Sun, 31 Jul 2005 09:45:34 +0200 Subject: Erlang REPOS 1.3 beta 1 Message-ID: <42EC819E.2030905@erlang-fr.org> Hello, I just wanted to let you know that we have updated the Erlang REPOS environment. REPOS stands for Repository of Erlang-Projects.Org Software selection. It is a collection of major ready-to-use Erlang software. Erlang REPOS is distributed as a CDROM image (ISO). You can use every program included in the REPOS environment either directly from the CDROM, from your hard drive, or from a USB key. This version is a major improvement over the 1.1 release: * Repos 1.3 is now based on Erlang/OTP R10B-6. * ejabberd has been upgraded to version 0.9.1 * J-EAI has been upgraded to version 1.0 beta 5 * Dialyzer has been upgraded to version 1.3 * Yaws is now included in version 1.55 * Tsunami benchmark tool has been upgraded to version 1.0.3 * MacOSX Tiger support * Bluetail Ticket Tracker has been added * Jabberlang 0.1 has been added This version includes the latest J-EAI version (1.0 beta 5), as well as the Tkabber XMPP client, to use with ejabberd and J-EAI. Various Erlang libraries are also included in the REPOS environment and can be directly use in software development. Next steps are now to make the platform even more usefull. For that we will need more contributors: - To write more repos:check/1 unit tests. - To add new features. I am for example thinking about embedding a complete configuration for Emacs (With Erlang mode, Distel and esense). I am also planning to add install script, that will only install parts relevant to the local architecture, and change every file to read/write mode. - To improve documentation and testing. Please let us know if you are interested. Do not hesitate to report problems or success using the Erlang REPOS environment. Details: http://www.erlang-projects.org/Public/projects/workgroups/repos/erlang_repos_1.3-b1/view Download: http://downloads.erlang-projects.org/repos/repos-1.3_b1.iso.tar.gz Cheers, -- Micka?l R?mond http://www.3pblog.net/