From kent@REDACTED Fri Jun 1 16:57:14 2001 From: kent@REDACTED (Kent Boortz) Date: 01 Jun 2001 16:57:14 +0200 Subject: The update R7B-3 is released In-Reply-To: Kent Boortz's message of "05 Mar 2001 18:33:17 +0100" Message-ID: Bug fix release : otp_src_R7B-3 >From snapshot : 20010601 This is a bug fix release R7B-3. You can download the full source distribution or the patch that take R7B-2 up to R7B-3. http://www.erlang.org/download/otp_src_R7B-3.tar.gz http://www.erlang.org/download/otp_src_R7B-2to3.patch.gz There are lots of bug fixes and improvements in this release, for more information read the distribution readme http://www.erlang.org/download/otp_src_R7B-3.readme To patch the previos release "otp_src_R7B-2" you use a patch program that understand the "unified" patch format and do % cd otp_src_R7B-X % gzip -dc /otp_src_R7B-2to3.patch.gz | patch -p1 There is no support for building a Windows version from source. Note that the documentation is not updated, we will do that soon. Darwin/OS X support was added but is not completed. Read the README file for more information. For installation instructions please read the README that is part of the distribution, The OTP Team From kent@REDACTED Fri Jun 1 16:57:14 2001 From: kent@REDACTED (Kent Boortz) Date: 01 Jun 2001 16:57:14 +0200 Subject: The update R7B-3 is released In-Reply-To: Kent Boortz's message of "05 Mar 2001 18:33:17 +0100" Message-ID: Bug fix release : otp_src_R7B-3 >From snapshot : 20010601 This is a bug fix release R7B-3. You can download the full source distribution or the patch that take R7B-2 up to R7B-3. http://www.erlang.org/download/otp_src_R7B-3.tar.gz http://www.erlang.org/download/otp_src_R7B-2to3.patch.gz There are lots of bug fixes and improvements in this release, for more information read the distribution readme http://www.erlang.org/download/otp_src_R7B-3.readme To patch the previos release "otp_src_R7B-2" you use a patch program that understand the "unified" patch format and do % cd otp_src_R7B-X % gzip -dc /otp_src_R7B-2to3.patch.gz | patch -p1 There is no support for building a Windows version from source. Note that the documentation is not updated, we will do that soon. Darwin/OS X support was added but is not completed. Read the README file for more information. For installation instructions please read the README that is part of the distribution, The OTP Team From salcaraz@REDACTED Tue Jun 5 20:49:12 2001 From: salcaraz@REDACTED (Salvador Alcaraz) Date: Tue, 5 Jun 2001 20:49:12 +0200 (CEST) Subject: random number generation Message-ID: Hello, friends, from Spain: I am using random module and export functions seed/0 uniform/0 I need to initialize same process with diferent seeds. Should I have to use random:seed/3 function??? seed(A1,A2,A3) -> ran() Which is the meaning of A1, A2 , A3 parameters?? Thank you in advance _________________________________________________ Salvador Alcaraz Carrasco Division de Ingenieria Telematica Dpto. de Fisica y Arquitectura de Computadores Universidad Miguel Hernandez salcaraz@REDACTED http://obelix.umh.es __________________________________________________ From nick@REDACTED Tue Jun 5 10:48:19 2001 From: nick@REDACTED (Niclas Eklund) Date: Tue, 5 Jun 2001 10:48:19 +0200 (MET DST) Subject: The update R7B-3 is released In-Reply-To: Message-ID: Hello! To make things easier to grasp I'll highlight the most important changes in Orber. * Performance - the Inter-ORB communication overhead significantly reduced; how much depends on the arguments/return-values. * oe_register/oe_unregister - the overhead for registering IDL-specifications in the IFR significantly reduced. * New configuration parameters (see User's Guide - 5.2 Configuration): - orber_debug_level generate reports for abnormal situations (i.e. not necessarily an error). Hence, use only during development. - iiop_setup_connection_timeout which should be used at all time together with iiop_connection_timeout and iiop_timeout. * System exceptions - Orber now supports all system exceptions defined in the CORBA-2.4 specification. The documentation also contain a listing of all exceptions and why/when they might be thrown. Please note that it's the "short version"; for an extensive description see http://www.omg.org/cgi-bin/doc?formal/01-02-01. /Niclas On 1 Jun 2001, Kent Boortz wrote: > Bug fix release : otp_src_R7B-3 > >From snapshot : 20010601 > > This is a bug fix release R7B-3. You can download the full source > distribution or the patch that take R7B-2 up to R7B-3. > > http://www.erlang.org/download/otp_src_R7B-3.tar.gz > http://www.erlang.org/download/otp_src_R7B-2to3.patch.gz > > There are lots of bug fixes and improvements in this release, for more > information read the distribution readme > > http://www.erlang.org/download/otp_src_R7B-3.readme > > To patch the previos release "otp_src_R7B-2" you use a patch program > that understand the "unified" patch format and do > > % cd otp_src_R7B-X > % gzip -dc /otp_src_R7B-2to3.patch.gz | patch -p1 > > There is no support for building a Windows version from source. > > Note that the documentation is not updated, we will do that soon. > > Darwin/OS X support was added but is not completed. Read the README file > for more information. > > For installation instructions please read the README that is part of > the distribution, > > The OTP Team From bjorn@REDACTED Tue Jun 5 14:30:17 2001 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 05 Jun 2001 14:30:17 +0200 Subject: Binaries in R8 In-Reply-To: James Hague's message of "Thu, 31 May 2001 10:09:15 -0500" References: Message-ID: So far we have speeded up binary matching somewhat. If time permits we will speed up binary constructing too. There might be other improvements as well. /Bjorn James Hague writes: > Does anyone on the Erlang team know what's changing--if anything--in > relation to binaries for R8? Mostly I'm hoping to see some real support for > little-endian matching, as the "/little" qualifier is fairly ambiguous > and/or broken (i.e. what does it mean to say that a bitfield is little > endian, if you can't specify that an entire word is little endian?). > Several times now I've been wanting to write code to pick apart and create > binary files, but if those files are little endian internally, then it is > much more difficult. Binaries are so beautiful otherwise! > > James > -- Bj?rn Gustavsson Ericsson Utvecklings AB bjorn@REDACTED ?T2/UAB/F/P BOX 1505 +46 8 727 56 87 125 25 ?lvsj? From rv@REDACTED Wed Jun 6 11:25:10 2001 From: rv@REDACTED (Robert Virding) Date: Wed, 06 Jun 2001 11:25:10 +0200 Subject: random number generation In-Reply-To: Your message of "Tue, 05 Jun 2001 20:49:12 +0200." Message-ID: <200106060925.LAA14235@trana.bluetail.com> The seed to the random number generator actually consists of three (positive) numbers. These can be set to the default values using seed/0 and to specific numbers using seed/3. Both functions return the current seed. There is no specific meaning the A1, A2, and A3 parameters, they are just three numbers which together make the seed. If you want to (re-)initialise a processes' seed to set (different) values then you must use seed/3. Robert Salvador Alcaraz writes: > >Hello, friends, from Spain: > >I am using random module and export functions > >seed/0 >uniform/0 > >I need to initialize same process with diferent seeds. > >Should I have to use random:seed/3 function??? > >seed(A1,A2,A3) -> ran() > >Which is the meaning of A1, A2 , A3 parameters?? > > >Thank you in advance > > >_________________________________________________ >Salvador Alcaraz Carrasco >Division de Ingenieria Telematica >Dpto. de Fisica y Arquitectura de Computadores >Universidad Miguel Hernandez > >salcaraz@REDACTED >http://obelix.umh.es >__________________________________________________ From etxuwig@REDACTED Wed Jun 6 11:47:11 2001 From: etxuwig@REDACTED (Ulf Wiger) Date: Wed, 6 Jun 2001 11:47:11 +0200 (MET DST) Subject: random number generation In-Reply-To: <200106060925.LAA14235@trana.bluetail.com> Message-ID: On Wed, 6 Jun 2001, Robert Virding wrote: >There is no specific meaning the A1, A2, and A3 parameters, they are >just three numbers which together make the seed. If you want to >(re-)initialise a processes' seed to set (different) values then >you must use seed/3. A handy way of doing this is of course: new_seed() -> {A1,A2,A3} = erlang:now(), random:seed(A1,A2,A3). /Uffe -- Ulf Wiger tfn: +46 8 719 81 95 Senior System Architect mob: +46 70 519 81 95 Strategic Product & System Management ATM Multiservice Networks Data Backbone & Optical Services Division Ericsson Telecom AB From comini@REDACTED Wed Jun 6 17:01:20 2001 From: comini@REDACTED (Marco Comini) Date: Wed, 6 Jun 2001 17:01:20 +0200 Subject: [E-CFP] AGP'01 2nd CFP Message-ID: !!! We apologize if you receive this message more than once. !!! !!! Please circulate !!! ------------------------------------------------ APPIA-GULP-PRODE'01 2nd Call for Papers Deadline July, 1st 2001 2001 JOINT CONFERENCE ON DECLARATIVE PROGRAMMING Evora, Portugal September 26-28 2001 http://www.di.uevora.pt/~agp01 Following APPIA-GULP-PRODE'00 in La Habana (Cuba) the next joint APPIA, GULP, and PRODE conference will be held in Evora (Portugal), in September 26-28, 2001. The aim of the Conference is to foster scientific meetings between Italian, Portuguese, Spanish and Latin America researchers, to improve the knowledge of the state of the art of declarative programming (through the invited talks) and to show the ongoing research done (through presentations of papers). Program Chairs Lu?s Moniz Pereira U. Nova de Lisboa (P.) Paulo Quaresma U. de Evora (P.) Program Committee Salvador Pinto Abreu (U. de Evora, P.) Jos? J. Alferes (U. Nova de Lisboa, P.) Patrizia Asirelli (IEI, CNR, I.) Pedro Barahona (U. Nova de Lisboa, P.) Elvira Pino Blanco (U. Polit?cnica de Catalu?a, E.) Marco Comini (U. di Udine, I.) Carlos Dam?sio (U. Nova de Lisboa, P.) Andrea Formisano (U. di Perugia, I.) Marisa Navarro G?mez (U. del Pais Vasco, E.) Sergio Greco (U. della Calabria, I.) Blas Carlos Ruiz Jimenez (U. de M?laga, E.) Alberto Martelli (U. di Torino, I.) Maurizio Martelli (U. di Genova, I.) Francisco Galan Morillo (U. de Sevilla, E.) Juan Jos? Moreno Navarro (U. Polit?cnica de Madrid, E.) Lu?s Moniz Pereira (U. Nova de Lisboa, P.) Fernando Saez P?rez (U. Complutense de Madrid, E.) Alberto Pettorossi (U. di Roma Tor Vergata, I.) Paulo Quaresma (U. de Evora, P.) Maria Jos? Ramirez (U. Polit?cnica de Valencia, E.) Gianfranco Rossi (U. di Parma, I.) Fernando Silva (U. do Porto, P.) Local Organizing Committee Iara de Almeida L?gia Ferreira V?tor Nogueira Lu?s Rato Topics: The technical program for the conference will include invited lectures and presentations of papers. Papers will be reviewed and accepted submissions will be available on WEB. Please specify whether the submitted paper is also under consideration (or accepted) for presentation in other meetings. Submissions of papers are welcome on all aspects of declarative programming, including, but not limited to: Theoretical Foundations: languages, semantics, procedures for declarative programming Implementation Program development tools Knowledge representation and reasoning Extensions Critical surveys Applications and evaluation Papers must not exceed 15 pages, including references and figures and must include a cover page containing the following: title of the paper, a 200-word abstract, keywords, postal and electronic mailing addresses, and voice and fax numbers of the contact author. Papers can be written in Italian, Portuguese, Spanish or, preferably, in English. Papers should be formatted according to a LaTeX style file, which can be retrieved from the conference web site http://www.di.uevora.pt/~agp01/manus.html. Accepted papers must be presented at the conference. Authors are invited to submit their papers electronically in (compressed uuencoded) postscript version to the following e-mail address: agp01@REDACTED IMPORTANT DATES Submissions: July 1 Notification of acceptance/rejection: July 27 Deadline for final text: September 7 Conference Site: The Conference is organized by the University of Evora http://www.uevora.pt, founded in 1559 in Evora (Portugal). Evora http://www.evora.net, located 140km southeast of Lisbon, and 60 km from the spanish border, is one of the oldest cities in the Iberian Peninsula and it was classified by UNESCO as World Heritage. As a Roman Town it was known as "Liberalitas Julia" and it became important during the late Middle Ages. Evora is also reknown for its architecture, with monuments varying in style from the Gothic and Manueline to the Moorish. For further details consult: http://www.di.uevora.pt/~agp01 -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Wed Jun 6 20:08:38 2001 From: vances@REDACTED (Vance Shipley) Date: Wed, 6 Jun 2001 14:08:38 -0400 Subject: Using Threads Message-ID: I want to run a threaded system so that device drivers can take advantage of threads for possibly slow operations. The first thing I note is that the documentation doesn't quite jive with the implementation. erl Starts the Erlang system. Any argument starting with a plus sign (+) is always interpreted as a system flag (described below), regardless of where it occurs on the command line. Arguments starting with a hyphen (-) are the start of a flag. A flag includes all following arguments up to the next argument starting with a hyphen. ... System Flags The erl script invokes the code for the Erlang virtual machine. This program supports the following flags: +A size Sets the pool size for device driver threads. Default is 0. OK, so then this should work (but doesn't): C:\>erl +A 10 erl unknown flag -root usage: erl [flags] [ -- [init_args] ] The flags are: -v turn on chatty mode, GCs will be reported e -l turn on auto load tracing -i module set the boot module (default init) -b fun set the boot function (default boot) -h number set minimum heap size in words (default 233 -# number set the number of items to be used in trace -B turn break handler off -P number set maximum number of processes on this nod valid range is [16-32768] -A number set number or threads in async thread pool valid range is [0-256] OK, so it didn't work. Now from the above usage message you might assume that the correct syntax would be: C:\>erl -A 10 Eshell V5.0.2 (abort with ^G) 1> erlang:system_info(thread_pool_size) . 0 2> However it isn't as you see. The correct syntax is: C:\>erl +A10 Eshell V5.0.2 (abort with ^G) 1> erlang:system_info(thread_pool_size). 10 2> Now the next question I have is why I can't get threads at all on any system other than Windows. Solaris is supposed to be supported but mine doesn't seam to work: $ erl +A10 Erlang (BEAM) emulator version 5.0.1.1 [source] Eshell V5.0.1.1 (abort with ^G) 1> erlang:system_info(thread_pool_size). 0 According to the release notes threading is supported on both Solaris & Windows: There is now support for operating system threads in drivers. The file driver has been modified to make use of this, which means that lengthy file operations no longer cause everything on the node to pause. Currently only supported on Solaris and Windows. The number of threads in the thread pool can be set with the emulator system flag -A. The default number of threads is 0, which means that this feature is turned off. The number of threads in a node can be obtained by the call erlang:info(thread_pool_size). So now I'm digging into the build stuff to determine why I have no working threads. Can anyone else get threads recognized on systems other than Windows? -Vance Motivity Telecom Inc. +1 519 579 5816 From david@REDACTED Wed Jun 6 23:55:03 2001 From: david@REDACTED (david wallin) Date: Wed, 6 Jun 2001 23:55:03 +0200 Subject: floating-point representation Message-ID: Hi all, I'm trying to create (32-bit) floats from binaries (using the bit syntax). Empirical tests showed this to be a naive idea : A = random:uniform(256), B = random:uniform(256), C = random:uniform(256), D = random:uniform(256), <> = <>. (depending on the values of A,B,C,D ; this can result in a badmatch). What kind of representation is used for floats (is this a local issue) ? Also, is there any way to tell what precision that erlang uses ? --david. From Sean.Hinde@REDACTED Thu Jun 7 14:11:43 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Thu, 7 Jun 2001 13:11:43 +0100 Subject: Using Threads Message-ID: <402DD461F109D411977E0008C791C312039F601F@imp02mbx.one2one.co.uk> Vance, > Now the next question I have is why I can't get threads at all on > any system other than Windows. Solaris is supposed to be supported > but mine doesn't seam to work: > > $ erl +A10 > Erlang (BEAM) emulator version 5.0.1.1 [source] > > Eshell V5.0.1.1 (abort with ^G) > 1> erlang:system_info(thread_pool_size). > 0 This works as below for the commercial Solaris version on my machine: erl +A10 Erlang (BEAM) emulator version 5.0.2.2 [threads] Eshell V5.0.2.2 (abort with ^G) 1> erlang:system_info(thread_pool_size). 10 The version I compiled from open source gives the same response as you got. I wonder if the little [threads] logo might give a clue ?? Rgds, Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From rickard.green@REDACTED Thu Jun 7 14:49:54 2001 From: rickard.green@REDACTED (Rickard Green) Date: Thu, 07 Jun 2001 14:49:54 +0200 Subject: Using Threads References: <402DD461F109D411977E0008C791C312039F601F@imp02mbx.one2one.co.uk> Message-ID: <3B1F7872.653129AB@uab.ericsson.se> Hello, The emulator is by default not built with thread support in the open source release. In order to enable thread support one is supposed to pass the "--enable-threads" argument to the configure script; but, due to a bug this doesn't work. One can easily bypass this bug, though. Do the following to build the open source release with thread support: # Set the ERL_TOP environment variable to the absolute path of the # top level directory cd $ERL_TOP ./configure --enable-threads $MY_OTHER_CONFIG_ARGS cd $ERL_TOP/erts/autoconf # These steps ./configure --enable-threads $MY_OTHER_CONFIG_ARGS # bypass the cd $ERL_TOP # bug. make make install If you succeeded "[threads]" should pop up on the first line in the Erlang shell. This bug will be fixed in the next patch to r7b01. /Rickard Green, Erlang/OTP, Ericsson UAB Sean Hinde wrote: > > Vance, > > > Now the next question I have is why I can't get threads at all on > > any system other than Windows. Solaris is supposed to be supported > > but mine doesn't seam to work: > > > > $ erl +A10 > > Erlang (BEAM) emulator version 5.0.1.1 [source] > > > > Eshell V5.0.1.1 (abort with ^G) > > 1> erlang:system_info(thread_pool_size). > > 0 > > This works as below for the commercial Solaris version on my machine: > > erl +A10 > Erlang (BEAM) emulator version 5.0.2.2 [threads] > > Eshell V5.0.2.2 (abort with ^G) > 1> erlang:system_info(thread_pool_size). > 10 > > The version I compiled from open source gives the same response as you got. > > I wonder if the little [threads] logo might give a clue ?? > > Rgds, > Sean > > NOTICE AND DISCLAIMER: > This email (including attachments) is confidential. If you have received > this email in error please notify the sender immediately and delete this > email from your system without copying or disseminating it or placing any > reliance upon its contents. We cannot accept liability for any breaches of > confidence arising through use of email. Any opinions expressed in this > email (including attachments) are those of the author and do not necessarily > reflect our opinions. We will not accept responsibility for any commitments > made by our employees outside the scope of our business. We do not warrant > the accuracy or completeness of such information. From fritchie@REDACTED Thu Jun 7 15:05:05 2001 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 07 Jun 2001 08:05:05 -0500 Subject: Using Threads In-Reply-To: Message of "Thu, 07 Jun 2001 13:11:43 BST." <402DD461F109D411977E0008C791C312039F601F@imp02mbx.one2one.co.uk> Message-ID: <200106071305.f57D55g65978@snookles.snookles.com> >>>>> "sh" == Sean Hinde writes: sh> [The +A flag] works as below for the commercial Solaris version on my sh> machine: sh> [...] sh> The version I compiled from open source gives the same response as sh> you got. If I recall correctly, I had to do a couple of things to get threads support with the open source release. I don't have time to confirm them before sending this message, but perhaps you can work it out from here. {shrug} Disclaimer: this is what I recall I had to do with open source R7B-1 under Linux, FreeBSD, and Solaris. YMMV. After running the top-level "configure", I had to manually edit erts/emulator/Makefile to add what configure should've added when using the I-forget-the-configure-flag-that-should've-enabled-os- thread-support (enable-threads?). The configure-substituted symbols are THR_DEFS (in CFLAGS) and THR_LIBS (in LIBS) (see erts/emulator/Makefile.in). The OS-appropriate values for the THR_* symbols can be found in erts/autoconf/configure. Solaris & Linux: THR_LIBS="-lpthread" THR_DEFS="-DUSE_THREADS -D_REENTRANT" THR_DEFS="$THR_DEFS -DPOSIX_THREADS -D_THREAD_SAFE" FreeBSD: THR_LIBS="-lc_r" THR_DEFS="-DUSE_THREADS -D_REENTRANT" THR_DEFS="$THR_DEFS -DPOSIX_THREADS -D_THREAD_SAFE" You'll come to much grief and woe if you don't include -DUSE_THREADS *and* -D_REENTRANT. If you did the build stuff correctly, you'd see this at startup: % erl Erlang (BEAM) emulator version 5.0.1.1 [source] [threads] Eshell V5.0.1.1 (abort with ^G) 1> Second, I recall (fuzzy memory) that the +A flag still didn't work. I had to set the ERL_THREAD_POOL_SIZE environment variable to get erlang:system_info(thread_pool_size) to spit out something non-zero. -Scott From Chandrashekhar.Mullaparthi@REDACTED Thu Jun 7 17:07:21 2001 From: Chandrashekhar.Mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Thu, 7 Jun 2001 16:07:21 +0100 Subject: reading internally formatted disk logs Message-ID: <402DD461F109D411977E0008C791C31203919B7F@imp02mbx.one2one.co.uk> I have a disk log opened with the following params: {type, halt} {format, internal} While the log is still open, I can write to terms to it, and read them using disk_log:chunk - but once the log is closed, there seems to be no way of reading from it. If I try disk_log:chunk after closing the log, I get {error, no_such_log}. So how does one read an internally formatted log once it has been closed?? tia, Chandru NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From david@REDACTED Thu Jun 7 23:58:52 2001 From: david@REDACTED (david wallin) Date: Thu, 7 Jun 2001 23:58:52 +0200 Subject: floating-point representation In-Reply-To: References: Message-ID: I tried to decipher my own mail and think some clarification might be in order... >Hi all, > >I'm trying to create (32-bit) floats from binaries (using the bit syntax). > >Empirical tests showed this to be a naive idea : > >A = random:uniform(256), >B = random:uniform(256), >C = random:uniform(256), >D = random:uniform(256), > ><> = <>. > >(depending on the values of A,B,C,D ; this can result in a badmatch). This is probably bit-combinations meaning "Infinity" and "NaN". It would be nice if this operation didn't fail but instead returned the above. (Is this possible and/or is it a performance penalty for this ?) > >What kind of representation is used for floats (is this a local issue) ? > >Also, is there any way to tell what precision that erlang uses ? > > >--david. From dne@REDACTED Fri Jun 8 00:32:23 2001 From: dne@REDACTED (Daniel =?iso-8859-1?q?N=E9ri?=) Date: 08 Jun 2001 00:32:23 +0200 Subject: Using Threads In-Reply-To: <200106071305.f57D55g65978@snookles.snookles.com> (Scott Lystig Fritchie's message of "Thu, 07 Jun 2001 08:05:05 -0500") References: <200106071305.f57D55g65978@snookles.snookles.com> Message-ID: <87hexrvs3s.fsf@nowhere.mayonnaise.net> Scott Lystig Fritchie writes: > FreeBSD: > > THR_LIBS="-lc_r" > THR_DEFS="-DUSE_THREADS -D_REENTRANT" > THR_DEFS="$THR_DEFS -DPOSIX_THREADS -D_THREAD_SAFE" > > You'll come to much grief and woe if you don't include -DUSE_THREADS > *and* -D_REENTRANT. Aren't you supposed to use the "-pthread" option[*] to cc, to compile/link pthreaded stuff on FreeBSD? Regards, --Daniel [*] http://www.FreeBSD.org/cgi/man.cgi?query=pthread&apropos=0&sektion=0&manpath=FreeBSD+4.3-RELEASE&format=html -- Daniel Neri dne@REDACTED From adept@REDACTED Fri Jun 8 11:46:57 2001 From: adept@REDACTED (Dmitry Astapov) Date: 08 Jun 2001 12:46:57 +0300 Subject: Anyone here uses erlang to work with Siemens EWSD, by any chance? Message-ID: <87u21r8fse.fsf@dimail.umc.com.ua> I'm especially interested in how to process TBCD (transposed binary-coded decimals) strings efficiently with erlang (encode/decode) and how to deal with several complex ASN.1 declaration using one generic piece of code (which boils down to various tricks with 'record's, and I'd rather not be reinvening the wheel). -- Dmitry Astapov //ADEpt (mail-to: adept@REDACTED) From arndt@REDACTED Fri Jun 8 14:05:30 2001 From: arndt@REDACTED (Arndt Jonasson) Date: 8 Jun 2001 12:05:30 GMT Subject: floating-point representation References: , Message-ID: <9fqf2a$5nm$1@news.du.uab.ericsson.se> In article , david wallin wrote: >I tried to decipher my own mail and think some clarification might be >in order... > > >>Hi all, >> >>I'm trying to create (32-bit) floats from binaries (using the bit syntax). >> >>Empirical tests showed this to be a naive idea : >> >>A = random:uniform(256), >>B = random:uniform(256), >>C = random:uniform(256), >>D = random:uniform(256), >> >><> = <>. >> >>(depending on the values of A,B,C,D ; this can result in a badmatch). > >This is probably bit-combinations meaning "Infinity" and "NaN". It >would be nice if this operation didn't fail but instead returned the >above. (Is this possible and/or is it a performance penalty for this >?) You are right; you can construct invalid floating-point bit patterns with this method, and since Erlang doesn't include "not-a-numbers" in the float data type, an exit is signalled. If you expect there to be non-numbers in the binary, you can use 'catch' to catch the exit: bytes_to_float(A,B,C,D) -> <> = <>, Float. convert(A,B,C,D) -> case catch bytes_to_float(A,B,C,D) of {'EXIT', _} -> not_a_number; F -> F end. >> >>What kind of representation is used for floats (is this a local issue) ? The representation is the IEEE 754 floating-point standard. This also happens to be the native representation on the platforms on which the commercial releases of Erlang runs. Here is a description of the standard: http://research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html >>Also, is there any way to tell what precision that erlang uses ? Internally, it uses the 'double' data type of C, which probably means 64 bits on most platforms. -- Arndt Jonasson arndt@REDACTED From Sean.Hinde@REDACTED Fri Jun 8 20:25:21 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Fri, 8 Jun 2001 19:25:21 +0100 Subject: Pitiful benchmark performance Message-ID: <402DD461F109D411977E0008C791C312039F603B@imp02mbx.one2one.co.uk> Hi, I've been browsing the Doug Bagley Shootout pages: http://www.bagley.org/~doug/shootout/ and Erlang does spectacularly badly in quite a few areas where it shouldn't really be quite that bad. I'm thinking of the Echo Client/Server which sends messages via an INET socket from one process to another. All the others use two Native processes but the Erlang one uses lightweight threads and still manages to be nearly three times slower than all the others. Also in Count Lines/Words/Chars Erlang uses 56Mbytes of RAM where the nearest other is Java at 7M and the majority come in with less than 2. Even at 8 octets per char it shouldn't use 25 times more memory.. Now I know that benchmarks are not representative etc etc but they do encapsulate quite a few common idioms and there shouldn't be a good reason for Erlang to do so badly. Maybe the HIPE guys can use (are using?) these in their investigations into optimal GC schemes etc for R8? This sort of thing does put people off, and I for one have bought a lot more RAM than my applications should need just in case... General moan over.. - Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From apomares@REDACTED Fri Jun 8 21:25:11 2001 From: apomares@REDACTED (Alejandro Pomares) Date: Fri, 8 Jun 2001 21:25:11 +0200 (CEST) Subject: Executable file Message-ID: Hello friends, from Spain: Is it posible to get an executable file from an Erlang code file? Thank you in advance --------------------------------------- Alejandro Pomares Universidad Miguel Hernandez Spain From jamesh@REDACTED Fri Jun 8 22:08:00 2001 From: jamesh@REDACTED (James Hague) Date: Fri, 8 Jun 2001 15:08:00 -0500 Subject: Pitiful benchmark performance Message-ID: > I've been browsing the Doug Bagley Shootout pages: > http://www.bagley.org/~doug/shootout/ and Erlang does > spectacularly badly in > quite a few areas where it shouldn't really be quite that bad. Some of the benchmarks are pretty convoluted in Erlang. I'm thinking specifically of one of the Hash benchmarks, which does a lot of conversion between lists and atoms inside an inner loop. To wit: From jamesh@REDACTED Fri Jun 8 22:09:58 2001 From: jamesh@REDACTED (James Hague) Date: Fri, 8 Jun 2001 15:09:58 -0500 Subject: Pitiful benchmark performance Message-ID: (I hit "send" by accident. Apologies.) > I've been browsing the Doug Bagley Shootout pages: > http://www.bagley.org/~doug/shootout/ and Erlang does > spectacularly badly in > quite a few areas where it shouldn't really be quite that bad. Some of the benchmarks are pretty convoluted in Erlang. I'm thinking specifically of one of the Hash benchmarks, which does a lot of conversion between lists and atoms inside an inner loop. To wit: doinserts1(10000, _) -> ok; doinserts1(I, H) -> Key = list_to_atom(lists:append( "foo_", integer_to_list(I))), ets:insert(H, { Key, I }), doinserts1(I+1, H). This is much more than a test of hashing. James From seb@REDACTED Fri Jun 8 22:15:53 2001 From: seb@REDACTED (Sebastian Strollo) Date: 08 Jun 2001 22:15:53 +0200 Subject: Pitiful benchmark performance In-Reply-To: James Hague's message of "Fri, 8 Jun 2001 15:09:58 -0500" References: Message-ID: James Hague writes: > Some of the benchmarks are pretty convoluted in Erlang. I'm thinking > specifically of one of the Hash benchmarks, which does a lot of conversion > between lists and atoms inside an inner loop. ... Hmm, further... This way: server_loop(Sock, Bytes + length(binary_to_list(Packet))); of counting the number of bytes received in the echo test (instead of just size(Packet)) creates quite an overhead... /Sebastian From seb@REDACTED Sat Jun 9 01:24:47 2001 From: seb@REDACTED (Sebastian Strollo) Date: 09 Jun 2001 01:24:47 +0200 Subject: Pitiful benchmark performance In-Reply-To: matthias@corelatus.com's message of "Fri, 8 Jun 2001 23:15:59 +0200" References: <15137.16527.785606.236931@corelatus.com> Message-ID: matthias@REDACTED writes: > Sebastian Strollo writes: > > > of counting the number of bytes received in the echo test (instead of > > just size(Packet)) creates quite an overhead... > > Yes, that's exactly what I thought. > > So I changed the program and ran tests comparing the original and > changed ones and there's no big difference. Hm, you are quite right. I guess when it comes to 19 bytes the difference is insignifcant. So I kept that change, made the constant a binary (using <<"...">>), changed the sockets to active mode and reran the tests. This gained me 10-20% which still leaves Erlang way behind the other implementations. One thought I had was that the other tests are forking and are not being charged for both processes CPU time. *But* I ran the C, perl and the Erlang version of the test on the same machine and just the difference in wall clock is really big. I don't know what is going on? Any takers? /Sebastian Results from a linux machine: % /usr/bin/time ./a.out 100000 server processed 1900000 bytes 0.32user 6.68system 0:07.07elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (95major+27minor)pagefaults 0swaps % /usr/bin/time ./echo.perl 100000 server processed 1900000 bytes 4.90user 7.91system 0:12.84elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (270major+225minor)pagefaults 0swaps % /usr/bin/time erl -noshell -noinput -s echo main 100000 server processed 1900000 bytes 32.98user 11.39system 0:45.78elapsed 96%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (632major+1099minor)pagefaults 0swaps % /usr/bin/time erl -noshell -noinput -s echo3 main 100000 server processed 1900000 bytes 26.44user 10.39system 0:38.45elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (633major+1106minor)pagefaults 0swaps --- echo.erl Sat Jun 9 00:12:09 2001 +++ echo3.erl Sat Jun 9 01:14:18 2001 @@ -4,10 +4,10 @@ %%% TBD - need to add check for valid response. --module(echo). +-module(echo3). -export([main/0, main/1, client/2, server/1]). --define(DATA, "Hello there sailor\n"). +-define(DATA, <<"Hello there sailor\n">>). main() -> main(['1']). main([Arg]) -> @@ -18,7 +18,7 @@ init:stop(). create_server_sock() -> - {ok, LSock} = gen_tcp:listen(0, [binary, {packet, 0}, {active, false}]), + {ok, LSock} = gen_tcp:listen(0, [binary]), LSock. socket_port(Sock) -> @@ -26,17 +26,16 @@ Port. client(N, ServerPort) -> - {ok, Sock} = gen_tcp:connect("localhost", ServerPort, - [binary, {packet, 0}, {active, false}]), + {ok, Sock} = gen_tcp:connect("localhost", ServerPort, [binary]), client_loop(N, Sock), gen_tcp:close(Sock). client_loop(0, Sock) -> ok; client_loop(N, Sock) -> ok = gen_tcp:send(Sock, ?DATA), - case gen_tcp:recv(Sock, 0) of - {ok, Packet} -> client_loop(N-1, Sock); - {error, closed} -> ok + receive + {tcp, Sock, _} -> client_loop(N-1, Sock); + {tcp_closed, Sock} -> ok end. server(LSock) -> @@ -45,11 +44,11 @@ gen_tcp:close(LSock). server_loop(Sock, Bytes) -> - case gen_tcp:recv(Sock, 0) of - {ok, Packet} -> + receive + {tcp, Sock, Packet} -> ok = gen_tcp:send(Sock, Packet), - server_loop(Sock, Bytes + length(binary_to_list(Packet))); - {error, closed} -> + server_loop(Sock, Bytes + size(Packet)); + {tcp_closed, Sock} -> io:format("server processed ~w bytes~n", [Bytes]), gen_tcp:close(Sock) end. From her@REDACTED Sat Jun 9 15:30:52 2001 From: her@REDACTED (Helmut Enck-Radana) Date: Sat, 09 Jun 2001 15:30:52 +0200 Subject: Executable file Message-ID: <5.0.2.1.0.20010609153026.00a64110@popmail.space.net> At 21:25 08-06-01, you wrote: >Is it posible to get an executable file from an Erlang code >file? Have a look at http://www.bluetail.com/~joe/sae_r7b/sae.html and http://www.bluetail.com/wiki/showPage?node=StandAloneErlang -- Helmut From etxuwig@REDACTED Sat Jun 9 15:48:15 2001 From: etxuwig@REDACTED (Ulf Wiger) Date: Sat, 9 Jun 2001 15:48:15 +0200 (MET DST) Subject: Pitiful benchmark performance In-Reply-To: Message-ID: On 9 Jun 2001, Sebastian Strollo wrote: >*But* I ran the C, perl and the Erlang version of the test on >the same machine and just the difference in wall clock is really >big. I don't know what is going on? Any takers? I don't know, but I've never had the impression that Erlang was very quick to initialize. What happens if the clock is started by triggering it inside an already running VM? Also, I don't know how much it affects the benchmark, but sockets basically behave like low-priority processes. This seems to work OK in a system that handles large volumes of I/O and doing some- thing significant with most of it (e.g. connection handling.) /Uffe -- Ulf Wiger tfn: +46 8 719 81 95 Senior System Architect mob: +46 70 519 81 95 Strategic Product & System Management ATM Multiservice Networks Data Backbone & Optical Services Division Ericsson Telecom AB From etxuwig@REDACTED Sat Jun 9 16:12:48 2001 From: etxuwig@REDACTED (Ulf Wiger) Date: Sat, 9 Jun 2001 16:12:48 +0200 (MET DST) Subject: Pitiful benchmark performance In-Reply-To: Message-ID: On Sat, 9 Jun 2001, Ulf Wiger wrote: >I don't know, but I've never had the impression that Erlang was >very quick to initialize. What happens if the clock is started >by triggering it inside an already running VM? (Since I can barely understand what I wrote myself:) By "triggering it", I meant first starting the VM, and then running the benchmark from within Erlang. At least, this could give a perspective on how much of the 19 sec lies in just starting Erlang. Of course, clocking by hand on my old Pentium 90, this still only saves you 2-3 seconds... What would happen if you e.g. change INPUT_REDUCTIONS in erl_vm.h from (2* CONTEXT_REDS) to CONTEXT_REDS, doubling the I/O poll frequency? (CONTEXT_REDS is set to 1000) If basically all that happens in the erlang vm is that some process is waiting for I/O, ports may not be polled with any predictably high frequency (then again, this is only guessing wildly. Perhaps it effectively gives the ports even higher priority...) /Uffe -- Ulf Wiger tfn: +46 8 719 81 95 Senior System Architect mob: +46 70 519 81 95 Strategic Product & System Management ATM Multiservice Networks Data Backbone & Optical Services Division Ericsson Telecom AB From radhia@REDACTED Sat Jun 9 20:35:05 2001 From: radhia@REDACTED (radhia@REDACTED) Date: Sat, 9 Jun 2001 20:35:05 +0200 (CEST) Subject: SAS'01: Early registration 15th June Message-ID: <200106091835.UAA11011@albatros.polytechnique.fr> ___________________________________________________________________________ [We apologize for any inconvience caused by multiple copies] SAS'01 Eigth International Static Analysis Symposium La Sorbonne, Paris, 16-18 July, 2001 http://www.ens.fr/sas01/ Registration information is available on the SAS'01 website. Early registration ends soon (15th June). ___________________________________________________________________________ From jocke@REDACTED Mon Jun 11 18:37:46 2001 From: jocke@REDACTED (Joakim G.) Date: Mon, 11 Jun 2001 18:37:46 +0200 Subject: Erlang success story Message-ID: <3B24F3DA.66046384@bluetail.com> FYI: http://www.networkcomputing.com/1212/1212f46.html Excerpt: "Alteon scores with the newest version of its iSD (Integrated Service Director) SSL acceleration product. Offering unmatched management via an integrated system based on Erlang (a programming language designed for developing robust distributed systems), this appliance performs like a champ, with more bang for the buck than the competition. The iSD 2.0 stole the performance category by serving up more than 530 SSL transactions per second. With its certificate-management features, including client-side certificate authentication and certificate-revocation lists, this product could have stopped there and we'd have been pleased with it. Instead, it keeps offering features that push it over the top, to beat F5's Big-IP 200 e-Commerce Controller." /Jocke From jocke@REDACTED Mon Jun 11 18:40:36 2001 From: jocke@REDACTED (Joakim G.) Date: Mon, 11 Jun 2001 18:40:36 +0200 Subject: Erlang success story References: <3B24F3DA.66046384@bluetail.com> Message-ID: <3B24F484.78A48E01@bluetail.com> Read it from the beginning here: http://www.networkcomputing.com/1212/1212f4.html "Joakim G." wrote: > > FYI: > > http://www.networkcomputing.com/1212/1212f46.html > > Excerpt: > > "Alteon scores with the newest version of its iSD (Integrated Service Director) SSL > acceleration product. Offering unmatched management via an integrated system based > on Erlang (a programming language designed for developing robust distributed > systems), this appliance performs like a champ, with more bang for the buck than the > competition. The iSD 2.0 stole the performance category by serving up more than 530 > SSL transactions per second. With its certificate-management features, including client-side > certificate authentication and certificate-revocation lists, this product could have stopped there and > we'd have been pleased with it. Instead, it keeps offering features that push it over the top, to beat F5's > Big-IP 200 e-Commerce Controller." > > /Jocke From jamesh@REDACTED Mon Jun 11 17:44:18 2001 From: jamesh@REDACTED (James Hague) Date: Mon, 11 Jun 2001 10:44:18 -0500 Subject: Pitiful benchmark performance Message-ID: Sean Hinde wrote: > > Now I know that benchmarks are not representative etc etc but they do > encapsulate quite a few common idioms and there shouldn't be > a good reason > for Erlang to do so badly. Maybe the HIPE guys can use (are > using?) these in > their investigations into optimal GC schemes etc for R8? Networking aside, some of these benchmarks are just plain bad. The List Operations benchmark involves accessing lists from both ends. Naturally, naive implementations of this are going to suffer in functional languages, and languages that implement "lists" as arrays (like Perl) are going to do better. Some of the others involve calls to lists:append inside inner loops. While it would be nice for Erlang to have a better showing--Bagley's page has gotten a lot of attention--I think these benchmarks are too trivial to be of much value. What *is* heartening though, is that Erlang still beats Python in most cases :) James From Erik.Johansson@REDACTED Mon Jun 11 17:59:55 2001 From: Erik.Johansson@REDACTED (Erik.Johansson) Date: Mon, 11 Jun 2001 17:59:55 +0200 Subject: Pitiful benchmark performance References: Message-ID: <00ef01c0f28f$8ed9d6e0$980cee82@it.uu.se> > Sean Hinde wrote: > > > > Now I know that benchmarks are not representative etc etc but they do > > encapsulate quite a few common idioms and there shouldn't be > > a good reason > > for Erlang to do so badly. Maybe the HIPE guys can use (are > > using?) these in > > their investigations into optimal GC schemes etc for R8? I think there are problems with the benchmark that casues the huge need of memory, and it is not certain that a bettter GC would make the problem go away. Still all benchmarks are interesting to us so we might steal some of them... A nice thing though is that Erlang behaves quite well on some benchmarks, for example nested loops, where it is one of the fastest non-natively compiled systems. (And HiPE is more than 6 times faster than BEAM on this benchmark putting Erlang in the ballpark of the statically typed smlnj.) /Erik From Sean.Hinde@REDACTED Mon Jun 11 18:27:04 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Mon, 11 Jun 2001 17:27:04 +0100 Subject: Pitiful benchmark performance Message-ID: <402DD461F109D411977E0008C791C312039F6046@imp02mbx.one2one.co.uk> > > Sean Hinde wrote: > > > > > > Now I know that benchmarks are not representative etc etc > but they do > > > encapsulate quite a few common idioms and there shouldn't be > > > a good reason > > > for Erlang to do so badly. Maybe the HIPE guys can use (are > > > using?) these in > > > their investigations into optimal GC schemes etc for R8? > > I think there are problems with the benchmark that casues the > huge need of memory, and it is not certain that a bettter GC > would make the problem go away. Still all benchmarks are > interesting to us so we might steal some of them... It would be nice to see Erlang up there in the top few. Personally I don't see what is wrong with the code for the word count benchmark - it doesn't seem to be breaking any obvious Erlang no-no. Hmmm. > A nice thing though is that Erlang behaves quite well on some > benchmarks, for example nested loops, where it is one of the > fastest non-natively compiled systems. (And HiPE is more than > 6 times faster than BEAM on this benchmark putting Erlang in > the ballpark of the statically typed smlnj.) That's good to hear. Looking forward to R8. - Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From jamesh@REDACTED Mon Jun 11 19:38:33 2001 From: jamesh@REDACTED (James Hague) Date: Mon, 11 Jun 2001 12:38:33 -0500 Subject: Pitiful benchmark performance Message-ID: > > A nice thing though is that Erlang behaves quite well on some > > benchmarks, for example nested loops, where it is one of the > > fastest non-natively compiled systems. (And HiPE is more than > > 6 times faster than BEAM on this benchmark putting Erlang in > > the ballpark of the statically typed smlnj.) > > That's good to hear. Looking forward to R8. Is HiPE part of R8? From Sean.Hinde@REDACTED Mon Jun 11 19:50:02 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Mon, 11 Jun 2001 18:50:02 +0100 Subject: Pitiful benchmark performance Message-ID: <402DD461F109D411977E0008C791C312039F6048@imp02mbx.one2one.co.uk> I heard somewhere that the next version of Hipe would be coming out later this year. I guess I put 2 and 2 together and made 20! Sean > -----Original Message----- > From: James Hague [mailto:jamesh@REDACTED] > Sent: 11 June 2001 18:39 > To: 'Erlang Questions' > Subject: RE: Pitiful benchmark performance > > > > > A nice thing though is that Erlang behaves quite well on some > > > benchmarks, for example nested loops, where it is one of the > > > fastest non-natively compiled systems. (And HiPE is more than > > > 6 times faster than BEAM on this benchmark putting Erlang in > > > the ballpark of the statically typed smlnj.) > > > > That's good to hear. Looking forward to R8. > > Is HiPE part of R8? > NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From kostis@REDACTED Mon Jun 11 20:09:12 2001 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 11 Jun 2001 20:09:12 +0200 (MET DST) Subject: Pitiful benchmark performance In-Reply-To: Mail from 'Sean Hinde ' dated: Mon, 11 Jun 2001 18:50:02 +0100 Message-ID: <200106111809.UAA13078@harpo.it.uu.se> James Hague asked: > > > A nice thing though is that Erlang behaves quite well on some > > > benchmarks, for example nested loops, where it is one of the > > > fastest non-natively compiled systems. (And HiPE is more than > > > 6 times faster than BEAM on this benchmark putting Erlang in > > > the ballpark of the statically typed smlnj.) > > > > That's good to hear. Looking forward to R8. > > Is HiPE part of R8? FIY, the HiPE group and members of the Erlang/OTP team held a very loooong meeting today and among other things, the following were decided: - If the Erlang/OTP team decides to release a pre-release of R8 during the summer (quite probable), HiPE will also be part of this pre-release. The supported platform will be SPARC *only*. - HiPE will be part of R8 (coming out sometime in September or so). The supported platform (for HiPE) will be the SPARC architecture and there one can observe the speedups reported by Erik Johansson in his mail. In addition, R8 might also include an x86 port of HiPE (which as of yesterday is passing the testsuite, but is currently quite far from generating code which is significantly faster than BEAM). Hope this info is interesting, Kostis Sagonas (for the HiPE team). From happi@REDACTED Mon Jun 11 21:44:41 2001 From: happi@REDACTED (Happi) Date: Mon, 11 Jun 2001 21:44:41 +0200 Subject: Pitiful benchmark performance References: <200106111809.UAA13078@harpo.it.uu.se> Message-ID: <001d01c0f2ae$f61c8770$c90b0a0a@LISA> Kostis Sagonas wrote: > FIY, the HiPE group and members of the Erlang/OTP team held a very > loooong meeting today and among other things, the following were > decided: > > - If the Erlang/OTP team decides to release a pre-release of R8 > during the summer (quite probable), HiPE will also be part of > this pre-release. The supported platform will be SPARC *only*. > > - HiPE will be part of R8 (coming out sometime in September or so). > The supported platform (for HiPE) will be the SPARC architecture > and there one can observe the speedups reported by Erik Johansson > in his mail. Just to clearify: HiPE will be an *unsupported* part of R8, that is, OTP will not support it. Use it at your own risk, and your own reward ;) (Hopefully, it will be fully supporten in R9.) One goal with this release is to get a user-base who can give us feedback, suggestions for improvements, and above all, realistic benchmarks. /Erik Johansson From kostis@REDACTED Mon Jun 11 21:59:03 2001 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 11 Jun 2001 21:59:03 +0200 (MET DST) Subject: Pitiful benchmark performance In-Reply-To: Mail from '"Happi" ' dated: Mon, 11 Jun 2001 21:44:41 +0200 Message-ID: <200106111959.VAA15586@harpo.it.uu.se> Erik Johansson wrote: > Just to clearify: HiPE will be an *unsupported* part of R8, that is, OTP > will not support it. Yes, but it will be supported by us though, right Erik ;) /Kostis. From complog@REDACTED Tue Jun 12 00:00:20 2001 From: complog@REDACTED (Logic Programming Rsrch Association) Date: Mon, 11 Jun 2001 16:00:20 -0600 Subject: CFP: CICLOPS 2001 Message-ID: <200106112200.f5BM0K503866@pippo.cs.nmsu.edu> ======================================================================== FIRST call for papers FIRST call for papers FIRST call for papers We apologize for multiple copies of this mail / posting ======================================================================== Post-Conference Workshop CICLOPS 2001 Colloquium on Implementation of Constraint and LOgic Programming Systems held in conjunction with the International Conference on Logic Programming Paphos, Cyprus December 1, 2001 ======================================================================== see also: http://www.cs.nmsu.edu/~complog/conferences/iclp01 ======================================================================== This workshop is a continuation of the series of workshops on Implementations of Logic Programming Systems, previously held with considerable success in Budapest (1993), Santa Margherita (1994) and Ithaca (1994), as well as the Compulog Net workshops on Parallelism and Implementation Technologies held in Madrid (1993-4), Utrecht (1995), Bonn (1996), Port Jefferson (1997), Manchester (1998), Las Cruces (1999), and London (2000). This year, the event will join forces with TRICS (Techniques for Implementing Constraint Programming Systems), previously held in Singapore (2000), thus providing coverage to implementation technology for both Logic and Constraint Programming. The last years have seen continuous progress in the computing technology available both for the academic and commercial computing environments. Examples include improved processor performance, increased memory capacity and bandwidth, faster networking technology, and Operating System support for cluster computing. Combined with recent advances in compilation technologies, these improvements are causing high-level languages to be regarded as good candidates for programming complex, real world applications. Logic Programming and Constraint Programming, in particular, seem to offer one of the best alternatives, as they couple a very high level of abstraction and a declarative nature with an extreme flexibility in the design of its execution model. The main intent of this workshop is to bring together, in an informal and friendly setting, key researchers on implementation technologies for logic and constraint-based languages and systems, in order to promote a much needed exchange of ideas and feedback on recent developments. The workshop is focused on design and implementation of logic and constraint programming systems whether sequential, parallel, or concurrent. Preference will be given to the analysis and description of implemented systems (or systems currently under implementation) and their associated techniques, problems found in their development or design, and steps taken towards the solution of these problems. Suggested topics of interest include, but are not limited to: - standard and non-standard implementation schemes e.g. modification of WAM, translation to C, native-code generation etc. - optimizing compilers and static analysis - parallelism and concurrency - techniques for the implementation of different extensions of logic and constraint programming, e.g., multi-paradigm languages and systems, non-monotonic reasoning systems, inductive logic programming - benchmarking and performance evaluation - Software design aspects of LP/CP systems: components, patterns, etc. - memory management and garbage collection - programming environments - tools for internet applications - experiences from using systems in real-life applications Contributions: -------------- Authors are invited to submit papers written in English and not exceeding 12 pages (10pts Font, letter-size paper, 1 inch margin). To speed up the process of refereeing, authors are requested to submit their paper in Postscript or PDF form by electronic mail to the workshop coordinator (e-mail address below). Conventional paper copies may be sent to the contact address below only if access to electronic media is not available. Submissions should contain full return mail and email address (if applicable), and FAX number (if applicable) of the contact author. Prospective authors are kindly requested to first send an indication of interest together with a paper title to the organizers. Deadline for submissions is September 5, 2001. Authors submitting by electronic mail will receive an acknowledgement (also by electronic mail) within 2-3 days. Accepted papers: ---------------- Submitted papers will be reviewed by at least two referees. Authors will be notified of the acceptance of their papers by September 23, 2001. To be included in the workshop proceedings (to be distributed to the workshop participants), revised versions of the accepted papers must be sent no later than October 5, 2001. Electronic versions of the accepted papers will be also made available through the workshop's homepage. Workshop Chairs: ---------------- I. Dutra (Federal University of Rio de Janeiro, Brazil) M. Henz (National University of Singapore, Singapore) Program committee: ------------------ M. Carro (Politechnic Univ. of Madrid, Spain) M. Carlsson (SICS, Sweden) B. Demoen (K.U. Leuven, Belgium) G. Gupta (Univ. of Texas at Dallas, USA) F. Laburthe (Bouygues E-lab, France) I. Niemela (Helsinki University of Technology, Finland) E. Pontelli (New Mexico State University, USA) V. Santos Costa (Federal University of Rio de Janeiro, Brazil) C. Schulte (Universitat des Saarlandes, Germany) F. Silva (Univ. of Porto, Portugal) Important dates: ---------------- Indication of interest: as soon as possible Submission of papers: September 5, 2001 Notification of authors: September 23, 2001 Final version sent by: October 5, 2001 ICLP'01 Conference: Nov. 26 - Dec. 1, 2001 Workshop: December 1, 2001 Contact address for Workshop Coordinator: ----------------------------------------- Enrico Pontelli Dept. Computer Science New Mexico State University Box 30001/CS, Las Cruces, NM 88003 Phone: +1 (505) 646-6239 FAX: +1 (505) 646-1002 email: epontell@REDACTED Workshop Web Site: ------------------ http://www.cs.nmsu.edu/~complog/conferences/iclp01 ======================================================================== From selwyn.dhinakar@REDACTED Tue Jun 12 05:23:59 2001 From: selwyn.dhinakar@REDACTED (Jacob.S.Dhinakar) Date: Tue, 12 Jun 2001 08:53:59 +0530 Subject: Hello Message-ID: <004c01c0f2ef$1eec92e0$2b077aa3@tatainfotech.com> Hello I am Dhinakar Jacob, Aerospace research student of Indian Institute of Science, Bangalore India. I need to design the parser for PL/1 language to C++. So if I have to do that what are all the basic things ( knowledge ) I need?? Can you help me in this regard....What are all the basic steps I got thru your web site it was very nice. But to design the parser what are all the steps I have to follow... Please let me know... My mail id : jacobs99@REDACTED www.dhinakarjacob.com With regards Jacob -------------- next part -------------- An HTML attachment was scrubbed... URL: From eleberg@REDACTED Tue Jun 12 13:38:57 2001 From: eleberg@REDACTED (Bengt Kleberg) Date: Tue, 12 Jun 2001 13:38:57 +0200 (MET DST) Subject: sae, io:format(), undefined Message-ID: <200106121138.NAA25550@avc280.etxb.ericsson.se> Greeitings, When creating a sae program doing io:format("hello, world") I get an error message when running: undef[{ io, format, 2}, ... The supplied example hello.erl program works. It uses erlang:display() instead of io:format(). Should I have added the io module to elink? Just doing elink -o hello -m hello io -s hello get_args does not work. Moreover, my original 'hello, world' program had a 0 arity entry function. This was not allowed by elink. Perhaps the argument of the entry function (it is a list, isn't it?) should be documented? Best Wishes, Bengt Kleberg From alexgc@REDACTED Tue Jun 12 14:12:52 2001 From: alexgc@REDACTED (alex) Date: Tue, 12 Jun 2001 14:12:52 +0200 Subject: Problems with ASN1 v1.3 (Erlang/OTP R7B-3) Message-ID: <20010612141252.A27901@bulma.dc.fi.udc.es> Hello everybody. I've been compiling an ASN1 schema without troubles with Erlang/OTP R7B-2 (ASN1 version 1.2.9.6); yesterday I downloaded the patch (R7B-2 to -3 patch), and compiled erlang. Now the new ASN1 compiler (version 1.3) can't compile the schema. The problem seems to be the COMPONETS tag: --- $ erlc -bber Ldap.asn Erlang ASN.1 version "1.3" compiling "./Ldap.asn" Compiler Options: [fast, ber, report_errors, {cwd,'./'}, {outdir,"./"}] Ldap.asn:syntax error at line 144: got 'COMPONENTS' expected '}' error --- I deleted the lines with this tag and it compiled. I'm using LDAP ASN1 specification, taken from RFC2551. Any suggestions? Thank you in advance. From Sean.Hinde@REDACTED Tue Jun 12 18:53:48 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Tue, 12 Jun 2001 17:53:48 +0100 Subject: Pitiful benchmark performance Message-ID: <402DD461F109D411977E0008C791C312039F604E@imp02mbx.one2one.co.uk> > I think there are problems with the benchmark that casues the > huge need of memory, and it is not certain that a bettter GC > would make the problem go away. Still all benchmarks are > interesting to us so we might steal some of them... I've had a better look at the Wordfreq benchmark: Most of the cpu time seems to be taken up in reading the file one line at a time rather than in the 4k chunks allowed in the test and used by all the other solutions. This appears to be unavoidable using the port mechanism to access stdin - the only options are to read one line at a time (up to a max per line - {line, 512}) or read the whole stream in one go (thus breaking the rules, but if done makes Erlang one of the fastest!). Does anyone know of a way to read from stdin in 4k chunks? No worries if not as I don't imagine any real erlang application uses this mechanism. Memory usage is interesting with this one. The memory is all grabbed by the runtime after reading three or four lines of the input file at the very beginning of the run. On my SPARC machine the beam process leaps to use 80M almost straight away and then never uses any more. The erlang thread grows almost immediately to 22M (as reported by erlang:process_info(self(), memory) ). This suggests that the Erlang runtime is perhaps being a little over eager in reserving memory in case it will be needed in the future. It is also not very clear where the other 60M has gone.. Maybe in the driver itself? I'd never be in favour of anything which slowed the whole thing down by doing lots of tiny memory allocations but maybe the balance is not quite right at the moment (stop press..) I just tried it on the latest patch release and the behaviour is somewhat improved :) The beam process only gets up to 24M with the erlang thread using 22M of that. The new dlmalloc allocator seems to be much more effective at controlling memory usage - though I still don't see why the erlang thread gets so big so quickly. The size of the input file is 1.7M.. Any ideas anyone? Is the thread memory behaviour by design? Also using the new erts the benchmark runs a bit slower: New: time /opt/erlang/5.0.2.5/erl -noinput -s wordfreq main < Input > output real 0m18.421s user 0m15.140s sys 0m3.110s Old: time /opt/erlang/5.0.2.1/erl -noinput -s wordfreq main < Input > output real 0m15.378s user 0m14.450s sys 0m0.550s lots more time spent in sys.. an inevitable consequence of reducing memory consumption so drastically? Or something else? Another slightly less scientific test using the Echo Client/Server sockets benchmark. It runs about the same speed under Solaris and NT on approximately the same speed machines.. apparently not an OS issue. Cheers, Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From Sean.Hinde@REDACTED Fri Jun 15 00:13:38 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Thu, 14 Jun 2001 23:13:38 +0100 Subject: Interesting benchmark performance (was RE: Pitiful benchmark perf ormance) Message-ID: <402DD461F109D411977E0008C791C312039F6066@imp02mbx.one2one.co.uk> Hi, > What would happen if you e.g. change INPUT_REDUCTIONS in erl_vm.h > from (2* CONTEXT_REDS) to CONTEXT_REDS, doubling the I/O poll > frequency? (CONTEXT_REDS is set to 1000) > > If basically all that happens in the erlang vm is that some > process is waiting for I/O, ports may not be polled with any > predictably high frequency (then again, this is only guessing > wildly. Perhaps it effectively gives the ports even higher > priority...) I didn't try this but did get the benchmark to spawn an additional process to do this: busy() -> busy(). The time to complete the benchmark came down to: real 0m34.032s user 0m22.190s sys 0m11.640s from: real 0m45.845s user 0m32.910s sys 0m11.790s while also completing some huge amount of additional work.. We have a number of machines working as pure mnesia servers running small transactions - pretty much exactly what this benchmark is simulating. I will do some more testing but it looks like in this sort of OLTP type applications where there is little else going on in the emulator except - read message from tcp/ip, do small work, wait for next message there might be a useful boost in throughput available. More thoughts anyone? - Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From etxuwig@REDACTED Fri Jun 15 09:29:30 2001 From: etxuwig@REDACTED (Ulf Wiger) Date: Fri, 15 Jun 2001 09:29:30 +0200 (MET DST) Subject: Interesting benchmark performance (was RE: Pitiful benchmark perf ormance) In-Reply-To: <402DD461F109D411977E0008C791C312039F6066@imp02mbx.one2one.co.uk> Message-ID: This was interesting. Erlang strikes me as more and more human. ;) I seem to function in the same way: when I have lots to do, I become more efficient; when I have little to do, I become very inefficient. Seriously, Erlang *has been* highly optimized for the very type of systems where there are lots of things going on at once. This reflects on the I/O, for example, where Erlang tries to handle up to hundreds of active I/O ports fairly, rather than handling only one extremely well. Perhaps for example the ETOS implementation could focus more squarely on efficiently running applications with much less concurrency? It seems to be very good at that already. /Uffe On Thu, 14 Jun 2001, Sean Hinde wrote: >Hi, > >> What would happen if you e.g. change INPUT_REDUCTIONS in erl_vm.h >> from (2* CONTEXT_REDS) to CONTEXT_REDS, doubling the I/O poll >> frequency? (CONTEXT_REDS is set to 1000) >> >> If basically all that happens in the erlang vm is that some >> process is waiting for I/O, ports may not be polled with any >> predictably high frequency (then again, this is only guessing >> wildly. Perhaps it effectively gives the ports even higher >> priority...) > >I didn't try this but did get the benchmark to spawn an additional process >to do this: > >busy() -> > busy(). > >The time to complete the benchmark came down to: > >real 0m34.032s >user 0m22.190s >sys 0m11.640s > >from: > >real 0m45.845s >user 0m32.910s >sys 0m11.790s > >while also completing some huge amount of additional work.. > >We have a number of machines working as pure mnesia servers running small >transactions - pretty much exactly what this benchmark is simulating. I will >do some more testing but it looks like in this sort of OLTP type >applications where there is little else going on in the emulator except - >read message from tcp/ip, do small work, wait for next message there might >be a useful boost in throughput available. > >More thoughts anyone? > >- Sean > > > > >NOTICE AND DISCLAIMER: >This email (including attachments) is confidential. If you have received >this email in error please notify the sender immediately and delete this >email from your system without copying or disseminating it or placing any >reliance upon its contents. We cannot accept liability for any breaches of >confidence arising through use of email. Any opinions expressed in this >email (including attachments) are those of the author and do not necessarily >reflect our opinions. We will not accept responsibility for any commitments >made by our employees outside the scope of our business. We do not warrant >the accuracy or completeness of such information. > > -- Ulf Wiger tfn: +46 8 719 81 95 Senior System Architect mob: +46 70 519 81 95 Strategic Product & System Management ATM Multiservice Networks Data Backbone & Optical Services Division Ericsson Telecom AB From denicola@REDACTED Fri Jun 15 09:37:38 2001 From: denicola@REDACTED (Rocco De Nicola) Date: Fri, 15 Jun 2001 09:37:38 +0200 Subject: PLI 2001 in Firenze Message-ID: CALL FOR PARTICIPATION AND EARLY REGISTRATION PLI 2001 Principles, Logics, and Implementations of high-level programming languages Firenze, ITALY September 2 - 8, 2001 http://music.dsi.unifi.it/pli01/ The colloquium on Principles, Logics, and Implementations of high- level programming languages is a collection of events aimed at the advancement of high-level programming languages. PLI 2001 includes the following conferences and workshops: ACM Sponsored Conferences: ICFP (September 3-5) Int. Conf. on Functional Programming General chair: Benjamin Pierce (Univ. Pennsylvania) Program chair: Xavier Leroy (INRIA Rocquencourt) Invited speakers: To be announced PPDP (September 5-7) Int. Conf. on Principles and Practice of Declarative Programming Conference chair: Rocco De Nicola (Univ. Firenze) Program chair: Harald S?ndergaard (Univ. Melbourne) Invited speakers: J. Esparza, A. Gordon, and D.A. Schmidt. ACM Sponsored Workshops: ? BABEL (Multi-language Infrastructure and Interoperability) ? HASKELL ? QAPL (Quantitative Aspects of Programming Languages) ? RULE (Rule-Based Programming) ? SAIG (Semantics, Applications, and Implementation of Program Generation) ? SCHEME (Scheme and Functional Programming) ? VCL (Verification and Computational Logic) Co-located Workshops: ? ERLANG ? FICS (Fixed Points in Computer Science) A detailed presentation of PLI2001, including schedule of events, travel and tourist information, is available at the WEB page http://music.dsi.unifi.it/pli01/. Registration and accommodation information and forms are available at http://music.dsi.unifi.it/pli01/registration/ Early registration rates apply until July 25. For informations about hotels please contact (mentioning PLI 2001) Giubbi Jet di Volo Viaggi Piazza San Jacopino, 34/r - 50144 Firenze Telephone: +39 055 3249074 - +39 055 350577 Fax: +39 055 366807 E-mail: incoming@REDACTED For all other informations mail to pli-org@REDACTED ------------------------------------------------------------------- Firenze is packed in September; do book accommodation as soon as possible. ------------------------------------------------------------------- -- <><><><><><><><><><><><><><><><><><><><><><><><><><> Prof. Rocco De Nicola Dip. Sistemi e Informatica Univ. di Firenze Via C. Lombroso 6/17 I-50134 FIRENZE (ITALY) tel. +39 055 4796733 fax +39 055 4796730 Web Page: http://www.dsi.unifi.it/~denicola/ <><><><><><><><><><><><><><><><><><><><><><><><><><> From mickael.remond@REDACTED Fri Jun 15 14:45:51 2001 From: mickael.remond@REDACTED (Mickael Remond) Date: Fri, 15 Jun 2001 14:45:51 +0200 Subject: Inets in R7B-3 Message-ID: <20010615144551.C23835@idealx.com> Hello, I encoutered problems with the new Inets version. What I discovered is that the new directive: MaxHeaderSize and MaxBodySize are mandatory in the new inets config file. Thus you need to change your old config file to put this two parameters. If they happens to be set two small or to be missing in the config file the requested URL will not be served by the Inets server. Moreover, the message in the log file is malformed and appears as a list, making the problem difficult to found. Would it be possible to set the two parameters above with a conservative default value if they are missing in the config file ? Thank you for your help. -- Micka?l R?mond From enano@REDACTED Fri Jun 15 18:00:52 2001 From: enano@REDACTED (Miguel Barreiro Paz) Date: Fri, 15 Jun 2001 18:00:52 +0200 (CEST) Subject: Inets in R7B-3 In-Reply-To: <20010615144551.C23835@idealx.com> Message-ID: Hi, > I encoutered problems with the new Inets version. > What I discovered is that the new directive: > MaxHeaderSize and MaxBodySize are mandatory in the new inets config > file. Well, it seems to work OK without them, but the default MaxHeaderSize value is 256 bytes IIRC, so the default headers sent by netscape for instance are far too long. I faced this same problem yesterday, after updating a machine to R7B-3; netscape was unable to retrieve web pages, but links was. Fortunately, tcpdump is your friend :) > Would it be possible to set the two parameters above with a conservative > default value if they are missing in the config file ? I second this. Regards, Miguel From spearce@REDACTED Fri Jun 15 23:33:44 2001 From: spearce@REDACTED (Shawn Pearce) Date: Fri, 15 Jun 2001 17:33:44 -0400 Subject: Using Erlang as a web stress tool Message-ID: <20010615173344.B3055@spearce.org> I'm considering building a web application testing tool using Erlang. What I am thinking is, modeling each simulated user as a Erlang process, and each browser connection as a Erlang process. Thus for 400 simulated users simulating Netscape we have 2000 processes. (4 browser connections per user.) Now since I have a quad CPU machine, I would most likely split these evenly into 4 nodes, leaving 500 processes per node. Most likely easy work for Erlang. The question is, could a single Erlang node handle 400 concurrent TCP/IP connections to the server(s) being tested? Does anyone think that I'm better off doing this in C/C++ where I have direct access to poll (and friends)? And the second question is, if I'm in Erlang, will using binaries slow me down due to the shear number of binaries being created and released from network traffic? I realize all of the binaries reside in the same memory heap, and aside from passing the data from the network socket to the browser connection process, the data won't get moved around anywhere. We're talking doing 100-200 HTTP requests per second by each node (making 400-800 total for the machine). Some of those requests will be able to recycle the TCP stream via HTTP KeepAlive, others won't. But at least 100 new TCP streams will be created (and closed) per second. BTW, the Erlang nodes are most likely to run on a Windows NT 4.0 Server system, in case that makes any difference. Just thought I'd ask. :-) -- Shawn. ``If this had been a real life, you would have received instructions on where to go and what to do.'' From svg@REDACTED Fri Jun 15 19:05:07 2001 From: svg@REDACTED (Vladimir Sekissov) Date: Fri, 15 Jun 2001 23:05:07 +0600 Subject: ANN: Dia EML diagrams plug-in (now for Windows) Message-ID: <20010615230507N.svg@disney.surnet.ru> EML diagrams plug-in now included in official Dia distribution (http://www.lysator.liu.se/~alla/dia). Windows version can be downloaded from http://hans.breuer.org/dia/ Send me your comments and suggestions. Best Regards, Vladimir Sekissov From Sean.Hinde@REDACTED Mon Jun 18 01:05:15 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Mon, 18 Jun 2001 00:05:15 +0100 Subject: Interesting benchmark performance (was RE: Pitiful benchmark perf ormance) Message-ID: <402DD461F109D411977E0008C791C312039F606B@imp02mbx.one2one.co.uk> > This was interesting. > > Erlang strikes me as more and more human. ;) > I seem to function in the same way: when I have lots to do, I > become more efficient; when I have little to do, I become very > inefficient. I too have this same scheduling quirk, although I would suggest the mechanism is not quite the same as the Erlang runtime one.. > Seriously, Erlang *has been* highly optimized for the very type > of systems where there are lots of things going on at once. This > reflects on the I/O, for example, where Erlang tries to handle up > to hundreds of active I/O ports fairly, rather than handling only > one extremely well. It says to me rather that Erlang is optimised for applications which are sufficiently complex that at least 2000 reductions are required following each external input or timeout. It seems to have little to do with just general stuff going on, and that for many applications Erlang would be otherwise superb at (web server, Mnesia based Online Transaction Processing server) there is a significant penalty. A small change to the runtime could mean that I don't have to run a busy loop in the background to get better performance out of my remote mnesia servers.. - Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From Sean.Hinde@REDACTED Mon Jun 18 12:16:45 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Mon, 18 Jun 2001 11:16:45 +0100 Subject: Using Erlang as a web stress tool Message-ID: <402DD461F109D411977E0008C791C312039F606D@imp02mbx.one2one.co.uk> > > I'm considering building a web application testing tool using Erlang. Great! > What I am thinking is, modeling each simulated user as a Erlang > process, and each browser connection as a Erlang process. Thus for > 400 simulated users simulating Netscape we have 2000 processes. (4 > browser connections per user.) > > Now since I have a quad CPU machine, I would most likely split these > evenly into 4 nodes, leaving 500 processes per node. Most likely easy > work for Erlang. Yes, we use a similar configuration in a couple of our systems. > The question is, could a single Erlang node handle 400 concurrent > TCP/IP connections to the server(s) being tested? Does anyone think > that I'm better off doing this in C/C++ where I have direct access to > poll (and friends)? The answer to this is abslutely yes! The situation where there appears to be some delay in responding to inputs is restricted to applications where not much is going on and the emulator seems to wait up to a millisecond (!) before doing poll or select again. You would not notice this at all in your application - it will appear to fly.. > And the second question is, if I'm in Erlang, will using binaries slow > me down due to the shear number of binaries being created and released > from network traffic? I realize all of the binaries reside > in the same > memory heap, and aside from passing the data from the network > socket to > the browser connection process, the data won't get moved around > anywhere. Binaries are extremely efficient in Erlang. You should be able to easily saturate your LAN connection by pumping binaries out of an Erlang node. > We're talking doing 100-200 HTTP requests per second by each node > (making 400-800 total for the machine). Some of those requests will > be able to recycle the TCP stream via HTTP KeepAlive, others > won't. But > at least 100 new TCP streams will be created (and closed) per second. This sounds pretty reasonable. It should be quite easy to mock up a quick test arrangement and see just how fast you can make it go. > BTW, the Erlang nodes are most likely to run on a Windows NT > 4.0 Server > system, in case that makes any difference. I've found broadly similar performance between equivalent processor speeds across NT and Solaris. > Just thought I'd ask. :-) Ask away, and enjoy! - Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From thomasl@REDACTED Mon Jun 18 15:15:58 2001 From: thomasl@REDACTED (Thomas Lindgren) Date: Mon, 18 Jun 2001 12:15:58 -0100 Subject: Interesting benchmark performance (was RE: Pitiful benchmark perf ormance) In-Reply-To: <402DD461F109D411977E0008C791C312039F606B@imp02mbx.one2one.co.uk> (message from Sean Hinde on Mon, 18 Jun 2001 00:05:15 +0100) References: <402DD461F109D411977E0008C791C312039F606B@imp02mbx.one2one.co.uk> Message-ID: <200106181315.f5IDFwo01237@lammgam.bluetail.com> > It says to me rather that Erlang is optimised for applications which are > sufficiently complex that at least 2000 reductions are required following > each external input or timeout. It seems to have little to do with just > general stuff going on, and that for many applications Erlang would be > otherwise superb at (web server, Mnesia based Online Transaction Processing > server) there is a significant penalty. Maybe the time has come to adaptively adjust the number of reductions before yielding (ie, rescheduling)? Here is a portable, straightforward approach: instead of compiling in a constant number of reductions, the number of remaining reductions should be loaded from the process structure when checking for yielding. The interesting part, then, is deciding how many reductions you get when you're scheduled. A simple approach is to permit the system to set the reductions-per-yield at runtime (per process or for the entire node), by using a BIF. But this must be supplemented by some way to measure activity, so that the decision can be made systematically. (Alternatively, one could take the approach that reductions-per-yield is set _only_ inside the runtime, to avoid messing around with BIFs.) A second, orthogonal, topic to consider is how well a "reduction" corresponds to a time tick. A reduction can vary quite a bit in the amount of time it requires, because of BIFs: today, there are ways to bump reductions when a long-running BIF begins. Another approach to yielding might be to measure the available time slice in hardware cycles, rather than procedure calls. All desktop processors have cycle counters, for example, so it is viable for a wide range of systems. Unfortunately, the counters are often somewhat messy to work with. Thomas -- Thomas Lindgren thomas+junk@REDACTED Alteon WebSystems From etxuwig@REDACTED Mon Jun 18 13:41:40 2001 From: etxuwig@REDACTED (Ulf Wiger) Date: Mon, 18 Jun 2001 13:41:40 +0200 (MET DST) Subject: Interesting benchmark performance (was RE: Pitiful benchmark perf ormance) In-Reply-To: <200106181315.f5IDFwo01237@lammgam.bluetail.com> Message-ID: Another option could be to allow "high-priority" ports, that are polled e.g. just before checking the normal priority process queue -- or just after, to reduce the starvation risk. Not that I'm against a time- or cycle-based scheduling model. /Uffe On Mon, 18 Jun 2001, Thomas Lindgren wrote: >Maybe the time has come to adaptively adjust the number of >reductions before yielding (ie, rescheduling)? > >Here is a portable, straightforward approach: instead of >compiling in a constant number of reductions, the number of >remaining reductions should be loaded from the process >structure when checking for yielding. > >The interesting part, then, is deciding how many reductions you >get when you're scheduled. A simple approach is to permit the >system to set the reductions-per-yield at runtime (per process >or for the entire node), by using a BIF. But this must be >supplemented by some way to measure activity, so that the >decision can be made systematically. (Alternatively, one could >take the approach that reductions-per-yield is set _only_ >inside the runtime, to avoid messing around with BIFs.) > >A second, orthogonal, topic to consider is how well a >"reduction" corresponds to a time tick. A reduction can vary >quite a bit in the amount of time it requires, because of BIFs: >today, there are ways to bump reductions when a long-running >BIF begins. > >Another approach to yielding might be to measure the available >time slice in hardware cycles, rather than procedure calls. All >desktop processors have cycle counters, for example, so it is >viable for a wide range of systems. Unfortunately, the counters >are often somewhat messy to work with. > > Thomas > -- Ulf Wiger tfn: +46 8 719 81 95 Senior System Architect mob: +46 70 519 81 95 Strategic Product & System Management ATM Multiservice Networks Data Backbone & Optical Services Division Ericsson Telecom AB From Sean.Hinde@REDACTED Mon Jun 18 14:50:37 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Mon, 18 Jun 2001 13:50:37 +0100 Subject: Interesting benchmark performance (was RE: Pitiful benchmark perf ormance) Message-ID: <402DD461F109D411977E0008C791C312039F606F@imp02mbx.one2one.co.uk> > Maybe the time has come to adaptively adjust the number of reductions > before yielding (ie, rescheduling)? Perhaps :) > Here is a portable, straightforward approach: instead of compiling in > a constant number of reductions, the number of remaining reductions > should be loaded from the process structure when checking for > yielding. As far as I can make out the current behaviour based on number of reductions generally works very well in the situation where each time the system is called upon to do some work there are at least 2000 to be done. One possible improvement (which I mailed before but it got lost) could be: The main emulator loop appears to be the code at the very end of sys.c. This calls schedule() and waits for it to return before any i/o is checked. schedule() in erl_process.c will *only* return when INPUT_REDUCTIONS have all been used up. As far as I can make out the only reason the echo/server benchmark doesn't deadlock is that there is enough going on in the background in an otherwise quiescent system to eventually use up the 2000 function calls required to check i/o. Maybe schedule() might be modified so that in addition to returning after INPUT_REDUCTIONS it keeps track of activity in each pass through all processes and returns if there has been no activity detected for a whole pass. This should ensure responsiveness is not compromised while not adding significantly to the context switching load. Since then I have tried to make the echo server benchmark run faster by adjusting the INPUT_REDUCTIONS and CONTEXT_REDS values and recompiling the emulator with absolutely no success. The only thing I have found to work is running a busy loop in the background!!! > The interesting part, then, is deciding how many reductions you get > when you're scheduled. A simple approach is to permit the system to > set the reductions-per-yield at runtime (per process or for the entire > node), by using a BIF. But this must be supplemented by some way to > measure activity, so that the decision can be made systematically. > (Alternatively, one could take the approach that reductions-per-yield > is set _only_ inside the runtime, to avoid messing around with BIFs.) The simplest method could be to make it a beam startup parameter rather than a constant. Then for weird applications the value could easily be tuned for max performance. > A second, orthogonal, topic to consider is how well a "reduction" > corresponds to a time tick. A reduction can vary quite a bit in the > amount of time it requires, because of BIFs: today, there are ways to > bump reductions when a long-running BIF begins. > > Another approach to yielding might be to measure the available time > slice in hardware cycles, rather than procedure calls. All desktop > processors have cycle counters, for example, so it is viable for a > wide range of systems. Unfortunately, the counters are often somewhat > messy to work with. It could be an interesting excercise. There appears to have been a large amout of effort put into all the BIFs to try to make them as fair as possible. Possibly this is heavier than making them all interruptible based on a time slice?? Still perplexed - Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From thomasl@REDACTED Mon Jun 18 18:23:16 2001 From: thomasl@REDACTED (Thomas Lindgren) Date: Mon, 18 Jun 2001 15:23:16 -0100 Subject: Interesting benchmark performance (was RE: Pitiful benchmark perf ormance) In-Reply-To: <402DD461F109D411977E0008C791C312039F606F@imp02mbx.one2one.co.uk> (message from Sean Hinde on Mon, 18 Jun 2001 13:50:37 +0100) References: <402DD461F109D411977E0008C791C312039F606F@imp02mbx.one2one.co.uk> Message-ID: <200106181623.f5IGNGe01559@lammgam.bluetail.com> > The simplest method could be to make it a beam startup parameter rather than > a constant. Then for weird applications the value could easily be tuned for > max performance. This could be sufficient if the system behaves in roughly the same way all the time (eg, amount of concurrency or I/O is roughly the same), though tuning (and possibly retuning) the system could be a chore if you have to restart it to change the magic number. (If performance is reasonably robust over #reductions, the need to change the number is probably small, though.) > It could be an interesting excercise. There appears to have been a large > amout of effort put into all the BIFs to try to make them as fair as > possible. Possibly this is heavier than making them all interruptible based > on a time slice?? Basically, the problem is that a reduction need not be well-correlated to a fixed elapsed time. When and if this is a problem, a cycle-based time slice could be (more) useful. Having BIFs being non-interruptible (or composed of "atomic pieces") is however a nice property, and I think we should keep it that way as far as possible. Basing rescheduling on elapsed cycles means we will at least switch "soon after" completing a long-running BIF, rather than completing a possibly large number of (costly?) reductions. Digression: one approach for dynamically modifying the schedule in regular Erlang could be the following. Assume we have a BIF CostlyOp(). We can then wrap it as: ... T1 = erlang:now(), CostlyOp(), T2 = erlang:now(), erlang:bump_reductions(time_to_reds(time_sub(T2,T1))), ... Where time_sub computes the elapsed time between T1 and T2, and time_to_reds/1 converts elapsed time into 'average reductions'. The drawbacks are code bloat, extra execution cost, imprecision (eg, no knowledge of how much time remains in this slice) and the need to manually place these wrappers. Thomas -- Thomas Lindgren thomas+junk@REDACTED Alteon WebSystems From per@REDACTED Mon Jun 18 15:28:48 2001 From: per@REDACTED (per@REDACTED) Date: Mon, 18 Jun 2001 15:28:48 +0200 (CEST) Subject: Interesting benchmark performance (was RE: Pitiful benchmark perf ormance) In-Reply-To: <402DD461F109D411977E0008C791C312039F606F@imp02mbx.one2one.co.uk> Message-ID: <200106181328.f5IDSmS12835@tordmule.bluetail.com> Sean Hinde wrote: >The main emulator loop appears to be the code at the very end of sys.c. This >calls schedule() and waits for it to return before any i/o is checked. > >schedule() in erl_process.c will *only* return when INPUT_REDUCTIONS have >all been used up. Of course not, that would be totally broken - when no processes are runnable, it will return immediately: case 0: /* No process at all */ return 0; This will happen when all processes are sitting in 'receive', which is in some sense the "normal" case. Note also that this will cause the subsequent poll()/select() to be *blocking* (until the next scheduled timeout, basically) instead of polling. >As far as I can make out the only reason the echo/server benchmark doesn't >deadlock is that there is enough going on in the background in an otherwise >quiescent system to eventually use up the 2000 function calls required to >check i/o. Unless your code is doing it, there is basically nothing "going on in the background". I.e. if there is nothing to do, the whole runtime will sit in a blocking poll()/select() call (you can easily verify this by using 'strace' or equivalent on the beam process), which should provide optimal responsiveness to I/O. So, while the improved benchmark performance with a background busy loop is "interesting", it seems to me that the explanations so far are on the wrong track - unfortunately I don't have a right track to suggest, but by all means keep looking.:-) I'm pretty sure it isn't due to any intentional optimization for a "busy system", though. --Per Hedeland per@REDACTED From Sean.Hinde@REDACTED Mon Jun 18 15:50:07 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Mon, 18 Jun 2001 14:50:07 +0100 Subject: Interesting benchmark performance (was RE: Pitiful benchmark perf ormance) Message-ID: <402DD461F109D411977E0008C791C312039F6070@imp02mbx.one2one.co.uk> > Of course not, that would be totally broken - when no processes are > runnable, it will return immediately: > > case 0: /* No process at all */ > return 0; Yes, I just didn't see it - I was looking for something which increased the reduction counter by a certain minimum each time and missed the obvious ;) > This will happen when all processes are sitting in 'receive', which is > in some sense the "normal" case. Note also that this will cause the > subsequent poll()/select() to be *blocking* (until the next scheduled > timeout, basically) instead of polling. Yes, I caught this - again is exactly the sort of thing one would expect in a nicely designed system. > Unless your code is doing it, there is basically nothing "going on in > the background". I.e. if there is nothing to do, the whole > runtime will > sit in a blocking poll()/select() call (you can easily verify this by > using 'strace' or equivalent on the beam process), which > should provide > optimal responsiveness to I/O. Totally agree. > So, while the improved benchmark performance with a > background busy loop > is "interesting", it seems to me that the explanations so far > are on the > wrong track - unfortunately I don't have a right track to suggest, but > by all means keep looking.:-) I'm pretty sure it isn't due to any > intentional optimization for a "busy system", though. I have tried the benchmark on R6B (sending the data as a list rather than a binary but otherwise the same) and get the following result: real 0m28.929s user 0m15.940s sys 0m11.820s This is much better than the best I managed with R7B. Maybe it is down to something in the new inet driver or driver framework. I'll pass this on to the OTP guys to see what they make of it. - Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From fritchie@REDACTED Mon Jun 18 17:10:09 2001 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 18 Jun 2001 10:10:09 -0500 Subject: Interesting benchmark performance (was RE: Pitiful benchmark perf ormance) In-Reply-To: Message of "Mon, 18 Jun 2001 13:41:40 +0200." Message-ID: <200106181510.f5IFAAg58213@snookles.snookles.com> >>>>> "uw" == Ulf Wiger writes: uw> Another option could be to allow "high-priority" ports, that are uw> polled e.g. just before checking the normal priority process queue uw> -- or just after, to reduce the starvation risk. There was a conversation not long ago over in comp.lang.functional about managing global state. One person noted that the latency to retrieve some "global" state from process S by process C can be a high-latency operation: it may be a while before S is scheduled to run, and when S sends its reply, it may be a while before C is scheduled to run again. A stray thought this weekend suggested a scheduling mechanism similar to doors, which are found in Spring, Solaris, and (I think) Linux. Doors have *very* low latency because the normal process scheduler is short-circuited: the door server is scheduled to run immediately after the client executes door_call(), and the client is rescheduled immediately after the server executes door_return(). From a wall-clock point of view, a door call looks an awful lot like a local procedure call: there is process context switching during the call & return, but they happen without interference from the regular scheduling algorithm(s). Doors are a synchronous communication mechanism. Erlang's are asynchronous. If a lower-latency communication scheme were added onto Erlang, how would you do it to get round trip times door-like? 1. The server process has an attribute that would cause it to preempt the message sending process immediately. When the server process yields or blocks, the message sending process is the next one scheduled so that it can get its reply right away. There are a couple of problems with this, I guess. One, there's no guarantee that the server will reply to the client right away. (But if you want door-like performance, the server must have an answer right away. Caveat emptor.) Two, every message sent to the server would be handled this way. This may or may not be bad. 2. Use a new syntactic thingie to explicitly denote door-like behavior. For example, use "Pid !! Message". It's an icky idea, but I thought I ought to mention it. It may even have merit, somehow. {shrug} 3. Have the server process have a second "address" to receive messages. Messages received via the real pid make no change in scheduling. Messages received via a second pid (pseudo-pid?) would cause door-like scheduling to occur. This would be a weird thing for Erlang: messages sent to two different pids actually get delivered to the same mailbox for the same process. However, it doesn't solve the problem of how to get door-like scheduling behavior when sending a reply back to the client ... unless the client has a pseudo-pid, too. 4. None of the above. -Scott --- Scott Lystig Fritchie Professional Governing: Is It Faked? From jamesh@REDACTED Mon Jun 18 21:24:03 2001 From: jamesh@REDACTED (James Hague) Date: Mon, 18 Jun 2001 14:24:03 -0500 Subject: Erlang array access benchmark Message-ID: I was perusing Doug Bagley's language shootout page a few evenings ago and noticed that there wasn't an Erlang entry for the Array Access benchmark. Not too surprising, given the state of array handling in Erlang, but I decided to give it a go anyway. My first submitted attempt used the process dictionary for the two arrays. The code was succinct, but the performance was horrible. The run-time of 40 seconds put it in the same ballpark as TCL. Then I rewrote it using ETS tables. As the benchmark was based around adding values into an array, I could use ets:update_counter, which was nice. Version 2 almost doubled the performance of version 1, clocking in at 22 seconds, better than guile and Ruby, but still behind ghc (a machine code generating Haskell compiler). Over the weekend I realized that only one array was being written into and the other was simply used for heavy indexing. So I initialized that array as a list, then converted it to a tuple. The other array was still an ets table. This doubled the performance again, dropping the execution time to ten seconds. In the process, Erlang passed ghc, Icon, Python, Perl, and Lua. Sure, Erlang is still 90 times slower than the C entry, but it was fun helping it slide up the scale a bit :) It is also interesting that for something as un-Erlang-like as array access, Erlang still fared better than Python and Perl. James From mickael.remond@REDACTED Mon Jun 18 22:34:08 2001 From: mickael.remond@REDACTED (Mickael Remond) Date: Mon, 18 Jun 2001 22:34:08 +0200 Subject: ANN: Dia EML diagrams plug-in (now for Windows) In-Reply-To: <20010615230507N.svg@disney.surnet.ru> Message-ID: <20010618223408.C537@erlang-fr.org> Vladimir Sekissov (svg@REDACTED) wrote: > EML diagrams plug-in now included in official Dia distribution > (http://www.lysator.liu.se/~alla/dia). > > Send me your comments and suggestions. Seems nice. By the way, has anybody any experience feedback on usage for real and complex system design ? Thank you in advance for your help. -- Micka?l R?mond http://www.erlang-fr.org/ From Chandrashekhar.Mullaparthi@REDACTED Thu Jun 14 16:09:08 2001 From: Chandrashekhar.Mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Thu, 14 Jun 2001 15:09:08 +0100 Subject: gen_tcp:send problem Message-ID: <402DD461F109D411977E0008C791C31203919BA7@imp02mbx.one2one.co.uk> Quite often, under high load, gen_tcp:send returns the following error. Has anyone seen this before?? what is causing this? From the server point of view, the connection still seems to be up - but the client side can't send anymore data using this socket. When in the process of sending a million messages, this occured about 16 times. The socket was opened as gen_tcp:open(Host,Port,[list]). {error, {badarg, {erlang, port_command, [#Port<0.37>, [84, 101, 108, 80, 0, 32, 0, 142, 0, 1, 0, 0, 255, 255, 255, 255, 255, 255, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 8, 256, 0, 121, 31, 0, 77, 0, 10, 0, 7, 0, 9, 0, 5, 0, 6, 0, 4, 0, 9, 0, 5, 0, 8, 0, 8, 0, 7, 0, 20, 0, 0, 0, 7, 0, 7, 0, 4, 0, 8, 0, 7, 0, 6, 0, 7, 0, 1, 0, 0, 0, 8, 0, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 14, 0, 50, 0, 48, 0, 48, 0, 49, 0, 48, 0, 54, 0, 49, 0, 51, 0, 48, 0, 57, 0, 48, 0, 48, 0, 48, 0, 48, 0, 1, 0, 1, 0, 15, 0, 50, 0, 51, 0, 52, 0, 70, 0, 51, 0, 48, 0, 48, 0, 48, 0, 48, 0, 48, 0, 49, 0, 49, 0, 50, 0, 51, 0, 52, 0, 1, 0, 0, 217, 29]]}, {prim_inet,send,2}, {tcp_client_server,connected,3}, {gen_fsm,handle_msg,7}, {proc_lib,init_p,5}}} tia, Chandru NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From Sean.Hinde@REDACTED Fri Jun 15 09:06:25 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Fri, 15 Jun 2001 08:06:25 +0100 Subject: Interesting benchmark performance (was RE: Pitiful benchmark perf ormance) Message-ID: <402DD461F109D411977E0008C791C312039F6067@imp02mbx.one2one.co.uk> > > What would happen if you e.g. change INPUT_REDUCTIONS in erl_vm.h > > from (2* CONTEXT_REDS) to CONTEXT_REDS, doubling the I/O poll > > frequency? (CONTEXT_REDS is set to 1000) > > > > If basically all that happens in the erlang vm is that some > > process is waiting for I/O, ports may not be polled with any > > predictably high frequency (then again, this is only guessing > > wildly. Perhaps it effectively gives the ports even higher > > priority...) > > I didn't try this but did get the benchmark to spawn an > additional process OK, here is my take on this: The main emulator loop appears to be the code at the very end of sys.c. This calls schedule() and waits for it to return before any i/o is checked. schedule() in erl_process.c will *only* return when INPUT_REDUCTIONS have all been used up. As far as I can make out the only reason the echo/server benchmark doesn't deadlock is that there is enough going on in the background in an otherwise quiescent system to eventually use up the 2000 function calls required to check i/o. Maybe schedule() might be modified so that in addition to returning after INPUT_REDUCTIONS it keeps track of activity in each pass through all processes and returns if there has been no activity detected for a whole pass. As far as I can make out this would not slow things down significantly in a busy system but would improve i/o responsiveness under normal load quite considerably. Please, someone who really understands these things, help me out here! Thanks, Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From Chandrashekhar.Mullaparthi@REDACTED Fri Jun 15 12:06:36 2001 From: Chandrashekhar.Mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Fri, 15 Jun 2001 11:06:36 +0100 Subject: FW: gen_tcp:send problem Message-ID: <402DD461F109D411977E0008C791C31203919BAB@imp02mbx.one2one.co.uk> I'm sorry if this arrives twice - but my original message seems to have been lost. -----Original Message----- From: Chandrashekhar Mullaparthi Sent: 14 June 2001 15:09 To: 'Erlang mailing list' Subject: gen_tcp:send problem Quite often, under high load, gen_tcp:send returns the following error. Has anyone seen this before?? what is causing this? From the server point of view, the connection still seems to be up - but the client side can't send anymore data using this socket. When in the process of sending a million messages, this occured about 16 times. The socket was opened as gen_tcp:open(Host,Port,[list]). {error, {badarg, {erlang, port_command, [#Port<0.37>, [84, 101, 108, 80, 0, 32, 0, 142, 0, 1, 0, 0, 255, 255, 255, 255, 255, 255, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 8, 256, 0, 121, 31, 0, 77, 0, 10, 0, 7, 0, 9, 0, 5, 0, 6, 0, 4, 0, 9, 0, 5, 0, 8, 0, 8, 0, 7, 0, 20, 0, 0, 0, 7, 0, 7, 0, 4, 0, 8, 0, 7, 0, 6, 0, 7, 0, 1, 0, 0, 0, 8, 0, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 14, 0, 50, 0, 48, 0, 48, 0, 49, 0, 48, 0, 54, 0, 49, 0, 51, 0, 48, 0, 57, 0, 48, 0, 48, 0, 48, 0, 48, 0, 1, 0, 1, 0, 15, 0, 50, 0, 51, 0, 52, 0, 70, 0, 51, 0, 48, 0, 48, 0, 48, 0, 48, 0, 48, 0, 49, 0, 49, 0, 50, 0, 51, 0, 52, 0, 1, 0, 0, 217, 29]]}, {prim_inet,send,2}, {tcp_client_server,connected,3}, {gen_fsm,handle_msg,7}, {proc_lib,init_p,5}}} tia, Chandru NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From peter@REDACTED Tue Jun 19 13:47:04 2001 From: peter@REDACTED (Peter H|gfeldt) Date: Tue, 19 Jun 2001 13:47:04 +0200 (MET DST) Subject: gen_tcp:send problem In-Reply-To: <402DD461F109D411977E0008C791C31203919BA7@imp02mbx.one2one.co.uk> Message-ID: In some way you have provided Data to gen_tcp:send(Socket, Data) that is not an I/O-list: there is an integer value of 256 in the list of data provided to port_command. For the definition of I/O-list, see for instance erl -man erlang, under the heading port_command. Regards, Peter ------------------------------------------------------------------------- Peter H?gfeldt e-mail : peter@REDACTED Open Telecom Platform Phone: : +46 (8) 727 57 58 Ericsson Utvecklings AB Mobile : +46 070-519 57 51 S-126 25 STOCKHOLM Fax: : +46 (8) 727 5775 Office address: Armborstv?gen 1, ?lvsj? On Thu, 14 Jun 2001, Chandrashekhar Mullaparthi wrote: > > Quite often, under high load, gen_tcp:send returns the following > error. Has anyone seen this before?? what is causing this? From the > server point of view, the connection still seems to be up - but the client > side > can't send anymore data using this socket. > > When in the process of sending a million messages, this occured about > 16 times. The socket was opened as gen_tcp:open(Host,Port,[list]). > > {error, {badarg, {erlang, > port_command, > [#Port<0.37>, > [84, > 101, > 108, > 80, > 0, > 32, > 0, > 142, > 0, > 1, > 0, > 0, > 255, > 255, > 255, > 255, > 255, > 255, > 255, > 255, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 5, > 8, > 256, > 0, > 121, > 31, > 0, > 77, > 0, > 10, > 0, > 7, > 0, > 9, > 0, > 5, > 0, > 6, > 0, > 4, > 0, > 9, > 0, > 5, > 0, > 8, > 0, > 8, > 0, > 7, > 0, > 20, > 0, > 0, > 0, > 7, > 0, > 7, > 0, > 4, > 0, > 8, > 0, > 7, > 0, > 6, > 0, > 7, > 0, > 1, > 0, > 0, > 0, > 8, > 0, > 255, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 0, > 14, > 0, > 50, > 0, > 48, > 0, > 48, > 0, > 49, > 0, > 48, > 0, > 54, > 0, > 49, > 0, > 51, > 0, > 48, > 0, > 57, > 0, > 48, > 0, > 48, > 0, > 48, > 0, > 48, > 0, > 1, > 0, > 1, > 0, > 15, > 0, > 50, > 0, > 51, > 0, > 52, > 0, > 70, > 0, > 51, > 0, > 48, > 0, > 48, > 0, > 48, > 0, > 48, > 0, > 48, > 0, > 49, > 0, > 49, > 0, > 50, > 0, > 51, > 0, > 52, > 0, > 1, > 0, > 0, > 217, > 29]]}, > {prim_inet,send,2}, > {tcp_client_server,connected,3}, > {gen_fsm,handle_msg,7}, > {proc_lib,init_p,5}}} > > tia, > Chandru > > > > NOTICE AND DISCLAIMER: > This email (including attachments) is confidential. If you have received > this email in error please notify the sender immediately and delete this > email from your system without copying or disseminating it or placing any > reliance upon its contents. We cannot accept liability for any breaches of > confidence arising through use of email. Any opinions expressed in this > email (including attachments) are those of the author and do not necessarily > reflect our opinions. We will not accept responsibility for any commitments > made by our employees outside the scope of our business. We do not warrant > the accuracy or completeness of such information. > > From Kenneth.Browne@REDACTED Thu Jun 21 15:44:58 2001 From: Kenneth.Browne@REDACTED (Kenneth Browne (EEI)) Date: Thu, 21 Jun 2001 15:44:58 +0200 Subject: Bypass module for INETS Message-ID: Hi, I'm having a problem trying to find a solution to this problem: Single Login ========================================================================================================== I have two web servers. One is using Apache (server1) and the other is using INETS (server2). I first logon to server1 using Apache's basic authentication. This opens up a frameset. The top frame has a HTML page from server1 and the bottom frame serves a page from server2. The problem is that server2 requires a user name and password (the same user name and password which was used to logon to server1) to logon to its site. How do I bypass the basic authentication on the INETS (server2) server. I don't want the login box to appear. I cannot use the allow directive because I will not know the IP address of the client (or any other details for that matter). I have tried using the http://username:password@REDACTED:port/dir but I have Java applets on the pages which I require from server2 and they wont load when I use this method. +----------+ +---------------------------+ | client |<--------------->| INETS (server2) | +----------+ | | ^ +---------------------------+ | v +------------------------------------+ | Apache (server1) | | | +------------------------------------+ To cut a long story short: I just want to have 1 login box which will allow me to login to both servers ????? Any help is appreciated !!!! -Ken Browne. reply to: kenneth.browne@REDACTED From eleberg@REDACTED Thu Jun 21 17:18:29 2001 From: eleberg@REDACTED (Bengt Kleberg) Date: Thu, 21 Jun 2001 17:18:29 +0200 (MET DST) Subject: sae, clarification needed, please Message-ID: <200106211518.RAA09372@etxb.ericsson.se> Greetings, Erlang R7B-3 sae-2.0 What modules are callable from my erlang program, without having to include them when doing the linking (elink)? In the example below I clearly have access to something called erlang:display/1 and erlang:stop/0. Where do they come from? ; cat > echo.erl -module( echo ). -export([main/1]). main(Args) -> echo( tl( Args ) ), erlang:stop(). echo( [] ) -> ok; echo( [H|T] ) -> erlang:display( H ), echo( T ). ^D ; ecc echo.erl "echo.erl" {ok,echo} ; elink -o echo -m echo -s ecat main ; ./echo 1 2 3 1 2 3 ; From Sean.Hinde@REDACTED Thu Jun 21 17:44:48 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Thu, 21 Jun 2001 16:44:48 +0100 Subject: sae, clarification needed, please Message-ID: <402DD461F109D411977E0008C791C312039F6082@imp02mbx.one2one.co.uk> > In the example below I clearly have access to something called > erlang:display/1 and erlang:stop/0. Where do they come from? They are BIFs. i.e. functions implemented in C directly into the beam emulator. There is a full list in bif.tab in erts/emulator/beam. Rgds, Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From svg@REDACTED Thu Jun 21 18:33:44 2001 From: svg@REDACTED (Vladimir Sekissov) Date: Thu, 21 Jun 2001 22:33:44 +0600 Subject: Bypass module for INETS In-Reply-To: References: Message-ID: <20010621223344J.svg@disney.surnet.ru> Kenneth.Browne> To cut a long story short: Kenneth.Browne> I just want to have 1 login box which will allow me to login to both servers ????? I think there are many ways. 1. Use mod_perl or mod_tcl etc ... to pass authentication request to INET server and process its response: +------+ HTML Form +------------------+ HTML Form +-------+ | |---------->| |---------->| | |Client| Response | Apache(mod_perl) | Response | INET | | |<----------| |<----------| | +------+ +------------------+ +-------+ 2. You can use mod_corba(http://www.netrinsics.com/Mod_Corba.html) and Erlang CORBA services to control Apache authentication phase. Regards, Vladimir Sekissov From voudheus@REDACTED Thu Jun 21 23:34:41 2001 From: voudheus@REDACTED (Karel Van Oudheusden) Date: Thu, 21 Jun 2001 23:34:41 +0200 Subject: Drivers Message-ID: <3B326871.6FC491F7@imec.be> Hello, I have written a 'C1.c' file which is supposed to be a linked-in Erlang driver: ----------------------------------------- #include #include "erl_driver.h" static int erlang_port; static long easy_start(); ..... ------------------------------------------ Except for a warning message, I have successfully compiled the file with the following command: gcc -c -I/mnt/hda2/kvo7B-1/otp_src_R7B-1/erts/emulator/beam C1.c -o C1 However, in the (old) documentation that I downloaded from the net, I am supposed to change the contents of the 'config.c' file before compilation and linking. The problem is I cannot find this file. Can somebody help me with this. And how do I link the file into the Erlang system? I am using RedHat 7.0. Where can I find the most up to date documentation on drivers? thanx, KVO. From vances@REDACTED Fri Jun 22 04:21:16 2001 From: vances@REDACTED (Vance Shipley) Date: Thu, 21 Jun 2001 22:21:16 -0400 Subject: Drivers In-Reply-To: <3B326871.6FC491F7@imec.be> Message-ID: Karel, I am also working on a driver now. The best documentation I have found is in the ERST User's Guide (3. How to implement an alternative carrier for the erlang distribution). This discusses an example linked in driver. > However, in the (old) documentation that I downloaded from the net, I am > supposed to change the contents of the 'config.c' file before > compilation and linking. The problem is I cannot find this file. Can > somebody help me with this. And how do I link the file into the Erlang You will however want to use the new Dynamic Driver Loader and Linker (erl_ddll) which is documented in the Kernel Reference Manual. This allows you to install your new driver on a running system without having to recompile the kernel. With this method you won't need to worry about the above. > I have written a 'C1.c' file which is supposed to be a linked-in Erlang > driver: > > ----------------------------------------- > #include > #include "erl_driver.h" You are including the newer header file (driver.h was the old) so that's a good start. > Except for a warning message, I have successfully compiled the file with > the following command: > > gcc -c -I/mnt/hda2/kvo7B-1/otp_src_R7B-1/erts/emulator/beam C1.c -o C1 To go the dll (dynamically linked library) route you'll need to add -fPIC. I found it quite a struggle to determine how all of this was supposed to work, especially because I chose to use threads, asynchronous I/O etc. My source code now serves to document the new features in the driver code (as best as I understand them). I do plan to release all of this when I'm done. Contact me directly if you need more help. -Vance Vance Shipley Motivity Telecom Inc. +1 519 579 5816 From nurjoohan@REDACTED Fri Jun 22 07:06:06 2001 From: nurjoohan@REDACTED (Nur Joo Han Guna Segar) Date: Fri, 22 Jun 2001 05:06:06 -0000 Subject: Encoding in base64 using awk, grep, sed, or find Message-ID: Hey there, I would like your help on this matter. My instructor have asked me to encode a text file into a base64 file by using grep, sed, awk or find. I would appreciate your help Thanks Joo Han _________________________________________________________________________ Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com. From fritchie@REDACTED Fri Jun 22 07:34:48 2001 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Fri, 22 Jun 2001 00:34:48 -0500 Subject: Drivers In-Reply-To: Message of "Thu, 21 Jun 2001 22:21:16 EDT." Message-ID: <200106220534.f5M5Ymg87339@snookles.snookles.com> >>>>> "vs" == Vance Shipley writes: vs> You will however want to use the new Dynamic Driver Loader and vs> Linker (erl_ddll) which is documented in the Kernel Reference vs> Manual. Another good source of info on that subject is by example. I'd found that the "byteorder" driver (found in the User Contributions area, http://www.erlang.org/user.html) was a good place to start. -Scott --- Scott Lystig Fritchie Professional Governing: Is It Faked? From mickael.remond@REDACTED Fri Jun 22 07:51:36 2001 From: mickael.remond@REDACTED (Mickael Remond) Date: Fri, 22 Jun 2001 07:51:36 +0200 Subject: Erlang conferences at Bordeaux (France) Message-ID: <20010622075136.D789@erlang-fr.org> During the "Libre Software Meeting", we will be presenting two conferences on the Erlang language (5th July): Dimitri Fontaine: Introduction to Erlang - Overview of Erlang features. Micka?l R?mond: Developping web services with Erlang and SOAP - This conference aims at showing that SOAP spreading can enlarge the application field of a functional language like Erlang, while leveraging fault tolerance and high-availability. More details on erlang-fr web site: http://www.erlang-fr.org/ Maybe I will see some of you here :-) -- Micka?l R?mond http://www.erlang-fr.org/ From voudheus@REDACTED Fri Jun 22 14:36:42 2001 From: voudheus@REDACTED (Van Oudheusden) Date: Fri, 22 Jun 2001 14:36:42 +0200 Subject: gen_fsm Message-ID: <3B333BDA.A143C8E1@imec.be> Hello again, It's firday afternoon and I'm getting a bit impatient with the code (see attachment). It's a simple instantiation of the 'gen_fsm' behaviour. Could somebody please tell me what I am doing wrong? Besides that, I have a more fundamental question: If I am not mistaken, the implementation of gen_fsm implies that one or more processes are spawned. My question is whether it is a good idea to keep this issue transparent from the user who is using the behaviour. Doesn't this influence the performance of his implementation? How efficient is a behavriour with respect to inheritance in an onject oriented program in which the same design pattern is implemented? any comments are welcome, KVO. -------------- next part -------------- -module(fsmkvo). -behaviour(gen_fsm). -import(lists, [append/2]). %% -------------------------------------------------------------------- %% CALLBACK functions: %% -------------------------------------------------------------------- %% Initialisation routine: init(Args) -> {ok, sleeping, [1]}. %% Termination routine: terminate(Reason,State,Data) -> ok. %% Stop the FSM no matter what state it is in: handle_event(stopit, AnyState, TheStateData) -> {stop, stopInvocation, []}; %% Print the State Data of the Current State of the FSM: handle_event(printState, AnyState, TheStateData) -> io:format(" Currently in state ~w~n", [AnyState]), io:format(" with state data: ~w~n", [TheStateData]), io:format(" ~n"), {next_state, AnyState, TheStateData}. %% The STATE MACHINE: sleeping({eat}, Data) -> %% ... actions for change from sleep to eat ... {next_state, eating, lists:append(Data,[2])}. eating({work}, Data) -> {next_state, working, lists:append(Data,[3])}. working({sleep}, Data) -> {next_state, sleeping, lists:append(Data,[4])}; working({eat}, Data) -> {next_state, eating, lists:append(Data,[5])}. %% -------------------------------------------------------------------- %% CALLBACK end. %% -------------------------------------------------------------------- %% ---------------------------------------------------------------------------------------------------------------- %% -------------------------------------------------------------------- %% interface routines: %% -------------------------------------------------------------------- start() -> gen_fsm:start({local,fsmkvo}, fsmkvo, [], []). letsEat() -> gen_fsm:send_event(fsmkvo, {eat}). letsWork() -> gen_fsm:send_event(fsmkvo, {work}). letsSleep() -> gen_fsm:send_event(fsmkvo, {sleep}). printState() -> gen_fsm:send_all_state_event(fsmkvo, printState). stop() -> gen_fsm:send_all_state_event(fsmkvo, stopit). %% -------------------------------------------------------------------- %% interface end. %% -------------------------------------------------------------------- From francesco@REDACTED Fri Jun 22 14:49:35 2001 From: francesco@REDACTED (Francesco Cesarini) Date: Fri, 22 Jun 2001 13:49:35 +0100 Subject: gen_fsm References: <3B333BDA.A143C8E1@imec.be> Message-ID: <3B333EDF.88148089@erlang-consulting.com> Hi, > It's firday afternoon and I'm getting a bit impatient with the code (see > attachment). It's a simple instantiation of the 'gen_fsm' behaviour. > Could somebody please tell me what I am doing wrong? you forgot to export your client functions, your states and your call back functions. Next time you are having problems, start the debugger, trace compiling the code and then spawn your process. > Besides that, I have a more fundamental question: If I am not > mistaken, the implementation of gen_fsm implies that one or more > processes are spawned. It depends on your implementation and needs. In that you are registering your process locally, you may only have one instance of it in your node. If you are not registering it, you may have one or more, and access it with the Pid. > My question is whether it is a good idea to keep this issue > transparent from the user who is using the behaviour. Doesn't this > influence the performance of his implementation? When dealing with processes (or behaviors) it is always a good idea to hide the fact that they are processes in a functional interface. It gives you more freedom when updating and maintaining the code. Readability and ease of maintenance should always go before performance. Regards, Francesco -- Francesco Cesarini http://www.erlang-consulting.com From hal@REDACTED Fri Jun 22 18:51:37 2001 From: hal@REDACTED (Hal Snyder) Date: 22 Jun 2001 11:51:37 -0500 Subject: gen_udp:open(...[{packet, 0}]) Message-ID: <87pubwlap2.fsf@ghidra.vail> What happened to raw IP support for UDP? Seems it was in R6 but is gone from OTP R7B. :-( We have an Erlang success story. About a year ago, we assigned a summer intern to write a proxy in Erlang, as a pilot project, for remote database dips done while handling calls on our IVR platform. The result was probably the most reliable app we have, running a pair of proxy servers set up so clients fail from one proxy to the other - also proxies can fail from one server to another at the remote SCP. During the past year, the proxies have sailed through almost weekly outages of the remote servers, including extended (rolling) blackouts in CA, etc. without losing a hit. No memory leaks, segfaults, or going off into the ozone. It just works. J?ttebra! (er, "Great!" - was that the right word?) The gen_udp:open.. {packet,0} change came to our attention when trying to extend the proxy system to other services on newer boxes running R7B. From vances@REDACTED Fri Jun 22 21:38:05 2001 From: vances@REDACTED (Vance Shipley) Date: Fri, 22 Jun 2001 15:38:05 -0400 Subject: gen_fsm In-Reply-To: <3B333BDA.A143C8E1@imec.be> Message-ID: > It's firday afternoon and I'm getting a bit impatient with the code (see > attachment). It's a simple instantiation of the 'gen_fsm' behaviour. > Could somebody please tell me what I am doing wrong? As has already been pointed out, you needed to export the functions which need to be called by outside modules. This includes those that the gen_fsm module must call. I like to do it this way: % export the callbacks common to gen_fsm behaviours -export([init/1, code_change/4, handle_event/3, handle_sync_event/4, handle_info/3, terminate/3]). % export the callbacks for (gen_fsm behaviour) states in this module -export([sleeping/2, eating/2, working/2]). % export the API functions -export([start/0, letsEat/0, letsWork/0, letsSleep/0, printState/0, stop/0]). > Besides that, I have a more fundamental question: If I am not mistaken, > the implementation of gen_fsm implies that one or more processes are > spawned. My question is whether it is a good idea to keep this issue > transparent from the user who is using the behaviour. Doesn't this > influence the performance of his implementation? How efficient is a > behavriour with respect to inheritance in an onject oriented program in > which the same design pattern is implemented? In fact there is only one process involved. The documentation can be a little misleading as it shows what appears to be a message flow between processes: Callback module gen_fsm ---------------- ------- gen_fsm:start_link -----> start a new fsm process Module:init/1 <----- looping gen_fsm:send_event -----> Module:StateName/2 <----- ..... However the introduction to this diagram says: "The relationship between the generic interface functions (and received messages) and the callback functions can be illustrated as follows:" Really this just shows the flow of execution which involves both your module (fsmkvo.erl) and the gen_fsm.erl module which implements the behaviour. gen_fsm.erl contains the main loop of the resulting program. You end up with one process which is executing code from both modules. There are not any messages being sent other than when your other modules execute gen_fsm:send_event/2 to send an event (message) into the state machine. The above diagram does however muddy the waters a bit 'cause it puts the 'gen_fsm:send_event' under the 'Callback Module' column. This isn't how you've done it. In your case you have an API function (e.g. letsEat/0) which will be called by a user (e.g. the shell user). Here we're running a shell to play with your code. The PID of the shell process is <0.39.0>. 1> self(). <0.39.0> We use your API function to start the gen_fsm process which now runs as PID <0.36.0> (and which you have registered with the name "fsmkvo"). 2> fsmkvo:start(). {ok,<0.36.0>} Now we use your API function to change the state: 3> fsmkvo:letsEat(). ok When we do this the shell process sources your module (fsmkvo.erl) to get the code for the start function. The shell process now executes this code. You defined letsEat/0 as: letsEat() -> gen_fsm:send_event(fsmkvo, {eat}). In turn the shell now calls gen_fsm:send_event/2 which is defined as: send_event(Name, Event) -> Name ! {'$gen_event', Event}, ok. So the shell process is really doing: 3> fsmkvo ! {'$gen_event', {eat}}. It is sending a message to your process. Since you included the gen_fsm behaviour your process includes both your code (fsmkvo.erl) and the code in gen_fsm.erl. The gen_fsm.erl module contains the code to handle receiving messages. The gen_fsm code runs your code to handle the implementation specific stuff. -Vance Vance Shipley Motivity Telecom Inc. Tel: +1 519 579 5816 From willem@REDACTED Sat Jun 23 17:16:23 2001 From: willem@REDACTED (Willem Broekema) Date: Sat, 23 Jun 2001 17:16:23 +0200 Subject: 'recbuf' and the size of received data Message-ID: <3B34B2C7.80309@pastelhorn.com> I have a problem receiving tcp packets with data larger than 1024 bytes. The connection is declared as: {ok, Sock} = gen_tcp:connect(Host, Port, [binary, {packet, 0}, {active, true}, {recbuf, 2048}]), And data is received with: receive {tcp, Sock, Data} -> ... end I wonder why the binary object Data is truncated to 1024 bytes, as I declared 'recbuf' to be twice that size. Is there another option I should use? Thanks for your help. - Willem From fritchie@REDACTED Sun Jun 24 09:19:46 2001 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Sun, 24 Jun 2001 02:19:46 -0500 Subject: 'recbuf' and the size of received data In-Reply-To: Message of "Sat, 23 Jun 2001 17:16:23 +0200." <3B34B2C7.80309@pastelhorn.com> Message-ID: <200106240719.f5O7Jkg05657@snookles.snookles.com> >>>>> "wb" == Willem Broekema writes: wb> I have a problem receiving tcp packets with data larger than 1024 wb> bytes. If you attempt to read N bytes from a TCP socket, you'll get at most N bytes, but all other bets are off. If the sender is transmitting in M byte chunks, you may get M bytes when attempting to read N bytes (assuming M < N), but you may not. If you need message boundaries, you need to use a different protocol (e.g. UDP) or you need to embed those boundaries into the protocol you're using over TCP. So, in your case, if recbuf is 2K, and you're gettings reads of 1K, you'll just have to do another read and then concatenate the results. There's no guarantee that your second read will be exactly 1K, either: it might be 1.5K, which means you'll have to "carry over" 0.5K into the next message you read. -Scott --- Scott Lystig Fritchie Professional Governing: Is It Faked? From raimo@REDACTED Mon Jun 25 10:24:17 2001 From: raimo@REDACTED (Raimo Niskanen) Date: Mon, 25 Jun 2001 10:24:17 +0200 Subject: gen_udp:open(...[{packet, 0}]) References: <87pubwlap2.fsf@ghidra.vail> Message-ID: <3B36F531.CF5FD586@erix.ericsson.se> Hal Snyder wrote: > > What happened to raw IP support for UDP? Seems it was in R6 but is > gone from OTP R7B. :-( > . . > > The gen_udp:open.. {packet,0} change came to our attention when trying > to extend the proxy system to other services on newer boxes running > R7B. We removed the {packet, 0} option since I thought it was misleading. An UDP socket _is_ packet oriented as it is, so {packet, 0} did not add anything. What do you mean with "raw IP"; UDP is as raw as it gets, is it not? / Raimo Niskanen, Ericsson Utvecklings AB, Erlang/OTP From hakan@REDACTED Mon Jun 25 17:44:47 2001 From: hakan@REDACTED (Hakan Mattsson) Date: Mon, 25 Jun 2001 17:44:47 +0200 (MET DST) Subject: 'recbuf' and the size of received data In-Reply-To: <200106240719.f5O7Jkg05657@snookles.snookles.com> Message-ID: On Sun, 24 Jun 2001, Scott Lystig Fritchie wrote: Scott> >>>>> "wb" == Willem Broekema writes: Scott> Scott> wb> I have a problem receiving tcp packets with data larger than 1024 Scott> wb> bytes. Scott> Scott> If you attempt to read N bytes from a TCP socket, you'll get at most N Scott> bytes, but all other bets are off. If the sender is transmitting in M Scott> byte chunks, you may get M bytes when attempting to read N bytes Scott> (assuming M < N), but you may not. If you need message boundaries, Scott> you need to use a different protocol (e.g. UDP) or you need to embed Scott> those boundaries into the protocol you're using over TCP. If you want to use TCP you may utilize the builtin Erlang emulator support for TPKT (http://www.ietf.org/rfc/rfc1006.txt). Open the socket with: gen_tcp:connect(Host, Port, [{packet, tpkt} | OtherOptions]) Add the TPKT header manually on the sending side: tpkt_send(Socket, MessageBody) when binary(MessageBody) -> Len = size(MessageBody) + 4, gen_tcp:send(Socket, [3, 0, ((Len) bsr 8) band 16#ff, (L) band 16#ff , MessageBody]). On the receiving side, the Erlang emulator will automatically assemble complete TPKT-packets before they are forwarded to the control process, where you strip the header off with: handle_info({tcp, Socket, <<3:8, _:8, Len:16, MessageBody/binary>>}, State) -> decode_body(MessageBody, State)... /H?kan --- H?kan Mattsson Ericsson Computer Science Laboratory http://www.ericsson.se/cslab/~hakan From martin@REDACTED Mon Jun 25 18:36:25 2001 From: martin@REDACTED (Martin Logan) Date: Mon, 25 Jun 2001 11:36:25 -0500 Subject: JInterface - Peer authentication error References: <39A13E8F.65AB8551@cellpt.com> <14753.22200.516070.872099@gargle.gargle.HOWL> Message-ID: <3B376889.D7987CD3@vailsys.com> Hello All, I am currently writing a proxy and a load balancer in erlang on linux and free-bsd boxes. I have to make an odbc connection to an Oracle db. I am having a bit of trouble installing the odbc libraries from the otp_src_R7B-3. If anyone on this list has done this or knows how to do this any help that you can give would be greatly appreciated. Thank you, Martin Logan From martin@REDACTED Mon Jun 25 19:14:06 2001 From: martin@REDACTED (Martin Logan) Date: Mon, 25 Jun 2001 12:14:06 -0500 Subject: Erlang ODBC install References: <39A13E8F.65AB8551@cellpt.com> <14753.22200.516070.872099@gargle.gargle.HOWL> <3B376889.D7987CD3@vailsys.com> Message-ID: <3B37715E.61A5C7C8@vailsys.com> Martin Logan wrote: > Hello All, > I am currently writing a proxy and a load balancer in erlang on linux > and free-bsd boxes. I have to make an odbc connection to an Oracle db. I am > having a bit of trouble installing the odbc libraries from the > otp_src_R7B-3. If anyone on this list has done this or knows how to do this > any help that you can give would be greatly appreciated. > Thank you, > Martin Logan From Sean.Hinde@REDACTED Mon Jun 25 19:52:40 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Mon, 25 Jun 2001 18:52:40 +0100 Subject: Erlang ODBC install Message-ID: <402DD461F109D411977E0008C791C312039F609D@imp02mbx.one2one.co.uk> > Martin Logan wrote: > > > Hello All, > > I am currently writing a proxy and a load balancer in > erlang on linux > > and free-bsd boxes. I have to make an odbc connection to an > Oracle db. I am > > having a bit of trouble installing the odbc libraries from the > > otp_src_R7B-3. If anyone on this list has done this or > knows how to do this > > any help that you can give would be greatly appreciated. As I understand it you need two things to get odbc working. You need a odbc driver manager which comes as standard with most linux distros (I don't know about free-bsd), and you need the driver itself. I've had it working using the postrgre-sql driver towards postgre-sql on linux by pretty much following the docs (though they are what might be called a bit thin :) As far as I am aware there is no free or open source odbc driver for Oracle. There is talk of one at easysoft.org but apparently the developer has left. One nice one is the Intersolv one (as tested by Ericsson) but it is very expensive.. I'll try to have a look at exactly what I did to get it working when I get back to my Linux machine. - Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From svg@REDACTED Mon Jun 25 20:41:42 2001 From: svg@REDACTED (Vladimir Sekissov) Date: Tue, 26 Jun 2001 00:41:42 +0600 Subject: Erlang ODBC install In-Reply-To: <402DD461F109D411977E0008C791C312039F609D@imp02mbx.one2one.co.uk> References: <402DD461F109D411977E0008C791C312039F609D@imp02mbx.one2one.co.uk> Message-ID: <20010626004142P.svg@disney.surnet.ru> Sean.Hinde> As far as I am aware there is no free or open source odbc driver for Oracle. Sean.Hinde> There is talk of one at easysoft.org but apparently the developer has left. Sean.Hinde> One nice one is the Intersolv one (as tested by Ericsson) but it is very Sean.Hinde> expensive.. I use Erlang under Linux with unixODBC (included at least in Mandrake distribution) and both Oracle and PostgreSQL. Regards, Vladimir Sekissov From garry@REDACTED Mon Jun 25 20:37:44 2001 From: garry@REDACTED (Garry Hodgson) Date: Mon, 25 Jun 2001 14:37:44 -0400 Subject: Seeking advice on port "pattern" (long) References: Message-ID: <3B3784F8.D43F84FB@sage.att.com> okay, i've been using erlang for a while now, and have learned enough to be dangerous. i've got my first app working nicely, but have this nagging feeling that i'm not doing things in the most elegant way. so i'm seeking advice on good practice for using the port mechanism to attach to legacy applications. in particular, if there are standard patterns for how to do this cleanly, i'd appreciate some clue. pointers to code examples would be especially helpful. the application is a server that provides a single point of contact for several other servers on different machines, each offering different kinds of network information queries. i've got a registered allocator process, which maintains a list of idle and busy nodes, a list of active servers running on those nodes, and maps a given request to a server who can handle it, starting a fresh one if needed. this is done via a spawn_link of a port server, which starts the appropriate legacy app (more on this later). requests come into the master, which asks the allocator for the approriate server, then passes the request on to that server. the complication comes in the port process. some of the legacy code is multithreaded, and can potentially have a mix of long running database queries and others that respond quickly. each message and corresponding response is tagged with a transaction id, so i can associate it with the correct caller on return. because i can't rewrite the legacy apps, i'm communicating with simple ascii strings (no length prefix), and must accumulate data until i get a complete response, as some of the queries return lengthy replies, and don't accumulate the whole thing before replying. so, my port module, absent some detail, is included below. it seems to me that it's much more complicated than it ought to be. spawning the separate alarm processes seems clumsy, vs. spawning a process per transaction with a receive/after/end statement. but either way, i've got to keep a list of 'em, so i can send the reply to the right one once i get a complete message. i'm (dimly) aware of the gen_server behavior, which i've avoided in favor of rolling my own for learning purposes. i'm open to going that route now if that's the right thing to do. so, that's more or less where i am. i love this language, and am just beginning to feel fluent in it. but i could use come pointers to gain Enlightenment. does anyone have any sage words of advice? thanks very much. % ----------------- code follows ------------------ % alarm process to handle timeouts, spawned when call comes in setalarm( Id ) -> spawn( port, doalarm, [ self(), 10000, Id ] ). doalarm( Caller, Timeout, Id ) -> receive cancel -> true after Timeout -> Caller ! { timeout, Id } end. % port:start() start( Service, ExtPrg ) -> init( Service, ExtPrg ). init( Service, ExtPrg ) -> Port = open_port( {spawn, ExtPrg}, [] ), loop( Service, Port, [], 1000, [] ). loop( Service, Port, Pending, NextTrans, Buffered ) -> receive { call, Caller, Request, Args } -> Job = { NextTrans, Caller, setalarm( NextTrans ) }, Port ! {self(), {command, encode( NextTrans, Request, Args ) } }, loop( Service, Port, [ Job|Pending ], NextTrans + 1, Buffered ); { Port, { data, Data } } -> AllData = Buffered++Data, case decode( AllData ) of { Id, Req, Result } -> case lists:keysearch( Id, 1, Pending ) of { value, { Id, Caller, Alarm } } -> Caller ! { ok, { Service, { Req, Result } } }, Alarm ! cancel, loop( Service, Port, lists:keydelete( Id, 1, Pending ), NextTrans, Buffered ); false -> screamloudly( "Got data for nonexistent transaction ~w", [ Id ] ), loop( Service, Port, Pending, NextTrans, Buffered ) end; { parseError, Details } -> io:format( "huh? got ~p ~n. i'll wait for more...~n", [Details] ), loop( Service, Port, Pending, NextTrans, AllData ) end; { timeout, Id } -> io:format( "port: timeout on Id ~w", [ Id ] ), case lists:keysearch( Id, 1, Pending ) of { value, { Id, Caller, Alarm } } -> Caller ! { error, timeout }, % loop( Service, Port, lists:keydelete( Id, 1, Pending ), NextTrans, [] ); exit( { error, "Application timed out" } ); false -> screamloudly( "Got timeout for nonexistent transaction ~w", [ Id ] ), loop( Service, Port, Pending, NextTrans, [] ) end; stop -> Port ! { self(), close }, receive {Port, closed} -> exit( normal ) end end. --- Garry Hodgson sometimes we ride on your horses Senior Hacker sometimes we walk alone Software Innovation Services sometimes the songs that we hear AT&T Labs are just songs of our own garry@REDACTED From tobbe@REDACTED Tue Jun 26 09:06:08 2001 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: 26 Jun 2001 09:06:08 +0200 Subject: Seeking advice on port "pattern" (long) In-Reply-To: Garry Hodgson's message of "Mon, 25 Jun 2001 14:37:44 -0400" References: <3B3784F8.D43F84FB@sage.att.com> Message-ID: Your program looks fine to me. Two comments: 1. You don't have to implement you own timer, look at the erlang man page for start_timer/3 and cancel_timer/1. 2. You could write your decoder in a continuation based style. So instead of having to do the append (AllData = Buffered++Data) your decoder returns either { Id, Req, Result } , { parseError, Details } , or {needmore, Cont}. When new data arrives, you feed it into your continuation ( Cont(Data) ), which again will return any of the three possibilities above. Cheers /Tobbe From bjarne@REDACTED Tue Jun 26 15:09:46 2001 From: bjarne@REDACTED (Bjarne =?iso-8859-1?Q?D=E4cker?=) Date: Tue, 26 Jun 2001 15:09:46 +0200 Subject: Seventh International Erlang/OTP User Conference References: <39A13E8F.65AB8551@cellpt.com> <14753.22200.516070.872099@gargle.gargle.HOWL> <3B376889.D7987CD3@vailsys.com> Message-ID: <3B38899A.D47842C4@erix.ericsson.se> Seventh International Erlang/OTP User Conference Stockholm - Thursday, September 27, 2001 Call for papers and attendance Erlang users and enthusiasts are all invited to this year's user conference sharing experiences and ideas about technology and applications. Please see the conference home page which aven contains a provisional conference programme (but further papers and presentations are invited). http://www.erlang.se/euc/01/ See you all in Stockholm in September and have a nice Summer Bjarne D?cker bjarne@REDACTED CSLab Ericsson From Sean.Hinde@REDACTED Tue Jun 26 15:52:26 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Tue, 26 Jun 2001 14:52:26 +0100 Subject: Seeking advice on port "pattern" (long) Message-ID: <402DD461F109D411977E0008C791C312039F60AC@imp02mbx.one2one.co.uk> > 2. You could write your decoder in a continuation > based style. So instead of having to do the > append (AllData = Buffered++Data) your decoder > returns either { Id, Req, Result } , { parseError, Details } , > or {needmore, Cont}. When new data arrives, you > feed it into your continuation ( Cont(Data) ), which > again will return any of the three possibilities above. Can you provide an example of what you mean? I know Joe posted one some time ago but I can't find it and haven't quite managed to figure it out for myself. Thanks, - Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From Ingela.Anderton@REDACTED Tue Jun 26 16:26:16 2001 From: Ingela.Anderton@REDACTED (UAB/L/K Ingela Anderton konsult OTP) Date: Tue, 26 Jun 2001 16:26:16 +0200 (MEST) Subject: Encoding in base64 using awk, grep, sed, or find References: Message-ID: <200106261426.QAA13610@luthien.du.uab.ericsson.se> I do not see how your question is related to Erlang! But if you want to use Erlang for base64 encoding the easiest way would be to use the library function in httpd_util encode_base64(ASCIIString) -> Base64String Types: ASCIIString = string() Base64String = string() encode_base64 encodes a plain ascii string to a Base64 encoded string. See RFC 1521 for a description of Base64 encoding. If you insist on implementing the function yourself that wold be a quite short erlang program, it would however not use awk, grep, sed or find. Nur Joo Han Guna Segar wrote: > Hey there, > I would like your help on this matter. > My instructor have asked me to encode a text file into a base64 file by > using grep, sed, awk or find. > I would appreciate your help > Thanks > Joo Han > > _________________________________________________________________________ > Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com. -- /m.v.h Ingela Ericsson Utvecklings AB Cellular/Mobile: +46 70 636 78 68 From tobbe@REDACTED Tue Jun 26 16:30:57 2001 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: 26 Jun 2001 16:30:57 +0200 Subject: Seeking advice on port "pattern" (long) In-Reply-To: Sean Hinde's message of "Tue, 26 Jun 2001 14:52:26 +0100" References: <402DD461F109D411977E0008C791C312039F60AC@imp02mbx.one2one.co.uk> Message-ID: > Can you provide an example of what you mean? I know Joe posted one some time > ago but I can't find it and haven't quite managed to figure it out for > myself. Ok, below I've whipped up something to show what I meant. Cheers /Tobbe %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -module(abc). -export([abc/1]). %%% %%% Parse strings looking like: "abc" %%% Example: %%% %%% 2> abc:abc("abcdefg"). %%% {ok,"abc","defg"} %%% 3> {_,C} = abc:abc("a"). %%% {needmore,#Fun} %%% 4> {_,C2} = C("b"). %%% {needmore,#Fun} %%% 5> C("cdef"). %%% {parse_error,{"a",'',"cdef"}} %%% 6> C2("cdef"). %%% {ok,"abc","def"} %%% abc(In) -> a(In,[]). a([$a|T],Acc) -> b(T, [$a|Acc]); a([],Acc) -> {needmore, fun(More) -> a(More, Acc) end}; a(In,Acc) -> parse_error(In,Acc). b([$b|T],Acc) -> c(T, [$b|Acc]); b([],Acc) -> {needmore, fun(More) -> b(More,Acc) end}; b(In,Acc) -> parse_error(In,Acc). c([$c|T],Acc) -> {ok, lists:reverse([$c|Acc]), T}; c([],Acc) -> {needmore, fun(More) -> c(More, Acc) end}; c(In,Acc) -> parse_error(In,Acc). parse_error(In,Acc) -> {parse_error, {lists:reverse(Acc), '', In}}. From gupta@REDACTED Wed Jun 27 04:21:56 2001 From: gupta@REDACTED (Gopal Gupta) Date: Tue, 26 Jun 2001 21:21:56 -0500 Subject: 2nd CFP: 4th PADL symposium Message-ID: <200106270221.f5R2LuU02411@herbrand.utdallas.edu> CALL FOR PAPERS!!! CALL FOR PAPERS!!! CALL FOR PAPERS!!! Fourth International Symposium on Practical Aspects of Declarative Languages http://www.cs.sunysb.edu/~padl2002 (PADL '02) Portland, Oregon, USA Jan 19-20, 2002 Co-located with POPL 2002 Declarative languages build on sound theoretical bases to provide attractive frameworks for application development. These languages have been successfully applied to vastly different real-world situations, ranging from data base management to active networks to software engineering to decision support systems. New developments in theory and implementation have opened up new application areas. At the same time, applications of declarative languages to novel problems raises numerous interesting research issues. Well-known questions include designing for scalability, language extensions for application deployment, and programming environments. Thus, applications drive the progress in the theory and implementation of declarative systems, and benefit from this progress as well. PADL provides a forum for researchers, practitioners, and implementors of declarative languages to exchange ideas on current and novel application areas and on the requirements for effective deployment of declarative systems. We invite papers dealing with practical applications of newly discovered results and techniques in logic, constraint, and functional programming. Papers dealing with practical applications of theoretical results, new techniques of implementation with considerable impact on an application, or innovative applications are particularly welcome. Position papers as well as papers that present works in progress are also welcome. The scope of PADL includes, but is not limited to: o Innovative applications of declarative languages o Declarative domain-specific languages and applications o New developments in declarative languages and their impact on applications o Practical experiences o Evaluation of implementation techniques on practical applications o Novel uses of declarative languages in the classroom The papers should highlight the practical contribution of the work and the relevance of declarative languages to achieve that end. PADL 2002 will co-locate with ACM POPL 2002, in Portland, Oregon. Previous PADLs were held in San Antonio (1999), Boston (2000), and Las Vegas (2001). Important Dates: o Paper Submission: Aug. 10, 2001 o Notification: Oct. 8, 2001 o Camera Ready: Nov. 5, 2001 o Symposium: Jan. 19-20, 2002 Paper Submission: Authors should submit an electronic copy of the full paper (written in English) in Postscript (Level 2) or PDF. Papers must be no longer than 15 pages, written in 11-point font and with single spacing. Since the final proceedings will be published as Lecture Notes in Computer Science by Springer Verlag, authors are strongly encouraged to use the LNCS paper formatting guidelines for their submission. Each submission must include, on its 1st page, the paper title; authors and their affiliations; contact author's email and postal addresses, telephone and fax numbers, abstract, and three to four keywords. The keywords will be used to assist us in selecting appropriate reviewers for the paper. If electronic submission is impossible, please contact the program co-chairs for information on how to submit hard copies. Program Committee (still being constituted). o Sergio Antoy, Portland State University, USA o Gopal Gupta, UT Dallas (Organizer) o Joxan Jaffar, National University of Singapore o Fergus Henderson, University of Melbourne, Australia o Shriram Krishnamurthi, Brown University, USA (Program Co-chair) o Andrew Kennedy, Microsoft Research, UK o Michael Leuschel, University of Southampton, UK o Kim Marriott, Monash University, Australia o John Peterson, Yale University, USA o Andreas Podelski, MPI, Germany o Enrico Pontelli, New Mexico State University, USA o C.R. Ramakrishnan, SUNY, Stony Brook, USA (Program Co-chair) o John Reppy, Bell Labs Lucent Technologies o Manuel Serrano, Universit'e de Nice, France o Olin Shivers, Georgia Tech, USA o Paul Tarau, University of North Texas, USA For more Information, please contact C.R. Ramakrishnan Computer Science Department SUNY at Stony Brook Stony Brook, NY 11794-4400 USA email: cram@REDACTED From pascal.brisset@REDACTED Wed Jun 27 13:15:21 2001 From: pascal.brisset@REDACTED (Pascal Brisset) Date: Wed, 27 Jun 2001 13:15:21 +0200 Subject: Security of binary_to_term ? Message-ID: <15161.49225.526286.218186@pcg.localdomain> erlang:binary_to_term/1 generally exits with 'badarg' when applied to invalid inputs. Is this behaviour guaranteed ? In other words, is it safe to decode untrusted data with binary_to_term ? The purpose is to send data between untrusted nodes with term_to_binary and binary_to_term over TCP, rather than with the erlang distribution protocol. --- Pascal Brisset +33141986741 -- ----- Cellicium | 73 avenue Carnot | 94230 Cachan | France ------ From klacke@REDACTED Wed Jun 27 14:19:57 2001 From: klacke@REDACTED (Klacke) Date: Wed, 27 Jun 2001 14:19:57 +0200 Subject: Security of binary_to_term ? In-Reply-To: <15161.49225.526286.218186@pcg.localdomain>; from pascal.brisset@cellicium.com on Wed, Jun 27, 2001 at 01:15:21PM +0200 References: <15161.49225.526286.218186@pcg.localdomain> Message-ID: <20010627141957.C24445@bluetail.com> On Wed, Jun 27, 2001 at 01:15:21PM +0200, Pascal Brisset wrote: > erlang:binary_to_term/1 generally exits with 'badarg' when applied to > invalid inputs. Is this behaviour guaranteed ? In other words, is it > safe to decode untrusted data with binary_to_term ? > > The purpose is to send data between untrusted nodes with > term_to_binary and binary_to_term over TCP, rather than with the > erlang distribution protocol. > A number of checks are done trying to validate the data, however I think there are some pathological cases left where the emulator dies. Think so anyway. An aside note: If you get the data over TCP, why should it be invalid. TCP ensures the data is non corrupted.... or maybe you are worrying over rouge nodes ??? /klacke -- Claes Wikstrom -- Caps lock is nowhere and Alteon WebSystems -- everything is under control http://www.bluetail.com/~klacke -- From garry@REDACTED Wed Jun 27 15:20:46 2001 From: garry@REDACTED (Garry Hodgson) Date: Wed, 27 Jun 2001 09:20:46 -0400 Subject: Seeking advice on port "pattern" (long) References: <402DD461F109D411977E0008C791C312039F60AC@imp02mbx.one2one.co.uk> Message-ID: <3B39DDAE.2458008A@sage.att.com> Torbjorn Tornkvist wrote: > > > Can you provide an example of what you mean? I know Joe posted one some time > > ago but I can't find it and haven't quite managed to figure it out for > > myself. > > Ok, below I've whipped up something to show what I meant. > > Cheers /Tobbe > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > -module(abc). > -export([abc/1]). > > %%% > %%% Parse strings looking like: "abc" > %%% Example: > %%% > %%% 2> abc:abc("abcdefg"). > %%% {ok,"abc","defg"} > %%% 3> {_,C} = abc:abc("a"). > %%% {needmore,#Fun} > %%% 4> {_,C2} = C("b"). > %%% {needmore,#Fun} > %%% 5> C("cdef"). > %%% {parse_error,{"a",'',"cdef"}} > %%% 6> C2("cdef"). > %%% {ok,"abc","def"} > %%% very cool. i reworked my code to use this notion, and it works just fine. thanks. -- Garry Hodgson sometimes we ride on your horses Senior Hacker sometimes we walk alone Software Innovation Services sometimes the songs that we hear AT&T Labs are just songs of our own garry@REDACTED From pascal.brisset@REDACTED Wed Jun 27 15:34:40 2001 From: pascal.brisset@REDACTED (Pascal Brisset) Date: Wed, 27 Jun 2001 15:34:40 +0200 Subject: Security of binary_to_term ? In-Reply-To: <20010627141957.C24445@bluetail.com> References: <15161.49225.526286.218186@pcg.localdomain> <20010627141957.C24445@bluetail.com> Message-ID: <15161.57584.399333.200453@pcg.localdomain> > An aside note: If you get the data over TCP, why should it be > invalid. TCP ensures the data is non corrupted.... or maybe you > are worrying over rouge nodes ??? Well this is what security is about, isn't it ? :) Actually I stumbled on one of those pathological cases, and I was wondering whether it was just a bug or whether additional checks were required anyway. $ erl Erlang (BEAM) emulator version 5.0.2.4 [source] Eshell V5.0.2.4 (abort with ^G) 1> binary_to_term(<<131,111,255,0,0,0>>). zsh: 30198 segmentation fault ./bin/erl -- Pascal Brisset +33141986741 -- ----- Cellicium | 73 avenue Carnot | 94230 Cachan | France ----- From Sean.Hinde@REDACTED Wed Jun 27 15:47:02 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Wed, 27 Jun 2001 14:47:02 +0100 Subject: Driver questions Message-ID: <402DD461F109D411977E0008C791C312039F60BB@imp02mbx.one2one.co.uk> All, I have a short piece of C code which calculates a checksum and I want to turn it into a driver. I have inserted it into the byteorder-1.0 contrib and all works fine. I have some questions though about the design choices involved in this. 1. When to use synchronous port_control as opposed to asynchronous port_command and driver_output? I notice that Tobbe uses the second version in the byteorder contrib, but the crypto application uses the first version. Is the syncronous one quicker? What else should I take into account for this type of application (as opposed to natively async stuff like ip drivers)? 2. In using port control how big is the return buffer by default? Can this be relied on (it appears not to be documented)? 3. What is the significance of the return value in the control case? Is it related to the length of the result? Is this different if PORT_CONTROL_FLAG_BINARY is set (as implied by a comment in crypto_drv.c)? 4. Is it neccessary to create a driver binary for the return value as in the crypto case: case DRV_INFO: *rbuf = (char*)(b = driver_alloc_binary(NUM_CRYPTO_FUNCS)); for (i = 0; i < NUM_CRYPTO_FUNCS; i++) { b->orig_bytes[i] = i + 1; } return NUM_CRYPTO_FUNCS; break; Or can one just use a local variable as in Tobbes example: char lsb[]="l"; char msb[]="m"; if (*(char *) &endian) driver_output(erlang_port, lsb, sizeof(lsb)); else driver_output(erlang_port, msb, sizeof(msb)); Is the difference down to the choice of sync or async? Or something else? 5. In both sync and async cases is it possible for any process to make use of the port and have answers returned directly, or only the process which opened the port? I have read the docs and pored over io.c but it would be nice to have some definitive answers Thanks in advance, - Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From bjarne@REDACTED Wed Jun 27 15:49:02 2001 From: bjarne@REDACTED (Bjarne =?iso-8859-1?Q?D=E4cker?=) Date: Wed, 27 Jun 2001 15:49:02 +0200 Subject: 2nd CFP: 4th PADL symposium Message-ID: <3B39E44E.DAB98A86@erix.ericsson.se> Erlang for PADL ?! -------------- next part -------------- An embedded message was scrubbed... From: Gopal Gupta Subject: 2nd CFP: 4th PADL symposium Date: Tue, 26 Jun 2001 21:21:56 -0500 Size: 6275 URL: From vances@REDACTED Wed Jun 27 16:41:29 2001 From: vances@REDACTED (Vance Shipley) Date: Wed, 27 Jun 2001 10:41:29 -0400 Subject: Security of binary_to_term ? In-Reply-To: <15161.57584.399333.200453@pcg.localdomain> Message-ID: > 1> binary_to_term(<<131,111,255,0,0,0>>). > zsh: 30198 segmentation fault ./bin/erl Well this isn't right though is it? I found that I also got a segmentation fault on Linux but not on WindowsNT. In any event segmentation faults are never proper behaviour. -Vance From vances@REDACTED Wed Jun 27 18:33:12 2001 From: vances@REDACTED (Vance Shipley) Date: Wed, 27 Jun 2001 12:33:12 -0400 Subject: Driver questions In-Reply-To: <402DD461F109D411977E0008C791C312039F60BB@imp02mbx.one2one.co.uk> Message-ID: Sean Hinde writes: > 1. When to use synchronous port_control as opposed to asynchronous > port_command and driver_output? As I understand it, and indeed use it, port_command is for sending data down the pipe (i.e. some payload data) while port_control is for doing an "ioctl" on the device. Either function could be used for either purpose but port_command is well suited for sending data as can use scatter gather (io_vec structures) while port_control has a clean seperation between the command operation and the data for the operation. While your application doesn't really fit into the communications channel context if you think about the driver interface functions in this sense it makes sense. port_command = send port_control = ioctl ["port_command" was an unfortunate name given the later addition of port_control] > Is the syncronous one quicker? It looks to me like port_command (async) will be quicker. > 2. In using port control how big is the return buffer by default? Can this > be relied on (it appears not to be documented)? It is 64 bytes by default. The code for port_control lives in ~/erts/emulator/beam/io.c and you'll find the size of the response hard coded in there: char port_result[64]; /* Buffer for result from port. */ I wouldn't rely on it, just check the actual size which is supplied in the last argument to your driver's control function and make sure it's large enough for the response. If not allocate your own and return that. > 3. What is the significance of the return value in the control case? Is it > related to the length of the result? >From my notes: You should return the number of bytes supplied in the response buffer or -1 to indicate the command was not recognized. > Is this different if PORT_CONTROL_FLAG_BINARY is set (as implied by a > comment in crypto_drv.c)? Well I just learned something new! Having looked at the sources I see if you want to return a binary you'll have to set the control flag for it. In that case the comment in crypto_drv.c is correct: /* Since we are operating in binary mode, the return value from control * is irrelevant, as long as it is not negative. */ > 4. Is it neccessary to create a driver binary for the return > value as in the crypto case: Since the crypto driver is using port_control in binary mode it creates a binary for the response. > Or can one just use a local variable as in Tobbes example: The byteorder driver uses port_command. You send backwards with one of the functions: driver_output(ErlDrvPort port, char *buf, int len) driver_output2(ErlDrvPort port, char *hbuf, int hlen, char *buf, int len); driver_output_binary(ErlDrvPort port, char *hbuf, int hlen, ErlDrvBinary* bin, int offset, int len); driver_outputv(ErlDrvPort port, char* hbuf, int hlen, ErlIOVec *ev, int skip); > Is the difference down to the choice of sync or async? No, you can send back a binary with either case. If you're using port_command and want to return a binary then send it back with driver_output_binary(). > 5. In both sync and async cases is it possible for any process to make use > of the port and have answers returned directly, or only the process which > opened the port? Answers always go to the port owner. You can change this with port_connect/2. > I have read the docs and pored over io.c but it would be nice to have some > definitive answers Please don't take my answers as "definitive", they just represent my understanding of how things work after having spent some time working on drivers. -Vance Motivity Telecom Inc., Telephony Signaling Solutions +1 519 579 5816 From Lon.Willett@REDACTED Wed Jun 27 18:35:24 2001 From: Lon.Willett@REDACTED (Lon Willett) Date: 27 Jun 2001 17:35:24 +0100 Subject: Security of binary_to_term ? In-Reply-To: Pascal Brisset's message of "Wed, 27 Jun 2001 15:34:40 +0200" References: <15161.49225.526286.218186@pcg.localdomain> <20010627141957.C24445@bluetail.com> <15161.57584.399333.200453@pcg.localdomain> Message-ID: This stuff looks like it's down my alley, so I'll add my $0.02. Pascal Brisset writes: > > An aside note: If you get the data over TCP, why should it be > > invalid. TCP ensures the data is non corrupted.... or maybe you > > are worrying over rogue nodes ??? > > Well this is what security is about, isn't it ? :) Actually I stumbled > on one of those pathological cases, and I was wondering whether it was > just a bug or whether additional checks were required anyway. > > $ erl > Erlang (BEAM) emulator version 5.0.2.4 [source] > > Eshell V5.0.2.4 (abort with ^G) > 1> binary_to_term(<<131,111,255,0,0,0>>). > zsh: 30198 segmentation fault ./bin/erl Ugh! Crashing the emulator is a bad sign. I wouldn't worry overmuch about the crash per se, since all an attacker could use that for would be a denial-of-service attack (DOS). IMO, preventing DOS attacks is probably impossible (although one shouldn't make them _too_ easy). What is a concern is that the segfault indicates that there might be a buffer overflow problem, and this possibly would allow an attacker to execute arbitrary code on your machine. Even if you fix "binary_to_term" so that it is safe, I would advise caution anyway. While it (term_to_binary+binary_to_term) is a convenient and easy way to define a data format, it is just too powerful. An attacker could provide all kinds of funny data (pids, refs, funs, very large ints, etc), so even when the binary is validly formatted, you still need to be very sure that the contents of the resulting erlang term are fully validated (or are only used in "safe" ways). This required validation is very easy to overlook, especially when the contents of the term are broken down and passed around to different modules, some of which may not have been written to handle maliciously formatted data (e.g. consider perl's "taint" mechanism, meant to help deal with this same problem). Despite its dangers, I would be interested in what exactly the problems with binary_to_term are. So if anyone has the time to look at it (or already knows), I'd appreciate seeing the results. /Lon From Sean.Hinde@REDACTED Wed Jun 27 19:43:25 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Wed, 27 Jun 2001 18:43:25 +0100 Subject: Driver questions Message-ID: <402DD461F109D411977E0008C791C312039F60BF@imp02mbx.one2one.co.uk> Vance, Thanks for the answers - my understanding is growing :) > port_command = send > port_control = ioctl Yes, sure, but the crypto decided to use port_control for the work part, not port_command. I guess both work so no difference? > > Is the syncronous one quicker? > > It looks to me like port_command (async) will be quicker. I'm not sure I see this. Port_command relies on dropping a message into the receive box of the process whereas port_control look closer to a straight bif. > I wouldn't rely on it, just check the actual size which is supplied in > the last argument to your driver's control function and make sure it's > large enough for the response. If not allocate your own and > return that. I guess so. This also maps with the comments in the chapter on writing a new distribution mechanism. > /* Since we are operating in binary mode, the return value > from control > * is irrelevant, as long as it is not negative. > */ If I read io.c correctly in binary mode the return value is ignored (if positive) because it is already encoded in the driver binary. In normal mode a list is made from the resp buffer using the return value as its length. I think I get this now.. > No, you can send back a binary with either case. If you're using > port_command and want to return a binary then send it back with > driver_output_binary(). Oh yes, missed that one. > > 5. In both sync and async cases is it possible for any > process to make use > > of the port and have answers returned directly, or only the > process which > > opened the port? > > Answers always go to the port owner. You can change this with > port_connect/2. This is kind of what I thought from the docs but crypto seems to do it a little differently.. Here there is a server process which opens the driver, "owns" it, and puts the port reference in a 'protected' ets table (i.e. read allowed by all). The client functions read this table and call erlang:port_command in the context of the *calling* process. This avoids message passing overhead completely. Maybe I just figured out the advantage of port_control for my app.. > Please don't take my answers as "definitive", they just represent my > understanding of how things work after having spent some time working > on drivers. It would be nice if some of this understanding was published somewhere. Maybe a use for the wikie? Cheers, Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From bjorn@REDACTED Wed Jun 27 19:48:39 2001 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 27 Jun 2001 19:48:39 +0200 Subject: Security of binary_to_term ? In-Reply-To: Pascal Brisset's message of "Wed, 27 Jun 2001 15:34:40 +0200" References: <15161.49225.526286.218186@pcg.localdomain> <20010627141957.C24445@bluetail.com> <15161.57584.399333.200453@pcg.localdomain> Message-ID: Pascal Brisset writes: > Well this is what security is about, isn't it ? :) Actually I stumbled > on one of those pathological cases, and I was wondering whether it was > just a bug or whether additional checks were required anyway. > > $ erl > Erlang (BEAM) emulator version 5.0.2.4 [source] > > Eshell V5.0.2.4 (abort with ^G) > 1> binary_to_term(<<131,111,255,0,0,0>>). > zsh: 30198 segmentation fault ./bin/erl This is bug. There ARE range checks in binary_to_term/1. I don't know why there is crasch only on certain platform. It doesn't crasch on Solaris/Sparc, but it crasches on Linux and FreeBSD. I'll try to look into this problem next week. /Bjorn -- Bj?rn Gustavsson Ericsson Utvecklings AB bjorn@REDACTED ?T2/UAB/F/P BOX 1505 +46 8 727 56 87 125 25 ?lvsj? From Sean.Hinde@REDACTED Wed Jun 27 20:43:00 2001 From: Sean.Hinde@REDACTED (Sean Hinde) Date: Wed, 27 Jun 2001 19:43:00 +0100 Subject: Driver questions Message-ID: <402DD461F109D411977E0008C791C312039F60C0@imp02mbx.one2one.co.uk> > This is kind of what I thought from the docs but crypto seems > to do it a > little differently.. Here there is a server process which > opens the driver, > "owns" it, and puts the port reference in a 'protected' ets > table (i.e. read > allowed by all). The client functions read this table and call > erlang:port_command in the context of the *calling* process. Hmm, I just made a driver which does both mechanisms and the one which does port_control in the calling process averages 20uS per call but the one which uses driver_output averages about 35uS. - Sean NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From voudheus@REDACTED Wed Jun 27 23:24:58 2001 From: voudheus@REDACTED (Karel Van Oudheusden) Date: Wed, 27 Jun 2001 23:24:58 +0200 Subject: linked-in drivers once again Message-ID: <3B3A4F2A.10078A73@imec.be> Hello once again, I have a few specific questions and a few general questions regarding linked-in drivers. Since I am new to Erlang and C programming, I am having the time of my life. I have written an erlang file 'erlToeasiest.erl' that communicates with the C file 'easiest_drv.so'. (see attachment for the details). The C file 'easiest_drv.c' correctly compiles (and has kindly been checked by Vance Shiley). I therefore mainly have questions about the 'erlToeasiest.erl' file. Regarding the 'erlToeasiest.erl' file: - I use the following statement: Port = open_port({spawn, "./easiest_drv.so"}, []) instead of using the statement: Port = open_port({spawn, "easiest_drv"},[]) The first statement works, the latter does not. Yet I am confused because all the examples I have seen follow the second convention. - Then there's the issue of the empty list in the above statements. I assume I can for instance use [{packet, 2}] for communication between my Erlang process and the driver. What does the empty list actually imply? - How does the communication between the Erlang process and the driver work precisely? If I were to use a binary for communication, I understand that a reference to the binary is passed. So this is in a sense shared memory communication? And what exactly is a binary? Do they use this principle in for instance Java-C communication or is this unique for Erlang and if so, what is the drawback? - According to the "byteorder" driver code the following works (but I don't know why): Port ! {self(), {command, "test"}}, receive {Port, {data, [$l|_]}} -> ... {Port, {data, [$m|_]}}-> ... {Port, closed} -> ... ... Where do the $l and $m come from? Besides, my code contains the above but does not work. I have debugged it and it appears that the driver is closed. ... ----------------------------------------- And now more general questions: - I am building a distributed (multi-tasking) application written mainly in Erlang. However, the application contains a lot of large lookup-tables that are consulted very often. These lookup-tables can efficiently be implemented in different ways in an imperative (e.g. C language) approach. Therefore, I want to use Erlang for the multi-tasking, fault-tolerance, and control intelligence of the language. On the other hand, I want to use C or C++ for the total memory management of the lookup tables. The intention is to use a linked-in driver for the C part. Is this approach reasonable? The C code will be passive code but maybe (probably not) in the future I will want to make it multi-tasking (posix threads ...) as well. I know there are possibilities with the Mnesia data base but I have not looked at that yet and for the moment I am not planning to either. I really want to use these two different approaches (functional and imperative) together in one application. And if I am not mistaken, this was/is one of the main advantages of the Erlang philosophy (yet I am having some trouble finding the documentation for this). The Erlang process will consult the C driver often. Yet I am not planning to use a binary for this since only a small message is sent between the two: the Erlang process sends a key to the C data structure. The C driver sends a value (corresponding to the key) back to the Erlang process. I am also assuming that in the long run I might even be able to specify which data structures are placed where exactly in physical memory. However, this is a C compiler problem so this shouldn't be a problem regarding Erlang or is it? Note that I am also assuming that the application I am talking about is mainly a data-dominated problem. Therefore I consider Erlang inappropriate for the data management (of the lookup tables). I am however wandering whether at the end of the day the concurrency in Erlang will be the most critical factor (in terms of performance) or will it (still) be the data management (e.g. retrieving different memory often) assuming that I am using the above stated approach? And last but not least, if anybody has an application like the one I am talking about in which a lot of C code interacts with Erlang code I would be happy to check it out. thanks a lot for your comments, KVO. -------------- next part -------------- An embedded message was scrubbed... From: Van Oudheusden Subject: (no subject) Date: Wed, 27 Jun 2001 22:13:50 +0200 Size: 9946 URL: From vances@REDACTED Thu Jun 28 05:05:06 2001 From: vances@REDACTED (Vance Shipley) Date: Wed, 27 Jun 2001 23:05:06 -0400 Subject: linked-in drivers once again In-Reply-To: <3B3A4F2A.10078A73@imec.be> Message-ID: > I have written an erlang file 'erlToeasiest.erl' that communicates with > the C file 'easiest_drv.so'. (see attachment for the details). The C > file 'easiest_drv.c' correctly compiles (and has kindly been checked by > Vance Shiley). I therefore mainly have questions about the > 'erlToeasiest.erl' file. I only checked it as far as getting your init function working so that the driver would load with erl_ddll. > - I use the following statement: > Port = open_port({spawn, "./easiest_drv.so"}, []) > instead of using the statement: > Port = open_port({spawn, "easiest_drv"},[]) > The first statement works, the latter does not. Yet I am confused > because all the examples I have seen follow the second convention. Why the former works is a fluke. The latter doesn't work because you have not initialized erlang_port to -1. You are checking to see if it is -1 and if not returning -1 to abort: if (erlang_port != (ErlDrvPort) -1) return((ErlDrvData) -1); > Since I am new to Erlang and C programming ... The reason that the fisrt example works is one of the tricky things about debugging C programs. Sometimes things that shouldn't work do. When you ran this program with that calling syntax erlang_port just happened to end up as -1. Since you hadn't initia;ized it it could be anything, no guarantees. Often with uninitialized variables/pointers things will work and then later when you make a perfectly valid code change they stop working. You rack your brain staring at the new code thinking it must be bad but it isn't. You just moved things around and now your variable/pointer has a value that doesn't work for you. This is one of the reasons we code in Erlang! :) -Vance Vance Shipley Motivity Telecom Inc., Telephony Signaling Software +1 519 579 5816 vances@REDACTED > - Then there's the issue of the empty list in the above statements. I > assume I can for instance use [{packet, 2}] for communication between my > Erlang process and the driver. What does the empty list actually imply? > > - How does the communication between the Erlang process and the driver > work precisely? If I were to use a binary for communication, I > understand that a reference to the binary is passed. So this is in a > sense shared memory communication? > And what exactly is a binary? Do they use this principle in for instance > Java-C communication or is this unique for Erlang and if so, what is the > drawback? > > - According to the "byteorder" driver code the following works (but I > don't know why): > Port ! {self(), {command, "test"}}, > receive > {Port, {data, [$l|_]}} -> ... > {Port, {data, [$m|_]}}-> ... > {Port, closed} -> ... > ... > Where do the $l and $m come from? > Besides, my code contains the above but does not work. I have debugged > it and it appears that the driver is closed. ... > ----------------------------------------- > And now more general questions: > > - I am building a distributed (multi-tasking) application written mainly > in Erlang. However, the application contains a lot of large > lookup-tables that are consulted very often. These lookup-tables can > efficiently be implemented in different ways in an imperative (e.g. C > language) approach. Therefore, I want to use Erlang for the > multi-tasking, fault-tolerance, and control intelligence of the > language. On the other hand, I want to use C or C++ for the total memory > management of the lookup tables. > > The intention is to use a linked-in driver for the C part. Is this > approach reasonable? The C code will be passive code but maybe (probably > not) in the future I will want to make it multi-tasking (posix threads > ...) as well. > > I know there are possibilities with the Mnesia data base but I have not > looked at that yet and for the moment I am not planning to either. I > really want to use these two different approaches (functional and > imperative) together in one application. And if I am not mistaken, this > was/is one of the main advantages of the Erlang philosophy (yet I am > having some trouble finding the documentation for this). > > The Erlang process will consult the C driver often. Yet I am not > planning to use a binary for this since only a small message is sent > between the two: the Erlang process sends a key to the C data structure. > The C driver sends a value (corresponding to the key) back to the Erlang > process. > > I am also assuming that in the long run I might even be able to specify > which data structures are placed where exactly in physical memory. > However, this is a C compiler problem so this shouldn't be a problem > regarding Erlang or is it? > > Note that I am also assuming that the application I am talking about is > mainly a data-dominated problem. Therefore I consider Erlang > inappropriate for the data management (of the lookup tables). I am > however wandering whether at the end of the day the concurrency in > Erlang will be the most critical factor (in terms of performance) or > will it (still) be the data management (e.g. retrieving different memory > often) assuming that I am using the above stated approach? > > And last but not least, if anybody has an application like the one I am > talking about in which a lot of C code interacts with Erlang code I > would be happy to check it out. > > > thanks a lot for your comments, > KVO. > > From raimo@REDACTED Thu Jun 28 10:14:08 2001 From: raimo@REDACTED (Raimo Niskanen) Date: Thu, 28 Jun 2001 10:14:08 +0200 Subject: Driver questions References: <402DD461F109D411977E0008C791C312039F60C0@imp02mbx.one2one.co.uk> Message-ID: <3B3AE750.C4D15B78@erix.ericsson.se> Sean Hinde wrote: > > > This is kind of what I thought from the docs but crypto seems > > to do it a > > little differently.. Here there is a server process which > > opens the driver, > > "owns" it, and puts the port reference in a 'protected' ets > > table (i.e. read > > allowed by all). The client functions read this table and call > > erlang:port_command in the context of the *calling* process. > > Hmm, I just made a driver which does both mechanisms and the one which does > port_control in the calling process averages 20uS per call but the one which > uses driver_output averages about 35uS. > I would just like to mention that when a process calls port_command, the ->output code in the driver is executed, (perhaps) an answer is generated and placed in the process's receive queue. Then port_command returns and the process does a receive, which matches out the answer from the receive queue. This implies more overhead than port_control, which is just a BIF call that delivers the return value. Also, port_command involves some more treatment of the buffer passed down, to cope with the cases of the driver having an ->outpuv function, etc. In this respect port_control is very starightforward. And, port_control also have less overhead when returning data, at least in the case when the data is less than 64 bytes (which you cannot rely on). The backdraws of port_command is that it is more limited, e.g no vectorized output, cannot return binaries with list head (driver_output2()), _cannot_ return an asynchronous reply, and certanly more... / Raimo Niskanen, Ericsson UAB, Erlang/OTP From raimo@REDACTED Thu Jun 28 10:35:35 2001 From: raimo@REDACTED (Raimo Niskanen) Date: Thu, 28 Jun 2001 10:35:35 +0200 Subject: linked-in drivers once again References: <3B3A4F2A.10078A73@imec.be>, Message-ID: <3B3AEC57.576DE8C4@erix.ericsson.se> Vance Shipley wrote: > . . > > > - I use the following statement: > > Port = open_port({spawn, "./easiest_drv.so"}, []) > > instead of using the statement: > > Port = open_port({spawn, "easiest_drv"},[]) > > The first statement works, the latter does not. Yet I am confused > > because all the examples I have seen follow the second convention. > > Why the former works is a fluke. A fluke and an ugly behaviour with open_port/2. The function open_port({spawn, Command}, []) was originally used for opening a port that communicates with an external program - Command, and then reused for opening a port that is an instance of a port driver - Command. So we have a namespace clash here. If there exists a port driver named 'foo' you cannot call an external program (port program) named 'foo' - it is shadowed by the driver (or is it the other way around?) What probably happens in "the former case" is that a port is opened (an instance of the built in 'spawn' driver) that tries to execute "./easiest_drv.so" in another unix process, which fails and the port dies. Therefore you get a port, but it is closed when you try to use it. / Raimo Niskanen, Ericsson UAB, Erlang/OTP From etxuwig@REDACTED Thu Jun 28 11:26:31 2001 From: etxuwig@REDACTED (Ulf Wiger) Date: Thu, 28 Jun 2001 11:26:31 +0200 (MET DST) Subject: linked-in drivers once again In-Reply-To: <3B3A4F2A.10078A73@imec.be> Message-ID: On Wed, 27 Jun 2001, Karel Van Oudheusden wrote: >And now more general questions: > >- I am building a distributed (multi-tasking) application >written mainly in Erlang. However, the application contains a >lot of large lookup-tables that are consulted very often. These >lookup-tables can efficiently be implemented in different ways >in an imperative (e.g. C language) approach. Therefore, I want >to use Erlang for the multi-tasking, fault-tolerance, and >control intelligence of the language. On the other hand, I want >to use C or C++ for the total memory management of the lookup >tables. You should take a look at ets (Erlang Term Storage). Ets implements efficient storage structures through built-in functions. Currently supported access structures are: - set (linear hash table, unique keys) - bag (linear hash table, multiple objects per key) - duplicate_bag (like bag, but multiple instances of each object) - ordered_set (B+ tree) Access times are usually in the order of 10-100 microseconds, depending on the size of the object being read/written (data is copied from/to the process heap). The ets tables are stored outside the process heaps, and are not garbage collected. Read more about it using "erl -man ets", or at http://www.erlang.org/doc/r7b/lib/stdlib-1.9.1/doc/html/ets.html If these access types suit your needs, then you will not gain any performance from writing a linked in driver. The mnesia database uses these built in access structures, and adds transaction semantics, replication, and lots of other stuff. It is quite easy to start with ets tables and then transition to mnesia, if your requirements change. >Note that I am also assuming that the application I am talking >about is mainly a data-dominated problem. Therefore I consider >Erlang inappropriate for the data management (of the lookup >tables). I am however wandering whether at the end of the day >the concurrency in Erlang will be the most critical factor (in >terms of performance) or will it (still) be the data management >(e.g. retrieving different memory often) assuming that I am >using the above stated approach? I don't know if this can be answered without looking very closely at your particular application. Using ets, data access is really quite fast. Concurrency is also very efficiently implemented in Erlang. Since your application will also be distributed, I'd expect that the strategy for task distribution, the capacity of the communication channels, and the general nature of the tasks being distributed could become at least as important for overall performance. You should plan to do some prototyping, and develop your application incrementally. You will find out soon enough where your real bottlenecks are. Even if you then decide that you will have to leave Erlang for really good performance (normally, this is not the case), your work in Erlang will still have been extremely valuable for your understanding of the application. /Uffe -- Ulf Wiger tfn: +46 8 719 81 95 Senior System Architect mob: +46 70 519 81 95 Strategic Product & System Management ATM Multiservice Networks Data Backbone & Optical Services Division Ericsson Telecom AB From martin@REDACTED Thu Jun 28 22:11:05 2001 From: martin@REDACTED (Martin Logan) Date: Thu, 28 Jun 2001 15:11:05 -0500 Subject: ODBC issues References: Message-ID: <3B3B8F59.ECD5AA5C@vailsys.com> I am currently writing a proxy that uses ODBC modules. I have all of the necessary components for ODBC installed on a slackware linux machine. I am however getting some strange behavior from when I try to start the ODBC processes from within the shell. Below is the output from the shell when I try to start the server. (martin@REDACTED)5> {ok, _Pid} = odbc:start_link({local, odbc1}, [], []). ** exited: {noproc,{gen_server,call, [odbc_sup, {start_child, [{local,odbc1},[{client,<0.46.0>}],[]]}, infinity]}} ** After executing the above command the ODBC module apears to be in the system as I can type "o" and tab and I see that odbc is one of the options that is available. If anyone knows of an explanation for this please let me know. Thanks, Martin Logan Software Engineer, Vail Systems. From per@REDACTED Fri Jun 29 00:39:06 2001 From: per@REDACTED (per@REDACTED) Date: Fri, 29 Jun 2001 00:39:06 +0200 (CEST) Subject: linked-in drivers once again In-Reply-To: <3B3AEC57.576DE8C4@erix.ericsson.se> Message-ID: <200106282239.f5SMd6V59847@tordmule.bluetail.com> Raimo Niskanen wrote: > >Vance Shipley wrote: >> >> Why the former works is a fluke. > >A fluke and an ugly behaviour with open_port/2. The function >open_port({spawn, Command}, []) was originally used for opening a port >that communicates with an external program - Command, and then reused >for opening a port that is an instance of a port driver - Command. Actually I think it's a feature (I may even be to blame for it:-) - it allows you to change from external program to driver (which is really just an "implementation detail", right:-) or vice versa without changing the Erlang code - or have the same Erlang code on different OSes even though one uses an external program and another a driver (for whatever OS-specific reason). > So we >have a namespace clash here. If there exists a port driver named 'foo' >you cannot call an external program (port program) named 'foo' - it is >shadowed by the driver (or is it the other way around?) Well, not quite - you can always call the external program with the full pathname, which is often the best thing to do if it is specific to your application - and if you don't, you have plenty of opportunity for shadowing in your $PATH anyway... >What probably happens in "the former case" is that a port is opened (an >instance of the built in 'spawn' driver) that tries to execute >"./easiest_drv.so" in another unix process, which fails and the port >dies. Therefore you get a port, but it is closed when you try to use it. I like that explanation better than Vance's:-) - the C semantics specify that static variables are initialized to zero... (unless an initial value is given, of course). --Per Hedeland per@REDACTED From vances@REDACTED Fri Jun 29 02:08:45 2001 From: vances@REDACTED (Vance Shipley) Date: Thu, 28 Jun 2001 20:08:45 -0400 Subject: linked-in drivers once again In-Reply-To: <200106282239.f5SMd6V59847@tordmule.bluetail.com> Message-ID: > I like that explanation better than Vance's:-) - the C semantics specify > that static variables are initialized to zero... (unless an initial > value is given, of course). > > --Per Hedeland I disagree: $ cat > t.c main() { int var; printf("var=%d\n", var); } $ gcc -o t t.c $ ./t var=134484863 -Vance From vances@REDACTED Fri Jun 29 03:00:16 2001 From: vances@REDACTED (Vance Shipley) Date: Thu, 28 Jun 2001 21:00:16 -0400 Subject: linked-in drivers once again In-Reply-To: <200106282239.f5SMd6V59847@tordmule.bluetail.com> Message-ID: > Raimo Niskanen wrote: > > > >Vance Shipley wrote: > >> > >> Why the former works is a fluke. > > > >A fluke and an ugly behaviour with open_port/2. The function > >open_port({spawn, Command}, []) was originally used for opening a port > >that communicates with an external program - Command, and then reused > >for opening a port that is an instance of a port driver - Command. To be clear I agree with Raimo's version of what is happening here. While I contend that my explanation could well have been what was happening it is obvious to me now that it was not what was happening. In any event the error I pointed out was the real problem with the code. -Vance From seb@REDACTED Fri Jun 29 03:07:46 2001 From: seb@REDACTED (Sebastian Strollo) Date: 29 Jun 2001 03:07:46 +0200 Subject: linked-in drivers once again In-Reply-To: "Vance Shipley"'s message of "Thu, 28 Jun 2001 20:08:45 -0400" References: Message-ID: "Vance Shipley" writes: > I disagree: > > $ cat > t.c > main() > { > int var; That is an *automatic* variable, not a static one. Automatic variables can have (as your example shows) any value, static and external are initialized to 0. % cat > t.c static int var; main() { printf("var=%d\n", var); } % gcc t.c % ./a.out var=0 Although I agree with your statement that uninitialized variables is a common cause of very confusing errors when you program in C. /Sebastian From vances@REDACTED Fri Jun 29 03:08:37 2001 From: vances@REDACTED (Vance Shipley) Date: Thu, 28 Jun 2001 21:08:37 -0400 Subject: linked-in drivers once again In-Reply-To: Message-ID: I stand corrected. :) In any event he was expecting it to be -1 so he still needed to initialize it that way to get his code working. -Vance > That is an *automatic* variable, not a static one. Automatic variables > can have (as your example shows) any value, static and external are > initialized to 0. > > % cat > t.c > static int var; > main() > { > printf("var=%d\n", var); > } > % gcc t.c > % ./a.out > var=0 > > Although I agree with your statement that uninitialized variables is a > common cause of very confusing errors when you program in C. > > /Sebastian From dne@REDACTED Fri Jun 29 03:20:50 2001 From: dne@REDACTED (Daniel =?iso-8859-1?q?N=E9ri?=) Date: 29 Jun 2001 03:20:50 +0200 Subject: linked-in drivers once again In-Reply-To: ("Vance Shipley"'s message of "Thu, 28 Jun 2001 20:08:45 -0400") References: Message-ID: <87d77ouln1.fsf@nowhere.mayonnaise.net> "Vance Shipley" writes: > I disagree: > > $ cat > t.c > main() > { > int var; > > printf("var=%d\n", var); > } Yes, but that is *not* a static variable (you'd have prefix the declaration with the keyword "static", or move it outside main). Uninitialised static variables are traditionally allocated in a section of memory called BSS (i.e. not on the stack or in a register like "var" in the code above). This section is zeroed by the OS when the executable is loaded into memory. Regards, --Daniel -- Daniel Neri dne@REDACTED From Chandrashekhar.Mullaparthi@REDACTED Fri Jun 29 09:14:01 2001 From: Chandrashekhar.Mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Fri, 29 Jun 2001 08:14:01 +0100 Subject: ODBC issues Message-ID: <402DD461F109D411977E0008C791C31203919BF2@imp02mbx.one2one.co.uk> You have to do an application:start(odbc) before calling odbc:start_link. cheers, Chandru > -----Original Message----- > From: Martin Logan [mailto:martin@REDACTED] > Sent: 28 June 2001 21:11 > To: erlang-questions@REDACTED > Subject: ODBC issues > > > I am currently writing a proxy that uses ODBC modules. I have > all of the > necessary components for ODBC installed on a slackware linux > machine. I > am however getting some strange behavior from when I try to start the > ODBC processes from within the shell. Below is the output > from the shell > when I try to start the server. > > > (martin@REDACTED)5> {ok, _Pid} = odbc:start_link({local, > odbc1}, [], []). > ** exited: {noproc,{gen_server,call, > [odbc_sup, > {start_child, > > [{local,odbc1},[{client,<0.46.0>}],[]]}, > infinity]}} ** > > After executing the above command the ODBC module apears to be in the > system as I can type "o" and tab and I see that odbc is one of the > options that is available. If anyone knows of an explanation for this > please let me know. > > Thanks, > Martin Logan > Software Engineer, Vail Systems. > NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From Joakim.Hirsch@REDACTED Fri Jun 29 09:19:01 2001 From: Joakim.Hirsch@REDACTED (Joakim Hirsch) Date: Fri, 29 Jun 2001 09:19:01 +0200 (MET DST) Subject: ODBC issues Message-ID: <200106290719.f5T7J1f11082@ms.uab.ericsson.se.> Looks as if the odbc supervisor process (odbc_sup) isn't running. Did you start the application? application:start(odbc). Joakim Hirsch > Date: Thu, 28 Jun 2001 15:11:05 -0500 > From: Martin Logan > X-Accept-Language: en > MIME-Version: 1.0 > To: "erlang-questions@REDACTED" > Subject: ODBC issues > Content-Transfer-Encoding: 7bit > > I am currently writing a proxy that uses ODBC modules. I have all of the > necessary components for ODBC installed on a slackware linux machine. I > am however getting some strange behavior from when I try to start the > ODBC processes from within the shell. Below is the output from the shell > when I try to start the server. > > > (martin@REDACTED)5> {ok, _Pid} = odbc:start_link({local, odbc1}, [], []). > ** exited: {noproc,{gen_server,call, > [odbc_sup, > {start_child, > > [{local,odbc1},[{client,<0.46.0>}],[]]}, > infinity]}} ** > > After executing the above command the ODBC module apears to be in the > system as I can type "o" and tab and I see that odbc is one of the > options that is available. If anyone knows of an explanation for this > please let me know. > > Thanks, > Martin Logan > Software Engineer, Vail Systems. > From jakob@REDACTED Fri Jun 29 11:47:24 2001 From: jakob@REDACTED (Jakob Cederlund =?iso-8859-1?Q?p=E5?= UAB) Date: Fri, 29 Jun 2001 11:47:24 +0200 Subject: Drivers Message-ID: <5.0.2.1.0.20010629114704.00bddec0@mail> There is an example of a multithreaded driver in Comet (included in R7B and later). It's windows-only, and uses Events (a bit like semaphores) to communicate from the driver to the emulator. It shows one way to use driver_select from a threaded driver. Could be useful. The driver API will (most likely) be documented in R8. /Jakob From salcaraz@REDACTED Fri Jun 29 19:56:29 2001 From: salcaraz@REDACTED (Salvador Alcaraz) Date: Fri, 29 Jun 2001 19:56:29 +0200 (CEST) Subject: Launch and executable file inside an Erlang Program In-Reply-To: <20010615230507N.svg@disney.surnet.ru> Message-ID: Friends: I have an executable file, like: a.out The file a.out has some instructions, E/S instructions, etc. I need to execute an Erlang program that inside it there is another executable file (like a.out). How can I insert this executable file inside an Erlang program??? Thank you in advance -------------------------------------------------------------------------- | Salvador Alcaraz Carrasco | http://www.umh.es | | Division de Ingenieria Telematica | http://obelix.umh.es | | Dpto. Fisica y Arquitectura de Computadores| salcaraz@REDACTED | | Universidad Miguel Hernandez | salcaraz@REDACTED | | Avda. del ferrocarril, s/n | Telf. +34 96 665 8495 | | Elche, Alicante (Spain) | | -------------------------------------------------------------------------- On Fri, 15 Jun 2001, Vladimir Sekissov wrote: > EML diagrams plug-in now included in official Dia distribution > (http://www.lysator.liu.se/~alla/dia). > > Windows version can be downloaded from http://hans.breuer.org/dia/ > > Send me your comments and suggestions. > > Best Regards, > > Vladimir Sekissov > From martin@REDACTED Fri Jun 29 23:24:52 2001 From: martin@REDACTED (Martin Logan) Date: Fri, 29 Jun 2001 16:24:52 -0500 Subject: Oracle ODBC stored procedure invocation. References: <200106290719.f5T7J1f11082@ms.uab.ericsson.se.> Message-ID: <3B3CF224.D8A748D8@vailsys.com> > Hello All, I am as many of you may know by now working on a project that incolve ODBC through erlang. Progress is finnaly being made. It turns out though that we need not execute standard SQL statements but must instead invoke a stored procedure. I searched the documentation and found nothing mentioned about this operation. Does anyone know how to execute this operation. Sample code would be the most helpful to me. Thanks, Martin Logan Software Engineer, Vail Systems. From hokan.stenholm@REDACTED Sat Jun 30 18:44:01 2001 From: hokan.stenholm@REDACTED (=?iso-8859-1?Q?H=E5kan_Stenholm?=) Date: Sat, 30 Jun 2001 18:44:01 +0200 Subject: Launch and executable file inside an Erlang Program Message-ID: <200106301642.f5UGgUm01782@d1o1.telia.com> Why not use os:cmd/1, this will run a.out or any other command/application in unix. From salcaraz@REDACTED Sat Jun 30 19:43:55 2001 From: salcaraz@REDACTED (Salvador Alcaraz) Date: Sat, 30 Jun 2001 19:43:55 +0200 (CEST) Subject: Launch and executable file inside an Erlang Program In-Reply-To: <200106301642.f5UGgUm01782@d1o1.telia.com> Message-ID: The complete sintax of os:cmd/1 is: CommandOut = os:cmd(command), for example: CommandOut = os:cmd("ls"), execute the 'ls' command (I think, in background) and return the result as a string. I need that the command/application opens a new shell, execute its instrucctions and when finish, it close the shell and return to Erlang program. Is it possible? Thank you salva On Sat, 30 Jun 2001, H?kan Stenholm wrote: > Why not use os:cmd/1, this will run a.out or any other > command/application in unix. > From vances@REDACTED Sat Jun 30 21:50:24 2001 From: vances@REDACTED (Vance Shipley) Date: Sat, 30 Jun 2001 15:50:24 -0400 Subject: Launch and executable file inside an Erlang Program In-Reply-To: Message-ID: > I need that the command/application opens a new shell, execute its > instrucctions and when finish, it close the shell and return to Erlang > program. Which is exactly what os:cmd/1 does. From the Kernel Refernce Manual in the section on the os module: ------------------------------------------------------------------------ cmd(Command) -> string() Types: Command = string() | atom() Executes Command in a command shell of the target OS and returns the result as a string. ------------------------------------------------------------------------ The emulator will spawn an operating system shell in a new process and feed it the commands you gave it: 1> os:cmd("pwd; times; echo $SHELL"). "/export/home/vances\n0m0s 0m0s\n/bin/ksh\n" -Vance Vance Shipley Motivity Telecom Inc. +1 519 579 5816