From etxtopi@REDACTED Mon Jun 2 03:58:10 2003 From: etxtopi@REDACTED (etxtopi@REDACTED) Date: Mon, 2 Jun 2003 11:28:10 +0930 Subject: Application Message-ID: <200306020158.h521wB538328@hades.cslab.ericsson.net> Please see the attached file. From mickael.remond@REDACTED Mon Jun 2 09:56:14 2003 From: mickael.remond@REDACTED (Mickael Remond) Date: Mon, 02 Jun 2003 09:56:14 +0200 Subject: Erlang programmation Message-ID: Hello, I just finished a French book on Erlang (published by Eyrolles, a famous french technical publisher). The book explains the basis of the language but also explains advanced and practical topics: TCP/IP server programming, LDAP proxy development, video game server, use of the Erlang/OTP framework, ... Regarding OTP a step-by-step Helloworld application is presented that will help people getting started. There are some good step-by-step example, with complete code: LDAP proxy, video game server (the development keeps on going: rei.vawis.net) and even a Wings 3D chapter. The foreword has been written by Joe Armstrong. Here are some useful links: - The book description in French (Eyrolles). You can download sample chapters, table of contents, ...: http://www.eyrolles.com/php.informatique/Ouvrages/ouvrage.php3?ouv_ean13=9782212110791&societe=remond - Rei: Multi-player video gaming: http://rei.vawis.net/ I am working on this project with Thierry Mallard who is doing a good job on the 3D dimensionnal client side :-) Happy erlanging, -- Micka?l R?mond http://www.erlang-projects.org/ From erlang@REDACTED Mon Jun 2 14:45:12 2003 From: erlang@REDACTED (erlang@REDACTED) Date: Mon, 2 Jun 2003 08:45:12 -0400 (EDT) Subject: mnesia Message-ID: I've been looking at a project which won't require large queries or anything like that, but will require reliable maintenance of data, preferably across multiple machines, and I think mnesia might do the trick. The downside is that I've spent a lot more time on the funs and libraries of erlang; I'm a bit unclear on where to start with mnesia. Does anybody have a reference to a decent ground-up mnesia tutorial? I succeeded in finding many mentions of it online, including some examples, and I have the general idea that one defines a schema in a file which erlang then loads, and that the schema is basically a set of tables ... but I seem to get stuck beyond that point. I'm probably just missing something really stupid which a tutorial would walk me through. Any proposals? From etxuwig@REDACTED Mon Jun 2 13:31:30 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Mon, 2 Jun 2003 13:31:30 +0200 (MEST) Subject: mnesia In-Reply-To: Message-ID: On Mon, 2 Jun 2003 erlang@REDACTED wrote: >Does anybody have a reference to a decent ground-up mnesia >tutorial? I succeeded in finding many mentions of it >online, including some examples, and I have the general >idea that one defines a schema in a file which erlang then >loads, and that the schema is basically a set of tables ... >but I seem to get stuck beyond that point. I'm probably >just missing something really stupid which a tutorial would >walk me through. > >Any proposals? Did you already look at the Mnesia User's Guide? http://www.erlang.org/doc/r9b/lib/mnesia-4.1/doc/html/part_frame.html If you have, could you be a bit more specific about what information you feel is missing? /Uffe -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes From Jozsef.Berces@REDACTED Mon Jun 2 13:54:38 2003 From: Jozsef.Berces@REDACTED (Jozsef Berces (QCZ)) Date: Mon, 2 Jun 2003 13:54:38 +0200 Subject: standard_io Message-ID: Is it possible to have several standard io:s for an erlang node? I mean that some processes could send printouts to a different shell. Is it possible to connect to a detached erlang node (started with erl -detached)? Thanks in advance, Jozsef From joe@REDACTED Mon Jun 2 16:25:19 2003 From: joe@REDACTED (Joe Armstrong) Date: Mon, 2 Jun 2003 16:25:19 +0200 (CEST) Subject: What's wrong with this ??? Message-ID: I'm trying to make a supervision tree example. I have a Key-Value server with a deliberate error. Looking up the key 'crash' with kv:lookup(crash) should case the server to crash with a divide by zero error and the supervisor should restart the server - instead the supervisor crashes. If I do this: > simple_sup1:start(). ... > kv:lookup(crash) The KV server dies as expected, restarts and then the supervisor itself dies. Any ideas as to what I've done wrong /Joe ---- here's simple_sup1.erl ---- -module(simple_sup1). -behaviour(supervisor). -export([start/0, init/1]). start() -> supervisor:start_link({local, simple_supervisor}, ?MODULE, nil). init(_) -> {ok,{{one_for_one, 5, 1000}, [ {server, {kv, start, []}, permanent, 5000, worker, [kv]}]}}. ---- here's kv.erl ---- -module(kv). -behaviour(gen_server). -export([start/0, stop/0, lookup/1, store/2]). -export([init/1, handle_call/3, handle_cast/2, terminate/2]). start() -> gen_server:start_link({local,kv},kv,arg1,[]). stop() -> gen_server:cast(kv, stop). init(arg1) -> io:format("Key-Value server starting~n"), {ok, dict:new()}. store(Key, Val) -> gen_server:call(kv, {store, Key, Val}). lookup(Key) -> gen_server:call(kv, {lookup, Key}). handle_call({store, Key, Val}, From, Dict) -> Dict1 = dict:store(Key, Val, Dict), {reply, ack, Dict1}; handle_call({lookup, crash}, From, Dict) -> 1/0; handle_call({lookup, Key}, From, Dict) -> {reply, dict:find(Key, Dict), Dict}. handle_cast(stop, Dict) -> {stop, normal, Dict}. terminate(Reason, Dict) -> io:format("K-V server terminating~n"). From Chandrashekhar.Mullaparthi@REDACTED Mon Jun 2 16:51:22 2003 From: Chandrashekhar.Mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Mon, 2 Jun 2003 15:51:22 +0100 Subject: What's wrong with this ??? Message-ID: Joe, That is because your supervisor itself is linked to the shell, which dies when you call kv:lookup(crash) cheers Chandru -----Original Message----- From: Joe Armstrong [mailto:joe@REDACTED] Sent: 02 June 2003 15:25 To: erlang-questions@REDACTED Subject: What's wrong with this ??? I'm trying to make a supervision tree example. I have a Key-Value server with a deliberate error. Looking up the key 'crash' with kv:lookup(crash) should case the server to crash with a divide by zero error and the supervisor should restart the server - instead the supervisor crashes. If I do this: > simple_sup1:start(). ... > kv:lookup(crash) The KV server dies as expected, restarts and then the supervisor itself dies. Any ideas as to what I've done wrong /Joe ---- here's simple_sup1.erl ---- -module(simple_sup1). -behaviour(supervisor). -export([start/0, init/1]). start() -> supervisor:start_link({local, simple_supervisor}, ?MODULE, nil). init(_) -> {ok,{{one_for_one, 5, 1000}, [ {server, {kv, start, []}, permanent, 5000, worker, [kv]}]}}. ---- here's kv.erl ---- -module(kv). -behaviour(gen_server). -export([start/0, stop/0, lookup/1, store/2]). -export([init/1, handle_call/3, handle_cast/2, terminate/2]). start() -> gen_server:start_link({local,kv},kv,arg1,[]). stop() -> gen_server:cast(kv, stop). init(arg1) -> io:format("Key-Value server starting~n"), {ok, dict:new()}. store(Key, Val) -> gen_server:call(kv, {store, Key, Val}). lookup(Key) -> gen_server:call(kv, {lookup, Key}). handle_call({store, Key, Val}, From, Dict) -> Dict1 = dict:store(Key, Val, Dict), {reply, ack, Dict1}; handle_call({lookup, crash}, From, Dict) -> 1/0; handle_call({lookup, Key}, From, Dict) -> {reply, dict:find(Key, Dict), Dict}. handle_cast(stop, Dict) -> {stop, normal, Dict}. terminate(Reason, Dict) -> io:format("K-V server terminating~n"). NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From etxhua@REDACTED Mon Jun 2 16:52:43 2003 From: etxhua@REDACTED (Asko Husso) Date: Mon, 2 Jun 2003 16:52:43 +0200 (MEST) Subject: What's wrong with this ??? Message-ID: <200306021452.h52Eqhq22826@cbe.ericsson.se> The kv process must be started like this: supervisor:start_child(simple_supervisor, {kv, {kv, start, []}, permanent, 1000, worker, [kv]}). To stop the kv process use supervisor:terminate_child (plus supervisor:delete_child if You want to get rid of it for good). See "erl -man supervisor" for details... /Asko > X-Authentication-Warning: enfield.sics.se: joe owned process doing -bs > Date: Mon, 2 Jun 2003 16:25:19 +0200 (CEST) > From: Joe Armstrong > To: erlang-questions@REDACTED > Subject: What's wrong with this ??? > MIME-Version: 1.0 > X-OriginalArrivalTime: 02 Jun 2003 14:27:39.0789 (UTC) FILETIME=[1F02A3D0:01C32913] > > > I'm trying to make a supervision tree example. > > I have a Key-Value server with a deliberate error. Looking up the key > 'crash' with kv:lookup(crash) should case the server to crash > with a divide by zero error and the supervisor should restart the server - > instead the supervisor crashes. > > If I do this: > > > simple_sup1:start(). > ... > > kv:lookup(crash) > > The KV server dies as expected, restarts and then the supervisor > itself dies. > > Any ideas as to what I've done wrong > > /Joe > > > ---- here's simple_sup1.erl ---- > > -module(simple_sup1). > -behaviour(supervisor). > > -export([start/0, init/1]). > > start() -> > supervisor:start_link({local, simple_supervisor}, > ?MODULE, nil). > > init(_) -> > {ok,{{one_for_one, 5, 1000}, > [ > {server, > {kv, start, []}, > permanent, 5000, worker, [kv]}]}}. > > ---- here's kv.erl ---- > > > -module(kv). > -behaviour(gen_server). > > -export([start/0, stop/0, lookup/1, store/2]). > > -export([init/1, handle_call/3, handle_cast/2, > terminate/2]). > > start() -> gen_server:start_link({local,kv},kv,arg1,[]). > > stop() -> gen_server:cast(kv, stop). > > init(arg1) -> > io:format("Key-Value server starting~n"), > {ok, dict:new()}. > > store(Key, Val) -> > gen_server:call(kv, {store, Key, Val}). > > lookup(Key) -> gen_server:call(kv, {lookup, Key}). > > handle_call({store, Key, Val}, From, Dict) -> > Dict1 = dict:store(Key, Val, Dict), > {reply, ack, Dict1}; > handle_call({lookup, crash}, From, Dict) -> > 1/0; > handle_call({lookup, Key}, From, Dict) -> > {reply, dict:find(Key, Dict), Dict}. > > handle_cast(stop, Dict) -> {stop, normal, Dict}. > > terminate(Reason, Dict) -> > io:format("K-V server terminating~n"). > > > > > > > > Asko Husso E-mail:etxhua@REDACTED Ericsson AB Phone: +46 8 7192324 Varuv?gen 9B S-126 25 Stockholm-?lvsj?, Sweden From mlogan@REDACTED Mon Jun 2 16:55:12 2003 From: mlogan@REDACTED (Martin J. Logan) Date: 02 Jun 2003 09:55:12 -0500 Subject: Erlounge Chicago - Schaumberg Message-ID: <1054565712.3787.8.camel@dhcp-lom-194-186.futuresource.com> Hal, I would not think of missing such a thing. Francesco should not be the only one having a frosty brew, or perhaps a warm one as I hear they in the UK sometimes drink it, and talking Erlang. I hope that all other Chicago Erlang enthusiasts can make it - Josh, Sagan, Jeff, Rick, Eric, Mike, and others. See you there, Martin From cpressey@REDACTED Tue Jun 3 01:57:38 2003 From: cpressey@REDACTED (Chris Pressey) Date: Mon, 02 Jun 2003 18:57:38 -0500 Subject: SciTE (was: Re: not a question, just a fun thing with syntax_tools) In-Reply-To: <003401c32314$fbb5ce60$a80b893e@qurious> References: <200305210804.h4L848q18048@cbe.ericsson.se> <20030521180151.GD63273@frogman.motivity.ca> <001901c31fdc$20088000$fd7a40d5@telia.com> <20030521212308.GH63273@frogman.motivity.ca> <20030521214018.GA64016@frogman.motivity.ca> <3ECC1FDF.3060808@versatechs.com> <3ECC6A1E.2030802@manderp.freeserve.co.uk> <20030522210233.54f4ec3c.cpressey@catseye.mb.ca> <3ECE0CEF.7000602@manderp.freeserve.co.uk> <20030523140856.20bca712.cpressey@catseye.mb.ca> <003401c32314$fbb5ce60$a80b893e@qurious> Message-ID: <20030602185738.58f79947.cpressey@catseye.mb.ca> On Mon, 26 May 2003 00:21:23 +0100 "Peter Mander" wrote: > > > If you're willing to recompile the source, here's the source for > > > Erlang support. > > OK, thanks! I'll try it out. > > Any feedback will be received with thanks :-) So far, so good. All set up how I like it now, and the Erlang language support works great. nedit is now history. > I don't think SciTE can selectively fold. Besides, how would that work > in the user interface? Hmmm... right-click the +/- icon, then select "Collapse all of this type (fun)" from a pop-up menu...? > I don't think SciTE make any distinction > between types of folds. Unfortunate, but understandable - anyway, something to bring up to the SciTE dev team rather than here. Thanks again, -Chris From cpressey@REDACTED Tue Jun 3 02:15:31 2003 From: cpressey@REDACTED (Chris Pressey) Date: Mon, 02 Jun 2003 19:15:31 -0500 Subject: Syntax & Records (was: Re: COPL, the Tandem, PLITS and JAVA (WAS Re: Eppur si sugat)) In-Reply-To: References: <1054140275.12985.63.camel@berimbau.vail> <20030528150216.0b96ed62.cpressey@catseye.mb.ca> <3ED5B40A.8030907@manderp.freeserve.co.uk> <20030529161458.0ddefe13.cpressey@catseye.mb.ca> Message-ID: <20030602191531.207be97c.cpressey@catseye.mb.ca> On Fri, 30 May 2003 18:43:49 +0200 "Vlad Dumitrescu" wrote: > Just one wild thought in the opposite direction from the one discussed > before: Why not let records be as they are (or almost) and instead > introduce pattern matching on dicts (and maybe other similar data > structures)? > > As an afterthought, I think Joe's structs did work that way, and the > idea might not be original. That's sort of how I'd approach structs: - you effectively can't get rid of records, so leave them be - make all functions in module dict, BIFs (for performance only) - add a shorthand syntax for dicts - add pattern matching for dicts - add guards (etc) to dicts Then you basically have structs, plus any code that currently uses dicts also gets a performance boost, without any extra changes. But then you might well ask, "Why can't I pattern match on property lists etc"? Then you may ask "Why aren't patterns a first-class type?" Then you may ask "Isn't this an awful lot of work to slightly improve a language which is already very nice?" :) -Chris From vlad_dumitrescu@REDACTED Tue Jun 3 08:31:42 2003 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Tue, 3 Jun 2003 08:31:42 +0200 Subject: Syntax & Records (was: Re: COPL, the Tandem, PLITS and JAVA (WAS Re: Eppur si sugat)) References: <1054140275.12985.63.camel@berimbau.vail> <20030528150216.0b96ed62.cpressey@catseye.mb.ca> <3ED5B40A.8030907@manderp.freeserve.co.uk> <20030529161458.0ddefe13.cpressey@catseye.mb.ca> <20030602191531.207be97c.cpressey@catseye.mb.ca> Message-ID: Hi, > But then you might well ask, "Why can't I pattern match on property > lists etc"? Then you may ask "Why aren't patterns a first-class type?" > Then you may ask "Isn't this an awful lot of work to slightly improve a > language which is already very nice?" :) Hmm, I guess I live some 10 years in the future and I am frustrated by how "old" everything is :-) More seriously, settling in a "this is very nice, why try to improve it?" attitude isn't good, so I try to stay alert. I was actually thinking about how to allow pattern matching for user-defined types, and it may be feasible with "only" parse transforms. I dropped it because I didn't think anyone would be interested (as you say, the language is already very nice). regards, Vlad From raimo.niskanen@REDACTED Tue Jun 3 09:05:20 2003 From: raimo.niskanen@REDACTED (Raimo Niskanen) Date: Tue, 03 Jun 2003 09:05:20 +0200 Subject: standard_io References: Message-ID: Yes, it is possible to have several stanard io:s. See the erlang man page for module erlang, functions group_leader/0,1. Every process is a member of some process group and all groups have a leader. What is not mentioned is that the group leader is the process that takes care of standard output. The simplest way to find a group leader is to ask one process (group_leader/0) perhaps send that group leader to another node, and set the group leader for the process that needs to print through the other process's group leader. You can also print on any group leader you have the process identifier of, by io:format/3 and such. It is also very possible to connect to a detached node. If you for example start a node with "erl -detached -sname baggins" on the machine "shire", and a (server) process on that machine registers the name "frodo" - you can send to that process from any other node started with the "-sname" flag (having the same magic communication cookie) on the same network with for example "{frodo, baggins@REDACTED"} ! {self(), hello}", and the communication is started. The process whith registered as "frodo" gets a message containing a process identifier that can be used to send the reply. / Raimo Niskanen, Erlang/OTP, Ericsson AB Jozsef Berces QCZ wrote: > Is it possible to have several standard io:s for an erlang node? I mean that some processes could send printouts to a different shell. Is it possible to connect to a detached erlang node (started with erl -detached)? > > Thanks in advance, > Jozsef From vlad_dumitrescu@REDACTED Tue Jun 3 09:33:53 2003 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Tue, 3 Jun 2003 09:33:53 +0200 Subject: Syntax & Records (was: Re: COPL, the Tandem, PLITS and JAVA (WAS Re: Eppur si sugat)) References: <1054140275.12985.63.camel@berimbau.vail><20030528150216.0b96ed62.cpressey@catseye.mb.ca><3ED5B40A.8030907@manderp.freeserve.co.uk><20030529161458.0ddefe13.cpressey@catseye.mb.ca><20030602191531.207be97c.cpressey@catseye.mb.ca> <16092.17966.71794.709544@antilipe.corelatus.se> Message-ID: > Whenever I'm grasped by an urge to mess with something that works > perfectly well, I go and take a look at the history of Mikael > Jackson's face. > > http://www.anomalies-unlimited.com/Jackson.html > > and then I tend to wander off and do something else ;-) Which is a perfectly sensible thing to do [*], but on the other hand if everyone through the history would only have done the sensible thing, would we have gotten where we are today? I know, some will say that we might have been better off without so much science and technology, but that's not something to discuss here. :-) What I mean is that if a not trivially dumb question can be asked, I think it should be asked even if the answer is "no, thanks". For one thing, not everything one thinks is obvious/clear/simple really is so for everybody; for another, the question might create "ripples" that might lead to a more meaningful question to be asked; and finally, nothing is perfect, so there is always is room for improvement. Back to the topic, records aren't really "working perfectly well", are they? They do the job, yes, but not very elegantly. regards, Vlad From vances@REDACTED Tue Jun 3 09:34:38 2003 From: vances@REDACTED (Vance Shipley) Date: Tue, 3 Jun 2003 03:34:38 -0400 Subject: Format of Pids Message-ID: <20030603073437.GT99427@frogman.motivity.ca> What can be said about the format of Pids? Pids (and references and ports) look like: <0.11.0> These numbers seem to be refered to as num, serial and creation. What do these things mean? I am working with them in erl_interface and would like to understand them better. For instance if I want to quickly test if a Pid exists I could do if(pid->serial). But is that right or is num the right field to check? Or do I really have to compare the whole structure? I seem to see that the first field is '0' if the Pid is local and something else if it's remote. Is that number the same on other nodes? I assume that these Pids are munged on the way through the distribution. Just curious ... -Vance From Bruce@REDACTED Tue Jun 3 10:08:34 2003 From: Bruce@REDACTED (Bruce Fitzsimons) Date: Tue, 3 Jun 2003 20:08:34 +1200 Subject: Gentoo: dev-lang/erlang References: <1053660475.1509.4.camel@localhost> <200305222134.29077.george@gentoo.org> <14e501c3240b$28a94d10$0a21970a@norris> <200305311418.31072.george@gentoo.org> Message-ID: <22b401c329a7$545dda60$0a21970a@norris> Hi George, ----- Original Message ----- From: "George Shapovalov" To: "Bruce Fitzsimons" Cc: Sent: Sunday, June 01, 2003 9:18 AM Subject: Re: Gentoo: dev-lang/erlang > Yup, incidentally I thought about this and was thinking about switching to > similar scheme when R10 comes out :). I would probably have done so from the > beginning had I not observed R8B and then R9B and assumed that this is a > complete version name, e.g no stuff is going to be added after the last > letter. And yea, the original submission by Charlie Mac had -8b in the name > :). > Sounds fine, its really not a big problem, I just thought it was good to fix. > The real question however is what to do with the present stuff - the R9 > versions. Its inconvenient version numbers vs having to move stuff that is > already in and is depended upon by some other packages. Basically it comes > down to the following questions: > > 1. How many more versions are planned in the R9 branch before R10 gets issued? > If R9c will be followed by R10 it might be better to just keep this mismatch > (9C will be 9d.ebuild) and switch to numeric versioning starting with R10 > (note, the ordering of versions will be preserved in portage, so that newver > versions will be picked up properly - thus the necessity of this mangling). > I suspect R9C will be the last R9 version (based on history) but someone on the list will have a better idea. Switching the scheme for R10 sounds fine to me. > 2. On a related note: how many gentoo usres are on this list? Klacke uses gentoo (the yaws webserver - http://yaws.hyber.org has an ebuild). And theres me :-) /Bruce From tobbe@REDACTED Tue Jun 3 12:46:23 2003 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: Tue, 03 Jun 2003 12:46:23 +0200 Subject: Gentoo: dev-lang/erlang In-Reply-To: <22b401c329a7$545dda60$0a21970a@norris> References: <1053660475.1509.4.camel@localhost> <200305222134.29077.george@gentoo.org> <14e501c3240b$28a94d10$0a21970a@norris> <200305311418.31072.george@gentoo.org> <22b401c329a7$545dda60$0a21970a@norris> Message-ID: <3EDC7C7F.70908@bluetail.com> I'm also using Gentoo. An ebuild for Erlang would lead to an ebuild for Yaws which would lead to an ebuild for the Bluetail Ticket Tracker.... To all fellow Erlang'ers: try out Gentoo if you have the possiblity, its very nice indeed. If you ever appreciated FreeBSD's ports system, you will love Gentoo's 'emerge', and if you have started to dislike RedHat's way of configuring your box you will like Gentoo's etc-config-files style as well. Cheers , Tobbe Bruce Fitzsimons wrote: >>2. On a related note: how many gentoo usres are on this list? >> >> > > >Klacke uses gentoo (the yaws webserver - http://yaws.hyber.org has an >ebuild). And theres me :-) > >/Bruce > > > From eleberg@REDACTED Tue Jun 3 12:58:58 2003 From: eleberg@REDACTED (Bengt Kleberg) Date: Tue, 3 Jun 2003 12:58:58 +0200 (MEST) Subject: erlc question, why no error message for erronous code? Message-ID: <200306031058.h53Awwq05350@cbe.ericsson.se> greetings, i had a (runtime) crash with ''badarith''. the code was: rate_class + 1 while not beeing a compiler writer, i somehow think it would be possible for the compiler to diagnose this kind of error*, would it not? *i belive this is a ''attempt to do arithmetic on an atom'' kind of error. bengt From joe@REDACTED Tue Jun 3 13:15:33 2003 From: joe@REDACTED (Joe Armstrong) Date: Tue, 3 Jun 2003 13:15:33 +0200 (CEST) Subject: erlc question, why no error message for erronous code? In-Reply-To: <200306031058.h53Awwq05350@cbe.ericsson.se> Message-ID: But it's not an error. catch (rate_class + 1) is well defined Now why anybody might want to write that is a mystery - but it it allowed (A warning might be better - oh and a pragma to turn off the warning) then you could write X = catch ( (yesIknowItlooksFunny) rate_class + 1) Many years ago I was programming some hardware - I wanted to send an illegal command to the hardware. The software would not allow this. I asked the compiler writer why I could not send a op code 23 to the hardware - he said "you can't it's illegal" I said - "but I want to test the hardware - see if the red light goes on when you send it an illegal command" He said, "you can't send an illegal command to the hardware" So I had to /dev/null his compiler and re-write the thing so I could do what I wanted. So from then on I was of the opinion that a compiler should allow anything even if it looks silly *provided* it is well-defined and safe. and thus it was :-) /Joe On Tue, 3 Jun 2003, Bengt Kleberg wrote: > greetings, > > i had a (runtime) crash with ''badarith''. the code was: > rate_class + 1 > > while not beeing a compiler writer, i somehow think it would be > possible for the compiler to diagnose this kind of error*, would it > not? > > *i belive this is a ''attempt to do arithmetic on an atom'' kind of error. > > > bengt > From eleberg@REDACTED Tue Jun 3 13:42:43 2003 From: eleberg@REDACTED (Bengt Kleberg) Date: Tue, 3 Jun 2003 13:42:43 +0200 (MEST) Subject: erlc question, why no error message for erronous code? Message-ID: <200306031142.h53Bghq10113@cbe.ericsson.se> > X-Authentication-Warning: enfield.sics.se: joe owned process doing -bs > Date: Tue, 3 Jun 2003 13:15:33 +0200 (CEST) > From: Joe Armstrong > To: Bengt Kleberg > But it's not an error. > > catch (rate_class + 1) > > is well defined > > Now why anybody might want to write that is a mystery - but it it allowed > > (A warning might be better - oh and a pragma to turn off the warning) > then you could write > > X = catch ( (yesIknowItlooksFunny) rate_class + 1) > yes, you are correct. i agree that a warning is the best solution. and it would be perfectly ok (imho) to turn off the warning by a compiler directive (thus not having to change the code). but i sure would like that warning, please. talking about warnings i would like the current erlc system (warnings if the right compiler switch is on) changed to its opposite: warnings, unless the right switch turns them off. rationale: experienced programmers that want to break the rules can turn warnings off. new programmers that need all the help they can get, do not know enought to turn the warnings on. bengt From vlad_dumitrescu@REDACTED Tue Jun 3 14:38:49 2003 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Tue, 3 Jun 2003 14:38:49 +0200 Subject: Handling huge amounts of data Message-ID: Hi, I am writing some programs that need to shuffle a lot of data (arrays/lists of some 5.000.000 items). I didn't want to use C/C++, so I settled to C#. None of the above, however, are especially good prototyping. I would have loved to be able to give Erlang a chance, but some simple tests say most of the time goes to garbage collection (and I need some speed when running too, not only when developing). Also, the memory usage seems to always go up (except when a process ends and that memory is freed)[*]. Besides that, it's occasionally causing "Abnormal termination"s. Is there some "magic" that will make Erlang work with this kind of problems, or is it just not the right niche? If the answer above is "no", has anyone any suggestion about what wo use instead? thanks in advance. Best regards, Vlad [*] Strange enough, there was a memory "up" when running "f(A):" in the shell, A being a huge list... Why would tha happen? From joe@REDACTED Tue Jun 3 14:41:58 2003 From: joe@REDACTED (Joe Armstrong) Date: Tue, 3 Jun 2003 14:41:58 +0200 (CEST) Subject: How do you stop a supervisor? Message-ID: Is there an approved way to stop a supervisor? Apart from just killing it that is. /Joe (there is no stop call only an undocumented terminate/2) From rickard.green@REDACTED Tue Jun 3 14:51:34 2003 From: rickard.green@REDACTED (Rickard Green) Date: Tue, 03 Jun 2003 14:51:34 +0200 Subject: Format of Pids References: <20030603073437.GT99427@frogman.motivity.ca> Message-ID: <3EDC99D6.60607@uab.ericsson.se> Hi, Vance Shipley wrote: > What can be said about the format of Pids? > > Pids (and references and ports) look like: > <0.11.0> > > These numbers seem to be refered to as num, > serial and creation. What do these things mean? Pids: NodeId is a node local number identifying another node. For example, on node a@REDACTED NodeId for node b@REDACTED can be 4711, but on node c@REDACTED NodeId for node b@REDACTED can be 17. 0 is special and means this node. On pre-R9B nodes NodeId is an 8-bit unsigned integer. On R9B (and later) nodes NodeId is a 32-bit (or 64-bit) unsigned integer. Creation is an integer which identifies "the creation of a node". Creation is incremented when a node is restarted. Currently creation is a 2-bit unsigned integer. 0 is special and means something like: actual creation used right now. Creation isn't shown when a pid is printed. Number and Serial are two integers identifying the process on the node which the process resides on. Currently number is a 15-bit unsigned integer and serial is a 3-bit unsigned integer. Number used to (pre-R9B) be used as index into the process table (an array), today the index might be a part of Number or a combination of Number and a part of Serial. Ports: #Port NodeId and Creation as for Pids. Id is an integer identifying the port on the node which the port resides on. Currently Id is an 18-bit unsigned integer. References: #Ref NodeId and Creation as for Pids. The three unsigned integers Id1, Id2, Id3 are (almost) unique for references created on the same node. Currently Id1 is an 18-bit unsigned integer, and Id2 and Id3 are 32-bit unsigned integers, i.e. there can at most be 2^82 unique references created on the same node. > I am working with them in erl_interface and > would like to understand them better. For instance if I want to quickly test if a Pid > exists I could do if(pid->serial). But is > that right or is num the right field to check? > Or do I really have to compare the whole > structure? Yes, when testing for equality you have to compare the hole structure. > I seem to see that the first field is '0' if > the Pid is local and something else if it's remote. Is that number > the same on other nodes? See above. > I assume that these Pids are munged > on the way through the distribution. > In the external format, nodename is used instead of NodeId. > Just curious ... > > -Vance Regards, Rickard Green, Erlang/OTP, Ericsson AB From etxuwig@REDACTED Tue Jun 3 15:20:23 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Tue, 3 Jun 2003 15:20:23 +0200 (MEST) Subject: How do you stop a supervisor? In-Reply-To: Message-ID: On Tue, 3 Jun 2003, Joe Armstrong wrote: > >Is there an approved way to stop a supervisor? The parent calling exit(Child, shutdown). There is no other clean way to do it. >Apart from just killing it that is. Doing exit(Supervisor, kill) is not the approved way. This will cause an {'EXIT', Supervisor, killed} message to propagate to the children, which will be interpreted as an error. Most likely the children will die too, but if they follow the OTP behavioural rules (started with proc_lib, etc.), you will get crash reports. >(there is no stop call only an undocumented terminate/2) terminate/2 is the gen_server callback, which is called if the supervisor terminates due to a coding fault in the supervisor. The supervisor.erl module is implemented as a gen_server callback. Calling the terminate/2 function directly will have quite unexpected consequences. /Uffe -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes From kent@REDACTED Tue Jun 3 16:00:01 2003 From: kent@REDACTED (Kent Boortz) Date: 03 Jun 2003 16:00:01 +0200 Subject: Handling huge amounts of data In-Reply-To: References: Message-ID: "Vlad Dumitrescu" writes: > I am writing some programs that need to shuffle a lot of data (arrays/lists > of some 5.000.000 items). I didn't want to use C/C++, so I settled to C#. > None of the above, however, are especially good prototyping. > > I would have loved to be able to give Erlang a chance, but some simple tests > say most of the time goes to garbage collection (and I need some speed when > running too, not only when developing). Also, the memory usage seems to > always go up (except when a process ends and that memory is freed)[*]. > Besides that, it's occasionally causing "Abnormal termination"s. > > Is there some "magic" that will make Erlang work with this kind of problems, > or is it just not the right niche? If you do message passing with huge terms you could try to compile an Erlang/OTP that uses a shared heap to avoid the copying of the data. You have to build Erlang/OTP from sources and configure like % ./configure --enable-shared-heap I don't know if the shared heap support is stable enough or if the garbage collector handle the case well but I think others on this list can fill you in on that, kent From etxmacr@REDACTED Tue Jun 3 16:48:47 2003 From: etxmacr@REDACTED (Mats Cronqvist) Date: Tue, 03 Jun 2003 16:48:47 +0200 Subject: Handling huge amounts of data In-Reply-To: Your message of "03 Jun 2003 16:00:01 +0200." Message-ID: <200306031448.h53Emlx03106@cbe2077.al.sw.ericsson.se> we (AXD301) have had some success in reducing garbage collect cost by increasing the heap size of data shuffling processes from erlang:spawn_opt/4 doc; {min_heap_size, Size}: Gives a minimum heap size in words. Setting this value higher than the system default might speed up some processes because less garbage collection is done. Setting too high value, however, might waste memory and slow down the system due to worse data locality. Therefore, it is recommended to use this option only for fine-tuning an application and to measure the exe- cution time with various Size values. > "Vlad Dumitrescu" writes: > > I am writing some programs that need to shuffle a lot of data (arrays/lists > > of some 5.000.000 items). I didn't want to use C/C++, so I settled to C#. > > None of the above, however, are especially good prototyping. > > > > I would have loved to be able to give Erlang a chance, but some simple tests > > say most of the time goes to garbage collection (and I need some speed when > > running too, not only when developing). Also, the memory usage seems to > > always go up (except when a process ends and that memory is freed)[*]. > > Besides that, it's occasionally causing "Abnormal termination"s. > > > > Is there some "magic" that will make Erlang work with this kind of problems, > > or is it just not the right niche? > > If you do message passing with huge terms you could try to compile an > Erlang/OTP that uses a shared heap to avoid the copying of the > data. You have to build Erlang/OTP from sources and configure like > > % ./configure --enable-shared-heap > > I don't know if the shared heap support is stable enough or if the > garbage collector handle the case well but I think others on this list > can fill you in on that, > > kent From etxuwig@REDACTED Tue Jun 3 17:47:07 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Tue, 3 Jun 2003 17:47:07 +0200 (MEST) Subject: lists:sort/2 Message-ID: I encountered some surprising results when using lists:sort/2. Given a list [{P,R}], I wanted a list sorted on P for all P < 3, and sorted on R for the remainder. I tried two different funs on the same list. The second one below produced the wanted result. The first one didn't. There is no help from the documentation in explaining why this happens, or indeed that there is a difference. Does anyone feel acquainted enough with the sorting algorithm to explain the phenomenon? Am I using the function in a way that was never intended? /Uffe L = [{1, 10.1}, {4, 10.4}, {1, 11.2}, {3, 10.3}, {2, 10.2}]. --> [{1,10.1000},{4,10.4000},{1,11.2000},{3,10.3000},{2,10.2000}] lists:sort( fun({P1,_}, {P2,_}) when P1 < 3 -> P1 < P2; ({_,R1}, {_,R2}) -> R1 < R2 end, L). --> [{1,10.1000},{4,10.4000},{1,11.2000},{2,10.2000},{3,10.3000}] lists:sort( fun({P1,_}, {P2,_}) when P2 < 3 -> P1 < P2; ({_,R1}, {_,R2}) -> R1 < R2 end, L). --> [{1,11.2000},{1,10.1000},{2,10.2000},{3,10.3000},{4,10.4000}] /Uffe -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes From erlang@REDACTED Tue Jun 3 17:07:18 2003 From: erlang@REDACTED (Peter-Henry Mander) Date: Tue, 03 Jun 2003 16:07:18 +0100 Subject: SciTE erlang support source code patches References: Message-ID: <3EDCB9A6.5090509@manderp.freeserve.co.uk> Hi James, You're welcome to have a look, I'm open to suggestions and improvements. I'd be interested to know if this also compiles properly under M$-Windows too. Unless you wish to avoid (re)compiling from source, perhaps the best for me to keep track of changes is for you to use the patches I sent to Chris Pressey which I've included. It's not perfect, I've noticed a mistake in the erlang.properties file, so I've added that too, and forwarded this mail to Chris for his benefit. Pete. James Hague wrote: >>I'm planning to feed back my work into the SciTE >>project so that Erlang becomes one of the many >>languages it supports. But before I do, is >>anyone interested in giving it a spin and offering >>some criticism? > > > After seeing your post I tried SciTE. Very nice! I'd be interested in > seeing your Erlang support for it. > > James > > -------------- next part -------------- A non-text attachment was scrubbed... Name: SciTE-1.53-Erlang.tgz Type: application/x-gtar Size: 30300 bytes Desc: not available URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: .SciTEUser.properties URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: erlang.properties URL: From etxuwig@REDACTED Tue Jun 3 18:10:52 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Tue, 3 Jun 2003 18:10:52 +0200 (MEST) Subject: lists:sort/2 In-Reply-To: <440A2703A54A8F4FB2AC2AE34F27129D215A6E@ESEALNT889.al.sw.ericsson.se> Message-ID: I will try thinking before posting next time. Maybe it's time to call it a day. :) /Uffe On Tue, 3 Jun 2003, Erik Reitsma (ETM) wrote: >> I encountered some surprising results when using >> lists:sort/2. Given a list [{P,R}], I wanted a list sorted >> on P for all P < 3, and sorted on R for the remainder. > >What result would you like when you compare {P1,R1} and {P2,R2}, and P1<3 and P2>=3? And what if P1>=3 and P2<3? It looks like you have not specified this. > >> lists:sort( >> fun({P1,_}, {P2,_}) when P1 < 3 -> >> P1 < P2; >> ({_,R1}, {_,R2}) -> >> R1 < R2 >> end, L). > >If this function even anti-symmetrical? Consider X1={1,10} and X2={4,9}. Then F(X1,X2) = true and F(X2,X1) = true. Your function should be such, that F(A,B) = not F(B,A) for all A and B. > >When you have defined the relation you want, I think you will be able to find a fun that really implements that relation. > >*Erik. > -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes From etxhua@REDACTED Tue Jun 3 18:31:12 2003 From: etxhua@REDACTED (Asko Husso) Date: Tue, 3 Jun 2003 18:31:12 +0200 (MEST) Subject: lists:sort/2 Message-ID: <200306031631.h53GVCq07985@cbe.ericsson.se> Uffe, try: lists:foldl( fun({P,R}, {Low, High}) when P < 3 -> {[{P,R}|Low], High}; (stop, {Low, High}) -> lists:keysort(1, Low) ++ lists:keysort(2, High); (X, {Low, High}) -> {Low, [X|High]} end, {[], []}, L ++ [stop]). The lists module is nice to have ;-) /Asko > X-Authentication-Warning: cbe1066.al.sw.ericsson.se: etxuwig owned process doing -bs > Date: Tue, 3 Jun 2003 18:10:52 +0200 (MEST) > From: Ulf Wiger > X-X-Sender: etxuwig@REDACTED > To: "Erik Reitsma (ETM)" > cc: erlang-questions@REDACTED > Subject: RE: lists:sort/2 > MIME-Version: 1.0 > X-OriginalArrivalTime: 03 Jun 2003 16:12:16.0555 (UTC) FILETIME=[E6AADFB0:01C329EA] > > > I will try thinking before posting next time. > Maybe it's time to call it a day. :) > > /Uffe > > On Tue, 3 Jun 2003, Erik Reitsma (ETM) wrote: > > >> I encountered some surprising results when using > >> lists:sort/2. Given a list [{P,R}], I wanted a list sorted > >> on P for all P < 3, and sorted on R for the remainder. > > > >What result would you like when you compare {P1,R1} and {P2,R2}, and P1<3 and P2>=3? And what if P1>=3 and P2<3? It looks like you have not specified this. > > > >> lists:sort( > >> fun({P1,_}, {P2,_}) when P1 < 3 -> > >> P1 < P2; > >> ({_,R1}, {_,R2}) -> > >> R1 < R2 > >> end, L). > > > >If this function even anti-symmetrical? Consider X1={1,10} and X2={4,9}. Then F(X1,X2) = true and F(X2,X1) = true. Your function should be such, that F(A,B) = not F(B,A) for all A and B. > > > >When you have defined the relation you want, I think you will be able to find a fun that really implements that relation. > > > >*Erik. > > > > -- > Ulf Wiger, Senior Specialist, > / / / Architecture & Design of Carrier-Class Software > / / / Strategic Product & System Management > / / / Ericsson AB, Connectivity and Control Nodes > Asko Husso E-mail:etxhua@REDACTED Ericsson AB Phone: +46 8 7192324 Varuv?gen 9B S-126 25 Stockholm-?lvsj?, Sweden From Erik.Stenman@REDACTED Tue Jun 3 18:06:22 2003 From: Erik.Stenman@REDACTED (Erik Stenman) Date: Tue, 3 Jun 2003 18:06:22 +0200 Subject: lists:sort/2 In-Reply-To: Message-ID: <001201c329ea$138ab790$2d9ab280@lamppc36> I can't see that any of your functions could be right Take the elements {1,2} and {4,1} {1,2} < {4,1} AND {4,1} < {1,2} !!! You have to make a total order like: F1 = fun ({X1,Y1},{X2,Y2}) -> if (X1 < 3) or (X2 < 3) -> X1 < X2; true -> Y1 < Y2 end end. Or F2 = fun ({X1,Y1},{X2,Y2}) -> if (X1 < 3) and (X2 < 3) -> X1 < X2; true -> Y1 < Y2 end end. Depending on what you mean by "I wanted a list sorted on P for all P < 3, and sorted on R for the remainder." (From your example I guess you want solution 1). 6> lists:sort(F1,L). [{1,11.2000},{1,10.1000},{2,10.2000},{3,10.3000},{4,10.4000}] > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Ulf Wiger > Sent: den 3 juni 2003 17:47 > To: erlang-questions@REDACTED > Subject: lists:sort/2 > > > > I encountered some surprising results when using > lists:sort/2. Given a list [{P,R}], I wanted a list sorted > on P for all P < 3, and sorted on R for the remainder. > I tried two different funs on the same list. The second one > below produced the wanted result. The first one didn't. There > is no help from the documentation in explaining why this > happens, or indeed that there is a difference. > > Does anyone feel acquainted enough with the sorting > algorithm to explain the phenomenon? Am I using the function > in a way that was never intended? > > /Uffe > > L = [{1, 10.1}, > {4, 10.4}, > {1, 11.2}, > {3, 10.3}, > {2, 10.2}]. > --> [{1,10.1000},{4,10.4000},{1,11.2000},{3,10.3000},{2,10.2000}] > > lists:sort( > fun({P1,_}, {P2,_}) when P1 < 3 -> > P1 < P2; > ({_,R1}, {_,R2}) -> > R1 < R2 > end, L). > --> [{1,10.1000},{4,10.4000},{1,11.2000},{2,10.2000},{3,10.3000}] > > lists:sort( > fun({P1,_}, {P2,_}) when P2 < 3 -> > P1 < P2; > ({_,R1}, {_,R2}) -> > R1 < R2 > end, L). > --> [{1,11.2000},{1,10.1000},{2,10.2000},{3,10.3000},{4,10.4000}] > > > /Uffe > -- > Ulf Wiger, Senior Specialist, > / / / Architecture & Design of Carrier-Class Software > / / / Strategic Product & System Management > / / / Ericsson AB, Connectivity and Control Nodes > > From Erik.Reitsma@REDACTED Tue Jun 3 17:59:18 2003 From: Erik.Reitsma@REDACTED (Erik Reitsma (ETM)) Date: Tue, 3 Jun 2003 17:59:18 +0200 Subject: lists:sort/2 Message-ID: <440A2703A54A8F4FB2AC2AE34F27129D215A6E@ESEALNT889.al.sw.ericsson.se> > I encountered some surprising results when using > lists:sort/2. Given a list [{P,R}], I wanted a list sorted > on P for all P < 3, and sorted on R for the remainder. What result would you like when you compare {P1,R1} and {P2,R2}, and P1<3 and P2>=3? And what if P1>=3 and P2<3? It looks like you have not specified this. > lists:sort( > fun({P1,_}, {P2,_}) when P1 < 3 -> > P1 < P2; > ({_,R1}, {_,R2}) -> > R1 < R2 > end, L). If this function even anti-symmetrical? Consider X1={1,10} and X2={4,9}. Then F(X1,X2) = true and F(X2,X1) = true. Your function should be such, that F(A,B) = not F(B,A) for all A and B. When you have defined the relation you want, I think you will be able to find a fun that really implements that relation. *Erik. From jesperw@REDACTED Tue Jun 3 18:12:31 2003 From: jesperw@REDACTED (Jesper Wilhelmsson) Date: Tue, 3 Jun 2003 18:12:31 +0200 (MET DST) Subject: Handling huge amounts of data In-Reply-To: References: Message-ID: On Tue, 3 Jun 2003, Kent Boortz wrote: > "Vlad Dumitrescu" writes: > > I am writing some programs that need to shuffle a lot of data (arrays/lists > > of some 5.000.000 items). I didn't want to use C/C++, so I settled to C#. > > None of the above, however, are especially good prototyping. > > > > I would have loved to be able to give Erlang a chance, but some simple tests > > say most of the time goes to garbage collection (and I need some speed when > > running too, not only when developing). Also, the memory usage seems to > > always go up (except when a process ends and that memory is freed)[*]. > > Besides that, it's occasionally causing "Abnormal termination"s. > > > > Is there some "magic" that will make Erlang work with this kind of problems, > > or is it just not the right niche? > > If you do message passing with huge terms you could try to compile an > Erlang/OTP that uses a shared heap to avoid the copying of the > data. You have to build Erlang/OTP from sources and configure like > > % ./configure --enable-shared-heap > > I don't know if the shared heap support is stable enough or if the > garbage collector handle the case well but I think others on this list > can fill you in on that, > > kent Shared heap should be stable. If not, please send me the program causing the trouble. There are many reasons for a system to collect garbage, some of them can be solved by setting the minimum heap size, some with the shared heap. Say the size of a process live data is growing and shrinking over and over again. When there is a lot of live data the heap grows during garbage collection (gc). At some later point a gc occurs when there is little live data causing the heap to shrink again, which leads to a new wave of gc's when the live data is accuculating the next time. Setting the minimum heap size for this process may be a good way to solve it if you can live with the fact that a single process will lock a huge ammount of heap space even in the periods of low live data. I think the shared heap is a better candidate for this kind of programs. Once a shared heap has grown, it won't shrink again. In the periods of low live data, other processes can use the heap space. And if you know when the heap is likely to be low on live data, perhaps it might be a good time to force a gc..? (Not that I want to encourage forcing gc's.) If you have a program where several processes accumulate tons of live data only to do some small thing and then die, setting the minimum heap size is probably a better choice than shared heap, since you can get rid of all those gs's just like that. __ ___( |_______________________ _______________________________________________ | | ,---. ,--.,--.,--. ( ( | || _ || o ) o ) p ) ) ) "Beware of bugs in the above code; | || (_) || r'| r'| -?--.( ( I have only proved it correct, not tried it." o,--' | `---' |_| |_| `-----' ) ) -- Donald Knuth _`----'______________________( (_______________________________________________ Jesper Wilhelmsson, jesperw@REDACTED +46 (0)18 471 1046 Computing Science Department, Uppsala University, Sweden +46 (0)733 207 207 ------------------------------------------------------------------------------- From vlad_dumitrescu@REDACTED Tue Jun 3 20:18:31 2003 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Tue, 3 Jun 2003 20:18:31 +0200 Subject: Handling huge amounts of data References: Message-ID: Thanks for the ideas. Kent wrote: > > If you do message passing with huge terms you could try to compile an > > Erlang/OTP that uses a shared heap to avoid the copying of the > > data. You have to build Erlang/OTP from sources and configure like Jesper wrote: > Shared heap should be stable. If not, please send me the program causing > the trouble. I am on Windows, so rebuilding the emulator isn't an option... I will try to set the minimum heap size. My app has a huge list (4,5 milion items) of numeric data that can be considered read-only and has to test it and filter it in different ways. I'd like to keep this stable data in a separate process, so that the temporary processing doesn't make that heap to be collected, but then there will be a lot of copying between processes instead. In the meanwhile, I found Lush (http://lush.sf.net) that is a simple Lisp with special features for numeric processing. It might be something worth looking at. regards, Vlad From vances@REDACTED Tue Jun 3 21:53:56 2003 From: vances@REDACTED (Vance Shipley) Date: Tue, 3 Jun 2003 15:53:56 -0400 Subject: Format of Pids In-Reply-To: <3EDC99D6.60607@uab.ericsson.se> References: <20030603073437.GT99427@frogman.motivity.ca> <3EDC99D6.60607@uab.ericsson.se> Message-ID: <20030603195356.GG6519@frogman.motivity.ca> Rickard, Thank you very much for your definitive response! On Tue, Jun 03, 2003 at 02:51:34PM +0200, Rickard Green wrote: } } Creation is an integer which identifies "the creation of a node". } Creation is incremented when a node is restarted. Currently creation } is a 2-bit unsigned integer. 0 is special and means something like: } actual creation used right now. Creation isn't shown when a pid is } printed. That raises a few questions. In ei (erl_interface) creation is a short. I have been setting creation with get_pid(). Since 29607 isn't going to fit in the 2-bit unsigned integer used for creation in the VM I wonder whether any of this is useful. The documentation, in describing ei_connect_init(), has this to say of creation: "creation identifies a specific instance of a C node. It can help prevent the node from receiving messages sent to an earlier process with the same registered name." This suggests that the creation will be included in the cnode's pid sent to the VM so that if it replies to that received pid and the nessage is received by a new instance of the cnode we will recognize that the destination pid is not ours. If this is the case how is the short used in for creation in the cnode mapped to the 2-bit unsigned integer used in the VM? I also missed this in the documentation: "A C node acting as a server will be assigned a creation number when it calls erl_publish() or erl_xpublish()." I don't see this happening. At least it isn't overwriting the creation I set (e.g. 29607). } > For instance if I want to quickly test if a Pid } > exists I could do if(pid->serial). But is } > that right or is num the right field to check? } > Or do I really have to compare the whole } > structure? } } Yes, when testing for equality you have to compare the hole } structure. Sorry I wasn't clear. I want to test if there is a valid pid stored in the erlang_pid struct. Is it safe to say that if ((erlang_pid *)->num == 0) that there is no valid pid stored there? That is to say that num is always greater than 0 in a pid (if we ignore the otp_ring process which I will never want to send to)? -Vance From cpressey@REDACTED Tue Jun 3 21:59:48 2003 From: cpressey@REDACTED (Chris Pressey) Date: Tue, 03 Jun 2003 14:59:48 -0500 Subject: erlc question, why no error message for erronous code? In-Reply-To: References: <200306031058.h53Awwq05350@cbe.ericsson.se> Message-ID: <20030603145948.777f3092.cpressey@catseye.mb.ca> On Tue, 3 Jun 2003 13:15:33 +0200 (CEST) Joe Armstrong wrote: > [...] > Many years ago I was programming some hardware - I wanted to send > an illegal command to the hardware. > > The software would not allow this. I asked the compiler writer why I could not > send a op code 23 to the hardware - he said "you can't it's illegal" > > I said - "but I want to test the hardware - see if the red light > goes on when you send it an illegal command" > > He said, "you can't send an illegal command to the hardware" Perfect place for a compile-time warning, though... warning: line 123: this code is fubar, but hey, you're the boss! -Chris From francesco@REDACTED Tue Jun 3 22:13:23 2003 From: francesco@REDACTED (Francesco Cesarini) Date: Tue, 03 Jun 2003 21:13:23 +0100 Subject: Handling huge amounts of data References: Message-ID: <3EDD0163.7020009@erlang-consulting.com> > > >[*] Strange enough, there was a memory "up" when running "f(A):" in the >shell, A being a huge list... Why would tha happen? > I assume it has to do with your arguments and results being stored in the shell process for the h() shell command. Every time you run f(A), you eat up chunks of memory not released by the GC. Had the same problem last week, when I was doing process_info/1 on all the processes from the shell. After every call, the memory usage was growing exponentially, with the VM grinding to a halt. Francesco -- http://www.umbria-rentals.com From jeinhorn@REDACTED Tue Jun 3 22:23:19 2003 From: jeinhorn@REDACTED (Jeff Einhorn) Date: 03 Jun 2003 15:23:19 -0500 Subject: Are dets:to_ets/ets:to_dets synchronous. Message-ID: <1054671798.13571.55.camel@dhcp-lom-195-14.futuresource.com> I was wondering if the commands dets:to_ets and/or ets:to_dets are suppose to be synchronous. At a glance it seems that when I try to read a large dets table from disk and convert it to an ets table my function seems to return before they are actually done. If the dets:to_ets and/or ets:to_dets functions are not synchronous is there any way to be notified when they are done? thanks for your time, -jeff -- Jeffrey M. Einhorn Software Engineer Web/Platform Development Group FutureSource, LLC jeinhorn@REDACTED http://www.futuresource.com From erlang@REDACTED Wed Jun 4 02:00:36 2003 From: erlang@REDACTED (erlang@REDACTED) Date: Tue, 3 Jun 2003 20:00:36 -0400 (EDT) Subject: mnesia developments Message-ID: In my further researches into the user manual I _think_ I have developed a clearer understanding of what the source of my confusion was. Correct me if I am wrong, please: the _.hrl file can be used to express a schema which will be loaded when mnesia is started or mnesia's schema can be created on the fly with shell commands (or some suitable function which encapsulates a set of them). What I'm not clear on is if there is some sneaky lack of equivalence between the two methods which isn't made entirely clear in the manual. From vlad_dumitrescu@REDACTED Wed Jun 4 08:38:34 2003 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Wed, 4 Jun 2003 08:38:34 +0200 Subject: Handling huge amounts of data References: <3EDD0163.7020009@erlang-consulting.com> Message-ID: > >[*] Strange enough, there was a memory "up" when running "f(A):" in the > >shell, A being a huge list... Why would tha happen? > > > > I assume it has to do with your arguments and results being stored in > the shell process for the h() shell command. Every time you run f(A), > you eat up chunks of memory not released by the GC. Had the same problem > last week, when I was doing process_info/1 on all the processes from the > shell. After every call, the memory usage was growing exponentially, > with the VM grinding to a halt. Ahaa! Okay, so I will have to run from a spawned process instead. Thanks. /Vlad From etxuwig@REDACTED Wed Jun 4 09:36:23 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Wed, 4 Jun 2003 09:36:23 +0200 (MEST) Subject: mnesia developments In-Reply-To: Message-ID: On Tue, 3 Jun 2003 erlang@REDACTED wrote: >In my further researches into the user manual I _think_ I >have developed a clearer understanding of what the source >of my confusion was. > >Correct me if I am wrong, please: > >the _.hrl file can be used to express a schema which will >be loaded when mnesia is started Well, perhaps not quite. There is no mnesia equivalent to SQL for schema declaration. You call functions to create schema and tables. A common way to create a mnesia schema is to write a function, say myModule:schema_init(), that does some or all of these things: - possibly delete any existing schema (mnesia:delete_schema(Nodes)) - create a mnesia schema (mnesia:create_schema(Nodes) - start mnesia (mnesia:start()) - create the needed tables (mnesia:create_table(...)) - possibly fill the tables with initial data This function should only be run once, when installing the system. It's good practice to collect the record definitions for the mnesia tables in a common .hrl file, which can be included by myModule above. Once all this is done, you should be able to start your erlang node and make sure mnesia is started. Mnesia should then read the schema you have created. Note that mnesia expects the erlang node name to be the same each time. It tags the schema with the node name used at creation time. >mnesia's schema can be created on the fly with shell >commands (or some suitable function which encapsulates a >set of them). Yes, the mnesia interface can be used during runtime, so you can create new tables, delete old ones, change replication properties or even the data representation in a table on the fly, without stopping your system. >What I'm not clear on is if there is some sneaky lack of >equivalence between the two methods which isn't made >entirely clear in the manual. There is basically only one method: - make sure mnesia is running. - call the appropriate functions to create tables, once only for each table. - after that, just start mnesia each time, and it should be able to load the schema definitions. /Uffe -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes From bjorn@REDACTED Wed Jun 4 11:47:47 2003 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 04 Jun 2003 11:47:47 +0200 Subject: erlc question, why no error message for erronous code? In-Reply-To: <200306031058.h53Awwq05350@cbe.ericsson.se> References: <200306031058.h53Awwq05350@cbe.ericsson.se> Message-ID: We plan have add warnings of this kind in the R10 release. /Bjorn Bengt Kleberg writes: > greetings, > > i had a (runtime) crash with ''badarith''. the code was: > rate_class + 1 > > while not beeing a compiler writer, i somehow think it would be > possible for the compiler to diagnose this kind of error*, would it > not? > > *i belive this is a ''attempt to do arithmetic on an atom'' kind of error. > > > bengt > -- Bj?rn Gustavsson Ericsson Utvecklings AB bjorn@REDACTED ?T2/UAB/F/P BOX 1505 +46 8 727 56 87 125 25 ?lvsj? From ingela@REDACTED Wed Jun 4 14:27:40 2003 From: ingela@REDACTED (Ingela Anderton) Date: Wed, 4 Jun 2003 14:27:40 +0200 (MEST) Subject: Oracle and ODBC erlang driver In-Reply-To: <200306041146.04161.yvan.godin@free.fr> Message-ID: On Wed, 4 Jun 2003, Yvan Godin wrote: > Le Mercredi 28 Mai 2003 18:25, vous avez ?crit : > > First of all .. thank for your answer > > > The erlang ODBC application consists of an erlang part and a C-part. > > The C-part delivered for the W2K platform is compiled to work with > > sqlserver. Have you recompiled it to link with your oracle driver? > Really needed ??? seem to me ODBC here to be ind?pendant from database drivers > ;-) I think you are right the dll should take care of that! I was in a hurry to go home and clearly not thinking straight ;-) > > > Please let me know how it goes. > > I have made some new tests and below the result I am busy this week with giving an Erlang/OTP course. I will look into it next week. However I think you would be better of using the supported interface rather than the deprecated one. /Ingela - OTP team From yvan.godin@REDACTED Wed Jun 4 11:46:03 2003 From: yvan.godin@REDACTED (Yvan Godin) Date: Wed, 4 Jun 2003 11:46:03 +0200 Subject: Oracle and ODBC erlang driver In-Reply-To: <16084.58135.314737.292796@gargle.gargle.HOWL> References: <1054033749.3ed34755e6932@imp.free.fr> <16084.58135.314737.292796@gargle.gargle.HOWL> Message-ID: <200306041146.04161.yvan.godin@free.fr> Le Mercredi 28 Mai 2003 18:25, vous avez ?crit : First of all .. thank for your answer > The erlang ODBC application consists of an erlang part and a C-part. > The C-part delivered for the W2K platform is compiled to work with > sqlserver. Have you recompiled it to link with your oracle driver? Really needed ??? seem to me ODBC here to be ind?pendant from database drivers ;-) > Please let me know how it goes. I have made some new tests and below the result With new erlang ODBC functions (odbc:connect(..)) and setting LOG on ODBC visual studio analyser I get a pop up with APPLICATION ERROR l'instruction 0x1f95f26d emploi l'adresse memoire 0x1f95f26d MEMORY CAN'T BE READ Then I made some test with the deprecated fonctions (modifed version of basic.erl in odbc example see code below) and seem be better here are the result (give me an answer ... the error msg may be due of bad code ...see below) yoldbc:init(). sqlConnect returns 1 sqlExecDirect returns 0 Select: Number of columns: 6 sqlBindColumn returns 0 sqlBindColumn returns 0 sqlFetch returns 0 Select: Column 1 data:"11" Select: Column 2 data:"CLIENT" =ERROR REPORT==== 3-Jun-2003::17:36:44 === Error in process <0.72.0> with exit value: {{badmatch,0},[{yoldbc,select,0},{yol dbc,init,0},{erl_eval,expr,3},{erl_eval,exprs,4},{shell,eval_loop,2}]} ** exited: {{badmatch,0}, [{yoldbc,select,0}, {yoldbc,init,0}, {erl_eval,expr,3}, {erl_eval,exprs,4}, {shell,eval_loop,2}]} ** =ERROR REPORT==== 3-Jun-2003::17:36:44 === ** Generic server odbc1 terminating ** Last message in was {'DOWN',#Ref<0.0.0.288>, process, <0.72.0>, {{badmatch,0}, [{yoldbc,select,0}, {yoldbc,init,0}, {erl_eval,expr,3}, {erl_eval,exprs,4}, {shell,eval_loop,2}]}} ** When Server state == {state,#Port<0.85>, undefined, <0.72.0>, undefined, on, undefined, undefined, undefined, connected, undefined, 0, true, true, [{#Ref<0.0.0.303>,2},{#Ref<0.0.0.302>,1}]} ** Reason for termination == ** {stopped,{'EXIT',<0.72.0>, {{badmatch,0}, [{yoldbc,select,0}, {yoldbc,init,0}, {erl_eval,expr,3}, {erl_eval,exprs,4}, {shell,eval_loop,2}]}}} **************MODIFIED VERSION of BASIC.ERL ************* -module(yoldbc). -export([init/0,stop/0]). -include("odbc.hrl"). -def.... here my connection macros init() -> % Start a new ODBC server. The application must already be started. {ok, Pid} = odbc:start_link({local, ?SERV},[], []), % Connect to the database (also loads the Driver). Ret = odbc:sqlConnect(?SERV, ?DSN, ?UID, ?PWD, infinity), io:format("sqlConnect returns ~p~n",[Ret]), select(), stop(). select() -> SelectStmt = "SELECT * FROM GROUPES", Ret1 = odbc:sqlExecDirect(?SERV, SelectStmt, infinity), io:format("sqlExecDirect returns ~p~n",[Ret1]), {?SQL_SUCCESS, NSelectedCols} = odbc:sqlNumResultCols(?SERV, infinity), io:format("Select: Number of columns: ~p~n", [NSelectedCols]), {ok,Ref1} = odbc:columnRef(), {ok,Ref2} = odbc:columnRef(), Ret6 = odbc:sqlBindColumn(?SERV, 1, Ref1, infinity), io:format("sqlBindColumn returns ~p~n",[Ret6]), Ret7 = odbc:sqlBindColumn(?SERV, 2, Ref2, infinity), io:format("sqlBindColumn returns ~p~n",[Ret7]), % Fetch the first row of selected rows. Ret8 = odbc:sqlFetch(?SERV, infinity), io:format("sqlFetch returns ~p~n",[Ret1]), {ok, ColValue1} = odbc:readData(?SERV, Ref1, infinity), io:format("Select: Column 1 data:~p~n",[ColValue1]), {ok, ColValue2} = odbc:readData(?SERV, Ref2, infinity), io:format("Select: Column 2 data:~p~n",[ColValue2]), ?SQL_NO_DATA = odbc:sqlFetch(?SERV, infinity), Ret12 = odbc:sqlCloseHandle(?SERV, infinity), io:format("sqlCloseHandle returns ~p~n",[Ret12]). stop() -> Ret = odbc:sqlDisConnect(?SERV, infinity), odbc:stop(?SERV). -- Cordialement Yvan GODIN WITBE/DSIC http://www.witbe.net From vances@REDACTED Wed Jun 4 16:11:42 2003 From: vances@REDACTED (Vance Shipley) Date: Wed, 4 Jun 2003 10:11:42 -0400 Subject: mnesia developments In-Reply-To: References: Message-ID: <20030604141142.GN6519@frogman.motivity.ca> Perhaps an example: -module(install). -export([start/0]). start() -> mnesia:create_schema([node()]), application:start(mnesia), mnesia:wait_for_tables([schema], 10000), mnesia:create_table(ds1Table, [{snmp, [{key, integer}]}, {attributes, [index, name, type]}]), mnesia:wait_for_tables([ds1Table], 10000), eva_sup:create_tables_log_snmp([node()]), mnesia:wait_for_tables([eventTable, alarmTable, alarm, eva_snmp_map, evaLogDiscriminatorTable, logTable, logTransferTable], 10000). Note the wait_for_tables/2 calls. This non-obvious step is important. In this example we're creating tables for snmp. (foo@REDACTED)1> c(install). {ok,install} (foo@REDACTED)2> install:start(). ok This creates a directory named Mnesia.foo@REDACTED which now contains: LATEST.LOG logTable.DCD alarmTable.DCD logTransferTable.DCD evaLogDiscriminatorTable.DCD schema.DAT eventTable.DCD This directory can be configured in your sys.config file: {mnesia, [{dir, "lib/erlang/lib/bar-0.1/priv/data"}]} -Vance From erlang@REDACTED Wed Jun 4 12:18:43 2003 From: erlang@REDACTED (Inswitch Solutions - Erlang Evaluation) Date: Wed, 4 Jun 2003 12:18:43 +0200 Subject: Logging error to a file in FSM Message-ID: <000a01c32a82$ae7823d0$1e00a8c0@design> Is it {ok, Pid} = gen_fsm:start(?MODULE, [Mod, Board, Trunk, Ch], [{debug,[{log_to_file, "log.txt"}]}]), the right way to log errors to "log.txt" file using FSM ? Thanks, Eduardo Figoli -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Wed Jun 4 19:20:47 2003 From: francesco@REDACTED (Francesco Cesarini) Date: Wed, 04 Jun 2003 18:20:47 +0100 Subject: Logging error to a file in FSM References: <000a01c32a82$ae7823d0$1e00a8c0@design> Message-ID: <3EDE2A6F.8080308@erlang-consulting.com> I would not hard code the Debug Options flag. I would instead use the sys module and turn the debugging on once the fsm has been started by the supervisor. Regards, Francesco Inswitch Solutions - Erlang Evaluation wrote: > > Is it > {ok, Pid} = gen_fsm:start(?MODULE, [Mod, Board, Trunk, Ch], > [{debug,[{log_to_file, "log.txt"}]}]), > > the right way to log errors to "log.txt" file using FSM ? > > > Thanks, > Eduardo Figoli > From erlang@REDACTED Wed Jun 4 23:30:15 2003 From: erlang@REDACTED (Erlang Question) Date: Wed, 4 Jun 2003 18:30:15 -0300 Subject: Inets Performance Message-ID: <000d01c32ae0$80d17f40$4200a8c0@sbnt.com> Hi! Let me show my problem... I?m using intes to develop a web interface having dinamicals forms with some text boxes to complete. In spite of access from the server (worst if access from a client) if the form have a big number of data to complete then the cgi request never arrives to the server or better said: the cgi-module doesn?t executes(ie:one hundred if it?s in server; in client no more than 10 aprox.) ... otherwise no problem, the cgi module executes successfull (if form have few elements in the form). I read all about inets configuration and the httpd_core documentation thinking that the problem may be in the max header size or max body size clause...but it doesn?t fix-up my problem... So now, I don?t have idea what could it be...it's be a performance problem of the inets web server? or a bad configuration in the httpd_conf? or what could it be? any suggestion are welcome...thanks in advance gabriel... -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay@REDACTED Thu Jun 5 06:54:45 2003 From: jay@REDACTED (Jay Nelson) Date: Wed, 04 Jun 2003 21:54:45 -0700 Subject: Handling huge amounts of data Message-ID: <4.2.2.20030604215207.00cf8420@duomark.com> Vlad, What are you doing with this data? Does it have any regularity that can be captured in a function? How much manipulation? Would a binary work for you instead of lists? If you gave a few more details, someone might offer some alternative ways of managing memory. Just because erlang doesn't have malloc and free doesn't mean that there aren't alternative ways of storing things. jay From vlad_dumitrescu@REDACTED Thu Jun 5 08:40:32 2003 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Thu, 5 Jun 2003 08:40:32 +0200 Subject: Handling huge amounts of data References: <4.2.2.20030604215207.00cf8420@duomark.com> Message-ID: > What are you doing with this data? Does it have any > regularity that can be captured in a function? How > much manipulation? Would a binary work for you > instead of lists? Well, the data is a list (between 500.000 - 1.500.000 items) of chunks of data that may be lists of bytes, or binaries and are 27 bytes each. For each of them, I need to compare it with all the others in a non-trivial way (some 1000 similar tests), and select only those that pass the test. The problem that I see is the globality of the search, which makes that I can't use a "divide and conquer" strategy [*]. Also there is no good locality of data access either. [*] More precisely, in the worst case "d&c" will fall back onto "serch all" after much work. I tried several ways of storing the data in memory, as list of tuples, list of binaries, ets table and in all cases the VM becomes slower and slower and slower until it breaks. I try to find something to do in Erlang in order to present the great results and maybe allow further applications. We are a Web application that uses J2EE, and that would make a great target, but this is the third implementation in 3 years (don't ask why) and I feel nobody will accept another one if this works... regards, Vlad From rickard.green@REDACTED Thu Jun 5 11:00:11 2003 From: rickard.green@REDACTED (Rickard Green) Date: Thu, 05 Jun 2003 11:00:11 +0200 Subject: Format of Pids References: <20030603073437.GT99427@frogman.motivity.ca> <3EDC99D6.60607@uab.ericsson.se> <20030603195356.GG6519@frogman.motivity.ca> Message-ID: <3EDF069B.5030601@uab.ericsson.se> Vance, erl_interface and ei are currently being cleaned up, but this wont be finished until (hopefully) R10B. Vance Shipley wrote: > Rickard, > > Thank you very much for your definitive response! > > On Tue, Jun 03, 2003 at 02:51:34PM +0200, Rickard Green wrote: > } > } Creation is an integer which identifies "the creation of a node". > } Creation is incremented when a node is restarted. Currently creation > } is a 2-bit unsigned integer. 0 is special and means something like: > } actual creation used right now. Creation isn't shown when a pid is > } printed. > > That raises a few questions. In ei (erl_interface) creation is a short. > > I have been setting creation with get_pid(). Since 29607 isn't going > to fit in the 2-bit unsigned integer used for creation in the VM I > wonder whether any of this is useful. > > The documentation, in describing ei_connect_init(), has this to say > of creation: > > "creation identifies a specific instance of a C node. It can > help prevent the node from receiving messages sent to an > earlier process with the same registered name." > > This suggests that the creation will be included in the cnode's pid > sent to the VM so that if it replies to that received pid and the > nessage is received by a new instance of the cnode we will recognize > that the destination pid is not ours. If this is the case how is > the short used in for creation in the cnode mapped to the 2-bit > unsigned integer used in the VM? > This is a bug. My guess (before I looked in the code) was that ei_connect_init() converted the creation passed to it into a valid creation and then stored it. My guess was wrong, though, it just stores the creation it has been passed as it is. When a pid, port or ref is encoded into external format, all creation bits except for the 2 least significant are dropped. This means that an ei_encode_pid() followed by an ei_decode_pid() on ei_self() will in most cases produce a pid not equal to ei_self(). > I also missed this in the documentation: > > "A C node acting as a server will be assigned a creation > number when it calls erl_publish() or erl_xpublish()." > > I don't see this happening. At least it isn't overwriting the > creation I set (e.g. 29607). > This is a documentation bug. erl_publish() intentionally drops the creation it receives from epmd (a design flaw in my opinion). > } > For instance if I want to quickly test if a Pid > } > exists I could do if(pid->serial). But is > } > that right or is num the right field to check? > } > Or do I really have to compare the whole > } > structure? > } > } Yes, when testing for equality you have to compare the hole > } structure. > > Sorry I wasn't clear. I want to test if there is a valid pid > stored in the erlang_pid struct. Is it safe to say that if > ((erlang_pid *)->num == 0) that there is no valid pid stored > there? That is to say that num is always greater than 0 in > a pid (if we ignore the otp_ring process which I will never > want to send to)? > I'm not sure that I understand you right, but my guess is that you want to mark an erlang_pid structure as invalid. Setting the num member to 0 won't work even if you consider as invalid since could be a pid of a process that you want to communicate with. I'd set the node member to "" instead, i.e. if (epid->node[0] == '\0') then epid isn't a valid pid. > -Vance > Regards, Rickard From Chandrashekhar.Mullaparthi@REDACTED Thu Jun 5 11:10:11 2003 From: Chandrashekhar.Mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Thu, 5 Jun 2003 10:10:11 +0100 Subject: Handling huge amounts of data Message-ID: Vlad, Have you tried using 64 bit erlang - your app would be a good stress for it :-) cheers Chandru -----Original Message----- From: Vlad Dumitrescu [mailto:vlad_dumitrescu@REDACTED] Sent: 05 June 2003 07:41 To: erlang-questions@REDACTED; Jay Nelson Subject: Re: Handling huge amounts of data > What are you doing with this data? Does it have any > regularity that can be captured in a function? How > much manipulation? Would a binary work for you > instead of lists? Well, the data is a list (between 500.000 - 1.500.000 items) of chunks of data that may be lists of bytes, or binaries and are 27 bytes each. For each of them, I need to compare it with all the others in a non-trivial way (some 1000 similar tests), and select only those that pass the test. The problem that I see is the globality of the search, which makes that I can't use a "divide and conquer" strategy [*]. Also there is no good locality of data access either. [*] More precisely, in the worst case "d&c" will fall back onto "serch all" after much work. I tried several ways of storing the data in memory, as list of tuples, list of binaries, ets table and in all cases the VM becomes slower and slower and slower until it breaks. I try to find something to do in Erlang in order to present the great results and maybe allow further applications. We are a Web application that uses J2EE, and that would make a great target, but this is the third implementation in 3 years (don't ask why) and I feel nobody will accept another one if this works... regards, Vlad NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From Chandrashekhar.Mullaparthi@REDACTED Thu Jun 5 11:18:21 2003 From: Chandrashekhar.Mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Thu, 5 Jun 2003 10:18:21 +0100 Subject: Inets Performance Message-ID: Assuming you have looked in the access_logs and error_logs and can't see anything, in the past I've recompiled the inets application with the debug flags turned on (look in httpd.hrl) - that should give you some traces to see where the problem is. I've had this problem in the past but increasing the values of MaxHeaderSize and MaxBodySize fixed it. Maybe you can also post your inets config to the list and see if someone can find anything fishy in there. cheers Chandru -----Original Message----- From: Erlang Question [mailto:erlang@REDACTED] Sent: 04 June 2003 22:30 To: erlang-questions@REDACTED Subject: Inets Performance Hi! Let me show my problem... I?m using intes to develop a web interface having dinamicals forms with some text boxes to complete. In spite of access from the server (worst if access from a client) if the form have a big number of data to complete then the cgi request never arrives to the server or better said: the cgi-module doesn?t executes(ie:one hundred if it?s in server; in client no more than 10 aprox.) ... otherwise no problem, the cgi module executes successfull (if form have few elements in the form). I read all about inets configuration and the httpd_core documentation thinking that the problem may be in the max header size or max body size clause...but it doesn?t fix-up my problem... So now, I don?t have idea what could it be...it's be a performance problem of the inets web server? or a bad configuration in the httpd_conf? or what could it be? any suggestion are welcome...thanks in advance gabriel... NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From chris.danx@REDACTED Thu Jun 5 10:31:04 2003 From: chris.danx@REDACTED (chris.danx) Date: Thu, 05 Jun 2003 10:31:04 +0200 Subject: Embedding in an application Message-ID: <3EDEFFC8.30008@ntlworld.com> Hi, Is it possible to embed the Erlang/OTP in another application? I'm looking to use a language to script parts of an application and am wondering if anyone has done this or if it's even possible? I am considering Lua, Javascript and Haskell but wondered if it's possible with the Erlang system. Cheers, Chris p.s. sorry if I'm being a bit vague, I don't really know how to describe it. Perhaps the best way to describe it is think "Emacs & Lisp only for diagramming" From vlad_dumitrescu@REDACTED Thu Jun 5 11:30:17 2003 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Thu, 5 Jun 2003 11:30:17 +0200 Subject: Handling huge amounts of data References: Message-ID: RE: Handling huge amounts of data>Vlad, >Have you tried using 64 bit erlang - your app would be a good stress for it :-) >cheers >Chandru Hi, No, I didn't -- I am using Windows.... regards, Vlad From vlad_dumitrescu@REDACTED Thu Jun 5 11:58:13 2003 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Thu, 5 Jun 2003 11:58:13 +0200 Subject: Embedding in an application References: <3EDEFFC8.30008@ntlworld.com> Message-ID: Hi, ----- Original Message ----- > Is it possible to embed the Erlang/OTP in another application? I'm > looking to use a language to script parts of an application and am > wondering if anyone has done this or if it's even possible? I am > considering Lua, Javascript and Haskell but wondered if it's possible > with the Erlang system. I may be wrong here, but when I researched the same topic, this is what I came up to: It is certainly possible, but it's not the funniest thing to do :-) Beam is a dynamic library, so any program can use it. However, one can only talk to the shell then, I think. The best way is to let Erlang and the application be separate and talk either via a port, or a TCP connection. You will have to design a protocol, and convert the data from Erlang to whatever language your app is written in, and back. As wire data format, the Erlang external format is a possibility; UBF is another. Or you can use your own. I'm not sure this answers all your wonderings, but I hope it gives an indication. regards, Vlad From jay@REDACTED Thu Jun 5 14:00:08 2003 From: jay@REDACTED (Jay Nelson) Date: Thu, 05 Jun 2003 05:00:08 -0700 Subject: Handling huge amounts of data In-Reply-To: References: <4.2.2.20030604215207.00cf8420@duomark.com> Message-ID: <4.2.2.20030605045246.00cf7ab0@duomark.com> At 08:40 AM 6/5/03 +0200, you wrote: >Well, the data is a list (between 500.000 - 1.500.000 items) of chunks of >data that may be lists of bytes, or binaries and are 27 bytes each. For each >of them, I need to compare it with all the others in a non-trivial way (some >1000 similar tests), and select only those that pass the test. Hmm, interesting that it is always the same size. Can you say more about the test? Is there any similarity in the data content? Any way to compress the data that is related to the search mechanism? Does the problem map to other similar problems? How many search requests do you get and how often? How long do you have to respond (latency and thruput)? >I tried several ways of storing the data in memory, as list of tuples, list >of binaries, ets table and in all cases the VM becomes slower and slower and >slower until it breaks. Lists or tuples should be roughly equivalent. 64-bit mode will only make the data set larger, unless you use binaries in which case it will have no difference. You may need to find a way to do any of the following: 1) Reduce the data set size (thru compression, functional simulation, process partitioning, etc) 2) Reduce the number of compares (have you looked at "dynamic programming") 3) Translate the data / problem to a different representation 4) Use an external app and connect via a port erlang is good for development efficiency and code clarity, but not for memory efficiency. From vlad_dumitrescu@REDACTED Thu Jun 5 14:39:10 2003 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Thu, 5 Jun 2003 14:39:10 +0200 Subject: Handling huge amounts of data References: <4.2.2.20030604215207.00cf8420@duomark.com> <4.2.2.20030605045246.00cf7ab0@duomark.com> Message-ID: Hi Jay, First things first, I have a programm running, running in Delphi. Doing the tests for one item takes some 30s, and since I have to choose at least 50000 of them, it's going to be running some 3 weeks. I guess any interpreted language will be slower than direct memory access. The good news is it's going to be run once a year, or so. > Hmm, interesting that it is always the same size. Can you say more > about the test? Is there any similarity in the data content? Any way > to compress the data that is related to the search mechanism? Does > the problem map to other similar problems? How many search requests > do you get and how often? How long do you have to respond (latency > and thruput)? I am not sure if everybody is interested in more details, and also I am not allowed to say too much, but the answer to most of the questions above is 'no, as far as I can tell'. The items are the numbers 1 to 27 and I have to filter out certain interdependencies, and the comparisons need to be done one by one. Thanks for all the good ideas. I'll look for another opportunity to use Erlang! regards, Vlad From vances@REDACTED Thu Jun 5 15:16:05 2003 From: vances@REDACTED (Vance Shipley) Date: Thu, 5 Jun 2003 09:16:05 -0400 Subject: Format of Pids In-Reply-To: <3EDF069B.5030601@uab.ericsson.se> References: <20030603073437.GT99427@frogman.motivity.ca> <3EDC99D6.60607@uab.ericsson.se> <20030603195356.GG6519@frogman.motivity.ca> <3EDF069B.5030601@uab.ericsson.se> Message-ID: <20030605131604.GE12072@frogman.motivity.ca> On Thu, Jun 05, 2003 at 11:00:11AM +0200, Rickard Green wrote: } } This is a documentation bug. erl_publish() intentionally drops the } creation it receives from epmd (a design flaw in my opinion). Indeed. Reading your comments about how creation is skinny'd down to 2 bits I began to wonder how I could figure out on startup that I needed to bump up creation to '01'. The idea of using get_pid() was a good one but using the last 2 bits of that isn't going to provide any guarantees of uniqueness. Then I realized that epmd can keep track of this stuff and of course that's why it does. Let's call it a bug. Especially since it's documented. :) -Vance From erlang@REDACTED Thu Jun 5 19:17:44 2003 From: erlang@REDACTED (Erlang Question) Date: Thu, 5 Jun 2003 14:17:44 -0300 Subject: Inets Performance Message-ID: <002301c32b86$700120c0$4200a8c0@sbnt.com> Hi again! Thanks people, but I still surprise with the behavior of the webserver... i read the logs (both: access and error) and anyone have nothing to show about it... more of that the acces_log doesn?t registry the access (obviously because the server doesn?t recive the request from the client). When I said "access from the server I mean when the client are the same webserver machine...browser in the directly from the server". Finally: I?m using... What OS/HW are you using? Windows 2000 p. Which version of OTP are you using? 9.0 Which version of inets are you using? 3.0 I guess Which http client are you using? windows 2000 p. and IE 6.0 PD to Micael : yes inswitch are registered in ericsson support but I always like to post in both sides...I like technical forums community; everybody helping mutually... ;) Thanks again people!...my best regards...gabriel! ----- Original Message ----- From: Erlang Question To: erlang-questions@REDACTED Sent: Wednesday, June 04, 2003 6:30 PM Subject: Inets Performance Hi! Let me show my problem... I?m using intes to develop a web interface having dinamicals forms with some text boxes to complete. In spite of access from the server (worst if access from a client) if the form have a big number of data to complete then the cgi request never arrives to the server or better said: the cgi-module doesn?t executes(ie:one hundred if it?s in server; in client no more than 10 aprox.) ... otherwise no problem, the cgi module executes successfull (if form have few elements in the form). I read all about inets configuration and the httpd_core documentation thinking that the problem may be in the max header size or max body size clause...but it doesn?t fix-up my problem... So now, I don?t have idea what could it be...it's be a performance problem of the inets web server? or a bad configuration in the httpd_conf? or what could it be? any suggestion are welcome...thanks in advance gabriel... -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sun Jun 8 23:28:01 2003 From: erlang@REDACTED (erlang@REDACTED) Date: Sun, 8 Jun 2003 17:28:01 -0400 (EDT) Subject: Good practice Message-ID: bash$ erl Erlang (JAM) emulator version 47.4.1 Eshell V47.4.1 (abort with ^G) 1> c(test1file). ./test1file.erl:172: Warning: variable 'Response' exported from ['case'] {ok,test1file} 2> >From the file in question: case checkvalid(Funcref, Correctlist) of false -> Response = false; Other -> Response = invoker(Other, Otherargs) end, Requester ! Response, Obviously the warning warns me of a problem. This I expect, and welcome. However, that doesn't mean I want to see warnings; I want to do things right. I could just put the message in the case statement, I suppose, but is there really a canonical way of doing the above without getting the complaint? From ulf.wiger@REDACTED Sun Jun 8 22:25:51 2003 From: ulf.wiger@REDACTED (Wiger Ulf) Date: Sun, 8 Jun 2003 22:25:51 +0200 Subject: Good practice References: Message-ID: <001501c32dfc$281b6b80$fd7a40d5@telia.com> Try this: Response = case checkvalid(Funcref, Correctlist) of false -> false; Other -> invoker(Other, Otherargs) end, Requester ! Response, ... /Uffe ----- Original Message ----- From: To: Sent: den 8 juni 2003 23:28 Subject: Good practice > bash$ erl > Erlang (JAM) emulator version 47.4.1 > > Eshell V47.4.1 (abort with ^G) > 1> c(test1file). > ./test1file.erl:172: Warning: variable 'Response' exported from ['case'] > {ok,test1file} > 2> > > > >From the file in question: > > case checkvalid(Funcref, Correctlist) of > false -> > Response = false; > Other -> > Response = invoker(Other, Otherargs) > end, > Requester ! Response, > > Obviously the warning warns me of a problem. This I expect, and welcome. > However, that doesn't mean I want to see warnings; I want to do things > right. I could just put the message in the case statement, I suppose, but > is there really a canonical way of doing the above without getting the > complaint? From svg@REDACTED Sun Jun 8 22:35:55 2003 From: svg@REDACTED (Vladimir Sekissov) Date: Mon, 09 Jun 2003 02:35:55 +0600 (YEKST) Subject: Good practice In-Reply-To: References: Message-ID: <20030609.023555.74729622.svg@surnet.ru> Good day, Some ways are: 1. Single value return: Response = case checkvalid(Funcref, Correctlist) of false -> false; Other -> invoker(Other, Otherargs) end, Requester ! Response, 2. Multiple values with pattern-matching and tuples/lists/records: {R1, R2} = case checkvalid(Funcref, Correctlist) of false -> {false, true}; Other -> invoker(Other, Otherargs) end, Requester1 ! R1, Requester2 ! R2, 3. Simplified non-local return with catch/throw to avoid many nested clauses: Response = case catch begin ... case checkvalid(Funcref, Correctlist) of false -> throw({ok, false}); _ -> ok end, ... case invoker(Other, Otherargs) of true -> throw({ok, true}); _ -> ok end, ... end of {ok, V} -> V; Error -> exit(Error) end, Requester ! Response, Best Regards, Vladimir Sekissov erlang> >From the file in question: erlang> erlang> case checkvalid(Funcref, Correctlist) of erlang> false -> erlang> Response = false; erlang> Other -> erlang> Response = invoker(Other, Otherargs) erlang> end, erlang> Requester ! Response, erlang> erlang> Obviously the warning warns me of a problem. This I expect, and welcome. erlang> However, that doesn't mean I want to see warnings; I want to do things erlang> right. I could just put the message in the case statement, I suppose, but erlang> is there really a canonical way of doing the above without getting the erlang> complaint? From francesco@REDACTED Mon Jun 9 08:24:08 2003 From: francesco@REDACTED (Francesco Cesarini) Date: Mon, 09 Jun 2003 07:24:08 +0100 Subject: Good practice References: Message-ID: <3EE42808.9000701@erlang-consulting.com> Respone = case checkvalid(Funcref, Correctlist) of false -> Response = false; Other -> Response = invoker(Other, Otherargs) end, Regards, Francesco -- http://www.erlang-consulting.com erlang@REDACTED wrote: >bash$ erl >Erlang (JAM) emulator version 47.4.1 > >Eshell V47.4.1 (abort with ^G) >1> c(test1file). >./test1file.erl:172: Warning: variable 'Response' exported from ['case'] >{ok,test1file} >2> > > >>From the file in question: > > case checkvalid(Funcref, Correctlist) of > false -> > Response = false; > Other -> > Response = invoker(Other, Otherargs) > end, > Requester ! Response, > >Obviously the warning warns me of a problem. This I expect, and welcome. >However, that doesn't mean I want to see warnings; I want to do things >right. I could just put the message in the case statement, I suppose, but >is there really a canonical way of doing the above without getting the >complaint? > > > > From erlang@REDACTED Mon Jun 9 11:31:49 2003 From: erlang@REDACTED (Peter-Henry Mander) Date: Mon, 09 Jun 2003 10:31:49 +0100 Subject: compile error (+solution) with GNU gcc version 3.3 in SuSE 8.2 Message-ID: <3EE45405.4010503@manderp.freeserve.co.uk> Hi SuSE+Erlang people, I've stumbled across a compilation problem with R9B-1 due to a deprecated file which is missing from gcc v3.3 as distributed in SuSE-8.2 (it was still there in gcc v3.2 of SuSE-8.1 and 8.0, albeit with a warning about deprecated/antiquated headers, and may be included in an update from SuSE which I haven't installed. I've included it in case you happen to need it) Along with numerous additional signed/unsigned comparison warnings compared to SuSE-8.2/gcc-3.2, I get a single error while 'make' visits the lib/orber/c_src directory: In file included from InitialReference.cc:26: InitialReference.hh:28:23: strstream.h: No such file or directory followed by other errors directly related to this one. After a quick look into /usr/include/g++/backward I found that the file strstream.h is missing in the SuSE-8.2 distribution, but not strstream, so it looks as if it's a SuSE problem, not a gcc problem. Putting the missing strtream.h file back into /usr/include/g++/backward solves the compilation problem, and everything installs successfully. Pete. -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: strstream.h URL: From serge@REDACTED Mon Jun 9 18:47:49 2003 From: serge@REDACTED (Serge Aleynikov) Date: Mon, 09 Jun 2003 12:47:49 -0400 Subject: Erlang Process doesn't receive messages Message-ID: <3EE4BA35.2050508@hq.idt.net> During the following test, I discovered a non-intuitive behavior that I'd appreciate to receive some help with. Given two processes: client (shown as a Erlang shell process), and server. The server implements the following loop in a process separate from client: loop() -> receive stop -> stopped; {test, From, Arg} -> timer:sleep(3000), From ! Arg, loop() end. Perform the commands below from the client process shell. What is not clear in this code is that despite the fact that the client has messages in its mailbox, the receive command times out without removing messages from the mailbox. Can anyone explain this? Thanks, Serge 94> Pid = server:start(). <0.139.0> 95> process_info(self(), messages). {messages,[]} 96> Pid ! {test, self(), 1}. {test,<0.142.0>,1} 97> process_info(self(), messages). {messages,[]} 98> process_info(self(), messages). {messages,[1]} 99> receive Msg -> Msg after 3000 -> timeout end. 1 100> Pid ! {test, self(), 1}. {test,<0.142.0>,1} 101> Pid ! {test, self(), 2}. {test,<0.142.0>,2} 102> process_info(self(), messages). {messages,[1]} 103> process_info(self(), messages). {messages,[1]} 104> process_info(self(), messages). {messages,[1,2]} 105> receive Msg -> Msg after 3000 -> timeout end. 1 106> receive Msg -> Msg after 3000 -> timeout end. timeout 107> receive Msg -> Msg after 3000 -> timeout end. timeout 108> process_info(self(), messages). {messages,[2]} 109> os:type(). {win32,nt} ------------------------ From svg@REDACTED Mon Jun 9 20:06:06 2003 From: svg@REDACTED (Vladimir Sekissov) Date: Tue, 10 Jun 2003 00:06:06 +0600 (YEKST) Subject: Erlang Process doesn't receive messages In-Reply-To: <3EE4BA35.2050508@hq.idt.net> References: <3EE4BA35.2050508@hq.idt.net> Message-ID: <20030610.000606.41654860.svg@surnet.ru> Good day, 4> receive Msg -> Msg after 3000 -> timeout end. 1 5> Msg. 1 6> You exported Msg declaration from `receive' and in second `receive' tried to compare with pattern Msg=1. Compiler warns you in such cases. Best Regards, Vladimir Sekissov serge> During the following test, I discovered a non-intuitive behavior that serge> I'd appreciate to receive some help with. serge> serge> Given two processes: client (shown as a Erlang shell process), and serge> server. The server implements the following loop in a process separate serge> from client: serge> serge> loop() -> serge> receive serge> stop -> serge> stopped; serge> {test, From, Arg} -> serge> timer:sleep(3000), serge> From ! Arg, serge> loop() serge> end. serge> serge> Perform the commands below from the client process shell. What is not serge> clear in this code is that despite the fact that the client has messages serge> in its mailbox, the receive command times out without removing messages serge> from the mailbox. serge> serge> Can anyone explain this? serge> serge> Thanks, serge> serge> Serge serge> serge> 94> Pid = server:start(). serge> <0.139.0> serge> 95> process_info(self(), messages). serge> {messages,[]} serge> 96> Pid ! {test, self(), 1}. serge> {test,<0.142.0>,1} serge> 97> process_info(self(), messages). serge> {messages,[]} serge> 98> process_info(self(), messages). serge> {messages,[1]} serge> 99> receive Msg -> Msg after 3000 -> timeout end. serge> 1 serge> 100> Pid ! {test, self(), 1}. serge> {test,<0.142.0>,1} serge> 101> Pid ! {test, self(), 2}. serge> {test,<0.142.0>,2} serge> 102> process_info(self(), messages). serge> {messages,[1]} serge> 103> process_info(self(), messages). serge> {messages,[1]} serge> 104> process_info(self(), messages). serge> {messages,[1,2]} serge> 105> receive Msg -> Msg after 3000 -> timeout end. serge> 1 serge> 106> receive Msg -> Msg after 3000 -> timeout end. serge> timeout serge> 107> receive Msg -> Msg after 3000 -> timeout end. serge> timeout serge> 108> process_info(self(), messages). serge> {messages,[2]} serge> 109> os:type(). serge> {win32,nt} serge> ------------------------ From ulf.wiger@REDACTED Mon Jun 9 20:03:34 2003 From: ulf.wiger@REDACTED (Wiger Ulf) Date: Mon, 9 Jun 2003 20:03:34 +0200 Subject: Erlang Process doesn't receive messages References: <3EE4BA35.2050508@hq.idt.net> Message-ID: <002101c32eb1$77ba6820$fd7a40d5@telia.com> The first time you use "receive Msg -> Msg ... end" and received 1, the pattern describes an unbound variable Msg (which matches anything). It is then bound to 1 (the first message). The next time you use the same expression, Msg is already bound, and the expression becomes a pattern match matching only the message 1. It still seems to work initially, since the following message was also 1. When you try it with the message 2, it no longer works, since 2 doesn't match the bound pattern. You may try this: f(Msg), receive Msg -> Msg after 3000 -> timeout end. This tells the shell to "forget" the variable Msg before evaluating the receive expression. /Uffe ----- Original Message ----- From: "Serge Aleynikov" To: Sent: den 9 juni 2003 18:47 Subject: Erlang Process doesn't receive messages > During the following test, I discovered a non-intuitive behavior that > I'd appreciate to receive some help with. > > Given two processes: client (shown as a Erlang shell process), and > server. The server implements the following loop in a process separate > from client: > > loop() -> > receive > stop -> > stopped; > {test, From, Arg} -> > timer:sleep(3000), > From ! Arg, > loop() > end. > > Perform the commands below from the client process shell. What is not > clear in this code is that despite the fact that the client has messages > in its mailbox, the receive command times out without removing messages > from the mailbox. > > Can anyone explain this? > > Thanks, > > Serge > > 94> Pid = server:start(). > <0.139.0> > 95> process_info(self(), messages). > {messages,[]} > 96> Pid ! {test, self(), 1}. > {test,<0.142.0>,1} > 97> process_info(self(), messages). > {messages,[]} > 98> process_info(self(), messages). > {messages,[1]} > 99> receive Msg -> Msg after 3000 -> timeout end. > 1 > 100> Pid ! {test, self(), 1}. > {test,<0.142.0>,1} > 101> Pid ! {test, self(), 2}. > {test,<0.142.0>,2} > 102> process_info(self(), messages). > {messages,[1]} > 103> process_info(self(), messages). > {messages,[1]} > 104> process_info(self(), messages). > {messages,[1,2]} > 105> receive Msg -> Msg after 3000 -> timeout end. > 1 > 106> receive Msg -> Msg after 3000 -> timeout end. > timeout > 107> receive Msg -> Msg after 3000 -> timeout end. > timeout > 108> process_info(self(), messages). > {messages,[2]} > 109> os:type(). > {win32,nt} > ------------------------ > From spearce@REDACTED Mon Jun 9 20:44:16 2003 From: spearce@REDACTED (Shawn Pearce) Date: Mon, 9 Jun 2003 14:44:16 -0400 Subject: Erlang Process doesn't receive messages In-Reply-To: <3EE4BA35.2050508@hq.idt.net> References: <3EE4BA35.2050508@hq.idt.net> Message-ID: <20030609184416.GD30778@spearce.org> In the shell, you have bound Msg to the value of 1. So during the first receive: 99> receive Msg -> Msg after 3000 -> timeout end. Msg is bound to 1. Now later on, in all other receive calls, you really have: 104> process_info(self(), messages). {messages,[1,2]} 105> receive 1 -> 1 after 3000 -> timeout end. 1 106> receive 1 -> 1 after 3000 -> timeout end. timeout as there is no message '1' in the mailbox. You must forget the binding of Msg using f(Msg) (which only works in the shell), or use another variable: 105> receive Msg1 -> Msg1 after 3000 -> timeout end. 1 106> receive Msg2 -> Msg2 after 3000 -> timeout end. 2 Serge Aleynikov wrote: > During the following test, I discovered a non-intuitive behavior that > I'd appreciate to receive some help with. > > Given two processes: client (shown as a Erlang shell process), and > server. The server implements the following loop in a process separate > from client: > > loop() -> > receive > stop -> > stopped; > {test, From, Arg} -> > timer:sleep(3000), > From ! Arg, > loop() > end. > > Perform the commands below from the client process shell. What is not > clear in this code is that despite the fact that the client has messages > in its mailbox, the receive command times out without removing messages > from the mailbox. > > Can anyone explain this? > > Thanks, > > Serge > > 94> Pid = server:start(). > <0.139.0> > 95> process_info(self(), messages). > {messages,[]} > 96> Pid ! {test, self(), 1}. > {test,<0.142.0>,1} > 97> process_info(self(), messages). > {messages,[]} > 98> process_info(self(), messages). > {messages,[1]} > 99> receive Msg -> Msg after 3000 -> timeout end. > 1 > 100> Pid ! {test, self(), 1}. > {test,<0.142.0>,1} > 101> Pid ! {test, self(), 2}. > {test,<0.142.0>,2} > 102> process_info(self(), messages). > {messages,[1]} > 103> process_info(self(), messages). > {messages,[1]} > 104> process_info(self(), messages). > {messages,[1,2]} > 105> receive Msg -> Msg after 3000 -> timeout end. > 1 > 106> receive Msg -> Msg after 3000 -> timeout end. > timeout > 107> receive Msg -> Msg after 3000 -> timeout end. > timeout > 108> process_info(self(), messages). > {messages,[2]} > 109> os:type(). > {win32,nt} > ------------------------ > -- Shawn. Be a better psychiatrist and the world will beat a psychopath to your door. From serge@REDACTED Mon Jun 9 20:26:54 2003 From: serge@REDACTED (Serge Aleynikov) Date: Mon, 09 Jun 2003 14:26:54 -0400 Subject: Erlang Process doesn't receive messages References: <3EE4BA35.2050508@hq.idt.net> <20030610.000606.41654860.svg@surnet.ru> Message-ID: <3EE4D16E.4050202@hq.idt.net> Thanks. I had a feeling that it's somehow related to side-effects of the shell, because when the "receive Msg -> Msg after 3000 -> timeout end." is incapsulated in a function, when that function is called the message queue poll worked correctly. Serge Vladimir Sekissov wrote: > Good day, > > 4> receive Msg -> Msg after 3000 -> timeout end. > 1 > 5> Msg. > 1 > 6> > > You exported Msg declaration from `receive' and in second `receive' > tried to compare with pattern Msg=1. > > Compiler warns you in such cases. > > Best Regards, > Vladimir Sekissov > > serge> During the following test, I discovered a non-intuitive behavior that > serge> I'd appreciate to receive some help with. > serge> > serge> Given two processes: client (shown as a Erlang shell process), and > serge> server. The server implements the following loop in a process separate > serge> from client: > serge> > serge> loop() -> > serge> receive > serge> stop -> > serge> stopped; > serge> {test, From, Arg} -> > serge> timer:sleep(3000), > serge> From ! Arg, > serge> loop() > serge> end. > serge> > serge> Perform the commands below from the client process shell. What is not > serge> clear in this code is that despite the fact that the client has messages > serge> in its mailbox, the receive command times out without removing messages > serge> from the mailbox. > serge> > serge> Can anyone explain this? > serge> > serge> Thanks, > serge> > serge> Serge > serge> > serge> 94> Pid = server:start(). > serge> <0.139.0> > serge> 95> process_info(self(), messages). > serge> {messages,[]} > serge> 96> Pid ! {test, self(), 1}. > serge> {test,<0.142.0>,1} > serge> 97> process_info(self(), messages). > serge> {messages,[]} > serge> 98> process_info(self(), messages). > serge> {messages,[1]} > serge> 99> receive Msg -> Msg after 3000 -> timeout end. > serge> 1 > serge> 100> Pid ! {test, self(), 1}. > serge> {test,<0.142.0>,1} > serge> 101> Pid ! {test, self(), 2}. > serge> {test,<0.142.0>,2} > serge> 102> process_info(self(), messages). > serge> {messages,[1]} > serge> 103> process_info(self(), messages). > serge> {messages,[1]} > serge> 104> process_info(self(), messages). > serge> {messages,[1,2]} > serge> 105> receive Msg -> Msg after 3000 -> timeout end. > serge> 1 > serge> 106> receive Msg -> Msg after 3000 -> timeout end. > serge> timeout > serge> 107> receive Msg -> Msg after 3000 -> timeout end. > serge> timeout > serge> 108> process_info(self(), messages). > serge> {messages,[2]} > serge> 109> os:type(). > serge> {win32,nt} > serge> ------------------------ From jay@REDACTED Tue Jun 10 03:02:02 2003 From: jay@REDACTED (Jay Nelson) Date: Mon, 09 Jun 2003 18:02:02 -0700 Subject: Piccola and Pi Calculus Message-ID: <4.2.2.20030609180034.00cec970@duomark.com> Anybody familiar with Piccola and the Pi Calculus. It looks like a formalization of the erlang concepts (apparently without any knowledge of the existence of erlang). http://iamwww.unibe.ch/~scg/Research/Piccola/ jay From erlang@REDACTED Tue Jun 10 05:32:26 2003 From: erlang@REDACTED (erlang@REDACTED) Date: Mon, 9 Jun 2003 23:32:26 -0400 (EDT) Subject: restricted execution Message-ID: This is my week for stupid questions ... on the plus side I'm finding erlang so compelling that I might actually gain some proficiency by dint of outright practice. This evening's topic for debate before the symposium: In my researches online (so many that by now my head spins) I see lots of references to safe, or safer, erlang, and various models of restricted execution. Well, that's nice. If I really, really wanted to run something relatively customised, I could always implement a virtual machine (not a possibility I have completely eliminated, but one I'd prefer to avoid). I'd rather not, and this is pretty much my bottom line, have to deal with non-standard forms of the language. What I do want to be able to do: run a user-provided process with some assurance that the only external data access it has is precisely that which I can provide it. I realise that quite likely the basic form of Erlang doesn't make provision for this, but looking at the facilities for cookies and so on, it strikes me that something ought to be possible. Is there a particularly common or usual answer to this? I've worked in telecom situations, and I know that quite a lot of kit works with shared secrets, which is pretty much what the cookies implement. Anyone have any suggestions? My ideal situation: spawn a process, give it in its arguments a few processes it can communicate with, and leave it with no other sources of information, nor external contact facilities. From fgonthier@REDACTED Tue Jun 10 04:27:08 2003 From: fgonthier@REDACTED (=?iso-8859-1?Q?Fran=E7ois-Denis_Gonthier?=) Date: Mon, 9 Jun 2003 22:27:08 -0400 Subject: Erlang-mode problem Message-ID: Hello all, This is my first message here so I don't really know if I'm posting from the right place. I've got a problem with erlang-mode with XEmacs in Windows. I will not describe all the kind of errors I get when I try to compile a file but it all comes down to using \ instead of /. The earlier messes up Erlang and/or XEmacs and I get 'no such file or directory' and other more or less weird errors. I'm not exactly an (X)Emacs enthusiast but I'm willing to use it when I know it's superior. I checked erlang-mode.el but that big file is way beyond my limited ELisp skills. Anyone found a solution? Fran?ois-Denis Gonthier From spearce@REDACTED Tue Jun 10 04:47:27 2003 From: spearce@REDACTED (Shawn Pearce) Date: Mon, 9 Jun 2003 22:47:27 -0400 Subject: restricted execution In-Reply-To: References: Message-ID: <20030610024727.GA31922@spearce.org> I'd go a different, but close approach: spawn a process, and in spawn_opt give it an optional security module: {security_manager, mysecmgr}. The module mysecmgr implements a 'security_manager' behavior, such as: -module(mysecmgr). -behavior(security_manager). -export_all. send_message(To, Message) -> case sets:exists(get(mysecmgr_sendlist), To) of true -> To ! Message; false -> exit(security_error) end. put(mysecmgr_safemod, _) -> exit(security_error); put(mysecmgr_sendlist, _) -> exit(security_error); put(Key, Value) -> erlang:put(Key, Value). remote_call(erlang, Function, Args) -> apply(erlang, Function, Args); remote_call(lists, Function, Args) -> apply(lists, Function, Args); remote_call(sets, Function, Args) -> apply(sets, Function, Args); remote_call(dict, Function, Args) -> apply(dict, Function, Args); remote_call(_, _, _) -> exit(security_error). The VM would have to pass these 3 critical calls (!/send_message, put and remote module call) through the security manager module, rather than doing it directly. But the security manager would have to have access to do the calls itself, without recursing into itself. Sort of weird I guess, and most likely impossible to implement without creating duplicate versions of every module ('safe' and 'unsafe') and making sure the security manager's apply/3 in remote_call/3 doesn't jump to the unsafe version of any module (as that might then be able to subvert the security manager). My other idea was to do this at compile time. Create a parse transform in erlc that cannot be bypassed, which would convert these 3 cases to calls to the security manager module, which is obtained from the process dictionary. Also define a module_info/1 flag which indicates if if the module is safe or not. The security manager module would be complied without the +safe parse transform option. When you spawn, only spawn into a safe module. If the safe module will call an unsafe module (or send a message, or attempt to modify the process dictionary), it goes through the security manager set in the process dictionary, which is an 'unsafe' module and can call the other unsafe modules, or rewrite the module name to call the 'safe' form. This of course leaves open the attack that a user may give you a .beam file which claims it is safe, but isn't. Here is where a bytecode verifier is needed to ensure remote calls are only done to the security manager, and the same with remote sends. Yeck, now we are getting up to the point of Java. I've been thinking about this a lot lately, as I have been contemplating building a system in Erlang where I'd want safe and unsafe processes, some running user code, others' not, and the user code should not have unlimited access to gen_tcp, file, etc. Since users would be required to supply source code, and have the node compile it, I can ensure it was built with the +safe parse transform, and ensure they go through my security module. Which would be effective, but slower. I don't know enough about the gen_tcp/gen_udp/ssl/file/code/etc. modules to know if just limiting remote sends is enough. Certainly a 'safe' process must also not be able to open ports, etc so some erlang module functions must also be offlimits to these processes. However, I see little point to restrict their use of sets, dict, lists, string. But ets/dets does make sense to restrict access to as well. erlang@REDACTED wrote: > This is my week for stupid questions ... on the plus side I'm finding > erlang so compelling that I might actually gain some proficiency by dint of > outright practice. > > This evening's topic for debate before the symposium: > > In my researches online (so many that by now my head spins) I see lots of > references to safe, or safer, erlang, and various models of restricted > execution. > > Well, that's nice. If I really, really wanted to run something relatively > customised, I could always implement a virtual machine (not a possibility > I have completely eliminated, but one I'd prefer to avoid). I'd rather not, > and this is pretty much my bottom line, have to deal with non-standard > forms of the language. > > What I do want to be able to do: run a user-provided process with some > assurance that the only external data access it has is precisely that which > I can provide it. > > I realise that quite likely the basic form of Erlang doesn't make provision > for this, but looking at the facilities for cookies and so on, it strikes > me that something ought to be possible. > > Is there a particularly common or usual answer to this? I've worked in > telecom situations, and I know that quite a lot of kit works with shared > secrets, which is pretty much what the cookies implement. Anyone have > any suggestions? > > My ideal situation: spawn a process, give it in its arguments a few > processes it can communicate with, and leave it with no other sources of > information, nor external contact facilities. > > -- Shawn. Art is the tree of life. Science is the tree of death. From serge@REDACTED Tue Jun 10 05:08:28 2003 From: serge@REDACTED (Serge Aleynikov) Date: Mon, 09 Jun 2003 23:08:28 -0400 Subject: Erlang-mode problem References: Message-ID: <3EE54BAC.4080006@hq.idt.net> Are you running X server on a Windows desktop and the Erlang VM on Linux/Unix, or the VM is also running on Windows, and you have an Emacs Windows port. If you are complitely in the Windows environment you can use pretty much any text editor besides Emacs for editing Erlang code. I found Multi-Edit quite productive as it can be set up to do auto-highlighting, compiling from within the editor, and others. Also, when using file references use the "/" notation instead of traditional Windows "\" style. Regards, Serge Fran?ois-Denis Gonthier wrote: > Hello all, > > This is my first message here so I don't really know if I'm posting from > the right place. > > I've got a problem with erlang-mode with XEmacs in Windows. I will not > describe all the kind of errors I get when I try to compile a file but it > all comes down to using \ instead of /. The earlier messes up Erlang and/or > XEmacs and I get 'no such file or directory' and other more or less weird > errors. > > I'm not exactly an (X)Emacs enthusiast but I'm willing to use it when I > know it's superior. I checked erlang-mode.el but that big file is way > beyond my limited ELisp skills. > > Anyone found a solution? > > Fran?ois-Denis Gonthier > > From serge@REDACTED Tue Jun 10 05:18:11 2003 From: serge@REDACTED (Serge Aleynikov) Date: Mon, 09 Jun 2003 23:18:11 -0400 Subject: SIP Stack Message-ID: <3EE54DF3.3040201@hq.idt.net> Does anybody have a SIP stack implementation in Erlang? Thanks. Serge From fgonthier@REDACTED Tue Jun 10 05:18:13 2003 From: fgonthier@REDACTED (=?iso-8859-1?Q?Fran=E7ois-Denis_Gonthier?=) Date: Mon, 9 Jun 2003 23:18:13 -0400 Subject: Erlang-mode problem In-Reply-To: <3EE54BAC.4080006@hq.idt.net> Message-ID: Cool! Quick answer! Unfortunately, I'm 100% in Windows using native XEmacs and running Windows version of the Erlang VM. As for using other editors, I'm still fond of GVim but (X)Emacs has some incredibles advantages over any other editor for interactive languages like Erlang. The source of the problem is not really erlang-mode but the Win32-native version of XEmacs. It's set to produce DOS paths by default. When I manually enter / paths in the shell window, it works just beautifully. -----Message d'origine----- De : Serge Aleynikov [mailto:serge@REDACTED] Envoy? : 9 juin 2003 23:08 ? : Fran?ois-Denis Gonthier Cc : erlang-questions@REDACTED Objet : Re: Erlang-mode problem Are you running X server on a Windows desktop and the Erlang VM on Linux/Unix, or the VM is also running on Windows, and you have an Emacs Windows port. If you are complitely in the Windows environment you can use pretty much any text editor besides Emacs for editing Erlang code. I found Multi-Edit quite productive as it can be set up to do auto-highlighting, compiling from within the editor, and others. Also, when using file references use the "/" notation instead of traditional Windows "\" style. From tobbe@REDACTED Tue Jun 10 08:49:11 2003 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: Tue, 10 Jun 2003 08:49:11 +0200 Subject: restricted execution In-Reply-To: References: Message-ID: <3EE57F67.5080900@bluetail.com> Lawrie Brown, , has done lots of work in this area. Check with him what the status is of his work. Cheers , Tobbe erlang@REDACTED wrote: >This is my week for stupid questions ... on the plus side I'm finding >erlang so compelling that I might actually gain some proficiency by dint of >outright practice. > >This evening's topic for debate before the symposium: > >In my researches online (so many that by now my head spins) I see lots of >references to safe, or safer, erlang, and various models of restricted >execution. > >Well, that's nice. If I really, really wanted to run something relatively >customised, I could always implement a virtual machine (not a possibility >I have completely eliminated, but one I'd prefer to avoid). I'd rather not, >and this is pretty much my bottom line, have to deal with non-standard >forms of the language. > >What I do want to be able to do: run a user-provided process with some >assurance that the only external data access it has is precisely that which >I can provide it. > >I realise that quite likely the basic form of Erlang doesn't make provision >for this, but looking at the facilities for cookies and so on, it strikes >me that something ought to be possible. > >Is there a particularly common or usual answer to this? I've worked in >telecom situations, and I know that quite a lot of kit works with shared >secrets, which is pretty much what the cookies implement. Anyone have >any suggestions? > >My ideal situation: spawn a process, give it in its arguments a few >processes it can communicate with, and leave it with no other sources of >information, nor external contact facilities. > > > From Lawrie.Brown@REDACTED Tue Jun 10 13:34:31 2003 From: Lawrie.Brown@REDACTED (Lawrie Brown) Date: Tue, 10 Jun 2003 21:34:31 +1000 Subject: restricted execution In-Reply-To: <3EE57F67.5080900@bluetail.com> References: <3EE57F67.5080900@bluetail.com> Message-ID: <20030610113431.GA29924@icarus.its.adfa.edu.au> On Tue, Jun 10, 2003 at 08:49:11AM +0200, Torbjorn Tornkvist wrote: > Lawrie Brown, , has done lots of work > in this area. Check with him what the status is of his work. > erlang@REDACTED wrote: .... > >What I do want to be able to do: run a user-provided process with some > >assurance that the only external data access it has is precisely that which > >I can provide it. > > > >I realise that quite likely the basic form of Erlang doesn't make provision > >for this, but looking at the facilities for cookies and so on, it strikes > >me that something ought to be possible. The essence of the answer is to ensure that the "untrusted" erlang node is forced to use a (more) trusted node to mediate all external communications to ensure it conforms to your policy. As it stands I don't believe the current OTP erlang ndes make it easy to do this (though by running the untrusted node in a chroot jail and seriously constraining its environment you could probably hack it up). It IS indeed one of the key features I intended for the "safe erlang" proposals I worked up with Dan Sahlin back in 1997. At that time we did a "proof of concept" as a shim layer (in Erlang) over the then current OTP implementation (which used the JAM machine then). Last year I worked on implenting the extensions for real using the EC compiler being developed by SERC, RMIT. Its about 95% done (the missing bit is the distribution protocol - which is pretty critical I'm the first to agree). Unfortunately work pressures mean this is on hold till I get time to finish it (or someone else does). Because I was busy coding, I haven't yet had a lot of time to write the work up (though I'll have a paper on some of the design decisions I had to make in the runtime for AUUG in Sept this year). What info there is you can source off my web pages on this at: http://www.unsw.adfa.edu.au/~lpb/papers/ssp02/index.html The AUUG paper should hopefully be up in the next month or so. Cheers Lawrie ------------------------------------ <*> ------------------------------------ Post: Dr Lawrie Brown, Computer Science, UNSW@REDACTED, Canberra 2600 Australia Phone: 02 6268 8816 Fax: 02 6268 8581 Web: http://www.adfa.edu.au/~lpb/ From etxmacr@REDACTED Tue Jun 10 09:23:10 2003 From: etxmacr@REDACTED (Mats Cronqvist) Date: Tue, 10 Jun 2003 09:23:10 +0200 Subject: Folding emacs Message-ID: <200306100723.h5A7NBx27910@cbe2077.al.sw.ericsson.se> following the recent discussion about folding editors i found that putting this in the .emacs file will activate the emacs (21+) folding support (cleverly named hideshow). it will only fold erlang function clauses. M-h/M-u hides/unhides entire buffer, M-s toggles hiding of function clause at point. (bug: hiding doesn't work if point is on top of the function name...) mats (add-hook 'erlang-mode-hook 'my-erlang-mode-hook) (defun my-erlang-mode-hook () (setq hs-special-modes-alist (cons '(erlang-mode "^\\([a-z][a-zA-Z0-9_]*\\|'[^\n']*[^\\]'\\)\\s *(" nil "%" erlang-end-of-clause) hs-special-modes-alist)) (hs-minor-mode 1) (local-set-key [?\M-s] 'hs-toggle-hiding) (local-set-key [?\M-h] 'hs-hide-all) (local-set-key [?\M-u] 'hs-show-all)) From erlang@REDACTED Wed Jun 11 01:54:31 2003 From: erlang@REDACTED (erlang@REDACTED) Date: Tue, 10 Jun 2003 19:54:31 -0400 (EDT) Subject: restricted execution In-Reply-To: <20030610113431.GA29924@icarus.its.adfa.edu.au> from "Lawrie Brown" at Jun 10, 2003 09:34:31 PM Message-ID: First: thank you very much for being so helpful :) > The essence of the answer is to ensure that the "untrusted" erlang node is > forced to use a (more) trusted node to mediate all external communications > to ensure it conforms to your policy. As it stands I don't believe the > current OTP erlang ndes make it easy to do this (though by running the > untrusted node in a chroot jail and seriously constraining its environment > you could probably hack it up). Actually, environment constraints are feasible in my envisioned application. I can almost certainly use a read-only filesystem, and only have network connectivity to other erlang-using machines, hence no generalised internet access (unless mediated by a trusted erlang node). The hard part is forcing other erlang nodes into not honouring spawn commands, but still accepting ordinary messages. As I understand it, if I can guarantee the host's behaviour, then constraining the behaviour of other erlang nodes requires messing around with the net_kernel? [snip reference to interesting paper] The only other alternative that I really see is pretty much implementation of a virtual machine... which might actually be the easier answer. From Lawrie.Brown@REDACTED Wed Jun 11 02:06:48 2003 From: Lawrie.Brown@REDACTED (Lawrie Brown) Date: Wed, 11 Jun 2003 10:06:48 +1000 Subject: restricted execution In-Reply-To: References: <20030610113431.GA29924@icarus.its.adfa.edu.au> Message-ID: <20030611000648.GA12341@icarus.its.adfa.edu.au> On Tue, Jun 10, 2003 at 07:54:31PM -0400, erlang@REDACTED wrote: > > The essence of the answer is to ensure that the "untrusted" erlang node is > > forced to use a (more) trusted node to mediate all external communications > > to ensure it conforms to your policy. As it stands I don't believe the ... > Actually, environment constraints are feasible in my envisioned application. > I can almost certainly use a read-only filesystem, and only have network > connectivity to other erlang-using machines, hence no generalised internet > access (unless mediated by a trusted erlang node). The hard part is > forcing other erlang nodes into not honouring spawn commands, but still > accepting ordinary messages. This is getting into an area I've not investigated in detail, but I'd suggest firstly looking at the node type - there is a "hidden" node type usually used for C-nodes that does not advertise itself, whether a normal OTP node can be coerced into being such I'm not sure. The other point is indeed to play with the list of applications launched at startup, especially the ones which manage the list of known nodes, which could then be limited. Let me know how you go. Cheers Lawrie ------------------------------------ <*> ------------------------------------ Post: Dr Lawrie Brown, Computer Science, UNSW@REDACTED, Canberra 2600 Australia Phone: 02 6268 8816 Fax: 02 6268 8581 Web: http://www.adfa.edu.au/~lpb/ From ekarttun@REDACTED Wed Jun 11 10:42:56 2003 From: ekarttun@REDACTED (Einar Karttunen) Date: Wed, 11 Jun 2003 11:42:56 +0300 Subject: embedding erlang Message-ID: <20030611084256.GC28179@melkinpaasi.cs.Helsinki.FI> Hello I am thinking of embedding an erlang runtime systems inside of an app. There would be C code handling the GUI side / startup, but there are various issues I am unsure about. The application has to maintain a high number of encrypted connections and calculate some numeric math. Is it possible to fit the erlang system inside of one file as opposed to the normal filesystem layout? How much space does the whole thing take if I would rip nonused libraries and the compiler out? Is this possible at all? How well are ssl sockets supported under windows currently? I think there have been some limitations in the past. (yes, the customers want a windows version too...) As an alternative for erlang we have C++ which would make things quite ugly. Any suggestions? - Einar Karttunen From kotkaus@REDACTED Wed Jun 11 09:38:08 2003 From: kotkaus@REDACTED (James Freddy) Date: Wed, 11 Jun 2003 00:38:08 -0700 (PDT) Subject: xmerl Message-ID: <20030611073808.51593.qmail@web10902.mail.yahoo.com> hi, i'm trying to use xmerl to output xml. i think what i want is to use xmerl:simple/3 and i followed the example on how to pass in the callback however the output is not xml. is there a simple example somehwere that shows how to output XML using xmerl? thanks. j __________________________________ Do you Yahoo!? Yahoo! Calendar - Free online calendar with sync to Outlook(TM). http://calendar.yahoo.com From etxuwig@REDACTED Wed Jun 11 13:54:24 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Wed, 11 Jun 2003 13:54:24 +0200 (MEST) Subject: xmerl In-Reply-To: <20030611073808.51593.qmail@web10902.mail.yahoo.com> Message-ID: On Wed, 11 Jun 2003, James Freddy wrote: >i'm trying to use xmerl to output xml. i think what i want >is to use xmerl:simple/3 and i followed the example on how >to pass in the callback however the output is not xml. is >there a simple example somehwere that shows how to output >XML using xmerl? Could you provide a little bit more detail about what you did and what the output was? I tried test:foo2() in xmerl-0.18, and it was broken. I then tried the following. Apart from a missing line break in the exported data, it looks pretty much as intended. /Uffe 1> xmerl_scan:file("../priv/testdata/simple.xml"). {{xmlElement,section, [], [], 1, [{xmlAttribute,heading,[],1,[],[],[],[],"heading1"}], [{xmlText,[{section,1}],1,[],"\n "}, {xmlElement,'P', [{section,1}], [], 2, [], [{xmlText,[{'P',2},{section,1}], 1, [], "This is a paragraph of text."}], [], 'P', [], {xmlNamespace,[],[]}}, {xmlText,[{section,1}],3,[],"\n "}, {xmlElement,section, [{section,1}], [], 4, [{xmlAttribute,heading,[],1,[],[],[],[]|...}], [{xmlText,[{section,4},{section,1}],1,[],"\n "}, {xmlElement, 'P', [{section|...},{...}], [], 2|...}, {xmlText,[{section,4},{section|...}],3,[],"\n "}, {xmlElement,table,[{...}|...],[]|...}, {xmlText,[{...}|...],5|...}], [], section, [], {xmlNamespace,[],[]}}, {xmlText,[{section,1}],5,[],"\n"}], [], section, [], {xmlNamespace,[],[]}}, "\n"} 2> xmerl:export(element(1,v(1)),xmerl_xml). ["", [["<","section",[[" ","heading","=\"","heading1","\""]],">"], ["\n ", [["<","P",">"],["This is a paragraph of text."],[""]], "\n ", [["<","section",[[" ","heading","=\"","heading2","\""]],">"], ["\n ", [["<","P",">"],["This is another paragraph"],[""]], "\n ", [["<","table",[[" ","border","=\"","1","\""]],">"], ["\n ", [["<","heading",">"], ["\n ", [["<","col",">"],["head1"],[""]], "\n ", [["<","col",">"],["head2"],[""]], "\n ", [["<","row",">"], ["\n ", [["<","col",">"],["col11"],[""]], "\n ", [["<","row",">"], ["\n ", [["<","col"|...],["col21"],[...]], "\n ", [[...]|...], "\n "], [""]], "\n "], [""]], "\n "], [""]], "\n"], [""]]] 3> io:format("~s~n", [v(2)]).

This is a paragraph of text.

This is another paragraph

head1head2 col11col12 col21col22
ok 5> {ok,B} = file:read_file("../priv/testdata/simple.xml"),io:format("~s~n", [binary_to_list(B)]).

This is a paragraph of text.

This is another paragraph

head1head2 col11col12 col21col22
ok -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes From vances@REDACTED Thu Jun 12 00:16:22 2003 From: vances@REDACTED (Vance Shipley) Date: Wed, 11 Jun 2003 18:16:22 -0400 Subject: bignum support in ei? Message-ID: <20030611221622.GR31639@frogman.motivity.ca> How do I receive a bignum in ei? Using ei_decode_long() doesn't work. When I send integers and use ei_print_term() to see what was received and ei_get_type() to see the type I get the following: Sent Received Type ---------------- --------------------------------- --------------------- 16#00000000 0 ERL_SMALL_INTEGER_EXT 16#00000001 1 ERL_SMALL_INTEGER_EXT 16#000000ff 1 ERL_SMALL_INTEGER_EXT 16#000001ff 65535 ERL_INTEGER_EXT 16#00ffffff 16777215 ERL_INTEGER_EXT 16#07ffffff 134217727 ERL_INTEGER_EXT 16#08ffffff #integer(2) = {65535,2303} ERL_SMALL_BIG_EXT 16#ffffffff #integer(2) = {65535,65535} ERL_SMALL_BIG_EXT 16#fffffffffffff #integer(3) = {65535,65535,65535} ERL_SMALL_BIG_EXT Seems like a strange place to make the last break but anyway. The ei_get_type() function returns a size of 0 for ERL_SMALL_INTEGER_EXT and ERL_INTEGER_EXT but when the type is ERL_SMALL_BIG_EXT size is the number of elements in the tuple. There is also an ERL_LARGE_BIG_EXT type defined but I don't know how big a number you need to get it. I see from the source that there is an ei_decode_bignum() however it is noted that it just skips past it. -Vance From erlang@REDACTED Thu Jun 12 02:14:08 2003 From: erlang@REDACTED (erlang@REDACTED) Date: Wed, 11 Jun 2003 20:14:08 -0400 (EDT) Subject: Restricted erlang Message-ID: In connection with my earlier questions on safe implementations of erlang, I've been looking at a lobotomised version of erlang from which all I/O except message-passing has been removed, as well as direct OS access. I can easily enough remove enough of the modules to make a big difference, but some of the BIFs will need removal too. Before I get involved with a huge chopping exercise, does anyone have advice or recommendations? Obviously if there's a relatively painless way of doing this (let's say, a simple removal of certain OTP components, reduction of some of the inner source code to stub procedures) I would like to do that. From kent@REDACTED Thu Jun 12 01:35:23 2003 From: kent@REDACTED (Kent Boortz) Date: 12 Jun 2003 01:35:23 +0200 Subject: bignum support in ei? In-Reply-To: <20030611221622.GR31639@frogman.motivity.ca> References: <20030611221622.GR31639@frogman.motivity.ca> Message-ID: Vance Shipley writes: > How do I receive a bignum in ei? Using ei_decode_long() doesn't work. You mean an Erlang integer that don't fit in a C long or unsigned long? No, there is no way to decode this in the current version of ei. > When I send integers and use ei_print_term() to see what was received > and ei_get_type() to see the type I get the following: > > Sent Received Type > ---------------- --------------------------------- --------------------- > 16#00000000 0 ERL_SMALL_INTEGER_EXT > 16#00000001 1 ERL_SMALL_INTEGER_EXT > 16#000000ff 1 ERL_SMALL_INTEGER_EXT > 16#000001ff 65535 ERL_INTEGER_EXT > 16#00ffffff 16777215 ERL_INTEGER_EXT > 16#07ffffff 134217727 ERL_INTEGER_EXT > 16#08ffffff #integer(2) = {65535,2303} ERL_SMALL_BIG_EXT > 16#ffffffff #integer(2) = {65535,65535} ERL_SMALL_BIG_EXT > 16#fffffffffffff #integer(3) = {65535,65535,65535} ERL_SMALL_BIG_EXT > > Seems like a strange place to make the last break but anyway. What do you mean? What break and why is it strange? > The ei_get_type() function returns a size of 0 for ERL_SMALL_INTEGER_EXT > and ERL_INTEGER_EXT but when the type is ERL_SMALL_BIG_EXT size is the > number of elements in the tuple. There is also an ERL_LARGE_BIG_EXT > type defined but I don't know how big a number you need to get it. > > I see from the source that there is an ei_decode_bignum() however it is > noted that it just skips past it. In R9C there will be a ei_decode_longlong() and ei_decode_ulonglong(), e.i. 64-bit integers. R9C on Unix will also include support for GMP bignums that can be enabled if you have the GMP library installed and configure before compiling from source using --with-gmp=PATH specify location of GNU MP include and lib --with-gmp use GNU MP (will search for it) Using GMP you can handle integers of all sizes, kent Ref: http://www.swox.com/gmp/ New functions (preleminary): int ei_decode_longlong(const char *buf, int *index, EI_LONGLONG *p); int ei_decode_ulonglong(const char *buf, int *index, EI_ULONGLONG *p); int ei_encode_longlong(char *buf, int *index, EI_LONGLONG p); int ei_encode_ulonglong(char *buf, int *index, EI_ULONGLONG p); int ei_x_encode_longlong(ei_x_buff* x, EI_LONGLONG n); int ei_x_encode_ulonglong(ei_x_buff* x, EI_ULONGLONG n); int ei_decode_bignum(const char *buf, int *index, mpz_t obj); int ei_encode_bignum(char *buf, int *index, mpz_t obj); int ei_x_encode_bignum(ei_x_buff *x, mpz_t obj); From Pekka.Hedqvist@REDACTED Thu Jun 12 01:36:36 2003 From: Pekka.Hedqvist@REDACTED (Pekka Hedqvist) Date: Thu, 12 Jun 2003 01:36:36 +0200 Subject: SIP Stack In-Reply-To: <3EE54DF3.3040201@hq.idt.net> Message-ID: <002601c33072$54402200$b35c413e@burkeniv> ftp://ftp.stacken.kth.se/pub/random/map/ > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED] On Behalf Of > Serge Aleynikov > Sent: Tuesday, June 10, 2003 5:18 AM > To: erlang-questions@REDACTED > Subject: SIP Stack > > > Does anybody have a SIP stack implementation in Erlang? > > Thanks. > > Serge > > From vances@REDACTED Thu Jun 12 04:20:44 2003 From: vances@REDACTED (Vance Shipley) Date: Wed, 11 Jun 2003 22:20:44 -0400 Subject: bignum support in ei? In-Reply-To: References: <20030611221622.GR31639@frogman.motivity.ca> Message-ID: <20030612022044.GX31639@frogman.motivity.ca> On Thu, Jun 12, 2003 at 01:35:23AM +0200, Kent Boortz wrote: } } You mean an Erlang integer that don't fit in a C long or unsigned long? } No, there is no way to decode this in the current version of ei. Actually I was trying to send 16#ffffffff which is 32 bits no? It would appear that if the upper nibble of a 32 bit integer is not zero it becomes an ERL_SMALL_BIG_EXT: } > 16#07ffffff 134217727 ERL_INTEGER_EXT } > 16#08ffffff #integer(2) = {65535,2303} ERL_SMALL_BIG_EXT ... } > Seems like a strange place to make the last break but anyway. } } What do you mean? What break and why is it strange? Seems strange to me. Why can I send this 32 bit value: 0000 1111 1111 1111 1111 1111 1111 1111 but not this one (without becoming a bignum): 0001 1111 1111 1111 1111 1111 1111 1111 } In R9C there will be a ei_decode_longlong() and ei_decode_ulonglong(), } e.i. 64-bit integers. I was wondering what would happen on a 64-bit emulator. The R9C changes sound good. I can wait, in the mean time I'm just working around the limitation for an all ones 32 value. -Vance From kent@REDACTED Thu Jun 12 04:51:54 2003 From: kent@REDACTED (Kent Boortz) Date: 12 Jun 2003 04:51:54 +0200 Subject: bignum support in ei? In-Reply-To: <20030612022044.GX31639@frogman.motivity.ca> References: <20030611221622.GR31639@frogman.motivity.ca> <20030612022044.GX31639@frogman.motivity.ca> Message-ID: Vance Shipley writes: > On Thu, Jun 12, 2003 at 01:35:23AM +0200, Kent Boortz wrote: > } > } You mean an Erlang integer that don't fit in a C long or unsigned long? > } No, there is no way to decode this in the current version of ei. > > Actually I was trying to send 16#ffffffff which is 32 bits no? As documented in $ERL_TOP/erts/emulator/internal_doc/erl_ext_dist.txt the 32 bit integer INTEGER_EXT in the external format is signed. So the largest erlang integer it can hold is 16#7fffffff. > It would appear that if the upper nibble of a 32 bit integer is not zero > it becomes an ERL_SMALL_BIG_EXT: > > } > 16#07ffffff 134217727 ERL_INTEGER_EXT > } > 16#08ffffff #integer(2) = {65535,2303} ERL_SMALL_BIG_EXT > ... > } > Seems like a strange place to make the last break but anyway. > } > } What do you mean? What break and why is it strange? > > Seems strange to me. Why can I send this 32 bit value: > > 0000 1111 1111 1111 1111 1111 1111 1111 > > but not this one (without becoming a bignum): > > 0001 1111 1111 1111 1111 1111 1111 1111 You are right, this is a bit strange. It comes from how integers are represented in the emulator. The external format doesn't specify that an integer has to be packed into the smallest possible container in the external format but I think both the Erlang encode functions and the ei encode functions should do that. The decoders have to be more tolerant. Note that you can still use ei_decode_long() to read the ERL_SMALL_BIG_EXT as long as it fits into a 32 bit signed integer. > } In R9C there will be a ei_decode_longlong() and ei_decode_ulonglong(), > } e.i. 64-bit integers. > > I was wondering what would happen on a 64-bit emulator. ei_decode_longlong() and ei_decode_long() will be exactly the same function on a 64-bit emulator. > The R9C changes sound good. I can wait, in the mean time I'm just > working around the limitation for an all ones 32 value. Not for production code but you can always test the latest snapshot from www.erlang.org, kent From mbj@REDACTED Thu Jun 12 08:30:41 2003 From: mbj@REDACTED (Martin Bjorklund) Date: Thu, 12 Jun 2003 08:30:41 +0200 (CEST) Subject: compiler bug Message-ID: <20030612.083041.15246038.mbj@bluetail.com> I just found a compiler bug in R8 and R9: --------------------------------- -module(h). -compile(export_all). h() -> if true -> 1; []; true -> 2 end. --------------------------------- 1> c(h). {ok,h} 2> h:h(). 1 /martin From vances@REDACTED Thu Jun 12 08:41:08 2003 From: vances@REDACTED (Vance Shipley) Date: Thu, 12 Jun 2003 02:41:08 -0400 Subject: bignum support in ei? In-Reply-To: References: <20030611221622.GR31639@frogman.motivity.ca> <20030612022044.GX31639@frogman.motivity.ca> Message-ID: <20030612064108.GZ31639@frogman.motivity.ca> On Thu, Jun 12, 2003 at 04:51:54AM +0200, Kent Boortz wrote: } } Note that you can still use ei_decode_long() to read the ERL_SMALL_BIG_EXT } as long as it fits into a 32 bit signed integer. Ah, right you are. Cool. -Vance From bjorn@REDACTED Thu Jun 12 10:12:48 2003 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 12 Jun 2003 10:12:48 +0200 Subject: compiler bug In-Reply-To: <20030612.083041.15246038.mbj@bluetail.com> References: <20030612.083041.15246038.mbj@bluetail.com> Message-ID: Your program is equivalent to the following program: h() -> if true -> 1; [] =:= true; true -> 2 end. It is not a bug in the compiler. /Bjorn Martin Bjorklund writes: > I just found a compiler bug in R8 and R9: > > --------------------------------- > -module(h). > -compile(export_all). > > h() -> > if true -> > 1; > []; > true -> > 2 > end. > --------------------------------- > > > 1> c(h). > {ok,h} > 2> h:h(). > 1 > > > > > > > /martin > -- Bj?rn Gustavsson Ericsson Utvecklings AB bjorn@REDACTED ?T2/UAB/F/P BOX 1505 +46 8 727 56 87 125 25 ?lvsj? From mbj@REDACTED Thu Jun 12 10:44:19 2003 From: mbj@REDACTED (Martin Bjorklund) Date: Thu, 12 Jun 2003 10:44:19 +0200 (CEST) Subject: compiler bug In-Reply-To: References: <20030612.083041.15246038.mbj@bluetail.com> Message-ID: <20030612.104419.17843926.mbj@bluetail.com> Bjorn Gustavsson wrote: > Your program is equivalent to the following program: > > h() -> > if > true -> > 1; > [] =:= true; true -> > 2 > end. > > It is not a bug in the compiler. Aha... New feature obviously... (from R8?) /martin From A.BETIS@REDACTED Thu Jun 12 12:33:53 2003 From: A.BETIS@REDACTED (BETIS Alexandre) Date: Thu, 12 Jun 2003 12:33:53 +0200 Subject: Problem with orber, alternate iiop and naming service rebind Message-ID: Hi, We are currently using the Erlang implementation of the CORBA Event Channel, replacing a previously existing one. We have one rather mischevious problem, though, and we can't seem to find our way out. Erlang version is 5.2 We have the following code to start our event channel: ----------------- -module(event_channel). -export([start/0,stop/0,start_orb/0,start_event/0]). start()-> start_orb(), start_event(). stop()-> orber:uninstall(). start_orb() -> orber:configure(iiop_port,1450), % Must use version 1.2 (for corba:add_alternate_iiop_address(...)) orber:configure(giop_version,{1,2}), % Start orber mnesia:start(), corba:orb_init([{domain,"MyDomain"}]), orber:install([node()],[{ifr_storage_type,ram_copies}]), orber:start(), ok. start_event() -> cosEventApp:install(), cosEventApp:start(), % Create event channel Channel = 'oe_CosEventComm_Channel':oe_create([{domain,"MyDomain"},{blocking,false}],[{persistent,true}]), % Add additional iiop address CompleteChannel = corba:add_alternate_iiop_address(Channel, "localhost", 1450), % Register full reference (with both iiop addresses) in naming service Naming = corba:string_to_object("corbaloc::1.0@REDACTED:1235/NameService"), Name = lname:new(["CosEventService"]), 'CosNaming_NamingContext':rebind(Naming,Name,CompleteChannel), CompleteChannel. ----------------- We would like our channel to listen for requests on two specific interfaces: a network card and the loopback (==localhost). Thus can we have robust CORBA connections locally to the machine where the event channel is running. So, we make our event channel listen on all interfaces (default), and we would like to publish (via the naming service) a double reference. With the previously used ORB, this resulted into having two IORs available under the same name in the naming service. With Orber, we could verify that only the network interface is visible in the registered CORBA reference. Our processes, when using the naming service, only get a reference to the network interface :b. When we lose the network, our local applications lose their event channel connection instead of using the local loopback. Apparently, the :rebind does not make the alternate IIOP visible in the IOR submitted to the naming service. Can anyone suggest a fix / workaround / hack for that? Alexandre Betis From nick@REDACTED Thu Jun 12 14:19:21 2003 From: nick@REDACTED (Niclas Eklund) Date: Thu, 12 Jun 2003 14:19:21 +0200 (MEST) Subject: Problem with orber, alternate iiop and naming service rebind In-Reply-To: Message-ID: Hello! I'm affraid that this might cause problems: CompleteChannel = corba:add_alternate_iiop_address(Channel,"localhost",1450), If you create the event channel on hostA, the IOR will contain: MainHost == hostA MainPort == 1450 Adding the alternate-IIOP component using the above returns an IOR which contains: MainHost == hostA MainPort == 1450 AltHost == localhost AltPort == 1450 If the client, located on hostB, cannot connect to MainHost/MainPort, it will try to use AltHost/AltPort. Since AltHost == localhost, which, from the clients point of view, is hostB, this will not work. When resolving the NameService you use a different port-number. Do you intend to store the IOR using another ORB:s NameService? You also use IIOP-version 1.0, which doesn't support alternate-IIOP components (compliant with the OMG specification). Naming = corba:string_to_object("corbaloc::1.0@REDACTED:1235/NameService"), Naturally, it helps if your other ORB also supports TAG_ALTERNATE_IIOP_ADDRESS. ;-) If you want to access the local NameService you should use the following: Naming = corba:resolve_initial_references("NameService"), It would help a lot if you could tell me which versions (cosEvent & Orber) you're using (cosEventApp:module_info(). & orber:module_info().) I cannot recommend that you use any other way to start the event channel besides the ones you find in the cosEvent documentation: %% Start a channel using the default configuration Ch = cosEventApp:start_channel(), %% ... or use configuration parameters. Ch = cosEventApp:start_channel(Options), Options == [{pull_interval, Seconds} | {typecheck, Boolean} | {maxEvents,Integer} |{blocking, Boolean}] Best Regards, Nick > Hi, > > We are currently using the Erlang implementation of the CORBA Event > Channel, replacing a previously existing one. > > We have one rather mischevious problem, though, and we can't seem to > find our way out. Erlang version is 5.2 > > We have the following code to start our event channel: > > ----------------- > > -module(event_channel). > -export([start/0,stop/0,start_orb/0,start_event/0]). > > start()-> > start_orb(), > start_event(). > > stop()-> > orber:uninstall(). > > start_orb() -> > orber:configure(iiop_port,1450), > > % Must use version 1.2 (for corba:add_alternate_iiop_address(...)) > orber:configure(giop_version,{1,2}), > > % Start orber > mnesia:start(), > corba:orb_init([{domain,"MyDomain"}]), > orber:install([node()],[{ifr_storage_type,ram_copies}]), > orber:start(), > ok. > > start_event() -> > cosEventApp:install(), > cosEventApp:start(), > > % Create event channel > Channel = 'oe_CosEventComm_Channel':oe_create([{domain,"MyDomain"},{blocking,false}],[{persistent,true}]), > > % Add additional iiop address > CompleteChannel = corba:add_alternate_iiop_address(Channel, "localhost", 1450), > > % Register full reference (with both iiop addresses) in naming service > Naming = corba:string_to_object("corbaloc::1.0@REDACTED:1235/NameService"), > Name = lname:new(["CosEventService"]), > 'CosNaming_NamingContext':rebind(Naming,Name,CompleteChannel), > > CompleteChannel. > > ----------------- > > We would like our channel to listen for requests on two specific > interfaces: a network card and the loopback (==localhost). Thus can we > have robust CORBA connections locally to the machine where the event > channel is running. > > So, we make our event channel listen on all interfaces (default), and > we would like to publish (via the naming service) a double reference. > With the previously used ORB, this resulted into having two IORs > available under the same name in the naming service. > > With Orber, we could verify that only the network interface is visible > in the registered CORBA reference. Our processes, when using the naming > service, only get a reference to the network interface :b. When we lose > the network, our local applications lose their event channel connection > instead of using the local loopback. > > Apparently, the :rebind does not make the alternate IIOP visible in the > IOR submitted to the naming service. > > Can anyone suggest a fix / workaround / hack for that? > > Alexandre Betis From henkbijker78@REDACTED Thu Jun 12 17:25:47 2003 From: henkbijker78@REDACTED (henkbijker78@REDACTED) Date: Thu, 12 Jun 2003 17:25:47 +0200 Subject: Compiling problems Message-ID: Hi all, I've got a winblowz sybase / erlang interface having trouble compiling c code with visual c++ 6.0. I am using the right libs, but i keep on getting the following unresolved externals. sybasedbi.obj : error LNK2001: unresolved external symbol _erl_free_compound@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_mk_tuple@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_mk_atom@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_print_term@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_cons@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_mk_empty_list@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_tl@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_hd@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_length@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_term_len@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_encode@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_element@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_decode@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _exit@REDACTED sybasedbi.obj : error LNK2001: unresolved external symbol _erl_init_nothreads@REDACTED sybasedbi.exe : fatal error LNK1120: 15 unresolved externals ... and yes, i have to compile with ms vc, cause of the sybase connectivity. Looks familiar to anyone? ... I'm using erl5.1.2, and including ei.lib & erl_interface.lib Thanks, _____________________________ Hendrik Bijker, Cell : 082-255-1630, ICQ : 215097587 ############################################################################ ######### The information contained in this message and or attachments is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any system and destroy and copies. ############################################################################ ######### From serge@REDACTED Thu Jun 12 18:32:18 2003 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 12 Jun 2003 12:32:18 -0400 Subject: Compiling problems References: Message-ID: <3EE8AB12.6060709@hq.idt.net> Try to recompile the erl_interface library using your compiler, include the built ei.lib in your project, and try to build the project. Alternatively, try to turn on the ANSI C compiler compatibility. If this doesn't help, perhaps, try to diable name mangling in the compiler (this is what causes @8 and @4 suffixes in your errors). I am not 100% sure if you can disable this option in the C++ compilation mode, as in C++ this option is directly related to parameter overloading... Regards, Serge -- ================================================================ | Serge Aleynikov Tel: (973) 438-3436 | MIS Telecom Fax: (973) 438-1457 | IDT Corp. serge@REDACTED ================================================================ henkbijker78@REDACTED wrote: > Hi all, > > I've got a winblowz sybase / erlang interface > having trouble compiling c code with visual c++ 6.0. I am using the right > libs, but i keep on getting the following unresolved externals. > > sybasedbi.obj : error LNK2001: unresolved external symbol > _erl_free_compound@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_mk_tuple@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_mk_atom@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_print_term@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_cons@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol > _erl_mk_empty_list@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_tl@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_hd@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_length@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_term_len@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_encode@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_element@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_decode@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol _exit@REDACTED > sybasedbi.obj : error LNK2001: unresolved external symbol > _erl_init_nothreads@REDACTED > sybasedbi.exe : fatal error LNK1120: 15 unresolved externals > > ... and yes, i have to compile with ms vc, cause of the sybase connectivity. > > Looks familiar to anyone? ... > > I'm using erl5.1.2, and including ei.lib & erl_interface.lib > > Thanks, > > _____________________________ > Hendrik Bijker, > Cell : 082-255-1630, > ICQ : 215097587 > > ############################################################################ > ######### > The information contained in this message and or attachments is intended > only for the person or entity to which it is addressed and may contain > confidential and/or privileged material. Any review, retransmission, > dissemination or other use of, or taking of any action in reliance upon, > this information by persons or entities other than the intended recipient > is prohibited. If you received this in error, please contact the sender and > delete the material from any system and destroy and copies. > ############################################################################ > ######### > From taavi@REDACTED Thu Jun 12 19:00:50 2003 From: taavi@REDACTED (Taavi Talvik) Date: Thu, 12 Jun 2003 20:00:50 +0300 (EEST) Subject: Compiling problems In-Reply-To: <3EE8AB12.6060709@hq.idt.net> Message-ID: <20030612195838.H93345-100000@valu.uninet.ee> On Thu, 12 Jun 2003, Serge Aleynikov wrote: Also following may help: extern "C" { your erlang ei_* include files or ei_ function prototypes } best regards, taavi > Try to recompile the erl_interface library using your compiler, include > the built ei.lib in your project, and try to build the project. > > Alternatively, try to turn on the ANSI C compiler compatibility. > > If this doesn't help, perhaps, try to diable name mangling in the > compiler (this is what causes @8 and @4 suffixes in your errors). I am > not 100% sure if you can disable this option in the C++ compilation > mode, as in C++ this option is directly related to parameter overloading... > > Regards, > > Serge > > -- > ================================================================ > | Serge Aleynikov Tel: (973) 438-3436 > | MIS Telecom Fax: (973) 438-1457 > | IDT Corp. serge@REDACTED > ================================================================ > > henkbijker78@REDACTED wrote: > > Hi all, > > > > I've got a winblowz sybase / erlang interface > > having trouble compiling c code with visual c++ 6.0. I am using the right > > libs, but i keep on getting the following unresolved externals. > > > > sybasedbi.obj : error LNK2001: unresolved external symbol > > _erl_free_compound@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_mk_tuple@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_mk_atom@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_print_term@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_cons@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol > > _erl_mk_empty_list@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_tl@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_hd@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_length@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_term_len@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_encode@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_element@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol _erl_decode@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol _exit@REDACTED > > sybasedbi.obj : error LNK2001: unresolved external symbol > > _erl_init_nothreads@REDACTED > > sybasedbi.exe : fatal error LNK1120: 15 unresolved externals > > > > ... and yes, i have to compile with ms vc, cause of the sybase connectivity. > > > > Looks familiar to anyone? ... > > > > I'm using erl5.1.2, and including ei.lib & erl_interface.lib > > > > Thanks, > > > > _____________________________ > > Hendrik Bijker, > > Cell : 082-255-1630, > > ICQ : 215097587 > > > > ############################################################################ > > ######### > > The information contained in this message and or attachments is intended > > only for the person or entity to which it is addressed and may contain > > confidential and/or privileged material. Any review, retransmission, > > dissemination or other use of, or taking of any action in reliance upon, > > this information by persons or entities other than the intended recipient > > is prohibited. If you received this in error, please contact the sender and > > delete the material from any system and destroy and copies. > > ############################################################################ > > ######### > > > > > ----------------------------------------------------------- Taavi Talvik | Internet: taavi@REDACTED AS Uninet | phone: +372 6800013 Parnu mnt. 105 | fax: +372 6800001 Tallinn 11312, Estonia | gsm: +372 56569996 From david.wallin@REDACTED Fri Jun 13 13:25:36 2003 From: david.wallin@REDACTED (david wallin) Date: Fri, 13 Jun 2003 12:25:36 +0100 Subject: crashing the debugger Message-ID: Slightly puzzled. Tried to run a program under the debugger (because it stops unexplainably and intermittently with the beam process taking alot of CPU), and when I try to interpret the main file I get this error message from the debugger: [...] ,105,115,116,95,116,111,95,105,110,116,101,103,101,114,47,49,44,32,83,10 5,122,101,115,41,32,59,10,10,9,95,32,45,62,32,117,110,100,101,102,105,11 0,101,100,10,10,32,32,32,32,101,110,100,46,10,10,10,37,37,37,32,45,45,45 ,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45 ,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45 ,45,45,45,45,45,45,45,45,45,45,45,45,45,45,32,91,116,104,101,32,101,110, 100,93,10,109,0,0,0,16,217,142,202,234,188,223,186,193,177,235,28,143,55 ,242,56,1>>} ** When Server state == {state,11,[],[],false,all,[<0.30.0>]} ** Reason for termination == ** {{badmatch,{function_clause,[{dbg_iload,guard_test,[{atom,98,false}]}, {dbg_iload,and_guard,1}, {dbg_iload,guard,1}, {dbg_iload,clause,1}, {dbg_iload,icr_clauses,1}, {dbg_iload,expr,1}, {dbg_iload,exprs,1}, {dbg_iload,clause,1}| more]}}, [{dbg_iserver,handle_call,3}, {gen_server,handle_msg,6}, {proc_lib,init_p,5}]} any help would be helpful. cheers, --david. From david.wallin@REDACTED Fri Jun 13 14:03:36 2003 From: david.wallin@REDACTED (david wallin) Date: Fri, 13 Jun 2003 13:03:36 +0100 Subject: crashing the debugger In-Reply-To: Message-ID: <0FFC1FF8-9D97-11D7-B92C-000393536F4E@ul.ie> Should perhaps mention that this is with version R9B-1. On Friday, June 13, 2003, at 12:25 PM, david wallin wrote: > > Slightly puzzled. > > Tried to run a program under the debugger (because it stops > unexplainably and intermittently with the beam process taking alot of > CPU), and when I try to interpret the main file I get this error > message from the debugger: > > [...] > ,105,115,116,95,116,111,95,105,110,116,101,103,101,114,47,49,44,32,83,1 > 05,122,101,115,41,32,59,10,10,9,95,32,45,62,32,117,110,100,101,102,105, > 110,101,100,10,10,32,32,32,32,101,110,100,46,10,10,10,37,37,37,32,45,45 > ,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,4 > 5,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45, > 45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,32,91,116,104,101,32,10 > 1,110,100,93,10,109,0,0,0,16,217,142,202,234,188,223,186,193,177,235,28 > ,143,55,242,56,1>>} > ** When Server state == {state,11,[],[],false,all,[<0.30.0>]} > ** Reason for termination == > ** > {{badmatch,{function_clause,[{dbg_iload,guard_test,[{atom,98,false}]}, > {dbg_iload,and_guard,1}, > {dbg_iload,guard,1}, > {dbg_iload,clause,1}, > {dbg_iload,icr_clauses,1}, > {dbg_iload,expr,1}, > {dbg_iload,exprs,1}, > {dbg_iload,clause,1}| > more]}}, > [{dbg_iserver,handle_call,3}, > {gen_server,handle_msg,6}, > {proc_lib,init_p,5}]} > > > any help would be helpful. > > cheers, > > --david. > --david. From serge@REDACTED Fri Jun 13 15:46:39 2003 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 13 Jun 2003 09:46:39 -0400 Subject: Compiling problems References: <3EE8AB12.6060709@hq.idt.net> Message-ID: <3EE9D5BF.7080900@hq.idt.net> I was using Borland's C++ compiler to compile the library. Please look at my previous posts that indicated the changes required to get it working. Perhaps you would need to do something similar in MS C++. Serge henkbijker78@REDACTED wrote: >>Try to recompile the erl_interface library using your compiler, include >>the built ei.lib in your project, and try to build the project. > > > I'm really having trouble with the c part of the program, so .... > aby pointers on how to recompile the library? (ei, and erl_interface) > ... also, when i call erl_init(NULL, 0), [compiled with /force:unresolved] > i get a application error: > "The instruction at xxx referenced memory at xxx. The memory could not be > written.", > I assume this also has to do with re-compile of the libs? (Althoug erl_init > is not unresolved)? > > >>Alternatively, try to turn on the ANSI C compiler compatibility. >> >>If this doesn't help, perhaps, try to diable name mangling in the >>compiler (this is what causes @8 and @4 suffixes in your errors). I am >>not 100% sure if you can disable this option in the C++ compilation >>mode, as in C++ this option is directly related to parameter > > overloading... > >>Regards, >> >>Serge >> >>-- >>================================================================ >>| Serge Aleynikov Tel: (973) 438-3436 >>| MIS Telecom Fax: (973) 438-1457 >>| IDT Corp. serge@REDACTED >>================================================================ >> >>henkbijker78@REDACTED wrote: >> >>>Hi all, >>> >>>I've got a winblowz sybase / erlang interface >>>having trouble compiling c code with visual c++ 6.0. I am using the >> > right > >>>libs, but i keep on getting the following unresolved externals. >>> >>>sybasedbi.obj : error LNK2001: unresolved external symbol >>>_erl_free_compound@REDACTED >>>sybasedbi.obj : error LNK2001: unresolved external symbol >> > _erl_mk_tuple@REDACTED > >>>sybasedbi.obj : error LNK2001: unresolved external symbol _erl_mk_atom@REDACTED >>>sybasedbi.obj : error LNK2001: unresolved external symbol >> > _erl_print_term@REDACTED > >>>sybasedbi.obj : error LNK2001: unresolved external symbol _erl_cons@REDACTED >>>sybasedbi.obj : error LNK2001: unresolved external symbol >>>_erl_mk_empty_list@REDACTED >>>sybasedbi.obj : error LNK2001: unresolved external symbol _erl_tl@REDACTED >>>sybasedbi.obj : error LNK2001: unresolved external symbol _erl_hd@REDACTED >>>sybasedbi.obj : error LNK2001: unresolved external symbol _erl_length@REDACTED >>>sybasedbi.obj : error LNK2001: unresolved external symbol >> > _erl_term_len@REDACTED > >>>sybasedbi.obj : error LNK2001: unresolved external symbol _erl_encode@REDACTED >>>sybasedbi.obj : error LNK2001: unresolved external symbol _erl_element@REDACTED >>>sybasedbi.obj : error LNK2001: unresolved external symbol _erl_decode@REDACTED >>>sybasedbi.obj : error LNK2001: unresolved external symbol _exit@REDACTED >>>sybasedbi.obj : error LNK2001: unresolved external symbol >>>_erl_init_nothreads@REDACTED >>>sybasedbi.exe : fatal error LNK1120: 15 unresolved externals >>> >>>... and yes, i have to compile with ms vc, cause of the sybase >> > connectivity. > >>>Looks familiar to anyone? ... >>> >>>I'm using erl5.1.2, and including ei.lib & erl_interface.lib >>> >>>Thanks, >>> >>>_____________________________ >>>Hendrik Bijker, >>>Cell : 082-255-1630, >>>ICQ : 215097587 >>> >>> >> > ############################################################################ > >>>######### >>>The information contained in this message and or attachments is intended >>>only for the person or entity to which it is addressed and may contain >>>confidential and/or privileged material. Any review, retransmission, >>>dissemination or other use of, or taking of any action in reliance upon, >>>this information by persons or entities other than the intended >> > recipient > >>>is prohibited. If you received this in error, please contact the sender >> > and > >>>delete the material from any system and destroy and copies. >>> >> > ############################################################################ > >>>######### >>> >> >> >> > From erlang@REDACTED Sat Jun 14 02:45:33 2003 From: erlang@REDACTED (erlang@REDACTED) Date: Fri, 13 Jun 2003 20:45:33 -0400 (EDT) Subject: More restricted execution silly ideas Message-ID: I had another thought on implementing a restricted execution environment for untrusted erlang code: How about, instead of trying to lock down an erlang virtual machine, instead putting the erlang code presented through a cleanser which pulls out any functions which are regarded as unsafe? Code which doesn't refer to anything unwanted passes through and is executed without a murmur. The rest is hacked up, probably won't even compile, and is left to bleed, or returned without running? Or simply specify a limited set of functionality, and inspect incoming code for violations? I suspect this might be easier than hacking up OTP and erlang. From spearce@REDACTED Sat Jun 14 02:13:33 2003 From: spearce@REDACTED (Shawn Pearce) Date: Fri, 13 Jun 2003 20:13:33 -0400 Subject: More restricted execution silly ideas In-Reply-To: References: Message-ID: <20030614001333.GA7888@spearce.org> I think this is kind of the direction I had been suggesting earlier. I thought about it more last night. If it was possible to read a .beam file in Erlang, perhaps we could do something like this: - Designate a directory path as "trusted" to the code server. - Any path not desginated is assumed untrusted. - Within a "trusted" directory path desginate a set of "safe" modules. All others are "restricted". When a module is loaded, if it comes from a "trusted" directory, the code server loads it as is. If the module is from an untrusted directory, the code server rips apart the .beam file and begins to "verify" it: - Remove all native HiPE code. - Any simple call (e.g. code was: lists:length(X)) where module and function were statically known at compile time: - Load module lists now (if not loaded). - Check type of directory it came from: - Trusted: - If a "safe" module, leave call alone. - If not "safe", module call must be 'fixed': rewrite call to be: SM = get(security_manager), SM:apply3(lists, length, [Arg]) - Untrusted: - Leave alone. - Any non-simple function call (those we can't determine either function or module name now, or that uses apply/3): - Rewrite call to be: SM = get(security_manager), SM:apply3(lists, length, [Arg]) - Any message send instruction: SM = get(security_manager), SM:send_message(To, Data) - Any unsafe BIF/erlang module calls: (put, spawn, open_port, etc.) SM = get(security_manager), SM:erlang_bif(BifName, [Arg]) Basically we allow 'system' code to run without penalty, and allow user 'unsafe' code to run with minimal penalty, basically anytime they want to access external resources. Basic calls to stdlib such as lists, sets, dict are all allowed as-is. Its ets, file, code, message sending, modifying the process dictionary, etc that are slower. By putting all calls through a security_manager module, the security_manager can decide how a call should react, and can either emulate the real call, or check access (based on arguments, etc) and forward or exit/1 as necessary. I thought I might dig around this weekend and see if I can find some of the old messages where the .beam file was discussed to see how practical this would really be. erlang@REDACTED wrote: > I had another thought on implementing a restricted execution environment for > untrusted erlang code: > > How about, instead of trying to lock down an erlang virtual machine, instead > putting the erlang code presented through a cleanser which pulls out any > functions which are regarded as unsafe? > > Code which doesn't refer to anything unwanted passes through and is executed > without a murmur. The rest is hacked up, probably won't even compile, and > is left to bleed, or returned without running? > > Or simply specify a limited set of functionality, and inspect incoming code > for violations? > > I suspect this might be easier than hacking up OTP and erlang. -- Shawn. You can't have your cake and let your neighbor eat it too. -- Ayn Rand From valentin@REDACTED Sat Jun 14 02:17:12 2003 From: valentin@REDACTED (Valentin) Date: Sat, 14 Jun 2003 02:17:12 +0200 Subject: Erlang Core dump References: <20030612195838.H93345-100000@valu.uninet.ee> Message-ID: <001701c3320a$4f39ddc0$15ee1ec4@moneymaker> Hi Any ideas what can cause segmentation violation (causing a core dump) in ERTS (Erlang Release 8)? We do have an Erlang production system that's running without a problem (other than usual human factor) for the last 8 months, and I'm puzzled with this problem that occurred on our test system -- we are busy testing an application that provides a consolidation between multiple SCCP and TCAP layers... anyway, the SCCP and TCAP drivers are passing binary PDUs and some control information through ERLANG run-time system, and I was wondering if there is a connection between garbage collection of the binary data and this crash? Would it make any difference if I change from binary data to list? Would there be a performance penalty if I stop using: int ei_x_encode_binary(ei_x_buff* x, const void *p, long len) and change to: int ei_x_encode_string_len(ei_x_buff* x, const char* s, int len) Note that binary data is not interpreted by Erlang in any way but just forwarded between layers. Any takers? Thanks in advance. Valentin. From erlang@REDACTED Sat Jun 14 04:39:48 2003 From: erlang@REDACTED (erlang@REDACTED) Date: Fri, 13 Jun 2003 22:39:48 -0400 (EDT) Subject: More restricted execution silly ideas In-Reply-To: <20030614001333.GA7888@spearce.org> from "Shawn Pearce" at Jun 13, 2003 08:13:33 PM Message-ID: > I think this is kind of the direction I had been suggesting earlier. [snip bit about defining trusted module directory] > When a module is loaded, if it comes from a "trusted" directory, > the code server loads it as is. How about if we just allow OTP contents, and then cherrypick from those which actual calls we will permit? > If the module is from an untrusted directory, the code server > rips apart the .beam file and begins to "verify" it: [snip approval algorithm] > Basically we allow 'system' code to run without penalty, and allow user > 'unsafe' code to run with minimal penalty, basically anytime they want > to access external resources. Basic calls to stdlib such as lists, > sets, dict are all allowed as-is. Its ets, file, code, message sending, > modifying the process dictionary, etc that are slower. By putting all > calls through a security_manager module, the security_manager can > decide how a call should react, and can either emulate the real call, > or check access (based on arguments, etc) and forward or exit/1 as > necessary. This seems a bit overhead-rich. The idea (to me, at least) is that if one can simply define a standard subset of standard functions which can be permitted (typically, aside from message passing and spawn/3, anything which has particular side-effects with reference to machine state), then this subset can be permitted without any interference at runtime, which is good for things like realtime behaviour. Ultimately, the execution becomes more functional, in the sense of isolation from side effects, until the program hands back the answers in the form of messages. In addition, I'm unsure why analysis of .beam files would be really preferable to analysis of .erl files ... If one were to simply slurp through standard erlang/OTP source code, and ruthlessly mutilate anything with outside references beyond, say, messages, then one could pretty much guarantee that what happens inside the virtual machine is the concern of that program and its spawn only (load on available resources notwithstanding). From spearce@REDACTED Sat Jun 14 19:07:29 2003 From: spearce@REDACTED (Shawn Pearce) Date: Sat, 14 Jun 2003 13:07:29 -0400 Subject: More restricted execution silly ideas In-Reply-To: References: <20030614001333.GA7888@spearce.org> Message-ID: <20030614170729.GA9746@spearce.org> erlang@REDACTED wrote: > > I think this is kind of the direction I had been suggesting earlier. > [snip bit about defining trusted module directory] > > When a module is loaded, if it comes from a "trusted" directory, > > the code server loads it as is. > > How about if we just allow OTP contents, and then cherrypick from those > which actual calls we will permit? I thought about that one, but then you are constrainted to providing only some features of OTP which are considered safe, and what do you do with the rest? Ignore it? What if I chose to offer most of mnesia (the user functions, mnesia:transaction, mnesia:match_object), but not the dets or ets module? I thought about this case last night while grilling dinner, and realized that a .erl + parse_transform -> safe_.beam is the only way to go and still save performance. The "administrator" can define what modules and functions are trusted to be called from untrusted code, what should go through a security_manager module (as I've described), and what should be flagged as illegal operations at compile time, preventing compile from occuring. > > If the module is from an untrusted directory, the code server > > rips apart the .beam file and begins to "verify" it: > [snip approval algorithm] > > Basically we allow 'system' code to run without penalty, and allow user > > 'unsafe' code to run with minimal penalty, basically anytime they want > > to access external resources. Basic calls to stdlib such as lists, > > sets, dict are all allowed as-is. Its ets, file, code, message sending, > > modifying the process dictionary, etc that are slower. By putting all > > calls through a security_manager module, the security_manager can > > decide how a call should react, and can either emulate the real call, > > or check access (based on arguments, etc) and forward or exit/1 as > > necessary. > > This seems a bit overhead-rich. The idea (to me, at least) is that if > one can simply define a standard subset of standard functions which can > be permitted (typically, aside from message passing and spawn/3, anything > which has particular side-effects with reference to machine state), then > this subset can be permitted without any interference at runtime, which is > good for things like realtime behaviour. Ultimately, the execution becomes > more functional, in the sense of isolation from side effects, until the > program hands back the answers in the form of messages. I think the end result allows overhead only where necessary. What if I want/need to offer the ability to read/write files in directory A, but disallow all access in any other directory? There must be a runtime check, and adding that check to file:open and dets:open through a dynamic security_manager module would be the way to do it. But the rest of the dets: routines could be allowed to stay exactly as-is. There is a concern that dynamic module calls have to always go through the security manager, and this will slow down the call. Is M:F(A,B) today faster than apply(M, F, [A,B])? Because if so, I was going to have the security_manager module implement something like: call(M,F) -> M:F(). call(M,F,A) -> M:F(A). call(M,F,A,B) -> M:F(A,B). call(M,F,A,B,C) -> M:F(A,B,C). call(M,F,A,B,C,D) -> M:F(A,B,C,D). callv(M,F,ArgList) -> apply(M, F, ArgList). So its not as ugly. I contend you won't get safe execution without losing some performance in some areas of the application, but you can still have the security_manager provide a soft-realtime bound on the overhead it adds. > In addition, I'm unsure why analysis of .beam files would be really preferable > to analysis of .erl files ... The only advantage of a .beam file is I can receive code from over the network which I did not compile locally. But given that Erlang/OTP is usually only sent in source form between parties, doing analysis of .erl files is acceptable. > If one were to simply slurp through standard erlang/OTP source code, and > ruthlessly mutilate anything with outside references beyond, say, messages, > then one could pretty much guarantee that what happens inside the virtual > machine is the concern of that program and its spawn only (load on available > resources notwithstanding). I'd rather not get into rebuilding OTP to be safe. OTP is big, and most likely we'll never be able to keep up with the entire erlang kernel, stdlib, plus OTP and all standard applications. Not to mention that gutting them may remove critical functionality. What if I need to use ets tables for performance, as they are used only as lookup tables by my "untrusted" code, while other code which is trusted can send messages to the ets table owner to update the table? -- Shawn. Nondeterminism means never having to say you are wrong. From hal@REDACTED Sat Jun 14 20:17:56 2003 From: hal@REDACTED (Hal Snyder) Date: Sat, 14 Jun 2003 13:17:56 -0500 Subject: Erlounge Schaumburg, XML vs UBF In-Reply-To: <87n0gwbenp.fsf@ghidra.vail> (Hal Snyder's message of "Thu, 05 Jun 2003 12:46:50 -0500") References: <87n0gwbenp.fsf@ghidra.vail> Message-ID: <87fzmch6az.fsf@ghidra.vail> Thanks to all who made the first Erlounge Schaumburg a great success. For the benefit of those who could not make it, the XML-vs-UBF debate is summarized: XML Disadvantages 1. Verbose. 2. Problem with whitespace in elements. 3. Grammar not completely context free. 4. No consistent way of using elements vs. attributes. 5. It's not beer. XML Advantages 1. People agreed on it. 2. Simple, text-based. 3. Easy to read (at least the small samples shown in tutorials). 4. Intuitively looks like HTML. 5. Lots of toolsets, like XSLT. 6. Self defining. UBF Disadvantages 1. None of us fully understands it yet (Joe wasn't there). 2. Unproven. 3. Not finished. 4. S-expression closing tag problem. UBF Advantages 1. Concise. 2. Easy to parse. 3. Direct attack on XML's most annoying drawbacks. 4. Language independent type schema. 5. Contract checker is intriguing. 6. Proof of concept is coded in Erlang. Attendees gravely considered the above and found unanimously that a) UBF is much better than XML, and b) more study is needed. From flaig@REDACTED Sun Jun 15 12:01:38 2003 From: flaig@REDACTED (Dr.Dr.Ruediger M.Flaig) Date: Sun, 15 Jun 2003 03:01:38 -0700 (PDT) Subject: The reason WHY OO sucks Message-ID: <20030615100138.86DF84486@sitemail.everyone.net> Dear Joe, dear Erlangers, let me return once more to the "Eppur si sugat" thread. A number of things have recently moved me to sum up a few of my thoughts about OO vs. FP, which I would like to share with you. What do you think about this? ** In OO, it is often pretty difficult to decide where to attach a method which is not tied to an object by logical necessity; and when there is a pyramid of objects always appearing together, shall we attach a method to the higher components (wool = shepherd.shear( shepherd.sheep[ n ] )) or to the lower ones (wool = shepherd.sheep[ n ].shear(), or rather wool = shepherd.sheep[ n ].get_shorn())? Does the shaving method logically belong to the sheep (get_shorn) or to the shepherd (shear)? For example, the proper placement of a ``dice'' wrapper function built around the system random generator is a matter of creed. From outsourcing it into a different module to implementing it separately in every object type, all things are possible and equally justificable. The textbook example for OO programming -- a presentation or graphics program where a ``redraw yourself now'' message is sent to all the objects in memory upon receipt of a window event (for graphical_object in object_list: graphical_object.redraw()) -- emphasizes the advantages of the latter, but it is of fallacious simplicity. Actions which affect one shred of data only, like the redraw message, are not that frequent; most things happening inside a program establish some kind of relation between previously unconnected items of data. Moreover, most real-life agglomerates of data are less clearly discriminable than a simple list or array of graphical objects. Apart from the problem with shear, does it make sense to place the sheep array directly in shepherd, or is it preferable to create an intermediate level shepherd.flock? Are there any attributes, modofications or accidentia which belong to the flock as a whole without affecting the individual sheep? In the Middle Ages, half a millenium of philosophical debates was wasted on this question, and the solution agreed upon in the end (Abaelard's) bears all the hallmarks of a silly compromise. Human thinking is based upon the concept of an action flowing from a subject to an object; ever since Kant's times, the phrase ``The sun warms the stone'' has been used as the favourite example for this, and it was Schopenhauer who traced this back to the dependence of human thinking on causal connections: Observation 1 -- the sun shines upon the stone; observation 2 -- the stone gets warmer; causal connection -- the stone gets warmer because the sun shines upon it; (re)construction of an action -- the sun warms the stone. This notion is therefore so dominant that even statements where no action is described are formed by projecting the content upon auxiliary verbs such as ``to have'' or ``to be'': Remus erat vir fortissimus, Remus was a hero -- the Latin version shows that a ``nominative object'' is needed for this projection. Still the ``active'' nature of the statement is there, and the thing may even be put into the passive voice, odd though that appears -- the normal Sanskrit form for expressing the past is by means of a passive participle: Ramena bhuto virah, literally ``By Ramah [was] performed the hero-being''. OO programming, however, does not fully acknowledge this. In OO, the basic structure is reflexive, as in graphical_object.redraw(). It is the subject of the action whose state may be changed by the invocation of a method, not the optional object which may be passed as a parameter to the method! Thus, the common x.y(z) formula does not mean ``x does y to z'' but ``x is modified by y'ing z''. This must be kept in mind when dealing with OO. Personally, I prefer a structure which emphasizes the predicate, such as wool = map( shear, shepherd.sheep ) and thus keeps apart things and actions. (Haskell type declarations were chosen to represent currying, but they also indicate the logical direction, even though Haskell rejects the concept of action: shear::Animal->Product and tend::[Animal]->(Animal->Product)->[Product].) Using this approach, shear may be passed as a functional parameter without the need for a subject. Who cares about the shepherd? Let him go to the pub as long as somebody will shear his flock for him. His state is not altered by the action (as he probably won't shear himself), so there is no need for him to be involved. It's the wool we are interested in. In general, data should be considered as an attachment to code, not vice versa. Though being an OO language, Python partly takes this into account by ``defining'' variables only by assignment. In this respect, it is very different from -- and superior to -- C++ and Java. ** Maybe this will be good as a bit of "selling ammunition", or at least to stimulate a few discussions... Lookign forward to hearing from you, Ruediger Marcus Flaig Am Dienstag, 3. Juni 2003 23:19 schrieben Sie: > Hi Ruediger, > > Nice mail, I like the bit about orks ... :-) > > My latin/Italian is non-existent - I wanted > to say "and yet it still sucketh" ... > > I entirely agree re: java C++. IMHO the *important* thing is > total isolation between "things" and "pure message passing" - hence Erlang. > > The language for sequential computations is irrelevant. > > I build systems from "communicating black boxes". COPL etc *is* the point > only achieved in languages like Erlang/Oz/Occam/PLITS > > cheers > > /Joe > > On Thu, 29 May 2003, Dr.Dr.Ruediger M.Flaig wrote: > > Hi all, > > > > from my bit of Italian, I think it ought to be "Eppur suga" -- the Latin > > -t is dropped in the 3rd person singular AFAIK, and it is certainly not > > reflexive... it does not suck itself, that were recursion. ;-))) > > > > Now for something more serious... I think the basic sentiment we all > > share is that Java sucks, and that C++ sucks. They do, and very badly > > too, and it is annoying to see that they have become a kind of de facto > > standard. > > > > I have used both (more than Erlang, I am afraid -- as a molecular > > biologist, I have to deal with megabyte arrays, and that is not quite > > what Erlang was designed for -- sinner repent -- well Ocaml is quite nice > > too ;-) ), and I am equally fed up with the cobolesque boilerplate of > > "private final static paralyzed karl.otto.gustav emil = > > (karl.otto.gustav) ((egon.kasimir) foo.bar().blah[anything])" and the > > orkish scrawls of > > "#define n(x,i,j) {*++(*x)->y %= (i>j)?&i--:--&j}", not to mention > > memory leaks and, and, and... Maybe OO itself does not suck but a concept > > without implementation is void. > > > > But I think we are missing the point. The important thing is not that OO > > is an inferior concept but that FP (or COPL, if you please) is a superior > > one. Erlang were still great even Stroustrup and Gosling had never come > > near a computer -- because it enables you to be productive, to write > > clear and concise code, to cope with errors and, last but not least, > > because it allows reasoning. > > > > This latest thing is what I have really been missing. Erlang is a > > functional language, so it should be possible to employ automatic theorem > > provers and all that kind of stuff. Cummings wrote about ML: "We are not > > trying to develop 'safer' programs by testing, but developing SAFE > > programs by reasoning." This is a very philosophical matter. Let me say > > that Java and C++ rely on induction, Erlang allows deduction, and Sir > > Karl Popper has said the rest about that, decades ago. So are there any > > theorem prover > > s for Erlang, as there are for Lisp and ML? Can we prove a program's > correctness by reasoning? If not, let's do that! > > > The other point is that the power of Erlang's concurrency is often > > underestimated by folks who are not interested in parallelism itself. I > > am a complete layman to that field but I think that neuronal networks > > would be a great thing do with Erlang. Has anybody tried this before? Are > > there any projects in similar fields? Personally, I was attracted to > > Erlang because I felt that simulation of complex biological processes > > would be easy to implement. Imagine you have a microbe with some 4000 > > genes, each with > > a state of activity and each influencing a couple of others. In Erlang, > this would be straightforward... I met a guy who did the like and spend a > year on that programming the basic mechanism in Java! So I think we could > get away from that "Erlang? Oh, that's for telephones only" image. > > > Has anybody worked on this yet? > > > > > > Sk??l, > > Ruediger Marcus Flaig > > > > > > > > > > Dr. Dr. Ruediger Marcus Flaig Institute for Immunology University of Heidelberg Im Neuenheimer Feld 305 D-69120 Heidelberg Tel. +49-172-7652946 | +49-6221-56-5402 | +49-6221-432963 _____________________________________________________________ Free eMail .... the way it should be.... http://www.hablas.com _____________________________________________________________ Select your own custom email address for FREE! Get you@REDACTED, No Ads, 6MB, IMAP, POP, SMTP & more! http://www.everyone.net/selectmail?campaign=tag From ulf.wiger@REDACTED Sun Jun 15 12:50:10 2003 From: ulf.wiger@REDACTED (Wiger Ulf) Date: Sun, 15 Jun 2003 12:50:10 +0200 Subject: The reason WHY OO sucks References: <20030615100138.86DF84486@sitemail.everyone.net> Message-ID: <000501c3332b$e507a680$fd7a40d5@telia.com> On June 15 2003, "Dr.Dr.Ruediger M.Flaig" wrote: >In OO, it is often pretty difficult to decide where to attach a >method which is not tied to an object by logical necessity; >and when there is a pyramid of objects always appearing >together, shall we attach a method to the higher components >(wool = shepherd.shear( shepherd.sheep[ n ] )) or to the lower >ones (wool = shepherd.sheep[ n ].shear(), or rather >wool = shepherd.sheep[ n ].get_shorn())? Does the shaving >method logically belong to the sheep (get_shorn) or to the >shepherd (shear)? This is indeed one of the bigger problems I've seen in large OO projects: where design rules have not been sufficiently detailed, different teams tend to implement methods at different levels in the class hierarchy, and it eventually becomes very difficult to reuse code or adapt to new features. /Uffe From erlang@REDACTED Sun Jun 15 13:41:37 2003 From: erlang@REDACTED (Peter-Henry Mander) Date: Sun, 15 Jun 2003 12:41:37 +0100 Subject: The reason WHY OO sucks References: <20030615100138.86DF84486@sitemail.everyone.net> <000501c3332b$e507a680$fd7a40d5@telia.com> Message-ID: <3EEC5B71.2020805@manderp.freeserve.co.uk> Hi guys, At least I'm not the only one working on a Sunday, I don't feel so isolated! (-: I can only voice my support for this being an excellent example of OO design issues that cause huge fisticuffs among programmers. It gets worse as you try to create abstract patterns of such. Maybe someone will suggest using the Visitor pattern, where the "ShearWool" object visits all "Sheep" instances in the "Flock" object in turn accumulating the "Wool" instances from each. It is counterintuitive though, since each "Sheep" needs to call the "visit" method on "ShearWool" passing a reference to self and I don't think that reflects reality (self shearing sheep!) Besides, "Shepherd" instance would have a hard time justifying its existance, especially being a drunk too. It disassociates the classes fairly nicely though, now the shepherd can delegate shearing to his "Dog", sorry, "Visitor" instance and get irresponsibly pissed at the local brewer, but it does require the designer to know the pattern, and anyone reading the code too. Pete. Wiger Ulf wrote: > On June 15 2003, "Dr.Dr.Ruediger M.Flaig" wrote: > > >>In OO, it is often pretty difficult to decide where to attach a >>method which is not tied to an object by logical necessity; >>and when there is a pyramid of objects always appearing >>together, shall we attach a method to the higher components >>(wool = shepherd.shear( shepherd.sheep[ n ] )) or to the lower >>ones (wool = shepherd.sheep[ n ].shear(), or rather >>wool = shepherd.sheep[ n ].get_shorn())? Does the shaving >>method logically belong to the sheep (get_shorn) or to the >>shepherd (shear)? > > > This is indeed one of the bigger problems I've seen in large OO > projects: where design rules have not been sufficiently detailed, > different teams tend to implement methods at different levels > in the class hierarchy, and it eventually becomes very difficult to > reuse code or adapt to new features. > > /Uffe > > From luke@REDACTED Sun Jun 15 13:41:46 2003 From: luke@REDACTED (Luke Gorrie) Date: 15 Jun 2003 13:41:46 +0200 Subject: Erlounge Schaumburg, XML vs UBF In-Reply-To: <87fzmch6az.fsf@ghidra.vail> References: <87n0gwbenp.fsf@ghidra.vail> <87fzmch6az.fsf@ghidra.vail> Message-ID: Hal Snyder writes: > UBF Disadvantages > > 2. Unproven. Surely at least UBF(A) _is_ proven, since it's just term_to_binary in disguise (and term_to_binary is wonderful!) > 4. S-expression closing tag problem. What's that? Cheers, Luke From luke@REDACTED Sun Jun 15 13:47:18 2003 From: luke@REDACTED (Luke Gorrie) Date: 15 Jun 2003 13:47:18 +0200 Subject: The reason WHY OO sucks In-Reply-To: <20030615100138.86DF84486@sitemail.everyone.net> References: <20030615100138.86DF84486@sitemail.everyone.net> Message-ID: See also http://www.elsewhere.org/cgi-bin/postmodern/ Cheers, Luke *ducking!!* "Dr.Dr.Ruediger M.Flaig" writes: > Dear Joe, > dear Erlangers, > > let me return once more to the "Eppur si sugat" thread. A number of > things have recently moved me to sum up a few of my thoughts about > OO vs. FP, which I would like to share with you. What do you think > about this? > > > ** In OO, it is often pretty difficult to decide where to attach a > method which is not tied to an object by logical necessity; and when > there is a pyramid of objects always appearing together, shall we > attach a method to the higher components (wool = shepherd.shear( > shepherd.sheep[ n ] )) or to the lower ones (wool = shepherd.sheep[ > n ].shear(), or rather wool = shepherd.sheep[ n ].get_shorn())? Does > the shaving method logically belong to the sheep (get_shorn) or to > the shepherd (shear)? > > For example, the proper placement of a ``dice'' wrapper function > built around the system random generator is a matter of creed. From > outsourcing it into a different module to implementing it separately > in every object type, all things are possible and equally > justificable. > > The textbook example for OO programming -- a presentation or > graphics program where a ``redraw yourself now'' message is sent to > all the objects in memory upon receipt of a window event (for > graphical_object in object_list: graphical_object.redraw()) -- > emphasizes the advantages of the latter, but it is of fallacious > simplicity. Actions which affect one shred of data only, like the > redraw message, are not that frequent; most things happening inside > a program establish some kind of relation between previously > unconnected items of data. > > Moreover, most real-life agglomerates of data are less clearly > discriminable than a simple list or array of graphical > objects. Apart from the problem with shear, does it make sense to > place the sheep array directly in shepherd, or is it preferable to > create an intermediate level shepherd.flock? Are there any > attributes, modofications or accidentia which belong to the flock as > a whole without affecting the individual sheep? In the Middle Ages, > half a millenium of philosophical debates was wasted on this > question, and the solution agreed upon in the end (Abaelard's) bears > all the hallmarks of a silly compromise. > > Human thinking is based upon the concept of an action flowing from a > subject to an object; ever since Kant's times, the phrase ``The sun > warms the stone'' has been used as the favourite example for this, > and it was Schopenhauer who traced this back to the dependence of > human thinking on causal connections: Observation 1 -- the sun > shines upon the stone; observation 2 -- the stone gets warmer; > causal connection -- the stone gets warmer because the sun shines > upon it; (re)construction of an action -- the sun warms the stone. > > This notion is therefore so dominant that even statements where no > action is described are formed by projecting the content upon > auxiliary verbs such as ``to have'' or ``to be'': Remus erat vir > fortissimus, Remus was a hero -- the Latin version shows that a > ``nominative object'' is needed for this projection. Still the > ``active'' nature of the statement is there, and the thing may even > be put into the passive voice, odd though that appears -- the normal > Sanskrit form for expressing the past is by means of a passive > participle: Ramena bhuto virah, literally ``By Ramah [was] performed > the hero-being''. > > OO programming, however, does not fully acknowledge this. In OO, the > basic structure is reflexive, as in graphical_object.redraw(). It is > the subject of the action whose state may be changed by the > invocation of a method, not the optional object which may be passed > as a parameter to the method! Thus, the common x.y(z) formula does > not mean ``x does y to z'' but ``x is modified by y'ing z''. This > must be kept in mind when dealing with OO. > > Personally, I prefer a structure which emphasizes the predicate, > such as wool = map( shear, shepherd.sheep ) and thus keeps apart > things and actions. (Haskell type declarations were chosen to > represent currying, but they also indicate the logical direction, > even though Haskell rejects the concept of action: > shear::Animal->Product and > tend::[Animal]->(Animal->Product)->[Product].) Using this approach, > shear may be passed as a functional parameter without the need for a > subject. Who cares about the shepherd? Let him go to the pub as long > as somebody will shear his flock for him. His state is not altered > by the action (as he probably won't shear himself), so there is no > need for him to be involved. It's the wool we are interested in. > > In general, data should be considered as an attachment to code, not > vice versa. Though being an OO language, Python partly takes this > into account by ``defining'' variables only by assignment. In this > respect, it is very different from -- and superior to -- C++ and > Java. ** > > Maybe this will be good as a bit of "selling ammunition", or at > least to stimulate a few discussions... > > Lookign forward to hearing from you, > Ruediger Marcus Flaig From hal@REDACTED Sun Jun 15 14:49:55 2003 From: hal@REDACTED (Hal Snyder) Date: Sun, 15 Jun 2003 07:49:55 -0500 Subject: Erlounge Schaumburg, XML vs UBF In-Reply-To: (Luke Gorrie's message of "15 Jun 2003 13:41:46 +0200") References: <87n0gwbenp.fsf@ghidra.vail> <87fzmch6az.fsf@ghidra.vail> Message-ID: <87fzmbv72k.fsf@ghidra.vail> Luke Gorrie writes: > Hal Snyder writes: > >> UBF Disadvantages >> >> 2. Unproven. > Surely at least UBF(A) _is_ proven, since it's just term_to_binary > in disguise (and term_to_binary is wonderful!) I worded that item poorly. UBF looks very promising. We are mulling over ideas for a couple pilot projects. But, because we haven't used it before, it's possible there are some gotchas. >> 4. S-expression closing tag problem. Just the idea that you can lose count of parens at the end. A couple links in passing, found while looking into background material for UBF: XML is not S-Expressions http://www.prescod.net/xml/sexprs.html Xml Sucks http://c2.com/cgi/wiki/wiki?XmlSucks From joachim.durchholz@REDACTED Mon Jun 16 14:47:55 2003 From: joachim.durchholz@REDACTED (Joachim Durchholz) Date: Mon, 16 Jun 2003 14:47:55 +0200 Subject: Functions in data structures Message-ID: <3EEDBC7B.9060807@web.de> Hi all, I've been toying with Erlang, preparing for a Real-World(TM) project, and such a state provokes a specific kind of questions. Mine are related to functions in data structures (I gather that Erlangese for "data structure" is "term" - is this correct?) Assume that a term contains a function. Further assume that this term is converted to binary, transmitted to a different machine, and converted back to an Erlang term; or assume it is stored in Mnesia and later read back into Erlang. The base line is: the term is taken out of the Erlang run-time system and later reconstructed. The question is whether the function can be called after reconstructing the term, under various circumstances: 1) Does it work if the function is a named function? 2) Does it work if the function is a named function, but the term is restored in an environment that doesn't contain that function? (E.g. the term is sent over the network and reconstructed in an Erlang run-time that doesn't have the same sources.) 3) Does it work if the function is anonymous? 4) Does it work if the function was constructed at run-time? (Most functional languages can construct functions at run-time by taking a given function and filling in a parameter with either a term or another function. I'm not sure that this is possible in Erlang, but I sincerely hope so...) Regards, Jo From etxuwig@REDACTED Mon Jun 16 15:09:59 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Mon, 16 Jun 2003 15:09:59 +0200 (MEST) Subject: Functions in data structures In-Reply-To: <3EEDBC7B.9060807@web.de> Message-ID: On Mon, 16 Jun 2003, Joachim Durchholz wrote: >Hi all, > >I've been toying with Erlang, preparing for a >Real-World(TM) project, and such a state provokes a >specific kind of questions. > >Mine are related to functions in data structures (I gather >that Erlangese for "data structure" is "term" - is this >correct?) > >Assume that a term contains a function. Further assume that >this term is converted to binary, transmitted to a >different machine, and converted back to an Erlang term; or >assume it is stored in Mnesia and later read back into >Erlang. The base line is: the term is taken out of the >Erlang run-time system and later reconstructed. The >question is whether the function can be called after >reconstructing the term, under various circumstances: The main issue when storing metadata persistently is how to deal with upgrades. An issue when sending metadata to another machine is whether that machine has the same code loaded. >1) Does it work if the function is a named function? Yes, given that the named module is loaded on that machine. >2) Does it work if the function is a named function, but the > term is restored in an environment that doesn't contain > that function? > (E.g. the term is sent over the network and reconstructed > in an Erlang run-time that doesn't have the same sources.) No. Worst case (?) is that the named function exists, but does something entirely different from what you expected. >3) Does it work if the function is anonymous? Anonymous functions are also tied to modules, so the same answer as above applies, except anonymous functions may also break if the module is replaced with an equivalent, but heavily reorganized module. This will not break (1). >4) Does it work if the function was constructed at run-time? > (Most functional languages can construct functions at run-time > by taking a given function and filling in a parameter with > either a term or another function. I'm not sure that this is > possible in Erlang, but I sincerely hope so...) Anonymous functions are constructed at runtime (the logic is static, but the closure is defined at runtime.) So the answer is the same as for (3). There is another option, seldomly exercized: you can send an abstract form and evaluate it on the other side. This is more stable in time and space than the other option, but at the cost of slightly worse performance. The performance issue can be partly overcome by on-the-fly compilation. /Uffe -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes From gunilla@REDACTED Mon Jun 16 15:13:21 2003 From: gunilla@REDACTED (Gunilla Arendt) Date: Mon, 16 Jun 2003 15:13:21 +0200 Subject: crashing the debugger References: Message-ID: <3EEDC271.E33B0318@erix.ericsson.se> You have found a bug in Debugger, it cannot handle a guard test "when false -> ..." and there seems to be such a guard in your code. Debugger uses its own mechanism to load and evaluate code, in order to enable stepwise execution, breakpoints etc. This means that Debugger is not suitable for finding the kind of error you describe. I would recommend you use a tool such as fprof, dbg or perhaps observer instead. Best regards, Gunilla david wallin wrote: > > Slightly puzzled. > > Tried to run a program under the debugger (because it stops > unexplainably and intermittently with the beam process taking alot of > CPU), and when I try to interpret the main file I get this error > message from the debugger: > > [...] > ,105,115,116,95,116,111,95,105,110,116,101,103,101,114,47,49,44,32,83,10 > 5,122,101,115,41,32,59,10,10,9,95,32,45,62,32,117,110,100,101,102,105,11 > 0,101,100,10,10,32,32,32,32,101,110,100,46,10,10,10,37,37,37,32,45,45,45 > ,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45 > ,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45,45 > ,45,45,45,45,45,45,45,45,45,45,45,45,45,45,32,91,116,104,101,32,101,110, > 100,93,10,109,0,0,0,16,217,142,202,234,188,223,186,193,177,235,28,143,55 > ,242,56,1>>} > ** When Server state == {state,11,[],[],false,all,[<0.30.0>]} > ** Reason for termination == > ** > {{badmatch,{function_clause,[{dbg_iload,guard_test,[{atom,98,false}]}, > {dbg_iload,and_guard,1}, > {dbg_iload,guard,1}, > {dbg_iload,clause,1}, > {dbg_iload,icr_clauses,1}, > {dbg_iload,expr,1}, > {dbg_iload,exprs,1}, > {dbg_iload,clause,1}| > more]}}, > [{dbg_iserver,handle_call,3}, > {gen_server,handle_msg,6}, > {proc_lib,init_p,5}]} > > any help would be helpful. > > cheers, > > --david. -- _____Gunilla Arendt______________________________________________ EAB/UKH/KD Erlang/OTP Product Development Gunilla.Arendt@REDACTED +46-8-7275730 ecn 851 5730 From francesco@REDACTED Mon Jun 16 23:47:24 2003 From: francesco@REDACTED (Francesco Cesarini) Date: Mon, 16 Jun 2003 22:47:24 +0100 Subject: Functions in data structures References: <3EEDBC7B.9060807@web.de> Message-ID: <3EEE3AEC.5040600@erlang-consulting.com> Hi Joachim, you posed some interesting questions... If you are planing on a real world application, I think the best approach is to test your questions and ideas. Set up two erlang nodes, convert your fun to a binary and zip it over. Or store it in mnesia. It will give you an immediate reply, often faster than looking it up in the documentation. And most certainly, more precise. That is what is so great and easy about Erlang. Good luck in your project, Francesco -- http://www.erlang-consulting.com Joachim Durchholz wrote: > Hi all, > > I've been toying with Erlang, preparing for a Real-World(TM) project, > and such a state provokes a specific kind of questions. > > Mine are related to functions in data structures (I gather that > Erlangese for "data structure" is "term" - is this correct?) > > Assume that a term contains a function. Further assume that this term > is converted to binary, transmitted to a different machine, and > converted back to an Erlang term; or assume it is stored in Mnesia and > later read back into Erlang. The base line is: the term is taken out > of the Erlang run-time system and later reconstructed. > The question is whether the function can be called after > reconstructing the term, under various circumstances: > > 1) Does it work if the function is a named function? > 2) Does it work if the function is a named function, but the > term is restored in an environment that doesn't contain > that function? > (E.g. the term is sent over the network and reconstructed > in an Erlang run-time that doesn't have the same sources.) > 3) Does it work if the function is anonymous? > 4) Does it work if the function was constructed at run-time? > (Most functional languages can construct functions at run-time > by taking a given function and filling in a parameter with > either a term or another function. I'm not sure that this is > possible in Erlang, but I sincerely hope so...) > > Regards, > Jo > > > From kotkaus@REDACTED Tue Jun 17 02:49:11 2003 From: kotkaus@REDACTED (James Freddy) Date: Mon, 16 Jun 2003 17:49:11 -0700 (PDT) Subject: xmerl newlines Message-ID: <20030617004911.74322.qmail@web10904.mail.yahoo.com> hi, i;m trying to output an xml files and with xmerl 0.15 i seem to recall newlines were put in for me. now i upgraded to 0.18 and the xml is one long string -- fine for word-wrap editor like emacs otherwise i bit hard to use. J __________________________________ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com From etxuwig@REDACTED Tue Jun 17 09:27:07 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Tue, 17 Jun 2003 09:27:07 +0200 (MEST) Subject: xmerl newlines In-Reply-To: <20030617004911.74322.qmail@web10904.mail.yahoo.com> Message-ID: On Mon, 16 Jun 2003, James Freddy wrote: >i;m trying to output an xml files and with xmerl 0.15 i >seem to recall newlines were put in for me. now i upgraded >to 0.18 and the xml is one long string -- fine for >word-wrap editor like emacs otherwise i bit hard to use. I get newlines when I export using xmerl_xml.erl in xmerl-0.18. Maybe I have hacked my version and forgotten about it... (: Perhaps Richard or Johan can explain if something has changed in this regard (lots changed between 0.15 and 0.18, but this one I don't recall.) /Uffe -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes From joe@REDACTED Tue Jun 17 10:42:35 2003 From: joe@REDACTED (Joe Armstrong) Date: Tue, 17 Jun 2003 10:42:35 +0200 (CEST) Subject: Bug ... in Erlang Message-ID: Try typing the following: > 1.0e309. Is this a bug? /Joe From fredrik.linder@REDACTED Tue Jun 17 12:03:54 2003 From: fredrik.linder@REDACTED (Fredrik Linder) Date: Tue, 17 Jun 2003 12:03:54 +0200 Subject: Functions in data structures In-Reply-To: Message-ID: -- ZIP -- > There is another option, seldomly exercized: you can send an > abstract form and evaluate it on the other side. This is > more stable in time and space than the other option, but > at the cost of slightly worse performance. The performance > issue can be partly overcome by on-the-fly compilation. > > /Uffe Interesting, is that implemented within Erlang? And if so, where can I read about it/how to use it? /Fredrik From luke@REDACTED Tue Jun 17 12:29:51 2003 From: luke@REDACTED (Luke Gorrie) Date: 17 Jun 2003 12:29:51 +0200 Subject: Functions in data structures In-Reply-To: References: Message-ID: "Fredrik Linder" writes: > -- ZIP -- > > There is another option, seldomly exercized: you can send an > > abstract form and evaluate it on the other side. This is > > more stable in time and space than the other option, but > > at the cost of slightly worse performance. The performance > > issue can be partly overcome by on-the-fly compilation. > > > > /Uffe > > Interesting, is that implemented within Erlang? And if so, where can I read > about it/how to use it? Consider this shell fragment: (x@REDACTED)1> Node = node(). x@REDACTED (x@REDACTED)2> F = fun() -> {fun_created_on, Node} end. #Fun The last return value, #Fun, is not a fun created directly from the expression {fun_created_on, Node}. Instead, the fun was created in erl_eval, and it captures the current variable bindings and the syntax-tree of the code it's actually supposed to run (i.e. {fun_created_on, Node}). Here is the bit of erl_eval that creates the fun: 1. expr({'fun',Line,{clauses,Cs}}, Bs, Lf) -> 2. %% This is a really ugly hack! 3. case length(element(3,hd(Cs))) of 4. 0 -> {value,fun () -> eval_fun(Cs, [], Bs, Lf) end,Bs}; If you actually call the fun: (x@REDACTED)3> F(). {fun_created_on,x@REDACTED} The code it executes is actually the eval_fun(...) from line (4) above. This then *interprets* the {fun_created_on, Node} expression, which was captured in the erl_eval fun along with the set of variable bindings. Very groovy! The nice part is that, so long as two nodes have compatible copies of erl_eval, they will always be able to run each others' code in this way. BTW, here is a code snippet out of Distel's backend which uses erl_eval directly: eval_expression(S) -> case parse_expr(S) of {ok, Parse} -> try_evaluation(Parse); {error, {_, erl_parse, Err}} -> {error, Err} end. try_evaluation(Parse) -> case catch erl_eval:exprs(Parse, []) of {value, V, _} -> {ok, flatten(io_lib:format("~p", [V]))}; {'EXIT', Reason} -> {error, Reason} end. parse_expr(S) -> {ok, Scan, _} = erl_scan:string(S), erl_parse:parse_exprs(Scan). Using that, you can create "portable" funs in your programs, like this: 1> distel:eval_expression("fun() -> hello() end."). {ok,"#Fun"} Although if you want the funs to include variable bindings, you'll need to pass them as a key-value list to that call to erl_eval:exprs, where I just use the empty list. Cheers, Luke From joachim.durchholz@REDACTED Tue Jun 17 12:47:54 2003 From: joachim.durchholz@REDACTED (Joachim Durchholz) Date: Tue, 17 Jun 2003 12:47:54 +0200 Subject: Functions in data structures In-Reply-To: <3EEE3AEC.5040600@erlang-consulting.com> References: <3EEDBC7B.9060807@web.de> <3EEE3AEC.5040600@erlang-consulting.com> Message-ID: <3EEEF1DA.1020405@web.de> Francesco Cesarini wrote: > If you are planing on a real > world application, This is indeed the case. > I think the best approach is to test your questions > and ideas. Set up two erlang nodes, convert your fun to a binary and zip > it over. Or store it in mnesia. It will give you an immediate reply, > often faster than looking it up in the documentation. And most > certainly, more precise. Er... right, but it won't help me if a future version of Erlang behaves differently. Since I'm a relative newbie, there's also the risk that Erlang is doing things slightly differently from what I expect. In other words, I risk misinterpreting the results of my experiments. Finally, In summary: experimental programming is fun, but it's better for verifying documentation than for finding out how things work. > Good luck in your project, Thanks :-) Regards, Jo From fredrik.linder@REDACTED Tue Jun 17 13:06:26 2003 From: fredrik.linder@REDACTED (Fredrik Linder) Date: Tue, 17 Jun 2003 13:06:26 +0200 Subject: Functions in data structures In-Reply-To: Message-ID: Thank you! This was very informative, and as you said, this IS groovy. :-) /Fredrik > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Luke Gorrie > Sent: den 17 juni 2003 12:30 > To: Fredrik Linder > Cc: Erlang Questions > Subject: Re: Functions in data structures > > > "Fredrik Linder" writes: > > > -- ZIP -- > > > There is another option, seldomly exercized: you can send an > > > abstract form and evaluate it on the other side. This is > > > more stable in time and space than the other option, but > > > at the cost of slightly worse performance. The performance > > > issue can be partly overcome by on-the-fly compilation. > > > > > > /Uffe > > > > Interesting, is that implemented within Erlang? And if so, > where can I read > > about it/how to use it? > > Consider this shell fragment: > > (x@REDACTED)1> Node = node(). > x@REDACTED > (x@REDACTED)2> F = fun() -> {fun_created_on, Node} end. > #Fun > > The last return value, #Fun, is not a fun created > directly from the expression {fun_created_on, Node}. Instead, the fun > was created in erl_eval, and it captures the current variable bindings > and the syntax-tree of the code it's actually supposed to run > (i.e. {fun_created_on, Node}). > > Here is the bit of erl_eval that creates the fun: > > 1. expr({'fun',Line,{clauses,Cs}}, Bs, Lf) -> > 2. %% This is a really ugly hack! > 3. case length(element(3,hd(Cs))) of > 4. 0 -> {value,fun () -> eval_fun(Cs, [], Bs, Lf) end,Bs}; > > If you actually call the fun: > > (x@REDACTED)3> F(). > {fun_created_on,x@REDACTED} > > The code it executes is actually the eval_fun(...) from line (4) > above. This then *interprets* the {fun_created_on, Node} expression, > which was captured in the erl_eval fun along with the set of variable > bindings. > > Very groovy! > > The nice part is that, so long as two nodes have compatible copies of > erl_eval, they will always be able to run each others' code in this > way. > > BTW, here is a code snippet out of Distel's backend which uses > erl_eval directly: > > eval_expression(S) -> > case parse_expr(S) of > {ok, Parse} -> > try_evaluation(Parse); > {error, {_, erl_parse, Err}} -> > {error, Err} > end. > > try_evaluation(Parse) -> > case catch erl_eval:exprs(Parse, []) of > {value, V, _} -> > {ok, flatten(io_lib:format("~p", [V]))}; > {'EXIT', Reason} -> > {error, Reason} > end. > > parse_expr(S) -> > {ok, Scan, _} = erl_scan:string(S), > erl_parse:parse_exprs(Scan). > > Using that, you can create "portable" funs in your programs, like > this: > > 1> distel:eval_expression("fun() -> hello() end."). > {ok,"#Fun"} > > Although if you want the funs to include variable bindings, you'll > need to pass them as a key-value list to that call to erl_eval:exprs, > where I just use the empty list. > > Cheers, > Luke > > From Erik.Reitsma@REDACTED Tue Jun 17 09:10:17 2003 From: Erik.Reitsma@REDACTED (Erik Reitsma (ETM)) Date: Tue, 17 Jun 2003 09:10:17 +0200 Subject: xmerl newlines Message-ID: <440A2703A54A8F4FB2AC2AE34F27129D215A8B@ESEALNT889.al.sw.ericsson.se> > i;m trying to output an xml files and with xmerl 0.15 > i seem to recall newlines were put in for me. now i > upgraded to 0.18 and the xml is one long string -- > fine for word-wrap editor like emacs otherwise i bit > hard to use. On the other hand, these newlines and tabs and spaces also appear in parsed XML. Therefore this is indeed more symmetrical. It would be nice if the newlines and tabs and spaces would be removed from the parsed XML (if it is not already supported and I missed some option). Now I do a pass through the parsed XML to remove all xmlText records that contain nothing but whitespace. *Erik. From kent@REDACTED Tue Jun 17 14:20:13 2003 From: kent@REDACTED (Kent Boortz) Date: 17 Jun 2003 14:20:13 +0200 Subject: Bug ... in Erlang In-Reply-To: References: Message-ID: Joe Armstrong writes: > Try typing the following: > > > 1.0e309. > > Is this a bug? You mean Eshell V5.2.3.6 (abort with ^G) 1> 1.0e309. ** 1: illegal float ** 1> Don't think it is a bug. On Solaris and Linux max double is defined as /usr/include/values.h:#define MAXDOUBLE 1.79769313486231570e+308 /usr/include/values.h:#define MINDOUBLE 4.94065645841246544e-324 kent From enano@REDACTED Tue Jun 17 14:32:36 2003 From: enano@REDACTED (Miguel Barreiro Paz) Date: Tue, 17 Jun 2003 14:32:36 +0200 (CEST) Subject: Bug ... in Erlang In-Reply-To: References: Message-ID: > You mean > > Eshell V5.2.3.6 (abort with ^G) > 1> 1.0e309. > ** 1: illegal float ** Solaris 8 on SPARC, OTP compiled 'by hand': Eshell V5.1.1 (abort with ^G) 1> 1.0e309. ** 1: illegal float ** Debian GNU/Linux on x86 "sid" (testing/unstable), OTP installed from the Debian package: Eshell V5.2.3.3 (abort with ^G) 1> 1.0e309. (hangs forever, eating a bit of CPU, memory not growing) From vlad_dumitrescu@REDACTED Tue Jun 17 14:49:56 2003 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Tue, 17 Jun 2003 14:49:56 +0200 Subject: ^G/^C in shell on Windows Message-ID: Hi, I noticed that ^G doesn't work for me: what it seems to do is just start another shell (or restart the other one), because I get the "Eshell V5.2.3.3" banner and the prompt is reset to 1. Pressing ^C on the other hand, prints the "BREAK: (a)bort....." message but doesn't wait for any input, it just ends execution. Sometimes not even the message gets printed but the program crashes (there comes a "The memory could not be read" error), but I think this latter has to do with port programs running. This is in erl.exe, not werl.exe. Werl works fine. Should I gather some more data and submit it for analysis, or is it a known thing? regards, Vlad From serge@REDACTED Tue Jun 17 15:16:29 2003 From: serge@REDACTED (Serge Aleynikov) Date: Tue, 17 Jun 2003 09:16:29 -0400 Subject: Bug ... in Erlang References: Message-ID: <3EEF14AD.1010606@hq.idt.net> I am not sure about Erlang, but in C (32bit) the highest value in the range of the double precision floating point type is 1.700000E+308 I suppose the proper behavior would be to get an illegal floating point exception thrown. 19> 1.799E+308. 1.00000e+0 20> 1.79E+308. 1.79000e+308 Serge Joe Armstrong wrote: > > Try typing the following: > > > 1.0e309. > > Is this a bug? > > /Joe > > From joe@REDACTED Tue Jun 17 15:36:55 2003 From: joe@REDACTED (Joe Armstrong) Date: Tue, 17 Jun 2003 15:36:55 +0200 (CEST) Subject: Bug ... in Erlang In-Reply-To: Message-ID: > Joe Armstrong writes: > > Try typing the following: > > > > > 1.0e309. > > > > Is this a bug? > > You mean > > Eshell V5.2.3.6 (abort with ^G) > 1> 1.0e309. > ** 1: illegal float ** > 1> This does not happen on my machine - the program just never returns It doesn't seem to use CPU time it just *hangs* (seem's like I'm not the only one too) - I verified that I'm using R9B-1 (latest and best TM) > > Don't think it is a bug. On Solaris and Linux max double is defined as > > /usr/include/values.h:#define MAXDOUBLE 1.79769313486231570e+308 > /usr/include/values.h:#define MINDOUBLE 4.94065645841246544e-324 > This I don't understand. My /usr/include/values.h doesn't define MAXDOUBLE I found this (eventually) #define GSL_DBL_MIN 2.2250738585072014e-308 #define GSL_DBL_MAX 1.7976931348623157e+308 in /usr/include/gsl/gsl_machine.h This is what I also found in a document describig the format of IEEE 754 reals. (I'm assuming that Erlang floats are actually IEEE 754 64 bit reals) is this so??? /Joe > kent > From tony@REDACTED Tue Jun 17 15:45:57 2003 From: tony@REDACTED (Tony Rogvall) Date: 17 Jun 2003 15:45:57 +0200 Subject: Bug ... in Erlang In-Reply-To: References: Message-ID: <1055857556.1753.5.camel@bit.hemma.se> On Tue, 2003-06-17 at 14:20, Kent Boortz wrote: > Joe Armstrong writes: > > Try typing the following: > > > > > 1.0e309. > > > > Is this a bug? > > You mean > > Eshell V5.2.3.6 (abort with ^G) > 1> 1.0e309. > ** 1: illegal float ** > 1> > No I guess joe meant (R9B-1 on linux) Eshell V5.2.3.3 (abort with ^G) 1> 1.0e309. (Nothing :-() I looks like the shell dies without a catch! -- Tony Rogvall -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part URL: From fredrik.linder@REDACTED Tue Jun 17 15:49:30 2003 From: fredrik.linder@REDACTED (Fredrik Linder) Date: Tue, 17 Jun 2003 15:49:30 +0200 Subject: Bug ... in Erlang In-Reply-To: <3EEF14AD.1010606@hq.idt.net> Message-ID: Would not the *proper* behaviour be to implement a big-float mechanism, in the same spirit as for integers? /Fredrik > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Serge Aleynikov > Sent: den 17 juni 2003 15:16 > To: Joe Armstrong > Cc: erlang-questions@REDACTED > Subject: Re: Bug ... in Erlang > > > I am not sure about Erlang, but in C (32bit) the highest value in the > range of the double precision floating point type is 1.700000E+308 > > I suppose the proper behavior would be to get an illegal floating point > exception thrown. > > 19> 1.799E+308. > 1.00000e+0 > 20> 1.79E+308. > 1.79000e+308 > > Serge > > Joe Armstrong wrote: > > > > Try typing the following: > > > > > 1.0e309. > > > > Is this a bug? > > > > /Joe > > > > > > > From peter@REDACTED Tue Jun 17 16:11:22 2003 From: peter@REDACTED (Peter Lund) Date: Tue, 17 Jun 2003 15:11:22 +0100 (BST) Subject: Bug ... in Erlang In-Reply-To: References: Message-ID: <8481.149.254.120.136.1055859082.squirrel@www.lundata.se> How many decimals would you have for the value 1/3? Unlimited precision perhaps? Peter > Would not the *proper* behaviour be to implement a big-float mechanism, > in the same spirit as for integers? > > /Fredrik From etxuwig@REDACTED Tue Jun 17 17:34:36 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Tue, 17 Jun 2003 17:34:36 +0200 (MEST) Subject: xmerl newlines In-Reply-To: <440A2703A54A8F4FB2AC2AE34F27129D215A8B@ESEALNT889.al.sw.ericsson.se> Message-ID: On Tue, 17 Jun 2003, Erik Reitsma (ETM) wrote: >> i;m trying to output an xml files and with xmerl 0.15 >> i seem to recall newlines were put in for me. now i >> upgraded to 0.18 and the xml is one long string -- >> fine for word-wrap editor like emacs otherwise i bit >> hard to use. > >On the other hand, these newlines and tabs and spaces also >appear in parsed XML. Therefore this is indeed more >symmetrical. There are many different requirements... (: I understand the problem as being that you've built a structure of tuples (or #xmlElement{}) in Erlang and want them exported with some pretty-printing. Is that right? A less than perfect pretty-print hack to xmerl_xml.erl illustrates how it could be done. xmerl_xml.erl is quite a small module, and easily modified into a local version of export that fits your needs perfectly: '#element#'(Tag, [], Attrs, Parents, E) -> [pp(length(Parents)), empty_tag(Tag, Attrs)]; '#element#'(Tag, Data, Attrs, Parents, E) -> [pp(length(Parents)), markup(Tag, Attrs, Data)]. pp(N) -> ["\n", lists:duplicate(N*4, $\s)]. This has the disadvantage of not putting the end tags where you'd expect them to be. To fix that, you have to copy xmerl_lib:markup/3 and modify it -- not that difficult perhaps. To accompany the above hack, you may want to remove any whitespace already in the structure: '#text#'(Text) -> case is_whitespace(Text) of true -> []; false -> export_text(Text) end. is_whitespace(" " ++ T) -> is_whitespace(T); is_whitespace("\n" ++ T) -> is_whitespace(T); is_whitespace("\t" ++ T) -> is_whitespace(T); is_whitespace([_|_]) -> false; is_whitespace([]) -> true. Or scan with the {space, normalize} option (see below.) >It would be nice if the newlines and tabs and spaces would >be removed from the parsed XML (if it is not already >supported and I missed some option). Now I do a pass >through the parsed XML to remove all xmlText records that >contain nothing but whitespace. I had implemented the {space,...} option wrongly in xmerl-0.15 and was set straight by those who use and understand XML. (: However, the option {space,normalize} _should_ do almost what you want. It will accumulate consecutive whitespace and replace it with one space. Close enough? /Uffe -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes From etxuwig@REDACTED Tue Jun 17 18:06:01 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Tue, 17 Jun 2003 18:06:01 +0200 (MEST) Subject: bug in erl_eval? Message-ID: Mats Cronqvist stumbled upon the following surprise: 1> F1 = fun() -> process_flag(trap_exit,true),receive quit -> ok end end. #Fun 2> P1 = spawn(F1). <0.40.0> 3> F2 = fun() -> link(P1), receive quit -> exit(foo) end end. #Fun 4> P2 = spawn(F2). <0.44.0> 5> dbg:tracer(). {ok,<0.48.0>} 6> dbg:p(P1,[m]). {ok,[{matched,nonode@REDACTED,1}]} 7> P2 ! quit. quit 8> (<0.40.0>) << {'EXIT',<0.44.0>,foo} 9> process_info(P1). [{current_function,{erl_eval,receive_clauses,4}}, {initial_call,{erlang,apply,2}}, {status,waiting}, {message_queue_len,0}, {messages,[]}, {links,[]}, {dictionary,[]}, {trap_exit,true}, {error_handler,error_handler}, {priority,normal}, {group_leader,<0.21.0>}, {heap_size,233}, {stack_size,17}, {reductions,45}, {garbage_collection,[{fullsweep_after,65535}]}] ** Where did the message go?! ** Putting the same code in a module and compiling it, the example works as expected: -module(funny). -compile(export_all). p1() -> spawn(fun() -> process_flag(trap_exit, true), receive quit -> ok end end). p2(P1) -> spawn(fun() -> link(P1), receive quit -> exit(foo) end end). Eshell V5.2.3.3 (abort with ^G) 1> P1 = funny:p1(). <0.30.0> 2> P2 = funny:p2(P1). <0.32.0> 3> P2 ! quit. quit 4> process_info(P1). [{current_function,{funny,'-p1/0-fun-0-',0}}, {initial_call,{erlang,apply,2}}, {status,waiting}, {message_queue_len,1}, {messages,[{'EXIT',<0.32.0>,foo}]}, {links,[]}, {dictionary,[]}, {trap_exit,true}, {error_handler,error_handler}, {priority,normal}, {group_leader,<0.21.0>}, {heap_size,233}, {stack_size,1}, {reductions,3}, {garbage_collection,[{fullsweep_after,65535}]}] Looking at erl_eval.erl, I think I understand what's happening. %% %% receive(Clauses, Bindings, LocalFuncHandler, Messages) %% receive_clauses(Cs, Bs, Lf, Ms) -> receive Val -> case match_clause(Cs, [Val], Bs, Lf) of {B, Bs1} -> merge_queue(Ms), exprs(B, Bs1, Lf); nomatch -> receive_clauses(Cs, Bs, Lf, [Val|Ms]) end end. %% %% receive_clauses(TimeOut, Clauses, TimeoutBody, Bindings, LocalFuncHandler) %% receive_clauses(T, Cs, TB, Bs, Lf, Ms) -> {_,_} = statistics(runtime), receive Val -> case match_clause(Cs, [Val], Bs, Lf) of {B, Bs1} -> merge_queue(Ms), exprs(B, Bs1, Lf); nomatch -> {_,T1} = statistics(runtime), if T == infinity -> receive_clauses(T, Cs, TB, Bs, Lf, [Val|Ms]); T-T1 =< 0 -> receive_clauses(0, Cs, TB, Bs, Lf, [Val|Ms]); true -> receive_clauses(T-T1, Cs, TB, Bs, Lf, [Val|Ms]) end end after T -> merge_queue(Ms), {B, Bs1} = TB, exprs(B, Bs1, Lf) end. merge_queue(Ms) -> send_all(recv_all(Ms), self()). The evaluator consumes each message and matches it; if it doesn't match, it puts the message in an accumulator and waits for the next message. So the message is "queued", but not visible to process_info/1. I cannot think of a great solution to this problem. One could match messages by polling process_info(self(), messages), but that doesn't give much responsiveness in the case where one has to wait for a message. Suggestions? Should we just live with this? /Uffe -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes From thomasl_erlang@REDACTED Tue Jun 17 18:41:22 2003 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 17 Jun 2003 09:41:22 -0700 (PDT) Subject: Bug ... in Erlang In-Reply-To: Message-ID: <20030617164122.16522.qmail@web41902.mail.yahoo.com> --- Fredrik Linder wrote: > Would not the *proper* behaviour be to implement a > big-float mechanism, in > the same spirit as for integers? If we want to do it *properly*, my suggestion is to take a page from what Common Lisp did. E.g., Best, Thomas __________________________________ Do you Yahoo!? The New Yahoo! Search - Faster. Easier. Bingo. http://search.yahoo.com From bjorn@REDACTED Tue Jun 17 15:00:50 2003 From: bjorn@REDACTED (=?ISO-8859-1?Q?Bj=F6rn_Bylander?=) Date: Tue, 17 Jun 2003 15:00:50 +0200 Subject: Bug ... in Erlang In-Reply-To: References: Message-ID: <3EEF1102.3040708@loxysoft.se> Windows 2000 Professional: Eshell V5.2 (abort with ^G) 1> 1.0e309. 1.00000e+0 Kent Boortz wrote: >You mean > > Eshell V5.2.3.6 (abort with ^G) > 1> 1.0e309. > ** 1: illegal float ** > 1> > > From raimo.niskanen@REDACTED Tue Jun 17 18:53:02 2003 From: raimo.niskanen@REDACTED (Raimo Niskanen) Date: Tue, 17 Jun 2003 18:53:02 +0200 Subject: Bug ... in Erlang References: Message-ID: Try also: 1> 1.b. On our R9B, patched, erl_eval starts a death spin. There is probably some error checking in the shell (evaluator) that is missing. Our soon to come R9C does not show the same error; the shell and erl_eval has been somewhat rewritten. Nor does "1.0e309." death spin. 17> 1.0e309. ** 1: illegal float ** 17> . ** 1: syntax error before: '.' ** Unfortunately there is a semantic difference with the new erl_scan (compared to the old), that it does not read input until the full stop if there is a scan error before that. Therefore the syntax error on the '.'. It can probably be fixed, but the question is if I must? The same applies for: 17> 16#. ** 1: illegal integer ** 17> . ** 1: syntax error before: '.' ** / Raimo Niskanen, Erlang/OTP, Ericsson AB. Joe Armstrong wrote: > > Try typing the following: > > > 1.0e309. > > Is this a bug? > > /Joe > From raimo.niskanen@REDACTED Tue Jun 17 18:57:46 2003 From: raimo.niskanen@REDACTED (Raimo Niskanen) Date: Tue, 17 Jun 2003 18:57:46 +0200 Subject: Bug ... in Erlang References: , , <3EEF1102.3040708@loxysoft.se> Message-ID: Can everybody try: 17> erlang:list_to_float("1.0e309"). =ERROR REPORT==== 17-Jun-2003::18:54:46 === Error in process <0.46.0> with exit value: {badarg,[{erlang,list_to_float,["1.0e309"]},{erl_eval,do_apply,5},{shell,eval_loop,2}]} ** exited: {badarg,[{erlang,list_to_float,["1.0e309"]}, {erl_eval,do_apply,5}, {shell,eval_loop,2}]} ** on all your fancy platforms and see what you get. We might have at least two competing problems here. / Raimo Niskanen, Erlang/OTP, Ericsson AB. Bj?rn Bylander wrote: > Windows 2000 Professional: > > Eshell V5.2 (abort with ^G) > 1> 1.0e309. > 1.00000e+0 > > Kent Boortz wrote: > >> You mean >> >> Eshell V5.2.3.6 (abort with ^G) >> 1> 1.0e309. >> ** 1: illegal float ** >> 1> >> > From daniel.neri@REDACTED Tue Jun 17 15:47:37 2003 From: daniel.neri@REDACTED (=?iso-8859-1?q?Daniel_N=E9ri?=) Date: Tue, 17 Jun 2003 15:47:37 +0200 Subject: Bug ... in Erlang References: Message-ID: <87isr4u87a.fsf@fnord.hq.sigicom.net> Miguel Barreiro Paz writes: > Solaris 8 on SPARC, OTP compiled 'by hand': > Eshell V5.1.1 (abort with ^G) > 1> 1.0e309. > ** 1: illegal float ** > > Debian GNU/Linux on x86 "sid" (testing/unstable), OTP installed from the > Debian package: > > Eshell V5.2.3.3 (abort with ^G) > 1> 1.0e309. > (hangs forever, eating a bit of CPU, memory not growing) I think we've had this discussion before: http://www.erlang.org/ml-archive/erlang-questions/200203/msg00049.html Regards, --Daniel From serge@REDACTED Tue Jun 17 20:07:34 2003 From: serge@REDACTED (Serge Aleynikov) Date: Tue, 17 Jun 2003 14:07:34 -0400 Subject: Suspected Spam: Re: Bug ... in Erlang References: Message-ID: <3EEF58E6.9000300@hq.idt.net> Running Win2000 Sp3: (node1@REDACTED)1> os:type(). {win32,nt} (node1@REDACTED)2> erlang:list_to_float("1.0e309"). 1.00000e+0 (node1@REDACTED)3> 1.b. hangs forever. Raimo Niskanen wrote: > Try also: > > 1> 1.b. > > On our R9B, patched, erl_eval starts a death spin. There is probably > some error checking in the shell (evaluator) that is missing. > > Our soon to come R9C does not show the same error; the shell and > erl_eval has been somewhat rewritten. Nor does "1.0e309." death spin. > > 17> 1.0e309. > ** 1: illegal float ** > 17> . > ** 1: syntax error before: '.' ** > > Unfortunately there is a semantic difference with the new erl_scan > (compared to the old), that it does not read input until the full stop > if there is a scan error before that. Therefore the syntax error on the > '.'. It can probably be fixed, but the question is if I must? > > The same applies for: > > 17> 16#. > ** 1: illegal integer ** > 17> . > ** 1: syntax error before: '.' ** > > / Raimo Niskanen, Erlang/OTP, Ericsson AB. > > > > Joe Armstrong wrote: > >> >> Try typing the following: >> >> > 1.0e309. >> >> Is this a bug? >> >> /Joe >> > > -- ================================================================ | Serge Aleynikov Tel: (973) 438-3436 | MIS Telecom Fax: (973) 438-1457 | IDT Corp. serge@REDACTED ================================================================ From lennart.ohman@REDACTED Tue Jun 17 21:24:36 2003 From: lennart.ohman@REDACTED (=?ISO-8859-1?Q?Lennart_=D6hman?=) Date: Tue, 17 Jun 2003 21:24:36 +0200 Subject: Functions in data structures References: <3EEDBC7B.9060807@web.de> <3EEE3AEC.5040600@erlang-consulting.com> <3EEEF1DA.1020405@web.de> Message-ID: <3EEF6AF4.8030001@st.se> > ... > Er... right, but it won't help me if a future version of Erlang behaves > differently. > ... That is one thing which is good with Erlang. It *is* being used for real-life systems. Meaning that the user community is rather restrictive to changes that breaks old code, unless such bring substantial advantages to most developers. /Lennart ------------------------------------------------------------- Lennart Ohman phone : +46-8-587 623 27 Sjoland & Thyselius Telecom AB cellular: +46-70-552 6735 Sehlstedtsgatan 6 fax : +46-8-667 8230 SE-115 28 STOCKHOLM, SWEDEN email : lennart.ohman@REDACTED From chris.danx@REDACTED Tue Jun 17 21:44:34 2003 From: chris.danx@REDACTED (chris.danx) Date: Tue, 17 Jun 2003 20:44:34 +0100 Subject: Bug ... in Erlang In-Reply-To: References: , , <3EEF1102.3040708@loxysoft.se> Message-ID: <3EEF6FA2.8070106@ntlworld.com> Raimo Niskanen wrote: > Can everybody try: > > 1.b. hangs > > 17> erlang:list_to_float("1.0e309"). hangs. Running R9B-1 on Linux i686 From byron.hale@REDACTED Tue Jun 17 22:22:56 2003 From: byron.hale@REDACTED (Byron Hale) Date: Tue, 17 Jun 2003 13:22:56 -0700 Subject: First regular annual Bay Area Erlang Beer Break (BAEBB 2003) In-Reply-To: Message-ID: <5.2.0.9.2.20030617132100.02e28310@einfo.com> I'm busy through June 23. Byron Hale byron.hale@REDACTED At 10:11 AM 5/31/2003 -0700, you wrote: >Hi, > >I'm not sure how many Erlang folks we have on this list live in the SF Bay >Area, but I thought it would be a good idea to find out ! > >Anyone in the general vicinity of the SF Bay area interested in drinking >coffee/beer on a weekday evening, in company of like minded people (but >with different opinions :) ? > >If you are, let me know if the date/time below works for you. > > >Default proposal would be to meet June 19 @ 7PM at Cafe Borrone, Menlo Park. > > >Cheers, >-Roger From joachim.durchholz@REDACTED Tue Jun 17 23:40:23 2003 From: joachim.durchholz@REDACTED (Joachim Durchholz) Date: Tue, 17 Jun 2003 23:40:23 +0200 Subject: Bug ... in Erlang In-Reply-To: References: , , <3EEF1102.3040708@loxysoft.se> Message-ID: <3EEF8AC7.3000106@web.de> Raimo Niskanen wrote: > Can everybody try: > > 17> erlang:list_to_float("1.0e309"). Here's my transcript: --- Erlang (BEAM) emulator version 5.2.3.3 [threads:0] Eshell V5.2.3.3 (abort with ^G) 1> erlang:list_to_float("1.0e309"). 1.00000e+0 2> --- Weirder and weirder... This is running on Win 2K. Regards, Jo From joachim.durchholz@REDACTED Tue Jun 17 23:42:44 2003 From: joachim.durchholz@REDACTED (Joachim Durchholz) Date: Tue, 17 Jun 2003 23:42:44 +0200 Subject: Functions in data structures In-Reply-To: <3EEF6AF4.8030001@st.se> References: <3EEDBC7B.9060807@web.de> <3EEE3AEC.5040600@erlang-consulting.com> <3EEEF1DA.1020405@web.de> <3EEF6AF4.8030001@st.se> Message-ID: <3EEF8B54.5010304@web.de> Lennart ?hman wrote: >> ... >> Er... right, but it won't help me if a future version of Erlang >> behaves differently. > > That is one thing which is good with Erlang. It *is* being used for > real-life systems. Meaning that the user community is rather restrictive > to changes that breaks old code, unless such bring substantial advantages > to most developers. Oh, right. Though I'm still reluctant to depend on undefined behaviour. Anyway, I have decided to send just plain data, no functions. Code will be sent in the form of software upgrades, just the way that people expect things to work. (Sending functions implies all sorts of security hassles anyway.) Regards, Jo From ulf.wiger@REDACTED Wed Jun 18 08:24:26 2003 From: ulf.wiger@REDACTED (Wiger Ulf) Date: Wed, 18 Jun 2003 08:24:26 +0200 Subject: Functions in data structures References: <3EEDBC7B.9060807@web.de> <3EEE3AEC.5040600@erlang-consulting.com> <3EEEF1DA.1020405@web.de> <3EEF6AF4.8030001@st.se> <3EEF8B54.5010304@web.de> Message-ID: <000d01c33562$47549ae0$53bfb5d4@telia.com> From: "Joachim Durchholz" > Anyway, I have decided to send just plain data, no functions. Code will > be sent in the form of software upgrades, just the way that people > expect things to work. (Sending functions implies all sorts of security > hassles anyway.) You may want to include a version indicator in the messages going between nodes. There are currently a few things that can mess up certain code upgrades: - record structures are compile-time dependent - mnesia can not have multiple simultaneous versions of the schema Thus, if you send records in messages between nodes, and have a fully distributed database, it is a bit more difficult to upgrade code one node at a time (an otherwise attractive option). The issue of mnesia can be overcome by not mixing in too much into one upgrade, and taking care to build a stable data model. That is, if your upgrade doesn't contain schema changes, you can upgrade one node at a time. The issue of messages between nodes is a bit trickier. I don't think there are any good guidelines about how to do this. Distribution transparency is very nice (_very_ nice), but it does make in-service upgrade a bit more difficult. UBF may offer some relief... The general recommendation is to exercise more care in communication that goes across node boundaries -- treat it as a set of formal interfaces. The main reason for this rambling is that you need to spend some time rather early thinking about what your upgrade requirements are, and how they affect your design choices. From etxuwig@REDACTED Wed Jun 18 10:14:50 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Wed, 18 Jun 2003 10:14:50 +0200 (MEST) Subject: mnemosyne-bug? Message-ID: Looking at the code in mnemosynce_lc:make_rec_defs/1, mk_rec_defs(TabAttrs) -> lists:map(fun({Tab,Attrs}) -> {attribute,0,record, {Tab, lists:map(fun(Attr) -> {record_field,0,{atom,0,Attr}} end, Attrs)}} end, TabAttrs). shouldn't it call mnesia:table_info(Tab, record_name) instead of naively assuming that the record_name is always the same as the table name? 3> mnesia:create_table(foo,[{attributes,[a1,a2,a3]},{record_name,rec}]). {atomic,ok} 4> mnesia:table_info(foo,record_name). rec 5> mnesia:dirty_write(foo, {foo,1,2,3}). ** exited: {aborted,{bad_type,{foo,1,2,3}}} ** 6> mnesia:dirty_write(foo, {rec,1,2,3}). ok /Uffe -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes From etxuwig@REDACTED Wed Jun 18 11:29:01 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Wed, 18 Jun 2003 11:29:01 +0200 (MEST) Subject: xmerl newlines In-Reply-To: <440A2703A54A8F4FB2AC2AE34F27129D215A93@ESEALNT889.al.sw.ericsson.se> Message-ID: On Wed, 18 Jun 2003, Erik Reitsma (ETM) wrote: >I think that that was James' issue. My "problem" was, that >a pretty-printed XML document is not parsed into the same >data structure as a non-pretty-printed. So: Yes, apologies for referring to James's problems in a reply to your mail. >Still, it would be easier if all xmlText records with empty >text would be removed during parsing, especially if it has >"siblings" that are xmlElement records. An option >{space,remove} would be nice... Such an option would be non-standard, ;), and perhaps difficult to implement well. However, there's a non-standard option called {acc_fun,F} with which one can do cute things. It would allow you to remove empty text segments today, with the slight side effect that the pos attribute will be wrong. I have attached a diff to xmerl_scan.erl (the 0.18 version, since the latest version in CVS is much altered.) This diff includes a backward-compatible extension to the acc_fun semantics, allowing you to cleanly skip elements and attributes that you do not want to include in the output from the scanner. The change allows the acc function to return a new value for the position counter, which is otherwise automatically incremented by 1 (old semantics). This change also makes it possible to split an object into multiple smaller objects (for whatever reason) while keeping the numbering sequential. I wanted to avoid having to call length(NewAcc) each time, as it's an O(N) operation. The following sample program illustrates three ways of scanning a document: 1) the default scan, which leaves whitespace untouched 2) normalizing spaces 3) normalizing spaces, then removing text elements that only contain one space. -module(tmp). -include("xmerl.hrl"). -export([file1/1, file2/1, file3/1]). file1(F) -> xmerl_scan:file(F). file2(F) -> xmerl_scan:file(F, [{space,normalize}]). file3(F) -> Acc = fun(#xmlText{value = " ", pos = P}, Acc, S) -> {Acc, P, S}; % new return format (X, Acc, S) -> {[X|Acc], S} end, xmerl_scan:file(F, [{space,normalize}, {acc_fun, Acc}]). I do not have developer access to the sowap project, so I have not updated the CVS version. /Uffe -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes -------------- next part -------------- 333,335c333,339 < %% The acc/3 function must return either {[X'|Acc], S'} or {Acc, S'}. < %% It is not allowed to make significant changes to X, such as altering < %% the object type. --- > %% The acc/3 function can return either {Acc?, S'} or {Acc', Pos', S'}, > %% where Pos' can be derived from X#xmlElement.pos, X#xmlText.pos, or > %% X#xmlAttribute.pos (whichever is the current object type.) > %% The acc/3 function is not allowed to redefine the type of object > %% being defined, but _is_ allowed to either ignore it or split it > %% into multiple objects (in which case {Acc',Pos',S'} should be returned.) > %% If {Acc',S'} is returned, Pos will be incremented by 1 by default. 1430,1431c1434,1441 < {NewAcc, S2} = AccF(Markup, Acc, S1), < scan_content(T1, S2, Pos+1, Name, Attrs, Space, Lang, Parents, NS, NewAcc); --- > {NewAcc, NewPos, NewS} = case AccF(Markup, Acc, S1) of > {Acc2, S2} -> > {Acc2, Pos+1, S2}; > {Acc2, Pos2, S2} -> > {Acc2, Pos2, S2} > end, > scan_content(T1, S2, NewPos, Name, Attrs, Space, Lang, > Parents, NS, NewAcc); 1446,1447c1456,1463 < {NewAcc, S4} = F(Text, Acc, S3), < scan_content(T1, S4, Pos+1, Name, Attrs, Space, Lang, Parents, NS, NewAcc). --- > {NewAcc, NewPos, NewS} = case F(Text, Acc, S3) of > {Acc4, S4} -> > {Acc4, Pos+1, S4}; > {Acc4, Pos4, S4} -> > {Acc4, Pos4, S4} > end, > scan_content(T1, NewS, NewPos, Name, Attrs, Space, Lang, > Parents, NS, NewAcc). From hakan@REDACTED Wed Jun 18 11:35:47 2003 From: hakan@REDACTED (Hakan Mattsson) Date: Wed, 18 Jun 2003 11:35:47 +0200 (MEST) Subject: Functions in data structures In-Reply-To: <000d01c33562$47549ae0$53bfb5d4@telia.com> Message-ID: On Wed, 18 Jun 2003, Wiger Ulf wrote: Uffe> The main reason for this rambling is that you need to spend some Uffe> time rather early thinking about what your upgrade requirements are, Uffe> and how they affect your design choices. I wrote some internal notes about things to think about when performing changes in the Mnesia application. The notes are from 1998 and a bit outdated, but I think that they still can serve as a source for inspiration when you think about different types of future changes that you need to prepare your own application for. /H?kan --- H?kan Mattsson Ericsson High Availability Software, DBMS Internals http://www.ericsson.com/cslab/~hakan/ -------------- next part -------------- Mnesia upgrade policy (PA3) =========================== This paper describes the upgrade policy of the Mnesia application. It is divided into the following chapters: o Architecture overview o Compatibility o Configuration management o Compatibility requirements on other applications o Upgrade scenarios o Remaining issues Architecture overview --------------------- Mnesia is a distributed DataBase Management System (DBMS), appropriate for telecommunications applications and other Erlang applications which require continuous operation and exhibit soft real-time properties. Mnesia is entirely implemented in in Erlang. Meta data about the persistent tables is stored in a public ets table, called mnesia_gvar. This ets table is accessed concurrently by several kinds of processes: * Static - on each node Mnesia has about 10 static processes. Eg. monitor, controller, tm, locker, recover, dumper... * Dynamic - there are several kinds of dynamic processes are created for various purposes. Eg. tab_copier, perform_dump, backup_master ... * Client processes created by Mnesia users. These invokes the Mnesia API to perform operations such as table lookups, table updates, table reconfigurations... All these kinds of processes communicates with each other both locally (on the same node) and remotely. Mnesia may either be configured to use local disc or be to totally disc less. All disc resident data is located under one directory and is hosted by disk_log and dets. The persistent tables may be backed up to external media. By default backups are hosted by disk_log. Compatibility ------------- Each release of Mnesia has a unique version identifier. If Mnesia is delivered to somebody outside the development team, the version identifier is always changed, even if only one file differs from the previous release of Mnesia. This means that the exact version of each file in Mnesia can be determined from the version identifier of the Mnesia application. Mnesia does NOT utilize the concept of OTP "patches". The smallest deliverable is currently the entire application. In future releases we will probably deliver binaries, source code and different kinds of documentation as separate packages. Changes of the version identifier follows a certain pattern, depending of how compatible the new release is with the previous dito. The version identifier consists of 3 integer parts: Major.Minor.Harmless (or just Major.Minor when Harmless is 0). If a harmless detail has been changed, the "Harmless" part is incremented. The change must not imply any incompatibility problems for the user. An application upgrade script (.appup) could be delivered in order to utilize code change in a running system, if required. If the change is more than a harmless detail (or if it is a harmless change but has implied a substantial rewrite of the code), the "Minor" part is incremented. The change must not imply any incompatibility problems for the user. An application upgrade script (.appup) for code change in a running system, is not always possible to deliver if the change is complex. A full restart of Mnesia (and all applications using Mnesia) may be required, potentially on all nodes simultaneously. All other kinds of changes implies the "Major" part to be incremented. The change may imply that users of Mnesia needs to rewrite their code, restart from backup or other inconveniences. Application upgrade script (.appup) for code change in a running system is probably too complex to write. In any case it is a good idea to always read the release notes. Configuration management ------------------------ Each major release constitutes an own development branch. Bugfixes, new functionality etc. are normally performed in the latest major development branch only. The last release of Mnesia is always the best and customers are encouraged to upgrade to it. Application upgrade scripts (.appup) are only written if they are explicitly requested by a customer. Preparing the code for a smoothless code change in a running system is very demanding and restrains productive development. The code that handles the upgrade is extremely hard to test, but it must be 100% correct. If it is not 100% correct it is totally useless since the error may easily escalate to a node restart or an inconsistent database. It may however not always be possible to enforce a customer to accept the latest release of incompatibility reasons. If a customer already has a running system and encounters a serious bug in Mnesia, we may be enforced to fix the bug in an old release. Such a bugfix is performed in a separate bugfix branch (dedicated for this particular bugfix). All source files in the release is labled with the version identifier (e.g. "mnesia_3.4.1"). An application upgrade script (.appup) is probably needed. Compatibility requirements on other applications ------------------------------------------------ Mnesia is entirely implemented in Erlang and depends heavily on the Erts, Kernel and StdLib applications. Changes of storage format in dets and disk_log may cause these files to be automatically upgraded to the new format, but only if Mnesia allows them to perform the "repair". As an effect of such a format upgrade it is likely that the Mnesia files cannot be read by older releases, but it may be acceptable that old releases not are forward compatible. Mnesia stores its data as binaries. Changes of the external binary format must also be backward compatible. Automatic conversion to the new format is required. Note, that Mnesia backups may be archived in years and that this requires Erts and Kernel to be backward compatible with very old binary formats and disk_log formats. Upgrade scenarios ----------------- There are a wide range of upgrade scenarios that needs to be analyzed before they occur in reality. Here follows a few of them: Backup format change. In the abstract header of the backup there is a version tag that identifies the format of the rest of the backup. Backups may be archived for a long period of time and it is important that old backups can be loaded into newer versions of the system. The backup format is Mnesia's survival format. Changes in the abstract backup format requires the backup version tag to be incremented and code to be written to convert the old format at backup load time. Minor change. The concrete format is an open format handled by a callback module and it is up to the callback module implementors to handle future changes. The default callback module uses disk_log and Mnesia relies heavily on disk_log's ability to automatically convert old disk_log files into new dito if the concrete format has been changed. Transaction log file format change. In the abstract header of the transaction log there is a version tag that identifies the format of the rest of the log. Changes in the abstract transaction log format requires the transaction log version tag to be incremented and code to be written to convert the old format when the log is dumped at startup. Minor change. The concrete format is hidden by disk_log and Mnesia relies on disk_log's ability to automatically convert old disk_log files into new dito if the concrete format has been changed. If the abstract format change is severe or if disk_log cannot handle old disk_log formats the entire Mnesia database has to be backed up to external media and then installed as fallback. It may in worst case be neccessary to implement a new backup callback module that does not make use of disk_log. Major change. Decision table log file format change. In the abstract header of the decision table log file there is a version tag that identifies the format of the rest of the log. The concrete format is hidden by disk_log and severe changes in the abstract or concrete formats are handled in the same way as complex changes of the transaction log file format: back the database up and re-install it as a fallback. Major change. Minor changes in the abstract format can be handled by conversion code run at startup. Minor change. .DAT file format change. .DAT files has no abstract version format identifier. The concrete format is hidden by dets and Mnesia relies on dets' ability to automatically convert old dets files into new dito if the concrete format has been changed. Changes in the abstract or incompatible changes in the concrete format are are handled in the same way as complex changes in the transaction log file format: back the database up and re-install it as a fallback. Core file format change. The core file is a debugging aid and contains essential information about the database. The core is automatically produced when Mnesia encounters a fatal error or manually by the user for the purpose of enclosing it into a trouble report. The core file consists of list of tagged tuples stored as a large binary. Future changes of the core format can easily be handled by adding a version tag first in the list if neccessary. Persistent schema format change. The schema is implemented as an ordinary table with table names as key and a list of table properties as single attribute. Each table property is represented as a tagged tuple and adding new properties will not break any code. Minor change. Incompatible changes of the schema representation can be handled in the same way as complex changes of the transaction log file format: back the database up and re-install it as a fallback. Major change. If the change is severe and impacts the backup format then it should not be performed at all. Renaming schema table to mnesia_schema. All internal (d)ets tables are prefixed with 'mnesia_' in order to avoid name conflicts with other applications. The only exception is the table named 'schema'. Renaming 'schema' to 'mnesia_schema' is a major change, that may breaks much customer code if it is not done very careful, since the name of the table is a part of the Mnesia API. The least intrusive change would be to leave the name 'schema' as a part of the API, but internally use the name 'mnesia_schema' and map all usages of 'schema' in all internal modules that accesses the schema table. It is however questionable if the change is worth the work effort. Transient schema format change The transient schema information is stored in a public ets table named mnesia_gvar. Changes in the transient schema format affects a lot of processes and it is not feasible to change it dynamically. A restart on all db_nodes is required. Minor change. Configuration parameter change. *** partly not supported *** Many of the configuration parameters ought to be possible to be changed dynamically in a running system: access_module backup_module debug dump_log_load_regulation dump_log_time_threshold dump_log_update_in_place dump_log_write_threshold event_module extra_db_nodes The remaining configuration parameters are only interesting at startup and any dynamic change of these should silently be ignored: auto_repair dir embedded_mnemosyne ignore_fallback_at_startup max_wait_for_decision schema_location Inter node protocol change. *** partly not supported *** When Mnesia on one node tries to connect to Mnesia on another node it negotiates with the other node about which inter node communication protocol to use. A list of all known protocols identifiers are sent to the other node which replies with the protocol that it prefers. The other node may also reject the connection. Always when the inter node protocol needs to be changed, the protocol identifier is changed. If the change is a compatible change and we need to upgrade the protocol in a running system things gets a little bit complicated. A new version of the software which understands both the old an new protocols must be loaded on all nodes as a first step. All processes that uses old code must somehow switch to use the new code. One severe problem here is to locate all processes that needs a code switch. Server processes are fairly straight forward to manage. Client processes that are waiting in a receive statement are far more difficult to first locate and then force a code switch. We may prepare for the code switch by letting the client be able to receive a special code switch message and then continue to wait for interesting messages. (gen_server does not handle this but since Mnesia does not use gen_server's for performance critical processes or crash sensitive processes Mnesia can handle this in many cases.) If the new protocol is a pure extension of the old dito we may go ahead and use the new protocol and bump up the protocol version identifier. More complex protocol changes are sometimes possible to handle, but since they are extreamly hard to test it is probably better to restart the system. If the change is an incompatible change, Mnesia must be restarted on all nodes. Intra node protocol change. Changing protocol between processes on the same node is slightly easier than changing inter node protocols. The difference is that we may iterate over all nodes and restart them one and one until the software has started to use the new protocols on all nodes. Adding a new static process. Adding a new static process in Mnesia is a minor change, but cannot be handled by the supervisor "application" if the shutdown order of the static processes are important. The most relieable approach here is to restart Mnesia. Adopt to new functionality in Erts, Kernel or StdLib: When was the new functionality introduced? Can Mnesia use the new functionality and still run on older Erlang/OTP releases? Changing a function in the Mnesia API. Changes of module name, function name, arguments or semantics of a function is likely to be a major change. Adding a brand new function is a minor change. Changing a Mnesia internal function. If it is possible to locate all processes that may invoke the function it may be worth to handle the code change in a running system. The safe approach is to restart the node. The processes must be able to handle 'sys' messages and export 'sys' callback functions. Minor change. Process state format change. If it is possible to locate all processes that may invoke the function it may be worth to handle the code change in a running system. The safe approach is to restart the node. The processes must be able to handle 'sys' messages and export 'sys' callback functions. Minor change. Code change to fix a harmless bug. Does these exist in reality or only in the Erlang book? Code change to fix an inconsistency bug. A bug like all others but the effect of it is an inconsistent database. Mnesia cannot repair the database and it is up to the Mnesia users to make it consistent again. There are three options here: - ignore the fact that the database is inconsistent - back up the database and make the backup consistent before installing it as a fallback. - fix the inconsistency on-line while all applications are accessing the database Code change to prevent an indefinite wait (deadlock, protocol mismatch). A bug like all others but the effect of it is hanging processes. Restart Mnesia on one or more nodes. Remaining issues ---------------- The following issues remains to be implemented: - Dynamic configuration parameter change - Prepare code switch of client code From etxuwig@REDACTED Wed Jun 18 11:56:41 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Wed, 18 Jun 2003 11:56:41 +0200 (MEST) Subject: mnemosyne-bug? In-Reply-To: Message-ID: I did intend to illustrate the problem below some more, but ran into another mnemosyne bug. 12> mnemosyne:string_to_handle("query [X || X <- table(foo)] end. "). {'EXIT',{function_clause,[{erl_parse,normalise, [{call, 0, {atom,0,'MNEMOSYNE RULE'}, [{tuple, 0, [{atom,0,data}, {atom,0,record_defs}]}]}]}, {erl_parse,normalise_list,1}, {erl_parse,normalise,1}, {erl_parse,normalise_list,1}, {erl_parse,normalise,1}, {mnemosyne_lc,'-one_lc_to_handle/1-fun-1-',2}, {lists,foldl,3}, {mnemosyne_lc,one_lc_to_handle,1}| more]}} Writing a module that includes some queries does work, so I wrote the following: -module(mnemtest). -compile(export_all). -include_lib("mnemosyne/include/mnemosyne.hrl"). -record(foo, {a1,a2,a3}). -record(bar, {a1,a2,a3}). foo() -> Q = query [X || X <- table(foo)] end, mnesia:transaction( fun() -> mnemosyne:eval(Q) end). bar() -> Q = query [X || X <- table(bar)] end, mnesia:transaction( fun() -> mnemosyne:eval(Q) end). Recalling that the table foo was created as: >3> mnesia:create_table(foo,[{attributes,[a1,a2,a3]},{record_name,rec}]). the record definition -record(foo, [a1,a2,a3]) is irrelevant, but mnemosyne will not compile the function foo() without it. I also created a table bar such that: 4> mnesia:create_table(bar,[{attributes,[a1,a2,a3]}]). {atomic,ok} 5> mnesia:dirty_write(bar, {bar,1,2,3}). ok 6> c(mnemtest). {ok,mnemtest} 7> mnemtest:foo(). {atomic,[]} 8> mnemtest:bar(). {atomic,[{bar,1,2,3}]} 9> ets:tab2list(foo). [{rec,1,2,3}] 10> ets:tab2list(bar). [{bar,1,2,3}] That is, the query in foo() above only compiles if given an incorrect record definition, and thus doesn't find anything in the table foo. However, finding all data in 'bar' (which has the same record name as the table name) works fine. /Uffe On Wed, 18 Jun 2003, Ulf Wiger wrote: > >Looking at the code in mnemosynce_lc:make_rec_defs/1, > >mk_rec_defs(TabAttrs) -> > lists:map(fun({Tab,Attrs}) -> > {attribute,0,record, > {Tab, lists:map(fun(Attr) -> > {record_field,0,{atom,0,Attr}} > end, Attrs)}} > end, TabAttrs). > > >shouldn't it call mnesia:table_info(Tab, record_name) >instead of naively assuming that the record_name is always >the same as the table name? > >3> mnesia:create_table(foo,[{attributes,[a1,a2,a3]},{record_name,rec}]). >{atomic,ok} >4> mnesia:table_info(foo,record_name). >rec >5> mnesia:dirty_write(foo, {foo,1,2,3}). >** exited: {aborted,{bad_type,{foo,1,2,3}}} ** >6> mnesia:dirty_write(foo, {rec,1,2,3}). >ok > > >/Uffe > -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes From raimo.niskanen@REDACTED Wed Jun 18 14:19:00 2003 From: raimo.niskanen@REDACTED (Raimo Niskanen) Date: Wed, 18 Jun 2003 14:19:00 +0200 Subject: Bug ... in Erlang References: Message-ID: These bug will be fixed in R9C: There was bug with list_to_float/1 returning weird floats on some Linuxes and on (all) Windowses. 1> erlang:display(list_to_float("1.0e309")). % Linux inf true 1> erlang:display(list_to_float("1.0e309")). % Windows 1.#INF000e+000 true 2> list_to_float("1.0e309"). 1.00000e+0 Here "inf" and "1.#INF000e+000" are infinity floats (that should not exist in Erlang). You can try to assign them to a variable and performing tests on them. For example list_to_float("1.0e309") > 1.0e308 -> true. But printing with io:format/2 (as the shell does) crashes on Linux and prints 1.00000e+0 on Windows. So, list_to_float/1 has been rewritten to use strtod() (previously atof() on Unixish systems and sscanf("%lf",..) on Windows) and check errors on all platforms. Hopefully no more malicious floats. There was a problem when the shell crashed, which it seldom does. It did when printing infinity floats on Linux. You could type ^G to get to the job handler menu and there start a new shell, but not many knew that. So, the shell handling has been rewritten to automatically start a new shell if it crashes. For snapshot builders and for us internally there was a problem with the new erl_scan: 1> 1.0e309. ** 1: illegal float ** 2> . ** 1: syntax error before: '.' ** The new scanner left unscanned data on the input stream after a scan error. Therefore the second '.' command. So, the new erl_scan has been rewritten to always scan to full stop ('.') while recording errors, and finally check if there were any. Note that you can still do (as before): 1> a. b. a 2> b. b / Raimo Niskanen, Erlang/OTP, Ericsson AB Joe Armstrong wrote: > > Try typing the following: > > > 1.0e309. > > Is this a bug? > > /Joe > From raimo.niskanen@REDACTED Wed Jun 18 14:42:51 2003 From: raimo.niskanen@REDACTED (Raimo Niskanen) Date: Wed, 18 Jun 2003 14:42:51 +0200 Subject: ^G/^C in shell on Windows References: Message-ID: Well, it is kind of a known thing... The command "erl" on windows corresponds to "erl -oldshell" on Unix, and the "-oldshell" option causes ^G to do just what you describes. Note also that almost no command line editing works. This shell can (on Unix) be used against non-terminals e.g pipes. ^C on Unix brings up the break handler menu as you describes, but rumours here say that we could not get the break handler menu to work on Windows so it was disabled. Therefore you exit the emulator, but you should not have gotten a break handler menu. That is probably a bug. / Raimo Niskanen, Erlang/OTP, Ericsson AB. Vlad Dumitrescu wrote: > Hi, > > I noticed that ^G doesn't work for me: what it seems to do is just start > another shell (or restart the other one), because I get the "Eshell > V5.2.3.3" banner and the prompt is reset to 1. > > Pressing ^C on the other hand, prints the "BREAK: (a)bort....." message but > doesn't wait for any input, it just ends execution. Sometimes not even the > message gets printed but the program crashes (there comes a "The memory > could not be read" error), but I think this latter has to do with port > programs running. > > This is in erl.exe, not werl.exe. Werl works fine. > > Should I gather some more data and submit it for analysis, or is it a known > thing? > > regards, > Vlad From ekarttun@REDACTED Wed Jun 18 14:59:17 2003 From: ekarttun@REDACTED (Einar Karttunen) Date: Wed, 18 Jun 2003 15:59:17 +0300 Subject: Bug ... in Erlang In-Reply-To: References: Message-ID: <20030618125917.GA20800@melkinpaasi.cs.Helsinki.FI> On 18.06 14:19, Raimo Niskanen wrote: > So, list_to_float/1 has been rewritten to use strtod() (previously > atof() on Unixish systems and sscanf("%lf",..) on Windows) and check > errors on all platforms. Hopefully no more malicious floats. > I have a patch for replacing various sscanf:s with strtod / strtol, if you are interested. Mainly a cosmetic thing, but should be a tiny bit faster too... ps. if you want it I can send it to the patches list. - Einar Karttunen From Erik.Reitsma@REDACTED Wed Jun 18 09:27:46 2003 From: Erik.Reitsma@REDACTED (Erik Reitsma (ETM)) Date: Wed, 18 Jun 2003 09:27:46 +0200 Subject: xmerl newlines Message-ID: <440A2703A54A8F4FB2AC2AE34F27129D215A93@ESEALNT889.al.sw.ericsson.se> > There are many different requirements... (: > > I understand the problem as being that you've built a > structure of tuples (or #xmlElement{}) in Erlang and want > them exported with some pretty-printing. Is that right? I think that that was James' issue. My "problem" was, that a pretty-printed XML document is not parsed into the same data structure as a non-pretty-printed. So: Some content is not parsed into the same structure as Some content using xmerl_scan:file/1 > I had implemented the {space,...} option wrongly in > xmerl-0.15 and was set straight by those who use and > understand XML. (: However, the option {space,normalize} > _should_ do almost what you want. It will accumulate > consecutive whitespace and replace it with one space. Close > enough? I did not know that this option existed. I had not read the source well enough. This option makes it easier to pick out the parts of empty text and throw them out. Still, it would be easier if all xmlText records with empty text would be removed during parsing, especially if it has "siblings" that are xmlElement records. An option {space,remove} would be nice... Now I have a function that walks through the #xmlElement{} tree and throws out what is the result of pretty printing, which works well enough for me to not start hacking xmerl myself :) *Erik. From joachim.durchholz@REDACTED Wed Jun 18 15:14:59 2003 From: joachim.durchholz@REDACTED (Joachim Durchholz) Date: Wed, 18 Jun 2003 15:14:59 +0200 Subject: Functions in data structures In-Reply-To: References: Message-ID: <3EF065D3.2050104@web.de> Hakan Mattsson wrote: > > I wrote some internal notes about things to think about when > performing changes in the Mnesia application. The notes are > from 1998 and a bit outdated, but I think that they still can > serve as a source for inspiration when you think about > different types of future changes that you need to prepare > your own application for. Thanks, these notes have been most useful. Many (most) of them seem to pertain to the development of Mnesia resp. Erlang itself, is this correct? Regards, Jo From joachim.durchholz@REDACTED Wed Jun 18 15:15:05 2003 From: joachim.durchholz@REDACTED (Joachim Durchholz) Date: Wed, 18 Jun 2003 15:15:05 +0200 Subject: Functions in data structures In-Reply-To: <000d01c33562$47549ae0$53bfb5d4@telia.com> References: <3EEDBC7B.9060807@web.de> <3EEE3AEC.5040600@erlang-consulting.com> <3EEEF1DA.1020405@web.de> <3EEF6AF4.8030001@st.se> <3EEF8B54.5010304@web.de> <000d01c33562$47549ae0$53bfb5d4@telia.com> Message-ID: <3EF065D9.3050200@web.de> Wiger Ulf wrote: > From: "Joachim Durchholz" >>Anyway, I have decided to send just plain data, no functions. Code will >>be sent in the form of software upgrades, just the way that people >>expect things to work. (Sending functions implies all sorts of security >>hassles anyway.) > > You may want to include a version indicator in the messages going > between nodes. There are currently a few things that can mess up > certain code upgrades: > > - record structures are compile-time dependent > - mnesia can not have multiple simultaneous versions of the schema > > Thus, if you send records in messages between nodes, and have a > fully distributed database, it is a bit more difficult to upgrade code > one node at a time (an otherwise attractive option). Fortunately, it's unlikely that full records will ever be sent directly. The general structure will be as follows: One server, distributed over several physical machines. Distribution is both for scalability and redundancy; there is no communication between server machines except Mnesia replication and some load-balancing protocol. The server is intended to run 24/7. Lots of clients, each sitting on one physical machine. Clients interact just with the server, not directly with each other. Clients will run for a short period and terminate. In this scenario, changes in the protocol between client and server are most likely to cause problems. Changes in the Mnesia schema on the server comes next. I've been thinking about adding a version number to each table name, and add a layer that retrieves and converts data from previous-version tables if the current-version table doesn't have the data. Updates simply go to the latest version of the table, resulting in a convert-on-the-fly schema. (The critical point here is conversion, which must be 100% correct - but that would have to be correct whether the conversion is done in batch mode or on the fly.) > The main reason for this rambling is that you need to spend some > time rather early thinking about what your upgrade requirements are, > and how they affect your design choices. I hope the above doesn't contain too many thinkos :-) Regards, Jo From vlad_dumitrescu@REDACTED Wed Jun 18 15:19:46 2003 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Wed, 18 Jun 2003 15:19:46 +0200 Subject: ^G/^C in shell on Windows References: Message-ID: Okay, thanks for the info. I'm testing an app that is most easily restarted by killing the runtime and restarting it, and werl is not so handy from a command prompt. I solved it by letting my app call init:stop() when it ends :-) regards, Vlad ----- Original Message ----- From: "Raimo Niskanen" > Well, it is kind of a known thing... > > The command "erl" on windows corresponds to "erl -oldshell" on Unix, and > the "-oldshell" option causes ^G to do just what you describes. Note > also that almost no command line editing works. This shell can (on Unix) > be used against non-terminals e.g pipes. > > ^C on Unix brings up the break handler menu as you describes, but > rumours here say that we could not get the break handler menu to work on > Windows so it was disabled. Therefore you exit the emulator, but you > should not have gotten a break handler menu. That is probably a bug. > > / Raimo Niskanen, Erlang/OTP, Ericsson AB. > > > > Vlad Dumitrescu wrote: > > Hi, > > > > I noticed that ^G doesn't work for me: what it seems to do is just start > > another shell (or restart the other one), because I get the "Eshell > > V5.2.3.3" banner and the prompt is reset to 1. > > > > Pressing ^C on the other hand, prints the "BREAK: (a)bort....." message but > > doesn't wait for any input, it just ends execution. Sometimes not even the > > message gets printed but the program crashes (there comes a "The memory > > could not be read" error), but I think this latter has to do with port > > programs running. > > > > This is in erl.exe, not werl.exe. Werl works fine. > > > > Should I gather some more data and submit it for analysis, or is it a known > > thing? > > > > regards, > > Vlad > > From fredrik.linder@REDACTED Wed Jun 18 16:29:03 2003 From: fredrik.linder@REDACTED (Fredrik Linder) Date: Wed, 18 Jun 2003 16:29:03 +0200 Subject: xmerl newlines In-Reply-To: <440A2703A54A8F4FB2AC2AE34F27129D215A93@ESEALNT889.al.sw.ericsson.se> Message-ID: > > I had implemented the {space,...} option wrongly in > > xmerl-0.15 and was set straight by those who use and > > understand XML. (: However, the option {space,normalize} > > _should_ do almost what you want. It will accumulate > > consecutive whitespace and replace it with one space. Close > > enough? > > I did not know that this option existed. I had not read the > source well enough. > This option makes it easier to pick out the parts of empty text > and throw them out. Still, it would be easier if all xmlText > records with empty text would be removed during parsing, > especially if it has "siblings" that are xmlElement records. An > option {space,remove} would be nice... > Now I have a function that walks through the #xmlElement{} tree > and throws out what is the result of pretty printing, which works > well enough for me to not start hacking xmerl myself :) Alternatively to traverse the #xmlElement{} tree after parsing you could check this during parsing using the function hooks available in xmerl_scan. Great addition! You then will have the possiblity to change the whole tree if you so desire. Good Luck /Fredrik From etxuwig@REDACTED Wed Jun 18 16:47:36 2003 From: etxuwig@REDACTED (Ulf Wiger) Date: Wed, 18 Jun 2003 16:47:36 +0200 (MEST) Subject: Functions in data structures In-Reply-To: <3EF065D9.3050200@web.de> Message-ID: On Wed, 18 Jun 2003, Joachim Durchholz wrote: >Changes in the Mnesia schema on the server comes next. I've >been thinking about adding a version number to each table >name, and add a layer that retrieves and converts data from >previous-version tables if the current-version table >doesn't have the data. Updates simply go to the latest >version of the table, resulting in a convert-on-the-fly >schema. (The critical point here is conversion, which must >be 100% correct - but that would have to be correct whether >the conversion is done in batch mode or on the fly.) There is a transform_table() function in mnesia, which can be used for making in-service schema updates and converting all affected data within a transaction. This requires the users of the data to be suspended during the transformation, and that the users jump into new versions of the code before they resume. One suggestion is to begin using mnesia through an access module and not using the mnesia functions directly. In fact, the only function you really need to stub is the mnesia:activity() function. If you consistently use your own wrapper function myMod:activity(Type, F) -> mnesia:activity(Type, F). You can later change this function to myMod:activity(Type, F) -> mnesia:activity(Type, F, [], myAccessModule) where you can do all sorts of esoteric stuff in myAccessModule (the rdbms contribution uses this method) *** Example: In the event of really tricky upgrades, you can use some clever tricks in this access module, basically hiding the nastiness from the applications. I've attached what should be regarded as a prototype attempt at handling redundancy reboot upgrades including schema changes to mnesia. The idea is that one begins the upgrade by loading an "upgrade module" on all nodes, containing no other functionality than such that prepares for the upgrade. The included mnesia access module provides a way to upgrade a table definition using a forward transform for already upgraded nodes, and a backward transform for not yet upgraded nodes. I've tested it for simple cases. It's never been used in a real live upgrade, but it perhaps illustrates the potential. /Uffe -- Ulf Wiger, Senior Specialist, / / / Architecture & Design of Carrier-Class Software / / / Strategic Product & System Management / / / Ericsson AB, Connectivity and Control Nodes -------------- next part -------------- %%% The contents of this file are subject to the Erlang Public License, %%% Version 1.0, (the "License"); you may not use this file except in %%% compliance with the License. You may obtain a copy of the License at %%% http://www.erlang.org/license/EPL1_0.txt %%% %%% Software distributed under the License is distributed on an "AS IS" %%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %%% the License for the specific language governing rights and limitations %%% under the License. %%% %%% The Original Code is sysDb-0.1 %%% %%% The Initial Developer of the Original Code is Ericsson Telecom %%% AB. Portions created by Ericsson are Copyright (C), 1998, Ericsson %%% Telecom AB. All Rights Reserved. %%% %%% Contributor(s): %%% %%% #0. BASIC INFORMATION %%% ---------------------------------------------------------- %%% %CCaseFile: sysDb.erl % %%% Author: Ulf Wiger %%% Description: mnesia rwapper for enhanced upgrade support %%% %%% Modules used: mnesia %%% %%% ---------------------------------------------------------- -module(sysDb). -id('94/190 55-CNA 121 70 Ux'). -vsn('/main/R7A/10'). -date('01-04-11'). -ifdef(debug). -define(dbg(Fmt, Args), io:format("~p-~p: " ++ Fmt, [?MODULE,?LINE|Args])). -else. -define(dbg(Fmt,Args), no_debug). -endif. %%% #2. EXPORT LISTS %%% ---------------------------------------------------------- %%% #2.1 EXPORTED INTERFACE FUNCTIONS %%% ---------------------------------------------------------- %% Substitute for mnesia:transaction/1. -export([transaction/1, activity/2]). -export([begin_upgrade/0, activate_new_tables/0, end_upgrade/0]). -export([table_transform/4, table_transform/5]). -export([init_tables/0]). %% Update -export([lock/4, write/5, delete/5, delete_object/5, read/5, match_object/5, all_keys/4, index_match_object/6, index_read/6, table_info/4]). %%% ---------------------------------------------------------- %%% #2.2 EXPORTED INTERNAL FUNCTIONS %%% ---------------------------------------------------------- -define(TEMP, sysDbTemp). -define(TEMP_BAG, sysDbTempBag). -define(LOCAL, sysDb_local). -define(GLOBALS, sysDbGlobals). -record(?TEMP, {key, value}). -record(?TEMP_BAG, {key, value}). -record(?LOCAL, {key, value}). -record(?GLOBALS, {key, value}). %%% #3. CODE %%% #--------------------------------------------------------- %%% #3.1 CODE FOR EXPORTED INTERFACE FUNCTIONS %%% #--------------------------------------------------------- %%%---------------------------------------------------------------------- %%% -type activity(Fun : function())-> %%% ResultFromFun. %%% Input: Function object for Mnesia transaction %%% Output: Result from %%% Exceptions: EXIT if transaction aborted %%% Description: This is a wrapper around mnesia:activity/1. %%% It starts a mnesia activity with this module as the activity %%% module. This enables all SYSDB integrity checks. %%%---------------------------------------------------------------------- transaction(Fun) -> case is_upgrade() of false -> mnesia:transaction(Fun); true -> case catch mnesia:activity(transaction, Fun, [], ?MODULE) of {'EXIT', Reason} -> {aborted, Reason}; Result -> {atomic, Result} end end. activity(Type, Fun) -> case is_upgrade() of false -> mnesia:activity(Type, Fun); true -> mnesia:activity(Type, Fun, [], ?MODULE) end. begin_upgrade() -> mnesia:activity(sync_dirty, fun() -> mnesia:write(#?GLOBALS{key = is_upgrade, value = true}) end). table_transform(Tab, FwdFun, BwdFun, NewAttributeList) -> RecName = mnesia:table_info(Tab, record_name), table_transform(Tab, FwdFun, BwdFun, NewAttributeList, RecName). table_transform(Tab, FwdFun, BwdFun, NewAttributeList, NewRecordName) -> Key = {conversion, Tab}, Val = {FwdFun, BwdFun, NewAttributeList, NewRecordName}, mnesia:activity(sync_dirty, fun() -> mnesia:write(#?GLOBALS{key = Key, value = Val}) end). activate_new_tables() -> Tabs = ets:match_object(?GLOBALS, #?GLOBALS{key = {conversion, '_'}, value = '_'}), lists:foreach( fun(#?GLOBALS{key = {conversion, Tab}}) -> mnesia:dirty_write(#?LOCAL{key = {version, Tab}, value = new}) end, Tabs). end_upgrade() -> %% We split the conversion/copy operations into multiple transactions %% In order to make sure there are no updates in the meantime. %% Currently, no action is taken to ensure that already started %% activities actually finish before we commit the table transforms. %% however, chances are high that ets, dirty, and sync_dirty operations %% are lightweight enough that they will finish well before the transforms %% below are activated. Regular transactions are trickier, since they %% *might* end up waiting behind on of our table locks, and then continue %% believing that an upgrade is still in progress. We should deal with %% this in lock() below... exactly how, I'm not sure. mnesia:activity(sync_dirty, fun() -> mnesia:write(#?GLOBALS{key = is_upgrade, value = wait}) end), Tabs = ets:match_object(?GLOBALS, #?GLOBALS{key = {conversion, '_'}, value = '_'}), lists:foreach( fun(#?GLOBALS{key = {conversion, Tab}, value = {FwF, BwF, NewAttrs, NewRecName}}) -> {atomic, ok} = mnesia:transform_table(Tab, FwF, NewAttrs, NewRecName) end, Tabs), ok=io:format("transforms complete~n", []), lists:foreach( fun(#?GLOBALS{key = {conversion, Tab}, value = {FwF, BwF, NewAttrs, NewRecName}}) -> F = fun() -> mnesia:delete({?GLOBALS, {conversion, Tab}}), copy_new_objs(Tab) end, mnesia:activity(transaction, F) end, Tabs), ok=io:format("copy complete~n", []), mnesia:activity(sync_dirty, fun() -> mnesia:delete({?GLOBALS, is_upgrade}) end). init_tables() -> Nodes = mnesia:system_info(running_db_nodes), create_table(?LOCAL, [{ram_copies, Nodes}, {type, set}, {local_content, true}, {attributes, record_info(fields, ?LOCAL)}]), create_table(?TEMP, [{ram_copies, Nodes}, {type, set}, {attributes, record_info(fields, ?TEMP)}]), create_table(?TEMP_BAG, [{ram_copies, Nodes}, {type, bag}, {attributes, record_info(fields, ?TEMP_BAG)}]), create_table(?GLOBALS, [{ram_copies, Nodes}, {type, set}, {attributes, record_info(fields, ?GLOBALS)}]). %% mnesia callbacks ===================================== %% for each callback, an internal function is implemented. %% the internal functions are not exported, since they are only %% meant to be used for operations generated by the dictionary. %% No integrity checks are performed on the internal functions. lock(ActivityId, Opaque, LockItem, LockKind) -> mnesia:lock(ActivityId, Opaque, LockItem, LockKind). write(ActivityId, Opaque, Tab, Rec, LockKind) -> case table_conversion_type(Tab) of none -> mnesia:write(ActivityId, Opaque, Tab, Rec, LockKind); {Version, FwdF, BwdF} -> {OldRec, NewRec} = case Version of old -> {Rec, FwdF(Rec)}; new -> {BwdF(Rec), Rec} end, TempKey = {_, RecKey} = temp_key(Tab, Rec), case mnesia:table_info(Tab, type) of bag -> %% in order to implement some of the other access %% functions relatively safely, we must move all %% objects with the same key to ?TEMP_BAG case mnesia:read(ActivityId, Opaque, ?TEMP_BAG, RecKey, LockKind) of [] -> OldObjs = mnesia:read(ActivityId, Opaque, Tab, RecKey, LockKind), lists:foreach( fun(X) -> TempKeyX = temp_key(Tab, X), mnesia:write(ActivityId, Opaque, ?TEMP_BAG, {TempKeyX, X}, LockKind) end, OldObjs); _ -> %% This has been done previously ok end, mnesia:write(ActivityId, Opaque, ?TEMP_BAG, {TempKey, NewRec}, LockKind); _ -> mnesia:write(ActivityId, Opaque, ?TEMP, #?TEMP{key = TempKey, value = NewRec}, LockKind) end, mnesia:write(ActivityId, Opaque, Tab, OldRec, LockKind) end. delete(ActivityId, Opaque, Tab, Key, LockKind) -> case table_conversion_type(Tab) of none -> mnesia:delete(ActivityId, Opaque, Tab, Key, LockKind); _ -> case mnesia:table_info(Tab, type) of bag -> mnesia:delete(ActivityId, Opaque, ?TEMP_BAG, {Tab, Key}, LockKind); _ -> %% set or ordered_set mnesia:delete(ActivityId, Opaque, ?TEMP, {Tab, Key}, LockKind) end, mnesia:delete(ActivityId, Opaque, Tab, Key, LockKind) end. delete_object(ActivityId, Opaque, Tab, Rec, LockKind) -> %% WARNING!!! %% We cannot be sure that a transform hits the target, since %% attributes which have been added/removed are most likely initialized %% with default values in one direction or the other. Thus, an object %% written during the upgrade may not be delete:able with this function. case table_conversion_type(Tab) of none -> mnesia:delete_object(ActivityId, Opaque, Tab, Rec, LockKind); {new, FwF, BwF} -> NewRec = FwF(Rec), Key = temp_key(Tab, Rec), mnesia:delete_object(ActivityId, Opaque, Tab, Rec, LockKind), TempTab = case mnesia:table_info(Tab, type) of bag -> ?TEMP_BAG; _ -> ?TEMP end, mnesia:delete_object(ActivityId, Opaque, TempTab, {Key, NewRec}, LockKind); {old, FwF, BwF} -> OldRec = BwF(Rec), Key = temp_key(Tab, Rec), mnesia:delete_object(ActivityId, Opaque, Tab, OldRec, LockKind), TempTab = case mnesia:table_info(Tab, type) of bag -> ?TEMP_BAG; _ -> ?TEMP end, mnesia:delete_object(ActivityId, Opaque, TempTab, {Key, Rec}, LockKind) end. read(ActivityId, Opaque, Tab, Key, LockKind) -> case table_conversion_type(Tab) of none -> mnesia:read(ActivityId, Opaque, Tab, Key, LockKind); {Version, FwF, BwF} -> case mnesia:table_info(Tab, type) of bag -> case mnesia:read(ActivityId, Opaque, ?TEMP_BAG, {Tab, Key}, LockKind) of [] -> mnesia:read(ActivityId, Opaque, Tab, Key, LockKind); Objs -> %% see the implementation of write() -- we know %% that the ?TEMP_BAG table holds all objects %% with Key. case Version of new -> [O || #?TEMP_BAG{value = O} <- Objs]; old -> [BwF(O) || #?TEMP_BAG{value = O} <- Objs] end end; _ -> case mnesia:read(ActivityId, Opaque, ?TEMP, {Tab, Key}, LockKind) of [] -> case {mnesia:read(ActivityId, Opaque, Tab, Key, LockKind), Version} of {[], _} -> []; {[Obj], new} -> [FwF(Obj)]; {[Obj], old} -> [Obj] end; [#?TEMP{value = Obj}] -> case Version of new -> [Obj]; old -> [BwF(Obj)] end end end end. match_object(ActivityId, Opaque, Tab, Pattern, LockKind) -> case table_conversion_type(Tab) of none -> mnesia:match_object(ActivityId, Opaque, Tab, Pattern, LockKind); {old, _, _} -> mnesia:match_object(ActivityId, Opaque, Tab, Pattern, LockKind); {new, FwF, BwF} -> match_object1(ActivityId, Opaque, Tab, Pattern, LockKind) end. match_object1(ActivityId, Opaque, Tab, Pattern, LockKind) -> case is_var(Pattern) of true -> %% must search whole table search_whole_tab(ActivityId, Opaque, Tab, Pattern, LockKind); false -> KeyPos = mnesia:table_info(Tab, keypos), if size(Pattern) >= KeyPos -> Key = element(KeyPos, Pattern), case is_ground(Key) of true -> Objs = read(ActivityId, Opaque, Tab, Key, LockKind), match_objs(Objs, Pattern); false -> %% must search whole table search_whole_tab(ActivityId, Opaque, Tab, Pattern, LockKind) end; true -> [] end end. match_objs([], _) -> []; match_objs(Objs, Pattern) -> Tab = ets:new(match_tab, [set]), MatchPat = {x, Pattern}, match_objs(Objs, Tab, MatchPat). match_objs([], Tab, Pattern) -> []; match_objs([O|Objs], Tab, Pattern) -> %% It's not trivial to write an Erlang version of the ETS pattern match, %% so I cheat and do this instead. ets:insert(Tab, {x, O}), case ets:match_object(Tab, Pattern) of [] -> match_objs(Objs, Tab, Pattern); [_] -> [O|match_objs(Objs, Tab, Pattern)] end. search_whole_tab(ActivityId, Opaque, Tab, Pattern, LockKind) -> ETS = ets:new(match_tab, [set]), MatchPat = {x, Pattern}, Keys = mnesia:all_keys(ActivityId, Opaque, Tab, ETS, Pattern, MatchPat, LockKind), search_whole_tab(Keys, ActivityId, Opaque, Tab, ETS, Pattern, MatchPat, LockKind). search_whole_tab([K|Keys], ActivityId, Opaque, Tab, ETS, Pattern, MatchPat, LockKind) -> Objs = read(ActivityId, Opaque, Tab, K, LockKind), MatchedObjs = match_objs(Objs, ETS, MatchPat), MatchedObjs ++ search_whole_tab(Keys, ActivityId, Opaque, Tab, ETS, Pattern, MatchPat, LockKind); search_whole_tab([], ActivityId, Opaque, Tab, ETS, Pattern, MatchPat, LockKind) -> []. %% is_var(Atom) -> bool(). is_var(P) when atom(P) -> case atom_to_list(P) of [$_] -> true; [$$|Cs] -> digits(Cs); Other -> false end; is_var(P) -> false. digits([C|Cs]) when integer(C), C >= $0, C =< $9 -> digits(Cs); digits([C|Cs]) -> false; digits([]) -> true. %% is_ground(Term) -> bool(). is_ground([H|T]) -> case is_ground(H) of true -> is_ground(T); false -> false end; is_ground([]) -> true; is_ground(P) when tuple(P) -> is_ground_tuple(P, 1, size(P)); is_ground(P) -> not is_var(P). is_ground_tuple(P, I, Size) when I > Size -> true; is_ground_tuple(P, I, Size) -> case is_ground(element(I, P)) of true -> is_ground_tuple(P, I+1, Size); false -> false end. %%% ================== The following callback functions have not been %%% ================== dealt with yet. all_keys(ActivityId, Opaque, Tab, LockKind) -> mnesia:all_keys(ActivityId, Opaque, Tab, LockKind). index_match_object(ActivityId, Opaque, Tab, Pattern, Attr, LockKind) -> mnesia:index_match_object(ActivityId, Opaque, Tab, Pattern, Attr, LockKind). index_read(ActivityId, Opaque, Tab, SecondaryKey, Attr, LockKind) -> mnesia:index_read(ActivityId, Opaque, Tab, SecondaryKey, Attr, LockKind). table_info(ActivityId, Opaque, Tab, InfoItem) -> mnesia:table_info(ActivityId, Opaque, Tab, InfoItem). %%% #--------------------------------------------------------- %%% #3.2 CODE FOR EXPORTED INTERNAL FUNCTIONS %%% #--------------------------------------------------------- %%% #--------------------------------------------------------- %%% #3.3 CODE FOR INTERNAL FUNCTIONS %%% #--------------------------------------------------------- temp_key(Tab, Obj) -> {Tab, key_of(Tab, Obj)}. key_of(Tab, Obj) -> Pos = mnesia:table_info(Tab, keypos), element(Pos, Obj). table_conversion_type(Tab) -> case ets:lookup(?GLOBALS, {conversion, Tab}) of [] -> none; [#?GLOBALS{value = {FwF, BwF, NewAttrs, NewRecName}}] -> Version = case ets:lookup(?LOCAL, {version, Tab}) of [] -> %% we're still using the old old; [#?LOCAL{value = new}] -> new end, {Version, FwF, BwF} end. copy_new_objs(Tab) -> case mnesia:table_info(Tab, type) of bag -> Pat = #?TEMP_BAG{key = {Tab, '_'}, value = '_'}, NewObjs = ets:match_object(?TEMP_BAG, Pat), Buf = ets:new(buf, [set]), KeyPos = mnesia:table_info(Tab, keypos), copy_bag_objs(NewObjs, Tab, KeyPos, Buf), ets:delete(Buf); _ -> %% ordered_set or set Pat = #?TEMP{key = {Tab, '_'}, value = '_'}, NewObjs = ets:match_object(?TEMP, Pat), lists:foreach( fun(#?TEMP{key = TempKey, value = Obj}) -> mnesia:delete({?TEMP, TempKey}), mnesia:write(Obj) end, NewObjs) end. copy_bag_objs([#?TEMP_BAG{key=TempKey, value=Obj}|Objs], Tab, KeyPos, Buf) -> Key = element(KeyPos, Obj), case ets:lookup(Buf, Key) of [] -> ets:insert(Buf, {Key, 1}), mnesia:delete({Tab, Key}), mnesia:delete({?TEMP_BAG, TempKey}), mnesia:write(Obj); _ -> mnesia:write(Obj) end, copy_bag_objs(Objs, Tab, KeyPos, Buf); copy_bag_objs([], Tab, KeyPos, Buf) -> ok. create_table(Tab, Opts) -> case mnesia:create_table(Tab, Opts) of {atomic, ok} -> ok; Other -> exit(Other) end. is_upgrade() -> case ets:lookup(?GLOBALS, is_upgrade) of [] -> false; [#?GLOBALS{value = wait}] -> receive after 100 -> is_upgrade() end; [#?GLOBALS{value = true}] -> true end. %%%---------------------------------------------------------------------- %%% #3.3.1 Code for Additional Actions (post-transaction triggers). %%%---------------------------------------------------------------------- %%% #4 CODE FOR TEMPORARY CORRECTIONS %%% #--------------------------------------------------------- From rvg@REDACTED Wed Jun 18 17:09:32 2003 From: rvg@REDACTED (Rudolph van Graan) Date: Wed, 18 Jun 2003 17:09:32 +0200 Subject: Getting data out of a closing socket Message-ID: <1F42BB97D787C048BCC519FF28048AC367E6@na.sdn.net.za> Hi all, I have a bit of a strange situation at hand. A socket in active, packet mode: {ok, Sock} = gen_tcp:connect(IP, Port, [list, {packet, 4},{active,true}]), Basically, I send a request on the socket to the other side gen_tcp:send(Sock,Datagram) Decoding events from the socket like this [hiding some other unimportant things like closing the socket etc]: run(Sock) -> receive What -> io:format("Recv ~w\n",[What]), run(Sock) after 1000 -> run(Sock) end; So, I only get this from the socket: Recv {tcp_closed,#Port<0.140>} Looking at a packet trace on our firewall, I can see data arriving before the close. I would have expected two messages here, first the data and then the close. I have also tried coding this some routing with a passive socket, same problem... Any ideas? Thanks, Rudolph From raimo.niskanen@REDACTED Wed Jun 18 20:33:49 2003 From: raimo.niskanen@REDACTED (Raimo Niskanen) Date: Wed, 18 Jun 2003 20:33:49 +0200 Subject: Bug ... in Erlang References: , , <20030618125917.GA20800@melkinpaasi.cs.Helsinki.FI> Message-ID: Why not, go ahead. It will not be included in R9C, though. / Raimo Niskanen, Erlang/OTP, Ericsson AB Einar Karttunen wrote: > On 18.06 14:19, Raimo Niskanen wrote: > >>So, list_to_float/1 has been rewritten to use strtod() (previously >>atof() on Unixish systems and sscanf("%lf",..) on Windows) and check >>errors on all platforms. Hopefully no more malicious floats. >> > > > I have a patch for replacing various sscanf:s with strtod / strtol, > if you are interested. Mainly a cosmetic thing, but should be a > tiny bit faster too... > > ps. if you want it I can send it to the patches list. > > - Einar Karttunen From per@REDACTED Wed Jun 18 21:02:58 2003 From: per@REDACTED (Per Bergqvist) Date: Wed, 18 Jun 2003 22:02:58 +0300 Subject: Getting data out of a closing socket In-Reply-To: <1F42BB97D787C048BCC519FF28048AC367E6@na.sdn.net.za> Message-ID: <200306182002.h5IK2wH25243@tessla.levonline.com> Hi Dolph, which erl release and os are you using ? /Per (good to see that erlang is live and kicking in za) ------------------- > Hi all, > > I have a bit of a strange situation at hand. A socket in active, packet > mode: > > {ok, Sock} = gen_tcp:connect(IP, Port, [list, {packet, > 4},{active,true}]), > > Basically, I send a request on the socket to the other side > > gen_tcp:send(Sock,Datagram) > > Decoding events from the socket like this [hiding some other unimportant > things like closing the socket etc]: > > run(Sock) -> > receive > What -> > io:format("Recv ~w\n",[What]), > run(Sock) > after > 1000 -> > run(Sock) > end; > > > So, > > I only get this from the socket: > > Recv {tcp_closed,#Port<0.140>} > > Looking at a packet trace on our firewall, I can see data arriving > before the close. I would have expected two messages here, first the > data and then the close. I have also tried coding this some routing with > a passive socket, same problem... Any ideas? > > Thanks, > > Rudolph > ========================================================= Per Bergqvist Synapse Systems AB Phone: +46 709 686 685 Email: per@REDACTED From Erik.Reitsma@REDACTED Thu Jun 19 09:11:31 2003 From: Erik.Reitsma@REDACTED (Erik Reitsma (ETM)) Date: Thu, 19 Jun 2003 09:11:31 +0200 Subject: Small bug in asn1 Message-ID: <440A2703A54A8F4FB2AC2AE34F27129D215A98@ESEALNT889.al.sw.ericsson.se> Hi, I tried to compile an ASN.1 file (using ber), and got a compiler error. It was caused by the following bug. In asn1ct_gen_ber_bin.erl, asn1ct_gen_ber.erl and asn1rt_ber_bin.erl the tag 'ObjectDescriptor' is mistakenly called 'OBJECT DESCRIPTOR'. Replacing 'OBJECT DESCRIPTOR' by 'ObjectDescriptor' in these three files (all files where it occurs), made it possible to compile the ASN.1 and use the generated code to decode messages correctly. Regards, *Erik. From jay@REDACTED Fri Jun 20 06:28:47 2003 From: jay@REDACTED (Jay Nelson) Date: Thu, 19 Jun 2003 21:28:47 -0700 Subject: BSD Firewall Message-ID: <4.2.2.20030619212337.00d261a0@duomark.com> I got my first fanless PCs to use as game servers. I have 3 machines and will make one a dedicated firewall and the other two load-balanced web / game servers. I will be running from flash memory and RAM, but will network to a bigger machine to hold any DB or log info. I am thinking to make the firewall machine a BSD setup (and probably the others as well), but have been using Linux up to now. A few questions for others more familiar with BSD: 1) Which should I use? NetBSD, FreeBSD or OpenBSD and why? I am inclined right now to use the one with the best tutorial on how to set up a diskless firewall. 2) Does erlang run better or differently on any of the three? 3) Who out there is using erlang + BSD and what kind of experiences have they encountered? 4) Any tricks to compiling erlang for these OSes? jay From ekarttun@REDACTED Fri Jun 20 07:47:07 2003 From: ekarttun@REDACTED (Einar Karttunen) Date: Fri, 20 Jun 2003 08:47:07 +0300 Subject: BSD Firewall In-Reply-To: <4.2.2.20030619212337.00d261a0@duomark.com> References: <4.2.2.20030619212337.00d261a0@duomark.com> Message-ID: <20030620054707.GA24353@melkki.cs.Helsinki.FI> On 19.06 21:28, Jay Nelson wrote: > 1) Which should I use? NetBSD, FreeBSD or OpenBSD > and why? I am inclined right now to use the one with the > best tutorial on how to set up a diskless firewall. That is ultimately a matter of taste imho: - NetBSD is the most elegant one - OpenBSD is quite focused on security - Setting things up initially is often easiest on FreeBSD Unless you have prior experience or feel like learning things I would go with FreeBSD... - Einar Karttunen From rvg@REDACTED Fri Jun 20 10:33:32 2003 From: rvg@REDACTED (Rudolph van Graan) Date: Fri, 20 Jun 2003 10:33:32 +0200 Subject: Efficient bitwise operations Message-ID: <1F42BB97D787C048BCC519FF28048AC367E9@na.sdn.net.za> Hi there again, Thanks again for all the answers previously - helped me a lot! I've had to write some bitwise operations yesterday, because I couldn't locate any bifs or operations that do the same: Bitwise roll left: lroll1(<>) -> <>. rroll(<>,Rolls) when Rolls > 0 -> Output = rroll1(<>), rroll(Output,Rolls-1); rroll(Byte,0) -> Byte. Bitwise roll right: lroll(<>,Rolls) when Rolls > 0 -> Output = lroll1(<>), lroll(Output,Rolls-1); lroll(Byte,0) -> rroll1(<>) -> <>. Bitwise high <-> low nibble swap: swap(<>) -> <>. My question here has to do with the efficiency of the above code. Normally, I would have used a single assembly instruction to achieve the same result, but even though the code works, how efficient can it really be? Are there any obvious things I've missed in the documentation? Thanks again! Rudolph From Chandrashekhar.Mullaparthi@REDACTED Fri Jun 20 11:52:15 2003 From: Chandrashekhar.Mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Fri, 20 Jun 2003 10:52:15 +0100 Subject: Efficient bitwise operations Message-ID: You can treat each byte as an integer and do bsl - bit shift left bsr - bit shift right These operators are described in the first chapter of the red book. 11> 1 bsl 1. 2 12> 1 bsl 2. 4 13> 4 bsr 1. 2 14> 4 bsr 2. 1 To swap nibbles swap(Int) -> ((Int band 15) bsl 4) + ((Int band 240) bsr 4). cheers Chandru -----Original Message----- From: Rudolph van Graan [mailto:rvg@REDACTED] Sent: 20 June 2003 09:34 To: erlang-questions@REDACTED Subject: Efficient bitwise operations Hi there again, Thanks again for all the answers previously - helped me a lot! I've had to write some bitwise operations yesterday, because I couldn't locate any bifs or operations that do the same: Bitwise roll left: lroll1(<>) -> <>. rroll(<>,Rolls) when Rolls > 0 -> Output = rroll1(<>), rroll(Output,Rolls-1); rroll(Byte,0) -> Byte. Bitwise roll right: lroll(<>,Rolls) when Rolls > 0 -> Output = lroll1(<>), lroll(Output,Rolls-1); lroll(Byte,0) -> rroll1(<>) -> <>. Bitwise high <-> low nibble swap: swap(<>) -> <>. My question here has to do with the efficiency of the above code. Normally, I would have used a single assembly instruction to achieve the same result, but even though the code works, how efficient can it really be? Are there any obvious things I've missed in the documentation? Thanks again! Rudolph NOTICE AND DISCLAIMER: This email (including attachments) is confidential. If you have received this email in error please notify the sender immediately and delete this email from your system without copying or disseminating it or placing any reliance upon its contents. We cannot accept liability for any breaches of confidence arising through use of email. Any opinions expressed in this email (including attachments) are those of the author and do not necessarily reflect our opinions. We will not accept responsibility for any commitments made by our employees outside the scope of our business. We do not warrant the accuracy or completeness of such information. From enano@REDACTED Fri Jun 20 19:46:02 2003 From: enano@REDACTED (Miguel Barreiro Paz) Date: Fri, 20 Jun 2003 19:46:02 +0200 (CEST) Subject: ia64 Message-ID: I had asked here some time ago if erlang would run on ia64, and got no answers. I have just had access to an ia64 box, and: Erlang (BEAM) emulator version 5.2.3.3 [64-bit] [source] Eshell V5.2.3.3 (abort with ^G) 1> P=open_port("/proc/cpuinfo",[stream,in]). 2> receive X -> X after 0 -> ok end. {#Port<0.26>, {data,"processor : 0\nvendor : GenuineIntel\narch : IA-64\nfamily : Itanium 2\nmodel : 0\nrevision : 7\narchrev : 0\nfeatures : branchlong\ncpu number : 0\ncpu regs : 4\ncpu MHz : 900.000000\nitc MHz : 900.000000\nBogoMIPS : 1346.37\n\nprocessor : 1\nvendor : GenuineIntel\narch : IA-64\nfamily : Itanium 2\nmodel : 0\nrevision : 7\narchrev : 0\nfeatures : branchlong\ncpu number : 0\ncpu regs : 4\ncpu MHz : 900.000000\nitc MHz : 900.000000\nBogoMIPS : 1346.37\n\n"}} The only glitch: for some reason configure identifies the system as ia64-unknown-linux-gnu while make expects ia64-unknown-linux. I just symlinked them everywhere - of course the correct action would be to find the reason why configure chooses the wrong name and fix it. Also, compilation stopped in the middle of gs, at: erl -pa ../ebin -s gs_make -s erlang halt -noshell (sleeping forever). I killed it and restarted the compilation, and that same line worked the second time. I haven't tested it heavily yet, but trivial tests seem to work. Regards, Miguel From olgeni@REDACTED Fri Jun 20 22:56:27 2003 From: olgeni@REDACTED (Jimmy Olgeni) Date: Fri, 20 Jun 2003 22:56:27 +0200 (CEST) Subject: Getting data out of a closing socket In-Reply-To: <200306182002.h5IK2wH25243@tessla.levonline.com> References: <200306182002.h5IK2wH25243@tessla.levonline.com> Message-ID: <20030620225443.R70423@olgeni.olgeni> Hi! On Wed, 18 Jun 2003, Per Bergqvist wrote: > which erl release and os are you using ? I'm having the same problem using R9B-1 (5.2.3.3) and a localhost socket :) -- jimmy From eleberg@REDACTED Sat Jun 21 09:25:48 2003 From: eleberg@REDACTED (Bengt Kleberg) Date: Sat, 21 Jun 2003 09:25:48 +0200 (MEST) Subject: BSD Firewall Message-ID: <200306210725.h5L7Pmq13482@cbe.ericsson.se> > X-Sender: jay-dm@REDACTED > Date: Thu, 19 Jun 2003 21:28:47 -0700 > To: erlang-questions@REDACTED > From: Jay Nelson > Subject: BSD Firewall .deleted > 1) Which should I use? NetBSD, FreeBSD or OpenBSD > and why? I am inclined right now to use the one with the > best tutorial on how to set up a diskless firewall. openbsd has as (one of) its design gooals to be a very secure system, out-of-the-box. which is a good thing for a firewall. on the other hand, if you can not find a tutorial for configuring it according to your wishes, is worthless. perhaps a good idea would be to find the tutorial first and choose the os after that? ...deleted > 3) Who out there is using erlang + BSD and what kind of > experiences have they encountered? i used erlang on openbsd around 1999. it just worked. > 4) Any tricks to compiling erlang for these OSes? at the time it compiled without tricks. bengt From roussel@REDACTED Sat Jun 21 18:04:12 2003 From: roussel@REDACTED (=?ISO-8859-1?Q?Roussel_J=E9r=F4me?=) Date: Sat, 21 Jun 2003 18:04:12 +0200 Subject: moving processes from node to node Message-ID: Hello, I know that there are a lot of talks about process migration. This is not really the point of my post. I would like to know something else : - is it possible to transfer a whole application (with all its modules) from one computer (using one erlang node) to another computer (with another erlang node)? I don't talk about process migration because I only want to copy all the files of an application from the hard drive of computer A to put all theses files onto the hard drive of computer B - is it possible next to check if all the needed modules are available on the computer B? - if all this is possible, is it possible, in one word, to move one application from one node to another (without doing it by hand) ??? Thanks for every reply, JrSky From eleberg@REDACTED Mon Jun 23 08:19:16 2003 From: eleberg@REDACTED (Bengt Kleberg) Date: Mon, 23 Jun 2003 08:19:16 +0200 (MEST) Subject: moving processes from node to node Message-ID: <200306230619.h5N6JGq28904@cbe.ericsson.se> > X-Injected-Via-Gmane: http://gmane.org/ > To: erlang-questions@REDACTED > From: Roussel J?r?me > Subject: moving processes from node to node > - is it possible to transfer a whole application (with all its modules) > from one computer (using one erlang node) to another computer (with > another erlang node)? I don't talk about process migration because I > only want to copy all the files of an application from the hard drive of > computer A to put all theses files onto the hard drive of computer B > > - is it possible next to check if all the needed modules are available > on the computer B? > > - if all this is possible, is it possible, in one word, to move one > application from one node to another (without doing it by hand) ??? ''by hand''? does this mean ''sneaker net'' (manually carrying the files on some media), or would ''scp '' also be ''by hand''? while waiting for the high tech solution (presumably there is already some otp module that does this) i have the following (bfmi) suggestion: 1 use the code module to find the modules (code:which/1) 2 use the file module to read them (file:read_file/1) 3 (possibly use rpc:call/4 with file:make_dir/1 to create correct directory on the other node) 4 using the binaries from step 2, write them on the other node (rpc:call/4 with file:write_file/2) bengt From matthias@REDACTED Mon Jun 23 09:53:36 2003 From: matthias@REDACTED (Matthias Lang) Date: Mon, 23 Jun 2003 09:53:36 +0200 Subject: Efficient bitwise operations In-Reply-To: <1F42BB97D787C048BCC519FF28048AC367E9@na.sdn.net.za> References: <1F42BB97D787C048BCC519FF28048AC367E9@na.sdn.net.za> Message-ID: <16118.45568.673524.869365@antilipe.corelatus.se> Rudolph van Graan writes: > I've had to write some bitwise operations yesterday, because I couldn't > locate any bifs or operations that do the same: Bengt has already mentioned bsl/bsr. Also, writing > lroll1(<>) -> > <>. works, but is rather more verbose than lr(<>) -> <>. the latter also generates shorter BEAM assembly (try the 'S' option when you compile). > My question here has to do with the efficiency of the above code. > Normally, I would have used a single assembly instruction to achieve the > same result, but even though the code works, how efficient can it really > be? Are there any obvious things I've missed in the documentation? The pragmatic approach is: if you are thinking about "how many assembly instructions does this bit manipulation turn into", then it's time to move the code to something more suited to the task, such as C or assembler. Either that, or stop thinking about it ;-) There are several ways of interfacing C code to Erlang, each with its own convenience/performance tradeoffs. They're described at http://www.erlang.org/doc/r9b/doc/tutorial/part_first.html Matthias From per.gustafsson@REDACTED Sun Jun 22 22:51:14 2003 From: per.gustafsson@REDACTED (Per Gustafsson) Date: Sun, 22 Jun 2003 22:51:14 +0200 (MET DST) Subject: Efficient bitwise operations In-Reply-To: <1F42BB97D787C048BCC519FF28048AC367E9@na.sdn.net.za> References: <1F42BB97D787C048BCC519FF28048AC367E9@na.sdn.net.za> Message-ID: There are better ways to implement rroll, lroll and swap lroll(Byte, Rolls) -> RestSize = 8-Rolls, <> = Byte <>. rroll(Byte, Rolls) -> FirstSize = 8-Rolls, <> = Byte, <>. or rroll(Byte, Rolls) -> lroll(Byte, 8-Rolls). swap(<>) -> <> Per On Fri, 20 Jun 2003, Rudolph van Graan wrote: > Hi there again, > > Thanks again for all the answers previously - helped me a lot! > > I've had to write some bitwise operations yesterday, because I couldn't > locate any bifs or operations that do the same: > > Bitwise roll left: > > lroll1(<>) -> > <>. > > rroll(<>,Rolls) when Rolls > 0 -> > Output = rroll1(<>), > rroll(Output,Rolls-1); > > rroll(Byte,0) -> > Byte. > > > Bitwise roll right: > > lroll(<>,Rolls) when Rolls > 0 -> > Output = lroll1(<>), > lroll(Output,Rolls-1); > > lroll(Byte,0) -> > > rroll1(<>) -> > <>. > > Bitwise high <-> low nibble swap: > > swap(<>) -> > <>. > > My question here has to do with the efficiency of the above code. > Normally, I would have used a single assembly instruction to achieve the > same result, but even though the code works, how efficient can it really > be? Are there any obvious things I've missed in the documentation? > > Thanks again! > > Rudolph > From fillemanjong@REDACTED Mon Jun 23 07:51:25 2003 From: fillemanjong@REDACTED (fille manjong) Date: Mon, 23 Jun 2003 05:51:25 +0000 Subject: BSD Firewall Message-ID: Jay, Why not stay with Linux? I'm using Bering 1.2 from LEAF ( http://leaf.sourceforge.net ) It's a one floppy Linux Firewall using Shorewall for Firewall configuration. There are lots of additional packages like ssh, etc. It works perfectly well for me. Best Regards Anders _________________________________________________________________ L?ttare att hitta dr?mresan med MSN Resor http://www.msn.se/resor/ From roussel@REDACTED Mon Jun 23 10:35:17 2003 From: roussel@REDACTED (=?ISO-8859-1?Q?Roussel_J=E9r=F4me?=) Date: Mon, 23 Jun 2003 10:35:17 +0200 Subject: moving processes from node to node In-Reply-To: <200306230619.h5N6JGq28904@cbe.ericsson.se> References: <200306230619.h5N6JGq28904@cbe.ericsson.se> Message-ID: Bengt Kleberg wrote: >>X-Injected-Via-Gmane: http://gmane.org/ >>To: erlang-questions@REDACTED >>From: Roussel J?r?me >>Subject: moving processes from node to node > > > >> - is it possible to transfer a whole application (with all its modules) >>from one computer (using one erlang node) to another computer (with >>another erlang node)? I don't talk about process migration because I >>only want to copy all the files of an application from the hard drive of >>computer A to put all theses files onto the hard drive of computer B >> >> - is it possible next to check if all the needed modules are available >>on the computer B? >> >> - if all this is possible, is it possible, in one word, to move one >>application from one node to another (without doing it by hand) ??? > > > ''by hand''? does this mean ''sneaker net'' (manually carrying the > files on some media), or would ''scp '' also be ''by > hand''? > By hand, this mean using scp (or rsync, or ...) and ''sneaker net''. I am looking for a way to do all this with only one command in erlang and every thing would be copied. For example, I would have a registry or a database of every application on a node. Another node connects to the first node, it sees the applications in tha database and says : I don't have this application and this one, so I download them. Now, if you (the user) want, these applications are on the computer and you can use it. > > while waiting for the high tech solution (presumably there is already > some otp module that does this) i have the following (bfmi) suggestion: > Do you think there is an otp module for this ??? It would be wonderful. > 1 use the code module to find the modules (code:which/1) > 2 use the file module to read them (file:read_file/1) > 3 (possibly use rpc:call/4 with file:make_dir/1 to create correct > directory on the other node) > 4 using the binaries from step 2, write them on the other node > (rpc:call/4 with file:write_file/2) > > > bengt > > Thanks for this solution. It seems great, J?r?me From rprice@REDACTED Mon Jun 30 22:35:53 2003 From: rprice@REDACTED (Roger Price) Date: Mon, 30 Jun 2003 22:35:53 +0200 (CEST) Subject: What decides the name of a module? Message-ID: What decides the name of a module? I wrote a program in file hello_all.erl which contains the declaration -module(hello) . When the file hello_all.erl is compiled, I get the executable hello_all.beam and compile:file/2 replies {ok,hello}. I am assuming that my module name is 'hello', but the command code:which(hello) returns 'non_existing'. The command code:which(hello_all) gives me "./ebin/hello_all.beam". Is the module name specified by the declaration -module(.) or implied by the source file name? - or are there two different notions of "module" that I am confusing? Any advice would be much appreciated. Best Regards, Roger