From etxuwig@REDACTED Wed Sep 1 09:10:51 1999 From: etxuwig@REDACTED (Ulf Wiger) Date: Wed, 1 Sep 1999 09:10:51 +0200 (MET DST) Subject: Garbage collection In-Reply-To: <37cb6f6e.119.0@flashmail.com> Message-ID: On Mon, 30 Aug 1999, Vlad Dumitrescu wrote: vlad.d>>Did this discussion start because someone was unhappy with vlad.d>the performance >of the Erlang garbage collector or is it vlad.d>pure (though interesting) >conjecture about how to speed up vlad.d>the system? > vlad.d> vlad.d>I started it, and it is on the "to do" list at erlang.org, so vlad.d>I suppose someone has experienced slow collections... The main problem, as I see it, with the current collector is that it blocks the entire runtime system when it starts collecting for one process. Thus, the GC may be per-process, but all other processes have to wait. Otherwise, the current collector seems pretty fast. Most GC times are very short, and we have determined that the AXD 301 spends something like 5% of its CPU resources on GC when going full blast with connection handling. (This after we have used a few tricks like custom heap size.) I have seen collections as long as 1 second on a 300 MHz UltraSPARC, but those are extremely rare and have appeared under very special circumstances -- with rapidly growing heap and no data that could be released. As I recall, it also involved a first attempt at generational GC, followed by a fullsweep GC with a resize of the heap in the middle. Even such GC times are reasonable as long as they don't make the whole system unresponsive. This is why I would like to see an effort to make the collector reentrant. /Uffe Ulf Wiger, Chief Designer AXD 301 Ericsson Telecom AB tfn: +46 8 719 81 95 Varuv?gen 9, ?lvsj? mob: +46 70 519 81 95 S-126 25 Stockholm, Sweden fax: +46 8 719 43 44 From tobbe@REDACTED Wed Sep 1 15:09:50 1999 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: 01 Sep 1999 23:09:50 +1000 Subject: The Wikie Web site ! Message-ID: Hi everyone ! I just want to inform you of the following nice Web site for discussions about Erlang related topics: http://fresh.vegetable.org:2000/down/erl/wikie_web/show?Welcome Cheers /Tobbe -- Torbj?rn T?rnkvist tobbe@REDACTED o{..}o ph: +61 3 9925 4089 fax:+61 3 9925 4094 moo! (oo)___ Software Engineering Research Centre (SERC) (______)\ / \ / \ From dg@REDACTED Wed Sep 1 20:47:39 1999 From: dg@REDACTED (David Gould) Date: Wed, 1 Sep 1999 11:47:39 -0700 Subject: Garbage collection In-Reply-To: ; from Ulf Wiger on Wed, Sep 01, 1999 at 09:10:51AM +0200 References: <37cb6f6e.119.0@flashmail.com> Message-ID: <19990901114739.A8949@hawk.oak.informix.com> On Wed, Sep 01, 1999 at 09:10:51AM +0200, Ulf Wiger wrote: > On Mon, 30 Aug 1999, Vlad Dumitrescu wrote: > > vlad.d>>Did this discussion start because someone was unhappy with > vlad.d>the performance >of the Erlang garbage collector or is it > vlad.d>pure (though interesting) >conjecture about how to speed up > vlad.d>the system? > > vlad.d> > vlad.d>I started it, and it is on the "to do" list at erlang.org, so > vlad.d>I suppose someone has experienced slow collections... > > The main problem, as I see it, with the current collector is that it blocks > the entire runtime system when it starts collecting for one process. Thus, > the GC may be per-process, but all other processes have to wait. > > Otherwise, the current collector seems pretty fast. Most GC times are very > short, and we have determined that the AXD 301 spends something like 5% of > its CPU resources on GC when going full blast with connection handling. > (This after we have used a few tricks like custom heap size.) > > I have seen collections as long as 1 second on a 300 MHz UltraSPARC, but > those are extremely rare and have appeared under very special circumstances > -- with rapidly growing heap and no data that could be released. As I > recall, it also involved a first attempt at generational GC, followed by a > fullsweep GC with a resize of the heap in the middle. > > Even such GC times are reasonable as long as they don't make the whole > system unresponsive. This is why I would like to see an effort to make the > collector reentrant. > > /Uffe > > Ulf Wiger, Chief Designer AXD 301 > Ericsson Telecom AB tfn: +46 8 719 81 95 > Varuv?gen 9, ?lvsj? mob: +46 70 519 81 95 > S-126 25 Stockholm, Sweden fax: +46 8 719 43 44 I might be interested in working on this. I am curious to know what has prevented this in the past, the GC itself seems fairly compact and looks (ok, I only spent 10 minutes peeking at it) fairly clean. Also, any thoughts on how to proceed or things I need to be aware of outside gcc.c are welcome. Thanks -dg -- David Gould dg@REDACTED 510.628.3783 or 510.305.9468 Informix Software 300 Lakeside Drive Oakland, CA 94612 You will cooperate with Microsoft, for the good of Microsoft and for your own survival. -- Navindra Umanee From klacke@REDACTED Thu Sep 2 11:22:52 1999 From: klacke@REDACTED (Klacke) Date: Thu, 2 Sep 1999 11:22:52 +0200 (CEST) Subject: Garbage collection In-Reply-To: <19990901114739.A8949@hawk.oak.informix.com> References: <37cb6f6e.119.0@flashmail.com> <19990901114739.A8949@hawk.oak.informix.com> Message-ID: <14286.16876.167652.625593@kricka.bluetail.com> David Gould writes: > I might be interested in working on this. I am curious to know what has > prevented this in the past, It's a bit hard, to make the gc non blocking ggc.c needs to rewritten into a reentrant collector. This is a bit hard and it also makes the collector a bit slower unless some real clever tricks are applied. Furthermore, when a process has garbed and the garb was not finished (since it took too long time) and execution was resumed instead, well then we have a fairly complex situation. Imagine that we need to send a message to such a "half-garbed" process, we need to allocate som memory there for the message. Where do we allocate that memory, hardly on any of the heaps that are "half-garbed" ?? (Or do we suspend the sender !!!, deadlocks ???) Thus the situation which lead to this "half-garb", was probably that the live data was large. Interupting the gc of this large data set leaves us with a whole lot of memory which is not released. The original two heaps of the process as well as the new heap we're garbing to. Three heaps. If a message arrives at the process, we need to have yet another memory area associated with the process where we can put the message, four heaps !!. Now then, at a later stage when we're garbing this 4-heap process, we need to make this gc reentrant as well otherwise there's no point in the exercise at all. Well, the above sounds just horrible, so some sound strategy must be applied here, I don't know which though. And last but not least, the process must be marked (and queued) for gc as well so that we can finnish off any lingering half-garbed processes when the system is idling, this is the easy part because here we can simply put the process on the run queue and mark it for GC. /klacke Claes Wikstr?m Tel: +46 (0)8 692 22 09 Bluetail AB Email: klacke@REDACTED Hantverkargatan 78 WWW: http://www.bluetail.com SE-112 38 Stockholm, SWEDEN From etxuwig@REDACTED Thu Sep 2 11:56:19 1999 From: etxuwig@REDACTED (Ulf Wiger) Date: Thu, 2 Sep 1999 11:56:19 +0200 (MET DST) Subject: Garbage collection In-Reply-To: <14286.16876.167652.625593@kricka.bluetail.com> Message-ID: On Thu, 2 Sep 1999, Klacke wrote: klacke>Furthermore, when a process has garbed and the garb was klacke>not finished (since it took too long time) and execution klacke>was resumed instead, well then we have a fairly complex klacke>situation. Imagine that we need to send a message to such klacke>a "half-garbed" process, we need to allocate som memory there klacke>for the message. Where do we allocate that memory, hardly klacke>on any of the heaps that are "half-garbed" ?? klacke> klacke>(Or do we suspend the sender !!!, deadlocks ???) Why would this cause deadlocks? The runtime system already uses flow control when a process sends a message to a port, right? If a process tries to send a message to another process which is half-garbed, it is suspended; if no other processes can run, the half-garbed process is allowed to continue. Since it doesn't depend on the process waiting to send a message, there won't be any deadlock. Or am I missing something? klacke>Thus the situation which lead to this "half-garb", was probably klacke>that the live data was large. Interupting the gc of this klacke>large data set leaves us with a whole lot of memory which klacke>is not released. The original two heaps of the process as klacke>well as the new heap we're garbing to. Three heaps. klacke>If a message arrives at the process, we need to have yet klacke>another memory area associated with the process where we klacke>can put the message, four heaps !!. Yes. If we suspend senders -- three heaps. This is a tradeoff between memory useage and realtime behaviour. The main point of the exercise is that designers will know fairly well which parts of a program may cause a costly gc, and can structure their processes accordingly (processes which have to be responsive can offload heavy tasks to other processes.) There is no point in doing that today, because the heavy gc will destroy real-time characteristics even if it happens in a background process! What you can do today is rewrite your Erlang code so that it performs the same job differently -- perhaps by forcing small garbage collections along the way. This is "unnecessary" work (since it's a workaround for shortcomings in the runtime system), and makes for slower code at the Erlang level. klacke>Now then, at a later stage when we're garbing klacke>this 4-heap process, we need to make this gc reentrant as well klacke>otherwise there's no point in the exercise at all. I don't think I follow you. It probably isn't a good idea to accept more data as we're garbing, because we can't be sure that the gc will ever terminate. Therefore, we should suspend senders during the gc, and if we do, the gc will behave as today, except that it may yield at certain stages. One aspect to consider is that the vast majority of collections which are cheap should not become significantly more expensive because we want to make the heavy ones reentrant. I don't know how important it is, or how to go about it. Perhaps a few clever checks (e.g. heap size) could reveal whether the gc will be a candidate for reentrant gc... /Uffe Ulf Wiger, Chief Designer AXD 301 Ericsson Telecom AB tfn: +46 8 719 81 95 Varuv?gen 9, ?lvsj? mob: +46 70 519 81 95 S-126 25 Stockholm, Sweden fax: +46 8 719 43 44 From SEAN.HINDE@REDACTED Thu Sep 2 12:03:00 1999 From: SEAN.HINDE@REDACTED (SEAN HINDE) Date: Thu, 2 Sep 1999 11:03:00 +0100 Subject: The Wikie Web site ! Message-ID: <360660a8.020899@smtpgate1.ONE2ONE.CO.UK> I could only make it work without the :2000 - excellent site, well worth popping in Cheers, Sean ---------- From: Torbjorn Tornkvist To: erlang-questions@REDACTED Subject: The Wikie Web site ! Date: 01 September 1999 18:11 Hi everyone ! I just want to inform you of the following nice Web site for discussions about Erlang related topics: http://fresh.vegetable.org:2000/down/erl/wikie_web/show?Welcome Cheers /Tobbe -- Torbj?rn T?rnkvist tobbe@REDACTED o{..}o ph: +61 3 9925 4089 fax:+61 3 9925 4094 moo! (oo)___ Software Engineering Research Centre (SERC) (______)\ / \ / \ From bjorn@REDACTED Thu Sep 2 13:09:45 1999 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 02 Sep 1999 13:09:45 +0200 Subject: Garbage collection In-Reply-To: Klacke's message of Thu, 2 Sep 1999 11:22:52 +0200 (CEST) References: <37cb6f6e.119.0@flashmail.com> <19990901114739.A8949@hawk.oak.informix.com> <14286.16876.167652.625593@kricka.bluetail.com> Message-ID: Klacke writes: > > > David Gould writes: > > I might be interested in working on this. I am curious to know what has > > prevented this in the past, > > > It's a bit hard, to make the gc non blocking ggc.c needs to rewritten > into a reentrant collector. This is a bit hard and it also makes > the collector a bit slower unless some real clever tricks are > applied. Yes, I think keeping the performance is the real problem. > > Furthermore, when a process has garbed and the garb was > not finished (since it took too long time) and execution > was resumed instead, well then we have a fairly complex > situation. Imagine that we need to send a message to such > a "half-garbed" process, we need to allocate som memory there > for the message. Where do we allocate that memory, hardly > on any of the heaps that are "half-garbed" ?? Messages sent to a process are never put directly onto the heap of the receiving process. (They used to be in older version of Erlang.) Instead, each message is put into a separately allocated message buffer outside the heap. Therefore, this should not be a problem. But you make must take good care to keep track of the state (size of data in messages and which messages that are new and so on). > > (Or do we suspend the sender !!!, deadlocks ???) > > Thus the situation which lead to this "half-garb", was probably > that the live data was large. Interupting the gc of this > large data set leaves us with a whole lot of memory which > is not released. The original two heaps of the process as > well as the new heap we're garbing to. Three heaps. That is a real problem. /Bjorn Gustavsson From klacke@REDACTED Thu Sep 2 13:23:24 1999 From: klacke@REDACTED (Klacke) Date: Thu, 2 Sep 1999 13:23:24 +0200 (CEST) Subject: Garbage collection In-Reply-To: References: <37cb6f6e.119.0@flashmail.com> <19990901114739.A8949@hawk.oak.informix.com> <14286.16876.167652.625593@kricka.bluetail.com> Message-ID: <14286.24108.348663.980548@kricka.bluetail.com> Bjorn Gustavsson writes: > > Messages sent to a process are never put directly onto the heap > of the receiving process. (They used to be in older version of Erlang.) > Instead, each message is put into a separately allocated message > buffer outside the heap. > I know, this the situation in open source erlang as well. However the message bugger queue is part of the root set and can thus not be easily fiddled with if the process is in an interupted gc state. > Therefore, this should not be a problem. But you make must take good > care to keep track of the state (size of data in messages and which > messages that are new and so on). > Either that, or suspend the sender which is a whole lot easier. I wrote "deadlocks ???" and I've thought a bit about that now and I don't think it's an issue. ?? Think hard. > > > > Thus the situation which lead to this "half-garb", was probably > > that the live data was large. Interupting the gc of this > > large data set leaves us with a whole lot of memory which > > is not released. The original two heaps of the process as > > well as the new heap we're garbing to. Three heaps. > > That is a real problem. > Yes, on the other hand one of the more common situations for a process to get a really huge heap is that somebody is bombarding it with messages and the process can't keep up with the processing of all async messages. If the sender is suspended, it changes that situation a bit. An other question, what about distributed messages if the sender is to be suspended. Set the dist port to blocked, thereby inhibiting *all* dist messages on that port just because one process is garbing ?? /klacke From etxuwig@REDACTED Thu Sep 2 09:31:26 1999 From: etxuwig@REDACTED (Ulf Wiger) Date: Thu, 2 Sep 1999 09:31:26 +0200 (MET DST) Subject: Garbage collection In-Reply-To: <19990901114739.A8949@hawk.oak.informix.com> Message-ID: On Wed, 1 Sep 1999, David Gould wrote: dg>I might be interested in working on this. I am curious to know what has dg>prevented this in the past, the GC itself seems fairly compact and looks dg>(ok, I only spent 10 minutes peeking at it) fairly clean. I believe that it was considered a good idea, but it wasn't acted on because other things were given higher priority. dg>Also, any thoughts on how to proceed or things I need to be aware of dg>outside gcc.c are welcome. Sorry, I can't help you there. I don't understand the GC -- I just use it. /Uffe Ulf Wiger, Chief Designer AXD 301 Ericsson Telecom AB tfn: +46 8 719 81 95 Varuv?gen 9, ?lvsj? mob: +46 70 519 81 95 S-126 25 Stockholm, Sweden fax: +46 8 719 43 44 From ug-erlang@REDACTED Thu Sep 2 16:20:41 1999 From: ug-erlang@REDACTED (David Brown) Date: Thu, 2 Sep 1999 07:20:41 -0700 (PDT) Subject: Garbage collection In-Reply-To: <14286.16876.167652.625593@kricka.bluetail.com> References: <37cb6f6e.119.0@flashmail.com> <19990901114739.A8949@hawk.oak.informix.com> <14286.16876.167652.625593@kricka.bluetail.com> Message-ID: <14286.34745.573718.74672@opus.davidb.org> >>>>> On Thu, 2 Sep 1999 11:22:52 +0200 (CEST), Klacke said: > David Gould writes: >> I might be interested in working on this. I am curious to know what has >> prevented this in the past, > It's a bit hard, to make the gc non blocking ggc.c needs to rewritten > into a reentrant collector. This is a bit hard and it also makes > the collector a bit slower unless some real clever tricks are > applied. > ... I found a bunch of good papers on garbage collection (realtime as well) on . One of the papers discusses a real-time garbage collector that only needs to coordinate with the "mutators" when the mutate memory. We have a nice advantage here that there isn't much mutation going on outside of making new objects. Their collector is single threaded, and single heaped. Moving to this model might help with efficiency, since we could avoid the copy when sending local messages. The collector just needs to be run periodically (easy with an interpreter) and only needs to be informed about mutations. It also eliminates issues with allocation in a process whose GC would be running. The difficulty here is can we guarantee that a process can run that needs to run. I'll look around and figure out which article it actually is that talks about it. David Brown From crd@REDACTED Thu Sep 2 16:37:28 1999 From: crd@REDACTED (Craig Dickson) Date: Thu, 2 Sep 1999 07:37:28 -0700 Subject: The Wikie Web site ! References: <360660a8.020899@smtpgate1.ONE2ONE.CO.UK> Message-ID: <002e01bef550$b0b20c80$b102a8c0@int2.inversenet.com> SEAN HINDE wrote: >I could only make it work without the :2000 Yes, this is a problem when you work behind a firewall that allows traffic only on known ports. When the Wikie was announced a few days ago on comp.lang.functional, I complained about the use of port 2000, so they made sure that port 80 worked as well. That being the case, I don't really see the point of also listening on port 2000, but if that's how they want to do it, I don't suppose it hurts anything... Craig From jhague@REDACTED Thu Sep 2 18:25:35 1999 From: jhague@REDACTED (James Hague) Date: Thu, 2 Sep 1999 11:25:35 -0500 (CDT) Subject: Garbage collection In-Reply-To: <14286.34745.573718.74672@opus.davidb.org> Message-ID: On Thu, 2 Sep 1999, David Brown wrote: > > Their collector is single threaded, and single heaped. Moving to this > model might help with efficiency, since we could avoid the copy when > sending local messages. The collector just needs to be run > periodically (easy with an interpreter) and only needs to be informed > about mutations. It also eliminates issues with allocation in a > process whose GC would be running. A nice benefit of a separate heap per process is that you can just throw out all the data for that process when it's killed. Sharing a single heap would potentially result in more frequent collections. It would be handy for passing large messages around, though. I often find myself writing a database server that other processes query with messages like "send me a list of all items that meet criteria X." In the case of having numerous processes making queries very frequently--i.e. thirty or more times a second--it would be nice to avoid the copy. But the point of Erlang is to not think about low-level efficiency unless you really need to, eh? :) James From ug-erlang@REDACTED Thu Sep 2 19:00:15 1999 From: ug-erlang@REDACTED (David Brown) Date: Thu, 2 Sep 1999 10:00:15 -0700 (PDT) Subject: Garbage collection In-Reply-To: References: <14286.34745.573718.74672@opus.davidb.org> Message-ID: <14286.44319.321068.758475@opus.davidb.org> >>>>> On Thu, 2 Sep 1999 11:25:35 -0500 (CDT), James Hague said: > But the point of Erlang is to not think about low-level efficiency unless > you really need to, eh? :) Yeah, it's supposed to be the job of the implementors of the language. When we discuss how the garbage collector works, we make ourselves to be implementors. I guess some good questions to ask would be, for typical Erlang programs (whatever typical): - Is it frequent that processes are created and destroyed, while making a bunch of data while alive. - Are there many longer lived processes that serve up data. I would also suspect that the whole memory footprint could be reduced with a single collection heap. This might help with the desired implementation that runs on a PDA. Dave Brown From etxuwig@REDACTED Thu Sep 2 19:09:48 1999 From: etxuwig@REDACTED (Ulf Wiger) Date: Thu, 2 Sep 1999 19:09:48 +0200 (MET DST) Subject: Garbage collection In-Reply-To: <14286.44319.321068.758475@opus.davidb.org> Message-ID: On Thu, 2 Sep 1999, David Brown wrote: ug-erl>I guess some good questions to ask would be, for typical Erlang ug-erl>programs (whatever typical): ug-erl> ug-erl>- Is it frequent that processes are created and destroyed, while ug-erl> making a bunch of data while alive. Yes. One example is httpd, which spawns one process per connection. ug-erl>- Are there many longer lived processes that serve up data. Yes. Examples: the code server, the I/O server (user), application_controller, ... Both patterns are very important and must be handled efficiently by the garbage collector. /Uffe Ulf Wiger, Chief Designer AXD 301 Ericsson Telecom AB tfn: +46 8 719 81 95 Varuv?gen 9, ?lvsj? mob: +46 70 519 81 95 S-126 25 Stockholm, Sweden fax: +46 8 719 43 44 From dg@REDACTED Thu Sep 2 19:58:23 1999 From: dg@REDACTED (David Gould) Date: Thu, 2 Sep 1999 10:58:23 -0700 Subject: Garbage collection In-Reply-To: <14286.34745.573718.74672@opus.davidb.org>; from David Brown on Thu, Sep 02, 1999 at 07:20:41AM -0700 References: <37cb6f6e.119.0@flashmail.com> <19990901114739.A8949@hawk.oak.informix.com> <14286.16876.167652.625593@kricka.bluetail.com> <14286.34745.573718.74672@opus.davidb.org> Message-ID: <19990902105823.B19448@hawk.oak.informix.com> On Thu, Sep 02, 1999 at 07:20:41AM -0700, David Brown wrote: > >>>>> On Thu, 2 Sep 1999 11:22:52 +0200 (CEST), Klacke said: > > > David Gould writes: > >> I might be interested in working on this. I am curious to know what has > >> prevented this in the past, > > > It's a bit hard, to make the gc non blocking ggc.c needs to rewritten > > into a reentrant collector. This is a bit hard and it also makes > > the collector a bit slower unless some real clever tricks are > > applied. > > > ... > > I found a bunch of good papers on garbage collection (realtime as > well) on . One of the papers This is Paul Wilsons group at U Texas, a great resource for memory management information. I have read a number of their papers but not specifically the realtime GC one. But I will. > discusses a real-time garbage collector that only needs to coordinate > with the "mutators" when the mutate memory. We have a nice advantage > here that there isn't much mutation going on outside of making new > objects. > > Their collector is single threaded, and single heaped. Moving to this > model might help with efficiency, since we could avoid the copy when > sending local messages. The collector just needs to be run > periodically (easy with an interpreter) and only needs to be informed > about mutations. It also eliminates issues with allocation in a > process whose GC would be running. Interesting. > I'll look around and figure out which article it actually is that > talks about it. Please. -dg -- David Gould dg@REDACTED 510.628.3783 or 510.305.9468 Informix Software 300 Lakeside Drive Oakland, CA 94612 You will cooperate with Microsoft, for the good of Microsoft and for your own survival. -- Navindra Umanee From dg@REDACTED Fri Sep 3 05:29:33 1999 From: dg@REDACTED (David Gould) Date: Thu, 2 Sep 1999 20:29:33 -0700 Subject: Garbage collection In-Reply-To: ; from James Hague on Thu, Sep 02, 1999 at 11:25:35AM -0500 References: <14286.34745.573718.74672@opus.davidb.org> Message-ID: <19990902202933.A27298@hawk.oak.informix.com> On Thu, Sep 02, 1999 at 11:25:35AM -0500, James Hague wrote: > On Thu, 2 Sep 1999, David Brown wrote: > > > > Their collector is single threaded, and single heaped. Moving to this > > model might help with efficiency, since we could avoid the copy when > > sending local messages. The collector just needs to be run > > periodically (easy with an interpreter) and only needs to be informed > > about mutations. It also eliminates issues with allocation in a > > process whose GC would be running. > > A nice benefit of a separate heap per process is that you can just throw > out all the data for that process when it's killed. Sharing a single heap > would potentially result in more frequent collections. This should not be a problem for a global heap garbage collector. Real programs do not allocate memory in a steady way or even in any kind of random distribution. They tend to have strong "phase" behaviour, allocating large numbers of similar objects, working for bit, and then freeing most of the previous allocations and allocating some new kind of object. So any realistic memory manager has to cope with this kind of thing. Process deallocation is just one kind of bursty allocator/collector workload. -dg -- David Gould dg@REDACTED 510.628.3783 or 510.305.9468 Informix Software 300 Lakeside Drive Oakland, CA 94612 You will cooperate with Microsoft, for the good of Microsoft and for your own survival. -- Navindra Umanee From patrickdlogan@REDACTED Fri Sep 3 08:38:42 1999 From: patrickdlogan@REDACTED (Patrick Logan) Date: Thu, 2 Sep 1999 23:38:42 -0700 (PDT) Subject: Garbage collection In-Reply-To: <19990902202933.A27298@hawk.oak.informix.com> References: <14286.34745.573718.74672@opus.davidb.org> <19990902202933.A27298@hawk.oak.informix.com> Message-ID: <14287.27472.973598.694433@c837917-a.potlnd1.or.home.com> David Gould writes: > > > > A nice benefit of a separate heap per process is that you can > > just throw out all the data for that process when it's killed. > > Sharing a single heap would potentially result in more frequent > > collections. > > This should not be a problem for a global heap garbage > collector. Real programs do not allocate memory in a steady way or > even in any kind of random distribution. They tend to have strong > "phase" behaviour, allocating large numbers of similar objects, > working for bit, and then freeing most of the previous allocations > and allocating some new kind of object. > > So any realistic memory manager has to cope with this kind of > thing. Process deallocation is just one kind of bursty > allocator/collector workload. Sorry for jumping into this conversation without having followed it all too closely. But I have a few thoughts. A problem with a heap per process seems to be how much heap to allocate to each process, and handling the processes that outgrow their heap. Would there be a way to implement a "Big Bag of Pages" (BIBOP) where a "page" is dedicated to a process but any process' pages would be scattered around the heap. When a process goes away you'd want to give those pages back to the system somehow without too much effort. There was a paper out of Indiana University a few years ago called something like "Don't Stop the BIBOP" which described using BIBOP pages to indicate information about the kinds of things allocated on any given page. Process information may fit into this scheme. I don't know. Just a thought. I am not a GC implementor. -- Patrick Logan patrickdlogan@REDACTED From ug-erlang@REDACTED Fri Sep 3 16:44:44 1999 From: ug-erlang@REDACTED (David Brown) Date: Fri, 3 Sep 1999 07:44:44 -0700 (PDT) Subject: Garbage collection In-Reply-To: <14287.27472.973598.694433@c837917-a.potlnd1.or.home.com> References: <14286.34745.573718.74672@opus.davidb.org> <19990902202933.A27298@hawk.oak.informix.com> <14287.27472.973598.694433@c837917-a.potlnd1.or.home.com> Message-ID: <14287.57052.53834.255088@opus.davidb.org> >>>>> On Thu, 2 Sep 1999 23:38:42 -0700 (PDT), Patrick Logan said: > A problem with a heap per process seems to be how much heap to > allocate to each process, and handling the processes that outgrow > their heap. Another approach would be to do something like Caml Light. Each process has a small heap. Objects that are smaller than a certain size are allocated from this heap. When a process does a minor collection, its live data is moved to the shared heap. This per-process heap is usually something small, like 32k. (The small heap is also only for non-mutable objects). The large heap is the collected in a more coordinated way. This ends up as a kind of hybrid copying and mark-sweep collector. The first generation is copying (but only for small objects), and the subsequent generation is collected normally. The large heap will need to use a real time collection algorithm, whereas the small heaps can just be copied through in their entirery. They can be chosen to be small enough that this collection runs fast enough. When a message is passed, it is checked to see which heap it is in, and only if it is in a small heap is it copied. That way, especially large messages will avoid the copy. I'm still looking for the article. Hopefully I will find it when I return from my trip (Monday is a holiday in the US). Dave Brown From luke@REDACTED Sat Sep 4 18:28:35 1999 From: luke@REDACTED (Luke Gorrie) Date: 05 Sep 1999 02:28:35 +1000 Subject: Grid in etk? Message-ID: <87k8q6hc98.fsf@baked.vegetable.org> I'm wondering what the best way to do a grid component with etk is? I mean something like the processes table in pman. Is it just a bunch of labels arranged with the 'grid' layout command? Cheers, Luke From randyk@REDACTED Mon Sep 6 05:31:13 1999 From: randyk@REDACTED (Randy Katz) Date: Sun, 5 Sep 1999 20:31:13 -0700 Subject: [randyk@ccsales.com: compile erlang] Message-ID: <19990905203113.D681@ccsales.com> Hi, Linux RedHat 6.0, kernel 2.2.5-15, erlang-47.4.1: make[4]: Entering directory `/usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src' gcc -g -O2 -I/usr/src/eddy/erlang-47.4.1/erts/autoconf/i586-unknown-linux -I. -I/usr/src/eddy/erlang-47.4.1/erts/system/emulator/runtime -DHAVE_CONFIG_H -DUSE_DNS -DERLANG_DAEMON_PORT=4369 -L/usr/src/eddy/erlang-47.4.1/erts/obj/i586-unknown-linux -o /usr/src/eddy/erlang-47.4.1/erts/bin/i586-unknown-linux/erl_call erl_call.c -lerl_interface -lei -lcurses -lresolv -ldl -lm /tmp/ccHciWb9.o: In function `get_hostent': /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:198: undefined reference to `erl_gethostbyaddr' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:201: undefined reference to `erl_gethostbyname' /tmp/ccHciWb9.o: In function `do_connect': /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:221: undefined reference to `erl_connect' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:239: undefined reference to `erl_gethostbyname' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:242: undefined reference to `erl_start_sys' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:243: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:246: undefined reference to `erl_connect' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:255: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:258: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:261: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:264: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:267: undefined reference to `erl_err_quit' /tmp/ccHciWb9.o:/usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:291: more undefined references to `erl_err_quit' follow /tmp/ccHciWb9.o: In function `read_stdin': /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:339: undefined reference to `erl_err_sys' /tmp/ccHciWb9.o: In function `main': /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:607: undefined reference to `erl_init_nothreads' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:638: undefined reference to `erl_gethostbyname' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:639: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:648: undefined reference to `erl_connect_xinit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:650: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:653: undefined reference to `erl_thishostname' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:665: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:679: undefined reference to `erl_connect' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:684: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:689: undefined reference to `erl_format' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:689: undefined reference to `erl_rpc' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:705: undefined reference to `erl_mk_binary' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:705: undefined reference to `erl_format' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:705: undefined reference to `erl_rpc' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:708: undefined reference to `erl_err_msg' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:710: undefined reference to `erl_format' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:710: undefined reference to `erl_rpc' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:711: undefined reference to `erl_err_msg' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:713: undefined reference to `erl_format' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:713: undefined reference to `erl_match' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:714: undefined reference to `erl_err_msg' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:724: undefined reference to `erl_mk_binary' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:724: undefined reference to `erl_format' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:724: undefined reference to `erl_rpc' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:726: undefined reference to `erl_err_msg' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:728: undefined reference to `erl_print_term' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:741: undefined reference to `erl_format' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:744: undefined reference to `erl_rpc' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:747: undefined reference to `erl_print_term' collect2: ld returned 1 exit status make[4]: *** [/usr/src/eddy/erlang-47.4.1/erts/bin/i586-unknown-linux/erl_call] Error 1 make[4]: Leaving directory `/usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src' make[3]: *** [opt] Error 2 make[3]: Leaving directory `/usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src' make[2]: *** [opt] Error 2 make[2]: Leaving directory `/usr/src/eddy/erlang-47.4.1/erts/system/erl_interface' make[1]: *** [opt] Error 2 make[1]: Leaving directory `/usr/src/eddy/erlang-47.4.1/erts/system' make: *** [emulator] Error 2 [root@REDACTED erlang-47.4.1]# ----- End forwarded message ----- From klacke@REDACTED Mon Sep 6 16:55:30 1999 From: klacke@REDACTED (Klacke) Date: Mon, 6 Sep 1999 16:55:30 +0200 (CEST) Subject: Grid in etk? In-Reply-To: <87k8q6hc98.fsf@baked.vegetable.org> References: <87k8q6hc98.fsf@baked.vegetable.org> Message-ID: <14291.54754.616466.259474@kricka.bluetail.com> > I'm wondering what the best way to do a grid component with etk is? I > mean something like the processes table in pman. Is it just a bunch of > labels arranged with the 'grid' layout command? Here's what I learned when I was doing some etk programming some time ago. Do everything exactly the same way it should have been done with tcl/tk. There are thousand of tcl/tk tutorials and books out there and any idiom that suits tcl/tk (with emphasis on tk) is suitable for etk as well. Basically all tk commands map one to one to etk. As for your specific question of how to do pman in etk, maybe just a grid with labels as you suggest, or a canvas with a bunch of *embedded* buttons ?? Have you found the wtour in lib/etk/examples/wtour/* ??? /klacke From randyk@REDACTED Tue Sep 7 08:25:50 1999 From: randyk@REDACTED (Randy A. Katz) Date: Mon, 06 Sep 1999 23:25:50 -0700 Subject: Linux compile problems - (help) Message-ID: <3.0.5.32.19990906232550.033c6da0@ccsales.com> Hello, I was able to successfully compile Erlang under FreeBSD using the port. But under RedHat Linux 6.0, kernel 2.2.5-15, I get the following messages after a ./configure, make: make[3]: Entering directory `/usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src' make -f i586-unknown-linux/Makefile TYPE=opt make[4]: Entering directory `/usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src' gcc -g -O2 -I/usr/src/eddy/erlang-47.4.1/erts/autoconf/i586-unknown-linux -I. -I/usr/src/eddy/erlang-47.4.1/erts/system/emulator/runtime -DHAVE_CONFIG_H -DUSE_DNS -DERLANG_DAEMON_PORT=4369 -L/usr/src/eddy/erlang-47.4.1/erts/obj/i586-unknown-linux -o /usr/src/eddy/erlang-47.4.1/erts/bin/i586-unknown-linux/erl_call erl_call.c -lerl_interface -lei -lcurses -lresolv -ldl -lm /tmp/ccghYwCd.o: In function `get_hostent': /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:198: undefined reference to `erl_gethostbyaddr' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:201: undefined reference to `erl_gethostbyname' /tmp/ccghYwCd.o: In function `do_connect': /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:221: undefined reference to `erl_connect' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:239: undefined reference to `erl_gethostbyname' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:242: undefined reference to `erl_start_sys' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:243: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:246: undefined reference to `erl_connect' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:255: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:258: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:261: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:264: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:267: undefined reference to `erl_err_quit' /tmp/ccghYwCd.o:/usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/er l_call.c:291: more undefined references to `erl_err_quit' follow /tmp/ccghYwCd.o: In function `read_stdin': /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:339: undefined reference to `erl_err_sys' /tmp/ccghYwCd.o: In function `main': /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:607: undefined reference to `erl_init_nothreads' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:638: undefined reference to `erl_gethostbyname' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:639: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:648: undefined reference to `erl_connect_xinit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:650: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:653: undefined reference to `erl_thishostname' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:665: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:679: undefined reference to `erl_connect' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:684: undefined reference to `erl_err_quit' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:689: undefined reference to `erl_format' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:689: undefined reference to `erl_rpc' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:705: undefined reference to `erl_mk_binary' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:705: undefined reference to `erl_format' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:705: undefined reference to `erl_rpc' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:708: undefined reference to `erl_err_msg' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:710: undefined reference to `erl_format' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:710: undefined reference to `erl_rpc' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:711: undefined reference to `erl_err_msg' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:713: undefined reference to `erl_format' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:713: undefined reference to `erl_match' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:714: undefined reference to `erl_err_msg' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:724: undefined reference to `erl_mk_binary' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:724: undefined reference to `erl_format' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:724: undefined reference to `erl_rpc' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:726: undefined reference to `erl_err_msg' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:728: undefined reference to `erl_print_term' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:741: undefined reference to `erl_format' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:744: undefined reference to `erl_rpc' /usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src/erl_call.c:747: undefined reference to `erl_print_term' collect2: ld returned 1 exit status make[4]: *** [/usr/src/eddy/erlang-47.4.1/erts/bin/i586-unknown-linux/erl_call] Error 1 make[4]: Leaving directory `/usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src' make[3]: *** [opt] Error 2 make[3]: Leaving directory `/usr/src/eddy/erlang-47.4.1/erts/system/erl_interface/src' make[2]: *** [opt] Error 2 make[2]: Leaving directory `/usr/src/eddy/erlang-47.4.1/erts/system/erl_interface' make[1]: *** [opt] Error 2 make[1]: Leaving directory `/usr/src/eddy/erlang-47.4.1/erts/system' make: *** [emulator] Error 2 [root@REDACTED erlang-47.4.1]# ------- What can I do to get this to compile under Linux? Do I need different libraries? Should I use a different distribution? Thank you, Randy Katz From klacke@REDACTED Tue Sep 7 10:23:31 1999 From: klacke@REDACTED (Klacke) Date: Tue, 7 Sep 1999 10:23:31 +0200 (CEST) Subject: Linux compile problems - (help) In-Reply-To: <3.0.5.32.19990906232550.033c6da0@ccsales.com> References: <3.0.5.32.19990906232550.033c6da0@ccsales.com> Message-ID: <14292.52099.728004.970807@kricka.bluetail.com> Randy A. Katz writes: > Hello, > > I was able to successfully compile Erlang under FreeBSD using the port. But > under RedHat Linux 6.0, kernel 2.2.5-15, I get the following messages after > a ./configure, make: > > .......... Yes really boring bug, the error is in erts/system/emulator/runtime/erl_posix_str.c and the following diff should fix the problem. runtime> diff -c erl_posix_str.c erl_posix_str.c.orig *** erl_posix_str.c Tue Sep 7 10:16:39 1999 --- erl_posix_str.c.orig Tue Sep 7 10:16:24 1999 *************** *** 362,368 **** #ifdef ENXIO case ENXIO: return "enxio"; #endif ! #if defined(EOPNOTSUPP) && (!defined(ENOTSUP) || (ENOTSUP != EOPNOTSUPP)) case EOPNOTSUPP: return "eopnotsupp"; #endif #ifdef EPERM --- 362,368 ---- #ifdef ENXIO case ENXIO: return "enxio"; #endif ! #ifdef EOPNOTSUPP case EOPNOTSUPP: return "eopnotsupp"; #endif #ifdef EPERM Maybe someone in erlang-maintainers could add this to www.erlang.org /klacke From per@REDACTED Tue Sep 7 10:55:36 1999 From: per@REDACTED (Per Hedeland) Date: Tue, 7 Sep 1999 10:55:36 +0200 (MET DST) Subject: Linux compile problems - (help) Message-ID: <199909070855.KAA08406@aalborg.du.uab.ericsson.se> >Yes really boring bug, the error is in >erts/system/emulator/runtime/erl_posix_str.c > >and the following diff should fix the problem. [ ... ] >Maybe someone in erlang-maintainers could add this to www.erlang.org Hm, that patch (though non-reversed:-) has been on the Bugs&Fixes page (http://www.erlang.org/faq/bugs_and_fixes.html) since May - I don't quite see how it would fix Randy's problem though, the errors he got were link-time rather than compile-time? (And the missing symbols should have been found in .../erlang-47.4.1/erts/obj/i586-unknown-linux/liberl_interface.a at that point.) But perhaps this is a "secondary" problem, after retrying make when the initial one failed with the "real" error or somesuch? --Per Hedeland per@REDACTED From klacke@REDACTED Tue Sep 7 11:28:34 1999 From: klacke@REDACTED (Klacke) Date: Tue, 7 Sep 1999 11:28:34 +0200 (CEST) Subject: Linux compile problems - (help) In-Reply-To: <199909070855.KAA08406@aalborg.du.uab.ericsson.se> References: <199909070855.KAA08406@aalborg.du.uab.ericsson.se> Message-ID: <14292.56002.162671.962256@kricka.bluetail.com> Per Hedeland writes: > > Hm, that patch (though non-reversed:-) has been on the Bugs&Fixes page > (http://www.erlang.org/faq/bugs_and_fixes.html) since May - Oops, I didn't see that. > I don't > quite see how it would fix Randy's problem though, the errors he got > were link-time rather than compile-time? (And the missing symbols should > have been found in > .../erlang-47.4.1/erts/obj/i586-unknown-linux/liberl_interface.a at that > point.) But perhaps this is a "secondary" problem, after retrying make > when the initial one failed with the "real" error or somesuch? > Well I think it does fix Randys compile problems. I had similar problems on redhat 6.0 /klacke From klacke@REDACTED Tue Sep 7 23:37:24 1999 From: klacke@REDACTED (Klacke) Date: Tue, 7 Sep 1999 23:37:24 +0200 (CEST) Subject: File/Directory dialogs in ETK? In-Reply-To: <000001bef23f$ea7a5620$9fe879c3@noddy.nl.unisys.com> References: <000001bef23f$ea7a5620$9fe879c3@noddy.nl.unisys.com> Message-ID: <14293.34196.698143.489115@kricka.bluetail.com> Bruce Fitzsimons writes: > Hello again, > > I've finished the basic rewrite of Toolbar into ETK, I'm now working on the > maintenance dialogs. > > Does anyone have any File/Directory dialogs that I can reuse? I'm reluctant > to reinvent any wheels. > > If I have to, I'm planning to write a set of standard "common dialogs" in > ETK for directory searching and file selection(at least). Does that sound > useful? It does indeed. As for file dialogs. About a year ago I did a small attempt at working with a small gui builder for tcl/tk called vtcl. I wrote a little erlang program that took the tcl output from vtcl and makes an erlang program out of it. It doesn't work very well but it;s a start. I include the program as well as getfile.tcl which is generated by vtcl, it translates fairly well, the generated erlang code looks abit ugly but a M-x indent-region in emacs makes the code (almost) readable. Here's the erlang code ...... -------------- next part -------------- A non-text attachment was scrubbed... Name: vtcl.erl Type: application/octet-stream Size: 11231 bytes Desc: erlang code URL: -------------- next part -------------- And here's some tcl code which can loaded and fiddled with in vtcl as well as translated by vtcl.erl into an erlang file -------------- next part -------------- A non-text attachment was scrubbed... Name: getfile.tcl Type: application/octet-stream Size: 7680 bytes Desc: tcl code URL: -------------- next part -------------- Do what you please with this, throw it away, rewrite it or whatever. /klacke From mikl@REDACTED Fri Sep 10 07:35:09 1999 From: mikl@REDACTED (Mickael Remond) Date: Fri, 10 Sep 1999 07:35:09 +0200 Subject: List questions Message-ID: <19990910073509.A2070@louxor.home> I was very surprised with the result of the following commands : 19> [40]. "(" 20> [45]. "-" 21> [60]. "<" 22> [80]. "P" In fact, the result is displayed as a character corresponding to the ascii code in the list. This is quite surprising at first. In a complex command I search for a bug because the function was not supposed to return strings. I guess this has no consequence on the program logic, but you must be very careful to avoid be trapped by what you see. Have funs-> -- Mickael Remond mikl@REDACTED ICQ : 2265396 From crd@REDACTED Thu Sep 9 23:15:39 1999 From: crd@REDACTED (Craig Dickson) Date: Thu, 9 Sep 1999 14:15:39 -0700 Subject: List questions References: <19990910073509.A2070@louxor.home> Message-ID: <029301befb08$7abefe40$b102a8c0@int2.inversenet.com> Since Erlang has no separate type for text characters, a text string is just a list of integers. Because of this, it has no sure way of knowing whether any arbitrary list of integers is meant to be a text string, so when displaying them, it guesses. Your lists below are composed entirely of integers that correspond to printable ASCII characters, so it figures they might be meant to be strings, and displays them as such. (The actual criterion may be a bit different; I haven't looked at the code. But I believe that's the basic idea.) Craig ----- Original Message ----- From: Mickael Remond To: Sent: Thursday, 9 September 1999 10:35 pm Subject: List questions > > I was very surprised with the result of the following commands : > > 19> [40]. > "(" > 20> [45]. > "-" > 21> [60]. > "<" > 22> [80]. > "P" > > > In fact, the result is displayed as a character corresponding to the > ascii code in the list. > > This is quite surprising at first. In a complex command I search for a > bug because the function was not supposed to return strings. > > I guess this has no consequence on the program logic, but you must be > very careful to avoid be trapped by what you see. > > > Have funs-> > > -- > Mickael Remond > mikl@REDACTED > ICQ : 2265396 > From patrickdlogan@REDACTED Thu Sep 9 23:44:13 1999 From: patrickdlogan@REDACTED (Patrick Logan) Date: Thu, 9 Sep 1999 14:44:13 -0700 (PDT) Subject: List questions In-Reply-To: <19990910073509.A2070@louxor.home> References: <19990910073509.A2070@louxor.home> Message-ID: <14296.10551.718341.191147@c837917-a.potlnd1.or.home.com> Mickael Remond writes: > > I was very surprised with the result of the following commands : > > 19> [40]. > "(" > 20> [45]. > "-" > 21> [60]. > "<" > 22> [80]. > "P" Those of us raised on late night TV in America in the 1970s will simply quote: "It's a dessert topping *and* a floor wax!" What this means for Erlang is a list of ASCII integers is a list of numbers *and* a string! Now where did I put my Bass-O-Matic? -- Patrick Logan patrickdlogan@REDACTED From tobbe@REDACTED Thu Sep 9 23:46:47 1999 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: 10 Sep 1999 07:46:47 +1000 Subject: List questions In-Reply-To: Mickael Remond's message of "Fri, 10 Sep 1999 07:35:09 +0200" References: <19990910073509.A2070@louxor.home> Message-ID: > I was very surprised with the result of the following commands : > > 19> [40]. > "(" For many years we had the following behaviour in the shell: 19> "(". [40] Until we got fed up with it. So nowadays the shell tries to be nice and do it for you whenever possible. Cheers /Tobbe -- Torbj?rn T?rnkvist tobbe@REDACTED o{..}o ph: +61 3 9925 4089 fax:+61 3 9925 4094 moo! (oo)___ Software Engineering Research Centre (SERC) (______)\ / \ / \ From klacke@REDACTED Fri Sep 10 09:51:54 1999 From: klacke@REDACTED (Klacke) Date: Fri, 10 Sep 1999 09:51:54 +0200 (CEST) Subject: List questions In-Reply-To: <14296.10551.718341.191147@c837917-a.potlnd1.or.home.com> References: <19990910073509.A2070@louxor.home> <14296.10551.718341.191147@c837917-a.potlnd1.or.home.com> Message-ID: <14296.47258.413543.571223@kricka.bluetail.com> > Those of us raised on late night TV in America in the 1970s will > simply quote: > > "It's a dessert topping *and* a floor wax!" > Uhhh... we're all lost in Europe, I can imagine a disgusting commercial though. As for the: > > 19> [40]. > > "(" behaviour, I'd say it's almost a bug, and it should be fixed. The proper solution is to subtag integers as characters (that is: to introduce a builtin char type) which has the characteristics of an integer except that a new guard test like: f(C) when char(C) -> returns true for chars but not for ints, and $X + 2. should also return a char .... and so on. I think the OTP folks are/have been doing something like this ???? In an old implementation of Erlang where Tony Rogvall and I were experimenting with something we called the "bitsyntax" (briefly described at: http://www.bluetail.com/klacke/binaries.ps) we implemented a scheme with subtaggged integers, it never made it into any releases though. This was done in "rage" by Tony since he got fed up with the weird printouts from the shell. Cheers /klacke From rv@REDACTED Fri Sep 10 10:22:40 1999 From: rv@REDACTED (Robert Virding) Date: Fri, 10 Sep 1999 10:22:40 +0200 Subject: List questions In-Reply-To: Your message of "Fri, 10 Sep 1999 09:51:54 +0200." <14296.47258.413543.571223@kricka.bluetail.com> Message-ID: <199909100822.KAA09927@duva.bluetail.com> Klacke writes: > > Those of us raised on late night TV in America in the 1970s will > > simply quote: > > > > "It's a dessert topping *and* a floor wax!" > >Uhhh... we're all lost in Europe, I can imagine a >disgusting commercial though. > >As for the: > > > > 19> [40]. > > > "(" > >behaviour, I'd say it's almost a bug, and it should be fixed. > >The proper solution is to subtag integers as characters >(that is: to introduce a builtin char type) which has >the characteristics of an integer except that a new >guard test like: > >f(C) when char(C) -> > >returns true for chars but not for ints, and > >$X + 2. > >should also return a char .... and so on. > > >I think the OTP folks are/have been doing something like this ???? This has been discussed many times and made it into Standard Erlang (which only exists as a spec). There were plans how characters could be introduced into OTP and still be backwards compatible (this is as trivial as it sounds). I don't know if it is still in the pipe-line. The current solution in which the shell prints lists of integers as strings if the values are within range is definitely not perfect but probably better than always printing them as lists. For those who are interested the actual work is done in io_lib:fwrite with the ~p and ~P options. RTFM. Robert From svg@REDACTED Mon Sep 13 22:04:41 1999 From: svg@REDACTED (svg@REDACTED) Date: Tue, 14 Sep 1999 02:04:41 +0600 Subject: What's wrong, help please. Message-ID: <19990914020441V.svg@disney.surnet.ru> Then I was trying to execute a mnesia query, based on two tables, such as from eva/examples/simple_adaptation.erl : Handle = query [{A.name, A.sender, A.cause, A.severity, AlarmDef.class} || A <- table(alarm), AlarmDef <- table(alarmTable), A.name = AlarmDef.name] end, {atomic, Alarms} = mnesia:transaction(fun() -> mnemosyne:eval(Handle) end), ... I had always got error, such as : {'EXIT',{badarg,{ets,lookup,[db_get,[mnemosyne_catalog,{image,alarm}]]}}} ? Query, based on one table, executed well. My environment are RedHat Linux6.0 and Erlang47.4.1 installed from rpm distribution. Regards. --- Vladimir Sekissov svg@REDACTED From dgud@REDACTED Tue Sep 14 07:52:38 1999 From: dgud@REDACTED (Dan Gudmundsson) Date: Tue, 14 Sep 1999 07:52:38 +0200 (MET DST) Subject: What's wrong, help please. In-Reply-To: <19990914020441V.svg@disney.surnet.ru> References: <19990914020441V.svg@disney.surnet.ru> Message-ID: <14301.58022.737797.723969@rian> Try starting mnemosyne first. application:start(mnemosyne). The reason is that simple queries don't need the application to be started, while more complex, in this case 2 tables, queries requires mnemosyne. /Dan svg@REDACTED writes: > > Then I was trying to execute a mnesia query, based on two tables, > such as from eva/examples/simple_adaptation.erl : > > Handle = query [{A.name, A.sender, A.cause, A.severity, AlarmDef.class} || > A <- table(alarm), > AlarmDef <- table(alarmTable), > A.name = AlarmDef.name] end, > {atomic, Alarms} = mnesia:transaction(fun() -> mnemosyne:eval(Handle) end), > ... > > I had always got error, such as : > {'EXIT',{badarg,{ets,lookup,[db_get,[mnemosyne_catalog,{image,alarm}]]}}} ? > > Query, based on one table, executed well. > > My environment are RedHat Linux6.0 and Erlang47.4.1 installed from rpm distribution. > > Regards. > --- > Vladimir Sekissov svg@REDACTED From jim@REDACTED Fri Sep 17 02:02:21 1999 From: jim@REDACTED (Jim Larson) Date: Thu, 16 Sep 1999 17:02:21 -0700 Subject: "Forwarding" of gen_server calls? Message-ID: <199909170002.RAA12770@lefse.jetcafe.org> I'm using using open-source Erlang (erlang_base-47.4.1) on FreeBSD 2.2.8. I'm writing a server S1 using the gen_server behaviour. One of the handle_call() clauses makes a "tail call" to server S2 (which also uses gen_server), i.e. handle_call(Request, From, State) -> %% ... some pre-processing goes on ... Reply = gen_server:call(S2, Arg), {reply, Reply, NewState} However, the call to S2 may block for a long period of time, thus blocking all calls to S1 in the interim. What would be nice is a "tail call" function in the gen_server package that would allows you to forward the From "continuation" to another gen_server module, but returns to its caller immediately. handle_call(Request, From, State) -> %% ... the same pre-processing goes on ... %% This call won't block case gen_server:tail_call(S2, Arg, From) of ok -> {noreply, NewState}; % S2 will reply to our caller {error, Reason} -> %% error looking up S2 {reply, {error, SomeReason}, SomeState} end. The implementation for the simplest clause of tail_call would be something like: tail_call(Pid, Request, From) when pid(Pid), node(Pid) == node() -> Pid ! {'$gen_call', From, Request}, ok. [Okay, so it doesn't quite handle all error cases, since the original gen_server caller will only catch 'EXIT' messages from the Pid of the server it originally called. I'm sure there are ways to deal with this.] I realize that there are other workarounds, including: - returning appropriate information from S1, and allowing the original client process to call S2; - having S1 spawn a new process to make the blocking call; - explicitly passing the From "continuation" as an argument to a special method of S2. However, these workarounds are unappealing. Am I overlooking any other options or existing infrastructure for accomplishing what I want? I await the comments of those wiser in the ways of Erlang than I. Jim From cesarini@REDACTED Fri Sep 17 08:35:59 1999 From: cesarini@REDACTED (F. Cesarini) Date: Fri, 17 Sep 1999 08:35:59 +0200 Subject: "Forwarding" of gen_server calls? Message-ID: <37E1E14F.C6C1B1A5@csd.uu.se> Hi Jim, if you save the From argument in your handle_call function, you can later use it in the gen_server:reply(From, Reply) function call (When S2 asynchronously replies to S1's query). The preferred solution, however is to pass the From argument to S2 (Using an asynchronous function) so as to make S2 call the gen_server:reply/2 function. e.g. in S1: handle_call(Request, From, State) -> %% ... some pre-processing goes on ... Reply = gen_server:cast(S2, {From,Arg}), {noreply, NewState} in S2: handle_cast({From, Arg}, State) -> Reply = whatever(State, Arg), gen_server:reply(From, Reply), {noreply, State}. Cheers, Francesco Jim Larson wrote: > > I'm using using open-source Erlang (erlang_base-47.4.1) on FreeBSD > 2.2.8. > > I'm writing a server S1 using the gen_server behaviour. One of > the handle_call() clauses makes a "tail call" to server S2 (which > also uses gen_server), i.e. > > handle_call(Request, From, State) -> > %% ... some pre-processing goes on ... > Reply = gen_server:call(S2, Arg), > {reply, Reply, NewState} > > However, the call to S2 may block for a long period of time, thus > blocking all calls to S1 in the interim. > > What would be nice is a "tail call" function in the gen_server > package that would allows you to forward the From "continuation" > to another gen_server module, but returns to its caller immediately. > > handle_call(Request, From, State) -> > %% ... the same pre-processing goes on ... > %% This call won't block > case gen_server:tail_call(S2, Arg, From) of > ok -> > {noreply, NewState}; % S2 will reply to our caller > {error, Reason} -> > %% error looking up S2 > {reply, {error, SomeReason}, SomeState} > end. > > The implementation for the simplest clause of tail_call would be > something like: > > tail_call(Pid, Request, From) when pid(Pid), node(Pid) == node() -> > Pid ! {'$gen_call', From, Request}, > ok. > > [Okay, so it doesn't quite handle all error cases, since the original > gen_server caller will only catch 'EXIT' messages from the Pid of > the server it originally called. I'm sure there are ways to deal > with this.] > > I realize that there are other workarounds, including: > > - returning appropriate information from S1, and > allowing the original client process to call S2; > > - having S1 spawn a new process to make the blocking call; > > - explicitly passing the From "continuation" as an > argument to a special method of S2. > > However, these workarounds are unappealing. Am I overlooking any > other options or existing infrastructure for accomplishing what I > want? > > I await the comments of those wiser in the ways of Erlang than I. > > Jim From etxuwig@REDACTED Fri Sep 17 10:40:17 1999 From: etxuwig@REDACTED (Ulf Wiger) Date: Fri, 17 Sep 1999 10:40:17 +0200 (MET DST) Subject: "Forwarding" of gen_server calls? In-Reply-To: <199909170002.RAA12770@lefse.jetcafe.org> Message-ID: On Thu, 16 Sep 1999, Jim Larson wrote: jim>What would be nice is a "tail call" function in the gen_server jim>package that would allows you to forward the From "continuation" jim>to another gen_server module, but returns to its caller immediately. jim> jim> handle_call(Request, From, State) -> jim> %% ... the same pre-processing goes on ... jim> %% This call won't block jim> case gen_server:tail_call(S2, Arg, From) of jim> ok -> jim> {noreply, NewState}; % S2 will reply to our caller jim> {error, Reason} -> jim> %% error looking up S2 jim> {reply, {error, SomeReason}, SomeState} jim> end. jim> jim>The implementation for the simplest clause of tail_call would be jim>something like: jim> jim> tail_call(Pid, Request, From) when pid(Pid), node(Pid) == node() -> jim> Pid ! {'$gen_call', From, Request}, jim> ok. What you're proposing is quite doable in erlang 47.4.0, but becomes tricky in the commercially available R5B release. I'll explain why below. Another way to do it (as you also point out) would be to let the client function handle the redirect: req(To, Request) -> case gen_server:call(To, Request) of {redirect, NewTo, NewRequest} -> req(NewTo, NewRequest); Reply -> Reply end. It leads to a few more messages, but unless you have extreme response-time requirements, or the calls are passed across node limits, it probably doesn't matter much. Since it's common practice to implement API functions as "wrappers" around the gen_server:call() (don't let the implementation show in the interface), the client won't know that it's been redirected. jim>[Okay, so it doesn't quite handle all error cases, since the original jim>gen_server caller will only catch 'EXIT' messages from the Pid of jim>the server it originally called. I'm sure there are ways to deal jim>with this.] Actually, in erlang-47.4.0, the client only detects timeouts and nodedowns in gen:call(). However, in R5B (the latest commercial release), and in the upcoming Open Source release, gen:call() uses a new one-way monitor function to make sure the server doesn't crash while processing the call. This is needed in order to avoid unnecessary latency. The problem, from your perspective, becomes that the client will continue to monitor the first server, even though the call is passed to another server. A function like gen_server:tail_call() wouldn't be able to do much about that, since it would execute in a process other than the one doing the monitoring. jim>However, these workarounds are unappealing. Am I overlooking any jim>other options or existing infrastructure for accomplishing what I jim>want? It's difficult to tell without knowing the specifics of the problem, but sometimes it's possible to let the client to part of the pre-processing. You can also let the client read an ETS table (e.g. maintained by S1) in order to determine which server to address. /Uffe Ulf Wiger, Chief Designer AXD 301 Ericsson Telecom AB tfn: +46 8 719 81 95 Varuv?gen 9, ?lvsj? mob: +46 70 519 81 95 S-126 25 Stockholm, Sweden fax: +46 8 719 43 44 From cesarini@REDACTED Fri Sep 17 12:14:50 1999 From: cesarini@REDACTED (Francesco Cesarini) Date: Fri, 17 Sep 1999 12:14:50 +0200 (MET DST) Subject: "Forwarding" of gen_server calls? In-Reply-To: Message-ID: > jim>[Okay, so it doesn't quite handle all error cases, since the original > jim>gen_server caller will only catch 'EXIT' messages from the Pid of > jim>the server it originally called. I'm sure there are ways to deal > jim>with this.] > > Actually, in erlang-47.4.0, the client only detects timeouts and nodedowns > in gen:call(). However, in R5B (the latest commercial release), and in the > upcoming Open Source release, gen:call() uses a new one-way monitor > function to make sure the server doesn't crash while processing the call. > This is needed in order to avoid unnecessary latency. > > The problem, from your perspective, becomes that the client will continue > to monitor the first server, even though the call is passed to another > server. A function like gen_server:tail_call() wouldn't be able to do much > about that, since it would execute in a process other than the one doing > the monitoring. The above isn't an issue if you place S1 and S2 in the same supervisor tree with a one-for-all or rest-for-one relationship (Where S2 is started before S1) (Or have an application coordinator triggering a restart if they are in different applications). As there is a clear dependency between S1 and S2, if S1 exits abnormally, S2 should also be terminated and restarted. It is obvious that different process/application types have different robustness requirements. I have however always pushed the idea that one should always try to keep the restart strategy as simple as possible, avoiding the handling of all special cases which can occur (And decentralize in a generic module if some special cases have to be catered for as they occur often), as these work arounds often lead to new bugs instead of solving their predefined tasks. Thereof, if S2 exits, make S1 exit as well, and restart them. Cheers, Francesco From tmb-erlang@REDACTED Mon Sep 20 16:06:36 1999 From: tmb-erlang@REDACTED (tmb-erlang@REDACTED) Date: Mon, 20 Sep 1999 07:06:36 -0700 (PDT) Subject: string performance Message-ID: <199909201406.HAA14485@ shell1.ncal.verio.com> In many ways, Erlang looks very good for building distributed web applications, but its string performance is very poor in my benchmarks: string append is orders of magnitude slower than in Perl, and characters take many bytes to store (compared to one byte per character in other languages). I'm curious whether there are any plans to address this. One approach would be to transparently switch representations between lists and strings, like Tcl does. That would be completely backwards compatible. An alternative would be to define a separate string type, define new pattern matching syntax for true strings or prohibit pattern matching on true strings altogether; this would be backwards compatible, but less interopreable between old and new code (I still prefer this latter choice). So, what are the plans? I know I can use byte arrays, but that doesn't seem like it's quite the same as having a real string type. Thanks, Thomas. From etxuwig@REDACTED Mon Sep 20 16:26:50 1999 From: etxuwig@REDACTED (Ulf Wiger) Date: Mon, 20 Sep 1999 16:26:50 +0200 (MET DST) Subject: string performance In-Reply-To: <199909201406.HAA14485@ shell1.ncal.verio.com> Message-ID: One thing you can do today is to use binaries and deep lists to speed up processing in web applications. This has to do with the fact that an Erlang port accepts anything that is a mix of binaries and byte lists. Basically, anything that works with erlang:list_to_binary([...]) will work with a port, and with web applications, everything will eventually go through a port. Thus, you can append two strings by writing [String1,String2], and you can also use erlang:list_to_binary/1 on strings which do not need further manipulation. Also, binary_to_list(list_to_binary(DeepList)) can be significantly faster than lists:flatten(DeepList). There is talk of a string syntax, but more importantly (I think) is the upcoming bit syntax, which will allow you to manipulate binaries directly, including pattern matching of binary data. The syntax is not set yet, but an early proposal suggested something like: parse(<"GET ", What/binary, <"\r\n"> | Tail>) -> {Fields, Contents} = parse_tail(Tail, empty, []), {What, Fields, Contents}. parse_tail( | Cont>, A) -> {Cont, Ack}; parse_tail( | Tail>, A) -> parse_tail(Tail, [B|A]). (Example of parsing the binary data of a HTTP request) Take the syntax with a grain of salt. It has changed since then, and I don't know the latest details. /Uffe On Mon, 20 Sep 1999 tmb-erlang@REDACTED wrote: tmb-er>In many ways, Erlang looks very good for building distributed tmb-er>web applications, but its string performance is very poor in my tmb-er>benchmarks: string append is orders of magnitude slower than tmb-er>in Perl, and characters take many bytes to store (compared to one tmb-er>byte per character in other languages). tmb-er> tmb-er>I'm curious whether there are any plans to address this. tmb-er>One approach would be to transparently switch representations tmb-er>between lists and strings, like Tcl does. That would be tmb-er>completely backwards compatible. An alternative would be to tmb-er>define a separate string type, define new pattern matching tmb-er>syntax for true strings or prohibit pattern matching on true tmb-er>strings altogether; this would be backwards compatible, but less tmb-er>interopreable between old and new code (I still prefer this latter tmb-er>choice). tmb-er> tmb-er>So, what are the plans? I know I can use byte arrays, but that tmb-er>doesn't seem like it's quite the same as having a real string type. tmb-er> tmb-er>Thanks, tmb-er>Thomas. tmb-er> Ulf Wiger, Chief Designer AXD 301 Ericsson Telecom AB tfn: +46 8 719 81 95 Varuv?gen 9, ?lvsj? mob: +46 70 519 81 95 S-126 25 Stockholm, Sweden fax: +46 8 719 43 44 From klacke@REDACTED Mon Sep 20 16:27:29 1999 From: klacke@REDACTED (Klacke) Date: Mon, 20 Sep 1999 16:27:29 +0200 (CEST) Subject: string performance In-Reply-To: <199909201406.HAA14485@ shell1.ncal.verio.com> References: <199909201406.HAA14485@ shell1.ncal.verio.com> Message-ID: <14310.17489.681924.598919@kricka.bluetail.com> > In many ways, Erlang looks very good for building distributed > web applications, but its string performance is very poor in my > benchmarks: string append is orders of magnitude slower than > in Perl, and characters take many bytes to store (compared to one > byte per character in other languages). > > I'm curious whether there are any plans to address this. > One approach would be to transparently switch representations > between lists and strings, like Tcl does. That would be > completely backwards compatible. An alternative would be to > define a separate string type, define new pattern matching > syntax for true strings or prohibit pattern matching on true > strings altogether; this would be backwards compatible, but less > interopreable between old and new code (I still prefer this latter > choice). > > So, what are the plans? I know I can use byte arrays, but that > doesn't seem like it's quite the same as having a real string type. The last thing I did while I was still at Ericsson was an ambitios atempt do address not only the string issue but a number of other related buffer management issues together with Tony Rogvall. We worked with it for almost a year and the end result was a working implementation and a report. The implementation was never released neither as open source nor internally in ericsson other than to our colleagues. The results from the prototype were quite promising though, both in regards to execution efficency as well as clarity of the code. A report describing the proposed language extensions is available at: http://www.bluetail.com/klacke/binaries.ps and I'll be presenting this at the Erlang user conf which is due real soon now. Now, this work was a prototype and it would need quite a lot of work to implement it properly and I have no idea what plans the OTP group have. Furthermore, the implementation was jam specific and I know the OTP group are going for the BEAM machine instead. /klacke Claes Wikstr?m Tel: +46 (0)8 692 22 09 Bluetail AB Email: klacke@REDACTED Hantverkargatan 78 WWW: http://www.bluetail.com SE-112 38 Stockholm, SWEDEN From luke@REDACTED Tue Sep 21 07:59:20 1999 From: luke@REDACTED (Luke Gorrie) Date: 21 Sep 1999 15:59:20 +1000 Subject: hex2dec parsing for Inets Message-ID: <87u2oo9553.fsf@baked.vegetable.org> Bug report for Inets in httpd_util code: hex2dec(X) when X>=$0,X=<$9 -> X-$0; hex2dec($A) -> 10; hex2dec($B) -> 11; hex2dec($C) -> 12; hex2dec($D) -> 13; hex2dec($E) -> 14; hex2dec($F) -> 15. Lower-case should be handled too for compatibility (eg. with Lynx). I added a "hex2dec(C) when C >= $a, C =< $f -> 10 + C - $a." clause to the same code in 'pico' to make lynx posts work. What's a better place to send this sort of mail, to the email addresses listed in the source files? I've done that in the past but I'm not sure of the convention since those addresses were put there in pre-opensource days. Cheers, Luke From seb@REDACTED Tue Sep 21 15:57:53 1999 From: seb@REDACTED (Sebastian Strollo) Date: 21 Sep 1999 15:57:53 +0200 Subject: hex2dec parsing for Inets In-Reply-To: Luke Gorrie's message of "21 Sep 1999 15:59:20 +1000" References: <87u2oo9553.fsf@baked.vegetable.org> Message-ID: Luke Gorrie writes: > Bug report for Inets in httpd_util code: ... Thanks. Fixed it. > What's a better place to send this sort of mail, to the email > addresses listed in the source files? I've done that in the past but > I'm not sure of the convention since those addresses were put there in > pre-opensource days. You are more than welcome to send fixes (preferrably as diff's) to erlang-maintainers@REDACTED This is an alias which leads to a number of individuals who activly work on Erlang development. One of them will pick up the fix, see if it fits and then include it in the source. -- Sebastian From tmb@REDACTED Wed Sep 22 08:39:40 1999 From: tmb@REDACTED (tmb@REDACTED) Date: Tue, 21 Sep 1999 23:39:40 -0700 (PDT) Subject: string performance Message-ID: <199909220639.XAA21291@ shell1.ncal.verio.com> Hi, thanks for the response. I'm glad to see that binaries and deep lists are available for speeding things up. I like the lightweight, dynamic, and distributed character of Erlang; the string processing seems like the weakest point right now to me, in particular given the profusion of text-based Internet protocols and standards (HTTP, XML, POP, SMTP, etc.). Maybe it's just because I don't have much experience with it, but the problem I see with relying on deep lists of binaries is that it may make code harder to maintain and debug. It certainly makes it harder to explain to a new Erlang programmer how to do string processing. > There is talk of a string syntax, but more importantly (I think) is the > upcoming bit syntax, which will allow you to manipulate binaries directly, > including pattern matching of binary data. I think that kind of syntax would be great. I don't think, however, that using the same data type for strings is such a good idea: if the distinction between binary and text isn't made early on in the evolution of a language, it's very difficult to retrofit code later, in particular when issues like UNICODE support come up. Also, binary and text should behave (print) differently in an interactive development environment. It seems to me that the syntax, implementation, and functionality for "binary" and "text" could be almost the same; but carrying around one extra bit of type information to distinguish the two cases would seem very useful to me. In addition to the syntax, there is also the question of what a good underlying representation of strings should be in a mostly functional language. Trees of chunks, lists of blocks, and pointer pairs (start/end) into a buffer that's extensible at both ends but otherwise unmodifiable are all possibilities. Before the language changes, I'm wondering: for getting things done right now, are there any efficient and powerful string packages around that can deal with large amounts of variable text and handle things like string substitutions? Packages that avoid converting back and forth between various list types and binaries? Cheers, Thomas. From jfk@REDACTED Wed Sep 22 21:39:14 1999 From: jfk@REDACTED (Joergen Froejk Kjaersgaard) Date: Wed, 22 Sep 1999 21:39:14 +0200 Subject: Minimal embedded Erland system Message-ID: <37E93062.DAD7FF9F@informaticon.dk> I have tried to make a minimal embedded Erlang system to use as a template. However, when i start the system - even in embedded mode - I get an Erlang shell prompt. The files involved are: test.erl: ====== -module(test). -behaviour(application). -export([hello_sayer/0, start/2, stop/1]). hello_sayer() -> io:put_chars("hello, world\n"), receive _ -> [] end. start(Type, Args) -> io:put_chars("Application started\n"), P = spawn(test, hello_sayer, []), {ok, P, P}. stop(P) -> io:put_chars("Application has exited\n"), exit(P). test.app: ======= {application, test, [{description, "A simple test application"}, {vsn, "1.0"}, {id, "Test"}, {modules, [{test,"1.0"}]}, {maxP, infinity}, {maxT, infinity}, {registered, []}, {applications, [kernel, stdlib]}, {included_applications, []}, {env, []}, {mod, {test, []}}, {start_phases, []}]}. test.rel: ===== {release, {"test","1.0"}, {erts, "4.4"}, [{test, "1.0", permanent}, {kernel, "2.1.4", permanent}, {stdlib, "1.5.2", permanent}]}. I make a boot script using systools:make_script() and start the system using "erl -boot test -mode embedded -pz /home/jfk/test/ebin". The system writes "Application started" and "hello, world" as expected but then a beam shell prompt appears! I expected the system to wait forever (because nothing is ever sent to the started process). What am I doing wrong? How can I make a minimal system which can be started from a Unix shell, write something to standard output and then exit to the Unix shell? -- J?rgen Fr?jk Kj?rsgaard, Systemkonsulent (Systems Consultant) Informaticon ApS * N?rrev?nget 143 * DK-8310 Tranbjerg J Tlf: 8672 0093 * E-mail: jfk@REDACTED * http://www.informaticon.dk -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobbe@REDACTED Wed Sep 22 23:17:00 1999 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: 23 Sep 1999 07:17:00 +1000 Subject: Minimal embedded Erland system In-Reply-To: Joergen Froejk Kjaersgaard's message of "Wed, 22 Sep 1999 21:39:14 +0200" References: <37E93062.DAD7FF9F@informaticon.dk> Message-ID: > I have tried to make a minimal embedded Erlang system to use as a > template. However, when i start the system - even in embedded mode - I > get an Erlang shell prompt. Try out the following program (included below). Cheers /Tobbe -module(pipe). %%% ==================================================================== %%% Example how to use Erlang in a series of Unix pipes %%% --------------------------------------------------- %%% First create a file with some in data: %%% %%% cat > indata %%% 1 2 3 4 5 %%% 5 4 3 2 1 %%% ^D (NB: Control-D) %%% %%% Then run the following: %%% %%% erl -s pipe -noinput < indata | tee outdata %%% 1 2 3 4 5 %%% 5 4 3 2 1 %%% %%% (Also, inspect the created file: cat outdata ) %%% ==================================================================== -export([start/0]). -export([init/0]). start() -> spawn(?MODULE,init,[]). init() -> Port = open_port({fd, 0, 1},[eof]), loop(Port). loop(Port) -> receive {Port, {data, What}} -> Port ! {self(), {command, What}}, loop(Port); {Port, eof} -> halt() end. From klacke@REDACTED Thu Sep 23 09:23:11 1999 From: klacke@REDACTED (Klacke) Date: Thu, 23 Sep 1999 09:23:11 +0200 (CEST) Subject: Minimal embedded Erland system In-Reply-To: <37E93062.DAD7FF9F@informaticon.dk> References: <37E93062.DAD7FF9F@informaticon.dk> Message-ID: <14313.54623.782400.435463@kricka.bluetail.com> > I make a boot script using systools:make_script() and start the system > using "erl -boot test -mode embedded -pz /home/jfk/test/ebin". The If you replace the "-mode embedded " with -noshell, the shell isn't started. (Also note that the erlc compiler knows how to compile .rel files, we can just call erlc test.rel to produce the script+boot files.) An even easier way to make a small script to run your erlang code is erl -noshell -s test > shell, write something to standard output and then exit to the Unix io:format(... writes to stdout, halt() exits the erlang system back to the unix shell. There is no _easy_ way to write to stderr. /klacke From etxuwig@REDACTED Thu Sep 23 10:20:00 1999 From: etxuwig@REDACTED (Ulf Wiger) Date: Thu, 23 Sep 1999 10:20:00 +0200 (MET DST) Subject: Minimal embedded Erland system In-Reply-To: <37E93062.DAD7FF9F@informaticon.dk> Message-ID: On Wed, 22 Sep 1999, Joergen Froejk Kjaersgaard wrote: jfk>I have tried to make a minimal embedded Erlang system to use as a jfk>template. However, when i start the system - even in embedded mode jfk>- I get an Erlang shell prompt. The '-mode embedded' is a misnomer. What it means is that the erlang system will preload all the modules instead of loading on demand. /Uffe Ulf Wiger, Chief Designer AXD 301 Ericsson Telecom AB tfn: +46 8 719 81 95 Varuv?gen 9, ?lvsj? mob: +46 70 519 81 95 S-126 25 Stockholm, Sweden fax: +46 8 719 43 44 From joe@REDACTED Thu Sep 23 11:08:12 1999 From: joe@REDACTED (Joe Armstrong) Date: Thu, 23 Sep 1999 11:08:12 +0200 (CEST) Subject: Minimal embedded Erland system In-Reply-To: <37E93062.DAD7FF9F@informaticon.dk> Message-ID: On Wed, 22 Sep 1999, Joergen Froejk Kjaersgaard wrote: > I have tried to make a minimal embedded Erlang system to use as a > template. However, when i start the system - even in embedded mode - I > get an Erlang shell prompt. > ... Ummm :-) > test.erl: > ====== > -module(test). > -behaviour(application). > -export([hello_sayer/0, start/2, stop/1]). > > hello_sayer() -> > io:put_chars("hello, world\n"), > receive _ -> [] > end. ... Minimal! - the very fact that you call "io:" means you have to pull in a dozen or so modules and it won't be very minimal! ... cut ... > test.app: > ======= > {application, test, > [{description, "A simple test application"}, ... etc ... Using applications is the wrong way to go if you are making miminimal systems. To find out how to do this read section 1.4 of "System principles" very carefully. It's at http://www.erlang.org/doc/doc/doc/system_principles/system_principles.html#1.4 You will need to pre-load exactly two modules to make a minimal system init.erl and erl_prim_loader.erl In http://www.erlang.org/examples/examples-2.0.html under the section sos you will find a description of a complete mini OS which can you simple io - using sos instead of io you can write: -module(sos_test1). -export([main/0]). main() -> sos:write("Hello world\n"). To run this we must first compile sos_test1 using the compiler in the Erlang development environment. Once we have done this then we can run the program as follows: unix> sos sos_test1 Hello world We can time this as follows: > time sos sos_test1 Hello world 0.09u 0.04s 0:00.37 35.1 The details are in http://www.erlang.org/examples/examples-2.0.html Actually sos.erl is worthly of study since it *is* an entire applocation OS in a few hundred lines. /Joe -- Joe Armstrong, Bluetail AB, tel: +46 8 692 22 11 Hantverkargatan 78, fax: +46 8 654 70 71 SE-112 38 Stockholm, Sweden info: www.bluetail.com From joe@REDACTED Thu Sep 23 11:15:13 1999 From: joe@REDACTED (Joe Armstrong) Date: Thu, 23 Sep 1999 11:15:13 +0200 (CEST) Subject: Minimal embedded Erland system In-Reply-To: <14313.54623.782400.435463@kricka.bluetail.com> Message-ID: On Thu, 23 Sep 1999, Klacke wrote: > > > > > I make a boot script using systools:make_script() and start the system > > using "erl -boot test -mode embedded -pz /home/jfk/test/ebin". The > > If you replace the "-mode embedded " with -noshell, the shell isn't > started. (Also note that the erlc compiler knows how to compile > .rel files, we can just call erlc test.rel to produce the script+boot > files.) > > An even easier way to make a small script to run your erlang code is > erl -noshell -s test > > > shell, write something to standard output and then exit to the Unix > > io:format(... writes to stdout, halt() exits the erlang system back > to the unix shell. > > There is no _easy_ way to write to stderr. > > > /klacke If you want to make a minimal system you should not use io but hit the IO file descriptors *directly* here is cat.erl which does this: main(_) -> Port = erlang:open_port_prim({fd,0,1}, [eof, binary]), loop(Port). loop(Port) -> case read(Port) of eof -> true; {ok, X} -> write(Port, [X]), loop(Port) end. read(Port) -> receive {Port, {data, Bytes}} -> {ok, Bytes}; {Port, eof} -> eof end. write(Port, X) -> Port ! {self(), {command, X}}. If the next release of OSE contains the shared library and tiny application builder code stuff then this will compile to a 872 byte executable. <> /Joe -- Joe Armstrong, Bluetail AB, tel: +46 8 692 22 11 Hantverkargatan 78, fax: +46 8 654 70 71 SE-112 38 Stockholm, Sweden info: www.bluetail.com From justint@REDACTED Tue Sep 28 13:05:17 1999 From: justint@REDACTED (justint) Date: Tue, 28 Sep 1999 05:05:17 -0600 Subject: Building Erlang and rc? Message-ID: <37F0A0ED.A81E9AFB@automallusa.com> So Im trying to build erlang on a sun running solaris 7... I run the configure, no errors and then all I get is this error (so what is this rc stuff anyway)? any idea would rock... err please respond via mail and messages board, thanks Justin bash-2.03# make cd erts/system && ERL_TOP=/temp/erlang-47.4.1 make EMULATOR=jam opt make[1]: Entering directory `/temp/erlang-47.4.1/erts/system' make[2]: Entering directory `/temp/erlang-47.4.1/erts/system/emulator/zlib' make -f sparc-sun-solaris2.7/Makefile TYPE=opt make[3]: Entering directory `/temp/erlang-47.4.1/erts/system/emulator/zlib' rc /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/libz.a /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/adler32.o /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/compress.o /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/crc32.o /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/uncompr.o /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/deflate.o /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/trees.o /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/zutil.o /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/inflate.o /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/infblock.o /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/inftrees.o /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/infcodes.o /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/infutil.o /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/inffast.o make[3]: rc: Command not found make[3]: *** [/temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/libz.a] Error 127 make[3]: Leaving directory `/temp/erlang-47.4.1/erts/system/emulator/zlib' make[2]: *** [opt] Error 2 make[2]: Leaving directory `/temp/erlang-47.4.1/erts/system/emulator/zlib' make[1]: *** [opt] Error 2 make[1]: Leaving directory `/temp/erlang-47.4.1/erts/system' make: *** [emulator] Error 2 From vlad.dumitrescu@REDACTED Tue Sep 28 09:12:21 1999 From: vlad.dumitrescu@REDACTED (Vlad Dumitrescu) Date: Mon, 27 Sep 1999 23:12:21 -800 Subject: micro or pico erlang Message-ID: <37f05c45.18d.0@flashmail.com> Hi! A topic on the "to do" list is making a small version of erlang (that will fit on a floppy, I think the goal is). Does anyone work at it? Or an even smaller one? Would it even be possible to have Erlang+applications run in 64k or even 32k? regards, /Vlad ______________________________________________________ Get Your FREE FlashMail Address now at http://www.flashmail.com It's Free, Easy, & Fun !!! From klacke@REDACTED Tue Sep 28 11:44:50 1999 From: klacke@REDACTED (Klacke) Date: Tue, 28 Sep 1999 11:44:50 +0200 (CEST) Subject: Building Erlang and rc? In-Reply-To: <37F0A0ED.A81E9AFB@automallusa.com> References: <37F0A0ED.A81E9AFB@automallusa.com> Message-ID: <14320.36370.595253.133680@kricka.bluetail.com> justint writes: > So Im trying to build erlang on a sun running solaris 7... > I run the configure, no errors and then all I get is this error (so > what is this rc stuff anyway)? > any idea would rock... err please respond via mail and messages board, > thanks > Justin > > bash-2.03# make > cd erts/system && ERL_TOP=/temp/erlang-47.4.1 make EMULATOR=jam opt > make[1]: Entering directory `/temp/erlang-47.4.1/erts/system' > make[2]: Entering directory > `/temp/erlang-47.4.1/erts/system/emulator/zlib' > make -f sparc-sun-solaris2.7/Makefile TYPE=opt > make[3]: Entering directory > `/temp/erlang-47.4.1/erts/system/emulator/zlib' > rc /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/libz.a > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/adler32.o > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/compress.o > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/crc32.o > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/uncompr.o > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/deflate.o > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/trees.o > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/zutil.o > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/inflate.o > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/infblock.o > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/inftrees.o > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/infcodes.o > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/infutil.o > /temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/inffast.o > make[3]: rc: Command not found > make[3]: *** [/temp/erlang-47.4.1/erts/obj/sparc-sun-solaris2.7/libz.a] > Error 127 > make[3]: Leaving directory > `/temp/erlang-47.4.1/erts/system/emulator/zlib' > make[2]: *** [opt] Error 2 > make[2]: Leaving directory > `/temp/erlang-47.4.1/erts/system/emulator/zlib' > make[1]: *** [opt] Error 2 > make[1]: Leaving directory `/temp/erlang-47.4.1/erts/system' > make: *** [emulator] Error 2 > Weird stuff, The 'rc' however are flags to the 'ar' command which is supposed to run here. I.e: ar rc /.../libz.a All the dot.o files Looks like some kind of configure error. Hmmmm .... I just talked to some of the other bluetail guys here who built erlang 47.4.1 on solaris 2.7 a couple of months ago and they said something about adding /usr/ccs/bin to the path. (We don't have solaris 2.7 here any longer so I can't try it out) Try that. ---- diversion ---- By the way, what about the version numbering on Solaris ... gets me pretty confused, solaris 7.0, 5.7 and 2.7 are all the same or what ??? ---------------------- Cheers /klacke Claes Wikstr?m Tel: +46 (0)8 692 22 09 Bluetail AB Email: klacke@REDACTED Hantverkargatan 78 WWW: http://www.bluetail.com SE-112 38 Stockholm, SWEDEN From SEAN.HINDE@REDACTED Tue Sep 28 11:54:00 1999 From: SEAN.HINDE@REDACTED (SEAN HINDE) Date: Tue, 28 Sep 1999 10:54:00 +0100 Subject: Problem with make:all() Message-ID: <36073178.280899@smtpgate1.ONE2ONE.CO.UK> All, When using an Emakefile with the source files in a directory other than the current one e.g '../src/xxx.erl', make:all() currently recompiles all files regardless of date created. This has been driving me cray for a while so I finally got around to fixing it in 47.4.1. 161c161 < case exists(append(File, code:objfile_extension())) of --- > case exists(append(filename:basename(File), code:objfile_extension())) of 170c170 < {ok, Obj} = file:file_info(append(File, code:objfile_extension())), --- > {ok, Obj} = file:file_info(append(filename:basename(File), code:objfile_extension())), Regards, Sean Hinde From per@REDACTED Tue Sep 28 12:30:49 1999 From: per@REDACTED (Per Hedeland) Date: Tue, 28 Sep 1999 12:30:49 +0200 (MET DST) Subject: Building Erlang and rc? Message-ID: <199909281030.MAA21585@aalborg.du.uab.ericsson.se> >Hmmmm .... I just talked to some of the other bluetail guys here >who built erlang 47.4.1 on solaris 2.7 a couple of months ago >and they said something about adding /usr/ccs/bin to the path. >(We don't have solaris 2.7 here any longer so I can't try it out) Oh yes, you need to have /usr/ccs/bin in your $PATH to build pretty much anything on Solaris - regardless of version. That's where not just 'ar', but also 'as', 'ld', and various of other developer stuff traditionally lives on Solaris. >---- diversion ---- >By the way, what about the version numbering on Solaris ... >gets me pretty confused, solaris 7.0, 5.7 and 2.7 are all the >same or what ??? >---------------------- ---- more diversion:-) ------ Actually none of those are (politically:-) correct Solaris version numbers, the proper one is '7', which includes *SunOS* 5.7, CDE x.y, and some other stuff. According to the tradition (Solaris 2.x includes SunOS 5.x) it ought to have been Solaris 2.7, but they made a last- minute decision to change it (and missed a couple of places, such as the CD:-) - I guess someone decided that it was a more "impressive" number or something. Next version will be Solaris 8, with SunOS 5.8 (currently in late beta)... But who are we to talk, going from 4.7.3 to 47.4.0.:-) --Per Hedeland per@REDACTED From vances@REDACTED Tue Sep 28 23:32:35 1999 From: vances@REDACTED (Vance Shipley) Date: Tue, 28 Sep 1999 17:32:35 -0400 Subject: make tests Message-ID: <003001bf09f8$faea9440$0abdc5c0@desk.motivity.ca> There seems to be something set up for making tests in some of the Makefiles. If this is working how do you run it? -Vance From klacke@REDACTED Wed Sep 29 11:35:55 1999 From: klacke@REDACTED (Klacke) Date: Wed, 29 Sep 1999 11:35:55 +0200 (CEST) Subject: make tests In-Reply-To: <003001bf09f8$faea9440$0abdc5c0@desk.motivity.ca> References: <003001bf09f8$faea9440$0abdc5c0@desk.motivity.ca> Message-ID: <14321.56699.269405.215357@kricka.bluetail.com> > There seems to be something set up for making tests > in some of the Makefiles. If this is working how do > you run it? > There exists a HUGE test suite for the entire erlang system. The test suite is pretty tied up to the local build environment at Ericsson. You don't want to run it. When we assembled the open source erlang kit, we discussed whether to try to relase the test suite as well. It wasn't feasible though. Besides it's not that interesting unless you are hacking mnesia or the kernel or something like that. Cheers /klacke Claes Wikstr?m Tel: +46 (0)8 692 22 09 Bluetail AB Email: klacke@REDACTED Hantverkargatan 78 WWW: http://www.bluetail.com SE-112 38 Stockholm, SWEDEN From luke@REDACTED Wed Sep 29 12:37:22 1999 From: luke@REDACTED (Luke Gorrie) Date: 29 Sep 1999 20:37:22 +1000 Subject: make tests In-Reply-To: Klacke's message of "Wed, 29 Sep 1999 11:35:55 +0200 (CEST)" References: <003001bf09f8$faea9440$0abdc5c0@desk.motivity.ca> <14321.56699.269405.215357@kricka.bluetail.com> Message-ID: <87emfi6m1p.fsf@baked.vegetable.org> Klacke writes: > When we assembled the open source erlang kit, we discussed > whether to try to relase the test suite as well. It wasn't > feasible though. Besides it's not that interesting unless > you are hacking mnesia or the kernel or something like > that. Actually, I was hacking Mnesia's transaction manager today and was about to ask on Wikie if we could have a play with the test suite. :-) I think it would be great if it was released as open source, but understand if it's a bit tangled up in larger things. I certainly plan on doing some kernel hacking before too long too. ;-) P.S., someone mentioned that the wait before restart on aborted transactions had been reduced in the closed-source Mnesia. Could someone tell me the new value? Cheers, Luke From vances@REDACTED Wed Sep 29 17:54:08 1999 From: vances@REDACTED (Vance Shipley) Date: Wed, 29 Sep 1999 11:54:08 -0400 Subject: make tests In-Reply-To: <14321.56699.269405.215357@kricka.bluetail.com> Message-ID: <003a01bf0a92$dd92ed80$0abdc5c0@desk.motivity.ca> Klacke, There really ought to be a "make test" stage in the build though. Once I've completed a port I want to know that it works. A comprehensive test suite that tested each feature would be best but something that at least stressed the system a bit more than pulling a shell prompt would be nice. I have been using your estone.erl code to do so here. Is that a pretty good way to test? -Vance } There exists a HUGE test suite for the entire } erlang system. The test suite is pretty tied up to } the local build environment at Ericsson. You don't } want to run it. } } When we assembled the open source erlang kit, we discussed } whether to try to relase the test suite as well. It wasn't } feasible though. Besides it's not that interesting unless } you are hacking mnesia or the kernel or something like } that. } } Cheers } } /klacke From Olivier.Lefevre@REDACTED Wed Sep 29 18:38:59 1999 From: Olivier.Lefevre@REDACTED (Olivier.Lefevre@REDACTED) Date: Wed, 29 Sep 1999 12:38:59 -0400 Subject: Building Erlang and rc? In-Reply-To: <14320.36370.595253.133680@kricka.bluetail.com> Message-ID: > Looks like some kind of configure error. > > Hmmmm .... I just talked to some of the other bluetail guys here > who built erlang 47.4.1 on solaris 2.7 a couple of months ago > and they said something about adding /usr/ccs/bin to the path. I was bitten by this recently, too, with another program (emacs). My prb was that I had told configure (through $CPP) to use "cc -E" as cpp, which is ANSI cpp, and not /usr/ccs/bin/cpp, which is (or so I gather) the K&R cpp. You won't believe how muich it screwed up things. GNU stuff needs /usr/ccs/bin/cpp. That is probably the issue here too because the only other ar is /usr/xpg4/bin/ar, which 1/ almost nobody has in his path anyhow and 2/ accepts the rc flags, too. -- O.L. PS: Pls ignore crap below. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. From klacke@REDACTED Wed Sep 29 22:53:13 1999 From: klacke@REDACTED (Klacke) Date: Wed, 29 Sep 1999 22:53:13 +0200 (CEST) Subject: safedets Message-ID: <14322.31801.899127.579042@duva.bluetail.com> I've been hacking madly on the dets server lately. Basically I've been having two problems with it that are dependant on each other. 1. It takes a very long time to create a dets file with say 1. million entries in it. 2. If such a file is not properly closed, it takes ages to repair it. My first attempt was to hack the dets server and pull some of the hash-ndex structures that are now on disc up into RAM. This helped speed with about 30% but increased memory consumption substantially. Not ok. My second attempt was to scrap dets alltogether and use gdbm instead. So I wrote a linked in driver to gdbm. It turns out that gdbm is about equally fast as dets on lookups, and about 4 times faster on the first 10.000 items on insertions. However on the last 10.000 objects (Inserting 1 million itemes) gdbm is >100 times slower. It starts to degenerate around 100.000 items. So, this is good news, dets was better than I thought. Much better. Nevertheless I attacked the second problem instead and came up with the idea to use 2 dets files instead. I posted the code to http://www.erlang.org/user.html#safedets-1.0 Each safedets:open_file/2 makes a directory holding the 2 detsfiles + a log. safedets files are never repaired. If one of the dets files need to be repaired, the other doesn't. A regular file copy is performed. This is much much faster that a logical dets copy which is performed by the repair process. Another thought for the mnesia guys (Dan+Hakan), maybe it's agood idea to apply the same tactics on all mnesia files that are disc_only_copies. This way we can have really large mnesia databases. Any thoughts ... ?? /klacke From hakan@REDACTED Wed Sep 29 23:16:32 1999 From: hakan@REDACTED (Hakan Mattsson) Date: Wed, 29 Sep 1999 23:16:32 +0200 (MET DST) Subject: make tests In-Reply-To: <87emfi6m1p.fsf@baked.vegetable.org> Message-ID: On 29 Sep 1999, Luke Gorrie wrote: luke>Actually, I was hacking Mnesia's transaction manager today and was Great! What are you doing? luke> about to ask on Wikie if we could have a play with the test luke> suite. :-) I think it would be great if it was released as open luke> source, but understand if it's a bit tangled up in larger things. If your hacking is more serious than playing ;-), you will definitely need the test suite. The test suite of Mnesia is completely standalone, since it contains an own customized lightweight test server, developed for the purpose of testing and debugging Mnesia. The big test server of Erlang/OTP actually originated from this test server, once upon the time. I cannot see any problems with releasing the test suite of Mnesia (and possibly Mnemosyne and Mnesia Session) as open source, but this is up to the product manager of Erlang/OTP to decide. luke> P.S., someone mentioned that the wait before restart on aborted luke> transactions had been reduced in the closed-source Mnesia. Could luke> someone tell me the new value? In Mnesia 3.8, the implementation is as follows: random_time(Retries) -> UpperLimit = 500, Dup = Retries * Retries, MaxIntv = trunc(UpperLimit * (1-(50/((Dup)+50)))), case get(random_seed) of undefined -> {X, Y, Z} = erlang:now(), %% time() random:seed(X, Y, Z), Dup + random:uniform(MaxIntv); _ -> Dup + random:uniform(MaxIntv), end. /H?kan From vances@REDACTED Wed Sep 29 23:57:27 1999 From: vances@REDACTED (Vance Shipley) Date: Wed, 29 Sep 1999 17:57:27 -0400 Subject: short host names Message-ID: <005201bf0ac5$9e557920$0abdc5c0@desk.motivity.ca> All, I don't seem to be able to communicate between two nodes on a machine when using short node names. It works fine when I use long node names (e.g. erl -name foo). I start up a node on one terminal: $ erl -sname foo Erlang (BEAM) emulator version 47.4.1 Eshell V47.4.1 (abort with ^G) (foo@REDACTED)1> Then start another on another terminal: $ erl -sname bar Erlang (BEAM) emulator version 47.4.1 Eshell V47.4.1 (abort with ^G) (bar@REDACTED)1> Now I try to ping: foo@REDACTED)1> net_adm:ping(bar@REDACTED). pang Names doesn't work either: (bar@REDACTED)2> net_adm:names(). {ok,[]} I ran epmd with debugging turned on an it looks like epmd is getting the requests and responding properly. This is all under my port to Unixware 2.1.3 so it could well be my port is incomplete but I don't know what to look for at this point. -Vance From hakan@REDACTED Thu Sep 30 00:26:52 1999 From: hakan@REDACTED (Hakan Mattsson) Date: Thu, 30 Sep 1999 00:26:52 +0200 (MET DST) Subject: safedets In-Reply-To: <14322.31801.899127.579042@duva.bluetail.com> Message-ID: On Wed, 29 Sep 1999, Klacke wrote: klacke>2. If such a file is not properly closed, it takes ages klacke> to repair it. Yes, repair of dets-files is a pain. Recently, I introduced a new mechanism (auto_save) to dets which also aims to avoid the absurd repair times of possibly damaged dets-files. My approach was to delay the "not properly closed" mark until the first update actually was performed to the table. Later, when no updates has been performed for a while, the file was automatically marked as closed again. This approach makes the probability for repair lower, but does not eliminate it. klacke>Nevertheless I attacked the second problem instead klacke>and came up with the idea to use 2 dets files instead. klacke>I posted the code to klacke> klacke>http://www.erlang.org/user.html#safedets-1.0 klacke> klacke> klacke>Each safedets:open_file/2 makes a directory holding the klacke>2 detsfiles + a log. safedets files are never repaired. klacke>If one of the dets files need to be repaired, the other klacke>doesn't. A regular file copy is performed. This is much much klacke>faster that a logical dets copy which is performed by the klacke>repair process. klacke> klacke>Another thought for the mnesia guys (Dan+Hakan), maybe klacke>it's agood idea to apply the same tactics on all klacke>mnesia files that are disc_only_copies. This way we klacke>can have really large mnesia databases. Another approach is to avoid very large dets-files. By using the feature of fragmented tables in Mnesia, a table may be split into lots of fragments where each fragment is implemented as a normal Mnesia table. This will keep the dets-files small and therefore improve the repair time, especially together with the new auto_save mechanism in dets. klacke>Any thoughts ... ?? Your algorithm seems to be very promising, since it totally eliminates the need for repair. But, the drawback is performance and disk storage. - Have you made any measurements? - Appending to a log-file is not for free, but perhaps neglectible? - What happens with the availability of the dets table when the write threshold is reached? Does your algorithm allow table access (both reads and updates) during the log propagation or will the table access be blocked? /H?kan From tobbe@REDACTED Thu Sep 30 00:52:31 1999 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: 30 Sep 1999 08:52:31 +1000 Subject: safedets In-Reply-To: Hakan Mattsson's message of "Thu, 30 Sep 1999 00:26:52 +0200 (MET DST)" References: Message-ID: > By using the feature of fragmented tables in Mnesia, > a table may be split into lots of fragments where > each fragment is implemented as a normal Mnesia > table. I guess this is something new (i.e not in the Open Source Erlang) ? Can you give an example of how to do this ? /Tobbe From hakan@REDACTED Thu Sep 30 01:52:09 1999 From: hakan@REDACTED (Hakan Mattsson) Date: Thu, 30 Sep 1999 01:52:09 +0200 (MET DST) Subject: safedets In-Reply-To: Message-ID: On 30 Sep 1999, Torbjorn Tornkvist wrote: tobbe>> By using the feature of fragmented tables in Mnesia, tobbe>> a table may be split into lots of fragments where tobbe>> each fragment is implemented as a normal Mnesia tobbe>> table. tobbe> tobbe>I guess this is something new (i.e not in the Open Source Erlang) ? Yes, it is "new". tobbe>Can you give an example of how to do this ? Ok, here follows an extract from the User's Guide, about fragmented tables in Mnesia: /H?kan The Concept ----------- A concept of table fragmentation has been introduced in order to cope with very large tables. The idea is to split a table into several more manageable fragments. Each fragment is implemented as a first class Mnesia table and may be replicated, have indecies etc. as any other table. But the tables may neither have local_content nor have the snmp connection activated. In order to be able to access a record in a fragmented table, Mnesia must determine to which fragment the actual record belongs. This is done by the mnesia_frag module, which implements the mnesia_access callback behaviour. Please, read the documentation about mnesia:activity/4 to see how mnesia_frag can be used as a mnesia_access callback module. At each record access mnesia_frag first computes a hash value from the record key. Secondly the name of the table fragment is determined from the hash value. And finally the actual table access is performed by the same functions as for non-fragmented tables. When the key is not known beforehand, all fragments are searched for matching records. The following piece of code illustrates how an existing Mnesia table is converted to be a fragmented table and how more fragments are added later on. Eshell V4.7.3.3 (abort with ^G) (a@REDACTED)1> mnesia:start(). ok (a@REDACTED)2> mnesia:system_info(running_db_nodes). [b@REDACTED,c@REDACTED,a@REDACTED] (a@REDACTED)3> Tab = dictionary. dictionary (a@REDACTED)4> mnesia:create_table(Tab, [{ram_copies, [a@REDACTED, b@REDACTED]}]). {atomic,ok} (a@REDACTED)5> Write = fun(Keys) -> [mnesia:write({Tab,K,-K}) || K <- Keys], ok end. #Fun (a@REDACTED)6> mnesia:activity(sync_dirty, Write, [lists:seq(1, 256)], mnesia_frag). ok (a@REDACTED)7> mnesia:change_table_frag(Tab, {activate, []}). {atomic,ok} (a@REDACTED)8> mnesia:table_info(Tab, frag_properties). [{base_table,dictionary}, {foreign_key,undefined}, {n_doubles,0}, {n_fragments,1}, {next_n_to_split,1}, {node_pool,[a@REDACTED,b@REDACTED,c@REDACTED]}] (a@REDACTED)9> Info = fun(Item) -> mnesia:table_info(Tab, Item) end. #Fun (a@REDACTED)10> Dist = mnesia:activity(sync_dirty, Info, [frag_dist], mnesia_frag). [{c@REDACTED,0},{a@REDACTED,1},{b@REDACTED,1}] (a@REDACTED)11> mnesia:change_table_frag(Tab, {add_frag, Dist}). {atomic,ok} (a@REDACTED)12> Dist2 = mnesia:activity(sync_dirty, Info, [frag_dist], mnesia_frag). [{b@REDACTED,1},{c@REDACTED,1},{a@REDACTED,2}] (a@REDACTED)13> mnesia:change_table_frag(Tab, {add_frag, Dist2}). {atomic,ok} (a@REDACTED)14> Dist3 = mnesia:activity(sync_dirty, Info, [frag_dist], mnesia_frag). [{a@REDACTED,2},{b@REDACTED,2},{c@REDACTED,2}] (a@REDACTED)15> mnesia:change_table_frag(Tab, {add_frag, Dist3}). {atomic,ok} (a@REDACTED)16> Read = fun(Key) -> mnesia:read({Tab, Key}) end. #Fun (a@REDACTED)17> mnesia:activity(transaction, Read, [12], mnesia_frag). [{dictionary,12,-12}] (a@REDACTED)18> mnesia:activity(sync_dirty, Info, [frag_size], mnesia_frag). [{dictionary,64}, {dictionary_frag2,64}, {dictionary_frag3,64}, {dictionary_frag4,64}] (a@REDACTED)19> Fragmentation Properties ------------------------ There is a table property called frag_properties and may be read with mnesia:table_info(Tab, frag_properties). The fragmentation properties is a list of tagged tuples with the arity 2. By default the list is empty, but when it is non-empty it triggers Mnesia to regard the table as fragmented. The fragmentation properties are: {n_fragments, Int} n_fragments regulates how many fragments that the table currently has. This property may explictly be set at table creation and later be changed with {add_frag, NodesOrDist} or del_frag. n_fragments defaults to 1. {node_pool, List} The node pool contains a list of nodes and may explicitly be set at table creation and later be changed with {add_node, Node} or {del_node, Node}. At table creation Mnesia tries to distribute the replicas of each fragment evenly over all the nodes in the node pool. Hopefully all nodes will end up with the same number of replicas. node_pool defaults to the return value from mnesia:system_info(db_nodes). {n_ram_copies, Int} Regulates how many ram_copies replicas that each fragment should have. This property may explicitly be set at table creation. The default is 0, but if n_disc_copies and n_disc_only_copies also are 0, n_ram_copies will default be set to 1. {n_disc_copies, Int} Regulates how many disc_copies replicas that each fragment should have. This property may explicitly be set at table creation. The default is 0. {n_disc_only_copies, Int} Regulates how many disc_only_copies replicas that each fragment should have. This property may explicitly be set at table creation. The default is 0. {foreign_key, ForeignKey} ForeignKey may either be the atom undefined or the tuple {ForeignTab, Attr}, where Attr denotes an attribute which should be interpreted as a key in another fragmented table named ForeignTab. Mnesia will ensure that the number of fragments in this table and in the foreign table are always the same. When fragments are added or deleted Mnesia will automatically propagate the operation to all fragmented tables that has a foreign key referring to this table. Instead of using the record key to determine which fragment to access, the value of the Attr field is used. This feature makes it possible to automatically co-locate records in different tables to the same node. foreign_key defaults to undefined. However if the foreign key is set to something else it will cause the default values of the other fragmentation properties to be the same values as the actual fragmentation properties of the foreign table. Eshell V4.7.3.3 (abort with ^G) (a@REDACTED)1> mnesia:start(). ok (a@REDACTED)2> PrimProps = [{n_fragments, 7}, {node_pool, [node()]}]. [{n_fragments,7},{node_pool,[a@REDACTED]}] (a@REDACTED)3> mnesia:create_table(prim_dict, [{frag_properties, PrimProps}, {attributes,[prim_key,prim_val]}]). {atomic,ok} (a@REDACTED)4> SecProps = [{foreign_key, {prim_dict, sec_val}}]. [{foreign_key,{prim_dict,sec_val}}] (a@REDACTED)5> mnesia:create_table(sec_dict, [{frag_properties, SecProps}, (a@REDACTED)5> {attributes, [sec_key, sec_val]}]). {atomic,ok} (a@REDACTED)6> Write = fun(Rec) -> mnesia:write(Rec) end. #Fun (a@REDACTED)7> PrimKey = 11. 11 (a@REDACTED)8> SecKey = 42. 42 (a@REDACTED)9> mnesia:activity(sync_dirty, Write, [{prim_dict, PrimKey, -11}], mnesia_frag). ok (a@REDACTED)10> mnesia:activity(sync_dirty, Write, [{sec_dict, SecKey, PrimKey}], mnesia_frag). ok (a@REDACTED)11> mnesia:change_table_frag(prim_dict, {add_frag, [node()]}). {atomic,ok} (a@REDACTED)12> SecRead = fun(PrimKey, SecKey) -> mnesia:read({sec_dict, PrimKey}, SecKey, read) end. #Fun (a@REDACTED)13> mnesia:activity(transaction, SecRead, [PrimKey, SecKey], mnesia_frag). [{sec_dict,42,11}] (a@REDACTED)14> Info = fun(Tab, Item) -> mnesia:table_info(Tab, Item) end. #Fun (a@REDACTED)15> mnesia:activity(sync_dirty, Info, [prim_dict, frag_size], mnesia_frag). [{prim_dict,0}, {prim_dict_frag2,0}, {prim_dict_frag3,0}, {prim_dict_frag4,1}, {prim_dict_frag5,0}, {prim_dict_frag6,0}, {prim_dict_frag7,0}, {prim_dict_frag8,0}] (a@REDACTED)16> mnesia:activity(sync_dirty, Info, [sec_dict, frag_size], mnesia_frag). [{sec_dict,0}, {sec_dict_frag2,0}, {sec_dict_frag3,0}, {sec_dict_frag4,1}, {sec_dict_frag5,0}, {sec_dict_frag6,0}, {sec_dict_frag7,0}, {sec_dict_frag8,0}] (a@REDACTED)17> Management of Fragmented Tables ------------------------------- The function mnesia:change_table_frag(Tab, Change) is intended to be used for reconfiguration of fragmented tables. The Change argument should have one of the following values: {activate, FragProps} Activates the fragmentation properties of an existing table. FragProps should either contain {node_pool, Nodes} or be empty. deactivate Deactivates the fragmentation properties of a table. The number of fragments must be 1. No other tables may refer to this table in its foreign key. {add_frag, NodesOrDist} Adds one new fragment to a fragmented table. All records in one of the old fragments will be rehashed and about half of them will be moved to the new (last) fragment. All other fragmented tables, which refers to this table in their foreign key, will automatically get a new fragment, and their records will also be dynamically rehashed in the same manner as for the main table. The NodesOrDist argument may either be a list of nodes or the result from mnesia:table_info(Tab, frag_dist). The NodesOrDist argument is assumed to be a sorted list with the best nodes to host new replicas first in the list. The new fragment will get the same number of replicas as the first fragment (see n_ram_copies, n_disc_copies and n_disc_only_copies). The NodesOrDist list must at least contain one element for each replica that needs to be allocated. del_frag Deletes one fragment from a fragmented table. All records in the last fragment will be moved to one of the other fragments. All other fragmented tables which refers to this table in their foreign key, will automatically loose their last fragment and their records will also be dynamically rehashed in the same manner as for the main table. {add_node, Node} Adds a new node to the node_pool. The new node pool will affect the list returned from mnesia:table_info(Tab, frag_dist). {del_node, Node} Deletes a new node from the node_pool. The new node pool will affect the list returned from mnesia:table_info(Tab, frag_dist). Extensions of Existing Functions -------------------------------- The function mnesia:create_table/2 is used to create a brand new fragmented table, by setting the table property frag_properties to some proper values. The function mnesia:delete_table/2 is used to delete a fragmented table including all its fragments. There must however not exist any other fragmented tables which refers to this table in their foreign key. The function mnesia:table_table/2 now understands the frag_properties item. If the function mnesia:table_info/2 is invoked in the activity context of the mnesia_frag module, information of several new items may be obtained: base_table the name of the fragmented table n_fragments the actual number of fragments node_pool the pool of nodes n_ram_copies n_disc_copies n_disc_only_copies the number of replicas with storage type ram_copies, disc_copies and disc_only_copies respectively. The actual values are dynamically derived from the first fragment. The first fragment serves as a protype and when the actual values needs to be computed (e.g. when adding new fragments) they are simply determined by counting the number of each replicas for each storage type. This means, when the functions mnesia:add_table_copy/3, mnesia:del_table_copy/2 and mnesia:change_table_copy_type/2 are applied on the first fragment, it will affect the settings on n_ram_copies, n_disc_copies, and n_disc_only_copies. foreign_key the foreign key. foreigners all other tables that refers to this table in their foreign key. frag_names the names of all fragments. frag_dist a sorted list of {Node, Count} tuples which is sorted in increasing Count order. The Count is the total number of replicas that this fragmented table hosts on each Node. The list always contains at least all nodes in the node_pool. The nodes which not belongs to the node_pool will be put last in the list even if their Count is lower. frag_size a list of {Name, Size} tuples where Name is a fragment Name and Size is how many records it contains. frag_memory a list of {Name, Memory} tuples where Name is a fragment Name and Memory is how much memory it occupies. size total size of all fragments memory the total memory of all fragments Load Balancing -------------- There are several algorithms for distributing records in a fragmented table evenly over a pool of nodes. No one is best, it simply depends of the application needs. Here follows some examples of situations which may need some attention: permanent change of nodes when a new permanent db_node is introduced or dropped, it may be time to change the pool of nodes and re-distribute the replicas evenly over the new pool of nodes. It may also be time to add or delete a fragment before the replicas are re-distributed. size/memory threshold when the total size or total memory of a fragmented table (or a single fragment) exceeds some application specific threshold, it may be time to dynamically add a new fragment in order obtain a better distribution of records. temporary node down when a node temporarily goes down it may be time to compensate some fragments with new replicas in order to keep the desired level of redundancy. When the node comes up again it may be time to remove the superfluous replica. overload threshold when the load on some node is exceeds some application specific threshold, it may be time to either add or move some fragment replicas to nodes with lesser load. Extra care should be taken if the table has a foreign key relation to some other table. In order to avoid severe performance penalties, the same re-distribution must be performed for all of the related tables. Use mnesia:change_table_frag/2 to add new fragments and apply the usual schema manipulation functions (such as mnesia:add_table_copy/3, mnesia:del_table_copy/2 and mnesia:change_table_copy_type/2) on each fragment to perform the actual re-distribution. From dg@REDACTED Thu Sep 30 01:57:29 1999 From: dg@REDACTED (David Gould) Date: Wed, 29 Sep 1999 16:57:29 -0700 Subject: safedets In-Reply-To: <14322.31801.899127.579042@duva.bluetail.com>; from Klacke on Wed, Sep 29, 1999 at 10:53:13PM +0200 References: <14322.31801.899127.579042@duva.bluetail.com> Message-ID: <19990929165729.F27137@hawk.oak.informix.com> On Wed, Sep 29, 1999 at 10:53:13PM +0200, Klacke wrote: > > > I've been hacking madly on the dets server lately. > Basically I've been having two problems with it > that are dependant on each other. > > 1. It takes a very long time to create a dets file with say > 1. million entries in it. > 2. If such a file is not properly closed, it takes ages > to repair it. > > My first attempt was to hack the dets server and pull some > of the hash-ndex structures that are now on disc up into RAM. > This helped speed with about 30% but increased memory > consumption substantially. Not ok. > > My second attempt was to scrap dets alltogether and use > gdbm instead. So I wrote a linked in driver to gdbm. > It turns out that gdbm is about equally fast as dets on > lookups, and about 4 times faster on the first 10.000 items > on insertions. > > However on the last 10.000 objects (Inserting 1 million itemes) > gdbm is >100 times slower. It starts to degenerate > around 100.000 items. So, this is good news, dets was > better than I thought. Much better. Cool. One other thing to look at is Berkeley DB instead of gdbm. See www.sleepycat.com for more info and downloads. -dg -- David Gould dg@REDACTED 510.628.3783 or 510.305.9468 Informix Software 300 Lakeside Drive Oakland, CA 94612 You will cooperate with Microsoft, for the good of Microsoft and for your own survival. -- Navindra Umanee From luke@REDACTED Thu Sep 30 08:39:08 1999 From: luke@REDACTED (Luke Gorrie) Date: 30 Sep 1999 16:39:08 +1000 Subject: Code changing Message-ID: <87n1u4vr77.fsf@baked.vegetable.org> G'day all, A couple of questions about code changes in an erlang system. First, we're having a problem changing code in etk-using modules. The module itself is a gen_event and copes with code changes okay, but the command functions given to tk objects don't seem to work anymore. After a code change, I get: tkfun error: 1 : {function_clause,{messaging_gui, module_lambda_1, [10574021,[],{}]}} I've tried to fix this with: code_change(OldVsn, State, Extra) -> etk:configure(State#state.button, [{command, fun view_message/0}]), {ok, State}. But that didn't seem to do the trick - but then I don't really know Tk, maybe I'm appending another command rather than replacing it. How should I make etk callbacks update-safe, or am I wrong about what the problem is? The other issue is with gen_event and code_change. If I don't make a code_change/3 function, my gen_event processes don't survive code changes. I put this down to some code in gen_event: {ok, NewState} = Module:code_change(OldVsn, H#handler.state, Extra), Whereas gen_server and gen_fsm use a catch on this call, and (gen_server at least) work with modules that don't declare code_change. This seems to mean that it's not okay to omit code_change/3 from gen_event modules, and so I've attached a patch to erlang.el's gen_event skeleton to add code_change incase it's useful to anyone else. Is it actually ok to omit code_change from gen_servers? I had a little trace through gen_server and sys but I'm still not clear on what ultimately happens with the caught error from gen_server's call to a non-existent code_change/3. Cheers, Luke -------------- next part -------------- A non-text attachment was scrubbed... Name: elisp-gen_event.patch Type: application/x-patch Size: 880 bytes Desc: Patch adding code_change/3 to gen_event skeleton URL: -------------- next part -------------- From luke@REDACTED Thu Sep 30 11:23:07 1999 From: luke@REDACTED (Luke Gorrie) Date: 30 Sep 1999 19:23:07 +1000 Subject: make tests In-Reply-To: Hakan Mattsson's message of "Wed, 29 Sep 1999 23:16:32 +0200 (MET DST)" References: Message-ID: <87yadou51g.fsf@baked.vegetable.org> Hakan Mattsson writes: > On 29 Sep 1999, Luke Gorrie wrote: > > luke>Actually, I was hacking Mnesia's transaction manager today and was > > Great! What are you doing? I'm just toying around really. I read a transaction processing book a while ago and I've been looking for an excuse to play around with that stuff since. I've been looking at trying out a more optimistic locking strategy with deadlock detection rather than avoidance, though now that I can see how the present pessimistic locking works I think its pretty cool. I'm not sure how practical each approach is, so I'm only really poking around in the name of fun. :) > luke> about to ask on Wikie if we could have a play with the test > luke> suite. :-) I think it would be great if it was released as open > luke> source, but understand if it's a bit tangled up in larger things. > > If your hacking is more serious than playing ;-), you will > definitely need the test suite. Even for playing :-), I already broke the deadlock avoidance making a naive change to abort fewer transactions yesterday. I'm too scared to use a customised Mnesia with my programs without running it through the test suite. So, I think it would be nifty if the test suite was open sourced too, but I'm only playing around so it's not too important for me. Now that I applied the fix to shorten the restart delay, even the artificial test case I made runs pretty quick so I'm happy. :-) Cheers, Luke From peter.olin@REDACTED Thu Sep 30 15:38:36 1999 From: peter.olin@REDACTED (Peter Olin) Date: Thu, 30 Sep 1999 15:38:36 +0200 Subject: short host names References: <005201bf0ac5$9e557920$0abdc5c0@desk.motivity.ca> Message-ID: <37F367DC.491C4453@ericsson.com> Could it possibly just be so simple that you don't have the same cookie on the different nodes? (This is the default unless you somehow have specified a cookie). You are not specifying a cookie on the command line, so unless you have an .erlang.cookie file in your working/home directory, the nodes will have different cookies. erl -sname foo -setcookie ballerina erl -sname bar -setcookie ballerina (ballerina can of course be replaced with your favorite cookie, like singoalla, brago_crackers or possibly marie_crackers in case of an emergency. Try to stay away from soft cakes though, an italian chocolate cake is a bit on the fatty side :-) /Peter Vance Shipley wrote: > > All, > > I don't seem to be able to communicate between two > nodes on a machine when using short node names. It > works fine when I use long node names > (e.g. erl -name foo). > > I start up a node on one terminal: > > $ erl -sname foo > Erlang (BEAM) emulator version 47.4.1 > > Eshell V47.4.1 (abort with ^G) > (foo@REDACTED)1> > > Then start another on another terminal: > > $ erl -sname bar > Erlang (BEAM) emulator version 47.4.1 > > Eshell V47.4.1 (abort with ^G) > (bar@REDACTED)1> > > Now I try to ping: > > foo@REDACTED)1> net_adm:ping(bar@REDACTED). > pang > > Names doesn't work either: > > (bar@REDACTED)2> net_adm:names(). > {ok,[]} > > I ran epmd with debugging turned on an it looks like epmd > is getting the requests and responding properly. > > This is all under my port to Unixware 2.1.3 so it could > well be my port is incomplete but I don't know what to look > for at this point. > > -Vance -- /Peter Olin (mailto:peter.olin@REDACTED)