From brian@REDACTED Tue Mar 1 00:15:29 2005 From: brian@REDACTED (Brian Buchanan) Date: Mon, 28 Feb 2005 15:15:29 -0800 (PST) Subject: Calling internal functions - foo::bar() ? In-Reply-To: References: Message-ID: <20050228150820.A34850@mail.ncircle.com> > Today many people use export_all so that they have a convenient way > to test internal functions in the shell. > > This has the unfortunate side-effect of letting real modules call > not-explicitly-exported functions, and that in turn means that > export-lists can go stale because you don't notice when they're > wrong. > > To remove 'export_all' is dangerous and it requires you to retest > your program. (What if you were "accidentally" calling unexported > functions while testing before?). This sucks because typically the > time you want to remove export_all is when you've just finished > testing and you notice it in 'cvs diff' before checkin -- i.e. > exactly when you don't want to make any changes. > > If you do make the effort to correct your export lists and remove > your export_all's at some point then later you'll be tempted to put > them back in while hacking and debugging new versions of your > libraries. The way that I deal with this is to put the test cases for a module at the bottom of the source file for that module, wrapped in -ifdef(tests). My test harness, called from "make test", compiles each module using cover:compile(module, [{d, tests}]), executes the test cases, prints the results of the test, and warns of any uncovered functions. This doesn't address the desire to test internal functions from the shell, of course, but it solves my testability problem. Testing from the shell could be addressed by manually compiling the module and specifying export_all as an option to the compiler -- NOT putting it in the source file. This precludes accidentally checking in source code that includes "-compile(export_all)", because it's never there in the first place. You could also easily add a make target or variable that causes all your modules to be built +debug_info +export_all for writing/debugging/testing, and builds them without those options for production builds. - Brian -- Brian Buchanan Principal Engineer nCircle Network Security http://www.ncircle.com/ From ok@REDACTED Tue Mar 1 00:25:29 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 1 Mar 2005 12:25:29 +1300 (NZDT) Subject: Calling internal functions - foo::bar() ? Message-ID: <200502282325.j1SNPTJ6480152@atlas.otago.ac.nz> Rather than have a way to call non-exported functions, wouldn't it be cleaner to do what Hugs does, and let you choose in the shell which module to be in? (The :module command, if you know Hugs.) That is, each expression you type in the shell is evaluated inside the scope of some module, and you can change which module that is. This still lets you call any function in any module, but it _doesn't_ give you any _syntax_ for that. Perhaps mod() Report the current module name mod(Module_Name) Make Module_Name the current module name. From joe.armstrong@REDACTED Tue Mar 1 09:29:07 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Tue, 1 Mar 2005 09:29:07 +0100 Subject: Streaming Input Message-ID: This is slightly more difficult (not a lot) you're now talking about re-entrant parsers. The general form of these is as follows: Assume you have some parsing function F which parses a binary B. Calling F(B) returns done, P, F'} where P is a parse and F' a new parsing function or {more, F'} where F' is a new parsing function. You could call it like this: Socket = ... F = init(), loop(Socket, F). where loop(Socket, F) -> receive {tcp,Socket,Bin} -> case F(Bin) of {done,Parse, F1} -> ... do something with Parse ... loop(Socket, F1); {more, F1} -> loop(Socket, F1) end; ... Now you have to define F Let's give an example. Suppose the input has "begin" ... "end" symbols or #N (one byte followed by N bytes) init() -> fun(B) -> top(binary_to_list(B)) end. top("begin" + T) -> collect_body(T, []); top([$#,N|T]) -> collect_bytes(T, N, []); top(X) -> fun(B) -> top(X ++ binary_to_list(B)) end. collect_body("end" ++ T, L) -> {done, reverse(L), list_to_binary(T)}; collect_body([H|T], L) -> collect_body(T, [H|L]); collect_body([], L) -> fun(B) -> collect_body(binary_to_list(B), L) end. etc. This is pretty simple code, the point to note is the last clause of every group collect_body([], L) is called when we "run out of stuff to parse" - what do we want to do then? Answer: wait for "More" data and then call collect_body(More, L) - that's just what the last clause says: collect_body([], L) -> fun(B) -> collect_body(binary_to_list(B), L) end. This kind of code is very easy if you just "follow the pattern" and don't think (TM) BTW you have to get the code right first time - debugging this is not easy if you make a silly mistake :-) Cheers /Joe > -----Original Message----- > From: orbitz [mailto:orbitz@REDACTED] > Sent: den 28 februari 2005 22:55 > To: Joe Armstrong (AL/EAB) > Cc: erlang-questions@REDACTED > Subject: Re: Streaming Input > > > I'm not sure that'll work in my situation necessarily. In this > protocol, only some objects have a size specification, and > others don't. > And the ones that don't can be variable size. It uses > prefix/suffix to > say when decoding should start and end. Also I don't know how much I > need until I've identified what type it is and started extracting it, > and since sometimes are variable in size and the protocol > uses a suffix > to tell me when to stop decoding that type I can't figure out > how much I > need. Perhaps my original idea of figuring out what type it is then > sending to a special extract function for that type is no good? It > seems simpler that way since I don't need to keep track of state, but > more prone to issues since I need to go back to this waiting function > every time I run out of data but haven't finished decoding my object. > > Thanks > > Joe Armstrong (AL/EAB) wrote: > > > use binaries - that's what they are for > > > > First write something like this: > > > > extract(BinIn, Need, BinAcc) -> > > Got = size(BinIn), > > if > > Got > Need -> > > {Before, After} = split_binary(BinIn, Need), > > Result = concat_binary([BinAcc, Before]), > > {done, Result, After}; > > Got == Need -> > > Result = concat_binary([BinAcc,BinIn]), > > {done, Result, <<>>}; > > Got < Need -> > > BinAcc1 = concat_binary([BinAcc, BinIn]), > > {more, Need - Got, BinAcc1} > > end. > > > > > > > > here > > > > in extract(BinIn, Need, BinAcc) > > More and Sofar are binaries > > Need is the required block length > > > > if size(BinIn) > Need we split the block into two chunks > > and return {done, Bin, After} Bin = is the data you need > > otherwise {more, Need-Got, BinAcc} > > > > BinAcc is a binary accumulator containing all the data > received so far. > > > > Then just arrange so code to call this > > > > Cheers > > > > /Joe > > > > > > > > > > > > > > > >>-----Original Message----- > >>From: owner-erlang-questions@REDACTED > >>[mailto:owner-erlang-questions@REDACTED]On Behalf Of orbitz > >>Sent: den 27 februari 2005 07:49 > >>To: erlang-questions@REDACTED > >>Subject: Streaming Input > >> > >> > >>I am working with a protocol where the size of the > following block is > >>told to me so I can just convert the next N bytes to, say, > a string. > >>The problem is though, I'm trying to write this so it handles > >>a stream > >>properly, so in the binary I have could be all N bytes that I > >>need, or > >>something less than N. So at first I tried: > >> > >>extract_string(Tail, 0, Res) -> > >> {ok, {string, Res}, Tail}; > >>extract_string(<>, Length, Res) -> > >> extract_string(Tail, Length - 1, lists:append(Res, [H])); > >>extract_string(<<>>, Length, Res) -> > >> case dispatch_message() of > >> {decode, _, Data} -> > >> extract_string(Data, Length, Res) > >> end. > >> > >>When the binary is empty but I still need more data it waits > >>for more. > >>I don't know if this is the proper idiom (it seems gross to > >>me but I am > >>unsure of how to do it otherwise). This is incredibly slow > though. > >>With a long string that I need to extract it takes a lot of > >>CPU and far > >>too long. So I decided to do: > >> > >>extract_string(Data, Length, _) -> > >> <> = Data, > >> {ok, {string, binary_to_list(String)}, Tail}. > >> > >>In terms of CPU and time this is much much better, but if I > >>don't have > >>all N bytes it won't work. Any suggestions? > >> > >> > >> > > > > > > > > > > > From ulf.wiger@REDACTED Tue Mar 1 10:13:14 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Tue, 1 Mar 2005 10:13:14 +0100 Subject: the point of 'public' digraphs? Message-ID: I noticed that it's possible to specify the attribute 'public' in digraph:new/1. Shouldn't it be very clearly stated in the manual then that digraph is by no means thread safe? One example is the new_edge_id/1 function, below: %% %% Generate a "unique" edge identifier (relative this graph) %% ['$e' | N] %% new_edge_id(G) -> NT = G#graph.ntab, [{'$eid', K}] = ets:lookup(NT, '$eid'), true = ets:delete(NT, '$eid'), true = ets:insert(NT, {'$eid', K+1}), ['$e' | K]. The ntab table is a bag table, so update_counter can't be used (BTW, the man page for ets:update_counter() doesn't state the perhaps obvious fact that it doesn't support bag tables.) Here are two possible interleavings that may cause the function to fail: Example 1: ---------- Process A Process B NT = G#graph.ntab, [{'$eid', K}] = ets:lookup(NT, '$eid'), true = ets:delete(NT, '$eid'), % scheduled out NT = G#graph.ntab, [{'$eid', K}] = ets:lookup(NT, '$eid'), % crashes since all instances of '$eid' gone Example 2: ---------- Process A Process B NT = G#graph.ntab, [{'$eid', K}] = ets:lookup(NT, '$eid'), % scheduled out NT = G#graph.ntab, [{'$eid', K}] = ets:lookup(NT, '$eid'), true = ets:delete(NT, '$eid'), true = ets:insert(NT, {'$eid', K+1}), ['$e' | K]. % ... scheduled out true = ets:delete(NT, '$eid'), true = ets:insert(NT, {'$eid', K+1}), ['$e' | K]. Process A and Process B now get the same key, and '$eid' is incremented by 1 instead of 2, which may be a smaller problem. These two problems can occur even with today's scheduler, but if we move to multi-pro architectures, the probability of such race conditions occurring increases tremendously. Since digraph contains several functions that are implemented using multiple ets accesses, the most reasonable fix seems to be to remove the 'public' option. As a more backwards compatible, but IMO worse alternative, clearly document that some kind of locking mechanism is required to protect accesses to a public digraph. /Uffe From samuel@REDACTED Tue Mar 1 08:35:45 2005 From: samuel@REDACTED (Samuel Rivas) Date: Tue, 1 Mar 2005 08:35:45 +0100 Subject: tv not showing ets table In-Reply-To: References: Message-ID: <20050301073545.GA11716@crusher.lfcia.pri> Anders Nygren wrote: > Hi > I create two ets tables with > ets:new(workers,[named_table,set,protected]), > ets:new(queue,[named_table,ordered_set,protected]), > > tv only shows worker as default. > If I change options to show system tables the table queue is shown, > with my process as the owner. > > So why does tv think that the table queue is a system table? "System tables" are harcoded in tv_main.hrl, yet I don't know what that means ... -- Samuel From casper2000a@REDACTED Tue Mar 1 10:42:26 2005 From: casper2000a@REDACTED (Casper) Date: Tue, 1 Mar 2005 15:42:26 +0600 Subject: Erlang port driver question In-Reply-To: Message-ID: Hi All, I am developing an Erlang Port driver. That will receive/send data from another device. There's one C function, which will start receiving some data. For that, I need to pass a buffer pointer and it's length, like below. dataReceive(void *buffer, unsigned bufsize) This function is asynchronous. There's a data receiving queue, which will receive events from that device. Once the data is received, it will put an Acknowledgement event to that queue. I have to use the initial buffer given to dataReceive function to receive that data in my program. My question is, how can I keep a track of the *buffer given to dataReceive. There can be many Erlang Processes start dataReceive on many devices, through one Port Driver. How can I keep track of the *buffer variable? I prefer not to keep a Map of those buffers in the C side. Can a dummy buffer create on Erlang side, call the dataReceive, don't free that buffer in C Port side, receive data to that and finally arrive on the Erlang side? Please advice me! - Eranga From thomasl_erlang@REDACTED Tue Mar 1 15:13:21 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 1 Mar 2005 06:13:21 -0800 (PST) Subject: Calling internal functions - foo::bar() ? In-Reply-To: <200502282325.j1SNPTJ6480152@atlas.otago.ac.nz> Message-ID: <20050301141321.13139.qmail@web41905.mail.yahoo.com> --- "Richard A. O'Keefe" wrote: > Rather than have a way to call non-exported > functions, wouldn't > it be cleaner to do what Hugs does, and let you > choose in the > shell which module to be in? (The :module command, > if you know > Hugs.) That is, each expression you type in the > shell is > evaluated inside the scope of some module, and you > can change > which module that is. Sounds good to me. An interim solution might be to do the following: - if local call f(...) is a shell default, use it - otherwise, prepend the "current module" m, and call m:f(...) That way, we can leave calling of unexported functions for later. Having :: could be used in regular code as well; it might be preferrable to export_all, not to mention the "internal exports" one is forced to do. For one thing, it's easier to check. I recommend including :: as an experimental extension before casting it in stone. Finally, I currently prefer to use the following idiom: % erlc +export_all mod.erl rather than inserting it in mod.erl Best, Thomas __________________________________ Do you Yahoo!? Yahoo! Mail - You care about security. So do we. http://promotions.yahoo.com/new_mail From sean.hinde@REDACTED Tue Mar 1 15:35:41 2005 From: sean.hinde@REDACTED (Sean Hinde) Date: Tue, 1 Mar 2005 14:35:41 +0000 Subject: Calling internal functions - foo::bar() ? In-Reply-To: <20050301141321.13139.qmail@web41905.mail.yahoo.com> References: <20050301141321.13139.qmail@web41905.mail.yahoo.com> Message-ID: <2e37d84a75a7484dffa6e008a54775a4@mac.com> > > Having :: could be used in regular code as well; it > might be preferrable to export_all, not to mention the > "internal exports" one is forced to do. For one thing, > it's easier to check. That way we can just tell everyone that Erlang is simply a version of C++. Success and riches will surely follow :) Sean From vlad_dumitrescu@REDACTED Tue Mar 1 21:14:36 2005 From: vlad_dumitrescu@REDACTED (vladdu) Date: Tue, 1 Mar 2005 21:14:36 +0100 Subject: Emacs mode References: Message-ID: <20050301201436.612EA46962@bang.trapexit.org> Hi, With every new release, my .emacs needs to be adjusted in order to find the new erlang-start and friends. I am using a link to the erlang directory, but the prefix for the tools application changes, so it still needs an edit. The only way I can think of is to create a link outside the erlang tree, but it doesn't feel right. Doeas anyone have another solution? I suppose everybody has to deal with this... best regards, Vlad _________________________________________________________ Sent using Mail2Forum (http://m2f.sourceforge.net) From csanto@REDACTED Tue Mar 1 21:53:32 2005 From: csanto@REDACTED (Corrado Santoro) Date: Tue, 01 Mar 2005 21:53:32 +0100 Subject: Emacs mode In-Reply-To: <20050301201436.612EA46962@bang.trapexit.org> References: <20050301201436.612EA46962@bang.trapexit.org> Message-ID: <4224D64C.9060104@diit.unict.it> I'm using this in .emacs: (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.4/emacs" load-path)) (setq erlang-root-dir "/usr/local/lib/erlang") (setq erlang-electric-commands '()) (setq erlang-indent-level 2) (require 'erlang-start) Ciao, --C. vladdu wrote: > Hi, > > With every new release, my .emacs needs to be adjusted in order to find the new erlang-start and friends. I am using a link to the erlang directory, but the prefix for the tools application changes, so it still needs an edit. > > The only way I can think of is to create a link outside the erlang tree, but it doesn't feel right. > > Doeas anyone have another solution? I suppose everybody has to deal with this... > > best regards, > Vlad > _________________________________________________________ > Sent using Mail2Forum (http://m2f.sourceforge.net) -- ====================================================== Eng. Corrado Santoro, Ph.D. University of Catania - Engineering Faculty Department of Computer Science and Telecommunications Engineering Viale A. Doria, 6 - 95125 CATANIA (ITALY) Tel: +39 095 7382380 Fax: +39 095 7382397 +39 095 7382365 +39 095 7382364 EMail: csanto@REDACTED Personal Home Page: http://www.diit.unict.it/users/csanto NUXI Home Page: http://nuxi.iit.unict.it ====================================================== From vlad_dumitrescu@REDACTED Tue Mar 1 22:28:07 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Tue, 1 Mar 2005 22:28:07 +0100 Subject: Emacs mode References: <20050301201436.612EA46962@bang.trapexit.org> <4224D64C.9060104@diit.unict.it> Message-ID: From: "Corrado Santoro" > I'm using this in .emacs: > (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.4/emacs" > load-path)) Hi, and thank you for the answer. What I meant was that for example after installing R10B-3, the above won't work, because the path should be .../tools-2.4.1/... regards, Vlad From vlad_dumitrescu@REDACTED Tue Mar 1 22:28:07 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Tue, 1 Mar 2005 22:28:07 +0100 Subject: Emacs mode References: <20050301201436.612EA46962@bang.trapexit.org> <4224D64C.9060104@diit.unict.it> Message-ID: From: "Corrado Santoro" > I'm using this in .emacs: > (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.4/emacs" > load-path)) Hi, and thank you for the answer. What I meant was that for example after installing R10B-3, the above won't work, because the path should be .../tools-2.4.1/... regards, Vlad From erlang@REDACTED Tue Mar 1 05:28:19 2005 From: erlang@REDACTED (Michael McDaniel) Date: Mon, 28 Feb 2005 20:28:19 -0800 Subject: _Erlang_Programmation_ Message-ID: <20050301042819.GQ3858@fangora.autosys.us> Can anyone tell me if Mickael Remond's book, _Erlang_Programmation_, is going to be available in English this year? thanks, Michael McDaniel From yerl@REDACTED Wed Mar 2 00:33:07 2005 From: yerl@REDACTED (yerl) Date: Wed, 02 Mar 2005 00:33:07 +0100 Subject: Emacs mode In-Reply-To: References: <20050301201436.612EA46962@bang.trapexit.org> <4224D64C.9060104@diit.unict.it> Message-ID: <4224FBB3.5020600@club-internet.fr> Hi Vlad! Try this tip: 1. Create a "~/.elisp" directory on your home directory. 2. Copy the files from the directory "/usr/lib/erlang/lib/tools-2.4.XXX/emacs/*" to "~/.elisp". 3. Put the lines below on your "~/.emacs.el" : This works with any version of OTP because it is undependent from the OTP install directory. ------------------------------------------------------------------CUT AND PAST ---------------------------------------------------------- ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; ;; Erlang MODE ;; ;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; (setq load-path (cons "~/.elisp" load-path)) (setq erlang-root-dir "/usr/lib/erlang") (setq exec-path (cons "/usr/lib/erlang/lib/bin" exec-path)) (require 'erlang-start) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; ;; Erlang man ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; (defun erlang-man () "Ouvre le manuel Erlang a la page correspondant a la fonction sous le curseur" (interactive) (erlang-man-function (current-word)) ) (global-set-key "\C-c\h" 'erlang-man) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; UNCOMMENT TO ENABLE DISTEL MODE ;; ERLANG DISTEL MODE FOR EMACS ;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;(let ((distel-dir "~/.elisp/distel")) ; (unless (member distel-dir load-path) ; ;; Add distel-dir to the end of load-path ; (setq load-path (append load-path (list distel-dir))))) ;(require 'distel) ;(add-hook 'erlang-mode-hook 'distel-erlang-mode-hook) ------------------------------------------------------------------CUT AND PAST ---------------------------------------------------------- cheers /yerl Vlad Dumitrescu a ?crit : > From: "Corrado Santoro" > >> I'm using this in .emacs: >> (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.4/emacs" >> load-path)) > > > Hi, and thank you for the answer. > > What I meant was that for example after installing R10B-3, the above > won't work, because the path should be .../tools-2.4.1/... > > regards, > Vlad > From anders.nygren@REDACTED Wed Mar 2 02:36:16 2005 From: anders.nygren@REDACTED (Anders Nygren) Date: Tue, 1 Mar 2005 19:36:16 -0600 Subject: http problem Message-ID: Hi I have a program that does URL="http://148.233.175.237:80/callejero/callejeroXMLfeed2.asp?name=ford&cp=01020&tu=3" Headers=[], Request={URL,Headers}, Timeout=Srv_config#http_server.timeout, Reply=http:request(get,Request,[{timeout,Timeout}],[]) , On linux with otp-R10B-0 Erlang (BEAM) emulator version 5.4 [source] [hipe] it works fine. But on Windows XP otp-R10B-3 Erlang 5.4.3 it crashes like shown below. Does anyone have any ideas what can be wrong? /Anders Nygren =ERROR REPORT==== 1-Mar-2005::18:58:32 === ** Generic server httpc_manager terminating ** Last message in was {request,{request, undefined, <0.117.0>, 0, http, {"148.233.175.237",80}, "/callejero/callejeroXMLfeed2.asp", "?name=ford&cp=01020&tu=3", get, {http_request_h, undefined, "keep-alive", undefined, undefined, undefined, undefined, undefined, undefined, undefined, undefined, undefined, undefined, undefined, undefined, undefined, undefined, "148.233.175.237", undefined, undefined, undefined, undefined, undefined, undefined, undefined, undefined, undefined, [], undefined, undefined, undefined, undefined, "0", undefined, undefined, undefined, undefined, undefined, undefined, []}, {[],[]}, {http_options,5000,true,[],false,false}}} ** When Server state == {state,[],50,{options,{undefined,[]},3,400}} ** Reason for termination == ** {{badmatch,{error,badarg}}, [{httpc_manager,handle_request,2}, {gen_server,handle_msg,6}, {proc_lib,init_p,5}]} From vlad_dumitrescu@REDACTED Wed Mar 2 08:27:09 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Wed, 2 Mar 2005 08:27:09 +0100 Subject: Emacs mode References: <20050301201436.612EA46962@bang.trapexit.org> <4224D64C.9060104@diit.unict.it> <4224FBB3.5020600@club-internet.fr> Message-ID: From: "yerl" > Try this tip: > 1. Create a "~/.elisp" directory on your home directory. > 2. Copy the files from the directory > "/usr/lib/erlang/lib/tools-2.4.XXX/emacs/*" to "~/.elisp". > 3. Put the lines below on your "~/.emacs.el" : Yes, thank you. I suppose having an external directory is unavoidable. > This works with any version of OTP because it is undependent from the > OTP install directory. But still needs some manual handling whenever erlang emacs files get updated... Maybe a slightly better idea is to make ~/.elisp a link to the tools/emacs directory, and update it at the same time as the link to the new OTP distribution. best regards, Vlad From mickael.remond@REDACTED Wed Mar 2 09:55:31 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Wed, 02 Mar 2005 09:55:31 +0100 Subject: _Erlang_Programmation_ In-Reply-To: <20050301042819.GQ3858@fangora.autosys.us> References: <20050301042819.GQ3858@fangora.autosys.us> Message-ID: <42257F83.3080108@erlang-fr.org> Michael McDaniel wrote: > Can anyone tell me if Mickael Remond's book, _Erlang_Programmation_, is going to be > available in English this year? I think that this is not going to happens this year has the project has not started yet and has I have to find an English publisher willing to buy the right to the French publisher Eyrolles first. I am working on it however. If you know English publisher that could be interested, please, do not hesitate to ask them to contact me. -- Micka?l R?mond From joe.armstrong@REDACTED Wed Mar 2 11:18:52 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Wed, 2 Mar 2005 11:18:52 +0100 Subject: Sound/video drivers Message-ID: Questions about sound: I'd like to do the following: - From a microphone connected to the sound card of my (Linux) PC I'd like to get I get {raw,Bin} sound messages here raw might mean that Bin represents 20 KHz, 8 bit samples or something. - I'd like to send these to a codec process and get back (say) {codec23, Bin'} messages. - I ship these over the net with UDP/RTP (or something) - then turn these back into {raw, Bin''} messages and - send these to my soundcard So: 1) How can I get sound into/out of my PC to an Erlang program? The input is a microphone connected to my sound card Output is through the sound card I'm thinking of some raw driver here: I'd like to say: P = open_sound( Options), loop(P). loop(P) -> receive {P, Sound} -> .... received Sound ... end. I guess Options must say something about sampling rates etc. 2) What is sound? ie in the above program what is the type of Sound? - Assume Sound is a tuple {Descriptor, Bin} Where Bin is just sound data, then Descriptor would have to describe Bin What would descriptor be? Something like {EncodeingMethod, Args} but what would these be. 3) How can I encode sound with codecs? I'd like to write: encode(From, To, Codec) -> receive {From, Sound} -> {Sound1, Codec1} = Codec(Sound), To ! {self(), Sound1}, encode(From, To, Codec1) end. Video? - same questions for video - with microphone replaced by web_cam Has anybody some Erlang code that does all of this? Cheers /Joe From flaig@REDACTED Wed Mar 2 11:23:11 2005 From: flaig@REDACTED (flaig@REDACTED) Date: Wed, 2 Mar 2005 11:23:11 +0100 Subject: _Erlang_Programmation_ Message-ID: <200503021023.j22ANBdf011927@ger5.wwwserver.net> Maybe Springer in Heidelberg might be interested? They are a German company but 99% of their computer stuff is in English. Here's their web page: http://www.springeronline.com/sgw/cda/frontpage/0,11855,1-146-0-0-0,00.html (Actually, the problem with them is that they have high intellectual pretensions but actually most of their folks are rather dumbass. So don't talk to them about how good Erlang really is but just keep on telling them how widely it is used. Mak??? th???m $?????? a $???riou$ advantage.) Keep on hacking, R??diger M. Flaig Am Mittwoch, 2. M??rz 2005 09:55 schrieben Sie: > Michael McDaniel wrote: > > Can anyone tell me if Mickael Remond's book, _Erlang_Programmation_, is > > going to be available in English this year? > > I think that this is not going to happens this year has the project has > not started yet and has I have to find an English publisher willing to > buy the right to the French publisher Eyrolles first. > > I am working on it however. If you know English publisher that could be > interested, please, do not hesitate to ask them to contact me. -- Diese E-Mail wurde mit http://www.mail-inspector.de verschickt Mail Inspector ist ein kostenloser Service von http://www.is-fun.net Der Absender dieser E-Mail hatte die IP: 129.206.124.135 From micke@REDACTED Wed Mar 2 11:39:50 2005 From: micke@REDACTED (Michael Fogeborg) Date: Wed, 02 Mar 2005 11:39:50 +0100 Subject: java JMeter vs Erlang Tsunami In-Reply-To: <42257F83.3080108@erlang-fr.org> References: <20050301042819.GQ3858@fangora.autosys.us> <42257F83.3080108@erlang-fr.org> Message-ID: <6.2.1.2.0.20050302113538.08140048@mail.online.no> have anyone here compared these two ? From mickael.remond@REDACTED Wed Mar 2 11:54:10 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Wed, 02 Mar 2005 11:54:10 +0100 Subject: java JMeter vs Erlang Tsunami In-Reply-To: <6.2.1.2.0.20050302113538.08140048@mail.online.no> References: <20050301042819.GQ3858@fangora.autosys.us> <42257F83.3080108@erlang-fr.org> <6.2.1.2.0.20050302113538.08140048@mail.online.no> Message-ID: <42259B52.3050001@erlang-fr.org> Michael Fogeborg wrote: > have anyone here compared these two ? Yes. JMeter is a toy. You can hardly simulate more than 200 simultaneous users hitting a web server. This is something you can use to get a rough idea of your pages response time but you can do real benchmark performance. And JMeter is not clustered. JMeter can use protocol not (yet?) available under Tsunami (Most notably JDBC which is pretty much Java only; If Tsunami implements database benchmark, it is likely to use direct database protocol or ODBC). Otherwise Tsunami is made for real performance benchmark. Tsunami has been use to benchmark a static platform (Apache). Tsunami simulated 25000 simultaneous users (on 4 standar machines), half of the connexion being HTTPS. Tsunami is very powerfull in what you can do (in scenario), use a stochastic model to have more realistic user behaviours,.... I could keep on for pages :-) Tsunami is a little rough around the edge, but if you plan to use it, you can contact me to have some starting tips :-) -- Micka?l R?mond http://www.erlang-projects.org/ From mickael.remond@REDACTED Wed Mar 2 12:02:24 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Wed, 02 Mar 2005 12:02:24 +0100 Subject: Sound/video drivers In-Reply-To: References: Message-ID: <42259D40.9040504@erlang-fr.org> Joe Armstrong (AL/EAB) wrote: > Questions about sound: > > I'd like to do the following: > > - From a microphone connected to the sound card of my (Linux) PC I'd like to get I get {raw,Bin} sound > messages here raw might mean that Bin represents 20 KHz, 8 bit samples or something. > > - I'd like to send these to a codec process and get back (say) {codec23, Bin'} messages. > > - I ship these over the net with UDP/RTP (or something) > > - then turn these back into {raw, Bin''} messages and > > - send these to my soundcard > > So: > > > 1) How can I get sound into/out of my PC to an Erlang program? If your target is only Linux, you have devices to send sound to. Try to cat a file to /dev/audio for example and hear the nice result :-) /dev/audio is for playing sound. /dev/dsp is for reading sound. However this solution is Linux only and might not be what you want. Another solution would be to use SDL, which is a multiplatform library. An Erlang SDL binding currently exist. You will need to dig into the SDL sound API. I wonder how you plan to cut the sound into small Erlang messages and to synchronize playing of those messages at the other end (One sample should not be replayed before the previous one ends). -- Micka?l R?mond http://www.erlang-projects.org/ From joe.armstrong@REDACTED Wed Mar 2 13:36:28 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Wed, 2 Mar 2005 13:36:28 +0100 Subject: Gulp Message-ID: OMG anyone? Some quotes from the amazing Meta-Object Facility Specification available at http://www.omg.org/docs/formal/02-04-03.pdf The Meta Object Facility Specification is what is used to descibe the meta-meta model of a UML model (was that right?)- this 358 pages mastodon document is full of little gems like (section 2.2.1, page 34): The metamodel layer is comprised of the descriptions (i.e., meta-metadata) that define the structure and semantics of metadata. Meta-metadata is informally aggregated as metamodels. A metamodel is an "abstract language" for describing different kinds of data; that is, a language without a concrete syntax or notation. I'm having slight problems understanding this: Let's try an put the clause "describing the meta-meta model of a UML-model" into my language understanding engine. "describing the meta-meta model of a UML-model" => "describing X" where X = "the meta-meta model of a UML-model" I think that "the meta-meta model of a UML-model" => "meta-meta-meta model" and "describing X" => a meta-model of X so this should be a meta-meta-meta-meta model But is this "a language without a concrete syntax or notation" This is the puzzling bit. /Joe From luke@REDACTED Wed Mar 2 14:00:10 2005 From: luke@REDACTED (Luke Gorrie) Date: 02 Mar 2005 14:00:10 +0100 Subject: Gulp References: Message-ID: "Joe Armstrong \(AL/EAB\)" writes: > OMG anyone? > Some quotes from the amazing Meta-Object Facility Specification available at > > http://www.omg.org/docs/formal/02-04-03.pdf I looked at the MOF in detail a long time ago when I was into CORBA. I think it's the sort of thing that a teenager reading Descartes would design. Not one of Brisbane's greatest contributions to the world. From ulf.wiger@REDACTED Wed Mar 2 14:12:08 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Wed, 2 Mar 2005 14:12:08 +0100 Subject: Gulp Message-ID: Well, Joe, it's actually quite elementary, but you may need to read the UML 2.0 Infrastructure Final Adopted Spec in order to see the light. In it, on page 38, you can read: "UML is defined as a model that is based on MOF used as a metamodel," If this is not clear enough, on page 39 we receive further enlightenment: "One of the primary uses of the UML 2.0 Infrastructure specification is that it should be reused when creating other metamodels. The UML metamodel reuses the Infrastructure Library in two different ways: - All of the UML metamodel is instantiated from meta-metaclasses that are defined in the Infrastructure Library. - The UML metamodel imports and specializes all metaclasses in the Infrastructure Library. As was discussed earlier, it is possible for a model to be used as a metamodel, and here we make use of this fact. The Infrastructure Library is in one capacity used as a meta-metamodel and in the other aspect as a metamodel, and is thus reused in two dimensions." Just to make sure that everyone fully understands the structure, page 40 hammers it home: "When dealing with meta-layers to define languages there are generally three layers that always has to be taken into account: - the language specification, or the metamodel, - the user specification, or the model, and - objects of the model. This structure can be applied recursively many times so that we get a possibly infinite number of meta-layers; what is a metamodel in one case can be a model in another case, and this is what happens with UML and MOF. UML is a language specification (metamodel) from which users can define their own models. Similarly, MOF is also a language specification (metamodel) from which users can define their own models. From the perspective of MOF, however, UML is viewed as a user (i.e., the members of the OMG that have developed the language) specification that is based on MOF as a language specification. In the four-layer metamodel hierarchy, MOF is commonly referred to as a meta-metamodel, even though strictly speaking it is a metamodel." /Uffe > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Joe Armstrong > (AL/EAB) > Sent: den 2 mars 2005 13:36 > To: Erlang (E-mail) > Cc: Hans Brolin (AS/EAB); helen work > Subject: Gulp > > > > OMG anyone? > > Some quotes from the amazing Meta-Object Facility > Specification available at > > http://www.omg.org/docs/formal/02-04-03.pdf > > The Meta Object Facility Specification is what is used to > descibe the meta-meta model of a UML model (was that right?)- > this 358 pages mastodon > document is full of little gems like (section 2.2.1, page 34): > > The metamodel layer is comprised of the descriptions > (i.e., meta-metadata) > that define the structure and semantics of metadata. > Meta-metadata is informally aggregated > as metamodels. A metamodel is an "abstract language" for > describing different kinds of data; > that is, a language without a concrete syntax or notation. > > I'm having slight problems understanding this: > > Let's try an put the clause "describing the meta-meta > model of a UML-model" > into my language understanding engine. > > > "describing the meta-meta model of a UML-model" > > => "describing X" where X = "the meta-meta model of a UML-model" > > I think that > > "the meta-meta model of a UML-model" => "meta-meta-meta model" > > and "describing X" => a meta-model of X > > so this should be a meta-meta-meta-meta model > > But is this > > "a language without a concrete syntax or notation" > > This is the puzzling bit. > > > /Joe > From chandrashekhar.mullaparthi@REDACTED Wed Mar 2 15:03:29 2005 From: chandrashekhar.mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Wed, 2 Mar 2005 14:03:29 +0000 Subject: Calling internal functions - foo::bar() ? In-Reply-To: References: Message-ID: On 28 Feb 2005, at 22:13, Luke Gorrie wrote: > Hi, > > I have an Erlang feature request: I'd like to be able to call > unexported functions of modules from the Erlang shell. > Me too! I've tried wrapping the -compile(export_all) directive in -ifdef(debug)/-endif and then use a Makefile to compile with the debug option or not. But then I can't deliver this system to test as it is not what will go live. So this feature would be immensely useful. I'm afraid of removing the export_all from a module. > P.S., who can beat this? > > $ find . -name "*.erl" | wc -l > 813 > $ find . -name "*.erl" | xargs grep '^-compile(export_all)' | wc > -l > 323 > $ echo $[ 323 * 100 / 813 ] > 39 > $ find . -name "*.erl" | wc -l 852 $ find . -name "*.erl" | xargs grep '^-compile(export_all)' | wc -l 219 $ echo $[219 * 100/852] 25 From joe.armstrong@REDACTED Wed Mar 2 15:05:57 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Wed, 2 Mar 2005 15:05:57 +0100 Subject: Gulp Message-ID: > Some quotes from the amazing Meta-Object Facility Specification available at > http://www.omg.org/docs/formal/02-04-03.pdf > The metamodel layer is comprised of the descriptions > (i.e., meta-metadata) that define the structure and semantics of metadata. > Meta-metadata is informally aggregated as metamodels. A metamodel is an > "abstract language" for describing different kinds of data; > that is, a language without a concrete syntax or notation. I re-read this *again* - It looks like an April fool joke. Then I checked the date - the documents says April 2002 The 2'nd hit of the Google query "omg mof 1 april" is: OMG Process Deadlines ... OMG Process Deadlines Last updated: Saturday, 19-Feb-2005 02:15:18 ... RFP (2nd Revised Submission Deadline) MOF 2.0 Versioning RFP (2nd Revised ... April 1, 2005, Org. ... www.omg.org/schedule/tcdeadlines.html - 29k - Cached - Similar pages Now follow the link - it leads to a page caled "OMG process deadlines" Half way down the pages is: April 1, 2005 Org. Structure Metamodel RFP (Voting List Deadline) So there we have it. Is the OMG MOF model an elaborate April Fools joke? /Joe From kostis@REDACTED Wed Mar 2 15:40:04 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 2 Mar 2005 15:40:04 +0100 (MET) Subject: Calling internal functions - foo::bar() ? In-Reply-To: Mail from 'Chandrashekhar Mullaparthi ' dated: Wed, 2 Mar 2005 14:03:29 +0000 Message-ID: <200503021440.j22Ee4dU019737@spikklubban.it.uu.se> Chandrashekhar Mullaparthi wrote: > Me too! I've tried wrapping the -compile(export_all) directive in > -ifdef(debug)/-endif and then use a Makefile to compile with the debug > option or not. But then I can't deliver this system to test as it is > not what will go live. So this feature would be immensely useful. Being somebody who never used the -compile(export_all) directive in his life, I really would very much like to understand why this proposed addition would be "immensely useful" especially since currently there are very nice alternatives: erlc +export_all File or c(File, [export_all]). which nicely do the trick (whenever *really* needed). > I'm afraid of removing the export_all from a module. Hint: don't add it in the first place. Kostis > > P.S., who can beat this? Depends on your definition of "beat" :) $ find lib/hipe -name "*.erl" | wc -l 204 $ find lib/hipe -name "*.erl" | xargs grep '^-compile(export_all)' | wc -l 0 From mats.cronqvist@REDACTED Wed Mar 2 16:07:40 2005 From: mats.cronqvist@REDACTED (Mats Cronqvist) Date: Wed, 02 Mar 2005 16:07:40 +0100 Subject: Calling internal functions - foo::bar() ? In-Reply-To: <200503021440.j22Ee4dU019737@spikklubban.it.uu.se> References: <200503021440.j22Ee4dU019737@spikklubban.it.uu.se> Message-ID: <4225D6BC.9050907@ericsson.com> Kostis Sagonas wrote: > Being somebody who never used the -compile(export_all) directive > in his life, I really would very much like to understand why this > proposed addition would be "immensely useful" especially since > currently there are very nice alternatives: > > erlc +export_all File > or > c(File, [export_all]). > > which nicely do the trick (whenever *really* needed). perhaps in an environment where you cannot use erlc? perhaps because there are 27 include files in places known only to the makefile, or the source code is not available? purely hypothetical of course. mats From ulf.wiger@REDACTED Wed Mar 2 16:28:39 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Wed, 2 Mar 2005 16:28:39 +0100 Subject: an erlang monad? Message-ID: I have had reason to do some rather complex code using a 'dict' dictionary. I find myself constantly wishing for a library function (and eventually writing it myself) that does the following: with(Dict, Actions) -> lists:foldl(fun(F,D) -> F(D) end, Dict, Actions). That is, fold over a list of functions that update the dictionary. This, I believe, essentially amounts to a monad in other languages. ;-) Even if this is just me misunderstanding monads (like everyone else), I find the function very useful. What this does is basically free you from the absurd D1, D2, D3, etc. gymnastics, where you will eventually, inevitably mess up the numbering. An example from my 'builder' module on Jungerl: Dict = with(Dict0, [fun(D) -> read_options_file(D) end, fun(D) -> store_options(Options, D) end, fun(D) -> post_process_options(D) end, fun(D) -> {Path, [App, Vsn]} = get_app(D), store_options([{app_name, list_to_atom(App)}, {app_vsn, Vsn}], D) end, fun(D) -> [Descr, Id, Reg, Apps, Env, Mod, Phases] = read_app_file(D), Vsn = fetch(app_vsn, D), store_options([{app, [{vsn,Vsn}, Descr, Id, Reg, Apps, Env, Mod, Phases]}], D) end, fun(D) -> case dict:find(rel_name, D) of {ok,_} -> D; error -> RelName = atom_to_list( fetch(app_name, D)), dict:store(rel_name, RelName, D) end end, fun(D) -> RelFname = get_rel_filename(D), case file:consult(RelFname) of {ok, [{release,{_Name,_RelVsn}, _RelApps}= Rel]} -> ?report(debug,"rel_src = ~p~n",[Rel]), dict:store(rel_src,Rel, D); _ -> D end end, fun(D) -> AppInfo = find_apps(D), ?report(debug, "find_apps(D) -> ~p~n", [AppInfo]), dict:store(app_info, AppInfo, D) end, fun(D) -> BootVars = boot_vars(D), dict:store(boot_vars, BootVars, D) end]), From erlang@REDACTED Wed Mar 2 16:30:18 2005 From: erlang@REDACTED (Michael McDaniel) Date: Wed, 2 Mar 2005 07:30:18 -0800 Subject: _Erlang_Programmation_ In-Reply-To: <42257F83.3080108@erlang-fr.org> References: <20050301042819.GQ3858@fangora.autosys.us> <42257F83.3080108@erlang-fr.org> Message-ID: <20050302153018.GO3858@fangora.autosys.us> On Wed, Mar 02, 2005 at 09:55:31AM +0100, Mickael Remond wrote: > Michael McDaniel wrote: > >Can anyone tell me if Mickael Remond's book, _Erlang_Programmation_, is > >going to be > >available in English this year? > > I think that this is not going to happens this year has the project has > not started yet and has I have to find an English publisher willing to > buy the right to the French publisher Eyrolles first. > > I am working on it however. If you know English publisher that could be > interested, please, do not hesitate to ask them to contact me. > > -- > Micka?l R?mond ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Thanks for the information. I am ready to buy it in French rather than waiting. No, I don't really know French but think I could at the least wade through the examples. The page http://oreilly.com/oreilly/author/intro.csp contains the line "* Send proposals and proposal inquiries to proposals@REDACTED" ~Michael From joe.armstrong@REDACTED Wed Mar 2 16:44:49 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Wed, 2 Mar 2005 16:44:49 +0100 Subject: size of an IO list Message-ID: How about a BIF to compute the size of an I/O list? Why? I have made a deep list that I want to output (guess what's it's HTML :-) I need to make an HTTP header element "Content-Length: ..." Which is the size of the flattened deep list , but I don't want to flatten it (horrors) or traverse it (in Erlang) to find the length. Suitable case for a BIF? /Joe From chandrashekhar.mullaparthi@REDACTED Wed Mar 2 17:02:17 2005 From: chandrashekhar.mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Wed, 2 Mar 2005 16:02:17 +0000 Subject: Calling internal functions - foo::bar() ? In-Reply-To: <200503021440.j22Ee4dU019737@spikklubban.it.uu.se> References: <200503021440.j22Ee4dU019737@spikklubban.it.uu.se> Message-ID: <73278D2F-8B34-11D9-98AB-000A95865434@t-mobile.co.uk> On 2 Mar 2005, at 14:40, Kostis Sagonas wrote: > Chandrashekhar Mullaparthi wrote: > >> Me too! I've tried wrapping the -compile(export_all) directive in >> -ifdef(debug)/-endif and then use a Makefile to compile with the debug >> option or not. But then I can't deliver this system to test as it is >> not what will go live. So this feature would be immensely useful. > > Being somebody who never used the -compile(export_all) directive > in his life, I really would very much like to understand why this > proposed addition would be "immensely useful" especially since > currently there are very nice alternatives: > > erlc +export_all File > or > c(File, [export_all]). > > which nicely do the trick (whenever *really* needed). > Neither the compiler nor the source code is included in any(almost - except when we use yaws) system we produce. Our test team don't like us changing modules in the test installation (understandably) - so troubleshooting becomes a bit of a hassle. Typically, this is what I have to do if I want to troubleshoot in the testplant. 1. Take a backup of the existing module 2. Compile correct version of module with +export_all option 3. FTP to testplant (sometimes via two hops because security folks are sometimes Business Prevention Officers) 4. Load new module 5. Do stuff 6. Backout the patch It'll be nice to be able to just do this. 5. Do stuff. >> I'm afraid of removing the export_all from a module. > > Hint: don't add it in the first place. Well, when there are 15 different services live and 40 erlang nodes to support each with different versions of software, it is easier said than done. cheers Chandru From hakan@REDACTED Wed Mar 2 17:38:28 2005 From: hakan@REDACTED (Hakan Mattsson) Date: Wed, 2 Mar 2005 17:38:28 +0100 (CET) Subject: size of an IO list In-Reply-To: References: Message-ID: On Wed, 2 Mar 2005, Joe Armstrong (AL/EAB) wrote: JA> How about a BIF to compute the size of an I/O list? JA> JA> Why? JA> JA> I have made a deep list that I want to output (guess what's it's HTML :-) JA> JA> I need to make an HTTP header element "Content-Length: ..." JA> JA> Which is the size of the flattened deep list , but I don't want to flatten it (horrors) JA> or traverse it (in Erlang) to find the length. JA> JA> Suitable case for a BIF? What about Bin = list_to_binary(List), size(Bin)? /H?kan From luke@REDACTED Wed Mar 2 17:42:56 2005 From: luke@REDACTED (Luke Gorrie) Date: 02 Mar 2005 17:42:56 +0100 Subject: Calling internal functions - foo::bar() ? References: <200503021440.j22Ee4dU019737@spikklubban.it.uu.se> Message-ID: Kostis Sagonas writes: > currently there are very nice alternatives: > > erlc +export_all File > or > c(File, [export_all]). even on my development machine it would be more like this: "Touch" the file (e.g. insert blank line, save, undo, save.) 'make debug' Reload the right module ... do what I want ... "Touch" the file again 'make' Reload the right module And more complicated if the code is running on a target machine or the source file I have is hacked/broken. In practice I've found this way too inconvenient even when it comes to adding +debug_info to run the debugger and I do that /way/ less than poking at internal functions. Thankfully the +debug_info problem is solved once-and-for-all with a line in ~/.bashrc: export ERL_COMPILER_OPTIONS='[debug_info]' too dangerous to do the export_all equivalent though :-) From erlq@REDACTED Wed Mar 2 17:57:31 2005 From: erlq@REDACTED (Rob) Date: Wed, 02 Mar 2005 08:57:31 -0800 Subject: _Erlang_Programmation_ In-Reply-To: <20050302153018.GO3858@fangora.autosys.us> References: <20050301042819.GQ3858@fangora.autosys.us> <42257F83.3080108@erlang-fr.org> <20050302153018.GO3858@fangora.autosys.us> Message-ID: <4225F07B.2020309@itsbeen.sent.com> Michael McDaniel wrote: > On Wed, Mar 02, 2005 at 09:55:31AM +0100, Mickael Remond wrote: > >>Michael McDaniel wrote: >> >>>Can anyone tell me if Mickael Remond's book, _Erlang_Programmation_, is >>>going to be >>>available in English this year? >> >>I think that this is not going to happens this year has the project has >>not started yet and has I have to find an English publisher willing to >>buy the right to the French publisher Eyrolles first. >> >>I am working on it however. If you know English publisher that could be >>interested, please, do not hesitate to ask them to contact me. >> >>-- >>Micka?l R?mond > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > Thanks for the information. I am ready to buy it in French rather than waiting. > No, I don't really know French but think I could at the least wade through the > examples. > > The page http://oreilly.com/oreilly/author/intro.csp contains the line > > "* Send proposals and proposal inquiries to proposals@REDACTED" > > > ~Michael I don' think oreilly would go for an erlang book, from their Q & A: ---- We're NOT looking for: * Books that overlap too heavily with our existing books. * Books on proprietary technologies that don't have a huge user base. * Books on miniscule (i.e., personal or nascent) products, even if they are open source. * Books on topics that have dismal sales despite quality books being available. (If you're addressing a topic where good books have sold dismally in the past (for instance, LISP, LaTeX, or Web-based training), you have a much higher threshold to clear with your proposal. Convince us why there is a revival of interest in your topic, or why your approach to a deadly topic will provoke interest nonetheless.) * Books that have been rejected by other publishers, in most cases. ---- Seems there is a strong anti-lisp bent there and I think erlang would fall under the "looks lisp-ish" category and I think they would question the audience size. But I like the model, typography and production used for the Ruby book. They have a deal to buy the PDF, paper book or both - http://pragmaticprogrammer.com/authors/index.html Seems to be good quality and maybe more open to taking a risk. And since they are distributed by oreilly, you get the best of both worlds - great exposure because the only have a few title so far (all current topics and no slumming with MCSE Certification type books), they are just beginning to branch out (e.g. http://pragmaticprogrammer.com/titles/gwd/index.html) and they have the world's best distribution channel for computer related books. I've cc'd them, maybe it is enough to start a dialog to see if there is a fit. Rob From thomasl_erlang@REDACTED Wed Mar 2 18:16:22 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 2 Mar 2005 09:16:22 -0800 (PST) Subject: _Erlang_Programmation_ In-Reply-To: <4225F07B.2020309@itsbeen.sent.com> Message-ID: <20050302171622.52561.qmail@web41903.mail.yahoo.com> --- Rob wrote: > they are just > beginning to branch out (e.g. > http://pragmaticprogrammer.com/titles/gwd/index.html) > and they have the > world's best distribution channel for computer > related books. Good point; O'Reilly is attractive but not the only game in town. I think that, e.g., Apress has a similar target audience. (Practitioners rather than academics, that is.) http://www.apress.com/about/writeForUs.html So there is some scope for shopping around. Best, Thomas __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From pascal.brisset@REDACTED Wed Mar 2 18:19:40 2005 From: pascal.brisset@REDACTED (Pascal Brisset) Date: Wed, 2 Mar 2005 18:19:40 +0100 Subject: an erlang monad? In-Reply-To: References: Message-ID: <20050302171845.5B2F41C001DA@mwinf0604.wanadoo.fr> > That is, fold over a list of functions that update > the dictionary. This, I believe, essentially amounts > to a monad in other languages. ;-) Even if this is > just me misunderstanding monads (like everyone else), > I find the function very useful. It looks even more like an iterated version the popular "maybe monad" if each function is allowed to abort the computation: with(X, []) -> X; with(X, [F|Fs]) -> case F(X) of {ok, Y} -> with(Y, Fs); {error, E} -> {error, E} end. > What this does is basically free you from the absurd > D1, D2, D3, etc. gymnastics, where you will eventually, > inevitably mess up the numbering. Ah, so we are not the only ones... I was ashamed that we had to write a dedicated linter to catch all these numbering errors :-) -- Pascal Brisset From mccratch@REDACTED Wed Mar 2 18:25:47 2005 From: mccratch@REDACTED (Matthias Kretschmer) Date: Wed, 2 Mar 2005 18:25:47 +0100 Subject: an erlang monad? In-Reply-To: References: Message-ID: <20050302182547.47c5ee3a@localhost.localdomain> On Wed, 2 Mar 2005 16:28:39 +0100 "Ulf Wiger \(AL/EAB\)" wrote: > > I have had reason to do some rather complex code using > a 'dict' dictionary. I find myself constantly wishing > for a library function (and eventually writing it > myself) that does the following: > > with(Dict, Actions) -> > lists:foldl(fun(F,D) -> > F(D) > end, Dict, Actions). > > That is, fold over a list of functions that update > the dictionary. This, I believe, essentially amounts > to a monad in other languages. ;-) Even if this is > just me misunderstanding monads (like everyone else), > I find the function very useful. This is something I use monads for, too. Hiding some sort of state or other information (serialization of operation or doing i/o is not that much of a concern using Erlang :-)). I wrote a little parse_transform'er to make the notation easier: monad(MT, Action0, Action1, 1 = mreturn(1), Action2, X = Action3, Action4, Action5(X)). this would be roughly like (assuming MT.>>= is an infix operation): Action0 MT.>>= fun (_) -> Action1 MT.>>= fun (_) -> MT.return(1) MT.>>= fun (1) -> Action2 MT.>>= fun (_) -> Action3 MT.>>= fun (X) -> Action4 MT.>>= fun (_) -> Action5(X) end end end end end trying to resemble basically Haskell's do-notation -- except that instead of "X <- Action" "X = Action" (the MT is the monad type specification -- basically a tuple containing the "bind"/">>=", "return", etc. functions). Your with/2 function may be easily translated to this notation. I like this notation, because it may be easily used for other types of monads, too, not only the basic state carrying monad and it is possible to build a set of basic combinators that may be used with any monad type. Like backtracking parsers or other stuff. This way you may even drop the fun (D) -> ... end and just write the expressions, using appropriate monad values for reading and writing the state (e.g. wrapping the basic dictionary operations). I think a really nice do-notation or something like that would be very nice to have in Erlang. I am always missing it (though I am currently quited happy with my transformation-hack). -- cheers Matthias Kretschmer From erlang@REDACTED Wed Mar 2 18:42:20 2005 From: erlang@REDACTED (Michael McDaniel) Date: Wed, 2 Mar 2005 09:42:20 -0800 Subject: _Erlang_Programmation_ In-Reply-To: <4225F07B.2020309@itsbeen.sent.com> References: <20050301042819.GQ3858@fangora.autosys.us> <42257F83.3080108@erlang-fr.org> <20050302153018.GO3858@fangora.autosys.us> <4225F07B.2020309@itsbeen.sent.com> Message-ID: <20050302174220.GR3858@fangora.autosys.us> On Wed, Mar 02, 2005 at 08:57:31AM -0800, Rob wrote: > Michael McDaniel wrote: > >On Wed, Mar 02, 2005 at 09:55:31AM +0100, Mickael Remond wrote: > > > >>Michael McDaniel wrote: > >> > >>>Can anyone tell me if Mickael Remond's book, _Erlang_Programmation_, is > >>>going to be > >>>available in English this year? > >> > >>I think that this is not going to happens this year has the project has > >>not started yet and has I have to find an English publisher willing to > >>buy the right to the French publisher Eyrolles first. > >> > >>I am working on it however. If you know English publisher that could be > >>interested, please, do not hesitate to ask them to contact me. > >> > >>-- > >>Micka?l R?mond > > > >^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >Thanks for the information. I am ready to buy it in French rather than > >waiting. > >No, I don't really know French but think I could at the least wade through > >the > >examples. > > > >The page http://oreilly.com/oreilly/author/intro.csp contains the line > > > > "* Send proposals and proposal inquiries to proposals@REDACTED" > > > > > >~Michael > > I don' think oreilly would go for an erlang book, from their Q & A: > ---- > We're NOT looking for: > > * Books that overlap too heavily with our existing books. > * Books on proprietary technologies that don't have a huge user base. > * Books on miniscule (i.e., personal or nascent) products, even if > they are open source. > * Books on topics that have dismal sales despite quality books > being available. (If you're addressing a topic where good books have > sold dismally in the past (for instance, LISP, LaTeX, or Web-based > training), you have a much higher threshold to clear with your proposal. > Convince us why there is a revival of interest in your topic, or why > your approach to a deadly topic will provoke interest nonetheless.) > * Books that have been rejected by other publishers, in most cases. > ---- > > Seems there is a strong anti-lisp bent there and I think erlang would > fall under the "looks lisp-ish" category and I think they would question > the audience size. > > But I like the model, typography and production used for the Ruby book. > They have a deal to buy the PDF, paper book or both - > http://pragmaticprogrammer.com/authors/index.html > Seems to be good quality and maybe more open to taking a risk. And since > they are distributed by oreilly, you get the best of both worlds - great > exposure because the only have a few title so far (all current topics > and no slumming with MCSE Certification type books), they are just > beginning to branch out (e.g. > http://pragmaticprogrammer.com/titles/gwd/index.html) and they have the > world's best distribution channel for computer related books. > > I've cc'd them, maybe it is enough to start a dialog to see if there is > a fit. > > Rob ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Thanks for the extra info, Rob. I did forget to mention that O'Reilly has 'Publishing Partners' so contacting them (O'Reilly) may provoke interest or forwarding of idea to Partners. The page I originally sent has the Partners in left column. ~Michael From luna@REDACTED Wed Mar 2 19:01:49 2005 From: luna@REDACTED (Daniel Luna) Date: Wed, 2 Mar 2005 19:01:49 +0100 (CET) Subject: size of an IO list In-Reply-To: References: Message-ID: On Wed, 2 Mar 2005, Joe Armstrong (AL/EAB) wrote: > How about a BIF to compute the size of an I/O list? > > Why? > > I have made a deep list that I want to output (guess what's it's HTML :-) > > I need to make an HTTP header element "Content-Length: ..." > > Which is the size of the flattened deep list , but I don't want to flatten it (horrors) > or traverse it (in Erlang) to find the length. > > Suitable case for a BIF? There is lists:flatlength/1 that does this, unfortunately it's not a BIF. /Luna -- Daniel Luna | Top reasons that I have a beard: luna@REDACTED | a) Laziness. http://www.update.uu.se/~luna/ | b) I can. Don't look at my homepage (it stinks).| c) I can get away with it. From james.hague@REDACTED Wed Mar 2 19:11:52 2005 From: james.hague@REDACTED (James Hague) Date: Wed, 2 Mar 2005 12:11:52 -0600 Subject: Calling internal functions - foo::bar() ? In-Reply-To: References: <200503021440.j22Ee4dU019737@spikklubban.it.uu.se> Message-ID: Does the compiler not take advantage of functions being internal to a module? For example, if a function always returns a tuple which is immediately unpacked, then the tuple construction can be avoided and the values returned in Erlang registers. At least I recall reading about this somewhere. If this is the case, then you wouldn't be able to call internal functions directly. From ulf@REDACTED Wed Mar 2 21:15:06 2005 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 02 Mar 2005 21:15:06 +0100 Subject: an erlang monad? In-Reply-To: <20050302182547.47c5ee3a@localhost.localdomain> References: <20050302182547.47c5ee3a@localhost.localdomain> Message-ID: Den 2005-03-02 18:25:47 skrev Matthias Kretschmer : > This is something I use monads for, too. Hiding some sort of state or > other information (serialization of operation or doing i/o is not that > much of a concern using Erlang :-)). I wrote a little parse_transform'er > to make the notation easier: > > monad(MT, > Action0, > Action1, > 1 = mreturn(1), > Action2, > X = Action3, > Action4, > Action5(X)). > > this would be roughly like (assuming MT.>>= is an infix operation): > > Action0 MT.>>= fun (_) -> > Action1 MT.>>= fun (_) -> > MT.return(1) MT.>>= fun (1) -> > Action2 MT.>>= fun (_) -> > Action3 MT.>>= fun (X) -> > Action4 MT.>>= fun (_) -> > Action5(X) end end end end end > > trying to resemble basically Haskell's do-notation -- except that > instead of "X <- Action" "X = Action" (the MT is the monad type > specification -- basically a tuple containing the "bind"/">>=", > "return", etc. functions). Great! Couple this with John Hughes' ?DELAY and ?FORCE macros, and you've almost got yourself a real language (except for the lack of static typing, of course) ;-) > This way you may even drop the fun (D) -> ... end and just write the > expressions, using appropriate monad values for reading and writing the > state (e.g. wrapping the basic dictionary operations). I played around for a while with macros in order to avoid having to write fun(D) -> ... end all the time, but eventually I gave up. /Uffe From luke@REDACTED Wed Mar 2 22:54:54 2005 From: luke@REDACTED (Luke Gorrie) Date: 02 Mar 2005 22:54:54 +0100 Subject: an erlang monad? References: Message-ID: "Ulf Wiger \(AL/EAB\)" writes: > I have had reason to do some rather complex code using > a 'dict' dictionary. I find myself constantly wishing > for a library function (and eventually writing it > myself) that does the following: > > with(Dict, Actions) -> > lists:foldl(fun(F,D) -> > F(D) > end, Dict, Actions). > > That is, fold over a list of functions that update > the dictionary. This, I believe, essentially amounts > to a monad in other languages. ;-) Even if this is > just me misunderstanding monads (like everyone else), > I find the function very useful. > > What this does is basically free you from the absurd > D1, D2, D3, etc. gymnastics, where you will eventually, > inevitably mess up the numbering. > > An example from my 'builder' module on Jungerl: > > Dict = with(Dict0, > [fun(D) -> read_options_file(D) end, > fun(D) -> store_options(Options, D) end, > fun(D) -> post_process_options(D) end, ... But surely the state monad is just a poor man's process dictionary.. D = make_ref(), put(D, mk_dict(default_options() ++ Options)), read_options_file(D), store_options(Options, D), post_process_options(D), ... erase(D). From erlang@REDACTED Thu Mar 3 01:59:23 2005 From: erlang@REDACTED (Michael McDaniel) Date: Wed, 2 Mar 2005 16:59:23 -0800 Subject: inets ssl proxy In-Reply-To: <20050224054129.GC14094@fangora.autosys.us> References: <20050223013404.GM14094@fangora.autosys.us> <16924.39226.611947.37660@gargle.gargle.HOWL> <20050224054129.GC14094@fangora.autosys.us> Message-ID: <20050303005923.GV3858@fangora.autosys.us> Well, I have not yet been successful. I thought it might be because I upgraded OpenSSL (which got installed in /usr/local) but still had the default SuSE OpenSSL installed elsewhere. I recompiled Erlang against OpenSSL 0.9.7e and still problems. Perhaps it is something else wrong with this machine as I also get Yaws problems like ----------------- ERROR erlang code crashed: File: /var/yaws/www/arg.yaws:59 Reason: {function_clause,[{prim_inet,peername,[{sslsocket,5,<0.63.0>}]}, {m2,out,1}, {yaws_server,deliver_dyn_part,8}, {yaws_server,aloop,3}, {yaws_server,acceptor0,2}, {proc_lib,init_p,5}]} Req: {http_request,'GET',{abs_path,"/arg.yaws"},{1,1}} ----------------- when I try to https://localhost/arg.yaws from this (localhost) machine, with Yaws running on this (localhost) machine. Most of the other https://localhost Yaws pages work fine. Anyway, back to original problem, Internet (for https://test.xquad.com ...) - - - - - - - + - - - - - - - - -- | +-------+----------+ | | | autosys.us | runs web proxy SQUID 3.0-PRE3 port 3128 | | +-------+----------+ | | | +-------+------------+ | | | fangora.autosys.us | | aka 'localhost' | | | +--------------------+ $ uname -a Linux fangora 2.6.4-52-default #1 Wed Apr 7 02:08:30 UTC 2004 i686 i686 i386 GNU/Linux $ erl Erlang (BEAM) emulator version 5.4.4 [source] [hipe] Eshell V5.4.4 (abort with ^G) 1> c:c(hd). {ok,hd} 2> hd:start(). https://test.xquad.com, error, esslerrssl https://autosys.us, ok, {{"HTTP/1.1",200,"OK"}, [{"connection","Keep-Alive"}, {"date","Wed, 02 Mar 2005 22:56:42 GMT"}, {"server","Apache"}, {"content-length","16364"}, {"content-type","text/html; charset=ISO-8859-1"}, {"keep-alive","timeout=15, max=100"}], etc., etc., etc. https://localhost, ok, {{"HTTP/1.1",200,"OK"}, [{"connection","Keep-Alive"}, {"date","Wed, 02 Mar 2005 22:56:43 GMT"}, {"server","Apache"}, {"content-length","16364"}, {"content-type","text/html; charset=ISO-8859-1"}, {"keep-alive","timeout=15, max=100"}], etc., etc., etc. 3> q(). ok 4> $ So, ssl is apparently working, just not when it has to go through the proxy server. And, apparently the reason for this is that, when going through the proxy, the very first thing that is sent is encrypted data rather than a connect request, and the proxy returns an invalid request error. I would not have bothered with this any more right now, except I think maybe that it is somehow related to my Yaws problem, and I *really* need to have Yaws working reliably over SSL. The hd module is below, thanks for any feedback. ~Michael ----------------- -module(hd). -export([start/0, hds/0, hds1/0, hds2/0]). start() -> {H, C} = hds() , io:format("https://test.xquad.com, ~p, ~p ~n", [H,C]), {H1, C1} = hds1() , io:format("https://autosys.us, ~p, ~p ~n", [H1, C1]), {H2, C2} = hds1() , io:format("https://localhost, ~p, ~p ~n", [H2, C2]) . hds() -> application:start(inets) , application:start(ssl) , {ok, Ciphers} = ssl:ciphers() , http:set_options([{proxy, {{"autosys.us", 3128 }, ["localhost"]}}]) , http:request(get, {"https://test.xquad.com", [ {"Referer", "https://test.xquad.com"}, {"Host", "fangora.autosys.us"} ]}, [{verify, 0}, {ciphers, Ciphers}], []). hds1() -> application:start(inets) , application:start(ssl) , {ok, Ciphers} = ssl:ciphers() , http:set_options([{proxy, {{"autosys.us", 3128 }, ["autosys.us"]}}]) , http:request(get, {"https://autosys.us", [ {"Referer", "https://autosys.us"}, {"Host", "fangora.autosys.us"} ]}, [{verify, 0}, {ciphers, Ciphers}], []). hds2() -> application:start(inets) , application:start(ssl) , {ok, Ciphers} = ssl:ciphers() , http:set_options([{proxy, {{"autosys.us", 3128 }, ["localhost"]}}]) , http:request(get, {"https://localhost", [ {"Referer", "https://localhost"}, {"Host", "fangora.autosys.us"} ]}, [{verify, 0}, {ciphers, Ciphers}], []). ----------------- On Wed, Feb 23, 2005 at 09:41:29PM -0800, Michael McDaniel wrote: > Thank you, Ingela. I will keep trying ssl options to try to get it > working. Maybe I have some other problem also - I'll keep trying. > > From my direct-connect machine, without using proxy, I do not need > to set any ssl options at all and it works fine. Although, now > as I think of it, my direct-connect machine is using R10B-3 > with inets-4.0.1 and my proxy machine is all R10B-3 (inets-4.2). > > ~Michael > > > On Wed, Feb 23, 2005 at 03:54:50PM +0100, Ingela Anderton wrote: > > > > You need to specify some ssl options! Something like: > > > > SSLOptions = [{certfile, CertFile}, {keyfile, CertFile}], > > http:request(get, {URL, []}, [{ssl, SSLOptions}], []), > > > > Michael McDaniel wrote: > > > Should the following work, or am I doing something wrong? > > > > > > -module(hd). > > > -export([hd/3]). > > > > > > > > > hd(Proxy, Port, URL) -> > > > application:start(inets) , > > > application:start(ssl) , > > > > > > http:set_options( [ {proxy, {{Proxy,Port}, ["localhost"]} } ] ) , > > > > > > http:request(get, > > > {URL, [ {"Version", "HTTP/1.0"}, {"Referer", URL}, > > > {"Host", "www.myhost.org"} ]}, [], []). > > > > > > $ uname -a > > > Linux fangora 2.6.4-52-default #1 Wed Apr 7 02:08:30 UTC 2004 i686 i686 i386 GNU/Linux > > > $ erl > > > Erlang (BEAM) emulator version 5.4.4 [source] [hipe] > > > > > > Eshell V5.4.4 (abort with ^G) > > > 1> c:c(hd). > > > {ok,hd} > > > 2> hd:hd("my.proxy.org",3128,"https://test.xquad.com"). > > > {error,esslerrssl} > > > 3> hd:hd("my.proxy.org",3128,"http://www.quickbots.com"). > > > {ok,{{"HTTP/1.0",200,"OK"}, > > > [{"date","Wed, 23 Feb 2005 01:34:04 GMT"}, > > > {"via","1.0 cougora.autosys.us (squid/3.0-PRE3)"}, > > > {"etag","\"15481-1ee2-da976100\""}, > > > {"server","Apache"}, > > > {"content-length","7906"}, > > > {"content-type","text/html; charset=ISO-8859-1"}, > > > {"last-modified","Wed, 15 Jan 2003 21:45:08 GMT"}, > > > {"accept-ranges","bytes"}, > > > {"x-cache","MISS from cougora.autosys.us"}, > > > {"x-cache-lookup","HIT from cougora.autosys.us:3128"}, > > > {"proxy-connection","keep-alive"}], > > > "\r\n\r\n\r\n > > 4> halt(). > > > $ > > > > > > The problem appears to be (looking at a trace) that the initial packets being sent by > > > hd:hd("my.proxy.org",3128,"https://test.xquad.com"). > > > to my.proxy.org is all encrypted, rather than being a request that the proxy server > > > can read, so the proxy server passes back the error INVALID REQUEST. As you can see, > > > the http:// request works great via proxy. > > > > > > > > > thank you, > > > > > > ~Michael > > > > -- > > /m.v.h Ingela > > > > //The highway of life is always under construction. // > > > > |\ _,,,--,,_ ,) > > /,`.-'`' -, ;-;;' > > |,4- ) )-,_ ) /\ > > '---''(_/--' (_/-' > > > > Ericsson AB - OTP team > > > > Cellular/Mobile: +46 70 636 78 68 > > > > > > > > From anders.nygren@REDACTED Thu Mar 3 03:08:44 2005 From: anders.nygren@REDACTED (Anders Nygren) Date: Wed, 2 Mar 2005 20:08:44 -0600 Subject: http problem In-Reply-To: References: Message-ID: On Tue, 1 Mar 2005 19:36:16 -0600, Anders Nygren wrote: > Hi > I have a program that does > URL="http://148.233.175.237:80/callejero/callejeroXMLfeed2.asp?name=ford&cp=01020&tu=3" > Headers=[], > Request={URL,Headers}, > Timeout=Srv_config#http_server.timeout, > Reply=http:request(get,Request,[{timeout,Timeout}],[]) , > > On linux with otp-R10B-0 Erlang (BEAM) emulator version 5.4 [source] > [hipe] it works fine. > But on Windows XP otp-R10B-3 Erlang 5.4.3 it crashes like shown below. > Some more testing and tracing has led me to the place of the error, but I still don't know why. prim_inet.erl: open1(Type, Family) -> case open0(Type) of {ok, S} -> case ctl_cmd(S,?INET_REQ_OPEN,[Family]) of {ok, _} -> {ok,S}; Error -> close(S), Error end; Error -> Error end. <0.145.0>: call inet:open(-1, {0, 0, 0, 0, 0, 0, 0, 0}, 0, [{reuseaddr,true},{active,false},{packet,0},{mode,binary}], stream, inet6, inet6_tcp) <0.145.0>: call prim_inet:open(stream, inet6) <0.145.0>: call prim_inet:open1(stream, 2) <0.145.0>: call prim_inet:open0(stream) <0.145.0>: call erlang:open_port({spawn, tcp_inet}, [binary]) <0.145.0>: call prim_inet:ctl_cmd(#Port<0.279>, 1, [2]) <0.145.0>: call erlang:port_control(#Port<0.279>, 1, [2])<========= Here it fails <0.145.0>: call erlang:list_to_atom("einval") <0.145.0>: call prim_inet:close(#Port<0.279>) <0.145.0>: call erlang:unlink(#Port<0.279>) or from the shell 7>Port=erlang:open_port({spawn,tcp_inet},[binary]). #Port<0.36> 8>erlang:port_control(Port,1,[2]). [0,101,105,110,118,97,108] The funny thing is that I can set up a tcp connection from the windows machine to the linux machine. But not from the windows machine to the internet. Any ideas anyone???? /Anders Nygren From david.nospam.hopwood@REDACTED Thu Mar 3 03:47:33 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Thu, 03 Mar 2005 02:47:33 +0000 Subject: Gulp In-Reply-To: References: Message-ID: <42267AC5.1060402@blueyonder.co.uk> Joe Armstrong (AL/EAB) wrote: > OMG anyone? > > Some quotes from the amazing Meta-Object Facility Specification available at > > http://www.omg.org/docs/formal/02-04-03.pdf Here's an analogy that may help to explain why this is silly: "We've designed a language X for writing compilers. What language should we implement the compiler for X in? I know -- let's design another language Y specifically to do that. It will have a 358-page spec, but won't be expressive enough to do anything useful other than write a compiler for X." -- David Hopwood From ulf@REDACTED Thu Mar 3 06:43:32 2005 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 03 Mar 2005 06:43:32 +0100 Subject: an erlang monad? In-Reply-To: References: Message-ID: Den 2005-03-02 22:54:54 skrev Luke Gorrie : > But surely the state monad is just a poor man's process dictionary.. > > D = make_ref(), > put(D, mk_dict(default_options() ++ Options)), > read_options_file(D), > store_options(Options, D), > post_process_options(D), > ... > erase(D). Well, if you want to be all imperative about it, yes. (: But one of the things I want to do this time is to be able to undo easily if something goes wrong. With a functional data structure, this is trivial. /Uffe From vlad_dumitrescu@REDACTED Thu Mar 3 08:41:42 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Thu, 3 Mar 2005 08:41:42 +0100 Subject: an erlang monad? References: <20050302182547.47c5ee3a@localhost.localdomain> Message-ID: Hi, Just a thought about this. It is important to be able to access the state even somewhere in-between the sequence, and it should be easy to do so too. The reasons are two: * one may need this state later on (I almost never have more than 3 levels where the intermediary state isn't needed) * easily inserting (and removing) debugging traces is very useful for finding out what went wrong. regards, Vlad From micke@REDACTED Thu Mar 3 10:29:01 2005 From: micke@REDACTED (Michael Fogeborg) Date: Thu, 03 Mar 2005 10:29:01 +0100 Subject: Gulp In-Reply-To: <42267AC5.1060402@blueyonder.co.uk> References: <42267AC5.1060402@blueyonder.co.uk> Message-ID: <6.2.1.2.0.20050303102350.027b2860@mail.online.no> At 03:47 2005-03-03, David Hopwood wrote: >Joe Armstrong (AL/EAB) wrote: >> OMG anyone? >> Some quotes from the amazing Meta-Object Facility Specification >> available at >> http://www.omg.org/docs/formal/02-04-03.pdf > >Here's an analogy that may help to explain why this is silly: > > "We've designed a language X for writing compilers. What language > should we implement the compiler for X in? I know -- let's design > another language Y specifically to do that. It will have a 358-page > spec, but won't be expressive enough to do anything useful other than > write a compiler for X." >-- >David Hopwood On the first course at Uppsala Uni' we had to implement Lisp in Lisp. ( Fun! ) It took one page with comments... not very advanced since it only took one page I guess. --- From dietmar@REDACTED Thu Mar 3 10:49:08 2005 From: dietmar@REDACTED (Dietmar Schaefer) Date: Thu, 03 Mar 2005 10:49:08 +0100 Subject: creating a target system Message-ID: <4226DD94.3030804@ast.dfs.de> Hi ! I am trying to build a target system. Here is my cmmc.rel file {release, {"4dp-cmmc", "1.0"}, { erts, [kernel, stdlib, sasl, mnesia, snmp, cmmc-foc]}. calling target_system:create("cmmc"). I get: ** exited: {undef,[{sys_pre_expand,module, [[{function, 1, foo, 0, [{clause, 1, [], [], [{call, 1, {remote,1|...}, [{...}]}]}]}], []]}, {shell,expand_records,2}, {shell,exprs,6}, {shell,eval_loop,3}]} ** Using the mysystem.rel file from "System Principles" I get the same ! BTW: I am using R10B2 on FC2 Regards Dietmar From ulf.wiger@REDACTED Thu Mar 3 11:30:40 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Thu, 3 Mar 2005 11:30:40 +0100 Subject: an erlang monad? Message-ID: One problem with the with/2 is that is not obvious how to pass intermediate results to the next function without having to put everything in the dictionary. This is how I decided to do it, after tiring once more of constantly renumbering variables: Dict1 = with( Dict, [fun(D) -> {_BeforeApp, _Dict1} = pickup_key(before, App, D) end, fun({BeforeApp, D}) -> {RequiredBy, Dict1} = pickup_key(required_by, App, D), {BeforeApp, RequiredBy, Dict1} end, fun({BeforeApp, RequiredBy, D}) -> dict:store(App, #info{waitfor = BeforeApp ++ Apps, required_by = RequiredBy, requires = Apps}, D) end, fun(D) -> enter_key(required_by, Apps, fun(#info{required_by = RB} = I) -> I#info{required_by = [App|RB]} end, App, D) end, fun(D) -> case Ps of undefined -> D; Phases when is_list(Phases) -> load_phases(Phases, App, D) end end]), {reply, ok, S#state{dict = Dict1}}; On a beauty scale of 1-10, it's definitely not a 10. But I don't think it's all that bad either. /Uffe > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Vlad Dumitrescu > Sent: den 3 mars 2005 08:42 > To: erlang-questions@REDACTED > Subject: Re: an erlang monad? > > > Hi, > > Just a thought about this. > > It is important to be able to access the state even somewhere > in-between the > sequence, and it should be easy to do so too. > > The reasons are two: > * one may need this state later on (I almost never have > more than 3 levels > where the intermediary state isn't needed) > * easily inserting (and removing) debugging traces is very > useful for finding > out what went wrong. > > regards, > Vlad > From tobbe@REDACTED Thu Mar 3 12:12:36 2005 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: Thu, 03 Mar 2005 12:12:36 +0100 Subject: creating a target system In-Reply-To: <4226DD94.3030804@ast.dfs.de> References: <4226DD94.3030804@ast.dfs.de> Message-ID: <4226F124.8000703@nortel.com> A nice HowTo on this topic would be nice! Any takers ? You can upload it at: http://www.trapexit.org/docs/howto/ Cheers, Tobbe Dietmar Schaefer wrote: > Hi ! > > > I am trying to build a target system. > > Here is my cmmc.rel file > > > > > {release, > {"4dp-cmmc", "1.0"}, > { erts, > [kernel, > stdlib, > sasl, > mnesia, > snmp, > cmmc-foc]}. > > > calling target_system:create("cmmc"). > > > I get: > > ** exited: {undef,[{sys_pre_expand,module, > [[{function, > 1, > foo, > 0, > [{clause, > 1, > [], > [], > [{call, > 1, > {remote,1|...}, > [{...}]}]}]}], > []]}, > {shell,expand_records,2}, > {shell,exprs,6}, > {shell,eval_loop,3}]} ** > > > Using the mysystem.rel file from "System Principles" I get the same ! > > > BTW: I am using R10B2 on FC2 > > > > Regards > > > Dietmar From ulf.wiger@REDACTED Thu Mar 3 13:07:07 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Thu, 3 Mar 2005 13:07:07 +0100 Subject: creating a target system Message-ID: I wrote a release tutorial a long time ago. I don't have the time right now to reformat it. If someone else wants to do it, let me know. /Uffe > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Torbjorn > Tornkvist > Sent: den 3 mars 2005 12:13 > To: Dietmar Schaefer > Cc: erlang-questions@REDACTED > Subject: Re: creating a target system > > > > A nice HowTo on this topic would be nice! > Any takers ? You can upload it at: > > http://www.trapexit.org/docs/howto/ > > Cheers, Tobbe > > > Dietmar Schaefer wrote: > > > Hi ! > > > > > > I am trying to build a target system. > > > > Here is my cmmc.rel file > > > > > > > > > > {release, > > {"4dp-cmmc", "1.0"}, > > { erts, > > [kernel, > > stdlib, > > sasl, > > mnesia, > > snmp, > > cmmc-foc]}. > > > > > > calling target_system:create("cmmc"). > > > > > > I get: > > > > ** exited: {undef,[{sys_pre_expand,module, > > [[{function, > > 1, > > foo, > > 0, > > [{clause, > > 1, > > [], > > [], > > [{call, > > 1, > > {remote,1|...}, > > [{...}]}]}]}], > > []]}, > > {shell,expand_records,2}, > > {shell,exprs,6}, > > {shell,eval_loop,3}]} ** > > > > > > Using the mysystem.rel file from "System Principles" I get > the same ! > > > > > > BTW: I am using R10B2 on FC2 > > > > > > > > Regards > > > > > > Dietmar > > From joe.armstrong@REDACTED Thu Mar 3 13:46:21 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Thu, 3 Mar 2005 13:46:21 +0100 Subject: Call for papers Message-ID: Call For Papers ~~~~~~~~~~~~~~~ Fourth ACM SIGPLAN Erlang Workshop Tallin, Estonia, September 25, 2005-03-01 Satellite event of ACM SIGPLAN International Conference of Functional programming, September 26-28, 2005. See http://www.erlang.se/workshop/2005/ Important dates and submission details ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Submission 6 June 2005 Notification 17 June 2005 Final 15 July 2005 Submission Length 6 - 12 pages in ACM conference format Workshop date 25 September 2005 Workshop Chair: Kostis Sagonas, Uppsala University, Sweden Program Chair: Joe Armstrong. Ericsson AB. Program Committee: To be announced. The workshop will have printed proceedings which will be distributed to the participants at the time of the workshop. Subsequently, the papers will be included in the ACM Digital Library. Authors are therefore advised to use the ACM SIGPLAN style file for conference proceedings when preparing their submissions. Authors should submit papers in postscript or PDF, formatted for A4 paper, to Joe Armstrong(joe.armstrong@REDACTED). The length should be restricted to 12 pages in standard ACM format (see http://www.cs.indiana.edu/icfp04/acm.html) Joe Armstrong Program Chair From heinrich@REDACTED Thu Mar 3 14:29:40 2005 From: heinrich@REDACTED (Heinrich Venter) Date: Thu, 3 Mar 2005 15:29:40 +0200 Subject: Snmpa_local_db example? Message-ID: Hi Does any one have an example of how to use snmpa_local_db that works? Is any one actually using it? I am either not using it correctly, or it is fundamentally flawed. I would prefer to know which before I attempt to correct the library :) Any help? -]-[einrich Get LookForMe for business using CellFind http://asp1.rocketseed.com/RocketSeed/mail/433a32353a313131303032373a33303639373a2d323a323138 This will take you to the Launchpad site for more info http://asp1.rocketseed.com/RocketSeed/mail/433a32353a313131303032373a33303639373a2d323a333738 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: MTk2LjMwLjc5LjE1NQ== Type: image/gif Size: 14964 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: MTk2LjMwLjc5LjE1NQ== Type: image/gif Size: 620 bytes Desc: not available URL: From hasse@REDACTED Thu Mar 3 15:28:08 2005 From: hasse@REDACTED (hasse@REDACTED) Date: Thu, 3 Mar 2005 15:28:08 +0100 Subject: creating a target system In-Reply-To: <4226DD94.3030804@ast.dfs.de> References: <4226DD94.3030804@ast.dfs.de> Message-ID: <16935.7928.518106.582517@gargle.gargle.HOWL> [Dietmar Schaefer:] > I am trying to build a target system. > > Here is my cmmc.rel file > > {release, > {"4dp-cmmc", "1.0"}, > { erts, > [kernel, > stdlib, > sasl, > mnesia, > snmp, > cmmc-foc]}. > > calling target_system:create("cmmc"). > > I get: > > ** exited: {undef,[{sys_pre_expand,module, > [[{function, > 1, > foo, > 0, > [{clause, > 1, > [], > [], > [{call, > 1, > {remote,1|...}, > [{...}]}]}]}], > []]}, > {shell,expand_records,2}, > {shell,exprs,6}, > {shell,eval_loop,3}]} ** The shell shouldn't try to expand records unless sys_pre_expand can be loaded. This will be fixed in next release of R10B. Thanks. Best regards, Hans Bolinder, Erlang/OTP From ulf.wiger@REDACTED Thu Mar 3 19:03:03 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Thu, 3 Mar 2005 19:03:03 +0100 Subject: start phases Message-ID: I decided to implement a prototype hack of the application_controller. The idea is that we should be able to extend the start phase concept significantly, and get cleaner (and perhaps faster) execution of complex system start sequences. While you can still specify dependencies in the 'applications' attribute as before, you can now also specify dependencies for each start phase, like so: {start_phases, [{Phase, Arg, [{run_before, [app() | phase()]} | {run_after, [app() | phase()]}]}]} The old way of specifying start_phases still works, and means no funny dependencies. I imagine that one would make start phase specifications part of the application API, basically stating "after this phase, the following resources are available", etc. The Application Controller will start applications and run start phases concurrently as much as possible.(*) To me, this is a fairly logical extension of the idea of application dependencies. Since it's declarative, one could write a tool that reasons about the most efficient order in which to start things, in order to achieve the optimal level of concurrency, while maintaining the safety guarantee. One could also analyse dependencies off-line, visualize them, and warn about loops, to a much finer degree than today. The whole thing is essentially a wild spin-off from AXD 301's start phase support. As far as I know, the patch is backwards compatible, but applying the patch on an OTP R10B-2 was difficult to say the least, since it required some changes in the boot script. This means you have to apply some of the patches, re-build the boot scripts by hand, and then do some copying by hand, before you can apply the last patches and run the system. (**) I built it, but don't really know what to do with it now. Does anyone feel like testing it? If so, let me know. /Uffe (*) One could also speculate that instead of the 'risky_shell' flag, one might specify after which start phase(s) the shell should be made available. The shell process could then simply call application_coordinator:await_start_phase(App, Phase), which returns 'ok' when all is set. (**) This is OT, but the make script uses a pre-built script, which was made using a special emulator. It is not re-built - ever! It's just copied into the final build as the default script. From peter@REDACTED Thu Mar 3 19:06:07 2005 From: peter@REDACTED (Peter H|gfeldt) Date: Thu, 3 Mar 2005 19:06:07 +0100 (MET) Subject: creating a target system In-Reply-To: <4226DD94.3030804@ast.dfs.de> Message-ID: Hi, as Hasse pointed out the problem has to do with the shell, which uses the sys_pre_expand module in the compiler application regardless of that modules availability (a bug). If you add `compiler' to your .rel file and start all over again, I am pretty sure it will work. /Peter On Thu, 3 Mar 2005, Dietmar Schaefer wrote: > Hi ! > > > I am trying to build a target system. > > Here is my cmmc.rel file > > > > > {release, > {"4dp-cmmc", "1.0"}, > { erts, > [kernel, > stdlib, > sasl, > mnesia, > snmp, > cmmc-foc]}. > > > calling target_system:create("cmmc"). > > > I get: > > ** exited: {undef,[{sys_pre_expand,module, > [[{function, > 1, > foo, > 0, > [{clause, > 1, > [], > [], > [{call, > 1, > {remote,1|...}, > [{...}]}]}]}], > []]}, > {shell,expand_records,2}, > {shell,exprs,6}, > {shell,eval_loop,3}]} ** > > > Using the mysystem.rel file from "System Principles" I get the same ! > > > BTW: I am using R10B2 on FC2 > > > > Regards > > > Dietmar > From taavi@REDACTED Thu Mar 3 19:53:34 2005 From: taavi@REDACTED (Taavi Talvik) Date: Thu, 3 Mar 2005 20:53:34 +0200 (EET) Subject: start phases In-Reply-To: References: Message-ID: <20050303204116.R879@m4.uninet.ee> On Thu, 3 Mar 2005, Ulf Wiger (AL/EAB) wrote: > The idea is that we should be able to extend the start phase concept > significantly, and get cleaner (and perhaps faster) execution of complex > system start sequences. > > While you can still specify dependencies in the 'applications' attribute > as before, you can now also specify dependencies for each start phase, > like so: > > {start_phases, [{Phase, Arg, [{run_before, [app() | phase()]} | > {run_after, [app() | phase()]}]}]} Look at FreeBSD(NetBSD?) startup scripts. They are shell scripts, which have keywords in first fiew lines. On the startup keywords are extracted and topologically sorted. Then they are executed in appropriate order. Example: #!/bin/sh # # REQUIRE: networking syslog # REQUIRE: usr # BEFORE: LOGIN # PROVIDE: dns nscd # KEYWORD: FreeBSD man rcorder(8) rc(8) etc. Just having run_before/run_after are probably not enough. best regards, taavi From raimo@REDACTED Thu Mar 3 09:14:10 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 03 Mar 2005 09:14:10 +0100 Subject: size of an IO list References: , Message-ID: lists:flatlength/1 works on a deep list of numbers, but not if the list contains binaries, nor if the list or one of its sublists is an improper list with a binary at the tail, as allowed for I/O lists. luna@REDACTED (Daniel Luna) writes: > On Wed, 2 Mar 2005, Joe Armstrong (AL/EAB) wrote: > > How about a BIF to compute the size of an I/O list? > > > > Why? > > > > I have made a deep list that I want to output (guess what's it's HTML :-) > > > > I need to make an HTTP header element "Content-Length: ..." > > > > Which is the size of the flattened deep list , but I don't want to flatten it (horrors) > > or traverse it (in Erlang) to find the length. > > > > Suitable case for a BIF? > > There is lists:flatlength/1 that does this, unfortunately it's not a BIF. > > /Luna > -- > Daniel Luna | Top reasons that I have a beard: > luna@REDACTED | a) Laziness. > http://www.update.uu.se/~luna/ | b) I can. > Don't look at my homepage (it stinks).| c) I can get away with it. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From sean.hinde@REDACTED Fri Mar 4 00:01:21 2005 From: sean.hinde@REDACTED (Sean Hinde) Date: Thu, 3 Mar 2005 23:01:21 +0000 Subject: size of an IO list In-Reply-To: References: Message-ID: <88933fb3ca3630b3813b0c9e95fd06eb@mac.com> I have also wished for this many times. I often find myself going to extraordinary lengths to add up the total length in the same pass as creating some complex io_list. This makes for horrible code. asn.1 encoding would also benefit greatly. So Yes Please! (from a paying customer :-) ) Sean On 3 Mar 2005, at 08:14, Raimo Niskanen wrote: > lists:flatlength/1 works on a deep list of numbers, but not if the > list contains binaries, nor if the list or one of its sublists > is an improper list with a binary at the tail, as allowed for > I/O lists. > > > luna@REDACTED (Daniel Luna) writes: > >> On Wed, 2 Mar 2005, Joe Armstrong (AL/EAB) wrote: >>> How about a BIF to compute the size of an I/O list? >>> >>> Why? >>> >>> I have made a deep list that I want to output (guess what's it's >>> HTML :-) >>> >>> I need to make an HTTP header element "Content-Length: ..." >>> >>> Which is the size of the flattened deep list , but I don't want to >>> flatten it (horrors) >>> or traverse it (in Erlang) to find the length. >>> >>> Suitable case for a BIF? >> >> There is lists:flatlength/1 that does this, unfortunately it's not a >> BIF. >> >> /Luna >> -- >> Daniel Luna | Top reasons that I have a >> beard: >> luna@REDACTED | a) Laziness. >> http://www.update.uu.se/~luna/ | b) I can. >> Don't look at my homepage (it stinks).| c) I can get away with it. > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB From alexey@REDACTED Fri Mar 4 00:15:03 2005 From: alexey@REDACTED (Alexey Shchepin) Date: Fri, 04 Mar 2005 01:15:03 +0200 Subject: kpoll and epoll support in erlang Message-ID: <87y8d45nk8.fsf@alex.sevcom.net> Hi! Some time ago there was a patch[1] that implements epoll support for linux. It seems it is applied now, but by some reason epoll support is replaced with kpoll support (except one comment in erts/emulator/sys/unix/sys.h). Now linux-2.6 kernels have epoll by default and no kpoll. Is there any plans to return epoll support in erlang? [1] http://www.erlang.org/ml-archive/erlang-questions/200202/msg00134.html From tobbe@REDACTED Fri Mar 4 00:55:10 2005 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: Fri, 04 Mar 2005 00:55:10 +0100 Subject: creating a target system In-Reply-To: <4226DD94.3030804@ast.dfs.de> References: <4226DD94.3030804@ast.dfs.de> Message-ID: <4227A3DE.9090301@nortelnetworks.com> Dietmar Schaefer wrote: > Hi ! > > > I am trying to build a target system. > > Ok, you can now find a first version of a "OTP Release Handling Tutorial" written by Ulf Wiger at: http://www.trapexit.org/docs/howto/release_handling_tutorial.html Cheers, Tobbe From bengt.kleberg@REDACTED Fri Mar 4 08:21:39 2005 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 04 Mar 2005 08:21:39 +0100 Subject: the problem of using _Var instead of _ Message-ID: <42280C83.602@ericsson.com> greetings, i have been tought(sp?) that it is better to use _Var than _ when ignoring parts of a pattern match. the reason beeing that i get a small amount of documentation, but still reap the benefits of ignorance :-) the advice is sound. i have used it for four years without problem. then today i had (equivilent of) the following pattern in a function head: fun( [H1|_T], [H2|_T] ) -> {H1, H2}. and it crashed with {function_clause,...}. however this works: fun( [H1|_], [H2|_] ) -> {H1, H2}. so i would like to mention that _Var is almost, but not entirely, totally unlike _ . as an aside: i thought of putting this somewhere on the trapexit forums (to avoid sending senseless ramblings to the mail list). however, there is no ''common mistakes'' or ''gotchas'' in the forum. perhaps there should be? bengt From tpatro@REDACTED Fri Mar 4 08:38:54 2005 From: tpatro@REDACTED (Tamas Patrovics) Date: Fri, 4 Mar 2005 08:38:54 +0100 Subject: Erlang documentation sources Message-ID: Hi, I'm looking for the documentation sources from which the erlang documentation (man, html) is generated? Are they available somewhere? They don't seem to be generated from the code, so I suppose there is a documentation source (iSGML? XML?) which is used to generate documentation in various format.. I went to the Erlang download page, but I could only find the html and man versions there. Thanks in advance, /Tamas From vlad_dumitrescu@REDACTED Fri Mar 4 08:46:24 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Fri, 4 Mar 2005 08:46:24 +0100 Subject: the problem of using _Var instead of _ References: <42280C83.602@ericsson.com> Message-ID: From: "Bengt Kleberg" > fun( [H1|_T], [H2|_T] ) -> > {H1, H2}. > and it crashed with {function_clause,...}. however this works: Hi, Could it be that you really meant to write fun( [H1|_T1], [H2|_T2] ) -> {H1, H2}. ? Using _ matches different tails, but using _T only matches if the tails are identical. regards, Vlad From bengt.kleberg@REDACTED Fri Mar 4 08:53:34 2005 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 04 Mar 2005 08:53:34 +0100 Subject: the problem of using _Var instead of _ In-Reply-To: References: Message-ID: <422813FE.7080402@ericsson.com> Vlad Dumitrescu wrote: ...deleted > > Using _ matches different tails, but using _T only matches if the tails are > identical. yes, there is a difference between _ and _. that was the observation i tried to convey. the design guidelines that mandate _Var instead of _ failed to mention this. so i have operated under the assumption that _ and _ are the same for over four years now. bengt From mccratch@REDACTED Fri Mar 4 09:16:45 2005 From: mccratch@REDACTED (Matthias Kretschmer) Date: Fri, 4 Mar 2005 09:16:45 +0100 Subject: an erlang monad? In-Reply-To: References: <20050302182547.47c5ee3a@localhost.localdomain> Message-ID: <20050304091645.75a9d47b@localhost.localdomain> On Thu, 3 Mar 2005 08:41:42 +0100 "Vlad Dumitrescu" wrote: > Hi, > > Just a thought about this. > > It is important to be able to access the state even somewhere in-between the > sequence, and it should be easy to do so too. > > The reasons are two: > * one may need this state later on (I almost never have more than 3 levels > where the intermediary state isn't needed) > * easily inserting (and removing) debugging traces is very useful for finding > out what went wrong. > > regards, > Vlad that's very easy with my monad implementation, but of course it has drawbacks: one always has to give around the monad-type (as with Haskell, where everything is packed into IO a, ST a or whatever). You can simply read the state anywhere where you are generating monadic values. This makes it very simple: just read the state, print it to screen and you're, done. Just another thought: Another way to makes this nicer would be to use other combinators that make life easier. E.g. something like (forgive the pseudo-code): lift(Fun, [Action0, Action1, ...]) --> apply(Fun, [X0, X1, ...] <- (X0 <- Action0 >>= X1 <- Action1 >>- ...)) where Action_n is of course a monad-value. This would make it easier to write good looking code without using a parse_transform'er. The drawback is, that one is loosing some power if one is only using this lift function (this would be more a arrow-style combinator), because Action_n doesn't know of the outcome of Action_1 to Action_{n-1} and can't decide what to do depending on their outcome. -- Matthias Kretschmer From tobbe@REDACTED Fri Mar 4 09:28:39 2005 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: Fri, 04 Mar 2005 09:28:39 +0100 Subject: creating a target system In-Reply-To: <20050304081801.6cc6617d.pmander@newport-networks.com> References: <4226DD94.3030804@ast.dfs.de> <4227A3DE.9090301@nortelnetworks.com> <20050304081801.6cc6617d.pmander@newport-networks.com> Message-ID: <42281C37.80400@nortel.com> Peter-Henry Mander wrote: >Hi Tobbe, > >I know this is a small detail, but I think the page title ought to refer >to "Erlang" and not "Gentoo Linux" (-: > > Thanx for pointing that out. It is fixed now. Cheers, Tobbe From vlad_dumitrescu@REDACTED Fri Mar 4 10:01:09 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Fri, 4 Mar 2005 10:01:09 +0100 Subject: an erlang monad? References: <20050302182547.47c5ee3a@localhost.localdomain> <20050304091645.75a9d47b@localhost.localdomain> Message-ID: From: "Matthias Kretschmer" > > It is important to be able to access the state even somewhere in-between the > > sequence, and it should be easy to do so too. > that's very easy with my monad implementation, but of course it has drawbacks: one always has to give around the monad-type (as with Haskell, where everything is packed into IO a, ST a or whatever). You can simply read the state anywhere where you are generating monadic values. This makes it very simple: just read the state, print it to screen and you're, done. Yes, I agree. The other solution I was thinking about and that don't fullfill this requirement (but is much easier to read) is simply using a call chain: fun_1(...(fun_n(State))...) > Another way to makes this nicer would be to use other combinators that make life easier. E.g. something like (forgive the pseudo-code): > > lift(Fun, [Action0, Action1, ...]) --> > apply(Fun, [X0, X1, ...] <- (X0 <- Action0 >>= X1 <- Action1 >>- ...)) Please forgive my ignorance, but do you really think the above is easier to read and less error-prone than the "hated" A1=fun_1(Start), A2=fun_2(A1), ... A_n=fun_n(A_k), ? Yes, it's easy to mix up the variable names, but the code flow is cleaner, uncluttered by plenty of 'fun()->...end's. If I get to mre than three chained calls, I usually switch to using more meaningful names than A1, A2 - which takes slightly more time (for thinking of a good name and typing it as it's longer than 2 chars), but I feel it saves debugging time. [*] Also, in my code it happens often that I need to do pattern matching - how would it be accounted for in such a monadic style? regards, Vlad [*] I don't mean to say that I didn't ever mixed up my variables, or that it won't happen again! :-) From ulf@REDACTED Fri Mar 4 10:25:29 2005 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 04 Mar 2005 10:25:29 +0100 Subject: creating a target system In-Reply-To: <4227A3DE.9090301@nortelnetworks.com> References: <4226DD94.3030804@ast.dfs.de> <4227A3DE.9090301@nortelnetworks.com> Message-ID: Den 2005-03-04 00:55:10 skrev Torbjorn Tornkvist : > Ok, you can now find a first version of a "OTP Release Handling Tutorial" > written by Ulf Wiger at: > > http://www.trapexit.org/docs/howto/release_handling_tutorial.html > > Cheers, Tobbe One thing that should be added to the tutorial is that the file example.rel contains hard-coded versions of kernel, stdlib and sasl. Depending on your version of OTP, the building of the boot script may fail. Fortunately, the error message is quite helpful in showing what needs to be changed. Another important detail is that the sys.config file contains hard-coded node names. You need to make sure that the node names are correct in your environment for the example to work. /Uffe From dietmar@REDACTED Fri Mar 4 11:09:53 2005 From: dietmar@REDACTED (Dietmar Schaefer) Date: Fri, 04 Mar 2005 11:09:53 +0100 Subject: mnesia creation of database failed Message-ID: <422833F1.1050504@ast.dfs.de> Hi ! Most recently I changed my directory layout ( The way how I origanize source,bin, etc) When I now try to create my database tables I get: creation of database scalarVariables failed reason.{bad_type, scalarVariables, {disc_copies,cmmc@REDACTED}} I am quite shure I dod not change anything else ! (?) Any hints ? Dietmar From ulf@REDACTED Fri Mar 4 11:52:17 2005 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 04 Mar 2005 11:52:17 +0100 Subject: mnesia creation of database failed In-Reply-To: <422833F1.1050504@ast.dfs.de> References: <422833F1.1050504@ast.dfs.de> Message-ID: Den 2005-03-04 11:09:53 skrev Dietmar Schaefer : > Hi ! > > > Most recently I changed my directory layout ( The way how I origanize > source,bin, etc) > > When I now try to create my database tables I get: > > creation of database scalarVariables failed reason.{bad_type, > scalarVariables, > > {disc_copies,cmmc@REDACTED}} > > I am quite shure I dod not change anything else ! (?) I'm not quite sure if it isn't just the format of the error message, but the reason could be that the value of disc_copies should be a list of node names, not just a single node. /Uffe From thomasl_erlang@REDACTED Fri Mar 4 11:57:24 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Fri, 4 Mar 2005 02:57:24 -0800 (PST) Subject: start phases In-Reply-To: 6667 Message-ID: <20050304105724.45487.qmail@web41906.mail.yahoo.com> --- "Ulf Wiger (AL/EAB)" wrote: > While you can still specify dependencies in the > 'applications' attribute > as before, you can now also specify dependencies for > each start phase, > like so: /.../ This sounds like a good idea. It subsumes a number of ad hoc hacks (mnesia:wait_for_tables, anyone?) and may eventually get rid of a number of existing race conditions in OTP as well: the shell, as mentioned, as well as orber, apparently, and perhaps more than that. That is, today you can't rely on an application being fully started (and its API functions working) even when application:start(App) has returned. Likewise for failover, etc. This can be annoying. Cellpoint used (and their later incarnation perhaps still uses) an approach similar to what you propose, but entirely separate from the applications/releases/... framework. Each subsystem depended on a collection of others, did not advertise itself until they all were available, and de-advertised when and if one disappeared. This worked fairly well. I think Per Bergqvist and Adam Aquilon were the guys behind it. The drawback, as I recall, is that it can be difficult to see why the system refuses to start. Some sort of inspection tool, as you suggest, may be necessary. Best, Thomas __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From mccratch@REDACTED Fri Mar 4 12:12:22 2005 From: mccratch@REDACTED (Matthias Kretschmer) Date: Fri, 4 Mar 2005 12:12:22 +0100 Subject: an erlang monad? In-Reply-To: References: <20050302182547.47c5ee3a@localhost.localdomain> <20050304091645.75a9d47b@localhost.localdomain> Message-ID: <20050304121222.1c70651a@localhost.localdomain> On Fri, 4 Mar 2005 10:01:09 +0100 "Vlad Dumitrescu" wrote: > > Another way to makes this nicer would be to use other combinators that make > life easier. E.g. something like (forgive the pseudo-code): > > > > lift(Fun, [Action0, Action1, ...]) --> > > apply(Fun, [X0, X1, ...] <- (X0 <- Action0 >>= X1 <- Action1 >>- ...)) > > Please forgive my ignorance, but do you really think the above is easier to read > and less error-prone than the "hated" > A1=fun_1(Start), > A2=fun_2(A1), > ... > A_n=fun_n(A_k), > ? Yes I think so, because I don't have to care about the state and state naming. Remember, my X_n's are not the state, they are just values returned by the monad-values while using/changing the state. E.g. if in one step you want to read a value from you dictionary. So if I have a chain of dictionary changing functions which don't return anything I want to bother about, it would be looking like: lift(fun (_, _, _, _, ...) -> ok end, [Action0, Action1, Action2, ...]). you could even reduce it to some lift/1, if you don't care about the possible values carried by the actions and you are just interested in the state itself. In my humble opinion it is much nicer, because I don't have to care about the state directly as in the sequence given above by you. > > Yes, it's easy to mix up the variable names, but the code flow is cleaner, > uncluttered by plenty of 'fun()->...end's. If I get to mre than three chained > calls, I usually switch to using more meaningful names than A1, A2 - which takes > slightly more time (for thinking of a good name and typing it as it's longer > than 2 chars), but I feel it saves debugging time. [*] > > Also, in my code it happens often that I need to do pattern matching - how would > it be accounted for in such a monadic style? well in fact using my transformer it is quite easy, simply putting a case statement in the chain. The transformer is very simple and you have to introduce a new monad-statement in each case clause or call a function returning a monad value, so it is not as nice as Haskell's do in that concern, but it is usable. -- cheer Matthias Kretschmer From ulf@REDACTED Fri Mar 4 12:17:50 2005 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 04 Mar 2005 12:17:50 +0100 Subject: start phases In-Reply-To: <20050303204116.R879@m4.uninet.ee> References: <20050303204116.R879@m4.uninet.ee> Message-ID: Since the dependencies are specified in a key-value list, one could of course add other options. One such option could be to await a globally (or locally) registered name. But I'm not actually convinced that it adds much. The start phases are quite powerful, in that they map to a function, and you can have as many as you want in each application (well...;) One thought was that it could reduce the need to have included applications. We've found that while creating an applciation hierarchy allows you to synchronize a lot of complex activities during startup (and also simplifies takeover), they're a pain to manage, and it's very difficult to add included applications to an already released application. Introducing start phase dependencies could allow us to build much more modular and extensible systems. As an extension of the work, I think the application_controller should be completely re-written (sorry Martin ;). The whole permit_app() thing should go away entirely (does anyone actually use that?) It severely clutters up the code and could be accomplished by hooking into the message passing API to the application_controller (which came much later.) In general, the AC implements several different functions in the same process, which makes it difficult to understand and modify. /Uffe Den 2005-03-03 19:53:34 skrev Taavi Talvik : > On Thu, 3 Mar 2005, Ulf Wiger (AL/EAB) wrote: > >> The idea is that we should be able to extend the start phase concept >> significantly, and get cleaner (and perhaps faster) execution of complex >> system start sequences. >> >> While you can still specify dependencies in the 'applications' attribute >> as before, you can now also specify dependencies for each start phase, >> like so: >> >> {start_phases, [{Phase, Arg, [{run_before, [app() | phase()]} | >> {run_after, [app() | phase()]}]}]} > > Look at FreeBSD(NetBSD?) startup scripts. They are shell scripts, > which have keywords in first fiew lines. On the startup keywords > are extracted and topologically sorted. Then they are executed in > appropriate order. > > Example: > #!/bin/sh > # > > # REQUIRE: networking syslog > # REQUIRE: usr > # BEFORE: LOGIN > # PROVIDE: dns nscd > # KEYWORD: FreeBSD > > man rcorder(8) rc(8) etc. > > Just having run_before/run_after are probably not enough. > > best regards, > taavi > > -- Anv?nder Operas banbrytande e-postklient: http://www.opera.com/m2/ From ulf@REDACTED Fri Mar 4 12:39:45 2005 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 04 Mar 2005 12:39:45 +0100 Subject: start phases In-Reply-To: <20050304105724.45487.qmail@web41906.mail.yahoo.com> References: <20050304105724.45487.qmail@web41906.mail.yahoo.com> Message-ID: Den 2005-03-04 11:57:24 skrev Thomas Lindgren : > The drawback, as I recall, is that it can be difficult > to see why the system refuses to start. Some sort of > inspection tool, as you suggest, may be necessary. I found while debugging my prototype that it was fairly easy to figure out who was waiting for what by just looking at the dictionary of my application_coordinator. It was sometimes more difficult to debug errors due to functions being called too early (partly because if e.g. c:erlangrc() kicks in before the file driver is ready, you most likely don't have the I/O subsystem ready either. In later phases, it should be easier.) Another recent hack was a local registry that allowed for registration of non-unique keys. Essentially, each process could register any number of unique keys (any erlang term) and any number of non-unique ones. In addition, you could fold over the process registry based on 'select' criteria (using the ets:select() syntax) and apply some function for each key. One idea was that you could publish services this way, perhaps parameterized (e.g. {megaco, LinkId}). Another idea was that you could use it to group processes in different categories -- essentially, it becomes a set indexes through which one can locate functions in the system. The final idea was to come up with a way to inspect and debug processes in huge systems, with several hundred thousand processes. This seemed to work excellently. I'm not sure you have to tie these two ideas together, but I think they both help to simplify specification and inspection of large systems. /Uffe From mats.cronqvist@REDACTED Fri Mar 4 13:24:38 2005 From: mats.cronqvist@REDACTED (Mats Cronqvist) Date: Fri, 04 Mar 2005 13:24:38 +0100 Subject: the problem of using _Var instead of _ In-Reply-To: <422813FE.7080402@ericsson.com> References: <422813FE.7080402@ericsson.com> Message-ID: <42285386.3050107@ericsson.com> ifaik, the only difference between _X and X is that the "unused variable" warning is not issued for _X. imho, the whole _Var thing is a kludge that should be avoided. mats Bengt Kleberg (AL/EAB) wrote: > Vlad Dumitrescu wrote: > ...deleted > >>Using _ matches different tails, but using _T only matches if the tails are >>identical. > > > yes, there is a difference between _ and _. that was the > observation i tried to convey. > > the design guidelines that mandate _Var instead of _ failed to mention > this. so i have operated under the assumption that _ and _ are the > same for over four years now. > > > bengt From valentin@REDACTED Fri Mar 4 14:24:37 2005 From: valentin@REDACTED (Valentin Micic) Date: Fri, 4 Mar 2005 15:24:37 +0200 Subject: mnesia creation of database failed References: <422833F1.1050504@ast.dfs.de> Message-ID: <015c01c520bd$93337ce0$0100a8c0@MONEYMAKER2> Or... trying to create table in ram_only schema? V. ----- Original Message ----- From: "Ulf Wiger" To: Sent: Friday, March 04, 2005 12:52 PM Subject: Re: mnesia creation of database failed Den 2005-03-04 11:09:53 skrev Dietmar Schaefer : > Hi ! > > > Most recently I changed my directory layout ( The way how I origanize > source,bin, etc) > > When I now try to create my database tables I get: > > creation of database scalarVariables failed reason.{bad_type, > scalarVariables, > > {disc_copies,cmmc@REDACTED}} > > I am quite shure I dod not change anything else ! (?) I'm not quite sure if it isn't just the format of the error message, but the reason could be that the value of disc_copies should be a list of node names, not just a single node. /Uffe From olgeni@REDACTED Sat Mar 5 17:14:54 2005 From: olgeni@REDACTED (Jimmy Olgeni) Date: Sat, 5 Mar 2005 17:14:54 +0100 (CET) Subject: distel and R10 Message-ID: <20050305171419.E63657@olgeni.olgeni> Hi, Is anybody working on distel compatibility with R10? :-) -- jimmy From luke@REDACTED Sat Mar 5 17:42:14 2005 From: luke@REDACTED (Luke Gorrie) Date: 05 Mar 2005 17:42:14 +0100 Subject: distel and R10 References: <20050305171419.E63657@olgeni.olgeni> Message-ID: Jimmy Olgeni writes: > Hi, > > Is anybody working on distel compatibility with R10? :-) I moved Distel into the Jungerl a little ways back and the version in there should be R10 compatible I think? If not then please let me know, I'm stuck in R9-land! From ncharpentier@REDACTED Sat Mar 5 20:03:37 2005 From: ncharpentier@REDACTED (Nicolas Charpentier) Date: Sat, 05 Mar 2005 20:03:37 +0100 Subject: Sound/video drivers In-Reply-To: References: Message-ID: <422A0289.9080409@free.fr> Joe Armstrong (AL/EAB) wrote: > > Has anybody some Erlang code that does all of this? > Hi, I don't have all of this, but I have a driver to portaudio, a portable audio library. The website of my project is http://ncharpentier.free.fr/ErlPhone . As the site is, for now, a french only site, you can find sources here http://ncharpentier.free.fr/ErlPhone/third/erlPhone_src.tgz . Your comments are welcome, Regards, Nicolas Charpentier From vlad_dumitrescu@REDACTED Sat Mar 5 21:27:04 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Sat, 5 Mar 2005 21:27:04 +0100 Subject: Poll: Packages use Message-ID: Hi, While trying to figure out a good model for Erlang projects, for use with the Eclipse plugin, I thought it would be interesting to find out the answer to the following questions: * How many people use packages in their applications? it would also be interesting to know if anyone uses them in a commercial setting. * How many people plan to start using packages in 6 months to one year from now? Also a related, more general question: * Should the use of packages become widespread, would it be useful to adopt some naming scheme so that package name clashes are avoided? Thank you. Best regards, Vlad From maruthavanan_s@REDACTED Sun Mar 6 14:04:18 2005 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Sun, 06 Mar 2005 13:04:18 +0000 Subject: memory leakage problem in erl_interface Message-ID: Hi, I have built a c exe with erl interface. When I send more messages from erlang to c or c to erlang my exe keeps on consuming memory drastically, which leads to memory leakage problem. Here are my codes. How to reduce my memory usage. Please help! I am in hurry int main(int argc, char **argv) { int id_number = 1; int creation=1,msg_from_enode(ETERM *); char *cookie="secretcookie"; char *nodename="enode@REDACTED"; int loop = 1; /* Loop flag */ int got; /* Result of receive */ unsigned char buf[BUFSIZE]; /* Buffer for incoming message */ ErlMessage emsg; /* Incoming message */ HANDLE mutex=NULL; ETERM *fromp, *tuplep, *fnp, *argtuple; erl_init(NULL, 0); if( erl_connect_init(id_number, cookie, (short)creation) == -1) erl_err_quit("erl_connect_init"); if ((sockfd = erl_connect(nodename)) < 0) erl_err_quit("ERROR: erl_connect failed"); printf("Connection established...\n"); erl_eterm_release(); while (loop) { memset(&emsg,0,sizeof(emsg)); got = erl_receive_msg(sockfd, buf, BUFSIZE, &emsg); if (got == ERL_TICK) { } else if (got == ERL_ERROR) { loop = 0; } else { switch (emsg.type) { case ERL_REG_SEND: mutex=CreateMutex(NULL,FALSE,"erlang3"); FileIP=fopen("input.txt","a"); WaitForSingleObject(mutex,INFINITE); erl_print_term(FileIP,emsg.msg); fclose(FileIP); fromp = erl_element(2, emsg.msg); tuplep = erl_element(3, emsg.msg); fnp = erl_element(1, tuplep); argtuple = erl_element(2, tuplep); if (strncmp(ERL_ATOM_PTR(fnp), "enode", 5) == 0) { int i=0,tl; tl = ERL_TUPLE_SIZE(argtuple); msg_from_enode(argtuple); } erl_free_compound(emsg.from); erl_free_compound(emsg.msg); erl_free_compound(fromp); erl_free_compound(tuplep); erl_free_compound(fnp); erl_free_compound(argtuple); erl_eterm_release(); ReleaseMutex(mutex); CloseHandle(mutex); break; case ERL_EXIT: break; } } } printf("ERL Got error"); getch(); } after receiving the message from C we are calling some functions in C void msg_from_enode(ETERM *tuplep) { function_number=ERL_INT_VALUE(erl_element(1,tuplep)); /*based up on the function number we are parsing the parameters from the tuple and we are calling the functions in C*/ erl_free_compound(tuplep); //printf("Compound Released\n"); } For sending message from C to erlang we use the following code: void send_3to_enode(int e1,int e2,int e3) { ETERM *t1; int Result; DWORD result=0; HANDLE mutex=NULL; mutex=CreateMutex(NULL,FALSE,"erlang3"); result=WaitForSingleObject(mutex,INFINITE); t1=erl_format("{~i,~i,~i}",e1,e2,e3); File3P=fopen("file3.txt","a"); erl_print_term(File3P,t1); Result=erl_reg_send(sockfd, "enodereceiver", t1); erl_free_compound(t1); erl_eterm_release(); ReleaseMutex(mutex); CloseHandle(mutex); fclose(File3P); } thanks, Maruthavanan _________________________________________________________________ On the road to retirement? Check out MSN Life Events for advice on how to get there! http://lifeevents.msn.com/category.aspx?cid=Retirement From vlad_dumitrescu@REDACTED Sun Mar 6 19:46:27 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Sun, 6 Mar 2005 19:46:27 +0100 Subject: Poll: Packages use References: <422A2C23.9070906@mallard.com> Message-ID: ----- Original Message ----- From: "Thierry Mallard" >> * Should the use of packages become widespread, would it be useful to >> adopt some naming scheme so that package name clashes are avoided? > Something like Java naming convention ( org.erlang.package ) ? That was my first thought too, but it feels a little clumsy. > Another option would be to make something like a service-tree : > net.http.server.yaws > net.xmpp.server.ejabberd I think that might be a better idea. Not everything is a service, of course, and the Java structure does have the advantage that it makes the top levels unique. > BTW, great job for Erlide, even in its current state : It really > demonstrate the possibilities of an integrated multi-OS plateform, IMHO. Thank you for the kind words. So far, it's been nice to be able to concentrate on whatever I felt like for, but soon there will be long lists of bug reports ;-) best regards, Vlad From klacke@REDACTED Sun Mar 6 21:23:38 2005 From: klacke@REDACTED (klacke@REDACTED) Date: Sun, 6 Mar 2005 21:23:38 +0100 Subject: inets ssl proxy In-Reply-To: <20050303005923.GV3858@fangora.autosys.us> References: <20050223013404.GM14094@fangora.autosys.us> <16924.39226.611947.37660@gargle.gargle.HOWL> <20050224054129.GC14094@fangora.autosys.us> <20050303005923.GV3858@fangora.autosys.us> Message-ID: <20050306202338.GA19104@hyber.org> On Wed, Mar 02, 2005 at 04:59:23PM -0800, Michael McDaniel wrote: > Well, I have not yet been successful. I thought it might be > because I upgraded OpenSSL (which got installed in /usr/local) > but still had the default SuSE OpenSSL installed elsewhere. > I recompiled Erlang against OpenSSL 0.9.7e and still problems. > > Perhaps it is something else wrong with this machine as I also get > Yaws problems like > ----------------- > ERROR erlang code crashed: > File: /var/yaws/www/arg.yaws:59 > Reason: {function_clause,[{prim_inet,peername,[{sslsocket,5,<0.63.0>}]}, > {m2,out,1}, > {yaws_server,deliver_dyn_part,8}, > {yaws_server,aloop,3}, > {yaws_server,acceptor0,2}, > {proc_lib,init_p,5}]} > Req: {http_request,'GET',{abs_path,"/arg.yaws"},{1,1}} > ----------------- This was just a ridiculous bug with the arg.yaws code which didn't check weather it was an ssl socket or a regular socket. Fixed in CVS. As for your problems with the inets http client I can't help you .... /klacke -- Claes Wikstrom -- Caps lock is nowhere and http://www.hyber.org -- everything is under control From erlang@REDACTED Sun Mar 6 22:13:09 2005 From: erlang@REDACTED (Michael McDaniel) Date: Sun, 6 Mar 2005 13:13:09 -0800 Subject: inets ssl proxy In-Reply-To: <20050306202338.GA19104@hyber.org> References: <20050223013404.GM14094@fangora.autosys.us> <16924.39226.611947.37660@gargle.gargle.HOWL> <20050224054129.GC14094@fangora.autosys.us> <20050303005923.GV3858@fangora.autosys.us> <20050306202338.GA19104@hyber.org> Message-ID: <20050306211309.GQ3217@fangora.autosys.us> On Sun, Mar 06, 2005 at 09:23:38PM +0100, klacke@REDACTED wrote: > On Wed, Mar 02, 2005 at 04:59:23PM -0800, Michael McDaniel wrote: > > Well, I have not yet been successful. I thought it might be > > because I upgraded OpenSSL (which got installed in /usr/local) > > but still had the default SuSE OpenSSL installed elsewhere. > > I recompiled Erlang against OpenSSL 0.9.7e and still problems. > > > > Perhaps it is something else wrong with this machine as I also get > > Yaws problems like > > ----------------- > > ERROR erlang code crashed: > > File: /var/yaws/www/arg.yaws:59 > > Reason: {function_clause,[{prim_inet,peername,[{sslsocket,5,<0.63.0>}]}, > > {m2,out,1}, > > {yaws_server,deliver_dyn_part,8}, > > {yaws_server,aloop,3}, > > {yaws_server,acceptor0,2}, > > {proc_lib,init_p,5}]} > > Req: {http_request,'GET',{abs_path,"/arg.yaws"},{1,1}} > > ----------------- > > > This was just a ridiculous bug with the arg.yaws code > which didn't check weather it was an ssl socket or a regular > socket. Fixed in CVS. > > As for your problems with the inets http client > I can't help you .... > > > /klacke > > -- > Claes Wikstrom -- Caps lock is nowhere and > http://www.hyber.org -- everything is under control ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Thank you, I will get arg.yaws from CVS. I was told that the inets http client problem was a bug also. Fix to be sent to erlang-questions sometime. ~Michael From ok@REDACTED Sun Mar 6 22:57:56 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 7 Mar 2005 10:57:56 +1300 (NZDT) Subject: the problem of using _Var instead of _ Message-ID: <200503062157.j26Lvudv002327@atlas.otago.ac.nz> Bengt Kleberg wrote: i have operated under the assumption that _ and _ are the same for over four years now. The language description made it quite clear that this is not so. In ML, CAML, Haskell, Clean, Prolog, Mercury, and Erlang, in fact in _all_ the programming languages I've used which have pattern matching standardly built into their procedure-calling machinery, "anonymous variable" syntax gives you a new distinct variable each time you use it, while this is not true for any other kind of variable, no matter how spelled. Variable name New each Warn if Warn if matches regexp time? =1 occ? >1 occ? /_/ yes no no /_[a-zA-Z0-9_]+/ no no no[%] /[A-Z][a-zA-Z0-9_]*/ no yes no [%] It looks as though there _should_ be a warning. From matthias@REDACTED Mon Mar 7 08:26:38 2005 From: matthias@REDACTED (Matthias Lang) Date: Mon, 7 Mar 2005 08:26:38 +0100 Subject: the problem of using _Var instead of _ In-Reply-To: <200503062157.j26Lvudv002327@atlas.otago.ac.nz> References: <200503062157.j26Lvudv002327@atlas.otago.ac.nz> Message-ID: <16940.558.81435.579045@antilipe.corelatus.se> Bengt Kleberg wrote: > i have operated under the assumption that _ and _ are the > same for over four years now. As far as I can tell, the compiler treats _ as "don't give any warnings". Your problem (which has happened to me by accident a couple of times too) would be solved if the compiler inverted the warning behaviour compared to normal variables, i.e.: which appears only once in a clause: warning which appears multiple times in a clause: no warning _ which appears only once in a clause: no warning _ which appears multiple times in a clause: warning Matthias From bengt.kleberg@REDACTED Mon Mar 7 09:18:23 2005 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 07 Mar 2005 09:18:23 +0100 Subject: the problem of using _Var instead of _ In-Reply-To: <200503062157.j26Lvudv002327@atlas.otago.ac.nz> References: <200503062157.j26Lvudv002327@atlas.otago.ac.nz> Message-ID: <422C0E4F.20102@ericsson.com> Richard A. O'Keefe wrote: ...deleted > in _all_ the programming languages I've used which have pattern > matching standardly built into their procedure-calling machinery, > "anonymous variable" syntax gives you a new distinct variable > each time you use it, while this is not true for any other kind of > variable, no matter how spelled. my mistake was to think that all variables that start with '_' are anonymous variables. since i managed to write code for four years without realising my mistake, i thought it might be possible that others are also making this mistake. if so, it seemed to me to be for the best to explain the difference between variables starting with '_' that are one character long, and those that are longer. to avoid sending this (superflous) explanation to the mail list i wanted to know if this could not become part of a ''stupid mistakes to avoid'' page on trapexit.org . however, emails to adm@REDACTED bounce. the address is erronously expanded to jocke@REDACTED sorry for the confusion. bengt From bjorn@REDACTED Mon Mar 7 11:20:18 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 07 Mar 2005 11:20:18 +0100 Subject: size of an IO list In-Reply-To: References: Message-ID: As usual, the hardest part is usually coming up with a good name. After some discussion, Kenneth and I come up with the following suggestion for TWO new BIFs. iolist_size(IOList) iolist_to_binary(IOList) where iolist_to_binary/1 will work like this iolist_to_binary(IOList) -> size(list_to_binary([IOList])). but be much faster (and obviously not build the temporary binary). The reason for the new iolist_to_binary/1 BIF is that is not clear that iolist_size/1 is any way releated to list_to_binary/1. Comments? Other name suggestions? /Bjorn "Joe Armstrong \(AL/EAB\)" writes: > How about a BIF to compute the size of an I/O list? > > Why? > > I have made a deep list that I want to output (guess what's it's HTML :-) > > I need to make an HTTP header element "Content-Length: ..." > > Which is the size of the flattened deep list , but I don't want to flatten it (horrors) > or traverse it (in Erlang) to find the length. > > Suitable case for a BIF? > > /Joe > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bjorn@REDACTED Mon Mar 7 11:25:00 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 07 Mar 2005 11:25:00 +0100 Subject: Calling internal functions - foo::bar() ? In-Reply-To: References: <200503021440.j22Ee4dU019737@spikklubban.it.uu.se> Message-ID: After some discussion in the OTP group, we have decided NOT to implement any support for calling a local function. The reason is that it could prevent us from implementing (or drastically complicate) future optimizations in the compiler. (We haven't implemented the kind of optimization mentioned below, but we hope to do it in a future release.) /Bjorn James Hague writes: > Does the compiler not take advantage of functions being internal to a > module? For example, if a function always returns a tuple which is > immediately unpacked, then the tuple construction can be avoided and > the values returned in Erlang registers. At least I recall reading > about this somewhere. > > If this is the case, then you wouldn't be able to call internal > functions directly. > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From ingela@REDACTED Mon Mar 7 11:24:34 2005 From: ingela@REDACTED (Ingela Anderton) Date: Mon, 7 Mar 2005 11:24:34 +0100 Subject: inets ssl proxy References: <20050223013404.GM14094@fangora.autosys.us> <16924.39226.611947.37660@gargle.gargle.HOWL> <20050224054129.GC14094@fangora.autosys.us> <20050303005923.GV3858@fangora.autosys.us> <20050306202338.GA19104@hyber.org> <20050306211309.GQ3217@fangora.autosys.us> Message-ID: <16940.11234.661415.75846@gargle.gargle.HOWL> Michael McDaniel wrote: > I was told that the inets http client problem was a bug also. Fix to be > sent to erlang-questions sometime. Yes the bug was that the ssl tunnel was not set up on the proxy, e.i. the extra request "CONNECT host:port HTTP/1.1" was never sent. It will be fixed for the next open source release. -- /m.v.h Ingela Ericsson AB - OTP team From thomasl_erlang@REDACTED Mon Mar 7 11:58:31 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 7 Mar 2005 02:58:31 -0800 (PST) Subject: Poll: Packages use In-Reply-To: 6667 Message-ID: <20050307105831.96115.qmail@web41908.mail.yahoo.com> --- Vlad Dumitrescu wrote: > * How many people use packages in their > applications? > it would also be interesting to know if anyone > uses them in a commercial > setting. > * How many people plan to start using packages in 6 > months to one year from > now? While the underlying problem is relevant, I am, after some tinkering, not convinced the current 'packages' are the answer. I guess my answer vector would be (no, no, not me). The first issue is undesirable behaviour: basically, if atom 'm' is used as a module name, to which actual module ("object code") do we refer? For a global name space, it's obvious, but packages answer that question in a deeply unsatisfactory way. http://www.erlang.org/ml-archive/erlang-questions/200304/msg00288.html The second issue is, possibly, due to my lack of experience: using packages didn't seem to yield shorter, "better" or more flexible code than the status quo, in practice. (Again, I'll admit that practice wasn't extensive.) So, I found no compelling reason to switch, and a good reason not to switch. Best, Thomas __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From rrerlang@REDACTED Mon Mar 7 12:01:16 2005 From: rrerlang@REDACTED (Robert Raschke) Date: Mon, 7 Mar 2005 11:01:16 +0000 Subject: memory leakage problem in erl_interface In-Reply-To: Message-ID: <5654d5f984eac04cf1cc4bb0742f483e@tombob.com> Hi Maruthavanan, > I have built a c exe with erl interface. When I send more messages from > erlang to c or c to erlang my exe keeps on consuming memory drastically, > which leads to memory leakage problem. I would recommend trying to memory profile your code, using something like Purify, for example. Alternatively, you could attempt to recast your code using the ei library, which has a much, much simpler memory allocation scheme, leaving you in more control. Having skimmed your code excerpts, it looks like you are cleaning up the eterms properly. That would suggest to me, that the leak might lie elsewhere (e.g., in the functions you call from msg_from_enode()). Robby From thomasl_erlang@REDACTED Mon Mar 7 12:16:56 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 7 Mar 2005 03:16:56 -0800 (PST) Subject: Calling internal functions - foo::bar() ? In-Reply-To: 6667 Message-ID: <20050307111656.98385.qmail@web41901.mail.yahoo.com> --- Bjorn Gustavsson wrote: ... > (We haven't implemented the kind of optimization > mentioned below, > but we hope to do it in a future release.) ... > James Hague writes: > > > Does the compiler not take advantage of functions > being internal to a > > module? For example, if a function always returns > a tuple which is > > immediately unpacked, then the tuple construction > can be avoided and > > the values returned in Erlang registers. At least > I recall reading > > about this somewhere. > > > > If this is the case, then you wouldn't be able to > call internal > > functions directly. Note that you can handle this by a simple high-level transformation. For example, assume is an unboxed tuple and f/1 is originally a function returning a tuple {A,B}. Then rewrite it as follows: f(X) -> = f_internal(X), {A,B}. internal_f(X) -> Unboxed_definition_of_F. (So internal_f/1 returns rather than {E1,E2}) Here's the trick: inside the module, all local calls to f are then rewritten to call internal_f, while outside the module all remote calls (including ::, if available) go to the ordinary f. Local calls will then exploit unboxed tuples, while remote calls still use the default call/return convention. I believe some compilers (GHC?) do this as a consequence of other optimizations (inlining, etc). For example, a call f(E) would be optimized more or less like this: {P1,P2} = f(E) ==> (inline f/1) {P1,P2} = (begin = internal_f(E), {A,B} end) ==> (simplify mildly) = internal_f(E), P1=A, P2=B Voila, the useless tuple is gone. Best, Thomas __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From sean.hinde@REDACTED Mon Mar 7 12:53:42 2005 From: sean.hinde@REDACTED (Sean Hinde) Date: Mon, 7 Mar 2005 11:53:42 +0000 Subject: size of an IO list In-Reply-To: References: Message-ID: On 7 Mar 2005, at 10:20, Bjorn Gustavsson wrote: > As usual, the hardest part is usually coming up with a good name. > After some discussion, Kenneth and I come up with the following > suggestion for TWO new BIFs. > > iolist_size(IOList) > iolist_to_binary(IOList) > > where iolist_to_binary/1 will work like this > > iolist_to_binary(IOList) -> > size(list_to_binary([IOList])). > > but be much faster (and obviously not build the temporary binary). > > The reason for the new iolist_to_binary/1 BIF is that is not clear > that iolist_size/1 is any way releated to list_to_binary/1. I'm not quite sure I get this. iolist_to_binary([1,<<2,3,4>>]) would just return 4? Not a binary? Otherwise great. I usually type io_list with an underscore, but too many underscores are also problematic io_list_to_binary.. Don't mind. Sean From bjorn@REDACTED Mon Mar 7 13:43:15 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 07 Mar 2005 13:43:15 +0100 Subject: size of an IO list In-Reply-To: References: Message-ID: Sorry, I meant that iolist_size/1 should be implemented like iolist_size(IOList) -> size(list_to_binary([IOList])). /Bjorn Sean Hinde writes: > On 7 Mar 2005, at 10:20, Bjorn Gustavsson wrote: > > > As usual, the hardest part is usually coming up with a good name. > > After some discussion, Kenneth and I come up with the following > > suggestion for TWO new BIFs. > > > > iolist_size(IOList) > > iolist_to_binary(IOList) > > > > where iolist_to_binary/1 will work like this > > > > iolist_to_binary(IOList) -> > > size(list_to_binary([IOList])). > > > > but be much faster (and obviously not build the temporary binary). > > > > The reason for the new iolist_to_binary/1 BIF is that is not clear > > that iolist_size/1 is any way releated to list_to_binary/1. > > I'm not quite sure I get this. iolist_to_binary([1,<<2,3,4>>]) would > just return 4? Not a binary? > > Otherwise great. I usually type io_list with an underscore, but too > many underscores are also problematic io_list_to_binary.. Don't mind. > > Sean > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From vlad_dumitrescu@REDACTED Mon Mar 7 14:09:36 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Mon, 7 Mar 2005 14:09:36 +0100 Subject: Calling internal functions - foo::bar() ? References: <20050307111656.98385.qmail@web41901.mail.yahoo.com> Message-ID: Hi, From: "Thomas Lindgren" > Note that you can handle this by a simple high-level > transformation. That's a nice idea. However, I am not convinced that exposing internal functions is a good idea. If it is a good idea, then why not remove -export() completely, and use any optimizations to all intra-module calls? Like someone said before, having an accessible function will make people use it even if it shouldn't be used directly, creating a basic setup for Maintenance Hell (TM). Encapsulation is a good principle for a reason. And also internal functions need not be documented ;-) Yes, it was said that only the shell should be able to access those, but this can't be guaranteed (or if it can it would cost too much). My personal feeling is that testing internal functionality should be done by using (exported) test functions. This has its own drawbacks of course, but it feels better on the whole. best regards, Vlad From thomasl_erlang@REDACTED Mon Mar 7 15:01:53 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 7 Mar 2005 06:01:53 -0800 (PST) Subject: Calling internal functions - foo::bar() ? In-Reply-To: 6667 Message-ID: <20050307140153.82309.qmail@web41904.mail.yahoo.com> --- Vlad Dumitrescu wrote: > Hi, > > From: "Thomas Lindgren" > > Note that you can handle this by a simple > high-level > > transformation. > > That's a nice idea. I regret I didn't think of it first :-) > However, I am not convinced that exposing internal > functions is a good idea. If > it is a good idea, then why not remove -export() > completely, and use any > optimizations to all intra-module calls? Well, the previous optimization only shows there is no real need to forbid :: on grounds of performance. Introducing :: may, as you say, still be a bad idea. I find myself wavering on that issue: on one hand, you may be right in that it's hell on maintenance; on the other hand, I still find it preferrable to -compile(export_all). xref and similar tools could detect anomalies either way, either by looking for :: or by disregarding export_all and warning about calls that do not use explicitly exported functions. > My personal feeling is that testing internal > functionality should be done by > using (exported) test functions. This has its own > drawbacks of course, but it > feels better on the whole. Well, that's another reasonable argument for avoiding ::, at least as part of the language (rather than as a shell hack). (Finally, on the gripping hand, there might be yet another, superior approach to the problem somewhere.) Best, Thomas __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From kostis@REDACTED Mon Mar 7 17:46:17 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 7 Mar 2005 17:46:17 +0100 (MET) Subject: Calling internal functions - foo::bar() ? In-Reply-To: Mail from 'Thomas Lindgren ' dated: Mon, 7 Mar 2005 06:01:53 -0800 (PST) Message-ID: <200503071646.j27GkHMJ019118@spikklubban.it.uu.se> "Thomas Lindgren" wrote: > ... > Well, the previous optimization only shows there is no > real need to forbid :: on grounds of performance. Nope, that's NOT what your transformation is showing. It only shows that you can avoid boxing return values in tuples for calls within the module and still be able to call the function from outside if you are willing to pay in code size. It does NOT say anything about other types of transformations/optimizations that you can do for module-local functions (e.g., removing unreachable clauses, type tests, profiting from sophisticated static analyses, etc). It is of course true that for _all_ optimizations you can always keep the original code around (and pay in code size) so that :: calls are properly serviced from that code. But this is really a nuisance, not to mention the fact that it can really drive you crazy if you are chasing a bug/behaviour that is present only on the optimized code. > Introducing :: may, as you say, still be a bad idea. Indeed it is. A very bad one in fact. Best, Kostis. From thomasl_erlang@REDACTED Mon Mar 7 18:59:09 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 7 Mar 2005 09:59:09 -0800 (PST) Subject: Calling internal functions - foo::bar() ? In-Reply-To: <200503071646.j27GkHMJ019118@spikklubban.it.uu.se> Message-ID: <20050307175909.15763.qmail@web41908.mail.yahoo.com> --- Kostis Sagonas wrote: > "Thomas Lindgren" wrote: > > ... > > Well, the previous optimization only shows there > is no > > real need to forbid :: on grounds of performance. > > Nope, that's NOT what your transformation is > showing. It only shows > that you can avoid boxing return values in tuples > for calls within > the module and still be able to call the function > from outside if > you are willing to pay in code size. > > It does NOT say anything about other types of > transformations/optimizations > that you can do for module-local functions (e.g., > removing unreachable > clauses, type tests, profiting from sophisticated > static analyses, etc). Fair enough, though I was responding to the justification at hand. Note that such a compiler will face precisely the same situation with a module using export_all. Giving up on optimization there may or may not be acceptable. (My own view is, of course, to use module merging, profile-based optimization, etc.) As you subsequently mention, a compiler can in principle generate multiple copies of functions to take advantage of local optimization opportunities, including for exported functions. (And indeed, some compilers do.) Even a compiler using mere inlining will have the issue that the same original "source code" can appear in several places in the final code, optimized for different contexts. That this sort of optimization may require more sophisticated debugging support and yield more difficult compiler bugs than we have today is a challenge to debugger and compiler implementors, in my mind, not a reason to forbid that sort of features. It's not like these problems are insurmountable. (On the other hand, by all means, do build a compelling per-module compiler that, e.g., does not tolerate export_all, and your users/developers may come to accept that sort of restriction. I for one certainly wouldn't mind having it around.) > > Introducing :: may, as you say, still be a bad > idea. > > Indeed it is. A very bad one in fact. Given the prevalence of export_all all the way to deployed code, there still seems to be a customer, er, developer need for this sort of functionality :-) Well, if a solution can't be found, maybe there is a reasonable compromise somewhere. Best, Thomas __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From kostis@REDACTED Mon Mar 7 20:36:12 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 7 Mar 2005 20:36:12 +0100 (MET) Subject: Calling internal functions - foo::bar() ? In-Reply-To: Mail from 'Thomas Lindgren ' dated: Mon, 7 Mar 2005 09:59:09 -0800 (PST) Message-ID: <200503071936.j27JaCQ4010970@spikklubban.it.uu.se> "Thomas Lindgren" replied to my post: > > > > Introducing :: may, as you say, still be a bad idea. > > > > Indeed it is. A very bad one in fact. > > Given the prevalence of export_all all the way to > deployed code, there still seems to be a customer, er, > developer need for this sort of functionality :-) Just in case some of my points were too implicit, let me make them more explicit: Most of us agree that "export_all" is a terrible kludge, but some feel that is is occasionally helpful in developing/debugging. However bad it may be, note that at least it has the property that it allows the compiler to optimize internal functions in modules that do NOT contain an "export_all". This is something detectable at compilation time (i.e., when the .beam file is generated) and trivially also detectable by using "grep". The BEAM/native code code compiler is thus allowed to say: "Aha, the programmer has used an export_all in this module; I will punish her by not performing any optimizations in that module, and will only optimize the internal functions in the rest of her modules." The latter is possible _because_ there is currently no construct to call internal functions. In the presence of a :: construct, no sophisticated optimizations are possible to perform without having a fall-back mechanism. This puts an unnecessarily big implementation effort to the compiler's optimizer and significantly increases the size of the generated .beam files. Good that the OTP team opted against it. Best, Kostis From james.hague@REDACTED Mon Mar 7 16:27:38 2005 From: james.hague@REDACTED (James Hague) Date: Mon, 7 Mar 2005 09:27:38 -0600 Subject: size of an IO list In-Reply-To: References: Message-ID: > iolist_size(IOList) -> > size(list_to_binary([IOList])). I still think that IOLists should be able to include atoms. That is, '' should be considered as "" in an IOList. I have run into this situation a number of times already. It's much cleaner (and generates less garbage) to be able to include atoms directly in IOLists. But I've heard the argument against this (might break old code), so I won't push it. I just wanted another opportunity to put this on the table :) From kostis@REDACTED Mon Mar 7 22:11:34 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 7 Mar 2005 22:11:34 +0100 (MET) Subject: size of an IO list In-Reply-To: Mail from 'James Hague ' dated: Mon, 7 Mar 2005 09:27:38 -0600 Message-ID: <200503072111.j27LBY55014522@spikklubban.it.uu.se> James Hague wrote: > > iolist_size(IOList) -> > > size(list_to_binary([IOList])). > > I still think that IOLists should be able to include atoms. That is, > '' should be considered as "" in an IOList. I have run > into this situation a number of times already. It's much cleaner (and > generates less garbage) to be able to include atoms directly in > IOLists. I am trying to understand these IOLists and why this is an interesting data type to have in the first place, so please excuse my ignorance. Before we agree on what their 'size' is, can we first agree on their definition and if we are happy with it, possibly also add this as a new guard? (An is_iolist/1 Erlang function will do as answer.) Best, Kostis From sean.hinde@REDACTED Mon Mar 7 22:55:36 2005 From: sean.hinde@REDACTED (Sean Hinde) Date: Mon, 7 Mar 2005 21:55:36 +0000 Subject: OS X os_mon broken Message-ID: <9908d4ab376d89325ec0723084e51708@mac.com> Hi, os_mon-1.6.2 is still broken under OS X darwin. The following patch restores what was in my original patch. This makes the solution much less susceptible to changes in the output of uptime. Regards, Sean --- cpu_sup.erl.orig Fri Jun 25 14:25:58 2004 +++ cpu_sup.erl Mon Mar 7 21:50:04 2005 @@ -274,11 +274,11 @@ end; get_int_measurement(Request, #state{os_type = {unix, darwin}}) -> D = os:cmd("uptime") -- "\n", - {ok, [L1, L5, L15], _} = io_lib:fread("~*s~*s~*s~*s~*s~*s~*s~f,~f,~f", D), + [L15, L5, L1|_] = lists:reverse(string:tokens(D, ", ")), case Request of - ?avg1 -> sunify(L1); - ?avg5 -> sunify(L5); - ?avg15 -> sunify(L15); + ?avg1 -> sunify(list_to_float(L1)); + ?avg5 -> sunify(list_to_float(L5)); + ?avg15 -> sunify(list_to_float(L15)); ?ping -> 4711; ?nprocs -> Ps = os:cmd("/bin/ps -ax | /usr/bin/wc -l"), From carsten@REDACTED Mon Mar 7 23:30:06 2005 From: carsten@REDACTED (Carsten Schultz) Date: Mon, 7 Mar 2005 23:30:06 +0100 Subject: size of an IO list In-Reply-To: <200503072111.j27LBY55014522@spikklubban.it.uu.se> References: <200503072111.j27LBY55014522@spikklubban.it.uu.se> Message-ID: <20050307223005.GC4866@penne.localnet> Hi! On Mon, Mar 07, 2005 at 10:11:34PM +0100, Kostis Sagonas wrote: > I am trying to understand these IOLists and why this is an interesting > data type to have in the first place, so please excuse my ignorance. An IO list is a deep list (aka tree) of binarys and integers in the range from 0 to 255. You use them to prepare output you want to write to eg a file later on. The advantage over a simple list of ints is that List1++List2 has to copy List1 while [List1,List2] only has to allocate to cells. > Before we agree on what their 'size' is, can we first agree on their > definition and if we are happy with it, possibly also add this as a > new guard? > > (An is_iolist/1 Erlang function will do as answer.) Such a function would take time proportional to the length of the list, which is why it probably would not be seen as suitable in guards. OTOH size is allowed, although some see this is a design error. Will io_list_size be allowed? I hope that the following is accurate: is_iolist([]) -> true; is_iolist([H|T]) -> is_iolist_head(H) andalso is_iolist(T); is_iolist(_) -> false. is_iolist_head(I) when is_integer(I) -> (I>=0) and (I=<255); is_iolist_head(B) when is_binary(B) -> true; is_iolist_head(X) -> is_iolist(X). Greetings, Carsten -- Carsten Schultz (2:38, 33:47) http://carsten.codimi.de/ PGP/GPG key on the pgp.net key servers, fingerprint on my home page. From luke@REDACTED Tue Mar 8 02:39:37 2005 From: luke@REDACTED (Luke Gorrie) Date: 08 Mar 2005 02:39:37 +0100 Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) References: <200503071936.j27JaCQ4010970@spikklubban.it.uu.se> Message-ID: Kostis Sagonas writes: > Just in case some of my points were too implicit, let me make them > more explicit: We agree that export_all is bad. What we disagree on is the relative importance of troubleshooting tools vs. compiler optimization. This is hardly surprising since one of us is troubleshooting running systems all day and the other is developing compiler optimizations. :-) I did my thing with a parse transform. My problem now is how to handle 'make' dependencies in the presence of parse transforms. (Anyone know?) I can compile this module: -module(x). -compile({parse_transform,internal_exports}). a() -> working. z(X) -> {ok, X}. and I can call its internal functions in the shell: 5> x:a(). working 6> x:z(42). {ok,42} but they are not really exported. The trick is based on the one that Erik Stenman sent me when we had this thread years ago: 7> x:'EXPORTER'(). [{{a,0},#Fun},{{z,1},#Fun}] i.e. the parse transform has synthesized an 'EXPORTER' function that returns funs for everything. Of course I have hacked the shell to support this interface (patch below) but it can also be used in a more klunky way: 8> internal_exports:apply(x, a, []). working which might be bearable with this user_default function: iapply(M, F, A) -> internal_exports:apply(M, F, A). but I'll stick with the customized shell for now. The parse transform is lib/msc/src/internal_exports.erl in jungerl. Cheers, Luke P.S. Good luck Mats on getting this adopted for AXD301. :-) -------------- next part -------------- A non-text attachment was scrubbed... Name: erl_eval.patch Type: text/x-patch Size: 1204 bytes Desc: internal_exports patch to erl_eval.erl URL: From luke@REDACTED Tue Mar 8 03:50:41 2005 From: luke@REDACTED (Luke Gorrie) Date: 08 Mar 2005 03:50:41 +0100 Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) References: <200503071936.j27JaCQ4010970@spikklubban.it.uu.se> Message-ID: Luke Gorrie writes: > I did my thing with a parse transform. My problem now is how to handle > 'make' dependencies in the presence of parse transforms. (Anyone know?) I found a nice way to globally enable this for files compiled on my local machine: # This in ~/.bashrc: export ERL_COMPILER_OPTIONS='[debug_info,{parse_transform,internal_exports}]' % This in ~/.erlang: code:add_pathz("/where/the/parse/transform/module/is/ebin"). From bjorn@REDACTED Tue Mar 8 08:23:46 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 08 Mar 2005 08:23:46 +0100 Subject: OS X os_mon broken In-Reply-To: <9908d4ab376d89325ec0723084e51708@mac.com> References: <9908d4ab376d89325ec0723084e51708@mac.com> Message-ID: There will be several bug fixes in os_mon in R10B-4, including a bug fix to cpu_sup for Max OS X. My bug fix is similar to yours in that it looks at the uptime output starting at the end. /Bjorn %% Get the load average using uptime. D = os:cmd("uptime") -- "\n", %% Here is a sample uptime string from Mac OS 10.3.8: %% "11:17 up 12 days, 20:39, 2 users, load averages: 1.07 0.95 0.66" %% The safest way to extract the load averages seems to be grab everything %% after the last colon and then do an fread on that. Avg = lists:reverse(hd(string:tokens(lists:reverse(D), ":"))), {ok,[L1,L5,L15],_} = io_lib:fread("~f ~f ~f", Avg), case Request of ?avg1 -> sunify(L1); ?avg5 -> sunify(L5); ?avg15 -> sunify(L15); ?ping -> 4711; ?nprocs -> Ps = os:cmd("/bin/ps -ax | /usr/bin/wc -l"), {ok, [N], _} = io_lib:fread("~d", Ps), N-1 end; Sean Hinde writes: > Hi, > > os_mon-1.6.2 is still broken under OS X darwin. > > The following patch restores what was in my original patch. This makes > the solution much less susceptible to changes in the output of uptime. > > Regards, > Sean > > --- cpu_sup.erl.orig Fri Jun 25 14:25:58 2004 > +++ cpu_sup.erl Mon Mar 7 21:50:04 2005 > @@ -274,11 +274,11 @@ > end; > get_int_measurement(Request, #state{os_type = {unix, darwin}}) -> > D = os:cmd("uptime") -- "\n", > - {ok, [L1, L5, L15], _} = > io_lib:fread("~*s~*s~*s~*s~*s~*s~*s~f,~f,~f", D), > + [L15, L5, L1|_] = lists:reverse(string:tokens(D, ", ")), > case Request of > - ?avg1 -> sunify(L1); > - ?avg5 -> sunify(L5); > - ?avg15 -> sunify(L15); > + ?avg1 -> sunify(list_to_float(L1)); > + ?avg5 -> sunify(list_to_float(L5)); > + ?avg15 -> sunify(list_to_float(L15)); > ?ping -> 4711; > ?nprocs -> > Ps = os:cmd("/bin/ps -ax | /usr/bin/wc -l"), > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From heinrich@REDACTED Tue Mar 8 10:40:01 2005 From: heinrich@REDACTED (Heinrich Venter) Date: Tue, 8 Mar 2005 11:40:01 +0200 Subject: Snmp_generic error Message-ID: Hi I give up :) The snmp_generic module in snmp-4.1.1 does not work. It gives the same badmatch error for mnesia, persistant and volatile tables, the same as snmp-4.0.1 ** User error: Invalid return value {'EXIT',{{badmatch,false},[{snmp_generic,table_info,1},{snmp_generic_mne sia,table_func,4},{snmpa_agent,get_next_values_all_rows,6},{snmpa_agent, get_next_table,4},{snmpa_agent,next_loop_varbinds,5},{snmpa_agent,proces s_pdu,4},{snmpa_agent,handle_pdu,7},{snmpa_agent,handle_info,2}]}} from {snmp_generic,table_func,[{connectionTable,mnesia}]} (get_next) As far as I can tell it is because the table_info function tries to collect information about the table from an internal store where it does not exist. This only happens with table_func. Variable_func works with the volatile store. It would seem the only way to make this work is to resort to my own table functions based on the exaple in the snmp user guide, or to remove the tables completely. I used the guidelines given at http://www.drxyzzy.org/mnesia-snmp/ but cant get it to work. If any one has gotten this to work, I would dearly like to hear from you to compare notes. Thanks -]-[einrich Get LookForMe for business using CellFind http://asp1.rocketseed.com/RocketSeed/mail/433a32353a313131383330373a33303639373a2d323a323138 This will take you to the Launchpad site for more info http://asp1.rocketseed.com/RocketSeed/mail/433a32353a313131383330373a33303639373a2d323a333738 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: MTk2LjMwLjc5LjE1NQ== Type: image/gif Size: 14964 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: MTk2LjMwLjc5LjE1NQ== Type: image/gif Size: 620 bytes Desc: not available URL: From mats.cronqvist@REDACTED Tue Mar 8 10:46:09 2005 From: mats.cronqvist@REDACTED (Mats Cronqvist) Date: Tue, 08 Mar 2005 10:46:09 +0100 Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) In-Reply-To: References: <200503071936.j27JaCQ4010970@spikklubban.it.uu.se> Message-ID: <422D7461.9060200@ericsson.com> Luke Gorrie wrote: > Kostis Sagonas writes: > >>Just in case some of my points were too implicit, let me make them >>more explicit: > > > We agree that export_all is bad. What we disagree on is the relative > importance of troubleshooting tools vs. compiler optimization. This is > hardly surprising since one of us is troubleshooting running systems > all day and the other is developing compiler optimizations. :-) i was just going to comment to Kostis that for the AXD301, the choice between a 5% compiler optimization and 5% better turnaround time on bug fixes is a no-brainer (in favor of better debugging). alas, while i personally like the M::F shell-only proposal, i don't see it as being very helpful in AXD301 development. a) it's only really useful to the guy who wrote the code... b) ...and the function is reasonably side-effect free c) we already have the problem that the shell is abused on live nodes (by non-developers) if i thought otherwise i'd be mail-bombing the OTP guys right now. > The parse transform is lib/msc/src/internal_exports.erl in jungerl. > > Cheers, > Luke > > P.S. Good luck Mats on getting this adopted for AXD301. :-) i'll just explain to the bosses that it's object-oriented and they'll give me a raise :> From sean.hinde@REDACTED Tue Mar 8 11:02:13 2005 From: sean.hinde@REDACTED (Sean Hinde) Date: Tue, 8 Mar 2005 10:02:13 +0000 Subject: OS X os_mon broken In-Reply-To: References: <9908d4ab376d89325ec0723084e51708@mac.com> Message-ID: <7b38f4113706e391442d06b2fc588d02@mac.com> Excellent, thank you. Sean On 8 Mar 2005, at 07:23, Bjorn Gustavsson wrote: > There will be several bug fixes in os_mon in R10B-4, including a bug > fix to cpu_sup for Max OS X. > > My bug fix is similar to yours in that it looks at the uptime output > starting at the end. > > /Bjorn > > %% Get the load average using uptime. > D = os:cmd("uptime") -- "\n", > %% Here is a sample uptime string from Mac OS 10.3.8: > %% "11:17 up 12 days, 20:39, 2 users, load averages: 1.07 0.95 > 0.66" > %% The safest way to extract the load averages seems to be grab > everything > %% after the last colon and then do an fread on that. > Avg = lists:reverse(hd(string:tokens(lists:reverse(D), ":"))), > {ok,[L1,L5,L15],_} = io_lib:fread("~f ~f ~f", Avg), > > case Request of > ?avg1 -> sunify(L1); > ?avg5 -> sunify(L5); > ?avg15 -> sunify(L15); > ?ping -> 4711; > ?nprocs -> > Ps = os:cmd("/bin/ps -ax | /usr/bin/wc -l"), > {ok, [N], _} = io_lib:fread("~d", Ps), > N-1 > end; > > Sean Hinde writes: > >> Hi, >> >> os_mon-1.6.2 is still broken under OS X darwin. >> >> The following patch restores what was in my original patch. This makes >> the solution much less susceptible to changes in the output of uptime. >> >> Regards, >> Sean >> >> --- cpu_sup.erl.orig Fri Jun 25 14:25:58 2004 >> +++ cpu_sup.erl Mon Mar 7 21:50:04 2005 >> @@ -274,11 +274,11 @@ >> end; >> get_int_measurement(Request, #state{os_type = {unix, darwin}}) -> >> D = os:cmd("uptime") -- "\n", >> - {ok, [L1, L5, L15], _} = >> io_lib:fread("~*s~*s~*s~*s~*s~*s~*s~f,~f,~f", D), >> + [L15, L5, L1|_] = lists:reverse(string:tokens(D, ", ")), >> case Request of >> - ?avg1 -> sunify(L1); >> - ?avg5 -> sunify(L5); >> - ?avg15 -> sunify(L15); >> + ?avg1 -> sunify(list_to_float(L1)); >> + ?avg5 -> sunify(list_to_float(L5)); >> + ?avg15 -> sunify(list_to_float(L15)); >> ?ping -> 4711; >> ?nprocs -> >> Ps = os:cmd("/bin/ps -ax | /usr/bin/wc -l"), >> > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From maruthavanan_s@REDACTED Tue Mar 8 11:29:42 2005 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Tue, 08 Mar 2005 10:29:42 +0000 Subject: Problem in erlang interface Message-ID: Hi, I want to establish communication between C node and Erlang node, for that I?m using ei_connect_init() function and ei_connect. There is no [problem in establishing connection between two processes but when I tried to receive message from C node using the library function ei_receive_msg(), the function fails with value ERL_EXIT. In ei_receive_msg() I have used the following parameters ei_xreceive_msg(sockfd,&msg,&buff) sockfd=obtained on establishing connection with C node msg is declared as erlang_msg msg; and buff is obtained by ei_x_buff buff; ei_x_new(&buff); _________________________________________________________________ Is your PC infected? Get a FREE online computer virus scan from McAfee? Security. http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963 From kostis@REDACTED Tue Mar 8 12:22:21 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 8 Mar 2005 12:22:21 +0100 (MET) Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) In-Reply-To: Mail from 'Mats Cronqvist ' dated: Tue, 08 Mar 2005 10:46:09 +0100 Message-ID: <200503081122.j28BMLZP023645@spikklubban.it.uu.se> Mats Cronqvist wrote: > > We agree that export_all is bad. What we disagree on is the relative > > importance of troubleshooting tools vs. compiler optimization. This is > > hardly surprising since one of us is troubleshooting running systems > > all day and the other is developing compiler optimizations. :-) > > i was just going to comment to Kostis that for the AXD301, the choice between > a 5% compiler optimization and 5% better turnaround time on bug fixes is a > no-brainer (in favor of better debugging). What my mail tried to express (and apparently it failed) was that for a Programming Language Implementation the choice of satisfying the occasional convenience of *some* users (no matter how important they are) vs. adding a construct that effectively prohibits optimization and thus penalizing *all* its user community should also be a no-brainer. The difference is not that of a compiler writter vs. developer, it is a matter of quantifying over the user community. Erlang, as any language, has to have a compilation mode where analysis or optimizations are allowed. Being able to call arbitrary functions from anywhere (when the compilation was done without +export_all or some other option of the form +I_want_to_call_all_functions_from_anywhere) prohibits not only optimizations, but also reasoning about properties of the code. This is my point. I do not object to the :: construct, I strongly object to it when not explicitly telling the compiler that you reserve the right to use it. The default should be compilation *without* the ability to call internal functions. The opposite is bad not only for my compiler writter wishes, but also software-engineering wise. Best, Kostis From mats.cronqvist@REDACTED Tue Mar 8 12:58:21 2005 From: mats.cronqvist@REDACTED (Mats Cronqvist) Date: Tue, 08 Mar 2005 12:58:21 +0100 Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) In-Reply-To: <200503081122.j28BMLZP023645@spikklubban.it.uu.se> References: <200503081122.j28BMLZP023645@spikklubban.it.uu.se> Message-ID: <422D935D.2040101@ericsson.com> Kostis Sagonas wrote: > Mats Cronqvist wrote: > > > > We agree that export_all is bad. What we disagree on is the relative > > > importance of troubleshooting tools vs. compiler optimization. This is > > > hardly surprising since one of us is troubleshooting running systems > > > all day and the other is developing compiler optimizations. :-) > > > > i was just going to comment to Kostis that for the AXD301, the choice between > > a 5% compiler optimization and 5% better turnaround time on bug fixes is a > > no-brainer (in favor of better debugging). > > What my mail tried to express (and apparently it failed) was that for > a Programming Language Implementation the choice of satisfying the > occasional convenience of *some* users (no matter how important they > are) vs. adding a construct that effectively prohibits optimization > and thus penalizing *all* its user community should also be a no-brainer. i don't feel you failed in expressing yourself in the previous post. i just don't agree, or rather, i think you're over-simplifying. if you excuse me paraphrasing; here you have a choice between satisfying *some* users (the ones that need to integrate and debug, possibly live, systems w/o access to the source code), vs. penalizing *some* users (the ones that need a potential moderate performance increase). as for the AXD301, we of course want both :>. however, to us increased reliability is more important than increased speed. mats From klacke@REDACTED Tue Mar 8 13:34:42 2005 From: klacke@REDACTED (klacke@REDACTED) Date: Tue, 8 Mar 2005 13:34:42 +0100 Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) In-Reply-To: <422D935D.2040101@ericsson.com> References: <200503081122.j28BMLZP023645@spikklubban.it.uu.se> <422D935D.2040101@ericsson.com> Message-ID: <20050308123442.GA14360@hyber.org> On Tue, Mar 08, 2005 at 12:58:21PM +0100, Mats Cronqvist wrote: > here you have a choice between satisfying *some* users (the ones that > need to integrate and debug, possibly live, systems w/o access to the > source code), vs. penalizing *some* users (the ones that need a potential > moderate performance increase). > as for the AXD301, we of course want both :>. however, to us increased > reliability is more important than increased speed. > Speaking for myself. The ability to call private functions in _live_ systems far outweigh performance considerations in the systems I have been involved in lately. Therefore I almost always use -compile(export_all) in most of my modules. Later ... way later when a module has stabilized, I might (if I get the time) do a later overhaul and explicitly export the API functions. So I am apparently valuing debugability higher than sw engineering aspects as well as performance aspects. /klacke -- Claes Wikstrom -- Caps lock is nowhere and http://www.hyber.org -- everything is under control From rrerlang@REDACTED Tue Mar 8 14:59:43 2005 From: rrerlang@REDACTED (Robert Raschke) Date: Tue, 8 Mar 2005 13:59:43 +0000 Subject: Problem in erlang interface In-Reply-To: Message-ID: <41a88056467dd19cd8379326f91e2113@tombob.com> Hi, maruthavanan_s@REDACTED writes: > I want to establish communication between C node and Erlang node, for that > I'm using ei_connect_init() function and ei_connect. There is no [problem in > establishing connection between two processes but when I tried to receive > message from C node using the library function ei_receive_msg(), the > function fails with value ERL_EXIT. One of the problems I came across trying to run a C node (I now use the port mechanism and a C program that communicates via stdin/out) is nicely captured in a comment in some old code of mine: void main_message_loop(int fd, char *mynodeid) { [...] /* This loop can be very problematic if the handle_message() function takes a long time. Erlang expects to get answers to ERL_TICK messages in order to keep the connection alive. If an operation dealt with by handle_regular_message() takes a long time, then these tick will not get answered, leading Erlang to think this C node has died. */ while (ok) { result = ei_xreceive_msg(fd, &msg, &x_in); switch (result) { case ERL_TICK: break; case ERL_MSG: switch (msg.msgtype) { case ERL_EXIT: ok = 0; break case ERL_SEND: case ERL_REG_SEND: handle_message(.....); ei_send(.....); /* return results */ break; } break; } } } Could it be that the Erlang side thinks your node has died? In that case you may have to look into threading to help you deal with long running commands. Robby From luke@REDACTED Tue Mar 8 15:03:18 2005 From: luke@REDACTED (Luke Gorrie) Date: 08 Mar 2005 15:03:18 +0100 Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) References: <200503081122.j28BMLZP023645@spikklubban.it.uu.se> Message-ID: Kostis Sagonas writes: > What my mail tried to express (and apparently it failed) was that for > a Programming Language Implementation the choice of satisfying the > occasional convenience of *some* users (no matter how important they > are) vs. adding a construct that effectively prohibits optimization > and thus penalizing *all* its user community should also be a no-brainer. If you want a lot of real people to benefit from compiler optimizations then please finish HiPE so that we can actually use it. Lack of backtraces, memory leaks in code reloading, unclear limit on volume of native code that can be loaded -- these are limitations imposed for your convenience that prohibit actual use by most of the user community. There's no use targeting idealized users who don't really exist and "punishing" us poor fools who do. On the other hand, if there really are people who never need to call unexported functions, don't get into trouble when using a different compiler for production vs. development, select exactly the right code to native compile, and never have bugs that need live troubleshooting, then please tell us who they are - they should be doing my job. The rumour I heard is they're all programming in ML and Haskell :-( -Luke (sad that Richard C. has moved on to new and exciting things and all my years of nagging him for HiPE backtraces were wasted :-) .. IMHO. From mikael.karlsson@REDACTED Tue Mar 8 16:26:04 2005 From: mikael.karlsson@REDACTED (Mikael Karlsson) Date: Tue, 8 Mar 2005 16:26:04 +0100 Subject: Compiling NBAP ASN.1 signalling Message-ID: <200503081626.05291.mikael.karlsson@creado.com> Hi, Has anybody had any success in compiling the ASN.1 signal specification part of www.3gpp.org Technical Spec 25.433 (V5.9.0) NBAP signalling? I am having problems (with the NBAP-IEs part.) Cheers Mikael From anders.nygren@REDACTED Tue Mar 8 16:33:04 2005 From: anders.nygren@REDACTED (Anders Nygren) Date: Tue, 8 Mar 2005 09:33:04 -0600 Subject: Bug in mnesia:dirty_update_counter Message-ID: There seems to be a bug in mnesia:dirty_update_counter. The first time dirty_update_counter(Tab,Key,Incr) is called with a specific Key the counter is set to 0. The offending code seems to be mnesia_tm:do_update_op(....) do_update_op(Tid, Storage, {{Tab, K}, {RecName, Incr}, update_counter}) -> {NewObj, OldObjs} = case catch mnesia_lib:db_update_counter(Storage, Tab, K, Incr) of NewVal when integer(NewVal), NewVal >= 0 -> {{RecName, K, NewVal}, [{RecName, K, NewVal - Incr}]}; _ -> Zero = {RecName, K, 0}, mnesia_lib:db_put(Storage, Tab, Zero), {Zero, []} end, commit_update(?catch_val({Tab, commit_work}), Tid, Tab, K, NewObj, OldObjs), element(3, NewObj); It always stores 0 when there is no record with the specified Key. /Anders Nygren From kostis@REDACTED Tue Mar 8 16:51:16 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 8 Mar 2005 16:51:16 +0100 (MET) Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) In-Reply-To: Mail from 'Luke Gorrie ' dated: 08 Mar 2005 15:03:18 +0100 Message-ID: <200503081551.j28FpGoE025274@spikklubban.it.uu.se> Luke Gorrie wrote: > If you want a lot of real people to benefit from compiler > optimizations then please finish HiPE so that we can actually use it. > Lack of backtraces, memory leaks in code reloading, unclear limit on > volume of native code that can be loaded -- these are limitations > imposed for your convenience that prohibit actual use by most of the > user community. Seems that you are somehow misinformed. None of these limitations is imposed for "our convenience". On the contrary I would say. - "Lack of backtraces" This is directly related to what I was describing in my previous mails. For performance reasons, optimized code cannot be polluted by debugging code / considerations. (There is a reason why eg. GCC has a -g option that at least till some time ago could not be used in conjunction with -O3.) Stack backtraces CANNOT be as accurate as those of BEAM without a performance penalty (in time, code space, or both). For example, HiPE passes many arguments in machine registers. If an exception occurs later on, their values are lost and cannot appear in the backtrace. Automatic inlining disturbs the calling sequence, etc. Bottomline: This is not out of convenience, it is done for performance. - "Memory leaks in code reloading" + "limit on volume of native code that can be loaded" Both these are limitations imposed to HiPE by BEAM, which actually *complicate* our life rather than making it "more convenient". We have been begging the OTP team to address these issues for at least the last four/five years. Doing so is not straightforward, but all of us are working on this. Once BEAM's runtime system is ready for it, we will do this. Hope this clarifies things. Kostis > (sad that Richard C. has moved on to new and exciting things and > all my years of nagging him for HiPE backtraces were wasted :-) FYI, Richard C. is actually the guy who took OUT the quite minimal backtrace information that earlier versions of HiPE were generating. (This was done to accomodate the new try-based exceptions of BEAM, again not done for HiPE's "convenience".) From raimo@REDACTED Tue Mar 8 12:46:44 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 08 Mar 2005 12:46:44 +0100 Subject: Parameterized/Abstract modules Message-ID: [Dan]: I took a quick glance at it today, and thought that I should be able to do something like: -module(api, [Server]). -export([start/0,foo/1]). start() -> new(spawn(fun() -> init() end)). foo(Arg) -> call(foo,Arg). call(Op,Args) -> Server ! {Op, Args}, receive {Server,Res} -> Res end. I.e. I want to have functions that can be called without being parameterized (sp?). I can implement this with two modules the first one is not a parameterized module, it spawns a process or does other init work, calls new in the real parameterized module, but I thought it was a ugly workaround and it also HIDES the api from the user. Raimo and I talked about it here during lunch and came up with the following suggestions. A new export list, for functions that should not be parameterized, which can be called directly, e.g. api:start() or api:help(). -non_parameterized_exports([start/0,help/0]). and a macro or bif which is called at the end of the "constructors". start() -> Pid = spawn(fun() ->..end), erlang:create_parameterized_module(?MODULE, [Pid]). %% or ?MAKE_PARAMETERIZED_MODULE([Pid]). That way the programmer can decide the name of the "constructor" it doesn't have to be new(..) with exactly the same number of arguments as module parameters and in the same order, it could be named init, start or creat :-) For backwards compability a new(Params) function can be generated if -non_parameterized_exports() is not present. But do we really have to be backwards compatible, is anyone using it alot? Comments? Name suggestions? /Dan and Raimo -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From ulf.wiger@REDACTED Wed Mar 9 10:04:23 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Wed, 9 Mar 2005 10:04:23 +0100 Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) Message-ID: Luke: > If you want a lot of real people to benefit from compiler > optimizations then please finish HiPE so that we can actually > use it. In fairness, OTP nowadays contains lots of stuff that came from the HiPE team, and that we all benefit from. My personal view on the original question (whether it should be possible to call local functions): I do not like the idea of establishing a system that essentially makes all functions 'exported'. For debugging, I can imagine a few ways to achieve the same effect. - The parse_transform option that Luke suggested. If I understood it correctly, it _doesn't_ export the local functions per se, but rather a wrapper function through which you can reach all local functions. I don't see how this interferes much with the optimization potential. - If debug_info is present, one could do the same thing using a support function that reads the debug_info chunk and recompiles it on the fly with export_all, or inserting a wrapper function as in Luke's parse_transform. I think that one of the big problems in large, complex systems is that the code tends to become difficult to follow, with too many module hops etc. Another problem I think I see is that people do low-level optimizations in order to make the code go faster. My own experience is that code thus 'optimized' is quite often less efficient than textbook Erlang run through a reasonably efficient compiler. We have stated on occasion that native code compilation has no positive effect on the performance-critical code in AXD 301. This is based on performance measurements where we've tried to hipe-compile parts of our code. My own take on this is that the code is written such that it leaves very little for the compiler to work with (the large number of module hops is one factor working against hipe.) But I am absolutely convinced that the code could be made faster (perhaps much faster) by re-writing it in a way that it becomes both more readable and easier for a compiler to optimize(*). Since our code appears to be fast enough, and robust enough, as it is, there is little-to-no short-term incentive to do this. Besides not making the code faster, such pseudo-optimizations - clutter up the code, - making it more difficult to understand, - which leads to systems that are more difficult to debug, - which increases the need for hacks, like calling local functions from the shell - which makes it more difficult for the compiler to optimize the code - which increases the perceived need for 'manual optimizations', which clutters up the code, etc. In my mind, this is not a question about speed. It's a question about conceptual integrity. If the debugging need can be addressed through a reasonable workaround, then let's not change the language. /Uffe (*) I would like to state for the record that I _do not_ mean that the code isn't competently written. It is -- in many places very competently written. But there are many forces at work in large projects, and some of the programming habits were established at a time when the Erlang compiler did very few optimizations (and the computers were not nearly as fast as today), and optimization by hand was an absolute must in order to meet the performance requirements. From vlad_dumitrescu@REDACTED Wed Mar 9 10:37:11 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Wed, 9 Mar 2005 10:37:11 +0100 Subject: Parameterized/Abstract modules References: Message-ID: ----- Original Message ----- From: "Raimo Niskanen" > I.e. I want to have functions that can be called without being > parameterized (sp?). You mean something like "static" functions? The problem as I see it is that the APIs are usually located in the same file as the implementation. For abstract modules, it isn't as easy to use, because the module name is no longer an atom, but has to be passed around in a variable. Maybe separating the API from the implementation isn't a bad idea even for normal gen_servers and friends... Regarding the issue at hand, the compiler could check that the module variables are not referenced inside that function, and that it only calls such static functions (or external ones, of course) and generate them in a separate internal module. Or actually, the parametrized module would be differently named (because static calls should use the name in the source file). regards, Vlad From dgud@REDACTED Wed Mar 9 11:32:54 2005 From: dgud@REDACTED (Dan Gudmundsson) Date: Wed, 9 Mar 2005 11:32:54 +0100 Subject: Parameterized/Abstract modules In-Reply-To: References: Message-ID: <16942.53462.344358.770503@rian.du.uab.ericsson.se> Maybe "static" functions, atleast I want define/overload the constructor and check the arguments and do some init work, before returning the module/object to the caller. Hmm, the proposal with a erlang:create_parameterized_module(?MODULE, [Params]). opens up the possibilty to change the parameters as well, by returning a new module in the calls. A0 = tree:new(this), A1 = A0:insert(turns_erlang), A2 = A1:insert(towards_oo), Good or bad? Code upgrades on these modules are hard aswell, you can never change the length of the parameter list.. /Dan Vlad Dumitrescu writes: > ----- Original Message ----- > From: "Raimo Niskanen" > > I.e. I want to have functions that can be called without being > > parameterized (sp?). > > You mean something like "static" functions? > > The problem as I see it is that the APIs are usually located in the same file as > the implementation. For abstract modules, it isn't as easy to use, because the > module name is no longer an atom, but has to be passed around in a variable. > > Maybe separating the API from the implementation isn't a bad idea even for > normal gen_servers and friends... > > Regarding the issue at hand, the compiler could check that the module variables > are not referenced inside that function, and that it only calls such static > functions (or external ones, of course) and generate them in a separate internal > module. Or actually, the parametrized module would be differently named (because > static calls should use the name in the source file). > > regards, > Vlad > -- Dan Gudmundsson Project: Mnesia, Erlang/OTP Ericsson Utvecklings AB Phone: +46 8 727 5762 UAB/F/P Mobile: +46 70 519 9469 S-125 25 Stockholm Visit addr: Armborstv 1 From dgud@REDACTED Wed Mar 9 11:37:02 2005 From: dgud@REDACTED (Dan Gudmundsson) Date: Wed, 9 Mar 2005 11:37:02 +0100 Subject: Bug in mnesia:dirty_update_counter In-Reply-To: References: Message-ID: <16942.53710.42304.79283@rian.du.uab.ericsson.se> It's a (broken) feature..:-) I probably can't change that behaviour, but I can document it.. /Dan Anders Nygren writes: > There seems to be a bug in mnesia:dirty_update_counter. > The first time dirty_update_counter(Tab,Key,Incr) is called with a specific Key > the counter is set to 0. > The offending code seems to be mnesia_tm:do_update_op(....) > > > do_update_op(Tid, Storage, {{Tab, K}, {RecName, Incr}, update_counter}) -> > {NewObj, OldObjs} = > case catch mnesia_lib:db_update_counter(Storage, Tab, K, Incr) of > NewVal when integer(NewVal), NewVal >= 0 -> > {{RecName, K, NewVal}, [{RecName, K, NewVal - Incr}]}; > _ -> > Zero = {RecName, K, 0}, > mnesia_lib:db_put(Storage, Tab, Zero), > {Zero, []} > end, > commit_update(?catch_val({Tab, commit_work}), Tid, Tab, > K, NewObj, OldObjs), > element(3, NewObj); > > It always stores 0 when there is no record with the specified Key. > > /Anders Nygren From thomasl_erlang@REDACTED Wed Mar 9 12:06:48 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 9 Mar 2005 03:06:48 -0800 (PST) Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) In-Reply-To: 6667 Message-ID: <20050309110648.64311.qmail@web41908.mail.yahoo.com> --- "Ulf Wiger (AL/EAB)" wrote: > - The parse_transform option that Luke suggested. If > I understood > it correctly, it _doesn't_ export the local > functions per se, > but rather a wrapper function through which you > can reach all > local functions. I don't see how this interferes > much with > the optimization potential. Alas, for optimization purposes it's the moral equivalent of exporting the function. The transform introduces an unknown caller to all functions; if the compiler does not account for this, certain optimizations will be unsafe. (In particular, if you call the function from a "sneak export" unknown to the compiler, you roll the dice and take your chances. Depending on the optimization in question, you are likely to get core dumps (type analysis), corrupt data (reuse analysis), etc.) > - If debug_info is present, one could do the same > thing using a > support function that reads the debug_info chunk > and recompiles > it on the fly with export_all, or inserting a > wrapper function > as in Luke's parse_transform. Some systems support "deoptimization" to enable debugging; recompiling the code for debugging is essentially that approach. This seems to work. The current code manager would have to change to handle this: first, code purging will be a problem (loading new modules will eventually kill off processes that should be alive); second, the existing call stack will still return to the old, undebuggable code. Some modification might be needed, thus. (A more flexible general scheme for loading and unloading code might be quite useful for this and similar purposes.) I think this might be a useful way forward. One could see it as "unsealing" and "sealing" the module(s), if you will, which could replace export_all entirely. (Whether everyone will be happy to provide a readable representation of the source in their shipped systems is another question :-) E.g., http://www.erlang.org/ml-archive/erlang-questions/200101/msg00106.html (But there might be workarounds to that too.) Best, Thomas __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From bjorn@REDACTED Wed Mar 9 12:14:11 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 09 Mar 2005 12:14:11 +0100 Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) In-Reply-To: <20050309110648.64311.qmail@web41908.mail.yahoo.com> References: <20050309110648.64311.qmail@web41908.mail.yahoo.com> Message-ID: Thomas Lindgren writes: > --- "Ulf Wiger (AL/EAB)" > wrote: [...] > > Alas, for optimization purposes it's the moral > equivalent of exporting the function. The transform > introduces an unknown caller to all functions; if the > compiler does not account for this, certain > optimizations will be unsafe. Since the wrapper function presumably is exported, the optimizer will not generate unsafe code, but will simply do less optimization. > > (In particular, if you call the function from a "sneak > export" unknown to the compiler, you roll the dice and > take your chances. Depending on the optimization in > question, you are likely to get core dumps (type > analysis), corrupt data (reuse analysis), etc.) I assume that the parse transformation inserts one additional exported function; otherwise there would be now way to call the function. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From thomasl_erlang@REDACTED Wed Mar 9 15:13:58 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 9 Mar 2005 06:13:58 -0800 (PST) Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) In-Reply-To: 6667 Message-ID: <20050309141358.12066.qmail@web41901.mail.yahoo.com> --- Bjorn Gustavsson wrote: > Thomas Lindgren writes: > > > Alas, for optimization purposes it's the moral > > equivalent of exporting the function. The > transform > > introduces an unknown caller to all functions; if > the > > compiler does not account for this, certain > > optimizations will be unsafe. > > Since the wrapper function presumably is exported, > the optimizer will not generate unsafe code, but > will simply do less optimization. Sure, so it's "the moral equivalent of exporting the function". Given a debug wrapper exporting all functions (even indirectly, as in the parse transform example), a conscientous compiler is obviously back at the export_all level of optimization. Ignoring the wrapper, on the other hand, leads to unsafe optimization as described previously. There may be ways around this dilemma, of course. Best, Thomas __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From ulf.wiger@REDACTED Wed Mar 9 15:23:11 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Wed, 9 Mar 2005 15:23:11 +0100 Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) Message-ID: Still, such a workaround, if it can be applied conveniently in the field (which may be difficult if the requirement is "no source code available"), would satisfy those backward people who favour debuggability over speed. ;-) That is, if by a simple command, one can convert a module from "normal mode" to having all local functions reachable, I would think that most people would be happy. Those who care more about debugging than speed at the time, will happily pay the performance price. Nobody else will have to. /Uffe -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED]On Behalf Of Thomas Lindgren Sent: den 9 mars 2005 15:14 To: erlang-questions@REDACTED Subject: Re: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) --- Bjorn Gustavsson wrote: > Thomas Lindgren writes: > > > Alas, for optimization purposes it's the moral > > equivalent of exporting the function. The > transform > > introduces an unknown caller to all functions; if > the > > compiler does not account for this, certain > > optimizations will be unsafe. > > Since the wrapper function presumably is exported, > the optimizer will not generate unsafe code, but > will simply do less optimization. Sure, so it's "the moral equivalent of exporting the function". Given a debug wrapper exporting all functions (even indirectly, as in the parse transform example), a conscientous compiler is obviously back at the export_all level of optimization. Ignoring the wrapper, on the other hand, leads to unsafe optimization as described previously. There may be ways around this dilemma, of course. Best, Thomas __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From thomasl_erlang@REDACTED Wed Mar 9 15:36:35 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 9 Mar 2005 06:36:35 -0800 (PST) Subject: Bug in mnesia:dirty_update_counter In-Reply-To: 6667 Message-ID: <20050309143635.5367.qmail@web41904.mail.yahoo.com> --- Anders Nygren wrote: > There seems to be a bug in > mnesia:dirty_update_counter. > The first time dirty_update_counter(Tab,Key,Incr) is > called with a specific Key > the counter is set to 0. I'd say that is a feature, actually ... If only ets:update_counter had the same behaviour (or _nearly_ the same: behaving _as_if_ the counter had value 0 before this operation). I could then replace case catch ets:update_counter(Tab, Key, Incr) of {'EXIT',_} -> ets:insert(Tab, {Key, Incr}), Incr; N -> N end with (the silently initializing version): ets:update_counter(Tab, Key, Incr) Pretty please, someone? :-) Best, Thomas __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From vlad_dumitrescu@REDACTED Wed Mar 9 15:48:48 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Wed, 9 Mar 2005 15:48:48 +0100 Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) References: Message-ID: ----- Original Message ----- From: "Ulf Wiger (AL/EAB)" > That is, if by a simple command, one can convert a module > from "normal mode" to having all local functions reachable, > I would think that most people would be happy. Those who > care more about debugging than speed at the time, will > happily pay the performance price. Nobody else will > have to. But even with this command, one would have to reload the affected module, which might cause processes to be killed, and then the state of the system is changed... Besides, the same code compiled with different flags might cause different behaviours, so maybe the bug won't manifest itself but with the optimized version... Isn't it easy and fun to debug live systems? :-) regards, Vlad From anders.nygren@REDACTED Wed Mar 9 16:03:11 2005 From: anders.nygren@REDACTED (Anders Nygren) Date: Wed, 9 Mar 2005 09:03:11 -0600 Subject: Bug in mnesia:dirty_update_counter In-Reply-To: <16942.53710.42304.79283@rian.du.uab.ericsson.se> References: <16942.53710.42304.79283@rian.du.uab.ericsson.se> Message-ID: On Wed, 9 Mar 2005 11:37:02 +0100, Dan Gudmundsson wrote: > > It's a (broken) feature..:-) > > I probably can't change that behaviour, but I can document it.. > Is there any technical reason, or just the dreaded backward compability problem? /Anders From vances@REDACTED Wed Mar 9 16:08:59 2005 From: vances@REDACTED (Vance Shipley) Date: Wed, 9 Mar 2005 10:08:59 -0500 Subject: Bug in mnesia:dirty_update_counter In-Reply-To: <20050309143635.5367.qmail@web41904.mail.yahoo.com> References: <20050309143635.5367.qmail@web41904.mail.yahoo.com> Message-ID: <20050309150859.GK45756@frogman.motivity.ca> If only mnesia:dirty_update_counter/3 behaved the same as ets:update_counter/3: mnesia:dirty_update_counter(Tab, Key, Incr) -> ets:update_counter(Tab, Key, {Pos,Incr,Threshold,SetValue}) -> ets:update_counter(Tab, Key, {Pos,Incr}) -> ets:update_counter(Tab, Key, Incr) -> With the ets version you can have, for example, a modulo 128 counter with: ets:update_counter(Tab, Key, {Pos, 1, 127, 0}). -Vance From anders.nygren@REDACTED Wed Mar 9 16:21:40 2005 From: anders.nygren@REDACTED (Anders Nygren) Date: Wed, 9 Mar 2005 09:21:40 -0600 Subject: Bug in mnesia:dirty_update_counter In-Reply-To: <20050309143635.5367.qmail@web41904.mail.yahoo.com> References: <20050309143635.5367.qmail@web41904.mail.yahoo.com> Message-ID: On Wed, 9 Mar 2005 06:36:35 -0800 (PST), Thomas Lindgren wrote: > > --- Anders Nygren wrote: > > There seems to be a bug in > > mnesia:dirty_update_counter. > > The first time dirty_update_counter(Tab,Key,Incr) is > > called with a specific Key > > the counter is set to 0. > > I'd say that is a feature, actually ... If only > ets:update_counter had the same behaviour (or _nearly_ > the same: behaving _as_if_ the counter had value 0 > before this operation). > > I could then replace > > case catch ets:update_counter(Tab, Key, Incr) of > {'EXIT',_} -> ets:insert(Tab, {Key, Incr}), Incr; > N -> N > end > > with (the silently initializing version): > > ets:update_counter(Tab, Key, Incr) > > Pretty please, someone? :-) > While I agee that the ets behaviour is annoying, I really dont like the current mnesia behaviour. I currently have to count things where the set of keys is not known in advance so I have to do case mnesia:dirty_update_counter(Tab,Key,Incr) of 0 -> mnesia:dirty_update_counter(Tab,Key,Incr); N -> N end Currently Incr is always >0 so I dont have to worry about the case of the counter decreasing to =< 0. Which I guess would turn the above into something like case Incr>0 of true -> case mnesia:dirty_update_counter(Tab,Key,Incr) of 0 -> mnesia:dirty_update_counter(Tab,Key,Incr); N -> N end; false -> mnesia:dirty_update_counter(Tab,Key,Incr) end /Anders From anders.nygren@REDACTED Wed Mar 9 16:33:38 2005 From: anders.nygren@REDACTED (Anders Nygren) Date: Wed, 9 Mar 2005 09:33:38 -0600 Subject: How to setup your own Web server with Yaws In-Reply-To: <20050309004826.0ACFF46939@bang.trapexit.org> References: <20050309004826.0ACFF46939@bang.trapexit.org> Message-ID: On Wed, 9 Mar 2005 01:48:25 +0100, tobbe wrote: > > Yet another HowTo is now available at: > > http://www.trapexit.org/docs/howto/howto_setup_yaws.html > A small request, it would be nice with a howto on using yaws in embedded mode too. Since there is very little information on that in the Yaws documentation. /Anders From mikael.karlsson@REDACTED Wed Mar 9 16:48:18 2005 From: mikael.karlsson@REDACTED (Mikael Karlsson) Date: Wed, 9 Mar 2005 16:48:18 +0100 Subject: Compiling NBAP ASN.1 signalling In-Reply-To: <9475d08266935e76a73642ad4dcb1e95@mac.com> References: <200503081626.05291.mikael.karlsson@creado.com> <9475d08266935e76a73642ad4dcb1e95@mac.com> Message-ID: <200503091648.18899.mikael.karlsson@creado.com> Great! thanks Sean. I get the problem in exactly the same place. Good to know you are not alone :-) Cheers Mikael onsdag 09 mars 2005 00:56 skrev Sean Hinde: > Hi, > > On 8 Mar 2005, at 15:26, Mikael Karlsson wrote: > > Hi, > > > > Has anybody had any success in compiling the ASN.1 > > signal specification part of www.3gpp.org > > Technical Spec 25.433 (V5.9.0) NBAP signalling? > > I am having problems (with the NBAP-IEs part.) > > I thought it would be fun to try (it beats watching the DVD of daughter > for the 50th time anyway :-) ) > > I got problems in the same file. > > I attach the files having removed the syntax errors introduced by the > copy/paste from MS Word to save anyone else working on this problem > having to do it. Beware, I have tweaked a couple of places in trying to > get it to work. "TODO" appears on every line I have changed, with the > original part in a comment From luke@REDACTED Wed Mar 9 17:51:29 2005 From: luke@REDACTED (Luke Gorrie) Date: 09 Mar 2005 17:51:29 +0100 Subject: Debunking The Expensive Exported-Function Call Myth References: <200503021440.j22Ee4dU019737@spikklubban.it.uu.se> Message-ID: Bjorn Gustavsson writes: > After some discussion in the OTP group, we have decided NOT > to implement any support for calling a local function. Fair enough. > James Hague writes: > > > Does the compiler not take advantage of functions being internal to a > > module? For example, if a function always returns a tuple which is > > immediately unpacked, then the tuple construction can be avoided and > > the values returned in Erlang registers. At least I recall reading > > about this somewhere. > > > > If this is the case, then you wouldn't be able to call internal > > functions directly. This one seems easy. Suppose you have this code: foo() -> {A, B} = bar(), ...code... bar() -> {1, 2}. then the optimized version (using Core Erlang syntax for in registers syntax) is something like: foo() -> = bar(), ...code... bar() -> <1, 2>. then to support calling bar/0 externally you'd need a tiny little wrapper that calls the optimized one and packs the result into a tuple: foo() -> = opt(), ...code... opt() -> <1, 2>. bar() -> = opt(), {A, B}. This way the optimization would also benefit people using export_all. Note: That includes everyone using embedded yaws. :-) From thomasl_erlang@REDACTED Wed Mar 9 18:29:03 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 9 Mar 2005 09:29:03 -0800 (PST) Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) In-Reply-To: 6667 Message-ID: <20050309172903.75349.qmail@web41905.mail.yahoo.com> --- "Ulf Wiger (AL/EAB)" wrote: > > > Still, such a workaround, if it can be applied > conveniently > in the field (which may be difficult if the > requirement is > "no source code available"), would satisfy those > backward > people who favour debuggability over speed. ;-) > > That is, if by a simple command, one can convert a > module > from "normal mode" to having all local functions > reachable, > I would think that most people would be happy. Those > who > care more about debugging than speed at the time, > will > happily pay the performance price. Nobody else will > have to. --- "Ulf Wiger (AL/EAB)" wrote: > That is, if by a simple command, one can convert a > module > from "normal mode" to having all local functions > reachable, > I would think that most people would be happy. Those > who > care more about debugging than speed at the time, > will > happily pay the performance price. Nobody else will > have to. Well, I'd classify the above as falling under your second alternative (fall back to unoptimized/deoptimized/... code for debugging), and what I wrote was really about the first one (provide an extra debug wrapper that actually exports all functions). Your second alternative sounds fine to me, keeping in mind the previous caveat that code loading will have to change to avoid nasty surprises. (Also, as Vlad points out, assuming that the debugging does not concern compiler bugs :-) The alternative of having a per-module compiler that doesn't do much when export_all is used is also fine by me, as mentioned before. It can still be a useful tool, when applicable. Best, Thomas __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From maruthavanan_s@REDACTED Thu Mar 10 06:30:52 2005 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Thu, 10 Mar 2005 05:30:52 +0000 Subject: Problem in erlang interface In-Reply-To: <41a88056467dd19cd8379326f91e2113@tombob.com> Message-ID: hi i am trying to use ei as you said. but i still have some problems...i am using windows XP with SP2 we have a large number of message to be passed to erlang and receive from erlang. char *cookie="secretcookie"; char *nodename="enode"; char *Node;int loop=1; if (ei_connect_init(&ec, nodename, cookie, id_number) < 0) { } Node=ei_thisnodename(&ec); if ((sockfd = ei_connect(&ec,Node)) < 0) { printf("ERROR: erl_connect failed"); exit(0); } printf("Connection established...\n"); while (loop) { ei_x_buff x; ei_x_new_with_version(&x); got=ei_xreceive_msg(sockfd,&msg,&x); } Executing the above code we are able to get success in ei_connect and we are able to get a non-zero 'sockfd' value but immediately after connection in while loop ei_xreceive_msg() returns an ERL_ERROR value what could be the reason for this. Is any thing missing on the above code Thanks Maruthavanan. _________________________________________________________________ Don?t just search. Find. Check out the new MSN Search! http://search.msn.click-url.com/go/onm00200636ave/direct/01/ From maruthavanan_s@REDACTED Thu Mar 10 07:13:29 2005 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Thu, 10 Mar 2005 06:13:29 +0000 Subject: Problem in erlang interface In-Reply-To: Message-ID: hi One small doubt i am trying to connect erlang node as server and C node as client. Is this possible in erlang using the following code. Thanks maruthavanan >From: "maruthavanan s" >To: erlang-questions@REDACTED >Subject: Re: Problem in erlang interface >Date: Thu, 10 Mar 2005 05:30:52 +0000 > >hi > >i am trying to use ei as you said. but i still have some problems...i am >using windows XP with SP2 we have a large number of message to be passed to >erlang and receive from erlang. > > >char *cookie="secretcookie"; >char *nodename="enode"; >char *Node;int loop=1; >if (ei_connect_init(&ec, nodename, cookie, id_number) < 0) >{ >} >Node=ei_thisnodename(&ec); >if ((sockfd = ei_connect(&ec,Node)) < 0) >{ > printf("ERROR: erl_connect failed"); > exit(0); >} >printf("Connection established...\n"); >while (loop) >{ > ei_x_buff x; > ei_x_new_with_version(&x); > got=ei_xreceive_msg(sockfd,&msg,&x); >} > >Executing the above code we are able to get success in ei_connect and we >are able to get a non-zero 'sockfd' value >but immediately after connection in while loop ei_xreceive_msg() returns an >ERL_ERROR value > >what could be the reason for this. >Is any thing missing on the above code > >Thanks >Maruthavanan. > >_________________________________________________________________ >Don?t just search. Find. Check out the new MSN Search! >http://search.msn.click-url.com/go/onm00200636ave/direct/01/ > _________________________________________________________________ Express yourself instantly with MSN Messenger! Download today - it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ From thomas.xa.johnsson@REDACTED Thu Mar 10 08:38:32 2005 From: thomas.xa.johnsson@REDACTED (Thomas Johnsson XA (LN/EAB)) Date: Thu, 10 Mar 2005 08:38:32 +0100 Subject: Bug in mnesia:dirty_update_counter Message-ID: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB0@esealmw107.eemea.ericsson.se> > I could then replace > > case catch ets:update_counter(Tab, Key, Incr) of > {'EXIT',_} -> ets:insert(Tab, {Key, Incr}), Incr; > N -> N > end While update_counter is atomic & thread-safe if the key is already in the table, the above is not thread safe: two or more processes might simultaneously do the ets:update_counter, get 'EXIT', and insert.... A thread-safe version which takes into account that the key does not exist previously in the table, should use ets:insert_new (new as of R10 ?) : case ets:insert_new(Tab, {Key, Incr}) of true -> Incr; %% really was new key, insert succeeded false -> ets:update_counter(Tab, Key, Incr) end If there's many simultaneous insert_new s, only one will success, the others perform update_counter. -- Thomas From csanto@REDACTED Thu Mar 10 09:02:08 2005 From: csanto@REDACTED (Corrado Santoro) Date: Thu, 10 Mar 2005 09:02:08 +0100 Subject: An "is_string" BIF? Message-ID: <422FFF00.9030609@diit.unict.it> Dear all, I need an "is_string" function, something that behaves as follows: is_string ("ABC") -> true is_string ([65,66,67]) -> true is_string (['1','2',three]) -> false ... I know that I could do that by defining a suitable function, but I would like to use it in a guard. Is there a sort of BIF? Cheers, --Corrado -- ====================================================== Eng. Corrado Santoro, Ph.D. University of Catania - Engineering Faculty Department of Computer Science and Telecommunications Engineering Viale A. Doria, 6 - 95125 CATANIA (ITALY) Tel: +39 095 7382380 Fax: +39 095 7382397 +39 095 7382365 +39 095 7382364 EMail: csanto@REDACTED Personal Home Page: http://www.diit.unict.it/users/csanto NUXI Home Page: http://nuxi.iit.unict.it ====================================================== From bertil@REDACTED Thu Mar 10 09:17:11 2005 From: bertil@REDACTED (Bertil Karlsson) Date: Thu, 10 Mar 2005 09:17:11 +0100 Subject: Compiling NBAP ASN.1 signalling In-Reply-To: <200503091648.18899.mikael.karlsson@creado.com> References: <200503081626.05291.mikael.karlsson@creado.com> <9475d08266935e76a73642ad4dcb1e95@mac.com> <200503091648.18899.mikael.karlsson@creado.com> Message-ID: <42300287.4050704@erix.ericsson.se> Hi, We (Kenneth and I) have been trying those specs and discovered a couple of bugs in the asn1 compiler that I'm now going to fix. It should be interesting to know if we run into the same errors as you did. The compiler reported errors on constants in the NBAP-Constants.asn module but somehow it passed the compiler. Then it failed on the NBAP-IEs.asn module on some INTEGER constraints and the NBAP-IEs.asn, NBAP-PDU-Contents.asn and NBUP-PDU-Descriptions.asn modules on some information object handling. Though, we have not seen any syntax errors in the ASN1 specs. /Bertil Mikael Karlsson wrote: > Great! > > thanks Sean. I get the problem in exactly > the same place. Good to know you are not > alone :-) > > Cheers > Mikael > > onsdag 09 mars 2005 00:56 skrev Sean Hinde: > >>Hi, >> >>On 8 Mar 2005, at 15:26, Mikael Karlsson wrote: >> >>>Hi, >>> >>>Has anybody had any success in compiling the ASN.1 >>>signal specification part of www.3gpp.org >>>Technical Spec 25.433 (V5.9.0) NBAP signalling? >>>I am having problems (with the NBAP-IEs part.) >> >>I thought it would be fun to try (it beats watching the DVD of daughter >>for the 50th time anyway :-) ) >> >>I got problems in the same file. >> >>I attach the files having removed the syntax errors introduced by the >>copy/paste from MS Word to save anyone else working on this problem >>having to do it. Beware, I have tweaked a couple of places in trying to >>get it to work. "TODO" appears on every line I have changed, with the >>original part in a comment > > > From thomas.xa.johnsson@REDACTED Thu Mar 10 09:21:36 2005 From: thomas.xa.johnsson@REDACTED (Thomas Johnsson XA (LN/EAB)) Date: Thu, 10 Mar 2005 09:21:36 +0100 Subject: An "is_string" BIF? Message-ID: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB1@esealmw107.eemea.ericsson.se> This really exposes an ugly wart on the language, in that one cannot use any expression in a guard, making it impossible to use abstractions there, e.g. glorbgraph(Graph) when mygraph:curiousproperty(Graph) -> ... -- Thomas -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED]On Behalf Of Corrado Santoro Sent: den 10 mars 2005 09:02 To: erlang-questions@REDACTED Subject: An "is_string" BIF? Dear all, I need an "is_string" function, something that behaves as follows: is_string ("ABC") -> true is_string ([65,66,67]) -> true is_string (['1','2',three]) -> false ... I know that I could do that by defining a suitable function, but I would like to use it in a guard. Is there a sort of BIF? Cheers, --Corrado -- From mccratch@REDACTED Thu Mar 10 10:13:21 2005 From: mccratch@REDACTED (mccratch@REDACTED) Date: Thu, 10 Mar 2005 10:13:21 +0100 Subject: An "is_string" BIF? In-Reply-To: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB1@esealmw107.eemea.ericsson.se> References: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB1@esealmw107.eemea.ericsson.se> Message-ID: <20050310091321.GA6854@localhost.localdomain> On 2005-03-10 at 09:21:36 (+0100), Thomas Johnsson XA (LN/EAB) wrote: > This really exposes an ugly wart on the language, in that > one cannot use any expression in a guard, making it impossible > to use abstractions there, e.g. > > glorbgraph(Graph) when mygraph:curiousproperty(Graph) -> ... > > -- Thomas I would be interested why this isn't allowed. If I recall correctly it has something to do with how guards are evaluated? Or maybe I am now confusing something. -- Cheers Matthias Kretschmer From vlad_dumitrescu@REDACTED Thu Mar 10 10:14:39 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Thu, 10 Mar 2005 10:14:39 +0100 Subject: An "is_string" BIF? References: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB1@esealmw107.eemea.ericsson.se> Message-ID: From: "Thomas Johnsson XA (LN/EAB)" > This really exposes an ugly wart on the language, in that > one cannot use any expression in a guard, making it impossible > to use abstractions there The real problem is that guards must be guaranteed to be side-effect free... I think (but am no compiler expert) that it should be possible to (very) conservatively tag functions as 'side-effect free'. Hopefully these functions will not be so restricted as to become useless [*] The other problem is that guards should evaluate in bounded time. This is much harder to control, but maybe it's not a hard requirement? regards, Vlad [*] They should only call other 'side-effect free' functions - since we don't have global compilation, there is no guarantee that external calls are side-effect free, so this limits usefulness quite a lot. Bifs should be ok, but not necessarily across releases. So it seems this is a big can of worms :-) From matthias@REDACTED Thu Mar 10 10:25:17 2005 From: matthias@REDACTED (Matthias Lang) Date: Thu, 10 Mar 2005 10:25:17 +0100 Subject: An "is_string" BIF? In-Reply-To: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB1@esealmw107.eemea.ericsson.se> References: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB1@esealmw107.eemea.ericsson.se> Message-ID: <16944.4733.267416.481949@antilipe.corelatus.se> Thomas Johnsson XA (LN/EAB) writes: > This really exposes an ugly wart on the language, in that > one cannot use any expression in a guard, If the restriction were removed, a number of problems arise. Those problems are discussed here: http://www.erlang.org/faq/x868.html (question 9.4) and here: http://www.erlang.org/ml-archive/erlang-questions/200402/msg00257.html and here: http://www.erlang.org/ml-archive/erlang-questions/200406/msg00046.html and here: http://www.erlang.org/ml-archive/erlang-questions/200309/msg00382.html and here: http://www.erlang.org/ml-archive/erlang-questions/200309/msg00263.html and here: http://www.erlang.org/ml-archive/erlang-questions/200309/msg00219.html and here: http://www.erlang.org/ml-archive/erlang-questions/200204/msg00079.html Matthias From joe.armstrong@REDACTED Thu Mar 10 10:37:37 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Thu, 10 Mar 2005 10:37:37 +0100 Subject: An "is_string" BIF? Message-ID: > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Thomas Johnsson > XA (LN/EAB) > Sent: den 10 mars 2005 09:22 > To: erlang-questions@REDACTED > Subject: RE: An "is_string" BIF? > > > This really exposes an ugly wart on the language, in that > one cannot use any expression in a guard, making it impossible > to use abstractions there, e.g. > > glorbgraph(Graph) when mygraph:curiousproperty(Graph) -> ... > It's not an ugly wart - it's deliberate. Guards are merely extensions of pattern matching - they must be side effect free and capably of efficient compilation. If you allowed user defined guards people would soon write things like this: guard(a, P) -> P ! foo, false; guard(b, _) -> true. and use them foo(X, Y) when guard(X, Y) -> .... foo(X, Y) -> ... evaluating foo(a, P) would cause code in the second clause to trigger, but a side effect started in the first clause would also happen. The only way of forbidding this is either to delay any side effects in the user clause (ie delay Pid ! foo until the value of the guard is true) or restrict guards to side-effect free code. Guard tests are also inlined in the pattern matching algorithms in what should be an optimal manner. /Joe -- Thomas > > > > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Corrado Santoro > Sent: den 10 mars 2005 09:02 > To: erlang-questions@REDACTED > Subject: An "is_string" BIF? > > > Dear all, > > I need an "is_string" function, something that behaves as follows: > > is_string ("ABC") -> true > is_string ([65,66,67]) -> true > is_string (['1','2',three]) -> false > ... > > I know that I could do that by defining a suitable function, > but I would > like to use it in a guard. Is there a sort of BIF? > > Cheers, > --Corrado > > -- > From dietmar@REDACTED Thu Mar 10 12:09:37 2005 From: dietmar@REDACTED (Dietmar Schaefer) Date: Thu, 10 Mar 2005 12:09:37 +0100 Subject: How to use mnesia:subscribe Message-ID: <42302AF1.3080301@ast.dfs.de> Hi ! Actually - I need to know how (and where) to receive the event. Any sample code ?? regards Dietmar From hakan@REDACTED Thu Mar 10 15:54:55 2005 From: hakan@REDACTED (Hakan Mattsson) Date: Thu, 10 Mar 2005 15:54:55 +0100 (CET) Subject: How to use mnesia:subscribe In-Reply-To: <42302AF1.3080301@ast.dfs.de> References: <42302AF1.3080301@ast.dfs.de> Message-ID: On Thu, 10 Mar 2005, Dietmar Schaefer wrote: DS> Actually - I need to know how (and where) to receive the event. DS> DS> Any sample code ?? The events are sent as messages to the process that invokes mnesia:subscribe/1. See the Mnesia User's Guide about the event handling details: http://www.erlang.se/doc/doc-5.4.3/lib/mnesia-4.2/doc/html/Mnesia_chap5.html#5.7 /H?kan % erl Erlang (BEAM) emulator version 5.4.3 [source] [hipe] [threads:0] Eshell V5.4.3 (abort with ^G) 1> mnesia:start(). ok 2> mnesia:create_table(t, []). {atomic,ok} 3> mnesia:subscribe({table, t, simple}). {ok,nonode@REDACTED} 4> [mnesia:dirty_write({t, K, K * K}) || K <- lists:seq(1, 3)]. [ok,ok,ok] 5> mnesia:dirty_delete(t, 2). ok 6> flush(). Shell got {mnesia_table_event,{write,{t,1,1},{dirty,<0.30.0>}}} Shell got {mnesia_table_event,{write,{t,2,4},{dirty,<0.30.0>}}} Shell got {mnesia_table_event,{write,{t,3,9},{dirty,<0.30.0>}}} Shell got {mnesia_table_event,{delete,{t,2},{dirty,<0.30.0>}}} ok 7> From erlang@REDACTED Thu Mar 10 16:06:19 2005 From: erlang@REDACTED (Michael McDaniel) Date: Thu, 10 Mar 2005 07:06:19 -0800 Subject: MySQL Message-ID: <20050310150619.GS15629@fangora.autosys.us> I would like to use MySQL as the database backend to my Erlang program. I am using Erlang R10B-3 on x86 Linux. I would like to use MySQL v5.x (because a PHP program needs access to same data populated by Erlang program). I found a MySQL module written by Magnus Alhtorp at http://www.erlang-projects.org/Public/projects/libraries/mysql_erlang_module/view It appears that the password routines are failing with MySQL v5.x and I do not understand enough about Erlang or encryption routines to modify it. QUESTIONS: 1) Has anyone been successful in compiling the Erlang odbc module to work with MySQL? How did you do it? (I haven't been able to get it to compile). 2) Does anyone have an alternative method of using MySQL with Erlang? thank you for any assistance, ~Michael From erlang@REDACTED Thu Mar 10 16:24:39 2005 From: erlang@REDACTED (Michael McDaniel) Date: Thu, 10 Mar 2005 07:24:39 -0800 Subject: MySQL In-Reply-To: <20050310150619.GS15629@fangora.autosys.us> References: <20050310150619.GS15629@fangora.autosys.us> Message-ID: <20050310152439.GU15629@fangora.autosys.us> On Thu, Mar 10, 2005 at 07:06:19AM -0800, Michael McDaniel wrote: > I would like to use MySQL as the database backend to my Erlang program. I am using > Erlang R10B-3 on x86 Linux. I would like to use MySQL v5.x (because a PHP program > needs access to same data populated by Erlang program). > > I found a MySQL module written by Magnus Alhtorp at > http://www.erlang-projects.org/Public/projects/libraries/mysql_erlang_module/view > It appears that the password routines are failing with MySQL v5.x and I do not > understand enough about Erlang or encryption routines to modify it. > > QUESTIONS: > 1) Has anyone been successful in compiling the Erlang odbc module to work with MySQL? > How did you do it? (I haven't been able to get it to compile). > > 2) Does anyone have an alternative method of using MySQL with Erlang? > > > thank you for any assistance, > > ~Michael > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ P.S. I did find some archived messages regarding this subject, but as yet have not got Erlang <-> MySQL working. ~Michael From ingela@REDACTED Thu Mar 10 16:28:30 2005 From: ingela@REDACTED (Ingela Anderton) Date: Thu, 10 Mar 2005 16:28:30 +0100 Subject: MySQL References: <20050310150619.GS15629@fangora.autosys.us> Message-ID: <16944.26526.159145.628573@gargle.gargle.HOWL> Michael McDaniel wrote: [...] > QUESTIONS: > 1) Has anyone been successful in compiling the Erlang odbc module to work with MySQL? > How did you do it? (I haven't been able to get it to compile). Well I know odbc has been a pain in the as to compile. It was a legacy thing! Now I have finally found some time to write a configure script for odbc so hopefully it will be much less painful to compile odbc from the next open source release an on. -- /Ingela - OTP team From joe.armstrong@REDACTED Thu Mar 10 16:59:07 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Thu, 10 Mar 2005 16:59:07 +0100 Subject: Why no is_octet(N) guard? Message-ID: I disagree - please not more special purpose guards when a general mechanism will do and why just is_octet? what about is_digit, is_alpha_numberic, is_ascii is_unicode is_bananna? This is what macros are for -define(DIGIT(X), X >= $0, X =< $9). Then use it like this: foo(X) when ?DIGIT(X) -> ... /Joe > -----Original Message----- > From: peter [mailto:erlang@REDACTED] > Sent: den 10 mars 2005 10:51 > To: erlang-questions@REDACTED > Subject: Why no is_octet(N) guard? > > > > Now you need 3 guards to test if an argument is_octet: > > foo(N) when is_integer(N), 0= do_stuff; > foo(_) -> do_other_stuff. > > it would have been much nicer if this chould be written with > a is_octet() guard. > > foo(N) when is_octet(N) -> do_stuff; > foo(_) -> do_other_stuff. > > I find the need of the is_octet guard quite common, so why > not supply one extra > guard for this soon?Peter Lund > _________________________________________________________ > Sent using Mail2Forum (http://m2f.sourceforge.net) > From mats.cronqvist@REDACTED Thu Mar 10 17:20:50 2005 From: mats.cronqvist@REDACTED (Mats Cronqvist) Date: Thu, 10 Mar 2005 17:20:50 +0100 Subject: An "is_string" BIF? In-Reply-To: References: Message-ID: <423073E2.3030809@ericsson.com> i personally would also like an is_string guard, primarily becasue it would make much code a lot easier to read. i agree with joe that one should be very careful in introducing new guards. but surely "string" is a much more important special case than e.g. "bananna". mats Joe Armstrong (AL/EAB) wrote: > I disagree - please not more special purpose guards when a general mechanism will do > and why just is_octet? what about is_digit, is_alpha_numberic, is_ascii > is_unicode is_bananna? Corrado Santoro wrote: >>Dear all, >> >>I need an "is_string" function, something that behaves as follows: >> >>is_string ("ABC") -> true >>is_string ([65,66,67]) -> true >>is_string (['1','2',three]) -> false >>... >> >>I know that I could do that by defining a suitable function, >>but I would >>like to use it in a guard. Is there a sort of BIF? >> >>Cheers, >>--Corrado >> >>-- >> From richardc@REDACTED Thu Mar 10 19:37:02 2005 From: richardc@REDACTED (Richard Carlsson) Date: Thu, 10 Mar 2005 19:37:02 +0100 Subject: Parameterized/Abstract modules In-Reply-To: References: Message-ID: <423093CE.2030303@csd.uu.se> Raimo Niskanen wrote: > I.e. I want to have functions that can be called without being > parameterized (sp?). Yes, my intention was that you should be able to write -static([f/1, g/2]). but I never got around to implementing it. (It should be rather easy, though.) /Richard From kostis@REDACTED Thu Mar 10 22:03:46 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 10 Mar 2005 22:03:46 +0100 (MET) Subject: Why no is_octet(N) guard? In-Reply-To: Mail from '"Joe Armstrong \(AL/EAB\)" ' dated: Thu, 10 Mar 2005 16:59:07 +0100 Message-ID: <200503102103.j2AL3kbI019763@spikklubban.it.uu.se> Joe Armstrong wrote: > I disagree - please not more special purpose guards when a general mechanism > will do and why just is_octet? what about is_digit, is_alpha_numberic, > is_ascii, is_unicode, is_bananna? All this discussion is somehow unnecessary. Erlang at some point *should* be extended with the ability to allow arbitrary user-defined guards. Conservatively checking whether a (module-local for a start) function is side-effect free is not exactly rocket science... In any case, this is what I really want to reply to with my post: > This is what macros are for > > -define(DIGIT(X), X >= $0, X =< $9). Even disregarding for a moment that macros are not what a language should be forcing users to rely upon -- and that the Erlang preprocessor is not really the most well-though and flexible macro expansion system around -- the whole discussion about the is_string/1 guard really shows a limitation and the brain-damage of the is_list/1 guard. If the is_list/1 guard were really testing that something is a proper list, then one could simply write the is_string/1 guard as a macro with the currently available Erlang guards. Alas, it is just a very misleading name for is_cons_or_nil/1. IMO, this sucks. Best, Kostis From tobbe@REDACTED Thu Mar 10 22:02:52 2005 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: Thu, 10 Mar 2005 22:02:52 +0100 Subject: MySQL In-Reply-To: <20050310150619.GS15629@fangora.autosys.us> References: <20050310150619.GS15629@fangora.autosys.us> Message-ID: <4230B5FC.4040507@nortel.com> Michael McDaniel wrote: >I would like to use MySQL as the database backend to my Erlang program. I am using >Erlang R10B-3 on x86 Linux. I would like to use MySQL v5.x (because a PHP program >needs access to same data populated by Erlang program). > >I found a MySQL module written by Magnus Alhtorp at >http://www.erlang-projects.org/Public/projects/libraries/mysql_erlang_module/view >It appears that the password routines are failing with MySQL v5.x and I do not >understand enough about Erlang or encryption routines to modify it. > >QUESTIONS: >1) Has anyone been successful in compiling the Erlang odbc module to work with MySQL? > How did you do it? (I haven't been able to get it to compile). > >2) Does anyone have an alternative method of using MySQL with Erlang? > > >thank you for any assistance, > >~Michael > > > Well, you could try out this code that I wrote for the purpose of authenticating trapexit.org users: http://www.trapexit.org/emysql.erl http://www.trapexit.org/emysql.hrl and example of how to use it: http://www.trapexit.org/obfuscate.erl The implementation is based on this description: http://www.trapexit.org/MySQL-Protocol.html The code isn't really ready for release yet. My intention was to improve especially how data is retrieved from the MySQL server. This has been done in a better way by Magnus Ahltorp I belive, and you'll find the code here: http://www.stacken.kth.se/projekt/yxa/*mysql*-0.1.tar.gz Cheers, Tobbe From erlang@REDACTED Thu Mar 10 23:01:57 2005 From: erlang@REDACTED (Michael McDaniel) Date: Thu, 10 Mar 2005 14:01:57 -0800 Subject: MySQL In-Reply-To: <4230B5FC.4040507@nortel.com> References: <20050310150619.GS15629@fangora.autosys.us> <4230B5FC.4040507@nortel.com> Message-ID: <20050310220156.GF15629@fangora.autosys.us> On Thu, Mar 10, 2005 at 10:02:52PM +0100, Torbjorn Tornkvist wrote: > Michael McDaniel wrote: > > >I would like to use MySQL as the database backend to my Erlang program. I > >am using > >Erlang R10B-3 on x86 Linux. I would like to use MySQL v5.x (because a PHP > >program > >needs access to same data populated by Erlang program). > > > >I found a MySQL module written by Magnus Alhtorp at > >http://www.erlang-projects.org/Public/projects/libraries/mysql_erlang_module/view > >It appears that the password routines are failing with MySQL v5.x and I do > >not > >understand enough about Erlang or encryption routines to modify it. > > > >QUESTIONS: > >1) Has anyone been successful in compiling the Erlang odbc module to work > >with MySQL? > > How did you do it? (I haven't been able to get it to compile). > > > >2) Does anyone have an alternative method of using MySQL with Erlang? > > > > > >thank you for any assistance, > > > >~Michael > > > > > > > Well, you could try out this code that I wrote for the purpose of > authenticating trapexit.org users: > > http://www.trapexit.org/emysql.erl > http://www.trapexit.org/emysql.hrl > > and example of how to use it: > > http://www.trapexit.org/obfuscate.erl > > The implementation is based on this description: > > http://www.trapexit.org/MySQL-Protocol.html > > The code isn't really ready for release yet. My intention was > to improve especially how data is retrieved from the MySQL > server. This has been done in a better way by Magnus Ahltorp > I belive, and you'll find the code here: > > http://www.stacken.kth.se/projekt/yxa/*mysql*-0.1.tar.gz > > > Cheers, Tobbe -------------------------------------------------------------------- Thank you, Tobbe. I hope to work on this some more tonight. ~Michael From csanto@REDACTED Thu Mar 10 23:47:55 2005 From: csanto@REDACTED (Corrado Santoro) Date: Thu, 10 Mar 2005 23:47:55 +0100 Subject: Why no is_octet(N) guard? In-Reply-To: <200503102103.j2AL3kbI019763@spikklubban.it.uu.se> References: <200503102103.j2AL3kbI019763@spikklubban.it.uu.se> Message-ID: <1110494875.4230ce9b1fbfe@www.cdc.unict.it> > Joe Armstrong wrote: > > > I disagree - please not more special purpose guards when a general > mechanism > will do and why just is_octet? what about is_digit, > is_alpha_numberic, > > is_ascii, is_unicode, is_bananna? That's the point! And thus the definitive answer should be "let us allow user- defined functions in guards"! Quoting Kostis Sagonas : > All this discussion is somehow unnecessary. Erlang at some point > *should* be extended with the ability to allow arbitrary user-defined > guards. Conservatively checking whether a (module-local for a start) > function is side-effect free is not exactly rocket science... Well, my two cents. I know that there was already a discussion on these issues and I'll re-read it soon. When I'm invoking a BIF, I'm "calling a function"; when I'm invoking a user- defined function, I'm "calling a function": from the semantic point of view, there's no difference. So, if we stop the discussion here, there is no reason for not allowing the user-defined guards. But if we go further and talk about side-effects, it is clear that a user-defined guards *should not* cause any change of state. So we could think to something like: -module(test). -safe([my_guard/1]). my_guard (X) -> ... my_func (X) when my_guard (X) -> ... This could solves two problems: - the programmer is forced to understand that user-defined guards could be dangerous and thus explicitly signal "safe" guards; - the compiler must not analyze if a guard is user-defined, checking also its subsequent other function calls, searching for a possible side-effects. Now the issue is compiling user-defined guards, since they cannot be translated into matching op-codes in function heads. Proposals: 1. let the compiler to understand if the guard is a BIF or not, and behave accordingly; 2. introduce another reserved keyword for user-defined guards, something like: -module(test). -safe([my_guard/1]). my_guard (X) -> ... my_func (X) when is_atom(X) if my_guard (X) -> ... Why? Once again, the programmer is completely aware that there are two mechanisms in guards: BIF--side-effect free, fast and directly compiled--and user-defined functions--with side-effects, not directly compiled, etc.. BTW.... at this point, I do not understand why standard Erlang "if" cannot invoke user-defined functions: it is not a side-effect problem but I think that it's related to the fact that an "if" is compiled like a "when". Isn't it? IMHO, it is no so good to see something like: case my_func() of true -> ... false -> ... end, Ciao, --Corrado -- ====================================================== Eng. Corrado Santoro, Ph.D. University of Catania - Engineering Faculty Department of Computer Science and Telecommunications Engineering Viale A. Doria, 6 - 95125 CATANIA (ITALY) Tel: +39 095 7382380 Fax: +39 095 338280 +39 095 7382365 +39 095 7382364 EMail: csanto@REDACTED Personal Home Page: http://www.diit.unict.it/users/csanto NUXI Home Page: http://nuxi.iit.unict.it ====================================================== ------------------------------------------------- This mail sent through IMP: http://www.cdc.unict.it/ From ok@REDACTED Fri Mar 11 00:42:32 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 11 Mar 2005 12:42:32 +1300 (NZDT) Subject: Why no is_octet(N) guard? Message-ID: <200503102342.j2ANgWgF028320@atlas.otago.ac.nz> I note that my 'abstract patterns' proposal could handle pretty much any non-recursive side-effect free test you might want in a guard. Abstract patterns are guaranteed, *without* any cross-module analysis, to (1) have no side effects, and (2) terminate in finite time. Writing an abstract pattern for 'is_octet' is a no-brainer; writing an abstract pattern for 'is_string' is impossible, as it should be. Arguably, guards should look _less_ like function calls than they do, and instead of enlarging the set of names that could be used as guards and in function bodies may have been a mistake. From heinrich@REDACTED Fri Mar 11 07:31:29 2005 From: heinrich@REDACTED (Heinrich Venter) Date: Fri, 11 Mar 2005 08:31:29 +0200 Subject: MySQL Message-ID: Michael I am using the ODBC driver with erlang at the moment. Here is what I have found works I am assuming you are building OTP from source on Linux of some sort? After doing the "configure --enable-odbc" you should edit the resulting make file in the directory /lib/odbc/c_src/i686-pc-linux-gnu/Makefile I made the following changes: I commented out the test for TYPE and forced "TYPEMARKER = " #ifeq ($(TYPE),debug) #TYPEMARKER = .debug #else TYPEMARKER = #endif I removed the checks for Solaris MAKEFILE = makefiles/Makefile.unix.src #ifeq ($(OTP_RELEASE),yes) ifneq ($(BITS64),yes) #ifeq ($(findstring solaris,$(TARGET)),solaris) UNIX_TARGET = $(BINDIR)/odbcserver EXE_TARGET = $(UNIX_TARGET) #endif endif #endif endif Lastly I changed the ODBC library variables to what works on my RedHat system ODBCROOT = /usr ODBCLDFLAGS = -L$(ODBCROOT)/lib ODBCINCLUDE = $(ODBCROOT)/include After doing a "make" I cd lib/odbc/c_src and run make again. After this I do "make install" It is great news that Ingela is updating the configure script! -]-[einrich Get LookForMe for business using CellFind http://asp1.rocketseed.com/RocketSeed/mail/433a32353a313132373133353a33303639373a2d323a323138 This will take you to the Launchpad site for more info http://asp1.rocketseed.com/RocketSeed/mail/433a32353a313132373133353a33303639373a2d323a333738 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: MTk2LjMwLjc5LjE1NQ== Type: image/gif Size: 14964 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: MTk2LjMwLjc5LjE1NQ== Type: image/gif Size: 620 bytes Desc: not available URL: From thomas.xa.johnsson@REDACTED Fri Mar 11 10:35:27 2005 From: thomas.xa.johnsson@REDACTED (Thomas Johnsson XA (LN/EAB)) Date: Fri, 11 Mar 2005 10:35:27 +0100 Subject: Guards and side effects Message-ID: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB6@esealmw107.eemea.ericsson.se> (Subduing the hardcore functional programmer in me for a bit ... :-) I see really no compelling reason to prohibit side effects in guards, in fact there may well be sensible uses, for instance, communicating with another process to find out the value of the guard, consult ets tables or other data bases, call a memo function .... The only tricky place, as far as I can see, is guard evaluation in a receive: to evaluate guards to select a message in the own process in-queue, it makes sense *not* to allow another 'recursive' receive to find the value of the guard, but to issue a run time error instead. ( An aside: this is akin to black hole:ing in implementing a lazy functional language, where in the process of evaluating a 'thunk' you need the value of the same thunk) There are many opportunities for the programmer to shoot him/herself in the foot, but to hold the programmer in strict harness in this particular corner of the language, and nowhere else, I find that misguided. -- Thomas From ulf.wiger@REDACTED Fri Mar 11 11:08:48 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Fri, 11 Mar 2005 11:08:48 +0100 Subject: Guards and side effects Message-ID: > -----Original Message----- > From: Thomas Johnsson XA (LN/EAB) > Sent: den 11 mars 2005 10:35 > To: erlang-questions@REDACTED > Subject: Guards and side effects > > > (Subduing the hardcore functional programmer in me for a bit ... :-) > > I see really no compelling reason to prohibit side effects in guards, > in fact there may well be sensible uses, for instance, communicating > with another process to find out the value of the guard, > consult ets tables or other data bases, call a memo function .... I completely disagree. (-: I would like to see a way to declare new guard expressions, but think those guard expressions should be declarative and fall strictly within the realm of pattern matching. I think it is perfectly reasonable and sound to clearly differentiate between functions that modify data (and perhaps produce side-effects) and expressions that test the validity of data in a side-effect free manner. The purpose of user-defined guards should be to allow for more expressive (and/or more intuitive) match expressions, not to introduce new semantics. /Uffe From mickael.remond@REDACTED Fri Mar 11 11:24:20 2005 From: mickael.remond@REDACTED (Mickael Remond) Date: Fri, 11 Mar 2005 11:24:20 +0100 Subject: Guards and side effects In-Reply-To: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB6@esealmw107.eemea.ericsson.se> References: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB6@esealmw107.eemea.ericsson.se> Message-ID: <423171D4.5090001@erlang-fr.org> Thomas Johnsson XA (LN/EAB) wrote: > (Subduing the hardcore functional programmer in me for a bit ... :-) > > I see really no compelling reason to prohibit side effects in guards, > in fact there may well be sensible uses, for instance, communicating > with another process to find out the value of the guard, consult ets tables > or other data bases, call a memo function .... Getting information from a process has no side effect. I think there is a conceptual problem with side effect in guards. Functions clause are selected by guards, but what happens if the guard do real side-effect processing and the clause if finally not selected ? It will result in side effect generated bu clause that are different from the executed clause. In my opinion, this seems at least at first sight conter intuitive and can lead to program very hard to debug. -- Micka?l R?mond From vlad_dumitrescu@REDACTED Fri Mar 11 11:28:05 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Fri, 11 Mar 2005 11:28:05 +0100 Subject: Guards and side effects References: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB6@esealmw107.eemea.ericsson.se> Message-ID: Hi, From: "Thomas Johnsson XA (LN/EAB)" > There are many opportunities for the programmer to shoot him/herself in the foot, > but to hold the programmer in strict harness in this particular corner of the > language, and nowhere else, I find that misguided. Good point, I think. The language proper should not be restricted by what are mostly implementation/optimization issues. And extra power might open possibilities that now are closed. However, Erlang has a very pragmatic background. If the extra power means a noticeable slowdown in applications that don't use that power, it's not going to be easy to convince users it's good idea. Also, a more powerful language also needs more top-of-the-range developers. Not that the existing ones aren't good enough, but the new generation might be even more tempted choose Java, because all it's safeguards don't let them shoot themselves in the foot as easily [*] So IMHO it is a very difficult balance to keep. best regards, Vlad [*] More precisely, in Java it looks like it's more safe. One can do silly stuff there too :-) From osxroolz@REDACTED Fri Mar 11 11:54:02 2005 From: osxroolz@REDACTED (osxroolz@REDACTED) Date: Fri, 11 Mar 2005 02:54:02 -0800 Subject: Guards and side effects Message-ID: <1110538442.23603.218090774@webmail.messagingengine.com> I see no compelling reason to prohibit pointer manipulation in Erlang. Just imagine how many really useful things you could do if you could manipulate pointers. Communicate with other processes, re-examine garbage-collected data and cause behaviour in your program which you never imagined possible!!! This would be a real performance boaster. There are many opportunities for the programmer to shoot him/herself in the foot, but to hold the programmer in strict harness in this particular corner of the language, and nowhere else, I find that misguided. iMan --------- "Thomas Johnsson XA (LN/EAB)" wrote (Subduing the hardcore functional programmer in me for a bit ... :-) I see really no compelling reason to prohibit side effects in guards, in fact there may well be sensible uses, for instance, communicating with another process *** There are many opportunities for the programmer to shoot him/herself in the foot, but to hold the programmer in strict harness in this particular corner of the language, and nowhere else, I find that misguided. -- Thomas -- osxroolz@REDACTED From zoltan.peter.toth@REDACTED Fri Mar 11 12:17:57 2005 From: zoltan.peter.toth@REDACTED (Zoltan Peter Toth) Date: Fri, 11 Mar 2005 12:17:57 +0100 Subject: Optimized lists Message-ID: <42317E65.3020607@ericsson.com> Hi, Currently there's no length information stored in the list elements, so length/1 is an O(n) operation. (This is not so nice if an application wants to safeguard against operating on too long lists, but even length itself may last too long.) However, list handling BIFs seem to count as a fixed number of reductions, irrespectively of list length. (See http://www.erlang.org/ml-archive/erlang-questions/200502/msg00168.html) Idea: Store in each list element the length of the remaining part of the list. This way, length() would become an O(1) operation, [this may be a benefit if it's used frequently enough] but also, the list BIFs could count as variable number of reductions (based on length), keeping the real time behaviour. Tail recursion would also work, as the length information in the tail of the list would be correct too. (Currently I think it is possible to call a BIF on a long list that keeps the beam busy in that process so long that the net_ticktime elapses and other nodes lose contact with the node. Am I wrong ?) How feasible this would be to implement ? (The more ambitious solution would be that if a list BIF would take more than 1000 reductions (the limit for rescheduling) then it is scheduled out. This part may be tricky, though.) Cheers, Zoltan From tobbe@REDACTED Fri Mar 11 12:24:17 2005 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: Fri, 11 Mar 2005 12:24:17 +0100 Subject: Guards and side effects In-Reply-To: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB6@esealmw107.eemea.ericsson.se> References: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB6@esealmw107.eemea.ericsson.se> Message-ID: <42317FE1.1030809@nortel.com> Thomas Johnsson XA (LN/EAB) wrote: >(Subduing the hardcore functional programmer in me for a bit ... :-) > >I see really no compelling reason to prohibit side effects in guards, >in fact there may well be sensible uses, for instance, communicating >with another process to find out the value of the guard, consult ets tables >or other data bases, call a memo function .... > > I must say that I totally disagree here! Side-effects must be kept as few as possible! The hardest problem I find, when debugging code, are those that involves side-effects, i.e ets, process dict., message sending, etc To open up guards to be allowed to do this would be a nightmare. User defined guards which is guaranteed to be side-effect free may be ok, but then we have the termination problem of course. Cheers, Tobbe From mccratch@REDACTED Fri Mar 11 12:32:20 2005 From: mccratch@REDACTED (mccratch@REDACTED) Date: Fri, 11 Mar 2005 12:32:20 +0100 Subject: Guards and side effects In-Reply-To: References: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB6@esealmw107.eemea.ericsson.se> Message-ID: <20050311113220.GA6859@localhost.localdomain> On 2005-03-11 at 11:28:05 (+0100), Vlad Dumitrescu wrote: > Hi, > > From: "Thomas Johnsson XA (LN/EAB)" > > There are many opportunities for the programmer to shoot him/herself in the > foot, > > but to hold the programmer in strict harness in this particular corner of the > > language, and nowhere else, I find that misguided. > > Good point, I think. The language proper should not be restricted by what are > mostly implementation/optimization issues. And extra power might open > possibilities that now are closed. > > However, Erlang has a very pragmatic background. If the extra power means a > noticeable slowdown in applications that don't use that power, it's not going to > be easy to convince users it's good idea. Well the extra power is just "syntactic sugar" for something that is already possible in Erlang. Not being able to express some condition in guards just forces one to use another approach which might look ugly. In fact I think it would be really possible to introduce the extension (using arbitrary boolean expressions in guards) using the parse_transform mechanism for at least functions and if expressions (preserving the semantics of a receive statement I think is very difficult). It is just some work and might reduce the performance compared to an implementation in the compiler (of the generated code). Performance loss for old code should not occur, because a guard expression using only expressions allowed today in guards can be compiled as it is now. -- Cheers Matthias Kretschmer From luke@REDACTED Fri Mar 11 12:44:42 2005 From: luke@REDACTED (Luke Gorrie) Date: 11 Mar 2005 12:44:42 +0100 Subject: Guards and side effects References: Message-ID: "Ulf Wiger \(AL/EAB\)" writes: > I think it is perfectly reasonable and sound to clearly > differentiate between functions that modify data (and perhaps > produce side-effects) and expressions that test the validity > of data in a side-effect free manner. I'm with you, I like guards the way they are. I think the main issue is that guards look so nice: foo(X) when guard1(X) -> code1; foo(X) when guard2(X) -> code2; ... but if you need a non-guard test then it's way more verbose: foo(X) -> case test1(X) of true -> code1; false -> case test2(X) of true -> code2; false -> ... end end Seems a bit unnecessary. The related and bigger problem for me is code like: case foo() of {ok, F} -> case bar() of {ok, B} -> ... ; Err = {error, Reason} -> Err end; Err = {error, Reason} -> Err end I'm looking forward to trying out the new try-catch on this code once we manage to upgrade to R10. Cheers, Luke From vlad_dumitrescu@REDACTED Fri Mar 11 13:02:49 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Fri, 11 Mar 2005 13:02:49 +0100 Subject: Optimized lists References: <42317E65.3020607@ericsson.com> Message-ID: From: "Zoltan Peter Toth" > Idea: > Store in each list element the length of the remaining part of the list. The basic idea is thinkable, but I see one big disadvantage: strings are already stored in a bloated way (8 bytes per char). On the other hand, it may be useful to have in any case strings as a type or a nicer handling of binaries_as_strings. regards, Vlad From matthias@REDACTED Fri Mar 11 13:10:01 2005 From: matthias@REDACTED (Matthias Lang) Date: Fri, 11 Mar 2005 13:10:01 +0100 Subject: Optimized lists In-Reply-To: <42317E65.3020607@ericsson.com> References: <42317E65.3020607@ericsson.com> Message-ID: <16945.35481.531656.375088@antilipe.corelatus.se> Zoltan Peter Toth writes: > Idea: > Store in each list element the length of the remaining part of the list. [...] > How feasible this would be to implement ? Pretty straightforward: -module(zoltan). -compile([export_all]). from_list(List) -> from_list(lists:reverse(List), [], 0). from_list([], Acc, _) -> Acc; from_list([H|T], Acc, N) -> from_list(T, [{H, N}|Acc], N+1). to_list(List) -> [H || {H,_} <- List]. length([]) -> 0; length([{_, N}|_]) -> N + 1. I'm not sure many people will use it, given that it imposes even more space overhead than lists already have. Implementing it inside the runtime would be possible too, with likely space and performance gains. I'm sure a bunch of people would implement it right away, if only the Erlang source code wasn't kept a closely guarded secret. Matthias From thomas.xa.johnsson@REDACTED Fri Mar 11 13:14:31 2005 From: thomas.xa.johnsson@REDACTED (Thomas Johnsson XA (LN/EAB)) Date: Fri, 11 Mar 2005 13:14:31 +0100 Subject: Guards and side effects Message-ID: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB7@esealmw107.eemea.ericsson.se> The torch I threw in seems to have caught fire -- good! (:-) The 'Erlang way' is to do run time checks instead of compile time / static checks -- my argument is merely drawing that to it's logical conclusion. I wasn't advocating profligate use of side effects as a style of programming. But in cleaning up the language and thus allowing general expressions including calls of user defined functions in guards, one must also potenitally allow side effects there. With my examples I was also pointing out that here might be legitimate uses of side effects even with a declarative/functional thinking & style. Torbjorn Tornkvist: > The hardest problem I find, when debugging > code, are those that involves side-effects, i.e > ets, process dict., message sending, etc > To open up guards to be allowed to do this > would be a nightmare. I agree that these kinds of problems are often nightmarish, but I don't see how generalising guards noticeably worsens this nightmare. > User defined guards which is guaranteed to > be side-effect free may be ok, but then we > have the termination problem of course. A *runtime* check & crash when the expression evaluation is doing something fishy in e.g. guard evaluation in receive! -- Thomas From luke@REDACTED Fri Mar 11 13:22:57 2005 From: luke@REDACTED (Luke Gorrie) Date: 11 Mar 2005 13:22:57 +0100 Subject: Optimized lists References: <42317E65.3020607@ericsson.com> Message-ID: Zoltan Peter Toth writes: > Currently there's no length information stored in the list elements, so > length/1 is an O(n) operation. You might want to look in jungerl/lib/ermacs/src/cord.erl. This is an efficient string data structure built on a balanced tree (with ~2KB binaries for leaves). Length check is O(1), space is about one byte per character, and I've loaded 100MB log files with it. Boehm's idea. He used them for efficient immutable strings in C. Of course cords aren't IO-lists. From zoltan.peter.toth@REDACTED Fri Mar 11 13:47:36 2005 From: zoltan.peter.toth@REDACTED (Zoltan Peter Toth) Date: Fri, 11 Mar 2005 13:47:36 +0100 Subject: Optimized lists In-Reply-To: <16945.35481.531656.375088@antilipe.corelatus.se> References: <16945.35481.531656.375088@antilipe.corelatus.se> Message-ID: <42319368.4060808@ericsson.com> Matthias Lang wrote: > Zoltan Peter Toth writes: > > > Idea: > > Store in each list element the length of the remaining part of the > list. > [...] > > > How feasible this would be to implement ? > > Pretty straightforward: ... > I'm not sure many people will use it, given that it imposes even more > space overhead than lists already have. > > Implementing it inside the runtime would be possible too, with likely > space and performance gains. I'm sure a bunch of people would > implement it right away, if only the Erlang source code wasn't kept a > closely guarded secret. Hi Matthias, The Erlang solution is really pretty straightforward, and may be an option for new code that uses only the new list structure. But it would not solve the problem when we get a long conventional list that we don't want to handle and want to check its size with length(). Further on, transforming to/from conventional list would also be O(n). Therefore I propose a change to the implementation of the Erlang list construct in the emulator. (Afaik, currently the head structures are linked into a list and each of them has a pointer to the next head and the actual list element. What I propose is to include a length field into the head structure. So it would not consume too much extra memory I believe.) Cheers, Zoltan From mikael.karlsson@REDACTED Fri Mar 11 14:01:51 2005 From: mikael.karlsson@REDACTED (Mikael Karlsson) Date: Fri, 11 Mar 2005 14:01:51 +0100 Subject: Compiling NBAP ASN.1 signalling In-Reply-To: <42300287.4050704@erix.ericsson.se> References: <200503081626.05291.mikael.karlsson@creado.com> <200503091648.18899.mikael.karlsson@creado.com> <42300287.4050704@erix.ericsson.se> Message-ID: <200503111401.51305.mikael.karlsson@creado.com> torsdag 10 mars 2005 09:17 skrev Bertil Karlsson: > Hi, > > We (Kenneth and I) have been trying those specs and discovered a couple > of bugs in the asn1 compiler that I'm now going to fix. It should be > interesting to know if we run into the same errors as you did. > The compiler reported errors on constants in the NBAP-Constants.asn > module but somehow it passed the compiler. Yes > Then it failed on the > NBAP-IEs.asn module on some INTEGER constraints and the NBAP-IEs.asn, > NBAP-PDU-Contents.asn and NBUP-PDU-Descriptions.asn modules on some > information object handling. I noticed that NBAP-PDU-xxx.asn failed too. As well as NBAP-IEs.asn > Though, we have not seen any syntax errors in the ASN1 specs. No, they compile with other tools. I sent you Seans polished .asn files, as they did not seem to make it into the mailinglist (maybe because of the 600k attachements :-). I hope you can find out something from that as well. It would be really good to know if you can fix this. Thanks Mikael > > /Bertil > > Mikael Karlsson wrote: > > Great! > > > > thanks Sean. I get the problem in exactly > > the same place. Good to know you are not > > alone :-) > > > > Cheers > > Mikael > > > > onsdag 09 mars 2005 00:56 skrev Sean Hinde: > >>Hi, > >> > >>On 8 Mar 2005, at 15:26, Mikael Karlsson wrote: > >>>Hi, > >>> > >>>Has anybody had any success in compiling the ASN.1 > >>>signal specification part of www.3gpp.org > >>>Technical Spec 25.433 (V5.9.0) NBAP signalling? > >>>I am having problems (with the NBAP-IEs part.) > >> > >>I thought it would be fun to try (it beats watching the DVD of daughter > >>for the 50th time anyway :-) ) > >> > >>I got problems in the same file. > >> > >>I attach the files having removed the syntax errors introduced by the > >>copy/paste from MS Word to save anyone else working on this problem > >>having to do it. Beware, I have tweaked a couple of places in trying to > >>get it to work. "TODO" appears on every line I have changed, with the > >>original part in a comment From matthias@REDACTED Fri Mar 11 15:24:02 2005 From: matthias@REDACTED (Matthias Lang) Date: Fri, 11 Mar 2005 15:24:02 +0100 Subject: Optimized lists In-Reply-To: <42319368.4060808@ericsson.com> References: <16945.35481.531656.375088@antilipe.corelatus.se> <42319368.4060808@ericsson.com> Message-ID: <16945.43522.360854.204978@antilipe.corelatus.se> Zoltan Peter Toth writes: > The Erlang solution is really pretty straightforward, and may > be an option for new code that uses only the new list structure. > But it would not solve the problem when we get a long conventional > list that we don't want to handle and want to check its size with > length(). > Further on, transforming to/from conventional list would also be O(n). > Therefore I propose a change to the implementation of the Erlang > list construct in the emulator. Transforming a conventional list to one with the size in every element will cost O(n), even if the emulator does it for you. Or is the idea that the emulator be changed to _only_ allow tail-size-in-every-element lists? The benefit is that length() becomes O(1). The cost is O(n) space for all lists and O(n) extra execution time for most string operations. Matthias From zoltan.peter.toth@REDACTED Fri Mar 11 15:51:58 2005 From: zoltan.peter.toth@REDACTED (Zoltan Peter Toth) Date: Fri, 11 Mar 2005 15:51:58 +0100 Subject: Optimized lists In-Reply-To: <16945.43522.360854.204978@antilipe.corelatus.se> References: <16945.43522.360854.204978@antilipe.corelatus.se> Message-ID: <4231B08E.1080602@ericsson.com> Hi, > Or is the idea that the emulator be changed to _only_ allow > tail-size-in-every-element lists? Yes, that's the idea. Quite a change, I admit. > The benefit is that length() becomes > O(1). The cost is O(n) space for all lists and O(n) extra execution > time for most string operations. True, but for list BIFs the extra time would probably mean memcpy-ing a struct that's one word longer. I don't have any profiling information about execution time of BIFs, but I guess this would not be the dominant part on todays CPUs. It won't be noticable if list operations concern only one or a few elements (like prepending an element to the list). If it becomes substantially slower, then probably we have such a long list where it would be risky to call length(). The same for the O(n) space increase: if it really matters, then calling length() would be a real time problem. Further on, this would allow to count the BIF reductions in a length-dependent way, which requires to get the length in O(1) steps (or faster :)). Regards, Zoltan From mscandar@REDACTED Thu Mar 10 20:07:19 2005 From: mscandar@REDACTED (Mark Scandariato) Date: Thu, 10 Mar 2005 14:07:19 -0500 Subject: Parameterized/Abstract modules In-Reply-To: <423093CE.2030303@csd.uu.se> References: <423093CE.2030303@csd.uu.se> Message-ID: <42309AE7.3010309@cisco.com> Perhaps (along the lines of Vlad's suggestion yesterday) the compiler could notice that functions that don't reference the module's parameters (or THIS) should be exported as "static"? Mark. Richard Carlsson wrote: > Raimo Niskanen wrote: > >> I.e. I want to have functions that can be called without being >> parameterized (sp?). > > > Yes, my intention was that you should be able to write > > -static([f/1, g/2]). > > but I never got around to implementing it. (It should be > rather easy, though.) > > /Richard From svg@REDACTED Fri Mar 11 18:24:09 2005 From: svg@REDACTED (Vladimir Sekissov) Date: Fri, 11 Mar 2005 22:24:09 +0500 (YEKT) Subject: Flow-Based Programming of Paul Morrison Message-ID: <20050311.222409.15666815.svg@surnet.ru> Good day, http://www.jpaulmorrison.com/fbp/index.shtml http://www.jpaulmorrison.com/cgi-bin/wiki.pl I've found materials on this site interesting for me and very close to Erlang way. #| In "Flow-Based Programming" (FBP), applications are defined as networks of "black box" processes, which exchange data across predefined connections. These black box processes can be reconnected endlessly to form different applications without having to be changed internally. It is thus naturally component-oriented. |# Best Regards, Vladimir Sekissov From sean.hinde@REDACTED Fri Mar 11 20:42:38 2005 From: sean.hinde@REDACTED (Sean Hinde) Date: Fri, 11 Mar 2005 19:42:38 +0000 Subject: Flow-Based Programming of Paul Morrison In-Reply-To: <20050311.222409.15666815.svg@surnet.ru> References: <20050311.222409.15666815.svg@surnet.ru> Message-ID: <2cf064e2ca220ef6f108dfe58db28dbb@mac.com> On 11 Mar 2005, at 17:24, Vladimir Sekissov wrote: > Good day, > > http://www.jpaulmorrison.com/fbp/index.shtml > http://www.jpaulmorrison.com/cgi-bin/wiki.pl > > I've found materials on this site interesting for me and very close to > Erlang way. > > #| > In "Flow-Based Programming" (FBP), applications are defined as > networks of "black box" processes, which exchange data across > predefined connections. These black box processes can be reconnected > endlessly to form different applications without having to be changed > internally. It is thus naturally component-oriented. > |# The authors response to Erlang is also quite interesting. It seems the message did not quite get through (so to speak) http://www.jpaulmorrison.com/cgi-bin/wiki.pl?ErlangLanguage I took a quick look at it, and I have a few points: ? I have trouble picturing running millions of transactions a day on it, but maybe you don't need such large volumes. ? I believe FBP is not really a programming language, so much as a 'coordination' language - one of its goals is to be able to easily interface different languages to each other, rather than provide all the things you might want to do in a single language. ? Looks like ports would have to be added onto Erlang - although you can probably simulate them by passing process Ids around. ? Can FP support a full "2-dimensional" net - I know you said you thought this wouldn't be necessary, but I have my doubts. Click here [1] for an example. Sean From david.nospam.hopwood@REDACTED Fri Mar 11 21:07:30 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Fri, 11 Mar 2005 20:07:30 +0000 Subject: Guards and side effects In-Reply-To: <1110538442.23603.218090774@webmail.messagingengine.com> References: <1110538442.23603.218090774@webmail.messagingengine.com> Message-ID: <4231FA82.6030501@blueyonder.co.uk> osxroolz@REDACTED wrote: > I see no compelling reason to prohibit pointer manipulation in Erlang. > Just imagine how many really useful things you could do if > you could manipulate pointers. Communicate with other processes, > re-examine garbage-collected data and cause behaviour in your program > which you never imagined possible!!! This would be a real performance > booster. > There are many opportunities for the programmer to shoot him/herself > in the foot, but to hold the programmer in strict harness in this > particular corner of the language, and nowhere else, I find that > misguided. Your implied argument (that allowing arbitrary function guards would lead to adding bad features such as unrestricted pointer manipulation) is a variant of the "slippery slope" fallacy: No-one is suggesting adding unrestricted pointers. Unrestricted pointers substantially change the semantic model of the language, creating the possibility of undefined behaviour that can be manifested far away from the construct that caused it. Function guards may have confusing effects in some cases, but those effects occur immediately, and are equivalent to similar code written using the 'case' construct. If guards are specified to be evaluated as-if in order, then the behaviour is always well-defined (and deterministic if the guard functions are). Please stick to criticising the specific construct that is being proposed. -- David Hopwood From serge@REDACTED Fri Mar 11 21:47:35 2005 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 11 Mar 2005 15:47:35 -0500 Subject: pattern match Message-ID: <423203E7.3090807@hq.idt.net> Is there any difference in doing a pattern match on a parameter in the following implementations, or this is just a matter of programming style? f({a, _B} = Param) -> Param. or f(Param = {a, _B}) -> Param. Thanks. Serge From richardc@REDACTED Fri Mar 11 21:50:07 2005 From: richardc@REDACTED (Richard Carlsson) Date: Fri, 11 Mar 2005 21:50:07 +0100 Subject: Parameterized/Abstract modules In-Reply-To: <42309AE7.3010309@cisco.com> References: <423093CE.2030303@csd.uu.se> <42309AE7.3010309@cisco.com> Message-ID: <4232047F.1040706@csd.uu.se> Mark Scandariato wrote: > Perhaps (along the lines of Vlad's suggestion yesterday) the compiler > could notice that functions that don't reference the module's > parameters (or THIS) should be exported as "static"? The short answer is no. Consider the following: -module(foo, [A]). -export([f/1]). f(X) -> 1 + g(X). g(X) -> h(X) - 1. h(X) -> X + A. Neither f nor g directly access A (or THIS), so both would be candidates for the sort of thing you suggest. However, h/1 needs access to THIS (or at least to A), so that must be passed through g/1, and must enter the module via the exported f/1 (even though this calling convention is handled "under the hood" for parameterized modules, and you normally don't need to think about it). So, to find those functions that could be automatically exported as static (i.e., not taking an implicit THIS parameter), you'd have to take the transitive closure over the functions which could access THIS, and could then remove the unnecessary THIS parameter from all the remaining functions (if any). I had actually indended to implement this as an optimization, but I just didn't have the time. It would be all right for non-exported functions, since these are not part of the external interface. But doing it for exported functions would mean that a particular function could switch unpredictably back and forth between being "static" and being a "member", simply depending on whether or not some indirectly called function could need access to the THIS parameter. So I don't think it's a good idea to let the compiler decide this property. /Richard From erlang-list@REDACTED Fri Mar 11 22:04:24 2005 From: erlang-list@REDACTED (Dominic Williams) Date: Fri, 11 Mar 2005 22:04:24 +0100 Subject: Emacs mode In-Reply-To: References: <20050301201436.612EA46962@bang.trapexit.org> <4224D64C.9060104@diit.unict.it> Message-ID: <423207D8.1090106@dominicwilliams.net> Vlad Dumitrescu wrote: >> I'm using this in .emacs: >> (setq load-path (cons "/usr/local/lib/erlang/lib/tools-2.4/emacs" >> load-path)) then: > What I meant was that for example after installing R10B-3, the above > won't work, because the path should be .../tools-2.4.1/... This works for me: (set 'erlang-home "/opt/local/lib/erlang/") (set 'erlang-lib (concat erlang-home "lib/")) (set 'erlang-bin (concat erlang-home "bin/")) (set 'erlang-emacs-dir (concat erlang-lib (file-name-completion "tools-" erlang-lib) "emacs")) (set 'load-path (cons erlang-emacs-dir load-path)) (set 'exec-path (cons erlang-bin exec-path)) (require 'erlang-start) Regards, Dominic Williams http://www.dominicwilliams.net ---- From erlang-list@REDACTED Fri Mar 11 22:32:28 2005 From: erlang-list@REDACTED (Dominic Williams) Date: Fri, 11 Mar 2005 22:32:28 +0100 Subject: Poll: Packages use In-Reply-To: <20050307105831.96115.qmail@web41908.mail.yahoo.com> References: <20050307105831.96115.qmail@web41908.mail.yahoo.com> Message-ID: <42320E6C.3050006@dominicwilliams.net> Vlad Dumitrescu wrote: >* How many people use packages in their > applications? Not me. > * How many people plan to start using packages in 6 > months to one year from > now? Not me. My experience from other languages makes me reject the feature a priori. The whole concept of packages is conceptually flawed, IMO. 1) In order to avoid potential name clashes, which are in fact very rare, they pollute the code all the time. Far from encouraging longer, meaningful names, you still get short, cryptic names, but at the end of a long string of package names which usually have nothing to do with the design. 2) They just reduce the probability of a name clash, but these are still possible. So we still need a mecanism to resolve clashes. 3) Module names and distribution of functions among modules need to evolve over time, as the design evolves, so we need refactoring tools to easily rename modules throughout the code. 4) package implementations tend (as in Java and Erlang) to tie the code down to the file structure, whereas I am interested in going in the opposite direction (no files). What I want, and hope to provide some day is: 1) refactoring tools to facilitate renaming my own modules 2) an Eiffel-like facility to use, within my own code, third party modules /under a different name/, to resolve clashes when they actually occur. And yes, I am aware that in Erlang modules can be called using names that are computed at runtime, but that doesn't seem too huge an obstacle. Regards, Dominic Williams http://www.dominicwilliams.net ---- From ulf@REDACTED Fri Mar 11 22:46:13 2005 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 11 Mar 2005 22:46:13 +0100 Subject: pattern match In-Reply-To: <423203E7.3090807@hq.idt.net> References: <423203E7.3090807@hq.idt.net> Message-ID: Den 2005-03-11 21:47:35 skrev Serge Aleynikov : > Is there any difference in doing a pattern match on a parameter in the > following implementations, or this is just a matter of programming style? > > f({a, _B} = Param) -> > Param. > > or > > f(Param = {a, _B}) -> > Param. Please see the discussion started with http://www.erlang.org/ml-archive/erlang-questions/200410/msg00198.html The short answer: it's a matter of programming style. Personally, I would pick the first alternative. From the thread, you will learn who agrees or disagrees, and why. Regards, Uffe From erlang-list@REDACTED Fri Mar 11 23:01:36 2005 From: erlang-list@REDACTED (Dominic Williams) Date: Fri, 11 Mar 2005 23:01:36 +0100 Subject: An "is_string" BIF? In-Reply-To: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB1@esealmw107.eemea.ericsson.se> References: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB1@esealmw107.eemea.ericsson.se> Message-ID: <42321540.2030609@dominicwilliams.net> Thomas Johnsson XA (LN/EAB) wrote: > This really exposes an ugly wart on the language, in that > one cannot use any expression in a guard, ... I like guards the way they are. > ... making it impossible to use abstractions there, e.g. > > glorbgraph(Graph) when mygraph:curiousproperty(Graph) -> ... glorbgraph(Graph) -> glorbgraph(Graph, mygraph:curiousproperty(Graph)). glorbgraph(Graph,true) -> ...; glorbgraph(Graph,false) -> .... Regards, Dominic Williams http://www.dominicwilliams.net ---- From kostis@REDACTED Fri Mar 11 23:53:45 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 11 Mar 2005 23:53:45 +0100 (MET) Subject: An "is_string" BIF? In-Reply-To: Mail from 'Dominic Williams ' dated: Fri, 11 Mar 2005 23:01:36 +0100 Message-ID: <200503112253.j2BMrjNI006917@spikklubban.it.uu.se> Dominic Williams wrote: > I like guards the way they are. It is good to know you are a hard-core Erlang afficionado, but if the semantics of guards were as simple as the translation you describe below, there would not be a need to have guards in the first place. > > ... making it impossible to use abstractions there, e.g. > > > > glorbgraph(Graph) when mygraph:curiousproperty(Graph) -> ... > > glorbgraph(Graph) -> > glorbgraph(Graph, mygraph:curiousproperty(Graph)). > > glorbgraph(Graph,true) -> > ...; > glorbgraph(Graph,false) -> > .... Best, Kostis From osxroolz@REDACTED Sat Mar 12 12:16:37 2005 From: osxroolz@REDACTED (osxroolz@REDACTED) Date: Sat, 12 Mar 2005 03:16:37 -0800 Subject: Guards and side effects In-Reply-To: <4231FA82.6030501@blueyonder.co.uk> References: <1110538442.23603.218090774@webmail.messagingengine.com> <4231FA82.6030501@blueyonder.co.uk> Message-ID: <1110626197.20061.218160961@webmail.messagingengine.com> > Your implied argument (that allowing arbitrary function guards would > lead to adding bad features such as unrestricted pointer manipulation) > is a variant of the "slippery slope" fallacy: > You infer incorrectely. The implication is that the arguments the original poster presented in favour of his language change, i.e. * that he can't think of any compelling reason to prohibit it * that he feels that the restriction is a "strict harness" and is therefore "misguided" are insufficient since they can be used to argue in favour of just about anything. > Please stick to criticising the specific construct that is being proposed. Even more talk? Like Elvis, I want action not talk. Looking through the erlanguage mailing archive, much friday afternoon talk about language change, but seldom backed by a proof-of-concept implementation. Should not the burden be on the person proposing the changes to put in some work and present evidence, rather than demanding that others show reason why change is not worth implementing? iMan -- osxroolz@REDACTED From ulf@REDACTED Sat Mar 12 13:40:00 2005 From: ulf@REDACTED (Ulf Wiger) Date: Sat, 12 Mar 2005 13:40:00 +0100 Subject: Guards and side effects In-Reply-To: <1110626197.20061.218160961@webmail.messagingengine.com> References: <1110538442.23603.218090774@webmail.messagingengine.com> <4231FA82.6030501@blueyonder.co.uk> <1110626197.20061.218160961@webmail.messagingengine.com> Message-ID: Den 2005-03-12 12:16:37 skrev : > Looking through the erlanguage mailing archive, much friday afternoon > talk about language change, but seldom backed by a proof-of-concept > implementation. Should not the burden be on the person proposing the > changes to put in some work and present evidence, rather than demanding > that others show reason why change is not worth implementing? A nice example of this is Joe Armstrong's "bang-bang", i.e. the !! notation for a synchronous process interaction. Not only did he implement the language feature -- he wrote a GUI framework with lots of example code using it. Not that it was adopted anyway, but a nice example of someone actually walking the talk it was. (: /Uffe From ulf@REDACTED Sat Mar 12 15:08:35 2005 From: ulf@REDACTED (Ulf Wiger) Date: Sat, 12 Mar 2005 15:08:35 +0100 Subject: Sending a message to another node In-Reply-To: <20050312130244.C6CB7469C8@bang.trapexit.org> References: <20050312130244.C6CB7469C8@bang.trapexit.org> Message-ID: Den 2005-03-12 14:02:44 skrev : > (one@REDACTED)13> {pid(0,35,0), 'two@REDACTED'} ! hello2. > > > > =ERROR REPORT==== 12-Mar-2005::07:54:48 === > > Error in process <0.54.0> on node 'one@REDACTED' with exit value: > {badarg,[{erl_e > > val,eval_op,3},{erl_eval,expr,5},{shell,exprs,6},{shell,eval_loop,3}]} > > > Am I using a wrong syntax for sending messages to a known pid, or > referencing the pid incorrectly at node two? If I know a pid on node > two, is it possible to send a message to it using the ! command? You are using the wrong syntax to begin with, and you shouldn't ever use pid/3 except for debugging. {Process, Node} ! Message assumes that Process is a registered name on Node. As such, it must be an atom. The syntax for sending to a pid is always Pid ! Message, but if you call pid(0,35,0), this generates a _local_ pid (as indicated by the first digit, the "node index"). The way to make a remote pid known is to send it somehow in a message. You should never assume that a pid will always have the same "number", nor that a node index of a remote node is the same for all sessions. Basically, you shouldn't make any assumptions about the value of av pid. /Uffe -- Anv?nder Operas banbrytande e-postklient: http://www.opera.com/m2/ From mikael.karlsson@REDACTED Sat Mar 12 18:08:25 2005 From: mikael.karlsson@REDACTED (Mikael Karlsson) Date: Sat, 12 Mar 2005 18:08:25 +0100 Subject: Flow-Based Programming of Paul Morrison In-Reply-To: <2cf064e2ca220ef6f108dfe58db28dbb@mac.com> References: <20050311.222409.15666815.svg@surnet.ru> <2cf064e2ca220ef6f108dfe58db28dbb@mac.com> Message-ID: <200503121808.25598.mikael.karlsson@creado.com> There is another component based model, Fractal, that I think could be interesting to look at from an Erlang perspective: http://fractal.objectweb.org/ http://fractal.objectweb.org/specification/index.html Cheers Mikael Fri 11 mars 2005 20:42 Sean Hinde wrote: > On 11 Mar 2005, at 17:24, Vladimir Sekissov wrote: > > Good day, > > > > http://www.jpaulmorrison.com/fbp/index.shtml > > http://www.jpaulmorrison.com/cgi-bin/wiki.pl > > > > I've found materials on this site interesting for me and very close to > > Erlang way. > > > > #| > > In "Flow-Based Programming" (FBP), applications are defined as > > networks of "black box" processes, which exchange data across > > predefined connections. These black box processes can be reconnected > > endlessly to form different applications without having to be changed > > internally. It is thus naturally component-oriented. > > > > |# > > The authors response to Erlang is also quite interesting. It seems the > message did not quite get through (so to speak) > > http://www.jpaulmorrison.com/cgi-bin/wiki.pl?ErlangLanguage > > I took a quick look at it, and I have a few points: > ? I have trouble picturing running millions of transactions a day on > it, but maybe you don't need such large volumes. > ? I believe FBP is not really a programming language, so much as a > 'coordination' language - one of its goals is to be able to easily > interface different languages to each other, rather than provide all > the things you might want to do in a single language. > ? Looks like ports would have to be added onto Erlang - although you > can probably simulate them by passing process Ids around. > ? Can FP support a full "2-dimensional" net - I know you said you > thought this wouldn't be necessary, but I have my doubts. Click here > [1] for an example. > > > Sean From kent@REDACTED Mon Mar 14 00:42:40 2005 From: kent@REDACTED (Kent Boortz) Date: Mon, 14 Mar 2005 00:42:40 +0100 Subject: Problem in erlang interface In-Reply-To: (maruthavanan s.'s message of "Tue, 08 Mar 2005 10:29:42 +0000") References: Message-ID: "maruthavanan s" writes: > I want to establish communication between C node and Erlang node, for that > Im using ei_connect_init() function and ei_connect. There is no [problem in > establishing connection between two processes but when I tried to receive > message from C node using the library function ei_receive_msg(), the > function fails with value ERL_EXIT. In ei_receive_msg() I have used the > following parameters > > ei_xreceive_msg(sockfd,&msg,&buff) > > sockfd=obtained on establishing connection with C node > > msg is declared as erlang_msg msg; > > and buff is obtained by > > ei_x_buff buff; > ei_x_new(&buff); Might not give more info as it mostly trace the connect and disconnect phase, but you can set EI_TRACELEVEL in your environment to a numeric value, to get verbose information from the ei library about what is being sent and received, kent -- Kent Boortz, Senior Software Developer MySQL AB, www.mysql.com Office: +46 18 174400 ext. 4450 (VoIP) Office: +46 8 59091063 Mobile: +46 70 2791171 From kent@REDACTED Mon Mar 14 01:00:37 2005 From: kent@REDACTED (Kent Boortz) Date: Mon, 14 Mar 2005 01:00:37 +0100 Subject: tv not showing ets table In-Reply-To: (Anders Nygren's message of "Mon, 28 Feb 2005 10:23:45 -0600") References: Message-ID: Anders Nygren writes: > I create two ets tables with > ets:new(workers,[named_table,set,protected]), > ets:new(queue,[named_table,ordered_set,protected]), > > tv only shows worker as default. > If I change options to show system tables the table queue is shown, > with my process as the owner. > > So why does tv think that the table queue is a system table? The "system table" hiding to try not to mix your tables with tables that you didn't create. As there are no name spaces, prefixes or other way to know if it is a table used internally in the standard libraries or not, there is a static list of table names in "tv_main.hrl" that is used. This list isn't perfect and "queue" happen to be in the list. The solution to list the names statically is a "hack". If a list of names is to be maintained, maybe tv itself should have an ets table with these names to hide, so that you can both view and edit this list, kent -- Kent Boortz, Senior Software Developer MySQL AB, www.mysql.com Office: +46 18 174400 ext. 4450 (VoIP) Office: +46 8 59091063 Mobile: +46 70 2791171 From ok@REDACTED Mon Mar 14 03:20:16 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 14 Mar 2005 15:20:16 +1300 (NZDT) Subject: Guards and side effects Message-ID: <200503140220.j2E2KG2K060720@atlas.otago.ac.nz> "Vlad Dumitrescu" wrote: Good point, I think. The language proper should not be restricted by what are mostly implementation/optimization issues. And extra power might open possibilities that now are closed. What is allowed in guards is NEITHER an implementation NOR an optimisation issue. What's more, there are *NO* 'possibilities that are now closed' with the single exception of the 'receive' construct. Proof: the Haskell report shows how Haskell guards are nothing more than syntactic sugar and provides explicit transformations which can be used to eliminate guards entirely. This elimination does not introduce order-of-magnitude changes in efficiency or expressiveness. Indeed, many functional programming languages including ML and its relative have no guards at all. (Nor do Lisp and Scheme, including Kali-Scheme, a version of Scheme with some similarities to Erlang.) Erlang was influenced by Prolog and concurrent logic programming languages. There are technical reasons why committed choice concurrent LP languages like Parlog, GHC, and FCP place strong constraints on what you can say in a guard. In the absence of logical variables, not only guards but even pattern matching are entirely dispensable. In Haskell, the choice between using guards and using 'if' is entirely stylistic. That is NOT a necessary truth about all functional programming languages. In particular, for a language which is supposed to be useful for soft real time programming, it is useful to know that certain constructions MUST terminate and in bounded time. I repeat my earlier observation that the real problem is not that guards need to be more like expressions in Erlang but that they are already too MUCH like expressions, and should be restricted further. For example, length/1 should not be allowed in guards. All the uses of length/1 I've seen in guards could either be replaced by body code using length/1 (with an increase in clarity) OR by length where for example X length>= 2 is true if X = [_,_|_]. This replacement would in some cases yield a substantial increase in efficiency. Naturally, this construct would not\ be made available in expressions; guards and expressions *in Erlang* should be distinct. (This is not a general truth about all functional programming languages, any more than the converse is.) I also repeat that abstract patterns give you a way to have user-defined "predicates" for use in guards that don't break the basic rules, and are entirely safe to use in 'receive'. From ok@REDACTED Mon Mar 14 03:49:54 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 14 Mar 2005 15:49:54 +1300 (NZDT) Subject: Optimized lists Message-ID: <200503140249.j2E2nsB4066577@atlas.otago.ac.nz> Zoltan Peter Toth wrote: Idea: Store in each list element the length of the remaining part of the list. Better idea: if you have an algorithm where you need a sequence data type where determining the size is O(1), USE THE EXISTING DATA TYPE THAT'S SUITED TO THE JOB, namely tuples. This way, length() would become an O(1) operation, So what is the length of [1|2]? What are you going to store as its length? [this may be a benefit if it's used frequently enough] There are three bad consequences of the proposed change. (1) The Warren Abstract Machine stores list pairs in 2 words. I don't know what BEAM uses. Probably 3 words. The change would thus add 33% to the size of every list. This leads to - less effective use of cache. This has a bigger effect than people commonly realise. Here are some measurements from a 1GHz G4 PowerPC running MacOS 10.3 Level Sequential access Random access L1 cache 1.7 GB/sec 2.1 GB/sec L2 cache 1.0 GB/sec 0.7 GB/sec main 0.4 GB/sec 0.05 GB/sec And here are some measurements from a 500MHz SunBlade100 (UltraSparc II) running Solaris 2.10: Level Sequential access Random access L1 cache 2.8 GB/sec 1.6 GB/sec L2 cache 1.4 GB/sec 0.8 GB/sec main 0.3 GB/sec 0.04 GB/sec Adding an extra word to every list cell could be enough to push a program from L1 cache to L2 cache, cutting speed by a factor of 2, or from L2 cache to main memory (a factor of 10 or more). (The same 40:1 ratio between main memory speed and L1 cache explains why compiling everything to native code doesn't always pay off.) (2) Making a cons cell would no longer be a simple matter of bumping a register (the required check for overflow can be amortised over several allocations; I don't know whether it is or not) and filling in a couple of words. The length field has to be computed and stored as well. So the time to make *every* list goes up by 33% as well, maybe more. (3) There are plenty of list-processing functions which, to my understanding, don't involve tail recursion at the moment, but with a slightly different abstract machine *could* be tail recursions. Here's one: append([X|Xs], L) -> [X | append(Xs, L)]; append([], L) -> L. Making this and similar functions tail recursive would be very good news for memory use (using O(1) stack space instead of O(n)). Such code is _very_ common, much more common than length/1 is. Switching to tail recursion here would be IMPOSSIBLE if each list cell carried its length. Bad consequences (1) and (2) mean that length/1 would have to be vastly commoner than it is now to pay off; bad consequence (3) means that we'd be locked out of an improvement that WOULD be an improvement. but also, the list BIFs could count as variable number of reductions (based on length), keeping the real time behaviour. There's a much simpler way to do that. (A) Ban length/1 from guards, replacing it by length comparisons with slightly different semantics. (B) Turn any "BIF" that can traverse an entire list into plain Erlang code. Then all 'reduction counting', time-slicing, &c, would be accurate without any special machinery. By the way, there are data structures with O(1) concatenation and O(lgN) length. I think at least one such data structure has been implemented in Erlang. The space and time constant factors are much higher than ordinary Erlang lists, of course. Erlang lists are for use when you don't care about length/1 all that much. From ok@REDACTED Mon Mar 14 04:25:16 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 14 Mar 2005 16:25:16 +1300 (NZDT) Subject: Guards and side effects Message-ID: <200503140325.j2E3PGWT065272@atlas.otago.ac.nz> David Hopwood claimed that osxroolz@REDACTED's ... implied argument (that allowing arbitrary function guards would lead to adding bad features such as unrestricted pointer manipulation) is a variant of the "slippery slope" fallacy: There are two replies to that. The first is that while the slippery slope argument is not deductively sound, it is *empirically* amazingly reliable. It so happens, for example, that there have been proposals to add mutable data structures to Erlang. Right now there is a proposal to add traditional assignment statements to Prolog. I kid you not. Give people a taste of something familiar and pretty soon they *do* want the whole thing. Not deductively conclusive, but empirically pretty reliable. The second is that that's *not* how the implied argument worked, or at least not how I read it. The way I read it was "disproof by analogy", that is, showing that an argument has an unsound form. That is, "if this FORM of argument is invalid when X='pointer manipulation' then you can't rely on this form of argument when X='unrestricted guards'., The burden of substantive proof that the argument *is* sound in this case rests with the proposer." In short, not a slippery slope argument at all. No-one is suggesting adding unrestricted pointers. I *will* make a slippery slope argument here, but it's an inductive generalisation. Time after time after time, I have seen a demand that this or that programming language be extended to allow C pointers to be passed around. I have no proof that it _must_ happen, only the observation that it _does_ happen. (See manual(i-3-9) and manual(i-3-15-5) in Quintus Prolog 3.4 for a fairly benign version. For a rather less benign example, see the "structs" package in Quintus Prolog 3.4, manual(k-10), manual(g-8-3-2), manual(l-3-22), which basically gives you C-in-Prolog. I could provide similar examples for other languages.) Given that Erlang is not a lazy language like Haskell or Clean, it is far from clear to me that there is any advantage in adopting an aspect of their syntax. From richardc@REDACTED Mon Mar 14 07:41:18 2005 From: richardc@REDACTED (Richard Carlsson) Date: Mon, 14 Mar 2005 07:41:18 +0100 Subject: Optimized lists In-Reply-To: <200503140249.j2E2nsB4066577@atlas.otago.ac.nz> References: <200503140249.j2E2nsB4066577@atlas.otago.ac.nz> Message-ID: <4235320E.9050308@csd.uu.se> Richard A. O'Keefe wrote: > (1) The Warren Abstract Machine stores list pairs in 2 words. I don't > know what BEAM uses. Probably 3 words. The change would thus > add 33% to the size of every list. The BEAM uses 2 words per cons cell, so it would be a 50% addition, which of course would make it even more horrific. Folks, the whole reason that lists are as efficient as they are, is that the implementation is dead simple. Adding words to the representation, or having different kinds of lists under the hood (forcing a runtime test for each access to a cons cell to see what kind it really is) generally kills performance. /Richard From bjorn@REDACTED Mon Mar 14 08:45:44 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 14 Mar 2005 08:45:44 +0100 Subject: Optimized lists In-Reply-To: <200503140249.j2E2nsB4066577@atlas.otago.ac.nz> References: <200503140249.j2E2nsB4066577@atlas.otago.ac.nz> Message-ID: "Richard A. O'Keefe" writes: [...] > > (2) Making a cons cell would no longer be a simple matter of bumping a > register (the required check for overflow can be amortised over > several allocations; I don't know whether it is or not) and filling > in a couple of words. The length field has to be computed and stored > as well. So the time to make *every* list goes up by 33% as well, > maybe more. The overflow test IS amortised over several allocations in Beam. And as Richard C already noted in another mail, cons cell take up two words, so the extra cost would be 50%. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From micke@REDACTED Mon Mar 14 09:59:13 2005 From: micke@REDACTED (micke) Date: Mon, 14 Mar 2005 09:59:13 +0100 Subject: Grid Message-ID: <42358185@epostleser.online.no> Any work done on this ( see below ) using Erlang in Sweden ? http://www.alltheweb.com/search?cat=web&cs=utf8&q=%2Bsite% 3Ase+EGEE+Enabling+Grids+for+E-science+in+Europe&rys=0&_sb_lang=any From joe.armstrong@REDACTED Mon Mar 14 10:50:38 2005 From: joe.armstrong@REDACTED (Joe Armstrong (AL/EAB)) Date: Mon, 14 Mar 2005 10:50:38 +0100 Subject: Optimized lists Message-ID: > Currently there's no length information stored in the list > elements, so > length/1 is an O(n) operation. > (This is not so nice if an application wants to safeguard against > operating on too long lists, but even length itself may last > too long.) > However, list handling BIFs seem to count as a fixed number of > reductions, irrespectively of list length. > (See > http://www.erlang.org/ml-archive/erlang-questions/200502/msg00 168.html) >Idea: > Store in each list element the length of the remaining part of the list. > This way, length() would become an O(1) operation, > [this may be a benefit if it's used frequently enough] > but also, the list BIFs could count as variable number > of reductions (based on length), keeping the real time behaviour. > Tail recursion would also work, as the length information in the > tail of the list would be correct too. No no no. Whenever you write an algorithm you must have some idea in the back of you head as to the sizes of the data structures involved. The one-data-structure fits all approach is doomed to failure. If I want to write a program that manipulates a file I need to have some idea of the limitations of the program. If the file is s mall (a few meg) I'd do manipulations in memory, with file-at-a-time operations. If the file was large Tera Bytes then I'd do things on disk. Between the small and large file categories there are many varients and almost unlimited scope for imagination. If you need or want "a list and a length" then just make a tuple {List, Len} and maintain the invarient that len is the length of List. There is absolutly no point in making specialised data structures in excess of a small number of pre-defined and efficiently implemented primitive data structures. If you really want "very large lists" (like infinite lists, then the length gets too big to be stored :-) - so then you need to make a abstract data type for lazy lists etc. > (Currently I think it is possible to call a BIF on a long list that keeps > the beam busy in that process so long that the net_ticktime elapses and > other nodes lose contact with the node. Am I wrong ?) If you have to call a BIF on a very long list like this your algorithm might be wrong :-) > How feasible this would be to implement ? > (The more ambitious solution would be that if a list BIF would take more > than 1000 reductions (the limit for rescheduling) then it is scheduled out. > This part may be tricky, though.) But all processes get rescheduled after something like 500 reductions - not just ones that do things to lists ... /Joe From thomasl_erlang@REDACTED Mon Mar 14 11:23:02 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 14 Mar 2005 02:23:02 -0800 (PST) Subject: Flow-Based Programming of Paul Morrison In-Reply-To: 6667 Message-ID: <20050314102303.13187.qmail@web41904.mail.yahoo.com> --- Vladimir Sekissov wrote: > Good day, > > http://www.jpaulmorrison.com/fbp/index.shtml > http://www.jpaulmorrison.com/cgi-bin/wiki.pl > > I've found materials on this site interesting for me > and very close to > Erlang way. Seems interesting. Building programs at a higher level than processes and messages could be the next step beyond today's Erlang, in my mind. Another related approach (which was mentioned on the list a couple of weeks ago) is workflow processing. Though that's usually done in a context quite different from, say, telecoms. Best, Thomas __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From csanto@REDACTED Mon Mar 14 17:30:07 2005 From: csanto@REDACTED (Corrado Santoro) Date: Mon, 14 Mar 2005 17:30:07 +0100 Subject: Grid In-Reply-To: <42358185@epostleser.online.no> References: <42358185@epostleser.online.no> Message-ID: <4235BC0F.5040903@diit.unict.it> micke wrote: > Any work done on this ( see below ) using Erlang in Sweden ? I'm doing some research on grid computing, but it seems that no one uses Erlang for grids, even if it is a very attractive language for such large parallel and distributed systems. I used Erlang to write a simulator for resource allocation algorithm in a grid. And I exploited multiple process support for simulating a 400-nodes grid (a process for each node). It worked well! But, for a real environment, I think that we *must* accept the monstruosities of the Globus toolkit, with its XMLRPC/SOAP/WebServices, etc.... It is a standard, today :-( Ciao, --Corrado -- ====================================================== Eng. Corrado Santoro, Ph.D. University of Catania - Engineering Faculty Department of Computer Science and Telecommunications Engineering Viale A. Doria, 6 - 95125 CATANIA (ITALY) Tel: +39 095 7382144 Int. (5) 8035 +39 095 7382380 +39 095 7382365 +39 095 7382364 VoIP: sip:8035@REDACTED Fax: +39 095 7382397 EMail: csanto@REDACTED Personal Home Page: http://www.diit.unict.it/users/csanto NUXI Home Page: http://nuxi.iit.unict.it ====================================================== From yerl@REDACTED Mon Mar 14 18:32:49 2005 From: yerl@REDACTED (yerl@REDACTED) Date: Mon, 14 Mar 2005 17:32:49 +0000 Subject: www_tools-1.0 PATCH Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: www_tools-1.0.patch URL: From geoffw@REDACTED Mon Mar 14 20:52:13 2005 From: geoffw@REDACTED (Geoff White) Date: Mon, 14 Mar 2005 11:52:13 -0800 Subject: Erlang port of R10B-3 to OpenBSD 3.7-current Message-ID: <4235EB6D.50500@cybertribe.com> Folks, I have successfully finished a port of the latest Eralng language (R10B-3) to OpenBSD 3.7 (current) I would like to find people willing to test this port, it will save people who want to run erlang on OpenBSD (a great combination for router work), a lot of time and trouble. For those of you who don't us BSD systems, a "port" is equivalent to a .rpm in Linux systems. It allows one to simply cd to the /usr/ports/lang/erlang directory and type make && make install the erlang tar ball is downloaded, patched so it will build on OpenBSD, and installed. It can be removed with a simple pkg_delete command Erlang is the primary development language of DroidOS, a stripped down version of OpenBSD that runs on the soekris 4x01 series boxes used in SnortDroid (www.snortdroid.com) drop a line if you have questions or are interested in testing this port geoffw From dgud@REDACTED Tue Mar 15 13:13:29 2005 From: dgud@REDACTED (Dan Gudmundsson) Date: Tue, 15 Mar 2005 13:13:29 +0100 Subject: Parameterized/Abstract modules In-Reply-To: <423093CE.2030303@csd.uu.se> References: <423093CE.2030303@csd.uu.se> Message-ID: <16950.53609.556869.330022@rian.du.uab.ericsson.se> Shouldn't we turn it around then and have the "static"/"normal functions" in the ordinary export list and make a new list for the paramterized functions? i.e. -parameterized_export([init/0,foo/1...]). What do you think of the other suggestion, removing the autogenerated new/lenght(Params) and letting the coder call a bif when (s)he wants to create a new parameterized module? So we can overload the "constructor" function. /Dan Richard Carlsson writes: > Raimo Niskanen wrote: > > I.e. I want to have functions that can be called without being > > parameterized (sp?). > > Yes, my intention was that you should be able to write > > -static([f/1, g/2]). > > but I never got around to implementing it. (It should be > rather easy, though.) > > /Richard From anders.nygren@REDACTED Wed Mar 16 01:16:06 2005 From: anders.nygren@REDACTED (Anders Nygren) Date: Tue, 15 Mar 2005 18:16:06 -0600 Subject: How to do a takeover? Message-ID: Hi I have started looking at how to do a takeover. In my tests I discovered that I need to do a global:reregister_name to "move" the name of my server to the new node, and how to move some state information from the old to the new node. The way my current design works is that I have a number of gen_servers. When one server needs to call another server it spawns a worker that makes the call, so as to not block. This gives me the problem that I have a lot of workers that must terminated correctly before the takeover can be finalized. But I dont understand how to make a controlled transfer of messages in the mailbox, or linked worker processes. My best guess now is that I have to stop using gen_server:call to other servers and instead use an interface between my servers that always sends messages to processes with globally registered names. /Anders Nygren From james.hague@REDACTED Wed Mar 16 03:20:25 2005 From: james.hague@REDACTED (James Hague) Date: Tue, 15 Mar 2005 20:20:25 -0600 Subject: Debunking The Expensive Exported-Function Call Myth In-Reply-To: References: <200503021440.j22Ee4dU019737@spikklubban.it.uu.se> Message-ID: On 09 Mar 2005 17:51:29 +0100, Luke Gorrie wrote: > > then the optimized version (using Core Erlang syntax for > in registers syntax) is something like: > > foo() -> = bar(), ...code... > bar() -> <1, 2>. Very cool! I did not know about that! I think I erred in providing the easiest possible example to shoot down :) An alternate example is that you could determine the return type of a local function via "type discovery," then propagate that information to the calling function. That is, if you knew a local function always returned a list, then specialized code could be used to process that value after it is returned. James From james.hague@REDACTED Wed Mar 16 03:32:20 2005 From: james.hague@REDACTED (James Hague) Date: Tue, 15 Mar 2005 20:32:20 -0600 Subject: Guards and side effects In-Reply-To: <1110626197.20061.218160961@webmail.messagingengine.com> References: <1110538442.23603.218090774@webmail.messagingengine.com> <4231FA82.6030501@blueyonder.co.uk> <1110626197.20061.218160961@webmail.messagingengine.com> Message-ID: On Sat, 12 Mar 2005 03:16:37 -0800, osxroolz@REDACTED wrote: > > Even more talk? Like Elvis, I want action not talk. > > Looking through the erlanguage mailing archive, much friday afternoon > talk about language change, but seldom backed by a proof-of-concept > implementation. Should not the burden be on the person proposing the > changes to put in some work and present evidence, rather than demanding > that others show reason why change is not worth implementing? The reason here is simple. Erlang is a complex beast. I have submitted (and had accepted) small changes to the Erlang system. But changes to core language can be far from trivial. If it can be done as a parse transform or a straighforward compiler-level change, then that's one thing. But anything that requires a change to the BEAM emulator or a new BIF or whatnot, that's a big undertaking unless the person already understands the innards of the compiler and runtime. It's not like there's lots of documentation about the GC interactions of various functions in the emulator, for example. Note: I'm not the person who suggested this change :) James From casper2000a@REDACTED Wed Mar 16 07:43:20 2005 From: casper2000a@REDACTED (Casper) Date: Wed, 16 Mar 2005 12:43:20 +0600 Subject: Port driver memory free In-Reply-To: <422D935D.2040101@ericsson.com> Message-ID: <20050316063919.2B8CC4001C3@mail.omnibis.com> Hi All, There are 2 parameters, **rbuf and rlen in Port Driver entry "control. In PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in **rbuf. Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use that without creating a new ErlDrvBinary. My questions are, 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf without allocating a new ErlDrvBinary, should I still call driver_free_binary or driver_free to free that **rbuf memory before "control" function returns? 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new driver_binary and assign to **rbuf, what will happen to the previous 64 buffer which was in **rbuf? Will the port or erlang garbage collecter free that? Or do I need to free it inside the "control" function by calling driver_free_binary or driver_free? 3. I (2) above, what will be the case if I use driver_realloc or driver_realloc_binary? Will the previous 64 byes get released? 4. I (2) above, I still need to call driver_free_binary to free the newly created driver_binary inside "control" function, correct? 5. If I call "set_port_control_flags(dd->port, 0)" to set the port output to non-binary, do I need to free or clear the non-used space of **rbuf? 6. In above cased which free function, driver_free_binary or driver_free and which allocation function, driver_alloc, driver_alloc_binary, driver_realloc, driver_realloc_binary should I use? Thanks in advance! - Eranga From tobbe@REDACTED Wed Mar 16 09:29:18 2005 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: Wed, 16 Mar 2005 09:29:18 +0100 Subject: How to do a takeover? In-Reply-To: References: Message-ID: <4237EE5E.2020206@nortel.com> Anders Nygren wrote: >Hi >I have started looking at how to do a takeover. >In my tests I discovered that I need to do a global:reregister_name >to "move" the name of my server to the new node, and how to move some >state information from the old to the new node. > >The way my current design works is that I have a number of gen_servers. >When one server needs to call another server it spawns a worker that makes the >call, so as to not block. This gives me the problem that I have a lot of workers >that must terminated correctly before the takeover can be finalized. > >But I dont understand how to make a controlled transfer of messages in the >mailbox, or linked worker processes. > >My best guess now is that I have to stop using gen_server:call to other servers >and instead use an interface between my servers that always sends messages to >processes with globally registered names. > >/Anders Nygren > > This tutorial by Ulf Wiger, at least touches on the topic: http://www.trapexit.org/docs/howto/release_handling_tutorial.html Cheers, Tobbe From ulf.wiger@REDACTED Wed Mar 16 09:36:19 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Wed, 16 Mar 2005 09:36:19 +0100 Subject: How to do a takeover? Message-ID: One way to handle takeover is to use globally registered names, as you suggest. A reasonably simple way to coordinate transfer during takeover is to use the following sequence: (Context, application instance {A, N1} is supposed to take over from {A, N2}, where N1, N2 are node names.) 1) From e.g. a start_phase function, issue a gen_server call to processes in {A, N1} that should transfer global names and state. 2) In the handle_call for each process P, a) first call global:re_register_name() to move the name from {A,N2} to {A,N1}. This should prevent new requests from coming in to {P,N2} b) Issue a gen_server:call({P,N2}, takeover_state), which signals the process locally registered as P on node N2 to hand over the state. This message should be processed _after_ all external requests in {P,N2}, due to the FIFO semantics of gen_server. c) {P,N1} now has the global name and the state, and can return control to the gen_server. d) {P,N2} could perhaps relay all further messages to {P,N1} until it is terminated, but it shouldn't be strictly necessary. One thing to consider is that each "top application" is moved as one entity, and as soon as takeover is finished, the old instance is terminated. This can cause problems in applications that assume that they will always be on the same node (e.g. applications using SNMP). One way to address this is to introduce a wrapper application that includes all such applications. This has been done for many years in e.g. AXD 301. In AXD 301, this didn't solve all problems, so we made a special "distributed application controller" that is able to coordinate takeover of several applications in parallel. For the longest time, though, making one large application of included O&M applications worked just fine. /Uffe > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED]On Behalf Of Anders Nygren > Sent: den 16 mars 2005 01:16 > To: erlang-questions@REDACTED > Subject: How to do a takeover? > > > Hi > I have started looking at how to do a takeover. > In my tests I discovered that I need to do a global:reregister_name > to "move" the name of my server to the new node, and how to move some > state information from the old to the new node. > > The way my current design works is that I have a number of > gen_servers. > When one server needs to call another server it spawns a > worker that makes the > call, so as to not block. This gives me the problem that I > have a lot of workers > that must terminated correctly before the takeover can be finalized. > > But I dont understand how to make a controlled transfer of > messages in the > mailbox, or linked worker processes. > > My best guess now is that I have to stop using > gen_server:call to other servers > and instead use an interface between my servers that always > sends messages to > processes with globally registered names. > > /Anders Nygren > From thomasl_erlang@REDACTED Wed Mar 16 13:49:03 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 16 Mar 2005 04:49:03 -0800 (PST) Subject: Debunking The Expensive Exported-Function Call Myth In-Reply-To: 6667 Message-ID: <20050316124904.71761.qmail@web41906.mail.yahoo.com> --- James Hague wrote: > An alternate example is that you could determine the > return type of a > local function via "type discovery," then propagate > that information > to the calling function. That is, if you knew a > local function always > returned a list, then specialized code could be used > to process that > value after it is returned. Indeed. Type analysis has shown itself to be useful in other dynamic languages, so it might well be so in Erlang too. Type analysis has been done for donkey's years too; my introduction to its "serious use" was Peter Van Roy's thesis in 1990. Reasonably low-risk thus. For Erlang, there are two extra hurdles, both due to code loading. The first hurdle is that a function exported from a module can in principle be called with any argument in the future, since we can't say if code will be loaded in the future that calls an exported function. This obviously dilutes the available type information (used when optimizing the function body). The second hurdle is the converse, that a remote call can return anything or might not exist (since your analyzer again needs to be conservative in the face of code loading, and you don't know the future code of the called module). So function calls to other modules yield no information either. As an example, consider this: socket_info(Socket) -> case inet:peername(Socket) of {ok, {Address, Port}} -> case inet:gethostbyaddr(Address) of {ok, Hostent} -> ...; {error, Reason} -> ... end; {error, Reason} -> {error, Reason} end. Here, you can't say anything about what inet:peername/1 or inet:gethostbyaddr/1 will return since module inet may be replaced in the future. If get_peerhost/1 is exported, Socket could be anything too. When I experimented with type analysis for optimization at the start of the Hipe project (mid-90s), I came to the conclusion that a straightforward per-module type analysis was no better than a per-function type analysis on "real code" (OTP at the time). Bummer. There may be ways around that, of course. One approach is to deprecate export_all and so on, so that more code is local and analyzers may gain more information. That seems to be the road taken by Hipe and OTP, as far as I can tell. A second one is to use a more robust analysis framework, which permits multiple specialized versions of functions. While this is much harder to get right, it might be feasible. My preferred approach is a third one: module merging and profile-driven optimization on top of that, with type analysis as one part. See an Uppsala tech report from 1996 and my EUC 2001 paper for an overview, not including type analysis as such. If this third approach was used with the example above, I would expect the subsequent compiler to see something like this: %% latest(inet) checks that local definitions of %% inet:peername and inet:gethostbyaddr are still valid %% %% is_socket(Sock) tests that Sock is a socket/port, %% because profiling has shown that this is usually %% the case. %% %% local_inet_peername/1 and local_inet_gethostbyaddr/1 %% are made local and are now visible to the analyzer %% and optimizer. socket_info(Socket) when is_socket(Socket), latest(inet) -> case local_inet_peername(Socket) of {ok, {Address, Port}} -> case local_inet_gethostbyaddr(Address) of {ok, Hostent} -> ...; {error, Reason} -> ... end; {error, Reason} -> {error, Reason} end; (Note that this may, e.g., enable inlining or unboxing of the returned tuples too, as a follow-on optimization.) To end on a down-beat, there are certainly drawbacks to this "third way" too. Profile-based optimization is always more difficult to put into practical use. More infrastructure is needed to handle code loading. There may be more code in total than before due to merging and duplication (though hopefully the hot/important code is smaller). No doubt there are other drawbacks as well. Still, I consider it quite promising. Best, Thomas __________________________________ Do you Yahoo!? Make Yahoo! your home page http://www.yahoo.com/r/hs From thomasl_erlang@REDACTED Wed Mar 16 14:14:34 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 16 Mar 2005 05:14:34 -0800 (PST) Subject: Debunking The Expensive Exported-Function Call Myth In-Reply-To: <20050316124904.71761.qmail@web41906.mail.yahoo.com> Message-ID: <20050316131434.94039.qmail@web41907.mail.yahoo.com> --- Thomas Lindgren wrote: > As an example, consider this: > > socket_info(Socket) -> ... > If > get_peerhost/1 is exported, Socket could be anything > too. Um, that is, socket_info/1, not get_peerhost/1. Best, Thomas __________________________________ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail From vlad_dumitrescu@REDACTED Wed Mar 16 16:02:58 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Wed, 16 Mar 2005 16:02:58 +0100 Subject: Debunking The Expensive Exported-Function Call Myth References: <20050316124904.71761.qmail@web41906.mail.yahoo.com> Message-ID: From: "Thomas Lindgren" > For Erlang, there are two extra hurdles, both due to > code loading. Just a thought: one could envision a code loading system that would check the type signatures of exported functions and the types of the arguments to external calls from that module, and only accept values that fit with the existing ones. This would make Erlang almost statically typed, though, and also still has a problem: several interdependent modules should be possible to be reloaded at the same time even if type signatures are different, if those changes are only for "intra-cluster" calls. Solvable, but not easy at all. regards, Vlad From kostis@REDACTED Thu Mar 17 01:14:39 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 17 Mar 2005 01:14:39 +0100 (MET) Subject: Bugs in Erlang code Message-ID: <200503170014.j2H0EdKr026952@spikklubban.it.uu.se> [ Apologies for the plug for Dialyzer, but I am currently quite excited. ] Intro: As part of Dialyzer, we have been developing a type analysis for Erlang that is significantly different that the one the current version of Dialyzer is using. It is not totally usable yet, but it is very powerful. Stay tuned for its release! I am really amazed at some of the bugs we have been able to find in supposedly well-tested Erlang code using the new type analysis. Many of them, we've reported directly to the OTP team and they are already fixed. Some others we've finding every day as the analysis gets stronger and stronger. For example, today I've come across the following code (in kernel/src/rpc.erl). ======================================================================= yield(Key) when pid(Key) -> {value, R} = do_yield(Key, infinite), R. nb_yield(Key, infinite) when pid(Key) -> do_yield(Key, infinite); nb_yield(Key, Timeout) when pid(Key), integer(Timeout), Timeout >= 0 -> do_yield(Key, Timeout). nb_yield(Key) when pid(Key) -> do_yield(Key, 0). do_yield(Key, Timeout) -> receive {Key,{promise_reply,R}} -> {value,R} after Timeout -> timeout end. ========================================================================= There are actually two (and arguably three) errors in the above code! Dialyzer found them. QUIZ for the seasoned Erlang programmers: Who can spot them? Cheers, Kostis From alexey@REDACTED Thu Mar 17 02:58:17 2005 From: alexey@REDACTED (Alexey Shchepin) Date: Thu, 17 Mar 2005 03:58:17 +0200 Subject: Bugs in Erlang code In-Reply-To: <200503170014.j2H0EdKr026952@spikklubban.it.uu.se> (Kostis Sagonas's message of "Thu, 17 Mar 2005 01:14:39 +0100 (MET)") References: <200503170014.j2H0EdKr026952@spikklubban.it.uu.se> Message-ID: <87d5tzqbie.fsf@alex.sevcom.net> Hello, Kostis! On Thu, 17 Mar 2005 01:14:39 +0100 (MET), you said: KS> I am really amazed at some of the bugs we have been able to find in KS> supposedly well-tested Erlang code using the new type analysis. KS> Many of them, we've reported directly to the OTP team and they are already KS> fixed. Some others we've finding every day as the analysis gets stronger KS> and stronger. For example, today I've come across the following code (in KS> kernel/src/rpc.erl). KS> ======================================================================= KS> yield(Key) when pid(Key) -> KS> {value, R} = do_yield(Key, infinite), KS> R. do_yield can return 'timeout', so there can be 'badmatch' error, and second argument should be 'infinity'. KS> KS> nb_yield(Key, infinite) when pid(Key) -> KS> do_yield(Key, infinite); Again 'infinity'. KS> nb_yield(Key, Timeout) when pid(Key), integer(Timeout), Timeout >= 0 -> KS> do_yield(Key, Timeout). KS> KS> nb_yield(Key) when pid(Key) -> KS> do_yield(Key, 0). KS> KS> do_yield(Key, Timeout) -> KS> receive KS> {Key,{promise_reply,R}} -> KS> {value,R} KS> after Timeout -> KS> timeout KS> end. KS> ========================================================================= The rest looks ok for me. KS> There are actually two (and arguably three) errors in the above code! KS> Dialyzer found them. KS> QUIZ for the seasoned Erlang programmers: Who can spot them? From luke@REDACTED Thu Mar 17 05:15:03 2005 From: luke@REDACTED (Luke Gorrie) Date: 17 Mar 2005 05:15:03 +0100 Subject: Bugs in Erlang code References: <200503170014.j2H0EdKr026952@spikklubban.it.uu.se> Message-ID: Kostis Sagonas writes: > QUIZ for the seasoned Erlang programmers: Who can spot them? Interesting that bugs becomes obvious once you know they're there. I don't even think it's knowing the bugs exist that makes them obvious, it's just carefully reading the code - which one happens to rarely do except when there's known to be an obscure bug. I've got a stone-age dializer here: http://fresh.homeunix.net/~luke/misc/emacs/pbook.pdf http://fresh.homeunix.net/~luke/misc/emacs/pbook.el It takes time to use but it's time well spent when I can afford to. From bjorn@REDACTED Thu Mar 17 11:33:23 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 17 Mar 2005 11:33:23 +0100 Subject: Bugs in Erlang code In-Reply-To: <200503170014.j2H0EdKr026952@spikklubban.it.uu.se> References: <200503170014.j2H0EdKr026952@spikklubban.it.uu.se> Message-ID: Thanks! I have written the missing test cases for async_call/4 and friends, and corrected the two bugs ("infinity" instead of "infinite"). Matching against {value,R} in yield/1 is not a bug, since do_yield(Key, Atom) cannot return anything else than {value,R} (or crash if Atom is not infinity). /Bjorn P.S. Here is the updated code (which will be included in R10B-4). yield(Key) when is_pid(Key) -> {value,R} = do_yield(Key, infinity), R. nb_yield(Key, infinity=Inf) when is_pid(Key) -> do_yield(Key, Inf); nb_yield(Key, Timeout) when is_pid(Key), is_integer(Timeout), Timeout >= 0 -> do_yield(Key, Timeout). nb_yield(Key) when is_pid(Key) -> do_yield(Key, 0). do_yield(Key, Timeout) -> receive {Key,{promise_reply,R}} -> {value,R} after Timeout -> timeout end. Kostis Sagonas writes: > [ Apologies for the plug for Dialyzer, but I am currently quite excited. ] > > Intro: As part of Dialyzer, we have been developing a type > analysis for Erlang that is significantly different > that the one the current version of Dialyzer is using. > It is not totally usable yet, but it is very powerful. > Stay tuned for its release! > > > I am really amazed at some of the bugs we have been able to find > in supposedly well-tested Erlang code using the new type analysis. > > Many of them, we've reported directly to the OTP team and they are > already fixed. Some others we've finding every day as the analysis > gets stronger and stronger. For example, today I've come across > the following code (in kernel/src/rpc.erl). > > ======================================================================= > yield(Key) when pid(Key) -> > {value, R} = do_yield(Key, infinite), > R. > > nb_yield(Key, infinite) when pid(Key) -> > do_yield(Key, infinite); > nb_yield(Key, Timeout) when pid(Key), integer(Timeout), Timeout >= 0 -> > do_yield(Key, Timeout). > > nb_yield(Key) when pid(Key) -> > do_yield(Key, 0). > > do_yield(Key, Timeout) -> > receive > {Key,{promise_reply,R}} -> > {value,R} > after Timeout -> > timeout > end. > ========================================================================= > > There are actually two (and arguably three) errors in the above code! > > Dialyzer found them. > > QUIZ for the seasoned Erlang programmers: Who can spot them? > > Cheers, > Kostis > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From mats.cronqvist@REDACTED Thu Mar 17 12:19:52 2005 From: mats.cronqvist@REDACTED (Mats Cronqvist) Date: Thu, 17 Mar 2005 12:19:52 +0100 Subject: Optimized lists In-Reply-To: <200503140249.j2E2nsB4066577@atlas.otago.ac.nz> References: <200503140249.j2E2nsB4066577@atlas.otago.ac.nz> Message-ID: <423967D8.4010402@ericsson.com> Richard A. O'Keefe wrote: [..] > (1) The Warren Abstract Machine stores list pairs in 2 words. I don't > know what BEAM uses. Probably 3 words. The change would thus > add 33% to the size of every list. This leads to > - less effective use of cache. This has a bigger effect than > people commonly realise. Here are some measurements from a 1GHz > G4 PowerPC running MacOS 10.3 > > Level Sequential access Random access > L1 cache 1.7 GB/sec 2.1 GB/sec > L2 cache 1.0 GB/sec 0.7 GB/sec > main 0.4 GB/sec 0.05 GB/sec > > And here are some measurements from a 500MHz SunBlade100 (UltraSparc II) > running Solaris 2.10: > > Level Sequential access Random access > L1 cache 2.8 GB/sec 1.6 GB/sec > L2 cache 1.4 GB/sec 0.8 GB/sec > main 0.3 GB/sec 0.04 GB/sec in the AXD301 we have also found that cache is massively important. in one of our benchmarks a doubling of the cache size improved performance ~50% (not a typical value). indeed, we often see a loss of performance when we try hipe-compiling (our applications are not well-suited to hipe, and we lose on the cache). question; how did you make the measurements on memory access speeds? [...] > (3) There are plenty of list-processing functions which, to my understanding, > don't involve tail recursion at the moment, but with a slightly different > abstract machine *could* be tail recursions. Here's one: > > append([X|Xs], L) -> [X | append(Xs, L)]; > append([], L) -> L. > > Making this and similar functions tail recursive would be very good > news for memory use (using O(1) stack space instead of O(n)). Such code > is _very_ common, much more common than length/1 is. Switching to tail > recursion here would be IMPOSSIBLE if each list cell carried its length. what changes to the emulator would that be? mats From thomasl_erlang@REDACTED Thu Mar 17 15:34:28 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 17 Mar 2005 06:34:28 -0800 (PST) Subject: Optimized lists In-Reply-To: 6667 Message-ID: <20050317143428.41896.qmail@web41908.mail.yahoo.com> --- Mats Cronqvist wrote: > > (3) There are plenty of list-processing functions > which, to my understanding, > > don't involve tail recursion at the moment, > but with a slightly different > > abstract machine *could* be tail recursions. > Here's one: > > > > append([X|Xs], L) -> [X | append(Xs, L)]; > > append([], L) -> L. > > > > Making this and similar functions tail > recursive would be very good > > news for memory use (using O(1) stack space > instead of O(n)). Such code > > is _very_ common, much more common than > length/1 is. Switching to tail > > recursion here would be IMPOSSIBLE if each > list cell carried its length. > > > what changes to the emulator would that be? I can only chime in with Richard: getting tail recursion can be very advantageous, particularly in native code. Consider: each stack frame of append requires at least 2 words in itself (return address + saved X), which is 4 extra words of memory traffic per list element (store, load). Not to mention the tail recursive version requiring O(1) stack space rather than O(n). What is needed: a garbage collector that handles pointers from old to new data (there are plenty of existing ones that do), and a compiler that rewrites code to use tail recursion. Prolog might serve as inspiration. A source level example: append([X|Xs], L) -> [X | append(Xs, L)]; append([], L) -> L. would become something like this (or perhaps something a bit nicer): % wrapper append([], L) -> L; append([X|Xs], L) -> Lst = [X|empty], app(Xs, L, Lst), Lst. % inner loop, destructive update of tail of Lst app([X|Xs], L, Out_tail) -> New_out_tail = [X|empty], setcdr(Out_tail, New_out_tail), app(Xs, L, New_out_tail); app([], L, Lst) -> setcdr(Lst, L). That is, the last tail of the output list Lst is destructively updated in each iteration. The setcdr/2 operation can be implemented as a "trailed store", a not-taken test and a store in the usual case. In this case it's possible to speed this up even more, since writing the 'empty' in the tail of the list cell is only done for the sake of the GC :-) So if the GC can handle it, you might leave the tail of the cell uninitialized instead. (I guess this serves as testimony of a youth ill-spent with optimizing Prolog :-) The equivalent technique can be used for tuples as well. Best, Thomas __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From yerl@REDACTED Thu Mar 17 16:38:16 2005 From: yerl@REDACTED (yerl@REDACTED) Date: Thu, 17 Mar 2005 15:38:16 +0000 Subject: HTTP Proxy module in Erlang Message-ID: An HTML attachment was scrubbed... URL: From james.hague@REDACTED Thu Mar 17 16:04:41 2005 From: james.hague@REDACTED (James Hague) Date: Thu, 17 Mar 2005 09:04:41 -0600 Subject: Optimized lists In-Reply-To: <423967D8.4010402@ericsson.com> References: <200503140249.j2E2nsB4066577@atlas.otago.ac.nz> <423967D8.4010402@ericsson.com> Message-ID: > in the AXD301 we have also found that cache is massively important. in one of > our benchmarks a doubling of the cache size improved performance ~50% (not a > typical value). indeed, we often see a loss of performance when we try > hipe-compiling (our applications are not well-suited to hipe, and we lose on the > cache). Hmmm...makes me wonder if cdr-coding (or the like) would help out. It's traditionally not done in in Lisp systems, because you can modify existing cons cells in Lisp. But not in Erlang. You'd lose tag bits, though, which is a huge deal. I'd bet that someone looked into this a long time back :) From csanto@REDACTED Thu Mar 17 16:16:09 2005 From: csanto@REDACTED (Corrado Santoro) Date: Thu, 17 Mar 2005 16:16:09 +0100 Subject: HTTP Proxy module in Erlang In-Reply-To: References: Message-ID: <42399F39.7030305@diit.unict.it> yerl@REDACTED wrote: > Hi All! > I'm wondering if there is any HTTP Proxy module implementation in the > OTP tree or somewhere else. Any doc about how to create a simple HTTP > Proxy is very welcome? > Regards, > /ycrux > Do you need a "real web proxy system" or a library to write proxies? I have something for the latter case, i.e. a library that acts as a web proxy, also allowing to perform some user-defined manipulation of transiting data. Ciao, --C. -- ====================================================== Eng. Corrado Santoro, Ph.D. University of Catania - Engineering Faculty Department of Computer Science and Telecommunications Engineering Viale A. Doria, 6 - 95125 CATANIA (ITALY) Tel: +39 095 7382144 Int. (5) 8035 +39 095 7382380 +39 095 7382365 +39 095 7382364 VoIP: sip:8035@REDACTED Fax: +39 095 7382397 EMail: csanto@REDACTED Personal Home Page: http://www.diit.unict.it/users/csanto NUXI Home Page: http://nuxi.iit.unict.it ====================================================== From thomasl_erlang@REDACTED Thu Mar 17 16:42:20 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 17 Mar 2005 07:42:20 -0800 (PST) Subject: Optimized lists In-Reply-To: 6667 Message-ID: <20050317154220.93451.qmail@web41901.mail.yahoo.com> --- James Hague wrote: > Hmmm...makes me wonder if cdr-coding (or the like) > would help out. > It's traditionally not done in in Lisp systems, > because you can modify > existing cons cells in Lisp. But not in Erlang. > You'd lose tag bits, > though, which is a huge deal. I'd bet that someone > looked into this a > long time back :) Not that I know of, at least. (Well, vague hand-waving apart.) In my opinion, after having read most of what's been published on the topic, cdr coding is both quite interesting and quite a mess to implement. We might call it the femme fatale of language hackers?(*) Best, Thomas (*) Actually, I'd nominate compile-time reuse for that title. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From jay@REDACTED Thu Mar 17 16:53:09 2005 From: jay@REDACTED (Jay Nelson) Date: Thu, 17 Mar 2005 07:53:09 -0800 Subject: HTTP Proxy module in Erlang Message-ID: <4239A7E5.7040102@duomark.com> I wrote a tutorial on a TCP Proxy a while back. You can still read it at http://www.duomark.com/erlang/tutorials/proxy.html Part 3 has never been written. jay From raimo@REDACTED Thu Mar 17 09:16:34 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 17 Mar 2005 09:16:34 +0100 Subject: Port driver memory free References: <422D935D.2040101@ericsson.com>, <20050316063919.2B8CC4001C3@mail.omnibis.com> Message-ID: The short answer is: do not free! To elaborate: * In binary mode, you must return a binary: ErlDrvBinary *binp = driver_alloc_binary(len); /* Copy len bytes of data to to binp->orig_bytes */ *rbuf = (char *)binp; return len; The caller (the emulator) will increment the refcount and take care of deallocation when the binary is no longer used. * In list mode, if the supplied result buffer is too small, you should allocate a new result buffer using driver_alloc(size), and the caller (the emulator) will free it it has fetched the result from it. Otherwise just write the result in the supplied result buffer. The emulator compares the returned buffer address with the supplied buffer address to see if you return an allocated buffer. So, if you set *rbuf, it _must_ be to something allocated with driver_alloc(size), because it _will_ be freed with driver_free(*rbuf). You can of course return an allocated buffer for any size, also less than 64 bytes. casper2000a@REDACTED (Casper) writes: > Hi All, > > There are 2 parameters, **rbuf and rlen in Port Driver entry "control. In > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in **rbuf. > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use that > without creating a new ErlDrvBinary. My questions are, > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf without > allocating a new ErlDrvBinary, should I still call driver_free_binary or > driver_free to free that **rbuf memory before "control" function returns? > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > driver_binary and assign to **rbuf, what will happen to the previous 64 > buffer which was in **rbuf? Will the port or erlang garbage collecter free > that? Or do I need to free it inside the "control" function by calling > driver_free_binary or driver_free? > > 3. I (2) above, what will be the case if I use driver_realloc or > driver_realloc_binary? Will the previous 64 byes get released? > > 4. I (2) above, I still need to call driver_free_binary to free the newly > created driver_binary inside "control" function, correct? > > 5. If I call "set_port_control_flags(dd->port, 0)" to set the port output to > non-binary, do I need to free or clear the non-used space of **rbuf? > > 6. In above cased which free function, driver_free_binary or driver_free and > which allocation function, driver_alloc, driver_alloc_binary, > driver_realloc, driver_realloc_binary should I use? > > Thanks in advance! > - Eranga > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo@REDACTED Thu Mar 17 09:31:26 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 17 Mar 2005 09:31:26 +0100 Subject: Bugs in Erlang code References: <200503170014.j2H0EdKr026952@spikklubban.it.uu.se>, <200503170014.j2H0EdKr026952@spikklubban.it.uu.se>, <87d5tzqbie.fsf@alex.sevcom.net> Message-ID: If the second argument to do_yield is corrected to 'infinity' it will not return 'timeout', I would say that is not a bug, just mere ugly. So the bug is that 'infinity' is misspelled as 'infinite' in two places. alexey@REDACTED (Alexey Shchepin) writes: > Hello, Kostis! > > On Thu, 17 Mar 2005 01:14:39 +0100 (MET), you said: > > KS> I am really amazed at some of the bugs we have been able to find in > KS> supposedly well-tested Erlang code using the new type analysis. > > KS> Many of them, we've reported directly to the OTP team and they are already > KS> fixed. Some others we've finding every day as the analysis gets stronger > KS> and stronger. For example, today I've come across the following code (in > KS> kernel/src/rpc.erl). > > KS> ======================================================================= > KS> yield(Key) when pid(Key) -> > KS> {value, R} = do_yield(Key, infinite), > KS> R. > > do_yield can return 'timeout', so there can be 'badmatch' error, and second > argument should be 'infinity'. > > KS> > KS> nb_yield(Key, infinite) when pid(Key) -> > KS> do_yield(Key, infinite); > > Again 'infinity'. > > KS> nb_yield(Key, Timeout) when pid(Key), integer(Timeout), Timeout >= 0 -> > KS> do_yield(Key, Timeout). > KS> > KS> nb_yield(Key) when pid(Key) -> > KS> do_yield(Key, 0). > KS> > KS> do_yield(Key, Timeout) -> > KS> receive > KS> {Key,{promise_reply,R}} -> > KS> {value,R} > KS> after Timeout -> > KS> timeout > KS> end. > KS> ========================================================================= > > The rest looks ok for me. > > KS> There are actually two (and arguably three) errors in the above code! > > KS> Dialyzer found them. > > KS> QUIZ for the seasoned Erlang programmers: Who can spot them? > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From ok@REDACTED Fri Mar 18 00:01:57 2005 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 18 Mar 2005 12:01:57 +1300 (NZDT) Subject: Optimized lists Message-ID: <200503172301.j2HN1vuQ105331@atlas.otago.ac.nz> The question about CDR-coding came up. The experience Quintus had putting Prolog on the Xerox Lisp Machines (the Dandelion, Dandetiger, Daybreak &c: 1108, 1109, 1185, 1186) may be informative here. The Xerox machines ran Interlisp-D. It's the classic CDR-coding system. On this machine, the memory bus was 16 bits wide, and the address space' was 24 bits. A cons cell was 32 bits, holding a 24-bit CAR and an 8-bit CDR. The CDR codes included CDR is NIL CDR is a pointer to cell N on the current page CDR is the pointer _in_ cell N on the current page and there may have been an "I've moved to another address" case or not, I can't recall. They were able to divide up the word this way because they used the BiBOP (Big Bag Of Pages) technique for run time typing (_what_ a word is depends on _where_ it is) and because their garbage collector (a version of reference counting that assumes the reference count is 1 unless there is evidence to the contrary) put its associated information elsewhere. The Xerox machines were microcoded. They were the same machines that ran Mesa (with different microcode) for the Xerox Star office machines, and the same machines that ran Smalltalk (with different microcode). Xerox Quintus Prolog was given 4k of microcode space and 4MB of main address space. We wrote specifications of all our instructions in Interlisp, only to discover that the microcoder didn't read it. That worked out OK; Herb Jellinek translated for him. This was important, because any unimplemented instruction could trap out to Interlisp, so we could run with all our instructions in Lisp, all of them in microcode, or any mix, for debugging. Nice system. The big thing was that we just didn't have *space* in our block of microcode to implement CDR-coding, so we didn't. We did the traditional WAM thing of 2 pointers per cons cell, except we put the tag in the top 8 bytes where the microcode dispatch hardware could find it. So there were two list-processing languages on the _same_ hardware using similar levels of technology (non-optimisation to virtual instructions emulated in microcode) with the microcode written by the _same_ very clever microcoder. And Prolog did basic list processing about 5 times as fast as Lisp. A large part of that was Prolog's light-weight procedure calling mechanism (heck, benchmarks doing nothing but procedure calls ran _faster_ in WAM- emulated code than native C code did on Sun-2 computers...), but another large part was _not_ doing CDR coding. Lightweight function calls (including as much TRO as possible) and lightweight access to major data structures (excluding CDR coding). 5 times faster. From casper2000a@REDACTED Fri Mar 18 01:39:10 2005 From: casper2000a@REDACTED (casper2000a@REDACTED) Date: Fri, 18 Mar 2005 06:39:10 +0600 Subject: Port driver memory free In-Reply-To: References: <422D935D.2040101@ericsson.com>, <20050316063919.2B8CC4001C3@mail.omnibis.com> Message-ID: <1111106350.423a232eb157b@www.omnibis.com> Hi Raimo, Thanks for the detailed explaination. Still I have 2 questions. 1. In PORT_CONTROL_FLAG_BINARY mode, control function should return a binary. So I usually allocate a binary by calling driver_alloc_binary function. As per your explaination the emulater uses driver_free to deallocate it. Will this create a problem? (allocate using driver_alloc_binary and deallocate using driver_free) 2. Erlang Maual says at page ERTS->driver_entry->control "Note that this binary must be freed". - Eranga Quoting Raimo Niskanen : > Quoting Raimo Niskanen : > The short answer is: do not free! > > To elaborate: > * In binary mode, you must return a binary: > ErlDrvBinary *binp = driver_alloc_binary(len); > /* Copy len bytes of data to to binp->orig_bytes */ > *rbuf = (char *)binp; > return len; > The caller (the emulator) will increment the refcount and take care > of deallocation when the binary is no longer used. > * In list mode, if the supplied result buffer is too small, you > should allocate a new result buffer using driver_alloc(size), > and the caller (the emulator) will free it it has fetched the > result from it. Otherwise just write the result in the supplied > result buffer. The emulator compares the returned buffer address > with the supplied buffer address to see if you return an allocated > buffer. So, if you set *rbuf, it _must_ be to something allocated > with driver_alloc(size), because it _will_ be freed with > driver_free(*rbuf). You can of course return an allocated buffer > for any size, also less than 64 bytes. > > > > casper2000a@REDACTED (Casper) writes: > > > Hi All, > > > > There are 2 parameters, **rbuf and rlen in Port Driver entry \"control. In > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in **rbuf. > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use that > > without creating a new ErlDrvBinary. My questions are, > > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf without > > allocating a new ErlDrvBinary, should I still call driver_free_binary or > > driver_free to free that **rbuf memory before \"control\" function returns? > > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > > driver_binary and assign to **rbuf, what will happen to the previous 64 > > buffer which was in **rbuf? Will the port or erlang garbage collecter free > > that? Or do I need to free it inside the \"control\" function by calling > > driver_free_binary or driver_free? > > > > 3. I (2) above, what will be the case if I use driver_realloc or > > driver_realloc_binary? Will the previous 64 byes get released? > > > > 4. I (2) above, I still need to call driver_free_binary to free the newly > > created driver_binary inside \"control\" function, correct? > > > > 5. If I call \"set_port_control_flags(dd->port, 0)\" to set the port output > to > > non-binary, do I need to free or clear the non-used space of **rbuf? > > > > 6. In above cased which free function, driver_free_binary or driver_free > and > > which allocation function, driver_alloc, driver_alloc_binary, > > driver_realloc, driver_realloc_binary should I use? > > > > Thanks in advance! > > - Eranga > > > > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > --------------This mail sent through OmniBIS.com-------------- From raimo@REDACTED Fri Mar 18 09:00:06 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 18 Mar 2005 09:00:06 +0100 Subject: Port driver memory free References: <20050316063919.2B8CC4001C3@mail.omnibis.com>, , <1111106350.423a232eb157b@www.omnibis.com> Message-ID: 1. No, driver_free is used in list mode; directly after the result list has been created. In binary mode the binary is not freed, it is passed to the receiving process and will be properly freed once the garbage collector finds it is not used anymore. 2. I guess the manual is unclear or even wrong. Like for plain result buffer you cannot free until after you have returned, which is impossible, so therefore, conceptually, the emulator frees the result buffer - even when it is a binary. (I hope I do not have to eat this mail if that statement proves wrong) Generally, if you allocate a binary you will have to free it when you do not use it anymore, but in this special case of returning it as a result; if you want to hang on to it you will have to increment its refcount before you return. But _if_ you keep the binary after returning it - you can not change it because the emulator, compiler and all programmers assume that a term can not change value. As a special feature, from R10B, you can also return NULL instead of a binary, which will arrive as [] to the receiving erlang process which is handy when you do not want to return anything, and want to do it fast. casper2000a@REDACTED writes: > Hi Raimo, > > Thanks for the detailed explaination. Still I have 2 questions. > > 1. In PORT_CONTROL_FLAG_BINARY mode, control function should return a binary. So I usually > allocate a binary by calling driver_alloc_binary function. As per your explaination the emulater uses > driver_free to deallocate it. Will this create a problem? (allocate using driver_alloc_binary and > deallocate using driver_free) > > 2. Erlang Maual says at page ERTS->driver_entry->control "Note that this binary must be freed". > > - Eranga > > > > Quoting Raimo Niskanen : > Quoting Raimo Niskanen : > > > The short answer is: do not free! > > > > To elaborate: > > * In binary mode, you must return a binary: > > ErlDrvBinary *binp = driver_alloc_binary(len); > > /* Copy len bytes of data to to binp->orig_bytes */ > > *rbuf = (char *)binp; > > return len; > > The caller (the emulator) will increment the refcount and take care > > of deallocation when the binary is no longer used. > > * In list mode, if the supplied result buffer is too small, you > > should allocate a new result buffer using driver_alloc(size), > > and the caller (the emulator) will free it it has fetched the > > result from it. Otherwise just write the result in the supplied > > result buffer. The emulator compares the returned buffer address > > with the supplied buffer address to see if you return an allocated > > buffer. So, if you set *rbuf, it _must_ be to something allocated > > with driver_alloc(size), because it _will_ be freed with > > driver_free(*rbuf). You can of course return an allocated buffer > > for any size, also less than 64 bytes. > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > Hi All, > > > > > > There are 2 parameters, **rbuf and rlen in Port Driver entry \"control. In > > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in **rbuf. > > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use that > > > without creating a new ErlDrvBinary. My questions are, > > > > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf without > > > allocating a new ErlDrvBinary, should I still call driver_free_binary or > > > driver_free to free that **rbuf memory before \"control\" function returns? > > > > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > > > driver_binary and assign to **rbuf, what will happen to the previous 64 > > > buffer which was in **rbuf? Will the port or erlang garbage collecter free > > > that? Or do I need to free it inside the \"control\" function by calling > > > driver_free_binary or driver_free? > > > > > > 3. I (2) above, what will be the case if I use driver_realloc or > > > driver_realloc_binary? Will the previous 64 byes get released? > > > > > > 4. I (2) above, I still need to call driver_free_binary to free the newly > > > created driver_binary inside \"control\" function, correct? > > > > > > 5. If I call \"set_port_control_flags(dd->port, 0)\" to set the port output > > to > > > non-binary, do I need to free or clear the non-used space of **rbuf? > > > > > > 6. In above cased which free function, driver_free_binary or driver_free > > and > > > which allocation function, driver_alloc, driver_alloc_binary, > > > driver_realloc, driver_realloc_binary should I use? > > > > > > Thanks in advance! > > > - Eranga > > > > > > > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > --------------This mail sent through OmniBIS.com-------------- > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From casper2000a@REDACTED Fri Mar 18 09:08:00 2005 From: casper2000a@REDACTED (Casper) Date: Fri, 18 Mar 2005 14:08:00 +0600 Subject: Port driver memory free In-Reply-To: Message-ID: <20050318080336.86CE340001B@mail.omnibis.com> Hi, Today I noticed a funny thing with Port drivers "control" call. I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is (without reinitializing), since the space I need to output the binary message is less than 64 bytes. When I decode the received binary message in the emulator side, up to 7 bytes were exactly as I encoded in the C Port side. After that 8 position onward I get some other value. Then I created a new binary using driver_alloc_binary, even though the output (encoded, total) is less than 64 bytes and tried, and viola, the correct message came to emulator side. If that a bug or something I do wrong? Thanks! - Eranga -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen Sent: Thursday, March 17, 2005 2:17 PM To: erlang-questions@REDACTED Subject: Re: Port driver memory free The short answer is: do not free! To elaborate: * In binary mode, you must return a binary: ErlDrvBinary *binp = driver_alloc_binary(len); /* Copy len bytes of data to to binp->orig_bytes */ *rbuf = (char *)binp; return len; The caller (the emulator) will increment the refcount and take care of deallocation when the binary is no longer used. * In list mode, if the supplied result buffer is too small, you should allocate a new result buffer using driver_alloc(size), and the caller (the emulator) will free it it has fetched the result from it. Otherwise just write the result in the supplied result buffer. The emulator compares the returned buffer address with the supplied buffer address to see if you return an allocated buffer. So, if you set *rbuf, it _must_ be to something allocated with driver_alloc(size), because it _will_ be freed with driver_free(*rbuf). You can of course return an allocated buffer for any size, also less than 64 bytes. casper2000a@REDACTED (Casper) writes: > Hi All, > > There are 2 parameters, **rbuf and rlen in Port Driver entry "control. In > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in **rbuf. > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use that > without creating a new ErlDrvBinary. My questions are, > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf without > allocating a new ErlDrvBinary, should I still call driver_free_binary or > driver_free to free that **rbuf memory before "control" function returns? > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > driver_binary and assign to **rbuf, what will happen to the previous 64 > buffer which was in **rbuf? Will the port or erlang garbage collecter free > that? Or do I need to free it inside the "control" function by calling > driver_free_binary or driver_free? > > 3. I (2) above, what will be the case if I use driver_realloc or > driver_realloc_binary? Will the previous 64 byes get released? > > 4. I (2) above, I still need to call driver_free_binary to free the newly > created driver_binary inside "control" function, correct? > > 5. If I call "set_port_control_flags(dd->port, 0)" to set the port output to > non-binary, do I need to free or clear the non-used space of **rbuf? > > 6. In above cased which free function, driver_free_binary or driver_free and > which allocation function, driver_alloc, driver_alloc_binary, > driver_realloc, driver_realloc_binary should I use? > > Thanks in advance! > - Eranga > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo@REDACTED Fri Mar 18 09:31:20 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 18 Mar 2005 09:31:20 +0100 Subject: Port driver memory free References: , <20050318080336.86CE340001B@mail.omnibis.com> Message-ID: Yes, you do it wrong. The returned value _must_ be a binary, (or NULL for R10B or later). The code that takes care of the value assumes it is a binary, so you have introduced something in your system that is assumed to be a binary, but is actually stack allocated plain memory. The emulator will probably crash much later because memory has been trashed. You will get partially correct behaviour because the receiving code looks for the contents of the binary, and finds something there. Unfortunately the header is not correct and it has not been allocated with driver_alloc_binary(), but will later be freed with driver_free_binary(), and both the contents and header will be overwritten with garbage as soon as the emulator C stack grows. Do not do that - you will get hurt! casper2000a@REDACTED (Casper) writes: > Hi, > > Today I noticed a funny thing with Port drivers "control" call. > > I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is (without > reinitializing), since the space I need to output the binary message is less > than 64 bytes. > > When I decode the received binary message in the emulator side, up to 7 > bytes were exactly as I encoded in the C Port side. After that 8 position > onward I get some other value. > > Then I created a new binary using driver_alloc_binary, even though the > output (encoded, total) is less than 64 bytes and tried, and viola, the > correct message came to emulator side. > > If that a bug or something I do wrong? > > Thanks! > - Eranga > > > > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > Sent: Thursday, March 17, 2005 2:17 PM > To: erlang-questions@REDACTED > Subject: Re: Port driver memory free > > The short answer is: do not free! > > To elaborate: > * In binary mode, you must return a binary: > ErlDrvBinary *binp = driver_alloc_binary(len); > /* Copy len bytes of data to to binp->orig_bytes */ > *rbuf = (char *)binp; > return len; > The caller (the emulator) will increment the refcount and take care > of deallocation when the binary is no longer used. > * In list mode, if the supplied result buffer is too small, you > should allocate a new result buffer using driver_alloc(size), > and the caller (the emulator) will free it it has fetched the > result from it. Otherwise just write the result in the supplied > result buffer. The emulator compares the returned buffer address > with the supplied buffer address to see if you return an allocated > buffer. So, if you set *rbuf, it _must_ be to something allocated > with driver_alloc(size), because it _will_ be freed with > driver_free(*rbuf). You can of course return an allocated buffer > for any size, also less than 64 bytes. > > > > casper2000a@REDACTED (Casper) writes: > > > Hi All, > > > > There are 2 parameters, **rbuf and rlen in Port Driver entry "control. In > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in **rbuf. > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use that > > without creating a new ErlDrvBinary. My questions are, > > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf without > > allocating a new ErlDrvBinary, should I still call driver_free_binary or > > driver_free to free that **rbuf memory before "control" function returns? > > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > > driver_binary and assign to **rbuf, what will happen to the previous 64 > > buffer which was in **rbuf? Will the port or erlang garbage collecter free > > that? Or do I need to free it inside the "control" function by calling > > driver_free_binary or driver_free? > > > > 3. I (2) above, what will be the case if I use driver_realloc or > > driver_realloc_binary? Will the previous 64 byes get released? > > > > 4. I (2) above, I still need to call driver_free_binary to free the newly > > created driver_binary inside "control" function, correct? > > > > 5. If I call "set_port_control_flags(dd->port, 0)" to set the port output > to > > non-binary, do I need to free or clear the non-used space of **rbuf? > > > > 6. In above cased which free function, driver_free_binary or driver_free > and > > which allocation function, driver_alloc, driver_alloc_binary, > > driver_realloc, driver_realloc_binary should I use? > > > > Thanks in advance! > > - Eranga > > > > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From kostis@REDACTED Fri Mar 18 09:58:49 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 18 Mar 2005 09:58:49 +0100 (MET) Subject: Another bug in Erlang code Message-ID: <200503180858.j2I8wnwc029802@spikklubban.it.uu.se> Another QUIZ inspired by bugs identified by the new type inference. Who can spot the error in the following xmerl/src/xmerl.erl code? ============================================================================ %% @doc Exports simple XML content directly, without further context. export_simple_content(Data, Callback) when atom(Callback) -> export_content(xmerl_lib:expand_content(Data), callbacks(Callback)); export_simple_content(Data, Callbacks) when list(Callbacks) -> export_content(xmerl_lib:expand_element(Data), Callbacks). %% @spec export_content(Content, Callbacks) -> term() %% Content = [Element] %% Callback = [atom()] %% @doc Exports normal XML content directly, without further context. export_content([#xmlText{value = Text} | Es], CBs) -> [apply_text_cb(CBs, Text) | export_content(Es, CBs)]; export_content([#xmlPI{} | Es], CBs) -> export_content(Es, CBs); export_content([#xmlComment{} | Es], CBs) -> export_content(Es, CBs); export_content([#xmlDecl{} | Es], CBs) -> export_content(Es, CBs); export_content([E | Es], CBs) -> [export_element(E, CBs) | export_content(Es, CBs)]; export_content([], _CBs) -> []. ============================================================================ Hint: Just use your common sense from the information in function names. Cheers, Kostis PS. The maintainer of xmerl might want to get in touch with me. I have a decently-sized list of discrepancies. From richardc@REDACTED Fri Mar 18 11:02:01 2005 From: richardc@REDACTED (Richard Carlsson) Date: Fri, 18 Mar 2005 11:02:01 +0100 Subject: Another bug in Erlang code In-Reply-To: <200503180858.j2I8wnwc029802@spikklubban.it.uu.se> References: <200503180858.j2I8wnwc029802@spikklubban.it.uu.se> Message-ID: <423AA719.9090109@csd.uu.se> Kostis Sagonas wrote: > Another QUIZ inspired by bugs identified by the new type inference. > > Who can spot the error in the following xmerl/src/xmerl.erl code? > > export_simple_content(Data, Callback) when atom(Callback) -> > export_content(xmerl_lib:expand_content(Data), > callbacks(Callback)); > export_simple_content(Data, Callbacks) when list(Callbacks) -> > export_content(xmerl_lib:expand_element(Data), Callbacks). Mea culpa. I guess nobody's ever called export_simple_content with a list of callbacks, or this would have shown up. Just a copy/paste error; cf. the function export_simple_element/2, some lines below. Thanks for finding it. :-) /R From casper2000a@REDACTED Fri Mar 18 11:04:57 2005 From: casper2000a@REDACTED (Casper) Date: Fri, 18 Mar 2005 16:04:57 +0600 Subject: Port driver memory free In-Reply-To: Message-ID: <20050318100033.8143C40001B@mail.omnibis.com> Raimo, Actually I allocate a binary. Here's what I do, ----------------------- C Port ----------------------------- int gen_reply(char **rbuf, int rlen, int *rindex, void *buffer, int buflen) { ErlDrvBinary *drv_binary; char *p; *rindex = 0; ........ ........ for (p = NULL;;) { if (ei_encode_version(p, rindex) || ei_encode_tuple_header(p, rindex, 2) || ei_encode_atom(p, rindex, "ok") || ei_encode_binary(p, rindex, buffer, buflen)) { DBG("ei_encode failed"); return((int) ERL_DRV_ERROR_ERRNO); } if (p) break; //if (*rindex > rlen) { *rbuf = driver_alloc_binary(*rindex); //} drv_binary = (ErlDrvBinary *) *rbuf; p = drv_binary->orig_bytes; *rindex = 0; } drv_binary->orig_size = *rindex; ........ ---------------------------------------------------------- I had to comment out the "(*rindex > rlen)" part since when it was there, what I received in the emulator side was garbage after 8th char position. ------------------------- Emulator ----------------------- case catch binary_to_term(erlang:port_control(Port, Command, term_to_binary(Args))) of {'EXIT', Reason} -> ....... ---------------------------------------------------------- On the emulator side binary_to_term BIF didn't give any error. But once receive the term, the binary part contain incorrect data after 8th char pos. Thanks! - Eranga -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen Sent: Friday, March 18, 2005 2:31 PM To: erlang-questions@REDACTED Subject: Re: Port driver memory free Yes, you do it wrong. The returned value _must_ be a binary, (or NULL for R10B or later). The code that takes care of the value assumes it is a binary, so you have introduced something in your system that is assumed to be a binary, but is actually stack allocated plain memory. The emulator will probably crash much later because memory has been trashed. You will get partially correct behaviour because the receiving code looks for the contents of the binary, and finds something there. Unfortunately the header is not correct and it has not been allocated with driver_alloc_binary(), but will later be freed with driver_free_binary(), and both the contents and header will be overwritten with garbage as soon as the emulator C stack grows. Do not do that - you will get hurt! casper2000a@REDACTED (Casper) writes: > Hi, > > Today I noticed a funny thing with Port drivers "control" call. > > I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is (without > reinitializing), since the space I need to output the binary message is less > than 64 bytes. > > When I decode the received binary message in the emulator side, up to 7 > bytes were exactly as I encoded in the C Port side. After that 8 position > onward I get some other value. > > Then I created a new binary using driver_alloc_binary, even though the > output (encoded, total) is less than 64 bytes and tried, and viola, the > correct message came to emulator side. > > If that a bug or something I do wrong? > > Thanks! > - Eranga > > > > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > Sent: Thursday, March 17, 2005 2:17 PM > To: erlang-questions@REDACTED > Subject: Re: Port driver memory free > > The short answer is: do not free! > > To elaborate: > * In binary mode, you must return a binary: > ErlDrvBinary *binp = driver_alloc_binary(len); > /* Copy len bytes of data to to binp->orig_bytes */ > *rbuf = (char *)binp; > return len; > The caller (the emulator) will increment the refcount and take care > of deallocation when the binary is no longer used. > * In list mode, if the supplied result buffer is too small, you > should allocate a new result buffer using driver_alloc(size), > and the caller (the emulator) will free it it has fetched the > result from it. Otherwise just write the result in the supplied > result buffer. The emulator compares the returned buffer address > with the supplied buffer address to see if you return an allocated > buffer. So, if you set *rbuf, it _must_ be to something allocated > with driver_alloc(size), because it _will_ be freed with > driver_free(*rbuf). You can of course return an allocated buffer > for any size, also less than 64 bytes. > > > > casper2000a@REDACTED (Casper) writes: > > > Hi All, > > > > There are 2 parameters, **rbuf and rlen in Port Driver entry "control. In > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in **rbuf. > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use that > > without creating a new ErlDrvBinary. My questions are, > > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf without > > allocating a new ErlDrvBinary, should I still call driver_free_binary or > > driver_free to free that **rbuf memory before "control" function returns? > > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > > driver_binary and assign to **rbuf, what will happen to the previous 64 > > buffer which was in **rbuf? Will the port or erlang garbage collecter free > > that? Or do I need to free it inside the "control" function by calling > > driver_free_binary or driver_free? > > > > 3. I (2) above, what will be the case if I use driver_realloc or > > driver_realloc_binary? Will the previous 64 byes get released? > > > > 4. I (2) above, I still need to call driver_free_binary to free the newly > > created driver_binary inside "control" function, correct? > > > > 5. If I call "set_port_control_flags(dd->port, 0)" to set the port output > to > > non-binary, do I need to free or clear the non-used space of **rbuf? > > > > 6. In above cased which free function, driver_free_binary or driver_free > and > > which allocation function, driver_alloc, driver_alloc_binary, > > driver_realloc, driver_realloc_binary should I use? > > > > Thanks in advance! > > - Eranga > > > > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From luke@REDACTED Fri Mar 18 13:22:22 2005 From: luke@REDACTED (Luke Gorrie) Date: 18 Mar 2005 13:22:22 +0100 Subject: Another bug in Erlang code References: <200503180858.j2I8wnwc029802@spikklubban.it.uu.se> Message-ID: Kostis Sagonas writes: > Another QUIZ inspired by bugs identified by the new type inference. > > > Who can spot the error in the following xmerl/src/xmerl.erl code? For fun I blindly transliterated this code into Lisp to see if CMUCL's type inferencer would find the error. It did not, and the reason seems to be that in Lisp you can't propagate type information nearly so much as in Erlang and still get "no false positives". This suggests to me a very bright future for Dialyzer :-) The difference seems to be that in Erlang you can rely on individual functions within a module to stay the same so it's safe to propagate the inferred types of individual functions. In Lisp any single function can change by itself so you never know if there will /really/ be an error at runtime. The positive side for Lisp is that you still can propagate types that are explicitly delcared, the type checker is built into the compiler (used for optimization as well), and the user-interface is very convenient. If I add enough type declarations to my Lisp version to make the error detectable (lame to have to, I admit) then it's able to underline the offending expression the moment I compile the file: http://fresh.homeunix.net/~luke/misc/xx1.png and if I put the mouse over that expression then it prints a reasonably intelligible explanation at the bottom of the window: http://fresh.homeunix.net/~luke/misc/xx2.png I would love to see Dialyzer similarly integrated into the normal workflow (e.g. compiler) instead of being a separate linting tool. That way I would use it all the time, the same way everyone uses CMUCL's discrepency analyzer at all times. I also posted to the CMUCL list in case they have some ideas for doing more type propagation with this example as a motivation. Cool stuff! Cheers, Luke From thomasl_erlang@REDACTED Fri Mar 18 13:56:17 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Fri, 18 Mar 2005 04:56:17 -0800 (PST) Subject: Another bug in Erlang code In-Reply-To: 6667 Message-ID: <20050318125617.82891.qmail@web41906.mail.yahoo.com> --- Luke Gorrie wrote: > The difference seems to be that in Erlang you can > rely on individual > functions within a module to stay the same so it's > safe to propagate > the inferred types of individual functions. In Lisp > any single > function can change by itself so you never know if > there will /really/ > be an error at runtime. I recall seeing that CMUCL can do block compilation, which might be just the thing to use to get "modules". Though I haven't tried it myself. > The positive side for Lisp is that you still can > propagate types that > are explicitly delcared, the type checker is built > into the compiler > (used for optimization as well), and the > user-interface is very > convenient. Also note that several Lisp compilers are capable of very impressive performance if you add enough type declarations. There are occasional challenges of this sort on comp.lang.lisp for those interested. (The Lisp type system per se can be useful inspiration too.) > If I add enough type declarations to my Lisp version > to make the error > detectable (lame to have to, I admit) then it's able > to underline the > offending expression the moment I compile the file: DrScheme or MrSpidey or whatever it's called is also capable of that sort of thing, though for Scheme. Cf. soft typing. Best, Thomas __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From kostis@REDACTED Sun Mar 20 18:49:37 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Sun, 20 Mar 2005 18:49:37 +0100 (MET) Subject: Bugs in Erlang code -- a common one? Message-ID: <200503201749.j2KHnbTj018064@spikklubban.it.uu.se> Some of the bugs in Erlang code that we have discovered using the new type inference (albeit currently the hard way) are really subtle and extend beyond simple type clashes. For example, the code of snmp/src/compile/snmpc.erl at some point reads: ----------------------------------------------------------------------- define_cols(...) -> SIok = case SubIndex of {Parent,[_SI]} when Parent =/= NameOfEntry -> snmpc_lib:print_error( "Invalid parent ~p for table column ~p (should be ~p).", [Parent,NameOfCol,NameOfEntry],Oline), false; {NameOfEntry,[SI]} when SI =/= SubIndex -> snmpc_lib:print_error( "Invalid column number ~p for column ~p.", [SI,NameOfCol],Oline), false; {NameOfEntry,[SubIndex]} -> ok; .... ------------------------------------------------------------------------ and a moment's thought can reveal that there is something fishy here. By the way, this is the second occurrence of this type of error that we've found this week in OTP code (there was a similar one in gen_fsm). I am wondering how frequent is this type of error in Erlang code and whether this type of error can eventually be detected by the BEAM compiler. Best, Kostis. From ulf@REDACTED Mon Mar 21 02:49:11 2005 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 21 Mar 2005 02:49:11 +0100 Subject: Bugs in Erlang code -- a common one? In-Reply-To: <200503201749.j2KHnbTj018064@spikklubban.it.uu.se> References: <200503201749.j2KHnbTj018064@spikklubban.it.uu.se> Message-ID: Den 2005-03-20 18:49:37 skrev Kostis Sagonas : > For example, the code of snmp/src/compile/snmpc.erl at some point > reads: > > ----------------------------------------------------------------------- > define_cols(...) -> > SIok = case SubIndex of > {Parent,[_SI]} when Parent =/= NameOfEntry -> > snmpc_lib:print_error( > "Invalid parent ~p for table column ~p (should be > ~p).", > [Parent,NameOfCol,NameOfEntry],Oline), > false; > {NameOfEntry,[SI]} when SI =/= SubIndex -> > snmpc_lib:print_error( > "Invalid column number ~p for column ~p.", > [SI,NameOfCol],Oline), > false; > {NameOfEntry,[SubIndex]} -> > ok; > .... > ------------------------------------------------------------------------ > > and a moment's thought can reveal that there is something fishy here. Maybe it's because it's 2:30 am (don't ask), but revealing the fishiness required more than a moment's thought for me. (-: I will resist the urge to try to investigate what kind of code the compiler actually puts out. However... The clause you left out (in order to save space?) contained something that at after a moment's thought was a bit fishy too, but I will not fault Dialyzer for missing it: _Q -> snmpc_lib:print_error( "Invalid parent for column ~p.",[NameOfCol],Oline) end, In consideration of others who may be reading this list in the wee hours, I will save you the trouble of chasing the actual return value of snmpc_lib:print_error/3 -- it's 'ok'. I spent a few moments trying to figure out if that could have been deliberate. I decided it probably wasn't. Such implicit return values are usually to the detriment of the reader anywy. The ill effect in this case ought to be one error printout too many. /Uffe (now going to bed) From luke@REDACTED Mon Mar 21 03:28:22 2005 From: luke@REDACTED (Luke Gorrie) Date: 21 Mar 2005 03:28:22 +0100 Subject: Bugs in Erlang code -- a common one? References: <200503201749.j2KHnbTj018064@spikklubban.it.uu.se> Message-ID: Kostis Sagonas writes: > Some of the bugs in Erlang code that we have discovered using the > new type inference (albeit currently the hard way) are really > subtle and extend beyond simple type clashes. Interestingly the fishy thing that I see could also not be detected without false positives in Lisp. Functionalness pays off yet again! From bjorn@REDACTED Mon Mar 21 07:57:01 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 21 Mar 2005 07:57:01 +0100 Subject: Bugs in Erlang code -- a common one? In-Reply-To: <200503201749.j2KHnbTj018064@spikklubban.it.uu.se> References: <200503201749.j2KHnbTj018064@spikklubban.it.uu.se> Message-ID: Kostis Sagonas writes: > > By the way, this is the second occurrence of this type of error that > we've found this week in OTP code (there was a similar one in gen_fsm). > I am wondering how frequent is this type of error in Erlang code and > whether this type of error can eventually be detected by the BEAM compiler. > Since it has happened already twice, it seems like a good idea such a test to the compiler. We'll try to do it for R11B (or possibly for a future maintenance release of R10B). /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From dietmar@REDACTED Mon Mar 21 13:47:49 2005 From: dietmar@REDACTED (Dietmar Schaefer) Date: Mon, 21 Mar 2005 13:47:49 +0100 Subject: Mnesia problems Message-ID: <423EC275.8000304@ast.dfs.de> Hi I am trying to processing a table using DoAction = fun() -> mnesia:foldl(Action,[],bufferedMessages,write)end, The DoAction is a little bit too long and complicated to be listed here but maybe you know what's going on here ? I get: transaction DoAction aborted {badarg,[{ets,match_object,[377,{{locks,bufferedMessages,'______WHOLETABLE_____'},'_'}]},{mnesia_locker,need_lock,4},{mnesia_locker,rlock,3},{mnesia,read,5},{mnesia,do_foldl,8},{mnesia,foldl,6},{mnesia_tm,apply_fun,3},{mnesia_tm,execute_transaction,5},{cmmc_db,processBufferedMessages,0},{cmmc_com,decode,1}]} Regards Dietmar From geoffw@REDACTED Mon Mar 21 13:57:33 2005 From: geoffw@REDACTED (Geoff White) Date: Mon, 21 Mar 2005 04:57:33 -0800 Subject: R10B-3 install/porting question ecc? ear? elink? escript? Message-ID: <423EC4BD.9010406@cybertribe.com> Greetz, I am currently working on the port of R10B-3 to OpenBSD 3.7. I thought I had the port all but done, when I discovered (at the install phase) that I don't actually have executables for the following... ecc ear elink escript The Install; installs symbolic links that go to $(ERLANG_BINDIR) but these files don't seem to have been built and placed there. I can't find any evidence of them by searching in the distribution. I also don't see any mention of them in the R10B documentation area of the web site ( http://www.erlang.se/doc/doc-5.4.3/doc/ ). As I create the port, the missing files are flagged as errors and it is creating problems. Can anyone enlighten me as to what is going on, or what I'm doing wrong. Of course this is a BSD build environment cheers, geoffw From erlang@REDACTED Mon Mar 21 14:35:11 2005 From: erlang@REDACTED (Peter-Henry Mander) Date: Mon, 21 Mar 2005 13:35:11 +0000 Subject: R10B-3 install/porting question ecc? ear? elink? escript? In-Reply-To: <423EC4BD.9010406@cybertribe.com> References: <423EC4BD.9010406@cybertribe.com> Message-ID: <20050321133511.76b0a20a.erlang@manderp.freeserve.co.uk> Hi Geoff, The following thread may be useful to you. http://www.erlang.org/ml-archive/erlang-questions/200502/msg00206.html Pete. On Mon, 21 Mar 2005 04:57:33 -0800 Geoff White wrote: > Greetz, > I am currently working on the port of R10B-3 to OpenBSD 3.7. I thought I > had the port all but done, when I discovered (at the install phase) > that I don't actually have executables for the following... > > ecc > ear > elink > escript > > The Install; installs symbolic links that go to $(ERLANG_BINDIR) but > these files don't seem to have been built and placed there. I can't find > any evidence of them by searching in the distribution. I also don't > see any mention of them in the R10B documentation area of the web site ( > http://www.erlang.se/doc/doc-5.4.3/doc/ ). As I create the port, the > missing files are flagged as errors and it is creating problems. Can > anyone enlighten me as to what is going on, or what I'm doing wrong. Of > course this is a BSD build environment > > cheers, > > geoffw > > > -- "The Tao of Programming flows far away and returns on the wind of morning." From dgud@REDACTED Mon Mar 21 15:19:06 2005 From: dgud@REDACTED (Dan Gudmundsson) Date: Mon, 21 Mar 2005 15:19:06 +0100 Subject: Mnesia problems In-Reply-To: <423EC275.8000304@ast.dfs.de> References: <423EC275.8000304@ast.dfs.de> Message-ID: <16958.55258.777729.638337@rian.du.uab.ericsson.se> Hmm, that shouldn't happen, looks like a bug, can you email me the code or a snippet of it that reproduces the fault. /Dan Dietmar Schaefer writes: > Hi > > I am trying to processing a table using > > DoAction = fun() -> mnesia:foldl(Action,[],bufferedMessages,write)end, > > The DoAction is a little bit too long and complicated to be listed here > but maybe you know what's going > on here ? > > > I get: > > transaction DoAction aborted > {badarg,[{ets,match_object,[377,{{locks,bufferedMessages,'______WHOLETABLE_____'},'_'}]},{mnesia_locker,need_lock,4},{mnesia_locker,rlock,3},{mnesia,read,5},{mnesia,do_foldl,8},{mnesia,foldl,6},{mnesia_tm,apply_fun,3},{mnesia_tm,execute_transaction,5},{cmmc_db,processBufferedMessages,0},{cmmc_com,decode,1}]} > > > > Regards > > > Dietmar From yerl@REDACTED Mon Mar 21 17:35:17 2005 From: yerl@REDACTED (yerl@REDACTED) Date: Mon, 21 Mar 2005 16:35:17 +0000 Subject: R10B-3 install/porting question ecc? ear? elink? escript? Message-ID: An HTML attachment was scrubbed... URL: From james.hague@REDACTED Mon Mar 21 20:29:41 2005 From: james.hague@REDACTED (James Hague) Date: Mon, 21 Mar 2005 13:29:41 -0600 Subject: Should there be warnings about atoms in binary expressions? Message-ID: Currently you can have expressions like this: <> and <> = Binary where obviously it should have been Size and Type, but you don't get any warnings from the compiler. Feels like there should be warnings, given the myriad of warnings introduced in the last release of Erlang. James From kostis@REDACTED Mon Mar 21 20:56:43 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 21 Mar 2005 20:56:43 +0100 (MET) Subject: Should there be warnings about atoms in binary expressions? In-Reply-To: Mail from 'James Hague ' dated: Mon, 21 Mar 2005 13:29:41 -0600 Message-ID: <200503211956.j2LJuhci028716@spikklubban.it.uu.se> James Hague wrote: > ... > where obviously it should have been Size and Type, but you don't get > any warnings from the compiler. Feels like there should be warnings, > given the myriad of warnings introduced in the last release of Erlang. Yes, after a request from me, the OTP folks added this one. In R10B-4 you will get: ./foo.erl:5: Warning: binary construction will fail because of a type mismatch Stay tuned for its upcoming release! Kostis From erlang-list@REDACTED Mon Mar 21 22:16:58 2005 From: erlang-list@REDACTED (Dominic Williams) Date: Mon, 21 Mar 2005 22:16:58 +0100 Subject: Bug with module_info in R10B-3 Message-ID: <423F39CA.40409@dominicwilliams.net> Hello, The following bug appeared sometime after R9C, and is at least present in R10B-3 under Windows XP: when recompiling a modified module, its old exports appear to be kept by module_info. E.g., after compiling test.erl: ----------------- -module(test). -export([foo/0]). foo() -> ok. ----------------- test:module_info(exports) returns [{foo,0},{module_info,0},{module_info,1}] After modifying and recompiling test.erl thus: ----------------- -module(test). -export([bar/0]). bar() -> ok. ----------------- test:module_info(exports) now returns [{foo,0},{bar,0},{module_info,0},{module_info,1}] Regards, Dominic, on behalf of the www.xpdojo.org team. From casper2000a@REDACTED Tue Mar 22 02:44:41 2005 From: casper2000a@REDACTED (casper2000a@REDACTED) Date: Tue, 22 Mar 2005 07:44:41 +0600 Subject: Port driver memory free In-Reply-To: <005801c52e80$efed11f0$07f4f7dc@wavenet.lk> References: <005801c52e80$efed11f0$07f4f7dc@wavenet.lk> Message-ID: <1111455881.423f7889b745e@www.omnibis.com> Raimo, Actually I allocate a binary. Here's what I do, ----------------------- C Port ----------------------------- int gen_reply(char **rbuf, int rlen, int *rindex, void *buffer, int buflen) { ErlDrvBinary *drv_binary; char *p; *rindex = 0; ........ ........ for (p = NULL;;) { if (ei_encode_version(p, rindex) || ei_encode_tuple_header(p, rindex, 2) || ei_encode_atom(p, rindex, "ok") || ei_encode_binary(p, rindex, buffer, buflen)) { DBG("ei_encode failed"); return((int) ERL_DRV_ERROR_ERRNO); } if (p) break; //if (*rindex > rlen) { *rbuf = driver_alloc_binary(*rindex); //} drv_binary = (ErlDrvBinary *) *rbuf; p = drv_binary->orig_bytes; *rindex = 0; } drv_binary->orig_size = *rindex; ........ ---------------------------------------------------------- I had to comment out the "(*rindex > rlen)" part since when it was there, what I received in the emulator side was garbage after 8th char position. ------------------------- Emulator ----------------------- case catch binary_to_term(erlang:port_control(Port, Command, term_to_binary(Args))) of {'EXIT', Reason} -> ....... ---------------------------------------------------------- On the emulator side binary_to_term BIF didn't give any error. But once receive the term, the binary part contain incorrect data after 8th char pos. Thanks! - Eranga -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen Sent: Friday, March 18, 2005 2:31 PM To: erlang-questions@REDACTED Subject: Re: Port driver memory free Yes, you do it wrong. The returned value _must_ be a binary, (or NULL for R10B or later). The code that takes care of the value assumes it is a binary, so you have introduced something in your system that is assumed to be a binary, but is actually stack allocated plain memory. The emulator will probably crash much later because memory has been trashed. You will get partially correct behaviour because the receiving code looks for the contents of the binary, and finds something there. Unfortunately the header is not correct and it has not been allocated with driver_alloc_binary(), but will later be freed with driver_free_binary(), and both the contents and header will be overwritten with garbage as soon as the emulator C stack grows. Do not do that - you will get hurt! casper2000a@REDACTED (Casper) writes: > Hi, > > Today I noticed a funny thing with Port drivers \"control\" call. > > I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is (without > reinitializing), since the space I need to output the binary message is less > than 64 bytes. > > When I decode the received binary message in the emulator side, up to 7 > bytes were exactly as I encoded in the C Port side. After that 8 position > onward I get some other value. > > Then I created a new binary using driver_alloc_binary, even though the > output (encoded, total) is less than 64 bytes and tried, and viola, the > correct message came to emulator side. > > If that a bug or something I do wrong? > > Thanks! > - Eranga > > > > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > Sent: Thursday, March 17, 2005 2:17 PM > To: erlang-questions@REDACTED > Subject: Re: Port driver memory free > > The short answer is: do not free! > > To elaborate: > * In binary mode, you must return a binary: > ErlDrvBinary *binp = driver_alloc_binary(len); > /* Copy len bytes of data to to binp->orig_bytes */ > *rbuf = (char *)binp; > return len; > The caller (the emulator) will increment the refcount and take care > of deallocation when the binary is no longer used. > * In list mode, if the supplied result buffer is too small, you > should allocate a new result buffer using driver_alloc(size), > and the caller (the emulator) will free it it has fetched the > result from it. Otherwise just write the result in the supplied > result buffer. The emulator compares the returned buffer address > with the supplied buffer address to see if you return an allocated > buffer. So, if you set *rbuf, it _must_ be to something allocated > with driver_alloc(size), because it _will_ be freed with > driver_free(*rbuf). You can of course return an allocated buffer > for any size, also less than 64 bytes. > > > > casper2000a@REDACTED (Casper) writes: > > > Hi All, > > > > There are 2 parameters, **rbuf and rlen in Port Driver entry \"control. In > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in **rbuf. > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use that > > without creating a new ErlDrvBinary. My questions are, > > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf without > > allocating a new ErlDrvBinary, should I still call driver_free_binary or > > driver_free to free that **rbuf memory before \"control\" function returns? > > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > > driver_binary and assign to **rbuf, what will happen to the previous 64 > > buffer which was in **rbuf? Will the port or erlang garbage collecter free > > that? Or do I need to free it inside the \"control\" function by calling > > driver_free_binary or driver_free? > > > > 3. I (2) above, what will be the case if I use driver_realloc or > > driver_realloc_binary? Will the previous 64 byes get released? > > > > 4. I (2) above, I still need to call driver_free_binary to free the newly > > created driver_binary inside \"control\" function, correct? > > > > 5. If I call \"set_port_control_flags(dd->port, 0)\" to set the port output > to > > non-binary, do I need to free or clear the non-used space of **rbuf? > > > > 6. In above cased which free function, driver_free_binary or driver_free > and > > which allocation function, driver_alloc, driver_alloc_binary, > > driver_realloc, driver_realloc_binary should I use? > > > > Thanks in advance! > > - Eranga > > > > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB --------------This mail sent through OmniBIS.com-------------- From geoffw@REDACTED Tue Mar 22 03:09:51 2005 From: geoffw@REDACTED (Geoff White) Date: Mon, 21 Mar 2005 18:09:51 -0800 Subject: Essential patches to R10B-3? Message-ID: <423F7E6F.4020001@cybertribe.com> Are there any esssential patches that I should apply to R10B-3 for my port to OpenBSD? I have it building correctly for OpenBSD 3.7 and installing properly, all that is left to test is make uninstall. So now I'd like to know if there are any "show stopper" patches that really should be applied to R10B-3 (or should I just release this and wait for R10B-4 or more) geoffw From vances@REDACTED Tue Mar 22 05:44:12 2005 From: vances@REDACTED (Vance Shipley) Date: Mon, 21 Mar 2005 23:44:12 -0500 Subject: Port driver memory free In-Reply-To: <1111455881.423f7889b745e@www.omnibis.com> References: <005801c52e80$efed11f0$07f4f7dc@wavenet.lk> <1111455881.423f7889b745e@www.omnibis.com> Message-ID: <20050322044412.GG84433@frogman.motivity.ca> Eranga, Your problem is that you have copied my (copyrighted) code written for use with erlang:port_call/3 and reused it with erlang:port_control/3. The difference between the two is that port_call/3 uses the external term format as encoded by the ei library while port_control/3 uses the driver binary format. -Vance Motivity Telecom Inc. +1 519 240 3684 vances@REDACTED From geoffw@REDACTED Tue Mar 22 08:23:48 2005 From: geoffw@REDACTED (Geoff White) Date: Mon, 21 Mar 2005 23:23:48 -0800 Subject: Erlang otp_r10B-3 OpenBSD3.7 port is now ready for testing. Message-ID: <423FC804.5020109@cybertribe.com> I am pleased to announce a new port of Erlang otp_R10B-3 to OpenBSD 3.7 (beta). The port can be downloaded from... http://ww.snortdroid.org/DroidOS/erlang_otp_R10B-3_port.tgz Hopefully the OpenBSD folks will consider this port to replace the hopelessly outdated port of 4.7 that currently represents Erlang in their ports tree. Please note that this port, enables SAE but it has not been tested. Also the jinterface has been disabled due to lack of real jvm support on OpenBSD. YOU MUST BE RUNNING OpenBSD 3.7 (current) in order to use this port. I have not tried it with earlier releases, it may work but then again, it may not, but you are welcome to try, please report back any results. I have also only built and tested on i386 boxes, if anyone is running OpenBSD on anything else, I'd be interested in knowing how it fares. This port will install erl erlc ear ecc elink escript esh into /usr/local/bin. The bulk of the erlang system is installed in /usr/local/lib/erlang. You should unpack this tarball into /usr/ports/, make sure you move the existing erlang port to erlang.old. If you want to play with SAE, you will need beam_evm and erlang.ear. After your make and make install, you will find beam_evm in w-otp_R10B-3/otp_R10B-3/bin/$(TARGET-ARCH) and erlang.ear in w-otp_R10B-3/otp_R10B-3/erts/boot/src. Please let me know if you do download and do some testing, I'd like to know about successes as well as failures. I'd like to submit this to the OpenBSD folks as soon as possible doubt it will be in time to go on the 3.7 CD but maybe it will get incorporated into the ports tree in the next few months. Send feedback to me at geoffw at cybertribe.com From geoffw@REDACTED Tue Mar 22 08:26:29 2005 From: geoffw@REDACTED (Geoff White) Date: Mon, 21 Mar 2005 23:26:29 -0800 Subject: CORRECTION: Erlang OpenBSD port Message-ID: <423FC8A5.2050301@cybertribe.com> the correct URL for download is... http://www.snortdroid.org/DroidOS/erlang_otp_R10B-3_port.tgz geoffw From bjorn@REDACTED Tue Mar 22 08:51:11 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 22 Mar 2005 08:51:11 +0100 Subject: Bug with module_info in R10B-3 In-Reply-To: <423F39CA.40409@dominicwilliams.net> References: <423F39CA.40409@dominicwilliams.net> Message-ID: The bug will be corrected in R10B-4. /Bjorn Dominic Williams writes: > Hello, > > The following bug appeared sometime after R9C, and is at least present > in R10B-3 under Windows XP: when recompiling a modified module, its > old exports appear to be kept by module_info. > > E.g., after compiling test.erl: > > ----------------- > -module(test). > -export([foo/0]). > > foo() -> ok. > ----------------- > > test:module_info(exports) returns > [{foo,0},{module_info,0},{module_info,1}] > > After modifying and recompiling test.erl thus: > > ----------------- > -module(test). > -export([bar/0]). > > bar() -> ok. > ----------------- > > test:module_info(exports) now returns > [{foo,0},{bar,0},{module_info,0},{module_info,1}] > > Regards, > > Dominic, on behalf of the www.xpdojo.org team. > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From raimo@REDACTED Tue Mar 22 10:06:16 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 22 Mar 2005 10:06:16 +0100 Subject: Port driver memory free References: <005801c52e80$efed11f0$07f4f7dc@wavenet.lk>, <1111455881.423f7889b745e@www.omnibis.com> Message-ID: Ok, to repeat once again. For port_control in binary mode, you must _always_ return a binary. You can _not_ return the result in the preallocated buffer *rbuf -it can only be used in non-binary mode. So, your loop that runs twice, once for size determination, and once for encoding, should end in: if (p) break; *rbuf = driver_alloc_binary(*rindex); drv_binary = (ErlDrvBinary *) *rbuf; p = drv_binary->orig_bytes; *rindex = 0; } Now p always gets set for the second iteration, and *rbuf always gets set to a binary. That ought to work. casper2000a@REDACTED writes: > Raimo, > > Actually I allocate a binary. Here's what I do, > > ----------------------- C Port ----------------------------- > int gen_reply(char **rbuf, int rlen, int *rindex, void *buffer, int buflen) { > ErlDrvBinary *drv_binary; > char *p; > > *rindex = 0; > ........ > ........ > for (p = NULL;;) { > if (ei_encode_version(p, rindex) > || ei_encode_tuple_header(p, rindex, 2) > || ei_encode_atom(p, rindex, "ok") > || ei_encode_binary(p, rindex, buffer, buflen)) { > DBG("ei_encode failed"); > return((int) ERL_DRV_ERROR_ERRNO); > } > if (p) > break; > //if (*rindex > rlen) { > *rbuf = driver_alloc_binary(*rindex); > //} > drv_binary = (ErlDrvBinary *) *rbuf; > p = drv_binary->orig_bytes; > *rindex = 0; > } > > drv_binary->orig_size = *rindex; > ........ > ---------------------------------------------------------- > > I had to comment out the "(*rindex > rlen)" part since when it was there, what I received in the > emulator side was garbage after 8th char position. > > ------------------------- Emulator ----------------------- > case catch binary_to_term(erlang:port_control(Port, Command, > term_to_binary(Args))) of > {'EXIT', Reason} -> > ....... > ---------------------------------------------------------- > > On the emulator side binary_to_term BIF didn't give any error. But once receive the term, the binary > part contain incorrect data after 8th char pos. > > Thanks! > - Eranga > > > > > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > Sent: Friday, March 18, 2005 2:31 PM > To: erlang-questions@REDACTED > Subject: Re: Port driver memory free > > Yes, you do it wrong. The returned value _must_ be a binary, > (or NULL for R10B or later). The code that takes care of the value > assumes it is a binary, so you have introduced something in your > system that is assumed to be a binary, but is actually stack allocated > plain memory. The emulator will probably crash much later because > memory has been trashed. > > You will get partially correct behaviour because the receiving code > looks for the contents of the binary, and finds something there. > Unfortunately the header is not correct and it has not been allocated > with driver_alloc_binary(), but will later be freed with > driver_free_binary(), and both the contents and header will be > overwritten with garbage as soon as the emulator C stack grows. > > Do not do that - you will get hurt! > > > > casper2000a@REDACTED (Casper) writes: > > > Hi, > > > > Today I noticed a funny thing with Port drivers \"control\" call. > > > > I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is > (without > > reinitializing), since the space I need to output the binary message is > less > > than 64 bytes. > > > > When I decode the received binary message in the emulator side, up to 7 > > bytes were exactly as I encoded in the C Port side. After that 8 > position > > onward I get some other value. > > > > Then I created a new binary using driver_alloc_binary, even though the > > output (encoded, total) is less than 64 bytes and tried, and viola, the > > correct message came to emulator side. > > > > If that a bug or something I do wrong? > > > > Thanks! > > - Eranga > > > > > > > > -----Original Message----- > > From: owner-erlang-questions@REDACTED > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > Sent: Thursday, March 17, 2005 2:17 PM > > To: erlang-questions@REDACTED > > Subject: Re: Port driver memory free > > > > The short answer is: do not free! > > > > To elaborate: > > * In binary mode, you must return a binary: > > ErlDrvBinary *binp = driver_alloc_binary(len); > > /* Copy len bytes of data to to binp->orig_bytes */ > > *rbuf = (char *)binp; > > return len; > > The caller (the emulator) will increment the refcount and take care > > of deallocation when the binary is no longer used. > > * In list mode, if the supplied result buffer is too small, you > > should allocate a new result buffer using driver_alloc(size), > > and the caller (the emulator) will free it it has fetched the > > result from it. Otherwise just write the result in the supplied > > result buffer. The emulator compares the returned buffer address > > with the supplied buffer address to see if you return an allocated > > buffer. So, if you set *rbuf, it _must_ be to something allocated > > with driver_alloc(size), because it _will_ be freed with > > driver_free(*rbuf). You can of course return an allocated buffer > > for any size, also less than 64 bytes. > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > Hi All, > > > > > > There are 2 parameters, **rbuf and rlen in Port Driver entry \"control. > In > > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in > **rbuf. > > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use > that > > > without creating a new ErlDrvBinary. My questions are, > > > > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf > without > > > allocating a new ErlDrvBinary, should I still call driver_free_binary > or > > > driver_free to free that **rbuf memory before \"control\" function > returns? > > > > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > > > driver_binary and assign to **rbuf, what will happen to the previous > 64 > > > buffer which was in **rbuf? Will the port or erlang garbage collecter > free > > > that? Or do I need to free it inside the \"control\" function by calling > > > driver_free_binary or driver_free? > > > > > > 3. I (2) above, what will be the case if I use driver_realloc or > > > driver_realloc_binary? Will the previous 64 byes get released? > > > > > > 4. I (2) above, I still need to call driver_free_binary to free the > newly > > > created driver_binary inside \"control\" function, correct? > > > > > > 5. If I call \"set_port_control_flags(dd->port, 0)\" to set the port > output > > to > > > non-binary, do I need to free or clear the non-used space of **rbuf? > > > > > > 6. In above cased which free function, driver_free_binary or > driver_free > > and > > > which allocation function, driver_alloc, driver_alloc_binary, > > > driver_realloc, driver_realloc_binary should I use? > > > > > > Thanks in advance! > > > - Eranga > > > > > > > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > --------------This mail sent through OmniBIS.com-------------- > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From casper2000a@REDACTED Tue Mar 22 11:57:55 2005 From: casper2000a@REDACTED (Casper) Date: Tue, 22 Mar 2005 16:57:55 +0600 Subject: Port driver memory free In-Reply-To: Message-ID: <20050322105331.0DDD1400029@mail.omnibis.com> Hi Raimo, Thank you so much for enlighten me on the subject. I was very unclear about the binary mode and list mode and I thought even on the Binary mode I can still use the already allocated memory in the **rbuf, if the total output is less than 64 bytes. Today I loaded my Erlang work and used "TOP" unix command to monitor the Beam process and the CPU load. I noticed that Beam process memory size goes up. Initially it was consuming about 36MB. But after about 4 hours later, i.e. after passing more than 800,000 messages between Erlang and C Port, I noticed that memory usage of Beam has gone up to 100MB. As you instructed I don't use driver_free_binary. Could that be because I don't free it? I checked my C code thoroughly but cannot find any other memory leak area. Please advice! Thanks! - Eranga -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen Sent: Tuesday, March 22, 2005 3:06 PM To: erlang-questions@REDACTED Subject: Re: Port driver memory free Ok, to repeat once again. For port_control in binary mode, you must _always_ return a binary. You can _not_ return the result in the preallocated buffer *rbuf -it can only be used in non-binary mode. So, your loop that runs twice, once for size determination, and once for encoding, should end in: if (p) break; *rbuf = driver_alloc_binary(*rindex); drv_binary = (ErlDrvBinary *) *rbuf; p = drv_binary->orig_bytes; *rindex = 0; } Now p always gets set for the second iteration, and *rbuf always gets set to a binary. That ought to work. casper2000a@REDACTED writes: > Raimo, > > Actually I allocate a binary. Here's what I do, > > ----------------------- C Port ----------------------------- > int gen_reply(char **rbuf, int rlen, int *rindex, void *buffer, int buflen) { > ErlDrvBinary *drv_binary; > char *p; > > *rindex = 0; > ........ > ........ > for (p = NULL;;) { > if (ei_encode_version(p, rindex) > || ei_encode_tuple_header(p, rindex, 2) > || ei_encode_atom(p, rindex, "ok") > || ei_encode_binary(p, rindex, buffer, buflen)) { > DBG("ei_encode failed"); > return((int) ERL_DRV_ERROR_ERRNO); > } > if (p) > break; > //if (*rindex > rlen) { > *rbuf = driver_alloc_binary(*rindex); > //} > drv_binary = (ErlDrvBinary *) *rbuf; > p = drv_binary->orig_bytes; > *rindex = 0; > } > > drv_binary->orig_size = *rindex; > ........ > ---------------------------------------------------------- > > I had to comment out the "(*rindex > rlen)" part since when it was there, what I received in the > emulator side was garbage after 8th char position. > > ------------------------- Emulator ----------------------- > case catch binary_to_term(erlang:port_control(Port, Command, > term_to_binary(Args))) of > {'EXIT', Reason} -> > ....... > ---------------------------------------------------------- > > On the emulator side binary_to_term BIF didn't give any error. But once receive the term, the binary > part contain incorrect data after 8th char pos. > > Thanks! > - Eranga > > > > > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > Sent: Friday, March 18, 2005 2:31 PM > To: erlang-questions@REDACTED > Subject: Re: Port driver memory free > > Yes, you do it wrong. The returned value _must_ be a binary, > (or NULL for R10B or later). The code that takes care of the value > assumes it is a binary, so you have introduced something in your > system that is assumed to be a binary, but is actually stack allocated > plain memory. The emulator will probably crash much later because > memory has been trashed. > > You will get partially correct behaviour because the receiving code > looks for the contents of the binary, and finds something there. > Unfortunately the header is not correct and it has not been allocated > with driver_alloc_binary(), but will later be freed with > driver_free_binary(), and both the contents and header will be > overwritten with garbage as soon as the emulator C stack grows. > > Do not do that - you will get hurt! > > > > casper2000a@REDACTED (Casper) writes: > > > Hi, > > > > Today I noticed a funny thing with Port drivers \"control\" call. > > > > I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is > (without > > reinitializing), since the space I need to output the binary message is > less > > than 64 bytes. > > > > When I decode the received binary message in the emulator side, up to 7 > > bytes were exactly as I encoded in the C Port side. After that 8 > position > > onward I get some other value. > > > > Then I created a new binary using driver_alloc_binary, even though the > > output (encoded, total) is less than 64 bytes and tried, and viola, the > > correct message came to emulator side. > > > > If that a bug or something I do wrong? > > > > Thanks! > > - Eranga > > > > > > > > -----Original Message----- > > From: owner-erlang-questions@REDACTED > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > Sent: Thursday, March 17, 2005 2:17 PM > > To: erlang-questions@REDACTED > > Subject: Re: Port driver memory free > > > > The short answer is: do not free! > > > > To elaborate: > > * In binary mode, you must return a binary: > > ErlDrvBinary *binp = driver_alloc_binary(len); > > /* Copy len bytes of data to to binp->orig_bytes */ > > *rbuf = (char *)binp; > > return len; > > The caller (the emulator) will increment the refcount and take care > > of deallocation when the binary is no longer used. > > * In list mode, if the supplied result buffer is too small, you > > should allocate a new result buffer using driver_alloc(size), > > and the caller (the emulator) will free it it has fetched the > > result from it. Otherwise just write the result in the supplied > > result buffer. The emulator compares the returned buffer address > > with the supplied buffer address to see if you return an allocated > > buffer. So, if you set *rbuf, it _must_ be to something allocated > > with driver_alloc(size), because it _will_ be freed with > > driver_free(*rbuf). You can of course return an allocated buffer > > for any size, also less than 64 bytes. > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > Hi All, > > > > > > There are 2 parameters, **rbuf and rlen in Port Driver entry \"control. > In > > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in > **rbuf. > > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use > that > > > without creating a new ErlDrvBinary. My questions are, > > > > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf > without > > > allocating a new ErlDrvBinary, should I still call driver_free_binary > or > > > driver_free to free that **rbuf memory before \"control\" function > returns? > > > > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > > > driver_binary and assign to **rbuf, what will happen to the previous > 64 > > > buffer which was in **rbuf? Will the port or erlang garbage collecter > free > > > that? Or do I need to free it inside the \"control\" function by calling > > > driver_free_binary or driver_free? > > > > > > 3. I (2) above, what will be the case if I use driver_realloc or > > > driver_realloc_binary? Will the previous 64 byes get released? > > > > > > 4. I (2) above, I still need to call driver_free_binary to free the > newly > > > created driver_binary inside \"control\" function, correct? > > > > > > 5. If I call \"set_port_control_flags(dd->port, 0)\" to set the port > output > > to > > > non-binary, do I need to free or clear the non-used space of **rbuf? > > > > > > 6. In above cased which free function, driver_free_binary or > driver_free > > and > > > which allocation function, driver_alloc, driver_alloc_binary, > > > driver_realloc, driver_realloc_binary should I use? > > > > > > Thanks in advance! > > > - Eranga > > > > > > > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > --------------This mail sent through OmniBIS.com-------------- > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From casper2000a@REDACTED Tue Mar 22 12:03:14 2005 From: casper2000a@REDACTED (Casper) Date: Tue, 22 Mar 2005 17:03:14 +0600 Subject: Port driver memory free In-Reply-To: <20050322044412.GG84433@frogman.motivity.ca> Message-ID: <20050322105853.11CEF400029@mail.omnibis.com> Vance, Regarding port_call and port_control, I think both can use ei library functions to encode/decode terms, correct? If not, please advice how they've being done. I am sorry if I did something offending. But to my understanding the code I published only carries a very generic Erlang-C port code. Even on the Erlang manuals or samples, the same method is shown. Thanks! - Eranga -----Original Message----- From: Vance Shipley [mailto:vances@REDACTED] Sent: Tuesday, March 22, 2005 10:44 AM To: casper2000a@REDACTED Cc: erlang-questions@REDACTED; raimo@REDACTED Subject: Re: Port driver memory free Eranga, Your problem is that you have copied my (copyrighted) code written for use with erlang:port_call/3 and reused it with erlang:port_control/3. The difference between the two is that port_call/3 uses the external term format as encoded by the ei library while port_control/3 uses the driver binary format. -Vance Motivity Telecom Inc. +1 519 240 3684 vances@REDACTED From raimo@REDACTED Tue Mar 22 12:24:43 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 22 Mar 2005 12:24:43 +0100 Subject: Port driver memory free References: , <20050322105331.0DDD1400029@mail.omnibis.com> Message-ID: It is not certain you have a leak. It may be just fragmented memory. Try erlang:memory(), and/or also erlang:memory(binary) to see how the emulator's internal view of allocated memory varies over the time, and compare that to the crude figure from 'top'. It is probably so that after your application has started it takes some minutes / an hour for it to reach a quiescent state. And in that state there is a lot of binaries allocated, being allocated and being freed, so if erlang:memory(binary) asymtotically rises to a mean value, fluctuating around it, there is no problem. If on the other hand, erlang:memory(binary) rises linearily forever, you have a memory leak of binaries. The value from 'top' can differ a lot from what erlang:memory() reports, due to memory fragmentation. You can also read "erl -man instrument" for an advanced feature (I have never used) to analyze memory allocation in the emulator. casper2000a@REDACTED (Casper) writes: > Hi Raimo, > > Thank you so much for enlighten me on the subject. I was very unclear about > the binary mode and list mode and I thought even on the Binary mode I can > still use the already allocated memory in the **rbuf, if the total output is > less than 64 bytes. > > Today I loaded my Erlang work and used "TOP" unix command to monitor the > Beam process and the CPU load. I noticed that Beam process memory size goes > up. Initially it was consuming about 36MB. But after about 4 hours later, > i.e. after passing more than 800,000 messages between Erlang and C Port, I > noticed that memory usage of Beam has gone up to 100MB. > > As you instructed I don't use driver_free_binary. Could that be because I > don't free it? I checked my C code thoroughly but cannot find any other > memory leak area. > > Please advice! > > Thanks! > - Eranga > > > > > > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > Sent: Tuesday, March 22, 2005 3:06 PM > To: erlang-questions@REDACTED > Subject: Re: Port driver memory free > > Ok, to repeat once again. For port_control in binary mode, you must > _always_ return a binary. You can _not_ return the result in the > preallocated buffer *rbuf -it can only be used in non-binary > mode. > > So, your loop that runs twice, once for size determination, and > once for encoding, should end in: > if (p) > break; > *rbuf = driver_alloc_binary(*rindex); > drv_binary = (ErlDrvBinary *) *rbuf; > p = drv_binary->orig_bytes; > *rindex = 0; > } > Now p always gets set for the second iteration, and *rbuf always > gets set to a binary. That ought to work. > > > casper2000a@REDACTED writes: > > > Raimo, > > > > Actually I allocate a binary. Here's what I do, > > > > ----------------------- C Port ----------------------------- > > int gen_reply(char **rbuf, int rlen, int *rindex, void *buffer, int > buflen) { > > ErlDrvBinary *drv_binary; > > char *p; > > > > *rindex = 0; > > ........ > > ........ > > for (p = NULL;;) { > > if (ei_encode_version(p, rindex) > > || ei_encode_tuple_header(p, rindex, 2) > > || ei_encode_atom(p, rindex, "ok") > > || ei_encode_binary(p, rindex, buffer, > buflen)) { > > DBG("ei_encode failed"); > > return((int) ERL_DRV_ERROR_ERRNO); > > } > > if (p) > > break; > > //if (*rindex > rlen) { > > *rbuf = driver_alloc_binary(*rindex); > > //} > > drv_binary = (ErlDrvBinary *) *rbuf; > > p = drv_binary->orig_bytes; > > *rindex = 0; > > } > > > > drv_binary->orig_size = *rindex; > > ........ > > ---------------------------------------------------------- > > > > I had to comment out the "(*rindex > rlen)" part since when it was there, > what I received in the > > emulator side was garbage after 8th char position. > > > > ------------------------- Emulator ----------------------- > > case catch binary_to_term(erlang:port_control(Port, Command, > > > term_to_binary(Args))) of > > {'EXIT', Reason} -> > > ....... > > ---------------------------------------------------------- > > > > On the emulator side binary_to_term BIF didn't give any error. But once > receive the term, the binary > > part contain incorrect data after 8th char pos. > > > > Thanks! > > - Eranga > > > > > > > > > > -----Original Message----- > > From: owner-erlang-questions@REDACTED > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > Sent: Friday, March 18, 2005 2:31 PM > > To: erlang-questions@REDACTED > > Subject: Re: Port driver memory free > > > > Yes, you do it wrong. The returned value _must_ be a binary, > > (or NULL for R10B or later). The code that takes care of the value > > assumes it is a binary, so you have introduced something in your > > system that is assumed to be a binary, but is actually stack allocated > > plain memory. The emulator will probably crash much later because > > memory has been trashed. > > > > You will get partially correct behaviour because the receiving code > > looks for the contents of the binary, and finds something there. > > Unfortunately the header is not correct and it has not been allocated > > with driver_alloc_binary(), but will later be freed with > > driver_free_binary(), and both the contents and header will be > > overwritten with garbage as soon as the emulator C stack grows. > > > > Do not do that - you will get hurt! > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > Hi, > > > > > > Today I noticed a funny thing with Port drivers \"control\" call. > > > > > > I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is > > (without > > > reinitializing), since the space I need to output the binary message is > > less > > > than 64 bytes. > > > > > > When I decode the received binary message in the emulator side, up to 7 > > > bytes were exactly as I encoded in the C Port side. After that 8 > > position > > > onward I get some other value. > > > > > > Then I created a new binary using driver_alloc_binary, even though the > > > output (encoded, total) is less than 64 bytes and tried, and viola, the > > > correct message came to emulator side. > > > > > > If that a bug or something I do wrong? > > > > > > Thanks! > > > - Eranga > > > > > > > > > > > > -----Original Message----- > > > From: owner-erlang-questions@REDACTED > > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > > Sent: Thursday, March 17, 2005 2:17 PM > > > To: erlang-questions@REDACTED > > > Subject: Re: Port driver memory free > > > > > > The short answer is: do not free! > > > > > > To elaborate: > > > * In binary mode, you must return a binary: > > > ErlDrvBinary *binp = driver_alloc_binary(len); > > > /* Copy len bytes of data to to binp->orig_bytes */ > > > *rbuf = (char *)binp; > > > return len; > > > The caller (the emulator) will increment the refcount and take care > > > of deallocation when the binary is no longer used. > > > * In list mode, if the supplied result buffer is too small, you > > > should allocate a new result buffer using driver_alloc(size), > > > and the caller (the emulator) will free it it has fetched the > > > result from it. Otherwise just write the result in the supplied > > > result buffer. The emulator compares the returned buffer address > > > with the supplied buffer address to see if you return an allocated > > > buffer. So, if you set *rbuf, it _must_ be to something allocated > > > with driver_alloc(size), because it _will_ be freed with > > > driver_free(*rbuf). You can of course return an allocated buffer > > > for any size, also less than 64 bytes. > > > > > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > > > Hi All, > > > > > > > > There are 2 parameters, **rbuf and rlen in Port Driver entry > \"control. > > In > > > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in > > **rbuf. > > > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use > > that > > > > without creating a new ErlDrvBinary. My questions are, > > > > > > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf > > without > > > > allocating a new ErlDrvBinary, should I still call driver_free_binary > > or > > > > driver_free to free that **rbuf memory before \"control\" function > > returns? > > > > > > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > > > > driver_binary and assign to **rbuf, what will happen to the previous > > 64 > > > > buffer which was in **rbuf? Will the port or erlang garbage collecter > > free > > > > that? Or do I need to free it inside the \"control\" function by > calling > > > > driver_free_binary or driver_free? > > > > > > > > 3. I (2) above, what will be the case if I use driver_realloc or > > > > driver_realloc_binary? Will the previous 64 byes get released? > > > > > > > > 4. I (2) above, I still need to call driver_free_binary to free the > > newly > > > > created driver_binary inside \"control\" function, correct? > > > > > > > > 5. If I call \"set_port_control_flags(dd->port, 0)\" to set the port > > output > > > to > > > > non-binary, do I need to free or clear the non-used space of **rbuf? > > > > > > > > 6. In above cased which free function, driver_free_binary or > > driver_free > > > and > > > > which allocation function, driver_alloc, driver_alloc_binary, > > > > driver_realloc, driver_realloc_binary should I use? > > > > > > > > Thanks in advance! > > > > - Eranga > > > > > > > > > > > > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > --------------This mail sent through OmniBIS.com-------------- > > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From casper2000a@REDACTED Tue Mar 22 12:45:39 2005 From: casper2000a@REDACTED (Casper) Date: Tue, 22 Mar 2005 17:45:39 +0600 Subject: Port driver memory free In-Reply-To: <20050322105853.11CEF400029@mail.omnibis.com> Message-ID: <20050322114115.EC646400029@mail.omnibis.com> Only difference is call use external term format, so need to use driver_alloc, but control doesn't so use driver_alloc_binary, Correct? - Eranga -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED] On Behalf Of Casper Sent: Tuesday, March 22, 2005 5:03 PM To: 'Vance Shipley' Cc: erlang-questions@REDACTED; raimo@REDACTED Subject: RE: Port driver memory free Vance, Regarding port_call and port_control, I think both can use ei library functions to encode/decode terms, correct? If not, please advice how they've being done. I am sorry if I did something offending. But to my understanding the code I published only carries a very generic Erlang-C port code. Even on the Erlang manuals or samples, the same method is shown. Thanks! - Eranga -----Original Message----- From: Vance Shipley [mailto:vances@REDACTED] Sent: Tuesday, March 22, 2005 10:44 AM To: casper2000a@REDACTED Cc: erlang-questions@REDACTED; raimo@REDACTED Subject: Re: Port driver memory free Eranga, Your problem is that you have copied my (copyrighted) code written for use with erlang:port_call/3 and reused it with erlang:port_control/3. The difference between the two is that port_call/3 uses the external term format as encoded by the ei library while port_control/3 uses the driver binary format. -Vance Motivity Telecom Inc. +1 519 240 3684 vances@REDACTED From raimo@REDACTED Tue Mar 22 13:06:13 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 22 Mar 2005 13:06:13 +0100 Subject: Port driver memory free References: <20050322105853.11CEF400029@mail.omnibis.com>, <20050322114115.EC646400029@mail.omnibis.com> Message-ID: Rather correct... driver_entry->call() should return memory allocated with driver_alloc(), and the contents should be in external term format as from the ei_* functions. You do not have to allocate memory if the return value fits in the preallocated buffer. driver_entry->control() should return memory allocated with driver_alloc(), and the contents is interpreted as a list of integers [0..255]. You do not have to allocate memory if the return value fits in the preallocated buffer. If you set the PORT_CONTROL_FLAG_BINARY flag, driver_entry->control() should instead return a binary allocated with driver_alloc_binary() and it is interpreted as a binary (!), except if you (do not allocate a binary and) set the return buffer to NULL which is interpreted as an empty list (allowed from R10B). The preallocated return buffer can not be used. Note that you do not have to encode the result value using ei_* functions when you use driver_entry->control(). The encoding of the binary (or list) is entirely up to the driver and its calling erlang code. It is probably faster to encode the data in a suitable way of your own. casper2000a@REDACTED (Casper) writes: > Only difference is call use external term format, so need to use > driver_alloc, but control doesn't so use driver_alloc_binary, Correct? > > - Eranga > > > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Casper > Sent: Tuesday, March 22, 2005 5:03 PM > To: 'Vance Shipley' > Cc: erlang-questions@REDACTED; raimo@REDACTED > Subject: RE: Port driver memory free > > Vance, > > Regarding port_call and port_control, I think both can use ei library > functions to encode/decode terms, correct? If not, please advice how they've > being done. > > I am sorry if I did something offending. But to my understanding the code I > published only carries a very generic Erlang-C port code. Even on the Erlang > manuals or samples, the same method is shown. > > Thanks! > - Eranga > > > > > -----Original Message----- > From: Vance Shipley [mailto:vances@REDACTED] > Sent: Tuesday, March 22, 2005 10:44 AM > To: casper2000a@REDACTED > Cc: erlang-questions@REDACTED; raimo@REDACTED > Subject: Re: Port driver memory free > > Eranga, > > Your problem is that you have copied my (copyrighted) code written > for use with erlang:port_call/3 and reused it with erlang:port_control/3. > The difference between the two is that port_call/3 uses the external > term format as encoded by the ei library while port_control/3 uses the > driver binary format. > > -Vance > > > Motivity Telecom Inc. > +1 519 240 3684 > vances@REDACTED > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From casper2000a@REDACTED Tue Mar 22 13:13:50 2005 From: casper2000a@REDACTED (Casper) Date: Tue, 22 Mar 2005 18:13:50 +0600 Subject: Port driver memory free In-Reply-To: Message-ID: <20050322120926.7CF85400042@mail.omnibis.com> Raimo, Thanks a lot for your advice. I will check memory usage that way. - Eranga -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen Sent: Tuesday, March 22, 2005 5:25 PM To: erlang-questions@REDACTED Subject: Re: Port driver memory free It is not certain you have a leak. It may be just fragmented memory. Try erlang:memory(), and/or also erlang:memory(binary) to see how the emulator's internal view of allocated memory varies over the time, and compare that to the crude figure from 'top'. It is probably so that after your application has started it takes some minutes / an hour for it to reach a quiescent state. And in that state there is a lot of binaries allocated, being allocated and being freed, so if erlang:memory(binary) asymtotically rises to a mean value, fluctuating around it, there is no problem. If on the other hand, erlang:memory(binary) rises linearily forever, you have a memory leak of binaries. The value from 'top' can differ a lot from what erlang:memory() reports, due to memory fragmentation. You can also read "erl -man instrument" for an advanced feature (I have never used) to analyze memory allocation in the emulator. casper2000a@REDACTED (Casper) writes: > Hi Raimo, > > Thank you so much for enlighten me on the subject. I was very unclear about > the binary mode and list mode and I thought even on the Binary mode I can > still use the already allocated memory in the **rbuf, if the total output is > less than 64 bytes. > > Today I loaded my Erlang work and used "TOP" unix command to monitor the > Beam process and the CPU load. I noticed that Beam process memory size goes > up. Initially it was consuming about 36MB. But after about 4 hours later, > i.e. after passing more than 800,000 messages between Erlang and C Port, I > noticed that memory usage of Beam has gone up to 100MB. > > As you instructed I don't use driver_free_binary. Could that be because I > don't free it? I checked my C code thoroughly but cannot find any other > memory leak area. > > Please advice! > > Thanks! > - Eranga > > > > > > -----Original Message----- > From: owner-erlang-questions@REDACTED > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > Sent: Tuesday, March 22, 2005 3:06 PM > To: erlang-questions@REDACTED > Subject: Re: Port driver memory free > > Ok, to repeat once again. For port_control in binary mode, you must > _always_ return a binary. You can _not_ return the result in the > preallocated buffer *rbuf -it can only be used in non-binary > mode. > > So, your loop that runs twice, once for size determination, and > once for encoding, should end in: > if (p) > break; > *rbuf = driver_alloc_binary(*rindex); > drv_binary = (ErlDrvBinary *) *rbuf; > p = drv_binary->orig_bytes; > *rindex = 0; > } > Now p always gets set for the second iteration, and *rbuf always > gets set to a binary. That ought to work. > > > casper2000a@REDACTED writes: > > > Raimo, > > > > Actually I allocate a binary. Here's what I do, > > > > ----------------------- C Port ----------------------------- > > int gen_reply(char **rbuf, int rlen, int *rindex, void *buffer, int > buflen) { > > ErlDrvBinary *drv_binary; > > char *p; > > > > *rindex = 0; > > ........ > > ........ > > for (p = NULL;;) { > > if (ei_encode_version(p, rindex) > > || ei_encode_tuple_header(p, rindex, 2) > > || ei_encode_atom(p, rindex, "ok") > > || ei_encode_binary(p, rindex, buffer, > buflen)) { > > DBG("ei_encode failed"); > > return((int) ERL_DRV_ERROR_ERRNO); > > } > > if (p) > > break; > > //if (*rindex > rlen) { > > *rbuf = driver_alloc_binary(*rindex); > > //} > > drv_binary = (ErlDrvBinary *) *rbuf; > > p = drv_binary->orig_bytes; > > *rindex = 0; > > } > > > > drv_binary->orig_size = *rindex; > > ........ > > ---------------------------------------------------------- > > > > I had to comment out the "(*rindex > rlen)" part since when it was there, > what I received in the > > emulator side was garbage after 8th char position. > > > > ------------------------- Emulator ----------------------- > > case catch binary_to_term(erlang:port_control(Port, Command, > > > term_to_binary(Args))) of > > {'EXIT', Reason} -> > > ....... > > ---------------------------------------------------------- > > > > On the emulator side binary_to_term BIF didn't give any error. But once > receive the term, the binary > > part contain incorrect data after 8th char pos. > > > > Thanks! > > - Eranga > > > > > > > > > > -----Original Message----- > > From: owner-erlang-questions@REDACTED > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > Sent: Friday, March 18, 2005 2:31 PM > > To: erlang-questions@REDACTED > > Subject: Re: Port driver memory free > > > > Yes, you do it wrong. The returned value _must_ be a binary, > > (or NULL for R10B or later). The code that takes care of the value > > assumes it is a binary, so you have introduced something in your > > system that is assumed to be a binary, but is actually stack allocated > > plain memory. The emulator will probably crash much later because > > memory has been trashed. > > > > You will get partially correct behaviour because the receiving code > > looks for the contents of the binary, and finds something there. > > Unfortunately the header is not correct and it has not been allocated > > with driver_alloc_binary(), but will later be freed with > > driver_free_binary(), and both the contents and header will be > > overwritten with garbage as soon as the emulator C stack grows. > > > > Do not do that - you will get hurt! > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > Hi, > > > > > > Today I noticed a funny thing with Port drivers \"control\" call. > > > > > > I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is > > (without > > > reinitializing), since the space I need to output the binary message is > > less > > > than 64 bytes. > > > > > > When I decode the received binary message in the emulator side, up to 7 > > > bytes were exactly as I encoded in the C Port side. After that 8 > > position > > > onward I get some other value. > > > > > > Then I created a new binary using driver_alloc_binary, even though the > > > output (encoded, total) is less than 64 bytes and tried, and viola, the > > > correct message came to emulator side. > > > > > > If that a bug or something I do wrong? > > > > > > Thanks! > > > - Eranga > > > > > > > > > > > > -----Original Message----- > > > From: owner-erlang-questions@REDACTED > > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > > Sent: Thursday, March 17, 2005 2:17 PM > > > To: erlang-questions@REDACTED > > > Subject: Re: Port driver memory free > > > > > > The short answer is: do not free! > > > > > > To elaborate: > > > * In binary mode, you must return a binary: > > > ErlDrvBinary *binp = driver_alloc_binary(len); > > > /* Copy len bytes of data to to binp->orig_bytes */ > > > *rbuf = (char *)binp; > > > return len; > > > The caller (the emulator) will increment the refcount and take care > > > of deallocation when the binary is no longer used. > > > * In list mode, if the supplied result buffer is too small, you > > > should allocate a new result buffer using driver_alloc(size), > > > and the caller (the emulator) will free it it has fetched the > > > result from it. Otherwise just write the result in the supplied > > > result buffer. The emulator compares the returned buffer address > > > with the supplied buffer address to see if you return an allocated > > > buffer. So, if you set *rbuf, it _must_ be to something allocated > > > with driver_alloc(size), because it _will_ be freed with > > > driver_free(*rbuf). You can of course return an allocated buffer > > > for any size, also less than 64 bytes. > > > > > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > > > Hi All, > > > > > > > > There are 2 parameters, **rbuf and rlen in Port Driver entry > \"control. > > In > > > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in > > **rbuf. > > > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use > > that > > > > without creating a new ErlDrvBinary. My questions are, > > > > > > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf > > without > > > > allocating a new ErlDrvBinary, should I still call driver_free_binary > > or > > > > driver_free to free that **rbuf memory before \"control\" function > > returns? > > > > > > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > > > > driver_binary and assign to **rbuf, what will happen to the previous > > 64 > > > > buffer which was in **rbuf? Will the port or erlang garbage collecter > > free > > > > that? Or do I need to free it inside the \"control\" function by > calling > > > > driver_free_binary or driver_free? > > > > > > > > 3. I (2) above, what will be the case if I use driver_realloc or > > > > driver_realloc_binary? Will the previous 64 byes get released? > > > > > > > > 4. I (2) above, I still need to call driver_free_binary to free the > > newly > > > > created driver_binary inside \"control\" function, correct? > > > > > > > > 5. If I call \"set_port_control_flags(dd->port, 0)\" to set the port > > output > > > to > > > > non-binary, do I need to free or clear the non-used space of **rbuf? > > > > > > > > 6. In above cased which free function, driver_free_binary or > > driver_free > > > and > > > > which allocation function, driver_alloc, driver_alloc_binary, > > > > driver_realloc, driver_realloc_binary should I use? > > > > > > > > Thanks in advance! > > > > - Eranga > > > > > > > > > > > > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > --------------This mail sent through OmniBIS.com-------------- > > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From tobias.lindahl@REDACTED Tue Mar 22 15:47:42 2005 From: tobias.lindahl@REDACTED (Tobias Lindahl) Date: Tue, 22 Mar 2005 15:47:42 +0100 (MET) Subject: Closing a dets Message-ID: Hi all, Is there a way of waiting for a dets to be properly closed after dets:close/1 has been called. The problem is that I want to shut down the erlang node with an exit status (using halt(ExitStatus)), but wait until the dets is properly closed. Cheers, Tobias From casper2000a@REDACTED Wed Mar 23 03:28:39 2005 From: casper2000a@REDACTED (casper2000a@REDACTED) Date: Wed, 23 Mar 2005 08:28:39 +0600 Subject: Port driver memory free In-Reply-To: References: , <20050322105331.0DDD1400029@mail.omnibis.com> Message-ID: <1111544919.4240d457907bc@www.omnibis.com> Raimo, As you instructed I ran erlang:memory(), erlang:memory(binary) - "erlang_memory.txt" erlang:system_info(allocated_areas), erlang:system_info(garbage_collection) - "erlang_system_info_allocated_areas.txt" erlang:system_info(allocator) - "erlang_system_info_allocator.txt" As I can see Erlang memory fluctuates around a 10MB total memory and 100KB binary memory marks. Those outputs are in the attached text files, respectively. Also I have attached to "TOP" output in the file "mem_usage.txt". As you can see, when the program started around 12:50pm, Beam was using 56MB. At 18:55pm, it has gone up to 153MB. Looks like there's a memory leak somewhere. Is there anyway to check the C port memory allocation? How much it has been allocated? Or can that memory being some cache or something? If you find time, please go through my dump files and see if you can identify where those memory gone. Thanks in advance! - Eranga Quoting Raimo Niskanen : > Quoting Raimo Niskanen : > It is not certain you have a leak. It may be just fragmented memory. > Try erlang:memory(), and/or also erlang:memory(binary) to see how > the emulator\'s internal view of allocated memory varies over the > time, and compare that to the crude figure from \'top\'. > > It is probably so that after your application has started it takes > some minutes / an hour for it to reach a quiescent state. And in > that state there is a lot of binaries allocated, being allocated > and being freed, so if erlang:memory(binary) asymtotically rises > to a mean value, fluctuating around it, there is no problem. If > on the other hand, erlang:memory(binary) rises linearily forever, > you have a memory leak of binaries. > > The value from \'top\' can differ a lot from what erlang:memory() > reports, due to memory fragmentation. > > You can also read \"erl -man instrument\" for an advanced feature > (I have never used) to analyze memory allocation in the emulator. > > > > casper2000a@REDACTED (Casper) writes: > > > Hi Raimo, > > > > Thank you so much for enlighten me on the subject. I was very unclear > about > > the binary mode and list mode and I thought even on the Binary mode I can > > still use the already allocated memory in the **rbuf, if the total output > is > > less than 64 bytes. > > > > Today I loaded my Erlang work and used \"TOP\" unix command to monitor the > > Beam process and the CPU load. I noticed that Beam process memory size > goes > > up. Initially it was consuming about 36MB. But after about 4 hours later, > > i.e. after passing more than 800,000 messages between Erlang and C Port, I > > noticed that memory usage of Beam has gone up to 100MB. > > > > As you instructed I don\'t use driver_free_binary. Could that be because I > > don\'t free it? I checked my C code thoroughly but cannot find any other > > memory leak area. > > > > Please advice! > > > > Thanks! > > - Eranga > > > > > > > > > > > > -----Original Message----- > > From: owner-erlang-questions@REDACTED > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > Sent: Tuesday, March 22, 2005 3:06 PM > > To: erlang-questions@REDACTED > > Subject: Re: Port driver memory free > > > > Ok, to repeat once again. For port_control in binary mode, you must > > _always_ return a binary. You can _not_ return the result in the > > preallocated buffer *rbuf -it can only be used in non-binary > > mode. > > > > So, your loop that runs twice, once for size determination, and > > once for encoding, should end in: > > if (p) > > break; > > *rbuf = driver_alloc_binary(*rindex); > > drv_binary = (ErlDrvBinary *) *rbuf; > > p = drv_binary->orig_bytes; > > *rindex = 0; > > } > > Now p always gets set for the second iteration, and *rbuf always > > gets set to a binary. That ought to work. > > > > > > casper2000a@REDACTED writes: > > > > > Raimo, > > > > > > Actually I allocate a binary. Here\'s what I do, > > > > > > ----------------------- C Port ----------------------------- > > > int gen_reply(char **rbuf, int rlen, int *rindex, void *buffer, int > > buflen) { > > > ErlDrvBinary *drv_binary; > > > char *p; > > > > > > *rindex = 0; > > > ........ > > > ........ > > > for (p = NULL;;) { > > > if (ei_encode_version(p, rindex) > > > || ei_encode_tuple_header(p, rindex, 2) > > > || ei_encode_atom(p, rindex, \"ok\") > > > || ei_encode_binary(p, rindex, buffer, > > buflen)) { > > > DBG(\"ei_encode failed\"); > > > return((int) ERL_DRV_ERROR_ERRNO); > > > } > > > if (p) > > > break; > > > //if (*rindex > rlen) { > > > *rbuf = driver_alloc_binary(*rindex); > > > //} > > > drv_binary = (ErlDrvBinary *) *rbuf; > > > p = drv_binary->orig_bytes; > > > *rindex = 0; > > > } > > > > > > drv_binary->orig_size = *rindex; > > > ........ > > > ---------------------------------------------------------- > > > > > > I had to comment out the \"(*rindex > rlen)\" part since when it was > there, > > what I received in the > > > emulator side was garbage after 8th char position. > > > > > > ------------------------- Emulator ----------------------- > > > case catch binary_to_term(erlang:port_control(Port, Command, > > > > > term_to_binary(Args))) of > > > {\'EXIT\', Reason} -> > > > ....... > > > ---------------------------------------------------------- > > > > > > On the emulator side binary_to_term BIF didn\'t give any error. But once > > receive the term, the binary > > > part contain incorrect data after 8th char pos. > > > > > > Thanks! > > > - Eranga > > > > > > > > > > > > > > > -----Original Message----- > > > From: owner-erlang-questions@REDACTED > > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > > Sent: Friday, March 18, 2005 2:31 PM > > > To: erlang-questions@REDACTED > > > Subject: Re: Port driver memory free > > > > > > Yes, you do it wrong. The returned value _must_ be a binary, > > > (or NULL for R10B or later). The code that takes care of the value > > > assumes it is a binary, so you have introduced something in your > > > system that is assumed to be a binary, but is actually stack allocated > > > plain memory. The emulator will probably crash much later because > > > memory has been trashed. > > > > > > You will get partially correct behaviour because the receiving code > > > looks for the contents of the binary, and finds something there. > > > Unfortunately the header is not correct and it has not been allocated > > > with driver_alloc_binary(), but will later be freed with > > > driver_free_binary(), and both the contents and header will be > > > overwritten with garbage as soon as the emulator C stack grows. > > > > > > Do not do that - you will get hurt! > > > > > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > > > Hi, > > > > > > > > Today I noticed a funny thing with Port drivers \\\"control\\\" call. > > > > > > > > I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is > > > (without > > > > reinitializing), since the space I need to output the binary message > is > > > less > > > > than 64 bytes. > > > > > > > > When I decode the received binary message in the emulator side, up to > 7 > > > > bytes were exactly as I encoded in the C Port side. After that 8 > > > position > > > > onward I get some other value. > > > > > > > > Then I created a new binary using driver_alloc_binary, even though the > > > > output (encoded, total) is less than 64 bytes and tried, and viola, > the > > > > correct message came to emulator side. > > > > > > > > If that a bug or something I do wrong? > > > > > > > > Thanks! > > > > - Eranga > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: owner-erlang-questions@REDACTED > > > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > > > Sent: Thursday, March 17, 2005 2:17 PM > > > > To: erlang-questions@REDACTED > > > > Subject: Re: Port driver memory free > > > > > > > > The short answer is: do not free! > > > > > > > > To elaborate: > > > > * In binary mode, you must return a binary: > > > > ErlDrvBinary *binp = driver_alloc_binary(len); > > > > /* Copy len bytes of data to to binp->orig_bytes */ > > > > *rbuf = (char *)binp; > > > > return len; > > > > The caller (the emulator) will increment the refcount and take care > > > > of deallocation when the binary is no longer used. > > > > * In list mode, if the supplied result buffer is too small, you > > > > should allocate a new result buffer using driver_alloc(size), > > > > and the caller (the emulator) will free it it has fetched the > > > > result from it. Otherwise just write the result in the supplied > > > > result buffer. The emulator compares the returned buffer address > > > > with the supplied buffer address to see if you return an allocated > > > > buffer. So, if you set *rbuf, it _must_ be to something allocated > > > > with driver_alloc(size), because it _will_ be freed with > > > > driver_free(*rbuf). You can of course return an allocated buffer > > > > for any size, also less than 64 bytes. > > > > > > > > > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > > > > > Hi All, > > > > > > > > > > There are 2 parameters, **rbuf and rlen in Port Driver entry > > \\\"control. > > > In > > > > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in > > > **rbuf. > > > > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use > > > that > > > > > without creating a new ErlDrvBinary. My questions are, > > > > > > > > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf > > > without > > > > > allocating a new ErlDrvBinary, should I still call > driver_free_binary > > > or > > > > > driver_free to free that **rbuf memory before \\\"control\\\" function > > > returns? > > > > > > > > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > > > > > driver_binary and assign to **rbuf, what will happen to the previous > > > 64 > > > > > buffer which was in **rbuf? Will the port or erlang garbage > collecter > > > free > > > > > that? Or do I need to free it inside the \\\"control\\\" function by > > calling > > > > > driver_free_binary or driver_free? > > > > > > > > > > 3. I (2) above, what will be the case if I use driver_realloc or > > > > > driver_realloc_binary? Will the previous 64 byes get released? > > > > > > > > > > 4. I (2) above, I still need to call driver_free_binary to free the > > > newly > > > > > created driver_binary inside \\\"control\\\" function, correct? > > > > > > > > > > 5. If I call \\\"set_port_control_flags(dd->port, 0)\\\" to set the port > > > output > > > > to > > > > > non-binary, do I need to free or clear the non-used space of **rbuf? > > > > > > > > > > 6. In above cased which free function, driver_free_binary or > > > driver_free > > > > and > > > > > which allocation function, driver_alloc, driver_alloc_binary, > > > > > driver_realloc, driver_realloc_binary should I use? > > > > > > > > > > Thanks in advance! > > > > > - Eranga > > > > > > > > > > > > > > > > > > -- > > > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > > > > > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > > > --------------This mail sent through OmniBIS.com-------------- > > > > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > --------------This mail sent through OmniBIS.com-------------- -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: mem_usage.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: erlang_memory.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: erlang_system_info_allocated_areas.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: erlang_system_info_allocator.txt URL: From casper2000a@REDACTED Wed Mar 23 03:32:46 2005 From: casper2000a@REDACTED (casper2000a@REDACTED) Date: Wed, 23 Mar 2005 08:32:46 +0600 Subject: Port driver memory free In-Reply-To: References: <20050322105853.11CEF400029@mail.omnibis.com>, <20050322114115.EC646400029@mail.omnibis.com> Message-ID: <1111545166.4240d54e4f31b@www.omnibis.com> Thanks Raimo and Vance for you generous advice! - Eranga Quoting Raimo Niskanen : > Quoting Raimo Niskanen : > Rather correct... > > driver_entry->call() should return memory allocated with > driver_alloc(), and the contents should be in external term format > as from the ei_* functions. You do not have to allocate memory > if the return value fits in the preallocated buffer. > > driver_entry->control() should return memory allocated with > driver_alloc(), and the contents is interpreted as a list of > integers [0..255]. You do not have to allocate memory if the > return value fits in the preallocated buffer. > > If you set the PORT_CONTROL_FLAG_BINARY flag, > driver_entry->control() should instead return a binary allocated > with driver_alloc_binary() and it is interpreted as a binary (!), > except if you (do not allocate a binary and) set the return buffer > to NULL which is interpreted as an empty list (allowed from R10B). > The preallocated return buffer can not be used. > > Note that you do not have to encode the result value using ei_* > functions when you use driver_entry->control(). The encoding > of the binary (or list) is entirely up to the driver and its > calling erlang code. It is probably faster to encode the data > in a suitable way of your own. > > > > casper2000a@REDACTED (Casper) writes: > > > Only difference is call use external term format, so need to use > > driver_alloc, but control doesn\'t so use driver_alloc_binary, Correct? > > > > - Eranga > > > > > > -----Original Message----- > > From: owner-erlang-questions@REDACTED > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Casper > > Sent: Tuesday, March 22, 2005 5:03 PM > > To: \'Vance Shipley\' > > Cc: erlang-questions@REDACTED; raimo@REDACTED > > Subject: RE: Port driver memory free > > > > Vance, > > > > Regarding port_call and port_control, I think both can use ei library > > functions to encode/decode terms, correct? If not, please advice how > they\'ve > > being done. > > > > I am sorry if I did something offending. But to my understanding the code > I > > published only carries a very generic Erlang-C port code. Even on the > Erlang > > manuals or samples, the same method is shown. > > > > Thanks! > > - Eranga > > > > > > > > > > -----Original Message----- > > From: Vance Shipley [mailto:vances@REDACTED] > > Sent: Tuesday, March 22, 2005 10:44 AM > > To: casper2000a@REDACTED > > Cc: erlang-questions@REDACTED; raimo@REDACTED > > Subject: Re: Port driver memory free > > > > Eranga, > > > > Your problem is that you have copied my (copyrighted) code written > > for use with erlang:port_call/3 and reused it with erlang:port_control/3. > > The difference between the two is that port_call/3 uses the external > > term format as encoded by the ei library while port_control/3 uses the > > driver binary format. > > > > -Vance > > > > > > Motivity Telecom Inc. > > +1 519 240 3684 > > vances@REDACTED > > > > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > --------------This mail sent through OmniBIS.com-------------- From casper2000a@REDACTED Wed Mar 23 06:15:40 2005 From: casper2000a@REDACTED (Casper) Date: Wed, 23 Mar 2005 11:15:40 +0600 Subject: Port driver memory free In-Reply-To: <1111544919.4240d457907bc@www.omnibis.com> Message-ID: <20050323051116.A472E40000A@mail.omnibis.com> More updates.... Today when I checked the system, the Beam memory consumption was 253MB. - Eranga -----Original Message----- From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED] On Behalf Of casper2000a@REDACTED Sent: Wednesday, March 23, 2005 8:29 AM To: erlang-questions@REDACTED Cc: raimo@REDACTED Subject: Re: Port driver memory free Raimo, As you instructed I ran erlang:memory(), erlang:memory(binary) - "erlang_memory.txt" erlang:system_info(allocated_areas), erlang:system_info(garbage_collection) - "erlang_system_info_allocated_areas.txt" erlang:system_info(allocator) - "erlang_system_info_allocator.txt" As I can see Erlang memory fluctuates around a 10MB total memory and 100KB binary memory marks. Those outputs are in the attached text files, respectively. Also I have attached to "TOP" output in the file "mem_usage.txt". As you can see, when the program started around 12:50pm, Beam was using 56MB. At 18:55pm, it has gone up to 153MB. Looks like there's a memory leak somewhere. Is there anyway to check the C port memory allocation? How much it has been allocated? Or can that memory being some cache or something? If you find time, please go through my dump files and see if you can identify where those memory gone. Thanks in advance! - Eranga Quoting Raimo Niskanen : > Quoting Raimo Niskanen : > It is not certain you have a leak. It may be just fragmented memory. > Try erlang:memory(), and/or also erlang:memory(binary) to see how > the emulator\'s internal view of allocated memory varies over the > time, and compare that to the crude figure from \'top\'. > > It is probably so that after your application has started it takes > some minutes / an hour for it to reach a quiescent state. And in > that state there is a lot of binaries allocated, being allocated > and being freed, so if erlang:memory(binary) asymtotically rises > to a mean value, fluctuating around it, there is no problem. If > on the other hand, erlang:memory(binary) rises linearily forever, > you have a memory leak of binaries. > > The value from \'top\' can differ a lot from what erlang:memory() > reports, due to memory fragmentation. > > You can also read \"erl -man instrument\" for an advanced feature > (I have never used) to analyze memory allocation in the emulator. > > > > casper2000a@REDACTED (Casper) writes: > > > Hi Raimo, > > > > Thank you so much for enlighten me on the subject. I was very unclear > about > > the binary mode and list mode and I thought even on the Binary mode I can > > still use the already allocated memory in the **rbuf, if the total output > is > > less than 64 bytes. > > > > Today I loaded my Erlang work and used \"TOP\" unix command to monitor the > > Beam process and the CPU load. I noticed that Beam process memory size > goes > > up. Initially it was consuming about 36MB. But after about 4 hours later, > > i.e. after passing more than 800,000 messages between Erlang and C Port, I > > noticed that memory usage of Beam has gone up to 100MB. > > > > As you instructed I don\'t use driver_free_binary. Could that be because I > > don\'t free it? I checked my C code thoroughly but cannot find any other > > memory leak area. > > > > Please advice! > > > > Thanks! > > - Eranga > > > > > > > > > > > > -----Original Message----- > > From: owner-erlang-questions@REDACTED > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > Sent: Tuesday, March 22, 2005 3:06 PM > > To: erlang-questions@REDACTED > > Subject: Re: Port driver memory free > > > > Ok, to repeat once again. For port_control in binary mode, you must > > _always_ return a binary. You can _not_ return the result in the > > preallocated buffer *rbuf -it can only be used in non-binary > > mode. > > > > So, your loop that runs twice, once for size determination, and > > once for encoding, should end in: > > if (p) > > break; > > *rbuf = driver_alloc_binary(*rindex); > > drv_binary = (ErlDrvBinary *) *rbuf; > > p = drv_binary->orig_bytes; > > *rindex = 0; > > } > > Now p always gets set for the second iteration, and *rbuf always > > gets set to a binary. That ought to work. > > > > > > casper2000a@REDACTED writes: > > > > > Raimo, > > > > > > Actually I allocate a binary. Here\'s what I do, > > > > > > ----------------------- C Port ----------------------------- > > > int gen_reply(char **rbuf, int rlen, int *rindex, void *buffer, int > > buflen) { > > > ErlDrvBinary *drv_binary; > > > char *p; > > > > > > *rindex = 0; > > > ........ > > > ........ > > > for (p = NULL;;) { > > > if (ei_encode_version(p, rindex) > > > || ei_encode_tuple_header(p, rindex, 2) > > > || ei_encode_atom(p, rindex, \"ok\") > > > || ei_encode_binary(p, rindex, buffer, > > buflen)) { > > > DBG(\"ei_encode failed\"); > > > return((int) ERL_DRV_ERROR_ERRNO); > > > } > > > if (p) > > > break; > > > //if (*rindex > rlen) { > > > *rbuf = driver_alloc_binary(*rindex); > > > //} > > > drv_binary = (ErlDrvBinary *) *rbuf; > > > p = drv_binary->orig_bytes; > > > *rindex = 0; > > > } > > > > > > drv_binary->orig_size = *rindex; > > > ........ > > > ---------------------------------------------------------- > > > > > > I had to comment out the \"(*rindex > rlen)\" part since when it was > there, > > what I received in the > > > emulator side was garbage after 8th char position. > > > > > > ------------------------- Emulator ----------------------- > > > case catch binary_to_term(erlang:port_control(Port, Command, > > > > > term_to_binary(Args))) of > > > {\'EXIT\', Reason} -> > > > ....... > > > ---------------------------------------------------------- > > > > > > On the emulator side binary_to_term BIF didn\'t give any error. But once > > receive the term, the binary > > > part contain incorrect data after 8th char pos. > > > > > > Thanks! > > > - Eranga > > > > > > > > > > > > > > > -----Original Message----- > > > From: owner-erlang-questions@REDACTED > > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > > Sent: Friday, March 18, 2005 2:31 PM > > > To: erlang-questions@REDACTED > > > Subject: Re: Port driver memory free > > > > > > Yes, you do it wrong. The returned value _must_ be a binary, > > > (or NULL for R10B or later). The code that takes care of the value > > > assumes it is a binary, so you have introduced something in your > > > system that is assumed to be a binary, but is actually stack allocated > > > plain memory. The emulator will probably crash much later because > > > memory has been trashed. > > > > > > You will get partially correct behaviour because the receiving code > > > looks for the contents of the binary, and finds something there. > > > Unfortunately the header is not correct and it has not been allocated > > > with driver_alloc_binary(), but will later be freed with > > > driver_free_binary(), and both the contents and header will be > > > overwritten with garbage as soon as the emulator C stack grows. > > > > > > Do not do that - you will get hurt! > > > > > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > > > Hi, > > > > > > > > Today I noticed a funny thing with Port drivers \\\"control\\\" call. > > > > > > > > I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is > > > (without > > > > reinitializing), since the space I need to output the binary message > is > > > less > > > > than 64 bytes. > > > > > > > > When I decode the received binary message in the emulator side, up to > 7 > > > > bytes were exactly as I encoded in the C Port side. After that 8 > > > position > > > > onward I get some other value. > > > > > > > > Then I created a new binary using driver_alloc_binary, even though the > > > > output (encoded, total) is less than 64 bytes and tried, and viola, > the > > > > correct message came to emulator side. > > > > > > > > If that a bug or something I do wrong? > > > > > > > > Thanks! > > > > - Eranga > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: owner-erlang-questions@REDACTED > > > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > > > Sent: Thursday, March 17, 2005 2:17 PM > > > > To: erlang-questions@REDACTED > > > > Subject: Re: Port driver memory free > > > > > > > > The short answer is: do not free! > > > > > > > > To elaborate: > > > > * In binary mode, you must return a binary: > > > > ErlDrvBinary *binp = driver_alloc_binary(len); > > > > /* Copy len bytes of data to to binp->orig_bytes */ > > > > *rbuf = (char *)binp; > > > > return len; > > > > The caller (the emulator) will increment the refcount and take care > > > > of deallocation when the binary is no longer used. > > > > * In list mode, if the supplied result buffer is too small, you > > > > should allocate a new result buffer using driver_alloc(size), > > > > and the caller (the emulator) will free it it has fetched the > > > > result from it. Otherwise just write the result in the supplied > > > > result buffer. The emulator compares the returned buffer address > > > > with the supplied buffer address to see if you return an allocated > > > > buffer. So, if you set *rbuf, it _must_ be to something allocated > > > > with driver_alloc(size), because it _will_ be freed with > > > > driver_free(*rbuf). You can of course return an allocated buffer > > > > for any size, also less than 64 bytes. > > > > > > > > > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > > > > > Hi All, > > > > > > > > > > There are 2 parameters, **rbuf and rlen in Port Driver entry > > \\\"control. > > > In > > > > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in > > > **rbuf. > > > > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use > > > that > > > > > without creating a new ErlDrvBinary. My questions are, > > > > > > > > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf > > > without > > > > > allocating a new ErlDrvBinary, should I still call > driver_free_binary > > > or > > > > > driver_free to free that **rbuf memory before \\\"control\\\" function > > > returns? > > > > > > > > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > > > > > driver_binary and assign to **rbuf, what will happen to the previous > > > 64 > > > > > buffer which was in **rbuf? Will the port or erlang garbage > collecter > > > free > > > > > that? Or do I need to free it inside the \\\"control\\\" function by > > calling > > > > > driver_free_binary or driver_free? > > > > > > > > > > 3. I (2) above, what will be the case if I use driver_realloc or > > > > > driver_realloc_binary? Will the previous 64 byes get released? > > > > > > > > > > 4. I (2) above, I still need to call driver_free_binary to free the > > > newly > > > > > created driver_binary inside \\\"control\\\" function, correct? > > > > > > > > > > 5. If I call \\\"set_port_control_flags(dd->port, 0)\\\" to set the port > > > output > > > > to > > > > > non-binary, do I need to free or clear the non-used space of **rbuf? > > > > > > > > > > 6. In above cased which free function, driver_free_binary or > > > driver_free > > > > and > > > > > which allocation function, driver_alloc, driver_alloc_binary, > > > > > driver_realloc, driver_realloc_binary should I use? > > > > > > > > > > Thanks in advance! > > > > > - Eranga > > > > > > > > > > > > > > > > > > -- > > > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > > > > > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > > > --------------This mail sent through OmniBIS.com-------------- > > > > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > --------------This mail sent through OmniBIS.com-------------- From osxroolz@REDACTED Wed Mar 23 08:18:58 2005 From: osxroolz@REDACTED (osxroolz@REDACTED) Date: Tue, 22 Mar 2005 23:18:58 -0800 Subject: Port driver memory free In-Reply-To: <20050323051116.A472E40000A@mail.omnibis.com> References: <20050323051116.A472E40000A@mail.omnibis.com> Message-ID: <1111562338.27289.230214619@webmail.messagingengine.com> On Wed, 23 Mar 2005 11:15:40 +0600, "Casper" said: > More updates.... > > Today when I checked the system, the Beam memory consumption was 253MB. Most powerful tool in debugging is diffrential analysis. Differential analysis means you take version of the code with desirable properties and version of program with ugly properties and step by step make them more and more the same. In this process you one day make one step which turns good program into evil program and then that is the time when you know exactly which step take you to hell. Your case your having troubles with drivers. You say > Is there anyway to check the C port memory allocation? the obvious answer is "turn your linking driver into own program which communicates with Erlang through a socket". Now unix tell you exactly where memory being used. Maybe you then have good version of your program which let you think about which step is needed to make it evil out of control memory eating beast. But now your not using deferential analysis. You are using ask mailing list a lot of clueless questions type of analysis. iMan ---- -- osxroolz@REDACTED From casper2000a@REDACTED Wed Mar 23 08:28:53 2005 From: casper2000a@REDACTED (Casper) Date: Wed, 23 Mar 2005 13:28:53 +0600 Subject: Port driver memory free In-Reply-To: <1111562338.27289.230214619@webmail.messagingengine.com> Message-ID: <20050323072429.95F55400028@mail.omnibis.com> iMan, Thanks for your advice. I would like to do what you mentioned below. But to do that, don't I have to change the architecture of Port Driver to a C Port or Node? In which case I will need to use different encode/decode and communication methods. With that I will not be able to see what's exactly happening with the current Port Driver scenario. I will not get dirver->control or driver->call scenarios. To give you an understanding, the C code that I wrote myself is very minimal to send and receive data to and from the underline protocol stack. Most of the C code is written to encode/decode objects pass between Erlang and C. So if I have to change the Architecture to convert my Port driver to an external Port or Node, I loose the track of the memory leak point. Please correct me if I am wrong. - Eranga -----Original Message----- From: osxroolz@REDACTED [mailto:osxroolz@REDACTED] Sent: Wednesday, March 23, 2005 1:19 PM To: Casper; casper2000a@REDACTED; erlang-questions@REDACTED Cc: raimo@REDACTED Subject: RE: Port driver memory free On Wed, 23 Mar 2005 11:15:40 +0600, "Casper" said: > More updates.... > > Today when I checked the system, the Beam memory consumption was 253MB. Most powerful tool in debugging is diffrential analysis. Differential analysis means you take version of the code with desirable properties and version of program with ugly properties and step by step make them more and more the same. In this process you one day make one step which turns good program into evil program and then that is the time when you know exactly which step take you to hell. Your case your having troubles with drivers. You say > Is there anyway to check the C port memory allocation? the obvious answer is "turn your linking driver into own program which communicates with Erlang through a socket". Now unix tell you exactly where memory being used. Maybe you then have good version of your program which let you think about which step is needed to make it evil out of control memory eating beast. But now your not using deferential analysis. You are using ask mailing list a lot of clueless questions type of analysis. iMan ---- -- osxroolz@REDACTED From thomas.xa.johnsson@REDACTED Wed Mar 23 08:58:12 2005 From: thomas.xa.johnsson@REDACTED (Thomas Johnsson XA (LN/EAB)) Date: Wed, 23 Mar 2005 08:58:12 +0100 Subject: Parse transformery (Was: Re: Calling internal functions - foo::bar() ?) Message-ID: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EADA@esealmw107.eemea.ericsson.se> (Wading through some backlogs ...:-) klacke@REDACTED: > > Speaking for myself. The ability to call private functions > in _live_ systems far outweigh performance considerations > in the systems I have been involved in lately. > > Therefore I almost always use -compile(export_all) > in most of my modules. Later ... way later when a > module has stabilized, I might (if I get the time) > do a later overhaul and explicitly export the API > functions. So how about the following suggestion: - You write your -export declarations, properly, for soft.eng. purposes. - For development & debuggning, put in a -compile(export_all) also! - Let the compiler and/or otp *warn* if you're using something exported by an export_all but not a proper -export. I imagine that to make a proper job of the last point, the compiler needs to put additional information in the beam file, namely mark the functions export_all:ed but not exported. -- Thomas From bjorn@REDACTED Wed Mar 23 09:24:24 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 23 Mar 2005 09:24:24 +0100 Subject: Erlang/OTP R10B-4 has been released Message-ID: Bug fix release : otp_src_R10B-4 Build from snapshot : 2005-03-23 This is a bug fix release 4 for the R10B release. You can download the full source distribution from http://www.erlang.org/download/otp_src_R10B-4.tar.gz http://www.erlang.org/download/otp_src_R10B-4.readme Note: To unpack the TAR archive you need a GNU TAR compatible program. For instance, on MacOS X you need to use the 'gnutar' command; you can't use the 'tar' command or StuffIt to unpack the sources. For installation instructions please read the README that is part of the distribution. The Windows binary distribution can be downloaded from http://www.erlang.org/download/otp_win32_R10B-4.exe The documentation at http://www.erlang.org will be updated. You can also download the complete HTML documentation or the Unix manual files http://www.erlang.org/download/otp_doc_html_R10B-4.tar.gz http://www.erlang.org/download/otp_doc_man_R10B-4.tar.gz For some OTP applications there are more detailed release notes in the HTML documentation. We also want to thank those that sent us patches, suggestions and bug reports, The OTP Team From raimo@REDACTED Wed Mar 23 09:52:31 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 23 Mar 2005 09:52:31 +0100 Subject: Port driver memory free References: <20050322105331.0DDD1400029@mail.omnibis.com>, , <1111544919.4240d457907bc@www.omnibis.com> Message-ID: You should have a closer look at what the OS thinks about the Beam process' memory allocation. If you are running linux, firs get the pid of the Beam process, e.g from os:getpid(). Then have a look at /proc/[BeamPid]/maps and /proc/[BeamPid]/statm, see proc(5 or 4) man page, and watch how they change over time. casper2000a@REDACTED writes: > Raimo, > > As you instructed I ran > erlang:memory(), erlang:memory(binary) - "erlang_memory.txt" > erlang:system_info(allocated_areas), erlang:system_info(garbage_collection) - > "erlang_system_info_allocated_areas.txt" > erlang:system_info(allocator) - "erlang_system_info_allocator.txt" > > As I can see Erlang memory fluctuates around a 10MB total memory and 100KB binary memory > marks. Those outputs are in the attached text files, respectively. > > Also I have attached to "TOP" output in the file "mem_usage.txt". As you can see, when the program > started around 12:50pm, Beam was using 56MB. At 18:55pm, it has gone up to 153MB. > > Looks like there's a memory leak somewhere. Is there anyway to check the C port memory > allocation? How much it has been allocated? Or can that memory being some cache or something? > > If you find time, please go through my dump files and see if you can identify where those memory > gone. > > Thanks in advance! > - Eranga > > > > Quoting Raimo Niskanen : > Quoting Raimo Niskanen : > > > It is not certain you have a leak. It may be just fragmented memory. > > Try erlang:memory(), and/or also erlang:memory(binary) to see how > > the emulator\'s internal view of allocated memory varies over the > > time, and compare that to the crude figure from \'top\'. > > > > It is probably so that after your application has started it takes > > some minutes / an hour for it to reach a quiescent state. And in > > that state there is a lot of binaries allocated, being allocated > > and being freed, so if erlang:memory(binary) asymtotically rises > > to a mean value, fluctuating around it, there is no problem. If > > on the other hand, erlang:memory(binary) rises linearily forever, > > you have a memory leak of binaries. > > > > The value from \'top\' can differ a lot from what erlang:memory() > > reports, due to memory fragmentation. > > > > You can also read \"erl -man instrument\" for an advanced feature > > (I have never used) to analyze memory allocation in the emulator. > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > Hi Raimo, > > > > > > Thank you so much for enlighten me on the subject. I was very unclear > > about > > > the binary mode and list mode and I thought even on the Binary mode I can > > > still use the already allocated memory in the **rbuf, if the total output > > is > > > less than 64 bytes. > > > > > > Today I loaded my Erlang work and used \"TOP\" unix command to monitor the > > > Beam process and the CPU load. I noticed that Beam process memory size > > goes > > > up. Initially it was consuming about 36MB. But after about 4 hours later, > > > i.e. after passing more than 800,000 messages between Erlang and C Port, I > > > noticed that memory usage of Beam has gone up to 100MB. > > > > > > As you instructed I don\'t use driver_free_binary. Could that be because I > > > don\'t free it? I checked my C code thoroughly but cannot find any other > > > memory leak area. > > > > > > Please advice! > > > > > > Thanks! > > > - Eranga > > > > > > > > > > > > > > > > > > -----Original Message----- > > > From: owner-erlang-questions@REDACTED > > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > > Sent: Tuesday, March 22, 2005 3:06 PM > > > To: erlang-questions@REDACTED > > > Subject: Re: Port driver memory free > > > > > > Ok, to repeat once again. For port_control in binary mode, you must > > > _always_ return a binary. You can _not_ return the result in the > > > preallocated buffer *rbuf -it can only be used in non-binary > > > mode. > > > > > > So, your loop that runs twice, once for size determination, and > > > once for encoding, should end in: > > > if (p) > > > break; > > > *rbuf = driver_alloc_binary(*rindex); > > > drv_binary = (ErlDrvBinary *) *rbuf; > > > p = drv_binary->orig_bytes; > > > *rindex = 0; > > > } > > > Now p always gets set for the second iteration, and *rbuf always > > > gets set to a binary. That ought to work. > > > > > > > > > casper2000a@REDACTED writes: > > > > > > > Raimo, > > > > > > > > Actually I allocate a binary. Here\'s what I do, > > > > > > > > ----------------------- C Port ----------------------------- > > > > int gen_reply(char **rbuf, int rlen, int *rindex, void *buffer, int > > > buflen) { > > > > ErlDrvBinary *drv_binary; > > > > char *p; > > > > > > > > *rindex = 0; > > > > ........ > > > > ........ > > > > for (p = NULL;;) { > > > > if (ei_encode_version(p, rindex) > > > > || ei_encode_tuple_header(p, rindex, 2) > > > > || ei_encode_atom(p, rindex, \"ok\") > > > > || ei_encode_binary(p, rindex, buffer, > > > buflen)) { > > > > DBG(\"ei_encode failed\"); > > > > return((int) ERL_DRV_ERROR_ERRNO); > > > > } > > > > if (p) > > > > break; > > > > //if (*rindex > rlen) { > > > > *rbuf = driver_alloc_binary(*rindex); > > > > //} > > > > drv_binary = (ErlDrvBinary *) *rbuf; > > > > p = drv_binary->orig_bytes; > > > > *rindex = 0; > > > > } > > > > > > > > drv_binary->orig_size = *rindex; > > > > ........ > > > > ---------------------------------------------------------- > > > > > > > > I had to comment out the \"(*rindex > rlen)\" part since when it was > > there, > > > what I received in the > > > > emulator side was garbage after 8th char position. > > > > > > > > ------------------------- Emulator ----------------------- > > > > case catch binary_to_term(erlang:port_control(Port, Command, > > > > > > > term_to_binary(Args))) of > > > > {\'EXIT\', Reason} -> > > > > ....... > > > > ---------------------------------------------------------- > > > > > > > > On the emulator side binary_to_term BIF didn\'t give any error. But once > > > receive the term, the binary > > > > part contain incorrect data after 8th char pos. > > > > > > > > Thanks! > > > > - Eranga > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: owner-erlang-questions@REDACTED > > > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > > > Sent: Friday, March 18, 2005 2:31 PM > > > > To: erlang-questions@REDACTED > > > > Subject: Re: Port driver memory free > > > > > > > > Yes, you do it wrong. The returned value _must_ be a binary, > > > > (or NULL for R10B or later). The code that takes care of the value > > > > assumes it is a binary, so you have introduced something in your > > > > system that is assumed to be a binary, but is actually stack allocated > > > > plain memory. The emulator will probably crash much later because > > > > memory has been trashed. > > > > > > > > You will get partially correct behaviour because the receiving code > > > > looks for the contents of the binary, and finds something there. > > > > Unfortunately the header is not correct and it has not been allocated > > > > with driver_alloc_binary(), but will later be freed with > > > > driver_free_binary(), and both the contents and header will be > > > > overwritten with garbage as soon as the emulator C stack grows. > > > > > > > > Do not do that - you will get hurt! > > > > > > > > > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > > > > > Hi, > > > > > > > > > > Today I noticed a funny thing with Port drivers \\\"control\\\" call. > > > > > > > > > > I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is > > > > (without > > > > > reinitializing), since the space I need to output the binary message > > is > > > > less > > > > > than 64 bytes. > > > > > > > > > > When I decode the received binary message in the emulator side, up to > > 7 > > > > > bytes were exactly as I encoded in the C Port side. After that 8 > > > > position > > > > > onward I get some other value. > > > > > > > > > > Then I created a new binary using driver_alloc_binary, even though the > > > > > output (encoded, total) is less than 64 bytes and tried, and viola, > > the > > > > > correct message came to emulator side. > > > > > > > > > > If that a bug or something I do wrong? > > > > > > > > > > Thanks! > > > > > - Eranga > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: owner-erlang-questions@REDACTED > > > > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen > > > > > Sent: Thursday, March 17, 2005 2:17 PM > > > > > To: erlang-questions@REDACTED > > > > > Subject: Re: Port driver memory free > > > > > > > > > > The short answer is: do not free! > > > > > > > > > > To elaborate: > > > > > * In binary mode, you must return a binary: > > > > > ErlDrvBinary *binp = driver_alloc_binary(len); > > > > > /* Copy len bytes of data to to binp->orig_bytes */ > > > > > *rbuf = (char *)binp; > > > > > return len; > > > > > The caller (the emulator) will increment the refcount and take care > > > > > of deallocation when the binary is no longer used. > > > > > * In list mode, if the supplied result buffer is too small, you > > > > > should allocate a new result buffer using driver_alloc(size), > > > > > and the caller (the emulator) will free it it has fetched the > > > > > result from it. Otherwise just write the result in the supplied > > > > > result buffer. The emulator compares the returned buffer address > > > > > with the supplied buffer address to see if you return an allocated > > > > > buffer. So, if you set *rbuf, it _must_ be to something allocated > > > > > with driver_alloc(size), because it _will_ be freed with > > > > > driver_free(*rbuf). You can of course return an allocated buffer > > > > > for any size, also less than 64 bytes. > > > > > > > > > > > > > > > > > > > > casper2000a@REDACTED (Casper) writes: > > > > > > > > > > > Hi All, > > > > > > > > > > > > There are 2 parameters, **rbuf and rlen in Port Driver entry > > > \\\"control. > > > > In > > > > > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in > > > > **rbuf. > > > > > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use > > > > that > > > > > > without creating a new ErlDrvBinary. My questions are, > > > > > > > > > > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf > > > > without > > > > > > allocating a new ErlDrvBinary, should I still call > > driver_free_binary > > > > or > > > > > > driver_free to free that **rbuf memory before \\\"control\\\" function > > > > returns? > > > > > > > > > > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new > > > > > > driver_binary and assign to **rbuf, what will happen to the previous > > > > 64 > > > > > > buffer which was in **rbuf? Will the port or erlang garbage > > collecter > > > > free > > > > > > that? Or do I need to free it inside the \\\"control\\\" function by > > > calling > > > > > > driver_free_binary or driver_free? > > > > > > > > > > > > 3. I (2) above, what will be the case if I use driver_realloc or > > > > > > driver_realloc_binary? Will the previous 64 byes get released? > > > > > > > > > > > > 4. I (2) above, I still need to call driver_free_binary to free the > > > > newly > > > > > > created driver_binary inside \\\"control\\\" function, correct? > > > > > > > > > > > > 5. If I call \\\"set_port_control_flags(dd->port, 0)\\\" to set the port > > > > output > > > > > to > > > > > > non-binary, do I need to free or clear the non-used space of **rbuf? > > > > > > > > > > > > 6. In above cased which free function, driver_free_binary or > > > > driver_free > > > > > and > > > > > > which allocation function, driver_alloc, driver_alloc_binary, > > > > > > driver_realloc, driver_realloc_binary should I use? > > > > > > > > > > > > Thanks in advance! > > > > > > - Eranga > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > > > > > > > > > > -- > > > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > > > > > --------------This mail sent through OmniBIS.com-------------- > > > > > > > > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > --------------This mail sent through OmniBIS.com-------------- > > > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From osxroolz@REDACTED Wed Mar 23 11:58:34 2005 From: osxroolz@REDACTED (osxroolz@REDACTED) Date: Wed, 23 Mar 2005 02:58:34 -0800 Subject: Port driver memory free In-Reply-To: <20050323072429.95F55400028@mail.omnibis.com> References: <20050323072429.95F55400028@mail.omnibis.com> Message-ID: <1111575514.8130.230223774@webmail.messagingengine.com> On Wed, 23 Mar 2005 13:28:53 +0600, "Casper" said: > if I have to change the Architecture to convert my Port driver to an > external Port or Node, I loose the track of the memory leak point. Your saying oposite of what makes sense. When you divide into two processes then you do not loose track but you have better track. Three results One. Memory leak goes away. That was point of exorcise. Now you know the leaking it is in interface code. Very short example with just leaking code and nothing more possible now. Maybe the mistake in how you call interface. Two. beam continuse leaking. P{roblem allmost certain in erlang code. Three. c process continuse leaking. Problem is in C code. iMan -- osxroolz@REDACTED From casper2000a@REDACTED Wed Mar 23 12:22:17 2005 From: casper2000a@REDACTED (Casper) Date: Wed, 23 Mar 2005 17:22:17 +0600 Subject: Port driver memory free In-Reply-To: <1111575514.8130.230223774@webmail.messagingengine.com> Message-ID: <20050323111756.CE11E400074@mail.omnibis.com> iMan, You are right. But what I am saying is if I have to change the architecture of the program, for example change the Port Driver to a C Port or C Node, then I loose the efficiency which I targeted by choosing Port Driver method in the first place. For example, to my understanding, to split Erlang and my C code, I need to change the Port Driver behavior. There won't be driver->control, driver->call, driver->event, etc. I will not use driver_alloc_binary, driver_free_binary functions. Then I will not know the correct method of writing a Port Driver. Well, if I convert the code to be a something other than Port Driver, yes, I will know that Port Driver part created the problem. If there's no any other workaround solution to identify the problem, I guess I will do that and see. Thanks! - Eranga -----Original Message----- From: osxroolz@REDACTED [mailto:osxroolz@REDACTED] Sent: Wednesday, March 23, 2005 4:59 PM To: Casper; erlang-questions@REDACTED Subject: RE: Port driver memory free On Wed, 23 Mar 2005 13:28:53 +0600, "Casper" said: > if I have to change the Architecture to convert my Port driver to an > external Port or Node, I loose the track of the memory leak point. Your saying oposite of what makes sense. When you divide into two processes then you do not loose track but you have better track. Three results One. Memory leak goes away. That was point of exorcise. Now you know the leaking it is in interface code. Very short example with just leaking code and nothing more possible now. Maybe the mistake in how you call interface. Two. beam continuse leaking. P{roblem allmost certain in erlang code. Three. c process continuse leaking. Problem is in C code. iMan -- osxroolz@REDACTED From olgeni@REDACTED Wed Mar 23 12:24:01 2005 From: olgeni@REDACTED (Jimmy Olgeni) Date: Wed, 23 Mar 2005 12:24:01 +0100 (CET) Subject: Erlang/OTP R10B-4 has been released In-Reply-To: References: Message-ID: <20050323122242.R31481@dev1.localdomain.net> Hi, On Wed, 23 Mar 2005, Bjorn Gustavsson wrote: > Bug fix release : otp_src_R10B-4 > Build from snapshot : 2005-03-23 I just upgraded and now I get this message from Dialyzer when building the first PLT: =ERROR REPORT==== 23-Mar-2005::12:22:27 === Error: [hipe:781]: ERROR: {{case_clause, {dets, "/usr/local/lib/erlang/lib/dialyzer-1.2.0/plt/dialyzer_init_plt"}}, [{hipe_main,icode_ssa_type_info,3}, {hipe_main,icode_ssa,3}, {hipe_main,compile_icode,4}, {hipe,finalize_fun,2}, {hipe,finalize_fun_fixpoint,3}, {hipe,finalize,5}, {hipe,compile_finish,3}, {hipe, '-run_compiler_1/3-fun-0-', 4}]} Any idea would be welcome :-) -- jimmy From kostis@REDACTED Wed Mar 23 12:34:33 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 23 Mar 2005 12:34:33 +0100 (MET) Subject: Erlang/OTP R10B-4 has been released In-Reply-To: Mail from 'Jimmy Olgeni ' dated: Wed, 23 Mar 2005 12:24:01 +0100 (CET) Message-ID: <200503231134.j2NBYXtS020220@spikklubban.it.uu.se> Jimmy Olgeni wrote: > I just upgraded and now I get this message from Dialyzer when building > the first PLT: In general it is very difficult to guarantee that a software program (the Dialyzer version we distributed in December 2004 in this case) will work with a future version of another program (the OTP release of today in this case). Please bear with us for a day or two and we will be releasing a new Dialyzer version. We would be doing this anyway. Best, Kostis From tobias.lindahl@REDACTED Wed Mar 23 12:39:23 2005 From: tobias.lindahl@REDACTED (Tobias Lindahl) Date: Wed, 23 Mar 2005 12:39:23 +0100 (MET) Subject: Erlang/OTP R10B-4 has been released In-Reply-To: <20050323122242.R31481@dev1.localdomain.net> References: <20050323122242.R31481@dev1.localdomain.net> Message-ID: Hi Jimmy and all other Dialyzer users, This was even faster than I could imagine ;) In R10B-4 there has been changes in the interface between Dialyzer and HiPE. This means that the available Dialyzer will not not work with R10B-4, but we are working full time to make the new Dialyzer ready for a new release that does. Tobias On Wed, 23 Mar 2005, Jimmy Olgeni wrote: > > Hi, > > On Wed, 23 Mar 2005, Bjorn Gustavsson wrote: > > > Bug fix release : otp_src_R10B-4 > > Build from snapshot : 2005-03-23 > > I just upgraded and now I get this message from Dialyzer when building > the first PLT: > > =ERROR REPORT==== 23-Mar-2005::12:22:27 === > Error: [hipe:781]: ERROR: {{case_clause, > {dets, > "/usr/local/lib/erlang/lib/dialyzer-1.2.0/plt/dialyzer_init_plt"}}, > [{hipe_main,icode_ssa_type_info,3}, > {hipe_main,icode_ssa,3}, > {hipe_main,compile_icode,4}, > {hipe,finalize_fun,2}, > {hipe,finalize_fun_fixpoint,3}, > {hipe,finalize,5}, > {hipe,compile_finish,3}, > {hipe, > '-run_compiler_1/3-fun-0-', > 4}]} > > Any idea would be welcome :-) > > -- > jimmy > From hasse@REDACTED Wed Mar 23 13:43:06 2005 From: hasse@REDACTED (Hans Bolinder) Date: Wed, 23 Mar 2005 13:43:06 +0100 Subject: Closing a dets In-Reply-To: References: Message-ID: <16961.25690.805769.594924@gargle.gargle.HOWL> [Tobias Lindahl:] > Is there a way of waiting for a dets to be properly closed after > dets:close/1 has been called. The problem is that I want to shut down the > erlang node with an exit status (using halt(ExitStatus)), but wait until > the dets is properly closed. You should call dets:sync/1 before calling dets:close/1 if you want to be sure that data have been written to disk (dets:sync/1 calls file:sync/1 which calls fsync(3C)). (This may take some time if you close many Dets tables.) But I assume what you want is to be sure that no Dets table will need to be repaired when opened next time. For that it should suffice to call dets:close/1 without calling dets:sync/1. Or have you found a counterexample? Best regards, Hans Bolinder, Erlang/OTP From tobias.lindahl@REDACTED Wed Mar 23 14:31:10 2005 From: tobias.lindahl@REDACTED (Tobias Lindahl) Date: Wed, 23 Mar 2005 14:31:10 +0100 (MET) Subject: Closing a dets In-Reply-To: <16961.25690.805769.594924@gargle.gargle.HOWL> References: <16961.25690.805769.594924@gargle.gargle.HOWL> Message-ID: Actually, while constructing an counterexample I realised that the problem is not related to shutting down the node. It seems to be related to converting a sufficently large ets table to dets. The following example works when the ets is small, but breaks when it grows larger. Tobias ===== Example ===== -module(t). -export([t/2]). t(Name, Size) -> Ets = build_ets(Size), {ok, Dets} = dets:open_file(Name, []), ok = dets:from_ets(Dets, Ets), ok = dets:sync(Dets), ok = dets:close(Dets), ets:delete(Ets). build_ets(Size) -> Ets = ets:new(t_ets, [public, set]), build_ets(Ets, Size). build_ets(Ets, N) when N < 0-> Ets; build_ets(Ets, N) -> ets:insert(Ets, {N, N+1}), build_ets(Ets, N-1). Eshell V5.5 (abort with ^G) 1> t:t("/tmp/foo", 100). true 2> {ok, Dets} = dets:open_file("/tmp/foo", [{access, read}]). {ok,"/tmp/foo"} 3> dets:lookup(Dets, 1). [{1,2}] Eshell V5.5 (abort with ^G) 1> t:t("/tmp/foo", 1000). true 2> {ok, Dets} = dets:open_file("/tmp/foo", [{access, read}]). {ok,"/tmp/foo"} 3> dets:lookup(Dets, 1). {error,{file_error,"/tmp/foo",{0,ebadf}}} On Wed, 23 Mar 2005, Hans Bolinder wrote: > [Tobias Lindahl:] > > Is there a way of waiting for a dets to be properly closed after > > dets:close/1 has been called. The problem is that I want to shut down the > > erlang node with an exit status (using halt(ExitStatus)), but wait until > > the dets is properly closed. > > You should call dets:sync/1 before calling dets:close/1 if you want to > be sure that data have been written to disk (dets:sync/1 calls > file:sync/1 which calls fsync(3C)). (This may take some time if you > close many Dets tables.) > > But I assume what you want is to be sure that no Dets table will need > to be repaired when opened next time. For that it should suffice to > call dets:close/1 without calling dets:sync/1. > > Or have you found a counterexample? > > Best regards, > > Hans Bolinder, Erlang/OTP > From edmundd@REDACTED Wed Mar 23 16:19:39 2005 From: edmundd@REDACTED (Edmund Dengler) Date: Wed, 23 Mar 2005 10:19:39 -0500 (EST) Subject: Direct to Postgresql process/module? Message-ID: Hi folks! Just checking (before spending time developing ourselves), if anybody has (or knows of) a direct from Erlang to Postgresql process/module that we could use? We are currently using the ODBC component, but we are getting bad round trip times for simple queries (39 msec/query), which is too long for a system we are developing. We are hoping that by directly connecting to Postgresql via a TCP socket and using the protocol, we can reduce the round trip costs. Regards! Ed From carsten@REDACTED Wed Mar 23 23:51:08 2005 From: carsten@REDACTED (Carsten Schultz) Date: Wed, 23 Mar 2005 23:51:08 +0100 Subject: [Erlyaws-list] Yaws HTTPS Requests In-Reply-To: <20050323205605.GK10107@fangora.autosys.us> References: <20050323205605.GK10107@fangora.autosys.us> Message-ID: <20050323225108.GA23034@penne.localnet> Hi Micheal! On Wed, Mar 23, 2005 at 12:56:05PM -0800, Michael McDaniel wrote: > Does anyone have Yaws accepting HTTPS requests and responding? Sure. > If so, how did you get it to work? I just followed the instructions in the documentation. Maybe you could show us your yaws.conf I am using some version of 1.49 and have compiled Erlang myself. No special reason why I am lagging behind version-wise, though. Greetings, Carsten -- Carsten Schultz (2:38, 33:47) http://carsten.codimi.de/ PGP/GPG key on the pgp.net key servers, fingerprint on my home page. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From yvonne@REDACTED Thu Mar 24 11:03:03 2005 From: yvonne@REDACTED (Yvonne Erlandsson) Date: Thu, 24 Mar 2005 10:03:03 -0000 Subject: International Erlang jobs Message-ID: <000c01c53058$aed11340$155189d9@Yvonne> Hi, Sorry for sending something like this; however we sent this to everyone on our private mailing list and many suggested we also send it to the general Erlang-questions list. Hope it is ok! We have launched an Erlang Jobs website where we are posting some quite interesting opportunities and are therefore inviting anyone to pay a visit and, either apply to the position/s that you would be interested in or register to get future job updates. The URL is http://www.erlang-consulting.com/jobs_fs.html We have positions in four continents we are working hard to fill up, so please also help us to spread the word to anyone out there that might be looking or will be in the future looking for work within Erlang/OTP. Yvonne -- http://www.erlang-consulting.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.lindahl@REDACTED Thu Mar 24 12:22:13 2005 From: tobias.lindahl@REDACTED (Tobias Lindahl) Date: Thu, 24 Mar 2005 12:22:13 +0100 (MET) Subject: Dialyzer v1.3.0 released Message-ID: Once again we are proud to release a new version of Dialyzer. Dialyzer v 1.3.0 is available for download at: http://www.it.uu.se/research/group/hipe/dialyzer/snapshots/dialyzer-1.3.0.tar.gz Please note that this version needs Erlang/OTP R10B-4 to work properly. Enjoy! Tobias and Kostis Release notes: Version 1.3.0 ------------- - Requires the presence of an Erlang/OTP R10B-4 system. - Dialyzer is significantly (approx 40%) faster since it now uses 'ets' rather than 'dets' for its PLT. - Slightly improved the precision of the analysis. - Fixed a number of false positives that Dialyzer 1.2.0 was spitting out. - In the GUI version, Dialyzer now reports the list of modules that should be included in the modules to analyze in order to possibly improve the accuracy of the reported results. - Some more information is displayed when calling a function or closure with arguments of the wrong type. - The record guard now allows discrepancies involving tuples that are known to be records to be displayed as #rec{} rather than {'rec',_,...,_}. - Added -q option which makes the command-line version of Dialyzer a bit more silent. From vlad_dumitrescu@REDACTED Thu Mar 24 12:45:17 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Thu, 24 Mar 2005 12:45:17 +0100 Subject: Abstract patterns Message-ID: Hi, I just wonder if there are any plans to implement Richard O'Keefe's abstract patterns or something similar? As a slightly related "feature", I just added to Jungerl, in the msc directory, a module called rec_info that retrieves record information from modules (using code taken from shell.erl) and also allows converting the record into a keylist. It has no error handling yet, and it probably works only with R10. best regards, Vlad From olgeni@REDACTED Thu Mar 24 13:09:27 2005 From: olgeni@REDACTED (Jimmy Olgeni) Date: Thu, 24 Mar 2005 13:09:27 +0100 (CET) Subject: Dialyzer v1.3.0 released In-Reply-To: References: Message-ID: <20050324130526.D32855@dev1.localdomain.net> On Thu, 24 Mar 2005, Tobias Lindahl wrote: > Once again we are proud to release a new version of Dialyzer. Maybe somebody should have a look into stdlib, I hope there are not that many improper lists :-) {dict,counter_bkt,3}: Cons will produce a non-proper list since its 2nd arg is number()! {dict,mk_seg,1}: The clause matching on the integer 4 will never match; argument is of type the integer 16 {dict,mk_seg,1}: The clause matching on the integer 8 will never match; argument is of type the integer 16 {dict,mk_seg,1}: The clause matching on the integer 32 will never match; argument is of type the integer 16 {digraph,new_edge_id,1}: Cons will produce a non-proper list since its 2nd arg is number()! {digraph,new_vertex_id,1}: Cons will produce a non-proper list since its 2nd arg is number()! {qlc_pt,element_calls,3}: Cons will produce a non-proper list since its 2nd arg is {'var',0,reference()}! {qlc,abstract,2}: Call to '++'/2 will produce a non-proper list; 2nd arg is 'more'! {qlc,append_loop,2}: Cons will produce a non-proper list since its 2nd arg is (() -> any())! {qlc,cache,4}: Cons will produce a non-proper list since its 2nd arg is (() -> possibly_improper_list())! {qlc,cache_recall,2}: Cons will produce a non-proper list since its 2nd arg is (() -> possibly_improper_list())! {qlc,no_more,2}: Function has no local return {qlc,size_bin,2}: Cons will produce a non-proper list since its 2nd arg is binary() | tuple()! {qlc,no_dups,2}: Cons will produce a non-proper list since its 2nd arg is (() -> possibly_improper_list())! {qlc,ucache3,7}: Cons will produce a non-proper list since its 2nd arg is (() -> possibly_improper_list())! {qlc,ucache_recall,3}: Cons will produce a non-proper list since its 2nd arg is (() -> possibly_improper_list())! {ets,qlc_next,2}: Call to '++'/2 will produce a non-proper list; 2nd arg is (() -> (() -> (() -> (() -> any()) | possibly_improper_list()) | possibly_improper_list()) | possibly_improper_list())! {ets,qlc_prev,2}: Call to '++'/2 will produce a non-proper list; 2nd arg is (() -> (() -> (() -> (() -> any()) | possibly_improper_list()) | possibly_improper_list()) | possibly_improper_list())! {ets,qlc_select,1}: Call to '++'/2 will produce a non-proper list; 2nd arg is (() -> (() -> (() -> (() -> any()) | possibly_improper_list()) | possibly_improper_list()) | possibly_improper_list())! {erl_scan,more,7}: Call to '++'/2 will produce a non-proper list; 2nd arg is 'eof'! {erl_scan,done,5}: Call to '++'/2 will produce a non-proper list; 2nd arg is 'eof'! {erl_scan,tokens,3}: Call to '++'/2 will produce a non-proper list; 2nd arg is 'eof'! {erl_tar,create_header,3}: Cons will produce a non-proper list since its 2nd arg is binary()! {pg,master_loop,2}: Function has no local return {slave,relay1,1}: Function has no local return {pool,start,2}: Unsafe use of tuple as a fun in call to {lists,map,2} {pool,stat_loop,2}: Function has no local return {shell,eval_loop,3}: Function has no local return {shell,restricted_eval_loop,4}: Function has no local return -- jimmy From hasse@REDACTED Thu Mar 24 13:23:00 2005 From: hasse@REDACTED (Hans Bolinder) Date: Thu, 24 Mar 2005 13:23:00 +0100 Subject: Closing a dets In-Reply-To: References: <16961.25690.805769.594924@gargle.gargle.HOWL> Message-ID: <16962.45348.528987.416518@gargle.gargle.HOWL> [Tobias Lindahl:] > Actually, while constructing an counterexample I realised that the problem > is not related to shutting down the node. It seems to be related to > converting a sufficently large ets table to dets. You have found a bug in Dets. In R10B-5 the following patch will be included: stdlib/src/dets_v9.erl: 1884a1885,1886 > may_grow(#head{access = read}=Head, _N, _How) -> > {Head, ok}; stdlib/src/dets_v8.erl: 1075a1076,1077 > may_grow(#head{access = read}=Head, _N, _How) -> > {Head, ok}; Best regards, Hans Bolinder, Erlang/OTP From bjorn@REDACTED Thu Mar 24 13:54:32 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 24 Mar 2005 13:54:32 +0100 Subject: Dialyzer v1.3.0 released In-Reply-To: <20050324130526.D32855@dev1.localdomain.net> References: <20050324130526.D32855@dev1.localdomain.net> Message-ID: We like improper lists. And we turn off the warnings for improper lists when we run Dialyzer. /Bjorn Jimmy Olgeni writes: > On Thu, 24 Mar 2005, Tobias Lindahl wrote: > > > Once again we are proud to release a new version of Dialyzer. > > Maybe somebody should have a look into stdlib, I hope there are not that many improper lists :-) -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From thomasl_erlang@REDACTED Thu Mar 24 14:59:03 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 24 Mar 2005 05:59:03 -0800 (PST) Subject: International Erlang jobs In-Reply-To: 6667 Message-ID: <20050324135903.71421.qmail@web41901.mail.yahoo.com> --- Yvonne Erlandsson wrote: > We have launched an Erlang Jobs website ... > We have positions in four continents we are working > hard to fill up, /.../ When I see this, I realize that Erlang has grown up :-) (And, god help me, it _is_ kind of cool to read multiple job ads asking for Erlang expertise.) -- Thomas __________________________________ Do you Yahoo!? Make Yahoo! your home page http://www.yahoo.com/r/hs From erlang-list@REDACTED Thu Mar 24 16:31:17 2005 From: erlang-list@REDACTED (Dominic Williams) Date: Thu, 24 Mar 2005 16:31:17 +0100 Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: References: <20050324130526.D32855@dev1.localdomain.net> Message-ID: <4242DD45.6010008@dominicwilliams.net> Hi Bjorn, > We like improper lists. This sounds interesting. I don't yet have enough Erlang (or Lisp) experience to understand, though. I would really appreciate an explanation or an example! Thanks, Dominic Williams http://www.dominicwilliams.net ---- From thomasl_erlang@REDACTED Thu Mar 24 17:07:35 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 24 Mar 2005 08:07:35 -0800 (PST) Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: 6667 Message-ID: <20050324160735.8609.qmail@web41903.mail.yahoo.com> --- Dominic Williams wrote: > This sounds interesting. I don't yet have enough > Erlang (or Lisp) experience to understand, though. > I would really appreciate an explanation or an > example! My guess is that they like the following (implementation-) property: [X|Y] requires 2 words, while {X,Y} requires 3 words. So storing a pair as a cons saves 33% memory. Also, it's generally somewhat faster to access conses than tuples :-) Best, Thomas __________________________________ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail From bfulg@REDACTED Fri Mar 25 02:39:59 2005 From: bfulg@REDACTED (Brent Fulgham) Date: Thu, 24 Mar 2005 17:39:59 -0800 (PST) Subject: Dialyzer v1.3.0 released In-Reply-To: 6667 Message-ID: <20050325013959.38307.qmail@web81202.mail.yahoo.com> --- Tobias Lindahl wrote: > Once again we are proud to release a new version of > Dialyzer. > > Dialyzer v 1.3.0 is available for download at: Has anyone successfully used Dialyzer in a Windows environment? (If so, care to share your install script?) Thanks, -Brent From vlad_dumitrescu@REDACTED Fri Mar 25 22:59:38 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Fri, 25 Mar 2005 22:59:38 +0100 Subject: windows build Message-ID: Hi, Is it something in my setup, or is there soething weird in the windows build scripts? There aren't any ebin directories in the source archive (R10B-4) and they aren't being created in the build process, so I have to create them manually, which is a bore... It used to work before... :-) best regards, Vlad From vances@REDACTED Fri Mar 25 23:35:40 2005 From: vances@REDACTED (Vance Shipley) Date: Fri, 25 Mar 2005 17:35:40 -0500 Subject: wrong path for config.cache Message-ID: <20050325223540.GA19758@frogman.motivity.ca> In the recently released R10B-4 the configure files are creating a config.cache file which is ../config.cache relative to ERL_TOP. I discovered this because I didn't have write permission there. :) -Vance From robert.virding@REDACTED Sat Mar 26 22:30:00 2005 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 26 Mar 2005 22:30:00 +0100 Subject: Guards and side effects In-Reply-To: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB6@esealmw107.eemea.ericsson.se> References: <44D1EB33A5E44D4BBAFD4FB9E8CCCB4719EAB6@esealmw107.eemea.ericsson.se> Message-ID: <4245D458.1060604@telia.com> Sorry for missing this but I have not been on for a while. I think that Joe said it all in another thread: It's not an ugly wart - it's deliberate. Guards are merely extensions of pattern matching - they must be side effect free and capably of efficient compilation. Soon you hve the whole application in guards. Look at the Common Lisp format function. Robert Thomas Johnsson XA (LN/EAB) wrote: >(Subduing the hardcore functional programmer in me for a bit ... :-) > >I see really no compelling reason to prohibit side effects in guards, >in fact there may well be sensible uses, for instance, communicating >with another process to find out the value of the guard, consult ets tables >or other data bases, call a memo function .... > >The only tricky place, as far as I can see, is guard evaluation >in a receive: to evaluate guards to select a message in the own process in-queue, >it makes sense *not* to allow another 'recursive' receive to find the value of the guard, >but to issue a run time error instead. >( An aside: this is akin to black hole:ing in implementing a lazy functional language, where in >the process of evaluating a 'thunk' you need the value of the same thunk) > >There are many opportunities for the programmer to shoot him/herself in the foot, >but to hold the programmer in strict harness in this particular corner of the >language, and nowhere else, I find that misguided. > >-- Thomas > > > From vlad_dumitrescu@REDACTED Sun Mar 27 17:40:06 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Sun, 27 Mar 2005 17:40:06 +0200 Subject: Dialyzer win32 Message-ID: Hi, > Has anyone successfully used Dialyzer in a Windows > environment? (If so, care to share your install > script?) First problem is that linking to hipe_icode_type.hrl doesn't work: the -include directive doesn't follow links, because erl isn't a cygwin application. So I changed setup.sh to do a copy instead. Next problem is also related to the fact that erl is a Windows application, while the paths sent to it from setup.sh are cygwin-like. I am attaching a patch that tries to detect a cygwin installation and act accordingly. Haven't tested in a non-cygwin environment. best regards, Vlad -------------- next part -------------- A non-text attachment was scrubbed... Name: setup.sh.patch Type: application/octet-stream Size: 1702 bytes Desc: not available URL: From vlad_dumitrescu@REDACTED Sun Mar 27 19:33:37 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Sun, 27 Mar 2005 19:33:37 +0200 Subject: windows build Message-ID: Sorry I took your time, it was my setup: somehow a flawed tar implementation was used... regards, Vlad From james.hague@REDACTED Mon Mar 28 05:54:22 2005 From: james.hague@REDACTED (James Hague) Date: Sun, 27 Mar 2005 21:54:22 -0600 Subject: Improper lists [was: Dialyzer v1.3.0 released] Message-ID: Thomas Lindgren wrote: >So storing a pair as a cons saves 33% memory. Also, >it's generally somewhat faster to access conses than >tuples :-) In theory you could have a special type tag for "two element tuple" (and even three and four elements, depending how far you want to go), so there would be no benefit to storing a pair as a cons. The tradeoff is that that you have to handle those new type tags all over the place in the runtime, of course. James From matthias@REDACTED Mon Mar 28 10:11:45 2005 From: matthias@REDACTED (Matthias Lang) Date: Mon, 28 Mar 2005 10:11:45 +0200 Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: References: Message-ID: <16967.48193.531773.651618@antilipe.corelatus.se> James Hague writes: > In theory you could have a special type tag for "two element tuple" Apart from not being a tuple, isn't that exactly what a cons cell is?! Matt From vlad_dumitrescu@REDACTED Mon Mar 28 11:05:34 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Mon, 28 Mar 2005 11:05:34 +0200 Subject: Dialyzer wishlist Message-ID: Hi, I might have missed this possibility, but I'd like to be able to create a PLT for the whole OTP, so that it won't have to analyze the whole of it from scratch. Of course, one has to be able to choose different PLTs. Also, is it possible to do a global analyzis but exclude warnings from the standard libraries? I'd guess that if the libs were in a PLT, there would be no warnings from them? best regards, Vlad From james.hague@REDACTED Mon Mar 28 16:28:34 2005 From: james.hague@REDACTED (James Hague) Date: Mon, 28 Mar 2005 08:28:34 -0600 Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: <16967.48193.531773.651618@antilipe.corelatus.se> References: <16967.48193.531773.651618@antilipe.corelatus.se> Message-ID: > > In theory you could have a special type tag for "two element tuple" > > Apart from not being a tuple, isn't that exactly what a cons cell is?! Yes, of course. But "not being a tuple" is what matters :) You have two ways of representing a pair: {a,b} [a|b] But is [a|b], when used to store two arbitrary values, an efficiency hack or simply taking full advantage of cons cells? Certainly {a,b} is more, uh, "Erlangian." James From james.hague@REDACTED Mon Mar 28 18:04:48 2005 From: james.hague@REDACTED (James Hague) Date: Mon, 28 Mar 2005 10:04:48 -0600 Subject: What does the beam '%live' instruction do? Message-ID: Erlang: sum(A,B) -> A + B. BEAM: {bif,'+',{f,0},[{x,0},{x,1}],{x,0}}. {'%live',1}. <-- What does this do? return. From kostis@REDACTED Mon Mar 28 18:03:41 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 28 Mar 2005 18:03:41 +0200 (MEST) Subject: Dialyzer wishlist In-Reply-To: Mail from '"Vlad Dumitrescu" ' dated: Mon, 28 Mar 2005 11:05:34 +0200 Message-ID: <200503281603.j2SG3fUL013073@spikklubban.it.uu.se> Vlad Dumitrescu wrote: > I might have missed this possibility, but I'd like to be able to create a > PLT for the whole OTP, so that it won't have to analyze the whole of it from > scratch. Of course, one has to be able to choose different PLTs. > > Also, is it possible to do a global analyzis but exclude warnings from the > standard libraries? I'd guess that if the libs were in a PLT, there would be > no warnings from them? Both these suggestions are on our TODO list. The second could happen soon and we might release a patch for it. The first is better addressed by having pre-built PLTs in selected applications of the OTP distribution. Cannot expect to have this before R10B-5 and even this scenario is *very* optimistic. Best, Kostis From carsten@REDACTED Mon Mar 28 19:47:48 2005 From: carsten@REDACTED (Carsten Schultz) Date: Mon, 28 Mar 2005 19:47:48 +0200 Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: References: <16967.48193.531773.651618@antilipe.corelatus.se> Message-ID: <20050328174747.GA11772@penne.localnet> Hi! On Mon, Mar 28, 2005 at 08:28:34AM -0600, James Hague wrote: > > > In theory you could have a special type tag for "two element tuple" > > > > Apart from not being a tuple, isn't that exactly what a cons cell is?! > > Yes, of course. But "not being a tuple" is what matters :) You have > two ways of representing a pair: > > {a,b} > [a|b] > > But is [a|b], when used to store two arbitrary values, an efficiency > hack or simply taking full advantage of cons cells? Certainly {a,b} > is more, uh, "Erlangian." Some of the messages were like Cons will produce a non-proper list since its 2nd arg is (() -> possibly_improper_list())! This looks to me like a lazy list, so I can understand that [ | ] is used. Carsten -- Carsten Schultz (2:38, 33:47) http://carsten.codimi.de/ PGP/GPG key on the pgp.net key servers, fingerprint on my home page. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From joelr1@REDACTED Mon Mar 28 22:27:55 2005 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 28 Mar 2005 21:27:55 +0100 Subject: RUDP, shared libraries, etc. Message-ID: <20050328212755.18949@smtp.gmail.com> Folks, I'm writing client/server poker software and have a few questions regarding Erlang... Is there a reliable UDP implementation for Erlang? I would like my server to support at least 10,000 simultaneous users and it seems that I could only do 1024 with gen_tcp due to a Unix select limitation. This seems like a non-issue with UDP but what's the best way to ensure that messages are ordered and positively delivered? The documentation states that messages sent from process A to B are indeed ordered but does this apply to UDP? I would like to build the user interface using the Torque Game Engine (www.garagegames.com) and it uses a custom main on Windows and Mac OSX which makes it very hard to build it into a shared library to be called from Erlang. What is the best way to interface with Erlang here? Can I make the Erlang runtime and my code into a shared library to be loaded into C++? P.S. I'm using Common Lisp right now Thanks, Joel -- http://wagerlabs.com/tech From serge@REDACTED Mon Mar 28 23:28:08 2005 From: serge@REDACTED (Serge Aleynikov) Date: Mon, 28 Mar 2005 16:28:08 -0500 Subject: process groups - bug or feature? Message-ID: <424876E8.40902@hq.idt.net> Hi All, I wonder if this is a bug or a feature: Erlang (BEAM) emulator version 5.4.3 [source] [hipe] [threads:8] Eshell V5.4.3 (abort with ^G) (srv@REDACTED)1> nodes(). [] (srv@REDACTED)2> pg2:create(test). ok (srv@REDACTED)3> pg2:get_members(test). [] (srv@REDACTED)4> pg2:join(test, self()). ok (srv@REDACTED)5> pg2:get_members(test). [<0.36.0>] (srv@REDACTED)6> pg2:join(test, self()). ok (srv@REDACTED)7> pg2:get_members(test). [<0.36.0>,<0.36.0>] While this is not documented, I would intuitively assume that the same process should not be allowed to join a process group more than once. Could someone explain if this is a bug or a feature? Also pg module sends motifications to other members of a group when a member joins/leaves the group. pg2 does not. Is there a reason not to notify other group members of membership changes? Thanks, Serge From kostis@REDACTED Mon Mar 28 23:46:32 2005 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 28 Mar 2005 23:46:32 +0200 (MEST) Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: Mail from 'Carsten Schultz ' dated: Mon, 28 Mar 2005 19:47:48 +0200 Message-ID: <200503282146.j2SLkWEu029145@spikklubban.it.uu.se> Carsten Schultz wrote: > > Some of the messages were like > > Cons will produce a non-proper list since its 2nd arg is=20 > (() -> possibly_improper_list())! > > This looks to me like a lazy list, so I can understand that [ | ] is used. Indeed. I would personally like to exclude these from the list of Dialyzer discrepancies, but it is not very clear at this point how to recognize lazy lists -- which BTW is the only type of improper lists that I can see strong arguments for liking. For the rest of the uses of improper lists, the disadvantages outweigh the benefits, IMO. Cheers, Kostis. From frank@REDACTED Tue Mar 29 00:13:19 2005 From: frank@REDACTED (Frank Seesink) Date: Mon, 28 Mar 2005 17:13:19 -0500 Subject: MS Windows build use of versions in directory names/paths Message-ID: <4248817F.1050500@mail.wvnet.edu> I'm relatively new to Erlang, so please excuse me if this has been covered. I've searched the FAQs and not found a reasoning, so thought I would post here. What exactly is the reasoning behind using version numbers in the directory names/paths in the Windows prebuilt binaries? That is, the latest version of Erlang as I write this is Erlang/OTP R10B-4 (release 23 Mar 2005). Performing a vanilla/standard install, the user has Erland installed in x:\Program Files\erl5.4.5 and there is a subdirectory x:\Program Files\erl5.4.5\erts-5.4.5 Why does the Erlang development team (or the Windows installer creator) just install to x:\Program Files\erl\ with a subdirectory x:\Program Files\erl\erts ? Here is why I ask. I notice that often open source apps do this sort of thing, especially when they have ties to the *nix world. I also know that in the *nix world, it is common to decompress appX version A.B.C in a directory such as ./appX-A.B.C, then create a symlink ./app-X -> ./appX-A.B.C This allows the path to the application to remain constant, regardless of updates/changes. If appX version A.B.D comes out, that is simply decompressed to its own folder and the 'app-X' symlink changed to point to the new version. However, this is not how things are done typically in the Windows world. I've gotten used to certain open source projects installing in a different directory with reach version release. For example, the game BZFlag v2.0.0 installed in x:\Program Files\BZFlag2.0.0 whereas BZFlag 2.0.2 installed in x:\Program Files\BZFlag2.0.2 This is annoying enough, as my personal firewall rules must be adjusted with each release. Average Windows users will not deal with this well, as it gets to be quite annoying. But this is tolerable with games as you typically do not build/layer other applications on top of them. However, programming languages like Erlang exist almost exclusively for just that reason: to layer other applications on top of them. Other examples might be Perl, PHP, etc. Now in the Windows world, one of the most common variants of Perl is ActiveState's ActivePerl, and their installer by default will install Perl in x:\Perl The documentation on PHP similary advocates installing in a directory such as x:\PHP And with a new version release, updating is simply a matter of replacing the old files with the new ones in most cases (barring those users who heavily customize/add/change/delete files from the distribution). The advantage of this approach is that the PATH to a given language's files/engine remains the same. This makes script writing easier, not to mention the above firewall rule situation where you have application-level firewalling. There is no "moving target" as it were. This issue came up for me as my introduction to Erlang came from looking into ejabberd, the Erlang-based Jabber/XMPP server (http://ejabberd.jabber.ru). Installing/running ejabberd requires Erlang, of course. And the ejabberd installer offers to create a Windows service so that a user can run ejabberd on system startup. However, the installation of this NT service involves launching Erlang and having it fire up ejabberd. This, of course, necessitates putting the PATH to the current installed version of Erlang into the appropriate place in the Windows Registry (HKLM\SYSTEM\CurrentControlSet\Services\ejabberd...) where the NT service is set. (Note this is not something many users will understand.) If a new version of Windows software comes out, most people simply either run the latest installer on top of the old install, OR often they will first uninstall the old version before installing the new one. I tend to do the latter so not sure how well the former might avoid this. That is, if I install R10B-4 on top of R10B-3, I'm not sure if it will simply keep the current path or not. But using the "uninstall, then install" approach results in a lot of things suddenly breaking. The ejabberd service is no longer pointing to the right directory for Erlang. Any scripts or other items which also required this path are also snafu'd. Note this does NOT happen with updates to Perl or PHP, for example. So I was wondering why the Erlang team chose this approach. Noting that many other open source projects do NOT resort to this in Windows by default--e.g., Apache1/2 webservers, AWStats, Dev-PHP, Ethereal, FeedReader, FileZilla/Server, Neverball, OpenSSL, PuTTY, WinMerge--it seems it might be best NOT to put a version number in the path, but rather let the user find the version either by looking at a README in the installed directory, or in the case of Erlang, simply by running the shell. Especially noting that updating Erlang itself at present can result in almost ANY Erlang app breaking, and that the number of Erlang apps will be >= 1 (otherwise why install Erlang?), this means with each release of Erlang, users may have to go through and not just install the latest Erlang, but they will have to go adjusting any and all Erlang apps such as ejabberd which need the path information to work properly. Thanks for any insight into this. At present I resort to digging through the system and adjusting the path where necessary, but I can assure you that most Windows users will NOT do this. The end result is that adoption of Erlang as a language--just as I started playing due to ejabberd--will be hindered as others may be frustrated/annoyed by this process (and I believe rightfully so). I know the folks behind ejabberd swear by Erlang, and it definitely has some interesting features. But even their image will be tarnished any time someone updates Erlang only to find that suddenly ejabberd will not run anymore. P.S. I tried adjusting the install path for Erlang, but the best I could get was x:\Program Files\erl Unfortunately, the subdirectory for ERTS is still x:\Program Files\erl\erts-5.4.5 And as ejabberd uses this path, BOOM! From erlang@REDACTED Tue Mar 29 01:30:48 2005 From: erlang@REDACTED (Michael McDaniel) Date: Mon, 28 Mar 2005 15:30:48 -0800 Subject: 'please report this bug' Message-ID: <20050328233048.GF10107@fangora.autosys.us> I discovered the following consistency check problem due to an incorrect assignment I had in some real code. I distilled down to the nonsense code below to reproduce the problem. ~Michael $ uname -a Linux fangora 2.6.4-52-default #1 Wed Apr 7 02:08:30 UTC 2004 i686 i686 i386 GNU/Linux $ erlc foo.erl foo: function out/1+15: Internal consistency check failed - please report this bug. Instruction: {get_tuple_element,{x,1},2,{x,2}} Error: {bad_type,{needed,{tuple_element,3}},{actual,{tuple,2}}}: ./foo.erl:17: Warning: variable 'Arg' is unused ./foo.erl:22: Warning: variable 'Rr' is unused ./foo.erl:27: Warning: wrong number of arguments in format call ./foo.erl:33: Warning: this clause cannot match because a previous clause at line 25 always matches ------------------------------------------------------------------------------- % a nonsense program to demonstrate the above Internal consistency check bug % -module(foo). -author('Michael McDaniel dba Automated Systems, erlang@REDACTED'). -compile(export_all). -record( ics, { a = "a" , b = 'b' , x = "" }). out(Arg) -> Tag = true , R = #ics{x=233} , { _, Rr} = R#ics{} , %% WRONG assignment if Tag -> [ { html, io_lib:format('~s~s~n~p~n~s~n', [R#ics.a , R#ics.b , R#ics.x ]) } ] ; true -> [ { html, io_lib:format('~s~n', [Tag])} ] end . %% end From vances@REDACTED Tue Mar 29 06:39:59 2005 From: vances@REDACTED (Vance Shipley) Date: Mon, 28 Mar 2005 23:39:59 -0500 Subject: run_erl/to_erl on FreeBSD Message-ID: <20050329043958.GD836@blank.motivity.ca> Is anyone able to use R10B-4 to_erl with FreeBSD? Here (4.10-STABLE) I see run_erl fail to open the pipe to write on and to_erl gets EOF reading from it's pipe. I spent quite a bit of time today trying to get to the bottom of it but I'm stumped. -Vance From vlad_dumitrescu@REDACTED Tue Mar 29 08:42:27 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Tue, 29 Mar 2005 08:42:27 +0200 Subject: MS Windows build use of versions in directory names/paths References: <4248817F.1050500@mail.wvnet.edu> Message-ID: Hi, This isn't an answer, but a partial workaround. I use Junction Link Magic (http://www.rekenwonder.com/linkmagic.htm) to create a hard link to the current Erlang distribution directory (only works with NTFS). However, this doesn't solve the problem with the erts directory... regards, Vlad From raimo@REDACTED Tue Mar 29 08:49:49 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 29 Mar 2005 08:49:49 +0200 Subject: What does the beam '%live' instruction do? References: Message-ID: It does nothing. It is an internal instruction that tells subsequent compiler passes that there in this case are 1 X registers that contain valid values. The presence of this instruction facilitates optimization. It does not need to be around in the assembly listing, but is there anyway. The beam code validator (the last compiler pass) checks that there are as many valid X registers as this instruction states; if it were not, somthing would be wrong. The instruction is not stored in .beam code, so it is not present in a disassembly listing. james.hague@REDACTED (James Hague) writes: > Erlang: > > sum(A,B) -> A + B. > > BEAM: > > {bif,'+',{f,0},[{x,0},{x,1}],{x,0}}. > {'%live',1}. <-- What does this do? > return. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo@REDACTED Tue Mar 29 09:04:36 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 29 Mar 2005 09:04:36 +0200 Subject: RUDP, shared libraries, etc. References: <20050328212755.18949@smtp.gmail.com> Message-ID: Erlang uses poll() instead of select() on platforms where it exists, and it does not have the 1024 file descriptor limitation of select(); however, the OS itself may limit the number of allowed file descriptors. You can use "ulimit -n" to change the number of allowed file descriptors for a shell. On my Solaris 9 machine, I cannot exceed 1024 unless being root, but then it does not stop for at least 524288 (I did not try further). The same applies to Red Hat 9 Linux. How efficient it is for half a Mega file descriptors I do not know... UDP has its quirks that gen_udp reflects, e.g message ordering is not guaranteed. It is not related to Erlang inter-process communiation at all. About Erlang as a shared library: the Erlang emulator (Beam) also needs to be running main(). You might write interface code in C to dynamically link with the game engine, and communicate through an Erlang port program. I do not know, I hope someone else will give a better answer on this point. joelr1@REDACTED (Joel Reymont) writes: > Folks, > > I'm writing client/server poker software and have a few questions > regarding Erlang... > > Is there a reliable UDP implementation for Erlang? I would like my server > to support at least 10,000 simultaneous users and it seems that I could > only do 1024 with gen_tcp due to a Unix select limitation. > > This seems like a non-issue with UDP but what's the best way to ensure > that messages are ordered and positively delivered? The documentation > states that messages sent from process A to B are indeed ordered but does > this apply to UDP? > > I would like to build the user interface using the Torque Game Engine > (www.garagegames.com) and it uses a custom main on Windows and Mac OSX > which makes it very hard to build it into a shared library to be called > from Erlang. What is the best way to interface with Erlang here? Can I > make the Erlang runtime and my code into a shared library to be loaded > into C++? > > P.S. I'm using Common Lisp right now > > Thanks, Joel > > -- > http://wagerlabs.com/tech > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From matthias@REDACTED Tue Mar 29 09:16:12 2005 From: matthias@REDACTED (Matthias Lang) Date: Tue, 29 Mar 2005 09:16:12 +0200 Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: References: <16967.48193.531773.651618@antilipe.corelatus.se> Message-ID: <16969.188.79545.705496@antilipe.corelatus.se> James Hague writes: > But is [a|b], when used to store two arbitrary values, an efficiency > hack or simply taking full advantage of cons cells? Certainly {a,b} > is more, uh, "Erlangian." It's possible to see it the other way around. Cons cells are universal. Build whatever you want with them: lists, trees, ... It's tuples that are the efficiency hack with the arbitrary size limit. Why not go fully erlangian and eliminate cons cells entirely and just use tuples for everything, i.e. "String" could be sugar for {$S, {$t, {$r, {$i, {$n, {$g, {}}}}}}} you could even include the length of the tail at every step for added popularity ;-) Matt From raimo@REDACTED Tue Mar 29 09:36:43 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 29 Mar 2005 09:36:43 +0200 Subject: MS Windows build use of versions in directory names/paths References: <4248817F.1050500@mail.wvnet.edu> Message-ID: Well, that was a long question... Erlang comes from the Unix world, and is built for handling hot code upgrades, so if you are a commercial customer and installs an OTP patch, the patched application is installed in a subdirectory with a new version number. The old version is kept in case you would want to back out the patch later on. That is the main reason for the version numbers on the directories. The patch script is the same for Windows and Unix, and I guess it is not even aware of the platform. Large parts of OTP (at least the code server) also depends on this directory structure. And rewriting all code would cost too much. But, as a side note. In our internal development environment we have a directory tree without version numbers. Version handling is done in ClearCase. So, probably, if you would take a Windows (or Unix) installation and strip all version suffixes from the Erlang installation tree, and rebuild the start scripts (the start script compiler takes an undocumented flag to build without version numbers) - it would work. _But_ OTP patch handling would _not_ work, and that is why the installation tree does not look that way. This is an area where Windows and Unix security approaches still clashes head on. E.g ZoneAlarm on windows checking which installed program that tries to get out - that is not really what a firewall is supposed to do. In the Unix world a firewall is a separate box, and it checks which user and which machine that tries to get out and knows nothing of program installation pathes. The ZoneAlarm check is really a Worm check. Why ejabberd uses the registry is another question. Erlang/OTP and many others has moved away from using the registry. I think the Erlang/OTP installer fixes the path, so if ejabberd used the Erlang that was in the path, there would be no problem (guessing here...) frank@REDACTED (Frank Seesink) writes: > I'm relatively new to Erlang, so please excuse me if this has been > covered. I've searched the FAQs and not found a reasoning, so thought I > would post here. > > What exactly is the reasoning behind using version numbers in the > directory names/paths in the Windows prebuilt binaries? That is, the > latest version of Erlang as I write this is Erlang/OTP R10B-4 (release > 23 Mar 2005). Performing a vanilla/standard install, the user has > Erland installed in > > x:\Program Files\erl5.4.5 > > and there is a subdirectory > > x:\Program Files\erl5.4.5\erts-5.4.5 > > Why does the Erlang development team (or the Windows installer creator) > just install to > > x:\Program Files\erl\ > > with a subdirectory > > x:\Program Files\erl\erts > ? > > Here is why I ask. I notice that often open source apps do this sort of > thing, especially when they have ties to the *nix world. I also know > that in the *nix world, it is common to decompress appX version A.B.C in > a directory such as ./appX-A.B.C, then create a symlink > > ./app-X -> ./appX-A.B.C > > This allows the path to the application to remain constant, regardless > of updates/changes. If appX version A.B.D comes out, that is simply > decompressed to its own folder and the 'app-X' symlink changed to point > to the new version. > > However, this is not how things are done typically in the Windows world. > I've gotten used to certain open source projects installing in a > different directory with reach version release. For example, the game > BZFlag v2.0.0 installed in > > x:\Program Files\BZFlag2.0.0 > > whereas BZFlag 2.0.2 installed in > > x:\Program Files\BZFlag2.0.2 > > This is annoying enough, as my personal firewall rules must be adjusted > with each release. Average Windows users will not deal with this well, > as it gets to be quite annoying. But this is tolerable with games as > you typically do not build/layer other applications on top of them. > > However, programming languages like Erlang exist almost exclusively for > just that reason: to layer other applications on top of them. Other > examples might be Perl, PHP, etc. Now in the Windows world, one of the > most common variants of Perl is ActiveState's ActivePerl, and their > installer by default will install Perl in > > x:\Perl > > The documentation on PHP similary advocates installing in a directory > such as > > x:\PHP > > And with a new version release, updating is simply a matter of replacing > the old files with the new ones in most cases (barring those users who > heavily customize/add/change/delete files from the distribution). > > The advantage of this approach is that the PATH to a given language's > files/engine remains the same. This makes script writing easier, not to > mention the above firewall rule situation where you have > application-level firewalling. There is no "moving target" as it were. > > This issue came up for me as my introduction to Erlang came from looking > into ejabberd, the Erlang-based Jabber/XMPP server > (http://ejabberd.jabber.ru). Installing/running ejabberd requires > Erlang, of course. And the ejabberd installer offers to create a > Windows service so that a user can run ejabberd on system startup. > > However, the installation of this NT service involves launching Erlang > and having it fire up ejabberd. This, of course, necessitates putting > the PATH to the current installed version of Erlang into the appropriate > place in the Windows Registry > (HKLM\SYSTEM\CurrentControlSet\Services\ejabberd...) where the NT > service is set. (Note this is not something many users will understand.) > > If a new version of Windows software comes out, most people simply > either run the latest installer on top of the old install, OR often they > will first uninstall the old version before installing the new one. I > tend to do the latter so not sure how well the former might avoid this. > That is, if I install R10B-4 on top of R10B-3, I'm not sure if it will > simply keep the current path or not. > > But using the "uninstall, then install" approach results in a lot of > things suddenly breaking. The ejabberd service is no longer pointing to > the right directory for Erlang. Any scripts or other items which also > required this path are also snafu'd. Note this does NOT happen with > updates to Perl or PHP, for example. > > So I was wondering why the Erlang team chose this approach. Noting that > many other open source projects do NOT resort to this in Windows by > default--e.g., Apache1/2 webservers, AWStats, Dev-PHP, Ethereal, > FeedReader, FileZilla/Server, Neverball, OpenSSL, PuTTY, WinMerge--it > seems it might be best NOT to put a version number in the path, but > rather let the user find the version either by looking at a README in > the installed directory, or in the case of Erlang, simply by running the > shell. Especially noting that updating Erlang itself at present can > result in almost ANY Erlang app breaking, and that the number of Erlang > apps will be >= 1 (otherwise why install Erlang?), this means with each > release of Erlang, users may have to go through and not just install the > latest Erlang, but they will have to go adjusting any and all Erlang > apps such as ejabberd which need the path information to work properly. > > Thanks for any insight into this. At present I resort to digging > through the system and adjusting the path where necessary, but I can > assure you that most Windows users will NOT do this. The end result is > that adoption of Erlang as a language--just as I started playing due to > ejabberd--will be hindered as others may be frustrated/annoyed by this > process (and I believe rightfully so). I know the folks behind ejabberd > swear by Erlang, and it definitely has some interesting features. But > even their image will be tarnished any time someone updates Erlang only > to find that suddenly ejabberd will not run anymore. > > P.S. I tried adjusting the install path for Erlang, but the best I could > get was > > x:\Program Files\erl > > Unfortunately, the subdirectory for ERTS is still > > x:\Program Files\erl\erts-5.4.5 > > And as ejabberd uses this path, BOOM! > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From matthias@REDACTED Tue Mar 29 10:41:47 2005 From: matthias@REDACTED (Matthias Lang) Date: Tue, 29 Mar 2005 10:41:47 +0200 Subject: RUDP, shared libraries, etc. In-Reply-To: <20050328212755.18949@smtp.gmail.com> References: <20050328212755.18949@smtp.gmail.com> Message-ID: <16969.5323.707153.380369@antilipe.corelatus.se> Joel Reymont writes: > Is there a reliable UDP implementation for Erlang? Not that I know of. Though I know people have played around with implementing parts of TCP in Erlang (most recently: Luke Gorrie), but I'd guess that none of that is intended/ready for serious use. > I would like my server > to support at least 10,000 simultaneous users and it seems that I could > only do 1024 with gen_tcp due to a Unix select limitation. A relatively conservative way around that limitation is to run multiple Erlang VMs on the same machine. Erlang is intended for that sort of design. A little bonus would be that your application would then be ready to scale to multiple machines. > This seems like a non-issue with UDP but what's the best way to ensure > that messages are ordered and positively delivered? The documentation > states that messages sent from process A to B are indeed ordered but does > this apply to UDP? UDP makes few guarantees. A UDP message may be quietly lost in transmission. A UDP message might arrive twice. UDP messages might arrive out-of-order. This is a property of UDP, it's not Erlang-specific. If you run UDP packets over the big, bad internet, you'll see the above happening, especially "quietly lost". If you run UDP packets over a switched ethernet, you'll rarely, if ever, see UDP misbehaving. UDP will seem reliable. YMMV. > I would like to build the user interface using the Torque Game Engine > (www.garagegames.com) and it uses a custom main on Windows and Mac OSX > which makes it very hard to build it into a shared library to be called > from Erlang. What is the best way to interface with Erlang here? See http://www.erlang.se/doc/doc-5.0.1/doc/tutorial/part_frame.html There are several ways to interface Erlang, each has advantages and disadvantages. I'd suggest starting with something simple first, e.g. running the two as seperate OS processes and communicating over a TCP socket. Invent your own simple protocol. If you want to make things more complicated, take a look at erl_interface. But be warned: the mailing list is littered with the corpses of beginners who have used erl_interface or linked-in drivers without understanding them and then become completely baffled by the resulting memory-corruption problems. > Can I make the Erlang runtime and my code into a shared library > to be loaded into C++? For practical purposes: no. Matt From bjorn@REDACTED Tue Mar 29 11:02:44 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 29 Mar 2005 11:02:44 +0200 Subject: What does the beam '%live' instruction do? In-Reply-To: References: Message-ID: It does nothing. It is there just to help the optimizing passes. /Bjorn James Hague writes: > Erlang: > > sum(A,B) -> A + B. > > BEAM: > > {bif,'+',{f,0},[{x,0},{x,1}],{x,0}}. > {'%live',1}. <-- What does this do? > return. > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bjorn@REDACTED Tue Mar 29 11:21:02 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 29 Mar 2005 11:21:02 +0200 Subject: 'please report this bug' In-Reply-To: <20050328233048.GF10107@fangora.autosys.us> References: <20050328233048.GF10107@fangora.autosys.us> Message-ID: Thanks! The enclosed patch corrects the problem. /Bjorn Michael McDaniel writes: > I discovered the following consistency check problem due to an incorrect assignment > I had in some real code. I distilled down to the nonsense code below to reproduce > the problem. > > ~Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: beam_validator.diff Type: text/x-patch Size: 5627 bytes Desc: beam_validator patch URL: -------------- next part -------------- -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From thomasl_erlang@REDACTED Tue Mar 29 11:34:25 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 29 Mar 2005 01:34:25 -0800 (PST) Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: 6667 Message-ID: <20050329093426.96942.qmail@web41905.mail.yahoo.com> --- James Hague wrote: > Thomas Lindgren wrote: > > >So storing a pair as a cons saves 33% memory. Also, > >it's generally somewhat faster to access conses > than > >tuples :-) > > In theory you could have a special type tag for "two > element tuple" > (and even three and four elements, depending how far > you want to go), > so there would be no benefit to storing a pair as a > cons. The > tradeoff is that that you have to handle those new > type tags all over > the place in the runtime, of course. Yeah, nothing really prevents that (apart from a small matter of implementation :-) The data representation itself is basically just a big trade-off, one which is deemed to be reasonably fast and compact for most Erlang programs and not too difficult to work with in the runtime system. (For an overview of various alternatives for Erlang, take a look at Mikael Petterson's report. I believe his tag scheme is the one in use nowadays. http://www.it.uu.se/research/reports/2000-029/ ) Best, Thomas __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From thomasl_erlang@REDACTED Tue Mar 29 11:42:09 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 29 Mar 2005 01:42:09 -0800 (PST) Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: 6667 Message-ID: <20050329094209.91936.qmail@web41904.mail.yahoo.com> --- Carsten Schultz wrote: > Some of the messages were like > > Cons will produce a non-proper list since its > 2nd arg is > (() -> possibly_improper_list())! > > This looks to me like a lazy list, so I can > understand that [ | ] is > used. Unless destructive assignment has crept back in, that doesn't seem lazy to me. (For readers hazy on the details: laziness entails evaluating an expression at most once, which is usually implemented by a reference cell pointing to a closure. When the value is first demanded, the closure is evaluated and the reference cell is updated with the result. Any subsequent demands will then use the evaluated result. Erlang doesn't have reference cells, though ... as far as I know. Subsequent demands will then just re-evaluate the closure.) Best, Thomas __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From her@REDACTED Tue Mar 29 12:03:05 2005 From: her@REDACTED (Helmut Enck-Radana) Date: Tue, 29 Mar 2005 12:03:05 +0200 Subject: MS Windows build use of versions in directory names/paths In-Reply-To: References: <4248817F.1050500@mail.wvnet.edu> Message-ID: <6.2.1.2.0.20050329113712.02fd9e50@paradigma-software.de> Hi, At 08:42 2005-03-29, Vlad wrote: >I use Junction Link Magic (http://www.rekenwonder.com/linkmagic.htm) to >create a hard link to the current Erlang distribution directory (only >works with NTFS). May be only nitpicking: AFAIK NTFS links to folders (junctions) are more like symbolic links, while NTFS hard links only can point to files. (Folders in NTFS aren't files.) There is a difference between a folder and a link to the folder: When you delete the folder, the content is gone and you can't reach it via the link. Links to files behave like hard links in a POSIX system. Unfortunately both aren't well supported by the Windows system tools. In Windows Explorer you can't see the difference between a link and a copy, and Windows Backup even treats links as if they were copies. -- helmut From vlad_dumitrescu@REDACTED Tue Mar 29 15:00:34 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Tue, 29 Mar 2005 15:00:34 +0200 Subject: MS Windows build use of versions in directory names/paths References: <4248817F.1050500@mail.wvnet.edu> <6.2.1.2.0.20050329113712.02fd9e50@paradigma-software.de> Message-ID: From: "Helmut Enck-Radana" > May be only nitpicking: AFAIK NTFS links to folders (junctions) are more > like symbolic links, while NTFS hard links only can point to files. > (Folders in NTFS aren't files.) There is a difference between a folder and > a link to the folder: When you delete the folder, the content is gone and > you can't reach it via the link. Links to files behave like hard links in a > POSIX system. Yes, I mixed up my metaphors -- you are right. > Unfortunately both aren't well supported by the Windows system tools. In > Windows Explorer you can't see the difference between a link and a copy, > and Windows Backup even treats links as if they were copies. Also true. For my own purposes, I can live with that. Your mileage may vary. regards, Vlad From ulf.wiger@REDACTED Tue Mar 29 15:24:00 2005 From: ulf.wiger@REDACTED (Ulf Wiger (AL/EAB)) Date: Tue, 29 Mar 2005 15:24:00 +0200 Subject: send/receive faster than ets:lookup? Message-ID: Rather than spending the next hour or so meditating over the C code in BEAM, I thought I'd ask the list: I slapped together a small benchmark to show that the cost of sending/receiving a data object in erlang is no more expensive than reading same object from an ETS table (I know: an 'apples and oranges' comparison). The result surprised me somewhat. It appears as if a client-server call is _much_ more efficient than ets:lookup() for large data. The OTP version is R10B-2: First runs with payload being lists:seq(1,10). 55> eptest:run(1000). [{proc,{2081,2047,2045}},{ets,{1987,1049,1376}},{empty,107}] 56> eptest:run(1000). [{proc,{2098,2064,8537}},{ets,{1935,1050,1384}},{empty,107}] 57> eptest:run(1000). [{proc,{2103,2070,2068}},{ets,{1978,1028,1375}},{empty,110}] 58> eptest:run(1000). [{proc,{2083,2048,2125}},{ets,{1821,1042,1389}},{empty,112}] 59> eptest:run(1000). [{proc,{2114,2083,2080}},{ets,{1886,1046,1382}},{empty,110}] Explanation: - The 'proc' pass runs the same test three times: one process sends a {self(), get} message to a server process, and receives {Server, Data} back; repeat a 1000 times. i.e. 1000 client-server calls took 2,064 ms - The 'ets' pass writes Data to an initially empty ets table, using the iteration counter as key; the second number is for writing same data again, overwriting the old -- thus, no re-hashing; the third number is for ets:lookup(). I.e. 1000 ets:lookup() took 1,38 ms. It doesn't feel too surprising that ets:lookup() beats the client-server call, for small payload. - The 'empty' pass just runs through an empty loop 1000 times. Next, same test but with payload being lists:seq(1,1000). 61> eptest:run(1000). [{proc,{26812,32757,27390}},{ets,{60121,37657,47838}},{empty,101}] 62> eptest:run(1000). [{proc,{27575,28201,26403}},{ets,{47367,37911,50910}},{empty,100}] 63> eptest:run(1000). [{proc,{26866,27328,26295}},{ets,{51932,38646,48290}},{empty,103}] 64> eptest:run(1000). [{proc,{26635,31830,26444}},{ets,{44664,40211,47878}},{empty,110}] 65> eptest:run(1000). [{proc,{27073,38103,29029}},{ets,{41096,37969,48585}},{empty,104}] Now, what happened here? Why did ets:lookup() get so expensive all of a sudden? 2,6 ms for the client-server call vs. 4.78 ms for the ets:lookup(). The ratio is 1.8x. For lists:seq(1,10000), the ratio is the same. I checked the key distribution in the ets table. For 1000 keys, sequential 1-1000, the average bucket length was 3.9, and the biggest bucket contained 12 objects. 9 out of 256 buckets were empty. But the difference seemed the same even for much smaller N. Is the term copy between process heaps more efficient than the copy from ets to process heap? Benchmark source attached. /Uffe -------------- next part -------------- A non-text attachment was scrubbed... Name: eptest.erl Type: application/octet-stream Size: 2490 bytes Desc: eptest.erl URL: From vlad_dumitrescu@REDACTED Tue Mar 29 16:04:13 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Tue, 29 Mar 2005 16:04:13 +0200 Subject: MS Windows build use of versions in directory names/paths References: <4248817F.1050500@mail.wvnet.edu> Message-ID: From: "Raimo Niskanen" > Why ejabberd uses the registry is another question. Erlang/OTP > and many others has moved away from using the registry. I think > the Erlang/OTP installer fixes the path, so if ejabberd used the > Erlang that was in the path, there would be no problem (guessing > here...) I don't know about ejabberd, but I guess it's the same issue I got when running an erlang node as a Windows service: erlsrv.exe uses the registry. By default the VM used is the one in erlsrv's directory which is $ERL_TOP/erts-n.n.n By using a link to the current erlang directory, one can write for example erlsrv -m c:\program\erl\bin\erl.exe It would be nicer if cygwin links would work outside the cygwin environment - but I guess then it wouldn't be Windows anymore :-) regards, Vlad From james.hague@REDACTED Tue Mar 29 16:27:29 2005 From: james.hague@REDACTED (James Hague) Date: Tue, 29 Mar 2005 08:27:29 -0600 Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: <16969.188.79545.705496@antilipe.corelatus.se> References: <16967.48193.531773.651618@antilipe.corelatus.se> <16969.188.79545.705496@antilipe.corelatus.se> Message-ID: Matthias Lang wrote: > > It's possible to see it the other way around. Cons cells are > universal. True. But the way I see it, the fundamental difference is that lists are designed to contain an arbitrary number of values, whereas tuples contain a fixed number. If I see {10, 3}, then I know that those are a pair of values with some meaning. If I see [10, 3], however, then I don't know if that's a pair or two separate values or what. Tuples make data be atomic. Separating the two also makes output unambiguous: {1,2} displays as {1,2} [1|2] displays as [1|2] but {1,[2,3]} displays as {1,[2,3]} [1|[2,3]] displays as [1,2,3] James From tty@REDACTED Tue Mar 29 17:36:49 2005 From: tty@REDACTED (tty@REDACTED) Date: Tue, 29 Mar 2005 10:36:49 -0500 Subject: RUDP, shared libraries, etc. Message-ID: You can increase your tcp limit using ulimit. $> ulimit -n 11000 as root. For some reason 'ulimit -n unlimited' did not work for me on Linux (SuSE). tee -------- Original Message -------- From: "Joel Reymont" Apparently from: owner-erlang-questions@REDACTED To: Subject: RUDP, shared libraries, etc. Date: Mon, 28 Mar 2005 21:27:55 +0100 > Folks, > > I'm writing client/server poker software and have a few questions > regarding Erlang... > > Is there a reliable UDP implementation for Erlang? I would like my server > to support at least 10,000 simultaneous users and it seems that I could > only do 1024 with gen_tcp due to a Unix select limitation. > > This seems like a non-issue with UDP but what's the best way to ensure > that messages are ordered and positively delivered? The documentation > states that messages sent from process A to B are indeed ordered but does > this apply to UDP? > > I would like to build the user interface using the Torque Game Engine > (www.garagegames.com) and it uses a custom main on Windows and Mac OSX > which makes it very hard to build it into a shared library to be called > from Erlang. What is the best way to interface with Erlang here? Can I > make the Erlang runtime and my code into a shared library to be loaded > into C++? > > P.S. I'm using Common Lisp right now > > Thanks, Joel > > -- > http://wagerlabs.com/tech From erlang@REDACTED Tue Mar 29 17:47:41 2005 From: erlang@REDACTED (Michael McDaniel) Date: Tue, 29 Mar 2005 07:47:41 -0800 Subject: 'please report this bug' In-Reply-To: References: <20050328233048.GF10107@fangora.autosys.us> Message-ID: <20050329154740.GP10107@fangora.autosys.us> Erlang/OTP developers ROCK!! Thank you for your quick response. ~Michael On Tue, Mar 29, 2005 at 11:21:02AM +0200, Bjorn Gustavsson wrote: > Thanks! > > The enclosed patch corrects the problem. > > /Bjorn > > Michael McDaniel writes: > > > I discovered the following consistency check problem due to an incorrect assignment > > I had in some real code. I distilled down to the nonsense code below to reproduce > > the problem. > > > > ~Michael > > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From carsten@REDACTED Tue Mar 29 18:59:47 2005 From: carsten@REDACTED (Carsten Schultz) Date: Tue, 29 Mar 2005 18:59:47 +0200 Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: <20050329094209.91936.qmail@web41904.mail.yahoo.com> References: <20050329094209.91936.qmail@web41904.mail.yahoo.com> Message-ID: <20050329165946.GA5349@blofield.math.tu-berlin.de> On Tue, Mar 29, 2005 at 01:42:09AM -0800, Thomas Lindgren wrote: > > --- Carsten Schultz wrote: > > > Some of the messages were like > > > > Cons will produce a non-proper list since its > > 2nd arg is > > (() -> possibly_improper_list())! > > > > This looks to me like a lazy list, so I can > > understand that [ | ] is > > used. > > Unless destructive assignment has crept back in, that > doesn't seem lazy to me. Of course, you are right. So they are simply non-strict or whatever. To get real laziness, one caching process per cons cell could be used ;-) Carsten -- Carsten Schultz (2:38, 33:47) http://carsten.codimi.de/ PGP/GPG key on the pgp.net key servers, fingerprint on my home page. From vlad_dumitrescu@REDACTED Tue Mar 29 19:48:21 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Tue, 29 Mar 2005 19:48:21 +0200 Subject: MS Windows build use of versions in directory names/paths References: <4248817F.1050500@mail.wvnet.edu> Message-ID: > By using a link to the current erlang directory, one can write for example > > erlsrv -m c:\program\erl\bin\erl.exe Sorry, I did speak too early. The Windows service manager stores also the path to erlsrv.exe, which means that one also has to create a link $ERL_TOP/erts to $ERL_TOP/erts-n.n.n /Vlad From klacke@REDACTED Tue Mar 29 21:33:55 2005 From: klacke@REDACTED (klacke@REDACTED) Date: Tue, 29 Mar 2005 21:33:55 +0200 Subject: RUDP, shared libraries, etc. In-Reply-To: <20050328212755.18949@smtp.gmail.com> References: <20050328212755.18949@smtp.gmail.com> Message-ID: <20050329193355.GA783@hyber.org> On Mon, Mar 28, 2005 at 09:27:55PM +0100, Joel Reymont wrote: > I would like to build the user interface using the Torque Game Engine > (www.garagegames.com) and it uses a custom main on Windows and Mac OSX > which makes it very hard to build it into a shared library to be called > from Erlang. What is the best way to interface with Erlang here? Can I > make the Erlang runtime and my code into a shared library to be loaded > into C++? > It's always highly problematic to integrate two programs that both require main() for filedescriptor polling. Start off by keeping Erlang and Torque in separate processes and invent some IPC mechansim between them. TCP sockets over loopback is fast. /klacke -- Claes Wikstrom -- Caps lock is nowhere and http://www.hyber.org -- everything is under control From joelr1@REDACTED Tue Mar 29 21:53:30 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 29 Mar 2005 20:53:30 +0100 Subject: RUDP, shared libraries, etc. In-Reply-To: <16969.5323.707153.380369@antilipe.corelatus.se> References: <16969.5323.707153.380369@antilipe.corelatus.se> Message-ID: <20050329205330.27783@smtp.gmail.com> > Matthias Lang wrote: >A relatively conservative way around that limitation is to run >multiple Erlang VMs on the same machine. Erlang is intended for that >sort of design. A little bonus would be that your application would >then be ready to scale to multiple machines. Is there an idiom to implement this scalability in Erlang? That is how do I run multiple instances of the VM on the same machine and have external C code see them as one single server? Thanks, Joel -- http://wagerlabs.com/tech From vlad_dumitrescu@REDACTED Tue Mar 29 22:12:33 2005 From: vlad_dumitrescu@REDACTED (Vlad Dumitrescu) Date: Tue, 29 Mar 2005 22:12:33 +0200 Subject: Taking over a global name Message-ID: Hi, Since this is something that isn't easy to test thouroughly, because of all the strange race conditions, I hope someone will have a little spare time to look at the following code and review it. The problem to be solved is how to do a takeover between two processes, without OTP startup files (as Ulf showed us earlier). One major requirement is that no messages should be lost or come out of order. The code doesn't handle errors, but is supposed to show the bare bones of my implementation. Please feel free to criticize, and show simpler implementations if they are possible. The crux lies in takeover/2: the current process should be put on hold (so it doesn't eat any more messages), the name is re_registered (the new process must be in a special state, so that no messages are eaten), old process flushes any messages to the new ones, and then the new one gets to run freely. This is probably very similar to the application start phases described by Ulf, but useful if one wants to do it with "simple" processes. Is there a reason why there isn't a re_register bif for local names? best regards, Vlad -------------- next part -------------- A non-text attachment was scrubbed... Name: takeover.erl Type: application/octet-stream Size: 920 bytes Desc: not available URL: From joelr1@REDACTED Tue Mar 29 22:16:03 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 29 Mar 2005 21:16:03 +0100 Subject: RUDP, shared libraries, etc. In-Reply-To: <20050329193355.GA783@hyber.org> References: <20050329193355.GA783@hyber.org> Message-ID: <20050329211603.13847@smtp.gmail.com> > klacke@REDACTED wrote: >Start off by keeping Erlang and Torque in separate processes >and invent some IPC mechansim between them. >TCP sockets over loopback is fast. There's not my logic in my GUI as it simply does what the server tells it. I think I'll just have the clients represented as ports in Erlang and confine all Erlang to the server side. I'm now looking for the idiom to implement load-balancing and scalability so that a bunch of Erlang nodes look like one server to the clients. All pointers are appreciated! Thanks, Joel -- http://wagerlabs.com/tech From joelr1@REDACTED Tue Mar 29 23:06:04 2005 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 29 Mar 2005 22:06:04 +0100 Subject: Standalone Erlang Message-ID: <20050329220604.8240@smtp.gmail.com> Folks, Is Standalone Erlang still used? I don't find elink in my R10B tree on Mac OSX. What is the proper way to package an Erlang app for installation on Windows? Is is the same for other platforms? Thanks, Joel -- http://wagerlabs.com/tech From sean.hinde@REDACTED Wed Mar 30 00:16:12 2005 From: sean.hinde@REDACTED (Sean Hinde) Date: Tue, 29 Mar 2005 23:16:12 +0100 Subject: run_erl/to_erl on FreeBSD In-Reply-To: <20050329043958.GD836@blank.motivity.ca> References: <20050329043958.GD836@blank.motivity.ca> Message-ID: <82b7d977b3afa950c10e7b109e194da1@mac.com> Hi Vance, The only thing that changed (in R10B-2 as far as I can see) is that the call to mknod in run_erl.c was replaced with a call to mkfifo in run_erl.c. This fixed things for OS X, but maybe not all BSDs are equal in this regard. Regards, Sean On 29 Mar 2005, at 05:39, Vance Shipley wrote: > > Is anyone able to use R10B-4 to_erl with FreeBSD? > > Here (4.10-STABLE) I see run_erl fail to open the pipe to > write on and to_erl gets EOF reading from it's pipe. I > spent quite a bit of time today trying to get to the bottom > of it but I'm stumped. > > -Vance > From anders@REDACTED Wed Mar 30 00:12:10 2005 From: anders@REDACTED (Anders Ramsell) Date: Wed, 30 Mar 2005 00:12:10 +0200 Subject: MS Windows build use of versions in directory names/paths In-Reply-To: <4248817F.1050500@mail.wvnet.edu> References: <4248817F.1050500@mail.wvnet.edu> Message-ID: <4249D2BA.7010006@theheartofgold.org> Frank Seesink wrote: > I'm relatively new to Erlang, so please excuse me if this has been > covered. I've searched the FAQs and not found a reasoning, so thought I > would post here. > > What exactly is the reasoning behind using version numbers in the > directory names/paths in the Windows prebuilt binaries? That is, the > latest version of Erlang as I write this is Erlang/OTP R10B-4 (release > 23 Mar 2005). Performing a vanilla/standard install, the user has > Erland installed in > > x:\Program Files\erl5.4.5 > > and there is a subdirectory > > x:\Program Files\erl5.4.5\erts-5.4.5 > > [snip] > I can see how this can be annoying for some uses but I would still not like any major change to the current situation. We typically have to support several versions of our application and these in turn (often) use different versions of Erlang. This means that I need to have several versions of Erlang installed in parallell on my development environment. With the current system this is a breeze. I simply install the versions of Erlang I need. I can then work using any of the installed versions without risk of using a newer compiler than I want, or an older stdlib, or ... The reason this works so well is because the versions install in completely different structures. Not in spite of it. > This issue came up for me as my introduction to Erlang came from looking > into ejabberd, the Erlang-based Jabber/XMPP server > (http://ejabberd.jabber.ru). Installing/running ejabberd requires > Erlang, of course. And the ejabberd installer offers to create a > Windows service so that a user can run ejabberd on system startup. > > However, the installation of this NT service involves launching Erlang > and having it fire up ejabberd. This, of course, necessitates putting > the PATH to the current installed version of Erlang into the appropriate > place in the Windows Registry > (HKLM\SYSTEM\CurrentControlSet\Services\ejabberd...) where the NT > service is set. (Note this is not something many users will understand.) > > [snip] > I don't know how ejarrerd works and this is not an argument either for it or against it. Our application, which is run in a Windows environment, is built to work with a specific version of Erlang. Each version is verified for one Erlang version only. Running it using a different version of Erlang is NOT supported. To make this work our installer checks for the required version of Erlang. If that version is not found it will be installed. Once the correct version of Erlang is available that version is used by the installer to run the erlsrv application to install or update the Windows services. The services will then be registered with a full path to the required Erlang version. Selecting the version of Erlang is (in our case) up to us. Not our users. (They usually have no idea what Erlang is.) If they do install another application using a different version of Erlang that is not a problem. Our stuff still works just fine. /Anders From matthias@REDACTED Wed Mar 30 00:34:26 2005 From: matthias@REDACTED (Matthias Lang) Date: Wed, 30 Mar 2005 00:34:26 +0200 Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: References: <16967.48193.531773.651618@antilipe.corelatus.se> <16969.188.79545.705496@antilipe.corelatus.se> Message-ID: <16969.55282.442093.27746@antilipe.corelatus.se> Matt> > It's possible to see it the other way around. Cons cells are Matt> > universal. ^^^^^^^^^^ James> True. But the way I see it, the fundamental difference is that lists James> are designed... ^^^^^ All lists are made of cons cells. But not all cons cells are part of a list. Wasn't that the whole point? http://www.gigamonkeys.com/book/beyond-lists-other-uses-for-conses.html http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-16.html#%_sec_2.3.3 Matt From matthias@REDACTED Wed Mar 30 00:55:23 2005 From: matthias@REDACTED (Matthias Lang) Date: Wed, 30 Mar 2005 00:55:23 +0200 Subject: RUDP, shared libraries, etc. In-Reply-To: <20050329205330.27783@smtp.gmail.com> References: <16969.5323.707153.380369@antilipe.corelatus.se> <20050329205330.27783@smtp.gmail.com> Message-ID: <16969.56539.217254.247377@antilipe.corelatus.se> Joel Reymont writes: > >A relatively conservative way around that limitation is to run > >multiple Erlang VMs on the same machine. Erlang is intended for that > >sort of design. A little bonus would be that your application would > >then be ready to scale to multiple machines. > Is there an idiom to implement this scalability in Erlang? That is how do > I run multiple instances of the VM on the same machine and have external > C code see them as one single server? I was thinking "Erlang has nice support for writing distributed Erlang programs, i.e. systems which run on multiple Erlang VMs which may or may not be on the same machine. That may or may not be useful for your particular problem." The Erlang Book has a whole chapter about this. http://www.erlang.org/download/erlang-book-part1.pdf I have a feeling that you were after a much more concrete answer. I don't have one. Matt From bjorn@REDACTED Wed Mar 30 09:24:39 2005 From: bjorn@REDACTED (Bjorn Gustavsson) Date: 30 Mar 2005 09:24:39 +0200 Subject: run_erl/to_erl on FreeBSD In-Reply-To: <82b7d977b3afa950c10e7b109e194da1@mac.com> References: <20050329043958.GD836@blank.motivity.ca> <82b7d977b3afa950c10e7b109e194da1@mac.com> Message-ID: Yes, that is correct. I changed mknod() to mkfifo() because mknod() on Mac OS X requires the user to be root, even when creating a FIFO. mkfifo() is the recommended way to create a FIFO, and is supposed to work on all modern Unix platforms, so I found no reason to continue to use mknod() on platforms where it works. Our test case for run_erl/to_erl runs succesfully on FreeBSD. We use the following version of FreeBSD: FreeBSD 5.0-RELEASE #0: Thu Jan 16 22:16:53 GMT 2003 Which version of FreeBSD do you use? Does run_erl run using R10B-2 or R10B-3? /Bjorn Sean Hinde writes: > Hi Vance, > > The only thing that changed (in R10B-2 as far as I can see) is that > the call to mknod in run_erl.c was replaced with a call to mkfifo in > run_erl.c. This fixed things for OS X, but maybe not all BSDs are > equal in this regard. > > Regards, > Sean > > > > On 29 Mar 2005, at 05:39, Vance Shipley wrote: > > > > > Is anyone able to use R10B-4 to_erl with FreeBSD? > > > > Here (4.10-STABLE) I see run_erl fail to open the pipe to > > write on and to_erl gets EOF reading from it's pipe. I > > spent quite a bit of time today trying to get to the bottom > > of it but I'm stumped. > > > > -Vance > > > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From joelr1@REDACTED Wed Mar 30 21:24:30 2005 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 30 Mar 2005 20:24:30 +0100 Subject: OpenGL GUI widgets Message-ID: <20050330202430.24463@smtp.gmail.com> Howdy! Is there a library of OpenGL GUI interface elements for Erlang? I'm looking at Wings 3D right now but have not figured the answer yet. It's clear that it's built on top of SDL/OpenGL but I can't find documentation on the UI toolkit. Thanks, Joel -- http://wagerlabs.com/tech From keymon@REDACTED Wed Mar 30 22:12:09 2005 From: keymon@REDACTED (=?ISO-8859-1?Q?H=E9ctor_Rivas_G=E1ndara?=) Date: Wed, 30 Mar 2005 22:12:09 +0200 Subject: EPI: Erlang Plus Interface (http://epi.sf.net) Message-ID: <44da525e721985df3a4e4d43f2e5a1c8@wanadoo.es> The Erlang Plus Interface library (EPI) is a tool set of C++ classes to easily build applications in C++ what comunicates with Erlang. You can download it from http://epi.sourceforge.net. The intention of the library is to cover the holes that EI library offers: - An object oriented implementation - A simple API - Extensibility - Abstraction of comunication mechanism - Simplified management of incoming messages (mailboxes) EPI is realased under the LGPL license. For details please see the file "COPYING" distributed with EPI. EPI works in a similar manner than jinterface the library for java http://www.erlang.se/doc/doc-5.0.1/lib/jinterface-1.2/doc/. It have some differences: * All comunication is done throw MailBoxes, no matter if you use self or auto managed nodes. * Epi abstracts the comunication mechanism, and you can extend it with new mechanism (using ErlangTransport interface and ErlangTransportFactory) NOTE: EPI is a alfa version and is not complete and probably has a lot of bugs. Testers are wellcome! Example of use: #include "Config.hpp" #include #include #include #include "epi.hpp" // Use namespaces using namespace epi::type; using namespace epi::error; using namespace epi::node; int main () { const std::string LOCALNODE = "pepito@REDACTED"; const std::string REMOTENODE = "pepita@REDACTED"; const std::string COOKIE = "one_cookie"; try { // Create the node AutoNode node(LOCALNODE, COOKIE); // Get a mailbox. The node has the pointer owership!!! MailBox *mailbox = node.createMailBox(); // Create the tuple {self(), hello} ErlTermPtr<> tuple(new ErlTuple(mailbox->self(), new ErlAtom("hello"))); // Send the term to a server in the remote node mailbox->send(REMOTENODE, "reply_server", tuple.get()); // Receive the response ErlTermPtr<> received(mailbox->receive()); // Print it std::cout << "Received response: " << received->toString() << std::endl; } catch (EpiException &e) { std::cout << "Exception catched: " << e.getMessage() << std::endl; return 1; } return 0; } The correspondind erlang node: -module(reply_server). -export([start/0, loop/0]). start() -> Pid=spawn(reply_server, loop, []), register(reply_server, Pid), Pid. loop() -> receive {Pid, Msg} -> io:format("Received: ~w from ~w~n", [Msg, Pid]), Pid!Msg; X -> io:format("Received: ~w~n", [X]) end, loop(). Bird's eye view: * To manage connections automaticlyuse epi::node::AutoNode, use epi::node::LocalNode otherwise * You can explore the mailbox queue content using the interface QueueGuard from the GenericQueue.hpp file and the associated method in MailBox class. Some useful guards are provided: o PatternMatchingGuard: It searchs a message that matches a pattern (an ErlTerm with variables). o CommandQueueGuard: Uses the Command+Decorator patterns to attach an command to be executed when the guard matches. o MatchingCommandGuard: Like the CommandQueueGuard, but with pattern matching o ComposedGuard: Uses the composite pattern to compose guards * Is highly recomended the use of std::auto_ptr and epi::type::ErlTermPtr. -- Saudos Keymon From rpettit@REDACTED Wed Mar 30 23:23:04 2005 From: rpettit@REDACTED (Rick Pettit) Date: Wed, 30 Mar 2005 15:23:04 -0600 Subject: Erlang security and the Internet Message-ID: <20050330212304.GD11895@vailsys.com> Is anyone out there running OTP nodes directly on the net (i.e. not behind a firewall)? A quick scan of the archives turned up the research project "Safe Erlang". Perhaps I should start there? -Rick From mikael.karlsson@REDACTED Wed Mar 30 23:33:03 2005 From: mikael.karlsson@REDACTED (Mikael Karlsson) Date: Wed, 30 Mar 2005 23:33:03 +0200 Subject: compiling R10B-4 odbc on SuSE Linux 9.2 Message-ID: <200503302333.03732.mikael.karlsson@creado.com> The new configure makes it really easy to compile the odbc driver on Linux now. Still some minor issues (that could be included in the next release maybe :-) lib/Makefile - still checks for sparc-solaris to include odbc, fix: otp_src_R10B-4/lib> diff Makefile Makefile~ 68c68 < pman $(SSL_APP) toolbar tv observer odbc \ --- > pman $(SSL_APP) toolbar tv observer \ On SuSE the odbc root is /usr so this could be added to the configure(.in) files: otp_src_R10B-4/lib/odbc> diff configure.in configure.in~ 67c67 < for dir in /opt/local/pgm/odbc /usr/local/odbc /usr/odbc /usr --- > for dir in /opt/local/pgm/odbc /usr/local/odbc /usr/odbc Regards Mikael From rpettit@REDACTED Thu Mar 31 00:19:37 2005 From: rpettit@REDACTED (Rick Pettit) Date: Wed, 30 Mar 2005 16:19:37 -0600 Subject: EPI: Erlang Plus Interface (http://epi.sf.net) In-Reply-To: <44da525e721985df3a4e4d43f2e5a1c8@wanadoo.es> References: <44da525e721985df3a4e4d43f2e5a1c8@wanadoo.es> Message-ID: <20050330221937.GB20439@vailsys.com> On Wed, Mar 30, 2005 at 10:12:09PM +0200, H?ctor Rivas G?ndara wrote: > > The Erlang Plus Interface library (EPI) is a tool set of C++ classes to > easily build applications in C++ what comunicates with Erlang. > > You can download it from http://epi.sourceforge.net. > > The intention of the library is to cover the holes that EI library > offers: > > - An object oriented implementation > - A simple API > - Extensibility > - Abstraction of comunication mechanism > - Simplified management of incoming messages (mailboxes) > > > EPI is realased under the LGPL license. For details please see the file > "COPYING" distributed with EPI. > > EPI works in a similar manner than jinterface the library for java > http://www.erlang.se/doc/doc-5.0.1/lib/jinterface-1.2/doc/. > > It have some differences: > > * All comunication is done throw MailBoxes, no matter if you use > self or auto managed nodes. > * Epi abstracts the comunication mechanism, and you can extend it > with new mechanism (using ErlangTransport interface and > ErlangTransportFactory) > > NOTE: EPI is a alfa version and is not complete and probably has a lot > of bugs. Testers are wellcome! Very nice, and thank you. I hope to have time to play with it soon. -Rick From erlang@REDACTED Thu Mar 31 02:10:27 2005 From: erlang@REDACTED (Michael McDaniel) Date: Wed, 30 Mar 2005 16:10:27 -0800 Subject: Erlang security and the Internet In-Reply-To: <20050330212304.GD11895@vailsys.com> References: <20050330212304.GD11895@vailsys.com> Message-ID: <20050331001027.GW10107@fangora.autosys.us> On Wed, Mar 30, 2005 at 03:23:04PM -0600, Rick Pettit wrote: > Is anyone out there running OTP nodes directly on the net (i.e. not behind a > firewall)? > > A quick scan of the archives turned up the research project "Safe Erlang". > Perhaps I should start there? ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Here's a thread on similar subject, though it is about getting through firewalls w/SSL enabled Erlang nodes: http://www.erlang.org/ml-archive/erlang-questions/200412/msg00100.html ~Michael > > -Rick From david.nospam.hopwood@REDACTED Thu Mar 31 03:06:57 2005 From: david.nospam.hopwood@REDACTED (David Hopwood) Date: Thu, 31 Mar 2005 02:06:57 +0100 Subject: Improper lists [was: Dialyzer v1.3.0 released] In-Reply-To: <16969.55282.442093.27746@antilipe.corelatus.se> References: <16967.48193.531773.651618@antilipe.corelatus.se> <16969.188.79545.705496@antilipe.corelatus.se> <16969.55282.442093.27746@antilipe.corelatus.se> Message-ID: <424B4D31.9070500@blueyonder.co.uk> Matthias Lang wrote: > Matt> > It's possible to see it the other way around. Cons cells are > Matt> > universal. ^^^^^^^^^^ > > James> True. But the way I see it, the fundamental difference is that lists > James> are designed... ^^^^^ > > All lists are made of cons cells. > But not all cons cells are part of a list. > > Wasn't that the whole point? Depends on the language. In Haskell for example, and in other languages where cons cells are defined as the algebraic data type List t ::= Nil | Cons t (List t), lists are always proper. > http://www.gigamonkeys.com/book/beyond-lists-other-uses-for-conses.html This representation of trees uses only proper lists (where each list is the right spine of the corresponding subtree). -- David Hopwood From tobbe@REDACTED Thu Mar 31 04:35:24 2005 From: tobbe@REDACTED (Torbjorn Tornkvist) Date: Thu, 31 Mar 2005 04:35:24 +0200 Subject: Erlang security and the Internet In-Reply-To: <20050330212304.GD11895@vailsys.com> References: <20050330212304.GD11895@vailsys.com> Message-ID: <424B61EC.2030202@nortel.com> Rick Pettit wrote: >Is anyone out there running OTP nodes directly on the net (i.e. not behind a >firewall)? > >A quick scan of the archives turned up the research project "Safe Erlang". >Perhaps I should start there? > >-Rick > > It should be possible to run the Erlang distribution over SSH I think. See the ssh code in the Jungerl CVS repository. Cheers, Tobbe From raimo@REDACTED Thu Mar 31 08:30:15 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 31 Mar 2005 08:30:15 +0200 Subject: OpenGL GUI widgets References: <20050330202430.24463@smtp.gmail.com> Message-ID: The interface library that is used is called Esdl and can be found on Sourceforge: http://sourceforge.net/projects/esdl joelr1@REDACTED (Joel Reymont) writes: > Howdy! > > Is there a library of OpenGL GUI interface elements for Erlang? > > I'm looking at Wings 3D right now but have not figured the answer yet. > It's clear that it's built on top of SDL/OpenGL but I can't find > documentation on the UI toolkit. > > Thanks, Joel > > -- > http://wagerlabs.com/tech > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo@REDACTED Thu Mar 31 08:30:15 2005 From: raimo@REDACTED (Raimo Niskanen) Date: 31 Mar 2005 08:30:15 +0200 Subject: OpenGL GUI widgets References: <20050330202430.24463@smtp.gmail.com> Message-ID: The interface library that is used is called Esdl and can be found on Sourceforge: http://sourceforge.net/projects/esdl joelr1@REDACTED (Joel Reymont) writes: > Howdy! > > Is there a library of OpenGL GUI interface elements for Erlang? > > I'm looking at Wings 3D right now but have not figured the answer yet. > It's clear that it's built on top of SDL/OpenGL but I can't find > documentation on the UI toolkit. > > Thanks, Joel > > -- > http://wagerlabs.com/tech > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From joelr1@REDACTED Thu Mar 31 16:39:24 2005 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 31 Mar 2005 15:39:24 +0100 Subject: Bit twiddling Message-ID: <20050331153924.3956@smtp.gmail.com> Folks, What's the most effective way to do bit twiddling in Erlang? Should I convert the int into a list of bits? How would I efficiently do this? I'm looking to figure out the highest and lowest bit set in an integer as well as count the number of bits set. Thanks, Joel -- http://wagerlabs.com/tech From ingela@REDACTED Thu Mar 31 17:05:08 2005 From: ingela@REDACTED (Ingela Anderton) Date: Thu, 31 Mar 2005 17:05:08 +0200 Subject: compiling R10B-4 odbc on SuSE Linux 9.2 References: <200503302333.03732.mikael.karlsson@creado.com> Message-ID: <16972.4516.182655.18432@gargle.gargle.HOWL> Mikael Karlsson wrote: > The new configure makes it really easy to compile the > odbc driver on Linux now. That is almost the whole point in writing the new configure ;) Glad you like it :) > Still some minor issues (that could be included in the > next release maybe :-) > > lib/Makefile - still checks for sparc-solaris > to include odbc, fix: > otp_src_R10B-4/lib> diff Makefile Makefile~ > 68c68 > < pman $(SSL_APP) toolbar tv observer odbc \ > --- > > pman $(SSL_APP) toolbar tv observer \ I am sure I can convince the responsible person to make this change! > On SuSE the odbc root is /usr so this could be added > to the configure(.in) files: > otp_src_R10B-4/lib/odbc> diff configure.in configure.in~ > 67c67 > < for dir in /opt/local/pgm/odbc /usr/local/odbc /usr/odbc /usr > --- > > for dir in /opt/local/pgm/odbc /usr/local/odbc /usr/odbc This I have added! -- /Ingela - OTP team From thomasl_erlang@REDACTED Thu Mar 31 17:56:00 2005 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 31 Mar 2005 07:56:00 -0800 (PST) Subject: Bit twiddling In-Reply-To: 6667 Message-ID: <20050331155600.47010.qmail@web41904.mail.yahoo.com> --- Joel Reymont wrote: > Folks, > > What's the most effective way to do bit twiddling in > Erlang? > > Should I convert the int into a list of bits? How > would I efficiently do this? No, a list of bits is not the way to go. > I'm looking to figure out the highest and lowest bit > set in an integer as > well as count the number of bits set. I won't claim these to be ultimately fast, but they can serve as a starting point. All of the following work by breaking down the integer into a number of subfields, then using the subfield to index into a table, and finally summing the results. The table implementation is what differs. Note: these won't work for negative numbers. 1. The straightforward approach is to define a function as a lookup table, e.g. something like: -module(bits). -export([count_byte/1, ...]). %% simple 4-bit lookup table count_bits(2#0000) -> 0; count_bits(2#0001) -> 1; count_bits(2#0010) -> 1; ... count_bits(2#1111) -> 4. %% count bits per integer count(X) -> count(X, 0). count(0, N) -> N; count(X, N) -> count(X bsr 4, N + count_bits(X band 15)). %% if you have, say, a 4-byte int, then try unrolling %% the above loop into eight count_bits calls and see %% if that's faster 2. The lookup table could alternatively be implemented as an ets-table, which you have to populate beforehand. Ets is a bit messier to use, but if you are indexing on a large subword (a short, even?), it might scale better than using a function as a table like above. Here is the equivalent of the above loop. count_bits(0, N) -> N; count_bits(X, N) -> case ets:lookup(?bit_table, X band ?mask) of [{_X, NumBits}] -> count_bits(X bsr ?shift, N+NumBits); [] -> %% oops exit({bad_value, X band ?mask}); end. 3. Or a third kind of lookup table: use the humble tuple as a vector. In this case, create a tuple where position N holds the number of bits in N (or N-1): Create table: Table = list_to_tuple([0, 1, 1, ...]) Note that you can create very large tuples if you like, but do pass the Table around in the loop once you have created it. Repeatedly evaluating Table = ... will end in tears, performancewise. Index into it: ... Index = X band 15, NumBits = element(Index+1, Table), ... (Since tuple indexing is 1-based, we have to adjust Index) Highest and lowest bit can be coded as variations of the above. Best, Thomas __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From rpettit@REDACTED Thu Mar 31 18:23:48 2005 From: rpettit@REDACTED (Rick Pettit) Date: Thu, 31 Mar 2005 10:23:48 -0600 Subject: Erlang security and the Internet In-Reply-To: <424B61EC.2030202@nortel.com> References: <20050330212304.GD11895@vailsys.com> <424B61EC.2030202@nortel.com> Message-ID: <20050331162348.GA14657@vailsys.com> On Thu, Mar 31, 2005 at 04:35:24AM +0200, Torbjorn Tornkvist wrote: > Rick Pettit wrote: > > >Is anyone out there running OTP nodes directly on the net (i.e. not behind > >a > >firewall)? > > > >A quick scan of the archives turned up the research project "Safe Erlang". > >Perhaps I should start there? > > > >-Rick > > > > > It should be possible to run the Erlang distribution > over SSH I think. See the ssh code in the Jungerl > CVS repository. > > Cheers, Tobbe Thanks much for the help, -Rick From mscandar@REDACTED Thu Mar 31 19:26:52 2005 From: mscandar@REDACTED (Mark Scandariato) Date: Thu, 31 Mar 2005 12:26:52 -0500 Subject: Bit twiddling In-Reply-To: <20050331153924.3956@smtp.gmail.com> References: <20050331153924.3956@smtp.gmail.com> Message-ID: <424C32DC.6070903@cisco.com> If you have a binary you could do something hideous like: %%%%%%%%%%%%%%%% -module(foo). -export([twiddle/1]). twiddle(Bin) when is_binary(Bin) -> Bits = 8 * size(Bin), twiddle(Bin, -1, Bits+1, 0, 0, Bits-1). twiddle(_, _, _, 0, _, -1) -> none; twiddle(_, H, L, C, _, -1) -> {H-1,L-1,C}; twiddle(Bin, H, L, C, Pre, Post) -> <<_:Pre, X:1, _:Post>> = Bin, {H1,L1} = hilo(X, H, L, Post+1), twiddle(Bin, H1, L1, C+X, Pre+1, Post-1). hilo(0, H, L, _) -> {H, L}; hilo(1, H, L, C) -> {max(H,C), min(L,C)}. min(A, B) when A < B -> A; min(_, B) -> B. max(A, B) when A > B -> A; max(_, B) -> B. %%%%%%%%%%%%%%%% 1> foo:twiddle(<<9876:16>>). {13,2,6} 2> Mark. Joel Reymont wrote: > Folks, > > What's the most effective way to do bit twiddling in Erlang? > > Should I convert the int into a list of bits? How would I efficiently do this? > > I'm looking to figure out the highest and lowest bit set in an integer as > well as count the number of bits set. > > Thanks, Joel > From luke@REDACTED Thu Mar 31 20:04:32 2005 From: luke@REDACTED (Luke Gorrie) Date: 31 Mar 2005 20:04:32 +0200 Subject: Bit twiddling References: <20050331153924.3956@smtp.gmail.com> Message-ID: "Joel Reymont" writes: > I'm looking to figure out the highest and lowest bit set in an integer as > well as count the number of bits set. If it needs to be fast then be sure to see what mathematician assembler hackers have to say! hackmem http://www.inwap.com/pdp10/hbaker/hakmem/hakmem.html Hacker's Delight http://www.hackersdelight.org/ Count number of On bits in an integer http://www-db.stanford.edu/~manku/bitcount/bitcount.html Highest bit set is floor(log2(N)) if you figure out the floating point, though Erlang doesn't have a log2 BIF. Tony Rogvall has a very interesting "how many bits is this integer" function that makes the runtime system do the real work in jungerl/lib/ssh/src/ssh_bits.erl: isize(N) when N > 0 -> case term_to_binary(N) of <> -> isize_byte(X); <> -> isize_bytes([X3,X2,X1,X0]); <> -> K = S - 1, <<_:K/binary, Top>> = Ds, isize_byte(Top)+K*8; <> -> K = S - 1, <<_:K/binary, Top>> = Ds, isize_byte(Top)+K*8 end; isize(0) -> 0. i.e. encode the number in the binary Erlang external term format and then pull that apart to see the length header :-) From mscandar@REDACTED Thu Mar 31 23:19:39 2005 From: mscandar@REDACTED (Mark Scandariato) Date: Thu, 31 Mar 2005 16:19:39 -0500 Subject: Bit twiddling In-Reply-To: References: <20050331153924.3956@smtp.gmail.com> Message-ID: <424C696B.2090406@cisco.com> So, based on http://www-db.stanford.edu/~manku/bitcount/bitcount.html, something like these might do (for 32bit or smaller ints): %% number of bits set % bits0 works on arbitrary integers. bits0(N) when N >= 0-> bits0(N, 0). bits0(0, C) -> C; bits0(N, C) -> bits0((N band (N-1)), C+1). bits1(0) -> 0; bits1(N) when N > 0, N < 16#100000000 -> bits1(N, [1, 16#55555555, 2, 16#33333333, 4, 16#0F0F0F0F, 8, 16#00FF00FF, 16, 16#0000FFFF]). bits1(N, []) -> N; bits1(N, [S, B|Magic]) -> bits1(((N bsr S) band B) + (N band B), Magic). %% log2, aka, position of the high bit log2(N) when N > 0, N < 16#100000000 -> log2(N, 0, [16, 16#FFFF0000, 8, 16#FF00, 4, 16#F0, 2, 16#C, 1, 16#2]). log2(_, C, []) -> C; log2(N, C, [S,B|Magic]) -> if (N band B) == 0 -> log2(N, C, Magic); true -> log2((N bsr S), (C bor S), Magic) end. %% trailing zero bits, aka position of the lowest set bit. tzb0(N) when N > 0, N < 16#100000000 -> tzb0(N, 32, [16, 16#0000FFFF, 8, 16#00FF00FF, 4, 16#0F0F0F0F, 2, 16#33333333, 1, 16#55555555]). tzb0(_, Z, []) -> Z-1; tzb0(N, Z, [S, B|Magic]) -> if (N band B) == 0 -> tzb(N, Z, Magic); true -> tzb(0(N bsl S), (Z - S), Magic) end. tzb1(N) when N > 0, N < 16#100000000 -> Mod = {32,0,1,26,2,23,27,0,3,16,24,30, 28,11,0,13,4,7,17,0,25,22,31,15, 29,10,12,6,0,21,14,9,5,20,8,19,18}, P = (-N band N) rem 37, element(P+1, Mod). Although my naive "walk the binary, bit-by-bit" will provide, in O(N), all three results on arbitrary N-bit sequences. Mark. Luke Gorrie wrote: > "Joel Reymont" writes: > > >>I'm looking to figure out the highest and lowest bit set in an integer as >>well as count the number of bits set. > > > If it needs to be fast then be sure to see what mathematician > assembler hackers have to say! > > hackmem > http://www.inwap.com/pdp10/hbaker/hakmem/hakmem.html > > Hacker's Delight > http://www.hackersdelight.org/ > > Count number of On bits in an integer > http://www-db.stanford.edu/~manku/bitcount/bitcount.html > > Highest bit set is floor(log2(N)) if you figure out the floating > point, though Erlang doesn't have a log2 BIF. Tony Rogvall has a very > interesting "how many bits is this integer" function that makes the > runtime system do the real work in jungerl/lib/ssh/src/ssh_bits.erl: > > isize(N) when N > 0 -> > case term_to_binary(N) of > <> -> > isize_byte(X); > <> -> > isize_bytes([X3,X2,X1,X0]); > < Ds:S/binary>> -> > K = S - 1, > <<_:K/binary, Top>> = Ds, > isize_byte(Top)+K*8; > < Ds:S/binary>> -> > K = S - 1, > <<_:K/binary, Top>> = Ds, > isize_byte(Top)+K*8 > end; > isize(0) -> 0. > > i.e. encode the number in the binary Erlang external term format and > then pull that apart to see the length header :-) > From mscandar@REDACTED Thu Mar 31 23:26:13 2005 From: mscandar@REDACTED (Mark Scandariato) Date: Thu, 31 Mar 2005 16:26:13 -0500 Subject: Bit twiddling In-Reply-To: <424C696B.2090406@cisco.com> References: <20050331153924.3956@smtp.gmail.com> <424C696B.2090406@cisco.com> Message-ID: <424C6AF5.8040804@cisco.com> Mark Scandariato wrote: > So, based on http://www-db.stanford.edu/~manku/bitcount/bitcount.html, > something like these might do (for 32bit or smaller ints): > > %% number of bits set > % bits0 works on arbitrary integers. > bits0(N) when N >= 0-> bits0(N, 0). > bits0(0, C) -> C; > bits0(N, C) -> > bits0((N band (N-1)), C+1). > > bits1(0) -> 0; > bits1(N) when N > 0, N < 16#100000000 -> > bits1(N, [1, 16#55555555, > 2, 16#33333333, > 4, 16#0F0F0F0F, > 8, 16#00FF00FF, > 16, 16#0000FFFF]). > > bits1(N, []) -> N; > bits1(N, [S, B|Magic]) -> > bits1(((N bsr S) band B) + (N band B), Magic). > > > %% log2, aka, position of the high bit > log2(N) when N > 0, N < 16#100000000 -> > log2(N, 0, [16, 16#FFFF0000, 8, 16#FF00, 4, 16#F0, 2, 16#C, 1, 16#2]). > > log2(_, C, []) -> C; > log2(N, C, [S,B|Magic]) -> > if (N band B) == 0 -> log2(N, C, Magic); > true -> log2((N bsr S), (C bor S), Magic) > end. > > %% trailing zero bits, aka position of the lowest set bit. > tzb0(N) when N > 0, N < 16#100000000 -> > tzb0(N, 32, [16, 16#0000FFFF, > 8, 16#00FF00FF, > 4, 16#0F0F0F0F, > 2, 16#33333333, > 1, 16#55555555]). > > tzb0(_, Z, []) -> Z-1; > tzb0(N, Z, [S, B|Magic]) -> > if (N band B) == 0 -> tzb(N, Z, Magic); > true -> tzb(0(N bsl S), (Z - S), Magic) > end. Oops - that should have been: tzb0(N, Z, [S, B|Magic]) -> if (N band B) == 0 -> tzb0(N, Z, Magic); true -> tzb0((N bsl S), (Z - S), Magic) end. > > > tzb1(N) when N > 0, N < 16#100000000 -> > Mod = {32,0,1,26,2,23,27,0,3,16,24,30, > 28,11,0,13,4,7,17,0,25,22,31,15, > 29,10,12,6,0,21,14,9,5,20,8,19,18}, > P = (-N band N) rem 37, > element(P+1, Mod). > > > Although my naive "walk the binary, bit-by-bit" will provide, in O(N), > all three results on arbitrary N-bit sequences. > > Mark. > > Luke Gorrie wrote: > >> "Joel Reymont" writes: >> >> >>> I'm looking to figure out the highest and lowest bit set in an >>> integer as >>> well as count the number of bits set. >> >> >> >> If it needs to be fast then be sure to see what mathematician >> assembler hackers have to say! >> >> hackmem >> http://www.inwap.com/pdp10/hbaker/hakmem/hakmem.html >> >> Hacker's Delight >> http://www.hackersdelight.org/ >> >> Count number of On bits in an integer >> http://www-db.stanford.edu/~manku/bitcount/bitcount.html >> >> Highest bit set is floor(log2(N)) if you figure out the floating >> point, though Erlang doesn't have a log2 BIF. Tony Rogvall has a very >> interesting "how many bits is this integer" function that makes the >> runtime system do the real work in jungerl/lib/ssh/src/ssh_bits.erl: >> >> isize(N) when N > 0 -> >> case term_to_binary(N) of >> <> -> >> isize_byte(X); >> <> -> >> isize_bytes([X3,X2,X1,X0]); >> <> Ds:S/binary>> -> >> K = S - 1, >> <<_:K/binary, Top>> = Ds, >> isize_byte(Top)+K*8; >> <> Ds:S/binary>> -> >> K = S - 1, >> <<_:K/binary, Top>> = Ds, >> isize_byte(Top)+K*8 >> end; >> isize(0) -> 0. >> >> i.e. encode the number in the binary Erlang external term format and >> then pull that apart to see the length header :-) >> > >