From seancribbs@REDACTED Sun Nov 1 02:51:01 2009 From: seancribbs@REDACTED (Sean Cribbs) Date: Sat, 31 Oct 2009 21:51:01 -0400 Subject: [ANN] Neotoma 1.2 Message-ID: <4AECE985.4060301@gmail.com> I'm pleased to announce another update to Neotoma. The major features and changes in this release: * Generated parsers no longer require the neotoma libraries at compile-time or runtime, and are thus truly standalone. * All parse_transforms were removed and replaced with code-generation. * All nonterminals are escaped with single-quotes so you can use Erlang reserved words if desired. * All parser combinator functions are prefixed with p_ so as to reduce name-collision with your nonterminals. Thanks to Tony Arcieri and Kevin Smith for encouraging me to complete these features, and Caio Ariede for his code contribution. What is Neotoma? Neotoma is a packrat parser-generator for Erlang for Parsing Expression Grammars (PEGs). It consists of a parsing-combinator library with memoization routines, a parser for PEGs, and a utility to generate parsers from PEGs. It is inspired by treetop, a Ruby library with similar aims, and parsec, the parser-combinator library for Haskell. Browse & Fork: http://github.com/seancribbs/neotoma Download: http://github.com/seancribbs/neotoma/downloads Source: git clone git://github.com/seancribbs/neotoma.git Discussion: http://groups.google.com/group/neotoma-erl Cheers, Sean Cribbs From tedhenry10@REDACTED Sun Nov 1 06:31:27 2009 From: tedhenry10@REDACTED (Ted Henry) Date: Sat, 31 Oct 2009 22:31:27 -0700 Subject: erlang for programming a text editor Message-ID: With 100 core chips probably coming to desktop computers in the near future, I wonder about writing a multiprocess vi-like or emacs-like text editor (probably more emacs-like in implementation.) Sometimes single threaded editors really bog down when searching multiple files, syntax highlighting multiple files. I'm not that knowledgeable in the Erlang world. It seems like Erlang's fault tolerance and ability to upgrade code without restarting would not be big advantages for a text editor. Also a text editor isn't usually distributed across multiple physical machines. I think the main advantage would be Erlang's processes for concurrency. Would Erlang offer a good advantage over a programming language with software transactional memory (e.g. Haskell, Clojure)? Would *you* use Erlang to program a text editor or desktop application? Very interested to read your thoughts. Ted From kagato@REDACTED Sun Nov 1 06:54:14 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Sun, 1 Nov 2009 00:54:14 -0500 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: References: Message-ID: <27C8C8F9-D002-4949-B11A-9A0FA3C64203@souja.net> Ted, Well, if I were doing this, I probably wouldn't do it in Erlang. Text handling in Erlang is byzantine at best, so an editor very well could be an exercise in pain. I wouldn't bet that Erlang's good distribution / parallelization would offset the rest of the trouble. I'm unsure how production-ready it is, but Reia might make certain bits easier, just due to its string handling alone. JRuby or Clojure might be a better fit just because Java has MUCH more mature Unicode handling and syntax highlighting. I don't know of anything for Erlang that even tries to do syntax highlighting, although you could probably get a good start on the parsers with Neotoma. In any case, I wish you the best of luck. It's good to see people trying to tackle useful problems with the right tools. On Nov 1, 2009, at 12:31 AM, Ted Henry wrote: > With 100 core chips probably coming to desktop computers in the near > future, > I wonder about writing a multiprocess vi-like or emacs-like text > editor > (probably more emacs-like in implementation.) Sometimes single > threaded > editors really bog down when searching multiple files, syntax > highlighting > multiple files. I'm not that knowledgeable in the Erlang world. It > seems > like Erlang's fault tolerance and ability to upgrade code without > restarting > would not be big advantages for a text editor. Also a text editor > isn't > usually distributed across multiple physical machines. I think the > main > advantage would be Erlang's processes for concurrency. Would Erlang > offer a > good advantage over a programming language with software transactional > memory (e.g. Haskell, Clojure)? Would *you* use Erlang to program a > text > editor or desktop application? > > Very interested to read your thoughts. > > Ted -- Jayson Vantuyl kagato@REDACTED From mats.westin@REDACTED Sun Nov 1 11:11:38 2009 From: mats.westin@REDACTED (Mats) Date: Sun, 1 Nov 2009 11:11:38 +0100 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: References: Message-ID: I think it's a good idea. I want to upgrade my editor while coding. I also want my editor to be distributed so I don't lose state when I restart my computer. A wonderful thought. Updating the source could be done 2-3 times a day. Update cycles never seen before with text editors. Bye bye Emacs. ;) Luke Gorrie made an emacs clone[1]. It could be a good start. The only part missing in Ermacs is LFE (Lisp Flavored Erlang). Think about the accumulated runtime of all users, if Ermacs was running 24/7 of the latest Erlang release. It could be a nice way of introducing "Test what you fly, fly what you test" to the QA routine (both for Ermacs and Erlang). [1] http://fresh.homeunix.net/~luke/ermacs/ On Sun, Nov 1, 2009 at 6:31 AM, Ted Henry wrote: > With 100 core chips probably coming to desktop computers in the near > future, > I wonder about writing a multiprocess vi-like or emacs-like text editor > (probably more emacs-like in implementation.) Sometimes single threaded > editors really bog down when searching multiple files, syntax highlighting > multiple files. I'm not that knowledgeable in the Erlang world. It seems > like Erlang's fault tolerance and ability to upgrade code without > restarting > would not be big advantages for a text editor. Also a text editor isn't > usually distributed across multiple physical machines. I think the main > advantage would be Erlang's processes for concurrency. Would Erlang offer a > good advantage over a programming language with software transactional > memory (e.g. Haskell, Clojure)? Would *you* use Erlang to program a text > editor or desktop application? > > Very interested to read your thoughts. > > Ted > From attila.r.nohl@REDACTED Sun Nov 1 13:13:39 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Sun, 1 Nov 2009 13:13:39 +0100 Subject: [erlang-questions] Process Dictionary vs Proplist in Web Frameworks In-Reply-To: References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Message-ID: <401d3ba30911010413v162e629egd979abecc9f4a59@mail.gmail.com> 2009/10/29, Geoff Cant : >[...] Erlang programmers > usually only have to think about the function body and arguments to work > out what its going to do. Sprinkling 'get' and 'put' through the code > means that an erlang programmer trying to understand your code now has > to read all the code to figure out why something is happening. The order > in which functions are called becomes important. The behaviour of > functions in other modules becomes important because now there's a > back-channel to propagate bugs, er, state between parts of the code. This is not related to the process dictionary. If the programmer implements e.g. a handle_call in gen_server 'A' (or anything that's called from that handle_call), he has to make sure that he doesn't call gen_server 'B' if there's a chance that 'B' called 'A' first - otherwise there would be a deadlock. In this case also a lot of other code gets important and it's fairly common that a handle_call (or a function called from handle_call) gets implemented... Like it or not, it's important that in what circumstances a function is called - this is a limitation in Erlang's "functional languageness", but actually this is necessary to do anything useful with the language. The process dictionary could be great for environment-like variables, which are only set once, but used in very many places and it's very inconvenient to pass around one more parameter. They don't show up in function traces - but they do show up in the output of erlang:process_info(), where e.g. the gen_server state is not shown, even though it would be dead useful. From erlang@REDACTED Sun Nov 1 15:54:45 2009 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 1 Nov 2009 15:54:45 +0100 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: <27C8C8F9-D002-4949-B11A-9A0FA3C64203@souja.net> References: <27C8C8F9-D002-4949-B11A-9A0FA3C64203@souja.net> Message-ID: <9b08084c0911010654t7456e032q368d4e321b9ef703@mail.gmail.com> On Sun, Nov 1, 2009 at 6:54 AM, Jayson Vantuyl wrote: > Ted, > > Well, if I were doing this, I probably wouldn't do it in Erlang. ?Text > handling in Erlang is byzantine at best, so an editor very well could be an > exercise in pain. I beg to differ. In my opinion Erlang is brilliant at handling text - text is stored in lists and list processing is blindingly fast and there are large numbers of library functions that work on lists. A very unpainful experience. I have written a full text edit in both Erlang and C - the C *was* painful Imagine how to implement "undo" in Erlang. Since data is immutable, you just revert to some old state by accessing the data at some historical point in the edit - trail all old states in a history stack and a difficult operation become trivial to program (and efficient because of the sharing) by popping the history stack. Easy in Erlang - difficult in a language with mutable state. I have written *many* text processing applications in Erlang and only once had problems. My usual experience is that I can write mind-boggelingly inefficient code which never the less executes blindingly fast. I always use the get-it-right then optimize philosophy. But the number of times I have to optimize is very very few. For text applications virtually *never*. Text processing is trivial (unless you're talking of Gigabytes, but then the problem is one of algorithms) Only one text application ever caused me a problem - writing a full text indexer for Gigabytes of data - the problem was not with Erlang but with my knowledge of algorithms - after I read "managing gigabyes" and implemented gamma encoding in Erlang the problem went away. (the ease of which I could change representations for lists, to (in this case) gamma encoded integers, was a big big win) Text process applications involve mainly list processing. If lists don't cut it then custom abstractions can be easily made. Erlang is *brilliant* at text applications. Lists are blindingly fast. The bit syntax is fantastic if you need to do things like huffman encoding or the like for tricky text representations. Complex data structures are trivial to create. A text editor hardy needs concurrency it should be way fast enough on one core - there is no "natural concurrency in the problem" - emacs used to run blindingly fast on a 40Mz PC with 128KB of memory - it's *not* a difficult problem making something like emacs run fast enough (unless you want to use emacs to search through GBytes of data, in which case you're probably should not be using emacs anyway) I wrote a simple emacs editor years ago - it's in the widgets subdirectory of http://www.sics.se/~joe/ex11/download/release-2.5.tgz The only tricky part was not the emacs logic, but the screen display and catching the keystrokes and mouse events. /Joe > > I wouldn't bet that Erlang's good distribution / parallelization would > offset the rest of the trouble. ?I'm unsure how production-ready it is, but > Reia might make certain bits easier, just due to its string handling alone. > > JRuby or Clojure might be a better fit just because Java has MUCH more > mature Unicode handling and syntax highlighting. ?I don't know of anything > for Erlang that even tries to do syntax highlighting, although you could > probably get a good start on the parsers with Neotoma. > > In any case, I wish you the best of luck. ?It's good to see people trying to > tackle useful problems with the right tools. > > On Nov 1, 2009, at 12:31 AM, Ted Henry wrote: > >> With 100 core chips probably coming to desktop computers in the near >> future, >> I wonder about writing a multiprocess vi-like or emacs-like text editor >> (probably more emacs-like in implementation.) Sometimes single threaded >> editors really bog down when searching multiple files, syntax highlighting >> multiple files. I'm not that knowledgeable in the Erlang world. It seems >> like Erlang's fault tolerance and ability to upgrade code without >> restarting >> would not be big advantages for a text editor. Also a text editor isn't >> usually distributed across multiple physical machines. I think the main >> advantage would be Erlang's processes for concurrency. Would Erlang offer >> a >> good advantage over a programming language with software transactional >> memory (e.g. Haskell, Clojure)? Would *you* use Erlang to program a text >> editor or desktop application? >> >> Very interested to read your thoughts. >> >> Ted > > > -- > Jayson Vantuyl > kagato@REDACTED > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From toby@REDACTED Sun Nov 1 17:08:16 2009 From: toby@REDACTED (Toby Thain) Date: Sun, 1 Nov 2009 11:08:16 -0500 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: References: Message-ID: <6F15D5EA-DD41-4B89-9C16-7FA44936B095@telegraphics.com.au> On 1-Nov-09, at 1:31 AM, Ted Henry wrote: > With 100 core chips probably coming to desktop computers in the > near future, > I wonder about writing a multiprocess vi-like or emacs-like text > editor > (probably more emacs-like in implementation.) Sometimes single > threaded > editors really bog down when searching multiple files, This would seem to be I/O bound. > syntax highlighting > multiple files. If you can't get syntax highlighting working fast enough on today's single core, your problems aren't going to be solved by language choice. --Toby > I'm not that knowledgeable in the Erlang world. It seems > like Erlang's fault tolerance and ability to upgrade code without > restarting > would not be big advantages for a text editor. Also a text editor > isn't > usually distributed across multiple physical machines. I think the > main > advantage would be Erlang's processes for concurrency. Would Erlang > offer a > good advantage over a programming language with software transactional > memory (e.g. Haskell, Clojure)? Would *you* use Erlang to program a > text > editor or desktop application? > > Very interested to read your thoughts. > > Ted From rapsey@REDACTED Sun Nov 1 17:50:11 2009 From: rapsey@REDACTED (Rapsey) Date: Sun, 1 Nov 2009 17:50:11 +0100 Subject: building a debug version of erlang Message-ID: <97619b170911010850s3071ed3j3552b8ea336061c7@mail.gmail.com> How do you compile a debug version of erlang? My servers are crashing and Valgrind is not telling me anything useful. Hopefully with debug information it will. Sergej From tedhenry10@REDACTED Sun Nov 1 22:47:13 2009 From: tedhenry10@REDACTED (Ted Henry) Date: Sun, 1 Nov 2009 13:47:13 -0800 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: <9b08084c0911010654t7456e032q368d4e321b9ef703@mail.gmail.com> References: <27C8C8F9-D002-4949-B11A-9A0FA3C64203@souja.net> <9b08084c0911010654t7456e032q368d4e321b9ef703@mail.gmail.com> Message-ID: On Sun, Nov 1, 2009 at 6:54 AM, Joe Armstrong wrote: > Text process applications involve mainly list processing. > If lists don't cut it then custom abstractions can be easily made. In the context of a text editor, by "custom abstractions can be easily made", do you mean something written in C (e.g. gap buffer) and linked into the Erlang program with an FFI? Ted From yogishb@REDACTED Sun Nov 1 22:07:37 2009 From: yogishb@REDACTED (Yogish Baliga) Date: Sun, 1 Nov 2009 13:07:37 -0800 (PST) Subject: Corrupt database in mnesia Message-ID: <244668.41236.qm@web112610.mail.gq1.yahoo.com> Hello all, On our production box, I have seen mnesia getting corrupt once in a while. LATEST.LOG, PREVIOUS.LOG becomes 150G or so and 1st fragment of .DCL file reaches 6G or so. Restarting mnesia again will take forever to recover. But after deleting LATEST.LOG, PREVIOUS.LOG file and restarting mnesia will recover pretty fast and the data is intact in the database. 2 questions regarding this issue 1. What is causing the corruption of .LOG and .DCL files? 2. Why is that removing LATEST.LOG, PREVIOUS.LOG and restarting mnesia recovers pretty fast and data is not lost? 3. How to prevent this from happening again? Thanx, -- baliga "Point of view is worth 80 IQ points" --Alan Kay http://dudefrommangalore.blogspot.com/ From ok@REDACTED Sun Nov 1 23:27:38 2009 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 2 Nov 2009 11:27:38 +1300 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: References: Message-ID: <6014999B-09C3-44F6-B022-188B24602160@cs.otago.ac.nz> On Nov 1, 2009, at 6:31 PM, Ted Henry wrote: > With 100 core chips probably coming to desktop computers in the near > future, > I wonder about writing a multiprocess vi-like or emacs-like text > editor > (probably more emacs-like in implementation.) David Warren used to use a home-brew text editor he'd written in Prolog, so why not? > > Would Erlang offer a > good advantage over a programming language with software transactional > memory (e.g. Haskell, Clojure)? Would *you* use Erlang to program a > text > editor or desktop application? What exactly do you want transactional memory _for_? Have you read Joe Armstrong's blog entry about STM in Erlang? One issue in a text editor is undo. I have my own small but powerful text editor written in C that I still use. The only thing I've ever really missed in it is a good undo. Erlang should at least remove the temptation to use data structures that make it hard to provide undo. (AVL DAGs in Erlang, anyone?) My editor could keep up with me on a PDP-11/60. So it's not things that require human involvement that would benefit from concurrency. Rendering with fancy fonts and bizarre hard-to-read colours is more demanding, but then, who needs to do _that_? So the most interesting questions are not "what language" or even "what features", but - what is it that we want extra computing power to DO when we are "editing", and - what concurrency pattern(s) is(are) useful to express that? From ok@REDACTED Mon Nov 2 00:13:28 2009 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 2 Nov 2009 12:13:28 +1300 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: <9b08084c0911010654t7456e032q368d4e321b9ef703@mail.gmail.com> References: <27C8C8F9-D002-4949-B11A-9A0FA3C64203@souja.net> <9b08084c0911010654t7456e032q368d4e321b9ef703@mail.gmail.com> Message-ID: On Nov 2, 2009, at 3:54 AM, Joe Armstrong wrote: > On Sun, Nov 1, 2009 at 6:54 AM, Jayson Vantuyl > wrote: >> Ted, >> >> Well, if I were doing this, I probably wouldn't do it in Erlang. >> Text >> handling in Erlang is byzantine at best, so an editor very well >> could be an >> exercise in pain. > > I beg to differ. > > In my opinion Erlang is brilliant at handling text - text is stored in > lists and list processing is blindingly fast and there are large > numbers of library functions > that work on lists. A very unpainful experience. For what it's worth, ermacs has a 'cords.erl' module written by Luke Gorrie that stores a document as a tree of binaries. The thing very few people realise is just how very LITTLE help strings give you for text processing. For example, back in 1979 a friend of mine wrote a batch text editor for IBM mainframes. (Think of it as an MVS version of sed(1), but a *lot* dumber.) It took him 150 pages of PL/I, and he had to fight PL/I strings every step of the way. When I started at Edinburgh and had to learn C, I wrote a version of his program in C, *with* undo, and it took me something between 10 and 15 pages of C (I no longer recall precisely). Why was C so much better? Because it DIDN'T have strings. It had completely general purpose data types (like arrays), and while it didn't actually help me much, it didn't get in my way either. It gave me the _building blocks_ I needed. Edinburgh had a succession of text editors written in Pop-2. There was the "77 editor" written by Boyer and Moore (of Boyer/Moore theorem prover fame). The architecture of that was later written up as a Xerox report that I have somewhere. Then there was the Pop Editor, which became the Display-Oriented Pop Editor (DOPE), which was rather Emacs-like. (If by any chance Dave Bowen should read this, hi!) What data structure did that use? A list of records, each with a block of characters and some counters, plus an overflow area for type-in. Not strings, as such. > > A text editor hardy needs concurrency it should be way fast enough > on one core - > there is no "natural concurrency in the problem" - emacs used to run > blindingly fast > on a 40Mz PC with 128KB of memory - it's *not* a difficult problem > making > something like emacs run fast enough (unless you want to use emacs > to search > through GBytes of data, in which case you're probably should not be > using emacs anyway) Grepping the R12B-5 sources on a 500 MHz CPU, 84 MHz memory machine took 20 seconds. Grepping a 500MHz file took 24 seconds. On a 2.2 GHz CPU, 667 Mhz memory machine, the same grep took 6 seconds. The same search would take maybe a second using Informatio Retrieval techniques. The limiting factor on the fast machine seems to be how fast the data can be pulled off the disc, and more cores would not help. One of the programs hanging around on my disc is SubEthaEdit, which is a collaborative editing program. Every so often I accidentally invoke it and am reminded of its existence. What about collaborative editing? What about fusing version control and collaborative editing? For code, isn't testing going to get a bigger payoff from multicore? From rvirding@REDACTED Mon Nov 2 02:38:55 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 2 Nov 2009 02:38:55 +0100 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: <9b08084c0911010654t7456e032q368d4e321b9ef703@mail.gmail.com> References: <27C8C8F9-D002-4949-B11A-9A0FA3C64203@souja.net> <9b08084c0911010654t7456e032q368d4e321b9ef703@mail.gmail.com> Message-ID: <3dbc6d1c0911011738i17100e4do3cb3b485caaf48fd@mail.gmail.com> 2009/11/1 Joe Armstrong > On Sun, Nov 1, 2009 at 6:54 AM, Jayson Vantuyl wrote: > > Ted, > > > > Well, if I were doing this, I probably wouldn't do it in Erlang. Text > > handling in Erlang is byzantine at best, so an editor very well could be > an > > exercise in pain. > > I beg to differ. > > In my opinion Erlang is brilliant at handling text - text is stored in > lists and list processing is blindingly fast and there are large > numbers of library functions > that work on lists. A very unpainful experience. > Joe is absolutely right here. The problem is not the lack of traditional string handling, the problem is not realising how powerful and easy it is to use lists instead. It is necessary to "think outside the box"* of strings. Robert * That is a terrible expression but I couldn't think of a better one just now. From leap@REDACTED Mon Nov 2 03:57:34 2009 From: leap@REDACTED (Michael Turner) Date: Mon, 02 Nov 2009 02:57:34 +0000 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: <3dbc6d1c0911011738i17100e4do3cb3b485caaf48fd@mail.gmail.com> Message-ID: But aren't text editors supposed to be written in TECO? So, obviously, the first order of business is a TECO interpreter implemented in Erlang, taking all possible advantage of its ease of hotswapping modules, and of seamlessly scaling to many processes and many processors. It could be a challenge. I believe that nobody has looked at this particular problem before. I feel as if I am standing silent, with a wild surmise, upon a peak in Darien. -michael turenr From ajuttner.list@REDACTED Mon Nov 2 08:20:12 2009 From: ajuttner.list@REDACTED (Alpar Juttner) Date: Mon, 02 Nov 2009 08:20:12 +0100 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: <9b08084c0911010654t7456e032q368d4e321b9ef703@mail.gmail.com> References: <27C8C8F9-D002-4949-B11A-9A0FA3C64203@souja.net> <9b08084c0911010654t7456e032q368d4e321b9ef703@mail.gmail.com> Message-ID: <1257146412.4639.34.camel@piko.site> On Sun, 2009-11-01 at 15:54 +0100, Joe Armstrong wrote: > A text editor hardy needs concurrency it should be way fast enough on one core - > there is no "natural concurrency in the problem" - emacs used to run > blindingly fast > on a 40Mz PC with 128KB of memory - it's *not* a difficult problem making > something like emacs run fast enough (unless you want to use emacs to search > through GBytes of data, in which case you're probably should not be > using emacs anyway) An editor like emacs does not need multicore, but it _does_ need concurrency, as it could provide much better responsiveness. When emacs blocks for a couple of seconds, it is already annoying. When it blocks for tens of seconds, it can be a considerable pain. * Several years ago, I had gave up using a Emacs-RMAIL, because fetching new mail took a long time on our environment and this process blocks the whole emacs. * Emacs can open remote files through ssh/ftp. A wonderful feature but opening and (auto)saving may take quite a while through the net and these processes also block whole emacs. * Emacs-w3 is also practically unusable due to its blocking behavior. In some important cases the non-blocking processing are solved in emacs (such as in case of the compile command or the gdb interface) but with complex design and sometimes with dirty hacking. An inherently concurrent design would immediately eliminate all of these issues. Regards, Alpar > > I wrote a simple emacs editor years ago - it's in the widgets subdirectory > of http://www.sics.se/~joe/ex11/download/release-2.5.tgz > > The only tricky part was not the emacs logic, but the screen display > and catching the > keystrokes and mouse events. > > /Joe > > > > > I wouldn't bet that Erlang's good distribution / parallelization would > > offset the rest of the trouble. I'm unsure how production-ready it is, but > > Reia might make certain bits easier, just due to its string handling alone. > > > > JRuby or Clojure might be a better fit just because Java has MUCH more > > mature Unicode handling and syntax highlighting. I don't know of anything > > for Erlang that even tries to do syntax highlighting, although you could > > probably get a good start on the parsers with Neotoma. > > > > In any case, I wish you the best of luck. It's good to see people trying to > > tackle useful problems with the right tools. > > > > On Nov 1, 2009, at 12:31 AM, Ted Henry wrote: > > > >> With 100 core chips probably coming to desktop computers in the near > >> future, > >> I wonder about writing a multiprocess vi-like or emacs-like text editor > >> (probably more emacs-like in implementation.) Sometimes single threaded > >> editors really bog down when searching multiple files, syntax highlighting > >> multiple files. I'm not that knowledgeable in the Erlang world. It seems > >> like Erlang's fault tolerance and ability to upgrade code without > >> restarting > >> would not be big advantages for a text editor. Also a text editor isn't > >> usually distributed across multiple physical machines. I think the main > >> advantage would be Erlang's processes for concurrency. Would Erlang offer > >> a > >> good advantage over a programming language with software transactional > >> memory (e.g. Haskell, Clojure)? Would *you* use Erlang to program a text > >> editor or desktop application? > >> > >> Very interested to read your thoughts. > >> > >> Ted > > > > > > -- > > Jayson Vantuyl > > kagato@REDACTED > > > > > > > > > > > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From rvirding@REDACTED Mon Nov 2 13:18:44 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 2 Nov 2009 13:18:44 +0100 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: References: <3dbc6d1c0911011738i17100e4do3cb3b485caaf48fd@mail.gmail.com> Message-ID: <3dbc6d1c0911020418k4084a418m43c538fb485ddbf@mail.gmail.com> 2009/11/2 Michael Turner > > But aren't text editors supposed to be written in TECO? > > So, obviously, the first order of business is a TECO interpreter > implemented in Erlang, taking all possible advantage of its ease of > hotswapping modules, and of seamlessly scaling to many processes and > many processors. > > It could be a challenge. I believe that nobody has looked at this > particular problem before. I feel as if I am standing silent, with a > wild surmise, upon a peak in Darien. > Ah, a true believer in the old gods! Yes, of course this is the true path to greatness, but unfortunately many young apprentices have no knowledge of the old ways. Robert From vinoski@REDACTED Mon Nov 2 15:27:52 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Mon, 2 Nov 2009 10:27:52 -0400 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: <1257146412.4639.34.camel@piko.site> References: <27C8C8F9-D002-4949-B11A-9A0FA3C64203@souja.net> <9b08084c0911010654t7456e032q368d4e321b9ef703@mail.gmail.com> <1257146412.4639.34.camel@piko.site> Message-ID: <65b2728e0911020627u74c294cdp2abc1f8f470a8b66@mail.gmail.com> On Mon, Nov 2, 2009 at 3:20 AM, Alpar Juttner wrote: > An editor like emacs does not need multicore, but it _does_ need > concurrency, as it could provide much better responsiveness. When emacs > blocks for a couple of seconds, it is already annoying. When it blocks > for tens of seconds, it can be a considerable pain. > I use a package called emacs-jabber for doing IM from within emacs. I've never had it block emacs on me. When I started looking into how it was doing event handling, I found that it uses something called fsm.el: which has the following interesting comment near the top of the source code: ;; fsm.el is an exercise in metaprogramming inspired by gen_fsm of ;; Erlang/OTP. It aims to make asynchronous programming in Emacs Lisp ;; easy and fun. By "asynchronous" I mean that long-lasting tasks ;; don't interfer with normal editing. It's based on gen_fsm -- we've come full circle. :-) More emacs packages should make use of this to avoid blocking. --steve From raimo+erlang-questions@REDACTED Mon Nov 2 18:46:16 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 2 Nov 2009 18:46:16 +0100 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <6a3ae47e0910280435j2a7225ccqdf556aa107a58946@mail.gmail.com> References: <20091021083803.GA32147@erix.ericsson.se> <200910211157176241641@its3.ch> <20091021122804.GA4311@erix.ericsson.se> <20091023135603.GA31042@erix.ericsson.se> <6a3ae47e0910280435j2a7225ccqdf556aa107a58946@mail.gmail.com> Message-ID: <20091102174616.GA18199@erix.ericsson.se> Thanks a lot for the feedback! Then this fix will be in the next release. On Wed, Oct 28, 2009 at 11:35:34AM +0000, Robert Raschke wrote: > Hi Raimo. > > On Fri, Oct 23, 2009 at 1:56 PM, Raimo Niskanen < > raimo+erlang-questions@REDACTED > > wrote: > > > After a week of bug hunting we can confirm this is really > > Microsoft's bug, but we have a workaround. > > (It may be Microsoft's and Comodo's bug, who knows...) > > > > It turns out if you mix overlapped I/O with non-overlapped > > (asyncronous with syncronous) on different pipes, in this > > case the pipe to inet_gethost.exe from erlang was opened > > by erlang as non-overlapped and inet_gethost.exe itself > > used overlapped on all its other sockets and pipes... > > > > ...alright if you mix ... I/O something gets not initialized > > in Microsoft's libs unless you do a gethostname() call > > before going into recv() of commands from erlang (we already > > found this out the hard way) but now also if you had certain > > firewalls installed it was not enough, you had to do > > gethostbyname() calls with longer domain names to > > work around Microsoft's bug. > > > > Since this felt like a way to shaky workaround we found > > another that seems much better: now erlang can open the > > pipe to this specific port program (inet_gethost.exe) > > for overlapped I/O so inet_gethost.exe now uses overlapped > > I/O only. This seems to avoid the bug. > > > > We reproduced the name lookup freeze with Comodo 2.4.19.185 > > and Microsoft's Client for ISA Server 4.0. Here is > > a snapshot build (without documentation) that > > does not freeze for neither of these firewalls. > > > > http://www.erlang.org/download/snapshots/otp_win32_R13B03_gethost_fix.exe > > > > Try it and see if it solves your problem. > > > > / Raimo Niskanen > > > > > > > Congratulations! You have a winner. > > I have gone through the following tests: > > R13B02-1 on W2003, no Firewall Client for ISA > inet:gethostbyname("localhost"). > OK > inet:gethostbyname("local machine name"). > OK > inet:gethostbyname("machinename.localdomain"). > OK > inet:gethostbyname("www.google.com"). > OK > inet:gethostbyname("www.cisco.com"). > OK > > R13B02-1 on W2003, with Firewall Client for ISA > inet:gethostbyname("localhost"). > OK > inet:gethostbyname("local machine name"). > OK > inet:gethostbyname("machinename.localdomain"). > OK > inet:gethostbyname("www.google.com"). > timeout > inet:gethostbyname("www.cisco.com"). > timeout > > > R13B03_gethost_fix on W2003, no Firewall Client for ISA > inet:gethostbyname("localhost"). > OK > inet:gethostbyname("local machine name"). > OK > inet:gethostbyname("machinename.localdomain"). > OK > inet:gethostbyname("www.google.com"). > OK > inet:gethostbyname("www.cisco.com"). > OK > > R13B03_gethost_fix on W2003, with Firewall Client for ISA > inet:gethostbyname("localhost"). > OK > inet:gethostbyname("local machine name"). > OK > inet:gethostbyname("machinename.localdomain"). > OK > inet:gethostbyname("www.google.com"). > OK > inet:gethostbyname("www.cisco.com"). > OK > > > Brilliant work for tracking down this obscure behaviour. > > Robby -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From torben.lehoff@REDACTED Mon Nov 2 21:43:02 2009 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 2 Nov 2009 21:43:02 +0100 Subject: [erlang-questions] How to get a supervisor to tell me a child's startfunc? In-Reply-To: References: Message-ID: Jim, Since this seems to receive little attention and I kind of wanted to know this too, I digged around a bit... I have not tried this out with dynamically created children, but for standard children something along the lines of: supervisor:which_children(SupRef) to get the list of children of a supervisor. The first element of the tuples in the list is the name of the child. sys:get_status(ChildName) will then give you details about the child. Example: 1> application:start(sasl). .... loads of things 2> supervisor:which_children(sasl_safe_sup). [{overload,<0.41.0>,worker,[overload]}, {alarm_handler,<0.40.0>,worker,dynamic}] 3> sys:get_status(overload). {status,<0.41.0>, {module,gen_server}, [[{'$ancestors',[sasl_safe_sup,sasl_sup,<0.37.0>]}, {'$initial_call',{overload,init,1}}], running,<0.39.0>,[], [overload, {state,0,0,0.8,5244,0.1,{0,0},clear}, overload,infinity]]} The '$initial_call' tag seems to provide at least which function was called. But it turns out that calling this on the supervisor is the solution! 11> sys:get_status(sasl_safe_sup). {status,<0.39.0>, {module,gen_server}, [[{'$ancestors',[sasl_sup,<0.37.0>]}, {'$initial_call',{supervisor,sasl,1}}], running,<0.38.0>,[], [sasl_safe_sup, {state,{local,sasl_safe_sup}, one_for_one, [{child,<0.41.0>,overload, {overload,start_link,[]}, permanent,2000,worker, [overload]}, {child,<0.40.0>,alarm_handler, {alarm_handler,start_link,[]}, permanent,2000,worker,dynamic}], {dict,0,16,16,8,80,48,{[],[],[],[],[],...},{{[],[],[],...}}}, 4,3600,[],sasl,safe}, supervisor,infinity]]} Fouth element of the child tagged tuples reveals how he children were started. Sorry about dragging you through the whole investigation, but I think it holds some value... at least as to how I think ;-) Cheers, Torben On Fri, Oct 30, 2009 at 17:45, Jim McCoy wrote: > I need to discover the startfunc being used by a variety of processes > started dynamically by a supervisor and I am having a hard time > figuring out how to dig this information out of the system. Process A > starts a bunch of children for supervisor S over its lifetime, process > B needs to do some maintenance or checking of which of these children > are running at various points in time, to do this B needs to > specifically know the start function that A used when it passed the > child spec for any particular child process in to S. This information > does not seem to be available directly from the supervisor, so I am > guessing that there is some other place I need to look but my various > search queries seem to be missing the magic keyword that will not lead > me back to the standard supervisor docs... > > Any hints? > > Thanks, > jim > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- http://www.linkedin.com/in/torbenhoffmann From torben.lehoff@REDACTED Mon Nov 2 21:45:54 2009 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 2 Nov 2009 21:45:54 +0100 Subject: [erlang-questions] module dependencies between applications when using .appup files In-Reply-To: <54E2F5E30BE5BC42AC4FF38C4545AAE1A352B9@UKLHREXCL02.activision.com> References: <54E2F5E30BE5BC42AC4FF38C4545AAE1A352B9@UKLHREXCL02.activision.com> Message-ID: Malcom, I think you should have a look at phased application start-up - to the best of my recollection the principles there also apply when you do an application upgrade. Cheers, Torben On Fri, Oct 30, 2009 at 19:33, Dowse, Malcolm wrote: > Hi, > > > > I have a question about the release handler and .appup files. > > > > I have two applications, app, and app_common, and a release which uses > both. app_common is a dependency of app. So if app is running, > app_common will be too.. but not the opposite. > > > > It is possible to force a module in app_common to load before one in > app. You specify {load_module, app_module, [app_common_module]} in > app.appup. This isn't very elegant, but it definitely works. > > > > The problem I'm having is that when upgrading the release, I want to > load a module in app before I load one in app_common. Without a command > like {load_module, app_module, [{reverse_dependency, > app_common_module}]} there seems to be no way of doing this. Adding > {load_module, app_common_module, [app_module]} to app_common.appup is > out of the question. > > > > Can anyone tell me any clever tricks/hacks to get around this? > > > > Thanks in advance, > > > > Malcolm > > -- http://www.linkedin.com/in/torbenhoffmann From rvirding@REDACTED Mon Nov 2 22:36:07 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 2 Nov 2009 22:36:07 +0100 Subject: [erlang-bugs] Re: Re: [erlang-questions] Erlang's process not killed by exit signalreasoned by "kill" In-Reply-To: <4AE9A0E8.7050101@erlang-consulting.com> References: <200910241907167657586@gmail.com> <200910242342216873502@gmail.com> <3dbc6d1c0910241310p53a30c23m58ba263d271f99fe@mail.gmail.com> <9b08084c0910281432m134e4f98p4154eb33ca224f52@mail.gmail.com> <3dbc6d1c0910290636u557f51e2y51e067d4d7f117fe@mail.gmail.com> <4AE9A0E8.7050101@erlang-consulting.com> Message-ID: <3dbc6d1c0911021336k705314c2xd581c4d0e5fc9ed0@mail.gmail.com> 2009/10/29 Ulf Wiger > Robert Virding wrote: > >> >> BUT when I do exit(kill) I send a type 2 signal with value 'kill' which >> doesn't behave as a type 3 'kill' signal though the name is the same. This >> is inconsistent, and I don't like inconsistencies. There are two different >> solutions: either send out real type 3 kill signals; or send out type 2 >> 'killed' signals as if I had received a type 3 signal. Both are >> consistent, >> but which is best? >> > > Why send out type 3 kill signals? > > What is that consistent with? > > I think the consistent behaviour would be to send out type 2 > 'killed' signals. I quite agree Ulf, I think that would be best too. I just wanted some form of discussion so we could get a change from what it is today so I presented the two alternatives as I saw them. Robert From info@REDACTED Mon Nov 2 23:59:05 2009 From: info@REDACTED (info) Date: Mon, 2 Nov 2009 23:59:05 +0100 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? References: <20091021083803.GA32147@erix.ericsson.se>, <200910211157176241641@its3.ch>, <20091021122804.GA4311@erix.ericsson.se>, <20091023135603.GA31042@erix.ericsson.se>, <6a3ae47e0910280435j2a7225ccqdf556aa107a58946@mail.gmail.com> Message-ID: <200911022359044040264@its3.ch> Hi both, Nothing to add ... It was not easy to find and resolve this problem ! I changed the firewall version and now everything is ok. Thank you to Raimo. John Hi Raimo. On Fri, Oct 23, 2009 at 1:56 PM, Raimo Niskanen < raimo+erlang-questions@REDACTED > wrote: > After a week of bug hunting we can confirm this is really > Microsoft's bug, but we have a workaround. > (It may be Microsoft's and Comodo's bug, who knows...) > > It turns out if you mix overlapped I/O with non-overlapped > (asyncronous with syncronous) on different pipes, in this > case the pipe to inet_gethost.exe from erlang was opened > by erlang as non-overlapped and inet_gethost.exe itself > used overlapped on all its other sockets and pipes... > > ...alright if you mix ... I/O something gets not initialized > in Microsoft's libs unless you do a gethostname() call > before going into recv() of commands from erlang (we already > found this out the hard way) but now also if you had certain > firewalls installed it was not enough, you had to do > gethostbyname() calls with longer domain names to > work around Microsoft's bug. > > Since this felt like a way to shaky workaround we found > another that seems much better: now erlang can open the > pipe to this specific port program (inet_gethost.exe) > for overlapped I/O so inet_gethost.exe now uses overlapped > I/O only. This seems to avoid the bug. > > We reproduced the name lookup freeze with Comodo 2.4.19.185 > and Microsoft's Client for ISA Server 4.0. Here is > a snapshot build (without documentation) that > does not freeze for neither of these firewalls. > > http://www.erlang.org/download/snapshots/otp_win32_R13B03_gethost_fix.exe > > Try it and see if it solves your problem. > > / Raimo Niskanen > > > Congratulations! You have a winner. I have gone through the following tests: R13B02-1 on W2003, no Firewall Client for ISA inet:gethostbyname("localhost"). OK inet:gethostbyname("local machine name"). OK inet:gethostbyname("machinename.localdomain"). OK inet:gethostbyname("www.google.com"). OK inet:gethostbyname("www.cisco.com"). OK R13B02-1 on W2003, with Firewall Client for ISA inet:gethostbyname("localhost"). OK inet:gethostbyname("local machine name"). OK inet:gethostbyname("machinename.localdomain"). OK inet:gethostbyname("www.google.com"). timeout inet:gethostbyname("www.cisco.com"). timeout R13B03_gethost_fix on W2003, no Firewall Client for ISA inet:gethostbyname("localhost"). OK inet:gethostbyname("local machine name"). OK inet:gethostbyname("machinename.localdomain"). OK inet:gethostbyname("www.google.com"). OK inet:gethostbyname("www.cisco.com"). OK R13B03_gethost_fix on W2003, with Firewall Client for ISA inet:gethostbyname("localhost"). OK inet:gethostbyname("local machine name"). OK inet:gethostbyname("machinename.localdomain"). OK inet:gethostbyname("www.google.com"). OK inet:gethostbyname("www.cisco.com"). OK Brilliant work for tracking down this obscure behaviour. Robby From garry@REDACTED Tue Nov 3 05:45:48 2009 From: garry@REDACTED (Garry Hodgson) Date: Mon, 02 Nov 2009 23:45:48 -0500 Subject: weird mneisa/escript problem in r13 Message-ID: <4AEFB57C.6020306@research.att.com> i am using mnesia from within some escript code, and i cannot get it to work on both r13b01 and r13b02-2. the error occurs when i call mnesia:create_schema( [Node] ). i specify an mnesia dir on line 2 of the escript code using one of: 1) %%! -mnesia dir '"test/bunderl"' 2) %%! -mneisa dir "test/bunderl" when i use version 1, it works on r13b01, but fails on r13b02-2 with: {"Cannot create Mnesia dir", "/home/garry/admin/\"test/bunderl\"", enoent}} and yes, the appropriate dirs exist, and it fails the same if i clean it out between runs. i am running it from /home/garry/admin, and test/bundle exists and is writable and all that. when i use version 2, it works on r13b02-2, but fails on r13b01 with: =ERROR REPORT==== 2-Nov-2009::23:28:06 === application_controller: bad term: test/bunderl escript: exception throw: {create_failed,bunderl@REDACTED, {application_load_error, {bad_environment_value,"test/bunderl"}}} am i doing something wrong, or is there a bug in one or the other of these releases? -- Garry Hodgson Lead Member of Technical Staff AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From kenneth.lundin@REDACTED Tue Nov 3 08:21:31 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 3 Nov 2009 08:21:31 +0100 Subject: [erlang-questions] Corrupt database in mnesia In-Reply-To: <244668.41236.qm@web112610.mail.gq1.yahoo.com> References: <244668.41236.qm@web112610.mail.gq1.yahoo.com> Message-ID: Hi, If you have a previous log with size 150G I suppose you have run with mensia overloaded for quite some time. What is your mnesia setup ? disk_copy tables or what? How big entries in the tables? The PREVIOUS.LOG is the transactions whose result is not yet written into the tables. I.e a backlog of things to do. When you delete the PREVIOUS and LATEST you will miss all the transactions not yet reflected in the tables. /Kenneth Erlang/OTP Ericsson On Sun, Nov 1, 2009 at 10:07 PM, Yogish Baliga wrote: > Hello all, > > ?On our production box, I have seen mnesia getting corrupt once in a while. LATEST.LOG, PREVIOUS.LOG becomes 150G or so and 1st fragment of .DCL file reaches 6G or so. Restarting mnesia again will take forever to recover. But after deleting LATEST.LOG, PREVIOUS.LOG file and restarting mnesia will recover pretty fast and the data is intact in the database. > > ?2 questions regarding this issue > > ? ?1. What is causing the corruption of .LOG and .DCL files? > ? ?2. Why is that removing LATEST.LOG, PREVIOUS.LOG and restarting mnesia recovers pretty fast and data is not lost? > ? ?3. How to prevent this from happening again? > > Thanx, > -- baliga > > > "Point of view is worth 80 IQ points" --Alan Kay > > http://dudefrommangalore.blogspot.com/ From yogishb@REDACTED Tue Nov 3 07:48:50 2009 From: yogishb@REDACTED (Yogish Baliga) Date: Mon, 2 Nov 2009 22:48:50 -0800 (PST) Subject: [erlang-questions] weird mneisa/escript problem in r13 In-Reply-To: <4AEFB57C.6020306@research.att.com> References: <4AEFB57C.6020306@research.att.com> Message-ID: <615795.67725.qm@web112611.mail.gq1.yahoo.com> Try -mnesia dir '"test/bunderl"' NOTE: double quote within single quotes/ -- baliga "Point of view is worth 80 IQ points" --Alan Kay http://dudefrommangalore.blogspot.com/ ________________________________ From: Garry Hodgson To: erlang-questions@REDACTED Sent: Mon, November 2, 2009 8:45:48 PM Subject: [erlang-questions] weird mneisa/escript problem in r13 i am using mnesia from within some escript code, and i cannot get it to work on both r13b01 and r13b02-2. the error occurs when i call mnesia:create_schema( [Node] ). i specify an mnesia dir on line 2 of the escript code using one of: 1) %%! -mnesia dir '"test/bunderl"' 2) %%! -mneisa dir "test/bunderl" when i use version 1, it works on r13b01, but fails on r13b02-2 with: {"Cannot create Mnesia dir", "/home/garry/admin/\"test/bunderl\"", enoent}} and yes, the appropriate dirs exist, and it fails the same if i clean it out between runs. i am running it from /home/garry/admin, and test/bundle exists and is writable and all that. when i use version 2, it works on r13b02-2, but fails on r13b01 with: =ERROR REPORT==== 2-Nov-2009::23:28:06 === application_controller: bad term: test/bunderl escript: exception throw: {create_failed,bunderl@REDACTED, {application_load_error, {bad_environment_value,"test/bunderl"}}} am i doing something wrong, or is there a bug in one or the other of these releases? -- Garry Hodgson Lead Member of Technical Staff AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From garry@REDACTED Tue Nov 3 14:05:30 2009 From: garry@REDACTED (Garry Hodgson) Date: Tue, 03 Nov 2009 08:05:30 -0500 Subject: [erlang-questions] weird mneisa/escript problem in r13 In-Reply-To: <615795.67725.qm@web112611.mail.gq1.yahoo.com> References: <4AEFB57C.6020306@research.att.com> <615795.67725.qm@web112611.mail.gq1.yahoo.com> Message-ID: <4AF02A9A.9090107@research.att.com> Yogish Baliga wrote: > Try > > -mnesia dir '"test/bunderl"' > > NOTE: double quote within single quotes/ i'm sorry i didn't make it clear in my post, but that's what version 1 is. -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From luke@REDACTED Tue Nov 3 14:45:27 2009 From: luke@REDACTED (Luke Gorrie) Date: Tue, 3 Nov 2009 14:45:27 +0100 Subject: erlang for programming a text editor Message-ID: <1d0fb8fc0911030545g11856752ubb634709f3f61562@mail.gmail.com> Ted Henry wrote: > Very interested to read your thoughts [on text editors in Erlang] The desire to rewrite Emacs in my-pet-language is a form of insanity! The only cure I know of is to actually do it, and then you'll realise that Emacs Lisp is a great language to write Emacs in and wonder what all the fuss is about :-) I have a strong suspicion that loving Emacs in the 21st century is also a form of insanity, but I'm not sure of the cure yet.. I think it involves making friends with Smalltalk hackers :-) -Luke P.S. Apologies for probable poor threading of this message. From gethemant@REDACTED Tue Nov 3 16:04:10 2009 From: gethemant@REDACTED (hemant) Date: Tue, 3 Nov 2009 20:34:10 +0530 Subject: Compiler crashes on ubuntu 9.10 Message-ID: Folks, I am getting compiler crashes while compiling rabbitmq-public-umbrella package and Alice (http://wiki.github.com/auser/alice). Typical compile error looks like: make[1]: Entering directory `/home/hemant/alice/deps/mochiweb' (cd src;make all) make[2]: Entering directory `/home/hemant/alice/deps/mochiweb/src' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/home/hemant/alice/deps/mochiweb/src' make[1]: Leaving directory `/home/hemant/alice/deps/mochiweb' {"init terminating in do_boot",{undef,[{make,all,[]},{erl_eval,do_apply,5},{erl_eval,expr,5},{init,start_it,1},{init,start_em,1}]}} Crash dump was written to: erl_crash.dump init terminating in do_boot () erl_crash.dump file is big and hence I am not attaching it here. But it can be downloaded from, http://backgroundrb.gnufied.org/erl_crash.dump location. Any idea, why is this happening? -- Let them talk of their oriental summer climes of everlasting conservatories; give me the privilege of making my own summer with my own coals. http://gnufied.org From goran.bage@REDACTED Tue Nov 3 16:55:05 2009 From: goran.bage@REDACTED (=?UTF-8?B?R8O2cmFuIELDpWdl?=) Date: Tue, 03 Nov 2009 16:55:05 +0100 Subject: R11 configuration problem Message-ID: <4AF05259.5000202@mobilearts.com> Hello, I tried to install R11R-5 on my new Imac running Snow Leopard and got this trying to configure: ... checking for IP version 6 support... yes checking for multicast support... rm: conftest.dSYM: is a directory yes checking how to correct for time adjustments... none checking if gethrvtime works and how to use it... not working checking if clock_gettime can be used to get process CPU time... not working checking for unreliable floating point execptions... and there it gets stuck, hangs forever (or a very long time). Anyone out there know what the problem is? No problems with R12, R13, and yes I need R11 as we have customer installations using R11 that we need to maintain. Cheers, -- -- Goran ------------------------- May the Snow be with you ---- Goran Bage, MobileArts, www.mobilearts.com Tjarhovsgatan 56, SE-116 28 Stockholm, Sweden email:goran.bage@REDACTED, phone: +46 733 358405 From mikpe@REDACTED Tue Nov 3 18:45:13 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Tue, 3 Nov 2009 18:45:13 +0100 Subject: [erlang-questions] R11 configuration problem In-Reply-To: <4AF05259.5000202@mobilearts.com> References: <4AF05259.5000202@mobilearts.com> Message-ID: <19184.27689.504103.803073@pilspetsen.it.uu.se> =?UTF-8?B?R8O2cmFuIELDpWdl?= writes: > Hello, > > I tried to install R11R-5 on my new Imac running Snow Leopard and > got this trying to configure: > ... > checking for IP version 6 support... yes > checking for multicast support... rm: conftest.dSYM: is a directory > yes > checking how to correct for time adjustments... none > checking if gethrvtime works and how to use it... not working > checking if clock_gettime can be used to get process CPU time... not working > checking for unreliable floating point execptions... > > and there it gets stuck, hangs forever (or a very long time). > Anyone out there know what the problem is? > No problems with R12, R13, and yes I need R11 as we have customer installations > using R11 that we need to maintain. There are two main changes related to that test between R11B-5 and R12B-5, the first is surrounding logic to only run the test if HiPE is enabled or the user requested the feature, the second is a check in the test itself against infinite SIGFPE loops. I suspect the latter is what's happening to you. The patch below backports that one fix to R11B-5, apply it and try ./configure again. --- otp_src_R11B-5/erts/configure.in.~1~ 2007-06-11 14:53:45.000000000 +0200 +++ otp_src_R11B-5/erts/configure.in 2009-11-03 18:41:17.000000000 +0100 @@ -1674,6 +1674,19 @@ AC_TRY_RUN([ volatile int erl_fp_exception; +/* + * We expect a single SIGFPE in this test program. + * Getting many more indicates an inadequate SIGFPE handler, + * e.g. using the generic handler on x86. + */ +static void new_fp_exception(void) +{ + if (++erl_fp_exception > 50) { + fprintf(stderr, "SIGFPE loop detected, bailing out\n"); + exit(1); + } +} + /* Is there no standard identifier for Darwin/MacOSX ? */ #if defined(__APPLE__) && defined(__MACH__) && !defined(__DARWIN__) #define __DARWIN__ 1 @@ -2017,7 +2030,7 @@ static void fpe_sig_action(int sig, sigi fpstate->mxcsr = 0x1F80; fpstate->sw &= ~0xFF; #endif - erl_fp_exception = 1; + new_fp_exception(); } static void do_init(void) @@ -2034,7 +2047,7 @@ static void do_init(void) static void fpe_sig_handler(int sig) { - erl_fp_exception = 1; + new_fp_exception(); } static void do_init(void) --- otp_src_R11B-5/erts/configure.~1~ 2007-06-11 19:16:18.000000000 +0200 +++ otp_src_R11B-5/erts/configure 2009-11-03 18:42:20.000000000 +0100 @@ -18197,6 +18197,19 @@ cat >>conftest.$ac_ext <<_ACEOF volatile int erl_fp_exception; +/* + * We expect a single SIGFPE in this test program. + * Getting many more indicates an inadequate SIGFPE handler, + * e.g. using the generic handler on x86. + */ +static void new_fp_exception(void) +{ + if (++erl_fp_exception > 50) { + fprintf(stderr, "SIGFPE loop detected, bailing out\n"); + exit(1); + } +} + /* Is there no standard identifier for Darwin/MacOSX ? */ #if defined(__APPLE__) && defined(__MACH__) && !defined(__DARWIN__) #define __DARWIN__ 1 @@ -18540,7 +18553,7 @@ static void fpe_sig_action(int sig, sigi fpstate->mxcsr = 0x1F80; fpstate->sw &= ~0xFF; #endif - erl_fp_exception = 1; + new_fp_exception(); } static void do_init(void) @@ -18557,7 +18570,7 @@ static void do_init(void) static void fpe_sig_handler(int sig) { - erl_fp_exception = 1; + new_fp_exception(); } static void do_init(void) From gethemant@REDACTED Tue Nov 3 19:58:13 2009 From: gethemant@REDACTED (hemant) Date: Wed, 4 Nov 2009 00:28:13 +0530 Subject: Compiler crashes on ubuntu 9.10 In-Reply-To: References: Message-ID: On Tue, Nov 3, 2009 at 8:34 PM, hemant wrote: > Folks, > > I am getting compiler crashes while compiling rabbitmq-public-umbrella > package and Alice (http://wiki.github.com/auser/alice). ?Typical > compile error looks like: > > make[1]: Entering directory `/home/hemant/alice/deps/mochiweb' > (cd src;make all) > make[2]: Entering directory `/home/hemant/alice/deps/mochiweb/src' > make[2]: Nothing to be done for `all'. > make[2]: Leaving directory `/home/hemant/alice/deps/mochiweb/src' > make[1]: Leaving directory `/home/hemant/alice/deps/mochiweb' > {"init terminating in > do_boot",{undef,[{make,all,[]},{erl_eval,do_apply,5},{erl_eval,expr,5},{init,start_it,1},{init,start_em,1}]}} > > Crash dump was written to: erl_crash.dump > init terminating in do_boot () > > > erl_crash.dump file is big and hence I am not attaching it here. But > it can be downloaded from, > http://backgroundrb.gnufied.org/erl_crash.dump location. > > > Any idea, why is this happening? Folks, Apologies for the mail. The problem was because of some missing packages From tedhenry10@REDACTED Tue Nov 3 23:16:05 2009 From: tedhenry10@REDACTED (Ted Henry) Date: Tue, 3 Nov 2009 14:16:05 -0800 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: <1257146412.4639.34.camel@piko.site> References: <27C8C8F9-D002-4949-B11A-9A0FA3C64203@souja.net> <9b08084c0911010654t7456e032q368d4e321b9ef703@mail.gmail.com> <1257146412.4639.34.camel@piko.site> Message-ID: On Sun, Nov 1, 2009 at 11:20 PM, Alpar Juttner wrote: > An editor like emacs does not need multicore, but it _does_ need > concurrency, as it could provide much better responsiveness. When emacs > blocks for a couple of seconds, it is already annoying. When it blocks > for tens of seconds, it can be a considerable pain. > > ? ? ?* Several years ago, I had gave up using a Emacs-RMAIL, because > ? ? ? ?fetching new mail took a long time on our environment and this > ? ? ? ?process blocks the whole emacs. > ? ? ?* Emacs can open remote files through ssh/ftp. A wonderful feature > ? ? ? ?but opening and (auto)saving may take quite a while through the > ? ? ? ?net and these processes also block whole emacs. > ? ? ?* Emacs-w3 is also practically unusable due to its blocking > ? ? ? ?behavior. Thanks for your response and examples. These issues seem more related to Emacs as an OS (as some non-Emacs users like to accuse) rather than Emacs as a simple text editor...not that I'd be shooting for the simple text editor. ;-) Ted From tedhenry10@REDACTED Tue Nov 3 23:17:37 2009 From: tedhenry10@REDACTED (Ted Henry) Date: Tue, 3 Nov 2009 14:17:37 -0800 Subject: [erlang-questions] Re: erlang for programming a text editor In-Reply-To: <1d0fb8fc0911030545g11856752ubb634709f3f61562@mail.gmail.com> References: <1d0fb8fc0911030545g11856752ubb634709f3f61562@mail.gmail.com> Message-ID: On Tue, Nov 3, 2009 at 5:45 AM, Luke Gorrie wrote: > The desire to rewrite Emacs in my-pet-language is a form of insanity! And what if you want to invent your own language and then implement an Emacs in that language? :-S Ted From ok@REDACTED Wed Nov 4 00:30:32 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 4 Nov 2009 12:30:32 +1300 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: <1257146412.4639.34.camel@piko.site> References: <27C8C8F9-D002-4949-B11A-9A0FA3C64203@souja.net> <9b08084c0911010654t7456e032q368d4e321b9ef703@mail.gmail.com> <1257146412.4639.34.camel@piko.site> Message-ID: On Nov 2, 2009, at 8:20 PM, Alpar Juttner wrote: > * Several years ago, I had gave up using a Emacs-RMAIL, because > fetching new mail took a long time on our environment and this > process blocks the whole emacs. I use "Mail" version 3.6 on a 2.2GHz intel Mac running MacOS 10.5.7. I can assure you that l o n g pauses to fetch new mail are by no means confined to Emacs. Activity Monitor reports that Mail is using 12 ... no wait ... 11 threads, no wait, it's 12 again. So simply having and using concurrency is no guarantee that you won't get long waits. Our sysadmins made a unilateral decision to move everyone's files off their local machines onto a file server. There are now frequent jarring waits while Mail waits for a file server in order to touch some file I don't particularly want it to touch at that time. The fundamental issue is *system design*. It's thinking about what kinds of delays there might be and arranging for the program to let you do something else (such as compose a message) while that's happening. For this, threads are a great help, but not a necessity. (Well, you _could_ write the whole thing in assembly code (:-) (:-).) > > An inherently concurrent design would immediately eliminate all of > these > issues. Mail is evidence that an inherently concurrent design *as such* need not eliminate *any* of these issues. Concurrency makes it *easier* to create good designs, but good design never just happens. From fedora789@REDACTED Wed Nov 4 00:41:14 2009 From: fedora789@REDACTED (fedora) Date: Tue, 3 Nov 2009 15:41:14 -0800 Subject: grep data at erlang console Message-ID: <35eed8d00911031541s3b34cf79wd3688d5bee4d2c8c@mail.gmail.com> Hi, any one know a quick way to grep or sort the long output of erlang console ? for example, process list, something should like grep( "keyword", i() ) , sort( i() ) , sort( nodes() ) . Thanks. John From roberto.aloi@REDACTED Wed Nov 4 01:04:57 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Wed, 4 Nov 2009 00:04:57 +0000 Subject: [erlang-questions] grep data at erlang console In-Reply-To: <35eed8d00911031541s3b34cf79wd3688d5bee4d2c8c@mail.gmail.com> References: <35eed8d00911031541s3b34cf79wd3688d5bee4d2c8c@mail.gmail.com> Message-ID: In these cases I usually run the shell from within Emacs and I do the grepping and all other operations there. Macros are also useful. Roberto Aloi Erlang Training and Consulting Ltd. http://www.erlang-consulting.com On 3 Nov 2009, at 23:41, fedora wrote: > Hi, any one know a quick way to grep or sort the long output of > erlang console ? for example, process list, something should like > grep( "keyword", i() ) , sort( i() ) , sort( nodes() ) . > > Thanks. > > John > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From yogishb@REDACTED Wed Nov 4 03:21:00 2009 From: yogishb@REDACTED (Yogish Baliga) Date: Tue, 3 Nov 2009 18:21:00 -0800 (PST) Subject: [erlang-questions] Corrupt database in mnesia In-Reply-To: References: <244668.41236.qm@web112610.mail.gq1.yahoo.com> Message-ID: <473554.14432.qm@web112615.mail.gq1.yahoo.com> Hello Kenneth, Thanx for the response. Mnesia is setup as disk_copy tables. These tables store the logs. So loosing them was not a major issues. I did not see Mnesia overloaded message in the Erlang VM log file. In any case, I figured out the root cause of huge transaction log file. I was emulating the bag semantics of the table at application level by storing the the log entries as list. Every write was fetching the existing list, adding the current log to it and writing it back to table. :-(. This has dramatic effect only after having a large number of entries in the list. The current implementation started as a hack for log storage and ended up in production environment. I am in the process of changing the table type of bag. Cheers, -- baliga "Point of view is worth 80 IQ points" --Alan Kay http://dudefrommangalore.blogspot.com/ ________________________________ From: Kenneth Lundin To: Yogish Baliga Cc: Erlang Questions Sent: Mon, November 2, 2009 11:21:31 PM Subject: Re: [erlang-questions] Corrupt database in mnesia Hi, If you have a previous log with size 150G I suppose you have run with mensia overloaded for quite some time. What is your mnesia setup ? disk_copy tables or what? How big entries in the tables? The PREVIOUS.LOG is the transactions whose result is not yet written into the tables. I.e a backlog of things to do. When you delete the PREVIOUS and LATEST you will miss all the transactions not yet reflected in the tables. /Kenneth Erlang/OTP Ericsson On Sun, Nov 1, 2009 at 10:07 PM, Yogish Baliga wrote: > Hello all, > > On our production box, I have seen mnesia getting corrupt once in a while. LATEST.LOG, PREVIOUS.LOG becomes 150G or so and 1st fragment of .DCL file reaches 6G or so. Restarting mnesia again will take forever to recover. But after deleting LATEST.LOG, PREVIOUS.LOG file and restarting mnesia will recover pretty fast and the data is intact in the database. > > 2 questions regarding this issue > > 1. What is causing the corruption of .LOG and .DCL files? > 2. Why is that removing LATEST.LOG, PREVIOUS.LOG and restarting mnesia recovers pretty fast and data is not lost? > 3. How to prevent this from happening again? > > Thanx, > -- baliga > > > "Point of view is worth 80 IQ points" --Alan Kay > > http://dudefrommangalore.blogspot.com/ ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From leap@REDACTED Wed Nov 4 03:55:48 2009 From: leap@REDACTED (Michael Turner) Date: Wed, 04 Nov 2009 02:55:48 +0000 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: Message-ID: "Good design never just happens." Great bumpersticker line! Copyright it immediately. The main useful purpose I can see for using Erlang to implement a text-editor-cum-general-purpose-shell-programming-environment (Emacs being exemplary) is to make a promotional statement about Erlang: your success says, "Look, Erlang is not just great for what it was designed for, it's also quite adequate for other purposes." Wings3D already does a pretty good job in that department, though. Offhand, I can't think of a good reason to write a 3D modeler in Erlang specifically. What Wings3D says, in effect, is that if you just want to write something complex for fun and learning value, you shouldn't worry that Erlang would hamper you significantly -- it might make some things harder, but not prohibitively so. Wings3D is better than some hypothetical Emacs clone in another (promotional) respect: your average *user* can relate to it better. Emacs grew up in the glass tty days, it's got a face (and a raison d'etre) that only a programmer could love. Most people these days don't even know what you mean when you say "text editor", any more than they know what you mean when you say "Erlang is Turing-complete." -michael turner On 11/3/2009, "Richard O'Keefe" wrote: > >On Nov 2, 2009, at 8:20 PM, Alpar Juttner wrote: >> * Several years ago, I had gave up using a Emacs-RMAIL, because >> fetching new mail took a long time on our environment and this >> process blocks the whole emacs. > >I use "Mail" version 3.6 on a 2.2GHz intel Mac running MacOS 10.5.7. > >I can assure you that l o n g pauses to fetch new mail are >by no means confined to Emacs. Activity Monitor reports that Mail >is using 12 ... no wait ... 11 threads, no wait, it's 12 again. >So simply having and using concurrency is no guarantee that you won't >get long waits. > >Our sysadmins made a unilateral decision to move everyone's files off >their local machines onto a file server. There are now frequent >jarring waits while Mail waits for a file server in order to touch >some file I don't particularly want it to touch at that time. > >The fundamental issue is *system design*. It's thinking about what >kinds >of delays there might be and arranging for the program to let you do >something else (such as compose a message) while that's happening. For >this, threads are a great help, but not a necessity. (Well, you _could_ >write the whole thing in assembly code (:-) (:-).) >> > >> An inherently concurrent design would immediately eliminate all of >> these >> issues. > >Mail is evidence that an inherently concurrent design *as such* need not >eliminate *any* of these issues. Concurrency makes it *easier* to >create >good designs, but good design never just happens. > > >________________________________________________________________ >erlang-questions mailing list. See http://www.erlang.org/faq.html >erlang-questions (at) erlang.org > > From yogishb@REDACTED Wed Nov 4 03:29:47 2009 From: yogishb@REDACTED (Yogish Baliga) Date: Tue, 3 Nov 2009 18:29:47 -0800 (PST) Subject: [erlang-questions] weird mneisa/escript problem in r13 In-Reply-To: <4AF02A9A.9090107@research.att.com> References: <4AEFB57C.6020306@research.att.com> <615795.67725.qm@web112611.mail.gq1.yahoo.com> <4AF02A9A.9090107@research.att.com> Message-ID: <224598.32231.qm@web112619.mail.gq1.yahoo.com> Looks like there is some shell interpretation going on. What is the output of application:get_all_env(mnesia)? Also can you try without quotes? -- baliga "Point of view is worth 80 IQ points" --Alan Kay http://dudefrommangalore.blogspot.com/ ________________________________ From: Garry Hodgson To: Yogish Baliga Cc: erlang-questions@REDACTED Sent: Tue, November 3, 2009 5:05:30 AM Subject: Re: [erlang-questions] weird mneisa/escript problem in r13 Yogish Baliga wrote: > Try > > -mnesia dir '"test/bunderl"' > > NOTE: double quote within single quotes/ i'm sorry i didn't make it clear in my post, but that's what version 1 is. -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From garry@REDACTED Wed Nov 4 04:50:46 2009 From: garry@REDACTED (Garry Hodgson) Date: Tue, 03 Nov 2009 22:50:46 -0500 Subject: [erlang-questions] weird mneisa/escript problem in r13 In-Reply-To: <224598.32231.qm@web112619.mail.gq1.yahoo.com> References: <4AEFB57C.6020306@research.att.com> <615795.67725.qm@web112611.mail.gq1.yahoo.com> <4AF02A9A.9090107@research.att.com> <224598.32231.qm@web112619.mail.gq1.yahoo.com> Message-ID: <4AF0FA16.1000709@research.att.com> Yogish Baliga wrote: > Looks like there is some shell interpretation going on. > > What is the output of application:get_all_env(mnesia)? > > Also can you try without quotes? i've tried it both ways, as mentioned in the original post. the problem is not getting this to work. i can do that easily. the problem is that it behaves so differently in two very similar versions r13b01 and r13b02. i see nothing in the release notes to inidicate such a change, so i thought i'd mention it, in case it was a bug. -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From tamas.nagy@REDACTED Wed Nov 4 09:49:00 2009 From: tamas.nagy@REDACTED (Tamas Nagy) Date: Wed, 4 Nov 2009 08:49:00 +0000 Subject: [erlang-questions] Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <401d3ba30911010413v162e629egd979abecc9f4a59@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <401d3ba30911010413v162e629egd979abecc9f4a59@mail.gmail.com> Message-ID: <6643BAB0-C16B-41FF-8370-893906FCD753@erlang-consulting.com> Hi Attila, Well if you would like to see the gen_server's state you can always do a sys:get_status/1. Regards, Tamas Tamas Nagy Erlang Training & Consulting http://www.erlang-consulting.com On 1 Nov 2009, at 12:13, Attila Rajmund Nohl wrote: > 2009/10/29, Geoff Cant : >> [...] Erlang programmers >> usually only have to think about the function body and arguments to >> work >> out what its going to do. Sprinkling 'get' and 'put' through the code >> means that an erlang programmer trying to understand your code now >> has >> to read all the code to figure out why something is happening. The >> order >> in which functions are called becomes important. The behaviour of >> functions in other modules becomes important because now there's a >> back-channel to propagate bugs, er, state between parts of the code. > > This is not related to the process dictionary. If the programmer > implements e.g. a handle_call in gen_server 'A' (or anything that's > called from that handle_call), he has to make sure that he doesn't > call gen_server 'B' if there's a chance that 'B' called 'A' first - > otherwise there would be a deadlock. In this case also a lot of other > code gets important and it's fairly common that a handle_call (or a > function called from handle_call) gets implemented... > > Like it or not, it's important that in what circumstances a function > is called - this is a limitation in Erlang's "functional > languageness", but actually this is necessary to do anything useful > with the language. > > The process dictionary could be great for environment-like variables, > which are only set once, but used in very many places and it's very > inconvenient to pass around one more parameter. They don't show up in > function traces - but they do show up in the output of > erlang:process_info(), where e.g. the gen_server state is not shown, > even though it would be dead useful. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From mryufeng@REDACTED Wed Nov 4 10:56:06 2009 From: mryufeng@REDACTED (Feng Yu) Date: Wed, 4 Nov 2009 17:56:06 +0800 Subject: [erlang-questions] Re: Unicast 40k messages, $1000 bounty (Hot Wheels!) In-Reply-To: References: Message-ID: <549b206a0911040156m5feec0ccw9b218a4d967e1081@mail.gmail.com> hi all, I achieve the goal with my patched otp R13B02-1. Server: #taskset -c 1 /usr/src/xxx/bin/erl -pa ebin mochiweb/ebin -boot start_sasl +A 8 +K true +P 1200000 -hotwheels cluster -hotwheels listen_port 8081 -name hotwheels@`hostname`s hotwheels start publish,info: [{memory,757453}, {owner,<0.84.0>}, {heir,none}, {name,subs}, {size,50000}, {node,'hotwheels@REDACTED'}, {named_table,false}, {type,set}, {keypos,1}, {protection,protected}] cost time: 871.379 Client: #taskset -c 0 /usr/src/xxx/bin/erl -boot start_sasl -pa ebin mochiweb/ebin +A 8 +K true +P 1200000 -hotwheels cluster -hotwheels listen_port 8081 -name debug (debug@REDACTED)8> bot:test(flashbot,50000). setup connection... setup done =INFO REPORT==== 4-Nov-2009::17:41:32 === setup: 5843.08ms, good: 50000, bad: 0, total run: 8593.22ms 0.0510ms | min 500.0000ms | 15943 - 31.89% 1000.0000ms | 16949 - 33.90% 1500.0000ms | 17108 - 34.22% 1349.0090ms | max ok My machine hardware: root@REDACTED:/proc/sys/net/ipv4# cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Pentium(R) Dual-Core CPU E5200 @ 2.50GHz stepping : 6 cpu MHz : 1200.000 cache size : 2048 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm lahf_lm bogomips : 4986.70 clflush size : 64 power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Pentium(R) Dual-Core CPU E5200 @ 2.50GHz stepping : 6 cpu MHz : 1200.000 cache size : 2048 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 apicid : 1 initial apicid : 1 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm lahf_lm bogomips : 4987.44 clflush size : 64 power management: Os: # uname -a Linux nd-desktop 2.6.31-14-generic #3 SMP Sun Nov 1 23:03:10 CST 2009 i686 GNU/Linux ?? ????????????????? http://blog.yufeng.info On Sat, Jul 18, 2009 at 2:14 PM, Joel Reymont wrote: > Folks, > > Janus has been renamed to Hot Wheels and now lives at > http://github.com/wagerlabs/hotwheels/tree/master. > > The goal is still to get a broadcast to 40k users in < 1s. > > I'm making several protocol changes, e.g. {packet, 2} > and moving away from JSON to a simple protocol where > every packet has a 12 byte timestamp to measure latency, > 1 byte for message type (1 = subscribe, 2 = unsubscribe), > and size + body for topic, event, etc. > > Thanks, Joel > > --- > Mac hacker with a performance bent > http://www.linkedin.com/in/joelreymont > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From litaocheng@REDACTED Wed Nov 4 11:20:12 2009 From: litaocheng@REDACTED (litao cheng) Date: Wed, 4 Nov 2009 18:20:12 +0800 Subject: [erlang-questions] Re: Unicast 40k messages, $1000 bounty (Hot Wheels!) In-Reply-To: <549b206a0911040156m5feec0ccw9b218a4d967e1081@mail.gmail.com> References: <549b206a0911040156m5feec0ccw9b218a4d967e1081@mail.gmail.com> Message-ID: wow, so cool, can you give me your OTP R13B02-1 patches ? thanks! 2009/11/4 Feng Yu > hi all, > I achieve the goal with my patched otp R13B02-1. > Server: > #taskset -c 1 /usr/src/xxx/bin/erl -pa ebin mochiweb/ebin -boot start_sasl > +A 8 +K true +P 1200000 -hotwheels cluster -hotwheels listen_port 8081 > -name hotwheels@`hostname`s hotwheels start > publish,info: [{memory,757453}, > {owner,<0.84.0>}, > {heir,none}, > {name,subs}, > {size,50000}, > {node,'hotwheels@REDACTED'}, > {named_table,false}, > {type,set}, > {keypos,1}, > {protection,protected}] > cost time: 871.379 > > Client: > #taskset -c 0 /usr/src/xxx/bin/erl -boot start_sasl -pa ebin mochiweb/ebin > +A 8 +K true +P 1200000 -hotwheels cluster -hotwheels listen_port 8081 > -name debug > (debug@REDACTED)8> bot:test(flashbot,50000). > setup connection... > setup done > > =INFO REPORT==== 4-Nov-2009::17:41:32 === > setup: 5843.08ms, good: 50000, bad: 0, total run: 8593.22ms > 0.0510ms | min > 500.0000ms | 15943 - 31.89% > 1000.0000ms | 16949 - 33.90% > 1500.0000ms | 17108 - 34.22% > 1349.0090ms | max > ok > > My machine hardware: > > root@REDACTED:/proc/sys/net/ipv4# cat /proc/cpuinfo > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 23 > model name : Pentium(R) Dual-Core CPU E5200 @ 2.50GHz > stepping : 6 > cpu MHz : 1200.000 > cache size : 2048 KB > physical id : 0 > siblings : 2 > core id : 0 > cpu cores : 2 > apicid : 0 > initial apicid : 0 > fdiv_bug : no > hlt_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 10 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm > constant_tsc > arch_perfmon pebs bts pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr > pdcm > lahf_lm > bogomips : 4986.70 > clflush size : 64 > power management: > > processor : 1 > vendor_id : GenuineIntel > cpu family : 6 > model : 23 > model name : Pentium(R) Dual-Core CPU E5200 @ 2.50GHz > stepping : 6 > cpu MHz : 1200.000 > cache size : 2048 KB > physical id : 0 > siblings : 2 > core id : 1 > cpu cores : 2 > apicid : 1 > initial apicid : 1 > fdiv_bug : no > hlt_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 10 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm > constant_tsc > arch_perfmon pebs bts pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr > pdcm > lahf_lm > bogomips : 4987.44 > clflush size : 64 > power management: > > Os: > # uname -a > Linux nd-desktop 2.6.31-14-generic #3 SMP Sun Nov 1 23:03:10 CST 2009 i686 > GNU/Linux > > ?? ????????????????? > http://blog.yufeng.info > > > > On Sat, Jul 18, 2009 at 2:14 PM, Joel Reymont wrote: > > > Folks, > > > > Janus has been renamed to Hot Wheels and now lives at > > http://github.com/wagerlabs/hotwheels/tree/master. > > > > The goal is still to get a broadcast to 40k users in < 1s. > > > > I'm making several protocol changes, e.g. {packet, 2} > > and moving away from JSON to a simple protocol where > > every packet has a 12 byte timestamp to measure latency, > > 1 byte for message type (1 = subscribe, 2 = unsubscribe), > > and size + body for topic, event, etc. > > > > Thanks, Joel > > > > --- > > Mac hacker with a performance bent > > http://www.linkedin.com/in/joelreymont > > > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > From torben.lehoff@REDACTED Wed Nov 4 12:49:01 2009 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Wed, 4 Nov 2009 12:49:01 +0100 Subject: [erlang-questions] erlang for programming a text editor In-Reply-To: References: Message-ID: Hi, Since "Look, Erlang is not just great for what it was designedfor, it's also quite adequate for other purposes." is part of the reason for wanting to inflict almost infinite pain on one-self - I mean, writing a text editor is hard work - I would like to point the attention to the original domain Erlang was supposed to address (nice form @ http://en.citizendium.org/wiki/Erlang%27s_original_domain): 1. Handling of a very large number of concurrent activities 2. Actions to be performed at a certain point in time or within a certain time 3. Systems distributed over several computers 4. Interaction with hardware 5. Very large software systems 6. Complex functionality such as feature interaction 7. Continuous operation for many years 8. Software maintenance (reconfiguration et cetera) without stopping the system 9. Stringent quality and reliability requirements 10. Fault tolerance both to hardware failures and software errors My argument, as that of many others smarter than me, is that whenever you have a problem domain that has needs that matches with the above list then Erlang might be the tool to use. Web servers, instant messaging and enterprise messaging are other areas that have similar requirements, and - surprise, surprise - quite a few Erlang solutions has turned up in these areas. Final remark: if you are going to invest your time and energy into something why not solve a new problem and try to make some money in the process? I love Erlang, Torben - High Priest of Erlang (self-proclaimed) On Wed, Nov 4, 2009 at 03:55, Michael Turner wrote: > "Good design never just happens." > > Great bumpersticker line! Copyright it immediately. > > The main useful purpose I can see for using Erlang to implement a > text-editor-cum-general-purpose-shell-programming-environment (Emacs > being exemplary) is to make a promotional statement about Erlang: your > success says, "Look, Erlang is not just great for what it was designed > for, it's also quite adequate for other purposes." > > Wings3D already does a pretty good job in that department, though. > Offhand, I can't think of a good reason to write a 3D modeler in Erlang > specifically. What Wings3D says, in effect, is that if you just want to > write something complex for fun and learning value, you shouldn't worry > that Erlang would hamper you significantly -- it might make some things > harder, but not prohibitively so. > > Wings3D is better than some hypothetical Emacs clone in another > (promotional) respect: your average *user* can relate to it better. > Emacs grew up in the glass tty days, it's got a face (and a raison > d'etre) that only a programmer could love. Most people these days > don't even know what you mean when you say "text editor", any more > than they know what you mean when you say "Erlang is Turing-complete." > > -michael turner > > > > On 11/3/2009, "Richard O'Keefe" wrote: > > > > >On Nov 2, 2009, at 8:20 PM, Alpar Juttner wrote: > >> * Several years ago, I had gave up using a Emacs-RMAIL, because > >> fetching new mail took a long time on our environment and this > >> process blocks the whole emacs. > > > >I use "Mail" version 3.6 on a 2.2GHz intel Mac running MacOS 10.5.7. > > > >I can assure you that l o n g pauses to fetch new mail are > >by no means confined to Emacs. Activity Monitor reports that Mail > >is using 12 ... no wait ... 11 threads, no wait, it's 12 again. > >So simply having and using concurrency is no guarantee that you won't > >get long waits. > > > >Our sysadmins made a unilateral decision to move everyone's files off > >their local machines onto a file server. There are now frequent > >jarring waits while Mail waits for a file server in order to touch > >some file I don't particularly want it to touch at that time. > > > >The fundamental issue is *system design*. It's thinking about what > >kinds > >of delays there might be and arranging for the program to let you do > >something else (such as compose a message) while that's happening. For > >this, threads are a great help, but not a necessity. (Well, you _could_ > >write the whole thing in assembly code (:-) (:-).) > >> > > > >> An inherently concurrent design would immediately eliminate all of > >> these > >> issues. > > > >Mail is evidence that an inherently concurrent design *as such* need not > >eliminate *any* of these issues. Concurrency makes it *easier* to > >create > >good designs, but good design never just happens. > > > > > >________________________________________________________________ > >erlang-questions mailing list. See http://www.erlang.org/faq.html > >erlang-questions (at) erlang.org > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- http://www.linkedin.com/in/torbenhoffmann From kenji.rikitake@REDACTED Wed Nov 4 13:41:46 2009 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Wed, 4 Nov 2009 21:41:46 +0900 Subject: Erlang logo licensing and usage examples Message-ID: <20091104124146.GA27799@k2r.org> I'd be glad if anyone tells me what is the licensing policies and procedures for the following logo pictures of Erlang: http://www.erlang.se/images/Erlanglogo.gif (well-known "e" sign) http://www.erlang.org/images/erlang-logo.png (wow, shaded :)) I'm asking this because it'll be nice if I could wear a shirt with the Erlang logo. Regards, Kenji Rikitake PS: my apology for asking non-technical question in erlang-questions. From steven.charles.davis@REDACTED Wed Nov 4 15:30:18 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 4 Nov 2009 06:30:18 -0800 (PST) Subject: Erlang logo licensing and usage examples In-Reply-To: <20091104124146.GA27799@k2r.org> References: <20091104124146.GA27799@k2r.org> Message-ID: <54fe998c-0df4-4d8f-ac42-a3217181e566@15g2000yqy.googlegroups.com> I'd buy one. It would be nice to have a banner quote from Joe on it e.g. "The world is parallel" (from the frontispiece of his book). In case the OTP team wanted to offer this, it's simple to do... http://www.cafepress.com/cp/customize/ /s On Nov 4, 6:41?am, Kenji Rikitake wrote: > I'd be glad if anyone tells me what is the licensing policies and > procedures for the following logo pictures of Erlang: > > http://www.erlang.se/images/Erlanglogo.gif > (well-known "e" sign) > > http://www.erlang.org/images/erlang-logo.png > (wow, shaded :)) > > I'm asking this because it'll be nice if I could wear a shirt with > the Erlang logo. > > Regards, > Kenji Rikitake > PS: my apology for asking non-technical question in erlang-questions. > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From bryantln@REDACTED Wed Nov 4 15:47:19 2009 From: bryantln@REDACTED (Lincoln Bryant) Date: Wed, 4 Nov 2009 09:47:19 -0500 Subject: [erlang-questions] Re: Erlang logo licensing and usage examples In-Reply-To: <54fe998c-0df4-4d8f-ac42-a3217181e566@15g2000yqy.googlegroups.com> References: <20091104124146.GA27799@k2r.org> <54fe998c-0df4-4d8f-ac42-a3217181e566@15g2000yqy.googlegroups.com> Message-ID: I'd like stickers, myself. Think about it: your server, your fridge, and your cat can all be powered by Erlang. Who wouldn't want that? On Wed, Nov 4, 2009 at 9:30 AM, Steve Davis wrote: > I'd buy one. > > It would be nice to have a banner quote from Joe on it e.g. "The world > is parallel" (from the frontispiece of his book). > > In case the OTP team wanted to offer this, it's simple to do... > http://www.cafepress.com/cp/customize/ > > /s > > On Nov 4, 6:41 am, Kenji Rikitake wrote: > > I'd be glad if anyone tells me what is the licensing policies and > > procedures for the following logo pictures of Erlang: > > > > http://www.erlang.se/images/Erlanglogo.gif > > (well-known "e" sign) > > > > http://www.erlang.org/images/erlang-logo.png > > (wow, shaded :)) > > > > I'm asking this because it'll be nice if I could wear a shirt with > > the Erlang logo. > > > > Regards, > > Kenji Rikitake > > PS: my apology for asking non-technical question in erlang-questions. > > > > ________________________________________________________________ > > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- Lincoln Bryant From jarrod@REDACTED Wed Nov 4 17:41:42 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Wed, 4 Nov 2009 11:41:42 -0500 Subject: Xapian linked-in/port driver for Erlang? Message-ID: Has anyone worked on a linked-in/port driver for Xapian ( full text indexer written in C++ ). Is anyone interested in working on such a thing? I really think Erlang would be a perfect network front end for Xapian. I have a Python/Twisted server that front ends Xapian, but I want it to scale better and Erlang will do that well. From rtrlists@REDACTED Wed Nov 4 18:45:53 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 4 Nov 2009 17:45:53 +0000 Subject: [erlang-questions] Xapian linked-in/port driver for Erlang? In-Reply-To: References: Message-ID: <6a3ae47e0911040945o2aec9224yc9505ae57928193b@mail.gmail.com> On Wed, Nov 4, 2009 at 4:41 PM, Jarrod Roberson wrote: > Has anyone worked on a linked-in/port driver for Xapian ( full text indexer > written in C++ ). > Is anyone interested in working on such a thing? I really think Erlang > would > be a perfect network front end for Xapian. > I have a Python/Twisted server that front ends Xapian, but I want it to > scale better and Erlang will do that well. > Ooooh, yes, I've been toying with that idea for some time now, but never had the time. I don't have much spare time, but would be happy to pitch in if I do. I'm not a fan of linked in drivers, I much prefer port programs. When a port program crashes, there's soo much less damage. Robby From junhufr@REDACTED Wed Nov 4 19:42:39 2009 From: junhufr@REDACTED (Jun HU) Date: Wed, 4 Nov 2009 14:42:39 -0400 Subject: Is erlang suitable to handle Constraint satisfaction problems ? Message-ID: Hello all, I'm a PhD student in computer science, currently i'm working constraint satisfaction problem, notable in frequency assignment problem. I'm just wondering that erlang is suitable for this kind of problems or not. Thanks in advance. From gleber.p@REDACTED Wed Nov 4 20:32:01 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 4 Nov 2009 20:32:01 +0100 Subject: What happened to jserl project? Message-ID: <14f0e3620911041132o228a104cr22afd0564bec0d4e@mail.gmail.com> Hello What happened to jserl project? It's homepage [1] on google code is not accessible any more. 1: http://code.google.com/p/jserl/ Best, Gleb Peregud Green Elephant Labs From nick@REDACTED Wed Nov 4 20:34:08 2009 From: nick@REDACTED (Nick Gerakines) Date: Wed, 4 Nov 2009 11:34:08 -0800 Subject: [erlang-questions] Xapian linked-in/port driver for Erlang? In-Reply-To: <6a3ae47e0911040945o2aec9224yc9505ae57928193b@mail.gmail.com> References: <6a3ae47e0911040945o2aec9224yc9505ae57928193b@mail.gmail.com> Message-ID: I was using Xapian with iplaywow for a while and it worked well. I've got some Erlang that builds the data files that the import tool uses. To pull results, I simply executed the command line query tool. Kevin Smith wrote an Erlang client library for Sphinx that's worth checking out. http://github.com/kevsmith/giza/ # Nick Gerakines On Wed, Nov 4, 2009 at 9:45 AM, Robert Raschke wrote: > On Wed, Nov 4, 2009 at 4:41 PM, Jarrod Roberson wrote: > >> Has anyone worked on a linked-in/port driver for Xapian ( full text indexer >> written in C++ ). >> Is anyone interested in working on such a thing? I really think Erlang >> would >> be a perfect network front end for Xapian. >> I have a Python/Twisted server that front ends Xapian, but I want it to >> scale better and Erlang will do that well. >> > > Ooooh, yes, I've been toying with that idea for some time now, but never had > the time. > > I don't have much spare time, but would be happy to pitch in if I do. I'm > not a fan of linked in drivers, I much prefer port programs. When a port > program crashes, there's soo much less damage. > > Robby > From torben.lehoff@REDACTED Wed Nov 4 20:38:56 2009 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Wed, 4 Nov 2009 20:38:56 +0100 Subject: [erlang-questions] Is erlang suitable to handle Constraint satisfaction problems ? In-Reply-To: References: Message-ID: Hi Jun, I don't think constraint satisfaction is one of Erlang's strengths. The syntax has roots in Prolog, but that is just about it - you won't get any of Prolog's wonderful inference algorithms. You should look at Mozart/Oz, Prolog or for constraint satisfaction. Unless the frequency assignment problem is something completely different than what I read from your description! Cheers, Torben On Wed, Nov 4, 2009 at 19:42, Jun HU wrote: > Hello all, > > I'm a PhD student in computer science, currently i'm working > constraint satisfaction problem, notable in frequency assignment > problem. I'm just wondering that erlang is suitable for this kind of > problems or not. > > > > Thanks in advance. > -- http://www.linkedin.com/in/torbenhoffmann From rvirding@REDACTED Wed Nov 4 21:06:56 2009 From: rvirding@REDACTED (Robert Virding) Date: Wed, 4 Nov 2009 21:06:56 +0100 Subject: [erlang-questions] Is erlang suitable to handle Constraint satisfaction problems ? In-Reply-To: References: Message-ID: <3dbc6d1c0911041206u55d32191sa32db86ec37a5b7b@mail.gmail.com> As mentioned there is no built-in support for constraint satisfaction in Erlang. There is, however, a Prolog interpreter implemented in erlang, Erlog, which could be useful if you need other Erlang features as well. There is no constraint satisfaction in erlog now but I am itching for a reason to add it. It seems like it could be a fun project. http://github.com/rvirding/erlog Robert 2009/11/4 Torben Hoffmann > Hi Jun, > > I don't think constraint satisfaction is one of Erlang's strengths. > > The syntax has roots in Prolog, but that is just about it - you won't get > any of Prolog's wonderful inference algorithms. > > You should look at Mozart/Oz, Prolog or for > constraint satisfaction. > > Unless the frequency assignment problem is something completely different > than what I read from your description! > > Cheers, > Torben > > On Wed, Nov 4, 2009 at 19:42, Jun HU wrote: > > > Hello all, > > > > I'm a PhD student in computer science, currently i'm working > > constraint satisfaction problem, notable in frequency assignment > > problem. I'm just wondering that erlang is suitable for this kind of > > problems or not. > > > > > > > > Thanks in advance. > > > > > > -- > http://www.linkedin.com/in/torbenhoffmann > From ok@REDACTED Thu Nov 5 00:46:14 2009 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 5 Nov 2009 12:46:14 +1300 Subject: [erlang-questions] Is erlang suitable to handle Constraint satisfaction problems ? In-Reply-To: <3dbc6d1c0911041206u55d32191sa32db86ec37a5b7b@mail.gmail.com> References: <3dbc6d1c0911041206u55d32191sa32db86ec37a5b7b@mail.gmail.com> Message-ID: <08D1614A-520D-4A0C-AA26-22F0DF9D0431@cs.otago.ac.nz> Actually, there was a (masters?) thesis on constraints in Erlang years and years ago. Thank you Google: An Extension of Erlang with Finite-Domain Constraints www.erlang.se/publications/xjobb/finite-domain-erlang.ps.gz Dan Sahlin wrote: An experiment has also been carried out adding finite domain constraints to Erlang. The work showed that although constraint programming does not fit so well into the functional programming framework, it was quite feasible to do. As telecom applications seem to lack the need for constraint solving, the system will not be included in current releases of the [Erlang/OTP] system. I suspect that "lack the need for" may really mean "are not yet written by people aware of the utility of". See http://www.starlikedesign.com/ for one possible counter-example. So I hope that the code hasn't been lost or bitrotted too much to be salvaged. As others have noted, modern Prolog systems like SICStus, Eclipse, and SWI often support contraint programming. Much depends on what _kind_ of constraints you have to deal with and the scale of the problem. >> From leap@REDACTED Thu Nov 5 04:47:24 2009 From: leap@REDACTED (Michael Turner) Date: Thu, 05 Nov 2009 03:47:24 +0000 Subject: [erlang-questions] Is erlang suitable to handle Constraint satisfaction problems ? In-Reply-To: <08D1614A-520D-4A0C-AA26-22F0DF9D0431@cs.otago.ac.nz> Message-ID: On 11/4/2009, "Richard O'Keefe" wrote: >As others have noted, modern Prolog systems like SICStus, >Eclipse, and SWI often support contraint programming. I pointed out ECLiPSe as being probably more appropriate for the needs of the original inquirer, especially since it's Prolog-for-constraint-programming-until-the-cows-come-home. The inevitable afterthought arrived: sometimes constraint programming has highly parallelizable stages, and for very large problems, good concurrency support could be a big help. So I went looking for how ECLiPSe might manage concurrency and found this, at http://www.eclipse-clp.org/: "ECLiPSe programs can execute in or-parallel on shared-memory multiprocessor hardware (this functionality is currently not actively maintained because of other priorities)." If the maintainer can't tell you that his app is broken in a certain respect, it means it's almost certainly broken in that respect. To qualify my first recommendation: ECLiPSe is probably better IF you don't need parallelized solvers. Maybe there's something worth doing with Erlang in this application space for big, highly parallelizable constraint programming. ECLiPSe-over-Erlang would be so ridiculously powerful that somebody might write Skynet in it by accident. If only I weren't so wrapped up right now in this project of implementing TECO in Erlang ... -michael turner P.S. The last two sentences of the above were meant in jest, just in case anybody found them confusing. Especially the one referring to Skynet; everybody knows that Skynet would have to be written in Emacs Lisp. On 11/4/2009, "Richard O'Keefe" wrote: >Actually, there was a (masters?) thesis on constraints in Erlang >years and years ago. Thank you Google: >An Extension of Erlang with Finite-Domain Constraints >www.erlang.se/publications/xjobb/finite-domain-erlang.ps.gz > >Dan Sahlin wrote: > An experiment has also been carried out adding > finite domain constraints to Erlang. The work showed > that although constraint programming does not fit so > well into the functional programming framework, it > was quite feasible to do. As telecom applications > seem to lack the need for constraint solving, the > system will not be included in current releases of > the [Erlang/OTP] system. > >I suspect that "lack the need for" may really mean "are not >yet written by people aware of the utility of". See >http://www.starlikedesign.com/ >for one possible counter-example. So I hope that the code >hasn't been lost or bitrotted too much to be salvaged. > >As others have noted, modern Prolog systems like SICStus, >Eclipse, and SWI often support contraint programming. >Much depends on what _kind_ of constraints you have to deal with >and the scale of the problem. >>> > > >________________________________________________________________ >erlang-questions mailing list. See http://www.erlang.org/faq.html >erlang-questions (at) erlang.org > > From hakan@REDACTED Thu Nov 5 15:11:48 2009 From: hakan@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Thu, 5 Nov 2009 15:11:48 +0100 Subject: [erlang-questions] weird mneisa/escript problem in r13 In-Reply-To: <4AF0FA16.1000709@research.att.com> References: <4AEFB57C.6020306@research.att.com> <615795.67725.qm@web112611.mail.gq1.yahoo.com> <4AF02A9A.9090107@research.att.com> <224598.32231.qm@web112619.mail.gq1.yahoo.com> <4AF0FA16.1000709@research.att.com> Message-ID: <922d05850911050611g3ece957cw5728f44ee4436720@mail.gmail.com> On Wed, Nov 4, 2009 at 4:50 AM, Garry Hodgson wrote: > Yogish Baliga wrote: >> >> Looks like there is some shell interpretation going on. >> >> What is the output of application:get_all_env(mnesia)? >> >> Also can you try without quotes? > > i've tried it both ways, as mentioned in the original post. the problem is > not getting this to work. i can do that easily. the problem is that it behaves > so differently in two very similar versions r13b01 and r13b02. i see nothing > in the release notes to inidicate such a change, so i thought i'd mention it, > in case it was a bug. It was a bug, but it isn't anymore. In earlier releases, the arguments on the escript line beginning with %%! was passed to the virtual machine via an environment variable. As a side effect of this the arguments was mangled a bit, causing some of the quotation marks to disappear. This was fixed in Erts 5.7.3 (OTP-1060). The arguments are now passed to the VM as they are written in the escript, without any mangling. Unfortunately this bug fix was not documented. I hope that has not caused you too much trouble. /H?kan --- H?kan Mattsson (uabhams) Erlang/OTP, Ericsson AB From garry@REDACTED Thu Nov 5 15:26:22 2009 From: garry@REDACTED (Garry Hodgson) Date: Thu, 05 Nov 2009 09:26:22 -0500 Subject: [erlang-questions] weird mneisa/escript problem in r13 In-Reply-To: <922d05850911050611g3ece957cw5728f44ee4436720@mail.gmail.com> References: <4AEFB57C.6020306@research.att.com> <615795.67725.qm@web112611.mail.gq1.yahoo.com> <4AF02A9A.9090107@research.att.com> <224598.32231.qm@web112619.mail.gq1.yahoo.com> <4AF0FA16.1000709@research.att.com> <922d05850911050611g3ece957cw5728f44ee4436720@mail.gmail.com> Message-ID: <4AF2E08E.8020504@research.att.com> H?kan Mattsson wrote: > It was a bug, but it isn't anymore. cool. > In earlier releases, the arguments on the escript line beginning with > %%! was passed to the virtual machine via an environment variable. As > a side effect of this the arguments was mangled a bit, causing some of > the quotation marks to disappear. This was fixed in Erts 5.7.3 > (OTP-1060). The arguments are now passed to the VM as they are written > in the escript, without any mangling. Unfortunately this bug fix was > not documented. I hope that has not caused you too much trouble. it wasn't a big deal. just wanted to mention it in case it was unknown. thanks for the info. -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From roberto.aloi@REDACTED Thu Nov 5 16:59:44 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Thu, 05 Nov 2009 15:59:44 +0000 Subject: Erlang Certificatino Survey Message-ID: <4AF2F670.6040609@erlang-consulting.com> Hi all, please forgive me about using this mailing list for a non-technical issue, but I'd really appreciate your feedback on the following. --- Erlang Training and Consulting Ltd in conjunction with the University of Kent intends to launch a pilot version of an e-certification platform for Erlang in the near future. To assist in this process, we have launched a survey on Erlang Certification. The objective of the study is to better understand the expectations of users, in order to ensure a well-focused product. We would appreciate a few minutes of your time in completing this survey. To take the survey, please go to: http://surveys.erlang-consulting.com/index.php?sid=21265&lang=en The results of the survey will be shared with third parties, in publications and as part of research into Certification. When doing so, all results will be completely anonymised and any individual-identifying information removed. Thank you for your time and cooperation. --- Roberto Aloi Erlang Training and Consulting Ltd. http://www.erlang-consulting.com University of Kent http://www.kent.ac.uk From clist@REDACTED Thu Nov 5 19:53:32 2009 From: clist@REDACTED (Angel Alvarez) Date: Thu, 5 Nov 2009 19:53:32 +0100 Subject: Elang discussions? - a call for a new format In-Reply-To: <20040114112027.GA9080@cgey.com> References: <20040114112027.GA9080@cgey.com> Message-ID: <200911051953.32627.clist@uah.es> El Mi?rcoles, 14 de Enero de 2004 Mickael Remond escribi?: > * Joe Armstrong [2004-01-14 11:58:55 +0100]: > > > Here are some questions > > > > 1) Is this a good idea? > > I do not know. Splitting the discussion list might make it difficult to > follow the discussions around erlang. > > > 2) Who can host this? > > 3) What is the best S/W for the list > > > > Does anybody have any first hand experience here - what is the > > best S/W to run???? > > We have some experience with running Sympa for several projects on our > server (Erlang-projects). It is an handy, robust tool with a web admin > interface (User friendly). > We can host such a list on Erlang-projects. > > > An alternative is a "write your own in Yaws" > > This could be an idea. Another solution that I thought about was mixing > forum things with an NNTP back-end. Old school users can use a news > reader to follow Erlang discussion. They could be publish in a forum > like interface on the web, where authenticated user could contribute. > I am think about writing that with Yaws (Web client for NNTP). > Seems that you guys did not split the erlang list after all. So i managed to download the complete archive 1999-2009 for learning porpouses!!! ...dont program defensively!!! .. let other proccess fix the error... ...threads use as much you can... ...Chitty chitty bang bang operators... ...Good desing nevers happens... ???and many other lnterestings subjects!!! I miss some links long gone now, but overall impresion is great!!! So much good computer science and effort deserve a bit of gratitude. Thanks to all! Well i have to come back to the future, All you have to do is drive my mouse directly toward that screen accelerating to 88 miles an hour.... /Angel -- No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. ->>----------------------------------------------- Clist UAH a.k.a Angel ---------------------------------[www.uah.es]-<<-- PUMPKIN - Everybody be cool this is a robbery! From idiay_tuanis@REDACTED Thu Nov 5 21:13:30 2009 From: idiay_tuanis@REDACTED (Jose Castro) Date: Thu, 5 Nov 2009 20:13:30 +0000 (GMT) Subject: Stand alone erlang deployment Message-ID: <59753.80288.qm@web25805.mail.ukl.yahoo.com> I'm sure this has come up before but here goes: Is there a "simple" way to deploy a program that is written in erlang in a system without doing an install that changes the environment or system files? the context of the question is this: I work in accademia and do some consulting, a large transnational in Costa Rica (where I live) requires a system that manages processes that supervise each other over a network of machines, my obvious recommendation was erlang. The company has an identical configuration policy for factory machines, they are managed remotely, so it's desired not to change the environment, registry or any system files and directories, ideally the install should copy a file/directory in the machine, create an icon for clicking, and be up and running WITHOUT install. Stand Alone Erlang is old, and reltool creates a minimal install, but an install just the same. Am I correct in this? I know there has to be a way, but for the company it's desireable for it to be simple. As a side note, the lack of a simple stand alone creating tool is a hurdle for Erlang adoption in certain companies (like this one) with strict configuration policies (it makes it hard to sneak in an erlang solution without management knowing about it :), here they are most problably going to default for the tool they have in the stardard configuration (perl). Jose From idiay_tuanis@REDACTED Thu Nov 5 21:48:47 2009 From: idiay_tuanis@REDACTED (Jose Castro) Date: Thu, 5 Nov 2009 20:48:47 +0000 (GMT) Subject: Stand alone erlang deploy Message-ID: <540410.44136.qm@web25808.mail.ukl.yahoo.com> This has probably come up before, but here goes: is there a simple way to deploy an erlang program/application without having to install erlang on the machine, that is: without modifying the environment or configuration and sistem files/directories. The context of the question is this: A large transnational in Costa Rica (where I live) requires a system that creates processes, monitors activitye and supervises the processes on a pool of factory machines, my obvious recommendation was erlang. The company has a idental configuration policy for their machines, so system files, directories, registry, and environment variables should not be impacted (they could, but require higher management aproval, and the impact of the system does not justify changing the configuration of all the machines). Idealy we should be able to drop a folder in the machine, create an Icon that when clicked the application runs, thats it. I know stand alone erlang but does not work with the current erlang release, and looking at reltool it appears that it does a minimal install, but an install just the same. Is there a "simple" way to do this? am I missing something? On a side note, the lack of a good simple "one file executable" deployment tool creates a hurdle for the acceptance of Erlang in institutions with a strict machine configuration policy, since it makes it mandatory for management to know and approve the approach, and harder to sneak a grass roots erlang solution that surprises management. From hawk.mattsson@REDACTED Fri Nov 6 11:59:07 2009 From: hawk.mattsson@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Fri, 6 Nov 2009 11:59:07 +0100 Subject: [erlang-questions] Stand alone erlang deployment In-Reply-To: <59753.80288.qm@web25805.mail.ukl.yahoo.com> References: <59753.80288.qm@web25805.mail.ukl.yahoo.com> Message-ID: <922d05850911060259l6c04b370t7962f20bfda106b1@mail.gmail.com> On Thu, Nov 5, 2009 at 9:13 PM, Jose Castro wrote: > ideally the install should copy a file/directory in the machine, > create an icon for clicking, and be up and running WITHOUT install. Besides the icon stuff, you can do this with Reltool. > Stand Alone Erlang is old, and reltool creates a minimal install, but > an install just the same. Am I correct in this? I know there has to be > a way, but for the company it's desireable for it to be simple. When you generate a target system with Reltool it will by default not need any installation script to be run. You can just copy the files to the new location and start the VM. If someone by some obscure reason wants to have the old behavior (with a mandatory installation script) it can still be achieved by setting the Reltool parameter relocatable to false when the target system is generated. /H?kan --- H?kan Mattsson (uabhams) Erlang/OTP, Ericsson AB From jlprasantha@REDACTED Fri Nov 6 14:37:39 2009 From: jlprasantha@REDACTED (prasantha kumara) Date: Fri, 6 Nov 2009 19:07:39 +0530 Subject: shut down the smpp smsc Message-ID: i have used the open smpp for smsc gateway development.so i want to shutdown the gateway.in that case i used stop() -> gen_smsc:call(?SERVER, die, 10000), method to do it. but here it sends an unbind request fromthe gateway server to the application. please can anybody let me know how to solve this matter.(because the gateway server should not sends unbind request tothe application) thanks & regards From nick@REDACTED Fri Nov 6 16:42:05 2009 From: nick@REDACTED (Niclas Eklund) Date: Fri, 6 Nov 2009 16:42:05 +0100 (CET) Subject: City-walk Message-ID: Hello! The evening before the EUC (Wednesday), we invite participants from abroad to a city-walk through central Stockholm (showing the castle, the old town etc) and a beer/meal together at a pub. If you care to join us, please send an e-mail to Henry and me. When and where? We'll meet up at the central station at 18:00, about 15-20 meters from the main entrance (ground level) there is a "large hole" in the floor (about 8 meters in diameter), with an iron fence around it. Best regards, Niclas Eklund @ Erlang/OTP From tony@REDACTED Fri Nov 6 15:47:39 2009 From: tony@REDACTED (Tony Rogvall) Date: Fri, 6 Nov 2009 15:47:39 +0100 Subject: type attribute pp bug Message-ID: <2CC4AA7A-6327-4E76-9B53-45C15BD2B220@rogvall.se> Hi! Found a "small" bug in erl_pp handling of the type declaration. The "code" compiles, but with -E flag will result in a crash. -module(type_bug). -type foo() :: integer(). > erlc -E type_bug.erl ./type_bug.erl:none: internal error in listing; crash reason: {function_clause, [{erl_pp,typeattr, [type,[{foo,{type,4,integer,[]},[]}],none]}, {erl_pp,lattribute,2}, {erl_pp,form,2}, {compile,'-src_listing/2-anonymous-0-',2}, {lists,foreach,2}, {compile,listing,3}, {compile,'-internal_comp/4-anonymous-1-',2}, {compile,fold_comp,3}]} ./type_bug.erl:4: Warning: type foo() is unused /Tony From anton.krasovsky@REDACTED Fri Nov 6 16:57:24 2009 From: anton.krasovsky@REDACTED (Anton Krasovsky) Date: Fri, 6 Nov 2009 15:57:24 +0000 Subject: Exceptions and gen_server Message-ID: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> What would be the standard way of returning/throwing an exception in a result to a gen_server:call? Would something like that be reasonable: a() -> case gen_server:call(..., a) of {ok, Value} -> Value; {error, Error} -> throw({error, Error}) end and handle_call(a, ...) -> case ... of {ok, Value} -> {reply, {ok, Value} , State}; {error, Error} -> {reply, {error, Error} , State} end Regards, Anton From roberto.aloi@REDACTED Fri Nov 6 17:42:13 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Fri, 06 Nov 2009 16:42:13 +0000 Subject: [erlang-questions] Exceptions and gen_server In-Reply-To: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> References: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> Message-ID: <4AF451E5.2090200@erlang-consulting.com> Hi Anton, I usually do the following: foo(Params) -> case gen_server:call(?MODULE, {foo, Params}) of {error, Reason} -> erlang:error({Reason, Params}); % I return params to understand what's going on... {ok, Value} -> Value end. handle_call({create, Name, Opts}, _From, State) -> ... Regards, Roberto Aloi Erlang Training and Consulting Ltd. http://www.erlang-consulting.com Anton Krasovsky wrote: > What would be the standard way of returning/throwing an exception in a > result to a gen_server:call? > > Would something like that be reasonable: > > a() -> > case gen_server:call(..., a) of > {ok, Value} -> Value; > {error, Error} -> throw({error, Error}) > end > > and > > handle_call(a, ...) -> > case ... of > {ok, Value} -> {reply, {ok, Value} , State}; > {error, Error} -> {reply, {error, Error} , State} > end > > Regards, > Anton > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From dmercer@REDACTED Fri Nov 6 18:08:35 2009 From: dmercer@REDACTED (David Mercer) Date: Fri, 6 Nov 2009 11:08:35 -0600 Subject: [erlang-questions] Exceptions and gen_server In-Reply-To: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> References: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> Message-ID: Is there any particular reason you want to throw the value rather than just erroring out? If you need to throw it to a catch, I'd probably do as you say, but otherwise, I'd just do a match and let an error happen when it doesn't match. Something like: a() -> {ok, Value} = gen_server:call(..., a), Value. Your handle_call looks OK to me. Cheers, David > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Anton Krasovsky > Sent: Friday, November 06, 2009 9:57 AM > To: erlang-questions@REDACTED > Subject: [erlang-questions] Exceptions and gen_server > > What would be the standard way of returning/throwing an exception in a > result to a gen_server:call? > > Would something like that be reasonable: > > a() -> > case gen_server:call(..., a) of > {ok, Value} -> Value; > {error, Error} -> throw({error, Error}) > end > > and > > handle_call(a, ...) -> > case ... of > {ok, Value} -> {reply, {ok, Value} , State}; > {error, Error} -> {reply, {error, Error} , State} > end > > Regards, > Anton > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org From christophe.romain@REDACTED Fri Nov 6 19:03:40 2009 From: christophe.romain@REDACTED (Christophe Romain) Date: Fri, 6 Nov 2009 19:03:40 +0100 Subject: [erlang-questions] Stand alone erlang deployment In-Reply-To: <59753.80288.qm@web25805.mail.ukl.yahoo.com> References: <59753.80288.qm@web25805.mail.ukl.yahoo.com> Message-ID: <20091106180340.GM3338@localhost> you can check CEAN, it does what you need. it creates a directory where you can launch erlang, without installing into the system, the directory can be stored on a remote filesystem as well. http://cean.process-one.net/ From hans.bolinder@REDACTED Fri Nov 6 21:51:52 2009 From: hans.bolinder@REDACTED (Hans Bolinder) Date: Fri, 6 Nov 2009 21:51:52 +0100 Subject: [erlang-questions] type attribute pp bug In-Reply-To: <2CC4AA7A-6327-4E76-9B53-45C15BD2B220@rogvall.se> References: <2CC4AA7A-6327-4E76-9B53-45C15BD2B220@rogvall.se> Message-ID: <19188.35944.768487.273894@ornendil.du.uab.ericsson.se> [Tony Rogvall:] > Found a "small" bug in erl_pp handling of the type declaration. > The "code" compiles, but with -E flag will result in a crash. Thanks! The bug has been corrected; snapshots downloaded after 10th October should be OK. The modules erl_expand_records, epp.erl, and compile.erl have been modified. Mail me if you need a patch. Best regards, Hans Bolinder, Erlang/OTP team, Ericsson From fjanon@REDACTED Sat Nov 7 03:08:51 2009 From: fjanon@REDACTED (Fred Janon) Date: Sat, 7 Nov 2009 10:08:51 +0800 Subject: Web servers and unicode support? Message-ID: <2fd0c7810911061808l2c1fc897ufa0f462237ab903@mail.gmail.com> Hi, I want to design a web application with support for European, Asian languages as well as English, meaning Unicode. I am very interested in doing that in Erlang with Mnesia or MySQL or Postgres (when I figure out if I can use Mnesia), but what would I need to use (web server, extensions) to support these languages/Unicode? I saw a few posting about not so great String support in Erlang, therefore even less for Unicode. What's the status? Thanks Fred From rumata-estor@REDACTED Sat Nov 7 03:19:38 2009 From: rumata-estor@REDACTED (Rumata) Date: Sat, 07 Nov 2009 05:19:38 +0300 Subject: [erlang-questions] Web servers and unicode support? In-Reply-To: <2fd0c7810911061808l2c1fc897ufa0f462237ab903@mail.gmail.com> References: <2fd0c7810911061808l2c1fc897ufa0f462237ab903@mail.gmail.com> Message-ID: <1257560378.5339.6.camel@rumbuntu> If you don't need any string manipulation (I didn't need for my site) you can treat all strings as binaries. Besides, there are asn1rt:utf8_list_to_binary/1 and asn1rt:utf8_list_to_binary/1 functions. On Sat, 2009-11-07 at 10:08 +0800, Fred Janon wrote: > Hi, > > I want to design a web application with support for European, Asian > languages as well as English, meaning Unicode. I am very interested in doing > that in Erlang with Mnesia or MySQL or Postgres (when I figure out if I can > use Mnesia), but what would I need to use (web server, extensions) to > support these languages/Unicode? I saw a few posting about not so great > String support in Erlang, therefore even less for Unicode. What's the > status? > > Thanks > > Fred From kevin@REDACTED Sat Nov 7 05:13:15 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Fri, 6 Nov 2009 23:13:15 -0500 Subject: [erlang-questions] Web servers and unicode support? In-Reply-To: <1257560378.5339.6.camel@rumbuntu> References: <2fd0c7810911061808l2c1fc897ufa0f462237ab903@mail.gmail.com> <1257560378.5339.6.camel@rumbuntu> Message-ID: There's also the unicode module in later releases (R13B-0 and later, I think). --Kevin On Nov 6, 2009, at 9:19 PM, Rumata wrote: > If you don't need any string manipulation (I didn't need for my site) > you can treat all strings as binaries. > > Besides, there are asn1rt:utf8_list_to_binary/1 and > asn1rt:utf8_list_to_binary/1 functions. > > On Sat, 2009-11-07 at 10:08 +0800, Fred Janon wrote: >> Hi, >> >> I want to design a web application with support for European, Asian >> languages as well as English, meaning Unicode. I am very interested >> in doing >> that in Erlang with Mnesia or MySQL or Postgres (when I figure out >> if I can >> use Mnesia), but what would I need to use (web server, extensions) to >> support these languages/Unicode? I saw a few posting about not so >> great >> String support in Erlang, therefore even less for Unicode. What's the >> status? >> >> Thanks >> >> Fred > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From rapsey@REDACTED Sat Nov 7 08:39:22 2009 From: rapsey@REDACTED (Rapsey) Date: Sat, 7 Nov 2009 08:39:22 +0100 Subject: [erlang-questions] Web servers and unicode support? In-Reply-To: References: <2fd0c7810911061808l2c1fc897ufa0f462237ab903@mail.gmail.com> <1257560378.5339.6.camel@rumbuntu> Message-ID: <97619b170911062339mae67b94s59ee1d230d8a0e80@mail.gmail.com> I'm writing a website in yaws/erlydtl/erlmongo right now and it works with UTF-8 just fine. Sergej On Sat, Nov 7, 2009 at 5:13 AM, Kevin A. Smith wrote: > There's also the unicode module in later releases (R13B-0 and later, I > think). > > --Kevin > > On Nov 6, 2009, at 9:19 PM, Rumata wrote: > > If you don't need any string manipulation (I didn't need for my site) >> you can treat all strings as binaries. >> >> Besides, there are asn1rt:utf8_list_to_binary/1 and >> asn1rt:utf8_list_to_binary/1 functions. >> >> On Sat, 2009-11-07 at 10:08 +0800, Fred Janon wrote: >> >>> Hi, >>> >>> I want to design a web application with support for European, Asian >>> languages as well as English, meaning Unicode. I am very interested in >>> doing >>> that in Erlang with Mnesia or MySQL or Postgres (when I figure out if I >>> can >>> use Mnesia), but what would I need to use (web server, extensions) to >>> support these languages/Unicode? I saw a few posting about not so great >>> String support in Erlang, therefore even less for Unicode. What's the >>> status? >>> >>> Thanks >>> >>> Fred >>> >> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From bryan.fink@REDACTED Sat Nov 7 14:00:55 2009 From: bryan.fink@REDACTED (Bryan Fink) Date: Sat, 7 Nov 2009 08:00:55 -0500 Subject: [erlang-questions] Web servers and unicode support? In-Reply-To: <2fd0c7810911061808l2c1fc897ufa0f462237ab903@mail.gmail.com> References: <2fd0c7810911061808l2c1fc897ufa0f462237ab903@mail.gmail.com> Message-ID: On Fri, Nov 6, 2009 at 9:08 PM, Fred Janon wrote: > I want to design a web application with support for European, ?Asian > languages as well as English, meaning Unicode. I am very interested in doing > that in Erlang with Mnesia or MySQL or Postgres (when I figure out if I can > use Mnesia), but what would I need to use (web server, extensions) to > support these languages/Unicode? I saw a few posting about not so great > String support in Erlang, therefore even less for Unicode. What's the > status? Hi, Fred. Like a few others here, I also have a website running on Erlang (mochiweb/webmachine/mnesia) right now, and it also handles UTF-8 just fine. For what it's worth, UTF-16 would have worked just as well, but UTF-8 was more convenient, given the set of services the website communicates with (solr, facebook, browsers, ...). For actually working with unicode text (not just storing & fetching), I've found the Starling project's links to the ICU library to be quite helpful: http://12monkeys.co.uk/starling/ The unpublicized xmerl_ucs module included with Erlang releases also has some useful tools for converting among character encodings, if you're unable to use R13B's unicode module. -Bryan From fjanon@REDACTED Sat Nov 7 14:44:39 2009 From: fjanon@REDACTED (Fred Janon) Date: Sat, 7 Nov 2009 21:44:39 +0800 Subject: [erlang-questions] Web servers and unicode support? In-Reply-To: References: <2fd0c7810911061808l2c1fc897ufa0f462237ab903@mail.gmail.com> Message-ID: <2fd0c7810911070544i23a79e9bida65da59932c0647@mail.gmail.com> Thanks everyone for your answers and pointers, really appreciated. I'll start working on it as soon as my present project permits, then. Fred On Sat, Nov 7, 2009 at 21:00, Bryan Fink wrote: > On Fri, Nov 6, 2009 at 9:08 PM, Fred Janon wrote: > > I want to design a web application with support for European, Asian > > languages as well as English, meaning Unicode. I am very interested in > doing > > that in Erlang with Mnesia or MySQL or Postgres (when I figure out if I > can > > use Mnesia), but what would I need to use (web server, extensions) to > > support these languages/Unicode? I saw a few posting about not so great > > String support in Erlang, therefore even less for Unicode. What's the > > status? > > Hi, Fred. Like a few others here, I also have a website running on > Erlang (mochiweb/webmachine/mnesia) right now, and it also handles > UTF-8 just fine. For what it's worth, UTF-16 would have worked just > as well, but UTF-8 was more convenient, given the set of services the > website communicates with (solr, facebook, browsers, ...). > > For actually working with unicode text (not just storing & fetching), > I've found the Starling project's links to the ICU library to be quite > helpful: > > http://12monkeys.co.uk/starling/ > > The unpublicized xmerl_ucs module included with Erlang releases also > has some useful tools for converting among character encodings, if > you're unable to use R13B's unicode module. > > -Bryan > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From caluml@REDACTED Sat Nov 7 19:53:04 2009 From: caluml@REDACTED (Calum) Date: Sat, 7 Nov 2009 18:53:04 +0000 Subject: Distributing things Message-ID: <635498b70911071053l6618ecfeif8ec2f7712ffa78@mail.gmail.com> Hello, I'm trying to get my head around Erlang, and I've decided to try to take an existing program, and distribute it. I'm using code from Joe's Erlang book which generates a prime of X digits. On my laptop, anything over 500 digits takes a while. If you have one a single node running the code, it just loops, calculates, tests if it's a prime, and increments the number, and repeats if it's not a prime. It's the "is_prime" test that seems to take the time, so that's the part I want to distribute. I imagine if you have more than one node, you want to kick off "n" is_prime tests, but of different values. What is the best way of going about this? Is it to kick off n, and wait for all to return before kicking off another n? This isn't so good if one of the nodes takes a very long time to return, as the rest are doing nothing. Or have a "global" "next value" variable, and each node just gets the next one when it's finished its current one? What would be the best way of maintaining this "next value" variable? How can I handle additional nodes "coming on line" as it's running? How can I handle a node going offline in the middle of this? Would you all just use the existing nodes() nodelist to do this, or would you need to extend that in some way? Is there a good tried and tested method of deciding how to break down a large job of 1000000 small jobs into batches for n nodes? Hope these questions makes sense to you, C From erlang@REDACTED Sat Nov 7 21:13:46 2009 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 7 Nov 2009 21:13:46 +0100 Subject: [erlang-questions] Distributing things In-Reply-To: <635498b70911071053l6618ecfeif8ec2f7712ffa78@mail.gmail.com> References: <635498b70911071053l6618ecfeif8ec2f7712ffa78@mail.gmail.com> Message-ID: <9b08084c0911071213h5f943d5fn690760bf0cdcbe93@mail.gmail.com> On Sat, Nov 7, 2009 at 7:53 PM, Calum wrote: > Hello, > > I'm trying to get my head around Erlang, and I've decided to try to > take an existing program, and distribute it. > I'm using code from Joe's Erlang book which generates a prime of X > digits. On my laptop, anything over 500 digits takes a while. > > If you have one a single node running the code, it just loops, > calculates, tests if it's a prime, and increments the number, and > repeats if it's not a prime. > It's the "is_prime" test that seems to take the time, so that's the > part I want to distribute. Hang on - the algorithm just generates a random integer and tests if it is a prime, if it's not a prime it tries again. It has to generate many candidates before it finds a prime. Why not start N parallel processes each of which generates and tests candidates then when either one of them succeeds you can stop. Sounds easier to me. /Joe > I imagine if you have more than one node, you want to kick off "n" > is_prime tests, but of different values. > > What is the best way of going about this? Is it to kick off n, and > wait for all to return before kicking off another n? This isn't so > good if one of the nodes takes a very long time to return, as the rest > are doing nothing. > Or have a "global" "next value" variable, and each node just gets the > next one when it's finished its current one? > What would be the best way of maintaining this "next value" variable? > > How can I handle additional nodes "coming on line" as it's running? > How can I handle a node going offline in the middle of this? > Would you all just use the existing nodes() nodelist to do this, or > would you need to extend that in some way? > > Is there a good tried and tested method of deciding how to break down > a large job of 1000000 small jobs into batches for n nodes? > > > Hope these questions makes sense to you, > > C > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From fabio.mazanatti@REDACTED Sat Nov 7 21:55:15 2009 From: fabio.mazanatti@REDACTED (Fabio Mazanatti) Date: Sat, 7 Nov 2009 18:55:15 -0200 Subject: [erlang-questions] Distributing things In-Reply-To: <9b08084c0911071213h5f943d5fn690760bf0cdcbe93@mail.gmail.com> References: <635498b70911071053l6618ecfeif8ec2f7712ffa78@mail.gmail.com> <9b08084c0911071213h5f943d5fn690760bf0cdcbe93@mail.gmail.com> Message-ID: <535117ab0911071255tc419e61j3da8c4af783df8bd@mail.gmail.com> Hi, the scenario I imagined for this exercise, based on Calum's text would be something like assigning a range of numbers to each node, starting with the lowest possible given by the entry parameter (the # of digits), controlling its distribution and wrapping things up when a prime number is found by one of the nodes. An example: 4 nodes, generating a prime with 6 digits, would start at 100.000. The entry point would assign a range from 100.000 to 105.000 to node 1, 105.001 to 110.000 to node 2, and so on. When a node finishes traversing a range, a new block is assigned to it, until one of them returns a valid value, and processing stops. Maybe(?) this is a weird construction, but the idea is to explore parallel processing of series, rather than disconected calls from many clients. I'm new to Erlang too, just finished reading Joe's book (BTW, great work!!), and am now exploring possibilities. Don't know if this matches with Calum's idea, but it's something I would like to explore :-) Cheers, Fabio Mazanatti On Sat, Nov 7, 2009 at 6:13 PM, Joe Armstrong wrote: > On Sat, Nov 7, 2009 at 7:53 PM, Calum wrote: > > Hello, > > > > I'm trying to get my head around Erlang, and I've decided to try to > > take an existing program, and distribute it. > > I'm using code from Joe's Erlang book which generates a prime of X > > digits. On my laptop, anything over 500 digits takes a while. > > > > If you have one a single node running the code, it just loops, > > calculates, tests if it's a prime, and increments the number, and > > repeats if it's not a prime. > > It's the "is_prime" test that seems to take the time, so that's the > > part I want to distribute. > > Hang on - the algorithm just generates a random integer and tests if > it is a prime, > if it's not a prime it tries again. It has to generate many candidates > before it finds a > prime. Why not start N parallel processes each of which generates and > tests candidates > then when either one of them succeeds you can stop. Sounds easier to me. > > /Joe > > > > I imagine if you have more than one node, you want to kick off "n" > > is_prime tests, but of different values. > > > > What is the best way of going about this? Is it to kick off n, and > > wait for all to return before kicking off another n? This isn't so > > good if one of the nodes takes a very long time to return, as the rest > > are doing nothing. > > Or have a "global" "next value" variable, and each node just gets the > > next one when it's finished its current one? > > What would be the best way of maintaining this "next value" variable? > > > > How can I handle additional nodes "coming on line" as it's running? > > How can I handle a node going offline in the middle of this? > > Would you all just use the existing nodes() nodelist to do this, or > > would you need to extend that in some way? > > > > Is there a good tried and tested method of deciding how to break down > > a large job of 1000000 small jobs into batches for n nodes? > > > > > > Hope these questions makes sense to you, > > > > C > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From caluml@REDACTED Sat Nov 7 21:57:02 2009 From: caluml@REDACTED (Calum) Date: Sat, 7 Nov 2009 20:57:02 +0000 Subject: [erlang-questions] Distributing things In-Reply-To: <9b08084c0911071213h5f943d5fn690760bf0cdcbe93@mail.gmail.com> References: <635498b70911071053l6618ecfeif8ec2f7712ffa78@mail.gmail.com> <9b08084c0911071213h5f943d5fn690760bf0cdcbe93@mail.gmail.com> Message-ID: <635498b70911071257y38c6e631k2b8c522537537298@mail.gmail.com> On Sat, Nov 7, 2009 at 8:13 PM, Joe Armstrong wrote: > Hang on - the algorithm just generates a random integer and tests if > it is a prime, > if it's not a prime it tries again. It has to generate many candidates > before it finds a > prime. Why not start N parallel processes each of which generates and > tests candidates > then when either one of them succeeds you can stop. Sounds easier to me. > > /Joe Heya Joe (crikey I feel honoured!) I thought it picked a random number, and then tested N-1 over and over to see if it was prime. I was trying to think how to coordinate the N value among all the processes so that they wouldn't try the same ones. I.e.: Main program selects 1234567 as a random number, and knows it has 3 Erlang nodes it can work on Node one tries 1234566 Node two tries 1234565 Node three tries 1234564 Then, when node one has found out that 1234566 isn't a prime (unsurprisingly!), it would have to know where the other nodes were at so it could know to check 1234563. Although I suppose it doesn't really matter (for this program) if they're all checking 123456* - but for some applications, they'd need to be in sync. The other thing I was wondering was what would happen if I fired up a new node in the middle of this running - how would it join in? Are there tried-and-tested software patterns to handle this sort of dynamic loading? C From caluml@REDACTED Sat Nov 7 22:02:39 2009 From: caluml@REDACTED (Calum) Date: Sat, 7 Nov 2009 21:02:39 +0000 Subject: [erlang-questions] Distributing things In-Reply-To: <535117ab0911071255tc419e61j3da8c4af783df8bd@mail.gmail.com> References: <635498b70911071053l6618ecfeif8ec2f7712ffa78@mail.gmail.com> <9b08084c0911071213h5f943d5fn690760bf0cdcbe93@mail.gmail.com> <535117ab0911071255tc419e61j3da8c4af783df8bd@mail.gmail.com> Message-ID: <635498b70911071302n17c39697k45e46bb3d08e35e5@mail.gmail.com> On Sat, Nov 7, 2009 at 8:55 PM, Fabio Mazanatti wrote: > Hi, > > the scenario I imagined for this exercise, based on Calum's text would be > something like assigning a range of numbers to each node, starting with the > lowest possible given by the entry parameter (the # of digits), controlling > its distribution and wrapping things up when a prime number is found by one > of the nodes. > > An example: 4 nodes, generating a prime with 6 digits, would start at > 100.000. The entry point would assign a range from 100.000 to 105.000 to > node 1, 105.001 to 110.000 to node 2, and so on. When a node finishes > traversing a range, a new block is assigned to it, until one of them returns > a valid value, and processing stops. Hello Fabio, Yes, this is the sort of thing I would do for this problem. However, for other problems, perhaps the splitting method isn't so clearcut. Also, your method only really allows for 20 nodes - what would you do if you suddenly had 50 nodes available to join in? Actually, in the real world, do Erlang node clusters vary in size a lot during operation, or do most people know how many nodes they have available at the start, and just program for that? Is my attempting to cope with varied node numbers something that just isn't really needed? From vances@REDACTED Sat Nov 7 22:03:36 2009 From: vances@REDACTED (Vance Shipley) Date: Sat, 7 Nov 2009 16:03:36 -0500 Subject: [erlang-questions] Exceptions and gen_server In-Reply-To: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> References: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> Message-ID: <20091107210335.GE92030@h216-235-12-174.host.egate.net> Some time ago I was asking myself whether it was better to throw an exception in the client or the server. In the end I decided your method was better because SASL handled it better. -Vance -------------------------------------------------------------------------- -module(foo_client). -export([do_local/2, do_remote/2]). do_local(Pid, Request) -> case gen_server:call(Pid, Request) of {ok, Value} -> Value; {error, Error} -> exit(Error) end. do_remote(Pid, Request) -> gen_server:call(Pid, Request). -------------------------------------------------------------------------- -module(foo_server). -behaviour(gen_server). -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). init(Arg) -> {ok, Arg}. handle_call(foo_local, _From, State) -> {reply, {ok, 42}, State}; handle_call(foo_remote, _From, State) -> {reply, 42, State}; handle_call(bar_local, _From, State) -> {reply, {error, badarg}, State}; handle_call(bar_remote, {Pid, _Tag}, State) -> exit(Pid, badarg), {noreply, State}. handle_cast(_Request, State) -> {noreply, State}. handle_info(_Info, State) -> {noreply, State}. terminate(_Reason, _State) -> ok. code_change(_OldVsn, State, _Extra) -> {ok, State}. -------------------------------------------------------------------------- 1> application:start(sasl). ok 2> {ok, S} = gen_server:start(foo_server, [], []). {ok,<0.45.0>} 3> foo_client:do_local(S, foo_local). 42 4> foo_client:do_local(S, bar_local). ** exception exit: badarg in function foo_client:do_local/2 5> foo_client:do_remote(S, foo_remote). 42 6> foo_client:do_remote(S, bar_remote). ** exception exit: badarg 7> On Fri, Nov 06, 2009 at 03:57:24PM +0000, Anton Krasovsky wrote: } What would be the standard way of returning/throwing an exception in a } result to a gen_server:call? } } Would something like that be reasonable: } } a() -> } case gen_server:call(..., a) of } {ok, Value} -> Value; } {error, Error} -> throw({error, Error}) } end } } and } } handle_call(a, ...) -> } case ... of } {ok, Value} -> {reply, {ok, Value} , State}; } {error, Error} -> {reply, {error, Error} , State} } end From anton.krasovsky@REDACTED Sat Nov 7 22:27:06 2009 From: anton.krasovsky@REDACTED (Anton Krasovsky) Date: Sat, 7 Nov 2009 21:27:06 +0000 Subject: [erlang-questions] Exceptions and gen_server In-Reply-To: References: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> Message-ID: <46167e6a0911071327r54c8e3ffl33c584c72597c907@mail.gmail.com> You're right, your approach is better. Regards, anton On Fri, Nov 6, 2009 at 5:08 PM, David Mercer wrote: > Is there any particular reason you want to throw the value rather than just > erroring out? ?If you need to throw it to a catch, I'd probably do as you > say, but otherwise, I'd just do a match and let an error happen when it > doesn't match. ?Something like: > > a() -> > ? ? ? ?{ok, Value} = gen_server:call(..., a), > ? ? ? ?Value. > > Your handle_call looks OK to me. > > Cheers, > > David > >> -----Original Message----- >> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On >> Behalf Of Anton Krasovsky >> Sent: Friday, November 06, 2009 9:57 AM >> To: erlang-questions@REDACTED >> Subject: [erlang-questions] Exceptions and gen_server >> >> What would be the standard way of returning/throwing an exception in a >> result to a gen_server:call? >> >> Would something like that be reasonable: >> >> a() -> >> case gen_server:call(..., a) of >> ? ? {ok, Value} -> Value; >> ? ? {error, Error} -> throw({error, Error}) >> end >> >> and >> >> handle_call(a, ...) -> >> case ... of >> ? ? {ok, Value} -> {reply, {ok, Value} , State}; >> ? ? {error, Error} -> {reply, {error, Error} , State} >> end >> >> Regards, >> Anton >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org > > From fabio.nunes@REDACTED Sat Nov 7 22:28:32 2009 From: fabio.nunes@REDACTED (Fabio Mazanatti Nunes) Date: Sat, 7 Nov 2009 19:28:32 -0200 Subject: [erlang-questions] Distributing things In-Reply-To: <635498b70911071302n17c39697k45e46bb3d08e35e5@mail.gmail.com> References: <635498b70911071053l6618ecfeif8ec2f7712ffa78@mail.gmail.com> <9b08084c0911071213h5f943d5fn690760bf0cdcbe93@mail.gmail.com> <535117ab0911071255tc419e61j3da8c4af783df8bd@mail.gmail.com> <635498b70911071302n17c39697k45e46bb3d08e35e5@mail.gmail.com> Message-ID: <535117ab0911071328s62874e13h5a50d31e2018b90d@mail.gmail.com> Hi Calum, as I said, my scenario may be weird, and indeed it is :-) But I can't see why it wouldn't use as many nodes as possible - I defined that the range interval was 5000, but could be any other number, fixed or derived from a calculation considering the # or digits and the # of active nodes, for instance. Anyway, my goal was to put an example of coordination and distribution. if each node works on a single number or a predefined range is not relevant - someone (a central module, for instance) has to control and pass the information along. You may know how many nodes will be available at production, but most of time you don't build your solution considering it. I'm working actively with Java EE, and always built applications targeting "clusters" (with any number of members), not a specific number of instances. If you need more power, you just add more nodes (or instances) to the environment. So, your idea of abstracting the number of actual nodes of a solution is the right way to go, at least to me :-) Ciao! On Sat, Nov 7, 2009 at 7:02 PM, Calum wrote: > On Sat, Nov 7, 2009 at 8:55 PM, Fabio Mazanatti > wrote: > > Hi, > > > > the scenario I imagined for this exercise, based on Calum's text would be > > something like assigning a range of numbers to each node, starting with > the > > lowest possible given by the entry parameter (the # of digits), > controlling > > its distribution and wrapping things up when a prime number is found by > one > > of the nodes. > > > > An example: 4 nodes, generating a prime with 6 digits, would start at > > 100.000. The entry point would assign a range from 100.000 to 105.000 to > > node 1, 105.001 to 110.000 to node 2, and so on. When a node finishes > > traversing a range, a new block is assigned to it, until one of them > returns > > a valid value, and processing stops. > > Hello Fabio, > > Yes, this is the sort of thing I would do for this problem. > However, for other problems, perhaps the splitting method isn't so > clearcut. > Also, your method only really allows for 20 nodes - what would you do > if you suddenly had 50 nodes available to join in? > > Actually, in the real world, do Erlang node clusters vary in size a > lot during operation, or do most people know how many nodes they have > available at the start, and just program for that? > Is my attempting to cope with varied node numbers something that just > isn't really needed? > From anton.krasovsky@REDACTED Sat Nov 7 22:33:33 2009 From: anton.krasovsky@REDACTED (Anton Krasovsky) Date: Sat, 7 Nov 2009 21:33:33 +0000 Subject: [erlang-questions] Exceptions and gen_server In-Reply-To: <20091107210335.GE92030@h216-235-12-174.host.egate.net> References: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> <20091107210335.GE92030@h216-235-12-174.host.egate.net> Message-ID: <46167e6a0911071333m18051578sa626d52c28ff6b45@mail.gmail.com> I suppose you'd get better stack trace in my case, but also I didn't realize you can do exit(Pid, Reason) to a caller process. Anton On Sat, Nov 7, 2009 at 9:03 PM, Vance Shipley wrote: > Some time ago I was asking myself whether it was better to > throw an exception in the client or the server. ?In the end > I decided your method was better because SASL handled it better. > > ? ? ? ?-Vance > > -------------------------------------------------------------------------- > -module(foo_client). > > -export([do_local/2, do_remote/2]). > > do_local(Pid, Request) -> > ? ? ? ?case gen_server:call(Pid, Request) of > ? ? ? ? ? ? ? ?{ok, Value} -> > ? ? ? ? ? ? ? ? ? ? ? ?Value; > ? ? ? ? ? ? ? ?{error, Error} -> > ? ? ? ? ? ? ? ? ? ? ? ?exit(Error) > ? ? ? ?end. > > do_remote(Pid, Request) -> > ? ? ? ?gen_server:call(Pid, Request). > > -------------------------------------------------------------------------- > -module(foo_server). > > -behaviour(gen_server). > > -export([init/1, handle_call/3, handle_cast/2, handle_info/2, > ? ? ? ? ? ? ? ? ? ? ? ?terminate/2, code_change/3]). > > init(Arg) -> > ? ? ? ?{ok, Arg}. > > handle_call(foo_local, _From, State) -> > ? ? ? ?{reply, {ok, 42}, State}; > handle_call(foo_remote, _From, State) -> > ? ? ? ?{reply, 42, State}; > handle_call(bar_local, _From, State) -> > ? ? ? ?{reply, {error, badarg}, State}; > handle_call(bar_remote, {Pid, _Tag}, State) -> > ? ? ? ?exit(Pid, badarg), > ? ? ? ?{noreply, State}. > > handle_cast(_Request, State) -> > ? ? ? ?{noreply, State}. > > handle_info(_Info, State) -> > ? ? ? ?{noreply, State}. > > terminate(_Reason, _State) -> > ? ? ? ?ok. > > code_change(_OldVsn, State, _Extra) -> > ? ? ? ?{ok, State}. > > -------------------------------------------------------------------------- > 1> application:start(sasl). > ok > 2> ?{ok, S} = gen_server:start(foo_server, [], []). > {ok,<0.45.0>} > 3> foo_client:do_local(S, foo_local). > 42 > 4> foo_client:do_local(S, bar_local). > ** exception exit: badarg > ? ? in function ?foo_client:do_local/2 > 5> foo_client:do_remote(S, foo_remote). > 42 > 6> foo_client:do_remote(S, bar_remote). > ** exception exit: badarg > 7> > > > On Fri, Nov 06, 2009 at 03:57:24PM +0000, Anton Krasovsky wrote: > } ?What would be the standard way of returning/throwing an exception in a > } ?result to a gen_server:call? > } > } ?Would something like that be reasonable: > } > } ?a() -> > } ?case gen_server:call(..., a) of > } ? ? ?{ok, Value} -> Value; > } ? ? ?{error, Error} -> throw({error, Error}) > } ?end > } > } ?and > } > } ?handle_call(a, ...) -> > } ?case ... of > } ? ? ?{ok, Value} -> {reply, {ok, Value} , State}; > } ? ? ?{error, Error} -> {reply, {error, Error} , State} > } ?end > From vances@REDACTED Sat Nov 7 22:42:08 2009 From: vances@REDACTED (Vance Shipley) Date: Sat, 7 Nov 2009 16:42:08 -0500 Subject: [erlang-questions] Exceptions and gen_server In-Reply-To: <46167e6a0911071333m18051578sa626d52c28ff6b45@mail.gmail.com> References: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> <20091107210335.GE92030@h216-235-12-174.host.egate.net> <46167e6a0911071333m18051578sa626d52c28ff6b45@mail.gmail.com> Message-ID: <20091107214207.GF92030@h216-235-12-174.host.egate.net> On Sat, Nov 07, 2009 at 09:33:33PM +0000, Anton Krasovsky wrote: } I suppose you'd get better stack trace in my case, but also I didn't } realize you can do exit(Pid, Reason) to a caller process. I do use this method to handle unknown calls in servers where I'd rather the server kept running than have it fail and expose a programming error in another process. handle_call(Request, From, State) -> ... {reply, Reply, State}; handle_call(_Request, {Pid, _Tag}, State) -> exit(Pid, badarg), {noreply, State}. This way if some process attempts gen_server:call(S, foo), where foo is not a supported API call, they get the appropriate response. -- -Vance From anton.krasovsky@REDACTED Sat Nov 7 22:53:27 2009 From: anton.krasovsky@REDACTED (Anton Krasovsky) Date: Sat, 7 Nov 2009 21:53:27 +0000 Subject: [erlang-questions] Exceptions and gen_server In-Reply-To: <20091107214207.GF92030@h216-235-12-174.host.egate.net> References: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> <20091107210335.GE92030@h216-235-12-174.host.egate.net> <46167e6a0911071333m18051578sa626d52c28ff6b45@mail.gmail.com> <20091107214207.GF92030@h216-235-12-174.host.egate.net> Message-ID: <46167e6a0911071353i7d8b1be9t284b5494539c445b@mail.gmail.com> True, I'd normally just print unexpected requests, but it's better this way. Anton On Sat, Nov 7, 2009 at 9:42 PM, Vance Shipley wrote: > On Sat, Nov 07, 2009 at 09:33:33PM +0000, Anton Krasovsky wrote: > } ?I suppose you'd get better stack trace in my case, but also I didn't > } ?realize you can do exit(Pid, Reason) to a caller process. > > I do use this method to handle unknown calls in servers where I'd > rather the server kept running than have it fail and expose a > programming error in another process. > > ? ? ? ?handle_call(Request, From, State) -> > ? ? ? ? ? ? ? ?... > ? ? ? ? ? ? ? ?{reply, Reply, State}; > ? ? ? ?handle_call(_Request, {Pid, _Tag}, State) -> > ? ? ? ? ? ? ? ?exit(Pid, badarg), > ? ? ? ? ? ? ? ?{noreply, State}. > > This way if some process attempts gen_server:call(S, foo), where > foo is not a supported API call, they get the appropriate response. > > -- > ? ? ? ?-Vance > From mpalmer@REDACTED Sun Nov 8 02:59:00 2009 From: mpalmer@REDACTED (Matthew Palmer) Date: Sun, 8 Nov 2009 12:59:00 +1100 Subject: Distributing things In-Reply-To: <635498b70911071302n17c39697k45e46bb3d08e35e5@mail.gmail.com> References: <635498b70911071053l6618ecfeif8ec2f7712ffa78@mail.gmail.com> <9b08084c0911071213h5f943d5fn690760bf0cdcbe93@mail.gmail.com> <535117ab0911071255tc419e61j3da8c4af783df8bd@mail.gmail.com> <635498b70911071302n17c39697k45e46bb3d08e35e5@mail.gmail.com> Message-ID: <20091108015900.GA8322@hezmatt.org> On Sat, Nov 07, 2009 at 09:02:39PM +0000, Calum wrote: > On Sat, Nov 7, 2009 at 8:55 PM, Fabio Mazanatti > wrote: > > Hi, > > > > the scenario I imagined for this exercise, based on Calum's text would be > > something like assigning a range of numbers to each node, starting with the > > lowest possible given by the entry parameter (the # of digits), controlling > > its distribution and wrapping things up when a prime number is found by one > > of the nodes. > > > > An example: 4 nodes, generating a prime with 6 digits, would start at > > 100.000. The entry point would assign a range from 100.000 to 105.000 to > > node 1, 105.001 to 110.000 to node 2, and so on. When a node finishes > > traversing a range, a new block is assigned to it, until one of them returns > > a valid value, and processing stops. > > Yes, this is the sort of thing I would do for this problem. > However, for other problems, perhaps the splitting method isn't so clearcut. So for these other problems, you would use a different method of distribution. For this problem, chunking the search space and distributing it to clients is a good method, so use it. "Right tool for the job" and all that. > Also, your method only really allows for 20 nodes - what would you do > if you suddenly had 50 nodes available to join in? Since the controlling process keeps control of the chunks, if more nodes join in mid-job, they just ask the controller for the next available chunk and go to it. > Actually, in the real world, do Erlang node clusters vary in size a > lot during operation, or do most people know how many nodes they have > available at the start, and just program for that? Honestly, I don't think that Erlang is the tool of choice for many big iron distributed computation problems. That's not to say that I don't think it's a good option (especially if you have more highly-optimised native code doing the number crunching, for maximum speed) but just that it isn't as widely used as the other, more well-known methods. > Is my attempting to cope with varied node numbers something that just > isn't really needed? It's your problem, not ours -- if it's necessary to solve your problem, then it's needed. Personally, I'd try the simplest thing that could possibly work first, which would be spawning a fixed number of processes to do the crunching, but if that was too slow in benchmarking (run a subset of the problem, time it, extrapolate) and, say, I wanted to start the search now but I had a pile of new hardware coming online next month, then I'd make sure that I could increase the number of available workers dynamically. - Matt From rumata-estor@REDACTED Sun Nov 8 08:42:29 2009 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Sun, 08 Nov 2009 10:42:29 +0300 Subject: Unload unused modules Message-ID: <1257666149.24645.15.camel@rumbuntu> Is there any way to unload unused module? I've been testing how many modules can be loaded to erlang. I came up to little less than 750 modules of 217.5 KB each. So beam.smp consumed 800 MB of memory. Now I'm sure that some of the loaded modules will not be used at least for a long time. How can I unload them? Dmitry Belyaev From vladdu55@REDACTED Sun Nov 8 13:24:09 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sun, 8 Nov 2009 13:24:09 +0100 Subject: [erlang-questions] Unload unused modules In-Reply-To: <1257666149.24645.15.camel@rumbuntu> References: <1257666149.24645.15.camel@rumbuntu> Message-ID: <95be1d3b0911080424k715748a8id225473d180b1bf6@mail.gmail.com> Hi! On Sun, Nov 8, 2009 at 08:42, Dmitry Belyaev wrote: > Is there any way to unload unused module? > > I've been testing how many modules can be loaded to erlang. I came up to > little less than 750 modules of 217.5 KB each. So beam.smp consumed 800 > MB of memory. > > Now I'm sure that some of the loaded modules will not be used at least > for a long time. How can I unload them? Yes, code:delete/1 and code:purge/1. If you know nobody uses tha code, the latter will also clean up the memory. regards, Vlad From rumata-estor@REDACTED Sun Nov 8 13:36:34 2009 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Sun, 08 Nov 2009 15:36:34 +0300 Subject: [erlang-questions] Unload unused modules In-Reply-To: <95be1d3b0911080424k715748a8id225473d180b1bf6@mail.gmail.com> References: <1257666149.24645.15.camel@rumbuntu> <95be1d3b0911080424k715748a8id225473d180b1bf6@mail.gmail.com> Message-ID: <1257683794.2860.1.camel@rumbuntu> Thank you. I searched for "unload" but there is "remove" in docs. On Sun, 2009-11-08 at 13:24 +0100, Vlad Dumitrescu wrote: > Hi! > > On Sun, Nov 8, 2009 at 08:42, Dmitry Belyaev wrote: > > Is there any way to unload unused module? > > > > I've been testing how many modules can be loaded to erlang. I came up to > > little less than 750 modules of 217.5 KB each. So beam.smp consumed 800 > > MB of memory. > > > > Now I'm sure that some of the loaded modules will not be used at least > > for a long time. How can I unload them? > > Yes, code:delete/1 and code:purge/1. If you know nobody uses tha code, > the latter will also clean up the memory. > > regards, > Vlad > From jeedward@REDACTED Sun Nov 8 14:50:25 2009 From: jeedward@REDACTED (John Edward) Date: Sun, 8 Nov 2009 05:50:25 -0800 (PST) Subject: MULTICONF-10 Call for papers Message-ID: <368038.43217.qm@web45915.mail.sp1.yahoo.com> MULTICONF-10 Call for papers ? The 2010 multi-conference (MULTICONF-10) (website: http://www.promoteresearch.org) will be held during July 12-14, 2010 in Orlando, Florida, USA. The primary goal of MULTICONF is to promote research and developmental activities in computer science, information technology, control engineering, and related fields. Another goal is to promote the dissemination of research to a multidisciplinary audience and to facilitate communication among researchers, developers, practitioners in different fields. The following conferences are planned to be organized as part of MULTICONF-10. ? International Conference on Artificial Intelligence and Pattern Recognition (AIPR-10) ?International Conference on Automation, Robotics and Control Systems (ARCS-10) International Conference on Bioinformatics, Computational Biology, Genomics and Chemoinformatics (BCBGC-10) International Conference on Computer Networks (CN-10) International Conference on Enterprise Information Systems and Web Technologies (EISWT-10) International Conference on High Performance Computing Systems (HPCS-10) International Conference on Information Security and Privacy (ISP-10) International Conference on Image and Video Processing and Computer Vision (IVPCV-10) International Conference on Software Engineering Theory and Practice (SETP-10) International Conference on Theoretical and Mathematical Foundations of Computer Science (TMFCS-10) ? We invite draft paper submissions. Please see the website http://www.promoteresearch.org for more details. ? Sincerely John Edward Publicity committee From steven.charles.davis@REDACTED Sun Nov 8 17:48:13 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 8 Nov 2009 08:48:13 -0800 (PST) Subject: SCTP accept? In-Reply-To: References: Message-ID: <7bb6ca9f-460b-4941-a421-cdd03910ca52@a21g2000yqc.googlegroups.com> I must be incredibly dense! Many words later, I'm still 100% unclear as to how to use gen_sctp effectively to do "http over sctp". The documentation shows an "Example of an Erlang SCTP Server which receives SCTP messages and prints them on the standard output." ...but does not really address how you reply unless you do everything from the one server process. To have any chance of scaling I would think that you'd want to process client requests in a number of worker processes at the server side (obviously)? With tcp you hand over the socket you get from accept -- but I'm entirely unclear what the equivalent of controlling_process(Socket, Pid) would be for associations? Any education would be very gratefully "accepted"! Best, Steve From erlang@REDACTED Sun Nov 8 18:32:00 2009 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 8 Nov 2009 18:32:00 +0100 Subject: [erlang-questions] Distributing things In-Reply-To: <635498b70911071257y38c6e631k2b8c522537537298@mail.gmail.com> References: <635498b70911071053l6618ecfeif8ec2f7712ffa78@mail.gmail.com> <9b08084c0911071213h5f943d5fn690760bf0cdcbe93@mail.gmail.com> <635498b70911071257y38c6e631k2b8c522537537298@mail.gmail.com> Message-ID: <9b08084c0911080932q782f8a63t34b242249ade777e@mail.gmail.com> On Sat, Nov 7, 2009 at 9:57 PM, Calum wrote: > On Sat, Nov 7, 2009 at 8:13 PM, Joe Armstrong wrote: >> Hang on - the algorithm just generates a random integer and tests if >> it is a prime, >> if it's not a prime it tries again. It has to generate many candidates >> before it finds a >> prime. Why not start N parallel processes each of which generates and >> tests candidates >> then when either one of them succeeds you can stop. Sounds easier to me. >> >> /Joe > > Heya Joe (crikey I feel honoured!) > > I thought it picked a random number, and then tested N-1 over and over > to see if it was prime. > I was trying to think how to coordinate the N value among all the > processes so that they wouldn't try the same ones. > I.e.: But the probability of that is 10^-N (N= number of digits) so for 80 digit primes 10^-80 > Main program selects 1234567 as a random number, and knows it has 3 > Erlang nodes it can work on > Node one tries 1234566 > Node two tries 1234565 > Node three tries 1234564 > Then, when node one has found out that 1234566 isn't a prime > (unsurprisingly!), it would have to know where the other nodes were at > so it could know to check 1234563. I'd just tell the N nodes "go generate random numbers and test if they are primes then tell me if you've got a prime" the chance they do the same work is very small and if they do who cares. The simplicity of the algorithm overweighs any potential inefficiency. Once you've got a prime tell all the nodes to stop calculating. > > Although I suppose it doesn't really matter (for this program) if > they're all checking 123456* - but for some applications, they'd need > to be in sync. > > The other thing I was wondering was what would happen if I fired up a > new node in the middle of this running - how would it join in? Are > there tried-and-tested software patterns to handle this sort of > dynamic loading? No - you could subscribe to node-up messages and fiddle with your algorithm /Joe > > C > From steven.charles.davis@REDACTED Sun Nov 8 19:37:35 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 8 Nov 2009 10:37:35 -0800 (PST) Subject: SCTP accept? In-Reply-To: <7bb6ca9f-460b-4941-a421-cdd03910ca52@a21g2000yqc.googlegroups.com> References: <7bb6ca9f-460b-4941-a421-cdd03910ca52@a21g2000yqc.googlegroups.com> Message-ID: <885603be-589e-40d3-9f28-2e6ef67c8981@t2g2000yqn.googlegroups.com> On Nov 8, 10:48?am, Steve Davis wrote: > I must be incredibly dense! Many words later, I'm still 100% unclear > as to how to use gen_sctp effectively to do "http over sctp". Same question, different way of saying it... Would it be fair to say that gen_sctp currently offers the many-to-one socket implementation: http://tools.ietf.org/html/draft-ietf-tsvwg-sctpsocket-19#section-4 ...but does not (yet?) offer the one-to-one socket implementation: http://tools.ietf.org/html/draft-ietf-tsvwg-sctpsocket-19#section-5 ...and thus is (necessarily?) incomplete? Is my current challenge best approached as: 1) write a one-to-one socket implementation, or 2) write a server layer (gen_event?) over the top of the many-to-one implementation ? /s From rvirding@REDACTED Sun Nov 8 19:47:29 2009 From: rvirding@REDACTED (Robert Virding) Date: Sun, 8 Nov 2009 19:47:29 +0100 Subject: [erlang-questions] Exceptions and gen_server In-Reply-To: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> References: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> Message-ID: <3dbc6d1c0911081047x689c99acm7b5a59cac836ea8@mail.gmail.com> There have been a number of different suggestions as to how you could program this. I think the main thing for you to decide is *HOW* you want to see the error, then it is relatively easy for you to program it. For example: - First off: do you want the error to crash the server or not? If so is links and exit signals a good enough way for you to see the error? - If the server is to live, how should your app see the error? Two valid suggestions which have been given here are: 1. Signal the error using exit/2. This "simulates" the exit signals as if the server had crashed. 2. Return an error value which either: returns an error value to the caller; or generates an error using erlang:error/1. Which is best for you depends on your app. All these solutions are easy to program so the main problem is for you to decide which is best for your app. One thing I would *NOT* do is use throw/1 to signal error! Throw is for non-local returns not errors. Using it for errors will just lead to confusion and maybe errors if other parts of app assume it is a non-local return. As try can differentiate between throw, exit and error it is important that they are used for the right thing. Robert P.S. I would LOVE to change catch now so it only catches throws, but I don't think it would be possible. :-( 2009/11/6 Anton Krasovsky > What would be the standard way of returning/throwing an exception in a > result to a gen_server:call? > > Would something like that be reasonable: > > a() -> > case gen_server:call(..., a) of > {ok, Value} -> Value; > {error, Error} -> throw({error, Error}) > end > > and > > handle_call(a, ...) -> > case ... of > {ok, Value} -> {reply, {ok, Value} , State}; > {error, Error} -> {reply, {error, Error} , State} > end > > Regards, > Anton > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From tamas.nagy@REDACTED Sun Nov 8 20:23:36 2009 From: tamas.nagy@REDACTED (Tamas Nagy) Date: Sun, 8 Nov 2009 19:23:36 +0000 Subject: Dialyzer problem on OSX 10.6 (Erlang R13B02-1) Message-ID: <7D879B13-07F9-4161-93BF-B1A44A138BFE@erlang-consulting.com> Hi, Am I the only one who cannot build a plt under OSX 10.6 in R13B02-1? No matter what applications I specify for the initial plt it just hangs eating processing power but not producing anything. Erlang was installed with homebrew, but that shouldn't matter. Or maybe it does. (beam is compiled for 64 bit) Here is the command I used: dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/erlang/ lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/lib/ kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/lib/ stdlib-1.16.3/ebin/ I gave waiting for it to finish after ~1 hour (on an old type white MacBook (MacBook2,1) with 2GB of RAM). The info I get from the dialyzer seems to suggest that it still assumes that erts version is 5.7 can the problem be around there? Checking whether the PLT /Users/lestat/.dialyzer_plt is up-to- date...dialyzer: Could not find the PLT: /Users/lestat/.dialyzer_plt Use the options: --build_plt to build a new one; or --add_to_plt to add to an existing PLT For example (in an installed Erlang/OTP system): dialyzer --build_plt -r $ERL_TOP/lib/erts-5.7/ebin\ $ERL_TOP/lib/kernel/ebin\ $ERL_TOP/lib/stdlib/ebin\ $ERL_TOP/lib/mnesia/ebin\ $ERL_TOP/lib/crypto/ebin Note that building a PLT such as the above may take 20 mins or so If I try to use a plt generated on another machine (on Ubuntu there does not seem to be a problem) it seems to work. P.S.: I've tried rebuilding erlang with and without disable-debug flag as well to no avail. Tamas Nagy Erlang Training & Consulting http://www.erlang-consulting.com From kostis@REDACTED Sun Nov 8 20:52:12 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Sun, 08 Nov 2009 21:52:12 +0200 Subject: [erlang-questions] Dialyzer problem on OSX 10.6 (Erlang R13B02-1) In-Reply-To: <7D879B13-07F9-4161-93BF-B1A44A138BFE@erlang-consulting.com> References: <7D879B13-07F9-4161-93BF-B1A44A138BFE@erlang-consulting.com> Message-ID: <4AF7216C.2030201@cs.ntua.gr> Tamas Nagy wrote: > Hi, > > Am I the only one who cannot build a plt under OSX 10.6 in R13B02-1? > > No matter what applications I specify for the initial plt it just hangs > eating processing power but not producing anything. > Erlang was installed with homebrew, but that shouldn't matter. Or maybe > it does. (beam is compiled for 64 bit) > > Here is the command I used: > dialyzer --build_plt -r > /usr/local/Cellar/erlang/R13B02-1/lib/erlang/lib/erts-5.7.3/ebin/ > /usr/local/Cellar/erlang/R13B02-1/lib/erlang/lib/kernel-2.13.3/ebin/ > /usr/local/Cellar/erlang/R13B02-1/lib/erlang/lib/stdlib-1.16.3/ebin/ > > I gave waiting for it to finish after ~1 hour (on an old type white > MacBook (MacBook2,1) with 2GB of RAM). None of the dialyzer developers has access to such a machine to see what's happening. Perhaps somebody else in that list might want to report whether this is reproducible or not. However, perhaps the following might help: when building the PLT, do you see a message: Compiling some key modules to native code... If so, can you disable hipe (64-bit Mac is not a platform we have ever tested due to no access to such a machine) to see whether this may be the culprit? Kostis From max.lapshin@REDACTED Sun Nov 8 21:56:05 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 8 Nov 2009 23:56:05 +0300 Subject: Why Beam.smp crashes when memory is over? Message-ID: Why does Erlang crashes, when physical memory is over? Error is: beam.smp(5454,0xb01aa000) malloc: *** mmap(size=41943040) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Isn't it possible to intercept this error and stop erlang process, that requests memory? From kagato@REDACTED Sun Nov 8 21:57:45 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Sun, 8 Nov 2009 12:57:45 -0800 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: From within Erlang, I don't believe so. From outside, maybe, with the right debugger. In general, I believe the heartbeat functionality (i.e. erl -heart) is supposed to handle this by letting it crash and restarting the whole application. On Nov 8, 2009, at 12:56 PM, Max Lapshin wrote: > Why does Erlang crashes, when physical memory is over? > > Error is: > > beam.smp(5454,0xb01aa000) malloc: *** mmap(size=41943040) failed > (error code=12) > > *** error: can't allocate region > > *** set a breakpoint in > malloc_error_break to debug > > > Isn't it possible to intercept this error and stop erlang process, > that requests memory? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- Jayson Vantuyl kagato@REDACTED From max.lapshin@REDACTED Sun Nov 8 22:02:31 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 9 Nov 2009 00:02:31 +0300 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: On Sun, Nov 8, 2009 at 11:57 PM, Jayson Vantuyl wrote: > From within Erlang, I don't believe so. And what are the problems? OS never crashes when memory is over, OOM killer does the job well. Why should die Erlang VM? From kagato@REDACTED Sun Nov 8 22:26:43 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Sun, 8 Nov 2009 13:26:43 -0800 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: Erlang needs to allocate memory in any number of situations. For example, assume that Erlang tried to tell your code. Should it generate a message? Should it call a function? Should it create an exception? Guess what all of these have in common? They allocate memory (which isn't there). You can try to work around this. You can have reserved memory just for this. However, there's still no clue on where it should happen. There is not a very good chance that this error will happen in the process that has all of the memory allocated. If you take the Linux OOM approach, you would have to scan all of the processes, weigh them, mix in some randomness, and message it. There's no memory for it do think much about the problem. Even if you killed it, that would just trigger the supervisor to restart it, even though we may not have actually stopped the memory leak. Worse, this means that an "out of memory error" can happen anywhere and must be handled everywhere, even the supervisors. Patching the supervisors to reliably handle this would be insane. Suddenly, reliability under load becomes impossible to guarantee. Even if you emulate Linux and provide an OOM-killer (i.e. kill processes based on randomness + heuristics to detect runaway processes), you introduce tons of random behavior into the VM, when a VM restart would be recognizable, loggable, and generally easier to debug. Exposing those errors creates an ugly situation. This extra error handling would cause an explosion of corner cases, decreases in reliability, and volumes of code (i.e. where bugs live). Inside of Erlang, the philosophy is to use supervisors and writing daemons to be able to recover from a restart. Heartbeat gives the same behavior for the entire VM. It's a philosophical design choice to try to handle critical faults rather than mask critical faults. It's really better than trying to handle this. It's seems obvious that there should be a better way to handle OOM, but it's is all devilishly difficult to do in any meaningfully portable (or useful) way. On Nov 8, 2009, at 1:02 PM, Max Lapshin wrote: > On Sun, Nov 8, 2009 at 11:57 PM, Jayson Vantuyl > wrote: >> From within Erlang, I don't believe so. > > And what are the problems? OS never crashes when memory is over, OOM > killer does the job well. > Why should die Erlang VM? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From anton.krasovsky@REDACTED Sun Nov 8 23:01:58 2009 From: anton.krasovsky@REDACTED (Anton Krasovsky) Date: Sun, 8 Nov 2009 22:01:58 +0000 Subject: [erlang-questions] Exceptions and gen_server In-Reply-To: <3dbc6d1c0911081047x689c99acm7b5a59cac836ea8@mail.gmail.com> References: <46167e6a0911060757x4c1ce0a7w9fa1e585cf3471ec@mail.gmail.com> <3dbc6d1c0911081047x689c99acm7b5a59cac836ea8@mail.gmail.com> Message-ID: <46167e6a0911081401s6b3a4cdld7e432dbca2b7f93@mail.gmail.com> Thanks for clarifying that, especially throw vs error part! Regards, Anton On Sun, Nov 8, 2009 at 6:47 PM, Robert Virding wrote: > There have been a number of different suggestions as to how you could > program this. I think the main thing for you to decide is *HOW* you want to > see the error, then it is relatively easy for you to program it. For > example: > > - First off: do you want the error to crash the server or not? If so is > links and exit signals a good enough way for you to see the error? > > - If the server is to live, how should your app see the error? Two valid > suggestions which have been given here are: > > 1. Signal the error using exit/2. This "simulates" the exit signals as if > the server had crashed. > 2. Return an error value which either: returns an error value to the caller; > or generates an error using erlang:error/1. Which is best for you depends on > your app. > > All these solutions are easy to program so the main problem is for you to > decide which is best for your app. > > One thing I would *NOT* do is use throw/1 to signal error! Throw is for > non-local returns not errors. Using it for errors will just lead to > confusion and maybe errors if other parts of app assume it is a non-local > return. As try can differentiate between throw, exit and error it is > important that they are used for the right thing. > > Robert > > P.S. I would LOVE to change catch now so it only catches throws, but I don't > think it would be possible. :-( > > 2009/11/6 Anton Krasovsky >> >> What would be the standard way of returning/throwing an exception in a >> result to a gen_server:call? >> >> Would something like that be reasonable: >> >> a() -> >> case gen_server:call(..., a) of >> ? ?{ok, Value} -> Value; >> ? ?{error, Error} -> throw({error, Error}) >> end >> >> and >> >> handle_call(a, ...) -> >> case ... of >> ? ?{ok, Value} -> {reply, {ok, Value} , State}; >> ? ?{error, Error} -> {reply, {error, Error} , State} >> end >> >> Regards, >> Anton >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > From pablo.platt@REDACTED Sun Nov 8 23:04:34 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Sun, 8 Nov 2009 14:04:34 -0800 (PST) Subject: some questions about beepbeep Message-ID: <854771.96531.qm@web112614.mail.gq1.yahoo.com> Hi, I'm trying to understand beepbeep web framework and ewgi. I like the ideas behind it. Is there a mailing list because I couldn't find one. Does anyone using it and can comment about it? Does it support arbitrary url dispatching like django and webmachine? I only saw examples like login_controller that handles /login/some/path I want to dispatch /admin/pages/15 to the admin_pages_controller. And sometimes you need to dispatch /user/15/album/12 to album_controller(user, album) Is there a form validation module? Thanks From tony@REDACTED Mon Nov 9 00:25:17 2009 From: tony@REDACTED (Tony Arcieri) Date: Sun, 8 Nov 2009 16:25:17 -0700 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: On Sun, Nov 8, 2009 at 1:57 PM, Jayson Vantuyl wrote: > In general, I believe the heartbeat functionality (i.e. erl -heart) is > supposed to handle this by letting it crash and restarting the whole > application. Yes, the design philosophy of Erlang is that your application should be stateless aside from things residing in dets tables or other external, stateful resources. Following the fail early, fail often approach, Erlang recovers from an out of memory condition by crashing the entire Erlang intrepreter, restarting it via heartbeat, and reloading your application. -- Tony Arcieri Medioh/Nagravision From tamas.nagy@REDACTED Mon Nov 9 00:35:12 2009 From: tamas.nagy@REDACTED (Tamas Nagy) Date: Sun, 8 Nov 2009 23:35:12 +0000 Subject: [erlang-questions] Dialyzer problem on OSX 10.6 (Erlang R13B02-1) In-Reply-To: <4AF7216C.2030201@cs.ntua.gr> References: <7D879B13-07F9-4161-93BF-B1A44A138BFE@erlang-consulting.com> <4AF7216C.2030201@cs.ntua.gr> Message-ID: Hi Kostis, This is what all I get: $ dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/erlang/ lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/lib/ kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/lib/ stdlib-1.16.3/ebin/ Creating PLT /Users/lestat/.dialyzer_plt ... I will try to rebuild erlang with disabling hipe to see if that helps. I am happy to run some tests/traces you would like to see the results of to help. Regards, Tamas Tamas Nagy Erlang Training & Consulting http://www.erlang-consulting.com On 8 Nov 2009, at 19:52, Kostis Sagonas wrote: > Tamas Nagy wrote: >> Hi, >> Am I the only one who cannot build a plt under OSX 10.6 in R13B02-1? >> No matter what applications I specify for the initial plt it just >> hangs eating processing power but not producing anything. >> Erlang was installed with homebrew, but that shouldn't matter. Or >> maybe it does. (beam is compiled for 64 bit) >> Here is the command I used: >> dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/ >> erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/ >> erlang/lib/kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/ >> lib/erlang/lib/stdlib-1.16.3/ebin/ >> I gave waiting for it to finish after ~1 hour (on an old type white >> MacBook (MacBook2,1) with 2GB of RAM). > > None of the dialyzer developers has access to such a machine to see > what's happening. Perhaps somebody else in that list might want to > report whether this is reproducible or not. > > However, perhaps the following might help: when building the PLT, do > you see a message: > > Compiling some key modules to native code... > > If so, can you disable hipe (64-bit Mac is not a platform we have > ever tested due to no access to such a machine) to see whether this > may be the culprit? > > Kostis From max.lapshin@REDACTED Mon Nov 9 05:56:38 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 9 Nov 2009 07:56:38 +0300 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: I understand your arguments. Yet, It would be great to have instrument, that tells supervisor, that child is eating too much memory. Look, if we are speaking about reliable platform, than it is better to have some predictable behaviour. Perhaps some startup options, like memory threshold for OOM killer? Supervisor will never grow up to 20 GB, but our rabbitmq production instance error_logger used to grow up to 20GB+ From max.lapshin@REDACTED Mon Nov 9 06:08:58 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 9 Nov 2009 08:08:58 +0300 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: Look, I have a program, that reads MPEG TS stream from network. There is some bug in my code and my ts_lander crashes beam. How is it possible to store state of external HTTP stream?? If Erlang VM wouldn't crash, but only kill the leaking process, I could buffer stream in other process and feed new mpeg ts lander after restart. But it is impossible =( From tony@REDACTED Mon Nov 9 06:43:25 2009 From: tony@REDACTED (Tony Arcieri) Date: Sun, 8 Nov 2009 22:43:25 -0700 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: On Sun, Nov 8, 2009 at 9:56 PM, Max Lapshin wrote: > I understand your arguments. Yet, It would be great to have > instrument, that tells supervisor, that child > is eating too much memory. Look, if we are speaking about reliable > platform, than it is better to have some > predictable behaviour. > I'm not really sure if we're speaking about the same thing then. Erlang is best utilized as a stateless platform, with a "fail early fail often" philosophy, and crashing the entire interpreter in the event of some underlying problem with the OS fits that entirely. You seem to be seeking a platform that lets you program defensively, handling errors rather than crashing and restoring with a clean state. In that regard, Erlang may not be the best language for you to work with, but then again the problem of successfully recovering from an out-of-resource condition is an extremely difficult one to handle well without scrapping it all and trying to restart with a clean state. I'm not really sure what alternative behavior you're proposing, either? -- Tony Arcieri Medioh/Nagravision From andrew@REDACTED Mon Nov 9 07:48:11 2009 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 9 Nov 2009 01:48:11 -0500 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: <20091109064811.GF9418@hijacked.us> On Sun, Nov 08, 2009 at 10:43:25PM -0700, Tony Arcieri wrote: > I'm not really sure if we're speaking about the same thing then. Erlang is > best utilized as a stateless platform, with a "fail early fail often" > philosophy, and crashing the entire interpreter in the event of some > underlying problem with the OS fits that entirely. You seem to be seeking a > platform that lets you program defensively, handling errors rather than > crashing and restoring with a clean state. In that regard, Erlang may not > be the best language for you to work with, but then again the problem of > successfully recovering from an out-of-resource condition is an extremely > difficult one to handle well without scrapping it all and trying to restart > with a clean state. > I agree that Erlang is best used when your application is as stateless as possible, or has clear transfer of responsibility (HTTP server, SMTP server, etc), but sometimes Erlang still has a lot of benefits even if you're lugging some state around, and that's when the current OOM behaviour is a little annoying. I don't say that the current behaviour is not the best in a lot of situations, but it'd be nice to have an alternative in cases where nuking the VM and having to recover or simply discard a lot of state is unpleasant. In an ideal world I'd like to see an *optional* OOM behaviour similar to the following: On startup, the VM pre-allocates enough memory to be able to attempt OOM recovery without additional malloc calls. If and when an OOM condition is detected, freeze the VM, iterate the process list and simply exit(FatPid, oom) the process with the most memory allocated (obviously this strategy is full of holes, epecially with regard to off-heap binaries), but *sometimes* it's better than just blowing away the whole thing. Then, if the malloc fails again, revert to the current behaviour. You could, of course try to be smarter about it (look at the process calling malloc, look at the reference counted binaries, try to avoid killing SASL or system processes), but the point is that sometimes the obvious offender is the only one you need to kill to restore a system's functionality. Of course, I'm no expert on this, these are merely musings from having (entirely due to my own fault) triggered plenty of OOM VM kills that would have been as easily solved if a single process had been killed. Andrew From sapan.gcet@REDACTED Mon Nov 9 08:16:39 2009 From: sapan.gcet@REDACTED (sapan shah) Date: Mon, 9 Nov 2009 12:46:39 +0530 Subject: accessing ets from the process on other node Message-ID: HI, I use digraph module to create graphs and apply some functions on it. The digraph is internally stored as ets tuples. I am using mapreduce architecture to speed up the computations on the digraph (using plists.erl from Stephen Marsh). So, When the graph is passed to a process on some other node on network, it exits with error. (may be because it is not able to access the ets tuples) Any solution??? Below is a small test example to generate the error %%%%%%%%%%%%%%%%%%%%%%%%%%% -module(pliststest). -compile(export_all). malt() -> [100, {nodes, [{'sap@REDACTED', 3}, {'sap@REDACTED', 3}]}]. test(G, List) -> Fun = fun(X) -> io:format("TROUBLE: ~p~n",[digraph:vertices(G)]), lists:foldr(fun(_Y,ACC) -> ACC+1 end, 0, X) end, Fuse = fun(X, Y) -> X+Y end, plists:runmany(Fun, Fuse, List, malt()). %%%%%%%%%%%%%%%%%%%%%%%%%%%% Also, PFA the plists.erl (from Stephen Marsh). -- Regards, Sapan Shah -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: plists.erl Type: text/x-erlang Size: 25663 bytes Desc: not available URL: From kiszl@REDACTED Mon Nov 9 08:35:53 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Mon, 9 Nov 2009 08:35:53 +0100 (CET) Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: <6058.194.88.55.211.1257752153.squirrel@localhost> Why don't you create a process, which regularly checks on the memory usage of those processes, and kills them when they begin to eat up too much memory? Zoltan. > Look, I have a program, that reads MPEG TS stream from network. There > is some bug in my code > and my ts_lander crashes beam. How is it possible to store state of > external HTTP stream?? > > If Erlang VM wouldn't crash, but only kill the leaking process, I > could buffer stream in other process and feed > new mpeg ts lander after restart. But it is impossible =( > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From max.lapshin@REDACTED Mon Nov 9 08:37:32 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 9 Nov 2009 10:37:32 +0300 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <6058.194.88.55.211.1257752153.squirrel@localhost> References: <6058.194.88.55.211.1257752153.squirrel@localhost> Message-ID: Yes, there are techniques to write watchdogs, but my question was: is it possible to prevent Erlang VM from crash? From rvirding@REDACTED Mon Nov 9 09:16:44 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 9 Nov 2009 09:16:44 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: <6058.194.88.55.211.1257752153.squirrel@localhost> Message-ID: <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> No. There is a major difference between handling OOM in an OS and in the BEAM. In an OS it usually at a per process level that memory runs out so it is easy to decide which process to kill so that the OS can continue. In the BEAM, however, it is the VM as a whole which has run out of memory not a specific, it is. therefore, much more difficult to work out which process is the culprit and to decide what to do. For example it might be that the process which causes the OOM is not the actual problem process, it might just the last straw. Or the actual cause may that the whole app might be generating large binaries too quickly. Or it might be that the whole app is spawning to many processes without any one process being the cause. Or ... In all these cases killing the process which triggered the OOM would be the Wrong Thing. We found that it was difficult to work out a reasonable strategy to handle the actual cause so we decided not handle it. "Don't catch an error which you can't handle" as the bard put it. Robert 2009/11/9 Max Lapshin > Yes, there are techniques to write watchdogs, but my question was: is > it possible to prevent Erlang VM from crash? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From max.lapshin@REDACTED Mon Nov 9 09:19:12 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 9 Nov 2009 11:19:12 +0300 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> Message-ID: Ok, so You advice is just to look after memory usage of "dangereous" processes? From rvirding@REDACTED Mon Nov 9 09:42:03 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 9 Nov 2009 09:42:03 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> Message-ID: <3dbc6d1c0911090042v682a9844v5a12b601c77dbda3@mail.gmail.com> That would be a start, it very much depends on your app. I am not really the right person to ask about details on chasing the culprit, but I do remember the discussions we had long ago on trying to work out a good solution. Which we couldn't. 2009/11/9 Max Lapshin > Ok, so You advice is just to look after memory usage of "dangereous" > processes? > From tony@REDACTED Mon Nov 9 09:45:10 2009 From: tony@REDACTED (Tony Rogvall) Date: Mon, 9 Nov 2009 09:45:10 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> Message-ID: <55BD1E0E-AA8A-462C-8BF0-08D1F5EAE1CC@rogvall.se> Interesting discussion! I have been working on a resource system for Erlang for nearly two years now. I have a working (tm) prototype where you can set resource limits like max_processes/max_ports/max_memory/max_time/max_reductions ... The limits are passed with spawn_opt and are inherited by the processes spawned. This means that if you spawn_opt(M,F,A[{max_memory, 1024*1024}]) the process will be able to use 1M words for it self and it's "subprocesses". This also means that the spawner will get 1M less to use (as designed right now). If a resource limit is reached the process crash with system_limt reason. There are still some details to work out before a release, but I will try to get it ready before end of this year. /Tony On 9 nov 2009, at 09.16, Robert Virding wrote: > No. > > There is a major difference between handling OOM in an OS and in the > BEAM. > In an OS it usually at a per process level that memory runs out so > it is > easy to decide which process to kill so that the OS can continue. In > the > BEAM, however, it is the VM as a whole which has run out of memory > not a > specific, it is. therefore, much more difficult to work out which > process is > the culprit and to decide what to do. For example it might be that the > process which causes the OOM is not the actual problem process, it > might > just the last straw. Or the actual cause may that the whole app > might be > generating large binaries too quickly. Or it might be that the whole > app is > spawning to many processes without any one process being the cause. > Or ... > In all these cases killing the process which triggered the OOM would > be the > Wrong Thing. We found that it was difficult to work out a reasonable > strategy to handle the actual cause so we decided not handle it. > > "Don't catch an error which you can't handle" as the bard put it. > > Robert > > 2009/11/9 Max Lapshin > >> Yes, there are techniques to write watchdogs, but my question was: is >> it possible to prevent Erlang VM from crash? >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> From kenneth.lundin@REDACTED Mon Nov 9 10:53:10 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 9 Nov 2009 10:53:10 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> Message-ID: On Mon, Nov 9, 2009 at 9:16 AM, Robert Virding wrote: > No. > > There is a major difference between handling OOM in an OS and in the BEAM. > In an OS it usually at a per process level that memory runs out so it is > easy to decide which process to kill so that the OS can continue. In the > BEAM, however, it is the VM as a whole which has run out of memory not a > specific, it is. therefore, much more difficult to work out which process is > the culprit and to decide what to do. For example it might be that the > process which causes the OOM is not the actual problem process, it might > just the last straw. Or the actual cause may that the whole app might be > generating large binaries too quickly. Or it might be that the whole app is > spawning to many processes without any one process being the cause. Or ... > In all these cases killing the process which triggered the OOM would be the > Wrong Thing. We found that it was difficult to work out a reasonable > strategy to handle the actual cause so we decided not handle it. > > "Don't catch an error which you can't handle" as the bard put it. > > Robert We have discussed this many times and have always come to the same conclusion as Robert expains above, that we can't know the right thing to do so we just do nothing and terminate the VM. What we could do is to make it easier for the user to prevent OOM situations and also to let him take the decision when it occurs or rather before it occurs. One way would be to let the user set a memory quota on a process with options at spawn time. When the process reaches it quota it can be automatically killed or the user can be notified in some way and take actions. We have also thought about ways for the user to monitor memory consumption and by this taking actions before the VM runs out of memory. /Kenneth, Erlang/OTP Ericsson From n.decker@REDACTED Mon Nov 9 11:31:19 2009 From: n.decker@REDACTED (Decker, Nils) Date: Mon, 9 Nov 2009 11:31:19 +0100 Subject: AW: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: Hello, I had a few occasions during developing that a runaway process caused my machine to swap and gind to a halt. (i should have used ulimit on beam) I wondered why there is no way to limit the size of a single process. It could be a simple option to spawn to limit the heap size of a process. (like ulimit) The process gets killed if it ever grows beyond the limit. A limit for the size of the message queue would be nice too, because a process with a few thousand entries is (most of the time) not going to cope with it anyway. A similar limit for ETS tables would be possible too. A memory limit for shared resources (large binaries) would be more difficult. Nils Decker From rumata-estor@REDACTED Mon Nov 9 12:05:41 2009 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Mon, 09 Nov 2009 14:05:41 +0300 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: <1257764741.2860.5.camel@rumbuntu> What if you'll have different nodes for different tasks instead of processes? Start one node to read stream and one (or many) other node to process the stream part. On Mon, 2009-11-09 at 08:08 +0300, Max Lapshin wrote: > Look, I have a program, that reads MPEG TS stream from network. There > is some bug in my code > and my ts_lander crashes beam. How is it possible to store state of > external HTTP stream?? > > If Erlang VM wouldn't crash, but only kill the leaking process, I > could buffer stream in other process and feed > new mpeg ts lander after restart. But it is impossible =( > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From tamas.nagy@REDACTED Mon Nov 9 12:35:18 2009 From: tamas.nagy@REDACTED (Tamas Nagy) Date: Mon, 9 Nov 2009 11:35:18 +0000 (GMT) Subject: [erlang-questions] Dialyzer problem on OSX 10.6 (Erlang R13B02-1) In-Reply-To: <3A65550C-AEAC-4232-822E-74CE96F7D493@erlang-consulting.com> Message-ID: <16321682.23851257766518669.JavaMail.root@zimbra> Hi Tino, Are you using homebrew? So far it seems that homebrew applies some c compiler magic (e.g optimizations) when building erlang which somehow borks dialyzer (strange) as compiling from source with the same ./configure options homebrew uses results in an erlang installation which works like a charm. Regards, Tamas ----- "Tino Breddin" wrote: > Hey Tamas, > > Here's the output for the command run on my machine (Macbook Pro Core > > Duo, running OSX 10.6.1 32-bit): > > dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/erlang/ > > lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/lib/ > > kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/lib/ > > stdlib-1.16.3/ebin/ > Creating PLT /Users/tino/.dialyzer_plt ... > Unknown functions: > compile:file/2 > compile:forms/2 > compile:noenv_forms/2 > compile:output_generated/1 > crypto:des3_cbc_decrypt/5 > crypto:start/0 > done in 46m22.52s > done (passed successfully) > > Apparently it takes some time to complete but you'll get there. > > Cheers, > Tino > > On Nov 8, 2009, at 11:35 PM, Tamas Nagy wrote: > > > Hi Kostis, > > > > This is what all I get: > > $ dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/ > > erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/ > > erlang/lib/kernel-2.13.3/ebin/ > /usr/local/Cellar/erlang/R13B02-1/lib/ > > erlang/lib/stdlib-1.16.3/ebin/ > > Creating PLT /Users/lestat/.dialyzer_plt ... > > > > I will try to rebuild erlang with disabling hipe to see if that > helps. > > > > I am happy to run some tests/traces you would like to see the > > results of to help. > > > > Regards, > > Tamas > > > > Tamas Nagy > > Erlang Training & Consulting > > http://www.erlang-consulting.com > > > > On 8 Nov 2009, at 19:52, Kostis Sagonas wrote: > > > >> Tamas Nagy wrote: > >>> Hi, > >>> Am I the only one who cannot build a plt under OSX 10.6 in > >>> R13B02-1? > >>> No matter what applications I specify for the initial plt it just > > >>> hangs eating processing power but not producing anything. > >>> Erlang was installed with homebrew, but that shouldn't matter. Or > > >>> maybe it does. (beam is compiled for 64 bit) > >>> Here is the command I used: > >>> dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/ > >>> erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/ > > >>> erlang/lib/kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/ > > >>> lib/erlang/lib/stdlib-1.16.3/ebin/ > >>> I gave waiting for it to finish after ~1 hour (on an old type > >>> white MacBook (MacBook2,1) with 2GB of RAM). > >> > >> None of the dialyzer developers has access to such a machine to see > > >> what's happening. Perhaps somebody else in that list might want to > > >> report whether this is reproducible or not. > >> > >> However, perhaps the following might help: when building the PLT, > > >> do you see a message: > >> > >> Compiling some key modules to native code... > >> > >> If so, can you disable hipe (64-bit Mac is not a platform we have > > >> ever tested due to no access to such a machine) to see whether this > > >> may be the culprit? > >> > >> Kostis > > > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > -- Tamas Nagy Erlang Training & Consulting http://www.erlang-consulting.com From clist@REDACTED Mon Nov 9 12:43:02 2009 From: clist@REDACTED (Angel Alvarez) Date: Mon, 9 Nov 2009 12:43:02 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <55BD1E0E-AA8A-462C-8BF0-08D1F5EAE1CC@rogvall.se> References: <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <55BD1E0E-AA8A-462C-8BF0-08D1F5EAE1CC@rogvall.se> Message-ID: <200911091243.02605.clist@uah.es> Well please let me say something I'm plain new but some things are pretty clear for me. The beauty of the erlang concept is "let it crash" , "don't program defensively" so the VM and the underlaying hardware are entities that can fail, that's it. What's the problem so? Joe said... If you want failure tolerance you need at least two nodes... From J.A thesis " ...Schneider [60, 59] answered this question by giving three properties that he thought a hardware system should have in order to be suitable for programming a fault-tolerant system. These properties Schneider called:..." 1. Halt on failure ? in the event of an error a processor should halt instead of performing a possibly erroneous operation.2 So on memory exhaustation the VM has to die and other node (erlang) will do the recovery. that's the distrirbution role, no only to span computations over several nodes to enhance performance but to provide resilence in the presence of fatal errors (non correctable). As a OS process the VM has to compete with other OS processes so in a shared deployment (a VM running on a server or a desktop) you cant be safe agaisnt a OOM trigered by other entities. Such resource control thing wil only augment process overhead and context switching in the VM. People new to erlang will be atracted to this hierarquical decomposition of tasks as joe stated in his thesis "If you cant run certaing task try doing something simpler" Many languages and VM's are incorporating erlang's good "multicore" features but not the erlang powerfull error handling concept and you guys want to kill the simpliticy incorporating many defensive capabilities to avoid fatality instead of just organize code to handle such fatality. ?whats next?, ?A mailbox maximum message queue control? Well, that's all i have to say about that, Forrest Gump. El Lunes, 9 de Noviembre de 2009 09:45:10 Tony Rogvall escribi?: > Interesting discussion! > > I have been working on a resource system for Erlang for nearly two > years now. > I have a working (tm) prototype where you can set resource limits like > max_processes/max_ports/max_memory/max_time/max_reductions ... > The limits are passed with spawn_opt and are inherited by the > processes spawned. > This means that if you spawn_opt(M,F,A[{max_memory, 1024*1024}]) the > process > will be able to use 1M words for it self and it's "subprocesses". This > also means > that the spawner will get 1M less to use (as designed right now). If a > resource limit > is reached the process crash with system_limt reason. > > There are still some details to work out before a release, but I will > try to get it ready before > end of this year. > > /Tony > > > > On 9 nov 2009, at 09.16, Robert Virding wrote: > > > No. > > > > There is a major difference between handling OOM in an OS and in the > > BEAM. > > In an OS it usually at a per process level that memory runs out so > > it is > > easy to decide which process to kill so that the OS can continue. In > > the > > BEAM, however, it is the VM as a whole which has run out of memory > > not a > > specific, it is. therefore, much more difficult to work out which > > process is > > the culprit and to decide what to do. For example it might be that the > > process which causes the OOM is not the actual problem process, it > > might > > just the last straw. Or the actual cause may that the whole app > > might be > > generating large binaries too quickly. Or it might be that the whole > > app is > > spawning to many processes without any one process being the cause. > > Or ... > > In all these cases killing the process which triggered the OOM would > > be the > > Wrong Thing. We found that it was difficult to work out a reasonable > > strategy to handle the actual cause so we decided not handle it. > > > > "Don't catch an error which you can't handle" as the bard put it. > > > > Robert > > > > 2009/11/9 Max Lapshin > > > >> Yes, there are techniques to write watchdogs, but my question was: is > >> it possible to prevent Erlang VM from crash? > >> > >> ________________________________________________________________ > >> erlang-questions mailing list. See http://www.erlang.org/faq.html > >> erlang-questions (at) erlang.org > >> > >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > Este correo no tiene dibujos. Las formas extra?as en la pantalla son letras. __________________________________________ Clist UAH a.k.a Angel __________________________________________ Warning: Microsoft_bribery.ISO contains OOXML code. From tony@REDACTED Mon Nov 9 14:11:34 2009 From: tony@REDACTED (Tony Rogvall) Date: Mon, 9 Nov 2009 14:11:34 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <200911091243.02605.clist@uah.es> References: <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <55BD1E0E-AA8A-462C-8BF0-08D1F5EAE1CC@rogvall.se> <200911091243.02605.clist@uah.es> Message-ID: <12D6682F-5EB6-428F-B28D-E65656AF17FF@rogvall.se> It is still the same "let it crash" concept using the resource limit system I am designing. But you can limit the crash in a more controlled way. Also you will be able to report interesting information about what is crashing and when. There is sometimes an issue when big systems crash. The restart may take a lot of time. Nodes must be synchronised, database tables must be repaired etc etc. I guess you can design this to be light and easy, but it is not always the case. /Tony On 9 nov 2009, at 12.43, Angel Alvarez wrote: > > Well please let me say something > > I'm plain new but some things are pretty clear for me. > > > The beauty of the erlang concept is "let it crash" , "don't program > defensively" > so the VM and the underlaying hardware are entities that can fail, > that's it. > > What's the problem so? > > Joe said... > > If you want failure tolerance you need at least two nodes... > > From J.A thesis > " ...Schneider [60, 59] answered this question by giving three > properties > that he thought a hardware system should have in order to be > suitable for > programming a fault-tolerant system. These properties Schneider > called:..." > > 1. Halt on failure ? in the event of an error a processor should halt > instead of performing a possibly erroneous operation.2 > > So on memory exhaustation the VM has to die and other node (erlang) > will do the recovery. > > that's the distrirbution role, no only to span computations over > several nodes to enhance performance > but to provide resilence in the presence of fatal errors (non > correctable). > > As a OS process the VM has to compete with other OS processes so in > a shared deployment (a VM running > on a server or a desktop) you cant be safe agaisnt a OOM trigered by > other entities. > > Such resource control thing wil only augment process overhead and > context switching in the VM. > > People new to erlang will be atracted to this hierarquical > decomposition of tasks as joe stated in his thesis > "If you cant run certaing task try doing something simpler" > > Many languages and VM's are incorporating erlang's good "multicore" > features but not the erlang powerfull error handling concept > and you guys want to kill the simpliticy incorporating many > defensive capabilities to avoid fatality instead of just organize > code to > handle such fatality. > > ?whats next?, ?A mailbox maximum message queue control? > > > Well, that's all i have to say about that, Forrest Gump. > > > El Lunes, 9 de Noviembre de 2009 09:45:10 Tony Rogvall escribi?: >> Interesting discussion! >> >> I have been working on a resource system for Erlang for nearly two >> years now. >> I have a working (tm) prototype where you can set resource limits >> like >> max_processes/max_ports/max_memory/max_time/max_reductions ... >> The limits are passed with spawn_opt and are inherited by the >> processes spawned. >> This means that if you spawn_opt(M,F,A[{max_memory, 1024*1024}]) the >> process >> will be able to use 1M words for it self and it's "subprocesses". >> This >> also means >> that the spawner will get 1M less to use (as designed right now). >> If a >> resource limit >> is reached the process crash with system_limt reason. >> >> There are still some details to work out before a release, but I will >> try to get it ready before >> end of this year. >> >> /Tony >> >> >> >> On 9 nov 2009, at 09.16, Robert Virding wrote: >> >>> No. >>> >>> There is a major difference between handling OOM in an OS and in the >>> BEAM. >>> In an OS it usually at a per process level that memory runs out so >>> it is >>> easy to decide which process to kill so that the OS can continue. In >>> the >>> BEAM, however, it is the VM as a whole which has run out of memory >>> not a >>> specific, it is. therefore, much more difficult to work out which >>> process is >>> the culprit and to decide what to do. For example it might be that >>> the >>> process which causes the OOM is not the actual problem process, it >>> might >>> just the last straw. Or the actual cause may that the whole app >>> might be >>> generating large binaries too quickly. Or it might be that the whole >>> app is >>> spawning to many processes without any one process being the cause. >>> Or ... >>> In all these cases killing the process which triggered the OOM would >>> be the >>> Wrong Thing. We found that it was difficult to work out a reasonable >>> strategy to handle the actual cause so we decided not handle it. >>> >>> "Don't catch an error which you can't handle" as the bard put it. >>> >>> Robert >>> >>> 2009/11/9 Max Lapshin >>> >>>> Yes, there are techniques to write watchdogs, but my question >>>> was: is >>>> it possible to prevent Erlang VM from crash? >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>>> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > > Este correo no tiene dibujos. Las formas extra?as en la pantalla son > letras. > __________________________________________ > > Clist UAH a.k.a Angel > __________________________________________ > Warning: Microsoft_bribery.ISO contains OOXML code. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From clist@REDACTED Mon Nov 9 14:54:10 2009 From: clist@REDACTED (Angel Alvarez) Date: Mon, 9 Nov 2009 14:54:10 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <12D6682F-5EB6-428F-B28D-E65656AF17FF@rogvall.se> References: <200911091243.02605.clist@uah.es> <12D6682F-5EB6-428F-B28D-E65656AF17FF@rogvall.se> Message-ID: <200911091454.10912.clist@uah.es> Well still there are many issues with this new approach Where are the maibox of processes located? With a heap pre process... Couldnt you trigger a memory exception on a remote process by just sending one message when the process is almost consuming its reserved memory? Systems other than embedded erlangs deploy ( form de current erlang movement as a server/desktop plataform) will suffer from resource contention beetween erlang VM and other OS processes. Port programs also need system resources... Well in the end your approach is still very interesting as a framework for continous erlang VM innovations... but please correct me if im wrong but I saw that memory carriers allowed to set several options on erlang VM start-up so, is stil posible to pacth those carriers to allow a safe memory reservation to let de VM manage properly a memory full condition by killing the offending process (sort of a OOM killer for the VM)? Just telling the VM not to "kill system process" and let the supervisors do the work... /Angel El Lunes, 9 de Noviembre de 2009 Tony Rogvall escribi?: > > It is still the same "let it crash" concept using the resource limit > system I am designing. > But you can limit the crash in a more controlled way. Also you will be > able to report > interesting information about what is crashing and when. > > There is sometimes an issue when big systems crash. The restart may > take a lot of time. > Nodes must be synchronised, database tables must be repaired etc etc. > I guess you can design this to be light and easy, but it is not always > the case. > > /Tony > > > > On 9 nov 2009, at 12.43, Angel Alvarez wrote: > > > > > Well please let me say something > > > > I'm plain new but some things are pretty clear for me. > > > > > > The beauty of the erlang concept is "let it crash" , "don't program > > defensively" > > so the VM and the underlaying hardware are entities that can fail, > > that's it. > > > > What's the problem so? > > > > Joe said... > > > > If you want failure tolerance you need at least two nodes... > > > > From J.A thesis > > " ...Schneider [60, 59] answered this question by giving three > > properties > > that he thought a hardware system should have in order to be > > suitable for > > programming a fault-tolerant system. These properties Schneider > > called:..." > > > > 1. Halt on failure ? in the event of an error a processor should halt > > instead of performing a possibly erroneous operation.2 > > > > So on memory exhaustation the VM has to die and other node (erlang) > > will do the recovery. > > > > that's the distrirbution role, no only to span computations over > > several nodes to enhance performance > > but to provide resilence in the presence of fatal errors (non > > correctable). > > > > As a OS process the VM has to compete with other OS processes so in > > a shared deployment (a VM running > > on a server or a desktop) you cant be safe agaisnt a OOM trigered by > > other entities. > > > > Such resource control thing wil only augment process overhead and > > context switching in the VM. > > > > People new to erlang will be atracted to this hierarquical > > decomposition of tasks as joe stated in his thesis > > "If you cant run certaing task try doing something simpler" > > > > Many languages and VM's are incorporating erlang's good "multicore" > > features but not the erlang powerfull error handling concept > > and you guys want to kill the simpliticy incorporating many > > defensive capabilities to avoid fatality instead of just organize > > code to > > handle such fatality. > > > > ?whats next?, ?A mailbox maximum message queue control? > > > > > > Well, that's all i have to say about that, Forrest Gump. > > > > > > El Lunes, 9 de Noviembre de 2009 09:45:10 Tony Rogvall escribi?: > >> Interesting discussion! > >> > >> I have been working on a resource system for Erlang for nearly two > >> years now. > >> I have a working (tm) prototype where you can set resource limits > >> like > >> max_processes/max_ports/max_memory/max_time/max_reductions ... > >> The limits are passed with spawn_opt and are inherited by the > >> processes spawned. > >> This means that if you spawn_opt(M,F,A[{max_memory, 1024*1024}]) the > >> process > >> will be able to use 1M words for it self and it's "subprocesses". > >> This > >> also means > >> that the spawner will get 1M less to use (as designed right now). > >> If a > >> resource limit > >> is reached the process crash with system_limt reason. > >> > >> There are still some details to work out before a release, but I will > >> try to get it ready before > >> end of this year. > >> > >> /Tony > >> > >> > >> > >> On 9 nov 2009, at 09.16, Robert Virding wrote: > >> > >>> No. > >>> > >>> There is a major difference between handling OOM in an OS and in the > >>> BEAM. > >>> In an OS it usually at a per process level that memory runs out so > >>> it is > >>> easy to decide which process to kill so that the OS can continue. In > >>> the > >>> BEAM, however, it is the VM as a whole which has run out of memory > >>> not a > >>> specific, it is. therefore, much more difficult to work out which > >>> process is > >>> the culprit and to decide what to do. For example it might be that > >>> the > >>> process which causes the OOM is not the actual problem process, it > >>> might > >>> just the last straw. Or the actual cause may that the whole app > >>> might be > >>> generating large binaries too quickly. Or it might be that the whole > >>> app is > >>> spawning to many processes without any one process being the cause. > >>> Or ... > >>> In all these cases killing the process which triggered the OOM would > >>> be the > >>> Wrong Thing. We found that it was difficult to work out a reasonable > >>> strategy to handle the actual cause so we decided not handle it. > >>> > >>> "Don't catch an error which you can't handle" as the bard put it. > >>> > >>> Robert > >>> > >>> 2009/11/9 Max Lapshin > >>> > >>>> Yes, there are techniques to write watchdogs, but my question > >>>> was: is > >>>> it possible to prevent Erlang VM from crash? > >>>> > >>>> ________________________________________________________________ > >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html > >>>> erlang-questions (at) erlang.org > >>>> > >>>> > >> > >> > >> ________________________________________________________________ > >> erlang-questions mailing list. See http://www.erlang.org/faq.html > >> erlang-questions (at) erlang.org > >> > >> > > > > > > Este correo no tiene dibujos. Las formas extra?as en la pantalla son > > letras. > > __________________________________________ > > > > Clist UAH a.k.a Angel > > __________________________________________ > > Warning: Microsoft_bribery.ISO contains OOXML code. > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > -- No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. __________________________________________ Clist UAH a.k.a Angel __________________________________________ China 'limpia' el Tibet para las Olimpiadas. From thomasl_erlang@REDACTED Mon Nov 9 14:25:09 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 9 Nov 2009 05:25:09 -0800 (PST) Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <20091109064811.GF9418@hijacked.us> References: <20091109064811.GF9418@hijacked.us> Message-ID: <663062.43522.qm@web111409.mail.gq1.yahoo.com> ----- Original Message ---- > From: Andrew Thompson > ... it'd be nice to have an > alternative in cases where nuking the VM and having to recover or simply > discard a lot of state is unpleasant. > > In an ideal world I'd like to see an *optional* OOM behaviour similar to > the following: > > On startup, the VM pre-allocates enough memory to be able to attempt OOM > recovery without additional malloc calls. If and when an OOM condition > is detected, freeze the VM, iterate the process list and simply > exit(FatPid, oom) the process with the most memory allocated (obviously > this strategy is full of holes, epecially with regard to off-heap > binaries), but *sometimes* it's better than just blowing away the whole > thing. Then, if the malloc fails again, revert to the current behaviour. Another way of doing that might be to hand the decision to (a modified version of) the supervisor hierarchy (if any). The supervisor could get info on sibling priority (if any) and/or memory usage as part of the request perhaps. Restart up the supervisor hierarchy to suit. That might give more structure to the restarting. Or you could do the OOM-killing in distributed erlang + unix instead. Run the buggy, memory-hungry stuff in a separate VM with a memory limit on the process, and manage the work from a "supervisor VM" using monitoring and rpc. Not quite as elegant, of course. Best, Thomas From tony@REDACTED Mon Nov 9 15:38:02 2009 From: tony@REDACTED (Tony Rogvall) Date: Mon, 9 Nov 2009 15:38:02 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <200911091454.10912.clist@uah.es> References: <200911091243.02605.clist@uah.es> <12D6682F-5EB6-428F-B28D-E65656AF17FF@rogvall.se> <200911091454.10912.clist@uah.es> Message-ID: <0C8EA69B-2D70-49A1-AF95-161D1C920A23@rogvall.se> Hi! On 9 nov 2009, at 14.54, Angel Alvarez wrote: > Well still there are many issues with this new approach > Yes! But it does not scare me ;-) > Where are the maibox of processes located? > > With a heap pre process... Depends on the implementation. But in general you could do something like, If the the data is shared then you split the share (memory_size/ref_count). If the data is copied then you must count it in. > > Couldnt you trigger a memory exception on a remote process by just > sending one message > when the process is almost consuming its reserved memory? > Yes. But that is the point. If it pass the limit the process will die. There are many special cases where you could think of using the memory in a better and more optimal way. Lets say you are reaching the memory limit you may switch to a compression algorithm for heap memory !? But lets keep it simple for the prototype, and see if it useful. > Systems other than embedded erlangs deploy ( form de current erlang > movement as a server/desktop plataform) > will suffer from resource contention beetween erlang VM and other OS > processes. > > Port programs also need system resources... > For loadable drivers using driver_alloc, one could possibly do something, otherwise it will be up to the driver designer to handle it. There is a max_ports in the prototype that limits number of open_ports. If sockets/files are mapped to single ports then it may help a bit. > Well in the end your approach is still very interesting as a > framework for continous erlang VM innovations... > Thanks. > but please correct me if im wrong but I saw that memory carriers > allowed to set several options on erlang VM start-up so, > I am not sure what you mean here? /Tony > is stil posible to pacth those carriers to allow a safe memory > reservation to let de VM manage properly a memory full > condition by killing the offending process (sort of a OOM killer for > the VM)? > > Just telling the VM not to "kill system process" and let the > supervisors do the work... > > /Angel > > > > El Lunes, 9 de Noviembre de 2009 Tony Rogvall escribi?: >> >> It is still the same "let it crash" concept using the resource limit >> system I am designing. >> But you can limit the crash in a more controlled way. Also you will >> be >> able to report >> interesting information about what is crashing and when. >> >> There is sometimes an issue when big systems crash. The restart may >> take a lot of time. >> Nodes must be synchronised, database tables must be repaired etc etc. >> I guess you can design this to be light and easy, but it is not >> always >> the case. >> >> /Tony >> >> >> >> On 9 nov 2009, at 12.43, Angel Alvarez wrote: >> >>> >>> Well please let me say something >>> >>> I'm plain new but some things are pretty clear for me. >>> >>> >>> The beauty of the erlang concept is "let it crash" , "don't program >>> defensively" >>> so the VM and the underlaying hardware are entities that can fail, >>> that's it. >>> >>> What's the problem so? >>> >>> Joe said... >>> >>> If you want failure tolerance you need at least two nodes... >>> >>> From J.A thesis >>> " ...Schneider [60, 59] answered this question by giving three >>> properties >>> that he thought a hardware system should have in order to be >>> suitable for >>> programming a fault-tolerant system. These properties Schneider >>> called:..." >>> >>> 1. Halt on failure ? in the event of an error a processor should >>> halt >>> instead of performing a possibly erroneous operation.2 >>> >>> So on memory exhaustation the VM has to die and other node (erlang) >>> will do the recovery. >>> >>> that's the distrirbution role, no only to span computations over >>> several nodes to enhance performance >>> but to provide resilence in the presence of fatal errors (non >>> correctable). >>> >>> As a OS process the VM has to compete with other OS processes so in >>> a shared deployment (a VM running >>> on a server or a desktop) you cant be safe agaisnt a OOM trigered by >>> other entities. >>> >>> Such resource control thing wil only augment process overhead and >>> context switching in the VM. >>> >>> People new to erlang will be atracted to this hierarquical >>> decomposition of tasks as joe stated in his thesis >>> "If you cant run certaing task try doing something simpler" >>> >>> Many languages and VM's are incorporating erlang's good "multicore" >>> features but not the erlang powerfull error handling concept >>> and you guys want to kill the simpliticy incorporating many >>> defensive capabilities to avoid fatality instead of just organize >>> code to >>> handle such fatality. >>> >>> ?whats next?, ?A mailbox maximum message queue control? >>> >>> >>> Well, that's all i have to say about that, Forrest Gump. >>> >>> >>> El Lunes, 9 de Noviembre de 2009 09:45:10 Tony Rogvall escribi?: >>>> Interesting discussion! >>>> >>>> I have been working on a resource system for Erlang for nearly two >>>> years now. >>>> I have a working (tm) prototype where you can set resource limits >>>> like >>>> max_processes/max_ports/max_memory/max_time/max_reductions ... >>>> The limits are passed with spawn_opt and are inherited by the >>>> processes spawned. >>>> This means that if you spawn_opt(M,F,A[{max_memory, 1024*1024}]) >>>> the >>>> process >>>> will be able to use 1M words for it self and it's "subprocesses". >>>> This >>>> also means >>>> that the spawner will get 1M less to use (as designed right now). >>>> If a >>>> resource limit >>>> is reached the process crash with system_limt reason. >>>> >>>> There are still some details to work out before a release, but I >>>> will >>>> try to get it ready before >>>> end of this year. >>>> >>>> /Tony >>>> >>>> >>>> >>>> On 9 nov 2009, at 09.16, Robert Virding wrote: >>>> >>>>> No. >>>>> >>>>> There is a major difference between handling OOM in an OS and in >>>>> the >>>>> BEAM. >>>>> In an OS it usually at a per process level that memory runs out so >>>>> it is >>>>> easy to decide which process to kill so that the OS can >>>>> continue. In >>>>> the >>>>> BEAM, however, it is the VM as a whole which has run out of memory >>>>> not a >>>>> specific, it is. therefore, much more difficult to work out which >>>>> process is >>>>> the culprit and to decide what to do. For example it might be that >>>>> the >>>>> process which causes the OOM is not the actual problem process, it >>>>> might >>>>> just the last straw. Or the actual cause may that the whole app >>>>> might be >>>>> generating large binaries too quickly. Or it might be that the >>>>> whole >>>>> app is >>>>> spawning to many processes without any one process being the >>>>> cause. >>>>> Or ... >>>>> In all these cases killing the process which triggered the OOM >>>>> would >>>>> be the >>>>> Wrong Thing. We found that it was difficult to work out a >>>>> reasonable >>>>> strategy to handle the actual cause so we decided not handle it. >>>>> >>>>> "Don't catch an error which you can't handle" as the bard put it. >>>>> >>>>> Robert >>>>> >>>>> 2009/11/9 Max Lapshin >>>>> >>>>>> Yes, there are techniques to write watchdogs, but my question >>>>>> was: is >>>>>> it possible to prevent Erlang VM from crash? >>>>>> >>>>>> ________________________________________________________________ >>>>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>>>> erlang-questions (at) erlang.org >>>>>> >>>>>> >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>>> >>> >>> >>> Este correo no tiene dibujos. Las formas extra?as en la pantalla son >>> letras. >>> __________________________________________ >>> >>> Clist UAH a.k.a Angel >>> __________________________________________ >>> Warning: Microsoft_bribery.ISO contains OOXML code. >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> >> > > > > -- > No imprima este correo si no es necesario. El medio ambiente est? en > nuestras manos. > __________________________________________ > > Clist UAH a.k.a Angel > __________________________________________ > China 'limpia' el Tibet para las Olimpiadas. From clist@REDACTED Mon Nov 9 17:21:41 2009 From: clist@REDACTED (Angel Alvarez) Date: Mon, 9 Nov 2009 17:21:41 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <0C8EA69B-2D70-49A1-AF95-161D1C920A23@rogvall.se> References: <200911091454.10912.clist@uah.es> <0C8EA69B-2D70-49A1-AF95-161D1C920A23@rogvall.se> Message-ID: <200911091721.41491.clist@uah.es> El Lunes, 9 de Noviembre de 2009 Tony Rogvall escribi?: > Hi! > > On 9 nov 2009, at 14.54, Angel Alvarez wrote: > > > Well still there are many issues with this new approach > > > Yes! But it does not scare me ;-) > > > Where are the maibox of processes located? > > > > With a heap pre process... > > Depends on the implementation. But in general you could do something > like, If the the data is shared then > you split the share (memory_size/ref_count). If the data is copied > then you must count it in. > > > > > Couldnt you trigger a memory exception on a remote process by just > > sending one message > > when the process is almost consuming its reserved memory? > > > Yes. But that is the point. If it pass the limit the process will die. > There are many special cases where you could think of using the memory > in a better and more optimal way. > Lets say you are reaching the memory limit you may switch to a > compression algorithm for heap memory !? > But lets keep it simple for the prototype, and see if it useful. > > > > Systems other than embedded erlangs deploy ( form de current erlang > > movement as a server/desktop plataform) > > will suffer from resource contention beetween erlang VM and other OS > > processes. > > > > Port programs also need system resources... > > > For loadable drivers using driver_alloc, one could possibly do > something, otherwise it will be > up to the driver designer to handle it. There is a max_ports in the > prototype that limits number of > open_ports. If sockets/files are mapped to single ports then it may > help a bit. > > > > Well in the end your approach is still very interesting as a > > framework for continous erlang VM innovations... > > > Thanks. > > > but please correct me if im wrong but I saw that memory carriers > > allowed to set several options on erlang VM start-up so, > > > I am not sure what you mean here? Well some bits can be controlled as you show on spawn_opt. In the other hand i mean that VM Safe memory as stated by others (Andrew, Tom ...) should be controlled on memory carriers: ( erts_alloc(3) ) In esence - A new emergency memory alocator in the alloc_util framework just in case the VM need memory to recover from a OOM. or perhaps - Using the segment allocator on mmap supported arquitectures to allow fast recovery for the full erlang vm (sort of checkpointing) using a special BIF you could instruct memory carriers to checkpoint the entire VM usining this allocator just in case the VM crashes. so on next run the enrire VM can be recovered parsing the last checkpint Still i think is better and more clean having two o more instances that bulletprooffing the VM /Angel > > /Tony > > > is stil posible to pacth those carriers to allow a safe memory > > reservation to let de VM manage properly a memory full > > condition by killing the offending process (sort of a OOM killer for > > the VM)? > > > > Just telling the VM not to "kill system process" and let the > > supervisors do the work... > > > > > > > /Angel > > > > > > > > El Lunes, 9 de Noviembre de 2009 Tony Rogvall escribi?: > >> > >> It is still the same "let it crash" concept using the resource limit > >> system I am designing. > >> But you can limit the crash in a more controlled way. Also you will > >> be > >> able to report > >> interesting information about what is crashing and when. > >> > >> There is sometimes an issue when big systems crash. The restart may > >> take a lot of time. > >> Nodes must be synchronised, database tables must be repaired etc etc. > >> I guess you can design this to be light and easy, but it is not > >> always > >> the case. > >> > >> /Tony > >> > >> > >> > >> On 9 nov 2009, at 12.43, Angel Alvarez wrote: > >> > >>> > >>> Well please let me say something > >>> > >>> I'm plain new but some things are pretty clear for me. > >>> > >>> > >>> The beauty of the erlang concept is "let it crash" , "don't program > >>> defensively" > >>> so the VM and the underlaying hardware are entities that can fail, > >>> that's it. > >>> > >>> What's the problem so? > >>> > >>> Joe said... > >>> > >>> If you want failure tolerance you need at least two nodes... > >>> > >>> From J.A thesis > >>> " ...Schneider [60, 59] answered this question by giving three > >>> properties > >>> that he thought a hardware system should have in order to be > >>> suitable for > >>> programming a fault-tolerant system. These properties Schneider > >>> called:..." > >>> > >>> 1. Halt on failure ? in the event of an error a processor should > >>> halt > >>> instead of performing a possibly erroneous operation.2 > >>> > >>> So on memory exhaustation the VM has to die and other node (erlang) > >>> will do the recovery. > >>> > >>> that's the distrirbution role, no only to span computations over > >>> several nodes to enhance performance > >>> but to provide resilence in the presence of fatal errors (non > >>> correctable). > >>> > >>> As a OS process the VM has to compete with other OS processes so in > >>> a shared deployment (a VM running > >>> on a server or a desktop) you cant be safe agaisnt a OOM trigered by > >>> other entities. > >>> > >>> Such resource control thing wil only augment process overhead and > >>> context switching in the VM. > >>> > >>> People new to erlang will be atracted to this hierarquical > >>> decomposition of tasks as joe stated in his thesis > >>> "If you cant run certaing task try doing something simpler" > >>> > >>> Many languages and VM's are incorporating erlang's good "multicore" > >>> features but not the erlang powerfull error handling concept > >>> and you guys want to kill the simpliticy incorporating many > >>> defensive capabilities to avoid fatality instead of just organize > >>> code to > >>> handle such fatality. > >>> > >>> ?whats next?, ?A mailbox maximum message queue control? > >>> > >>> > >>> Well, that's all i have to say about that, Forrest Gump. > >>> > >>> > >>> El Lunes, 9 de Noviembre de 2009 09:45:10 Tony Rogvall escribi?: > >>>> Interesting discussion! > >>>> > >>>> I have been working on a resource system for Erlang for nearly two > >>>> years now. > >>>> I have a working (tm) prototype where you can set resource limits > >>>> like > >>>> max_processes/max_ports/max_memory/max_time/max_reductions ... > >>>> The limits are passed with spawn_opt and are inherited by the > >>>> processes spawned. > >>>> This means that if you spawn_opt(M,F,A[{max_memory, 1024*1024}]) > >>>> the > >>>> process > >>>> will be able to use 1M words for it self and it's "subprocesses". > >>>> This > >>>> also means > >>>> that the spawner will get 1M less to use (as designed right now). > >>>> If a > >>>> resource limit > >>>> is reached the process crash with system_limt reason. > >>>> > >>>> There are still some details to work out before a release, but I > >>>> will > >>>> try to get it ready before > >>>> end of this year. > >>>> > >>>> /Tony > >>>> > >>>> > >>>> > >>>> On 9 nov 2009, at 09.16, Robert Virding wrote: > >>>> > >>>>> No. > >>>>> > >>>>> There is a major difference between handling OOM in an OS and in > >>>>> the > >>>>> BEAM. > >>>>> In an OS it usually at a per process level that memory runs out so > >>>>> it is > >>>>> easy to decide which process to kill so that the OS can > >>>>> continue. In > >>>>> the > >>>>> BEAM, however, it is the VM as a whole which has run out of memory > >>>>> not a > >>>>> specific, it is. therefore, much more difficult to work out which > >>>>> process is > >>>>> the culprit and to decide what to do. For example it might be that > >>>>> the > >>>>> process which causes the OOM is not the actual problem process, it > >>>>> might > >>>>> just the last straw. Or the actual cause may that the whole app > >>>>> might be > >>>>> generating large binaries too quickly. Or it might be that the > >>>>> whole > >>>>> app is > >>>>> spawning to many processes without any one process being the > >>>>> cause. > >>>>> Or ... > >>>>> In all these cases killing the process which triggered the OOM > >>>>> would > >>>>> be the > >>>>> Wrong Thing. We found that it was difficult to work out a > >>>>> reasonable > >>>>> strategy to handle the actual cause so we decided not handle it. > >>>>> > >>>>> "Don't catch an error which you can't handle" as the bard put it. > >>>>> > >>>>> Robert > >>>>> > >>>>> 2009/11/9 Max Lapshin > >>>>> > >>>>>> Yes, there are techniques to write watchdogs, but my question > >>>>>> was: is > >>>>>> it possible to prevent Erlang VM from crash? > >>>>>> > >>>>>> ________________________________________________________________ > >>>>>> erlang-questions mailing list. See http://www.erlang.org/faq.html > >>>>>> erlang-questions (at) erlang.org > >>>>>> > >>>>>> > >>>> > >>>> > >>>> ________________________________________________________________ > >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html > >>>> erlang-questions (at) erlang.org > >>>> > >>>> > >>> > >>> > >>> Este correo no tiene dibujos. Las formas extra?as en la pantalla son > >>> letras. > >>> __________________________________________ > >>> > >>> Clist UAH a.k.a Angel > >>> __________________________________________ > >>> Warning: Microsoft_bribery.ISO contains OOXML code. > >>> > >>> ________________________________________________________________ > >>> erlang-questions mailing list. See http://www.erlang.org/faq.html > >>> erlang-questions (at) erlang.org > >>> > >> > >> > > > > > > > > -- > > No imprima este correo si no es necesario. El medio ambiente est? en > > nuestras manos. > > __________________________________________ > > > > Clist UAH a.k.a Angel > > __________________________________________ > > China 'limpia' el Tibet para las Olimpiadas. > > -- Este correo no tiene dibujos. Las formas extra?as en la pantalla son letras. __________________________________________ Clist UAH a.k.a Angel __________________________________________ Artista -- (internet) --> Usuario final. As? los artistas cobran m?s y dicen menos paridas sobre lo que creen que es la pirater?a. From kagato@REDACTED Mon Nov 9 18:38:59 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Mon, 9 Nov 2009 09:38:59 -0800 Subject: AW: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: <70D081D9-DF8A-4079-9CD1-4B4BD1C31DCD@souja.net> This would be a very useful tunable. On Nov 9, 2009, at 2:31 AM, Decker, Nils wrote: > Hello, > > I had a few occasions during developing that a runaway process > caused my machine to swap and gind to a halt. (i should have used > ulimit on beam) > > I wondered why there is no way to limit the size of a single > process. It could be a simple option to spawn to limit the heap size > of a process. (like ulimit) > The process gets killed if it ever grows beyond the limit. A limit > for the size of the message queue would be nice too, because a > process with a few thousand entries is (most of the time) not going > to cope with it anyway. > A similar limit for ETS tables would be possible too. A memory limit > for shared resources (large binaries) would be more difficult. > > Nils Decker -- Jayson Vantuyl kagato@REDACTED From kagato@REDACTED Mon Nov 9 18:40:55 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Mon, 9 Nov 2009 09:40:55 -0800 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: <45953559-AE63-4DFF-B2C2-73191F2ABF91@souja.net> You could run two Erlang VMs (thus, keeping the HTTP connection and its state separate). Then the crasher wouldn't take the other one down. One with the processing and one with the networking, and have them communicate via distributed Erlang. You should still ulimit beam to keep the other one from getting OOM-killed though. Assuming you can get the throughput you need, this also would decompose some of the components, which might helps scale horizontally later. On Nov 8, 2009, at 9:08 PM, Max Lapshin wrote: > Look, I have a program, that reads MPEG TS stream from network. There > is some bug in my code > and my ts_lander crashes beam. How is it possible to store state of > external HTTP stream?? > > If Erlang VM wouldn't crash, but only kill the leaking process, I > could buffer stream in other process and feed > new mpeg ts lander after restart. But it is impossible =( > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From kagato@REDACTED Mon Nov 9 18:42:39 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Mon, 9 Nov 2009 09:42:39 -0800 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: <23132EFE-21FD-449B-990A-2A9D395AC562@souja.net> The problem is that there is no way for the VM to really know which process is leaking. The Linux OOM-killer has gone through a bunch of rewrites and it still doesn't always get it right. It might be possible to have a threshold trigger that would hit some sort of global alarm that could be application specific. That's probably the closest you could get though. On Nov 8, 2009, at 9:08 PM, Max Lapshin wrote: > Look, I have a program, that reads MPEG TS stream from network. There > is some bug in my code > and my ts_lander crashes beam. How is it possible to store state of > external HTTP stream?? > > If Erlang VM wouldn't crash, but only kill the leaking process, I > could buffer stream in other process and feed > new mpeg ts lander after restart. But it is impossible =( > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From torben.lehoff@REDACTED Mon Nov 9 21:31:14 2009 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 9 Nov 2009 21:31:14 +0100 Subject: [erlang-questions] Re: C Node and {'EXIT', Pid, noconnection} in Erlang In-Reply-To: References: Message-ID: Hi Max, I am doing some investigations into ports and nodes and ran across your thread here which nobody seems to have answered, so I will venture a guess. noconnection means that there is no connection to the node and since you are linking (in the original code) this will cause an EXIT signal with reason noconnection. So your ObjC node is probably not running in the eyes of the Erlang node. Why? Hard for me to see, but this is how I would interpret the error messages, perhaps others can shed some more light on what could cause this problem. Cheers, Torben On Tue, Oct 6, 2009 at 10:43, Max Lapshin wrote: > Problem disappeared when I remove link(NetStream) from erlang code. > It is strange for me, because no LINK messages haven't been sent to ObjC > node > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- http://www.linkedin.com/in/torbenhoffmann From tamas.nagy@REDACTED Mon Nov 9 22:42:29 2009 From: tamas.nagy@REDACTED (Tamas Nagy) Date: Mon, 9 Nov 2009 21:42:29 +0000 Subject: [erlang-questions] Dialyzer problem on OSX 10.6 (Erlang R13B02-1) In-Reply-To: <7BEEE594-5718-413E-BD77-9B1D6A37F0E2@gmail.com> References: <16321682.23851257766518669.JavaMail.root@zimbra> <7BEEE594-5718-413E-BD77-9B1D6A37F0E2@gmail.com> Message-ID: <75D3132C-E532-4DE1-A6B0-FAAEE5087E80@erlang-consulting.com> Hi everyone, The problem has been solved in homebrew. Aggressive optimisation for some reason doesn't go well with dialyzer. The error would probably have shown its ugly head in different applications as well. dialyzer was only the unlucky one to reliable trigger it. For all the gory details: http://github.com/mxcl/homebrew/issues/closed/#issue/120 Tamas Nagy Erlang Training & Consulting http://www.erlang-consulting.com On 9 Nov 2009, at 11:50, Tino Breddin wrote: > Hi Tamas, > > Yes I used homebrew as well. That sounds strange, might it be that > those optimizations have a negative impact on the Erlang performance > on Mac OSX? > > Cheers, > Tino > > On Nov 9, 2009, at 11:35 AM, Tamas Nagy wrote: > >> Hi Tino, >> >> Are you using homebrew? >> >> So far it seems that homebrew applies some c compiler magic (e.g >> optimizations) when building erlang which somehow borks dialyzer >> (strange) as compiling from source with the same ./configure >> options homebrew uses results in an erlang installation which works >> like a charm. >> >> Regards, >> Tamas >> ----- "Tino Breddin" wrote: >> >>> Hey Tamas, >>> >>> Here's the output for the command run on my machine (Macbook Pro >>> Core >>> >>> Duo, running OSX 10.6.1 32-bit): >>> >>> dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/ >>> erlang/ >>> >>> lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/ >>> lib/ >>> >>> kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/ >>> lib/ >>> >>> stdlib-1.16.3/ebin/ >>> Creating PLT /Users/tino/.dialyzer_plt ... >>> Unknown functions: >>> compile:file/2 >>> compile:forms/2 >>> compile:noenv_forms/2 >>> compile:output_generated/1 >>> crypto:des3_cbc_decrypt/5 >>> crypto:start/0 >>> done in 46m22.52s >>> done (passed successfully) >>> >>> Apparently it takes some time to complete but you'll get there. >>> >>> Cheers, >>> Tino >>> >>> On Nov 8, 2009, at 11:35 PM, Tamas Nagy wrote: >>> >>>> Hi Kostis, >>>> >>>> This is what all I get: >>>> $ dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/ >>>> erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/ >>>> erlang/lib/kernel-2.13.3/ebin/ >>> /usr/local/Cellar/erlang/R13B02-1/lib/ >>>> erlang/lib/stdlib-1.16.3/ebin/ >>>> Creating PLT /Users/lestat/.dialyzer_plt ... >>>> >>>> I will try to rebuild erlang with disabling hipe to see if that >>> helps. >>>> >>>> I am happy to run some tests/traces you would like to see the >>>> results of to help. >>>> >>>> Regards, >>>> Tamas >>>> >>>> Tamas Nagy >>>> Erlang Training & Consulting >>>> http://www.erlang-consulting.com >>>> >>>> On 8 Nov 2009, at 19:52, Kostis Sagonas wrote: >>>> >>>>> Tamas Nagy wrote: >>>>>> Hi, >>>>>> Am I the only one who cannot build a plt under OSX 10.6 in >>>>>> R13B02-1? >>>>>> No matter what applications I specify for the initial plt it just >>> >>>>>> hangs eating processing power but not producing anything. >>>>>> Erlang was installed with homebrew, but that shouldn't matter. Or >>> >>>>>> maybe it does. (beam is compiled for 64 bit) >>>>>> Here is the command I used: >>>>>> dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/ >>>>>> erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/ >>>>>> lib/ >>> >>>>>> erlang/lib/kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/ >>> >>>>>> lib/erlang/lib/stdlib-1.16.3/ebin/ >>>>>> I gave waiting for it to finish after ~1 hour (on an old type >>>>>> white MacBook (MacBook2,1) with 2GB of RAM). >>>>> >>>>> None of the dialyzer developers has access to such a machine to >>>>> see >>> >>>>> what's happening. Perhaps somebody else in that list might want >>>>> to >>> >>>>> report whether this is reproducible or not. >>>>> >>>>> However, perhaps the following might help: when building the PLT, >>> >>>>> do you see a message: >>>>> >>>>> Compiling some key modules to native code... >>>>> >>>>> If so, can you disable hipe (64-bit Mac is not a platform we have >>> >>>>> ever tested due to no access to such a machine) to see whether >>>>> this >>> >>>>> may be the culprit? >>>>> >>>>> Kostis >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >> >> -- >> Tamas Nagy >> Erlang Training & Consulting >> http://www.erlang-consulting.com >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > From tamas.nagy@REDACTED Mon Nov 9 23:50:04 2009 From: tamas.nagy@REDACTED (Tamas Nagy) Date: Mon, 9 Nov 2009 22:50:04 +0000 Subject: [erlang-questions] Dialyzer problem on OSX 10.6 (Erlang R13B02-1) In-Reply-To: <75D3132C-E532-4DE1-A6B0-FAAEE5087E80@erlang-consulting.com> References: <16321682.23851257766518669.JavaMail.root@zimbra> <7BEEE594-5718-413E-BD77-9B1D6A37F0E2@gmail.com> <75D3132C-E532-4DE1-A6B0-FAAEE5087E80@erlang-consulting.com> Message-ID: Hi, I have been misinforming the mailing list here (unintentionally, lack of information is the reason). The problem is not with aggressive optimisation or it depends how you look at it as homebrew on Snow Leopard uses the new llvm compiler as the default c compiler. This in case of erlang did not go too well. If somebody has the urge to get llvm compiled erlang working this is one issue which has to be solved. Tamas Nagy Erlang Training & Consulting http://www.erlang-consulting.com On 9 Nov 2009, at 21:42, Tamas Nagy wrote: > Hi everyone, > > The problem has been solved in homebrew. Aggressive optimisation for > some reason doesn't go well with dialyzer. > The error would probably have shown its ugly head in different > applications as well. dialyzer was only the unlucky one to reliable > trigger it. > > For all the gory details: > http://github.com/mxcl/homebrew/issues/closed/#issue/120 > > Tamas Nagy > Erlang Training & Consulting > http://www.erlang-consulting.com > > On 9 Nov 2009, at 11:50, Tino Breddin wrote: > >> Hi Tamas, >> >> Yes I used homebrew as well. That sounds strange, might it be that >> those optimizations have a negative impact on the Erlang >> performance on Mac OSX? >> >> Cheers, >> Tino >> >> On Nov 9, 2009, at 11:35 AM, Tamas Nagy wrote: >> >>> Hi Tino, >>> >>> Are you using homebrew? >>> >>> So far it seems that homebrew applies some c compiler magic (e.g >>> optimizations) when building erlang which somehow borks dialyzer >>> (strange) as compiling from source with the same ./configure >>> options homebrew uses results in an erlang installation which >>> works like a charm. >>> >>> Regards, >>> Tamas >>> ----- "Tino Breddin" wrote: >>> >>>> Hey Tamas, >>>> >>>> Here's the output for the command run on my machine (Macbook Pro >>>> Core >>>> >>>> Duo, running OSX 10.6.1 32-bit): >>>> >>>> dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/ >>>> erlang/ >>>> >>>> lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/ >>>> lib/ >>>> >>>> kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/ >>>> lib/ >>>> >>>> stdlib-1.16.3/ebin/ >>>> Creating PLT /Users/tino/.dialyzer_plt ... >>>> Unknown functions: >>>> compile:file/2 >>>> compile:forms/2 >>>> compile:noenv_forms/2 >>>> compile:output_generated/1 >>>> crypto:des3_cbc_decrypt/5 >>>> crypto:start/0 >>>> done in 46m22.52s >>>> done (passed successfully) >>>> >>>> Apparently it takes some time to complete but you'll get there. >>>> >>>> Cheers, >>>> Tino >>>> >>>> On Nov 8, 2009, at 11:35 PM, Tamas Nagy wrote: >>>> >>>>> Hi Kostis, >>>>> >>>>> This is what all I get: >>>>> $ dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/ >>>>> erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/ >>>>> erlang/lib/kernel-2.13.3/ebin/ >>>> /usr/local/Cellar/erlang/R13B02-1/lib/ >>>>> erlang/lib/stdlib-1.16.3/ebin/ >>>>> Creating PLT /Users/lestat/.dialyzer_plt ... >>>>> >>>>> I will try to rebuild erlang with disabling hipe to see if that >>>> helps. >>>>> >>>>> I am happy to run some tests/traces you would like to see the >>>>> results of to help. >>>>> >>>>> Regards, >>>>> Tamas >>>>> >>>>> Tamas Nagy >>>>> Erlang Training & Consulting >>>>> http://www.erlang-consulting.com >>>>> >>>>> On 8 Nov 2009, at 19:52, Kostis Sagonas wrote: >>>>> >>>>>> Tamas Nagy wrote: >>>>>>> Hi, >>>>>>> Am I the only one who cannot build a plt under OSX 10.6 in >>>>>>> R13B02-1? >>>>>>> No matter what applications I specify for the initial plt it >>>>>>> just >>>> >>>>>>> hangs eating processing power but not producing anything. >>>>>>> Erlang was installed with homebrew, but that shouldn't matter. >>>>>>> Or >>>> >>>>>>> maybe it does. (beam is compiled for 64 bit) >>>>>>> Here is the command I used: >>>>>>> dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/ >>>>>>> erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/ >>>>>>> lib/ >>>> >>>>>>> erlang/lib/kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/ >>>>>>> R13B02-1/ >>>> >>>>>>> lib/erlang/lib/stdlib-1.16.3/ebin/ >>>>>>> I gave waiting for it to finish after ~1 hour (on an old type >>>>>>> white MacBook (MacBook2,1) with 2GB of RAM). >>>>>> >>>>>> None of the dialyzer developers has access to such a machine to >>>>>> see >>>> >>>>>> what's happening. Perhaps somebody else in that list might >>>>>> want to >>>> >>>>>> report whether this is reproducible or not. >>>>>> >>>>>> However, perhaps the following might help: when building the PLT, >>>> >>>>>> do you see a message: >>>>>> >>>>>> Compiling some key modules to native code... >>>>>> >>>>>> If so, can you disable hipe (64-bit Mac is not a platform we have >>>> >>>>>> ever tested due to no access to such a machine) to see whether >>>>>> this >>>> >>>>>> may be the culprit? >>>>>> >>>>>> Kostis >>>>> >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>>> erlang-questions (at) erlang.org >>>>> >>> >>> -- >>> Tamas Nagy >>> Erlang Training & Consulting >>> http://www.erlang-consulting.com >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> > From yogishb@REDACTED Tue Nov 10 02:15:52 2009 From: yogishb@REDACTED (Yogish Baliga) Date: Mon, 9 Nov 2009 17:15:52 -0800 (PST) Subject: Mnesia stuck in transaction Message-ID: <386468.98356.qm@web112610.mail.gq1.yahoo.com> Hello Mnesia experts, I have a process hung with the following backtrace. What is the possible solution? This gen_server (workflow_manager_logs) is backed up with lot of messages delivered via RabbitMQ. Either I need a way to cancel this transaction or fix the database so that it will continue processing all messages. Thanx, -- baliga (workflow_gateway@REDACTED)11> io:format("~s", [erlang:binary_to_list(element(2, erlang:process_info(whereis(workflow_manager_logs), backtrace)))]). Program counter: 0xf0717490 (mnesia_tm:rec/2 + 28) CP: 0x00000000 (invalid) arity = 0 0xefa18404 Return addr 0xf071b7c0 (mnesia_tm:execute_outer/6 + 68) y(0) #Ref<0.0.0.1407> y(1) <0.104.0> 0xefa18410 Return addr 0xf07fe6e8 (mnesia:wrap_trans/6 + 20) y(0) async y(1) infinity y(2) 1 y(3) [] y(4) #Fun y(5) mnesia_frag 0xefa1842c Return addr 0xf13b0448 (workflow_manager_logs:handle_info/2 + 312) 0xefa18430 Return addr 0xf7e13124 (gen_server:handle_msg/5 + 932) y(0) [] y(1) Catch 0xf13b0448 (workflow_manager_logs:handle_info/2 + 312) y(2) [] y(3) {<0.264.0>,<0.274.0>,<<33 bytes>>} 0xefa18444 Return addr 0xf7db1c6c (proc_lib:init_p_do_apply/3 + 28) y(0) workflow_manager_logs y(1) {<0.264.0>,<0.274.0>,<<33 bytes>>} y(2) workflow_manager_logs y(3) <0.38.0> y(4) {{'basic.deliver',<<33 bytes>>,7,false,<<15 bytes>>,<<24 bytes>>},{content,60,none,<<29 bytes>>,[<<75 bytes>>]}} y(5) Catch 0xf7e13124 (gen_server:handle_msg/5 + 932) 0xefa18460 Return addr 0x08236c00 () y(0) Catch 0xf7db1c7c (proc_lib:init_p_do_apply/3 + 44) "Point of view is worth 80 IQ points" --Alan Kay http://dudefrommangalore.blogspot.com/ From yogishb@REDACTED Tue Nov 10 02:52:50 2009 From: yogishb@REDACTED (Yogish Baliga) Date: Mon, 9 Nov 2009 17:52:50 -0800 (PST) Subject: [erlang-questions] Mnesia stuck in transaction In-Reply-To: <386468.98356.qm@web112610.mail.gq1.yahoo.com> References: <386468.98356.qm@web112610.mail.gq1.yahoo.com> Message-ID: <692503.79190.qm@web112615.mail.gq1.yahoo.com> Not sure what actually happened. But I found the work-around. changed the table type to ram_copies Stopped mnesia Started mnesia Once gen_server message queue is drained, changed the table type of disc_copies -- baliga "Point of view is worth 80 IQ points" --Alan Kay http://dudefrommangalore.blogspot.com/ ________________________________ From: Yogish Baliga To: Erlang Questions Sent: Mon, November 9, 2009 5:15:52 PM Subject: [erlang-questions] Mnesia stuck in transaction Hello Mnesia experts, I have a process hung with the following backtrace. What is the possible solution? This gen_server (workflow_manager_logs) is backed up with lot of messages delivered via RabbitMQ. Either I need a way to cancel this transaction or fix the database so that it will continue processing all messages. Thanx, -- baliga (workflow_gateway@REDACTED)11> io:format("~s", [erlang:binary_to_list(element(2, erlang:process_info(whereis(workflow_manager_logs), backtrace)))]). Program counter: 0xf0717490 (mnesia_tm:rec/2 + 28) CP: 0x00000000 (invalid) arity = 0 0xefa18404 Return addr 0xf071b7c0 (mnesia_tm:execute_outer/6 + 68) y(0) #Ref<0.0.0.1407> y(1) <0.104.0> 0xefa18410 Return addr 0xf07fe6e8 (mnesia:wrap_trans/6 + 20) y(0) async y(1) infinity y(2) 1 y(3) [] y(4) #Fun y(5) mnesia_frag 0xefa1842c Return addr 0xf13b0448 (workflow_manager_logs:handle_info/2 + 312) 0xefa18430 Return addr 0xf7e13124 (gen_server:handle_msg/5 + 932) y(0) [] y(1) Catch 0xf13b0448 (workflow_manager_logs:handle_info/2 + 312) y(2) [] y(3) {<0.264.0>,<0.274.0>,<<33 bytes>>} 0xefa18444 Return addr 0xf7db1c6c (proc_lib:init_p_do_apply/3 + 28) y(0) workflow_manager_logs y(1) {<0.264.0>,<0.274.0>,<<33 bytes>>} y(2) workflow_manager_logs y(3) <0.38.0> y(4) {{'basic.deliver',<<33 bytes>>,7,false,<<15 bytes>>,<<24 bytes>>},{content,60,none,<<29 bytes>>,[<<75 bytes>>]}} y(5) Catch 0xf7e13124 (gen_server:handle_msg/5 + 932) 0xefa18460 Return addr 0x08236c00 () y(0) Catch 0xf7db1c7c (proc_lib:init_p_do_apply/3 + 44) "Point of view is worth 80 IQ points" --Alan Kay http://dudefrommangalore.blogspot.com/ From ok@REDACTED Tue Nov 10 05:36:44 2009 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 10 Nov 2009 17:36:44 +1300 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> Message-ID: On Nov 9, 2009, at 10:53 PM, Kenneth Lundin wrote: > What we could do is to make it easier for the user to prevent OOM > situations and also to > let him take the decision when it occurs or rather before it occurs. > > One way would be to let the user set a memory quota on a process with > options at spawn time. When the process reaches it quota it can be > automatically killed or the user can > be notified in some way and take actions. One of the reasons this hasn't been done is, I presume, the fact that it is quite difficult for a programmer to determine what the memory quota should be. It depends on - the nature of the code being run - the data it is processing (data loads tend to increase with the years) - the cleverness of the compiler - the data representation used by the Erlang system (e.g., whether pointers are 32 bits or 64, whether there are special tags for small arrays - the data representation used by library modules the code calls (change the representation of ordered sets and a bound that might have worked might not any more) - a system policy on whether it's better to be tight on memory and tolerate some processes crashing and needing to be restarted or whether it's best to keep more processing running at the cost of using more memory, a thing that might change at run time. I have long bemoaned the (measured!) fact that the size of a stack frame in C can vary by a factor of 10, so that determining the sizes for POSIX thread stacks is a game of Russian Roulette. You can determine suitable sizes for a particular release by running experiments, but the sizes thus determined are ONLY reliable for the specific release or releases you tried on the machine or machines that you tried it or them on. Set a limit, and you can *EXPECT* working processes to be crashed as a *NORMAL* outcome. Once some feature goes in, I expect to see messages in this mailing list "My program gets lots of killed processes due to Out-Of-Memory but it used to work and I haven't changed it." From clist@REDACTED Tue Nov 10 10:08:22 2009 From: clist@REDACTED (Angel Alvarez) Date: Tue, 10 Nov 2009 10:08:22 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: Message-ID: <200911101008.22771.clist@uah.es> El Martes, 10 de Noviembre de 2009 05:36:44 Richard O'Keefe escribi?: > > On Nov 9, 2009, at 10:53 PM, Kenneth Lundin wrote: > > What we could do is to make it easier for the user to prevent OOM > > situations and also to > > let him take the decision when it occurs or rather before it occurs. > > > > One way would be to let the user set a memory quota on a process with > > options at spawn time. When the process reaches it quota it can be > > automatically killed or the user can > > be notified in some way and take actions. > > One of the reasons this hasn't been done is, I presume, the fact that > it is quite difficult for a programmer to determine what the memory > quota should be. It depends on > - the nature of the code being run > - the data it is processing (data loads tend to increase with the > years) > - the cleverness of the compiler > - the data representation used by the Erlang system (e.g., whether > pointers are 32 bits or 64, whether there are special tags for small > arrays > - the data representation used by library modules the code calls > (change the representation of ordered sets and a bound that might > have worked might not any more) > - a system policy on whether it's better to be tight on memory and > tolerate some processes crashing and needing to be restarted or > whether it's best to keep more processing running at the cost of > using more memory, a thing that might change at run time. > > I have long bemoaned the (measured!) fact that the size of a stack > frame in C can vary by a factor of 10, so that determining the sizes > for POSIX thread stacks is a game of Russian Roulette. > > You can determine suitable sizes for a particular release by running > experiments, but the sizes thus determined are ONLY reliable for the > specific release or releases you tried on the machine or machines > that you tried it or them on. Set a limit, and you can *EXPECT* > working processes to be crashed as a *NORMAL* outcome. > > Once some feature goes in, I expect to see messages in this mailing > list "My program gets lots of killed processes due to Out-Of-Memory > but it used to work and I haven't changed it." I agree with You Richard Most of our production java code throws millions of nullpointer exceptions messages and management staff, and developers dont care, if users are happy. I presume many subtle errors came from those nullpointer errors but people got used to have those messages when the moved from other platforms to java and now is the norm among java developers. Its not acceptable for a platform that focuses on highly scalable, survivable and error tolerant systems that programmers were acustommed to see a lot of memory errors. Everyone here knows how to measure memory available at program life-time (im new but, i think i knows ), there is no need to cover such poor memory management with artificial limits, this is no C and the like. Virtual memory and Garbage collection should garantee enough memory provided the programmers know something about the algorithms and data structures (good unit testing). Imagine someone asks a question on the list, provides the offending code and everyother who is helping sees lot of memory errors on his code.. Should i waste my time helping someone when his code spurts so much memory errors and hi/her doesnt care? Should newbies like me be exposed to such poor desing and coding style? What's better? counting bits properly or just set the limit and wait the process for a memory crash?? How do you manage diferent compilers code size, or 32/64bits overhead? IMHO its up to the programmer concise memory usage expectation and this work has to be done on desing fase, and not let some black magic, i pressume this wil not be portable across many erlang versions and setups... Clearly for memory battles on process/threads and the like we have already many good languages like C despite most people here seems to want the VM to became a better OS... Classic behavior (let the VM crash) should be preserved for legacy systems and for academic and marketing stuff... I agree, Shielding the VM againts many error condictions and you probably will generate as much poor programmers. /Angel PS: I agree with Joe, Erlang best feature is a new mental model of dealing with faulty conditions more than the "multicore" promises > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- Este correo no tiene dibujos. Las formas extra?as en la pantalla son letras. __________________________________________ Clist UAH a.k.a Angel __________________________________________ HONEY BUNNY - Any of you *uckin' pricks move and I'll execute every mother*ucking last one of you. From chandrashekhar.mullaparthi@REDACTED Tue Nov 10 12:18:04 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 10 Nov 2009 11:18:04 +0000 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <55BD1E0E-AA8A-462C-8BF0-08D1F5EAE1CC@rogvall.se> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <55BD1E0E-AA8A-462C-8BF0-08D1F5EAE1CC@rogvall.se> Message-ID: 2009/11/9 Tony Rogvall > Interesting discussion! > > I have been working on a resource system for Erlang for nearly two years > now. > I have a working (tm) prototype where you can set resource limits like > max_processes/max_ports/max_memory/max_time/max_reductions ... > You don't happen to have a max_message_q_size option do you? Or does max_memory implicitly enforce this? Chandru From ulf.wiger@REDACTED Tue Nov 10 13:10:24 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 10 Nov 2009 13:10:24 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> Message-ID: <4AF95830.8060508@erlang-consulting.com> Richard O'Keefe wrote: >> One way would be to let the user set a memory quota on a process with >> options at spawn time. When the process reaches it quota it can be >> automatically killed or the user can >> be notified in some way and take actions. > > One of the reasons this hasn't been done is, I presume, the fact that > it is quite difficult for a programmer to determine what the memory > quota should be. It depends on > ... I implemented resource limits in erlhive - at the Erlang level rather than in the VM. The purpose was to be able to run foreign code safely in a hosted environment. Eliminating the possibility to do damage through traditional side-effects was relatively easy with a code transform, but two ways of staging a DoS attack would be to gobble RAM or CPU capacity. I approached this by inserting calls to a check function that sampled heap size, and started a "watchdog" process that would unceremoniously kill the program after a certain time. In short, I can see a need for such limits, and would like to include a reduction ceiling. The limits could be set after careful testing and high enough that they protect against runaway processes. A reduction limit could be checked at the end of each slice, perhaps. In my experience, per-process memory usage is fairly predictable in erlang. Does anyone have a different experience? BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From chandrashekhar.mullaparthi@REDACTED Tue Nov 10 13:25:54 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 10 Nov 2009 12:25:54 +0000 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <55BD1E0E-AA8A-462C-8BF0-08D1F5EAE1CC@rogvall.se> Message-ID: 2009/11/10 Tony Rogvall > Not yet. > > What kind of flavor do you have in mind? > > One proposal is to let sender crash when receiver in box is full. > A few strategies to choose from would be useful. * Crash sender * Crash receiver * Discard message cheers Chandru I would like to test a blocking version as well. This may sound like utterly > crazy, but we already have the infamous busy_port ;-) > Busy ports do block senders (mostly distribution stuff). > An other flavor is to drop the overflow message. This can however be done > by catch (Pid ! Message) if the crash semantics is implemented. > > A suggestion from Patrik Winroth is to be able to set thresholds on > resource limits, > say in percent from max limit. When threshold is reached then a warning > message > is sent to a logger process. > > The max_memory limit currently control number of words! that can be > allocated for heaps. > This is the sum of sizes of all heaps for the process and it's > "sub"-processes > (spawned by process that had the limit set in the first place). It is > implementation specific > if the message queue and/or the messages them selfs are located on the > process heap. > In the case where messages are none shared and located on a separate heap > fragments > this should be accounted for. > > /Tony > > > > > > On 10 nov 2009, at 12.18, Chandru wrote: > > 2009/11/9 Tony Rogvall >> >> Interesting discussion! >>> >>> I have been working on a resource system for Erlang for nearly two years >>> now. >>> I have a working (tm) prototype where you can set resource limits like >>> max_processes/max_ports/max_memory/max_time/max_reductions ... >>> >>> >> You don't happen to have a max_message_q_size option do you? Or does >> max_memory implicitly enforce this? >> >> Chandru >> > > From erlang@REDACTED Tue Nov 10 13:45:17 2009 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 10 Nov 2009 13:45:17 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <4AF95830.8060508@erlang-consulting.com> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <4AF95830.8060508@erlang-consulting.com> Message-ID: <9b08084c0911100445m11c77393y5646ca5a937d2176@mail.gmail.com> This is a very interesting problem. if processes have quotas, then how could you set the quota value? A perfectly correct process might just have a very deep stack, just once in its life and otherwise be fine. Whether to crash this process or not would depend upon what the other processes in the system happened to be doing at the time. This would be very unfortunate - it's like your program being hit by a cosmic ray - nasty. It creates a random non-deterministic coupling between things that are supposed to be independent. A possibility that just occurred to me might be to suspend processes that appear to be running wild until such a time as the overall memory situation looks good. Image two scheduler queues. One for well behaved programs. One for programs whose stacks and heaps are growing too fast. If memory is no problem we run programs from both queues. If memory is tight we run processes in the "problem" queue less often and with frequent garbs. Killing a program with a large stack and heap, just because their happens to be a temporary memory problem seems horrible, especially since the problem might go away if we wait a few milliseconds. Suspending a memory hungry process for a while, until memory is available seems less objectionable. Perhapse it could be swapped out to disk and pulled in a lot later. Killing things at random in the hope it might help sounds like a really bad idea. Process migration could solve this - move it to a machine that has got more memory. Suspending things seems ok - you might even suspend an errant process forever and reclaim the memory - but not kill it. Some other process could detect that the processes is not responding and kill it and thus all the semantics of the application would be obeyed (processes are allowed to be unresponsive, that's fine) and the semantics of the error recovery should say what to do in this case. Just killing processes when they have done nothing wrong is not a good idea. /Joe On Tue, Nov 10, 2009 at 1:10 PM, Ulf Wiger wrote: > Richard O'Keefe wrote: >>> >>> One way would be to let the user set a memory quota on a process with >>> options at spawn time. When the process reaches it quota it can be >>> automatically killed or the user can >>> be notified in some way and take actions. >> >> One of the reasons this hasn't been done is, I presume, the fact that >> it is quite difficult for a programmer to determine what the memory >> quota should be. ?It depends on >> ?... > > I implemented resource limits in erlhive - at the Erlang level rather > than in the VM. The purpose was to be able to run foreign code safely > in a hosted environment. Eliminating the possibility to do damage > through traditional side-effects was relatively easy with a code > transform, but two ways of staging a DoS attack would be to gobble > RAM or CPU capacity. I approached this by inserting calls to a check > function that sampled heap size, and started a "watchdog" process that > would unceremoniously kill the program after a certain time. > > In short, I can see a need for such limits, and would like to include > a reduction ceiling. The limits could be set after careful testing > and high enough that they protect against runaway processes. A reduction > limit could be checked at the end of each slice, perhaps. > > In my experience, per-process memory usage is fairly predictable in > erlang. Does anyone have a different experience? > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From tony@REDACTED Tue Nov 10 13:04:48 2009 From: tony@REDACTED (Tony Rogvall) Date: Tue, 10 Nov 2009 13:04:48 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <55BD1E0E-AA8A-462C-8BF0-08D1F5EAE1CC@rogvall.se> Message-ID: Not yet. What kind of flavor do you have in mind? One proposal is to let sender crash when receiver in box is full. I would like to test a blocking version as well. This may sound like utterly crazy, but we already have the infamous busy_port ;-) Busy ports do block senders (mostly distribution stuff). An other flavor is to drop the overflow message. This can however be done by catch (Pid ! Message) if the crash semantics is implemented. A suggestion from Patrik Winroth is to be able to set thresholds on resource limits, say in percent from max limit. When threshold is reached then a warning message is sent to a logger process. The max_memory limit currently control number of words! that can be allocated for heaps. This is the sum of sizes of all heaps for the process and it's "sub"- processes (spawned by process that had the limit set in the first place). It is implementation specific if the message queue and/or the messages them selfs are located on the process heap. In the case where messages are none shared and located on a separate heap fragments this should be accounted for. /Tony On 10 nov 2009, at 12.18, Chandru wrote: > 2009/11/9 Tony Rogvall > >> Interesting discussion! >> >> I have been working on a resource system for Erlang for nearly two >> years >> now. >> I have a working (tm) prototype where you can set resource limits >> like >> max_processes/max_ports/max_memory/max_time/max_reductions ... >> > > You don't happen to have a max_message_q_size option do you? Or does > max_memory implicitly enforce this? > > Chandru From kenneth.lundin@REDACTED Tue Nov 10 14:13:02 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 10 Nov 2009 14:13:02 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <9b08084c0911100445m11c77393y5646ca5a937d2176@mail.gmail.com> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <4AF95830.8060508@erlang-consulting.com> <9b08084c0911100445m11c77393y5646ca5a937d2176@mail.gmail.com> Message-ID: On Tue, Nov 10, 2009 at 1:45 PM, Joe Armstrong wrote: > This is a very interesting problem. if processes have quotas, then how > could you set the > quota value? If you for example have processes representing subscribers och half-calls or mobile phones it is pretty easy to know a reasonable size of such a process by measurement. It should be easy to say if it can grow up to 100 kwords, 1Mwords, 10 Mwords etc. Of course you should not try to set quota as tight as possible. The quota on process size is just a way to protect the system from total disaster if one of these processes happens to grow over all reasonable limits because of a bug. /Kenneth Erlang/OTP Ericsson > > ?A perfectly correct process might just have a very deep stack, just once in its > life and otherwise be fine. Whether to crash this process or not would > depend upon > what the other processes in the system happened to be doing at the > time. This would > be very unfortunate - it's like your program being hit by a cosmic ray > - nasty. It creates a random non-deterministic coupling between things > that are supposed to be independent. > > A possibility that just occurred to me might be to suspend processes > that appear to be > running wild until such a time as the overall memory situation looks good. > > Image two scheduler queues. One for well behaved programs. One for > programs whose > stacks and heaps are growing too fast. If memory is no problem we run > programs from both queues. If memory is tight we run processes in the > "problem" queue less often and with frequent garbs. > > Killing a program with a large stack and heap, just because their > happens to be a > temporary memory problem seems horrible, especially since the problem > might go away > if we wait a few milliseconds. > > Suspending a memory hungry process for a while, until memory is available seems > less objectionable. Perhapse it could be swapped out to disk and > pulled in a lot later. > Killing things at random in the hope it might help sounds like a > really bad idea. > Process migration could solve this - move it to a machine that has got > more memory. > > Suspending things seems ok - you might even suspend an errant process forever > and reclaim the memory - but not kill it. Some other process could > detect that the processes > is not responding and kill it and thus all the semantics of the > application would be obeyed > (processes are allowed to be unresponsive, that's fine) and the semantics of the > error recovery should say what to do in this case. > > Just killing processes when they have done nothing wrong is not a good idea. > > /Joe > > > > On Tue, Nov 10, 2009 at 1:10 PM, Ulf Wiger > wrote: >> Richard O'Keefe wrote: >>>> >>>> One way would be to let the user set a memory quota on a process with >>>> options at spawn time. When the process reaches it quota it can be >>>> automatically killed or the user can >>>> be notified in some way and take actions. >>> >>> One of the reasons this hasn't been done is, I presume, the fact that >>> it is quite difficult for a programmer to determine what the memory >>> quota should be. ?It depends on >>> ?... >> >> I implemented resource limits in erlhive - at the Erlang level rather >> than in the VM. The purpose was to be able to run foreign code safely >> in a hosted environment. Eliminating the possibility to do damage >> through traditional side-effects was relatively easy with a code >> transform, but two ways of staging a DoS attack would be to gobble >> RAM or CPU capacity. I approached this by inserting calls to a check >> function that sampled heap size, and started a "watchdog" process that >> would unceremoniously kill the program after a certain time. >> >> In short, I can see a need for such limits, and would like to include >> a reduction ceiling. The limits could be set after careful testing >> and high enough that they protect against runaway processes. A reduction >> limit could be checked at the end of each slice, perhaps. >> >> In my experience, per-process memory usage is fairly predictable in >> erlang. Does anyone have a different experience? >> >> BR, >> Ulf W >> -- >> Ulf Wiger >> CTO, Erlang Training & Consulting Ltd >> http://www.erlang-consulting.com >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ulf.wiger@REDACTED Tue Nov 10 14:19:55 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 10 Nov 2009 14:19:55 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <55BD1E0E-AA8A-462C-8BF0-08D1F5EAE1CC@rogvall.se> Message-ID: <4AF9687B.7000805@erlang-consulting.com> Tony Rogvall wrote: > Not yet. > > What kind of flavor do you have in mind? > > One proposal is to let sender crash when receiver in box is full. I would prefer it if messages are simply dropped. This would be symmetric with the distributed case - you can hardly crash a remote process, or a port in active mode. :) BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ulf.wiger@REDACTED Tue Nov 10 14:22:52 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 10 Nov 2009 14:22:52 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <55BD1E0E-AA8A-462C-8BF0-08D1F5EAE1CC@rogvall.se> Message-ID: <4AF9692C.2000703@erlang-consulting.com> Tony Rogvall wrote: > Not yet. > > What kind of flavor do you have in mind? > > One proposal is to let sender crash when receiver in box is full. > > I would like to test a blocking version as well. This may sound like > utterly crazy, but we already have the infamous busy_port ;-) ...and the punishing of senders if the receiver has a long message queue. I think these are abominations in the multicore world, and should be phased out. > A suggestion from Patrik Winroth is to be able to set thresholds on > resource limits, say in percent from max limit. This could be done as an extension of erlang:system_monitor(). It already has the ability to trigger on large heap and long GC, etc. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ulf.wiger@REDACTED Tue Nov 10 14:40:52 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 10 Nov 2009 14:40:52 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <9b08084c0911100445m11c77393y5646ca5a937d2176@mail.gmail.com> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <4AF95830.8060508@erlang-consulting.com> <9b08084c0911100445m11c77393y5646ca5a937d2176@mail.gmail.com> Message-ID: <4AF96D64.70305@erlang-consulting.com> Joe Armstrong wrote: > > Just killing processes when they have done nothing wrong is not a good idea. Well, it's optional, of course. :) Imagine, OTOH, a well-tested system where memory characteristics have been well charted for the foreseeable cases. It might be defensible to set resource limits so that everything we expect to see falls well within the limit, and stuff that we don't expect might trigger the killing of some process. If this is done on temporary processes, we should be able to accept it as long as the number of spurious kills is low. This is not much stranger than things that we do routinely in other cases: - If dets or disk_log notice that a file hasn't been properly closed, it 'repairs' the file - that is, it repairs the index. Corrupt objects are simply discarded, not repaired. - Replication in the AXD 301 and similar products was asynchronous with a bulking factor. Some failure cases could lead to dropped calls, but as long as they were few, it was acceptable. - Some complex state machines would bail out for unexpected sequences (I showed an example of this in my Structured Network Programming talk at EUC). This was a form of "complexity overload", and hugely unfair to the poor process running the code, as it was probably not a real failure case. - Mnesia's deadlock prevention algorithm, or indeed any deadlock prevention algo, will restart transactions if there is even the smallest chance of deadlock. Granted, this should be transparent if the transaction fun is well written, but there will be false positives, and this will affect performance. On the other hand, there can be situations where a rogue process gobbles up all available memory, rendering the VM unresponsive for several minutes (e.g. due to the infamous "loss of sharing"), or cases where a number of unexpectedly large processes "gang up" and kill the VM in one big memory spike. Or a difficult-to-reproduce bug that sends some application into an infinite retry situation rendering the system unusable. In all these cases, killing off the poor culprits, guilty or not, may well result in a less deadly disturbance for the system as a whole. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From saleyn@REDACTED Tue Nov 10 17:09:46 2009 From: saleyn@REDACTED (Serge Aleynikov) Date: Tue, 10 Nov 2009 11:09:46 -0500 Subject: [erlang-questions] Re: SCTP accept? In-Reply-To: <7bb6ca9f-460b-4941-a421-cdd03910ca52@a21g2000yqc.googlegroups.com> References: <7bb6ca9f-460b-4941-a421-cdd03910ca52@a21g2000yqc.googlegroups.com> Message-ID: <4AF9904A.7030602@aleynikov.org> On the contrary to TCP where a client socket corresponds to a single communication channel to be used between the server/client pair, with SCTP (when using one-to-many style interface, which is what's currently supported in the SCTP driver) you have one SCTP socket to work with and one association per client end-point. You can send replies from more than one Erlang server-side worker process as long as you pass it the Socket::sctp_socket() and AssocID::assoc_id() (the AssocID comes in the #sctp_assoc_change{} event when a new client association is established). Then you can use gen_sctp:send/3 or gen_sctp:send/4 functions to send your HTTP payload. Serge Steve Davis wrote: > I must be incredibly dense! Many words later, I'm still 100% unclear > as to how to use gen_sctp effectively to do "http over sctp". > > The documentation shows an > "Example of an Erlang SCTP Server which receives SCTP messages and > prints them on the standard output." > ...but does not really address how you reply unless you do everything > from the one server process. > > To have any chance of scaling I would think that you'd want to process > client requests in a number of worker processes at the server side > (obviously)? > > With tcp you hand over the socket you get from accept -- but I'm > entirely unclear what the equivalent of controlling_process(Socket, > Pid) would be for associations? > > Any education would be very gratefully "accepted"! > > Best, > Steve > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From jvliwanag@REDACTED Tue Nov 10 13:35:17 2009 From: jvliwanag@REDACTED (Jan Vincent) Date: Tue, 10 Nov 2009 20:35:17 +0800 Subject: Best way to implement a simple cache Message-ID: <3F36CDB2-757D-4204-920C-4B1307AC1DFB@gmail.com> Hi guys, I wanted your take on how to implement a simple small key-value cache, maybe holding around a few hundred tuples. The thing is, I don't want the cache to be consuming all my memory so that entries in the cache expires if it isn't read for some time or some maximum size is met. Jan Vincent Liwanag jvliwanag@REDACTED From freza@REDACTED Tue Nov 10 18:25:40 2009 From: freza@REDACTED (Jachym Holecek) Date: Tue, 10 Nov 2009 18:25:40 +0100 Subject: [erlang-questions] Best way to implement a simple cache In-Reply-To: <3F36CDB2-757D-4204-920C-4B1307AC1DFB@gmail.com> References: <3F36CDB2-757D-4204-920C-4B1307AC1DFB@gmail.com> Message-ID: <20091110172540.GA26341@hanele> Hello, # Jan Vincent 2009-11-10: > I wanted your take on how to implement a simple small key-value > cache, maybe holding around a few hundred tuples. The thing is, I > don't want the cache to be consuming all my memory so that entries > in the cache expires if it isn't read for some time or some maximum > size is met. An ETS table owned by a gen_server that runs periodic cleanup on it? -- Jachym From g@REDACTED Tue Nov 10 18:45:51 2009 From: g@REDACTED (Garrett Smith) Date: Tue, 10 Nov 2009 11:45:51 -0600 Subject: [erlang-questions] Best way to implement a simple cache In-Reply-To: <20091110172540.GA26341@hanele> References: <3F36CDB2-757D-4204-920C-4B1307AC1DFB@gmail.com> <20091110172540.GA26341@hanele> Message-ID: On Tue, Nov 10, 2009 at 11:25 AM, Jachym Holecek wrote: > Hello, > > # Jan Vincent 2009-11-10: >> I wanted your take on how to implement a simple small key-value >> cache, maybe holding around a few hundred tuples. The thing is, I >> don't want the cache to be consuming all my memory so that entries >> in the cache expires if it isn't read for some time or some maximum >> size is met. > > An ETS table owned by a gen_server that runs periodic cleanup on it? > I find myself writing purpose built gen_servers that maintain the cache their state for this sort of thing. To 'expire' items in the cache, you could run another process as a timer that calls an expire method on the gen_server. There are more moving parts here, but they're decoupled and avoid using ETS for what's a pretty simple caching requirement. Garrett From clist@REDACTED Tue Nov 10 20:10:13 2009 From: clist@REDACTED (Angel Alvarez) Date: Tue, 10 Nov 2009 20:10:13 +0100 Subject: package namespaces question Message-ID: <200911102010.14130.clist@uah.es> Hi Guys! Im currently reading erlang archives from 2004 On 10/05/2004 Someone talked about package namespaces ("Is erlang too small" thread) so i decided to just test this "new2 erlang knowledge. iv just borrowed some little code from a gs article... :-) the module starts like this: -module(gui). -export([init/0]). init() -> S = gs:start(), Win = gs:create(window, S, [{width, 350}, {height, 100}]), gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]), ... so on the erlang shell gui:init(). just runs the example.. So i changed the module to use package style names... -module(angel.tests.gui). -export([init/0]). init() -> S = gs:start(), Win = gs:create(window, S, [{width, 350}, {height, 100}]), gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]), ... and moved the gui.erl to $PWD/angel/test/gui.erl > sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code/angel/test> erlc gui.erl > cd ../../ > sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code/ but now firing at the shell results in : sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code> erl Erlang R13B01 (erts-5.7.2) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) 1> angel.test.gui:init(). ** exception error: undefined function 'angel.test.gs':start/0 in function 'angel.test.gui':init/0 What's wrong? Is needed the same convention on the rest of modules after you start using package name styles? From http://www.erlang.se/publications/packages.html "m2:g(X) becomes a call to foo.bar.m2 If this is not what was intended, the call can be written explicitly, as in -module(foo.bar.m1). -export([f/1]). f(X) -> fee.fie.foe.m2:g(X)." it seems bit embarrasing, i think is why perhaps no one is using this feature 5 years later... Thanks Angel -- No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. ->>----------------------------------------------- Clist UAH a.k.a Angel ---------------------------------[www.uah.es]-<<-- T? lo compras, yo lo copio. Todo legal. From mpquique@REDACTED Tue Nov 10 20:39:32 2009 From: mpquique@REDACTED (Enrique Marcote) Date: Tue, 10 Nov 2009 20:39:32 +0100 Subject: [erlang-questions] package namespaces question In-Reply-To: <200911102010.14130.clist@uah.es> References: <200911102010.14130.clist@uah.es> Message-ID: Hola Angel, Try importing gs. Saludos, Enrique El 10/11/2009, a las 20:10, Angel Alvarez escribi?: > Hi Guys! > > Im currently reading erlang archives from 2004 > > On 10/05/2004 Someone talked about package namespaces ("Is erlang too small" thread) so i decided to just test this "new2 erlang knowledge. > > iv just borrowed some little code from a gs article... :-) > > the module starts like this: > > > -module(gui). > -export([init/0]). > > init() -> > S = gs:start(), > Win = gs:create(window, S, [{width, 350}, {height, 100}]), > gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]), > ... > > so on the erlang shell gui:init(). just runs the example.. > So i changed the module to use package style names... > > > -module(angel.tests.gui). > -export([init/0]). > > init() -> > S = gs:start(), > Win = gs:create(window, S, [{width, 350}, {height, 100}]), > gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]), > ... > > and moved the gui.erl to $PWD/angel/test/gui.erl > > >> sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code/angel/test> erlc gui.erl >> cd ../../ >> sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code/ > > but now firing at the shell results in : > > sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code> erl > Erlang R13B01 (erts-5.7.2) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.7.2 (abort with ^G) > 1> angel.test.gui:init(). > ** exception error: undefined function 'angel.test.gs':start/0 > in function 'angel.test.gui':init/0 > > What's wrong? > Is needed the same convention on the rest of modules > after you start using package name styles? > > > From http://www.erlang.se/publications/packages.html > > "m2:g(X) becomes a call to foo.bar.m2 > > If this is not what was intended, the call can be written explicitly, as in > -module(foo.bar.m1). > -export([f/1]). > > f(X) -> fee.fie.foe.m2:g(X)." > > it seems bit embarrasing, i think is why perhaps no one is using this feature 5 years later... > > > Thanks Angel > -- > No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. > ->>----------------------------------------------- > Clist UAH a.k.a Angel > ---------------------------------[www.uah.es]-<<-- > > T? lo compras, yo lo copio. Todo legal. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From clist@REDACTED Tue Nov 10 20:58:05 2009 From: clist@REDACTED (Angel Alvarez) Date: Tue, 10 Nov 2009 20:58:05 +0100 Subject: [erlang-questions] package namespaces question In-Reply-To: References: <200911102010.14130.clist@uah.es> Message-ID: <200911102058.05513.clist@uah.es> Gracias Enrique! But how can i import all system modules and apps? Has gs become lib.gs or something alike? Well, the URLS from 2004 were bit old ive just found the current version at http://www.erlang.org/doc/man/packages.html I saw no prior examples of packcages used on the list certainly not back in 2008/2009 so an " almost deprecated" notice at the top of the page is all i need to stop at here and continue reading other things.... /Angel El Martes, 10 de Noviembre de 2009 Enrique Marcote escribi?: > Hola Angel, > > Try importing gs. > > Saludos, > > Enrique > > > El 10/11/2009, a las 20:10, Angel Alvarez escribi?: > > > Hi Guys! > > > > Im currently reading erlang archives from 2004 > > > > On 10/05/2004 Someone talked about package namespaces ("Is erlang too small" thread) so i decided to just test this "new2 erlang knowledge. > > > > iv just borrowed some little code from a gs article... :-) > > > > the module starts like this: > > > > > > -module(gui). > > -export([init/0]). > > > > init() -> > > S = gs:start(), > > Win = gs:create(window, S, [{width, 350}, {height, 100}]), > > gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]), > > ... > > > > so on the erlang shell gui:init(). just runs the example.. > > So i changed the module to use package style names... > > > > > > -module(angel.tests.gui). > > -export([init/0]). > > > > init() -> > > S = gs:start(), > > Win = gs:create(window, S, [{width, 350}, {height, 100}]), > > gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]), > > ... > > > > and moved the gui.erl to $PWD/angel/test/gui.erl > > > > > >> sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code/angel/test> erlc gui.erl > >> cd ../../ > >> sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code/ > > > > but now firing at the shell results in : > > > > sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code> erl > > Erlang R13B01 (erts-5.7.2) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] > > > > Eshell V5.7.2 (abort with ^G) > > 1> angel.test.gui:init(). > > ** exception error: undefined function 'angel.test.gs':start/0 > > in function 'angel.test.gui':init/0 > > > > What's wrong? > > Is needed the same convention on the rest of modules > > after you start using package name styles? > > > > > > From http://www.erlang.se/publications/packages.html > > > > "m2:g(X) becomes a call to foo.bar.m2 > > > > If this is not what was intended, the call can be written explicitly, as in > > -module(foo.bar.m1). > > -export([f/1]). > > > > f(X) -> fee.fie.foe.m2:g(X)." > > > > it seems bit embarrasing, i think is why perhaps no one is using this feature 5 years later... > > > > > > Thanks Angel > > -- > > No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. > > ->>----------------------------------------------- > > Clist UAH a.k.a Angel > > ---------------------------------[www.uah.es]-<<-- > > > > T? lo compras, yo lo copio. Todo legal. > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > -- Agua para todo? No, Agua para Todos. ->>----------------------------------------------- Clist UAH a.k.a Angel ---------------------------------[www.uah.es]-<<-- No le dar?a Cocacola Zero, ni a mi peor enemigo. Para eso est? el gas Mostaza que es mas piadoso. From kagato@REDACTED Tue Nov 10 21:22:12 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Tue, 10 Nov 2009 12:22:12 -0800 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <9b08084c0911100445m11c77393y5646ca5a937d2176@mail.gmail.com> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <4AF95830.8060508@erlang-consulting.com> <9b08084c0911100445m11c77393y5646ca5a937d2176@mail.gmail.com> Message-ID: <258CBDF8-A9E0-4F53-9CE1-BB68823A0867@souja.net> Dropping messages, suspending processes, and crashing processes is just a bad idea. Erlang's messaging is not "left-guarded" in the sense described by Hoare. That means that any behavior that suspends processes when the remote mailbox is full can either exhaust memory (i.e. what we have now) or arbitrarily deadlock the system. The only requirement for a deadlock is a loop in messaging anywhere, and we have more than a few of those. Dropping messages is probably the least disruptive, but (in applications that use OTP behaviors at least) it would just translate into gen_server:call timeouts, and we're back to defensive programming again. I'm all for a heuristic that sends a signal to a program when it's getting close (i.e. memory allocation failed and "reserved" memory is starting to be consumed so trigger an alarm of some sort), but anything proposed so far compromises the current behavior of the VM in ways that are awfully unpredictable. On Nov 10, 2009, at 4:45 AM, Joe Armstrong wrote: > This is a very interesting problem. if processes have quotas, then how > could you set the > quota value? > > A perfectly correct process might just have a very deep stack, just > once in its > life and otherwise be fine. Whether to crash this process or not would > depend upon > what the other processes in the system happened to be doing at the > time. This would > be very unfortunate - it's like your program being hit by a cosmic ray > - nasty. It creates a random non-deterministic coupling between things > that are supposed to be independent. > > A possibility that just occurred to me might be to suspend processes > that appear to be > running wild until such a time as the overall memory situation looks > good. > > Image two scheduler queues. One for well behaved programs. One for > programs whose > stacks and heaps are growing too fast. If memory is no problem we run > programs from both queues. If memory is tight we run processes in the > "problem" queue less often and with frequent garbs. > > Killing a program with a large stack and heap, just because their > happens to be a > temporary memory problem seems horrible, especially since the problem > might go away > if we wait a few milliseconds. > > Suspending a memory hungry process for a while, until memory is > available seems > less objectionable. Perhapse it could be swapped out to disk and > pulled in a lot later. > Killing things at random in the hope it might help sounds like a > really bad idea. > Process migration could solve this - move it to a machine that has got > more memory. > > Suspending things seems ok - you might even suspend an errant > process forever > and reclaim the memory - but not kill it. Some other process could > detect that the processes > is not responding and kill it and thus all the semantics of the > application would be obeyed > (processes are allowed to be unresponsive, that's fine) and the > semantics of the > error recovery should say what to do in this case. > > Just killing processes when they have done nothing wrong is not a > good idea. > > /Joe > > > > On Tue, Nov 10, 2009 at 1:10 PM, Ulf Wiger > wrote: >> Richard O'Keefe wrote: >>>> >>>> One way would be to let the user set a memory quota on a process >>>> with >>>> options at spawn time. When the process reaches it quota it can be >>>> automatically killed or the user can >>>> be notified in some way and take actions. >>> >>> One of the reasons this hasn't been done is, I presume, the fact >>> that >>> it is quite difficult for a programmer to determine what the memory >>> quota should be. It depends on >>> ... >> >> I implemented resource limits in erlhive - at the Erlang level rather >> than in the VM. The purpose was to be able to run foreign code safely >> in a hosted environment. Eliminating the possibility to do damage >> through traditional side-effects was relatively easy with a code >> transform, but two ways of staging a DoS attack would be to gobble >> RAM or CPU capacity. I approached this by inserting calls to a check >> function that sampled heap size, and started a "watchdog" process >> that >> would unceremoniously kill the program after a certain time. >> >> In short, I can see a need for such limits, and would like to include >> a reduction ceiling. The limits could be set after careful testing >> and high enough that they protect against runaway processes. A >> reduction >> limit could be checked at the end of each slice, perhaps. >> >> In my experience, per-process memory usage is fairly predictable in >> erlang. Does anyone have a different experience? >> >> BR, >> Ulf W >> -- >> Ulf Wiger >> CTO, Erlang Training & Consulting Ltd >> http://www.erlang-consulting.com >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From klas.johansson@REDACTED Tue Nov 10 22:18:10 2009 From: klas.johansson@REDACTED (Klas Johansson) Date: Tue, 10 Nov 2009 22:18:10 +0100 Subject: An English word stemmer Message-ID: Hi, I've dusted off an English word stemmer [1] I made quite a while ago. I've seen at least one other erlang stemmer before, and that's one by Hans Nilsson which was included in Joe's erlang book as well as in Ulf Wiger's rdbms. This stemmer is an implementation of the Porter2 stemming algorithm [2] and does its job fairly quickly at approximately 400,000 words per second on my 2.53 GHz Core 2 Duo MacBook Pro. I've tested it on the example vocabulary (available from [2]) and all words are stemmed according to the already prepared list of stemmed words. Code (and tests only a few clicks away) here: http://github.com/klajo/hacks/blob/master/stem/src/stem_en.erl There's also a Swedish version of the algorithm [3] in the repository which I whipped together just for the fun of it. The English algorithm does a better job at stemming words though. In case someone's interested... :-) Cheers, Klas [1] Read more on stemming for example here: http://en.wikipedia.org/wiki/Stemming [2] The Porter2 stemming algorithm http://snowball.tartarus.org/algorithms/english/stemmer.html [3] The Swedish stemming algorithm http://snowball.tartarus.org/algorithms/swedish/stemmer.html From icfp.publicity@REDACTED Tue Nov 10 22:22:12 2009 From: icfp.publicity@REDACTED (Wouter Swierstra) Date: Tue, 10 Nov 2009 22:22:12 +0100 Subject: ICFP 2010: Call for papers Message-ID: <53ff55480911101322o9528bf4jc1c036f364e2e66d@mail.gmail.com> ===================================================================== Call for Papers ICFP 2010: International Conference on Functional Programming Baltimore, Maryland, 27 -- 29 September 2010 http://www.icfpconference.org/icfp2010 ===================================================================== Important Dates (at 14:00 UTC) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Submission: 2 April 2010 Author response: 24 -- 25 May 2010 Notification: 7 June 2010 Final papers due: 12 July 2010 Scope ~~~~~ ICFP 2010 seeks original papers on the art and science of functional programming. Submissions are invited on all topics from principles to practice, from foundations to features, from abstraction to application. The scope includes all languages that encourage functional programming, including both purely applicative and imperative languages, as well as languages with objects or concurrency. Particular topics of interest include * Language Design: type systems; concurrency and distribution; modules; components and composition; metaprogramming; relations to object-oriented or logic programming; interoperability * Implementation: abstract machines; compilation; compile-time and run-time optimization; memory management; multi-threading; exploiting parallel hardware; interfaces to foreign functions, services, components or low-level machine resources * Software-Development Techniques: algorithms and data structures; design patterns; specification; verification; validation; proof assistants; debugging; testing; tracing; profiling * Foundations: formal semantics; lambda calculus; rewriting; type theory; monads; continuations; control; state; effects * Transformation and Analysis: abstract interpretation; partial evaluation; program transformation; program calculation; program proof * Applications and Domain-Specific Languages: symbolic computing; formal-methods tools; artificial intelligence; systems programming; distributed-systems and web programming; hardware design; databases; XML processing; scientific and numerical computing; graphical user interfaces; multimedia programming; scripting; system administration; security; education * Functional Pearls: elegant, instructive, and fun essays on functional programming The conference also solicits Experience Reports, which are short papers that provide evidence that functional programming really works or describe obstacles that have kept it from working in a particular application. Abbreviated instructions for authors ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ By 2 April 2010, 14:00 UTC, submit an abstract of at most 300 words and a full paper of at most 12 pages (6 pages for an Experience Report), including bibliography and figures. The deadline will be strictly enforced and papers exceeding the page limits will be summarily rejected. Authors have the option to attach supplementary material to a submission, on the understanding that reviewers may choose not to look at it. A submission will be evaluated according to its relevance, correctness, significance, originality, and clarity. It should explain its contributions in both general and technical terms, clearly identifying what has been accomplished, explaining why it is significant, and comparing it with previous work. The technical content should be accessible to a broad audience. Functional Pearls and Experience Reports are separate categories of papers that need not report original research results and must be marked as such at the time of submission. Detailed guidelines on both categories are on the conference web site. Each submission must adhere to SIGPLAN's republication policy, as explained on the web at http://www.acm.org/sigplan/republicationpolicy.htm. Proceedings will be published by ACM Press. Authors of accepted submissions are expected to transfer the copyright to the ACM. Presentations will be videotaped and released online if the presenter consents by signing an additional permission form at the time of the presentation. Formatting: Submissions must be in PDF format printable in black and white on US Letter sized paper and interpretable by Ghostscript. If this requirement is a hardship, make contact with the program chair at least one week before the deadline. Papers must adhere to the standard ACM conference format: two columns, nine-point font on a ten-point baseline, with columns 20pc (3.33in) wide and 54pc (9in) tall, with a column gutter of 2pc (0.33in). A suitable document template for LATEX is available from SIGPLAN at http://www.acm.org/sigs/sigplan/authorInformation.htm. Submission: Submissions will be accepted electronically at a URL to be named later. Improved versions of a paper may be submitted at any point before the submission deadline using the same web interface. Author response: Authors will have a 48-hour period, starting at 14:00 UTC on 24 May 2010, to read and respond to reviews. Special Journal Issue: There will be a special issue of the Journal of Functional Programming with papers from ICFP 2010. The program committee will invite the authors of select accepted papers to submit a journal version to this issue. Organization ~~~~~~~~~~~~ Conference Chair Paul Hudak, Yale University Program Chair Stephanie Weirich, University of Pennsylvania Program Committee: Umut Acar, Max Planck Institute for Software Systems Zena Ariola, University of Oregon James Cheney, University of Edinburgh Peter Dybjer, Chalmers University of Technology Robert Bruce Findler, Northwestern University Andy Gill, Kansas University Fritz Henglein, University of Copenhagen Michael Hicks, University of Maryland, College Park Patricia Johann, University of Strathclyde Andres L?h, Utrecht University Simon L. Peyton Jones, Microsoft Research Didier R?my, INRIA Paris-Rocquencourt John Reppy, University of Chicago Manuel Serrano, INRIA Sophia-Antipolis Matthieu Sozeau, Harvard University From wde@REDACTED Tue Nov 10 21:37:39 2009 From: wde@REDACTED (wde) Date: Tue, 10 Nov 2009 21:37:39 +0100 Subject: [erlang-questions] package namespaces question Message-ID: <20091110203735.1A06F2000063@mwinf2023.orange.fr> you can't import all "system "modules. you can't do somehting like that : import system.* you must import each module or you can prefix module names by a "." erlang package system is not really flexible. ======= le 10/11/2009, 20:58:05 vous ?criviez: ======= >Gracias Enrique! > >But how can i import all system modules and apps? > >Has gs become lib.gs or something alike? > >Well, the URLS from 2004 were bit old ive just found the current version at http://www.erlang.org/doc/man/packages.html > >I saw no prior examples of packcages used on the list certainly not back in 2008/2009 so an " almost deprecated" notice at the top >of the page is all i need to stop at here and continue reading other things.... > >/Angel > >El Martes, 10 de Noviembre de 2009 Enrique Marcote escribi?: >> Hola Angel, >> >> Try importing gs. >> >> Saludos, >> >> Enrique >> >> >> El 10/11/2009, a las 20:10, Angel Alvarez escribi?: >> >> > Hi Guys! >> > >> > Im currently reading erlang archives from 2004 >> > >> > On 10/05/2004 Someone talked about package namespaces ("Is erlang too small" thread) so i decided to just test this "new2 erlang knowledge. >> > >> > iv just borrowed some little code from a gs article... :-) >> > >> > the module starts like this: >> > >> > >> > -module(gui). >> > -export([init/0]). >> > >> > init() -> >> > S = gs:start(), >> > Win = gs:create(window, S, [{width, 350}, {height, 100}]), >> > gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]), >> > ... >> > >> > so on the erlang shell gui:init(). just runs the example.. >> > So i changed the module to use package style names... >> > >> > >> > -module(angel.tests.gui). >> > -export([init/0]). >> > >> > init() -> >> > S = gs:start(), >> > Win = gs:create(window, S, [{width, 350}, {height, 100}]), >> > gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]), >> > ... >> > >> > and moved the gui.erl to $PWD/angel/test/gui.erl >> > >> > >> >> sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code/angel/test> erlc gui.erl >> >> cd ../../ >> >> sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code/ >> > >> > but now firing at the shell results in : >> > >> > sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code> erl >> > Erlang R13B01 (erts-5.7.2) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] >> > >> > Eshell V5.7.2 (abort with ^G) >> > 1> angel.test.gui:init(). >> > ** exception error: undefined function 'angel.test.gs':start/0 >> > in function 'angel.test.gui':init/0 >> > >> > What's wrong? >> > Is needed the same convention on the rest of modules >> > after you start using package name styles? >> > >> > >> > From http://www.erlang.se/publications/packages.html >> > >> > "m2:g(X) becomes a call to foo.bar.m2 >> > >> > If this is not what was intended, the call can be written explicitly, as in >> > -module(foo.bar.m1). >> > -export([f/1]). >> > >> > f(X) -> fee.fie.foe.m2:g(X)." >> > >> > it seems bit embarrasing, i think is why perhaps no one is using this feature 5 years later... >> > >> > >> > Thanks Angel >> > -- >> > No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. >> > ->>----------------------------------------------- >> > Clist UAH a.k.a Angel >> > ---------------------------------[www.uah.es]-<<-- >> > >> > T? lo compras, yo lo copio. Todo legal. >> > >> > ________________________________________________________________ >> > erlang-questions mailing list. See http://www.erlang.org/faq.html >> > erlang-questions (at) erlang.org >> > >> >> > > > >-- >Agua para todo? No, Agua para Todos. >->>----------------------------------------------- > Clist UAH a.k.a Angel >---------------------------------[www.uah.es]-<<-- > >No le dar?a Cocacola Zero, ni a mi peor enemigo. Para eso est? el gas Mostaza que es mas piadoso. > >________________________________________________________________ >erlang-questions mailing list. See http://www.erlang.org/faq.html >erlang-questions (at) erlang.org = = = = = = = = = ========= = = = = = = = = = = wde wde@REDACTED 10/11/2009 From ok@REDACTED Tue Nov 10 22:52:07 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 11 Nov 2009 10:52:07 +1300 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <4AF95830.8060508@erlang-consulting.com> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <4AF95830.8060508@erlang-consulting.com> Message-ID: <229053ED-E213-4CF0-938B-AFC48F0FA3DE@cs.otago.ac.nz> On Nov 11, 2009, at 1:10 AM, Ulf Wiger wrote: > > In short, I can see a need for such limits, and would like to include > a reduction ceiling. The limits could be set after careful testing > and high enough that they protect against runaway processes. A > reduction > limit could be checked at the end of each slice, perhaps. > > In my experience, per-process memory usage is fairly predictable in > erlang. Does anyone have a different experience? I am not denying the *need* for limits. (Anyone else remember "engines" in Scheme?) I've had enough functions in enough languages go into infinite recursion that I can see the point of stopping them. However, Ulf Wiger has not addressed these points: - if the tagging scheme is changed (and it has in the past), memory requirements may change. They may either decrease (if, say, a special tag for 2-element tuples were introduced) or increase (if, say, list cells were made bigger). In particular, you may recall an EEP from me suggesting that there should be two representations for atoms: one physically unique as now and the other only logically unique as in LOGIX. There are three aims in that proposal: (1) reduce the need for symbol table locking when creating atoms, (2) reduce the impact of the symbol table size limit by making it possible for a process to create as many LOGIX-type atoms as it wants, and (3) make it possible to garbage-collect such atoms. If a process creates an atom now, the atom goes in shared storage and presumably is not "billed to" that process. With the LOGIX-style atoms proposal, many atoms would remain in the process's private heap and _would_ be billed to that process. If memory serves me correctly, the representation of binaries has changed in the past. There are certainly thresholds in binaries; as the performance characteristics of processors and the abilities of the Erlang compiler and HiPE change, they might well change, and if they do, the space needs of processes using binaries change. - if a data structure library changes its representation, memory requirements may change. One of the advantages of using a library is NOT having to know how it works inside. Suppose I need to keep track of N accounts. If I use gb_sets, from having looked at the source code I can tell that it will take something like 4N or 5N words, but without looking at the code, I would not know that. Looking at the code NOW tells me what the space cost is NOW. It does NOT guarantee me that the same space cost will apply in the next release. I can think of plausible reasons why the space cost might go up and plausible reasons why it might go down. - switching from one library module to another may change the space needs. For example, gb_sets and sets offer very similar interfaces. gb_sets uses binary search, sets uses hashing. In the course of maintenance, you might very well want to change from one to another. I haven't the faintest idea how much space N items in a sets: set would take, especially as there are tuning parameters in sets.erl which could very well be changed. It's fatally easy, when doing such maintenance, to forget to maintain the space bounds as well. - there are library modules where I have no idea how much space is needed. Come to think of it, that's nearly all of them. I certainly haven't the least notion what the space costs of 'supervisor' are. I've just assumed that they were small enough that I didn't need to bother. - Erlang data structures are made of things whose size is naturally measured in bytes (binaries) and things whose size is naturally measured in words (list cells, tuples), and other stuff which I'll ignore. When you switch from a 32-bit machine to a 64-bit machine (or vice versa) the size of *words* changes, but the size of *bytes* does not. This means that *right now* if you want your size bounds to port between 32-bit and 64-bit machines, even assuming everything else to remain unaltered, your size bounds *must* be expressed as B*bytes + W*words, not as bytes nor as words alone. Now it's open to someone to propose that space bounds (stack, heap, mailbox, individual messages sent, ...) *should* be expressed as {B,W} pairs, and such a proposal will get a respectful hearing from me. At any rate, I am not saying that space limits are an essentially bad idea, or that some means of providing such limits shouldn't be added to Erlang. What I'm saying is that - GETTING the limits right is never going to be easy - KEEPING the limits right is never going to be easy - EXPLAINING how to determine appropriate limits in fairly simple terms is going to be a very important part of Erlang documentation - CHECKING the limits, perhaps with some sort of load testing tools, is going to become an important part of development and maintenance. From ulf.wiger@REDACTED Tue Nov 10 23:26:34 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 10 Nov 2009 23:26:34 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <229053ED-E213-4CF0-938B-AFC48F0FA3DE@cs.otago.ac.nz> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <4AF95830.8060508@erlang-consulting.com> <229053ED-E213-4CF0-938B-AFC48F0FA3DE@cs.otago.ac.nz> Message-ID: <4AF9E89A.6080308@erlang-consulting.com> Richard O'Keefe wrote: > > I am not denying the *need* for limits. (Anyone else remember "engines" > in Scheme?) I've had enough functions in enough languages go into > infinite recursion that I can see the point of stopping them. > > However, Ulf Wiger has not addressed these points: > > - if the tagging scheme is changed (and it has in the past), > memory requirements may change. > ... Perhaps I'm colored by having worked on systems where every product release was preceded with months of relatively thorough testing, where monitoring memory usage during important operating conditions was routine. We would also tune the system by setting process heap size for important processes in order to minimize garbage collection. This obviously needed to be reviewed anytime something changed substantially. Another type of tuning is to limit the number of jobs of a given type that can execute concurrently. Again, this is dependent on the actual memory use of a process. When pushing high-throughput messaging systems to their limits, these activities tend to become necessary, and they have to be repeated for each significant revision of the software. A near-trivial case where memory requirements indeed will change is if one switches from 32-bit to 64-bit Erlang. This can cause memory usage in many Erlang applications to roughly double (depending on how much binaries are used). > At any rate, I am not saying that space limits are an essentially > bad idea, or that some means of providing such limits shouldn't be > added to Erlang. What I'm saying is that > - GETTING the limits right is never going to be easy > - KEEPING the limits right is never going to be easy > - EXPLAINING how to determine appropriate limits in fairly simple > terms is going to be a very important part of Erlang documentation > - CHECKING the limits, perhaps with some sort of load testing tools, > is going to become an important part of development and maintenance. Agreed. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ok@REDACTED Wed Nov 11 00:13:01 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 11 Nov 2009 12:13:01 +1300 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <9b08084c0911100445m11c77393y5646ca5a937d2176@mail.gmail.com> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <4AF95830.8060508@erlang-consulting.com> <9b08084c0911100445m11c77393y5646ca5a937d2176@mail.gmail.com> Message-ID: <9896D114-DC78-45C1-B1F9-2B86EE4C9E79@cs.otago.ac.nz> On Nov 11, 2009, at 1:45 AM, Joe Armstrong wrote: > A perfectly correct process might just have a very deep stack, just > once in its > life and otherwise be fine. Whether to crash this process or not would > depend upon > what the other processes in the system happened to be doing at the > time. Surely with per-process quotas the process would be crashed on exceeding its limit no matter what any other processes might be doing? I may be misunderstanding you. One thing that has plagued the UNIX world for years is over-commitment. A bunch of UNIX processes ask for lots of virtual memory each, and the operating system says yes to all of them, but doesn't actually give them the memory. When they try to touch the memory, it's then that the operating system actually allocates the pages. And it's then that the operating system discovers that "oops, each process is within its quota, but I don't actually have enough memory to go around." And it kills processes until it does. Quite often, the wrong ones. Some UNIX programmers (me amongst others) say this is ridiculous, a well written process won't ask for memory if it doesn't think it's going to need it, and if it can't have it, it should be told right now. Others say, no, it's normal practice to reserve a lot more than you are really likely to need, overcommitment is essential support. So if a process usually needs 10K but *might* need 10M, and there are process quotas, it has to ask for all 10M. And either the VM refuses to create new processes when the sum of the quotas exceeds available memory, in which case most of the memory might be lying idle because it was worst case requests, or the VM allows new processes to be created even so, in which case you can end up with a bunch of processes all within their quotas, but the VM runs out of memory anyway. Erlang already has 'hibernate', where a process can allow the VM to reclaim most of its memory. This suggests something that does the opposite. If a process can *tell* when it's going to need a lot more memory than usual, there could be an operation that says "please increase my quota to B bytes + W words, and if you can't do that just now, suspend me until you can". > A possibility that just occurred to me might be to suspend processes > that appear to be > running wild until such a time as the overall memory situation looks > good. A computation might use a lot of memory by setting up one large process. A computation might use a lot of memory by setting up a large number of small processes. The VM could be running out of memory, and there could be a large process, but _that_ process might be completely innocent. 200 (small) sheep weigh more than one (large) elephant and they breed a lot faster. A process might not be growing its own stack or heap at all, but none the less might be (indirectly) responsible for increasing demands on memory. Someone has already proposed a system whereby a process has to share its memory quota with the processes it spawns. By the way, I note that Java has precisely the same kind of problem. You can create a thread with a bound on its stack size, but the documentation for the relevant Thread constructor says: Allocates a new Thread object so that it ... has the specified stack size. ... The stack size is the approximate number of bytes of address space that the virtual machine is to allocate for this thread's stack. The effect of the stackSize parameter, if any, is highly platform dependent. [That sentence is bold in the original.] On some platforms, specifying a higher value for the stackSize parameter may allow a thread to achieve greater recursion depth before throwing a StackOverflowError. Similarly, specifying a lower value may allow a greater number of threads to exist concurrently without throwing an OutOfMemoryError (or other internal error). The details of the relationship between the value of the stackSize parameter and the maximum recursion depth and concurrency level are platform-dependent. On some platforms, the value of the stackSize parameter may have no effect whatsoever. [That sentence is bold in the original.] What happens when Java runs out of memory because the heap fills up? It's not completely clear from the documentation, but it appears that whichever process was running when an allocation attempt fails gets an OutOfMemory exception, even if that's the smallest process in the whole system. This exception doesn't seem to be handled in very many Java programs, at any rate I've often found perfectly good Java programs to crash with an unhandled OutOfMemory exception on my 4GB laptop because the default is 64MB. So, the Erlang VM crashes when memory runs out? That is, in practice, exactly what happens in Java. Can it be that Java programmers just don't expect the same quality of service from VMs as Erlang programmers do? (:-) (:-) (:-) From pablo.platt@REDACTED Wed Nov 11 00:25:29 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 10 Nov 2009 15:25:29 -0800 (PST) Subject: defining default functions for a module Message-ID: <424882.25194.qm@web112603.mail.gq1.yahoo.com> Hi, I have a module that acts as a router. It gets requests and pass them to callback modules. Callback modules should have a function for each case unless they need to do something specific. Is it ok to use erlang:function_exported/3 to check if the callback module defined the needed function and if not to execute the default defined in the router? Will it cost me in performance or is it against the way I should do things in erlang? I thought a behavior will solve my need but it only recommend a module structure for the compiler. Thanks From ok@REDACTED Wed Nov 11 00:28:22 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 11 Nov 2009 12:28:22 +1300 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <4AF95830.8060508@erlang-consulting.com> <9b08084c0911100445m11c77393y5646ca5a937d2176@mail.gmail.com> Message-ID: <1C66F88F-AC3C-4E1E-8565-F1543FFA39C4@cs.otago.ac.nz> On Nov 11, 2009, at 2:13 AM, Kenneth Lundin wrote: > If you for example have processes representing subscribers och > half-calls or mobile phones > it is pretty easy to know a reasonable size of such a process by > measurement. For THIS version of the program with THIS version of the Erlang libraries, compiler, VM and THIS build (because C space requirements can depend on C compiler settings) on THIS hardware with THIS workload. Measurement is a good thing, but when any of these factors changes, it's advisable to measure again. In particular, if I develop something on my box and measure and set an appropriate bound, and I send you the code, you have no reason to believe that the bound is appropriate for your circumstances until you have measured. Only the other day I was explaining to some students that the reason the SPARK subset of Ada can be used safely in embedded systems is that the SPARK subset excludes - recursion - pointers and dynamically allocated objects - dynamic array sizes so that the compiler can be sure everything fits. Erlang has all the things that make this hard. > It should be easy to say if it can grow up to 100 kwords, 1Mwords, 10 > Mwords etc. I can't do that for small C programs. I've seen C programs where the stack space varied by a factor of 10 between different hardware/ compiler/options choices. Come to that, with some C compilers supporting tail call optimisation and some not, it's quite easy to construct artificial examples where the memory needed by a thread could be as low as 1 KB or as high as 1 GB. > Of course you should not try to set quota as tight as possible. The > quota on process > size is just a way to protect the system from total disaster if one of > these processes > happens to grow over all reasonable limits because of a bug. There are two separable issues. A. How do we detect a runaway process and stop it ruining things for everyone else? Loose quotas are just fine for this. Even a factor of 10 too large is good enough. (Maybe.) B. How do we handle memory exhaustion without killing the VM *when no process has exceeded its quota*. Loose quotas actually create this problem. (In the present system, the quotas are as loose as they could possibly be...) I have no quarrel with optional loose quotas as a debugging/safety measure, a solution to problem A. It's just that we shouldn't mistake them for a solution to problem B, which is what I thought this thread was about. Tackling problem B should make a nice PhD for someone. (While there are some, um, questionable decisions in the design of the Java _language_, a lot of smart people have worked on Java _implementation_, and if there were a good solution to problem B I think some Java vendor would be boasting about it.) From ok@REDACTED Wed Nov 11 01:29:23 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 11 Nov 2009 13:29:23 +1300 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <4AF9E89A.6080308@erlang-consulting.com> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <4AF95830.8060508@erlang-consulting.com> <229053ED-E213-4CF0-938B-AFC48F0FA3DE@cs.otago.ac.nz> <4AF9E89A.6080308@erlang-consulting.com> Message-ID: <5794464D-37C2-4DB6-B298-4ADF0BD4DFA4@cs.otago.ac.nz> On Nov 11, 2009, at 11:26 AM, Ulf Wiger wrote: > Richard O'Keefe wrote: > >> I am not denying the *need* for limits. (Anyone else remember >> "engines" >> in Scheme?) I've had enough functions in enough languages go into >> infinite recursion that I can see the point of stopping them. >> However, Ulf Wiger has not addressed these points: >> - if the tagging scheme is changed (and it has in the past), >> memory requirements may change. > > ... > > Perhaps I'm colored by having worked on systems where every > product release was preceded with months of relatively > thorough testing, where monitoring memory usage during > important operating conditions was routine. Let's see if we can reach consensus here. Ulf, I'm *not* saying that you can't do what you DID do. That would be rude and stupid. What I'm saying is that you have to *KEEP* doing it, as indeed you did ("every product release"), and that you have to keep doing it even if YOUR code didn't change at all. How you do that thorough testing is something that needs to be written up clearly in some new Erlang book, which I wish you would write. I would certainly buy it. > A near-trivial case where memory requirements indeed will > change is if one switches from 32-bit to 64-bit Erlang. > This can cause memory usage in many Erlang applications to > roughly double (depending on how much binaries are used). I thought I already said that. This whole thread is rather interesting because it shows up the difference between *language* issues and *system* issues. There have, for example, been some interesting changes to the Java *language* over the years, but for many purposes the instrumentation interfaces and things like MX Beans are much more important. Erlang *has* monitoring tools, but I don't think they are anywhere near as well understood by many Erlang programmers as they should be, including me. I'm serious about the book! From kagato@REDACTED Wed Nov 11 01:32:04 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Tue, 10 Nov 2009 16:32:04 -0800 Subject: [erlang-questions] defining default functions for a module In-Reply-To: <424882.25194.qm@web112603.mail.gq1.yahoo.com> References: <424882.25194.qm@web112603.mail.gq1.yahoo.com> Message-ID: Maybe require a callback module to have a dummy function, or is the function name really dynamic? If it is dynamic (and your program is network accessible), are you sure you won't be leaking atoms (i.e. exposing an exciting DOS)? Also, have you considered gen_event? They're quite lovely, I assure you. On Nov 10, 2009, at 3:25 PM, Pablo Platt wrote: > Hi, > > I have a module that acts as a router. > It gets requests and pass them to callback modules. > Callback modules should have a function for each case unless they > need to do something specific. > > Is it ok to use erlang:function_exported/3 to check if the callback > module defined the needed function > and if not to execute the default defined in the router? > Will it cost me in performance or is it against the way I should do > things in erlang? > > I thought a behavior will solve my need but it only recommend a > module structure for the compiler. > > Thanks > > > From g@REDACTED Wed Nov 11 01:55:46 2009 From: g@REDACTED (Garrett Smith) Date: Tue, 10 Nov 2009 18:55:46 -0600 Subject: Mnesia reads with multiple table copies Message-ID: WIth Mnesia's location transparency, what is the scheme used to select a source for reads when there are multiple table copies? The factors that I imagine would be considered are: * local vs remote * ram vs disk * running vs stopped nodes E.g. one might assume that a local, ram based read would be chosen over a remote, disk base read. My apologies if this is well known -- I've poked around the docs and Google, but haven't turned up anything on this. Thanks, Garrett From kagato@REDACTED Wed Nov 11 01:57:04 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Tue, 10 Nov 2009 16:57:04 -0800 Subject: [erlang-questions] defining default functions for a module In-Reply-To: <406591.36865.qm@web112620.mail.gq1.yahoo.com> References: <424882.25194.qm@web112603.mail.gq1.yahoo.com> <406591.36865.qm@web112620.mail.gq1.yahoo.com> Message-ID: <4D4316B4-0AD9-4DF8-B1C0-7707C65475AC@souja.net> Basically, the router is a gen_event, the modules are gen_event handlers plugged into the router. Each handler implements the functions as something like: > handle_event({i_actually_handle_this,Params}) -> > do_something(Params); > handle_event(_) -> > unhandled. If the goal is route some message, that works pretty well. If you need the return value, it's not quite so easy, but it's an option that is "handled" for most cases. Just a thought. On Nov 10, 2009, at 4:48 PM, Pablo Platt wrote: > My callbacks are not dynamic so I won't be leaking atoms. > I can have a dummy function but it makes dev longer and modules less > readable. > I'm trying to do something similar to objects and classes in OO. > > How gen_event is related? > > From: Jayson Vantuyl > To: Pablo Platt > Cc: erlang-questions > Sent: Wed, November 11, 2009 2:32:04 AM > Subject: Re: [erlang-questions] defining default functions for a > module > > Maybe require a callback module to have a dummy function, or is the > function name really dynamic? If it is dynamic (and your program is > network accessible), are you sure you won't be leaking atoms (i.e. > exposing an exciting DOS)? > > Also, have you considered gen_event? They're quite lovely, I assure > you. > > On Nov 10, 2009, at 3:25 PM, Pablo Platt wrote: > > > Hi, > > > > I have a module that acts as a router. > > It gets requests and pass them to callback modules. > > Callback modules should have a function for each case unless they > need to do something specific. > > > > Is it ok to use erlang:function_exported/3 to check if the > callback module defined the needed function > > and if not to execute the default defined in the router? > > Will it cost me in performance or is it against the way I should > do things in erlang? > > > > I thought a behavior will solve my need but it only recommend a > module structure for the compiler. > > > > Thanks > > > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From pablo.platt@REDACTED Wed Nov 11 02:06:00 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 10 Nov 2009 17:06:00 -0800 (PST) Subject: [erlang-questions] defining default functions for a module In-Reply-To: <4D4316B4-0AD9-4DF8-B1C0-7707C65475AC@souja.net> References: <424882.25194.qm@web112603.mail.gq1.yahoo.com> <406591.36865.qm@web112620.mail.gq1.yahoo.com> <4D4316B4-0AD9-4DF8-B1C0-7707C65475AC@souja.net> Message-ID: <399518.46828.qm@web112618.mail.gq1.yahoo.com> I'm not sure but I don't think it's useful in my case. I'm trying to implement a CRUD - crate read update delete router for resources with callback functions. In the default case the router can do any one of the actions but callback functions can override this behavior. For example a callback function can check for permissions and won't let unauthorized user to delete a resource. ________________________________ From: Jayson Vantuyl To: Pablo Platt Cc: erlang-questions Sent: Wed, November 11, 2009 2:57:04 AM Subject: Re: [erlang-questions] defining default functions for a module Basically, the router is a gen_event, the modules are gen_event handlers plugged into the router. Each handler implements the functions as something like: > handle_event({i_actually_handle_this,Params}) -> > do_something(Params); > handle_event(_) -> > unhandled. If the goal is route some message, that works pretty well. If you need the return value, it's not quite so easy, but it's an option that is "handled" for most cases. Just a thought. On Nov 10, 2009, at 4:48 PM, Pablo Platt wrote: > My callbacks are not dynamic so I won't be leaking atoms. > I can have a dummy function but it makes dev longer and modules less readable. > I'm trying to do something similar to objects and classes in OO. > > How gen_event is related? > > From: Jayson Vantuyl > To: Pablo Platt > Cc: erlang-questions > Sent: Wed, November 11, 2009 2:32:04 AM > Subject: Re: [erlang-questions] defining default functions for a module > > Maybe require a callback module to have a dummy function, or is the function name really dynamic? If it is dynamic (and your program is network accessible), are you sure you won't be leaking atoms (i.e. exposing an exciting DOS)? > > Also, have you considered gen_event? They're quite lovely, I assure you. > > On Nov 10, 2009, at 3:25 PM, Pablo Platt wrote: > > > Hi, > > > > I have a module that acts as a router. > > It gets requests and pass them to callback modules. > > Callback modules should have a function for each case unless they need to do something specific. > > > > Is it ok to use erlang:function_exported/3 to check if the callback module defined the needed function > > and if not to execute the default defined in the router? > > Will it cost me in performance or is it against the way I should do things in erlang? > > > > I thought a behavior will solve my need but it only recommend a module structure for the compiler. > > > > Thanks > > > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From pablo.platt@REDACTED Wed Nov 11 01:48:12 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 10 Nov 2009 16:48:12 -0800 (PST) Subject: [erlang-questions] defining default functions for a module In-Reply-To: References: <424882.25194.qm@web112603.mail.gq1.yahoo.com> Message-ID: <406591.36865.qm@web112620.mail.gq1.yahoo.com> My callbacks are not dynamic so I won't be leaking atoms. I can have a dummy function but it makes dev longer and modules less readable. I'm trying to do something similar to objects and classes in OO. How gen_event is related? ________________________________ From: Jayson Vantuyl To: Pablo Platt Cc: erlang-questions Sent: Wed, November 11, 2009 2:32:04 AM Subject: Re: [erlang-questions] defining default functions for a module Maybe require a callback module to have a dummy function, or is the function name really dynamic? If it is dynamic (and your program is network accessible), are you sure you won't be leaking atoms (i.e. exposing an exciting DOS)? Also, have you considered gen_event? They're quite lovely, I assure you. On Nov 10, 2009, at 3:25 PM, Pablo Platt wrote: > Hi, > > I have a module that acts as a router. > It gets requests and pass them to callback modules. > Callback modules should have a function for each case unless they need to do something specific. > > Is it ok to use erlang:function_exported/3 to check if the callback module defined the needed function > and if not to execute the default defined in the router? > Will it cost me in performance or is it against the way I should do things in erlang? > > I thought a behavior will solve my need but it only recommend a module structure for the compiler. > > Thanks > > > ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From yogishb@REDACTED Wed Nov 11 03:09:56 2009 From: yogishb@REDACTED (Yogish Baliga) Date: Tue, 10 Nov 2009 18:09:56 -0800 (PST) Subject: gen_server:terminate/2 question Message-ID: <581363.81508.qm@web112618.mail.gq1.yahoo.com> I have a gen_server process listening for events on message bus. These messages are placed in its message box. The processing of these messages is slower than the rate of incoming message. When I stop the application (supervisor of which is a parent of the above mentioned gen-server), Will this gen_server get the terminate/2 call immediately or only after all the messages are processed OR should I have a wait logic in the terminate function to wait for the message box to be cleared? Thanx, -- baliga "Point of view is worth 80 IQ points" --Alan Kay http://dudefrommangalore.blogspot.com/ From bernie@REDACTED Wed Nov 11 06:57:55 2009 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 11 Nov 2009 16:57:55 +1100 Subject: [erlang-questions] gen_server:terminate/2 question In-Reply-To: <581363.81508.qm@web112618.mail.gq1.yahoo.com> References: <581363.81508.qm@web112618.mail.gq1.yahoo.com> Message-ID: <4AFA5263.90907@m5net.com> Yogish Baliga wrote: > I have a gen_server process listening for events on message bus. These messages are placed in its message box. The processing of these messages is slower than the rate of incoming message. > > When I stop the application (supervisor of which is a parent of the above mentioned gen-server), Will this gen_server get the terminate/2 call immediately or only after all the messages are processed OR should I have a wait logic in the terminate function to wait for the message box to be cleared? > It depends on the shutdown method you've specified in the supervisor. If you've nominated 'brutal_kill', I'm pretty sure even terminate/2 doesn't get called. If you've specified a timeout or 'infinity', I think the request to shutdown will be placed in the queue after all the other messages. If, of course, you reach the timeout before the message queue is cleared and the shutdown is processed, the supervisor will fall back to a kill which will bypass the queue (and, as above, will probably also mean terminate/2 doesn't get called). (Anyone please feel free to correct me on the above - I've only been using the gen_server and supervisor stuff for a few months and so could easily be under some misapprehensions). Remember also that if you want to ensure all the messages ever sent to the server are handled, you need to make sure the things sending them stop before you make the shutdown request. Cheers, Bernard From bengt.kleberg@REDACTED Wed Nov 11 08:53:20 2009 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 11 Nov 2009 08:53:20 +0100 Subject: link: CouchDB almost a GNOME dependency during this cycle. Message-ID: <1257926000.5227.56.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> So maybe next cycle? http://arstechnica.com/open-source/news/2009/11/gnome-roadmap-updated-version-3-to-arrive-in-2010.ars From mpquique@REDACTED Wed Nov 11 10:54:58 2009 From: mpquique@REDACTED (Enrique Marcote) Date: Wed, 11 Nov 2009 10:54:58 +0100 Subject: [erlang-questions] package namespaces question In-Reply-To: <200911102058.05513.clist@uah.es> References: <200911102010.14130.clist@uah.es> <200911102058.05513.clist@uah.es> Message-ID: <993F6C31-4F37-4E43-AFBD-2B9D7D460EAA@gmail.com> Hola Angel: Importing every module explicitly is not the worse thing, actually is even good, and besides that, the alternative (dot notation) is awful. Be warned that tools like dialyzer do not support packages. In our organization dialyzer is a fundamental tool and we prefer to sacrifice packages. I'm afraid that in the current state, packages will give you more headache than benefits. You may consider using a flat name space and agree on a prefix for every app. In your example you could do something like: gui.erl (for the application). gui_sup.erl (supervisor). gui_top_pane.erl ... and so on. It's not the nicest thing, but probably more convenient for the time being. Saludos, Enrique El 10/11/2009, a las 20:58, Angel Alvarez escribi?: > Gracias Enrique! > > But how can i import all system modules and apps? > > Has gs become lib.gs or something alike? > > Well, the URLS from 2004 were bit old ive just found the current version at http://www.erlang.org/doc/man/packages.html > > I saw no prior examples of packcages used on the list certainly not back in 2008/2009 so an " almost deprecated" notice at the top > of the page is all i need to stop at here and continue reading other things.... > > /Angel > > El Martes, 10 de Noviembre de 2009 Enrique Marcote escribi?: >> Hola Angel, >> >> Try importing gs. >> >> Saludos, >> >> Enrique >> >> >> El 10/11/2009, a las 20:10, Angel Alvarez escribi?: >> >>> Hi Guys! >>> >>> Im currently reading erlang archives from 2004 >>> >>> On 10/05/2004 Someone talked about package namespaces ("Is erlang too small" thread) so i decided to just test this "new2 erlang knowledge. >>> >>> iv just borrowed some little code from a gs article... :-) >>> >>> the module starts like this: >>> >>> >>> -module(gui). >>> -export([init/0]). >>> >>> init() -> >>> S = gs:start(), >>> Win = gs:create(window, S, [{width, 350}, {height, 100}]), >>> gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]), >>> ... >>> >>> so on the erlang shell gui:init(). just runs the example.. >>> So i changed the module to use package style names... >>> >>> >>> -module(angel.tests.gui). >>> -export([init/0]). >>> >>> init() -> >>> S = gs:start(), >>> Win = gs:create(window, S, [{width, 350}, {height, 100}]), >>> gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]), >>> ... >>> >>> and moved the gui.erl to $PWD/angel/test/gui.erl >>> >>> >>>> sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code/angel/test> erlc gui.erl >>>> cd ../../ >>>> sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code/ >>> >>> but now firing at the shell results in : >>> >>> sinosuke@REDACTED:~/Datos/Docs/Personal/Erlang/Code> erl >>> Erlang R13B01 (erts-5.7.2) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] >>> >>> Eshell V5.7.2 (abort with ^G) >>> 1> angel.test.gui:init(). >>> ** exception error: undefined function 'angel.test.gs':start/0 >>> in function 'angel.test.gui':init/0 >>> >>> What's wrong? >>> Is needed the same convention on the rest of modules >>> after you start using package name styles? >>> >>> >>> From http://www.erlang.se/publications/packages.html >>> >>> "m2:g(X) becomes a call to foo.bar.m2 >>> >>> If this is not what was intended, the call can be written explicitly, as in >>> -module(foo.bar.m1). >>> -export([f/1]). >>> >>> f(X) -> fee.fie.foe.m2:g(X)." >>> >>> it seems bit embarrasing, i think is why perhaps no one is using this feature 5 years later... >>> >>> >>> Thanks Angel >>> -- >>> No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. >>> ->>----------------------------------------------- >>> Clist UAH a.k.a Angel >>> ---------------------------------[www.uah.es]-<<-- >>> >>> T? lo compras, yo lo copio. Todo legal. >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> >> > > > > -- > Agua para todo? No, Agua para Todos. > ->>----------------------------------------------- > Clist UAH a.k.a Angel > ---------------------------------[www.uah.es]-<<-- > > No le dar?a Cocacola Zero, ni a mi peor enemigo. Para eso est? el gas Mostaza que es mas piadoso. From attila.r.nohl@REDACTED Wed Nov 11 10:57:31 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 11 Nov 2009 10:57:31 +0100 Subject: Fwd: [erlang-questions] defining default functions for a module In-Reply-To: <401d3ba30911110157m39f49bd2g8e4eb2d60f9a1a4a@mail.gmail.com> References: <424882.25194.qm@web112603.mail.gq1.yahoo.com> <401d3ba30911110157m39f49bd2g8e4eb2d60f9a1a4a@mail.gmail.com> Message-ID: <401d3ba30911110157r6ab412aavff8d113ba2395776@mail.gmail.com> Of course, I've wanted to send this to the list... ---------- Forwarded message ---------- From: Attila Rajmund Nohl Date: Wed, 11 Nov 2009 10:57:02 +0100 Subject: Re: [erlang-questions] defining default functions for a module To: Pablo Platt 2009/11/11, Pablo Platt : [...] > I thought a behavior will solve my need but it only recommend a module > structure for the compiler. It seems I'm not the only one missing inheritance :-) You can try code like this: case catch CBModule:cb_function(Parameters) of {'EXIT', {undef, _TraceBack}}=_Error -> call_default(Parameters); Result -> Result end, From tamas.nagy@REDACTED Wed Nov 11 11:01:18 2009 From: tamas.nagy@REDACTED (Tamas Nagy) Date: Wed, 11 Nov 2009 10:01:18 +0000 (GMT) Subject: [erlang-questions] gen_server:terminate/2 question In-Reply-To: <24372242.28271257933566121.JavaMail.root@zimbra> Message-ID: <8283198.28291257933678805.JavaMail.root@zimbra> Hi, One thing you should not forget is if you want the gen_server to do a terminate/2 you need to trap exits otherwise it will die immediately no matter what shutdown method is configured in the supervisor. Regards, Tamas ----- "Bernard Duggan" wrote: > Yogish Baliga wrote: > > I have a gen_server process listening for events on message bus. > These messages are placed in its message box. The processing of these > messages is slower than the rate of incoming message. > > > > When I stop the application (supervisor of which is a parent of the > above mentioned gen-server), Will this gen_server get the terminate/2 > call immediately or only after all the messages are processed OR > should I have a wait logic in the terminate function to wait for the > message box to be cleared? > > > It depends on the shutdown method you've specified in the supervisor. > > If you've nominated 'brutal_kill', I'm pretty sure even terminate/2 > doesn't get called. If you've specified a timeout or 'infinity', I > think the request to shutdown will be placed in the queue after all > the > other messages. If, of course, you reach the timeout before the > message > queue is cleared and the shutdown is processed, the supervisor will > fall > back to a kill which will bypass the queue (and, as above, will > probably > also mean terminate/2 doesn't get called). > > (Anyone please feel free to correct me on the above - I've only been > using the gen_server and supervisor stuff for a few months and so > could > easily be under some misapprehensions). > > Remember also that if you want to ensure all the messages ever sent > to > the server are handled, you need to make sure the things sending them > stop before you make the shutdown request. > > Cheers, > > Bernard > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- Tamas Nagy Erlang Training & Consulting http://www.erlang-consulting.com From kostis@REDACTED Wed Nov 11 11:08:43 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 11 Nov 2009 12:08:43 +0200 Subject: [erlang-questions] package namespaces question In-Reply-To: <993F6C31-4F37-4E43-AFBD-2B9D7D460EAA@gmail.com> References: <200911102010.14130.clist@uah.es> <200911102058.05513.clist@uah.es> <993F6C31-4F37-4E43-AFBD-2B9D7D460EAA@gmail.com> Message-ID: <4AFA8D2B.5040101@cs.ntua.gr> Enrique Marcote wrote: > Hola Angel: > > Importing every module explicitly is not the worse thing, actually is even good, and besides that, the alternative (dot notation) is awful. Be warned that tools like dialyzer do not support packages. In our organization dialyzer is a fundamental tool and we prefer to sacrifice packages. Good choice ;-), but you do not have to continue being on a dilemma... While it is true that versions of dialyzer up to now do not support packages, the version in the upcoming R13B03 will do so. Kostis Sagonas From ulf.wiger@REDACTED Wed Nov 11 11:38:47 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 11 Nov 2009 11:38:47 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <5794464D-37C2-4DB6-B298-4ADF0BD4DFA4@cs.otago.ac.nz> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <4AF95830.8060508@erlang-consulting.com> <229053ED-E213-4CF0-938B-AFC48F0FA3DE@cs.otago.ac.nz> <4AF9E89A.6080308@erlang-consulting.com> <5794464D-37C2-4DB6-B298-4ADF0BD4DFA4@cs.otago.ac.nz> Message-ID: <4AFA9437.6030007@erlang-consulting.com> Richard O'Keefe wrote: > > On Nov 11, 2009, at 11:26 AM, Ulf Wiger wrote: > >> Richard O'Keefe wrote: >> >>> I am not denying the *need* for limits. (Anyone else remember "engines" >>> in Scheme?) I've had enough functions in enough languages go into >>> infinite recursion that I can see the point of stopping them. >>> However, Ulf Wiger has not addressed these points: >>> - if the tagging scheme is changed (and it has in the past), >>> memory requirements may change. >> > ... >> >> Perhaps I'm colored by having worked on systems where every >> product release was preceded with months of relatively >> thorough testing, where monitoring memory usage during >> important operating conditions was routine. > > Let's see if we can reach consensus here. I think we do agree. Hopefully the volley will clarify things to the rest of the crowd - let's go another round and see if people will start groaning. :) > What I'm saying is that you have to *KEEP* doing it, > as indeed you did ("every product release"), and that > you have to keep doing it even if YOUR code didn't > change at all. Indeed. For high-availability products, this ought to be a given. But as Erlang is also used in many other settings, I would like to add an observation - perhaps obvious to all: Erlang has some limits /today/ that programmers have to live with. In many cases, the limits are high enough that they are no cause for concern. Some of the better-known limits are: - the number of simultaneous processes in the system (default: 32768, but can be raised up to 268435456) - the number of ETS tables (default: 1400, can be raised considerably) - the number of open ports and file descriptors (basically an OS limit) Other system limits can be found in http://www.erlang.org/doc/efficiency_guide/part_frame.html and most are implementation details and subject to change. What happens if you run into a system limit, e.g. if you try creating too many ets tables, is that the operation fails with an exception. Out of memory is a rather special case, in that it brings down the entire VM. Adding a few more limits that affect individual processes rather than suffering the node-global disaster of OOM, is thus nothing new, and in that sense shouldn't be terribly controversial in itself. The reasonable /default/ is of course that processes behave as today, i.e. the heap and message queue length limits et al would be /optional/. Having said this, it is certainly important to discuss which such limits would actually be useful in practice. > How you do that thorough testing is something that needs > to be written up clearly in some new Erlang book, which > I wish you would write. I would certainly buy it. Ok, at least one potential customer. I'll think about it. :) BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ulf.wiger@REDACTED Wed Nov 11 11:48:17 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 11 Nov 2009 11:48:17 +0100 Subject: [erlang-questions] Mnesia reads with multiple table copies In-Reply-To: References: Message-ID: <4AFA9671.4040400@erlang-consulting.com> Garrett Smith wrote: > WIth Mnesia's location transparency, what is the scheme used to select > a source for reads when there are multiple table copies? The factors > that I imagine would be considered are: > > * local vs remote > * ram vs disk > * running vs stopped nodes Yes, this is how it works. At table load time, mnesia stores a where_to_read value for each table in the mnesia_gvar table (an internal ets table). This value is updated dynamically as conditions change. Local reads are definitely favoured over remote, and RAM copies are favoured over disc_only. The preference logic is in mnesia_lib.erl BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From lfredlund@REDACTED Wed Nov 11 12:25:48 2009 From: lfredlund@REDACTED (=?ISO-8859-1?Q?Lars-=C5ke_Fredlund?=) Date: Wed, 11 Nov 2009 12:25:48 +0100 Subject: hipe_bifs missing in R13B02 Message-ID: <4AFA9F3C.3040105@fi.upm.es> Yes, they probably have never officially been part of the Erlang distribution but still filled a useful role, e.g. hipe_bifs:bytearray/2 for creating destructive quick arrays (check Kostis Sagonas posts to the erlang-questions mailing list for details :-) They seemed to be working in the first R13 release, but on downloading and compiling R13B02-1 they seem to be missing (or I messed up the recompilation). Any official word on their omission? They are really useful for some purposes... Lars-?ke Fredlund From clist@REDACTED Wed Nov 11 15:17:38 2009 From: clist@REDACTED (Angel Alvarez) Date: Wed, 11 Nov 2009 15:17:38 +0100 Subject: [erlang-questions] gen_server:terminate/2 question In-Reply-To: <8283198.28291257933678805.JavaMail.root@zimbra> References: <8283198.28291257933678805.JavaMail.root@zimbra> Message-ID: <200911111517.38336.clist@uah.es> I dont undertand this part i checked out again the docs but no clue "...If the gen_server is part of a supervision tree and is ordered by its supervisor to terminate, this function will be called with Reason=shutdown if the following conditions apply: - the gen_server has been set to trap exit signals, and - the shutdown strategy as defined in the supervisor's child specification is an integer timeout value, not brutal_kill. Otherwise, the gen_server will be immediately terminated." Why, the need to trap signals? has the supervisor started this gen_server with spawn_link. Does the gen_server behavior call terminate/2 only upon receiveing an exit mesage from the supervisor( so ti needs to trap this signal)? Please explain this, thanks!? El Mi?rcoles, 11 de Noviembre de 2009 Tamas Nagy escribi?: > Hi, > > One thing you should not forget is if you want the gen_server to do a terminate/2 you need to trap exits otherwise it will die immediately no matter what shutdown method is configured in the supervisor. > > Regards, > Tamas > > ----- "Bernard Duggan" wrote: > > > Yogish Baliga wrote: > > > I have a gen_server process listening for events on message bus. > > These messages are placed in its message box. The processing of these > > messages is slower than the rate of incoming message. > > > > > > When I stop the application (supervisor of which is a parent of the > > above mentioned gen-server), Will this gen_server get the terminate/2 > > call immediately or only after all the messages are processed OR > > should I have a wait logic in the terminate function to wait for the > > message box to be cleared? > > > > > It depends on the shutdown method you've specified in the supervisor. > > > > If you've nominated 'brutal_kill', I'm pretty sure even terminate/2 > > doesn't get called. If you've specified a timeout or 'infinity', I > > think the request to shutdown will be placed in the queue after all > > the > > other messages. If, of course, you reach the timeout before the > > message > > queue is cleared and the shutdown is processed, the supervisor will > > fall > > back to a kill which will bypass the queue (and, as above, will > > probably > > also mean terminate/2 doesn't get called). > > > > (Anyone please feel free to correct me on the above - I've only been > > using the gen_server and supervisor stuff for a few months and so > > could > > easily be under some misapprehensions). > > > > Remember also that if you want to ensure all the messages ever sent > > to > > the server are handled, you need to make sure the things sending them > > stop before you make the shutdown request. > > > > Cheers, > > > > Bernard > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > -- Agua para todo? No, Agua para Todos. ->>----------------------------------------------- Clist UAH a.k.a Angel ---------------------------------[www.uah.es]-<<-- T? lo compras, yo lo copio. Todo legal. From gleber.p@REDACTED Wed Nov 11 15:24:56 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 11 Nov 2009 15:24:56 +0100 Subject: [erlang-questions] Mnesia reads with multiple table copies In-Reply-To: <4AFA9671.4040400@erlang-consulting.com> References: <4AFA9671.4040400@erlang-consulting.com> Message-ID: <14f0e3620911110624u3ff15993n5f031002623a1500@mail.gmail.com> Here's my answer to the complementing question on StackOverflow: http://stackoverflow.com/questions/722665/in-a-mnesia-cluster-which-node-is-queried/726318#726318 On Wed, Nov 11, 2009 at 11:48, Ulf Wiger wrote: > Garrett Smith wrote: >> >> WIth Mnesia's location transparency, what is the scheme used to select >> a source for reads when there are multiple table copies? The factors >> that I imagine would be considered are: >> >> * local vs remote >> * ram vs disk >> * running vs stopped nodes > > Yes, this is how it works. > > At table load time, mnesia stores a where_to_read value > for each table in the mnesia_gvar table (an internal ets table). > This value is updated dynamically as conditions change. Local > reads are definitely favoured over remote, and RAM copies are > favoured over disc_only. The preference logic is in mnesia_lib.erl > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From g@REDACTED Wed Nov 11 15:37:01 2009 From: g@REDACTED (Garrett Smith) Date: Wed, 11 Nov 2009 08:37:01 -0600 Subject: [erlang-questions] Mnesia reads with multiple table copies In-Reply-To: <4AFA9671.4040400@erlang-consulting.com> References: <4AFA9671.4040400@erlang-consulting.com> Message-ID: On Wed, Nov 11, 2009 at 4:48 AM, Ulf Wiger wrote: > Garrett Smith wrote: >> >> WIth Mnesia's location transparency, what is the scheme used to select >> a source for reads when there are multiple table copies? The factors >> that I imagine would be considered are: >> >> * local vs remote >> * ram vs disk >> * running vs stopped nodes > > Yes, this is how it works. > > At table load time, mnesia stores a where_to_read value > for each table in the mnesia_gvar table (an internal ets table). > This value is updated dynamically as conditions change. Local > reads are definitely favoured over remote, and RAM copies are > favoured over disc_only. The preference logic is in mnesia_lib.erl Thanks! I see now also that you can get the info using: mnesia:table_info(Tab, where_to_read) From jvliwanag@REDACTED Wed Nov 11 16:41:49 2009 From: jvliwanag@REDACTED (Jan Vincent) Date: Wed, 11 Nov 2009 23:41:49 +0800 Subject: [erlang-questions] Best way to implement a simple cache In-Reply-To: References: <3F36CDB2-757D-4204-920C-4B1307AC1DFB@gmail.com> <20091110172540.GA26341@hanele> Message-ID: <5114DEA6-CC69-4B32-AC43-1962C4005E9A@gmail.com> Why wouldn't an ETS be good for this? On Nov 11, 2009, at 1:45 AM, Garrett Smith wrote: > On Tue, Nov 10, 2009 at 11:25 AM, Jachym Holecek wrote: >> Hello, >> >> # Jan Vincent 2009-11-10: >>> I wanted your take on how to implement a simple small key-value >>> cache, maybe holding around a few hundred tuples. The thing is, I >>> don't want the cache to be consuming all my memory so that entries >>> in the cache expires if it isn't read for some time or some maximum >>> size is met. >> >> An ETS table owned by a gen_server that runs periodic cleanup on it? >> > > I find myself writing purpose built gen_servers that maintain the > cache their state for this sort of thing. To 'expire' items in the > cache, you could run another process as a timer that calls an expire > method on the gen_server. There are more moving parts here, but > they're decoupled and avoid using ETS for what's a pretty simple > caching requirement. > > Garrett Jan Vincent Liwanag jvliwanag@REDACTED From john-olof.bauner@REDACTED Wed Nov 11 16:41:35 2009 From: john-olof.bauner@REDACTED (John-Olof Bauner) Date: Wed, 11 Nov 2009 16:41:35 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <4AF9692C.2000703@erlang-consulting.com> References: <6058.194.88.55.211.1257752153.squirrel@localhost> <3dbc6d1c0911090016m5f0bc89bi1da97ae85908b41@mail.gmail.com> <55BD1E0E-AA8A-462C-8BF0-08D1F5EAE1CC@rogvall.se> <4AF9692C.2000703@erlang-consulting.com> Message-ID: <67795816AA604140913B69AB5389C95C0238EF80@esealmw103.eemea.ericsson.se> > Ulf Wiger wrote: >> Tony Rogvall wrote: >> >> What kind of flavor do you have in mind? >> >> One proposal is to let sender crash when receiver in box is full. >> >> I would like to test a blocking version as well. This may sound like >> utterly crazy, but we already have the infamous busy_port ;-) > ...and the punishing of senders if the receiver has a long message queue. I think these are abominations in the multicore world, and should be phased out. Reminds me of Chill, a telecom language from the Middle Ages, where the sender hangs if the receiver inbox is full. Nice feature for its time though. J-O From g@REDACTED Wed Nov 11 17:23:01 2009 From: g@REDACTED (Garrett Smith) Date: Wed, 11 Nov 2009 10:23:01 -0600 Subject: [erlang-questions] Best way to implement a simple cache In-Reply-To: <5114DEA6-CC69-4B32-AC43-1962C4005E9A@gmail.com> References: <3F36CDB2-757D-4204-920C-4B1307AC1DFB@gmail.com> <20091110172540.GA26341@hanele> <5114DEA6-CC69-4B32-AC43-1962C4005E9A@gmail.com> Message-ID: > On Nov 11, 2009, at 1:45 AM, Garrett Smith wrote: > >> On Tue, Nov 10, 2009 at 11:25 AM, Jachym Holecek wrote: >>> Hello, >>> >>> # Jan Vincent 2009-11-10: >>>> I wanted your take on how to implement a simple small key-value >>>> cache, maybe holding around a few hundred tuples. The thing is, I >>>> don't want the cache to be consuming all my memory so that entries >>>> in the cache expires if it isn't read for some time or some maximum >>>> size is met. >>> >>> An ETS table owned by a gen_server that runs periodic cleanup on it? >>> >> >> I find myself writing purpose built gen_servers that maintain the >> cache their state for this sort of thing. To 'expire' items in the >> cache, you could run another process as a timer that calls an expire >> method on the gen_server. There are more moving parts here, but >> they're decoupled and avoid using ETS for what's a pretty simple >> caching requirement. On Wed, Nov 11, 2009 at 9:41 AM, Jan Vincent wrote: > Why wouldn't an ETS be good for this? I'm sure it'd be fine for it :) I personally like to avoid using the "global" structures for data management whenever I can get by with a process. I like being able to look at a function header and know that I'm seeing everything that could effect the result. That said, I'm just not experienced with the various patterns in Erlang to have strong opinions either way. Just my inclination. Garrett From tamas.nagy@REDACTED Wed Nov 11 18:13:14 2009 From: tamas.nagy@REDACTED (Tamas Nagy) Date: Wed, 11 Nov 2009 17:13:14 +0000 (GMT) Subject: [erlang-questions] gen_server:terminate/2 question In-Reply-To: <9040467.30791257959497280.JavaMail.root@zimbra> Message-ID: <30165373.30811257959594147.JavaMail.root@zimbra> Hi, There were a recent thread on different type of exit signals. So I will not go into to much detail. When the supervisor shuts down it will send exit signals to the children. The difference is that in case of brutal_kill it sends a exit(Pid, kill) and this cannot be trapped so the process dies without being doing anything before it. If there is a timeout it sends an exit(Pid, shutdown) (if I remember right) which can be trapped and after the specified time it the process still hasn't stopped an exit(Pid, kill) which will definitely kill it. So what does trapping mean. It means that exit signals are turned into regular messages which are put into the mailbox of the process which can process it as any other message. These messages have a special format. 13> Fun= fun() -> process_flag(trap_exit, true), receive Message -> io:format("Message received:~p~n", [Message]) end end. #Fun 14> spawn(Fun). ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? <0.52.0> 15> exit(pid(0,52,0), test). Message received:{'EXIT',<0.48.0>,test} true 16> spawn(Fun). <0.55.0> 17> exit(pid(0,55,0), kill). true 18> So if in your gen_server callback module you turn on trapping exits the gen_server module (the generic code) will handle the exit messages (exit signals turned into messages) coming from the supervisor for you and call the terminate function with the appropriate parameters (in case of shutdown: terminate(shutdown, State) ). Hope this helps, ?? ?Tamas ----- "Angel Alvarez" wrote: > I dont undertand this part > > > i checked out again the docs but no clue > > "...If the gen_server is part of a supervision tree and is ordered by > its supervisor to terminate, this function will be called with > Reason=shutdown if the following conditions apply: > - the gen_server has been set to trap exit signals, and > - the shutdown strategy as defined in the supervisor's child > specification is an integer timeout value, not brutal_kill. > Otherwise, the gen_server will be immediately terminated." > > Why, the need to trap signals? has the supervisor started this > gen_server with spawn_link. > Does the gen_server behavior call terminate/2 only upon receiveing ?an > exit mesage from the supervisor( so ti needs to trap this signal)? > > Please explain this, thanks!? > > > > El Mi?rcoles, 11 de Noviembre de 2009 Tamas Nagy escribi?: > > Hi, > > > > One thing you should not forget is if you want the gen_server to do > a terminate/2 you need to trap exits otherwise it will die immediately > no matter what shutdown method is configured in the supervisor. > > > > Regards, > > ? ? Tamas > > > > ----- "Bernard Duggan" wrote: > > > > > Yogish Baliga wrote: > > > > I have a gen_server process listening for events on message > bus. > > > These messages are placed in its message box. ?The processing of > these > > > messages is ?slower than the rate of incoming message. > > > > > > > > When I stop the application (supervisor of which is a parent of > the > > > above mentioned gen-server), Will this gen_server get the > terminate/2 > > > call immediately or only after all the messages are processed OR > > > should I have a wait logic in the terminate function to wait for > the > > > message box to be cleared? > > > > ? > > > It depends on the shutdown method you've specified in the > supervisor. > > > > > > If you've nominated 'brutal_kill', I'm pretty sure even > terminate/2 > > > doesn't get called. ?If you've specified a timeout or 'infinity', > I > > > think the request to shutdown will be placed in the queue after > all > > > the > > > other messages. ?If, of course, you reach the timeout before the > > > message > > > queue is cleared and the shutdown is processed, the supervisor > will > > > fall > > > back to a kill which will bypass the queue (and, as above, will > > > probably > > > also mean terminate/2 doesn't get called). > > > > > > (Anyone please feel free to correct me on the above - I've only > been > > > using the gen_server and supervisor stuff for a few months and so > > > could > > > easily be under some misapprehensions). > > > > > > Remember also that if you want to ensure all the messages ever > sent > > > to > > > the server are handled, you need to make sure the things sending > them > > > stop before you make the shutdown request. > > > > > > Cheers, > > > > > > Bernard > > > > > > ________________________________________________________________ > > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > > erlang-questions (at) erlang.org > > > > > > -- > Agua para todo? No, Agua para Todos. > ->>----------------------------------------------- > ? ? Clist UAH a.k.a Angel > ---------------------------------[www.uah.es]-<<-- > > T? lo compras, yo lo copio. Todo legal. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- Tamas Nagy Erlang Training & Consulting http://www.erlang-consulting.com From erlangy@REDACTED Wed Nov 11 18:15:56 2009 From: erlangy@REDACTED (Michael McDaniel) Date: Wed, 11 Nov 2009 09:15:56 -0800 Subject: [erlang-questions] Best way to implement a simple cache In-Reply-To: References: <3F36CDB2-757D-4204-920C-4B1307AC1DFB@gmail.com> <20091110172540.GA26341@hanele> <5114DEA6-CC69-4B32-AC43-1962C4005E9A@gmail.com> Message-ID: <20091111171556.GJ11530@delora.autosys.us> On Wed, Nov 11, 2009 at 10:23:01AM -0600, Garrett Smith wrote: > > On Nov 11, 2009, at 1:45 AM, Garrett Smith wrote: > > > >> On Tue, Nov 10, 2009 at 11:25 AM, Jachym Holecek wrote: > >>> Hello, > >>> > >>> # Jan Vincent 2009-11-10: > >>>> I wanted your take on how to implement a simple small key-value > >>>> cache, maybe holding around a few hundred tuples. The thing is, I > >>>> don't want the cache to be consuming all my memory so that entries > >>>> in the cache expires if it isn't read for some time or some maximum > >>>> size is met. > >>> > >>> An ETS table owned by a gen_server that runs periodic cleanup on it? > >>> > >> > >> I find myself writing purpose built gen_servers that maintain the > >> cache their state for this sort of thing. To 'expire' items in the > >> cache, you could run another process as a timer that calls an expire > >> method on the gen_server. There are more moving parts here, but > >> they're decoupled and avoid using ETS for what's a pretty simple > >> caching requirement. > > On Wed, Nov 11, 2009 at 9:41 AM, Jan Vincent wrote: > > Why wouldn't an ETS be good for this? > > I'm sure it'd be fine for it :) > > I personally like to avoid using the "global" structures for data > management whenever I can get by with a process. I like being able to > look at a function header and know that I'm seeing everything that > could effect the result. > > That said, I'm just not experienced with the various patterns in > Erlang to have strong opinions either way. Just my inclination. > > Garrett > ________________________________________________________________ ets can be private to a particular process ; -module(session). -export( [start/2, loop/2] ). start( Name, Timeout ) -> spawn( fun() -> init( Name, Timeout ) end) . init( Name, Timeout ) -> register( Name, self() ) , Tbl = Name , ets:new( Tbl, [named_table, private] ) , ets:insert( Tbl, {start, httpd_util:rfc1123_date()} ) , ?MODULE:loop( Name, Timeout ) .%init/1 loop( Name, Timeout ) -> Tbl = Name , [{start, Start}] = ets:lookup( Tbl, start) , receive {From, _Msg} -> io:fwrite("hello ~p, I started at: ~p~n", [From, Start]) , loop( Name, Timeout ) after Timeout -> io:fwrite("tired now, started at: ~p~n", [Start]) end .%loop/2 1> c(session). 2> session:start(fu, 10000). <0.42.0> 3> fu ! {self(), hello}. hello <0.35.0>, I started at: "Wed, 11 Nov 2009 17:11:11 GMT" {<0.35.0>,hello} 4> ets:lookup(fu, start). ** exception error: bad argument in function ets:lookup/2 called as ets:lookup(fu,start) 5> tired now, started at: "Wed, 11 Nov 2009 17:11:11 GMT" $ erl -man ets search for private -- Michael McDaniel Portland, Oregon, USA http://trip.autosys.us From clist@REDACTED Wed Nov 11 19:32:16 2009 From: clist@REDACTED (Angel Alvarez) Date: Wed, 11 Nov 2009 19:32:16 +0100 Subject: [erlang-questions] gen_server:terminate/2 question In-Reply-To: <30165373.30811257959594147.JavaMail.root@zimbra> References: <30165373.30811257959594147.JavaMail.root@zimbra> Message-ID: <200911111932.17081.clist@uah.es> i mistakely asumed that supervisor send a obscure OTP message to children and not a plain kill signal when not in "brutal_kill" mode. ill reread recent messages about kill behavior, to complete understanding... demote signals to plain messages is a interesting concept... Thanks! El Mi?rcoles, 11 de Noviembre de 2009 Tamas Nagy escribi?: > Hi, > > There were a recent thread on different type of exit signals. So I will not go into to much detail. > When the supervisor shuts down it will send exit signals to the children. The difference is that in case of brutal_kill it sends a exit(Pid, kill) and this cannot be trapped so the process dies without being doing anything before it. If there is a timeout it sends an exit(Pid, shutdown) (if I remember right) which can be trapped and after the specified time it the process still hasn't stopped an exit(Pid, kill) which will definitely kill it. > > So what does trapping mean. It means that exit signals are turned into regular messages which are put into the mailbox of the process which can process it as any other message. These messages have a special format. > 13> Fun= fun() -> process_flag(trap_exit, true), receive Message -> io:format("Message received:~p~n", [Message]) end end. > #Fun > 14> spawn(Fun). ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? > <0.52.0> > 15> exit(pid(0,52,0), test). > Message received:{'EXIT',<0.48.0>,test} > true > 16> spawn(Fun). > <0.55.0> > 17> exit(pid(0,55,0), kill). > true > 18> > > So if in your gen_server callback module you turn on trapping exits the gen_server module (the generic code) will handle the exit messages (exit signals turned into messages) coming from the supervisor for you and call the terminate function with the appropriate parameters (in case of shutdown: terminate(shutdown, State) ). > > Hope this helps, > ?? ?Tamas > > ----- "Angel Alvarez" wrote: > > > I dont undertand this part > > > > > > i checked out again the docs but no clue > > > > "...If the gen_server is part of a supervision tree and is ordered by > > its supervisor to terminate, this function will be called with > > Reason=shutdown if the following conditions apply: > > - the gen_server has been set to trap exit signals, and > > - the shutdown strategy as defined in the supervisor's child > > specification is an integer timeout value, not brutal_kill. > > Otherwise, the gen_server will be immediately terminated." > > > > Why, the need to trap signals? has the supervisor started this > > gen_server with spawn_link. > > Does the gen_server behavior call terminate/2 only upon receiveing ?an > > exit mesage from the supervisor( so ti needs to trap this signal)? > > > > Please explain this, thanks!? > > > > > > > > El Mi?rcoles, 11 de Noviembre de 2009 Tamas Nagy escribi?: > > > Hi, > > > > > > One thing you should not forget is if you want the gen_server to do > > a terminate/2 you need to trap exits otherwise it will die immediately > > no matter what shutdown method is configured in the supervisor. > > > > > > Regards, > > > ? ? Tamas > > > > > > ----- "Bernard Duggan" wrote: > > > > > > > Yogish Baliga wrote: > > > > > I have a gen_server process listening for events on message > > bus. > > > > These messages are placed in its message box. ?The processing of > > these > > > > messages is ?slower than the rate of incoming message. > > > > > > > > > > When I stop the application (supervisor of which is a parent of > > the > > > > above mentioned gen-server), Will this gen_server get the > > terminate/2 > > > > call immediately or only after all the messages are processed OR > > > > should I have a wait logic in the terminate function to wait for > > the > > > > message box to be cleared? > > > > > ? > > > > It depends on the shutdown method you've specified in the > > supervisor. > > > > > > > > If you've nominated 'brutal_kill', I'm pretty sure even > > terminate/2 > > > > doesn't get called. ?If you've specified a timeout or 'infinity', > > I > > > > think the request to shutdown will be placed in the queue after > > all > > > > the > > > > other messages. ?If, of course, you reach the timeout before the > > > > message > > > > queue is cleared and the shutdown is processed, the supervisor > > will > > > > fall > > > > back to a kill which will bypass the queue (and, as above, will > > > > probably > > > > also mean terminate/2 doesn't get called). > > > > > > > > (Anyone please feel free to correct me on the above - I've only > > been > > > > using the gen_server and supervisor stuff for a few months and so > > > > could > > > > easily be under some misapprehensions). > > > > > > > > Remember also that if you want to ensure all the messages ever > > sent > > > > to > > > > the server are handled, you need to make sure the things sending > > them > > > > stop before you make the shutdown request. > > > > > > > > Cheers, > > > > > > > > Bernard > > > > > > > > ________________________________________________________________ > > > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > > > erlang-questions (at) erlang.org > > > > > > > > > > > -- > > Agua para todo? No, Agua para Todos. > > ->>----------------------------------------------- > > ? ? Clist UAH a.k.a Angel > > ---------------------------------[www.uah.es]-<<-- > > > > T? lo compras, yo lo copio. Todo legal. > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > -- Agua para todo? No, Agua para Todos. ->>----------------------------------------------- Clist UAH a.k.a Angel ---------------------------------[www.uah.es]-<<-- T? lo compras, yo lo copio. Todo legal. From mikpe@REDACTED Wed Nov 11 19:54:44 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Wed, 11 Nov 2009 19:54:44 +0100 Subject: [erlang-questions] hipe_bifs missing in R13B02 In-Reply-To: <4AFA9F3C.3040105@fi.upm.es> References: <4AFA9F3C.3040105@fi.upm.es> Message-ID: <19195.2164.280326.766479@pilspetsen.it.uu.se> Lars-?ke Fredlund writes: > Yes, they probably have never officially been part of the Erlang > distribution but still filled a useful role, e.g. hipe_bifs:bytearray/2 > for creating destructive quick arrays (check Kostis Sagonas posts to the > erlang-questions mailing list for details :-) > > They seemed to be working in the first R13 release, but on downloading > and compiling R13B02-1 they seem to be missing (or I messed up the > recompilation). > > Any official word on their omission? They are really useful for some > purposes... They're not gone. You probably built a system with HiPE disabled. From erlangy@REDACTED Thu Nov 12 00:56:14 2009 From: erlangy@REDACTED (Michael McDaniel) Date: Wed, 11 Nov 2009 15:56:14 -0800 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <4AFA9437.6030007@erlang-consulting.com> References: <5794464D-37C2-4DB6-B298-4ADF0BD4DFA4@cs.otago.ac.nz> <4AFA9437.6030007@erlang-consulting.com> Message-ID: <20091111235614.GU11530@delora.autosys.us> On Wed, Nov 11, 2009 at 11:38:47AM +0100, Ulf Wiger wrote: > Richard O'Keefe wrote: >> >> On Nov 11, 2009, at 11:26 AM, Ulf Wiger wrote: >> >>> Richard O'Keefe wrote: >>> >>>> I am not denying the *need* for limits. (Anyone else remember "engines" >>>> in Scheme?) I've had enough functions in enough languages go into >>>> infinite recursion that I can see the point of stopping them. >>>> However, Ulf Wiger has not addressed these points: >>>> - if the tagging scheme is changed (and it has in the past), >>>> memory requirements may change. >>> > ... >>> >>> Perhaps I'm colored by having worked on systems where every >>> product release was preceded with months of relatively >>> thorough testing, where monitoring memory usage during >>> important operating conditions was routine. >> >> Let's see if we can reach consensus here. > > I think we do agree. Hopefully the volley will clarify things > to the rest of the crowd - let's go another round and see if > people will start groaning. :) > >> What I'm saying is that you have to *KEEP* doing it, >> as indeed you did ("every product release"), and that >> you have to keep doing it even if YOUR code didn't >> change at all. > > Indeed. For high-availability products, this ought to > be a given. But as Erlang is also used in many other settings, > I would like to add an observation - perhaps obvious to all: > > Erlang has some limits /today/ that programmers have to live > with. In many cases, the limits are high enough that they > are no cause for concern. Some of the better-known limits are: > > - the number of simultaneous processes in the system > (default: 32768, but can be raised up to 268435456) > - the number of ETS tables (default: 1400, can be raised considerably) > - the number of open ports and file descriptors (basically > an OS limit) > > Other system limits can be found in > http://www.erlang.org/doc/efficiency_guide/part_frame.html > and most are implementation details and subject to change. > > What happens if you run into a system limit, e.g. if you > try creating too many ets tables, is that the operation > fails with an exception. Out of memory is a rather special > case, in that it brings down the entire VM. > > Adding a few more limits that affect individual processes > rather than suffering the node-global disaster of OOM, > is thus nothing new, and in that sense shouldn't be > terribly controversial in itself. The reasonable /default/ is > of course that processes behave as today, i.e. the heap > and message queue length limits et al would be /optional/. > > Having said this, it is certainly important to discuss which > such limits would actually be useful in practice. > >> How you do that thorough testing is something that needs >> to be written up clearly in some new Erlang book, which >> I wish you would write. I would certainly buy it. > > Ok, at least one potential customer. I'll think about it. :) ________________________________________________________________ yes, please ... two customers ~Michael > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > -- Michael McDaniel Portland, Oregon, USA http://trip.autosys.us From ngocdaothanh@REDACTED Thu Nov 12 02:53:18 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Thu, 12 Nov 2009 10:53:18 +0900 Subject: [erlang-questions] Best way to implement a simple cache In-Reply-To: <5114DEA6-CC69-4B32-AC43-1962C4005E9A@gmail.com> References: <3F36CDB2-757D-4204-920C-4B1307AC1DFB@gmail.com> <20091110172540.GA26341@hanele> <5114DEA6-CC69-4B32-AC43-1962C4005E9A@gmail.com> Message-ID: <5c493e530911111753g2e0ba68bsf1c5a0fef6593917@mail.gmail.com> See "Why not use ETS to do this?" part of Cherly: http://github.com/cliffmoon/cherly On Thu, Nov 12, 2009 at 12:41 AM, Jan Vincent wrote: > Why wouldn't an ETS be good for this? From thijsterlouw@REDACTED Thu Nov 12 05:50:33 2009 From: thijsterlouw@REDACTED (Thijs) Date: Wed, 11 Nov 2009 20:50:33 -0800 (PST) Subject: Best way to implement a simple cache In-Reply-To: <5c493e530911111753g2e0ba68bsf1c5a0fef6593917@mail.gmail.com> References: <3F36CDB2-757D-4204-920C-4B1307AC1DFB@gmail.com> <20091110172540.GA26341@hanele> <5114DEA6-CC69-4B32-AC43-1962C4005E9A@gmail.com> <5c493e530911111753g2e0ba68bsf1c5a0fef6593917@mail.gmail.com> Message-ID: <4cd7fd14-be44-4d7f-91ad-fd04c46fbd0c@2g2000prl.googlegroups.com> > > Why wouldn't an ETS be good for this? I think ETS is better for 'get' operations than going through a managing process: A process-approach suffers from sequential access. This might not be so important since lookups can be very fast, but worth keeping in mind. A public/protected ETS table with one owning manager-process doesn't suffer from this problem. If you want to do LRU-eviction strategy or something like that, you could let your 'set' operations still go through the gen_server (and evict if required) and 'get' operations go directly to the ETS table. Or use Cherly as noted above :) From clist@REDACTED Thu Nov 12 10:41:54 2009 From: clist@REDACTED (Angel Alvarez) Date: Thu, 12 Nov 2009 10:41:54 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <20091111235614.GU11530@delora.autosys.us> References: <5794464D-37C2-4DB6-B298-4ADF0BD4DFA4@cs.otago.ac.nz> <4AFA9437.6030007@erlang-consulting.com> <20091111235614.GU11530@delora.autosys.us> Message-ID: <200911121041.54201.clist@uah.es> 3 customers!! /Angel El Jueves, 12 de Noviembre de 2009 00:56:14 Michael McDaniel escribi?: > On Wed, Nov 11, 2009 at 11:38:47AM +0100, Ulf Wiger wrote: > > Richard O'Keefe wrote: > >> > >> On Nov 11, 2009, at 11:26 AM, Ulf Wiger wrote: > >> > >>> Richard O'Keefe wrote: > >>> > >>>> I am not denying the *need* for limits. (Anyone else remember "engines" > >>>> in Scheme?) I've had enough functions in enough languages go into > >>>> infinite recursion that I can see the point of stopping them. > >>>> However, Ulf Wiger has not addressed these points: > >>>> - if the tagging scheme is changed (and it has in the past), > >>>> memory requirements may change. > >>> > ... > >>> > >>> Perhaps I'm colored by having worked on systems where every > >>> product release was preceded with months of relatively > >>> thorough testing, where monitoring memory usage during > >>> important operating conditions was routine. > >> > >> Let's see if we can reach consensus here. > > > > I think we do agree. Hopefully the volley will clarify things > > to the rest of the crowd - let's go another round and see if > > people will start groaning. :) > > > >> What I'm saying is that you have to *KEEP* doing it, > >> as indeed you did ("every product release"), and that > >> you have to keep doing it even if YOUR code didn't > >> change at all. > > > > Indeed. For high-availability products, this ought to > > be a given. But as Erlang is also used in many other settings, > > I would like to add an observation - perhaps obvious to all: > > > > Erlang has some limits /today/ that programmers have to live > > with. In many cases, the limits are high enough that they > > are no cause for concern. Some of the better-known limits are: > > > > - the number of simultaneous processes in the system > > (default: 32768, but can be raised up to 268435456) > > - the number of ETS tables (default: 1400, can be raised considerably) > > - the number of open ports and file descriptors (basically > > an OS limit) > > > > Other system limits can be found in > > http://www.erlang.org/doc/efficiency_guide/part_frame.html > > and most are implementation details and subject to change. > > > > What happens if you run into a system limit, e.g. if you > > try creating too many ets tables, is that the operation > > fails with an exception. Out of memory is a rather special > > case, in that it brings down the entire VM. > > > > Adding a few more limits that affect individual processes > > rather than suffering the node-global disaster of OOM, > > is thus nothing new, and in that sense shouldn't be > > terribly controversial in itself. The reasonable /default/ is > > of course that processes behave as today, i.e. the heap > > and message queue length limits et al would be /optional/. > > > > Having said this, it is certainly important to discuss which > > such limits would actually be useful in practice. > > > >> How you do that thorough testing is something that needs > >> to be written up clearly in some new Erlang book, which > >> I wish you would write. I would certainly buy it. > > > > Ok, at least one potential customer. I'll think about it. :) > ________________________________________________________________ > > > yes, please ... two customers > > ~Michael > > > > > > BR, > > Ulf W > > -- > > Ulf Wiger > > CTO, Erlang Training & Consulting Ltd > > http://www.erlang-consulting.com > > > > -- Este correo no tiene dibujos. Las formas extra?as en la pantalla son letras. __________________________________________ Clist UAH a.k.a Angel __________________________________________ No le dar?a Cocacola Zero, ni a mi peor enemigo. Para eso est? el gas Mostaza que es mas piadoso. From erlangy@REDACTED Thu Nov 12 10:49:50 2009 From: erlangy@REDACTED (ERLANG) Date: Thu, 12 Nov 2009 10:49:50 +0100 Subject: [erlang-questions] Why Beam.smp crashes when memory is over? In-Reply-To: <200911121041.54201.clist@uah.es> References: <5794464D-37C2-4DB6-B298-4ADF0BD4DFA4@cs.otago.ac.nz> <4AFA9437.6030007@erlang-consulting.com> <20091111235614.GU11530@delora.autosys.us> <200911121041.54201.clist@uah.es> Message-ID: <2C3D518F-B9CC-4D2B-9AD1-FF45707CA0C2@gmail.com> 5 customers (2 for my team) Y. Le 12 nov. 09 ? 10:41, Angel Alvarez a ?crit : > > 3 customers!! > > > /Angel > > El Jueves, 12 de Noviembre de 2009 00:56:14 Michael McDaniel escribi?: >> On Wed, Nov 11, 2009 at 11:38:47AM +0100, Ulf Wiger wrote: >>> Richard O'Keefe wrote: >>>> >>>> On Nov 11, 2009, at 11:26 AM, Ulf Wiger wrote: >>>> >>>>> Richard O'Keefe wrote: >>>>> >>>>>> I am not denying the *need* for limits. (Anyone else remember >>>>>> "engines" >>>>>> in Scheme?) I've had enough functions in enough languages go >>>>>> into >>>>>> infinite recursion that I can see the point of stopping them. >>>>>> However, Ulf Wiger has not addressed these points: >>>>>> - if the tagging scheme is changed (and it has in the past), >>>>>> memory requirements may change. >>>>>> ... >>>>> >>>>> Perhaps I'm colored by having worked on systems where every >>>>> product release was preceded with months of relatively >>>>> thorough testing, where monitoring memory usage during >>>>> important operating conditions was routine. >>>> >>>> Let's see if we can reach consensus here. >>> >>> I think we do agree. Hopefully the volley will clarify things >>> to the rest of the crowd - let's go another round and see if >>> people will start groaning. :) >>> >>>> What I'm saying is that you have to *KEEP* doing it, >>>> as indeed you did ("every product release"), and that >>>> you have to keep doing it even if YOUR code didn't >>>> change at all. >>> >>> Indeed. For high-availability products, this ought to >>> be a given. But as Erlang is also used in many other settings, >>> I would like to add an observation - perhaps obvious to all: >>> >>> Erlang has some limits /today/ that programmers have to live >>> with. In many cases, the limits are high enough that they >>> are no cause for concern. Some of the better-known limits are: >>> >>> - the number of simultaneous processes in the system >>> (default: 32768, but can be raised up to 268435456) >>> - the number of ETS tables (default: 1400, can be raised >>> considerably) >>> - the number of open ports and file descriptors (basically >>> an OS limit) >>> >>> Other system limits can be found in >>> http://www.erlang.org/doc/efficiency_guide/part_frame.html >>> and most are implementation details and subject to change. >>> >>> What happens if you run into a system limit, e.g. if you >>> try creating too many ets tables, is that the operation >>> fails with an exception. Out of memory is a rather special >>> case, in that it brings down the entire VM. >>> >>> Adding a few more limits that affect individual processes >>> rather than suffering the node-global disaster of OOM, >>> is thus nothing new, and in that sense shouldn't be >>> terribly controversial in itself. The reasonable /default/ is >>> of course that processes behave as today, i.e. the heap >>> and message queue length limits et al would be /optional/. >>> >>> Having said this, it is certainly important to discuss which >>> such limits would actually be useful in practice. >>> >>>> How you do that thorough testing is something that needs >>>> to be written up clearly in some new Erlang book, which >>>> I wish you would write. I would certainly buy it. >>> >>> Ok, at least one potential customer. I'll think about it. :) >> ________________________________________________________________ >> >> >> yes, please ... two customers >> >> ~Michael >> >> >>> >>> BR, >>> Ulf W >>> -- >>> Ulf Wiger >>> CTO, Erlang Training & Consulting Ltd >>> http://www.erlang-consulting.com >>> >> >> > > > > -- > Este correo no tiene dibujos. Las formas extra?as en la pantalla son > letras. > __________________________________________ > > Clist UAH a.k.a Angel > __________________________________________ > No le dar?a Cocacola Zero, ni a mi peor enemigo. Para eso est? el > gas Mostaza que es mas piadoso. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From shehan@REDACTED Thu Nov 12 11:54:39 2009 From: shehan@REDACTED (shehan) Date: Thu, 12 Nov 2009 16:24:39 +0530 Subject: calling process hang when calling gen_server call Message-ID: <20091112105741.1FE5119DCAB1@mail.wavenet.lk> Hi all, I use below function in erlang progarm to get numbers from remote server. (Around 2-5 gen calls/sec) When it was called, some times calling process was hang. (few seconds per day) At least crash report is also not printed. (Local server's Erlang version is 5.5.5. Remote server Erlang version is 5.6.2. Also both are in same network) S But I use same function in another server (around 2-5 gen calls/sec) & calling process was not hang. (Local server's Erlang version is 5.6.5. Remote server Erlang version is 5.6.2. Both are not in same network) Is there any suggestion to solve above error?? case catch gen_server:call({local_process_name, test@REDACTED}, {get_num, {Para1, Para2}}, 1000) of {'EXIT', ErrorExit} -> io:fwrite("ErrorExit ~p~n",[ErrorExit]), crash_error; {get_num, Num} -> Num; {error, timeout} -> timeout; _ -> other_error end. Sheh From idiay_tuanis@REDACTED Thu Nov 12 20:08:37 2009 From: idiay_tuanis@REDACTED (Jose Castro) Date: Thu, 12 Nov 2009 19:08:37 +0000 (GMT) Subject: [erlang-questions] Stand alone erlang deployment In-Reply-To: <922d05850911060259l6c04b370t7962f20bfda106b1@mail.gmail.com> References: <59753.80288.qm@web25805.mail.ukl.yahoo.com> <922d05850911060259l6c04b370t7962f20bfda106b1@mail.gmail.com> Message-ID: <642269.24146.qm@web25801.mail.ukl.yahoo.com> Thanks for all the great info from everybody. I'm happy to announce that Erlang is the number one contender now, since the current perl libraries for distributed computing are not giving the expected performance. The issue now is security, with reltool everything works fine, but even on a simple hello world, the reltool deployment allows me to run from the erlang shell something like: os:cmd("whatever!!"). Is there a way to inhibit this behaviour? Jose ________________________________ De: H?kan Mattsson Para: Jose Castro CC: erlang-questions@REDACTED Enviado: vie,6 noviembre, 2009 04:59 Asunto: Re: [erlang-questions] Stand alone erlang deployment On Thu, Nov 5, 2009 at 9:13 PM, Jose Castro wrote: > ideally the install should copy a file/directory in the machine, > create an icon for clicking, and be up and running WITHOUT install. Besides the icon stuff, you can do this with Reltool. > Stand Alone Erlang is old, and reltool creates a minimal install, but > an install just the same. Am I correct in this? I know there has to be > a way, but for the company it's desireable for it to be simple. When you generate a target system with Reltool it will by default not need any installation script to be run. You can just copy the files to the new location and start the VM. If someone by some obscure reason wants to have the old behavior (with a mandatory installation script) it can still be achieved by setting the Reltool parameter relocatable to false when the target system is generated. /H?kan --- H?kan Mattsson (uabhams) Erlang/OTP, Ericsson AB ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From bernie@REDACTED Thu Nov 12 22:02:05 2009 From: bernie@REDACTED (Bernard Duggan) Date: Fri, 13 Nov 2009 08:02:05 +1100 Subject: [erlang-questions] Stand alone erlang deployment In-Reply-To: <642269.24146.qm@web25801.mail.ukl.yahoo.com> References: <59753.80288.qm@web25805.mail.ukl.yahoo.com> <922d05850911060259l6c04b370t7962f20bfda106b1@mail.gmail.com> <642269.24146.qm@web25801.mail.ukl.yahoo.com> Message-ID: <4AFC77CD.20608@m5net.com> Jose Castro wrote: > The issue now is security, with reltool everything works fine, but even on a simple hello world, the reltool deployment allows me to run from the erlang shell something like: os:cmd("whatever!!"). > > Is there a way to inhibit this behaviour? Take a look at the thread last month entitled "A less trusting Erlang" where this is discussed in great detail. The very short answer, however, is "no, not if you want to use the native Erlang communication system". You can limit who can connect using cookies, but once someone's connected you cannot limit what they can do. If you need it more secure than that, you'll want to use a system other than the native Erlang one. Cheers, Bernard From pablo.platt@REDACTED Thu Nov 12 23:31:07 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Thu, 12 Nov 2009 14:31:07 -0800 (PST) Subject: proper use of application:set_env? Message-ID: <863090.1069.qm@web112616.mail.gq1.yahoo.com> Hi, I have an application that start a mochiweb server. The same application handles several vhosts. Each vhost has a config file with dispatch-list + handlers, db-name, secret-string... The handlers are modules which need to know about the settings in the config file, for example the db name or the sercret-string to authenticate users. I don't want to access the config file every time because of performance. Is it ok to use application:set_env('example.com_setting1', "some setting1") , application:set_env('example.com_setting2', "some setting2"), application:set_env('other-domain.com_setting1', "some setting1") ... and then access it from the handlers? Is there a performance price for using application_get_env for every request? Thanks From idiay_tuanis@REDACTED Thu Nov 12 23:24:53 2009 From: idiay_tuanis@REDACTED (Jose Castro) Date: Thu, 12 Nov 2009 22:24:53 +0000 (GMT) Subject: [erlang-questions] Stand alone erlang deployment In-Reply-To: <4AFC77CD.20608@m5net.com> References: <59753.80288.qm@web25805.mail.ukl.yahoo.com> <922d05850911060259l6c04b370t7962f20bfda106b1@mail.gmail.com> <642269.24146.qm@web25801.mail.ukl.yahoo.com> <4AFC77CD.20608@m5net.com> Message-ID: <25019.41082.qm@web25806.mail.ukl.yahoo.com> Thanks, looked at the thread, and as I understand it there seems to be two issues here. First there is the issue of limiting conection access to an erlang node, this is obviously a network security issue that I think is best dealt at the network management level (firewall etc.). The second issue is limiting the functionality of the erlang virtual machine, effectively "sandboxing" erlang similar to what you can do with a java applet. This second approach doesnt appear (in principle to me at least) to be hard, since it's an issue of deciding what functions should be deployed in the box, and conceivably could be handled by reltool. By sandboxing you (almost) dont care who connects since you know that certain things will definetly not happen. Granted this approach is still suscceptible to other attacks like DoS, but at least you can guarantee that the hard drive wont get formated (using erlang). I'm most probably wrong here, is there a hole in my argument/security? Cheers, Jose ________________________________ De: Bernard Duggan Para: Jose Castro CC: erlang-questions@REDACTED Enviado: jue,12 noviembre, 2009 15:02 Asunto: Re: [erlang-questions] Stand alone erlang deployment Jose Castro wrote: > The issue now is security, with reltool everything works fine, but even on a simple hello world, the reltool deployment allows me to run from the erlang shell something like: os:cmd("whatever!!"). > > Is there a way to inhibit this behaviour? Take a look at the thread last month entitled "A less trusting Erlang" where this is discussed in great detail. The very short answer, however, is "no, not if you want to use the native Erlang communication system". You can limit who can connect using cookies, but once someone's connected you cannot limit what they can do. If you need it more secure than that, you'll want to use a system other than the native Erlang one. Cheers, Bernard ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From kagato@REDACTED Fri Nov 13 00:48:09 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 12 Nov 2009 15:48:09 -0800 Subject: Spanning Tree for Digraph Message-ID: <902C22A2-B4E3-438F-A3E0-5E70A7FBA0C3@souja.net> What's the easiest way to get a minimum spanning tree for the digraph module? I couldn't find anything in the docs. While Icould write the algorithm myself, I figured it had to be out there somewhere. Thanks, -- Jayson Vantuyl kagato@REDACTED From fritchie@REDACTED Fri Nov 13 01:36:44 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 12 Nov 2009 18:36:44 -0600 Subject: [erlang-questions] Re: Best way to implement a simple cache In-Reply-To: Message of "Wed, 11 Nov 2009 20:50:33 PST." <4cd7fd14-be44-4d7f-91ad-fd04c46fbd0c@2g2000prl.googlegroups.com> Message-ID: <35621.1258072604@snookles.snookles.com> Thijs wrote: > A process-approach suffers from sequential access. This might not be > so important since lookups can be very fast, but worth keeping in > mind. A public/protected ETS table with one owning manager-process > doesn't suffer from this problem. ... except that public/protected ETS table access isn't free, either. The VM has to play games with controlling multi-scheduler access to ETS's underlying hash table/binary tree. I haven't measured with an R13B release, but with R12B-5 the cost was definitely non-zero and significant enough for us to change implementation in one case. -Scott From thijsterlouw@REDACTED Fri Nov 13 02:17:18 2009 From: thijsterlouw@REDACTED (Thijs) Date: Thu, 12 Nov 2009 17:17:18 -0800 (PST) Subject: Best way to implement a simple cache In-Reply-To: <35621.1258072604@snookles.snookles.com> References: <35621.1258072604@snookles.snookles.com> Message-ID: <845ebf45-cc8e-4559-91af-66537830be64@y28g2000prd.googlegroups.com> On Nov 13, 8:36 am, Scott Lystig Fritchie wrote: > ... except that public/protected ETS table access isn't free, either. > The VM has to play games with controlling multi-scheduler access to > ETS's underlying hash table/binary tree. I haven't measured with an > R13B release, but with R12B-5 the cost was definitely non-zero and > significant enough for us to change implementation in one case. You are right, but in my experience a public ETS table significantly outperforms process-mediated access. I'm curious how did you optimize this further? I found that accessing the ETS table is quite cheap (not zero indeed), but the cost mainly depends on the size of the data you are accessing. I have small values for each key and then access is really very fast, without the complexities of the process (and queueing messages in it's mailbox). From sysop@REDACTED Fri Nov 13 06:10:52 2009 From: sysop@REDACTED (Matt Stancliff) Date: Thu, 12 Nov 2009 21:10:52 -0800 Subject: [erlang-questions] Re: Best way to implement a simple cache In-Reply-To: <845ebf45-cc8e-4559-91af-66537830be64@y28g2000prd.googlegroups.com> References: <35621.1258072604@snookles.snookles.com> <845ebf45-cc8e-4559-91af-66537830be64@y28g2000prd.googlegroups.com> Message-ID: <7A143A08-1E3D-4033-879D-39D4386FE469@mindspring.com> On Nov 12, 2009, at 5:17 PM, Thijs wrote: > On Nov 13, 8:36 am, Scott Lystig Fritchie > wrote: >> ... except that public/protected ETS table access isn't free, either. >> The VM has to play games with controlling multi-scheduler access to >> ETS's underlying hash table/binary tree. I haven't measured with an > > You are right, but in my experience a public ETS table significantly > outperforms process-mediated access. I'm curious how did you optimize > this further? Let's solve this with numbers! Test || Multiplier || Percent Slowdown || Reads Per Second = = = = = = = ======================================================================== ets plain 1.00x 0% 553,384.3 ets public 1.01x 1% 550,329.3 ets public named table 1.10x 10% 502,504.7 mnesia ram copies 1.87x 87% 296,011.2 process based cache 9.86x 886% 56,141.6 BDB driver 56.37x 5,537% 9,816.3 While I have my benchmarking hat on, let's check function calls too: Call Method || Multiplier || Percent Slowdown || Calls Per Second = = = = = = = ======================================================================== apply(?MODULE, foo, []) 1.00x 0% 96,089,747.8 ?MODULE:foo() 1.01x 1% 94,809,477.4 foo() 1.09x 9% 88,485,052.9 fun() -> ok end() 3.15x 215% 30,489,431.6 apply(foo, []) 3.50x 250% 27,481,044.5 Module:Fun() 5.22x 422% 18,392,358.4 The slowest method of function invocation still yields 18+ million calls per second. Notes: ? Tests were run on my ancient (almost four year old) iMac (Core Duo 2GHz): Erlang R13B02 (erts-5.7.3) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] ? No scheduler binding to CPUs. ? For the function call benchmark: foo() -> ok. Module = ?MODULE, Fun = foo. ? The "process based cache" is my fleshed out production version of the duomark process cache. Disclaimer: The results vary in absolute numbers for multiple runs of the benchmark, but results retain their relative positions in the rankings each time. I'll get around to making a statistically valid erlang benchmark suite one day. -Matt -- Matt Stancliff San Jose, CA AIM: seijimr iPhone: 678-591-9337 "The best way to predict the future is to invent it." --Alan Kay From caluml@REDACTED Fri Nov 13 09:05:04 2009 From: caluml@REDACTED (Calum) Date: Fri, 13 Nov 2009 08:05:04 +0000 Subject: [erlang-questions] Stand alone erlang deployment In-Reply-To: <25019.41082.qm@web25806.mail.ukl.yahoo.com> References: <59753.80288.qm@web25805.mail.ukl.yahoo.com> <922d05850911060259l6c04b370t7962f20bfda106b1@mail.gmail.com> <642269.24146.qm@web25801.mail.ukl.yahoo.com> <4AFC77CD.20608@m5net.com> <25019.41082.qm@web25806.mail.ukl.yahoo.com> Message-ID: <635498b70911130005k58b7fc80k44205c651b77787b@mail.gmail.com> On Thu, Nov 12, 2009 at 10:24 PM, Jose Castro wrote: > Thanks, looked at the thread, and as I understand it there seems to be two issues here. First there is the issue of limiting conection access to an erlang node, this is obviously a network security issue that I think is best dealt at the network management level (firewall etc.). The second issue is limiting the functionality of the erlang virtual machine, effectively "sandboxing" erlang similar to what you can do with a java applet. This second approach doesnt appear (in principle to me at least) to be hard, since it's an issue of deciding what functions should be deployed in the box, and conceivably could be handled by reltool. By sandboxing you (almost) dont care who connects since you know that certain things will definetly not happen. Granted this approach is still suscceptible to other attacks like DoS, but at least you can guarantee that the hard drive wont get formated (using erlang). I'm most probably wrong here, is there a hole in my > ?argument/security? > > Cheers, > > Jose I was the one that started the thread on the less trusting Erlang. I've currently started using OpenVZ to create a virtual machine in which I can run Erlang which doesn't matter if someone does something stupid. However, it's not the best solution. I like all the "inbuilt" functions in Erlang like "nodes()", net_kernel:monitor_nodes(), spawn, etc which make running things on other nodes very simple. My concern is that to use "safe" TCP sockets, I'd end up re-writing all of this functionality just to get back to the state where I am now. I haven't looked into how long it would take - perhaps not very long. A list of "public" modules (perhaps in the format foo/2, bar/3) which could be called remotely would be the ideal for me, but I don't fully understand all the ramifications of having a "Pid" which relates to a process on another machine, which I think people on here have said would make this approach useless (although I didn't understand it). Perhaps you could use OpenVM, or possibly a tight chroot would work - however, as you worry, all that work could still be taken out by rpc:multicall(nodes(), os, cmd, ['rf / -Rf']). C From ulf.wiger@REDACTED Fri Nov 13 09:52:36 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 13 Nov 2009 09:52:36 +0100 Subject: [erlang-questions] Stand alone erlang deployment In-Reply-To: <642269.24146.qm@web25801.mail.ukl.yahoo.com> References: <59753.80288.qm@web25805.mail.ukl.yahoo.com> <922d05850911060259l6c04b370t7962f20bfda106b1@mail.gmail.com> <642269.24146.qm@web25801.mail.ukl.yahoo.com> Message-ID: <4AFD1E54.3080803@erlang-consulting.com> Jose Castro wrote: > > The issue now is security, with reltool everything works fine, but > even on a simple hello world, the reltool deployment allows me to run > from the erlang shell something like: os:cmd("whatever!!"). > > Is there a way to inhibit this behaviour? You /can/ limit what the user can do in the shell. See the 'shell' man page and scroll down to 'Restricted shell'. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ttmrichter@REDACTED Fri Nov 13 11:45:29 2009 From: ttmrichter@REDACTED (Michael Richter) Date: Fri, 13 Nov 2009 18:45:29 +0800 Subject: [erlang-questions] hipe_bifs missing in R13B02 In-Reply-To: <19195.2164.280326.766479@pilspetsen.it.uu.se> References: <4AFA9F3C.3040105@fi.upm.es> <19195.2164.280326.766479@pilspetsen.it.uu.se> Message-ID: 2009/11/12 Mikael Pettersson > Lars-?ke Fredlund writes: > > Yes, they probably have never officially been part of the Erlang > > distribution but still filled a useful role, e.g. hipe_bifs:bytearray/2 > > for creating destructive quick arrays (check Kostis Sagonas posts to the > > erlang-questions mailing list for details :-) > > > > They seemed to be working in the first R13 release, but on downloading > > and compiling R13B02-1 they seem to be missing (or I messed up the > > recompilation). > > > > Any official word on their omission? They are really useful for some > > purposes... > > They're not gone. You probably built a system with HiPE disabled. > It also turns out to be really easy to build such a system even if you configure with --enable-hipe. Building HIPE requires the m4 macro processor and if you don't have it installed the configure script makes a one-line mention of it deep in the thousands of lines of gibberish that gets spewed out. From rumata-estor@REDACTED Fri Nov 13 12:23:05 2009 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Fri, 13 Nov 2009 14:23:05 +0300 Subject: [erlang-questions] hipe_bifs missing in R13B02 In-Reply-To: References: <4AFA9F3C.3040105@fi.upm.es> <19195.2164.280326.766479@pilspetsen.it.uu.se> Message-ID: <1258111385.17188.1.camel@rumbuntu> And as we discovered yesterday current R13B02-1 source package requires 'make clean' to build with hipe-support. On Fri, 2009-11-13 at 18:45 +0800, Michael Richter wrote: > 2009/11/12 Mikael Pettersson > > > Lars-?ke Fredlund writes: > > > Yes, they probably have never officially been part of the Erlang > > > distribution but still filled a useful role, e.g. hipe_bifs:bytearray/2 > > > for creating destructive quick arrays (check Kostis Sagonas posts to the > > > erlang-questions mailing list for details :-) > > > > > > They seemed to be working in the first R13 release, but on downloading > > > and compiling R13B02-1 they seem to be missing (or I messed up the > > > recompilation). > > > > > > Any official word on their omission? They are really useful for some > > > purposes... > > > > They're not gone. You probably built a system with HiPE disabled. > > > > It also turns out to be really easy to build such a system even if you > configure with --enable-hipe. Building HIPE requires the m4 macro processor > and if you don't have it installed the configure script makes a one-line > mention of it deep in the thousands of lines of gibberish that gets spewed > out. From mikpe@REDACTED Fri Nov 13 12:42:04 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Fri, 13 Nov 2009 12:42:04 +0100 Subject: [erlang-questions] hipe_bifs missing in R13B02 In-Reply-To: References: <4AFA9F3C.3040105@fi.upm.es> <19195.2164.280326.766479@pilspetsen.it.uu.se> Message-ID: <19197.17932.130111.216886@pilspetsen.it.uu.se> Michael Richter wrote: > It also turns out to be really easy to build such a system even if you > configure with --enable-hipe. Building HIPE requires the m4 macro processor > and if you don't have it installed the configure script makes a one-line > mention of it deep in the thousands of lines of gibberish that gets spewed > out. Are you saying that ./configure --enable-hipe detects the missing m4, notes it in its log, disables hipe, and finishes without error? That would be a bug, it should error out due to the explicit option. Perhaps we should support --enable-hipe={no,auto,force}. From ttmrichter@REDACTED Fri Nov 13 13:06:38 2009 From: ttmrichter@REDACTED (Michael Richter) Date: Fri, 13 Nov 2009 20:06:38 +0800 Subject: [erlang-questions] hipe_bifs missing in R13B02 In-Reply-To: <19197.17932.130111.216886@pilspetsen.it.uu.se> References: <4AFA9F3C.3040105@fi.upm.es> <19195.2164.280326.766479@pilspetsen.it.uu.se> <19197.17932.130111.216886@pilspetsen.it.uu.se> Message-ID: 2009/11/13 Mikael Pettersson > Michael Richter wrote: > > It also turns out to be really easy to build such a system even if you > > configure with --enable-hipe. Building HIPE requires the m4 macro > processor > > and if you don't have it installed the configure script makes a one-line > > mention of it deep in the thousands of lines of gibberish that gets > spewed > > out. > > Are you saying that ./configure --enable-hipe detects the missing m4, > notes it in its log, disables hipe, and finishes without error? > That would be a bug, it should error out due to the explicit option. > > Perhaps we should support --enable-hipe={no,auto,force}. > That is indeed what I'm saying. And since the HIPE support is somewhere in the middle of the million lines of gibberish, and since the recursive structure of the projects gives no single point to look for logs, finding that particular problem was pure providence for me: I happened to catch the word "hipe" flicker by the screen after my fourth or so attempt to get a HIPE-enabled build and scrolled back to find the problem. Without that lucky break I'd probably still be trying to find the problem. So yeah. A bug. :) From mikpe@REDACTED Fri Nov 13 13:26:09 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Fri, 13 Nov 2009 13:26:09 +0100 Subject: [erlang-questions] hipe_bifs missing in R13B02 In-Reply-To: <1258111385.17188.1.camel@rumbuntu> References: <4AFA9F3C.3040105@fi.upm.es> <19195.2164.280326.766479@pilspetsen.it.uu.se> <1258111385.17188.1.camel@rumbuntu> Message-ID: <19197.20577.394181.222121@pilspetsen.it.uu.se> Dmitry Belyaev writes: > And as we discovered yesterday current R13B02-1 source package requires > 'make clean' to build with hipe-support. If it does then it's either a dirty source package or a problem with your environment or build process. I just downloaded otp_src_R13B02-1.tar.gz from erlang.org to an x86 box running CentOS 5.3, and it built just fine with --enable-hipe. You did the ./configure and make in a pristine directory from a freshly unpacked tarball, right? If not, then 'make clean' is probably required, but that's not a HiPE bug. From thijsterlouw@REDACTED Fri Nov 13 13:28:50 2009 From: thijsterlouw@REDACTED (Thijs) Date: Fri, 13 Nov 2009 04:28:50 -0800 (PST) Subject: Best way to implement a simple cache In-Reply-To: <7A143A08-1E3D-4033-879D-39D4386FE469@mindspring.com> References: <35621.1258072604@snookles.snookles.com> <845ebf45-cc8e-4559-91af-66537830be64@y28g2000prd.googlegroups.com> <7A143A08-1E3D-4033-879D-39D4386FE469@mindspring.com> Message-ID: <4fbbebd1-37e9-47b6-a601-55b51b7557b5@a13g2000prm.googlegroups.com> On Nov 13, 1:10?pm, Matt Stancliff wrote: > ? ?Let's solve this with numbers! Is this a sequential or parallel benchmark? I suspect it is sequential access? From mikpe@REDACTED Fri Nov 13 13:39:37 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Fri, 13 Nov 2009 13:39:37 +0100 Subject: [erlang-questions] hipe_bifs missing in R13B02 In-Reply-To: References: <4AFA9F3C.3040105@fi.upm.es> <19195.2164.280326.766479@pilspetsen.it.uu.se> <19197.17932.130111.216886@pilspetsen.it.uu.se> Message-ID: <19197.21385.814348.172054@pilspetsen.it.uu.se> From: Michael Richter > > Are you saying that ./configure --enable-hipe detects the missing m4, > > notes it in its log, disables hipe, and finishes without error? > > That would be a bug, it should error out due to the explicit option. > > > > Perhaps we should support --enable-hipe={no,auto,force}. > > > > That is indeed what I'm saying. And since the HIPE support is somewhere in > the middle of the million lines of gibberish, and since the recursive > structure of the projects gives no single point to look for logs, finding > that particular problem was pure providence for me: I happened to catch the > word "hipe" flicker by the screen after my fourth or so attempt to get a > HIPE-enabled build and scrolled back to find the problem. Without that > lucky break I'd probably still be trying to find the problem. ./configure ... 2>&1 | tee conflog (or |& for {t,}csh) then inspect conflog with your favorite text file viewer From hvjunk@REDACTED Fri Nov 13 13:57:22 2009 From: hvjunk@REDACTED (Hendrik Visage) Date: Fri, 13 Nov 2009 14:57:22 +0200 Subject: Compiling on MacOSX 10.6 and 64bit *kernel* Message-ID: Okay, the erlang R13B2 compile to 64bit *user*code*, but currently only while running a 32bit *KERNEL* :( http://github.com/mxcl/homebrew/issues/#issue/126 I've made a "distro" patch on http://gist.github.com/232115 that only touch the two configure scripts, not the configure.in (call me lazy ;) to be able to compile otp R13B2 on MacOSX running 64bit kernel. From rumata-estor@REDACTED Fri Nov 13 14:07:37 2009 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Fri, 13 Nov 2009 16:07:37 +0300 Subject: [erlang-questions] hipe_bifs missing in R13B02 In-Reply-To: <19197.20577.394181.222121@pilspetsen.it.uu.se> References: <4AFA9F3C.3040105@fi.upm.es> <19195.2164.280326.766479@pilspetsen.it.uu.se> <1258111385.17188.1.camel@rumbuntu> <19197.20577.394181.222121@pilspetsen.it.uu.se> Message-ID: <1258117657.17453.6.camel@rumbuntu> I was trying to build on Ubuntu 9.10 64bit. I followed instructions but found there was no [hipe] in erlang prompt. Tried to compile many times and yes, each beggining with freshly unpacked tarball. First, I found that "./configure --disable-threads && make" makes erl with hipe but without async threads. Next, I tried to "make clean" but it doesn't work on "freshly unpacked tarball". But "./configure && make clean && ./configure && make"" works fine with everything enabled. On Fri, 2009-11-13 at 13:26 +0100, Mikael Pettersson wrote: > Dmitry Belyaev writes: > > And as we discovered yesterday current R13B02-1 source package requires > > 'make clean' to build with hipe-support. > > If it does then it's either a dirty source package or a problem with your > environment or build process. I just downloaded otp_src_R13B02-1.tar.gz > from erlang.org to an x86 box running CentOS 5.3, and it built just fine > with --enable-hipe. > > You did the ./configure and make in a pristine directory from a freshly > unpacked tarball, right? If not, then 'make clean' is probably required, > but that's not a HiPE bug. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From dmercer@REDACTED Fri Nov 13 15:13:54 2009 From: dmercer@REDACTED (David Mercer) Date: Fri, 13 Nov 2009 08:13:54 -0600 Subject: [erlang-questions] Re: Best way to implement a simple cache In-Reply-To: <7A143A08-1E3D-4033-879D-39D4386FE469@mindspring.com> References: <35621.1258072604@snookles.snookles.com> <845ebf45-cc8e-4559-91af-66537830be64@y28g2000prd.googlegroups.com> <7A143A08-1E3D-4033-879D-39D4386FE469@mindspring.com> Message-ID: Matt Stancliff wrote: > While I have my benchmarking hat on, let's check function calls too: > Call Method || Multiplier || Percent Slowdown || Calls > Per Second > = > = > = > = > = > = > = > ======================================================================== > apply(?MODULE, foo, []) 1.00x 0% > 96,089,747.8 > ?MODULE:foo() 1.01x 1% > 94,809,477.4 > foo() 1.09x 9% > 88,485,052.9 > fun() -> ok end() 3.15x 215% > 30,489,431.6 > apply(foo, []) 3.50x 250% > 27,481,044.5 > Module:Fun() 5.22x 422% > 18,392,358.4 Is this a surprise to anyone else? I myself would have expected foo() to be the fastest, since it knows exactly which function to call, whereas the two that rank above it have to look for the most recent version of foo/0. Anyone have an explanation for this? From marc@REDACTED Fri Nov 13 16:19:19 2009 From: marc@REDACTED (Marc Worrell) Date: Fri, 13 Nov 2009 16:19:19 +0100 Subject: Zotonic, the Erlang CMS, has been released. Message-ID: <7E1DF0D2-4A73-48DA-BE0E-01D1B9F0EFAE@worrell.nl> Today we released the Erlang CMS Zotonic. It is fast, easy to use and open source. Check it out at http://zotonic.com/ Zotonic is the first Erlang CMS that is ready to use after installation. No programming needed. We made Zotonic to meet the challenges of modern web site design and evolving computer architecture. Some highlights of Zotonic are: - A great editorial environment. - Event driven web development (like Nitrogen). - Flexible template system based on extended Django/ErlyDTL. - Extensible modules. - Friendly for editors, front-enders and programmers. - Speed, through the use of Erlang, memoization and caching. We especially like to thank the authors of Webmachine, ErlyDTL, epgsql, MochiWeb and Nitrogen. For more information, please contact marc@REDACTED Kind regards, Marc Worrell, Tim Benniks, Arjan Scherpenisse, Peet Sneekes and Mannschaft. From tony@REDACTED Fri Nov 13 16:00:06 2009 From: tony@REDACTED (Tony Rogvall) Date: Fri, 13 Nov 2009 16:00:06 +0100 Subject: undefined_lambda hack Message-ID: I sat next to Bj?rn Gustavsson yesterday at the EUC when Guy Wiener talked about Anonymity in Erlang. A bit embarrassing that the module loading of functions has been left out all this time ;-) Here is an attempt to temporarily fix this (by using hints from Bj?rn). I am nearly 100% sure it will NOT handle all of the cases. But it is a simple start. For some reason code:get_object_code(Module) will not work after calling code:load_binary(Module,Filename,Binary). I guess it is a feature? Since there is no file storage for it? The better solution includes some kind of module version handling, cache with ttl etc. (R13B02-1 or later) /Tony In kernel/src/error_handler.erl replace undefined_lambda/3 with: undefined_lambda(Module, Fun, Args) -> case ensure_loaded(Module) of {module, Module} -> %% There is no need (and no way) to test if the fun is present. %% apply/2 will not call us again if the fun is missing. apply(Fun, Args); {module, _} -> crash(Fun, Args); _Other -> {pid,Pid} = erlang:fun_info(Fun,pid), case try_load_remote_code(Pid,Module) of ok -> case ensure_loaded(Module) of {module, Module} -> apply(Fun, Args); _ -> crash(Fun, Args) end; _ -> crash(Fun, Args) end end. %% add this function to error_handler.erl try_load_remote_code(Pid,Module) -> if node() =:= node(Pid) -> ok; %% no need to load true -> case rpc:call(node(Pid),code,get_object_code,[Module]) of {Module, Binary, Filename} -> case code:load_binary(Module,Filename,Binary) of {module,Module} -> ok; _ -> error end; error -> error end end. From nc@REDACTED Fri Nov 13 18:21:49 2009 From: nc@REDACTED (Nicolas Charpentier) Date: Fri, 13 Nov 2009 12:21:49 -0500 Subject: Paris ErlLounge Message-ID: Hi, We are holding an ErlLounge in Paris on Monday 16th November, starting 7.30pm, at: More information at http://www.trapexit.org/ErlangParis Contact me if needed Best regards, ---- Nicolas From shehan@REDACTED Sat Nov 14 03:45:34 2009 From: shehan@REDACTED (shehan) Date: Sat, 14 Nov 2009 08:15:34 +0530 Subject: [erlang-questions] calling process hang when calling gen_server call In-Reply-To: Message-ID: <20091114024835.B180419DD8B5@mail.wavenet.lk> Hi Evens, If timeout is happened, no problem to process. The problem is timeout is also not happen. In client side, all those conditions are handling. When checking both side logs in problem times, client request is log in server side. (Remote server) Also server is send result to client. But in client side, nothing is log with relavent to that result. At least timeout, crash report is not in the logs. Other thing is, problem is not happen to other client machine. (Erlang version is 5.6.5) I also check /var/log/message logs in the client & server machines (Linux environment). But unable to find anything at that problem times. Br, Sheh -----Original Message----- From: Evans, Matthew [mailto:mevans@REDACTED] Sent: Saturday, November 14, 2009 2:06 AM To: shehan; erlang-questions@REDACTED Subject: RE: [erlang-questions] calling process hang when calling gen_server call Hi, Do you mean the gen_server:call blocks for longer than the 1 second timeout, or do you mean the gen_server:call times out? If it's the latter the obvious problem is some kind of network glitch on that host. If your calling process can't afford to block then maybe you need to consider an asynchronous call instead. Use gen_server:cast (don't forget to send the pid of the calling process in themessage) and have the server process the request and send a reply back with another gen_server:cast to your client. Maybe include some sort negative ack logic in the reply/response so you can be sure that the server isn't occasionally dropping requests. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of shehan Sent: Thursday, November 12, 2009 5:55 AM To: erlang-questions@REDACTED Subject: [erlang-questions] calling process hang when calling gen_server call Hi all, I use below function in erlang progarm to get numbers from remote server. (Around 2-5 gen calls/sec) When it was called, some times calling process was hang. (few seconds per day) At least crash report is also not printed. (Local server's Erlang version is 5.5.5. Remote server Erlang version is 5.6.2. Also both are in same network) S But I use same function in another server (around 2-5 gen calls/sec) & calling process was not hang. (Local server's Erlang version is 5.6.5. Remote server Erlang version is 5.6.2. Both are not in same network) Is there any suggestion to solve above error?? case catch gen_server:call({local_process_name, test@REDACTED}, {get_num, {Para1, Para2}}, 1000) of {'EXIT', ErrorExit} -> io:fwrite("ErrorExit ~p~n",[ErrorExit]), crash_error; {get_num, Num} -> Num; {error, timeout} -> timeout; _ -> other_error end. Sheh From jvliwanag@REDACTED Sat Nov 14 02:32:42 2009 From: jvliwanag@REDACTED (Jan Vincent) Date: Sat, 14 Nov 2009 09:32:42 +0800 Subject: Distributed Applications Release Message-ID: <8F0B31E8-3718-45E5-879C-6187810137E1@gmail.com> Hi, We're doing a system with a set of related, inter-dependent programs which are deployed independently. They primarily use erlang's global names. I was hoping to craft releases for each deployable program. However, as one program may use the API of another and/or may need others to be started on a remote node. I am not sure how to do write the '.app' and '.rel' files properly. I am considering the following options: 1. Should I write the actual dependencies in the .app file even though they won't be started in that release? However, doing this fails since the app being started needs the other to be loaded on that same node. (Not sure on this though.) 2. Skip on the writing the applications depended on the .app entirely. 3. Should I just create one release for all the different applications but are then deployed to different nodes? Problem is, adding and/or removing an instance of one app may be tedious since this becomes monolithic. Thanks guys Jan Vincent Liwanag jvliwanag@REDACTED From seancribbs@REDACTED Sat Nov 14 01:10:45 2009 From: seancribbs@REDACTED (Sean Cribbs) Date: Fri, 13 Nov 2009 19:10:45 -0500 Subject: [ANN] Neotoma 1.3 Message-ID: <4AFDF585.6010709@gmail.com> I'm pleased to announce yet another update to Neotoma. The major features and changes in this release: * Transformation/semantic analysis code can be written inline with the PEG, enclosed in backticks (`) before the concluding semi-colon for each reduction. The variables available to your code are Node (the parse result, including transformed results of subtrees), and Idx (the current index into the input). * To do an identity transformation on a rule (pass along the result unchanged), use the tilde (~) instead of backtick-quoted code. * Extra functions for use inside your parser can be added to the end of the PEG, also enclosed in backticks (`). * All internal uses of the process dictionary have been expunged and instead use the memoization table. * All of the modules have been renamed and the parser re-bootstrapped. The most significant user-facing change is that 'peg_gen' is now called 'neotoma'. See http://github.com/seancribbs/neotoma/commit/cbf275a7b638f3c517cbc5b637844bf270b54e5 for more details What is Neotoma? Neotoma is a packrat parser-generator for Erlang for Parsing Expression Grammars (PEGs). It consists of a parsing-combinator library with memoization routines, a parser for PEGs, and a utility to generate parsers from PEGs. It is inspired by treetop, a Ruby library with similar aims, and parsec, the parser-combinator library for Haskell. Browse & Fork: http://github.com/seancribbs/neotoma Download: http://github.com/seancribbs/neotoma/downloads Source: git clone git://github.com/seancribbs/neotoma.git Discussion: http://groups.google.com/group/neotoma-erl Cheers, Sean Cribbs From ulf.wiger@REDACTED Fri Nov 13 19:53:08 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 13 Nov 2009 19:53:08 +0100 Subject: [erlang-questions] proper use of application:set_env? In-Reply-To: <863090.1069.qm@web112616.mail.gq1.yahoo.com> References: <863090.1069.qm@web112616.mail.gq1.yahoo.com> Message-ID: <4AFDAB14.6070804@erlang-consulting.com> Pablo Platt wrote: > Hi, > > I have an application that start a mochiweb server. > The same application handles several vhosts. > Each vhost has a config file with dispatch-list + handlers, db-name, secret-string... > The handlers are modules which need to know about the settings in the config file, for example the db name or the sercret-string to authenticate users. > I don't want to access the config file every time because of performance. > > Is it ok to use > application:set_env('example.com_setting1', "some setting1") , > application:set_env('example.com_setting2', "some setting2"), > application:set_env('other-domain.com_setting1', "some setting1") ... > and then access it from the handlers? It's not recommended, but technically, I don't really think there is a problem. The thing to remember is that many programs will check the environment variables once and assume they will not change after that. One needs to make certain that changes of the environment variables are reflected in the running programs. > Is there a performance price for using application_get_env for every request? If you specify the application name, it is one ets lookup. If you don't, it's two ets lookups - one to find the current application, based on the mapping of group leader to app, and then a lookup to find the env variable. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From tony@REDACTED Sat Nov 14 06:33:34 2009 From: tony@REDACTED (Tony Arcieri) Date: Fri, 13 Nov 2009 22:33:34 -0700 Subject: [neotoma] [ANN] Neotoma 1.3 In-Reply-To: <4AFDF585.6010709@gmail.com> References: <4AFDF585.6010709@gmail.com> Message-ID: Really glad to see an inline transformation syntax as this is one of my favorite features of Leex. Now that you are producing Erlang code AOT, will records be supported as they are in Leex? (since records are output verbatim in the resulting Erlang code from the original Leex files, and thus can get processed by EPP when the resulting Erlang code is compiled by the normal Erlang compiler) I have continued to develop my attempts at a minimalized, more Erlang-like Reia using Leex because I felt attempts at a Neotoma-based parser were killing my developmental gumption. I would still very much like to switch to Neotoma if I feel it addresses all my needs. I still worry about using a PEG for a large grammar, and I still do not have a good understanding of how PEGs work. -- Tony Arcieri Medioh/Nagravision From seancribbs@REDACTED Sat Nov 14 22:15:22 2009 From: seancribbs@REDACTED (Sean Cribbs) Date: Sat, 14 Nov 2009 16:15:22 -0500 Subject: Neotoma Screencast Message-ID: <4AFF1DEA.70805@gmail.com> Today I put up a ~10 minute practical screencast for Neotoma. If you're curious as to how to use the library, this may clear some things up for you. Enjoy! http://www.vimeo.com/7613228 See also: http://github.com/seancribbs/neotoma Cheers, Sean Cribbs From xma@REDACTED Sat Nov 14 23:48:17 2009 From: xma@REDACTED (Xavier Maillard) Date: Sat, 14 Nov 2009 23:48:17 +0100 Subject: [erlang-questions] Neotoma Screencast In-Reply-To: <4AFF1DEA.70805@gmail.com> (message from Sean Cribbs on Sat, 14 Nov 2009 16:15:22 -0500) References: <4AFF1DEA.70805@gmail.com> Message-ID: <200911142248.nAEMmHON009731@fed.local> Today I put up a ~10 minute practical screencast for Neotoma. If you're curious as to how to use the library, this may clear some things up for you. Enjoy! http://www.vimeo.com/7613228 Glad to read that. Is there anyway to watch to this video elsewhere ? Here I got a message indicating a wait queue of 1h33 :/ Regards. Xavier -- http://www.gnu.org http://www.april.org http://www.lolica.org From dmitriid@REDACTED Sun Nov 15 10:27:21 2009 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Sun, 15 Nov 2009 11:27:21 +0200 Subject: [erlang-questions] Zotonic, the Erlang CMS, has been released. In-Reply-To: <7E1DF0D2-4A73-48DA-BE0E-01D1B9F0EFAE@worrell.nl> References: <7E1DF0D2-4A73-48DA-BE0E-01D1B9F0EFAE@worrell.nl> Message-ID: <88A614C2-06B6-4595-B1E7-7FBC4D9AEBA6@gmail.com> Haven't looked at Zotonic in detail yet, but I can say "wow!" just by looking at the docs :) Looks hot! On Nov 13, 2009, at 17:19 , Marc Worrell wrote: > Today we released the Erlang CMS Zotonic. > > It is fast, easy to use and open source. Check it out at http://zotonic.com/ > > Zotonic is the first Erlang CMS that is ready to use after > installation. No programming needed. > We made Zotonic to meet the challenges of modern web site design and > evolving computer architecture. > > Some highlights of Zotonic are: > - A great editorial environment. > - Event driven web development (like Nitrogen). > - Flexible template system based on extended Django/ErlyDTL. > - Extensible modules. > - Friendly for editors, front-enders and programmers. > - Speed, through the use of Erlang, memoization and caching. > > We especially like to thank the authors of Webmachine, ErlyDTL, > epgsql, MochiWeb and Nitrogen. > > For more information, please contact marc@REDACTED > > Kind regards, > > Marc Worrell, Tim Benniks, Arjan Scherpenisse, Peet Sneekes and > Mannschaft. > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From roberto.aloi@REDACTED Sun Nov 15 14:51:12 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Sun, 15 Nov 2009 13:51:12 +0000 (GMT) Subject: [erlang-questions] Zotonic, the Erlang CMS, has been released. In-Reply-To: <7E1DF0D2-4A73-48DA-BE0E-01D1B9F0EFAE@worrell.nl> Message-ID: <23095856.37381258293072875.JavaMail.root@zimbra> Great job guys! I've been working on an Erlang Web CMS for quite a while now and yours looks very promising :) The admin panel reminds me a bit too much Wordpress, though :D I will install and try your CMS as soon as possible and I'll give you my feedback. I hope to release my CMS soon... :( It's interesting to see how I'm not the only Erlang developer working on these strange creatures... Regards, -- Roberto Aloi roberto.aloi@REDACTED http://www.erlang-consulting.com From marc@REDACTED Sun Nov 15 15:38:02 2009 From: marc@REDACTED (Marc Worrell) Date: Sun, 15 Nov 2009 15:38:02 +0100 Subject: [erlang-questions] Zotonic, the Erlang CMS, has been released. In-Reply-To: <23095856.37381258293072875.JavaMail.root@zimbra> References: <23095856.37381258293072875.JavaMail.root@zimbra> Message-ID: Hello Roberto, Are you working on erlang-web? Quite different from our approach, but also a lot in common. The Zotonic admin panel might look like WordPress when you see it. When you use it, you will discover some completely different concepts. :-) Though Tim promised a complete redesign, that even I haven't seen yet. Looking forward to your feedback and your cms! Regards, Marc Worrell On 15 nov 2009, at 14:51, Roberto Aloi wrote: > Great job guys! > > I've been working on an Erlang Web CMS for quite a while now and > yours looks very promising :) > The admin panel reminds me a bit too much Wordpress, though :D > I will install and try your CMS as soon as possible and I'll give > you my feedback. > I hope to release my CMS soon... :( > > It's interesting to see how I'm not the only Erlang developer > working on these strange creatures... > > Regards, > > -- > Roberto Aloi > roberto.aloi@REDACTED > http://www.erlang-consulting.com From seancribbs@REDACTED Sun Nov 15 16:53:33 2009 From: seancribbs@REDACTED (Sean Cribbs) Date: Sun, 15 Nov 2009 10:53:33 -0500 Subject: [erlang-questions] Neotoma Screencast In-Reply-To: <4d08db370911150300v3d27d985s92efbb5c514ecdbf@mail.gmail.com> References: <4AFF1DEA.70805@gmail.com> <200911142248.nAEMmHON009731@fed.local> <4d08db370911150300v3d27d985s92efbb5c514ecdbf@mail.gmail.com> Message-ID: <4B0023FD.6020905@gmail.com> I'll try again later, or on Youtube. In the meantime, you can download it from S3: https://seancribbs-com.s3.amazonaws.com/neotoma.mov Cheers, Sean Hynek Vychodil wrote: > 16 hours ago and still "Waiting in line" > > It seems that something bad has been happen. > > On Sat, Nov 14, 2009 at 11:48 PM, Xavier Maillard > wrote: > > > Today I put up a ~10 minute practical screencast for Neotoma. > If you're > curious as to how to use the library, this may clear some things > up for > you. Enjoy! > > http://www.vimeo.com/7613228 > > Glad to read that. Is there anyway to watch to this video > elsewhere ? Here I got a message indicating a wait queue of 1h33 :/ > > Regards. > > Xavier > -- > http://www.gnu.org > http://www.april.org > http://www.lolica.org > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try Good Data now for free: www.gooddata.com From kika@REDACTED Sun Nov 15 17:16:02 2009 From: kika@REDACTED (Cyril Pertsev) Date: Sun, 15 Nov 2009 19:16:02 +0300 Subject: inets: httpd_request_handler crash Message-ID: <4B002942.9010607@kikap.com> Hello, A while back I moved our code from mod_esi-based module to the 'raw' httpd:do handler because our software required a specific URL path we can't control (and to my understanding I can't use mod_esi to handle any URL I like, like, for example mod_perl does). So far so good, but sometimes I see following in the logs. =CRASH REPORT==== 15-Nov-2009::15:49:39 === crasher: initial call: httpd_request_handler:init/1 pid: <0.12084.62> registered_name: [] exception exit: {{badmatch,socket_closed}, [{httpd_response,send_body,3}, {httpd_response,generate_and_send_response,1}, {httpd_request_handler,handle_response,1}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} in function gen_server:terminate/6 ancestors: [<0.121.0>,httpd_acc_sup_1729,httpd_instance_sup_1729, httpd_sup,inets_sup,<0.54.0>] messages: [] links: [<0.118.0>] dictionary: [] trap_exit: false status: running heap_size: 6765 stack_size: 24 reductions: 16438 Apparently it depends on the client behaviour somehow, because when I tune a loadbalancer settings related to the 'http close', frequency of crashes change, but they never go away completely. Anyway, I tried to figure out what's happening and looked into the OTP code. What I see there is two clauses of send_body/3 (line 180 of httpd_response.erl) of which one is sending a list Body and requires httpd_socket:deliver to return 'ok' and second is a 'functional' version which calls a function and then delivers the result. This functional version handles all cases of httpd_socket:deliver including 'ok' and _ in which case it just returns with 'done'. Is there a reason for that? Apparently I'm hitting the first clause where I send a cooked body and httpd_socket:deliver returns socket_closed. Why the crash anyway? It's the exception I can't handle and process, since it happens deep inside httpd secret machinery. All I can do is just restart the failed instance with supervisor which is rather inefficient given our loads. Worse, sometimes this is accompanied by Nov 15 15:49:58 agaccel-demo2 kernel: beam.smp[5703]: segfault at 0000000000000028 rip 000000000044d874 rsp 0000000044332e60 error 4 Centos 5.3/x86_64 R13 Erlang R13B01 (erts-5.7.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] May be it's just a coincidence. Thanks for all advice and help you can offer. -- Best regards, Cyril From erik@REDACTED Sun Nov 15 21:40:01 2009 From: erik@REDACTED (Erik Rigtorp) Date: Sun, 15 Nov 2009 21:40:01 +0100 Subject: Erlang Latency Guide Message-ID: Hi! I enjoyed a chat with Patrik Nyblom during the EUC. We talked a bit about the scheduler and reducing latency in Erlang systems. I've written a fairly complex system in Erlang that requires low latencies. During the development I learned a few tricks on how to achieve low latencies in Erlang. This weekend I decided to write down my experiences and this resulted in the Erlang Latency Guide, http://rigtorp.se/latency.html. Here is a plain text version: Erlang Latency Guide Introduction Latency is a tricky subject, sometimes it's not even clear what or how to measure it. I've had the experience of writing a fairly complex system requiring low latencies in Erlang. Fortunately Erlang provides really good baseline performance. Most of the time you simply write your program and it will perform well. There are however a few tricks that can be used to lower the latencies of a specific path in the system. This document describes a few of these tricks. Yield Erlang allows you to design efficient concurrent systems without caring how processes are scheduled or how many cores the system is running on. When running Erlang with multiple schedulers (generally one per CPU-core) the runtime will balance the load between the schedulers by migrating processes to starved schedulers. There is no way to bind processes to schedulers or control how processes are migrated between schedulers. This introduces a non-deterministic behavior in the system and makes it hard to control latency. A common pattern is to have a demultiplexer that receives a message, sends it to some other process/processes and then performs some additional processing on the message: loop(State) -> receive Msg -> Pid = lookup_pid(Msg, State), Pid ! Msg, State2 = update_state(Msg, State), loop(State2) end. After the message has been sent the receiving process will be ready to execute, but unless the receiving process is on a different scheduler the demultiplexer will first finish executing. Ideally we would bind the demultiplexer to one scheduler and bind the receiving processes to the other schedulers, but that's not allowed in Erlang. Erlang provides only one simple, but powerful way to control scheduling: The built-in function (BIF) erlang:yield/0 lets processes voluntarily give up execution and let other processes get a chance to execute. The demultiplexer pattern can be modified by adding erlang:yield() after sending the message: loop(State) -> receive Msg -> Pid = lookup_pid(Msg, State), Pid ! Msg, erlang:yield(), State2 = update_state(Msg, State), loop(State2) end. After the message has been sent the demultiplexer will give up execution. If the demultiplexer and the receiver are on the same scheduler the receiver will execute before the demultiplexer finishes executing, if they are on different schedulers they will execute in parallel. Using the erlang:yield/0 BIF it's possible to control the scheduling of Erlang processes. If used correctly this can reduce the latency in a system. Network All network I/O in Erlang is implemented as an Erlang driver. The driver is interfaced by the module prim_inet which in turn is interfaced by the network related modules in the kernel application. There is a performance issue with the prim_inet:send/2 and prim_inet:recv/2 functions affecting all the network related modules. When calling prim_inet:send/2 or prim_inet:recv/2 the process will do a selective receive. If the process's message queue is long there will be a performance penalty from doing this selective receive. For receiving there is a simple solution to this problem: use the {active, once} socket option. A simple selective receive-free TCP receiver: loop(Sock) -> inet:setopts(Sock, [{active, once}]), receive {tcp, Sock, Data} -> loop(Sock); {tcp_error, Sock, Reason} -> exit(Reason); {tcp_closed, Sock} -> exit() end. To implement sending without doing a selective receive it is necessary to use the low-level port interface function erlang:port_command/2. Calling erlang:port_command(Sock, Data) on a TCP socket would send the data Data on the socket and return a reference Ref. The socket will reply by sending {inet_reply, Ref, Status} to the process that called erlang:port_command. A simple selective receive-free TCP writer: loop(Sock) -> receive {inet_reply, _, ok} -> loop(Sock); {inet_reply, _, Status} -> exit(Status); Msg -> try erlang:port_command(Sock, Msg) catch error:Error -> exit(Error) end, loop(Sock) end. Though not Erlang specific it is important to remember to tune the send and receive buffer sizes. If the TCP receive window is full data may be delayed up to one network round trip. For UDP, packets will be dropped. Distribution Erlang allows you to send messages between processes at different nodes on the same or different computers. It is also possible to interact with C-nodes (Erlang nodes implemented in C). The communication is done over TCP/IP and obviously this introduces latencies, especially when communicating between nodes on a network. Even when the nodes are running on the same computer they communicate using TCP /IP over the loopback interface. Different operating systems have widely different loopback performance (Solaris has lower latency than Linux). If your system uses the loopback interface it's a good idea to consider this. Further Reading ??? erts/preloaded/src/prim_inet.erl from the Erlang release ??? erts/emulator/drivers/common/inet_drv.c from the Erlang release CC ??? written by Erik Rigtorp From max.lapshin@REDACTED Sun Nov 15 22:53:55 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 16 Nov 2009 00:53:55 +0300 Subject: inet:setopts({packet, http}) for RTSP Message-ID: Hi, I want to get data from QuickTime Broadcaster. This tool uses RTSP protocol (http://en.wikipedia.org/wiki/Real_Time_Streaming_Protocol) This protocol seems to look like HTTP, but with other verbs. Here is example: ANNOUNCE rtsp://maxbook.local:8000/mystream.sdp RTSP/1.0\r\n I wanted to utilize http mode of socket, thinking that it will parse RTSP requests, but now it seems to be impossible. I understand, that it is unacceptable to atomify user input strings ("ANNOUNCE" -> announce), but is it the only reason not to parse it? From lfredlund@REDACTED Mon Nov 16 00:34:03 2009 From: lfredlund@REDACTED (=?UTF-8?B?TGFycy3DhWtlIEZyZWRsdW5k?=) Date: Mon, 16 Nov 2009 00:34:03 +0100 Subject: [erlang-questions] hipe_bifs missing in R13B02 In-Reply-To: References: <4AFA9F3C.3040105@fi.upm.es> <19195.2164.280326.766479@pilspetsen.it.uu.se> Message-ID: <4B008FEB.4080806@fi.upm.es> Michael Richter wrote: > 2009/11/12 Mikael Pettersson > > >> Lars-?ke Fredlund writes: >> > Yes, they probably have never officially been part of the Erlang >> > distribution but still filled a useful role, e.g. hipe_bifs:bytearray/2 >> > for creating destructive quick arrays (check Kostis Sagonas posts to the >> > erlang-questions mailing list for details :-) >> > >> > They seemed to be working in the first R13 release, but on downloading >> > and compiling R13B02-1 they seem to be missing (or I messed up the >> > recompilation). >> > >> > Any official word on their omission? They are really useful for some >> > purposes... >> >> They're not gone. You probably built a system with HiPE disabled. >> >> > > It also turns out to be really easy to build such a system even if you > configure with --enable-hipe. Building HIPE requires the m4 macro processor > and if you don't have it installed the configure script makes a one-line > mention of it deep in the thousands of lines of gibberish that gets spewed > out. > > Yes, that seems to be what had happened to me as well. No m4 installed, and the configure seemed ok from a superficial glance except complaining about wx linking... With m4 installed I have hipe_bifs again; thanks for the hint! /Lars-Ake From igorrs@REDACTED Mon Nov 16 03:17:50 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Mon, 16 Nov 2009 00:17:50 -0200 Subject: Impact of state size in mnesia_frag_hash Message-ID: Hello. I've begun the implementation of a module following the mnesia_frag_hash behaviour. With enough fragments, my hash_state could grow to a reasonable size (something around, say, 10 MB). Can you think of any significant performance impact of this implementation? For example: is the hash_state transmitted between nodes frequently enough to cause significant network traffic? Thanks. Igor. -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From jarrod@REDACTED Mon Nov 16 04:26:18 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Sun, 15 Nov 2009 22:26:18 -0500 Subject: Is there any existing library code to parse a DNS response in binary form? Message-ID: I am writing a native erlang Bonjour / ZeroConf implementation and need to parse the resulting binary DNS response I get. Before I start down the road of bit syntax and all that I figured something this fundamental had to already exist. When I did a Python version of this there was already library code I could use to parse the results. I have searched Google and come up with nothing useful. Thanks in advance. -- Jarrod Roberson www.vertigrated.com/blog/ 678.551.2852 From seancribbs@REDACTED Mon Nov 16 04:39:55 2009 From: seancribbs@REDACTED (Sean Cribbs) Date: Sun, 15 Nov 2009 22:39:55 -0500 Subject: [erlang-questions] Is there any existing library code to parse a DNS response in binary form? In-Reply-To: References: Message-ID: <4B00C98B.8000100@gmail.com> You might find Geoff Cant's enet project useful: http://github.com/archaelus/enet/ It appears he has a decoder for DNS, ARP and a bunch of other protocols. Sean Jarrod Roberson wrote: > I am writing a native erlang Bonjour / ZeroConf implementation and need to > parse the resulting binary DNS response I get. > Before I start down the road of bit syntax and all that I figured something > this fundamental had to already exist. > When I did a Python version of this there was already library code I could > use to parse the results. > I have searched Google and come up with nothing useful. Thanks in advance. > > From jarrod@REDACTED Mon Nov 16 04:41:20 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Sun, 15 Nov 2009 22:41:20 -0500 Subject: Is there any existing library code to parse a DNS response in binary form? In-Reply-To: References: Message-ID: On Sun, Nov 15, 2009 at 10:26 PM, Jarrod Roberson wrote: > I am writing a native erlang Bonjour / ZeroConf implementation and need to > parse the resulting binary DNS response I get. > Before I start down the road of bit syntax and all that I figured something > this fundamental had to already exist. > When I did a Python version of this there was already library code I could > use to parse the results. > I have searched Google and come up with nothing useful. Thanks in advance. > ok, I found something useful without Google :-) I started digging thru the Erlang library source and found, inets_dns:decode/1 it works like a charm, now I just need to figure out how to decode the TXT entries and I will be 100%. Jarrod Roberson 678.551.2852 From jarrod@REDACTED Mon Nov 16 06:04:21 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Mon, 16 Nov 2009 00:04:21 -0500 Subject: Struggling with multicast udp send Message-ID: here is what I am trying, it doesn't work. :-( send() -> {ok, S} = gen_udp:open(5353, [{reuseaddr, true},{ip,{224,0,0,251}},{broadcast, true}]), inet:setopts(S,[{add_membership,{{224,0,0,251},{0,0,0,0}}}]), gen_udp:send(S, {224,0,0,251}, 5353, "test"). here is what I get Erlang R13B02 (erts-5.7.3) [source] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] Eshell V5.7.3 (abort with ^G) 1> zcclient:send(). {error,eaddrnotavail} 2> I know this has to be a simple thing, Programming Erlang has a "broadcast" udp example that I am trying to make work. I am not sure that is the same thing as "multicast". Is it? From jarrod@REDACTED Mon Nov 16 06:05:07 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Mon, 16 Nov 2009 00:05:07 -0500 Subject: [erlang-questions] Is there any existing library code to parse a DNS response in binary form? In-Reply-To: <4B00C98B.8000100@gmail.com> References: <4B00C98B.8000100@gmail.com> Message-ID: On Sun, Nov 15, 2009 at 10:39 PM, Sean Cribbs wrote: > You might find Geoff Cant's enet project useful: > > http://github.com/archaelus/enet/ > > It appears he has a decoder for DNS, ARP and a bunch of other protocols. > > Sean thanks I found one that works perfect in the standard lib @ inet_dns:decode/1 From rapsey@REDACTED Mon Nov 16 06:41:06 2009 From: rapsey@REDACTED (Rapsey) Date: Mon, 16 Nov 2009 06:41:06 +0100 Subject: [erlang-questions] Struggling with multicast udp send In-Reply-To: References: Message-ID: <97619b170911152141u21d1c580w6d207df151a10788@mail.gmail.com> This is copy paste from my server: Dest and Source are IP addresses (Source is IP of local computer, can be {0,0,0,0}) {ok, Sock} = gen_udp:open(0, [{reuseaddr,true}, {ip, Source}, {multicast_ttl, P#tspar.ttl}, {multicast_loop, false}, binary]), inet:setopts(Sock, [{add_membership, {Dest, Source}}]); Sergej On Mon, Nov 16, 2009 at 6:04 AM, Jarrod Roberson wrote: > here is what I am trying, it doesn't work. :-( > > send() -> > {ok, S} = gen_udp:open(5353, [{reuseaddr, > true},{ip,{224,0,0,251}},{broadcast, true}]), > inet:setopts(S,[{add_membership,{{224,0,0,251},{0,0,0,0}}}]), > gen_udp:send(S, {224,0,0,251}, 5353, "test"). > > here is what I get > > Erlang R13B02 (erts-5.7.3) [source] [smp:4:4] [rq:4] [async-threads:0] > [kernel-poll:false] > > Eshell V5.7.3 (abort with ^G) > 1> zcclient:send(). > {error,eaddrnotavail} > 2> > > I know this has to be a simple thing, Programming Erlang has a "broadcast" > udp example that I am trying to make work. > I am not sure that is the same thing as "multicast". Is it? > From pablo.platt@REDACTED Mon Nov 16 06:55:11 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Sun, 15 Nov 2009 21:55:11 -0800 (PST) Subject: init.d script and a control script for starting an erlang app on boot in ubuntu Message-ID: <91026.80235.qm@web112610.mail.gq1.yahoo.com> Hi, Can someone post a simple init.d script with a control script for an erlang application on ubuntu? I've tried to look at yaws(also rabbitmq and ejabberd) as an example but it is a bit complicated and specific: http://github.com/klacke/yaws/blob/master/scripts/yaws.template http://github.com/klacke/yaws/blob/master/scripts/debian/yaws.init.d I'll appreciate an example for a control script that can start/stop/status/restart/reload(hot code update)/debug(attach an erlang shell). The script should also be able to restart the app (heart?). Thanks From jarrod@REDACTED Mon Nov 16 07:26:12 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Mon, 16 Nov 2009 01:26:12 -0500 Subject: [erlang-questions] Struggling with multicast udp send In-Reply-To: <97619b170911152141u21d1c580w6d207df151a10788@mail.gmail.com> References: <97619b170911152141u21d1c580w6d207df151a10788@mail.gmail.com> Message-ID: On Mon, Nov 16, 2009 at 12:41 AM, Rapsey wrote: > This is copy paste from my server: > Dest and Source are IP addresses (Source is IP of local computer, can be > {0,0,0,0}) > > {ok, Sock} = gen_udp:open(0, [{reuseaddr,true}, {ip, Source}, > {multicast_ttl, P#tspar.ttl}, > {multicast_loop, false}, > binary]), > inet:setopts(Sock, [{add_membership, {Dest, Source}}]); > Thanks, but that works great for listen, but I can't get it to work for sending. Also I don't want to pass in 0 for the port, that gives me a random port, I want to use port 5353. All the other programs can send on that port, why can't erlang? here is the output I get from iChat starting up. From: {192,168,0,105} Port: 5353 Data: {ok,{dns_rec,{dns_header,0,false,'query',false,false,false,false,false, 0}, [{dns_query,"_presence._tcp.local",ptr,in}], [{dns_rr,"_presence._tcp.local",ptr,in,0,4500, "jhr@REDACTED _presence._tcp.local",undefined, [],false}], [],[]}} I get similar results with the exact same port 5353 from SubEthaEdit and iTunes. here is what I get when I try and send the message. From: {192,168,0,105} Port: 57345 Data: {error,fmt} that is with passing in ZERO for the port on the :open() call. and here is what I get when I try and make it send on 5353 like the other programs 44> l(zcquery). {module,zcquery} 45> zcquery:send(). ** exception error: no match of right hand side value {error,eaddrinuse} in function zcquery:send/0 46> anyone have any arcane knowledge of how to get this to behave correctly? From sysop@REDACTED Mon Nov 16 07:55:38 2009 From: sysop@REDACTED (Matt Stancliff) Date: Sun, 15 Nov 2009 22:55:38 -0800 Subject: [erlang-questions] init.d script and a control script for starting an erlang app on boot in ubuntu In-Reply-To: <91026.80235.qm@web112610.mail.gq1.yahoo.com> References: <91026.80235.qm@web112610.mail.gq1.yahoo.com> Message-ID: <15F8750B-A91A-4995-9A80-9D6101C49C59@mindspring.com> Pablo, On Nov 15, 2009, at 9:55 PM, Pablo Platt wrote: > Can someone post a simple init.d script with a control script for an > erlang application on ubuntu? > > I'll appreciate an example for a control script that can start/stop/ > status/restart/reload(hot code update)/debug(attach an erlang shell). > The script should also be able to restart the app (heart?). The best setup I've found for node scripting is from rabbitmq: Control script: http://hg.rabbitmq.com/rabbitmq-server/file/default/scripts/rabbitmqctl Control module: http://hg.rabbitmq.com/rabbitmq-server/file/default/src/rabbit_control.erl You can also check out how ejabberd does it, but the rabbitmq way looks cleaner: Control script: https://forge.process-one.net/browse/ejabberd/branches/ejabberd-2.1.x/tools/ejabberdctl?r=HEAD One control module: https://forge.process-one.net/browse/ejabberd/branches/ejabberd-2.1.x/src/ejabberd.erl?r=HEAD Other control module: https://forge.process-one.net/browse/ejabberd/branches/ejabberd-2.1.x/src/ejabberd_ctl.erl?r=HEAD -Matt -- Matt Stancliff San Jose, CA AIM: seijimr iPhone: 678-591-9337 "The best way to predict the future is to invent it." --Alan Kay From rapsey@REDACTED Mon Nov 16 08:21:40 2009 From: rapsey@REDACTED (Rapsey) Date: Mon, 16 Nov 2009 08:21:40 +0100 Subject: [erlang-questions] Struggling with multicast udp send In-Reply-To: References: <97619b170911152141u21d1c580w6d207df151a10788@mail.gmail.com> Message-ID: <97619b170911152321v738d6c96p8350ea5a1a64fd04@mail.gmail.com> The code I pasted was for sending. I use it to transmit an IPTV multicast video stream. Why is source port important? If you wish to also receive on that port, you could always have a different regular udp socket used just for listening. Sergej On Mon, Nov 16, 2009 at 7:26 AM, Jarrod Roberson wrote: > On Mon, Nov 16, 2009 at 12:41 AM, Rapsey wrote: > > > This is copy paste from my server: > > Dest and Source are IP addresses (Source is IP of local computer, can be > > {0,0,0,0}) > > > > {ok, Sock} = gen_udp:open(0, [{reuseaddr,true}, {ip, Source}, > > {multicast_ttl, P#tspar.ttl}, > > {multicast_loop, false}, > > binary]), > > inet:setopts(Sock, [{add_membership, {Dest, Source}}]); > > > > Thanks, but that works great for listen, but I can't get it to work for > sending. > Also I don't want to pass in 0 for the port, that gives me a random port, I > want to use port 5353. > All the other programs can send on that port, why can't erlang? > here is the output I get from iChat starting up. > > From: {192,168,0,105} > Port: 5353 > Data: > {ok,{dns_rec,{dns_header,0,false,'query',false,false,false,false,false, > 0}, > [{dns_query,"_presence._tcp.local",ptr,in}], > [{dns_rr,"_presence._tcp.local",ptr,in,0,4500, > "jhr@REDACTED > _presence._tcp.local",undefined, > [],false}], > [],[]}} > > I get similar results with the exact same port 5353 from SubEthaEdit and > iTunes. > > here is what I get when I try and send the message. > > From: {192,168,0,105} > Port: 57345 > Data: {error,fmt} > > that is with passing in ZERO for the port on the :open() call. > > and here is what I get when I try and make it send on 5353 like the other > programs > > 44> l(zcquery). > {module,zcquery} > 45> zcquery:send(). > ** exception error: no match of right hand side value {error,eaddrinuse} > in function zcquery:send/0 > 46> > > anyone have any arcane knowledge of how to get this to behave correctly? > From sverker@REDACTED Mon Nov 16 09:40:07 2009 From: sverker@REDACTED (Sverker Eriksson) Date: Mon, 16 Nov 2009 09:40:07 +0100 Subject: [erlang-questions] inet:setopts({packet, http}) for RTSP In-Reply-To: References: Message-ID: <4B010FE7.1090608@erix.ericsson.se> Max Lapshin wrote: > Hi, I want to get data from QuickTime Broadcaster. This tool uses RTSP > protocol (http://en.wikipedia.org/wiki/Real_Time_Streaming_Protocol) > > This protocol seems to look like HTTP, but with other verbs. Here is example: > > ANNOUNCE rtsp://maxbook.local:8000/mystream.sdp RTSP/1.0\r\n > > I wanted to utilize http mode of socket, thinking that it will parse > RTSP requests, but now it seems to be impossible. > I understand, that it is unacceptable to atomify user input strings > ("ANNOUNCE" -> announce), but is it the only reason not to parse it? > > It works if you change "RTSP" to "HTTP" :-) 1> Bin = <<"ANNOUNCE rtsp://maxbook.local:8000/mystream.sdp HTTP/1.0\r\n">>. 2> erlang:decode_packet(http,Bin,[]). {ok,{http_request,"ANNOUNCE", {scheme,"rtsp","//maxbook.local:8000/mystream.sdp"}, {1,0}}, <<>>} The parsing is done in erts/emulator/beam/packet_parser.c: packet_parse_http() /Sverker, Erlang/OTP Ericsson From max.lapshin@REDACTED Mon Nov 16 09:44:11 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 16 Nov 2009 11:44:11 +0300 Subject: [erlang-questions] inet:setopts({packet, http}) for RTSP In-Reply-To: <4B010FE7.1090608@erix.ericsson.se> References: <4B010FE7.1090608@erix.ericsson.se> Message-ID: On Mon, Nov 16, 2009 at 11:40 AM, Sverker Eriksson wrote: > > It works if you change "RTSP" to "HTTP" :-) > It seems that use request parser from mochiweb is easier, than distribute binary patches to Quicktime Broadcaster =)) Yes, I've already found this fact and now experience problems, that with {packet, httph_bin}, {active, once}, after receiving {http, Socket, http_eoh}, there are messages in mailbox in format {http, Socket, {error, ..}}. I.e. further parsing of stream with format http headers is done even with active, once. From max.lapshin@REDACTED Mon Nov 16 14:27:47 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 16 Nov 2009 16:27:47 +0300 Subject: [erlang-questions] inet:setopts({packet, http}) for RTSP In-Reply-To: References: <4B010FE7.1090608@erix.ericsson.se> Message-ID: Yes, I have next problem with uncommon use of http packet mode. As it is impossible to use {packet, http} for reading non-HTTP request, I set {packet, line}, {active, once} to read first line of request, then I make {active, once}, {packet, httph_bin} to receive several lines of real http headers. When I get http_eoh, I switch to {packet, raw}, {active, once} and next received message is http_error. Trace dump shows me that message box is full of http_error messages: {active, once} doesn't limit http packetizer input and body gets into http parser. From sverker@REDACTED Mon Nov 16 15:05:40 2009 From: sverker@REDACTED (Sverker Eriksson) Date: Mon, 16 Nov 2009 15:05:40 +0100 Subject: [erlang-questions] inet:setopts({packet, http}) for RTSP In-Reply-To: References: <4B010FE7.1090608@erix.ericsson.se> Message-ID: <4B015C34.2010302@erix.ericsson.se> Max Lapshin wrote: > Yes, I have next problem with uncommon use of http packet mode. > > As it is impossible to use {packet, http} for reading non-HTTP > request, I set {packet, line}, {active, once} to read first line of > request, > then I make {active, once}, {packet, httph_bin} to receive several > lines of real http headers. > > When I get http_eoh, I switch to {packet, raw}, {active, once} and > next received message is http_error. > Trace dump shows me that message box is full of http_error messages: > {active, once} doesn't limit > http packetizer input and body gets into http parser. > That doesn't seem right. If you want to trouble shoot this (which I do not have time for at the moment), you should look in erts/emulator/drivers/common/inet_drv.c. This if-statement in tcp_reply_data() and tcp_reply_binary_data() if (desc->inet.active == INET_ONCE) desc->inet.active = INET_PASSIVE; should make it revert to passive mode after each received packet. /Sverker, Erlang/OTP Ericsson From max.lapshin@REDACTED Mon Nov 16 15:08:13 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 16 Nov 2009 17:08:13 +0300 Subject: [erlang-questions] inet:setopts({packet, http}) for RTSP In-Reply-To: <4B015C34.2010302@erix.ericsson.se> References: <4B010FE7.1090608@erix.ericsson.se> <4B015C34.2010302@erix.ericsson.se> Message-ID: I'll look more detailed at erlang source code and if I doesn't find bug in my code, I will make minimal test example. From igorrs@REDACTED Mon Nov 16 16:36:21 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Mon, 16 Nov 2009 13:36:21 -0200 Subject: Consistent hashing for Mnesia fragments (DRAFT) Message-ID: Hello. If you have the time to analyze the following idea and point some problems you find (and are not already in the text), I'd be very happy. http://igorrs.blogspot.com/2009/11/consistent-hashing-for-mnesia-fragments.html Also, any comments about the impact of the size of the hash state in the performance of Mnesia would be welcome. Thanks. Igor. -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From zabrane3@REDACTED Mon Nov 16 17:17:32 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 16 Nov 2009 17:17:32 +0100 Subject: gen_tcp very slow to fetch data Message-ID: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> Hi List ! New to Erlang, I'm trying to implement a simple URL fetcher. Here's my code (please, feel free to correct it if you find any bug or know a better approach): 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8---- -module(fetch). -export([url/1]). -define(TIMEOUT, 7000). -define(TCP_OPTS, [binary, {packet, raw}, {nodelay, true}, {active, true}]). url(Url) -> {ok, _Tag, Host, Port} = split_url(Url), Hdrs = [], Request = ["GET ", Url, " HTTP/1.1\r\n", Hdrs, "\r\n\r\n"], case catch gen_tcp:connect(Host, Port, ?TCP_OPTS) of {'EXIT', Why} -> {error, {socket_exit, Why}}; {error, Why} -> {error, {socket_error, Why}}; {ok, Socket} -> gen_tcp:send(Socket, list_to_binary(Request)), recv(Socket, list_to_binary([])) end. recv(Socket, Bin) -> receive {tcp, Socket, B} -> io:format(".", []), recv(Socket, concat_binary([Bin, B])); {tcp_closed, Socket} -> {ok, Bin}; Other -> {error, {socket, Other}} after ?TIMEOUT -> {error, {socket, timeout}} end. split_url([$h,$t,$t,$p,$:,$/,$/|T]) -> split_url(http, T); split_url(_X) -> {error, split_url}. split_url(Tag, X) -> case string:chr(X, $:) of 0 -> Port = 80, case string:chr(X,$/) of 0 -> {ok, Tag, X, Port}; N -> Site = string:substr(X,1,N-1), {ok, Tag, Site, Port} end; N1 -> case string:chr(X,$/) of 0 -> error; N2 -> PortStr = string:substr(X,N1+1, N2-N1-1), case catch list_to_integer(PortStr) of {'EXIT', _} -> {error, port_number}; Port -> Site = string:substr(X,1,N1-1), {ok, Tag, Site, Port} end end end. 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8------ When testing it, the receiving socket gets very very slow: $ erl 1> c(fetch). 2> Bin = fetch:url("http://www.google.com"). ......{error,{socket,timeout}} Am I missing something? What I like to get at the end is a very fast fetcher. Any hint? Regards Zabrane From gm4592@REDACTED Mon Nov 16 17:29:46 2009 From: gm4592@REDACTED (Madi, George) Date: Mon, 16 Nov 2009 11:29:46 -0500 Subject: Turning off echo in escript Message-ID: <1258388986.30511.2.camel@ocean> I'm having trouble turning off echo in escript. If I put the following into a module: ====================== -module(testecho). -compile(export_all). main(_) -> io:setopts( standard_io, [ {echo, false} ] ), P = io:get_line( standard_io, "Enter some stuff > "), io:format( "~nYou entered ~p~n", [ P ] ). ====================== and run it, I get the following: ====================== $ erl Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) 1> c(testecho). {ok,testecho} 2> testecho:main(hi). Enter some stuff > You entered "Hello\n" ok ====================== As you can see, "Hello" does not show up as I type, which is what I'm after. However, if I do the same in escript as follows: ====================== #!/usr/bin/env escript %% -*- erlang -*- main(_) -> io:setopts( standard_io, [ {echo, false} ] ), P = io:get_line( standard_io, "Enter some stuff > "), io:format( "~nYou entered ~p~n", [ P ] ). ====================== I get the following: ====================== $ ./testecho.sh Enter some stuff > Hello You entered "Hello\n" $ ====================== The echo does not turn off. How do I get the echo to turn off in escript? Thanks, A. G. Madi AT&T Chief Security Office (CSO) ***** The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential, proprietary, and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from all computers. GA623 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5201 bytes Desc: not available URL: From chandrashekhar.mullaparthi@REDACTED Mon Nov 16 18:53:00 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 16 Nov 2009 17:53:00 +0000 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> Message-ID: You are expecting the server to indicate end of response by closing the connection, but because you specify HTTP/1.1 in the request, the server is holding up your connection, and you are timing out. Try replacing HTTP/1.1 with HTTP/1.0 in your request, or parse the response to detect end of response. cheers Chandru 2009/11/16 zabrane Mikael > Hi List ! > > New to Erlang, I'm trying to implement a simple URL fetcher. > Here's my code (please, feel free to correct it if you find any bug or know > a better approach): > > > 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8---- > -module(fetch). > > -export([url/1]). > > -define(TIMEOUT, 7000). > -define(TCP_OPTS, [binary, {packet, raw}, {nodelay, true}, > {active, true}]). > > url(Url) -> > {ok, _Tag, Host, Port} = split_url(Url), > > Hdrs = [], > Request = ["GET ", Url, " HTTP/1.1\r\n", Hdrs, "\r\n\r\n"], > > case catch gen_tcp:connect(Host, Port, ?TCP_OPTS) of > {'EXIT', Why} -> > {error, {socket_exit, Why}}; > {error, Why} -> > {error, {socket_error, Why}}; > {ok, Socket} -> > gen_tcp:send(Socket, list_to_binary(Request)), > recv(Socket, list_to_binary([])) > end. > > recv(Socket, Bin) -> > receive > {tcp, Socket, B} -> > io:format(".", []), > recv(Socket, concat_binary([Bin, B])); > {tcp_closed, Socket} -> > {ok, Bin}; > Other -> > {error, {socket, Other}} > after > ?TIMEOUT -> > {error, {socket, timeout}} > end. > > > split_url([$h,$t,$t,$p,$:,$/,$/|T]) -> split_url(http, T); > split_url(_X) -> {error, split_url}. > > split_url(Tag, X) -> > case string:chr(X, $:) of > 0 -> > Port = 80, > case string:chr(X,$/) of > 0 -> > {ok, Tag, X, Port}; > N -> > Site = string:substr(X,1,N-1), > {ok, Tag, Site, Port} > end; > N1 -> > case string:chr(X,$/) of > 0 -> > error; > N2 -> > PortStr = string:substr(X,N1+1, N2-N1-1), > case catch list_to_integer(PortStr) of > {'EXIT', _} -> > {error, port_number}; > Port -> > Site = string:substr(X,1,N1-1), > {ok, Tag, Site, Port} > end > end > end. > > > 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8------ > > When testing it, the receiving socket gets very very slow: > $ erl > 1> c(fetch). > 2> Bin = fetch:url("http://www.google.com"). > ......{error,{socket,timeout}} > > Am I missing something? > What I like to get at the end is a very fast fetcher. Any hint? > > Regards > Zabrane > From erlangy@REDACTED Mon Nov 16 18:59:39 2009 From: erlangy@REDACTED (ERLANG) Date: Mon, 16 Nov 2009 18:59:39 +0100 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> Message-ID: <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> Hi Chandru ! That's fix my problem. Thanks. While googling a bit, I found two ways to read from the Socket: recv(Socket, Bin) -> receive {tcp, Socket, B} -> io:format(".", []), recv(Socket, concat_binary([Bin, B])); {tcp_closed, Socket} -> {ok, Bin}; Other -> {error, {socket, Other}} after ?TIMEOUT -> {error, {socket, timeout}} end. % version 2 with "gen_tcp:recv" recv2(Socket, Bin) -> case gen_tcp:recv(Socket, 0, ?TIMEOUT) of {ok, B} -> io:format(".", []), recv(Socket, concat_binary([Bin, B])); {error, closed} -> {ok, Bin}; {error, timeout} -> {error, {socket, timeout}}; Other -> {error, {socket, Other}} end. Which one is the best in my case (see below: fetch.erl)? Regards Zabrane Le 16 nov. 09 ? 18:53, Chandru a ?crit : > You are expecting the server to indicate end of response by closing > the > connection, but because you specify HTTP/1.1 in the request, the > server is > holding up your connection, and you are timing out. Try replacing > HTTP/1.1 > with HTTP/1.0 in your request, or parse the response to detect end of > response. > > cheers > Chandru > > 2009/11/16 zabrane Mikael > >> Hi List ! >> >> New to Erlang, I'm trying to implement a simple URL fetcher. >> Here's my code (please, feel free to correct it if you find any bug >> or know >> a better approach): >> >> >> 8 >> -----8 >> -----8 >> -----8-----8-----8-----8-----8-----8-----8-----8-----8-----8---- >> -module(fetch). >> >> -export([url/1]). >> >> -define(TIMEOUT, 7000). >> -define(TCP_OPTS, [binary, {packet, raw}, {nodelay, true}, >> {active, true}]). >> >> url(Url) -> >> {ok, _Tag, Host, Port} = split_url(Url), >> >> Hdrs = [], >> Request = ["GET ", Url, " HTTP/1.1\r\n", Hdrs, "\r\n\r\n"], >> >> case catch gen_tcp:connect(Host, Port, ?TCP_OPTS) of >> {'EXIT', Why} -> >> {error, {socket_exit, Why}}; >> {error, Why} -> >> {error, {socket_error, Why}}; >> {ok, Socket} -> >> gen_tcp:send(Socket, list_to_binary(Request)), >> recv(Socket, list_to_binary([])) >> end. >> >> recv(Socket, Bin) -> >> receive >> {tcp, Socket, B} -> >> io:format(".", []), >> recv(Socket, concat_binary([Bin, B])); >> {tcp_closed, Socket} -> >> {ok, Bin}; >> Other -> >> {error, {socket, Other}} >> after >> ?TIMEOUT -> >> {error, {socket, timeout}} >> end. >> >> >> split_url([$h,$t,$t,$p,$:,$/,$/|T]) -> split_url(http, T); >> split_url(_X) -> {error, split_url}. >> >> split_url(Tag, X) -> >> case string:chr(X, $:) of >> 0 -> >> Port = 80, >> case string:chr(X,$/) of >> 0 -> >> {ok, Tag, X, Port}; >> N -> >> Site = string:substr(X,1,N-1), >> {ok, Tag, Site, Port} >> end; >> N1 -> >> case string:chr(X,$/) of >> 0 -> >> error; >> N2 -> >> PortStr = string:substr(X,N1+1, N2-N1-1), >> case catch list_to_integer(PortStr) of >> {'EXIT', _} -> >> {error, port_number}; >> Port -> >> Site = string:substr(X,1,N1-1), >> {ok, Tag, Site, Port} >> end >> end >> end. >> >> >> 8 >> -----8 >> -----8 >> -----8-----8-----8-----8-----8-----8-----8-----8-----8-----8------ >> >> When testing it, the receiving socket gets very very slow: >> $ erl >> 1> c(fetch). >> 2> Bin = fetch:url("http://www.google.com"). >> ......{error,{socket,timeout}} >> >> Am I missing something? >> What I like to get at the end is a very fast fetcher. Any hint? >> >> Regards >> Zabrane >> From jarrod@REDACTED Mon Nov 16 20:56:11 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Mon, 16 Nov 2009 14:56:11 -0500 Subject: Encoding a DNS Query correctly Message-ID: I am writing a Bonjour/Zeroconf implemenation in native Erlang. This is basically multicast DNS. I have the listener and parser for the resulting DNS records working. What I can't get right is the encoding for sending. Here is what I am trying: send() -> {ok,S} = gen_udp:open(5353,[{reuseaddr,true}, {ip,{224,0,0,251}}, {multicast_ttl,4}, {multicast_loop,false}, {broadcast,true}, binary]), {ok,P} = inet_dns:encode(#dns_query{domain="_daap._tcp.local",type=ptr,class=in}), gen_udp:send(S,{224,0,0,251},5353,P), gen_udp:close(S). Here is what I am getting in the console when I call :send() 24> test:send(). ** exception error: {badrecord,dns_rec} in function inet_dns:encode/1 in call from test:send/0 25> If I can get over this hump I will have a basic implementation done. Anyone have any ideas? From garry@REDACTED Mon Nov 16 21:23:29 2009 From: garry@REDACTED (Garry Hodgson) Date: Mon, 16 Nov 2009 15:23:29 -0500 Subject: [erlang-questions] init.d script and a control script for starting an erlang app on boot in ubuntu In-Reply-To: <15F8750B-A91A-4995-9A80-9D6101C49C59@mindspring.com> References: <91026.80235.qm@web112610.mail.gq1.yahoo.com> <15F8750B-A91A-4995-9A80-9D6101C49C59@mindspring.com> Message-ID: <4B01B4C1.5030309@research.att.com> Matt Stancliff wrote: > The best setup I've found for node scripting is from rabbitmq: > Control script: > http://hg.rabbitmq.com/rabbitmq-server/file/default/scripts/rabbitmqctl > Control module: > http://hg.rabbitmq.com/rabbitmq-server/file/default/src/rabbit_control.erl i've recently begun writing my control scripts in escript. always hated all of the "-s foo bar baz" stuff. so for a server called "f2", for example, /etc/init.d/f2-server is a simple shell script that invokes one of "f2 start", "f2 stop", "f2 status", etc. it has worked out beautifully thus far. -- Garry Hodgson Lead Member of Technical Staff AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From jabronson@REDACTED Mon Nov 16 21:48:26 2009 From: jabronson@REDACTED (Joshua Bronson) Date: Mon, 16 Nov 2009 15:48:26 -0500 Subject: tk requirement? In-Reply-To: References: Message-ID: <299cc2dd0911161248n753344b5wd95d39236b3855bf@mail.gmail.com> Hey Erlang folks, I know relatively little about the Erlang distribution (my exposure is pretty much limited to installing it as a requirement of couchdb and rabbitmq-server). I was wondering if tk is (or has to be?) a hard requirement of the distribution or if (like e.g. Python) it's possible to build Erlang with a no_tk variant / USE flag / etc. in the case that you know you won't be needing to do any gui stuff. (This query is prompted by http://trac.macports.org/ticket/20006.) Thanks! Josh From paul.joseph.davis@REDACTED Mon Nov 16 22:00:25 2009 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Mon, 16 Nov 2009 16:00:25 -0500 Subject: [erlang-questions] tk requirement? In-Reply-To: <299cc2dd0911161248n753344b5wd95d39236b3855bf@mail.gmail.com> References: <299cc2dd0911161248n753344b5wd95d39236b3855bf@mail.gmail.com> Message-ID: On Mon, Nov 16, 2009 at 3:48 PM, Joshua Bronson wrote: > Hey Erlang folks, > > I know relatively little about the Erlang distribution (my exposure is > pretty much limited to installing it as a requirement of couchdb and > rabbitmq-server). I was wondering if tk is (or has to be?) a hard > requirement of the distribution or if (like e.g. Python) it's possible > to build Erlang with a no_tk variant / USE flag / etc. in the case > that you know you won't be needing to do any gui stuff. (This query is > prompted by http://trac.macports.org/ticket/20006.) > > Thanks! > > Josh > Josh, I've never had any issue with it using the system installed wx libraries on 10.5 or 10.6. Reading through the ticket, the library you want to check for linkage is probably $PREFIX/lib/erlang/lib/wx-0.98.1/priv/i386-apple-darwin9.6.0/wxe_driver.so. For instance: $ otool -L /usr/local/lib/erlang/lib/wx-0.98.1/priv/i386-apple-darwin9.6.0/wxe_driver.so /usr/local/lib/erlang/lib/wx-0.98.1/priv/i386-apple-darwin9.6.0/wxe_driver.so: /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0) /System/Library/Frameworks/QuickTime.framework/Versions/A/QuickTime (compatibility version 1.0.0, current version 1290.0.0) /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0) /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 136.0.0) /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 12.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.3) /usr/lib/libwx_macud_stc-2.8.0.dylib (compatibility version 2.6.0, current version 2.8.4) /usr/lib/libwx_macud-2.8.0.dylib (compatibility version 2.6.0, current version 2.8.4) /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.4.0) /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) Granted, that's not the same as disabling the whole thing altogether. HTH, Paul Davis From magnus@REDACTED Mon Nov 16 22:11:32 2009 From: magnus@REDACTED (Magnus Henoch) Date: Mon, 16 Nov 2009 21:11:32 +0000 Subject: Encoding a DNS Query correctly In-Reply-To: (Jarrod Roberson's message of "Mon, 16 Nov 2009 14:56:11 -0500") References: Message-ID: <84pr7i6upn.fsf@linux-b2a3.site> Jarrod Roberson writes: > 24> test:send(). > ** exception error: {badrecord,dns_rec} > in function inet_dns:encode/1 > in call from test:send/0 You're passing a dns_query record to inet_dns:encode/1, but it expects a dns_rec record. (If I knew the DNS protocol by heart, I'd tell you how to build one, but for now I'll have to refer to inet_dns.hrl.) -- Magnus Henoch, magnus@REDACTED Erlang Training and Consulting http://www.erlang-consulting.com/ From andrew@REDACTED Mon Nov 16 22:15:04 2009 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 16 Nov 2009 16:15:04 -0500 Subject: [erlang-questions] Encoding a DNS Query correctly In-Reply-To: References: Message-ID: <20091116211504.GC4286@hijacked.us> On Mon, Nov 16, 2009 at 02:56:11PM -0500, Jarrod Roberson wrote: > I am writing a Bonjour/Zeroconf implemenation in native Erlang. This is > basically multicast DNS. > I have the listener and parser for the resulting DNS records working. > What I can't get right is the encoding for sending. > > Here is what I am trying: > > send() -> > {ok,S} = gen_udp:open(5353,[{reuseaddr,true}, {ip,{224,0,0,251}}, > {multicast_ttl,4}, {multicast_loop,false}, {broadcast,true}, binary]), > {ok,P} = > inet_dns:encode(#dns_query{domain="_daap._tcp.local",type=ptr,class=in}), > gen_udp:send(S,{224,0,0,251},5353,P), > gen_udp:close(S). > > Here is what I am getting in the console when I call :send() > > 24> test:send(). > ** exception error: {badrecord,dns_rec} > in function inet_dns:encode/1 > in call from test:send/0 > 25> > > If I can get over this hump I will have a basic implementation done. > Anyone have any ideas? Maybe try adding -include_lib("kernel/src/inet_dns.hrl"). or something? It looks like you're missing the definition of the dns_rec record. Andrew From jarrod@REDACTED Mon Nov 16 23:02:35 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Mon, 16 Nov 2009 17:02:35 -0500 Subject: [erlang-questions] Encoding a DNS Query correctly In-Reply-To: <20091116211504.GC4286@hijacked.us> References: <20091116211504.GC4286@hijacked.us> Message-ID: thanks for the responses I now have it working! send(Domain) -> {ok,S} = gen_udp:open(5353,[{reuseaddr,true}, {ip,{224,0,0,251}}, {multicast_ttl,4}, {multicast_loop,false}, {broadcast,true}, binary]), P = #dns_rec{header=#dns_header{},qdlist=[#dns_query{domain=Domain,type=ptr,class= in}]}, gen_udp:send(S,{224,0,0,251},5353,inet_dns:encode(P)), gen_udp:close(S). From jabronson@REDACTED Mon Nov 16 23:03:11 2009 From: jabronson@REDACTED (Joshua Bronson) Date: Mon, 16 Nov 2009 17:03:11 -0500 Subject: [erlang-questions] tk requirement? In-Reply-To: References: <299cc2dd0911161248n753344b5wd95d39236b3855bf@mail.gmail.com> Message-ID: <299cc2dd0911161403u566da8b7k1fa1c382f97cf854@mail.gmail.com> Hey Paul, On Mon, Nov 16, 2009 at 4:00 PM, Paul Davis wrote: > On Mon, Nov 16, 2009 at 3:48 PM, Joshua Bronson > wrote: > > Hey Erlang folks, > > > > I know relatively little about the Erlang distribution (my exposure is > > pretty much limited to installing it as a requirement of couchdb and > > rabbitmq-server). I was wondering if tk is (or has to be?) a hard > > requirement of the distribution or if (like e.g. Python) it's possible > > to build Erlang with a no_tk variant / USE flag / etc. in the case > > that you know you won't be needing to do any gui stuff. (This query is > > prompted by http://trac.macports.org/ticket/20006.) > > > > Thanks! > > > > Josh > > > > Josh, > > I've never had any issue with it using the system installed wx > libraries on 10.5 or 10.6. Reading through the ticket, the library you > want to check for linkage is probably > $PREFIX/lib/erlang/lib/wx-0.98.1/priv/i386-apple-darwin9.6.0/wxe_driver.so. > > For instance: > > $ otool -L > /usr/local/lib/erlang/lib/wx-0.98.1/priv/i386-apple-darwin9.6.0/wxe_driver.so > > hrm, i looked in /usr/local/lib/erlang/lib and there's a wx-0.98.3 directory (must be the version the latest macports version (R13B02) ships with? though on my Snow Leopard machine "/usr/bin/wx-config --version" gives 2.8.8) but all it contains are the directories doc, ebin, include, and src; "find /opt/local/lib/erlang/lib/wx-0.98.3/ -name \*.so" gives no output. do i take it that by commenting out "port:wxWidgets" from my erlang Portfile's depends_build, I built erlang without wxwidgets? Granted, that's not the same as disabling the whole thing altogether. > indeed! would still love to know if there's actual official support for disabling wx/tk :) > HTH, > Paul Davis Thanks, Josh From jabronson@REDACTED Mon Nov 16 23:04:49 2009 From: jabronson@REDACTED (Joshua Bronson) Date: Mon, 16 Nov 2009 17:04:49 -0500 Subject: [erlang-questions] tk requirement? In-Reply-To: <299cc2dd0911161403u566da8b7k1fa1c382f97cf854@mail.gmail.com> References: <299cc2dd0911161248n753344b5wd95d39236b3855bf@mail.gmail.com> <299cc2dd0911161403u566da8b7k1fa1c382f97cf854@mail.gmail.com> Message-ID: <299cc2dd0911161404m24a789d7mdc40c0aa18e3aad7@mail.gmail.com> On Mon, Nov 16, 2009 at 5:03 PM, Joshua Bronson wrote: > hrm, i looked in /usr/local/lib/erlang/lib > by which I meant /opt/local/lib/erlang/lib of course, sorry! From aduston@REDACTED Tue Nov 17 01:34:42 2009 From: aduston@REDACTED (adam) Date: Mon, 16 Nov 2009 16:34:42 -0800 (PST) Subject: Constantly archiving old mnesia records Message-ID: <15dc2b46-1efd-488e-865a-be987ae28fa2@k4g2000yqb.googlegroups.com> I have an application in which I have several million new records coming in each day, and I need to make sure that no duplicate ids get past a certain point in the system. To do this, I'm using a distributed mnesia database. Duplicates can only occur within a few days of each other, so I want to ship all old ids off to an archive every week. I could do this by simply selecting old records, iterating through, saving them somewhere else, and deleting them, but I'm afraid of this locking the entire table, and I'm also afraid that this will cause the db file to get pretty bloated over time. The other way to do it is to make a module implementing the mnesia_frag_hash callback behavior which would create a fragment containing old records when I call mnesia:change_table_frag with add_frag. Then perhaps I could just move the data using the filesystem (?). Any suggestions for doing this? Thanks in advance! Adam From ngocdaothanh@REDACTED Tue Nov 17 01:51:23 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Tue, 17 Nov 2009 09:51:23 +0900 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> Message-ID: <5c493e530911161651w41e46aecu823bd4d36236815f@mail.gmail.com> >From inet's doc: http://www1.erlang.org/documentation/doc-4.9.1/lib/kernel-2.4.1/doc/html/inet.html If the active option is true, which is the default, everything received from the socket will be sent as messages to the receiving process. If the active option is set to false (passive mode), the process must explicitly receive incoming data by calling gen_tcp:recv/N or gen_udp:recv/N (depending on the type of socket). Note: Passive mode provides flow control; the other side will not be able send faster than the receiver can read. Active mode provides no flow control; a fast sender could easily overflow the receiver with incoming messages. Use active mode only if your high-level protocol provides its own flow control (for instance, acknowledging received messages) or the amount of data exchanged is small. On Tue, Nov 17, 2009 at 2:59 AM, ERLANG wrote: > Hi Chandru ! > > That's fix my problem. Thanks. > While googling a bit, I found two ways to read from the Socket: > > recv(Socket, Bin) -> > ? ?receive > ? ? ? ?{tcp, Socket, B} -> > ? ? ? ? ? ?io:format(".", []), > ? ? ? ? ? ?recv(Socket, concat_binary([Bin, B])); > ? ? ? ?{tcp_closed, Socket} -> > ? ? ? ? ? ?{ok, Bin}; > ? ? ? ?Other -> > ? ? ? ? ? ?{error, {socket, Other}} > ? ? ? ?after > ? ? ? ? ? ??TIMEOUT -> > ? ? ? ? ? ?{error, {socket, timeout}} > ? ?end. > > % version 2 with "gen_tcp:recv" > recv2(Socket, Bin) -> > ? ?case gen_tcp:recv(Socket, 0, ?TIMEOUT) of > ? ? ? ? {ok, B} -> > ? ? ? ? ? ? io:format(".", []), > ? ? ? ? ? ? recv(Socket, concat_binary([Bin, B])); > ? ? ? ? {error, closed} -> > ? ? ? ? ? ? {ok, Bin}; > ? ? ? ?{error, timeout} -> > ? ? ? ? ? ? {error, {socket, timeout}}; > ? ? ? ? Other -> > ? ? ? ? ? ? {error, {socket, Other}} > ? ? end. > > > Which one is the best in my case (see below: fetch.erl)? > > Regards > Zabrane > > Le 16 nov. 09 ? 18:53, Chandru a ?crit : > >> You are expecting the server to indicate end of response by closing the >> connection, but because you specify HTTP/1.1 in the request, the server is >> holding up your connection, and you are timing out. Try replacing HTTP/1.1 >> with HTTP/1.0 in your request, or parse the response to detect end of >> response. >> >> cheers >> Chandru >> >> 2009/11/16 zabrane Mikael >> >>> Hi List ! >>> >>> New to Erlang, I'm trying to implement a simple URL fetcher. >>> Here's my code (please, feel free to correct it if you find any bug or >>> know >>> a better approach): >>> >>> >>> >>> 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8---- >>> -module(fetch). >>> >>> -export([url/1]). >>> >>> -define(TIMEOUT, ? ?7000). >>> -define(TCP_OPTS, ? [binary, {packet, raw}, {nodelay, true}, >>> ? ? ? ? ? ? ? ? ? {active, true}]). >>> >>> url(Url) -> >>> ?{ok, _Tag, Host, Port} = split_url(Url), >>> >>> ?Hdrs = [], >>> ?Request = ["GET ", Url, " HTTP/1.1\r\n", Hdrs, "\r\n\r\n"], >>> >>> ?case catch gen_tcp:connect(Host, Port, ?TCP_OPTS) of >>> ? ?{'EXIT', Why} -> >>> ? ? ? ? ?{error, {socket_exit, Why}}; >>> ? ? ?{error, Why} -> >>> ? ? ? ? ?{error, {socket_error, Why}}; >>> ? ? ?{ok, Socket} -> >>> ? ? ? ? ?gen_tcp:send(Socket, list_to_binary(Request)), >>> ? ? ? ? ?recv(Socket, list_to_binary([])) >>> ?end. >>> >>> recv(Socket, Bin) -> >>> ?receive >>> ? ? ?{tcp, Socket, B} -> >>> ? ? ? ? ?io:format(".", []), >>> ? ? ? ? ?recv(Socket, concat_binary([Bin, B])); >>> ? ? ?{tcp_closed, Socket} -> >>> ? ? ? ? ?{ok, Bin}; >>> ? ? ?Other -> >>> ? ? ? ? ?{error, {socket, Other}} >>> after >>> ??TIMEOUT -> >>> ? ? ? ? ?{error, {socket, timeout}} >>> ?end. >>> >>> >>> split_url([$h,$t,$t,$p,$:,$/,$/|T]) -> ?split_url(http, T); >>> split_url(_X) ? ? ? ? ? ? ? ? ? ? ? -> ?{error, split_url}. >>> >>> split_url(Tag, X) -> >>> ?case string:chr(X, $:) of >>> ? ? ?0 -> >>> ? ? ? ? ?Port = 80, >>> ? ? ? ? ?case string:chr(X,$/) of >>> ? ? ? ? ? ? ?0 -> >>> ? ? ? ? ? ? ? ? ?{ok, Tag, X, Port}; >>> ? ? ? ? ? ? ?N -> >>> ? ? ? ? ? ? ? ? ?Site = string:substr(X,1,N-1), >>> ? ? ? ? ? ? ? ? ?{ok, Tag, Site, Port} >>> ? ? ? ? ?end; >>> ? ? ?N1 -> >>> ? ? ? ? ?case string:chr(X,$/) of >>> ? ? ? ? ? ? ?0 -> >>> ? ? ? ? ? ? ? ? ?error; >>> ? ? ? ? ? ? ?N2 -> >>> ? ? ? ? ? ? ? ? ?PortStr = string:substr(X,N1+1, N2-N1-1), >>> ? ? ? ? ? ? ? ? ?case catch list_to_integer(PortStr) of >>> ? ? ? ? ? ? ? ? ? ? ?{'EXIT', _} -> >>> ? ? ? ? ? ? ? ? ? ? ? ? ?{error, port_number}; >>> ? ? ? ? ? ? ? ? ? ? ?Port -> >>> ? ? ? ? ? ? ? ? ? ? ? ? ?Site = string:substr(X,1,N1-1), >>> ? ? ? ? ? ? ? ? ? ? ? ? ?{ok, Tag, Site, Port} >>> ? ? ? ? ? ? ? ? ?end >>> ? ? ? ? ?end >>> ?end. >>> >>> >>> >>> 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8------ >>> >>> When testing it, the receiving socket gets very very slow: >>> $ erl >>> 1> c(fetch). >>> 2> Bin = fetch:url("http://www.google.com"). >>> ......{error,{socket,timeout}} >>> >>> Am I missing something? >>> What I like to get at the end is a very fast fetcher. Any hint? >>> >>> Regards >>> Zabrane >>> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From kagato@REDACTED Tue Nov 17 02:34:47 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Mon, 16 Nov 2009 17:34:47 -0800 Subject: Problem With Common Test Message-ID: <9A3BAE41-5D12-4398-AB8F-D87A3CBAB4A5@souja.net> I'm trying to get common_test working. I have a few problems. First, the install procedure is byzantine. I'm not asking for an immediate fix, but really, there has to be a better way. Next, if I use the example (which includes ct.hrl), I get the following trying to do an erl -make. Recompile: src/isopod_SUITE /opt/local/lib/erlang/lib/common_test-1.4.1/include/ct.hrl:20: can't find include file "test_server.hrl" So, it appears that it should be using include_lib, instead of include. If I write the test without the include, it works, but I can't use the ?line macros. Third, when I try to load the Visual Test Server, I get this: > Erlang R13B (erts-5.7.1) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] > > Starting webtool... > Eshell V5.7.1 (abort with ^G) > (ct@REDACTED)1> WebTool is available at http://localhost:8888/ > Or http://127.0.0.1:8888/ > Starting vts... > Sending URL to netscape...sh: line 0: exec: netscape: not found > not running, starting netscape... > > Common Test starting (cwd is /Users/kagato/engineyard/swarm) > > CT failed to install configuration data. Please verify that the log directory exists and that write permission is set. > > > User switch command > --> q Ignoring the "netscape" error, the "CT failed to install configuration data" is a problem. It appears to be contently using the current directory (and even creates a variables file), but it always gives this error. Specifying config files hasn't helped. Fourth, when will I be able to usably get coverage data? The requirement that the .beam files be in the same directory as the .erl files is pretty much a nonstarter. The feature might as well not exist, since it's completely incompatible with any of the packaging tools. Thanks, -- Jayson Vantuyl kagato@REDACTED From dizzyd@REDACTED Tue Nov 17 03:32:41 2009 From: dizzyd@REDACTED (Dave Smith) Date: Mon, 16 Nov 2009 19:32:41 -0700 Subject: [erlang-questions] Problem With Common Test In-Reply-To: <9A3BAE41-5D12-4398-AB8F-D87A3CBAB4A5@souja.net> References: <9A3BAE41-5D12-4398-AB8F-D87A3CBAB4A5@souja.net> Message-ID: On Mon, Nov 16, 2009 at 6:34 PM, Jayson Vantuyl wrote: > I'm trying to get common_test working. ?I have a few problems. > > First, the install procedure is byzantine. ?I'm not asking for an immediate fix, but really, there has to be a better way. Agreed. As such, I don't usually follow the install procedure but puzzled out the necessary mojo to launch a CT test suite from the command line. Something along the lines of: erl -pa MyEbinPath -I MyInclude -noshell -s ct_run script_start -s erlang halt -logdir ALogDir -env TEST_DIR > Next, if I use the example (which includes ct.hrl), I get the following trying to do an erl -make. > > Recompile: src/isopod_SUITE > /opt/local/lib/erlang/lib/common_test-1.4.1/include/ct.hrl:20: can't find include file "test_server.hrl" > > So, it appears that it should be using include_lib, instead of include. ?If I write the test without the include, it works, but I can't use the ?line macros. To accomodate this...quirk..I usually programatically find the common_test path and add the include dir to the erl command line. > Fourth, when will I be able to usably get coverage data? ?The requirement that the .beam files be in the same directory as the .erl files is pretty much a nonstarter. ?The feature might as well not exist, since it's completely incompatible with any of the packaging tools. Yes, you can get coverage data by defining a cover.spec file (that's the name I use) with the following contents: {levels, details}. {incl_dirs_r, ["ebin"]}. Where the ebin directory is relative to where you are running erlang from the command line. You'll also need to add: -cover cover.spec to your erlang command line. HTH. common_test is really quite nice once you get it hooked up and running. Not sure why it's so byzantine. Maybe I'm just doing it Wrong. :) D. From jarrod@REDACTED Tue Nov 17 03:35:24 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Mon, 16 Nov 2009 21:35:24 -0500 Subject: Multicast UDP sending question Message-ID: I have gotten a good start on my program, my first REAL Erlang program. I have it listening for messages, reading them and parsing them. I also have it sending them. The one little thing that is bothering me is I can't SEND on Port 5353, I have tried everything. All the other applications on my machine and listen AND send on port 5353, SubEthaEdit, iTunes, iChat. The all report Port: 5353 when sending multicast messages. I really want my application to play nice and do the same thing, send on Port 5353. Here is my module as it stands now. -module(zeroconf). -include("zeroconf.hrl"). -export([open/0,start/0]). -export([stop/1,receiver/0]). -export([send/1]). -define(ADDR, {224,0,0,251}). -define(PORT, 5353). send(Domain) -> {ok,S} = gen_udp:open(0,[{broadcast,true}]), % I really want this Port to be 5353 :-( %{ok,S} = gen_udp:open(?PORT,[{reuseaddr,true}, {ip,?ADDR}, {broadcast,true}, {multicast_ttl,4}, {multicast_loop,false}, binary]), % this doesn't complain or throw errors but it also doesn't work :-( P = #dns_rec{header=#dns_header{},qdlist=[#dns_query{domain=Domain,type=ptr,class=in}]}, gen_udp:send(S,?ADDR,?PORT,inet_dns:encode(P)), gen_udp:close(S). open() -> {ok,S} = gen_udp:open(?PORT,[{reuseaddr,true}, {ip,?ADDR}, {multicast_ttl,4}, {multicast_loop,false}, binary]), inet:setopts(S,[{add_membership,{?ADDR,{0,0,0,0}}}]), S. close(S) -> gen_udp:close(S). start() -> S=open(), Pid=spawn(?MODULE,receiver,[]), gen_udp:controlling_process(S,Pid), {S,Pid}. stop({S,Pid}) -> close(S), Pid ! stop. receiver() -> receive {udp, _Socket, IP, InPortNo, Packet} -> io:format("~n~nFrom: ~p~nPort: ~p~nData: ~p~n",[IP,InPortNo,inet_dns:decode(Packet)]), receiver(); stop -> true; AnythingElse -> io:format("RECEIVED: ~p~n",[AnythingElse]), receiver() end. Here is what some output looks like. This is a QUERY from SubEthaEdit looking for other instances on the local network, notice that it says Port: 5353 From: {192,168,0,105} Port: 5353 Data: {ok,{dns_rec,{dns_header,0,true,'query',true,false,false,false,false,0}, [], [{dns_rr,"_see._tcp.local",ptr,in,0,0, "jhr@REDACTED",undefined,[], false}], [],[]}} Now here is a QUERY from my module looking for instances of iTunes on the local network, notice it says Port: 59795 With the code the way it is now, that port is random. I really want it to be 5353. From: {192,168,0,105} Port: 59795 Data: {ok,{dns_rec,{dns_header,0,false,'query',false,false,false,false,false, 0}, [{dns_query,"_daap._tcp.local",ptr,in}], [],[],[]}} Does anyone have any arcane insight in to UDP multicast at all? -- Jarrod Roberson 678.551.2852 From tushar.deshpande@REDACTED Tue Nov 17 07:45:46 2009 From: tushar.deshpande@REDACTED (Tushar Deshpande) Date: Tue, 17 Nov 2009 01:45:46 -0500 Subject: Why using pmap is not giving me the desired speedup ? Message-ID: <6b1b97d20911162245p61445058x3aaaf4048b628368@mail.gmail.com> Hi, I'm a newcomer to the Erlang world. I'm trying to learn parallel programming with Erlang. I tried to use Joe Armstrong's 'pmap' implementation to write a parallel quicksort. I imagined that I would get some observable speedup with parallel quicksort. I executed parallel quicksort on my dual-core notebook. I varied the erlang schedulers from 1 to 32. I ran quicksort on several lists with lengths 1, 10, 100, 1000, 10000 and 100000. Contrary to my expectations, parallel version took more time to run. [The pmap gave significant speedup for example on page 371 (ptests.erl) in the Joe Armstrong's book] What could be the reason ? Notice, that the pqsort is non-tail-recursive function. Does this have anything to do with my observations ? Wouls someone please help me ? Here's my quicksort implementation for easy reference. -module(qsort). -compile(export_all). -import(lib_parallel, [ pmap/2]). pqsort([]) -> ? []; pqsort([Pivot|T]) -> ? L1 = [X || X <- T, X < Pivot], ? L2 = [X || X <- T, X >= Pivot], ? [Left,Right] = pmap(fun pqsort/1, [L1,L2]), ? Left ++ [Pivot] ++ Right. qsort([]) -> ? []; qsort([Pivot|T]) -> ? L1 = [X || X <- T, X < Pivot], ? L2 = [X || X <- T, X >= Pivot], ? Left = qsort(L1), ? Right = qsort(L2), ? Left ++ [Pivot] ++ Right. -- Warm Regards, Tushar Deshpande From erlang@REDACTED Tue Nov 17 09:52:33 2009 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 17 Nov 2009 09:52:33 +0100 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> Message-ID: <9b08084c0911170052pc8cb3cbt2b943c169bf2eca7@mail.gmail.com> On Mon, Nov 16, 2009 at 6:53 PM, Chandru wrote: > You are expecting the server to indicate end of response by closing the > connection, but because you specify HTTP/1.1 in the request, the server is > holding up your connection, and you are timing out. Try replacing HTTP/1.1 > with HTTP/1.0 in your request, or parse the response to detect end of > response. This will get you into murkier waters - you'll have to check if a content length is defined and then read exactly this number of bytes. I wrote a tutorial about this a while back http://www.sics.se/~joe/tutorials/web_server/web_server.html You'll find a module called http_driver here that does the parsing and collects the appropriate number of bytes. /Joe I > > cheers > Chandru > > 2009/11/16 zabrane Mikael > >> Hi List ! >> >> New to Erlang, I'm trying to implement a simple URL fetcher. >> Here's my code (please, feel free to correct it if you find any bug or know >> a better approach): >> >> >> 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8---- >> -module(fetch). >> >> -export([url/1]). >> >> -define(TIMEOUT, ? ?7000). >> -define(TCP_OPTS, ? [binary, {packet, raw}, {nodelay, true}, >> ? ? ? ? ? ? ? ? ? ? {active, true}]). >> >> url(Url) -> >> ? ?{ok, _Tag, Host, Port} = split_url(Url), >> >> ? ?Hdrs = [], >> ? ?Request = ["GET ", Url, " HTTP/1.1\r\n", Hdrs, "\r\n\r\n"], >> >> ? ?case catch gen_tcp:connect(Host, Port, ?TCP_OPTS) of >> ? ? ?{'EXIT', Why} -> >> ? ? ? ? ? ?{error, {socket_exit, Why}}; >> ? ? ? ?{error, Why} -> >> ? ? ? ? ? ?{error, {socket_error, Why}}; >> ? ? ? ?{ok, Socket} -> >> ? ? ? ? ? ?gen_tcp:send(Socket, list_to_binary(Request)), >> ? ? ? ? ? ?recv(Socket, list_to_binary([])) >> ? ?end. >> >> recv(Socket, Bin) -> >> ? ?receive >> ? ? ? ?{tcp, Socket, B} -> >> ? ? ? ? ? ?io:format(".", []), >> ? ? ? ? ? ?recv(Socket, concat_binary([Bin, B])); >> ? ? ? ?{tcp_closed, Socket} -> >> ? ? ? ? ? ?{ok, Bin}; >> ? ? ? ?Other -> >> ? ? ? ? ? ?{error, {socket, Other}} >> after >> ? ??TIMEOUT -> >> ? ? ? ? ? ?{error, {socket, timeout}} >> ? ?end. >> >> >> split_url([$h,$t,$t,$p,$:,$/,$/|T]) -> ?split_url(http, T); >> split_url(_X) ? ? ? ? ? ? ? ? ? ? ? -> ?{error, split_url}. >> >> split_url(Tag, X) -> >> ? ?case string:chr(X, $:) of >> ? ? ? ?0 -> >> ? ? ? ? ? ?Port = 80, >> ? ? ? ? ? ?case string:chr(X,$/) of >> ? ? ? ? ? ? ? ?0 -> >> ? ? ? ? ? ? ? ? ? ?{ok, Tag, X, Port}; >> ? ? ? ? ? ? ? ?N -> >> ? ? ? ? ? ? ? ? ? ?Site = string:substr(X,1,N-1), >> ? ? ? ? ? ? ? ? ? ?{ok, Tag, Site, Port} >> ? ? ? ? ? ?end; >> ? ? ? ?N1 -> >> ? ? ? ? ? ?case string:chr(X,$/) of >> ? ? ? ? ? ? ? ?0 -> >> ? ? ? ? ? ? ? ? ? ?error; >> ? ? ? ? ? ? ? ?N2 -> >> ? ? ? ? ? ? ? ? ? ?PortStr = string:substr(X,N1+1, N2-N1-1), >> ? ? ? ? ? ? ? ? ? ?case catch list_to_integer(PortStr) of >> ? ? ? ? ? ? ? ? ? ? ? ?{'EXIT', _} -> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ?{error, port_number}; >> ? ? ? ? ? ? ? ? ? ? ? ?Port -> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ?Site = string:substr(X,1,N1-1), >> ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ok, Tag, Site, Port} >> ? ? ? ? ? ? ? ? ? ?end >> ? ? ? ? ? ?end >> ? ?end. >> >> >> 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8------ >> >> When testing it, the receiving socket gets very very slow: >> $ erl >> 1> c(fetch). >> 2> Bin = fetch:url("http://www.google.com"). >> ......{error,{socket,timeout}} >> >> Am I missing something? >> What I like to get at the end is a very fast fetcher. Any hint? >> >> Regards >> Zabrane >> > From peppe@REDACTED Tue Nov 17 10:30:49 2009 From: peppe@REDACTED (Peter Andersson) Date: Tue, 17 Nov 2009 10:30:49 +0100 Subject: [erlang-questions] Problem With Common Test In-Reply-To: <9A3BAE41-5D12-4398-AB8F-D87A3CBAB4A5@souja.net> References: <9A3BAE41-5D12-4398-AB8F-D87A3CBAB4A5@souja.net> Message-ID: <4B026D49.2010805@erix.ericsson.se> Hi, As far as installation goes, the documentation really needs an update! Before OTP R12, Common Test and Test Server weren't released as OTP applications (and e.g. didn't have version numbers), and then you would typically install these two apps separately from Erlang/OTP and generate the run_test script (unix/linux) in the same place. You can still do this, *but* what you would rather do these days is go to the common_test directory under Erlang/OTP lib and run "./install.sh local" there. Then you'd copy run_test from the CT priv/bin directory to where you want it (the directory names expanded in the script are absolute), or update $PATH with the CT priv/bin directory, or maybe create an alias or a link to the script. Note however that the installation procedure described in the User's Guide is only necessary if you want/need the run_test script!! As with any app that comes with OTP, you don't need an explicit install if you're going to use it from the Erlang shell, or from an Erlang program. Really, the installation chapter in the User's Guide needs to be rewritten. I'll get to it promptly! Thanks for pointing the problem out! About compiling test suites... Common Test has an auto compile feature (chapter 6.2), i.e. it will compile your suite modules for you before running the test. However, if you do want to compile the suite modules yourself, you need to add the common_test and test_server include directories to the compiler include path. Also, then, you should disable the CT auto compile feature with the 'no_auto_compile' flag/option when you run your test. About the vts mode... One major drawback of this mode *is* that it uses the current working directory as log directory. We've been wanting to replace the existing, and very limited, vts mode with a much more useful GUI mode. For that reason we haven't put effort into building all the newer CT features into this mode. That said, if you follow the instructions in chapter 6.4 to solve the "can't start netscape" problem (which is actually a problem with the Webtools app rather than CT), and you do have write permissions on the current working directory, vts really *should* work. Try to first get a browser started and if the configuration installation problem remains, get back to me, ok!? About code coverage... It's not a requirement for the code coverage analysis to work, that beam and source code modules are located in the same directory! It's only if you want to generate html result files you have this restriction. We will remove this restriction soon by making it possible for CT to take a source code path as input (probably by means of the cover spec file). It should be possible to locate the source code for both apps that follow the OTP directory convention, and for apps that don't. /Peter Ericsson AB, Erlang/OTP Jayson Vantuyl wrote: > I'm trying to get common_test working. I have a few problems. > > First, the install procedure is byzantine. I'm not asking for an immediate fix, but really, there has to be a better way. > > Next, if I use the example (which includes ct.hrl), I get the following trying to do an erl -make. > > Recompile: src/isopod_SUITE > /opt/local/lib/erlang/lib/common_test-1.4.1/include/ct.hrl:20: can't find include file "test_server.hrl" > > So, it appears that it should be using include_lib, instead of include. If I write the test without the include, it works, but I can't use the ?line macros. > > Third, when I try to load the Visual Test Server, I get this: > > > Erlang R13B (erts-5.7.1) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] > > > > Starting webtool... > > Eshell V5.7.1 (abort with ^G) > > (ct@REDACTED)1> WebTool is available at http://localhost:8888/ > > Or http://127.0.0.1:8888/ > > Starting vts... > > Sending URL to netscape...sh: line 0: exec: netscape: not found > > not running, starting netscape... > > > > Common Test starting (cwd is /Users/kagato/engineyard/swarm) > > > > CT failed to install configuration data. Please verify that the log directory exists and that write permission is set. > > > > > > User switch command > > --> q > > Ignoring the "netscape" error, the "CT failed to install configuration data" is a problem. It appears to be contently using the current directory (and even creates a variables file), but it always gives this error. Specifying config files hasn't helped. > > Fourth, when will I be able to usably get coverage data? The requirement that the .beam files be in the same directory as the .erl files is pretty much a nonstarter. The feature might as well not exist, since it's completely incompatible with any of the packaging tools. > > Thanks, > > From clist@REDACTED Tue Nov 17 10:32:21 2009 From: clist@REDACTED (Angel Alvarez) Date: Tue, 17 Nov 2009 10:32:21 +0100 Subject: [erlang-questions] Why using pmap is not giving me the desired speedup ? In-Reply-To: <6b1b97d20911162245p61445058x3aaaf4048b628368@mail.gmail.com> References: <6b1b97d20911162245p61445058x3aaaf4048b628368@mail.gmail.com> Message-ID: <200911171032.21602.clist@uah.es> El Martes, 17 de Noviembre de 2009 07:45:46 Tushar Deshpande escribi?: > Hi, > > I'm a newcomer to the Erlang world. I'm trying > to learn parallel programming with Erlang. Welcome!! im newbie too! > > I tried to use Joe Armstrong's 'pmap' implementation > to write a parallel quicksort. > > I imagined that I would get some observable > speedup with parallel quicksort. Well many people at blogs , on the lists and other groups talk about this. Where Joe talk about concurrency they ear "paralelism", those are not the same concepts. The more concurrency, you put in your programs the more overall paralelism youll get in the long term. But you can make a highly parallel program with almost no concurrency. I see this problem as the Haskell vs Erlang or Clojure vs Erlang wars and that why many other languages took some erlang features like procceses and distributions but left others maybe more interesting like a strong exception support. I saw on the list that erlang people change from "functional" to "concurrency oriented" when refering to erlang to help others change minds. > > I executed parallel quicksort on my dual-core > notebook. I varied the erlang schedulers from > 1 to 32. More schedulers than cores is a bad idea, they end contending for the CPU's Probably is best One scheduler per core ( maybe some core can be devoted to the async pool) > I ran quicksort on several lists with lengths > 1, 10, 100, 1000, 10000 and 100000. There is alway a balance beetween cost of computations and the overhead of distributing them. Also Amdahl's law marks the upper bound for speedups. Make several runs with 1 scheduler and promediate results then go for more cored up to the maximum you have (2) and promediate results. That have to measure speedups , OTHO your code seems wrong. where is the conditions that change using pqsort to plain qsort? It seems you keep spawning procceses also when you have pretty tiny lists down to one element. Somewhere on the run its better to call plain qsort that keep spawning because the cost for the plain function will be always better than than pmap way. Despite of ligth proccesses on Erlang that always applies, also on small data other algos as bubllesort sort can be as good as qsort or better... /Angel > > Contrary to my expectations, parallel version > took more time to run. [The pmap gave > significant speedup for example on page > 371 (ptests.erl) in the Joe Armstrong's book] > > What could be the reason ? > > Notice, that the pqsort is non-tail-recursive > function. Does this have anything to do > with my observations ? > > Wouls someone please help me ? > > Here's my quicksort implementation for > easy reference. > > -module(qsort). > -compile(export_all). > -import(lib_parallel, [ pmap/2]). > > pqsort([]) -> > ? []; > pqsort([Pivot|T]) -> > ? L1 = [X || X <- T, X < Pivot], > ? L2 = [X || X <- T, X >= Pivot], > ? [Left,Right] = pmap(fun pqsort/1, [L1,L2]), > ? Left ++ [Pivot] ++ Right. > > qsort([]) -> > ? []; > qsort([Pivot|T]) -> > ? L1 = [X || X <- T, X < Pivot], > ? L2 = [X || X <- T, X >= Pivot], > ? Left = qsort(L1), > ? Right = qsort(L2), > ? Left ++ [Pivot] ++ Right. > > -- > Warm Regards, > > Tushar Deshpande > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- __________________________________________ Clist UAH a.k.a Angel __________________________________________ Los dinosaurios murieron porque no ten?an un programa espacial. Larry Niven. From john.hughes@REDACTED Tue Nov 17 13:03:49 2009 From: john.hughes@REDACTED (John Hughes) Date: Tue, 17 Nov 2009 13:03:49 +0100 Subject: Announcement: new QuickCheck release and demo In-Reply-To: <1258394538.32553.ezmlm@erlang.org> References: <1258394538.32553.ezmlm@erlang.org> Message-ID: Quviq is happy to announce a new QuickCheck release, which supports race condition testing with a randomising scheduler, generation of test data from yecc grammars, and a new mechanism for testing C code from Erlang. A fifteen minute video demo, showing QuickCheck testing a circular buffer implementation in C, has been posted on Quviq's web site--see under "Demos" at quviq.com. From torben.lehoff@REDACTED Tue Nov 17 14:21:33 2009 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Tue, 17 Nov 2009 14:21:33 +0100 Subject: [erlang-questions] Multicast UDP sending question In-Reply-To: References: Message-ID: Hi Jarrod, I have found some old code where we set-up UDP multicast - I cannot remember the details, but it is not quite like the approach you are using: Transmit gen_udp:open(?PORT_NUM_TX_MULTI, [binary, {active, true}, {ip, GwIP}, inet, {multicast_ttl, 255}, {multicast_loop, false}, {multicast_if, GwIP}]), Receive: gen_udp:open(?PORT_NUM_RX_MULTI, [binary, {active, true}, {multicast_if, GwIP}, inet,{multicast_ttl, 255}, {multicast_loop, false}, {add_membership, {MultiAddr,GwIP}}]), The two ports PORT_NUM_TX_MULTI and PORT_NUM_RX_MULTI should be N and N+1 where N is your desired port number - as far as I understand UDP/TCP for Erlang the ports always comes in pairs due to some things with the C socket library used, but do check up on this by yourself or just try out the stuff above and see if it works ;-) Cheers, Torben On Tue, Nov 17, 2009 at 03:35, Jarrod Roberson wrote: > I have gotten a good start on my program, my first REAL Erlang program. > I have it listening for messages, reading them and parsing them. I also > have > it sending them. > The one little thing that is bothering me is I can't SEND on Port 5353, I > have tried everything. > All the other applications on my machine and listen AND send on port 5353, > SubEthaEdit, iTunes, iChat. > The all report Port: 5353 when sending multicast messages. > I really want my application to play nice and do the same thing, send on > Port 5353. > Here is my module as it stands now. > > -module(zeroconf). > > -include("zeroconf.hrl"). > > -export([open/0,start/0]). > -export([stop/1,receiver/0]). > -export([send/1]). > > -define(ADDR, {224,0,0,251}). > -define(PORT, 5353). > > send(Domain) -> > {ok,S} = gen_udp:open(0,[{broadcast,true}]), % I really want this Port > to be 5353 :-( > %{ok,S} = gen_udp:open(?PORT,[{reuseaddr,true}, {ip,?ADDR}, > {broadcast,true}, {multicast_ttl,4}, {multicast_loop,false}, binary]), % > this doesn't complain or throw errors but it also doesn't work :-( > P = > > #dns_rec{header=#dns_header{},qdlist=[#dns_query{domain=Domain,type=ptr,class=in}]}, > gen_udp:send(S,?ADDR,?PORT,inet_dns:encode(P)), > gen_udp:close(S). > > open() -> > {ok,S} = gen_udp:open(?PORT,[{reuseaddr,true}, {ip,?ADDR}, > {multicast_ttl,4}, {multicast_loop,false}, binary]), > inet:setopts(S,[{add_membership,{?ADDR,{0,0,0,0}}}]), > S. > > close(S) -> gen_udp:close(S). > > start() -> > S=open(), > Pid=spawn(?MODULE,receiver,[]), > gen_udp:controlling_process(S,Pid), > {S,Pid}. > > stop({S,Pid}) -> > close(S), > Pid ! stop. > > receiver() -> > receive > {udp, _Socket, IP, InPortNo, Packet} -> > io:format("~n~nFrom: ~p~nPort: ~p~nData: > ~p~n",[IP,InPortNo,inet_dns:decode(Packet)]), > receiver(); > stop -> true; > AnythingElse -> io:format("RECEIVED: ~p~n",[AnythingElse]), > receiver() > end. > > Here is what some output looks like. > > This is a QUERY from SubEthaEdit looking for other instances on the local > network, notice that it says Port: 5353 > From: {192,168,0,105} > Port: 5353 > Data: > {ok,{dns_rec,{dns_header,0,true,'query',true,false,false,false,false,0}, > [], > [{dns_rr,"_see._tcp.local",ptr,in,0,0, > "jhr@REDACTED",undefined,[], > false}], > [],[]}} > > Now here is a QUERY from my module looking for instances of iTunes on the > local network, notice it says Port: 59795 > With the code the way it is now, that port is random. I really want it to > be > 5353. > > From: {192,168,0,105} > Port: 59795 > Data: > {ok,{dns_rec,{dns_header,0,false,'query',false,false,false,false,false, > 0}, > [{dns_query,"_daap._tcp.local",ptr,in}], > [],[],[]}} > > Does anyone have any arcane insight in to UDP multicast at all? > > > -- > Jarrod Roberson > 678.551.2852 > -- http://www.linkedin.com/in/torbenhoffmann From v@REDACTED Tue Nov 17 15:15:14 2009 From: v@REDACTED (Valentin Micic) Date: Tue, 17 Nov 2009 16:15:14 +0200 Subject: [erlang-questions] Multicast UDP sending question In-Reply-To: Message-ID: <20091117141526.6A1D03D0D4E@mail.pharos-avantgard.com> You cannot open the same UDP port twice (or, shall I say -- bound the same UDP port to two different sockets). The story about N, N+1 port as well as explanation indicated that this is an Erlang restriction due to the socket library used is - well... pure fiction. I haven't dealt with multicast recently, but as far as I remember you do not have to do anything special to be able to send to a multicast address -- any standard UDP socket would do; you just need to reference the multicast address as a destination IP address in sendto call. OTOH, if you want to receive multicast traffic you would have to add membership -- this causes local MAC layer to assume additional link address, so it can receive such a packet from the wire (*) V/ (*) If you sending a packet from the same UDP socket that holds membership to a multicast group to which you're sending to, such packet should be received by a sending socket as well. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Torben Hoffmann Sent: 17 November 2009 03:22 PM To: Jarrod Roberson Cc: Erlang Subject: Re: [erlang-questions] Multicast UDP sending question Hi Jarrod, I have found some old code where we set-up UDP multicast - I cannot remember the details, but it is not quite like the approach you are using: Transmit gen_udp:open(?PORT_NUM_TX_MULTI, [binary, {active, true}, {ip, GwIP}, inet, {multicast_ttl, 255}, {multicast_loop, false}, {multicast_if, GwIP}]), Receive: gen_udp:open(?PORT_NUM_RX_MULTI, [binary, {active, true}, {multicast_if, GwIP}, inet,{multicast_ttl, 255}, {multicast_loop, false}, {add_membership, {MultiAddr,GwIP}}]), The two ports PORT_NUM_TX_MULTI and PORT_NUM_RX_MULTI should be N and N+1 where N is your desired port number - as far as I understand UDP/TCP for Erlang the ports always comes in pairs due to some things with the C socket library used, but do check up on this by yourself or just try out the stuff above and see if it works ;-) Cheers, Torben On Tue, Nov 17, 2009 at 03:35, Jarrod Roberson wrote: > I have gotten a good start on my program, my first REAL Erlang program. > I have it listening for messages, reading them and parsing them. I also > have > it sending them. > The one little thing that is bothering me is I can't SEND on Port 5353, I > have tried everything. > All the other applications on my machine and listen AND send on port 5353, > SubEthaEdit, iTunes, iChat. > The all report Port: 5353 when sending multicast messages. > I really want my application to play nice and do the same thing, send on > Port 5353. > Here is my module as it stands now. > > -module(zeroconf). > > -include("zeroconf.hrl"). > > -export([open/0,start/0]). > -export([stop/1,receiver/0]). > -export([send/1]). > > -define(ADDR, {224,0,0,251}). > -define(PORT, 5353). > > send(Domain) -> > {ok,S} = gen_udp:open(0,[{broadcast,true}]), % I really want this Port > to be 5353 :-( > %{ok,S} = gen_udp:open(?PORT,[{reuseaddr,true}, {ip,?ADDR}, > {broadcast,true}, {multicast_ttl,4}, {multicast_loop,false}, binary]), % > this doesn't complain or throw errors but it also doesn't work :-( > P = > > #dns_rec{header=#dns_header{},qdlist=[#dns_query{domain=Domain,type=ptr,clas s=in}]}, > gen_udp:send(S,?ADDR,?PORT,inet_dns:encode(P)), > gen_udp:close(S). > > open() -> > {ok,S} = gen_udp:open(?PORT,[{reuseaddr,true}, {ip,?ADDR}, > {multicast_ttl,4}, {multicast_loop,false}, binary]), > inet:setopts(S,[{add_membership,{?ADDR,{0,0,0,0}}}]), > S. > > close(S) -> gen_udp:close(S). > > start() -> > S=open(), > Pid=spawn(?MODULE,receiver,[]), > gen_udp:controlling_process(S,Pid), > {S,Pid}. > > stop({S,Pid}) -> > close(S), > Pid ! stop. > > receiver() -> > receive > {udp, _Socket, IP, InPortNo, Packet} -> > io:format("~n~nFrom: ~p~nPort: ~p~nData: > ~p~n",[IP,InPortNo,inet_dns:decode(Packet)]), > receiver(); > stop -> true; > AnythingElse -> io:format("RECEIVED: ~p~n",[AnythingElse]), > receiver() > end. > > Here is what some output looks like. > > This is a QUERY from SubEthaEdit looking for other instances on the local > network, notice that it says Port: 5353 > From: {192,168,0,105} > Port: 5353 > Data: > {ok,{dns_rec,{dns_header,0,true,'query',true,false,false,false,false,0}, > [], > [{dns_rr,"_see._tcp.local",ptr,in,0,0, > "jhr@REDACTED",undefined,[], > false}], > [],[]}} > > Now here is a QUERY from my module looking for instances of iTunes on the > local network, notice it says Port: 59795 > With the code the way it is now, that port is random. I really want it to > be > 5353. > > From: {192,168,0,105} > Port: 59795 > Data: > {ok,{dns_rec,{dns_header,0,false,'query',false,false,false,false,false, > 0}, > [{dns_query,"_daap._tcp.local",ptr,in}], > [],[],[]}} > > Does anyone have any arcane insight in to UDP multicast at all? > > > -- > Jarrod Roberson > 678.551.2852 > -- http://www.linkedin.com/in/torbenhoffmann From jarrod@REDACTED Tue Nov 17 15:45:09 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Tue, 17 Nov 2009 09:45:09 -0500 Subject: [erlang-questions] Multicast UDP sending question In-Reply-To: <20091117141526.6A1D03D0D4E@mail.pharos-avantgard.com> References: <20091117141526.6A1D03D0D4E@mail.pharos-avantgard.com> Message-ID: On Tue, Nov 17, 2009 at 9:15 AM, Valentin Micic wrote: > > You cannot open the same UDP port twice (or, shall I say -- bound the same > UDP port to two different sockets). The story about N, N+1 port as well as > explanation indicated that this is an Erlang restriction due to the socket > library used is - well... pure fiction. > thanks for the clarification. I am not sure how the Erlang internals work. Can you explain what {reuseaddr,true} does then, if not bind the same UDP port to multiple sockets. There are LOTS of applications on a OSX that all bind to 224.0.0.251:5353, is there a semantic different I am missing. Thanks for taking the time to explain this. From tony@REDACTED Tue Nov 17 16:03:43 2009 From: tony@REDACTED (Tony Rogvall) Date: Tue, 17 Nov 2009 16:03:43 +0100 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: <5c493e530911161651w41e46aecu823bd4d36236815f@mail.gmail.com> References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> <5c493e530911161651w41e46aecu823bd4d36236815f@mail.gmail.com> Message-ID: Do not forget about {active, once} mode. {active,once} will receive one message (depends on buffer size etc) the it will switch to passive mode. To get the next message you use inet:setopts(Socket, [{active,once}]) to activate it again. This mode enables a selective receive at the same time as it enables flow control. /Tony On 17 nov 2009, at 01.51, Ngoc Dao wrote: >> From inet's doc: > http://www1.erlang.org/documentation/doc-4.9.1/lib/kernel-2.4.1/doc/html/inet.html > > If the active option is true, which is the default, everything > received from the socket will be sent as messages to the receiving > process. If the active option is set to false (passive mode), the > process must explicitly receive incoming data by calling > gen_tcp:recv/N or gen_udp:recv/N (depending on the type of socket). > Note: Passive mode provides flow control; the other side will not be > able send faster than the receiver can read. Active mode provides no > flow control; a fast sender could easily overflow the receiver with > incoming messages. Use active mode only if your high-level protocol > provides its own flow control (for instance, acknowledging received > messages) or the amount of data exchanged is small. > > > On Tue, Nov 17, 2009 at 2:59 AM, ERLANG wrote: >> Hi Chandru ! >> >> That's fix my problem. Thanks. >> While googling a bit, I found two ways to read from the Socket: >> >> recv(Socket, Bin) -> >> receive >> {tcp, Socket, B} -> >> io:format(".", []), >> recv(Socket, concat_binary([Bin, B])); >> {tcp_closed, Socket} -> >> {ok, Bin}; >> Other -> >> {error, {socket, Other}} >> after >> ?TIMEOUT -> >> {error, {socket, timeout}} >> end. >> >> % version 2 with "gen_tcp:recv" >> recv2(Socket, Bin) -> >> case gen_tcp:recv(Socket, 0, ?TIMEOUT) of >> {ok, B} -> >> io:format(".", []), >> recv(Socket, concat_binary([Bin, B])); >> {error, closed} -> >> {ok, Bin}; >> {error, timeout} -> >> {error, {socket, timeout}}; >> Other -> >> {error, {socket, Other}} >> end. >> >> >> Which one is the best in my case (see below: fetch.erl)? >> >> Regards >> Zabrane >> >> Le 16 nov. 09 ? 18:53, Chandru a ?crit : >> >>> You are expecting the server to indicate end of response by closing the >>> connection, but because you specify HTTP/1.1 in the request, the server is >>> holding up your connection, and you are timing out. Try replacing HTTP/1.1 >>> with HTTP/1.0 in your request, or parse the response to detect end of >>> response. >>> >>> cheers >>> Chandru >>> >>> 2009/11/16 zabrane Mikael >>> >>>> Hi List ! >>>> >>>> New to Erlang, I'm trying to implement a simple URL fetcher. >>>> Here's my code (please, feel free to correct it if you find any bug or >>>> know >>>> a better approach): >>>> >>>> >>>> >>>> 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8---- >>>> -module(fetch). >>>> >>>> -export([url/1]). >>>> >>>> -define(TIMEOUT, 7000). >>>> -define(TCP_OPTS, [binary, {packet, raw}, {nodelay, true}, >>>> {active, true}]). >>>> >>>> url(Url) -> >>>> {ok, _Tag, Host, Port} = split_url(Url), >>>> >>>> Hdrs = [], >>>> Request = ["GET ", Url, " HTTP/1.1\r\n", Hdrs, "\r\n\r\n"], >>>> >>>> case catch gen_tcp:connect(Host, Port, ?TCP_OPTS) of >>>> {'EXIT', Why} -> >>>> {error, {socket_exit, Why}}; >>>> {error, Why} -> >>>> {error, {socket_error, Why}}; >>>> {ok, Socket} -> >>>> gen_tcp:send(Socket, list_to_binary(Request)), >>>> recv(Socket, list_to_binary([])) >>>> end. >>>> >>>> recv(Socket, Bin) -> >>>> receive >>>> {tcp, Socket, B} -> >>>> io:format(".", []), >>>> recv(Socket, concat_binary([Bin, B])); >>>> {tcp_closed, Socket} -> >>>> {ok, Bin}; >>>> Other -> >>>> {error, {socket, Other}} >>>> after >>>> ?TIMEOUT -> >>>> {error, {socket, timeout}} >>>> end. >>>> >>>> >>>> split_url([$h,$t,$t,$p,$:,$/,$/|T]) -> split_url(http, T); >>>> split_url(_X) -> {error, split_url}. >>>> >>>> split_url(Tag, X) -> >>>> case string:chr(X, $:) of >>>> 0 -> >>>> Port = 80, >>>> case string:chr(X,$/) of >>>> 0 -> >>>> {ok, Tag, X, Port}; >>>> N -> >>>> Site = string:substr(X,1,N-1), >>>> {ok, Tag, Site, Port} >>>> end; >>>> N1 -> >>>> case string:chr(X,$/) of >>>> 0 -> >>>> error; >>>> N2 -> >>>> PortStr = string:substr(X,N1+1, N2-N1-1), >>>> case catch list_to_integer(PortStr) of >>>> {'EXIT', _} -> >>>> {error, port_number}; >>>> Port -> >>>> Site = string:substr(X,1,N1-1), >>>> {ok, Tag, Site, Port} >>>> end >>>> end >>>> end. >>>> >>>> >>>> >>>> 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8------ >>>> >>>> When testing it, the receiving socket gets very very slow: >>>> $ erl >>>> 1> c(fetch). >>>> 2> Bin = fetch:url("http://www.google.com"). >>>> ......{error,{socket,timeout}} >>>> >>>> Am I missing something? >>>> What I like to get at the end is a very fast fetcher. Any hint? >>>> >>>> Regards >>>> Zabrane >>>> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From navaneethanit@REDACTED Tue Nov 17 18:08:46 2009 From: navaneethanit@REDACTED (NavTux) Date: Tue, 17 Nov 2009 09:08:46 -0800 (PST) Subject: Erlang IDE Message-ID: Hi, I need good interactive IDE for erlang for both windows and linux ,which one is best to work with erlang? It ll be good one if the IDE functions like Netbeans that means If any error happens it shows the tip to rectify From roberto.aloi@REDACTED Tue Nov 17 18:21:07 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Tue, 17 Nov 2009 17:21:07 +0000 (GMT) Subject: [erlang-questions] Erlang IDE In-Reply-To: Message-ID: <8672048.43021258478467903.JavaMail.root@zimbra> Well, what I usually do is to use Emacs + flymake. You get red marking for errors and blue ones for warnings. You also get the reason of the warning/error. Roberto Aloi Erlang Training and Consulting Ltd. http://www.erlang-consulting.com ----- Original Message ----- From: "NavTux" To: erlang-questions@REDACTED Sent: Tuesday, 17 November, 2009 17:08:46 GMT +00:00 GMT Britain, Ireland, Portugal Subject: [erlang-questions] Erlang IDE Hi, I need good interactive IDE for erlang for both windows and linux ,which one is best to work with erlang? It ll be good one if the IDE functions like Netbeans that means If any error happens it shows the tip to rectify ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org -- Roberto Aloi roberto.aloi@REDACTED http://www.erlang-consulting.com From v@REDACTED Tue Nov 17 18:48:44 2009 From: v@REDACTED (Valentin Micic) Date: Tue, 17 Nov 2009 19:48:44 +0200 Subject: [erlang-questions] Multicast UDP sending question In-Reply-To: Message-ID: <20091117174859.4597F3D0D4E@mail.pharos-avantgard.com> Let me give you a standard explanation: {reuseaddr,true}is mapped to SO_REUSEADDR flag, which is only used when you want to fire up something even though the port is still in a TIME_WAIT state (if you do not have SO_REUSEADDR set, you wouldn't be able to do that). I cannot explain how OSX does things, but I can speculate: - One may plumb a number of logical interfaces to the same physical interface and bound the same UDP port number to a specific interface (but then, this is not really the same port!). Each application may use a different logical interface to subscribe to the same multicast address, hence receive the same packet. - There may be a "higher level" API for multicast handling at the kernel level. I remember that Mickeysoft Windoze had something similar, but never really cared enough to test it... Maybe it is a similar kind of "innovation" that Apple does so well. V/ -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Jarrod Roberson Sent: 17 November 2009 04:45 PM To: Valentin Micic Cc: Torben Hoffmann; Erlang Subject: Re: [erlang-questions] Multicast UDP sending question On Tue, Nov 17, 2009 at 9:15 AM, Valentin Micic wrote: > > You cannot open the same UDP port twice (or, shall I say -- bound the same > UDP port to two different sockets). The story about N, N+1 port as well as > explanation indicated that this is an Erlang restriction due to the socket > library used is - well... pure fiction. > thanks for the clarification. I am not sure how the Erlang internals work. Can you explain what {reuseaddr,true} does then, if not bind the same UDP port to multiple sockets. There are LOTS of applications on a OSX that all bind to 224.0.0.251:5353, is there a semantic different I am missing. Thanks for taking the time to explain this. From jabronson@REDACTED Tue Nov 17 18:52:31 2009 From: jabronson@REDACTED (Joshua Bronson) Date: Tue, 17 Nov 2009 12:52:31 -0500 Subject: [erlang-questions] tk requirement? In-Reply-To: References: <299cc2dd0911161248n753344b5wd95d39236b3855bf@mail.gmail.com> <299cc2dd0911161403u566da8b7k1fa1c382f97cf854@mail.gmail.com> Message-ID: <299cc2dd0911170952m15de8c26h999aecf00e50bcfc@mail.gmail.com> On Mon, Nov 16, 2009 at 6:32 PM, Paul Davis wrote: > On Mon, Nov 16, 2009 at 5:03 PM, Joshua Bronson > wrote: > > Hey Paul, > > On Mon, Nov 16, 2009 at 4:00 PM, Paul Davis > > > wrote: > >> > >> On Mon, Nov 16, 2009 at 3:48 PM, Joshua Bronson > >> wrote: > >> > Hey Erlang folks, > >> > > >> > I know relatively little about the Erlang distribution (my exposure is > >> > pretty much limited to installing it as a requirement of couchdb and > >> > rabbitmq-server). I was wondering if tk is (or has to be?) a hard > >> > requirement of the distribution or if (like e.g. Python) it's possible > >> > to build Erlang with a no_tk variant / USE flag / etc. in the case > >> > that you know you won't be needing to do any gui stuff. (This query is > >> > prompted by http://trac.macports.org/ticket/20006.) > >> > > >> > Thanks! > >> > > >> > Josh > >> > > >> > >> Josh, > >> > >> I've never had any issue with it using the system installed wx > >> libraries on 10.5 or 10.6. Reading through the ticket, the library you > >> want to check for linkage is probably > >> > >> > $PREFIX/lib/erlang/lib/wx-0.98.1/priv/i386-apple-darwin9.6.0/wxe_driver.so. > >> > >> For instance: > >> > >> $ otool -L > >> > /usr/local/lib/erlang/lib/wx-0.98.1/priv/i386-apple-darwin9.6.0/wxe_driver.so > >> > > > > hrm, i looked in /usr/local/lib/erlang/lib and there's a wx-0.98.3 > directory > > (must be the version the latest macports version (R13B02) ships with? > though > > on my Snow Leopard machine "/usr/bin/wx-config --version" gives 2.8.8) > but > > all it contains are the directories doc, ebin, include, and src; "find > > /opt/local/lib/erlang/lib/wx-0.98.3/ -name \*.so" gives no output. > > do i take it that by commenting out "port:wxWidgets" from my erlang > > Portfile's depends_build, I built erlang without wxwidgets? > >> > >> Granted, that's not the same as disabling the whole thing altogether. > > > > indeed! would still love to know if there's actual official support for > > disabling wx/tk :) > > > >> > >> HTH, > >> Paul Davis > > > > Thanks, > > Josh > > Josh, > > Curiosity got the better of me. It appears that there is'nt a specific > option in the autoconf scripts to disable building the wx code, but it > does handle the inability just fine. I tracked it down to > AC_CONFIG_SUBDIRS in $(top_srcdir)/lib/configure.in running all the > configure scripts in subdirectories. The configure in wx will detect > when it can't be built and exits with an error message. Unfortunately > the docs on AC_CONFIG_SUBDIRS aren't very particular on what happens > if a sub-configure exits with an error. Seeing as though it works, it > looks like the exit status is ignored and the library won't be built > (due to no Makefiles being built). > > If you want to prevent any attempt at building wx widgets at all, the > quickest thing would be to just delete $(top_srcdir)/lib/wx/configure > before running the top level configure script. The longer approach > would be to add a --with option for it's configure that just exits the > processing, but that'd require autoconfing before building. > > HTH, > Paul Davis > Thanks Paul, it sounds like MacPorts package can definitely have a no_tk variant, supported either with a patch deleting the $(top_srcdir)/lib/wx/configure file or by adding a --with option for it (MacPorts can assume autoconf is available). Of course upstream support is always preferable... Is this the wrong mailing list to be asking whether the Erlang developers see value in adding this configure option? From rapsey@REDACTED Tue Nov 17 18:55:27 2009 From: rapsey@REDACTED (Rapsey) Date: Tue, 17 Nov 2009 18:55:27 +0100 Subject: [erlang-questions] Erlang IDE In-Reply-To: <8672048.43021258478467903.JavaMail.root@zimbra> References: <8672048.43021258478467903.JavaMail.root@zimbra> Message-ID: <97619b170911170955s4f13e477q3200204fd4026925@mail.gmail.com> Are you sure you need it? Erlang is not java/c#. Most people are completely satisfied with Emacs or TextMate. Personally I don't see any advantage in using a big IDE for Erlang. Sergej On Tue, Nov 17, 2009 at 6:21 PM, Roberto Aloi < roberto.aloi@REDACTED> wrote: > Well, what I usually do is to use Emacs + flymake. > You get red marking for errors and blue ones for warnings. > You also get the reason of the warning/error. > > Roberto Aloi > Erlang Training and Consulting Ltd. > http://www.erlang-consulting.com > > > ----- Original Message ----- > From: "NavTux" > To: erlang-questions@REDACTED > Sent: Tuesday, 17 November, 2009 17:08:46 GMT +00:00 GMT Britain, Ireland, > Portugal > Subject: [erlang-questions] Erlang IDE > > Hi, > > > I need good interactive IDE for erlang for both windows and > linux ,which one is best to work with erlang? > It ll be good one if the IDE functions like Netbeans that means If any > error happens it shows the tip to rectify > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > -- > Roberto Aloi > roberto.aloi@REDACTED > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From navaneethanit@REDACTED Tue Nov 17 19:31:02 2009 From: navaneethanit@REDACTED (NavTux) Date: Wed, 18 Nov 2009 00:01:02 +0530 Subject: Erlang IDE Message-ID: Rapsey gmail.com> writes: >Are you sure you need it? Erlang is not java/c#. Ya i am sure > Most people are completely > satisfied with Emacs or TextMate. Is any big differences arises between using IDE and non-IDE persons actually,i don't know for this language IDE preferable or not? > Personally I don't see any advantage in > using a big IDE for Erlang. > good:-) > Sergej > -------- regards, Navaneethan http://navaspot.wordpress.com From tony@REDACTED Tue Nov 17 19:09:59 2009 From: tony@REDACTED (Tony Rogvall) Date: Tue, 17 Nov 2009 19:09:59 +0100 Subject: [erlang-questions] Multicast UDP sending question In-Reply-To: References: <20091117141526.6A1D03D0D4E@mail.pharos-avantgard.com> Message-ID: <382D6F7C-5850-4704-834E-79AF146907A7@rogvall.se> Try the reuse port option? pass it to inet:setopts or gen_udp:open. You have to craft it a bit your self. reuse_port() -> [{raw,?SOL_SOCKET,?SO_REUSEPORT,<<1:32/native>>}]. where -define(SOL_SOCKET, 16#ffff). and %% enables duplicate address and port bindings -define(SO_REUSEPORT, 16#0200). This is tested on mac os x 10.4/10.5/10.6 Maybe it's the same in linux? /Tony On 17 nov 2009, at 15.45, Jarrod Roberson wrote: > On Tue, Nov 17, 2009 at 9:15 AM, Valentin Micic wrote: > >> >> You cannot open the same UDP port twice (or, shall I say -- bound the same >> UDP port to two different sockets). The story about N, N+1 port as well as >> explanation indicated that this is an Erlang restriction due to the socket >> library used is - well... pure fiction. >> > > thanks for the clarification. I am not sure how the Erlang internals work. > Can you explain what {reuseaddr,true} does then, if not bind the same UDP > port to multiple sockets. > There are LOTS of applications on a OSX that all bind to 224.0.0.251:5353, > is there a semantic different I am missing. > Thanks for taking the time to explain this. From mpalmer@REDACTED Tue Nov 17 20:35:20 2009 From: mpalmer@REDACTED (Matthew Palmer) Date: Wed, 18 Nov 2009 06:35:20 +1100 Subject: Multicast UDP sending question In-Reply-To: <382D6F7C-5850-4704-834E-79AF146907A7@rogvall.se> References: <20091117141526.6A1D03D0D4E@mail.pharos-avantgard.com> <382D6F7C-5850-4704-834E-79AF146907A7@rogvall.se> Message-ID: <20091117193520.GX25450@hezmatt.org> On Tue, Nov 17, 2009 at 07:09:59PM +0100, Tony Rogvall wrote: > Try the reuse port option? pass it to inet:setopts or gen_udp:open. > > You have to craft it a bit your self. > > reuse_port() -> > [{raw,?SOL_SOCKET,?SO_REUSEPORT,<<1:32/native>>}]. > > > where > > -define(SOL_SOCKET, 16#ffff). > > and > > %% enables duplicate address and port bindings > -define(SO_REUSEPORT, 16#0200). > > This is tested on mac os x 10.4/10.5/10.6 > > Maybe it's the same in linux? My Debian lenny system has that constant commented out, sounds like it's a work in progress: [...] #define SO_LINGER 13 #define SO_BSDCOMPAT 14 /* To add :#define SO_REUSEPORT 15 */ #define SO_PASSCRED 16 #define SO_PEERCRED 17 [...] - Matt From dmitriid@REDACTED Tue Nov 17 20:35:17 2009 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Tue, 17 Nov 2009 21:35:17 +0200 Subject: [erlang-questions] Erlang IDE In-Reply-To: References: Message-ID: Erlide, http://erlide.sourceforge.net/ or NetBeans >= 6.7 + Erlang Plugin, http://sourceforge.net/projects/erlybird/files/nb-erlang/ Feature-wise Erlide seems to be more advanced > Hi, > > > I need good interactive IDE for erlang for both windows and > linux ,which one is best to work with erlang? > It ll be good one if the IDE functions like Netbeans that means If any > error happens it shows the tip to rectify > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From v@REDACTED Tue Nov 17 20:45:29 2009 From: v@REDACTED (Valentin Micic) Date: Tue, 17 Nov 2009 21:45:29 +0200 Subject: [erlang-questions] Multicast UDP sending question In-Reply-To: Message-ID: <20091117194544.302573D0D4E@mail.pharos-avantgard.com> From: Samuel Rivas [mailto:samuel.rivas@REDACTED] >You also need SO_REUSEADDR to bind listening sockets in the same UDP >port to receive multicast traffic. > >If, for example, you open a socket listening in 224.0.0.251:5353 >without that option, you wouldn't be able to bind a socket listening >in 224.0.0.252.5353 in the same computer. But... I do not agree with the above, this is why. The way to get multicast working in Unix using system calls follows the sequence below: 1) Create a socket: socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP ) 2) Bind the socket to a particular UDP port bind( int Socket, sockaddr* pBindInfo, sizeof( socktaddr_in) 3) add membership to a multicast addresses: int setsockopt(int Socket, int lvl, int optname, const void *optval, socklen_t optlen); Naming of the arguments may vary but option name (optname) that is used to add multicast address membership is almost invariably defined as IP_ADD_MEMBERSHIP. Option value (optval) in this case contains a structure appropriate for IP_ADD_MEMBERSHIP which contains the actual multicast address. One may add membership to any number of multicast addresses to the same socket, by simply repeating the step 3). Therefore it is possible to receive traffic for 224.0.0.251 and 224.0.0.252 both coming over port 5353. I've looked to see how reuse port works, and this is what I've found: >From Richard Stevens (rstevens@REDACTED): This is a newer flag that appeared in the 4.4BSD multicasting code (although that code was from elsewhere, so I am not sure just who invented the new SO_REUSEPORT flag). What this flag lets you do is rebind a port that is already in use, but only if all users of the port specify the flag. I believe the intent is for multicasting apps, since if you're running the same app on a host, all need to bind the same port. But the flag may have other uses. For example the following is from a post in February: From Stu Friedberg (stuartf@REDACTED): SO_REUSEPORT is also useful for eliminating the try-10-times-to-bind hack in ftpd's data connection setup routine. Without SO_REUSEPORT, only one ftpd thread can bind to TCP (lhost, lport, INADDR_ANY, 0) in preparation for connecting back to the client. Under conditions of heavy load, there are more threads colliding here than the try-10-times hack can accomodate. With SO_REUSEPORT, things work nicely and the hack becomes unnecessary. If I may put forward my 2c worth, this flag doesn't do anything useful but creates confusion -- if we all reuse the same UDP port, who receives the traffic in a case of unicast? V/ From caluml@REDACTED Tue Nov 17 20:57:43 2009 From: caluml@REDACTED (Calum) Date: Tue, 17 Nov 2009 19:57:43 +0000 Subject: Remote "access" points Message-ID: <635498b70911171157n34972e66x1d66e68cb0f76152@mail.gmail.com> Hello all, I am looking through the Erlang source for any remote access points. My plan is to try restricting what can be called on a node. I've found the ones for spawn - partly because I was looking in the right place, and partly because they're helpfully commented in net_kernel.erl as %% %% The spawn/4 BIF ends up here. %% handle_call({spawn,M,F,A,Gleader},{From,Tag},State) when is_pid(From) -> do_spawn([no_link,{From,Tag},M,F,A,Gleader],[],State); By changing that to handle_call({spawn,M,F,A,Gleader},{From,Tag},State) when is_pid(From), M =:= 'public' -> do_spawn([no_link,{From,Tag},M,F,A,Gleader],[],State); I get the desired effect. Trying to spawn('paranoid@REDACTED', os, cmd, ['ls']) fails on the calling node (although the "paranoid" node dies a noisy death which isn't ideal, but is fine for testing). Spawning a public:* function works. However, I'm not able to find the place where rpc:*call requests arrive. Can anyone point me at the right place? As well as any other points at which remote nodes can start new processes. Another thing - putting a debug io:format doesn't seem to be a good way to debug which functions are being hit - is there a better way? Someone also mentioned that when you obtained a Pid from a spawn, you can do other things with it? Could someone explain what you can do with it? C From g@REDACTED Tue Nov 17 21:01:09 2009 From: g@REDACTED (Garrett Smith) Date: Tue, 17 Nov 2009 14:01:09 -0600 Subject: [erlang-questions] Erlang IDE In-Reply-To: References: Message-ID: On Tue, Nov 17, 2009 at 11:08 AM, NavTux wrote: > ? ? ?I need good interactive IDE for erlang for both windows and > linux ,which one is best to work with erlang? > It ll be good one if the IDE functions like Netbeans that means If any > error happens it shows the tip to rectify Have you looked at erlIDE? http://erlide.sourceforge.net This is probably your best bet, though when I looked at it (a few months ago), it was rough around the edges. Just to give you an idea how limited the Erlang IDE story is, I switched from Vim to Emacs as my main source editor in order to take advantages of its Erlang support. IMO, if you're going to invest a *lot* of time in Erlang, you'll want to consider a switch as well. Unfortunately, that's a huge transition of you do your work in Netbeans. Nevertheless, Emacs is the path of least resistance for productive development in Erlang. Another point to bear in mind... most Erlang projects use automake tools for builds... not a happy place for Windows development. I've restored to run virtualized Ubuntu on my Windows machines to do any Erlang development. Again, path of least resistance. You can certainly use Windows, but you're going to hit bumps along the way. Garrett From kenneth.lundin@REDACTED Tue Nov 17 21:15:04 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 17 Nov 2009 21:15:04 +0100 Subject: [erlang-questions] Erlang IDE In-Reply-To: References: Message-ID: I recommend ErlIDE if you really want an IDE , it is an Erlang plugin for Eclipse. It will work the same on both Linux and Windows. The more users we get on ErlIDE the better it will get. We are supporting the development. If you are used to Emacs you don't need an IDE to be productive in Erlang. And there are IDE like Erlang modes for Emacs as well. But ErlIDE is steadily improving and has features that will attract even the Emacs fans. /Kenneth Erlang/OTP Ericsson On Tue, Nov 17, 2009 at 9:01 PM, Garrett Smith wrote: > On Tue, Nov 17, 2009 at 11:08 AM, NavTux wrote: >> ? ? ?I need good interactive IDE for erlang for both windows and >> linux ,which one is best to work with erlang? >> It ll be good one if the IDE functions like Netbeans that means If any >> error happens it shows the tip to rectify > > Have you looked at erlIDE? > > http://erlide.sourceforge.net > > This is probably your best bet, though when I looked at it (a few > months ago), it was rough around the edges. > > Just to give you an idea how limited the Erlang IDE story is, I > switched from Vim to Emacs as my main source editor in order to take > advantages of its Erlang support. IMO, if you're going to invest a > *lot* of time in Erlang, you'll want to consider a switch as well. > Unfortunately, that's a huge transition of you do your work in > Netbeans. Nevertheless, Emacs is the path of least resistance for > productive development in Erlang. > > Another point to bear in mind... most Erlang projects use automake > tools for builds... not a happy place for Windows development. I've > restored to run virtualized Ubuntu on my Windows machines to do any > Erlang development. Again, path of least resistance. You can certainly > use Windows, but you're going to hit bumps along the way. > > Garrett > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From jarrod@REDACTED Tue Nov 17 21:42:23 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Tue, 17 Nov 2009 15:42:23 -0500 Subject: [erlang-questions] Erlang IDE In-Reply-To: References: Message-ID: On Tue, Nov 17, 2009 at 3:15 PM, Kenneth Lundin wrote: > I recommend ErlIDE if you really want an IDE , it is an Erlang plugin > for Eclipse. It will work > the same on both Linux and Windows. > > The more users we get on ErlIDE the better it will get. We are > supporting the development. > > If you are used to Emacs you don't need an IDE to be productive in > Erlang. And there are IDE like > Erlang modes for Emacs as well. > > But ErlIDE is steadily improving and has features that will attract > even the Emacs fans. > > /Kenneth Erlang/OTP Ericsson > > I second this opinion, I abandoned the NetBeans plugin and use ErlIDE. It does nice code completion, has nice error highlighting and auto compiles on save. you just need to keep a shell window open to your ebin dir and don't have to constantly compile everything. speeds up development a lot. From g@REDACTED Tue Nov 17 23:03:44 2009 From: g@REDACTED (Garrett Smith) Date: Tue, 17 Nov 2009 16:03:44 -0600 Subject: [erlang-questions] Erlang IDE In-Reply-To: References: Message-ID: On Tue, Nov 17, 2009 at 2:15 PM, Kenneth Lundin wrote: > But ErlIDE is steadily improving and has features that will attract > even the Emacs fans. That's great! The established Erlang toolset is excellent IMO but it's going to feel like a big step backward (justifiably or not) for Eclipse users. Garrett From wde@REDACTED Tue Nov 17 19:54:20 2009 From: wde@REDACTED (wde) Date: Tue, 17 Nov 2009 19:54:20 +0100 Subject: [erlang-questions] Re: Erlang IDE Message-ID: <20091117185419.7667C2000098@mwinf2111.orange.fr> I used ErlIDE with Eclipse and I found it very useful http://erlide.sourceforge.net/ ======= le 17/11/2009, 20:01:02 vous ?criviez: ======= >Rapsey gmail.com> writes: > >>Are you sure you need it? Erlang is not java/c#. > > Ya i am sure > >> Most people are completely >> satisfied with Emacs or TextMate. > > Is any big differences arises between using IDE and non-IDE persons >actually,i don't know for this language IDE preferable or not? > >> Personally I don't see any advantage in >> using a big IDE for Erlang. >> > > good:-) > >> Sergej >> > >-------- >regards, > >Navaneethan > >http://navaspot.wordpress.com > >________________________________________________________________ >erlang-questions mailing list. See http://www.erlang.org/faq.html >erlang-questions (at) erlang.org > > = = = = = = = = = ========= = = = = = = = = = = wde wde@REDACTED 17/11/2009 From wallentin.dahlberg@REDACTED Tue Nov 17 23:21:54 2009 From: wallentin.dahlberg@REDACTED (Wallentin Dahlberg) Date: Tue, 17 Nov 2009 23:21:54 +0100 Subject: [erlang-questions] Re: Erlang IDE In-Reply-To: <20091117185419.7667C2000098@mwinf2111.orange.fr> References: <20091117185419.7667C2000098@mwinf2111.orange.fr> Message-ID: I am tiyng down my hands so I don't start a flamewar (since I am Vim-user and utterly hate emacs =) . ErlIDE is a really good choice and I really recommend it. Try it! Regards, Bj?rn-Egil 2009/11/17 wde > I used ErlIDE with Eclipse and I found it very useful > http://erlide.sourceforge.net/ > > > > > > > ======= le 17/11/2009, 20:01:02 vous ?criviez: ======= > > >Rapsey gmail.com> writes: > > > >>Are you sure you need it? Erlang is not java/c#. > > > > Ya i am sure > > > >> Most people are completely > >> satisfied with Emacs or TextMate. > > > > Is any big differences arises between using IDE and non-IDE persons > >actually,i don't know for this language IDE preferable or not? > > > >> Personally I don't see any advantage in > >> using a big IDE for Erlang. > >> > > > > good:-) > > > >> Sergej > >> > > > >-------- > >regards, > > > >Navaneethan > > > >http://navaspot.wordpress.com > > > >________________________________________________________________ > >erlang-questions mailing list. See http://www.erlang.org/faq.html > >erlang-questions (at) erlang.org > > > > > > = = = = = = = = = ========= = = = = = = = = = = > > wde > wde@REDACTED > 17/11/2009 > > From ok@REDACTED Wed Nov 18 00:50:21 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 18 Nov 2009 12:50:21 +1300 Subject: [erlang-questions] Why using pmap is not giving me the desired speedup ? In-Reply-To: <6b1b97d20911162245p61445058x3aaaf4048b628368@mail.gmail.com> References: <6b1b97d20911162245p61445058x3aaaf4048b628368@mail.gmail.com> Message-ID: <8C9A9661-1854-4250-8EBE-7773832F92D6@cs.otago.ac.nz> On Nov 17, 2009, at 7:45 PM, Tushar Deshpande wrote: > I imagined that I would get some observable > speedup with parallel quicksort. > > Contrary to my expectations, parallel version > took more time to run. > > What could be the reason ? Why is parallel quicksort effective on shared memory multiprocessors? Because there is an O(n) in-place partitioning, following by sparking two activities, which need to be told only "where is the first element of my sublist" and "how many elements" (2 words), and which need no synchronisation between them, the only synchronisation being for the parent to wait for its two children. Indeed, we only need one child. psort(a, n) = if n is small then use insertion sort else choose pivot x partition (a,n) into (a,L), (a+L,E), (a+L+E,G) where L is the number of elements < x, E the number of elements = x, G the number > x. child = fork psort(a+L+E,G) psort(a,L) wait for child end if Tushar Deshpande's parallel quicksort, thanks to its use of pmap, creates two children, when it only needs to create one. Not only that, the left and right sublists have to be *sent* to each child, that's O(n) copying, the sorted versions have to be *received* from each child, that's O(n) copying again, and then they have to be concatenated. The sequential version has only to concatenate. It would be interesting to try pqsortB(L) -> pqsortB(L, length(L)). pqsortB(L, N) when N < 10 -> % tunable threshold lists:sort(L); pqsortB([Pivot|L], _) -> ppartitionB(L, Pivot, [], 0, [Pivot], [], 0). ppartitionB([], _, Lss, Nlss, Eql, Gtr, Ngtr) -> Self = self(), Pid = spawn(fun () -> Self!{self(),pqsortB(Gtr, Ngtr)} end), Left = pqsortB(Lss, Nlss), receive {Pid,Right} -> Left++(Eql++Right) end; ppartitionB([X|Xs], Pivot, Lss, Nlss, Eql, Gtr, Ngtr) -> if X < Pivot -> ppartitionB(Xs, Pivot, [X|Lss], Nlss+1, Eql, Gtr, Ngtr) ; X > Pivot -> ppartitionB(Xs, Pivot, Lss, Nlss, Eql, [X|Gtr], Ngtr+1) ; true -> ppartitionB(Xs, Pivot, Lss, Nlss, [X|Eql], Gtr, Ngtr) end. In fact it seemed interesting enough that I *did* try it. I measured the time to sort a list of 10,000 uniformly generated random floats, on a machine with a 500MHz CPU and 84 MHz memory. All times in this message are the smallest of 5 runs. 120 msec Tushar Deshpande's qsort 110 msec Sequential qsort with single partition pass as above and similarly switching over to lists:qsort for small N. 110 msec Same as above but not bothering with counts, checking for large -vs- small by [Pivot|L = [_,_,_|_]] or not and sorting 0 to 3 elements inline. 80 msec Same as previous, but qsort(L, T) == qsort(L)++T, to try to reduce the amount of concatenation. 930 msec Tushar Deshpande's pqsort (with a simplified version of pmap/2 that doesn't bother with error handling) 230 msec The pqsortB/1 function above. With Tushar Deshpande's pqsort/1 being 7.75 times slower than qsort/1, it's going to take a lot of cores just to catch up. My improved pqsortB is 4 times faster than his on a single core. I tried again with a 2.2GHz CPU, 667 MHz memory, 2-core machine. This time I needed 100,000 element lists to get useful times. 170 msec original qsort 120 msec 110 msec 100 msec best qsort from above -smp disable -smp enable 830 msec 1720 msec original pqsort 240 msec 310 msec improved pqsortB 110 msec 110 msec partition once, sort sublists using one child, combine. As a double check, with -smp disable, the emulator reported [rq:1] in the banner; with -smp enable, [smp:2:2] [rq:2]. Here's what I *suspect* is going on. The time ratios for the sequential versions are very close to the speed ratios of the main memories on the two machines. It looks as though memory is the main bottleneck. Now my laptop may have two cores, but they share one main memory. (And I think they share an L2 cache.) If the time of a computation is dominated by memory, dividing the work across several CPUs isn't going to help: you need to divide it across several memories. Another key point is that message passing is *copying*. It seems clear from the "-smp enable" column above that "the fewer the processes the better", which in this case actually means "the less COPYING of data between processes the better". -smp disable -smp enable #processes #cells copied 830 msec 1720 msec 32766 2800000 240 msec 310 msec 16383 1406112 110 msec 110 msec 1 100000 Here a "cell" means a list cell whose head is a float copied in a message from one process to another. #processes means the number of *additional* (temporary) processes. We have one mechanism in Erlang for both modelling concurrent systems and obtaining parallel speedups: logically separate processes communicating by message passing. In the former case, you create (at least) one Erlang process per logical activity, NOT for speed but for CLARITY, so that each logical activity can be modelled separately from the others. The deciding factor in the number of processes you create is the number of things to be modelled. When you are trying to get parallelism, there is seldom any point in having more than a small number of times as many processes as you have available cores. If you have lots of processes all wanting to be active at the same time, the costs of scheduling and message passing are likely to be high relative to the useful work. The best parallel quicksort above just used two processes (one the original, the other a temporary child). On a quad core machine, four processes would make sense. From xma@REDACTED Wed Nov 18 00:59:35 2009 From: xma@REDACTED (Xavier Maillard) Date: Wed, 18 Nov 2009 00:59:35 +0100 Subject: [erlang-questions] Erlang IDE In-Reply-To: <8672048.43021258478467903.JavaMail.root@zimbra> (message from Roberto Aloi on Tue, 17 Nov 2009 17:21:07 +0000 (GMT)) References: <8672048.43021258478467903.JavaMail.root@zimbra> Message-ID: <200911172359.nAHNxZCw008142@fed.local> Well, what I usually do is to use Emacs + flymake. You get red marking for errors and blue ones for warnings. You also get the reason of the warning/error. Can you share your settings ? Xavier -- http://www.gnu.org http://www.april.org http://www.lolica.org From steven.charles.davis@REDACTED Wed Nov 18 01:44:54 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 17 Nov 2009 16:44:54 -0800 (PST) Subject: Erlang IDE In-Reply-To: <200911172359.nAHNxZCw008142@fed.local> References: <8672048.43021258478467903.JavaMail.root@zimbra> <200911172359.nAHNxZCw008142@fed.local> Message-ID: It's vim and (when on windows) scite for me :) ...as has been stated -- I don't really feel a great need for "more". /s From adrianob@REDACTED Wed Nov 18 02:07:27 2009 From: adrianob@REDACTED (Adriano Bonat) Date: Tue, 17 Nov 2009 23:07:27 -0200 Subject: [erlang-questions] Erlang IDE In-Reply-To: <200911172359.nAHNxZCw008142@fed.local> References: <8672048.43021258478467903.JavaMail.root@zimbra> <200911172359.nAHNxZCw008142@fed.local> Message-ID: <6bb4c8cb0911171707x4e341699ofc49f96e674b0ab@mail.gmail.com> Some weeks ago I did give a try to Emacs when programming in Erlang, and I really liked it! :) I suggest that you take a look in Kevin Smith's blog, there is a screencast which explain in details how to setup flymake and Erlang's support to Emacs: http://weblog.hypotheticalabs.com/?page_id=318 On Tue, Nov 17, 2009 at 9:59 PM, Xavier Maillard wrote: > > ? Well, what I usually do is to use Emacs + flymake. > ? You get red marking for errors and blue ones for warnings. > ? You also get the reason of the warning/error. > > Can you share your settings ? > > ? ? ? ?Xavier > -- > http://www.gnu.org > http://www.april.org > http://www.lolica.org > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From seancribbs@REDACTED Wed Nov 18 03:26:43 2009 From: seancribbs@REDACTED (Sean Cribbs) Date: Tue, 17 Nov 2009 21:26:43 -0500 Subject: [erlang-questions] Erlang IDE In-Reply-To: <6bb4c8cb0911171707x4e341699ofc49f96e674b0ab@mail.gmail.com> References: <8672048.43021258478467903.JavaMail.root@zimbra> <200911172359.nAHNxZCw008142@fed.local> <6bb4c8cb0911171707x4e341699ofc49f96e674b0ab@mail.gmail.com> Message-ID: <4B035B63.6020209@gmail.com> I started with Kevin's setup (mostly because he coached me on how to use emacs), but switched to the emacs-starter-kit and customized it when I got more proficient. The original is here: http://github.com/technomancy/emacs-starter-kit You can see my fork here: http://github.com/seancribbs/emacs-starter-kit Sean Adriano Bonat wrote: > Some weeks ago I did give a try to Emacs when programming in Erlang, > and I really liked it! :) > > I suggest that you take a look in Kevin Smith's blog, there is a > screencast which explain in details how to setup flymake and Erlang's > support to Emacs: > > http://weblog.hypotheticalabs.com/?page_id=318 > > On Tue, Nov 17, 2009 at 9:59 PM, Xavier Maillard wrote: > >> Well, what I usually do is to use Emacs + flymake. >> You get red marking for errors and blue ones for warnings. >> You also get the reason of the warning/error. >> >> Can you share your settings ? >> >> Xavier >> -- >> http://www.gnu.org >> http://www.april.org >> http://www.lolica.org >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From tushar.erlang@REDACTED Wed Nov 18 06:18:31 2009 From: tushar.erlang@REDACTED (Tushar Deshpande) Date: Wed, 18 Nov 2009 00:18:31 -0500 Subject: [erlang-questions] Why using pmap is not giving me the desired speedup ? In-Reply-To: <8C9A9661-1854-4250-8EBE-7773832F92D6@cs.otago.ac.nz> References: <6b1b97d20911162245p61445058x3aaaf4048b628368@mail.gmail.com> <8C9A9661-1854-4250-8EBE-7773832F92D6@cs.otago.ac.nz> Message-ID: <6c20c4130911172118w3a43cf07v7d07cf1cf07f9aab@mail.gmail.com> Hi Richard and Angel, I appreciate your help a lot. I must admit that I naively assumed concurrency and parallelism to be same. I was underthe impression that concurrent code would give me speedup. Clearly, this is not the case. I think that precisely for this reason, using MapReduce makes a lot of sense. I wrote a distributed sort using Joe Armstrong's mapreduce implementation. I limited the number of spawn tasks to the number of cores. And, guess what, I got desired speedup. Distribted sort performed almost 20 times faster for an array with 100000 integers. I'm a grad student from State University of New York at Stony Brook and I'm studying Computer Architeture. I'm quite sure that I would learn a lot about multicore machines by writing some actual programs :) Warm Ragards, Tushar Deshpande On Tue, Nov 17, 2009 at 6:50 PM, Richard O'Keefe wrote: > > On Nov 17, 2009, at 7:45 PM, Tushar Deshpande wrote: > >> I imagined that I would get some observable >> speedup with parallel quicksort. >> >> Contrary to my expectations, parallel version >> took more time to run. >> > > >> What could be the reason ? >> > > Why is parallel quicksort effective on shared memory multiprocessors? > Because there is an O(n) in-place partitioning, following by sparking > two activities, which need to be told only "where is the first element > of my sublist" and "how many elements" (2 words), and which need no > synchronisation between them, the only synchronisation being for the > parent to wait for its two children. Indeed, we only need one child. > > psort(a, n) = > if n is small then > use insertion sort > else > choose pivot x > partition (a,n) into (a,L), (a+L,E), (a+L+E,G) > where L is the number of elements < x, E the > number of elements = x, G the number > x. > child = fork psort(a+L+E,G) > psort(a,L) > wait for child > end if > > Tushar Deshpande's parallel quicksort, thanks to its use of > pmap, creates two children, when it only needs to create one. > > Not only that, the left and right sublists have to be *sent* > to each child, that's O(n) copying, the sorted versions have > to be *received* from each child, that's O(n) copying again, > and then they have to be concatenated. The sequential version > has only to concatenate. > > It would be interesting to try > > pqsortB(L) -> > pqsortB(L, length(L)). > > pqsortB(L, N) when N < 10 -> % tunable threshold > lists:sort(L); > pqsortB([Pivot|L], _) -> > ppartitionB(L, Pivot, [], 0, [Pivot], [], 0). > > ppartitionB([], _, Lss, Nlss, Eql, Gtr, Ngtr) -> > Self = self(), > Pid = spawn(fun () -> Self!{self(),pqsortB(Gtr, Ngtr)} end), > Left = pqsortB(Lss, Nlss), > receive {Pid,Right} -> Left++(Eql++Right) end; > ppartitionB([X|Xs], Pivot, Lss, Nlss, Eql, Gtr, Ngtr) -> > if X < Pivot -> ppartitionB(Xs, Pivot, [X|Lss], Nlss+1, Eql, Gtr, Ngtr) > ; X > Pivot -> ppartitionB(Xs, Pivot, Lss, Nlss, Eql, [X|Gtr], Ngtr+1) > ; true -> ppartitionB(Xs, Pivot, Lss, Nlss, [X|Eql], Gtr, Ngtr) > end. > > In fact it seemed interesting enough that I *did* try it. > I measured the time to sort a list of 10,000 uniformly generated > random floats, on a machine with a 500MHz CPU and 84 MHz memory. > All times in this message are the smallest of 5 runs. > > 120 msec Tushar Deshpande's qsort > 110 msec Sequential qsort with single partition pass as above > and similarly switching over to lists:qsort for small N. > 110 msec Same as above but not bothering with counts, checking > for large -vs- small by [Pivot|L = [_,_,_|_]] or not > and sorting 0 to 3 elements inline. > 80 msec Same as previous, but qsort(L, T) == qsort(L)++T, to > try to reduce the amount of concatenation. > > 930 msec Tushar Deshpande's pqsort (with a simplified version of > pmap/2 that doesn't bother with error handling) > 230 msec The pqsortB/1 function above. > > With Tushar Deshpande's pqsort/1 being 7.75 times slower than qsort/1, > it's going to take a lot of cores just to catch up. My improved > pqsortB is 4 times faster than his on a single core. > > I tried again with a 2.2GHz CPU, 667 MHz memory, 2-core machine. > This time I needed 100,000 element lists to get useful times. > > 170 msec original qsort > 120 msec > 110 msec > 100 msec best qsort from above > > -smp disable -smp enable > 830 msec 1720 msec original pqsort > 240 msec 310 msec improved pqsortB > 110 msec 110 msec partition once, sort sublists > using one child, combine. > > As a double check, with -smp disable, the emulator reported > [rq:1] in the banner; with -smp enable, [smp:2:2] [rq:2]. > > Here's what I *suspect* is going on. The time ratios for > the sequential versions are very close to the speed ratios > of the main memories on the two machines. It looks as > though memory is the main bottleneck. > > Now my laptop may have two cores, but they share one main > memory. (And I think they share an L2 cache.) If the time > of a computation is dominated by memory, dividing the work > across several CPUs isn't going to help: you need to divide > it across several memories. > > Another key point is that message passing is *copying*. > It seems clear from the "-smp enable" column above that > "the fewer the processes the better", which in this case > actually means "the less COPYING of data between processes > the better". > > -smp disable -smp enable #processes #cells copied > 830 msec 1720 msec 32766 2800000 > 240 msec 310 msec 16383 1406112 > 110 msec 110 msec 1 100000 > > Here a "cell" means a list cell whose head is a float copied > in a message from one process to another. #processes means > the number of *additional* (temporary) processes. > > We have one mechanism in Erlang for both modelling concurrent > systems and obtaining parallel speedups: logically separate > processes communicating by message passing. In the former > case, you create (at least) one Erlang process per logical > activity, NOT for speed but for CLARITY, so that each logical > activity can be modelled separately from the others. The > deciding factor in the number of processes you create is the > number of things to be modelled. > > When you are trying to get parallelism, there is seldom any > point in having more than a small number of times as many > processes as you have available cores. If you have lots of > processes all wanting to be active at the same time, the > costs of scheduling and message passing are likely to be high > relative to the useful work. The best parallel quicksort > above just used two processes (one the original, the other > a temporary child). On a quad core machine, four processes > would make sense. > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From max.lapshin@REDACTED Wed Nov 18 06:23:08 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 18 Nov 2009 08:23:08 +0300 Subject: [erlang-questions] Why using pmap is not giving me the desired speedup ? In-Reply-To: <6c20c4130911172118w3a43cf07v7d07cf1cf07f9aab@mail.gmail.com> References: <6b1b97d20911162245p61445058x3aaaf4048b628368@mail.gmail.com> <8C9A9661-1854-4250-8EBE-7773832F92D6@cs.otago.ac.nz> <6c20c4130911172118w3a43cf07v7d07cf1cf07f9aab@mail.gmail.com> Message-ID: On Wed, Nov 18, 2009 at 8:18 AM, Tushar Deshpande wrote: > I'm a grad student from State University of New York > at Stony Brook and I'm studying Computer Architeture. > I'm quite sure that I would learn a lot about multicore > machines by writing some actual programs :) > Warm Ragards, > Tushar Deshpande Perhaps You can gain access to some cluster in your university and try your program on 30-100 computers. This would be interesting results for your education and for all of us =) From ulf.wiger@REDACTED Wed Nov 18 09:45:05 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 18 Nov 2009 09:45:05 +0100 Subject: [erlang-questions] Impact of state size in mnesia_frag_hash In-Reply-To: References: Message-ID: <4B03B411.3010806@erlang-consulting.com> Igor Ribeiro Sucupira wrote: > Hello. > > I've begun the implementation of a module following the > mnesia_frag_hash behaviour. > With enough fragments, my hash_state could grow to a reasonable size > (something around, say, 10 MB). Great! I have downloaded your code and will take a look at it when time allows (no promises as to when that might be...) This ought to come in very handy for fragmented tables where each fragment has considerably more than 2 GB of data :) (one example I have in mind is a sharding layer on top of Tokyo Tyrant; in which case it might also be interesting to add a Lua "BIF" to the Tyrant table that could calculate the new hash on each object in place and only return the objects that actually need to be moved to another bucket.) > Can you think of any significant performance impact of this > implementation? For example: is the hash_state transmitted between > nodes frequently enough to cause significant network traffic? It's not so much that which will be a problem. The frag_hash state will only be replicated when it's updated, which happens when e.g. changing the number of fragments. The bigger problem is that the frag_hash record is stored in ets, and retrieved for each access to the fragmented table (i.e. every read and every write). The ets:lookup() will cause the 10 MB structure to be copied to the heap of the calling process - once for every access! Another approach might be to create a fully replicated ordered_set disc_copy table and store the keys there. To speed things up, you can do raw ets accesses to this table, but you will have to ensure then that even extra db nodes (that don't have to be defined in the schema) are given a copy of the table. Otherwise, you have to use at least dirty reads to the table. This will provide distribution transparently, but carry a bit more overhead than raw ets operations. A reasonable compromise might be to check once if the table exists in ram (e.g using ets:table_info()), then choosing whether to use async_dirty or ets in a call to mnesia:activity/2. ...except this doesn't seem to work as well as one would hope. It seems that if you call mnesia:activity/2 inside a transaction, it will create a new transaction store and copy into it all objects updated in the outer transaction - even if the type of the nested transaction is dirty or ets! (in which case the copying will be entirely useless, as all the operations in the inner transaction will be mapped to dirty or ets counterparts that have no knowledge of transaction stores.) I can understand why this case isn't optimized, as normally you deserve a good spanking if you willfully run dirty transactions inside non-dirty ones. But in your case, I think it might have been a reasonable approach. You might be left with having to implement your own gb_sets equivalents based on ets, one using ets, and one using dirty_reads, and then pick the most suitable one depending on whether the keys table is local or not. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From erlang@REDACTED Wed Nov 18 09:56:55 2009 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 18 Nov 2009 09:56:55 +0100 Subject: programming tip: a note on encapsulation Message-ID: <9b08084c0911180056l53ca0695r781656018d6f76a9@mail.gmail.com> Here's a little programming technique I discovered yesterday, which I rather like. This is a method of hiding information using a higher order function, which is applicable to the callback style of programming that is used in things like gen_server. The basic idea is to replace code like this: foo({tag1, P, Q}, Var1, Var2, Var3) -> ... XX = ... Var1 ... foo({tag2, _, _}, Var1, Var2, Var3) -> ... with code like this: foo({tag1, P, Q}, F) -> ... XX = ... F(var1) ... In this code the first argument to foo is a pattern which must be matched, but arguments 2,3,4 etc. are just variables that are not used for pattern matching. They just define a global environment. Our problem is to represent this environment in a convenient manner. The code I was writing was for a little web server that I needed. The semantics of the web-server were provided by plug-in modules, I would say something like: web_server:start(myMod) This would start a web server whose semantics were determined by the plug-in module myMod. myMod had to export number of pre-defined functions, in particular there was a routine handle/4. handle had 4 arguments handle(Uri, Args, Pid, Root) So when I tried to fetch http://localhost:3000/some/page?a=1&b=2 I'd call: handle("/some/page", [{"a","1"},{"b","2"}], Pid, Root) Pid was the Pid of a database handler, and Root the root of a file system where the web server was installed. To access the data base I'd call lib_db:read(Pid, Key) and lib_db:write(Pid, key, Value). So a typical callback clause might look like this: handle("/counter", [{"name",N}], Pid, Root) -> Val = lib_db:rpc(Pid, {get, {name, N}}), TemplateFile = Root ++ "/abc.template", Result = expand_template(TemplateFile, [{"counter", Val}]), {ok, html, result}. So far so good - the problem with this is that all callback routines need the extra two arguments (Pid and Root) whether of not they use them. The code in the server that called the handler looked like this: Mod:handle(Uri, Args, Pid, Root) By using a higher order function we can make a much neater interface. We change the server code so that it reads: F = make_interface(Pid, Root), Mod:handle(Uri, Args, F), ... Where make_interface(Pid, Root) -> fun(root) -> Root; (X) -> lib_db:rpc(Pid, X) end. Now we can change the client callback code to: handle("/counter", [{"name",N}], F) -> Val = F({get, {name, N}}), TemplateFile = F(root) ++ "/abc.template", Result = expand_template(TemplateFile, [{"counter", Val}]), {ok, html, result}. This *reduces* the complexity of handle. The Pid and Root arguments vanish and the dependency to the module lib_db is eliminated. Any number of additional arguments to handle can be hidden inside F. From als@REDACTED Wed Nov 18 10:15:47 2009 From: als@REDACTED (Anthony Shipman) Date: Wed, 18 Nov 2009 20:15:47 +1100 Subject: [erlang-questions] programming tip: a note on encapsulation In-Reply-To: <9b08084c0911180056l53ca0695r781656018d6f76a9@mail.gmail.com> References: <9b08084c0911180056l53ca0695r781656018d6f76a9@mail.gmail.com> Message-ID: <200911182015.47855.als@iinet.net.au> On Wed, 18 Nov 2009 07:56:55 pm Joe Armstrong wrote: > We change the server code so that it reads: > > ? ? F = make_interface(Pid, Root), > ? ? Mod:handle(Uri, Args, F), > ? ? ... > > Where > > make_interface(Pid, Root) -> > ? ?fun(root) -> Root; > ? ? ? (X) ? ?-> lib_db:rpc(Pid, X) > ? ?end. That's a common pattern used with foldl et al e.g folder(A, B) -> fun(Arg, Result) -> Arg + Result + A + B end lists:foldl(folder(1, 2), 0, [2,4]) -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From leap@REDACTED Wed Nov 18 11:10:50 2009 From: leap@REDACTED (Michael Turner) Date: Wed, 18 Nov 2009 10:10:50 +0000 Subject: No subject In-Reply-To: <8C9A9661-1854-4250-8EBE-7773832F92D6@cs.otago.ac.nz> Message-ID: <8bzXghKw.1258539050.1327420.leap@gol.com> Some years ago, I attended a talk by Doug Clark about upcall implementations of IP. He stressed that the most important numbers you should know, when going into optimization of protocols at that level, were context-switch times and memory-copy times. Brutally basic operations, but both so important and frequent that they enjoy some degree of instruction-set-level support in any decent architecture. In some ways, context-switch costs are dominated by memory-copy times -- saving current state in memory, restoring interrupted state from memory -- so in a way, it almost comes down to the cost of copying. Quicksort is great, if you can do it in-place. But if you have to do O(n log n) copies of the data, well .... The discussion here reminds me that issues like these are still important in optimization. Which means you should always be revisiting the question: how important is optimization? The last time I had any serious issue involving quicksort, it was because a junior software engineer, not realizing there was a UNIX library function qsort(3), had translated some Pascal textbook quicksort to C. Not being very expert in C, he'd written unportable code, which segfaulted on a new platform. Upon investigating the issue, I discovered two things: (1) I could replace his two-dozen-lines-or-so lines of quicksort implementation/binary-search-lookup with just two lines, using library calls, BUT -- (2) the array he was sorting wasn't worth sorting anyway, because in real-world situations, nobody expected the list to get more than four or five items deep. Linear search of an unsorted array would have been faster than binary search on any sorted one, even without an unrolled loop or sentinel. He'd actually slowed down the system (slightly), he had taken time to implement this "pessimization" (expensive), and he had made systems crash (more expensive still), which required that somebody (me) debug his code (which was even more expensive than his original reimplementation of quicksort, since he was a junior employee and I was a contractor working at a higher hourly rate.) I really wish I was making this story up, but it did happen. Premature optimization truly is the root of all evil. Don't get me wrong, I love that Erlang two-liner version of quicksort. But it would be a service to the world if, in future editions of books and documents mentioning it, there was also a footnote pointing out that people should use quicksort as an optimization, only if necessary, and that Erlang will tend to execute this particular quicksort rather suboptimally. The cool thing about the two-liner is that it's a surprisingly clear expression of sorting, not that it's a good implementation in any other respect. -michael turner On 11/17/2009, "Richard O'Keefe" wrote: > >On Nov 17, 2009, at 7:45 PM, Tushar Deshpande wrote: >> I imagined that I would get some observable >> speedup with parallel quicksort. >> >> Contrary to my expectations, parallel version >> took more time to run. > >> >> What could be the reason ? > >Why is parallel quicksort effective on shared memory multiprocessors? >Because there is an O(n) in-place partitioning, following by sparking >two activities, which need to be told only "where is the first element >of my sublist" and "how many elements" (2 words), and which need no >synchronisation between them, the only synchronisation being for the >parent to wait for its two children. Indeed, we only need one child. > > psort(a, n) = > if n is small then > use insertion sort > else > choose pivot x > partition (a,n) into (a,L), (a+L,E), (a+L+E,G) > where L is the number of elements < x, E the > number of elements = x, G the number > x. > child = fork psort(a+L+E,G) > psort(a,L) > wait for child > end if > >Tushar Deshpande's parallel quicksort, thanks to its use of >pmap, creates two children, when it only needs to create one. > >Not only that, the left and right sublists have to be *sent* >to each child, that's O(n) copying, the sorted versions have >to be *received* from each child, that's O(n) copying again, >and then they have to be concatenated. The sequential version >has only to concatenate. > >It would be interesting to try > >pqsortB(L) -> > pqsortB(L, length(L)). > >pqsortB(L, N) when N < 10 -> % tunable threshold > lists:sort(L); >pqsortB([Pivot|L], _) -> > ppartitionB(L, Pivot, [], 0, [Pivot], [], 0). > >ppartitionB([], _, Lss, Nlss, Eql, Gtr, Ngtr) -> > Self = self(), > Pid = spawn(fun () -> Self!{self(),pqsortB(Gtr, Ngtr)} end), > Left = pqsortB(Lss, Nlss), > receive {Pid,Right} -> Left++(Eql++Right) end; >ppartitionB([X|Xs], Pivot, Lss, Nlss, Eql, Gtr, Ngtr) -> > if X < Pivot -> ppartitionB(Xs, Pivot, [X|Lss], Nlss+1, Eql, Gtr, >Ngtr) > ; X > Pivot -> ppartitionB(Xs, Pivot, Lss, Nlss, Eql, [X|Gtr], >Ngtr+1) > ; true -> ppartitionB(Xs, Pivot, Lss, Nlss, [X|Eql], Gtr, Ngtr) > end. > >In fact it seemed interesting enough that I *did* try it. >I measured the time to sort a list of 10,000 uniformly generated >random floats, on a machine with a 500MHz CPU and 84 MHz memory. >All times in this message are the smallest of 5 runs. > >120 msec Tushar Deshpande's qsort >110 msec Sequential qsort with single partition pass as above > and similarly switching over to lists:qsort for small N. >110 msec Same as above but not bothering with counts, checking > for large -vs- small by [Pivot|L = [_,_,_|_]] or not > and sorting 0 to 3 elements inline. > 80 msec Same as previous, but qsort(L, T) == qsort(L)++T, to > try to reduce the amount of concatenation. > >930 msec Tushar Deshpande's pqsort (with a simplified version of > pmap/2 that doesn't bother with error handling) >230 msec The pqsortB/1 function above. > >With Tushar Deshpande's pqsort/1 being 7.75 times slower than qsort/1, >it's going to take a lot of cores just to catch up. My improved >pqsortB is 4 times faster than his on a single core. > >I tried again with a 2.2GHz CPU, 667 MHz memory, 2-core machine. >This time I needed 100,000 element lists to get useful times. > > 170 msec original qsort > 120 msec > 110 msec > 100 msec best qsort from above > >-smp disable -smp enable > 830 msec 1720 msec original pqsort > 240 msec 310 msec improved pqsortB > 110 msec 110 msec partition once, sort sublists > using one child, combine. > >As a double check, with -smp disable, the emulator reported >[rq:1] in the banner; with -smp enable, [smp:2:2] [rq:2]. > >Here's what I *suspect* is going on. The time ratios for >the sequential versions are very close to the speed ratios >of the main memories on the two machines. It looks as >though memory is the main bottleneck. > >Now my laptop may have two cores, but they share one main >memory. (And I think they share an L2 cache.) If the time >of a computation is dominated by memory, dividing the work >across several CPUs isn't going to help: you need to divide >it across several memories. > >Another key point is that message passing is *copying*. >It seems clear from the "-smp enable" column above that >"the fewer the processes the better", which in this case >actually means "the less COPYING of data between processes >the better". > >-smp disable -smp enable #processes #cells copied > 830 msec 1720 msec 32766 2800000 > 240 msec 310 msec 16383 1406112 > 110 msec 110 msec 1 100000 > >Here a "cell" means a list cell whose head is a float copied >in a message from one process to another. #processes means >the number of *additional* (temporary) processes. > >We have one mechanism in Erlang for both modelling concurrent >systems and obtaining parallel speedups: logically separate >processes communicating by message passing. In the former >case, you create (at least) one Erlang process per logical >activity, NOT for speed but for CLARITY, so that each logical >activity can be modelled separately from the others. The >deciding factor in the number of processes you create is the >number of things to be modelled. > >When you are trying to get parallelism, there is seldom any >point in having more than a small number of times as many >processes as you have available cores. If you have lots of >processes all wanting to be active at the same time, the >costs of scheduling and message passing are likely to be high >relative to the useful work. The best parallel quicksort >above just used two processes (one the original, the other >a temporary child). On a quad core machine, four processes >would make sense. > > >________________________________________________________________ >erlang-questions mailing list. See http://www.erlang.org/faq.html >erlang-questions (at) erlang.org > > From roberto.aloi@REDACTED Wed Nov 18 11:14:02 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Wed, 18 Nov 2009 10:14:02 +0000 (GMT) Subject: [erlang-questions] Remote "access" points In-Reply-To: <14571703.44041258539137048.JavaMail.root@zimbra> Message-ID: <22244220.44061258539242065.JavaMail.root@zimbra> > Hello all, Hi, > I am looking through the Erlang source for any remote access points. > My plan is to try restricting what can be called on a node. > > I've found the ones for spawn - partly because I was looking in the > right place, and partly because they're helpfully commented in > net_kernel.erl as > > %% > %% The spawn/4 BIF ends up here. > %% > handle_call({spawn,M,F,A,Gleader},{From,Tag},State) when is_pid(From) > -> > do_spawn([no_link,{From,Tag},M,F,A,Gleader],[],State); > > By changing that to > handle_call({spawn,M,F,A,Gleader},{From,Tag},State) when > is_pid(From), > M =:= 'public' -> > do_spawn([no_link,{From,Tag},M,F,A,Gleader],[],State); > > I get the desired effect. Trying to spawn('paranoid@REDACTED', os, > cmd, ['ls']) fails on the calling node (although the "paranoid" node > dies a noisy death which isn't ideal, but is fine for testing). > Spawning a public:* function works. > > However, I'm not able to find the place where rpc:*call requests > arrive. Can anyone point me at the right place? As well as any other > points at which remote nodes can start new processes. I'm not sure you're heading to the right direction. I mean, tweaking the core code, identifying all entry points... What are you trying to do exactly? > Another thing - putting a debug io:format doesn't seem to be a good > way to debug which functions are being hit - is there a better way? I would suggest you to have a look to the dbg documentation: http://erlang.org/doc/man/dbg.html A simple tutorial from my blog: http://aloiroberto.wordpress.com/2009/02/23/tracing-erlang-functions/ > Someone also mentioned that when you obtained a Pid from a spawn, you > can do other things with it? Could someone explain what you can do > with it? Try to have a look to the concurrent programming section from the erlang.org website: http://erlang.org/course/concurrent_programming.html Also, here's explained the structure of an Erlang Pid: http://stackoverflow.com/questions/243363/can-someone-explain-the-structure-of-a-pid-in-erlang Regards, Roberto Aloi Erlang Training and Consulting Ltd. http://www.erlang-consulting.com From vladdu55@REDACTED Wed Nov 18 11:31:01 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 18 Nov 2009 11:31:01 +0100 Subject: [erlang-questions] Re: Erlang IDE In-Reply-To: References: <20091117185419.7667C2000098@mwinf2111.orange.fr> Message-ID: <95be1d3b0911180231k246151d9qda9de8ca8bad3621@mail.gmail.com> > ErlIDE is a really good choice and I really recommend it. Try it! Thanks for that comment! :-) If I may squeeze a related issue about erlide here... I fully agree that it's not a solution that fits all needs, but for those that use it or would like to use it, I would like to ask for a little help: We have a long list of things to do with erlide. There are only two of us and there is only so much time. In order to know where to focus the work, we would like to have your input. (By the way, a big "thank you" to all users who already are sending feedback!) If you can spare a few minutes, please go to http://erlide.uservoice.com/pages/9625-general and enter five things you want to see in erlide (either because they are missing or because they are working poorly) or vote for existing entries that match your wishes. If possible, please also state how many users you represent, if there are several at your company (in the body of the post or a comment) We will use this to set the priorities. The best way to make sure just your problem will get fixed is of course to also submit a patch. The second best is a detailed description of the desired use case or if it's a bug, of a way to reproduce it. Another option is to become a paying customer: send us a postcard, invite us to a beer or two, put a good word for us with Santa Claus, that kind of stuff ;-) As a last resort, we might even accept donations. Thanks a lot in advance! best regards, Vlad From icfp.publicity@REDACTED Wed Nov 18 12:13:56 2009 From: icfp.publicity@REDACTED (Wouter Swierstra) Date: Wed, 18 Nov 2009 12:13:56 +0100 Subject: ICFP '10: Second call for workshop proposals Message-ID: <53ff55480911180313v2ee8afa5g76f0777a8afa5f3f@mail.gmail.com> CALL FOR WORKSHOP AND CO-LOCATED EVENT PROPOSALS ICFP 2010 15th ACM SIGPLAN International Conference on Functional Programming September 27 - 29, 2010 Baltimore, Maryland http://www.icfpconference.org/icfp2010 The 15th ACM SIGPLAN International Conference on Functional Programming will be held in Baltimore, Maryland on September 27-29, 2010. ICFP provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. Proposals are invited for workshops (and other co-located events, such as tutorials) to be affiliated with ICFP 2010 and sponsored by SIGPLAN. These events should be more informal and focused than ICFP itself, include sessions that enable interaction among the attendees, and be fairly low-cost. The preference is for one-day events, but other schedules can also be considered. ---------------------------------------------------------------------- Submission details Deadline for submission: November 20, 2009 Notification of acceptance: December 18, 2009 Prospective organizers of workshops or other co-located events are invited to submit a completed workshop proposal form in plain text format to the ICFP 2010 workshop co-chairs (Derek Dreyer and Chris Stone), via email to icfp10-workshops at mpi-sws.org by November 20, 2009. (For proposals of co-located events other than workshops, please fill in the workshop proposal form and just leave blank any sections that do not apply.) Please note that this is a firm deadline. Organizers will be notified if their event proposal is accepted by December 18, 2009, and if successful, depending on the event, they will be asked to produce a final report after the event has taken place that is suitable for publication in SIGPLAN Notices. The proposal form is available at: http://www.icfpconference.org/icfp2010/icfp10-workshops-form.txt Further information about SIGPLAN sponsorship is available at: http://acm.org/sigplan/sigplan_workshop_proposal.htm ---------------------------------------------------------------------- Selection committee The proposals will be evaluated by a committee comprising the following members of the ICFP 2010 organizing committee, together with the members of the SIGPLAN executive committee. Workshop Co-Chair: Derek Dreyer (MPI-SWS) Workshop Co-Chair: Chris Stone (Harvey Mudd College) General Chair: Paul Hudak (Yale University) Program Chair: Stephanie Weirich (University of Pennsylvania) ---------------------------------------------------------------------- Further information Any queries should be addressed to the workshop co-chairs (Derek Dreyer and Chris Stone), via email to icfp10-workshops at mpi-sws.org. From fritchie@REDACTED Wed Nov 18 13:24:11 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 18 Nov 2009 06:24:11 -0600 Subject: R13B02 on 8/16 core box: all TCP communication hangs/frozen Message-ID: <13729.1258547051@snookles.snookles.com> Good morning, all. I posted a more detailed message over on the erlang-bugs mailing list but would like to reach a wider audience. I've encountered a rare, intermittent bug that has been driving me nuts to fix and is about to cause an important customer to be, er, extremely irritated. If you're not interested in bug reports, sorry in advance about the spam. I forgot to mention in the erlang-bugs posting that this appears to have hit us in a small lab (~8 machines) perhaps a handful of times in the last several weeks. Only today have we got some really good diagnostic info, alas. It appears repeatable but requires a ton of patience and time. -Scott [...] While running a distributed Erlang app on a box that probably wasn't very busy at the time, all of its peer nodes decided that the hung box had timed out (some whitespace and newlines flattened): ** Node foo@REDACTED not responding **** Removing (timedout) connection ** After running time of about 2 days, the VM appeared to lock up completely. Not possible to connect a remote shell. No "strace -f -p" output when connecting to the Erlang distribution TCP port reported by "epmd -names", nor any output when connecting to any of the TCP listener sockets owned by our Erlang applications inside the VM (connections via Telnet or "nc" would open in 0-10 seconds, usually, because most app listener sockets use a backlog size of 4096, but no sign of system call activity by the VM). The box in question is a few-months-old Dell (I can look up the model number if it's helpful), dual CPU, quad core + hyperthreading (or whatever Intel calls it today) for 16 virtual cores. If I start an Erlang shell without args: Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:16:16] [rq:16] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.3 (abort with ^G) (slf21203@REDACTED)1> The app is run via: env ERL_MAX_ETS_TABLES=10007 \ erl \ +A 64 +K true \ -noinput -noshell \ -kernel dist_auto_connect once \ -kernel net_ticktime 20 \ ... I've got output from "strace", "lsof", and a little bit of GDB stack backtraces from each Pthread. I've got a core file of the process via "gcore", so I can do other GDB stuff if it'd be helpful. I will almost certainly have at least 8 metric tons of extremely unhappy customer asking lots of anxious questions within 12-24 hours. Unhappy me. If there's other info that I can provide to the OTP team or other debugging parties, please let me know ASAP. And many thanks in advance! -Scott [... about 1200 lines of text deleted ...] From sedrik@REDACTED Wed Nov 18 13:51:40 2009 From: sedrik@REDACTED (Fredrik Andersson) Date: Wed, 18 Nov 2009 13:51:40 +0100 Subject: making new nodes and spawning functions on them inside of common_test Message-ID: <34e33440911180451ta73f198mfc30498d3564b006@mail.gmail.com> Hi everyone I want to use common_test to test our system in a more production like environment than what ordinary unit testing can do. To do this I have setup a small suite and am currently trying to spawn processes. This does not seem to work though and I would like to have some pointer as to what I am doing wrong and if this i even possible. This is a small sample test that does not work. It seems to hang in the receive statement for some reason. -module(pause_SUITE). -compile(export_all). -include_lib("common_test/include/ct.hrl"). %%%%%%%%%%%%%%%% %% common test callbacks %% %%%%%%%%%%%%%%%% all() -> [pause_resume_test]. init_per_suite(Config) -> error_logger:tty(false), slave:start(localhost, test_master), slave:start(localhost, test_slave), Config. end_per_suite(_Config) -> slave:stop('test_master@REDACTED'), slave:stop('test_slave@REDACTED'), ok. init_per_testcase(_TestCase, Config) -> Config. end_per_testcase(_TestCase, _Config) -> ok. %%%%%%%%%% %% test cases %% %%%%%%%%%% add_jobs(Pid) -> Pid ! success. pause_resume_test(_Config) -> spawn('test_master@REDACTED', ?MODULE, add_jobs, [self()]), receive success -> ok end, ok. From dionne@REDACTED Wed Nov 18 12:36:15 2009 From: dionne@REDACTED (Robert Dionne) Date: Wed, 18 Nov 2009 06:36:15 -0500 Subject: [erlang-questions] Erlang IDE In-Reply-To: <4B035B63.6020209@gmail.com> References: <8672048.43021258478467903.JavaMail.root@zimbra> <200911172359.nAHNxZCw008142@fed.local> <6bb4c8cb0911171707x4e341699ofc49f96e674b0ab@mail.gmail.com> <4B035B63.6020209@gmail.com> Message-ID: <596F1EFA-C0BF-4B71-B05C-9AE93093FD03@dionne-associates.com> For emacs users, I highly recommend Distel [1] It essentially makes emacs into a full featured IDE for Erlang. Cheers, Bob [1] http://bc.tech.coop/blog/070528.html [2] http://fresh.homeunix.net/~luke/distel/distel-euc.pdf On Nov 17, 2009, at 9:26 PM, Sean Cribbs wrote: > I started with Kevin's setup (mostly because he coached me on how to > use emacs), but switched to the emacs-starter-kit and customized it > when I got more proficient. > > The original is here: http://github.com/technomancy/emacs-starter-kit > > You can see my fork here: http://github.com/seancribbs/emacs-starter-kit > > Sean > > Adriano Bonat wrote: >> Some weeks ago I did give a try to Emacs when programming in Erlang, >> and I really liked it! :) >> >> I suggest that you take a look in Kevin Smith's blog, there is a >> screencast which explain in details how to setup flymake and Erlang's >> support to Emacs: >> >> http://weblog.hypotheticalabs.com/?page_id=318 >> >> On Tue, Nov 17, 2009 at 9:59 PM, Xavier Maillard wrote: >> >>> Well, what I usually do is to use Emacs + flymake. >>> You get red marking for errors and blue ones for warnings. >>> You also get the reason of the warning/error. >>> >>> Can you share your settings ? >>> >>> Xavier >>> -- >>> http://www.gnu.org >>> http://www.april.org >>> http://www.lolica.org >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >>> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> > From caluml@REDACTED Wed Nov 18 14:38:16 2009 From: caluml@REDACTED (Calum) Date: Wed, 18 Nov 2009 13:38:16 +0000 Subject: [erlang-questions] Remote "access" points In-Reply-To: <22244220.44061258539242065.JavaMail.root@zimbra> References: <14571703.44041258539137048.JavaMail.root@zimbra> <22244220.44061258539242065.JavaMail.root@zimbra> Message-ID: <635498b70911180538m5b005c09jda4d3409dd037fa2@mail.gmail.com> On Wed, Nov 18, 2009 at 10:14 AM, Roberto Aloi wrote: > > I'm not sure you're heading to the right direction. I mean, tweaking the core code, identifying all entry points... > What are you trying to do exactly? I am trying to modify Erlang so that remote nodes can't just run anything they like on other nodes. My plan is to identify the points at which the request to run a function enters the local node, and check if it is calling something in a module called "public". This would then allow the node owner to know what was being exposed, because if it wasn't in a module called public, it wouldn't be runnable. >> Another thing - putting a debug io:format doesn't seem to be a good >> way to debug which functions are being hit - is there a better way? > I would suggest you to have a look to the dbg documentation: > > http://erlang.org/doc/man/dbg.html > > A simple tutorial from my blog: > http://aloiroberto.wordpress.com/2009/02/23/tracing-erlang-functions/ Thanks - I'll have a look at that. >> Someone also mentioned that when you obtained a Pid from a spawn, you >> can do other things with it? Could someone explain what you can do >> with it? > Try to have a look to the concurrent programming section from the erlang.org website: > http://erlang.org/course/concurrent_programming.html I know that you can use it to send messages to the process it was returned from. What I'm curious about is is whether it can be subverted in any way? Let's say I've written a function called public:foo that just listens for messages, and does something simple. If I run Pid = spawn(SomeNode, public, foo, []). is there any way to abuse/manipulate the returned Pid to be able to do other things on SomeNode that aren't part of public:foo. Someone mentioned something called rex - but I didn't understand what part that played. > Also, here's explained the structure of an Erlang Pid: > http://stackoverflow.com/questions/243363/can-someone-explain-the-structure-of-a-pid-in-erlang I'll have a look at that after work - thanks. C From kenneth.lundin@REDACTED Wed Nov 18 15:15:57 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 18 Nov 2009 15:15:57 +0100 Subject: OTP R13B03 will be released on November 25:th Message-ID: Hi, In my presentation on the Erlang User Conference I announced that the next release (OTP R13B03) will be on November 23. THIS WAS INCORRECT and I apologize for the misleading information. The correct release date in November 25. We have as routine to always release on a Wednesday. /Kenneth, Erlang/OTP Ericsson From ulf.wiger@REDACTED Wed Nov 18 15:18:58 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 18 Nov 2009 15:18:58 +0100 Subject: [erlang-questions] Remote "access" points In-Reply-To: <635498b70911180538m5b005c09jda4d3409dd037fa2@mail.gmail.com> References: <14571703.44041258539137048.JavaMail.root@zimbra> <22244220.44061258539242065.JavaMail.root@zimbra> <635498b70911180538m5b005c09jda4d3409dd037fa2@mail.gmail.com> Message-ID: <4B040252.4080709@erlang-consulting.com> Calum wrote: > > Someone mentioned something called rex - but I didn't understand what > part that played. That's the registered name of the rpc server. It is a gen_server, so it's not terribly difficult to figure out how to fake a request without spawning anything on the remote node. Try this (not on a production node!): {rex,RemoteNode} ! {'$gen_call',{dummy_pid,dummy_ref},{call,erlang,halt,[],self()}}. If you can send a message to any process on a node, you can do evil things to that node in general. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From luke@REDACTED Wed Nov 18 15:28:18 2009 From: luke@REDACTED (Luke Gorrie) Date: Wed, 18 Nov 2009 15:28:18 +0100 Subject: [erlang-questions] R13B02 on 8/16 core box: all TCP communication hangs/frozen In-Reply-To: <13729.1258547051@snookles.snookles.com> References: <13729.1258547051@snookles.snookles.com> Message-ID: <1d0fb8fc0911180628p35864ec1ob843084c0ff625d@mail.gmail.com> Hi Scott! I suppose you knew the risk of idle speculation from the peanut gallery when you posted this to erlang-questions.. Can you clarify the timeline? First event (time T0) is foo@REDACTED being detected as "not responding" by its peers (i.e. doesn't answer TCP heartbeat) and second event is seeing that the emulator is totally wedged (time T1). Is it two days between T0 and T1? can you characterise the operational state of the machine in between? lsof shows not many open files / nothing scary. gdb shows the first thread in select() and the rest blocking on a pthreads event. Consistent with (a) normal operation (b) deadlock inside the emulator (c) screwed up file descriptor set or timeout in select(). I suppose (c) at least could be ruled out with gdb'ery. But it seems like the most interesting tidbit is: > connections via Telnet or "nc" would open in 0-10 seconds, usually, > because most app listener sockets use a backlog size of 4096, but no > sign of system call activity by the VM Are you saying that it sometimes takes several seconds to establish a loopback socket connection from telnet or netcat? That sounds extremely fishy! The kernel (not BEAM) is the one responsible for getting the socket to ESTABLISHED state and if that doesn't happen within a few milliseconds then it sounds like your kernel is performing very badly. Can the kernel possibly be very busy? (I watch 'vmstat 1' to check.) If it is busy you could run e.g. oprofile to find out where. One common cause is "too many open " hitting a bad performance case in a kernel data structure. Can be files, sockets, routing entries, iptables conntrack entries, etc (oprofile should make it easy to see which). If you still have the machine wedged I'd be curious to see a page of 'vmstat 1' output and also the full set of open sockets ('netstat -tlnp') and anything else that might be overloaded (are you using e.g. advanced routing or filewalling features?) If you want a sounding board let me know and I'll tell you my mobile number. Cheers, -Luke (damn I miss these problems!) P.S. If the list does NOT get two copies of this mail then it's annoyingly hard to post via Gmane. From xma@REDACTED Wed Nov 18 18:17:08 2009 From: xma@REDACTED (Xavier Maillard) Date: Wed, 18 Nov 2009 18:17:08 +0100 Subject: [erlang-questions] Re: Erlang IDE In-Reply-To: (message from Steve Davis on Tue, 17 Nov 2009 16:44:54 -0800 (PST)) References: <8672048.43021258478467903.JavaMail.root@zimbra> <200911172359.nAHNxZCw008142@fed.local> Message-ID: <200911181717.nAIHH85c012018@fed.local> It's vim and (when on windows) scite for me :) ...as has been stated -- I don't really feel a great need for "more". GNU Emacs because I do not need more as well :D Xavier -- http://www.gnu.org http://www.april.org http://www.lolica.org From hvjunk@REDACTED Wed Nov 18 18:18:38 2009 From: hvjunk@REDACTED (Hendrik Visage) Date: Wed, 18 Nov 2009 19:18:38 +0200 Subject: [erlang-questions] R13B02 on 8/16 core box: all TCP communication hangs/frozen In-Reply-To: <1d0fb8fc0911180628p35864ec1ob843084c0ff625d@mail.gmail.com> References: <13729.1258547051@snookles.snookles.com> <1d0fb8fc0911180628p35864ec1ob843084c0ff625d@mail.gmail.com> Message-ID: On 11/18/09, Luke Gorrie wrote: > Hi Scott! > > I suppose you knew the risk of idle speculation from the peanut > gallery when you posted this to erlang-questions.. > > But it seems like the most interesting tidbit is: > >> connections via Telnet or "nc" would open in 0-10 seconds, usually, >> because most app listener sockets use a backlog size of 4096, but no >> sign of system call activity by the VM > > Are you saying that it sometimes takes several seconds to establish a > loopback socket connection from telnet or netcat? That sounds > extremely fishy! The kernel (not BEAM) is the one responsible for > getting the socket to ESTABLISHED state and if that doesn't happen > within a few milliseconds then it sounds like your kernel is > performing very badly. My *first* question while reading the original description: "OS? Kernel version? patch levels?" (It's ot that obvious from the words "Dell", as I'd prefer to deploy Solaris x86, while you might prefer OpenSolaris and the next guy Windows.. and especially since the later Solaris and OpenSolaris have been looking at going to newer network TCP/IP stacks for zone related fun ;) Also: uptime of the different systems? another why to do the "debugging": truss -ffo /big/disk/tracefiles erl (or the equivalent strace options) and then to wait/initiate the "bug", that way we can have info into the long term kernel information (perhaps even also sotruss??) or else attach to the processes as you get to trigger this issue/bug. Also: You have checked that your kernel is capable of your number of TCP connections? > Can the kernel possibly be very busy? (I watch 'vmstat 1' to check.) > If it is busy you could run e.g. oprofile to find out where. One > common cause is "too many open " hitting a bad performance > case in a kernel data structure. Can be files, sockets, routing > entries, iptables conntrack entries, etc (oprofile should make it easy > to see which). > > If you still have the machine wedged I'd be curious to see a page of > 'vmstat 1' output and also the full set of open sockets ('netstat > -tlnp') and anything else that might be overloaded (are you using e.g. > advanced routing or filewalling features?) > > If you want a sounding board let me know and I'll tell you my mobile number. > > Cheers, > -Luke (damn I miss these problems!) > > P.S. If the list does NOT get two copies of this mail then it's > annoyingly hard to post via Gmane. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From jeffrey.horner@REDACTED Wed Nov 18 18:50:16 2009 From: jeffrey.horner@REDACTED (Jeffrey Horner) Date: Wed, 18 Nov 2009 11:50:16 -0600 Subject: Erlang logo licensing and usage examples Message-ID: Hi, Reviving this thread because I've been smitten with "Erlang: The Movie" on youtube (i'm a sucker for hacker/geek/pop culture). I've got a design up here for critique if anyone is interested: http://www.threadless.com/critique/56844/Erlang_The_Movie I'd like to use the erlang logo font for the title and clear any copryight issues. I'd also like to release the design to the public domain, copyleft, creative comons, etc. whatever works for the commuty. Let me know what you think. Jeff -- http://biostat.mc.vanderbilt.edu/JeffreyHorner From caluml@REDACTED Wed Nov 18 19:27:14 2009 From: caluml@REDACTED (Calum) Date: Wed, 18 Nov 2009 18:27:14 +0000 Subject: [erlang-questions] Remote "access" points In-Reply-To: <4B040252.4080709@erlang-consulting.com> References: <14571703.44041258539137048.JavaMail.root@zimbra> <22244220.44061258539242065.JavaMail.root@zimbra> <635498b70911180538m5b005c09jda4d3409dd037fa2@mail.gmail.com> <4B040252.4080709@erlang-consulting.com> Message-ID: <635498b70911181027s40bcdbfam5adb22b51144f052@mail.gmail.com> On Wed, Nov 18, 2009 at 2:18 PM, Ulf Wiger wrote: > That's the registered name of the rpc server. > It is a gen_server, so it's not terribly difficult > to figure out how to fake a request without spawning > anything on the remote node. > > Try this (not on a production node!): > > {rex,RemoteNode} ! > {'$gen_call',{dummy_pid,dummy_ref},{call,erlang,halt,[],self()}}. > > If you can send a message to any process on a node, you can > do evil things to that node in general. But I spy the module name in there - erlang. If I could filter the {call,erlang,halt,[],self()} part ( use the guard =:= 'public', for instance), then that would fail too? I do appreciate that I'm trying to do stuff that's "way above my pay grade" in terms of Erlang, but surely all remote module:function(Args) requests have to enter the remote node somewhere, so there must be a finite amount of places that would need checking. Unless in doing so, I'd end up crippling the local stuff too - that would be a problem. This is all basically in aid of getting a bunch of people who are interested in learning/experimenting with Erlang together to form a sort of public Erlang network, in a way that they could all use resources on each other's machines, but not in a way that allows any destructive actions - deleting files, spamming, whatever. If it's not possible to modify Erlang to do it, a virtual machine, suitably firewalled would be OK, as potentially could a decent chroot too. C From andrew@REDACTED Wed Nov 18 19:32:47 2009 From: andrew@REDACTED (Andrew Thompson) Date: Wed, 18 Nov 2009 13:32:47 -0500 Subject: [erlang-questions] Remote "access" points In-Reply-To: <635498b70911181027s40bcdbfam5adb22b51144f052@mail.gmail.com> References: <14571703.44041258539137048.JavaMail.root@zimbra> <22244220.44061258539242065.JavaMail.root@zimbra> <635498b70911180538m5b005c09jda4d3409dd037fa2@mail.gmail.com> <4B040252.4080709@erlang-consulting.com> <635498b70911181027s40bcdbfam5adb22b51144f052@mail.gmail.com> Message-ID: <20091118183247.GI4286@hijacked.us> On Wed, Nov 18, 2009 at 06:27:14PM +0000, Calum wrote: > On Wed, Nov 18, 2009 at 2:18 PM, Ulf Wiger > wrote: > > That's the registered name of the rpc server. > > It is a gen_server, so it's not terribly difficult > > to figure out how to fake a request without spawning > > anything on the remote node. > > > > Try this (not on a production node!): > > > > {rex,RemoteNode} ! > > {'$gen_call',{dummy_pid,dummy_ref},{call,erlang,halt,[],self()}}. > > > > If you can send a message to any process on a node, you can > > do evil things to that node in general. > > But I spy the module name in there - erlang. > If I could filter the {call,erlang,halt,[],self()} part ( use the > guard =:= 'public', for instance), then that would fail too? > But its a message not a function call? Are you proposing to filter all messages from other nodes as well as all calls to rpc:call or spawn? Andrew From ulf.wiger@REDACTED Wed Nov 18 19:33:13 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 18 Nov 2009 19:33:13 +0100 Subject: [erlang-questions] Remote "access" points In-Reply-To: <635498b70911181027s40bcdbfam5adb22b51144f052@mail.gmail.com> References: <14571703.44041258539137048.JavaMail.root@zimbra> <22244220.44061258539242065.JavaMail.root@zimbra> <635498b70911180538m5b005c09jda4d3409dd037fa2@mail.gmail.com> <4B040252.4080709@erlang-consulting.com> <635498b70911181027s40bcdbfam5adb22b51144f052@mail.gmail.com> Message-ID: <4B043DE9.3030501@erlang-consulting.com> Calum wrote: > > But I spy the module name in there - erlang. > If I could filter the {call,erlang,halt,[],self()} part ( use the > guard =:= 'public', for instance), then that would fail too? The problem is that you open up a huge security hole if you allow messages to be sent by default to any process and then try to filter out only those that you know could be dangerous. I think the only sensible default if you want to make it safe, is to deny everything you don't know to be safe. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From navaneethanit@REDACTED Wed Nov 18 19:41:26 2009 From: navaneethanit@REDACTED (NavTux) Date: Wed, 18 Nov 2009 10:41:26 -0800 (PST) Subject: why with Linux Message-ID: <7e9946a6-e350-4ada-83bc-a1ed0834f3fd@d9g2000prh.googlegroups.com> Hi Erlangies, In my previous post i had come to know that there is a difference to use erlang in windows rather than Linux peculiarly ubuntu,Here I should come to know that In what way which occurs? I mean there are all IDE s are available both of them,particularly Is any problem with windows related with erlang? then How can i feel the difference between them? ---------- regards, Navaneethan, http://navaspot.wordpress.com From caluml@REDACTED Wed Nov 18 19:42:52 2009 From: caluml@REDACTED (Calum) Date: Wed, 18 Nov 2009 18:42:52 +0000 Subject: [erlang-questions] Remote "access" points In-Reply-To: <20091118183247.GI4286@hijacked.us> References: <14571703.44041258539137048.JavaMail.root@zimbra> <22244220.44061258539242065.JavaMail.root@zimbra> <635498b70911180538m5b005c09jda4d3409dd037fa2@mail.gmail.com> <4B040252.4080709@erlang-consulting.com> <635498b70911181027s40bcdbfam5adb22b51144f052@mail.gmail.com> <20091118183247.GI4286@hijacked.us> Message-ID: <635498b70911181042t13d7f42ex6417cd2bf97aea03@mail.gmail.com> On Wed, Nov 18, 2009 at 6:32 PM, Andrew Thompson wrote: > But its a message not a function call? Are you proposing to filter all > messages from other nodes as well as all calls to rpc:call or spawn? Isn't it a message that requests a function gets executed? Somewhere in the request, the module name must be in there. I'd like to make an Erlang node that **only** allows remote nodes to call functions in the "public" module (whether by spawn, call, or any other means). If you add , M =:= 'public' to line 396 of lib/kernel/src/net_kernel.erl in otp_src_R13B02-1, then it prevents remote nodes from "spawn"ing anything on that node. I can't find the rpc:call equivalent though. From rickard.s.green@REDACTED Wed Nov 18 20:57:03 2009 From: rickard.s.green@REDACTED (Rickard Green) Date: Wed, 18 Nov 2009 20:57:03 +0100 Subject: [erlang-bugs] R13B02 on 8/16 core box: all TCP communication hangs/frozen In-Reply-To: <13253.1258546561@snookles.snookles.com> References: <13253.1258546561@snookles.snookles.com> Message-ID: <10d7e35f0911181157h2087b8c8sbb1684984825a8e9@mail.gmail.com> Hi Scott, I've attached a patch that fixes a race that could cause this situation. I haven't tested it thoroughly, but since our test suites haven't been able to trigger this situation, I thought that it would be best if you could test it at the same time as we are testing it. Please, let me know if it your problem is solved or not by this patch. If it doesn't cause any unexpected problems, it will appear in the soon to be released R13B03. How to apply the patch: $ ls otp_src_R13B02-1_OTP-8297.patch otp_src_R13B02-1.tar.gz $ gtar -zxf otp_src_R13B02-1.tar.gz $ patch -ZNp0 < otp_src_R13B02-1_OTP-8297.patch patching file otp_src_R13B02-1/erts/emulator/beam/erl_process.c $ cd otp_src_R13B02-1 $ # configure, make, and make install as usual... Regards, Rickard Green, Erlang/OTP, Ericsson AB 2009/11/18 Scott Lystig Fritchie : > Hi, all. I've scoured the erlang-patches, erlang-bugs, and > erlang-questions mailing list archives for help on this, but it doesn't > seem to have popped up yet? > > While running a distributed Erlang app on a box that probably wasn't > very busy at the time, all of its peer nodes decided that the hung box > had timed out (some whitespace and newlines flattened): > > ** Node foo@REDACTED not responding **** Removing (timedout) connection ** > > After running time of about 2 days, the VM appeared to lock up > completely. Not possible to connect a remote shell. No "strace -f -p" > output when connecting to the Erlang distribution TCP port reported by > "epmd -names", nor any output when connecting to any of the TCP listener > sockets owned by our Erlang applications inside the VM (connections via > Telnet or "nc" would open in 0-10 seconds, usually, because most app > listener sockets use a backlog size of 4096, but no sign of system call > activity by the VM). > > The box in question is a few-months-old Dell (I can look up the model > number if it's helpful), dual CPU, quad core + hyperthreading (or > whatever Intel calls it today) for 16 virtual cores. If I start an > Erlang shell without args: > > Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:16:16] [rq:16] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.7.3 (abort with ^G) > (slf21203@REDACTED)1> > > The app is run via: > > env ERL_MAX_ETS_TABLES=10007 \ > erl \ > +A 64 +K true \ > -noinput -noshell \ > -kernel dist_auto_connect once \ > -kernel net_ticktime 20 \ > ... > > I've got output from "strace", "lsof", and a little bit of GDB stack > backtraces from each Pthread. I've got a core file of the process via > "gcore", so I can do other GDB stuff if it'd be helpful. > > I will almost certainly have at least 8 metric tons of extremely unhappy > customer asking lots of anxious questions within 12-24 hours. Unhappy > me. If there's other info that I can provide to the OTP team or other > debugging parties, please let me know ASAP. And many thanks in advance! > > -Scott > > Output from "strace -f": > > [pid 28271] futex(0x2b3519c8d404, FUTEX_WAIT_PRIVATE, 1, NULL > [pid 28270] futex(0x2b3519c8d204, FUTEX_WAIT_PRIVATE, 1, NULL > [pid 28269] futex(0x2b3519c8d004, FUTEX_WAIT_PRIVATE, 201, NULL > [pid 28268] futex(0x2b3519c8ce04, FUTEX_WAIT_PRIVATE, 1, NULL > [pid 28267] futex(0x2b3519c8cc04, FUTEX_WAIT_PRIVATE, 1, NULL > [pid 28266] futex(0x2b3519c8ca04, FUTEX_WAIT_PRIVATE, 1, NULL > [pid 28265] futex(0x2b3519c8c804, FUTEX_WAIT_PRIVATE, 1, NULL > [pid 28264] futex(0x2b3519c8c604, FUTEX_WAIT_PRIVATE, 1, NULL > [pid 28263] futex(0x2b3519c8c404, FUTEX_WAIT_PRIVATE, 1, NULL > [pid 28262] futex(0x2b3519c8c204, FUTEX_WAIT_PRIVATE, 1, NULL > [pid 28261] futex(0x2b3519c8c004, FUTEX_WAIT_PRIVATE, 1, NULL > [pid 28260] futex(0x2b3519c8be04, FUTEX_WAIT_PRIVATE, 1, NULL > [pid 28259] futex(0x2b3519c8bc04, FUTEX_WAIT_PRIVATE, 783259, NULL > [pid 28258] futex(0x2b3519c8ba04, FUTEX_WAIT_PRIVATE, 4212125, NULL > [pid 28257] futex(0x2b3519c8b804, FUTEX_WAIT_PRIVATE, 11335295, NULL > [pid 28256] futex(0x2b3519c8b604, FUTEX_WAIT_PRIVATE, 11458135, NULL > [pid 28255] wait4(-1, > [pid 28254] futex(0x2aaaac92e3f4, FUTEX_WAIT_PRIVATE, 123, NULL > [pid 28253] futex(0x2aaaac92e364, FUTEX_WAIT_PRIVATE, 125, NULL > [pid 28252] futex(0x2aaaac92e2d4, FUTEX_WAIT_PRIVATE, 123, NULL > [pid 28251] futex(0x2aaaac92e244, FUTEX_WAIT_PRIVATE, 845, NULL > [pid 28250] futex(0x2aaaac92e1b4, FUTEX_WAIT_PRIVATE, 845, NULL > [pid 28249] futex(0x2aaaac92e124, FUTEX_WAIT_PRIVATE, 125, NULL > [pid 28248] futex(0x2aaaac92e094, FUTEX_WAIT_PRIVATE, 123, NULL > [pid 28247] futex(0x2aaaac92e004, FUTEX_WAIT_PRIVATE, 123, NULL > [pid 28246] futex(0x2aaaac92df74, FUTEX_WAIT_PRIVATE, 197345, NULL > [pid 28245] futex(0x2aaaac92dee4, FUTEX_WAIT_PRIVATE, 123, NULL > [pid 28244] futex(0x2aaaac92de54, FUTEX_WAIT_PRIVATE, 123, NULL > [pid 28243] futex(0x2aaaac92ddc4, FUTEX_WAIT_PRIVATE, 129, NULL > [pid 28242] futex(0x2aaaac92dd34, FUTEX_WAIT_PRIVATE, 125, NULL > [pid 28241] futex(0x2aaaac92dca4, FUTEX_WAIT_PRIVATE, 315, NULL > [pid 28240] futex(0x2aaaac92dc14, FUTEX_WAIT_PRIVATE, 129, NULL > [pid 28239] futex(0x2aaaac92db84, FUTEX_WAIT_PRIVATE, 152779, NULL > [pid 28238] futex(0x2aaaac92daf4, FUTEX_WAIT_PRIVATE, 121, NULL > [pid 28237] futex(0x2aaaac92da64, FUTEX_WAIT_PRIVATE, 121, NULL > [pid 28236] futex(0x2aaaac92d9d4, FUTEX_WAIT_PRIVATE, 121, NULL > [pid 28235] futex(0x2aaaac92d944, FUTEX_WAIT_PRIVATE, 123, NULL > [pid 28234] futex(0x2aaaac92d8b4, FUTEX_WAIT_PRIVATE, 1685, NULL > [pid 28233] futex(0x2aaaac92d824, FUTEX_WAIT_PRIVATE, 152827, NULL > [pid 28232] futex(0x2aaaac92d794, FUTEX_WAIT_PRIVATE, 127, NULL > [pid 28231] futex(0x2aaaac92d704, FUTEX_WAIT_PRIVATE, 123, NULL > [pid 28230] futex(0x2aaaac92d674, FUTEX_WAIT_PRIVATE, 119, NULL > [pid 28229] futex(0x2aaaac92d5e4, FUTEX_WAIT_PRIVATE, 125, NULL > [pid 28228] futex(0x2aaaac92d554, FUTEX_WAIT_PRIVATE, 127, NULL > [pid 28227] futex(0x2aaaac92d4c4, FUTEX_WAIT_PRIVATE, 783, NULL > [pid 28226] futex(0x2aaaac92d434, FUTEX_WAIT_PRIVATE, 129, NULL > [pid 28225] futex(0x2aaaac92d3a4, FUTEX_WAIT_PRIVATE, 125, NULL > [pid 28224] futex(0x2aaaac92d314, FUTEX_WAIT_PRIVATE, 158751, NULL > [pid 28223] futex(0x2aaaac92d284, FUTEX_WAIT_PRIVATE, 131, NULL > [pid 28222] futex(0x2aaaac92d1f4, FUTEX_WAIT_PRIVATE, 125, NULL > [pid 28221] futex(0x2aaaac92d164, FUTEX_WAIT_PRIVATE, 131, NULL > [pid 28220] futex(0x2aaaac92d0d4, FUTEX_WAIT_PRIVATE, 129, NULL > [pid 28219] futex(0x2aaaac92d044, FUTEX_WAIT_PRIVATE, 125, NULL > [pid 28218] futex(0x2aaaac92cfb4, FUTEX_WAIT_PRIVATE, 779, NULL > [pid 28217] futex(0x2aaaac92cf24, FUTEX_WAIT_PRIVATE, 127, NULL > [pid 28216] futex(0x2aaaac92ce94, FUTEX_WAIT_PRIVATE, 125, NULL > [pid 28215] futex(0x2aaaac92ce04, FUTEX_WAIT_PRIVATE, 123, NULL > [pid 28214] futex(0x2aaaac92cd74, FUTEX_WAIT_PRIVATE, 123, NULL > [pid 28213] futex(0x2aaaac92cce4, FUTEX_WAIT_PRIVATE, 121, NULL > [pid 28212] futex(0x2aaaac92cc54, FUTEX_WAIT_PRIVATE, 713, NULL > [pid 28211] futex(0x2aaaac92cbc4, FUTEX_WAIT_PRIVATE, 123, NULL > [pid 28210] futex(0x2aaaac92cb34, FUTEX_WAIT_PRIVATE, 795, NULL > [pid 28209] futex(0x2aaaac92caa4, FUTEX_WAIT_PRIVATE, 139, NULL > [pid 28208] futex(0x2aaaac92ca14, FUTEX_WAIT_PRIVATE, 127, NULL > [pid 28207] futex(0x2aaaac92c984, FUTEX_WAIT_PRIVATE, 121, NULL > [pid 28206] futex(0x2aaaac92c8f4, FUTEX_WAIT_PRIVATE, 123, NULL > [pid 28205] futex(0x2aaaac92c864, FUTEX_WAIT_PRIVATE, 121, NULL > [pid 28204] futex(0x2aaaac92c7d4, FUTEX_WAIT_PRIVATE, 127, NULL > [pid 28203] futex(0x2aaaac92c744, FUTEX_WAIT_PRIVATE, 125, NULL > [pid 28202] futex(0x2aaaac92c6b4, FUTEX_WAIT_PRIVATE, 129, NULL > [pid 28201] futex(0x2aaaac92c624, FUTEX_WAIT_PRIVATE, 127, NULL > [pid 28200] futex(0x2aaaac92c594, FUTEX_WAIT_PRIVATE, 133, NULL > [pid 28199] futex(0x2aaaac92c504, FUTEX_WAIT_PRIVATE, 129, NULL > [pid 28198] futex(0x2aaaac92c474, FUTEX_WAIT_PRIVATE, 139, NULL > [pid 28197] futex(0x2aaaac92c3e4, FUTEX_WAIT_PRIVATE, 1881, NULL > [pid 28196] futex(0x2aaaac92c354, FUTEX_WAIT_PRIVATE, 131, NULL > [pid 28195] futex(0x2aaaac92c2c4, FUTEX_WAIT_PRIVATE, 131, NULL > [pid 28194] futex(0x2aaaac92c234, FUTEX_WAIT_PRIVATE, 131, NULL > [pid 28193] futex(0x2aaaac92c1a4, FUTEX_WAIT_PRIVATE, 127, NULL > [pid 28192] futex(0x2aaaac92c114, FUTEX_WAIT_PRIVATE, 125, NULL > [pid 28191] futex(0x2aaaac92c084, FUTEX_WAIT_PRIVATE, 129, NULL > [pid 28190] futex(0x810ca4, FUTEX_WAIT_PRIVATE, 1, NULL > [pid 28189] read(6, [SLF: wait forever....] > > Output from "lsof -nP" > > COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME > beam.smp 28182 mmssys cwd DIR 8,3 4096 10485761 /m2h_data/data > beam.smp 28182 mmssys rtd DIR 8,1 4096 2 / > beam.smp 28182 mmssys txt REG 8,1 10282072 90608848 /usr/local/gemini/e > beam.smp 28182 mmssys mem REG 8,1 139416 48660505 /lib64/ld-2.5.so > beam.smp 28182 mmssys mem REG 8,1 1713096 48660507 /lib64/libc-2.5.so > beam.smp 28182 mmssys mem REG 8,1 23360 48660688 /lib64/libdl-2.5.so > beam.smp 28182 mmssys mem REG 8,1 615136 48660690 /lib64/libm-2.5.so > beam.smp 28182 mmssys mem REG 8,1 145592 48660704 /lib64/libpthread-2 > beam.smp 28182 mmssys mem REG 8,1 53448 48660706 /lib64/librt-2.5.so > beam.smp 28182 mmssys mem REG 8,1 18152 48660804 /lib64/libutil-2.5. > beam.smp 28182 mmssys mem REG 8,1 380464 89141483 /usr/lib64/libncurs > beam.smp 28182 mmssys mem REG 8,1 77360 90605001 /usr/local/gemini/e > beam.smp 28182 mmssys mem REG 8,1 1781932 90608896 /usr/local/gemini/e > beam.smp 28182 mmssys 0r CHR 1,3 3898 /dev/null > beam.smp 28182 mmssys 1w CHR 1,3 3898 /dev/null > beam.smp 28182 mmssys 2w CHR 1,3 3898 /dev/null > beam.smp 28182 mmssys 3r 0000 0,11 0 98551559 eventpoll > beam.smp 28182 mmssys 4r FIFO 0,6 98551560 pipe > beam.smp 28182 mmssys 5w FIFO 0,6 98551560 pipe > beam.smp 28182 mmssys 6r FIFO 0,6 98551561 pipe > beam.smp 28182 mmssys 7w FIFO 0,6 98551561 pipe > beam.smp 28182 mmssys 8u IPv4 98551569 TCP *:45315 (LISTEN) > beam.smp 28182 mmssys 9u IPv4 98551571 TCP 127.0.0.1:56642->12 > beam.smp 28182 mmssys 10r FIFO 0,6 98551574 pipe > beam.smp 28182 mmssys 11w REG 8,6 36288720 21397512 /foo_log/log/archiv > beam.smp 28182 mmssys 12u IPv4 98551581 TCP *:7585 (LISTEN) > beam.smp 28182 mmssys 13w FIFO 0,6 98551575 pipe > beam.smp 28182 mmssys 14u IPv4 98551584 TCP *:7576 (LISTEN) > beam.smp 28182 mmssys 15u IPv4 98551587 TCP A.B.C.140:45315 > beam.smp 28182 mmssys 16u IPv4 98551605 TCP A.B.C.140:56270 > beam.smp 28182 mmssys 17u IPv4 98551606 TCP A.B.C.140:43152 > beam.smp 28182 mmssys 18u IPv4 98551608 TCP A.B.C.140:53789 > beam.smp 28182 mmssys 19u IPv4 98551607 TCP A.B.C.140:42325 > beam.smp 28182 mmssys 20u IPv4 98551609 TCP A.B.C.140:2958- > beam.smp 28182 mmssys 21u IPv4 98551620 TCP A.B.C.140:55959 > beam.smp 28182 mmssys 22u sock 0,5 98697328 can't identify prot > beam.smp 28182 mmssys 23u sock 0,5 98696776 can't identify prot > beam.smp 28182 mmssys 24u sock 0,5 98698710 can't identify prot > beam.smp 28182 mmssys 25u sock 0,5 98695296 can't identify prot > beam.smp 28182 mmssys 26u sock 0,5 98698083 can't identify prot > beam.smp 28182 mmssys 27u sock 0,5 98698783 can't identify prot > beam.smp 28182 mmssys 28u sock 0,5 98698460 can't identify prot > beam.smp 28182 mmssys 30u sock 0,5 98698635 can't identify prot > beam.smp 28182 mmssys 31u sock 0,5 98685423 can't identify prot > beam.smp 28182 mmssys 32u sock 0,5 98698786 can't identify prot > beam.smp 28182 mmssys 33u sock 0,5 98698648 can't identify prot > beam.smp 28182 mmssys 34u sock 0,5 98698717 can't identify prot > beam.smp 28182 mmssys 36u sock 0,5 98686061 can't identify prot > beam.smp 28182 mmssys 37u sock 0,5 98698545 can't identify prot > beam.smp 28182 mmssys 38u sock 0,5 98686092 can't identify prot > beam.smp 28182 mmssys 41u sock 0,5 98698407 can't identify prot > beam.smp 28182 mmssys 42u sock 0,5 98698722 can't identify prot > beam.smp 28182 mmssys 43u sock 0,5 98698729 can't identify prot > > Output from GDB: > > # gdb /usr/local/gemini/ert/R13B02/lib/erlang/erts-5.7.3/bin/beam.smp ./beam.smpfoo-Nov18.0357orso.core.28182 > GNU gdb Fedora (6.8-27.el5) > Copyright (C) 2008 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "x86_64-redhat-linux-gnu"... > Reading symbols from /lib64/libutil.so.1...done. > Loaded symbols for /lib64/libutil.so.1 > Reading symbols from /lib64/libdl.so.2...done. > Loaded symbols for /lib64/libdl.so.2 > Reading symbols from /lib64/libm.so.6...done. > Loaded symbols for /lib64/libm.so.6 > Reading symbols from /usr/lib64/libncurses.so.5...done. > Loaded symbols for /usr/lib64/libncurses.so.5 > Reading symbols from /lib64/libpthread.so.0...done. > Loaded symbols for /lib64/libpthread.so.0 > Reading symbols from /lib64/librt.so.1...done. > Loaded symbols for /lib64/librt.so.1 > Reading symbols from /lib64/libc.so.6...done. > Loaded symbols for /lib64/libc.so.6 > Reading symbols from /lib64/ld-linux-x86-64.so.2...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/local/gemini/ert/R13B02/lib/erlang/lib/crypto-1.6.1/priv/lib/crypto_drv.so...done. > Loaded symbols for /usr/local/gemini/ert/R13B02/lib/erlang/lib/crypto-1.6.1/priv/lib/crypto_drv.so > Reading symbols from /usr/local/gemini/ert/R13B02/openssl/lib/libcrypto.so.0.9.8...done. > Loaded symbols for /usr/local/gemini/ert/R13B02/openssl/lib/libcrypto.so.0.9.8 > Core was generated by `/usr/local/gemini/ert/R13B02/lib/erlang/erts-5.7.3/bin/beam.smp'. > [New process 28189] > [New process 28190] > [New process 28191] > [New process 28192] > [New process 28193] > [New process 28194] > [New process 28195] > [New process 28196] > [New process 28197] > [New process 28198] > [New process 28199] > [New process 28200] > [New process 28201] > [New process 28202] > [New process 28203] > [New process 28204] > [New process 28205] > [New process 28206] > [New process 28207] > [New process 28208] > [New process 28209] > [New process 28210] > [New process 28211] > [New process 28212] > [New process 28213] > [New process 28214] > [New process 28215] > [New process 28216] > [New process 28217] > [New process 28218] > [New process 28219] > [New process 28220] > [New process 28221] > [New process 28222] > [New process 28223] > [New process 28224] > [New process 28225] > [New process 28226] > [New process 28227] > [New process 28228] > [New process 28229] > [New process 28230] > [New process 28231] > [New process 28232] > [New process 28233] > [New process 28234] > [New process 28235] > [New process 28236] > [New process 28237] > [New process 28238] > [New process 28239] > [New process 28240] > [New process 28241] > [New process 28242] > [New process 28243] > [New process 28244] > [New process 28245] > [New process 28246] > [New process 28247] > [New process 28248] > [New process 28249] > [New process 28250] > [New process 28251] > [New process 28252] > [New process 28253] > [New process 28254] > [New process 28255] > [New process 28256] > [New process 28257] > [New process 28258] > [New process 28259] > [New process 28260] > [New process 28261] > [New process 28262] > [New process 28263] > [New process 28264] > [New process 28265] > [New process 28266] > [New process 28267] > [New process 28268] > [New process 28269] > [New process 28270] > [New process 28271] > [New process 28182] > #0 0x0000003d05a0d2cb in read () from /lib64/libpthread.so.0 > (gdb) thread apply all info stack > > Thread 84 (process 28182): > #0 0x0000003d04ecc5e2 in select () from /lib64/libc.so.6 > #1 0x0000000000523290 in erts_sys_main_thread () at sys/unix/sys.c:3019 > #2 0x000000000044bb7f in erl_start (argc=34, argv=) > at beam/erl_init.c:1275 > #3 0x000000000042ee09 in main (argc=0, argv=0x0) at sys/unix/erl_main.c:29 > > Thread 83 (process 28271): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=16, rq=0x2b3519c8d3b0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > ---Type to continue, or q to quit--- > Thread 82 (process 28270): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=15, rq=0x2b3519c8d1b0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 81 (process 28269): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=14, rq=0x2b3519c8cfb0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > ---Type to continue, or q to quit--- > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 80 (process 28268): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=13, rq=0x2b3519c8cdb0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 79 (process 28267): > ---Type to continue, or q to quit--- > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=12, rq=0x2b3519c8cbb0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 78 (process 28266): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=11, rq=0x2b3519c8c9b0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > ---Type to continue, or q to quit--- > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 77 (process 28265): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=10, rq=0x2b3519c8c7b0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 76 (process 28264): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > ---Type to continue, or q to quit--- > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=9, rq=0x2b3519c8c5b0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 75 (process 28263): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=8, rq=0x2b3519c8c3b0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > ---Type to continue, or q to quit--- > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 74 (process 28262): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=7, rq=0x2b3519c8c1b0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 73 (process 28261): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > ---Type to continue, or q to quit--- > #1 0x0000000000494ab2 in sched_cnd_wait (no=6, rq=0x2b3519c8bfb0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 72 (process 28260): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=5, rq=0x2b3519c8bdb0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > ---Type to continue, or q to quit--- > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 71 (process 28259): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=4, rq=0x2b3519c8bbb0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 70 (process 28258): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=3, rq=0x2b3519c8b9b0) > ---Type to continue, or q to quit--- > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 69 (process 28257): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=2, rq=0x2b3519c8b7b0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > ---Type to continue, or q to quit--- > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 68 (process 28256): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x0000000000494ab2 in sched_cnd_wait (no=1, rq=0x2b3519c8b5b0) > at beam/erl_threads.h:634 > #2 0x000000000049f4c3 in schedule (p=, > calls=) at beam/erl_process.c:5992 > #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126 > #4 0x000000000049fd06 in sched_thread_func (vesdp=) > at beam/erl_process.c:3045 > #5 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 67 (process 28255): > #0 0x0000003d05a0e03f in waitpid () from /lib64/libpthread.so.0 > #1 0x000000000052448e in child_waiter (unused=) > at sys/unix/sys.c:2828 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > ---Type to continue, or q to quit--- > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 66 (process 28254): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 65 (process 28253): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > ---Type to continue, or q to quit--- > > Thread 64 (process 28252): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 63 (process 28251): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 62 (process 28250): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > ---Type to continue, or q to quit--- > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 61 (process 28249): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 60 (process 28248): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > ---Type to continue, or q to quit--- > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 59 (process 28247): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 58 (process 28246): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > ---Type to continue, or q to quit--- > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 57 (process 28245): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 56 (process 28244): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 55 (process 28243): > ---Type to continue, or q to quit--- > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 54 (process 28242): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 53 (process 28241): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > ---Type to continue, or q to quit--- > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 52 (process 28240): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 51 (process 28239): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > ---Type to continue, or q to quit--- > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 50 (process 28238): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 49 (process 28237): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > ---Type to continue, or q to quit--- > Thread 48 (process 28236): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 47 (process 28235): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 46 (process 28234): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > ---Type to continue, or q to quit--- > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 45 (process 28233): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 44 (process 28232): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > ---Type to continue, or q to quit--- > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 43 (process 28231): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 42 (process 28230): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > ---Type to continue, or q to quit--- > > Thread 41 (process 28229): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 40 (process 28228): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 39 (process 28227): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > ---Type to continue, or q to quit--- > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 38 (process 28226): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 37 (process 28225): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > ---Type to continue, or q to quit--- > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 36 (process 28224): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 35 (process 28223): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > ---Type to continue, or q to quit--- > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 34 (process 28222): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 33 (process 28221): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 32 (process 28220): > ---Type to continue, or q to quit--- > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 31 (process 28219): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 30 (process 28218): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > ---Type to continue, or q to quit--- > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 29 (process 28217): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 28 (process 28216): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > ---Type to continue, or q to quit--- > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 27 (process 28215): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 26 (process 28214): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > ---Type to continue, or q to quit--- > Thread 25 (process 28213): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 24 (process 28212): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 23 (process 28211): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > ---Type to continue, or q to quit--- > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 22 (process 28210): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 21 (process 28209): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > ---Type to continue, or q to quit--- > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 20 (process 28208): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 19 (process 28207): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > ---Type to continue, or q to quit--- > > Thread 18 (process 28206): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 17 (process 28205): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 16 (process 28204): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > ---Type to continue, or q to quit--- > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 15 (process 28203): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 14 (process 28202): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > ---Type to continue, or q to quit--- > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 13 (process 28201): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 12 (process 28200): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > ---Type to continue, or q to quit--- > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 11 (process 28199): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 10 (process 28198): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 9 (process 28197): > ---Type to continue, or q to quit--- > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 8 (process 28196): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 7 (process 28195): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > ---Type to continue, or q to quit--- > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 6 (process 28194): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 5 (process 28193): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > ---Type to continue, or q to quit--- > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 4 (process 28192): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 3 (process 28191): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x00000000004e583f in async_main (arg=) > at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > ---Type to continue, or q to quit--- > Thread 2 (process 28190): > #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 () > from /lib64/libpthread.so.0 > #1 0x000000000046b7bc in sys_msg_dispatcher_func ( > unused=) at beam/erl_threads.h:634 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > > Thread 1 (process 28189): > #0 0x0000003d05a0d2cb in read () from /lib64/libpthread.so.0 > #1 0x0000000000523f6d in signal_dispatcher_thread_func ( > unused=) at sys/unix/sys.c:2913 > #2 0x000000000057ca34 in thr_wrapper (vtwd=) > at common/ethread.c:475 > #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6 > (gdb) > (gdb) > (gdb) quit > > Abridged output from "cat /proc/cpuinfo": > > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 26 > model name : Intel(R) Xeon(R) CPU L5520 @ 2.27GHz > stepping : 5 > cpu MHz : 1596.000 > cache size : 8192 KB > physical id : 1 > siblings : 8 > core id : 0 > cpu cores : 4 > [...] > > > ________________________________________________________________ > erlang-bugs mailing list. See http://www.erlang.org/faq.html > erlang-bugs (at) erlang.org > > -------------- next part -------------- A non-text attachment was scrubbed... Name: otp_src_R13B02-1_OTP-8297.patch Type: text/x-patch Size: 602 bytes Desc: not available URL: From ulf.wiger@REDACTED Wed Nov 18 21:35:12 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 18 Nov 2009 21:35:12 +0100 Subject: [erlang-questions] Remote "access" points In-Reply-To: <635498b70911181042t13d7f42ex6417cd2bf97aea03@mail.gmail.com> References: <14571703.44041258539137048.JavaMail.root@zimbra> <22244220.44061258539242065.JavaMail.root@zimbra> <635498b70911180538m5b005c09jda4d3409dd037fa2@mail.gmail.com> <4B040252.4080709@erlang-consulting.com> <635498b70911181027s40bcdbfam5adb22b51144f052@mail.gmail.com> <20091118183247.GI4286@hijacked.us> <635498b70911181042t13d7f42ex6417cd2bf97aea03@mail.gmail.com> Message-ID: <4B045A80.1040303@erlang-consulting.com> Calum wrote: > On Wed, Nov 18, 2009 at 6:32 PM, Andrew Thompson wrote: > >> But its a message not a function call? Are you proposing to filter all >> messages from other nodes as well as all calls to rpc:call or spawn? > > Isn't it a message that requests a function gets executed? Somewhere > in the request, the module name must be in there. In a sense it is, but OTOH it is just a message. We happen to know in this case that the recipient of the message will interpret it as a request to call a particular function. We can also recognize the registered name. But if the message goes to <0.10.0> on the node in question - is it ok then? Clearly not, because in all Erlang nodes with a standard boot sequence, that /is/ the 'rex' process. It might also be that some application running in the system has some dispatcher pattern that, directly or indirectly, forwards any message to a given process. This could allow us to wrap the rpc request inside a dispatch request, and achieve the same effect anyway. We could also hunt down a supervisor and ask it to terminate its children - or send us a list of children so that we can get more information about ways to break the system. The possibilities are simply endless, if you allow users to connect using Distributed Erlang. It was designed for closed clusters, where all participants are trusted. You can provide a narrow interface by opening a socket. You can use something like UBF, CORBA, Thrift or HTTP, or why not write a BERT server in Erlang? http://github.com/blog/531-introducing-bert-and-bert-rpc BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ok@REDACTED Wed Nov 18 23:08:58 2009 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 19 Nov 2009 11:08:58 +1300 Subject: [erlang-questions] programming tip: a note on encapsulation In-Reply-To: <9b08084c0911180056l53ca0695r781656018d6f76a9@mail.gmail.com> References: <9b08084c0911180056l53ca0695r781656018d6f76a9@mail.gmail.com> Message-ID: <01699F7F-6016-432D-A59F-FCD72207CF00@cs.otago.ac.nz> On Nov 18, 2009, at 9:56 PM, Joe Armstrong wrote: > Here's a little programming technique I discovered yesterday, which I > rather like. It looks a whole lot like the way people used to do "objects" in T. (define (make-my-object ....) (lambda (Selector) (case Selector (field ....) (method (lambda (....) ....)) ....)))) with the equivalent of foo = new MyObject(...); foo.method(x, y) foo.field being (let ((Foo (make-my-object ....))) ((Foo 'method) X Y) (Foo 'field) It looks as though "Joe hates OO" may have to be rewritten! From kenneth.lundin@REDACTED Wed Nov 18 23:17:48 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 18 Nov 2009 23:17:48 +0100 Subject: [erlang-questions] Remote "access" points In-Reply-To: <4B045A80.1040303@erlang-consulting.com> References: <14571703.44041258539137048.JavaMail.root@zimbra> <22244220.44061258539242065.JavaMail.root@zimbra> <635498b70911180538m5b005c09jda4d3409dd037fa2@mail.gmail.com> <4B040252.4080709@erlang-consulting.com> <635498b70911181027s40bcdbfam5adb22b51144f052@mail.gmail.com> <20091118183247.GI4286@hijacked.us> <635498b70911181042t13d7f42ex6417cd2bf97aea03@mail.gmail.com> <4B045A80.1040303@erlang-consulting.com> Message-ID: > You can provide a narrow interface by opening a socket. > You can use something like UBF, CORBA, Thrift or HTTP, or why > not write a BERT server in Erlang? IThis is definately th easiest way to achieve what you are after. And if you are communicating between Erlang nodes you can of course use the erlang external format which is exactly what BERT does. It just prepends a header o the data. The other alternatives mentioned above is just more work and will not give you anything extra. /Kenneth Erlang/OTP Ericsson From jvliwanag@REDACTED Wed Nov 18 23:31:28 2009 From: jvliwanag@REDACTED (Jan Vincent) Date: Thu, 19 Nov 2009 06:31:28 +0800 Subject: Looking for Large Distributed Open Source Erlang OTP Project Message-ID: <5CAA10BB-D377-4C96-81E9-866C8F5C420F@gmail.com> Hi guys, I was wondering if there are any good large distributed open source Erlang application following OTP principles (with Applications and Releases, hot code upgrading, etc) that can be used a learning reference. I am looking for a whole system - not just one component (say, just the database part, etc). Most tutorials go on tackling small systems and don't really talk about making projects that take advantage of the whole OTP principles. Thanks, Jan Vincent Liwanag jvliwanag@REDACTED From jvliwanag@REDACTED Thu Nov 19 00:00:12 2009 From: jvliwanag@REDACTED (Jan Vincent) Date: Thu, 19 Nov 2009 07:00:12 +0800 Subject: [erlang-questions] Looking for Large Distributed Open Source Erlang OTP Project In-Reply-To: <4B04792E.3070709@gmail.com> References: <5CAA10BB-D377-4C96-81E9-866C8F5C420F@gmail.com> <4B04792E.3070709@gmail.com> Message-ID: <347C8C38-EDE2-4250-9D3E-A309EDD94AE7@gmail.com> As much as I respect very well the projects you've mentioned, they don't really follow the entirety of the OTP practices. They don't use OTP applications and releases. Thanks, Jan Vincent Liwanag jvliwanag@REDACTED On Nov 19, 2009, at 6:46 AM, Shelby Ramsey wrote: > You might want to take a look at ejabberd, couchdb, or even yaxe (a SIP server). > > SDR > > Jan Vincent wrote: >> Hi guys, >> >> I was wondering if there are any good large distributed open source Erlang application following OTP principles (with Applications and Releases, hot code upgrading, etc) that can be used a learning reference. I am looking for a whole system - not just one component (say, just the database part, etc). Most tutorials go on tackling small systems and don't really talk about making projects that take advantage of the whole OTP principles. >> >> Thanks, >> >> Jan Vincent Liwanag >> jvliwanag@REDACTED >> >> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> > From ok@REDACTED Thu Nov 19 00:04:42 2009 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 19 Nov 2009 12:04:42 +1300 Subject: No subject In-Reply-To: <8bzXghKw.1258539050.1327420.leap@gol.com> References: <8bzXghKw.1258539050.1327420.leap@gol.com> Message-ID: On Nov 18, 2009, at 11:10 PM, Michael Turner wrote: > -- so in a way, it almost comes down to the cost of copying. > Quicksort > is great, if you can do it in-place. But if you have to do O(n log n) > copies of the data, well .... It's actually O(log n). I came at this from another perspective, namely parallel programming. The old books go on at length about communication costs as well as computation costs. > > > The discussion here reminds me that issues like these are still > important > in optimization. Which means you should always be revisiting the > question: how important is optimization? The answer is "you never know until you have measured." > The last time I had any serious issue involving quicksort, it was > because > a junior software engineer, not realizing there was a UNIX library > function qsort(3), I teach (part of) a third year software engineering paper. (http://www.cs.otago.ac.nz/cosc345/ comments and suggestions always welcome) and "if you didn't write it you didn't wrong it" is a constant theme. > I really wish I was making this story up, but it did happen. > Premature > optimization truly is the root of all evil. Don't get me wrong, I > love > that Erlang two-liner version of quicksort. But it would be a service > to the world if, in future editions of books and documents mentioning > it, there was also a footnote pointing out that people should use > quicksort as an optimization, only if necessary, and that Erlang will > tend to execute this particular quicksort rather suboptimally. The > cool > thing about the two-liner is that it's a surprisingly clear expression > of sorting, not that it's a good implementation in any other respect. I'm glad you raised this, because there are several more issues. (1) The only known advantage of quicksort is that it can sort ARRAYS of NUMBERS without extra storage and quite quickly. It was known when it was invented that merge sort does fewer comparisons, so if the things you're sorting can't be compared in say under five machine instructions, you're probably better off with merge sort. If you are sorting LISTS, then you have already paid all the storage penalty needed by merge sort, and the space saving of quicksort has completely disappeared. (2) Quicksort has two known disadvantages. It isn't stable, which often doesn't matter, and it takes quadratic time in the worst case. Textbooks often say that the worst case is so unlikely you can forget about it, but it often crops up in practice. (There's a _reason_ the code I posted used a "fat pivot".) The Bentley and McIlroy "engineered" quicksort is now found in some C libraries, and it runs into this less often, but it can still happen. (3) The textbook algorithm for list quicksort is pretty much guaranteed to go quadratic a LOT. If the input is already in ascending or descending order, picking the first element is about the worst thing you can do. (4) The moral of your story as I see it is that it ALWAYS pays to look in the library and at least try things. Sorting 100,000 random floats on the slow machine, Deshpande's copy of the classic quicksort: 2.36 seconds Best sequential qsort posted yesterday: 1.86 seconds lists:sort/1 0.90 seconds Let's face it, "quicksort" is popular more because of its name than because of its merits. lists:sort/1 is a merge sort. > From g@REDACTED Thu Nov 19 01:03:41 2009 From: g@REDACTED (Garrett Smith) Date: Wed, 18 Nov 2009 18:03:41 -0600 Subject: [erlang-questions] Looking for Large Distributed Open Source Erlang OTP Project In-Reply-To: <5CAA10BB-D377-4C96-81E9-866C8F5C420F@gmail.com> References: <5CAA10BB-D377-4C96-81E9-866C8F5C420F@gmail.com> Message-ID: On Wed, Nov 18, 2009 at 4:31 PM, Jan Vincent wrote: > Hi guys, > > I was wondering if there are any good large distributed open source Erlang application following OTP principles (with Applications and Releases, hot code upgrading, etc) that can be used a learning reference. I am looking for a whole system - not just one component (say, just the database part, etc). Most tutorials go on tackling small systems and don't really talk about making projects that take advantage of the whole OTP principles. Excellent question! I haven't bumped into any Erlang projects that use releases, other than Erlang. I use them myself, but don't have a good feel for how a real-life upgrade cycle works with them. RabbitMQ uses OTP behaviors extensively. It has one app. No releases as far as I know. Garrett From kaiduanx@REDACTED Thu Nov 19 01:10:10 2009 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 18 Nov 2009 19:10:10 -0500 Subject: [erlang-questions] Looking for Large Distributed Open Source Erlang OTP Project In-Reply-To: References: <5CAA10BB-D377-4C96-81E9-866C8F5C420F@gmail.com> Message-ID: Ejabberd and YXA follow OTP application, and adding release handle to YXA/Ejabberd is not tough. kaiduan On Wed, Nov 18, 2009 at 7:03 PM, Garrett Smith wrote: > On Wed, Nov 18, 2009 at 4:31 PM, Jan Vincent wrote: >> Hi guys, >> >> I was wondering if there are any good large distributed open source Erlang application following OTP principles (with Applications and Releases, hot code upgrading, etc) that can be used a learning reference. I am looking for a whole system - not just one component (say, just the database part, etc). Most tutorials go on tackling small systems and don't really talk about making projects that take advantage of the whole OTP principles. > > Excellent question! > > I haven't bumped into any Erlang projects that use releases, other > than Erlang. I use them myself, but don't have a good feel for how a > real-life upgrade cycle works with them. > > RabbitMQ uses OTP behaviors extensively. It has one app. No releases > as far as I know. > > Garrett > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From g@REDACTED Thu Nov 19 01:31:08 2009 From: g@REDACTED (Garrett Smith) Date: Wed, 18 Nov 2009 18:31:08 -0600 Subject: [erlang-questions] why with Linux In-Reply-To: <7e9946a6-e350-4ada-83bc-a1ed0834f3fd@d9g2000prh.googlegroups.com> References: <7e9946a6-e350-4ada-83bc-a1ed0834f3fd@d9g2000prh.googlegroups.com> Message-ID: On Wed, Nov 18, 2009 at 12:41 PM, NavTux wrote: > Hi Erlangies, > > ? ? ? ? ? In my previous post i had come to know that there is a > difference to use erlang in windows rather than Linux peculiarly > ubuntu,Here I should come to know that In what way which occurs? > > ? ? ? ? I mean there are all IDE s are available both of > them,particularly Is any problem with windows related with erlang? > then I'm not aware of any major Erlang/Windows incompatibility issues, but I'm not spending *any* time on Windows with Erlang... This is because it's just a lot less painful for me to use Linux... This is mainly because of the widespread use of Windows incompatible makefiles in Erlang projects. In short, Windows is not a first-class citizen in the Erlang community. Or put more directly -- Windows requires an investment that most Erlang projects aren't willing to make. There are no doubt exceptions that I'm unaware of. I do know some hard core Erlang developers that use Windows to write code, but do no use Windows for end-to-end builds. The Eclipse plugin (erlide) could give some legs to Erlang development on Windows. I'm a big fan of taking the "path of least resistance" in tools -- I suspect you'll find that Windows introduces enough friction to your productivity that you might consider moving your dev environment to Linux. Any Windows champions want to take a crack at this? :) From steven.charles.davis@REDACTED Thu Nov 19 01:49:19 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 18 Nov 2009 16:49:19 -0800 (PST) Subject: why with Linux In-Reply-To: References: <7e9946a6-e350-4ada-83bc-a1ed0834f3fd@d9g2000prh.googlegroups.com> Message-ID: <9de176fe-891f-4931-88ac-b034dc5d9ad6@e23g2000yqd.googlegroups.com> On Nov 18, 6:31 pm, Garrett Smith wrote: > Any Windows champions want to take a crack at this? :) I use Erlang on both linux and windows depending on what machine I'm using at the time. I have no problems on either environment, and favor neither for development (though linux is a clear favorite for deployment). It's handy on both platforms to be able to write your own shell/batch scripts. This skill is probably more important on windows because (as you are no doubt aware), in most open source libraries (not all) the developer will more likely provide you with useful bash/shell scripts than (windows) batch scripts. /s From john@REDACTED Thu Nov 19 01:59:10 2009 From: john@REDACTED (John Kemp) Date: Wed, 18 Nov 2009 19:59:10 -0500 Subject: [erlang-questions] why with Linux In-Reply-To: References: <7e9946a6-e350-4ada-83bc-a1ed0834f3fd@d9g2000prh.googlegroups.com> Message-ID: <4B04985E.80100@jkemp.net> Garrett Smith wrote: > On Wed, Nov 18, 2009 at 12:41 PM, NavTux wrote: >> Hi Erlangies, >> >> In my previous post i had come to know that there is a >> difference to use erlang in windows rather than Linux peculiarly >> ubuntu,Here I should come to know that In what way which occurs? >> >> I mean there are all IDE s are available both of >> them,particularly Is any problem with windows related with erlang? >> then > > I'm not aware of any major Erlang/Windows incompatibility issues, I haven't checked recently, but it used to be the case (R11/12?) that the Erlang API for making symbolic links didn't work on Windows because Windows doesn't (didn't?) support Unix-style links in the fs. Regards, - johnk > but > I'm not spending *any* time on Windows with Erlang... From ngocdaothanh@REDACTED Thu Nov 19 02:05:25 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Thu, 19 Nov 2009 10:05:25 +0900 Subject: [erlang-questions] why with Linux In-Reply-To: <7e9946a6-e350-4ada-83bc-a1ed0834f3fd@d9g2000prh.googlegroups.com> References: <7e9946a6-e350-4ada-83bc-a1ed0834f3fd@d9g2000prh.googlegroups.com> Message-ID: <5c493e530911181705j119a2578x6d837a4a843f9b15@mail.gmail.com> > Is any problem with windows related with erlang? In general, for the development of non-Windows-specific applications (web, network etc. applications), Windows is the worst environment and Linux is the best one. So it is your problem if you use the worst one. Personally I don't see any value of Windows development environment other than Visual Studio. If I don't use Visual Studio, then I don't use Windows. On Thu, Nov 19, 2009 at 3:41 AM, NavTux wrote: > Hi Erlangies, > > ? ? ? ? ? In my previous post i had come to know that there is a > difference to use erlang in windows rather than Linux peculiarly > ubuntu,Here I should come to know that In what way which occurs? > > ? ? ? ? I mean there are all IDE s are available both of > them,particularly Is any problem with windows related with erlang? > then > > ? ? ? How can i feel the difference between them? From fritchie@REDACTED Thu Nov 19 03:56:21 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 18 Nov 2009 20:56:21 -0600 Subject: [erlang-questions] R13B02 on 8/16 core box: all TCP communication hangs/frozen In-Reply-To: Message of "Wed, 18 Nov 2009 15:28:18 +0100." <1d0fb8fc0911180628p35864ec1ob843084c0ff625d@mail.gmail.com> Message-ID: <8925.1258599381@snookles.snookles.com> Luke Gorrie wrote: lg> I suppose you knew the risk of idle speculation from the peanut lg> gallery when you posted this to erlang-questions.. Yes, yes, though perhaps posting only an abridged message here on erlang-questions was a mistake? {shrug} lg> Can you clarify the timeline? First event (time T0) is foo@REDACTED being lg> detected as "not responding" by its peers (i.e. doesn't answer TCP lg> heartbeat) and second event is seeing that the emulator is totally lg> wedged (time T1). Is it two days between T0 and T1? More like 2 seconds, or less maybe. lg> lsof shows not many open files / nothing scary. ... except that it should have had many more. The weird thing is that "lsof" didn't know how to identify the last dozen or so fds. There isn't an "lsof" package vs. kernel mismatch, I wonder if the app did a legitimate system call (but not close(2)) that managed to put those fds in a state where "lsof" can't decypher the kernel's data structures. lg> gdb shows the first thread in select() and the rest blocking on a lg> pthreads event. Consistent with (a) normal operation (b) deadlock lg> inside the emulator (c) screwed up file descriptor set or timeout in lg> select(). I suppose (c) at least could be ruled out with gdb'ery. Many thanks to Rickard Green for the wondrously quick patch. We'll start using it today, though it has taken a week or more for a single occurrance of this bug (or, rather, what we now suspect are all manifestations of the same thing). lg> But it seems like the most interesting tidbit is: lg> Are you saying that it sometimes takes several seconds to establish lg> a loopback socket connection from telnet or netcat? That sounds lg> extremely fishy! [...] Yes, I agree with your blame pointed at the kernel, but ... those sockets have a listen backlog of 4096. Strace shows no VM activity in any thread when a new TCP connection is actually established. My guess is that when the app does not bother calling accept(2), and then the listen backlog queue is full, the kernel will wait until one of the existing 4096 TCP connections leaves CLOSE_WAIT state, which then leaves an open slot in the listen queue? lg> Can the kernel possibly be very busy? (I watch 'vmstat 1' to check.) Somewhat busy, but not with any activity for the VM in question. No shortage of OS file descriptors, OS process slots, etc. For a little while, we suspected that the problem may have been within the VM, hitting the Erlang process limit. We use the default of 32K procs. But when that's happened in the past, we've still been able to scribble a log message via our custom gen_event handler for the SASL event logger. That handler doesn't keep a file handle open all the time, instead it opens, writes, closes for each event log entry. But we're opening the file handle in raw mode, so unless we managed to hit the max # of ports limit, we oughtta be OK. (And again, past experience suggests that we can still write the error message to the file reliably, or at least with high probability of success.) A peek with GDB at the # of Erlang processes, showed "142" so ... the too-many-procs theory was flawed. We also experimented with IP chains, to see if preventing an external TCP load balancer from tickling certain TCP ports every few seconds. But after waiting about 10 minutes with no change in VM behavior, we gave up. -Scott From erlang@REDACTED Thu Nov 19 08:59:38 2009 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 19 Nov 2009 08:59:38 +0100 Subject: [erlang-questions] programming tip: a note on encapsulation In-Reply-To: <01699F7F-6016-432D-A59F-FCD72207CF00@cs.otago.ac.nz> References: <9b08084c0911180056l53ca0695r781656018d6f76a9@mail.gmail.com> <01699F7F-6016-432D-A59F-FCD72207CF00@cs.otago.ac.nz> Message-ID: <9b08084c0911182359q39be28e7q5f925055124d8606@mail.gmail.com> I hadn't thought of it that way. This reminds me of Javascript. I define an object constructor make_my_thing(Data) -> fun(data) -> Data; (f1) -> fun(X) -> do_something(X, Data) end; (f2) -> fun(X, Y) -> do_something_else(X, Y, Data) end end. Then I can say F = make_my_thing(...) F(f1)(a, b) F(data) ... I could make mutable objects with: make_another_object(Data) -> Pid = spawn(fun() -> .... end), F = fun({call,X}) -> rpc(pid, X); (...) -> end. On Wed, Nov 18, 2009 at 11:08 PM, Richard O'Keefe wrote: > > On Nov 18, 2009, at 9:56 PM, Joe Armstrong wrote: > >> Here's a little programming technique I discovered yesterday, which I >> rather like. > > It looks a whole lot like the way people used to do "objects" in T. > ? (define (make-my-object ....) > ? ? (lambda (Selector) > ? ? ? (case Selector > ? ? ? ? (field ....) > ? ? ? ? (method (lambda (....) ....)) > ? ? ? ? ....)))) > with the equivalent of > ? ?foo = new MyObject(...); > ? ?foo.method(x, y) > ? ?foo.field > being > ? ?(let ((Foo (make-my-object ....))) > ? ? ?((Foo 'method) X Y) > ? ? ?(Foo 'field) > > It looks as though "Joe hates OO" may have to be rewritten! > > > > From zambal@REDACTED Thu Nov 19 09:28:57 2009 From: zambal@REDACTED (zambal) Date: Thu, 19 Nov 2009 00:28:57 -0800 (PST) Subject: programming tip: a note on encapsulation In-Reply-To: <9b08084c0911182359q39be28e7q5f925055124d8606@mail.gmail.com> References: <9b08084c0911180056l53ca0695r781656018d6f76a9@mail.gmail.com> <01699F7F-6016-432D-A59F-FCD72207CF00@cs.otago.ac.nz> <9b08084c0911182359q39be28e7q5f925055124d8606@mail.gmail.com> Message-ID: On 19 nov, 08:59, Joe Armstrong wrote: > Then I can say > > ? ?F = make_my_thing(...) > ? ?F(f1)(a, b) > ? ?F(data) ... Unfortunately F(f1)(a, b) doesn't parse. The only way to make that work is to write it as (F(f1))(a, b), which is horrible to read. I used to experiment a lot with using funs as kind of objects in Erlang, but in the end I always stopped using these techniques because of this superficial notation thing. Would it be difficult to change the parser rules to make a call like F(f1)(a, b) valid, or would that conflict too much with other (scoping) rules? - vincent From erlang@REDACTED Thu Nov 19 10:55:25 2009 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 19 Nov 2009 10:55:25 +0100 Subject: edoc, erlc and dialyser type spec consistency Message-ID: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> I'm trying to document my new library - *beautifully* - so I want to use edoc, the dialyzer etc. and anything I can throw at it to improve the code, but I'm confused about type specifications. The type specifications in edoc and erlc violate the principle of least astonishment, I haven't checked with the dialyzer. Question 1) Should type specifications end with DOT Whitespace? erlc says YES, edoc says NO -spec foo(In::string()) -> Out::string(). is legal in erlc %% @spec foo(In::string()) -> Out::string(). gives an error in edoc Question 2) Are type specifications without a function name symbol legal? The Edoc manual says the function name is optional. %% @spec (In::string()) -> Out::string() foo(X) -> Y. is *legal* in edoc and the function name is picked up from the *next* definition. But: -spec (In::string()) -> Out::string(). is *illegal* in erlc Question 3) What's up with atoms? A quick grep though stdlib shows that all atoms in type specifications are quoted - why is thi? Why is the convention for *atoms in code* different to *atoms in type specifications*???? Question 4) What's wrong with bool() - why is the name changed to boolean()? My suggestion: a) There should be ONE type/spec parser as part of the OTP distribution everybody should use this one and not their own b) Specs and types must finish with DOT WHITESPACE. c) Quoting atoms in type specs is illegal. d) Function names are required (ie not optional as in edoc) e) Short form of type names are legal bool(), int(). There rules obey certain meta principles: meta principle 1) "don't wear your fingers out" - I want to type foo(X::bool()) -> ok. and not foo(X::boolean()) -> 'ok' meta principle 2) "optional is bad" ie DOT whitespace is not optional - dropping the function symbol and inferring it from the next function is bad. (aside: the optional ';' in Javascript totally screwed up the parser and is responsible for lager numbers or really weird errors) meta principle 3) "make type specs etc look like Erlang" Similar syntax if possible, similar conventions. The sudden appearance of quotes, optional '.' violates the principle of least astonishment. /Joe From kiran.khaladkar@REDACTED Thu Nov 19 13:14:23 2009 From: kiran.khaladkar@REDACTED (kiran) Date: Thu, 19 Nov 2009 15:14:23 +0300 Subject: compiler bug Message-ID: <4B05369F.9020305@geodesic.com> hi, I am getting the following error when i compile the attached code erlc im_session.erl im_session: function process_request/5+3991: Internal consistency check failed - please report this bug. Instruction: {test_heap,6,3} Error: {{x,0},not_live}: ./im_session.erl:277: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:283: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:443: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:449: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:627: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:633: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:659: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:664: Warning: regexp:split/2: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:706: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:743: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:1251: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:1328: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:1412: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:1494: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead ./im_session.erl:1519: Warning: regexp:gsub/3: the regexp module is deprecated (will be removed in R15A); use the re module instead Details of my erlang installation are erlang:system_info(system_version). "Erlang R13B02 (erts-5.7.3) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]\n" compile:module_info(compile). [{options,[{cwd,"/net/isildur/ldisk/daily_build/otp_prebuild_r13b02.2009-09-21_11/otp_src_R13B02/lib/compiler/src"}, {outdir,"/net/isildur/ldisk/daily_build/otp_prebuild_r13b02.2009-09-21_11/otp_src_R13B02/lib/compiler/src/../ebin"}, {i,"/net/isildur/ldisk/daily_build/otp_prebuild_r13b02.2009-09-21_11/otp_src_R13B02/lib/compiler/src/../../stdlib/include"}, {i,"/net/isildur/ldisk/daily_build/otp_prebuild_r13b02.2009-09-21_11/otp_src_R13B02/lib/compiler/src/."}, warn_unused_import,inline,debug_info]}, {version,"4.6.3"}, {time,{2009,9,21,10,47,27}}, {source,"/net/isildur/ldisk/daily_build/otp_prebuild_r13b02.2009-09-21_11/otp_src_R13B02/lib/compiler/src/compile.erl"}] -- Kiran Khaladkar | Software Engineer Geodesic Limited | www.geodesic.com Tel: +91 22 4031 5800 - ext - 225 Mob: 9870587508 -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: im_session.erl URL: From bgustavsson@REDACTED Thu Nov 19 11:18:31 2009 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 19 Nov 2009 11:18:31 +0100 Subject: [erlang-questions] compiler bug In-Reply-To: <4B05369F.9020305@geodesic.com> References: <4B05369F.9020305@geodesic.com> Message-ID: <6672d0160911190218p3e374859h8be216cfd019fa5f@mail.gmail.com> On Thu, Nov 19, 2009 at 1:14 PM, kiran wrote: > hi, > > I am getting the following error when i compile the attached code > > erlc im_session.erl > im_session: function process_request/5+3991: > Internal consistency check failed - please report this bug. > Instruction: {test_heap,6,3} > Error: {{x,0},not_live}: Thanks for the bug report, but to be able to investigate the problem I'll need compilable source code. If header.hrl is the only file missing, you can send that. Otherwise I suggest that you'll do: erlc -P im_session.erl and send the resulting im_session.P file. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From kostis@REDACTED Thu Nov 19 11:16:14 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 19 Nov 2009 12:16:14 +0200 Subject: edoc, erlc and dialyser type spec consistency In-Reply-To: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> References: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> Message-ID: <4B051AEE.6080902@cs.ntua.gr> Joe Armstrong wrote: > I'm trying to document my new library - *beautifully* - so > I want to use edoc, the dialyzer etc. and anything I can throw at it > to improve the code, but I'm confused about type specifications. > > The type specifications in edoc and erlc violate the principle of > least astonishment, I haven't checked with the dialyzer. > You do not have to check this. I can assure you that Dialyzer uses the same specs as erlc. > Question 1) > > Should type specifications end with DOT Whitespace? > > erlc says YES, edoc says NO > > -spec foo(In::string()) -> Out::string(). is legal in erlc > %% @spec foo(In::string()) -> Out::string(). gives an error in edoc > > Question 2) > > Are type specifications without a function name symbol legal? > > The Edoc manual says the function name is optional. > > %% @spec (In::string()) -> Out::string() > > foo(X) -> Y. > > is *legal* in edoc and the function name is picked up from the > *next* definition. But: > > -spec (In::string()) -> Out::string(). > > is *illegal* in erlc > > Question 3) > > What's up with atoms? > > A quick grep though stdlib shows that all atoms in type specifications > are quoted - why is thi? Why is the convention for *atoms in code* > different to *atoms in type specifications*???? > As you point out, quoting atoms in specs is optional. It's equally optional in code, so I do not see any discrepancy or reason for astonishment here. I've personally adopted a habit to quote atoms in specs, because I've been bitten one too many from wanting to write e.g. atom() but written atom. Perhaps other users are more careful and disciplined than I am, so they do not need this. But it does not hurt to express intention that I want the 'ok' atom here and the ok() type there. > Question 4) > > What's wrong with bool() - why is the name changed to boolean()? > This was a conscious decision to have all base types match the corresponding type tests/guards. There is no is_bool/1 test, the BIF is called is_boolean/1. Similarly, quite long ago ref() was renamed to reference(). > My suggestion: > > a) There should be ONE type/spec parser as part of the OTP distribution > everybody should use this one and not their own > > b) Specs and types must finish with DOT WHITESPACE. > > c) Quoting atoms in type specs is illegal. > > d) Function names are required (ie not optional as in edoc) > > e) Short form of type names are legal > bool(), int(). > I very much agree with (a), (b) and (d). I strongly disagree with (c) because the corresponding action in code is legal. Why should it be illegal in specs? As for (e), it's currently very simple for the user to include type alias definitions as in: -type int() :: integer(). But if there is a strong demand, we can add hard-coded such declarations in R13B04. Kostis From kostis@REDACTED Thu Nov 19 11:17:36 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 19 Nov 2009 12:17:36 +0200 Subject: edoc, erlc and dialyser type spec consistency In-Reply-To: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> References: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> Message-ID: <4B051B40.1030007@cs.ntua.gr> Joe Armstrong wrote: > I'm trying to document my new library - *beautifully* - so > I want to use edoc, the dialyzer etc. and anything I can throw at it > to improve the code, but I'm confused about type specifications. > > The type specifications in edoc and erlc violate the principle of > least astonishment, I haven't checked with the dialyzer. > You do not have to check this. I can assure you that Dialyzer uses the same specs as erlc. > Question 1) > > Should type specifications end with DOT Whitespace? > > erlc says YES, edoc says NO > > -spec foo(In::string()) -> Out::string(). is legal in erlc > %% @spec foo(In::string()) -> Out::string(). gives an error in edoc > > Question 2) > > Are type specifications without a function name symbol legal? > > The Edoc manual says the function name is optional. > > %% @spec (In::string()) -> Out::string() > > foo(X) -> Y. > > is *legal* in edoc and the function name is picked up from the > *next* definition. But: > > -spec (In::string()) -> Out::string(). > > is *illegal* in erlc > > Question 3) > > What's up with atoms? > > A quick grep though stdlib shows that all atoms in type specifications > are quoted - why is thi? Why is the convention for *atoms in code* > different to *atoms in type specifications*???? > As you point out, quoting atoms in specs is optional. It's equally optional in code, so I do not see any discrepancy or reason for astonishment here. I've personally adopted a habit to quote atoms in specs, because I've been bitten one too many from wanting to write e.g. atom() but written atom. Perhaps other users are more careful and disciplined than I am, so they do not need this. But it does not hurt to express intention that I want the 'ok' atom here and the ok() type there. > Question 4) > > What's wrong with bool() - why is the name changed to boolean()? > This was a conscious decision to have all base types match the corresponding type tests/guards. There is no is_bool/1 test, the BIF is called is_boolean/1. Similarly, quite long ago ref() was renamed to reference(). > My suggestion: > > a) There should be ONE type/spec parser as part of the OTP distribution > everybody should use this one and not their own > > b) Specs and types must finish with DOT WHITESPACE. > > c) Quoting atoms in type specs is illegal. > > d) Function names are required (ie not optional as in edoc) > > e) Short form of type names are legal > bool(), int(). > I very much agree with (a), (b) and (d). I strongly disagree with (c) because the corresponding action in code is legal. Why should it be illegal in specs? As for (e), it's currently very simple for the user to include type alias definitions as in: -type int() :: integer(). But if there is a strong demand, we can add hard-coded such declarations in R13B04. Kostis From kostis@REDACTED Thu Nov 19 11:22:47 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 19 Nov 2009 12:22:47 +0200 Subject: edoc, erlc and dialyser type spec consistency In-Reply-To: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> References: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> Message-ID: <4B051C77.2060505@cs.ntua.gr> Joe Armstrong wrote: > I'm trying to document my new library - *beautifully* - so > I want to use edoc, the dialyzer etc. and anything I can throw at it > to improve the code, but I'm confused about type specifications. > > The type specifications in edoc and erlc violate the principle of > least astonishment, I haven't checked with the dialyzer. > You do not have to check this. dialyzer uses the same spec language as erlc. > Question 1) > > Should type specifications end with DOT Whitespace? > > erlc says YES, edoc says NO > > -spec foo(In::string()) -> Out::string(). is legal in erlc > %% @spec foo(In::string()) -> Out::string(). gives an error in edoc > > Question 2) > > Are type specifications without a function name symbol legal? > > The Edoc manual says the function name is optional. > > %% @spec (In::string()) -> Out::string() > > foo(X) -> Y. > > is *legal* in edoc and the function name is picked up from the > *next* definition. But: > > -spec (In::string()) -> Out::string(). > > is *illegal* in erlc > By the way, this happens because -specs can appear /anywhere/ in the file, while Edoc @specs *must* appear immediately before the corresponding function definitions, so it's easy for Edoc to fill in the missing function name. > Question 3) > > What's up with atoms? > > A quick grep though stdlib shows that all atoms in type specifications > are quoted - why is thi? Why is the convention for *atoms in code* > different to *atoms in type specifications*???? > As you point out, quoting atoms in specs is optional. It's equally optional in code, so I do not see any discrepancy or reason for astonishment here. I've personally adopted a habit to quote atoms in specs, because I've been bitten one too many from wanting to write e.g. atom() but written atom. Perhaps other users are more careful and disciplined than I am, so they do not need this. But it does not hurt to express intention that I want the 'ok' atom here and the ok() type there. > Question 4) > > What's wrong with bool() - why is the name changed to boolean()? > This was a conscious decision to have all base types match the corresponding type tests/guards. There is no is_bool/1 test, the BIF is called is_boolean/1. Similarly, quite long ago ref() was renamed to reference(). > My suggestion: > > a) There should be ONE type/spec parser as part of the OTP distribution > everybody should use this one and not their own > > b) Specs and types must finish with DOT WHITESPACE. > > c) Quoting atoms in type specs is illegal. > > d) Function names are required (ie not optional as in edoc) > > e) Short form of type names are legal > bool(), int(). > I very much agree with (a), (b) and (d). I strongly disagree with (c) because the corresponding action in code is legal. Why should it be illegal in specs? As for (e), it's currently very simple for the user to include type alias definitions as in: -type int() :: integer(). But if there is a strong demand, we can add hard-coded such declarations in R13B04. Kostis From kostis@REDACTED Thu Nov 19 11:24:01 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 19 Nov 2009 12:24:01 +0200 Subject: edoc, erlc and dialyser type spec consistency In-Reply-To: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> References: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> Message-ID: <4B051CC1.1090006@it.uu.se> Joe Armstrong wrote: > I'm trying to document my new library - *beautifully* - so > I want to use edoc, the dialyzer etc. and anything I can throw at it > to improve the code, but I'm confused about type specifications. > > The type specifications in edoc and erlc violate the principle of > least astonishment, I haven't checked with the dialyzer. > You do not have to check this. dialyzer uses the same spec language as erlc. > Question 1) > > Should type specifications end with DOT Whitespace? > > erlc says YES, edoc says NO > > -spec foo(In::string()) -> Out::string(). is legal in erlc > %% @spec foo(In::string()) -> Out::string(). gives an error in edoc > > Question 2) > > Are type specifications without a function name symbol legal? > > The Edoc manual says the function name is optional. > > %% @spec (In::string()) -> Out::string() > > foo(X) -> Y. > > is *legal* in edoc and the function name is picked up from the > *next* definition. But: > > -spec (In::string()) -> Out::string(). > > is *illegal* in erlc > By the way, this happens because -specs can appear /anywhere/ in the file, while Edoc @specs *must* appear immediately before the corresponding function definitions, so it's easy for Edoc to fill in the missing function name. > Question 3) > > What's up with atoms? > > A quick grep though stdlib shows that all atoms in type specifications > are quoted - why is thi? Why is the convention for *atoms in code* > different to *atoms in type specifications*???? > As you point out, quoting atoms in specs is optional. It's equally optional in code, so I do not see any discrepancy or reason for astonishment here. I've personally adopted a habit to quote atoms in specs, because I've been bitten one too many from wanting to write e.g. atom() but written atom. Perhaps other users are more careful and disciplined than I am, so they do not need this. But it does not hurt to express intention that I want the 'ok' atom here and the ok() type there. > Question 4) > > What's wrong with bool() - why is the name changed to boolean()? > This was a conscious decision to have all base types match the corresponding type tests/guards. There is no is_bool/1 test, the BIF is called is_boolean/1. Similarly, quite long ago ref() was renamed to reference(). > My suggestion: > > a) There should be ONE type/spec parser as part of the OTP distribution > everybody should use this one and not their own > > b) Specs and types must finish with DOT WHITESPACE. > > c) Quoting atoms in type specs is illegal. > > d) Function names are required (ie not optional as in edoc) > > e) Short form of type names are legal > bool(), int(). > I very much agree with (a), (b) and (d). I strongly disagree with (c) because the corresponding action in code is legal. Why should it be illegal in specs? As for (e), it's currently very simple for the user to include type alias definitions as in: -type int() :: integer(). But if there is a strong demand, we can add hard-coded such declarations in R13B04. Kostis From roberto.aloi@REDACTED Thu Nov 19 12:00:49 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Thu, 19 Nov 2009 11:00:49 +0000 Subject: [erlang-questions] Remote "access" points In-Reply-To: <4B043DE9.3030501@erlang-consulting.com> References: <14571703.44041258539137048.JavaMail.root@zimbra> <22244220.44061258539242065.JavaMail.root@zimbra> <635498b70911180538m5b005c09jda4d3409dd037fa2@mail.gmail.com> <4B040252.4080709@erlang-consulting.com> <635498b70911181027s40bcdbfam5adb22b51144f052@mail.gmail.com> <4B043DE9.3030501@erlang-consulting.com> Message-ID: <4B052561.6070906@erlang-consulting.com> Hi again, > The problem is that you open up a huge security hole if you > allow messages to be sent by default to any process and then > try to filter out only those that you know could be dangerous. > > I think the only sensible default if you want to make it safe, > is to deny everything you don't know to be safe. I strongly agree on this. Be conservative. Always. Erlang allow just all-or-nothing security for the nodes. Is there any specific reason you can't have a "public" node and a "restricted" one for your functionalities? Btw, the following article discuss how to secure Distributed Erlang Systems: http://dizzyd.com/sdist.pdf Regards, Roberto Aloi (@prof3ta on twitter) Erlang Training and Consulting Ltd. http://www.erlang-consulting.com From kenneth.lundin@REDACTED Thu Nov 19 13:02:44 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Thu, 19 Nov 2009 13:02:44 +0100 Subject: [erlang-questions] edoc, erlc and dialyser type spec consistency In-Reply-To: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> References: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> Message-ID: I agree with Kostis answers and can also inform you that we are working on a solution for a) below. The plan is: Edoc will then parse and use the same -spec and -type as Dialyzer. If a file contains both -spec/-type and edoc comments with @spec and @type you will have to choose which notation to use (as a compatiblity mode) for already existing edoc markup. /Kenneth Erlang/OTP Ericsson > My suggestion: > > ? ?a) There should be ONE type/spec parser as part of the OTP distribution > ? ? ? everybody should use this one and not their own > > ? ?b) Specs and types must finish with DOT WHITESPACE. > > ? ?c) Quoting atoms in type specs is illegal. > > ? ?d) Function names are required (ie not optional as in edoc) > > ? ?e) Short form of type names are legal > ? ? ? bool(), int(). > > There rules obey certain meta principles: > > ? ?meta principle 1) "don't wear your fingers out" > > ? - I want to type > > ? ? ? ?foo(X::bool()) -> ok. > > ? ?and not > > ? ? ? ?foo(X::boolean()) -> 'ok' > > ? ?meta principle 2) "optional is bad" > > ? ? ? ?ie DOT whitespace is not optional - dropping the function > ? ? ? ?symbol and inferring it from the next function is bad. > > ? ? ? ?(aside: the optional ';' in Javascript totally screwed up > ? ? ? ? ? ? ? ?the parser and is responsible for lager numbers > ? ? ? ? ? ? ? ?or really weird errors) > > ? ?meta principle 3) "make type specs etc look like Erlang" > > ? ?Similar syntax if possible, similar conventions. The sudden appearance > ? ?of quotes, optional '.' violates the principle of least astonishment. > > > /Joe > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From erlang@REDACTED Thu Nov 19 13:53:34 2009 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 19 Nov 2009 13:53:34 +0100 Subject: [erlang-questions] edoc, erlc and dialyser type spec consistency In-Reply-To: References: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> Message-ID: <9b08084c0911190453o41e5b560h2d5226aa690b24d3@mail.gmail.com> On Thu, Nov 19, 2009 at 1:02 PM, Kenneth Lundin wrote: > I agree with Kostis answers and can also inform you that we are working > on a solution for a) below. > > The plan is: > Edoc will then parse and use the same -spec and -type as Dialyzer. > If a file contains both -spec/-type and edoc comments with @spec and @type > you will have to choose which notation to use (as a compatiblity mode) for > already existing edoc markup. Excellent. One more question: - does the dialyzer pick up @specs from edoc comments if there are no -spec's in the code? /Joe > > /Kenneth Erlang/OTP Ericsson > >> My suggestion: >> >> ? ?a) There should be ONE type/spec parser as part of the OTP distribution >> ? ? ? everybody should use this one and not their own >> >> ? ?b) Specs and types must finish with DOT WHITESPACE. >> >> ? ?c) Quoting atoms in type specs is illegal. >> >> ? ?d) Function names are required (ie not optional as in edoc) >> >> ? ?e) Short form of type names are legal >> ? ? ? bool(), int(). >> >> There rules obey certain meta principles: >> >> ? ?meta principle 1) "don't wear your fingers out" >> >> ? - I want to type >> >> ? ? ? ?foo(X::bool()) -> ok. >> >> ? ?and not >> >> ? ? ? ?foo(X::boolean()) -> 'ok' >> >> ? ?meta principle 2) "optional is bad" >> >> ? ? ? ?ie DOT whitespace is not optional - dropping the function >> ? ? ? ?symbol and inferring it from the next function is bad. >> >> ? ? ? ?(aside: the optional ';' in Javascript totally screwed up >> ? ? ? ? ? ? ? ?the parser and is responsible for lager numbers >> ? ? ? ? ? ? ? ?or really weird errors) >> >> ? ?meta principle 3) "make type specs etc look like Erlang" >> >> ? ?Similar syntax if possible, similar conventions. The sudden appearance >> ? ?of quotes, optional '.' violates the principle of least astonishment. >> >> >> /Joe >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > From dionne@REDACTED Tue Nov 17 22:28:42 2009 From: dionne@REDACTED (Robert Dionne) Date: Tue, 17 Nov 2009 16:28:42 -0500 Subject: [erlang-questions] Erlang IDE In-Reply-To: References: Message-ID: <272DDCB1-4D05-48A3-B20F-7F06902FF7D7@dionne-associates.com> +1 for Emacs+Distel. If you're already an emacs user this combo is essentially an IDE, "who calls", symbolic debugging, etc.... On Nov 17, 2009, at 3:01 PM, Garrett Smith wrote: > On Tue, Nov 17, 2009 at 11:08 AM, NavTux > wrote: >> I need good interactive IDE for erlang for both windows and >> linux ,which one is best to work with erlang? >> It ll be good one if the IDE functions like Netbeans that means If >> any >> error happens it shows the tip to rectify > > Have you looked at erlIDE? > > http://erlide.sourceforge.net > > This is probably your best bet, though when I looked at it (a few > months ago), it was rough around the edges. > > Just to give you an idea how limited the Erlang IDE story is, I > switched from Vim to Emacs as my main source editor in order to take > advantages of its Erlang support. IMO, if you're going to invest a > *lot* of time in Erlang, you'll want to consider a switch as well. > Unfortunately, that's a huge transition of you do your work in > Netbeans. Nevertheless, Emacs is the path of least resistance for > productive development in Erlang. > > Another point to bear in mind... most Erlang projects use automake > tools for builds... not a happy place for Windows development. I've > restored to run virtualized Ubuntu on my Windows machines to do any > Erlang development. Again, path of least resistance. You can certainly > use Windows, but you're going to hit bumps along the way. > > Garrett > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From erlang@REDACTED Thu Nov 19 15:06:50 2009 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 19 Nov 2009 15:06:50 +0100 Subject: couchdb in Karmic Koala Message-ID: <9b08084c0911190606r6bfb7505lcd1d0d773f0a8210@mail.gmail.com> I just upgraded my ubuntu to Karmic Koala couchdb seems to have installed itself and a few other things (horray - well done ...) $ pwd /usr/lib/couchdb/erlang/lib $ ls couch-0.10.0 erlang-oauth etap ibrowse-1.5.2 mochiweb-r97 I think couchdb is running on my machine. A few questions: 1) Is couchdb running on my machine and how can I confirm this? 2) How is couchdb started and stopped? 3) Can I use couchdb for my own applications of is it reserved for system use? 4) Where is Erlang hidden away? 5) Can I use the hidden Erlang for my own applications 6) Can I distribute my own Erlang applications that make use of the Erlang that has (apparently) been installed in Karmic Koala Having Erlang on all Karmic Koala machines could lead to many exciting things :-) /Joe From rumata-estor@REDACTED Thu Nov 19 15:26:46 2009 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 19 Nov 2009 17:26:46 +0300 Subject: [erlang-questions] couchdb in Karmic Koala In-Reply-To: <9b08084c0911190606r6bfb7505lcd1d0d773f0a8210@mail.gmail.com> References: <9b08084c0911190606r6bfb7505lcd1d0d773f0a8210@mail.gmail.com> Message-ID: <1258640806.17050.14.camel@rumbuntu> 1) ps -A | grep couchdb or 2) /etc/init.d/couchdb {start|stop|status} 3) sure you can if you database name will not interfere any other As I remember couchdb package depends on erlang. So it may be installed as always in /usr/bin and /usr/lib/erlang On Thu, 2009-11-19 at 15:06 +0100, Joe Armstrong wrote: > I just upgraded my ubuntu to Karmic Koala > > couchdb seems to have installed itself and a few other things (horray > - well done ...) > > $ pwd > /usr/lib/couchdb/erlang/lib > $ ls > couch-0.10.0 erlang-oauth etap ibrowse-1.5.2 mochiweb-r97 > > I think couchdb is running on my machine. > > A few questions: > > 1) Is couchdb running on my machine and how can I confirm this? > 2) How is couchdb started and stopped? > 3) Can I use couchdb for my own applications of is it reserved for system use? > 4) Where is Erlang hidden away? > 5) Can I use the hidden Erlang for my own applications > 6) Can I distribute my own Erlang applications that make use of the > Erlang that has (apparently) been installed in Karmic Koala > > Having Erlang on all Karmic Koala machines could lead to many exciting > things :-) > > /Joe > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From luke@REDACTED Thu Nov 19 15:35:00 2009 From: luke@REDACTED (Luke Gorrie) Date: Thu, 19 Nov 2009 15:35:00 +0100 Subject: [erlang-questions] R13B02 on 8/16 core box: all TCP communication hangs/frozen In-Reply-To: <8925.1258599381@snookles.snookles.com> References: <1d0fb8fc0911180628p35864ec1ob843084c0ff625d@mail.gmail.com> <8925.1258599381@snookles.snookles.com> Message-ID: <1d0fb8fc0911190635p5adbeb1em5111f06d1ecb0a6d@mail.gmail.com> 2009/11/19 Scott Lystig Fritchie : > Yes, I agree with your blame pointed at the kernel, but ... those > sockets have a listen backlog of 4096. ?Strace shows no VM activity in > any thread when a new TCP connection is actually established. ?My guess > is that when the app does not bother calling accept(2), and then the > listen backlog queue is full, the kernel will wait until one of the > existing 4096 TCP connections leaves CLOSE_WAIT state, which then leaves > an open slot in the listen queue? Oh yes! The slow connection setup can be perfectly explained by the application not calling accept() often enough and the listen backlog becoming full. I just didn't understand the normal kernel behaviour with a full backlog. Hope I didn't distract you too much :-) FWIW I did a small experiment just to understand how Linux handles backlogged listen sockets better. I used a debian 2.6.26-2-amd64 kernel with default settings on the server side (no special features like SYN cookies enabled). I started a server with a 30 connection backlog and never calling accept(): server> gen_tcp:listen(9999, [{backlog,30}]). {ok,#Port<0.444>} Then on the client I opened 100 connections with netcat: client$ for i in $(seq 1 100); do (nc -w 180 server 9999 &); done and waited a few seconds for everything to settle and then ran netstat to see what sockets were open on each host: server$ netstat -an | grep 9999 | awk '{print $NF}' | sort | uniq -c | sort -n 1 LISTEN 29 SYN_RECV 31 ESTABLISHED client$ netstat -an | grep 9999 | awk '{print $NF}' | sort | uniq -c | sort -n 40 SYN_SENT 60 ESTABLISHED So the kernel has allowed 31 (BACKLOG+1?) connections to become ESTABLISHED and a further 29 (BACKLOG-1?) to enter SYN_RECV and seems to have ignored the other 40 connection requests. The client sees a full 60 connections in ESTABLISHED state (the server's ESTABLISHED + SYN_RECV) and the rest in SYN_SENT still resending their SYN packets. If I wait for an application timeout in netcat to close the sockets on the client side then the server looks like this: server$ netstat -an | grep 9999 | awk '{print $NF}' | sort | uniq -c | sort -n 1 LISTEN 31 CLOSE_WAIT Looks like the SYN_RECV sockets were discarded and the ESTABLISHED transitioned into the half-closed CLOSE_WAIT state. So if the server does start calling accept() it'll process 31 sockets that are already closed by the client. (Fair enough.) The SYN_RECV sockets make me curious. Linux seems to have a second backlog queue for when the normal one fills up. Connections in this queue seem to send a SYN+ACK (because the client reaches ESTABLISHED) but wait in SYN_RECV. The code in net/ipv4/tcp_ipv4.c's tcp_v4_conn_request refers to this as a "syn queue" containing "warm entries", but I haven't dug deep enough to see how it really works. Does anyone know? (Per Hedeland? :-)) I posted a tcpdump at http://fresh.homeunix.net/~luke/misc/9999.pcap in case anyone else is having a "geek out on Linux networking" week. Cheers, -Luke From igorrs@REDACTED Thu Nov 19 15:41:50 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Thu, 19 Nov 2009 12:41:50 -0200 Subject: Consistent hashing for Mnesia fragments - part 2 Message-ID: Hello, Ulf. By now, I have reimplemented the module using one of your suggestions (raw ets accesses to a disc_copies table): http://igorrs.blogspot.com/2009/11/consistent-hashing-for-mnesia-fragments_19.html Seems to be working fine now. Thank you. Igor. On Wed, Nov 18, 2009 at 6:45 AM, Ulf Wiger wrote: > Igor Ribeiro Sucupira wrote: >> >> Hello. >> >> I've begun the implementation of a module following the >> mnesia_frag_hash behaviour. >> With enough fragments, my hash_state could grow to a reasonable size >> (something around, say, 10 MB). > > Great! I have downloaded your code and will take a look at > it when time allows (no promises as to when that might be...) > > This ought to come in very handy for fragmented tables where > each fragment has considerably more than 2 GB of data :) > (one example I have in mind is a sharding layer on top of > Tokyo Tyrant; in which case it might also be interesting to > add a Lua "BIF" to the Tyrant table that could calculate > the new hash on each object in place and only return the > objects that actually need to be moved to another bucket.) > >> Can you think of any significant performance impact of this >> implementation? For example: is the hash_state transmitted between >> nodes frequently enough to cause significant network traffic? > > It's not so much that which will be a problem. The frag_hash state > will only be replicated when it's updated, which happens when > e.g. changing the number of fragments. > > The bigger problem is that the frag_hash record is stored in > ets, and retrieved for each access to the fragmented table > (i.e. every read and every write). The ets:lookup() will cause > the 10 MB structure to be copied to the heap of the calling > process - once for every access! > > Another approach might be to create a fully replicated > ordered_set disc_copy table and store the keys there. > To speed things up, you can do raw ets accesses to this table, > but you will have to ensure then that even extra db nodes (that > don't have to be defined in the schema) are given a copy of > the table. Otherwise, you have to use at least dirty reads to > the table. This will provide distribution transparently, but > carry a bit more overhead than raw ets operations. > > A reasonable compromise might be to check once if the table > exists in ram (e.g using ets:table_info()), then choosing > whether to use async_dirty or ets in a call to mnesia:activity/2. > > ...except this doesn't seem to work as well as one would hope. > It seems that if you call mnesia:activity/2 inside a transaction, > it will create a new transaction store and copy into it all objects > updated in the outer transaction - even if the type of the nested > transaction is dirty or ets! (in which case the copying will be > entirely useless, as all the operations in the inner transaction > will be mapped to dirty or ets counterparts that have no knowledge > of transaction stores.) > > I can understand why this case isn't optimized, as normally you > deserve a good spanking if you willfully run dirty transactions > inside non-dirty ones. But in your case, I think it might have > been a reasonable approach. > > You might be left with having to implement your own gb_sets > equivalents based on ets, one using ets, and one using > dirty_reads, and then pick the most suitable one depending on > whether the keys table is local or not. > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From erlang@REDACTED Thu Nov 19 15:51:52 2009 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 19 Nov 2009 15:51:52 +0100 Subject: [erlang-questions] couchdb in Karmic Koala In-Reply-To: <1258640806.17050.14.camel@rumbuntu> References: <9b08084c0911190606r6bfb7505lcd1d0d773f0a8210@mail.gmail.com> <1258640806.17050.14.camel@rumbuntu> Message-ID: <9b08084c0911190651k65abeb4du669c692dfeab4838@mail.gmail.com> On Thu, Nov 19, 2009 at 3:26 PM, Dmitry Belyaev wrote: > 1) ps -A | grep couchdb > ? or > 2) /etc/init.d/couchdb {start|stop|status} No such file (on my system) > 3) sure you can if you database name will not interfere any other > > As I remember couchdb package depends on erlang. So it may be installed > as always in /usr/bin and /usr/lib/erlang Right - seems to contain a stripped down OTP This should mean that *anybody* running Karmic Koala can just type "erl" to get an erlang up and running, and compile away etc? /Joe > > On Thu, 2009-11-19 at 15:06 +0100, Joe Armstrong wrote: >> I just upgraded my ubuntu to Karmic Koala >> >> couchdb seems to have installed itself and a few other things (horray >> - well done ...) >> >> $ pwd >> /usr/lib/couchdb/erlang/lib >> $ ls >> couch-0.10.0 ?erlang-oauth ?etap ?ibrowse-1.5.2 ?mochiweb-r97 >> >> I think couchdb is running on my machine. >> >> A few questions: >> >> 1) Is couchdb running on my machine and how can I confirm this? >> 2) How is couchdb started and stopped? >> 3) Can I use couchdb for my own applications of is it reserved for system use? >> 4) Where is Erlang hidden away? >> 5) Can I use the hidden Erlang for my own applications >> 6) Can I distribute my own Erlang applications that make use of the >> ? ? Erlang that has (apparently) been installed in Karmic Koala >> >> Having Erlang on all Karmic Koala machines could lead to many exciting >> things :-) >> >> /Joe >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > > From paul.joseph.davis@REDACTED Thu Nov 19 16:06:11 2009 From: paul.joseph.davis@REDACTED (Paul Joseph Davis) Date: Thu, 19 Nov 2009 10:06:11 -0500 Subject: [erlang-questions] couchdb in Karmic Koala In-Reply-To: <9b08084c0911190651k65abeb4du669c692dfeab4838@mail.gmail.com> References: <9b08084c0911190606r6bfb7505lcd1d0d773f0a8210@mail.gmail.com> <1258640806.17050.14.camel@rumbuntu> <9b08084c0911190651k65abeb4du669c692dfeab4838@mail.gmail.com> Message-ID: On Nov 19, 2009, at 9:51 AM, Joe Armstrong wrote: > On Thu, Nov 19, 2009 at 3:26 PM, Dmitry Belyaev > wrote: >> 1) ps -A | grep couchdb > >> or >> 2) /etc/init.d/couchdb {start|stop|status} > > No such file (on my system) > The default installed version is packaged to run as An on demand desktop service. To get the system service version you'll want to check apt. Something like: $ apt-cache search couchdb To get the name (just "couchdb" IIRC) and then: $ sudo apt-get install $pkgname >> 3) sure you can if you database name will not interfere any other >> >> As I remember couchdb package depends on erlang. So it may be >> installed >> as always in /usr/bin and /usr/lib/erlang > > Right - seems to contain a stripped down OTP > > This should mean that *anybody* running Karmic Koala can just type > "erl" to get an erlang up and running, and compile away etc? > > /Joe > >> >> On Thu, 2009-11-19 at 15:06 +0100, Joe Armstrong wrote: >>> I just upgraded my ubuntu to Karmic Koala >>> >>> couchdb seems to have installed itself and a few other things >>> (horray >>> - well done ...) >>> >>> $ pwd >>> /usr/lib/couchdb/erlang/lib >>> $ ls >>> couch-0.10.0 erlang-oauth etap ibrowse-1.5.2 mochiweb-r97 >>> >>> I think couchdb is running on my machine. >>> >>> A few questions: >>> >>> 1) Is couchdb running on my machine and how can I confirm this? >>> 2) How is couchdb started and stopped? >>> 3) Can I use couchdb for my own applications of is it reserved for >>> system use? >>> 4) Where is Erlang hidden away? >>> 5) Can I use the hidden Erlang for my own applications >>> 6) Can I distribute my own Erlang applications that make use of the >>> Erlang that has (apparently) been installed in Karmic Koala >>> >>> Having Erlang on all Karmic Koala machines could lead to many >>> exciting >>> things :-) >>> >>> /Joe >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >> >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From rapsey@REDACTED Thu Nov 19 16:15:36 2009 From: rapsey@REDACTED (Rapsey) Date: Thu, 19 Nov 2009 16:15:36 +0100 Subject: [erlang-questions] couchdb in Karmic Koala In-Reply-To: References: <9b08084c0911190606r6bfb7505lcd1d0d773f0a8210@mail.gmail.com> <1258640806.17050.14.camel@rumbuntu> <9b08084c0911190651k65abeb4du669c692dfeab4838@mail.gmail.com> Message-ID: <97619b170911190715l6dd08e5dod8e81f517abc82c9@mail.gmail.com> Pretty good writeup on CouchDB in ubuntu: http://arstechnica.com/open-source/reviews/2009/11/good-karma-ars-reviews-ubuntu-910.ars/6 http://arstechnica.com/open-source/reviews/2009/11/good-karma-ars-reviews-ubuntu-910.ars/7 Sergej On Thu, Nov 19, 2009 at 4:06 PM, Paul Joseph Davis < paul.joseph.davis@REDACTED> wrote: > > > > > On Nov 19, 2009, at 9:51 AM, Joe Armstrong wrote: > > On Thu, Nov 19, 2009 at 3:26 PM, Dmitry Belyaev >> wrote: >> >>> 1) ps -A | grep couchdb >>> >> >> or >>> 2) /etc/init.d/couchdb {start|stop|status} >>> >> >> No such file (on my system) >> >> > The default installed version is packaged to run as An on demand desktop > service. To get the system service version you'll want to check apt. > Something like: > > $ apt-cache search couchdb > > To get the name (just "couchdb" IIRC) and then: > > $ sudo apt-get install $pkgname > > > 3) sure you can if you database name will not interfere any other >>> >>> As I remember couchdb package depends on erlang. So it may be installed >>> as always in /usr/bin and /usr/lib/erlang >>> >> >> Right - seems to contain a stripped down OTP >> >> This should mean that *anybody* running Karmic Koala can just type >> "erl" to get an erlang up and running, and compile away etc? >> >> /Joe >> >> >>> On Thu, 2009-11-19 at 15:06 +0100, Joe Armstrong wrote: >>> >>>> I just upgraded my ubuntu to Karmic Koala >>>> >>>> couchdb seems to have installed itself and a few other things (horray >>>> - well done ...) >>>> >>>> $ pwd >>>> /usr/lib/couchdb/erlang/lib >>>> $ ls >>>> couch-0.10.0 erlang-oauth etap ibrowse-1.5.2 mochiweb-r97 >>>> >>>> I think couchdb is running on my machine. >>>> >>>> A few questions: >>>> >>>> 1) Is couchdb running on my machine and how can I confirm this? >>>> 2) How is couchdb started and stopped? >>>> 3) Can I use couchdb for my own applications of is it reserved for >>>> system use? >>>> 4) Where is Erlang hidden away? >>>> 5) Can I use the hidden Erlang for my own applications >>>> 6) Can I distribute my own Erlang applications that make use of the >>>> Erlang that has (apparently) been installed in Karmic Koala >>>> >>>> Having Erlang on all Karmic Koala machines could lead to many exciting >>>> things :-) >>>> >>>> /Joe >>>> >>>> ________________________________________________________________ >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>>> erlang-questions (at) erlang.org >>>> >>>> >>>> >>> >>> >>> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From masklinn@REDACTED Thu Nov 19 16:03:35 2009 From: masklinn@REDACTED (Masklinn) Date: Thu, 19 Nov 2009 16:03:35 +0100 Subject: [erlang-questions] couchdb in Karmic Koala In-Reply-To: <9b08084c0911190606r6bfb7505lcd1d0d773f0a8210@mail.gmail.com> References: <9b08084c0911190606r6bfb7505lcd1d0d773f0a8210@mail.gmail.com> Message-ID: <32FDBA49-391A-4F91-92D5-74B1FA65943E@masklinn.net> On 19 Nov 2009, at 15:06 , Joe Armstrong wrote: > A few questions: > > 1) Is couchdb running on my machine and how can I confirm this? > 2) How is couchdb started and stopped? > 3) Can I use couchdb for my own applications of is it reserved for system use? > 4) Where is Erlang hidden away? > 5) Can I use the hidden Erlang for my own applications > 6) Can I distribute my own Erlang applications that make use of the > Erlang that has (apparently) been installed in Karmic Koala > > Having Erlang on all Karmic Koala machines could lead to many exciting > things :-) > > /Joe Ars's piece on Karmic has a few informations: http://arstechnica.com/open-source/reviews/2009/11/good-karma-ars-reviews-ubuntu-910.ars/6 From kostis@REDACTED Thu Nov 19 17:39:32 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 19 Nov 2009 18:39:32 +0200 Subject: [erlang-questions] edoc, erlc and dialyser type spec consistency In-Reply-To: <9b08084c0911190453o41e5b560h2d5226aa690b24d3@mail.gmail.com> References: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> <9b08084c0911190453o41e5b560h2d5226aa690b24d3@mail.gmail.com> Message-ID: <4B0574C4.10308@cs.ntua.gr> Joe Armstrong wrote: > On Thu, Nov 19, 2009 at 1:02 PM, Kenneth Lundin > wrote: >> I agree with Kostis answers and can also inform you that we are working >> on a solution for a) below. >> >> The plan is: >> Edoc will then parse and use the same -spec and -type as Dialyzer. >> If a file contains both -spec/-type and edoc comments with @spec and @type >> you will have to choose which notation to use (as a compatiblity mode) for >> already existing edoc markup. > > Excellent. > > One more question: > > - does the dialyzer pick up @specs from edoc comments if there are > no -spec's in the code? The short answer is "No". The long answer is that even if there were some tool that would pick up @specs from edoc comments and feed them automatically to dialyzer, you probably would not want to use these @specs because quite many of them have suffered from code rot and are misleading or just wrong. See also war story documented in the following paper: http://www.it.uu.se/research/group/hipe/dialyzer/publications/wrangler.pdf Highly recommended to anyone who wants to add proper specs to his code. My advice: if you are about to start documenting your code using specs, start using proper -specs and run dialyzer periodically while you do so. Do not worry for that you may not be able to run edoc for a short while. It's just a matter of time for Edoc to be able to also handle proper -specs. Kostis From raould@REDACTED Thu Nov 19 19:29:46 2009 From: raould@REDACTED (Raoul Duke) Date: Thu, 19 Nov 2009 10:29:46 -0800 Subject: [erlang-questions] Re: programming tip: a note on encapsulation In-Reply-To: References: <9b08084c0911180056l53ca0695r781656018d6f76a9@mail.gmail.com> <01699F7F-6016-432D-A59F-FCD72207CF00@cs.otago.ac.nz> <9b08084c0911182359q39be28e7q5f925055124d8606@mail.gmail.com> Message-ID: <91a2ba3e0911191029w482c8910m1022d4124ceddc52@mail.gmail.com> > Unfortunately F(f1)(a, b) doesn't parse. The only way to make that > work is to write it as (F(f1))(a, b), which is horrible to read. cough cough LFE cough cough From pablo.platt@REDACTED Thu Nov 19 19:00:16 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Thu, 19 Nov 2009 10:00:16 -0800 (PST) Subject: [erlang-questions] init.d script and a control script for starting an erlang app on boot in ubuntu In-Reply-To: <15F8750B-A91A-4995-9A80-9D6101C49C59@mindspring.com> References: <91026.80235.qm@web112610.mail.gq1.yahoo.com> <15F8750B-A91A-4995-9A80-9D6101C49C59@mindspring.com> Message-ID: <170568.37904.qm@web112606.mail.gq1.yahoo.com> ________________________________ From: Matt Stancliff To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Mon, November 16, 2009 8:55:38 AM Subject: Re: [erlang-questions] init.d script and a control script for starting an erlang app on boot in ubuntu Pablo, On Nov 15, 2009, at 9:55 PM, Pablo Platt wrote: > Can someone post a simple init.d script with a control script for an erlang application on ubuntu? > > I'll appreciate an example for a control script that can start/stop/status/restart/reload(hot code update)/debug(attach an erlang shell). > The script should also be able to restart the app (heart?). > > The best setup I've found for node scripting is from rabbitmq: > Control script: http://hg.rabbitmq.com/rabbitmq-server/file/default/scripts/rabbitmqctl > Control module: http://hg.rabbitmq.com/rabbitmq-server/file/default/src/rabbit_control.erl I've tried to see how rabbitmq works and I'm a bit confused. The /etc/init.d/rabbitmq-server file define DAEMON=/usr/sbin/rabbitmq-multi and call $DAEMON start_all ${NODE_COUNT} > /var/log/rabbitmq/startup_log 2> /var/log/rabbitmq/startup_err /usr/sbin/rabbitmq-multi define SCRIPT=`basename $0` and call /usr/lib/bin/rabbitmq-multi: surabbitmq -s /bin/sh -c "/usr/lib/rabbitmq/bin/${SCRIPT} ${CMDLINE}" /usr/lib/bin/rabbitmq-multi starts the rabbitmq server by calling to rabbit_multi module start_all: exec ... -s rabbit_multi ... http://hg.rabbitmq.com/rabbitmq-server/file/9efe4a2ff6a5/src/rabbit_multi.erl The rabbit_multi module create starts the nodes and create the pid files. I didn't see a use of the files you link to rabbitmqctl and rabbit_control in the boot process. They are probably being used to control the server after boot. 1. Does it have to be so complicated? For example, why does /usr/sbin/rabbitmq-multi calls /usr/lib/bin/rabbitmq-multi instead of starting the server? Do I need to create and use pid files? 2. Does anyone have a simple boot script + control script that can be used with a mochiweb app? Thanks From clist@REDACTED Thu Nov 19 19:54:48 2009 From: clist@REDACTED (Angel Alvarez) Date: Thu, 19 Nov 2009 19:54:48 +0100 Subject: [erlang-questions] Why using pmap is not giving me the desired speedup ? In-Reply-To: <8C9A9661-1854-4250-8EBE-7773832F92D6@cs.otago.ac.nz> References: <6b1b97d20911162245p61445058x3aaaf4048b628368@mail.gmail.com> <8C9A9661-1854-4250-8EBE-7773832F92D6@cs.otago.ac.nz> Message-ID: <200911191954.49009.clist@uah.es> Great Post!!! Very educative, with the threshold and the "one child optimization". Thanks El Mi?rcoles, 18 de Noviembre de 2009 Richard O'Keefe escribi?: > > On Nov 17, 2009, at 7:45 PM, Tushar Deshpande wrote: > > I imagined that I would get some observable > > speedup with parallel quicksort. > > > > Contrary to my expectations, parallel version > > took more time to run. > > > > > What could be the reason ? > > Why is parallel quicksort effective on shared memory multiprocessors? > Because there is an O(n) in-place partitioning, following by sparking > two activities, which need to be told only "where is the first element > of my sublist" and "how many elements" (2 words), and which need no > synchronisation between them, the only synchronisation being for the > parent to wait for its two children. Indeed, we only need one child. > > psort(a, n) = > if n is small then > use insertion sort > else > choose pivot x > partition (a,n) into (a,L), (a+L,E), (a+L+E,G) > where L is the number of elements < x, E the > number of elements = x, G the number > x. > child = fork psort(a+L+E,G) > psort(a,L) > wait for child > end if > > Tushar Deshpande's parallel quicksort, thanks to its use of > pmap, creates two children, when it only needs to create one. > > Not only that, the left and right sublists have to be *sent* > to each child, that's O(n) copying, the sorted versions have > to be *received* from each child, that's O(n) copying again, > and then they have to be concatenated. The sequential version > has only to concatenate. > > It would be interesting to try > > pqsortB(L) -> > pqsortB(L, length(L)). > > pqsortB(L, N) when N < 10 -> % tunable threshold > lists:sort(L); > pqsortB([Pivot|L], _) -> > ppartitionB(L, Pivot, [], 0, [Pivot], [], 0). > > ppartitionB([], _, Lss, Nlss, Eql, Gtr, Ngtr) -> > Self = self(), > Pid = spawn(fun () -> Self!{self(),pqsortB(Gtr, Ngtr)} end), > Left = pqsortB(Lss, Nlss), > receive {Pid,Right} -> Left++(Eql++Right) end; > ppartitionB([X|Xs], Pivot, Lss, Nlss, Eql, Gtr, Ngtr) -> > if X < Pivot -> ppartitionB(Xs, Pivot, [X|Lss], Nlss+1, Eql, Gtr, > Ngtr) > ; X > Pivot -> ppartitionB(Xs, Pivot, Lss, Nlss, Eql, [X|Gtr], > Ngtr+1) > ; true -> ppartitionB(Xs, Pivot, Lss, Nlss, [X|Eql], Gtr, Ngtr) > end. > > In fact it seemed interesting enough that I *did* try it. > I measured the time to sort a list of 10,000 uniformly generated > random floats, on a machine with a 500MHz CPU and 84 MHz memory. > All times in this message are the smallest of 5 runs. > > 120 msec Tushar Deshpande's qsort > 110 msec Sequential qsort with single partition pass as above > and similarly switching over to lists:qsort for small N. > 110 msec Same as above but not bothering with counts, checking > for large -vs- small by [Pivot|L = [_,_,_|_]] or not > and sorting 0 to 3 elements inline. > 80 msec Same as previous, but qsort(L, T) == qsort(L)++T, to > try to reduce the amount of concatenation. > > 930 msec Tushar Deshpande's pqsort (with a simplified version of > pmap/2 that doesn't bother with error handling) > 230 msec The pqsortB/1 function above. > > With Tushar Deshpande's pqsort/1 being 7.75 times slower than qsort/1, > it's going to take a lot of cores just to catch up. My improved > pqsortB is 4 times faster than his on a single core. > > I tried again with a 2.2GHz CPU, 667 MHz memory, 2-core machine. > This time I needed 100,000 element lists to get useful times. > > 170 msec original qsort > 120 msec > 110 msec > 100 msec best qsort from above > > -smp disable -smp enable > 830 msec 1720 msec original pqsort > 240 msec 310 msec improved pqsortB > 110 msec 110 msec partition once, sort sublists > using one child, combine. > > As a double check, with -smp disable, the emulator reported > [rq:1] in the banner; with -smp enable, [smp:2:2] [rq:2]. > > Here's what I *suspect* is going on. The time ratios for > the sequential versions are very close to the speed ratios > of the main memories on the two machines. It looks as > though memory is the main bottleneck. > > Now my laptop may have two cores, but they share one main > memory. (And I think they share an L2 cache.) If the time > of a computation is dominated by memory, dividing the work > across several CPUs isn't going to help: you need to divide > it across several memories. > > Another key point is that message passing is *copying*. > It seems clear from the "-smp enable" column above that > "the fewer the processes the better", which in this case > actually means "the less COPYING of data between processes > the better". > > -smp disable -smp enable #processes #cells copied > 830 msec 1720 msec 32766 2800000 > 240 msec 310 msec 16383 1406112 > 110 msec 110 msec 1 100000 > > Here a "cell" means a list cell whose head is a float copied > in a message from one process to another. #processes means > the number of *additional* (temporary) processes. > > We have one mechanism in Erlang for both modelling concurrent > systems and obtaining parallel speedups: logically separate > processes communicating by message passing. In the former > case, you create (at least) one Erlang process per logical > activity, NOT for speed but for CLARITY, so that each logical > activity can be modelled separately from the others. The > deciding factor in the number of processes you create is the > number of things to be modelled. > > When you are trying to get parallelism, there is seldom any > point in having more than a small number of times as many > processes as you have available cores. If you have lots of > processes all wanting to be active at the same time, the > costs of scheduling and message passing are likely to be high > relative to the useful work. The best parallel quicksort > above just used two processes (one the original, the other > a temporary child). On a quad core machine, four processes > would make sense. > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. ->>----------------------------------------------- Clist UAH a.k.a Angel ---------------------------------[www.uah.es]-<<-- *J2EE es una arquitectura enorme, pesada y complicada dise?ada especialmente para que los departamentos de sistemas ?? justifiquen sus gastos y los gerentes puedan gastarse sus millones en IBM y ORACLE From rvirding@REDACTED Thu Nov 19 20:56:43 2009 From: rvirding@REDACTED (Robert Virding) Date: Thu, 19 Nov 2009 20:56:43 +0100 Subject: [erlang-questions] Re: programming tip: a note on encapsulation In-Reply-To: <91a2ba3e0911191029w482c8910m1022d4124ceddc52@mail.gmail.com> References: <9b08084c0911180056l53ca0695r781656018d6f76a9@mail.gmail.com> <01699F7F-6016-432D-A59F-FCD72207CF00@cs.otago.ac.nz> <9b08084c0911182359q39be28e7q5f925055124d8606@mail.gmail.com> <91a2ba3e0911191029w482c8910m1022d4124ceddc52@mail.gmail.com> Message-ID: <3dbc6d1c0911191156y55716a0elb36f93a9b1f482c7@mail.gmail.com> 2009/11/19 Raoul Duke > > Unfortunately F(f1)(a, b) doesn't parse. The only way to make that > > work is to write it as (F(f1))(a, b), which is horrible to read. > > cough cough LFE cough cough > (funcall (funcall f f1) a b) if I understood the binding correctly! :-) Robert From dima@REDACTED Thu Nov 19 21:36:43 2009 From: dima@REDACTED (Dmitry Vasiliev) Date: Thu, 19 Nov 2009 23:36:43 +0300 Subject: [erlang-questions] Erlang Latency Guide In-Reply-To: References: Message-ID: <4B05AC5B.7050504@hlabs.spb.ru> Erik Rigtorp wrote: > For receiving there is a simple solution to this problem: use the {active, > once} socket option. > > A simple selective receive-free TCP receiver: > > loop(Sock) -> > inet:setopts(Sock, [{active, once}]), > receive > {tcp, Sock, Data} -> > loop(Sock); > {tcp_error, Sock, Reason} -> > exit(Reason); > {tcp_closed, Sock} -> > exit() > end. Does it mean {active, once} more optimal than {active, false}? -- Dmitry Vasiliev http://hlabs.spb.ru http://twitter.com/hdima From rapsey@REDACTED Thu Nov 19 21:45:04 2009 From: rapsey@REDACTED (Rapsey) Date: Thu, 19 Nov 2009 21:45:04 +0100 Subject: [erlang-questions] Erlang Latency Guide In-Reply-To: <4B05AC5B.7050504@hlabs.spb.ru> References: <4B05AC5B.7050504@hlabs.spb.ru> Message-ID: <97619b170911191245t61c3706cl5cb66e5ff72518d0@mail.gmail.com> >From my testing {active,false} is the least optimal. I did those tests more than a year ago, perhaps something changed since then, but probably not. Sergej On Thu, Nov 19, 2009 at 9:36 PM, Dmitry Vasiliev wrote: > Erik Rigtorp wrote: > >> For receiving there is a simple solution to this problem: use the {active, >> once} socket option. >> >> A simple selective receive-free TCP receiver: >> >> loop(Sock) -> >> inet:setopts(Sock, [{active, once}]), >> receive >> {tcp, Sock, Data} -> >> loop(Sock); >> {tcp_error, Sock, Reason} -> >> exit(Reason); >> {tcp_closed, Sock} -> >> exit() >> end. >> > > Does it mean {active, once} more optimal than {active, false}? > > -- > Dmitry Vasiliev > http://hlabs.spb.ru > http://twitter.com/hdima > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From tomas.abrahamsson@REDACTED Thu Nov 19 22:24:30 2009 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Thu, 19 Nov 2009 22:24:30 +0100 Subject: [erlang-questions] R13B02 on 8/16 core box: all TCP communication hangs/frozen In-Reply-To: <1d0fb8fc0911190635p5adbeb1em5111f06d1ecb0a6d@mail.gmail.com> References: <1d0fb8fc0911180628p35864ec1ob843084c0ff625d@mail.gmail.com> <8925.1258599381@snookles.snookles.com> <1d0fb8fc0911190635p5adbeb1em5111f06d1ecb0a6d@mail.gmail.com> Message-ID: > The SYN_RECV sockets make me curious. Linux seems to have a second > backlog queue for when the normal one fills up. Connections in this > queue seem to send a SYN+ACK (because the client reaches ESTABLISHED) > but wait in SYN_RECV. The code in net/ipv4/tcp_ipv4.c's > tcp_v4_conn_request refers to this as a "syn queue" containing "warm > entries", but I haven't dug deep enough to see how it really works. > Does anyone know? (Per Hedeland? :-)) I believe you might be seeing what's described here: http://www.cs.rice.edu/CS/Systems/Web-measurement/paper/node3.html Here's the section describing the two queues: "When a connection establishment request (TCP SYN packet) from a client is received on [a] socket (Figure 1, position 1), the server TCP responds with a SYN-ACK TCP packet, creates a socket for the new, incomplete connection, and places it in the listen socket's SYN-RCVD queue. Later, when the client responds with an ACK packet to the server's SYN-ACK packet (position 2), the server TCP removes the socket created above from the SYN-RCVD queue and places it in the listen socket's queue of connections awaiting acceptance (accept queue). Each time the WWW server process executes the accept() system call (position 3), the first socket in the accept queue of the listen socket is removed and returned." I learned a lot from that paper. A PostScript and an html version is at: http://www.cs.rice.edu/CS/Systems/Web-measurement/ BRs Tomas From zabrane3@REDACTED Fri Nov 20 04:33:01 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Fri, 20 Nov 2009 04:33:01 +0100 Subject: Ready to use hash functions in Erlang Message-ID: <18a1db030911191933vacb048bx4826b49b7ad823e2@mail.gmail.com> Hi List ! Is there some "free/open source" hash functions implementation (like those: http://www.partow.net/programming/hashfunctions/index.html) in Erlang? Thanks Zabrane From ok@REDACTED Fri Nov 20 05:08:26 2009 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 20 Nov 2009 17:08:26 +1300 Subject: BERT Message-ID: <00982B68-416C-4E85-BF12-B2182C607467@cs.otago.ac.nz> BERT is a nice idea, but there's one thing I learned to be very scared of when working on Prolog implementations: reserved atoms. (Anyone who knows what numbervars/3 does will know the trouble caused by '$VAR'.) Instead of the bert-rpc.org specification saying that library writers should just make sure that 'bert' never turns up as the first element of a tuple, it should say that {bert,bert} decodes as 'bert', and ask library writers to always represent 'bert' as {bert,bert}. I mention it here because I've found the rule "make life simpler for users of an encoding even if it makes life harder for the encoder" a good one. From tony@REDACTED Fri Nov 20 05:18:31 2009 From: tony@REDACTED (Tony Arcieri) Date: Thu, 19 Nov 2009 21:18:31 -0700 Subject: [erlang-questions] Re: BERT In-Reply-To: <00982B68-416C-4E85-BF12-B2182C607467@cs.otago.ac.nz> References: <00982B68-416C-4E85-BF12-B2182C607467@cs.otago.ac.nz> Message-ID: Are you at Rubyconf or something? I'm trying to deconstruct the serendipity of this post. On Thu, Nov 19, 2009 at 9:08 PM, Richard O'Keefe wrote: > BERT is a nice idea, but there's one thing I learned to be very scared > of when working on Prolog implementations: reserved atoms. (Anyone > who knows what numbervars/3 does will know the trouble caused by '$VAR'.) > > Instead of the bert-rpc.org specification saying that library writers > should just make sure that 'bert' never turns up as the first element > of a tuple, it should say that {bert,bert} decodes as 'bert', and ask > library writers to always represent 'bert' as {bert,bert}. > > I mention it here because I've found the rule "make life simpler for > users of an encoding even if it makes life harder for the encoder" > a good one. > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- Tony Arcieri Medioh/Nagravision From gleber.p@REDACTED Fri Nov 20 09:41:55 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 20 Nov 2009 09:41:55 +0100 Subject: [erlang-questions] Ready to use hash functions in Erlang In-Reply-To: <18a1db030911191933vacb048bx4826b49b7ad823e2@mail.gmail.com> References: <18a1db030911191933vacb048bx4826b49b7ad823e2@mail.gmail.com> Message-ID: <14f0e3620911200041i703cc4b6wc2fa5f81a8fc2fc7@mail.gmail.com> There are few hash functions in Erlang's stdlib: erlang:phash/1,2 crypto:md5/1 crypto:sha/1 On Fri, Nov 20, 2009 at 04:33, zabrane Mikael wrote: > Hi List ! > > Is there some "free/open source" hash functions implementation (like those: > http://www.partow.net/programming/hashfunctions/index.html) > in Erlang? > > Thanks > Zabrane > From roberto@REDACTED Fri Nov 20 10:48:36 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 20 Nov 2009 10:48:36 +0100 Subject: [erlang-questions] Erlang Latency Guide In-Reply-To: <97619b170911191245t61c3706cl5cb66e5ff72518d0@mail.gmail.com> References: <4B05AC5B.7050504@hlabs.spb.ru> <97619b170911191245t61c3706cl5cb66e5ff72518d0@mail.gmail.com> Message-ID: > From my testing {active,false} is the least optimal. I did those tests more > than a year ago, perhaps something changed since then, but probably not. i confirm the results of these tests, where {active, false} < {active, once} < {active, true}, however for the latter you need to have a higher-level protocol to avoid overflooding the receiver from a fast sender. cheers, r. From roberto@REDACTED Fri Nov 20 11:17:20 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 20 Nov 2009 11:17:20 +0100 Subject: massive distribution Message-ID: dear all, we are using erlang for cloud computing applications, and i would like to know if it's possible to take advantage of the processing power of, let's say, 50,000 cores [i.e. from 6,000 to 12,000 machines, therefore erlang nodes]. i'm interested in knowing if someone has the information on how many interconnected nodes erlang has been proven able to support. sorry if this came along already somewhere, i could not find relevant info. thank you, r. From zabrane3@REDACTED Fri Nov 20 11:37:39 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Fri, 20 Nov 2009 11:37:39 +0100 Subject: [erlang-questions] Ready to use hash functions in Erlang In-Reply-To: <14f0e3620911200041i703cc4b6wc2fa5f81a8fc2fc7@mail.gmail.com> References: <18a1db030911191933vacb048bx4826b49b7ad823e2@mail.gmail.com> <14f0e3620911200041i703cc4b6wc2fa5f81a8fc2fc7@mail.gmail.com> Message-ID: <18a1db030911200237m320020c7lff53f35373e0bb4f@mail.gmail.com> Hi Gleb ! > There are few hash functions in Erlang's stdlib: > > erlang:phash/1,2 > crypto:md5/1 > crypto:sha/1 > Thanks for the poniter, I'am aware of these (phash is deprecated, use phash2 instead?). I'm interested in some fast and well distributed hash functions made by gurus in this domain like lookup3 (Bob Jenkins - http://burtleburtle.net/bob/hash/doobs.html), SuperFasthash (Paul Hiesh - http://www.azillionmonkeys.com/qed/hash.html) ... Does someone spend time on them or have an Erlang version? Thanks Zabrane > On Fri, Nov 20, 2009 at 04:33, zabrane Mikael wrote: > > Hi List ! > > > > Is there some "free/open source" hash functions implementation (like > those: > > http://www.partow.net/programming/hashfunctions/index.html) > > in Erlang? > > > > Thanks > > Zabrane > > > From gleber.p@REDACTED Fri Nov 20 11:43:18 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 20 Nov 2009 11:43:18 +0100 Subject: [erlang-questions] Ready to use hash functions in Erlang In-Reply-To: <18a1db030911200237m320020c7lff53f35373e0bb4f@mail.gmail.com> References: <18a1db030911191933vacb048bx4826b49b7ad823e2@mail.gmail.com> <14f0e3620911200041i703cc4b6wc2fa5f81a8fc2fc7@mail.gmail.com> <18a1db030911200237m320020c7lff53f35373e0bb4f@mail.gmail.com> Message-ID: <14f0e3620911200243w6301593fk8007f1f31fc83d93@mail.gmail.com> > phash is deprecated, use phash2 instead? Yep, I made a typo. > Does someone spend time on them or have an Erlang version? I haven't hear about any. There's also a sha224, sha256, sha384 and sha512 here: http://steve.vinoski.net/blog/2009/01/03/more-sha-in-erlang/ From mbj@REDACTED Fri Nov 20 11:48:06 2009 From: mbj@REDACTED (Martin Bjorklund) Date: Fri, 20 Nov 2009 11:48:06 +0100 (CET) Subject: [erlang-questions] Ready to use hash functions in Erlang In-Reply-To: <18a1db030911200237m320020c7lff53f35373e0bb4f@mail.gmail.com> References: <18a1db030911191933vacb048bx4826b49b7ad823e2@mail.gmail.com> <14f0e3620911200041i703cc4b6wc2fa5f81a8fc2fc7@mail.gmail.com> <18a1db030911200237m320020c7lff53f35373e0bb4f@mail.gmail.com> Message-ID: <20091120.114806.198376378.mbj@tail-f.com> Hi, zabrane Mikael wrote: > Thanks for the poniter, I'am aware of these (phash is deprecated, use phash2 > instead?). > > I'm interested in some fast and well distributed hash functions made by > gurus in this domain like lookup3 (Bob Jenkins - > http://burtleburtle.net/bob/hash/doobs.html) phash2() uses Bob Jenkins' function iff you pass it a binary. /martin From zabrane3@REDACTED Fri Nov 20 11:54:57 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Fri, 20 Nov 2009 11:54:57 +0100 Subject: [erlang-questions] Ready to use hash functions in Erlang In-Reply-To: <20091120.114806.198376378.mbj@tail-f.com> References: <18a1db030911191933vacb048bx4826b49b7ad823e2@mail.gmail.com> <14f0e3620911200041i703cc4b6wc2fa5f81a8fc2fc7@mail.gmail.com> <18a1db030911200237m320020c7lff53f35373e0bb4f@mail.gmail.com> <20091120.114806.198376378.mbj@tail-f.com> Message-ID: <18a1db030911200254v63b44f50ic4124fb2dfbba104@mail.gmail.com> Hi Martin ! > I'm interested in some fast and well distributed hash functions made by > > gurus in this domain like lookup3 (Bob Jenkins - > > http://burtleburtle.net/bob/hash/doobs.html) > > phash2() uses Bob Jenkins' function iff you pass it a binary. Excellent. And do you know if it was build in pure Erlang or as a linked Erlang driver? Where can I find the source code? Regards Zabrane. From mbj@REDACTED Fri Nov 20 11:59:06 2009 From: mbj@REDACTED (Martin Bjorklund) Date: Fri, 20 Nov 2009 11:59:06 +0100 (CET) Subject: [erlang-questions] Ready to use hash functions in Erlang In-Reply-To: <18a1db030911200254v63b44f50ic4124fb2dfbba104@mail.gmail.com> References: <18a1db030911200237m320020c7lff53f35373e0bb4f@mail.gmail.com> <20091120.114806.198376378.mbj@tail-f.com> <18a1db030911200254v63b44f50ic4124fb2dfbba104@mail.gmail.com> Message-ID: <20091120.115906.42132719.mbj@tail-f.com> zabrane Mikael wrote: > Hi Martin ! > > > I'm interested in some fast and well distributed hash functions made by > > > gurus in this domain like lookup3 (Bob Jenkins - > > > http://burtleburtle.net/bob/hash/doobs.html) > > > > phash2() uses Bob Jenkins' function iff you pass it a binary. > > > Excellent. And do you know if it was build in pure Erlang or as a linked > Erlang driver? > Where can I find the source code? It's a bif. See block_hash() in erts/emulator/beam/utils.c. /martin From pablo.platt@REDACTED Fri Nov 20 15:07:47 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Fri, 20 Nov 2009 06:07:47 -0800 (PST) Subject: ubuntu upstart job to run an erlang application Message-ID: <998105.35733.qm@web112602.mail.gq1.yahoo.com> Hi, I'm trying to create an upstart job on ubuntu to start a mochiweb app called myweb. My job looks like: start on filesystem and net-device-up stop on runlevel [06] script MYWEBPATH=/home/user/myweb exec erl -noinput -pa $MYWEBPATH/ebin $MYWEBPATH/deps/*/ebin -boot start_sasl -s myweb end script When I'm trying to start the job and check its status I'm getting: $sudo start myweb myweb start/running, process 6682 $sudo status myweb myweb stop/waiting So my mochiweb app doesn't run. When I'm using the start.sh script generated with the skeleton my app run fine. Am I missing a flag in the erl command or is it something else? Thanks From v@REDACTED Fri Nov 20 16:22:27 2009 From: v@REDACTED (Valentin Micic) Date: Fri, 20 Nov 2009 17:22:27 +0200 Subject: [erlang-questions] Multicast UDP sending question In-Reply-To: <4B06A28C.3040506@m5networks.com.au> Message-ID: <20091120152237.E62803D0D4E@mail.pharos-avantgard.com> Don't know what to tell you -- I've tried similar thing from Windoze and got what I've expected: Started two shells with following result: FIRST: Erlang (BEAM) emulator version 5.5.4 [async-threads:0] Eshell V5.5.4 (abort with ^G) (vhlr@REDACTED)1> gen_udp:open( 9999, [{reuseaddr, true}] ). {ok,#Port<0.97>} (vhlr@REDACTED)2> SECOND: Erlang (BEAM) emulator version 5.5.4 [async-threads:0] Eshell V5.5.4 (abort with ^G) (uscript@REDACTED)1> gen_udp:open( 9999, [{reuseaddr, true}] ). {error,eaddrinuse} (uscript@REDACTED)2> Hope this is helpful too. V/ -----Original Message----- From: Sam Bobroff [mailto:samb@REDACTED] Sent: 20 November 2009 04:07 PM To: Jarrod Roberson Cc: Valentin Micic; Torben Hoffmann; Erlang Subject: Re: [erlang-questions] Multicast UDP sending question Hello, Jarrod Roberson wrote: > On Tue, Nov 17, 2009 at 9:15 AM, Valentin Micic wrote: > > >> You cannot open the same UDP port twice (or, shall I say -- bound the same >> UDP port to two different sockets). The story about N, N+1 port as well as >> explanation indicated that this is an Erlang restriction due to the socket >> library used is - well... pure fiction. >> >> > > thanks for the clarification. I am not sure how the Erlang internals work. > Can you explain what {reuseaddr,true} does then, if not bind the same UDP > port to multiple sockets. > There are LOTS of applications on a OSX that all bind to 224.0.0.251:5353, > is there a semantic different I am missing. > Thanks for taking the time to explain this I was pretty sure that Linux's reuseaddr option *did* allow the same UDP port to be bound to more than one socket, so I thought I'd test it. It's easy to do with Erlang: I used the following test code on Ubuntu Linux 2.6.31: -module(udp). -export([start/0]). start() -> {ok, Port} = gen_udp:open(9999, [{reuseaddr, true}]), io:fwrite("Bound to port: ~p\n", [Port]), loop(). loop() -> receive Msg -> io:fwrite("Msg: ~p\n", [Msg]) end, loop(). Running this in one console and then using "netcat -u localhost 9999" in another and hitting enter a few times gives the following output: $ erl +B -noshell -run udp Bound to port: #Port<0.289> Msg: {udp,#Port<0.289>,{127,0,0,1},49272,"\n"} Msg: {udp,#Port<0.289>,{127,0,0,1},49272,"\n"} Msg: {udp,#Port<0.289>,{127,0,0,1},49272,"\n"} Leaving this running and running another copy of it in another shell produces: $ erl +B -noshell -run udp Bound to port: #Port<0.289> The bind has succeeded and so the same UDP port is now bound by both processes (with different sockets). Now hitting enter into the netcat terminal produces this in the second window: Msg: {udp,#Port<0.289>,{127,0,0,1},54312,"\n"} Msg: {udp,#Port<0.289>,{127,0,0,1},54312,"\n"} No output is produced in the first window -- under Linux the second binding of the port seems to get the traffic. But if you close the second erl then the traffic will return to the first process. I hope this was useful :-) Sam. From phmander@REDACTED Fri Nov 20 16:43:23 2009 From: phmander@REDACTED (Peter-Henry Mander) Date: Fri, 20 Nov 2009 15:43:23 +0000 Subject: Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 Message-ID: <38e3b9f40911200743p1f211988j9e03af6f3837f300@mail.gmail.com> Hi Gentlepeople, Does anyone else use the Erlang crypto application on Solaris 10, U8? We try to compile Erlang to run as a amd64 binary to access the 64Gb RAM on our servers. I know that crypto will load on Linux amd64, so the driver is capable of doing what's written on the tin. The symptom are appended below. R12B-5 can load the crypto_drv, but R13B0{2,3} cannot. I'm confident that the openssl-0.9.8i library is fine, we use stunnel linked to this and it works. We tried GCC-3.4.3 as supplied by Sun, and tried with GCC-4.4.2, compiling with the option -mcmodel=medium to see if adjusting the memory model would help, to no avail. This was prompted by indications that the relocation error was due to AMD ABI, address register relative memory access was spanning ranges beyond 2Gb in size, and would not fit in the 32bit offset. Help please? Yes, it works on Linux, no I can't consider deploying that option just yet. If compiling Erlang R13B as a 32bit runtime still enables access to the full 64Gb of memory, I may consider this option (I haven't tried yet). Pete. > ~/R12B-5/bin/erl Erlang (BEAM) emulator version 5.6.5 [source] [64-bit] [smp:8] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> application:start(crypto). ok > ~/R13B02/bin/erl Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [kernel-poll:false] Eshell V5.7.3 (abort with ^G) 1> PrivDir = code:priv_dir(crypto). "/Software/R13B02/lib/erlang/lib/crypto-1.6.1/priv" 2> LibDir1 = filename:join([PrivDir, "lib"]). "/Software/R13B02/lib/erlang/lib/crypto-1.6.1/priv/lib" 3> erl_ddll:load_driver(LibDir1, crypto_drv). {error,{open_error,-10}} 4> erl_ddll:format_error({open_error,-10}). "ld.so.1: beam.smp: fatal: relocation error: R_AMD64_PC32: file /Software/R13B02/lib/erlang/lib/crypto-1.6.1/priv/lib/crypto_drv.so: symbol main: value 0x28002cb378c does not fit" > ~/R13B03-2009-11-19/bin/erl Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [kernel-poll:false] Eshell V5.7.4 (abort with ^G) 1> application:start(crypto). =ERROR REPORT==== 20-Nov-2009::15:24:25 === Unable to load crypto_drv. Failed with error: "ld.so.1: beam.smp: fatal: relocation error: R_AMD64_32: file /Software/R13B03-2009-11-19/lib/erlang/lib/crypto-1.6.3/priv/lib/crypto_drv.so: symbol (unknown): value 0xfffffd7ffd514fdf does not fit" OpenSSL might not be installed on this system. {error,{shutdown,{crypto_app,start,[normal,[]]}}} =INFO REPORT==== 20-Nov-2009::15:24:25 === application: crypto exited: {shutdown,{crypto_app,start,[normal,[]]}} type: temporary From samb-bulk@REDACTED Fri Nov 20 17:21:11 2009 From: samb-bulk@REDACTED (Sam Bobroff) Date: Fri, 20 Nov 2009 11:21:11 -0500 Subject: [erlang-questions] Multicast UDP sending question In-Reply-To: <20091120152237.E62803D0D4E@mail.pharos-avantgard.com> References: <20091120152237.E62803D0D4E@mail.pharos-avantgard.com> Message-ID: <4B06C1F7.50604@m5networks.com.au> Valentin Micic wrote: > Don't know what to tell you -- I've tried similar thing from Windoze and got > what I've expected: > > Started two shells with following result: > > FIRST: > > Erlang (BEAM) emulator version 5.5.4 [async-threads:0] > > Eshell V5.5.4 (abort with ^G) > (vhlr@REDACTED)1> gen_udp:open( 9999, [{reuseaddr, true}] ). > {ok,#Port<0.97>} > (vhlr@REDACTED)2> > > SECOND: > > Erlang (BEAM) emulator version 5.5.4 [async-threads:0] > > Eshell V5.5.4 (abort with ^G) > (uscript@REDACTED)1> gen_udp:open( 9999, [{reuseaddr, true}] ). > {error,eaddrinuse} > (uscript@REDACTED)2> > > > Hope this is helpful too. > > V/ That IS quite interesting! I found some documentation on MSDN that seemed to indicate that SO_REUSEADDR was available under Windows, and that it actually did pretty much the same thing as Linux... So if that were the case then Erlang must be doing something differently. I grepped around in the source code and look what I found in "inet_setopts" in "inet_drv.c": case INET_OPT_REUSEADDR: #ifdef __WIN32__ continue; /* Bjorn says */ #else type = SO_REUSEADDR; DEBUGF(("inet_set_opts(%ld): s=%d, SO_REUSEADDR=%d\r\n", (long)desc->port, desc->s,ival)); break; #endif (From R13B02) So! reuseaddr does *nothing* in Erlang under Windows, but on other OSes it does! That certainly explains our different results. Perhaps the Erlang documentation for gen_udp should mention that this flag is unimplemented on Windows? Sam. From dmercer@REDACTED Fri Nov 20 17:28:54 2009 From: dmercer@REDACTED (David Mercer) Date: Fri, 20 Nov 2009 10:28:54 -0600 Subject: [erlang-questions] edoc, erlc and dialyser type spec consistency In-Reply-To: References: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> Message-ID: I agree also with Kostis, but I would also like to request that -type declarations permit recursive data types. Recursive data types are not really that uncommon, and are the reason I stopped using -spec/-type declarations. Another thing I'd like to have is the ability to define a -type as some sort of fun(...), and then in a -spec, be able to say that the spec for the function being spec'd is the type previously defined. E.g., -type funtype() :: fun((args()) -> return_type()). -spec f :: funtype (). Instead of having to retype the spec for every function with the same interface: -spec f(args()) -> return_type(). > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Kenneth Lundin > Sent: Thursday, November 19, 2009 6:03 AM > To: Joe Armstrong > Cc: Erlang; Richard Carlsson; Kostis Sagonas > Subject: Re: [erlang-questions] edoc, erlc and dialyser type spec > consistency > > I agree with Kostis answers and can also inform you that we are working > on a solution for a) below. > > The plan is: > Edoc will then parse and use the same -spec and -type as Dialyzer. > If a file contains both -spec/-type and edoc comments with @spec and @type > you will have to choose which notation to use (as a compatiblity mode) for > already existing edoc markup. > > /Kenneth Erlang/OTP Ericsson > > > My suggestion: > > > > ? ?a) There should be ONE type/spec parser as part of the OTP > distribution > > ? ? ? everybody should use this one and not their own > > > > ? ?b) Specs and types must finish with DOT WHITESPACE. > > > > ? ?c) Quoting atoms in type specs is illegal. > > > > ? ?d) Function names are required (ie not optional as in edoc) > > > > ? ?e) Short form of type names are legal > > ? ? ? bool(), int(). > > > > There rules obey certain meta principles: > > > > ? ?meta principle 1) "don't wear your fingers out" > > > > ? - I want to type > > > > ? ? ? ?foo(X::bool()) -> ok. > > > > ? ?and not > > > > ? ? ? ?foo(X::boolean()) -> 'ok' > > > > ? ?meta principle 2) "optional is bad" > > > > ? ? ? ?ie DOT whitespace is not optional - dropping the function > > ? ? ? ?symbol and inferring it from the next function is bad. > > > > ? ? ? ?(aside: the optional ';' in Javascript totally screwed up > > ? ? ? ? ? ? ? ?the parser and is responsible for lager numbers > > ? ? ? ? ? ? ? ?or really weird errors) > > > > ? ?meta principle 3) "make type specs etc look like Erlang" > > > > ? ?Similar syntax if possible, similar conventions. The sudden > appearance > > ? ?of quotes, optional '.' violates the principle of least astonishment. > > > > > > /Joe > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org From egil@REDACTED Fri Nov 20 17:28:26 2009 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Fri, 20 Nov 2009 17:28:26 +0100 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <38e3b9f40911200743p1f211988j9e03af6f3837f300@mail.gmail.com> References: <38e3b9f40911200743p1f211988j9e03af6f3837f300@mail.gmail.com> Message-ID: <4B06C3AA.2070801@erix.ericsson.se> Hi Pete, Could you send us the build log please so we can investigate this problem. We are specifically interested in the log entries regarding building crypto_drv.so. Regards, Bj?rn-Egil Erlang/OTP Peter-Henry Mander wrote: > Hi Gentlepeople, > > Does anyone else use the Erlang crypto application on Solaris 10, U8? > We try to compile Erlang to run as a amd64 binary to access the 64Gb > RAM on our servers. I know that crypto will load on Linux amd64, so > the driver is capable of doing what's written on the tin. > > The symptom are appended below. R12B-5 can load the crypto_drv, but > R13B0{2,3} cannot. > > I'm confident that the openssl-0.9.8i library is fine, we use stunnel > linked to this and it works. > > We tried GCC-3.4.3 as supplied by Sun, and tried with GCC-4.4.2, > compiling with the option -mcmodel=medium to see if adjusting the > memory model would help, to no avail. This was prompted by indications > that the relocation error was due to AMD ABI, address register > relative memory access was spanning ranges beyond 2Gb in size, and > would not fit in the 32bit offset. > > Help please? Yes, it works on Linux, no I can't consider deploying > that option just yet. If compiling Erlang R13B as a 32bit runtime > still enables access to the full 64Gb of memory, I may consider this > option (I haven't tried yet). > > Pete. > >> ~/R12B-5/bin/erl > Erlang (BEAM) emulator version 5.6.5 [source] [64-bit] [smp:8] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.6.5 (abort with ^G) > 1> application:start(crypto). > ok > > >> ~/R13B02/bin/erl > Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:8:8] [rq:8] > [async-threads:0] [kernel-poll:false] > > Eshell V5.7.3 (abort with ^G) > 1> PrivDir = code:priv_dir(crypto). > "/Software/R13B02/lib/erlang/lib/crypto-1.6.1/priv" > 2> LibDir1 = filename:join([PrivDir, "lib"]). > "/Software/R13B02/lib/erlang/lib/crypto-1.6.1/priv/lib" > 3> erl_ddll:load_driver(LibDir1, crypto_drv). > {error,{open_error,-10}} > 4> erl_ddll:format_error({open_error,-10}). > "ld.so.1: beam.smp: fatal: relocation error: R_AMD64_PC32: file > /Software/R13B02/lib/erlang/lib/crypto-1.6.1/priv/lib/crypto_drv.so: > symbol main: value 0x28002cb378c does not fit" > > >> ~/R13B03-2009-11-19/bin/erl > Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] [rq:8] > [async-threads:0] [kernel-poll:false] > > Eshell V5.7.4 (abort with ^G) > 1> application:start(crypto). > > =ERROR REPORT==== 20-Nov-2009::15:24:25 === > Unable to load crypto_drv. Failed with error: > "ld.so.1: beam.smp: fatal: relocation error: R_AMD64_32: file > /Software/R13B03-2009-11-19/lib/erlang/lib/crypto-1.6.3/priv/lib/crypto_drv.so: > symbol (unknown): value 0xfffffd7ffd514fdf does not fit" > OpenSSL might not be installed on this system. > {error,{shutdown,{crypto_app,start,[normal,[]]}}} > > =INFO REPORT==== 20-Nov-2009::15:24:25 === > application: crypto > exited: {shutdown,{crypto_app,start,[normal,[]]}} > type: temporary > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org From v@REDACTED Fri Nov 20 18:15:25 2009 From: v@REDACTED (Valentin Micic) Date: Fri, 20 Nov 2009 19:15:25 +0200 Subject: [erlang-questions] Multicast UDP sending question In-Reply-To: <4B06C1F7.50604@m5networks.com.au> Message-ID: <20091120171535.D54453D0D4E@mail.pharos-avantgard.com> Sam, SO_REUSADDR is a very *old* option with a definite semantics (and behavior) when it comes to TCP: 1) Servers are reusing the same local port (as in listeners) because clients are using different IP address and port. 2) Clients may reuse the same port (as in FTP client, for example) because servers utilizes different port (e.g. port 20 for commands and port 21 for transfer -- or was it the other way around). 3) Some other listener/connections may be started because connection enters TIME_WAIT state. This is possible because any association is presented using 5 elements: {Protocol, LocalIP, LocalPort, RemoteIP, RemotePort}. So it is easy to see that TCP may support local process reuse if and only if the remote half of the connection makes the association unique. In other words: {tcp, "127.0.0.1", 5001, "127.0.0.1", 20} {tcp, "127.0.0.1", 5001, "127.0.0.1", 21} Situation with UDP is not clear as there is no connection, hence no known remote half of the association to make it unique. Your test indicates -- even though you've managed to bind two different processes (e.g. Erlang runtimes) to the same UDP port; the behavior is unpredictable from the process' point of view. In other words, the process may receive traffic under some circumstances (say, started first), and not under another (e.g. started second, but may receive traffic if first process terminates). It would be interesting to test two Erlang runtimes (or any two independent processes) attempting to use the same TCP port as listeners specifying SO_REUSEADDR on both sockets. This situation would closely resemble UDP situation as it does not have remote half of the association, but only local resources: {tcp, LocalIp, LocalPort}. I remember that I've tested this once and it didn't work on any of the available unices: SunOS, HP/UX, DG/UX (don't remember if I tested it on SCO). V/ -----Original Message----- From: Sam Bobroff [mailto:samb-bulk@REDACTED] Sent: 20 November 2009 06:21 PM To: Valentin Micic Cc: 'Jarrod Roberson'; 'Torben Hoffmann'; 'Erlang' Subject: Re: [erlang-questions] Multicast UDP sending question Valentin Micic wrote: > Don't know what to tell you -- I've tried similar thing from Windoze and got > what I've expected: > > Started two shells with following result: > > FIRST: > > Erlang (BEAM) emulator version 5.5.4 [async-threads:0] > > Eshell V5.5.4 (abort with ^G) > (vhlr@REDACTED)1> gen_udp:open( 9999, [{reuseaddr, true}] ). > {ok,#Port<0.97>} > (vhlr@REDACTED)2> > > SECOND: > > Erlang (BEAM) emulator version 5.5.4 [async-threads:0] > > Eshell V5.5.4 (abort with ^G) > (uscript@REDACTED)1> gen_udp:open( 9999, [{reuseaddr, true}] ). > {error,eaddrinuse} > (uscript@REDACTED)2> > > > Hope this is helpful too. > > V/ That IS quite interesting! I found some documentation on MSDN that seemed to indicate that SO_REUSEADDR was available under Windows, and that it actually did pretty much the same thing as Linux... So if that were the case then Erlang must be doing something differently. I grepped around in the source code and look what I found in "inet_setopts" in "inet_drv.c": case INET_OPT_REUSEADDR: #ifdef __WIN32__ continue; /* Bjorn says */ #else type = SO_REUSEADDR; DEBUGF(("inet_set_opts(%ld): s=%d, SO_REUSEADDR=%d\r\n", (long)desc->port, desc->s,ival)); break; #endif (From R13B02) So! reuseaddr does *nothing* in Erlang under Windows, but on other OSes it does! That certainly explains our different results. Perhaps the Erlang documentation for gen_udp should mention that this flag is unimplemented on Windows? Sam. From rtrlists@REDACTED Fri Nov 20 18:18:13 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 20 Nov 2009 17:18:13 +0000 Subject: [erlang-questions] ubuntu upstart job to run an erlang application In-Reply-To: <998105.35733.qm@web112602.mail.gq1.yahoo.com> References: <998105.35733.qm@web112602.mail.gq1.yahoo.com> Message-ID: <6a3ae47e0911200918n217246b7l158774d70be5a58d@mail.gmail.com> On Fri, Nov 20, 2009 at 2:07 PM, Pablo Platt wrote: > > When I'm trying to start the job and check its status I'm getting: > $sudo start myweb > myweb start/running, process 6682 > $sudo status myweb > myweb stop/waiting > > Looks to me as if your app might be crashing just after startup. D'you have any logs in your app you could look at? Robby From kostis@REDACTED Fri Nov 20 19:35:37 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 20 Nov 2009 20:35:37 +0200 Subject: [erlang-questions] edoc, erlc and dialyser type spec consistency In-Reply-To: References: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> Message-ID: <4B06E179.3080802@cs.ntua.gr> David Mercer wrote: > I agree also with Kostis, but I would also like to request that -type > declarations permit recursive data types. I also want that... Be my guest ;-) > Recursive data types are not > really that uncommon, and are the reason I stopped using -spec/-type > declarations. But in the meantime, I do not see the fact that you cannot declare recursive data types as any show stopper. I usually declare them as follows: -type mytree() :: 'empty' | {'tree', atom(), _, _}. % this overapproximates % {'tree', atom(), mytree(), mytree()}. (or you can unfold the above definition as many levels as you feel like) > Another thing I'd like to have is the ability to define a -type as some sort > of fun(...), and then in a -spec, be able to say that the spec for the > function being spec'd is the type previously defined. E.g., > > -type funtype() :: fun((args()) -> return_type()). > -spec f :: funtype (). > > Instead of having to retype the spec for every function with the same > interface: > > -spec f(args()) -> return_type(). You managed to confuse me here... How can you have a "reusable" funtype? You cannot have functions with the same arguments in an Erlang module. Can you give us a concrete example of what you want to achieve and you cannot? Kostis From colm.dougan@REDACTED Fri Nov 20 19:59:20 2009 From: colm.dougan@REDACTED (Colm Dougan) Date: Fri, 20 Nov 2009 18:59:20 +0000 Subject: [erlang-questions] ubuntu upstart job to run an erlang application In-Reply-To: <998105.35733.qm@web112602.mail.gq1.yahoo.com> References: <998105.35733.qm@web112602.mail.gq1.yahoo.com> Message-ID: <24d4f39c0911201059s6e4530f6i8db9e4d0502fb6ff@mail.gmail.com> On Fri, Nov 20, 2009 at 2:07 PM, Pablo Platt wrote: > > So my mochiweb app doesn't run. > When I'm using the start.sh script generated with the skeleton my app > run fine. Is the 'HOME' environment variable defined? This is a gotcha I've run into in the past. colm@REDACTED:~$ env - erl erlexec: HOME must be set However : colm@REDACTED:~$ env - HOME="/home/colm" erl Eshell V5.6.5 (abort with ^G) 1> Colm From dmercer@REDACTED Fri Nov 20 20:31:22 2009 From: dmercer@REDACTED (David Mercer) Date: Fri, 20 Nov 2009 13:31:22 -0600 Subject: [erlang-questions] edoc, erlc and dialyser type spec consistency In-Reply-To: <4B06E179.3080802@cs.ntua.gr> References: <9b08084c0911190155o7343369bx9975156cbfe35f31@mail.gmail.com> <4B06E179.3080802@cs.ntua.gr> Message-ID: Kostis Sagonas wrote: > You managed to confuse me here... How can you have a "reusable" funtype? > You cannot have functions with the same arguments in an Erlang module. > Can you give us a concrete example of what you want to achieve and you > cannot? Let's say I am writing a parsing combinator library. I define a parser as: -type parser(A) :: fun((state(A)) -> parse_result(A)). The parameter is, say, the type of the tokens or something. So a parser that parses characters might have the following type: -type char_parser() :: parser(char()). Now I have a combinator that given two parser()'s, creates a parser that parses the first parser() followed by the second: -spec seq(parser(A), parser(A)) -> parser(A). That's good, until I want to start defining parsers. Every parser must share the same specification, as laid out by parser(): -spec parse_xml_tag(state(char())) -> parse_result(char()). -spec parse_edi_segment(state(char())) -> parse_result(char()). -spec parse_whitespace(state(char())) -> parse_result(char()). And so on. What I really want to say is that these functions are in accordance with the type specification of parser(_) (or, in this case, subtype char_parser()): -spec parse_xml_tag :: char_parser(). -spec parse_edi_segment :: char_parser(). -spec parse_whitespace :: char_parser(). That conveys my intent better, as a reader does not have to compare the function specs to the parser() type to see if he can pass it to seq/2 or not. Did that make sense? Cheers, David From torben.lehoff@REDACTED Fri Nov 20 21:09:23 2009 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 20 Nov 2009 21:09:23 +0100 Subject: [erlang-questions] Re: [neotoma] [ANN] Neotoma 1.3 In-Reply-To: References: <4AFDF585.6010709@gmail.com> Message-ID: Hi Tony, For what it is worth: I have implemented a parser for WikiCreole using Neotoma and it works quite fine. Not sure if that fits your notion of large, though ;-) This was with Neotoma 1.1 and right now I am just dying to upgrade to 1.3 and throw out a bunch of code that really has to be inline transformations. Cheers, Torben On Sat, Nov 14, 2009 at 06:33, Tony Arcieri wrote: > Really glad to see an inline transformation syntax as this is one of my > favorite features of Leex. > > Now that you are producing Erlang code AOT, will records be supported as > they are in Leex? (since records are output verbatim in the resulting > Erlang > code from the original Leex files, and thus can get processed by EPP when > the resulting Erlang code is compiled by the normal Erlang compiler) > > I have continued to develop my attempts at a minimalized, more Erlang-like > Reia using Leex because I felt attempts at a Neotoma-based parser were > killing my developmental gumption. I would still very much like to switch > to Neotoma if I feel it addresses all my needs. I still worry about using > a > PEG for a large grammar, and I still do not have a good understanding of > how > PEGs work. > > -- > Tony Arcieri > Medioh/Nagravision > -- http://www.linkedin.com/in/torbenhoffmann From per@REDACTED Fri Nov 20 21:54:12 2009 From: per@REDACTED (Per Hedeland) Date: Fri, 20 Nov 2009 21:54:12 +0100 (CET) Subject: [erlang-questions] R13B02 on 8/16 core box: all TCP communication hangs/frozen In-Reply-To: <1d0fb8fc0911190635p5adbeb1em5111f06d1ecb0a6d@mail.gmail.com> Message-ID: <200911202054.nAKKsCP3054204@pluto.hedeland.org> Luke Gorrie wrote: > >The SYN_RECV sockets make me curious. Linux seems to have a second >backlog queue for when the normal one fills up. Connections in this >queue seem to send a SYN+ACK (because the client reaches ESTABLISHED) >but wait in SYN_RECV. The code in net/ipv4/tcp_ipv4.c's >tcp_v4_conn_request refers to this as a "syn queue" containing "warm >entries", but I haven't dug deep enough to see how it really works. >Does anyone know? (Per Hedeland? :-)) Sorry Luke, I'm losing touch with the Linux kernel internals these days (though I don't really mean that I'm sorry about that:-). Up until what you wrote in this paragraph, I was assuming that you were seeing syncookies in action, and that they were now enabled by default on Linux, and you just didn't realize it (they are on FreeBSD - they even have their own man page!:-). But I guess not... --Per From per@REDACTED Fri Nov 20 22:00:51 2009 From: per@REDACTED (Per Hedeland) Date: Fri, 20 Nov 2009 22:00:51 +0100 (CET) Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <38e3b9f40911200743p1f211988j9e03af6f3837f300@mail.gmail.com> Message-ID: <200911202100.nAKL0o5K054276@pluto.hedeland.org> Peter-Henry Mander wrote: >"ld.so.1: beam.smp: fatal: relocation error: R_AMD64_PC32: file >/Software/R13B02/lib/erlang/lib/crypto-1.6.1/priv/lib/crypto_drv.so: >symbol main: value 0x28002cb378c does not fit" >From my limited experience of deciphering completely incomprehensible linker error messages (which seems to be a universal feature across *nixes), I would guess that you have a 32-bit beam and a 64-bit crypto_drv.so. What does 'file' say? >Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] [rq:8] Of course this is just what the build thought that it built.:-) --Per Hedeland From phmander@REDACTED Fri Nov 20 22:23:47 2009 From: phmander@REDACTED (Peter-Henry Mander) Date: Fri, 20 Nov 2009 21:23:47 +0000 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <200911202100.nAKL0o5K054276@pluto.hedeland.org> References: <38e3b9f40911200743p1f211988j9e03af6f3837f300@mail.gmail.com> <200911202100.nAKL0o5K054276@pluto.hedeland.org> Message-ID: <38e3b9f40911201323v783471c4k54ad226283b828a8@mail.gmail.com> Hi Per, A useful point, but no. I've been careful to ensure that the build is 64 bit: > /bin/file ./R13B03-2009-11-19-fix/lib/erlang/erts-5.7.4/bin/beam.smp ./R13B03-2009-11-19-fix/lib/erlang/erts-5.7.4/bin/beam.smp: ELF 64-bit LSB executable AMD64 Version 1, dynamically linked, not stripped > /bin/file ./R13B03-2009-11-19-fix/lib/erlang/lib/crypto-1.6.3/priv/lib/crypto_drv.so ./R13B03-2009-11-19-fix/lib/erlang/lib/crypto-1.6.3/priv/lib/crypto_drv.so: ELF 64-bit LSB dynamic lib AMD64 Version 1, dynamically linked, not stripped I'm not saying that nothing slipped the net, I've not checked every single file, but at least not the runtime nor the crypto driver. Bj?rn-Egil Dahlberg and Sverker Eriksson suggested that linker flags "-Wl,-Bsymbolic" applied to crypto_drv.so build might offer respite, but this was unsuccessful. I may have stuck in the wrong place, mind you. Pete. On Fri, Nov 20, 2009 at 9:00 PM, Per Hedeland wrote: > From my limited experience of deciphering completely incomprehensible > linker error messages (which seems to be a universal feature across > *nixes), I would guess that you have a 32-bit beam and a 64-bit > crypto_drv.so. What does 'file' say? On Fri, Nov 20, 2009 at 5:18 PM, Sverker Eriksson wrote: > On 64-bit Linux we added gcc option "-Wl,-Bsymbolic" to get crypto to link > in R13B03. Could be worth a try on solaris as well. From per@REDACTED Fri Nov 20 22:58:55 2009 From: per@REDACTED (Per Hedeland) Date: Fri, 20 Nov 2009 22:58:55 +0100 (CET) Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <38e3b9f40911201323v783471c4k54ad226283b828a8@mail.gmail.com> Message-ID: <200911202158.nAKLwtl7055453@pluto.hedeland.org> Peter-Henry Mander wrote: > >A useful point, but no. I've been careful to ensure that the build is 64 bit: OK, it was just a first guess.:-) >Bj?rn-Egil Dahlberg and Sverker Eriksson suggested that linker flags >"-Wl,-Bsymbolic" applied to crypto_drv.so build might offer respite, >but this was unsuccessful. I may have stuck in the wrong place, mind >you. Hm, they should of course know more about the Erlang/OTP build than I do, but the few words that are recognizable from the error message, like "relocation error" and "does not fit", don't really suggest that it is an issue with symbol resolution I think. My next guess would be a -fpic vs -fPIC problem when compiling crypto_drv - in many cases / architectures you can "get away" with using -fpic which is supposedly slightly more efficient, but my understanding is that -fPIC should always work and is sometimes required. Of course if your build actually used -fPIC (likely), this is moot. But then I recognized one more word from the message: "main". This kind of contradicts what I wrote above, but surely there shouldn't be a "main" symbol in crypto_drv. What does 'nm' say? --Per From wallentin.dahlberg@REDACTED Fri Nov 20 23:25:28 2009 From: wallentin.dahlberg@REDACTED (Wallentin Dahlberg) Date: Fri, 20 Nov 2009 23:25:28 +0100 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <200911202158.nAKLwtl7055453@pluto.hedeland.org> References: <38e3b9f40911201323v783471c4k54ad226283b828a8@mail.gmail.com> <200911202158.nAKLwtl7055453@pluto.hedeland.org> Message-ID: We haven't had time to investigate this problem yet. (Lots of release stuff going on today). However, it seems like similar problems exists on opensolaris + AMD64. For instance: http://bugs.opensolaris.org/bugdatabase/view_bug.do;jsessionid=a68c8de187c30929fb79755edd017?bug_id=5102797 This would be kind of bad. I don't have your build log in front of me so I can't see if this is the case but similar problems has been solved with -fPIC and -shared and not -G when using gcc on solaris. LDFLAGS += -shared CFLAGS += -fPIC // Bj?rn-Egil 2009/11/20 Per Hedeland > Peter-Henry Mander wrote: > > > >A useful point, but no. I've been careful to ensure that the build is 64 > bit: > > OK, it was just a first guess.:-) > > >Bj?rn-Egil Dahlberg and Sverker Eriksson suggested that linker flags > >"-Wl,-Bsymbolic" applied to crypto_drv.so build might offer respite, > >but this was unsuccessful. I may have stuck in the wrong place, mind > >you. > > Hm, they should of course know more about the Erlang/OTP build than I > do, but the few words that are recognizable from the error message, like > "relocation error" and "does not fit", don't really suggest that it is > an issue with symbol resolution I think. > > My next guess would be a -fpic vs -fPIC problem when compiling > crypto_drv - in many cases / architectures you can "get away" with using > -fpic which is supposedly slightly more efficient, but my understanding > is that -fPIC should always work and is sometimes required. Of course if > your build actually used -fPIC (likely), this is moot. > > But then I recognized one more word from the message: "main". This kind > of contradicts what I wrote above, but surely there shouldn't be a > "main" symbol in crypto_drv. What does 'nm' say? > > --Per > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From chandrashekhar.mullaparthi@REDACTED Fri Nov 20 23:33:51 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 20 Nov 2009 22:33:51 +0000 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <200911202158.nAKLwtl7055453@pluto.hedeland.org> References: <38e3b9f40911201323v783471c4k54ad226283b828a8@mail.gmail.com> <200911202158.nAKLwtl7055453@pluto.hedeland.org> Message-ID: 2009/11/20 Per Hedeland > Peter-Henry Mander wrote: > > > >A useful point, but no. I've been careful to ensure that the build is 64 > bit: > > OK, it was just a first guess.:-) > But then I recognized one more word from the message: "main". This kind > of contradicts what I wrote above, but surely there shouldn't be a > "main" symbol in crypto_drv. What does 'nm' say? > > Here is the output of nm on the build Peter is working on. > nm crypto_drv.so U AES_cbc_encrypt U AES_cfb128_encrypt U AES_set_decrypt_key U AES_set_encrypt_key U BF_cfb64_encrypt U BF_set_key U BN_CTX_free U BN_CTX_new U BN_add U BN_bin2bn U BN_bn2bin U BN_free U BN_mod_exp U BN_new U BN_num_bits U BN_pseudo_rand_range U BN_sub U CRYPTO_cleanup_all_ex_data U CRYPTO_num_locks U CRYPTO_set_dynlock_create_callback U CRYPTO_set_dynlock_destroy_callback U CRYPTO_set_dynlock_lock_callback U CRYPTO_set_id_callback U CRYPTO_set_locking_callback U CRYPTO_set_mem_functions U DES_ede3_cbc_encrypt U DES_ncbc_encrypt U DES_set_key U DH_check U DH_compute_key U DH_free U DH_generate_key U DH_generate_parameters U DH_new U DH_size U DSA_free U DSA_new U DSA_sign U DSA_size U DSA_verify U MD4 U MD4_Final U MD4_Init U MD4_Update U MD5 U MD5_Final U MD5_Init U MD5_Update U RAND_pseudo_bytes U RC2_cbc_encrypt U RC2_set_key U RC4 U RC4_set_key U RSA_free U RSA_new U RSA_private_decrypt U RSA_private_encrypt U RSA_public_decrypt U RSA_public_encrypt U RSA_sign U RSA_size U RSA_verify U SHA1 U SHA1_Final U SHA1_Init U SHA1_Update U SSLeay U SSLeay_version 0000000000017ff8 D _DYNAMIC 0000000000018318 b _END_ 0000000000017cf8 D _GLOBAL_OFFSET_TABLE_ w _Jv_RegisterClasses 00000000000023f8 T _PROCEDURE_LINKAGE_TABLE_ 0000000000000000 r _START_ 00000000000181c0 d __CTOR_END__ 00000000000181b8 d __CTOR_LIST__ 00000000000181d0 d __DTOR_END__ 00000000000181c8 d __DTOR_LIST__ 00000000000001d0 r __EH_FRAME_BEGIN__ 00000000000004d0 r __FRAME_END__ 00000000000181d8 d __JCR_END__ 00000000000181d8 d __JCR_LIST__ 00000000000181a0 D ___Argv w __deregister_frame_info_bases 0000000000007b70 t __do_global_ctors_aux 0000000000002a10 t __do_global_dtors_aux 00000000000181b0 D __dso_handle U __fpstart 00000000000181a8 D __longdouble_used w __register_frame_info_bases 00000000000182a8 D _edata 0000000000018318 B _end 0000000000018198 D _environ 0000000000007cf7 R _etext U _exit 0000000000007bc0 T _fini 0000000000007bb0 T _init 0000000000007bcc R _lib_version 0000000000002a00 T _mcount 0000000000002980 T _start U atexit 00000000000182c0 b completed.1 0000000000002c9f t control 0000000000018200 d crypto_driver_entry U driver_alloc U driver_alloc_binary U driver_free 0000000000002ab8 T driver_init U driver_realloc U driver_realloc_binary U driver_system_info 0000000000007797 t dyn_create_function 00000000000077d8 t dyn_destroy_function 00000000000077b4 t dyn_lock_function 0000000000018198 V environ U erl_drv_rwlock_create U erl_drv_rwlock_destroy U erl_drv_rwlock_rlock U erl_drv_rwlock_runlock U erl_drv_rwlock_rwlock U erl_drv_rwlock_rwunlock U erl_drv_thread_self U exit 0000000000002bfb t finish 0000000000002a60 t frame_dummy 00000000000077f6 t hmac_md5 00000000000079b1 t hmac_sha1 000000000000778c t id_function 0000000000002ac5 t init 0000000000007beb r libname.0 0000000000018310 b lock_vec 0000000000007729 t locking 00000000000076f2 t locking_function U main U memcpy U memset 00000000000182e0 b object.2 00000000000181e0 d p.0 0000000000007683 t return_binary 00000000000075ff t return_binary_shrink U set_port_control_flags 0000000000002c70 t start 0000000000002c95 t stop U strlen cheers, Chandru From phmander@REDACTED Fri Nov 20 23:43:05 2009 From: phmander@REDACTED (Peter-Henry Mander) Date: Fri, 20 Nov 2009 22:43:05 +0000 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: References: <38e3b9f40911201323v783471c4k54ad226283b828a8@mail.gmail.com> <200911202158.nAKLwtl7055453@pluto.hedeland.org> Message-ID: <38e3b9f40911201443xa03107ewa376b3ed0c99f733@mail.gmail.com> Yes, that bug looks awfully similar to mine, and the explanation also concurs with my fuzzy understanding of what is going on. "Will not fix" since 2004? Kind of bad, indeed. I tried -mcmodel=medium, which didn't solve the issue, and -mcmodel=large which broke *all* of the erl_interface C file compilation. Aside: The "medium/large memory model" is only available with GCC-4 for shared libraries, not GCC-3, and that led me on to another, different battle to compile GCC version 4. I succeeded thanks to Gilles Dauphin: http://www.infres.enst.fr/~dauphin/pkgsrcmywork/ The -fPIC flag is used pretty much everywhere I looked, and flew past very often as I went cross-eyed gazing at the compilation output, Matrix style. The -shared and -G flags, I don't recall seeing. I may try looking for this on Monday. Pete. On Fri, Nov 20, 2009 at 10:25 PM, Wallentin Dahlberg wrote: > We haven't had time to investigate this problem yet. (Lots of release stuff > going on today). > > However, it seems like similar problems exists on opensolaris + AMD64. > For instance: > http://bugs.opensolaris.org/bugdatabase/view_bug.do;jsessionid=a68c8de187c30929fb79755edd017?bug_id=5102797 > > This would be kind of bad. > > I don't have your build log in front of me so I can't see if this is the > case but similar problems has been solved with -fPIC and -shared and not -G > when using gcc on solaris. > > LDFLAGS += -shared > CFLAGS += -fPIC > > // Bj?rn-Egil From per@REDACTED Sat Nov 21 00:35:25 2009 From: per@REDACTED (Per Hedeland) Date: Sat, 21 Nov 2009 00:35:25 +0100 (CET) Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: Message-ID: <200911202335.nAKNZPkB057859@pluto.hedeland.org> Chandru wrote: >> >Here is the output of nm on the build Peter is working on. > >> nm crypto_drv.so > U _exit > U atexit > U exit > U main The presence of any of those seem seriously broken for a shared object I think - FWIW, our build of ancient but working R10B-10, 32-bit, gcc-3.4.3, Solaris 10, has none of them. I believe they're from crt1.o, which should only be linked into an executable. Our build log (lines folded): gcc -c -o ../priv/obj/sparc-sun-solaris2.10/crypto_drv.o -g -O2 \ -I/export/home/lab/build/otp/erts/sparc-sun-solaris2.10 \ -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -fPIC -I/usr/sfw/include \ -I/export/home/lab/build/otp/erts/emulator/beam \ -I/export/home/lab/build/otp/erts/emulator/sys/unix crypto_drv.c /usr/ucb/install -c -d ../priv/lib/sparc-sun-solaris2.10 ld -G -L/usr/sfw/lib -R/usr/sfw/lib \ -o ../priv/lib/sparc-sun-solaris2.10/crypto_drv.so \ ../priv/obj/sparc-sun-solaris2.10/crypto_drv.o -lcrypto Those are likely *not* correct for current versions or 64-bit though. And I believe Bj?rn-Egil has a point about -G vs -shared - but -G is a (Solaris) 'ld' option (and so is -R, though GNU ld may do the right thing with it if you are lucky), while -shared is a gcc (and GNU ld) option. I.e. basically 'ld -G -L -R' may need to be replaced with something like 'gcc -shared -L -Wl,-R,', assuming that you are using Solaris ld (which I think is the norm for gcc on Solaris). Certainly switching from ld to gcc -shared for building shared objects has worked well for us on some more modern *nixes. --Per From zabrane3@REDACTED Sat Nov 21 01:02:04 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Sat, 21 Nov 2009 01:02:04 +0100 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> <5c493e530911161651w41e46aecu823bd4d36236815f@mail.gmail.com> Message-ID: <18a1db030911201602n24ee3bb4ob4838492120f84ee@mail.gmail.com> Hi List ! While trying to learn how to write a simple TCP Web Server in Erlang which only dump what it gets to stdout, I realize that time to time, the HTTP requests get truncated when reaching the server. My main socket loop looks like: ------------------------------------------- -define(TCP_OPTIONS,[binary, {packet, 0}, {active, false}, {reuseaddr, true}]). ... loop_recv(Socket) case gen_tcp:recv(Socket, 0) of {ok, BinData} -> %% here, I'm assuming that all the HTTP request (Headers + Body) is in "BinData". Hope I'm right. io:format("BinData: ~p~n", []), ok; NotOK -> error_logger:info_report([{"gen_tcp:recv/2", NotOK}]), error end. ------------------------------------------- For some requests, the "io:format" prints a truncated data (in BinData): <<" http://www.foo.tv/images/v30/LoaderV3.swf?loop=false&quality=high&request=357&HTTP/1.0\r\nHost: www.foo.tv\r\nUser-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; fr; rv:1.9.0.2) Gecko/2008090512 Firefox/3.0.2\r\nAccept: text/h">> As you can see, the request isn't complete "... Accept: text/h". Am I doing somthing wrong? How can I fix it please? Regards Zabrane 2009/11/17 Tony Rogvall > Do not forget about {active, once} mode. > {active,once} will receive one message (depends on buffer size etc) > the it will switch to passive mode. To get the next message you use > inet:setopts(Socket, [{active,once}]) to activate it again. This mode > enables > a selective receive at the same time as it enables flow control. > > /Tony > > > On 17 nov 2009, at 01.51, Ngoc Dao wrote: > > >> From inet's doc: > > > http://www1.erlang.org/documentation/doc-4.9.1/lib/kernel-2.4.1/doc/html/inet.html > > > > If the active option is true, which is the default, everything > > received from the socket will be sent as messages to the receiving > > process. If the active option is set to false (passive mode), the > > process must explicitly receive incoming data by calling > > gen_tcp:recv/N or gen_udp:recv/N (depending on the type of socket). > > Note: Passive mode provides flow control; the other side will not be > > able send faster than the receiver can read. Active mode provides no > > flow control; a fast sender could easily overflow the receiver with > > incoming messages. Use active mode only if your high-level protocol > > provides its own flow control (for instance, acknowledging received > > messages) or the amount of data exchanged is small. > > > > > > On Tue, Nov 17, 2009 at 2:59 AM, ERLANG wrote: > >> Hi Chandru ! > >> > >> That's fix my problem. Thanks. > >> While googling a bit, I found two ways to read from the Socket: > >> > >> recv(Socket, Bin) -> > >> receive > >> {tcp, Socket, B} -> > >> io:format(".", []), > >> recv(Socket, concat_binary([Bin, B])); > >> {tcp_closed, Socket} -> > >> {ok, Bin}; > >> Other -> > >> {error, {socket, Other}} > >> after > >> ?TIMEOUT -> > >> {error, {socket, timeout}} > >> end. > >> > >> % version 2 with "gen_tcp:recv" > >> recv2(Socket, Bin) -> > >> case gen_tcp:recv(Socket, 0, ?TIMEOUT) of > >> {ok, B} -> > >> io:format(".", []), > >> recv(Socket, concat_binary([Bin, B])); > >> {error, closed} -> > >> {ok, Bin}; > >> {error, timeout} -> > >> {error, {socket, timeout}}; > >> Other -> > >> {error, {socket, Other}} > >> end. > >> > >> > >> Which one is the best in my case (see below: fetch.erl)? > >> > >> Regards > >> Zabrane > >> > >> Le 16 nov. 09 ? 18:53, Chandru a ?crit : > >> > >>> You are expecting the server to indicate end of response by closing the > >>> connection, but because you specify HTTP/1.1 in the request, the server > is > >>> holding up your connection, and you are timing out. Try replacing > HTTP/1.1 > >>> with HTTP/1.0 in your request, or parse the response to detect end of > >>> response. > >>> > >>> cheers > >>> Chandru > >>> > >>> 2009/11/16 zabrane Mikael > >>> > >>>> Hi List ! > >>>> > >>>> New to Erlang, I'm trying to implement a simple URL fetcher. > >>>> Here's my code (please, feel free to correct it if you find any bug or > >>>> know > >>>> a better approach): > >>>> > >>>> > >>>> > >>>> > 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8---- > >>>> -module(fetch). > >>>> > >>>> -export([url/1]). > >>>> > >>>> -define(TIMEOUT, 7000). > >>>> -define(TCP_OPTS, [binary, {packet, raw}, {nodelay, true}, > >>>> {active, true}]). > >>>> > >>>> url(Url) -> > >>>> {ok, _Tag, Host, Port} = split_url(Url), > >>>> > >>>> Hdrs = [], > >>>> Request = ["GET ", Url, " HTTP/1.1\r\n", Hdrs, "\r\n\r\n"], > >>>> > >>>> case catch gen_tcp:connect(Host, Port, ?TCP_OPTS) of > >>>> {'EXIT', Why} -> > >>>> {error, {socket_exit, Why}}; > >>>> {error, Why} -> > >>>> {error, {socket_error, Why}}; > >>>> {ok, Socket} -> > >>>> gen_tcp:send(Socket, list_to_binary(Request)), > >>>> recv(Socket, list_to_binary([])) > >>>> end. > >>>> > >>>> recv(Socket, Bin) -> > >>>> receive > >>>> {tcp, Socket, B} -> > >>>> io:format(".", []), > >>>> recv(Socket, concat_binary([Bin, B])); > >>>> {tcp_closed, Socket} -> > >>>> {ok, Bin}; > >>>> Other -> > >>>> {error, {socket, Other}} > >>>> after > >>>> ?TIMEOUT -> > >>>> {error, {socket, timeout}} > >>>> end. > >>>> > >>>> > >>>> split_url([$h,$t,$t,$p,$:,$/,$/|T]) -> split_url(http, T); > >>>> split_url(_X) -> {error, split_url}. > >>>> > >>>> split_url(Tag, X) -> > >>>> case string:chr(X, $:) of > >>>> 0 -> > >>>> Port = 80, > >>>> case string:chr(X,$/) of > >>>> 0 -> > >>>> {ok, Tag, X, Port}; > >>>> N -> > >>>> Site = string:substr(X,1,N-1), > >>>> {ok, Tag, Site, Port} > >>>> end; > >>>> N1 -> > >>>> case string:chr(X,$/) of > >>>> 0 -> > >>>> error; > >>>> N2 -> > >>>> PortStr = string:substr(X,N1+1, N2-N1-1), > >>>> case catch list_to_integer(PortStr) of > >>>> {'EXIT', _} -> > >>>> {error, port_number}; > >>>> Port -> > >>>> Site = string:substr(X,1,N1-1), > >>>> {ok, Tag, Site, Port} > >>>> end > >>>> end > >>>> end. > >>>> > >>>> > >>>> > >>>> > 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8------ > >>>> > >>>> When testing it, the receiving socket gets very very slow: > >>>> $ erl > >>>> 1> c(fetch). > >>>> 2> Bin = fetch:url("http://www.google.com"). > >>>> ......{error,{socket,timeout}} > >>>> > >>>> Am I missing something? > >>>> What I like to get at the end is a very fast fetcher. Any hint? > >>>> > >>>> Regards > >>>> Zabrane > >>>> > >> > >> > >> ________________________________________________________________ > >> erlang-questions mailing list. See http://www.erlang.org/faq.html > >> erlang-questions (at) erlang.org > >> > >> > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > From rvirding@REDACTED Sat Nov 21 01:10:07 2009 From: rvirding@REDACTED (Robert Virding) Date: Sat, 21 Nov 2009 01:10:07 +0100 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: <18a1db030911201602n24ee3bb4ob4838492120f84ee@mail.gmail.com> References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> <5c493e530911161651w41e46aecu823bd4d36236815f@mail.gmail.com> <18a1db030911201602n24ee3bb4ob4838492120f84ee@mail.gmail.com> Message-ID: <3dbc6d1c0911201610k99ffd11x9fe33ae7ac7d6c1@mail.gmail.com> The call to io:format is wrong as it is missing an argument to print with ~p. So it will print the "BinData: " leader and then crash. I am guessing you meant it to be: io:fornat("BinData: ~p~n", [BinData]), You should have got a warning from the compiler. Robert 2009/11/21 zabrane Mikael > Hi List ! > > While trying to learn how to write a simple TCP Web Server in Erlang which > only dump what it gets to stdout, I realize that time to time, the HTTP > requests get truncated when reaching the server. My main socket loop looks > like: > > ------------------------------------------- > -define(TCP_OPTIONS,[binary, {packet, 0}, {active, false}, {reuseaddr, > true}]). > ... > loop_recv(Socket) > case gen_tcp:recv(Socket, 0) of > {ok, BinData} -> > %% here, I'm assuming that all the HTTP request (Headers + > Body) is in "BinData". Hope I'm right. > io:format("BinData: ~p~n", []), > ok; > NotOK -> > error_logger:info_report([{"gen_tcp:recv/2", NotOK}]), > error > end. > ------------------------------------------- > > For some requests, the "io:format" prints a truncated data (in BinData): > > <<" > > http://www.foo.tv/images/v30/LoaderV3.swf?loop=false&quality=high&request=357&HTTP/1.0 > \r\nHost: > www.foo.tv\r\nUser-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; > fr; rv:1.9.0.2) Gecko/2008090512 Firefox/3.0.2\r\nAccept: text/h">> > > As you can see, the request isn't complete "... Accept: text/h". > > Am I doing somthing wrong? How can I fix it please? > > Regards > Zabrane > > From zabrane3@REDACTED Sat Nov 21 01:15:03 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Sat, 21 Nov 2009 01:15:03 +0100 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: <3dbc6d1c0911201610k99ffd11x9fe33ae7ac7d6c1@mail.gmail.com> References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> <5c493e530911161651w41e46aecu823bd4d36236815f@mail.gmail.com> <18a1db030911201602n24ee3bb4ob4838492120f84ee@mail.gmail.com> <3dbc6d1c0911201610k99ffd11x9fe33ae7ac7d6c1@mail.gmail.com> Message-ID: <18a1db030911201615v8ca1039n45717e8cbe918e65@mail.gmail.com> Thanks Robert. I just made a quick copy/past for the post and I forgot the "~p" ;-) So my problem is still there if you can help !! 2009/11/21 Robert Virding > The call to io:format is wrong as it is missing an argument to print with > ~p. So it will print the "BinData: " leader and then crash. I am guessing > you meant it to be: > > io:fornat("BinData: ~p~n", [BinData]), > > You should have got a warning from the compiler. > > Robert > > 2009/11/21 zabrane Mikael > >> Hi List ! >> >> While trying to learn how to write a simple TCP Web Server in Erlang which >> only dump what it gets to stdout, I realize that time to time, the HTTP >> requests get truncated when reaching the server. My main socket loop looks >> like: >> >> ------------------------------------------- >> -define(TCP_OPTIONS,[binary, {packet, 0}, {active, false}, {reuseaddr, >> true}]). >> ... >> loop_recv(Socket) >> case gen_tcp:recv(Socket, 0) of >> {ok, BinData} -> >> %% here, I'm assuming that all the HTTP request (Headers + >> Body) is in "BinData". Hope I'm right. >> io:format("BinData: ~p~n", []), >> ok; >> NotOK -> >> error_logger:info_report([{"gen_tcp:recv/2", NotOK}]), >> error >> end. >> ------------------------------------------- >> >> For some requests, the "io:format" prints a truncated data (in BinData): >> >> <<" >> >> http://www.foo.tv/images/v30/LoaderV3.swf?loop=false&quality=high&request=357&HTTP/1.0 >> \r\nHost: >> www.foo.tv\r\nUser-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; >> fr; rv:1.9.0.2) Gecko/2008090512 Firefox/3.0.2\r\nAccept: text/h">> >> >> As you can see, the request isn't complete "... Accept: text/h". >> >> Am I doing somthing wrong? How can I fix it please? >> >> Regards >> Zabrane >> >> From kenji.rikitake@REDACTED Sat Nov 21 07:31:16 2009 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Sat, 21 Nov 2009 15:31:16 +0900 Subject: [erlang-questions] Re: Erlang logo licensing and usage examples In-Reply-To: References: Message-ID: <20091121063116.GA66574@k2r.org> Very long time ago I saw Digital[1] logo rebuilt as an EPS file. I'd be very glad if Ericsson (or whoever has the legal rights of the Erlang logo(s)) to provide such vectorized files. [1]: Digital Equipment Corporation, later acquired by Compaq and HP. Kenji Rikitake In the message dated Wed, Nov 18, 2009 at 11:49:52AM -0600, Jeffrey Horner writes: > Hi, > > Reviving this thread because I've been smitten with "Erlang: The > Movie" on youtube (i'm a sucker for hacker/geek/pop culture). I've got > a design up here for critique if anyone is interested: > > http://www.threadless.com/critique/56844/Erlang_The_Movie > > I'd like to use the erlang logo font for the title and clear any > copryight issues. I'd also like to release the design to the public > domain, copyleft, creative comons, etc. whatever works for the > commuty. > > Let me know what you think. > > Jeff > -- > http://biostat.mc.vanderbilt.edu/JeffreyHorner From kenji.rikitake@REDACTED Sat Nov 21 07:47:31 2009 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Sat, 21 Nov 2009 15:47:31 +0900 Subject: [erlang-questions] Remote "access" points In-Reply-To: References: <14571703.44041258539137048.JavaMail.root@zimbra> <22244220.44061258539242065.JavaMail.root@zimbra> <635498b70911180538m5b005c09jda4d3409dd037fa2@mail.gmail.com> <4B040252.4080709@erlang-consulting.com> <635498b70911181027s40bcdbfam5adb22b51144f052@mail.gmail.com> <20091118183247.GI4286@hijacked.us> <635498b70911181042t13d7f42ex6417cd2bf97aea03@mail.gmail.com> <4B045A80.1040303@erlang-consulting.com> Message-ID: <20091121064731.GB66574@k2r.org> BERT-RPC looks to have a bright future, for the availability of implementation. And in general, you can use Erlang ssh module to provide authentication and cryptographic confidentiality. An example of mine (with a primitive test of SSH module) at the following report: Rikitake, K. and Nakao, K.: SSH Distribution Transport on Erlang Concurrent System, Proceedings of IPSJ Computer Security Symposium 2009 (CSS2009), IPSJ Symposium Series, Vol. 2009, No. 11, IPSJ, pp. 117-122 (2009). ISSN 1344-0640. And a PDF: http://www.ne.jp/asahi/bdx/info/depot/css2009-erlsshrpc-k2r-pub.pdf FYI. Kenji Rikitake From john.hughes@REDACTED Sat Nov 21 10:56:59 2009 From: john.hughes@REDACTED (John Hughes) Date: Sat, 21 Nov 2009 10:56:59 +0100 Subject: Interesting workshop on test automation Message-ID: This is a "heads up" about Automation of Software Test 2010 http://www.cs.allegheny.edu/ast2010/ part of the prestigious ACM/IEEE International Conference on Software Engineering, which is calling for presentations of industrial case studies on any aspect of test automation. There's plenty of interest in test automation in the Erlang community... whether it be with QuickCheck, Common Test, or EUnit. Wouldn't it be nice to see some of that represented at this event? There's a copy of the call-for-papers at http://quviq.com/documents/AST2010-CfP.pdf Note that you can submit either a short paper, or a presentation of at most 15 slides, but the programme committee chair must get it by January 20th. Don'f forget to check out the information on ICSE itself, which looks like being a really spectacular event: http://www.sbs.co.za/ICSE2010/ A trip to Cape Town, anybody? John Hughes From pablo.platt@REDACTED Sat Nov 21 15:02:23 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Sat, 21 Nov 2009 06:02:23 -0800 (PST) Subject: [erlang-questions] ubuntu upstart job to run an erlang application In-Reply-To: <24d4f39c0911201059s6e4530f6i8db9e4d0502fb6ff@mail.gmail.com> References: <998105.35733.qm@web112602.mail.gq1.yahoo.com> <24d4f39c0911201059s6e4530f6i8db9e4d0502fb6ff@mail.gmail.com> Message-ID: <933563.92345.qm@web112618.mail.gq1.yahoo.com> That works! I've added export HOME=/root Do I also need to add the erlang cookie dir to the PATH and export it? Do I need to add flags to erl? Thanks ________________________________ From: Colm Dougan To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Fri, November 20, 2009 8:59:20 PM Subject: Re: [erlang-questions] ubuntu upstart job to run an erlang application On Fri, Nov 20, 2009 at 2:07 PM, Pablo Platt wrote: > > So my mochiweb app doesn't run. > When I'm using the start.sh script generated with the skeleton my app > run fine. Is the 'HOME' environment variable defined? This is a gotcha I've run into in the past. colm@REDACTED:~$ env - erl erlexec: HOME must be set However : colm@REDACTED:~$ env - HOME="/home/colm" erl Eshell V5.6.5 (abort with ^G) 1> Colm ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From hermanccw@REDACTED Sun Nov 22 05:12:18 2009 From: hermanccw@REDACTED (Herman Chan) Date: Sat, 21 Nov 2009 20:12:18 -0800 Subject: Compile error on Mac OS X 10.6 Message-ID: <50da63430911212012m31b1bb9bu6841692d69a96811@mail.gmail.com> Hi all, I am new to Erlang and trying to compile from source in mac os x. I tried to compiled with ./configure --enable-threads --enable-smp-support --enable-kernel-poll, but got the following error. Ignoring the --cache-file argument since it can cause the system to be erroneously configured Disabling caching checking build system type... i386-apple-darwin10.0.0 checking host system type... i386-apple-darwin10.0.0 checking for gcc... gcc checking for C compiler default output file name... configure: error: C compiler cannot create executables See `config.log' for more details. Looking at config.log, it seems like the problem is in this line. configure:1771: gcc conftest.c >&5 ld: library not found for -lcrt1.10.5.o collect2: ld returned 1 exit status Has anyone got this error before? Any help is appreciated. From paul.joseph.davis@REDACTED Sun Nov 22 06:49:12 2009 From: paul.joseph.davis@REDACTED (Paul Joseph Davis) Date: Sun, 22 Nov 2009 00:49:12 -0500 Subject: [erlang-questions] Compile error on Mac OS X 10.6 In-Reply-To: <50da63430911212012m31b1bb9bu6841692d69a96811@mail.gmail.com> References: <50da63430911212012m31b1bb9bu6841692d69a96811@mail.gmail.com> Message-ID: <23C44192-C952-4D39-B94D-BCB53AA3EABB@gmail.com> Have you installed the Xcode developer tools? On Nov 21, 2009, at 11:12 PM, Herman Chan wrote: > Hi all, > > I am new to Erlang and trying to compile from source in mac os x. I > tried to compiled with ./configure --enable-threads > --enable-smp-support --enable-kernel-poll, but got the following > error. > > Ignoring the --cache-file argument since it can cause the system to be > erroneously configured > Disabling caching > checking build system type... i386-apple-darwin10.0.0 > checking host system type... i386-apple-darwin10.0.0 > checking for gcc... gcc > checking for C compiler default output file name... configure: error: > C compiler cannot create executables > See `config.log' for more details. > > Looking at config.log, it seems like the problem is in this line. > > configure:1771: gcc conftest.c >&5 > ld: library not found for -lcrt1.10.5.o > collect2: ld returned 1 exit status > > Has anyone got this error before? Any help is appreciated. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From jarrod@REDACTED Sun Nov 22 06:52:21 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Sun, 22 Nov 2009 00:52:21 -0500 Subject: How to match on a nested record against a dict Message-ID: I have the following data structure: it is a record as specified in inet_dns.hrl: {ok,{dns_rec,{dns_header,0,true,'query',true,false,false,false,false,0}, [], [{dns_rr,"_see._tcp.local",ptr,in,0,0, "jhr@REDACTED",undefined,[], false}], [],[]}} I have the following code where Sub is a dict: receiver(Sub) -> receive {udp, _Socket, IP, InPortNo, Packet} -> DnsRec = inet_dns:decode(Packet), io:format("~n~nFrom: ~p~nPort: ~p~nData: ~p~n",[IP,InPortNo,DnsRec]), receiver(Sub); stop -> true; AnythingElse -> %io:format("RECEIVED: ~p~n",[AnythingElse]), receiver(Sub) end. I want to match on the "_see._tcp.local" and see if it is in the dns_rr record and only do the io:format() if that is in my subscriptions dict. Can anyone point me in the right direction of what idiom to use? This pattern matching stuff is difficult for us olde school if () ers From igorrs@REDACTED Sun Nov 22 08:17:57 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Sun, 22 Nov 2009 05:17:57 -0200 Subject: [erlang-questions] How to match on a nested record against a dict In-Reply-To: References: Message-ID: Hi, Jarrod. I can't understand exactly what you want, so I'll give you an example. Suppose DomainsSet is a sets of domains and DnsRec is a dns_rec. This would be a list of the domains that are present in some resource record and in DomainsSet: [Domain || #dns_rr{domain = Domain} <- DnsRec#dns_rec.anlist, sets:is_element(Domain, DomainsSet)] Is that helpful? Igor. On Sun, Nov 22, 2009 at 3:52 AM, Jarrod Roberson wrote: > I have the following data structure: it is a record as specified in > inet_dns.hrl: > > {ok,{dns_rec,{dns_header,0,true,'query',true,false,false,false,false,0}, > ? ? ? ? ? ? ? ? ? [], > ? ? ? ? ? ? ? ? ? [{dns_rr,"_see._tcp.local",ptr,in,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ?"jhr@REDACTED",undefined,[], > ? ? ? ? ? ? ? ? ? ? ? ? ? ?false}], > ? ? ? ? ? ? ? ? ? [],[]}} > > I have the following code where Sub is a dict: > > receiver(Sub) -> > ? ?receive > ? ? ? ?{udp, _Socket, IP, InPortNo, Packet} -> > ? ? ? ? ? ?DnsRec = inet_dns:decode(Packet), > ? ? ? ? ? ? ? io:format("~n~nFrom: ~p~nPort: ~p~nData: > ~p~n",[IP,InPortNo,DnsRec]), > ? ? ? ? ? ? ? receiver(Sub); > ? ? ? ? ? stop -> > ? ? ? ? ? ? ? true; > ? ? ? ? AnythingElse -> > ? ? ? ? ? ? ? %io:format("RECEIVED: ~p~n",[AnythingElse]), > ? ? ? ? ? ? ? receiver(Sub) > ? ? ? end. > > I want to match on the "_see._tcp.local" and see if it is in the dns_rr > record and only do the io:format() if that is in my subscriptions dict. > Can anyone point me in the right direction of what idiom to use? This > pattern matching stuff is difficult for us olde school if () ers -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From zabrane3@REDACTED Sun Nov 22 12:26:48 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Sun, 22 Nov 2009 12:26:48 +0100 Subject: Other constants likes ?MODULE Message-ID: <18a1db030911220326j9139573w32c55aee92e522c8@mail.gmail.com> Hi List ! As for the ?MODULE which returns the module name, I'm wondering if there exist constants which return the "line number" (ex. ?LINE) and the "function name" (ex. ?FUNCTION) in any Erlang code? Regards Zabrane From kiszl@REDACTED Sun Nov 22 12:32:28 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sun, 22 Nov 2009 12:32:28 +0100 Subject: [erlang-questions] Other constants likes ?MODULE In-Reply-To: <18a1db030911220326j9139573w32c55aee92e522c8@mail.gmail.com> References: <18a1db030911220326j9139573w32c55aee92e522c8@mail.gmail.com> Message-ID: <4B09214C.2030504@tmit.bme.hu> Hi Zabrane, You can find the available predefined macros here: http://erlang.org/doc/reference_manual/macros.html#7.3 Regards, Zoltan. zabrane Mikael wrote: > Hi List ! > > As for the ?MODULE which returns the module name, I'm wondering if there > exist constants which > return the "line number" (ex. ?LINE) and the "function name" (ex. ?FUNCTION) > in any Erlang code? > > Regards > Zabrane > > From zabrane3@REDACTED Sun Nov 22 12:34:41 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Sun, 22 Nov 2009 12:34:41 +0100 Subject: [erlang-questions] Other constants likes ?MODULE In-Reply-To: <4B09214C.2030504@tmit.bme.hu> References: <18a1db030911220326j9139573w32c55aee92e522c8@mail.gmail.com> <4B09214C.2030504@tmit.bme.hu> Message-ID: <18a1db030911220334s265dcfa1rda5d16ccf0ec0bba@mail.gmail.com> Thanks Zoltan. Exactly was I was looking after Regards Zabrane 2009/11/22 Zoltan Lajos Kis > Hi Zabrane, > > You can find the available predefined macros here: > http://erlang.org/doc/reference_manual/macros.html#7.3 > > Regards, > Zoltan. > > > zabrane Mikael wrote: > >> Hi List ! >> >> As for the ?MODULE which returns the module name, I'm wondering if there >> exist constants which >> return the "line number" (ex. ?LINE) and the "function name" (ex. >> ?FUNCTION) >> in any Erlang code? >> >> Regards >> Zabrane >> >> >> > > From jarrod@REDACTED Sun Nov 22 16:55:35 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Sun, 22 Nov 2009 10:55:35 -0500 Subject: [erlang-questions] How to match on a nested record against a dict In-Reply-To: References: Message-ID: On Sun, Nov 22, 2009 at 2:17 AM, Igor Ribeiro Sucupira wrote: > Hi, Jarrod. > > I can't understand exactly what you want, so I'll give you an example. > > Suppose DomainsSet is a sets of domains and DnsRec is a dns_rec. This > would be a list of the domains that are present in some resource > record and in DomainsSet: > [Domain || #dns_rr{domain = Domain} <- DnsRec#dns_rec.anlist, > sets:is_element(Domain, DomainsSet)] > > Is that helpful? > > Igor. > > thanks it is a place for me to start From jabronson@REDACTED Sun Nov 22 22:49:13 2009 From: jabronson@REDACTED (Joshua Bronson) Date: Sun, 22 Nov 2009 13:49:13 -0800 (PST) Subject: tk requirement? In-Reply-To: <299cc2dd0911170952m15de8c26h999aecf00e50bcfc@mail.gmail.com> References: <299cc2dd0911161248n753344b5wd95d39236b3855bf@mail.gmail.com> <299cc2dd0911161403u566da8b7k1fa1c382f97cf854@mail.gmail.com> <299cc2dd0911170952m15de8c26h999aecf00e50bcfc@mail.gmail.com> Message-ID: <3d05541c-0888-423e-8975-5733516bc980@f10g2000vbl.googlegroups.com> On Nov 17, 12:52?pm, Joshua Bronson wrote: > Thanks Paul, it sounds like MacPorts package can definitely have a no_tk > variant, supported either with a patch deleting > the $(top_srcdir)/lib/wx/configure file or by adding a --with option for it > (MacPorts can assume autoconf is available). I asked about this on erlang-bugs and it turns out there's already a way to do this: On 2009/11/21 Bj?rn Gustavsson wrote: > There already is a mechanism to skip the building of certain applications, > namely to create a file named SKIP in the application directory: > > touch lib/wx/SKIP lib/gs/SKIP (from http://www.erlang.org/cgi-bin/ezmlm-cgi?2:mss:1606:200911:gpimfjjkfgneonnhogad) From colm.dougan@REDACTED Mon Nov 23 00:03:58 2009 From: colm.dougan@REDACTED (Colm Dougan) Date: Sun, 22 Nov 2009 23:03:58 +0000 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: <18a1db030911201615v8ca1039n45717e8cbe918e65@mail.gmail.com> References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> <5c493e530911161651w41e46aecu823bd4d36236815f@mail.gmail.com> <18a1db030911201602n24ee3bb4ob4838492120f84ee@mail.gmail.com> <3dbc6d1c0911201610k99ffd11x9fe33ae7ac7d6c1@mail.gmail.com> <18a1db030911201615v8ca1039n45717e8cbe918e65@mail.gmail.com> Message-ID: <24d4f39c0911221503r5db20eb6xf758e43e47150759@mail.gmail.com> On Sat, Nov 21, 2009 at 12:15 AM, zabrane Mikael wrote: > Thanks Robert. I just made a quick copy/past for the post and I forgot the > "~p" ;-) > So my problem is still there if you can help !! Your assumption that the initial gen_tcp:recv will give you a binary which always includes the full request headers is not correct. If you are going down that road you would need to write code to check for the the header terminator and if it isn't present then buffer and read more until you have the terminator. It isn't hard but it can be messy. Most erlang http servers these days use the {packet, http} socket option which is HTTP aware and which will read/parse headers one by one for you and will also indicate the end of the headers. mochiweb_http has a good example of how to use {packet, http} to read request headers : http://code.google.com/p/mochiweb/source/browse/trunk/src/mochiweb_http.erl Colm > > 2009/11/21 Robert Virding > >> The call to io:format is wrong as it is missing an argument to print with >> ~p. So it will print the "BinData: " leader and then crash. I am guessing >> you meant it to be: >> >> io:fornat("BinData: ~p~n", [BinData]), >> >> You should have got a warning from the compiler. >> >> Robert >> >> 2009/11/21 zabrane Mikael >> >>> Hi List ! >>> >>> While trying to learn how to write a simple TCP Web Server in Erlang which >>> only dump what it gets to stdout, I realize that time to time, the HTTP >>> requests get truncated when reaching the server. My main socket loop looks >>> like: >>> >>> ------------------------------------------- >>> -define(TCP_OPTIONS,[binary, {packet, 0}, {active, false}, {reuseaddr, >>> true}]). >>> ... >>> loop_recv(Socket) >>> ? case gen_tcp:recv(Socket, 0) of >>> ? ? ? ?{ok, BinData} -> >>> ? ? ? ? ? ? %% here, I'm assuming that all the HTTP request (Headers + >>> Body) is in "BinData". Hope I'm right. >>> ? ? ? ? ? ? io:format("BinData: ~p~n", []), >>> ? ? ? ? ? ? ok; >>> ? ? ? ?NotOK -> >>> ? ? ? ? ? ?error_logger:info_report([{"gen_tcp:recv/2", NotOK}]), >>> ? ? ? ? ? ?error >>> ? ?end. >>> ------------------------------------------- >>> >>> For some requests, the "io:format" prints a truncated data (in BinData): >>> >>> <<" >>> >>> http://www.foo.tv/images/v30/LoaderV3.swf?loop=false&quality=high&request=357&HTTP/1.0 >>> \r\nHost: >>> www.foo.tv\r\nUser-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; >>> fr; rv:1.9.0.2) Gecko/2008090512 Firefox/3.0.2\r\nAccept: text/h">> >>> >>> As you can see, the request isn't complete "... Accept: text/h". >>> >>> Am I doing somthing wrong? How can I fix it please? >>> >>> Regards >>> Zabrane >>> >>> > From hermanccw@REDACTED Mon Nov 23 02:06:04 2009 From: hermanccw@REDACTED (Herman Chan) Date: Sun, 22 Nov 2009 17:06:04 -0800 Subject: [erlang-questions] Compile error on Mac OS X 10.6 In-Reply-To: <23C44192-C952-4D39-B94D-BCB53AA3EABB@gmail.com> References: <50da63430911212012m31b1bb9bu6841692d69a96811@mail.gmail.com> <23C44192-C952-4D39-B94D-BCB53AA3EABB@gmail.com> Message-ID: <50da63430911221706h66f9e963u5d0f8cd4604f397d@mail.gmail.com> yes, I have XCode 3.1 installed. In fact, the error message seems to suggested that it can't link with crt1.10.5.o file, which is under a folder named /Developer/SDKs/MacOSX10.5.sdk/usr/lib. Is my terminal wired up to look into /Develoepr/SDKs/MacOSX1.5.sdk/usr/lib for library? Herman On Sat, Nov 21, 2009 at 9:49 PM, Paul Joseph Davis wrote: > Have you installed the Xcode developer tools? > > > > On Nov 21, 2009, at 11:12 PM, Herman Chan wrote: > >> Hi all, >> >> I am new to Erlang and trying to compile from source in mac os x. ?I >> tried to compiled with ./configure --enable-threads >> --enable-smp-support --enable-kernel-poll, but got the following >> error. >> >> Ignoring the --cache-file argument since it can cause the system to be >> erroneously configured >> Disabling caching >> checking build system type... i386-apple-darwin10.0.0 >> checking host system type... i386-apple-darwin10.0.0 >> checking for gcc... gcc >> checking for C compiler default output file name... configure: error: >> C compiler cannot create executables >> See `config.log' for more details. >> >> Looking at config.log, it seems like the problem is in this line. >> >> configure:1771: gcc ? ?conftest.c ?>&5 >> ld: library not found for -lcrt1.10.5.o >> collect2: ld returned 1 exit status >> >> Has anyone got this error before? ?Any help is appreciated. >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > From jarrod@REDACTED Mon Nov 23 05:18:28 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Sun, 22 Nov 2009 23:18:28 -0500 Subject: [erlang-questions] Compile error on Mac OS X 10.6 In-Reply-To: <50da63430911221706h66f9e963u5d0f8cd4604f397d@mail.gmail.com> References: <50da63430911212012m31b1bb9bu6841692d69a96811@mail.gmail.com> <23C44192-C952-4D39-B94D-BCB53AA3EABB@gmail.com> <50da63430911221706h66f9e963u5d0f8cd4604f397d@mail.gmail.com> Message-ID: On Sun, Nov 22, 2009 at 8:06 PM, Herman Chan wrote: > yes, I have XCode 3.1 installed. In fact, the error message seems to > suggested that it can't link with crt1.10.5.o file, which is under a > folder named /Developer/SDKs/MacOSX10.5.sdk/usr/lib. Is my terminal > wired up to look into /Develoepr/SDKs/MacOSX1.5.sdk/usr/lib for > library? > > Herman > if you are compling for 10.6 you should have Xcode 3.2.1 which is the latest. You can get the update from Apple Developers Connection. From fritchie@REDACTED Mon Nov 23 06:10:02 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Sun, 22 Nov 2009 23:10:02 -0600 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: Message of "Sat, 21 Nov 2009 00:35:25 +0100." <200911202335.nAKNZPkB057859@pluto.hedeland.org> Message-ID: <4846.1258953002@snookles.snookles.com> Hi, Per and Chandur and Peter-Henry ... I don't believe we've suffered from this exact problem, but perhaps it's related? At Gemini we build the VM with "./configure --prefix=/some/weird/place" and then really install the whole package to /some/other/dir/entirely. IIRC, we've had to add an "env LD_LIBRARY_PATH=/some/other/dir/entirely/...openssl/lib" wrapper in our /etc/init.d/appthingie script's invocation of our daemons. -Scott From jarrod@REDACTED Mon Nov 23 06:25:56 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Mon, 23 Nov 2009 00:25:56 -0500 Subject: [erlang-questions] How to match on a nested record against a dict In-Reply-To: References: Message-ID: On Sun, Nov 22, 2009 at 2:17 AM, Igor Ribeiro Sucupira wrote: > Hi, Jarrod. > > I can't understand exactly what you want, so I'll give you an example. > > Suppose DomainsSet is a sets of domains and DnsRec is a dns_rec. This > would be a list of the domains that are present in some resource > record and in DomainsSet: > [Domain || #dns_rr{domain = Domain} <- DnsRec#dns_rec.anlist, > sets:is_element(Domain, DomainsSet)] > > Is that helpful? > > Igor. > > Ok have tried and tried, and your snippet won't compile. I have both Erlang books and can't figure out how to do this logic. here is how I would do it in pseudo-code in languages that I am familar with. for (dns_rr in dns_rec.getAnList()) { if (dns_rr.domain == "_test._tcp.local") { // do some logic here } } receiver(Subs) -> receive {udp, _Socket, IP, InPortNo, Packet} -> {ok,DnsRec} = inet_dns:decode(Packet), % this is where I want the "domain" detecting logic from the dns_rr records io:format("~n~nFrom: ~p~nPort: ~p~nData: ~p~n",[IP,InPortNo,DnsRec]), receiver(Subs); stop -> true; AnythingElse -> %io:format("RECEIVED: ~p~n",[AnythingElse]), receiver(Subs) end. From rob@REDACTED Mon Nov 23 09:37:32 2009 From: rob@REDACTED (Rob Charlton) Date: Mon, 23 Nov 2009 08:37:32 +0000 Subject: [erlang-questions] How to match on a nested record against a dict In-Reply-To: References: Message-ID: <4B0A49CC.90307@erlang-consulting.com> Hi Jarrod, First off, to get the record definitions for #dns_rec and #dns_rr, you need to include this line in your module (normally near the top): -include_lib("kernel/src/inet_dns.hrl"). Now I would code it something like: receive {udp, _Socket, IP, InPortNo, Packet} -> process_dnsrec(Sub,inet_dns:decode(Packet)), receiver(Sub); ... process_dnsrec(Sub,{error,E}) -> %% do some error handling here (if required) ; process_dnsrec(Sub,{ok,#dnsrec{anlist=Responses}) -> %% Responses is a list of #dns_rr records, so we want to handle them one at a time process_dnsrec1(Sub,Responses). process_dnsrec1(_,[]) -> %% no more responses ; process_dnsrec1(Sub,[#dns_rr{domain=Dom}|Rest]) -> %% lookup the domain field from the #dns_rr record case dict:find(Dom,Sub) of [Result] -> io:format("Interesting domain ~p~n",[Result]); error -> %% do nothing for non-interesting domains ok end, process_dnsrec1(Sub,Rest). Cheers Rob Jarrod Roberson wrote: > I have the following data structure: it is a record as specified in > inet_dns.hrl: > > {ok,{dns_rec,{dns_header,0,true,'query',true,false,false,false,false,0}, > [], > [{dns_rr,"_see._tcp.local",ptr,in,0,0, > "jhr@REDACTED",undefined,[], > false}], > [],[]}} > > I have the following code where Sub is a dict: > > receiver(Sub) -> > receive > {udp, _Socket, IP, InPortNo, Packet} -> > DnsRec = inet_dns:decode(Packet), > io:format("~n~nFrom: ~p~nPort: ~p~nData: > ~p~n",[IP,InPortNo,DnsRec]), > receiver(Sub); > stop -> > true; > AnythingElse -> > %io:format("RECEIVED: ~p~n",[AnythingElse]), > receiver(Sub) > end. > > I want to match on the "_see._tcp.local" and see if it is in the dns_rr > record and only do the io:format() if that is in my subscriptions dict. > Can anyone point me in the right direction of what idiom to use? This > pattern matching stuff is difficult for us olde school if () ers > > From hvjunk@REDACTED Mon Nov 23 11:09:43 2009 From: hvjunk@REDACTED (Hendrik Visage) Date: Mon, 23 Nov 2009 12:09:43 +0200 Subject: list size 'causing VM "problems" Message-ID: Hi there, Yes, I know this code is not yet optimal (I'm still learning :), but it begs a few questions I'd like to understand from the VM etc. 1) I've run it fine with a small subset, but once I've loaded the 930k lines file, the VM sucks up a lot of RAM/Virtualmemory. Like a burst of about 2G (I have a 4G MacBookPro) and then once it returned in the erl shell, the VM starts to go balistic and consumes >7G of virtualmemory ;( Q1: why did the VM exhibit this behaviour? the garbage collector going bad/mad?? 2) I will push the data into an ETS of sorts, as I'll try to find duplicate files, but were thinking of an initial pull into a list, en then fron there do the tests etc. The idea might be to pull in one disk, and then compare it to another removal disk's files. Q2: Should I rather do this straight into an ETS/DETS? Q3: Should I preferably start to consider DETS 'cause of the size?? Q4: will Mnesia help in this case? %%-------------------------------------------------------------------- %% Function: process_line/1 %% Description: take a properly formated line, and parse it, and %% returns the tuple {Type,File,Hash} %% Line: "MD5 (/.file) = d41d8cd98f00b204e9800998ecf8427e" %% Nore some might be SHA1 in future. %%-------------------------------------------------------------------- process_line(Line) -> {match,[Type,File,Hash]}= re:run(Line, "\(.*\)[ ][\\(]\(.*\)[\\)][ ][=][ ]\([0-9a-f]*\)\n", [{capture,all_but_first,list}]), {Type,File,Hash}. %%-------------------------------------------------------------------- %% Function: read_lines/1 %% Description: read in all the lines from a "properly formatted" %% md5 output on MacOSX, returning a list with the tupples. %%-------------------------------------------------------------------- read_lines(IOfd) -> case file:read_line(IOfd) of {ok,Line} -> [process_line(Line)|read_lines(IOfd)]; eof -> [] end. From rvirding@REDACTED Mon Nov 23 11:18:41 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 23 Nov 2009 11:18:41 +0100 Subject: [erlang-questions] accessing ets from the process on other node In-Reply-To: References: Message-ID: <3dbc6d1c0911230218w22fa0ae6l50d6a77d3ee8ee5b@mail.gmail.com> Hi, I don't see that you got an answer to this question. By this time you have probably know anyway but ... Ets tables are local to a node. The id returned when creating an ets table is a number which is only relevant on the node on which the table was created. So to getting your program to work you would first have to distribute the ets table to all nodes on which you wish to use it and modify your algorithm to use the local tables. An alternative may be to use mnesia which is distributed and can distribute its data over many nodes. Sorry, there is nothing else to do. You could create a process on the main node to access the ets table and use its pid, but I am certain that this would be slow. Robert 2009/11/9 sapan shah > HI, > > I use digraph module to create graphs and apply some functions on it. The > digraph is internally stored as ets tuples. > I am using mapreduce architecture to speed up the computations on the > digraph (using plists.erl from Stephen Marsh). > So, When the graph is passed to a process on some other node on network, it > exits with error. (may be because it is not able to access the ets tuples) > Any solution??? > > Below is a small test example to generate the error > %%%%%%%%%%%%%%%%%%%%%%%%%%% > -module(pliststest). > -compile(export_all). > > malt() -> > [100, {nodes, [{'sap@REDACTED', 3}, {'sap@REDACTED', 3}]}]. > > test(G, List) -> > Fun = fun(X) -> io:format("TROUBLE: ~p~n",[digraph:vertices(G)]), > lists:foldr(fun(_Y,ACC) -> ACC+1 end, 0, X) end, > Fuse = fun(X, Y) -> X+Y end, > plists:runmany(Fun, Fuse, List, malt()). > %%%%%%%%%%%%%%%%%%%%%%%%%%%% > > Also, PFA the plists.erl (from Stephen Marsh). > > -- > Regards, > Sapan Shah > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From ulf.wiger@REDACTED Mon Nov 23 11:54:48 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 23 Nov 2009 11:54:48 +0100 Subject: [erlang-questions] list size 'causing VM "problems" In-Reply-To: References: Message-ID: <4B0A69F8.8080605@erlang-consulting.com> Hendrik Visage wrote: > Hi there, > > Yes, I know this code is not yet optimal (I'm still learning :), but > it begs a few questions I'd like to understand from the VM etc. > > 1) I've run it fine with a small subset, but once I've loaded the 930k > lines file, the VM sucks up a lot of RAM/Virtualmemory. Like a burst > of about 2G (I have a 4G MacBookPro) and then once it returned in the > erl shell, the VM starts to go balistic and consumes >7G of > virtualmemory ;( > Q1: why did the VM exhibit this behaviour? the garbage collector going bad/mad?? Since it consumes >7G, am I right in guessing that you're running 64-bit Erlang? If so (and in any case), you should really use 'binary' instead of 'list' in the regular expression option list. Using list representation of the string data, each byte will consume two heap words of memory - 8 bytes in 32-bit Erlang and 16 bytes in 64-bit. Regarding the GC, consider what it has to work with. You are building a very large data structure in a tight loop. The process will continuously run out of heap, triggering the GC. The GC will copy live data (which is going to be most of it) to another copy of the heap. If that's not enough, it will run a fullsweep, also looking at data that survived the previous GC (no garbage there, since the list just keeps growing). This creates yet another heap copy. Finally, it does a resize of the heap, if necessary. It is possible to pre-size the heap using spawn_opt() and the min_heap_size option. Given that you have a very large data structure, this may still turn out problematic. You should definitely try putting the data in ETS instead of accumulating it on the heap. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From phmander@REDACTED Mon Nov 23 12:19:39 2009 From: phmander@REDACTED (Peter-Henry Mander) Date: Mon, 23 Nov 2009 11:19:39 +0000 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <4846.1258953002@snookles.snookles.com> References: <200911202335.nAKNZPkB057859@pluto.hedeland.org> <4846.1258953002@snookles.snookles.com> Message-ID: <38e3b9f40911230319q24b894dex69135b8050308441@mail.gmail.com> Hi Scott, No, I don't think that the issue is to do with paths. The loader _does_ find the crypto_drv.so file, and fails to load it because "symbol (unknown): value 0xfffffd7ffd514fdf does not fit." I began to think that it was an AMD ABI "memory model" issue, where a 64 bit offset could not fit into a 32 bit value of an address register relative memory access instruction. Per pointed out that there are symbols in the crypto_drv.so file which suggest that the linker is being invoked with the wrong options, resulting in what smells like a stand-alone executable file with a main() function, not a shared library object. I'm going to continue hacking the Makefiles to see if adding the suggested -shared option to the crypto_drv linker command makes a positive difference. Pete. On Mon, Nov 23, 2009 at 5:10 AM, Scott Lystig Fritchie wrote: > Hi, Per and Chandur and Peter-Henry ... I don't believe we've suffered > from this exact problem, but perhaps it's related? ?At Gemini we build > the VM with "./configure --prefix=/some/weird/place" and then really > install the whole package to /some/other/dir/entirely. > > IIRC, we've had to add an "env > LD_LIBRARY_PATH=/some/other/dir/entirely/...openssl/lib" wrapper in our > /etc/init.d/appthingie script's invocation of our daemons. > > -Scott > From zabrane3@REDACTED Mon Nov 23 12:58:04 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 23 Nov 2009 12:58:04 +0100 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: <24d4f39c0911221503r5db20eb6xf758e43e47150759@mail.gmail.com> References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> <5c493e530911161651w41e46aecu823bd4d36236815f@mail.gmail.com> <18a1db030911201602n24ee3bb4ob4838492120f84ee@mail.gmail.com> <3dbc6d1c0911201610k99ffd11x9fe33ae7ac7d6c1@mail.gmail.com> <18a1db030911201615v8ca1039n45717e8cbe918e65@mail.gmail.com> <24d4f39c0911221503r5db20eb6xf758e43e47150759@mail.gmail.com> Message-ID: <18a1db030911230358w39eedf41wcbb79ab362402a2@mail.gmail.com> Hi Colm, Your assumption that the initial gen_tcp:recv will give you a binary > which always includes the full request headers is not correct. If you > are going down that road you would need to write code to check for the > the header terminator and if it isn't present then buffer and read > more until you have the terminator. Ok. Finding headers terminator (teh double CRLF) isn't difficult. What's difficult for me is to find the request end? How can I know that I've read all data in my request (GET or POST)? > It isn't hard but it can be messy. > Joe's code helped me a lot with his reentrant parser. Thanks by the way. Most erlang http servers these days use the {packet, http} socket > option which is HTTP aware and which will read/parse headers one by > one for you and will also indicate the end of the headers. > Lev Walkin introduced an example of his web server called yucan which uses {packer, http_bin}: http://lionet.livejournal.com/42016.html?thread=774432 Could someone explain the difference? > mochiweb_http has a good example of how to use {packet, http} to read > request headers : > > > http://code.google.com/p/mochiweb/source/browse/trunk/src/mochiweb_http.erl Thanks for the pointer Regards Zabrane From chandrashekhar.mullaparthi@REDACTED Mon Nov 23 13:25:04 2009 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 23 Nov 2009 12:25:04 +0000 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: <18a1db030911230358w39eedf41wcbb79ab362402a2@mail.gmail.com> References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> <5c493e530911161651w41e46aecu823bd4d36236815f@mail.gmail.com> <18a1db030911201602n24ee3bb4ob4838492120f84ee@mail.gmail.com> <3dbc6d1c0911201610k99ffd11x9fe33ae7ac7d6c1@mail.gmail.com> <18a1db030911201615v8ca1039n45717e8cbe918e65@mail.gmail.com> <24d4f39c0911221503r5db20eb6xf758e43e47150759@mail.gmail.com> <18a1db030911230358w39eedf41wcbb79ab362402a2@mail.gmail.com> Message-ID: 2009/11/23 zabrane Mikael > Hi Colm, > > Your assumption that the initial gen_tcp:recv will give you a binary > > which always includes the full request headers is not correct. If you > > are going down that road you would need to write code to check for the > > the header terminator and if it isn't present then buffer and read > > more until you have the terminator. > > > Ok. Finding headers terminator (teh double CRLF) isn't difficult. > What's difficult for me is to find the request end? > How can I know that I've read all data in my request (GET or POST)? > > There is no easy answer I'm afraid! You just have to read RFC2616 :) cheers Chandru From colm.dougan@REDACTED Mon Nov 23 13:27:04 2009 From: colm.dougan@REDACTED (Colm Dougan) Date: Mon, 23 Nov 2009 12:27:04 +0000 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: <18a1db030911230358w39eedf41wcbb79ab362402a2@mail.gmail.com> References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> <5c493e530911161651w41e46aecu823bd4d36236815f@mail.gmail.com> <18a1db030911201602n24ee3bb4ob4838492120f84ee@mail.gmail.com> <3dbc6d1c0911201610k99ffd11x9fe33ae7ac7d6c1@mail.gmail.com> <18a1db030911201615v8ca1039n45717e8cbe918e65@mail.gmail.com> <24d4f39c0911221503r5db20eb6xf758e43e47150759@mail.gmail.com> <18a1db030911230358w39eedf41wcbb79ab362402a2@mail.gmail.com> Message-ID: <24d4f39c0911230427i470c2f0au6b59417a7a08be1@mail.gmail.com> On Mon, Nov 23, 2009 at 11:58 AM, zabrane Mikael wrote: >> >> Your assumption that the initial gen_tcp:recv will give you a binary >> which always includes the full request headers is not correct. If you >> are going down that road you would need to write code to check for the >> the header terminator and if it isn't present then buffer and read >> more until you have the terminator. > > Ok. Finding headers terminator (teh double CRLF) isn't difficult. > What's difficult for me is to find the request end? > How can I know that I've read all data in my request (GET or POST)? erlang doesn't provide any magic way to read the request body (in the way it does for headers). You have to write that code yourself. You can look at either the yaws or mochiweb sources for straightforward erlang implementations of this. >> It isn't hard but it can be messy. > > Joe's code helped me a lot with his reentrant parser. Thanks by the way. >> >> Most erlang http servers these days use the {packet, http} socket >> option which is HTTP aware and which will read/parse headers one by >> one for you and will also indicate the end of the headers. > > Lev Walkin introduced an example of his web server called yucan which uses > {packer, http_bin}: > http://lionet.livejournal.com/42016.html?thread=774432 > Could someone explain the difference? With the "http" option you get the header key and value returned to you as strings, apart from well known header names which are returned as atoms. With "http_bin" the same applies, but where there were strings you get erlang binaries instead. Colm From mml6364@REDACTED Mon Nov 23 14:07:21 2009 From: mml6364@REDACTED (mliu) Date: Mon, 23 Nov 2009 05:07:21 -0800 (PST) Subject: ll_alloc reallocate crash Message-ID: We have a simple tcp server spawning many processes that manage accepted connections. these connections are long-lived, with long periods of idle waits (5 min). While stress-testing the server, we came across a strange memory-related crash (dump below). At the time of the crash, we have around 150k connections, and roughly 4 processes per conn (~600K). Server has 8Gb memory. Hoping someone has a clue of where we should look, within erlang or maybe os configuration. ulimit -a (CentOS) core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 71680 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) unlimited open files (-n) 900000 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 71680 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited sysctl -p net.ipv4.ip_forward = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 0 kernel.core_uses_pid = 1 net.ipv4.tcp_syncookies = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.shmmax = 68719476736 kernel.shmall = 4294967296 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 net.ipv4.tcp_no_metrics_save = 1 net.ipv4.tcp_moderate_rcvbuf = 1 net.core.netdev_max_backlog = 3000 Here's the crash dump: -- Slogan: ll_alloc: Cannot reallocate 2033104 bytes of memory (of type "fd_status"). System version: Erlang R13B01 (erts-5.7.2) [source] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:true] Compiled: Wed Aug 26 15:33:27 2009 Atoms: 9193 =memory total: 3396483592 processes: 2538096436 processes_used: 2538092308 system: 858387156 atom: 416385 atom_used: 395052 binary: 248350576 code: 3168606 ets: 209338444 =hash_table:atom_tab size: 6421 used: 4862 objs: 9193 depth: 7 =index_table:atom_tab size: 9216 limit: 1048576 entries: 9193 =hash_table:module_code size: 197 used: 118 objs: 151 depth: 3 =index_table:module_code size: 1024 limit: 65536 entries: 151 =hash_table:export_list size: 3203 used: 2324 objs: 4110 depth: 7 =index_table:export_list size: 5120 limit: 65536 entries: 4110 =hash_table:secondary_export_table size: 97 used: 0 objs: 0 depth: 0 =hash_table:process_reg size: 97 used: 53 objs: 74 depth: 3 =hash_table:fun_table size: 797 used: 516 objs: 807 depth: 6 =hash_table:node_table size: 5 used: 2 objs: 3 depth: 2 =hash_table:dist_table size: 5 used: 2 objs: 3 depth: 2 From rtrlists@REDACTED Mon Nov 23 14:08:34 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 23 Nov 2009 13:08:34 +0000 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <38e3b9f40911230319q24b894dex69135b8050308441@mail.gmail.com> References: <200911202335.nAKNZPkB057859@pluto.hedeland.org> <4846.1258953002@snookles.snookles.com> <38e3b9f40911230319q24b894dex69135b8050308441@mail.gmail.com> Message-ID: <6a3ae47e0911230508x14ef24b6s17d83a6fb13f2a6e@mail.gmail.com> Hi Pete, On Mon, Nov 23, 2009 at 11:19 AM, Peter-Henry Mander < phmander@REDACTED> wrote: > Hi Scott, > > No, I don't think that the issue is to do with paths. The loader > _does_ find the crypto_drv.so file, and fails to load it because > "symbol (unknown): value 0xfffffd7ffd514fdf does not fit." I began to > think that it was an AMD ABI "memory model" issue, where a 64 bit > offset could not fit into a 32 bit value of an address register > relative memory access instruction. > > Per pointed out that there are symbols in the crypto_drv.so file which > suggest that the linker is being invoked with the wrong options, > resulting in what smells like a stand-alone executable file with a > main() function, not a shared library object. > > I'm going to continue hacking the Makefiles to see if adding the > suggested -shared option to the crypto_drv linker command makes a > positive difference. > > And you are sure the right crypto_drv.so is getting picked, yes? You don't by any chance have another one on your system, which might be getting in the way? Potentially stupid question, I know, but worth asking nonetheless. Robby From steven.charles.davis@REDACTED Mon Nov 23 14:56:47 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 23 Nov 2009 05:56:47 -0800 (PST) Subject: list size 'causing VM "problems" In-Reply-To: References: Message-ID: <9a06d836-736f-4468-96ed-e7cf0d41547c@p19g2000vbq.googlegroups.com> Sounds very likely you are 64 bit which explains the 2 Gig, but I think that... {ok,Line} -> [process_line(Line)|read_lines(IOfd)]; ...would maybe not be "tail recursive" so the entire stack is being run hence chewing up 7G or more? What happens if you change the read_lines function to use an accumulator for the result? e.g... read_lines(IOfd) -> read_lines(IOfd, []). read_lines(IOfd, Acc) -> case file:read_line(IOfd) of {ok, Line} -> read_lines(IOfd, [process_line(Line) | Acc]); eof -> lists:reverse(Acc) end. On Nov 23, 4:09?am, Hendrik Visage wrote: > Hi there, > > Yes, I know this code is not yet optimal (I'm still learning :), but > it begs a few questions I'd like to understand from the VM etc. > > 1) I've run it fine with a small subset, but once I've loaded the 930k > lines file, the VM sucks up a lot of RAM/Virtualmemory. Like a burst > of about 2G (I have a 4G MacBookPro) and then once it returned in the > erl shell, the VM starts to go balistic and consumes >7G of > virtualmemory ;( > Q1: why did the VM exhibit this behaviour? the garbage collector going bad/mad?? > > 2) I will push the data into an ETS of sorts, as I'll try to find > duplicate files, but were thinking of an initial pull into a list, en > then fron there do the tests etc. The idea might be to pull in one > disk, and then compare it to another removal disk's files. > Q2: Should I rather do this straight into an ETS/DETS? > Q3: Should I preferably start to consider DETS 'cause of the size?? > Q4: will Mnesia help in this case? > > %%-------------------------------------------------------------------- > %% Function: process_line/1 > %% Description: take a properly formated line, and parse it, and > %% ? returns the tuple {Type,File,Hash} > %% Line: "MD5 (/.file) = d41d8cd98f00b204e9800998ecf8427e" > %% Nore some might be SHA1 in future. > %%-------------------------------------------------------------------- > process_line(Line) -> > ? ? {match,[Type,File,Hash]}= > ? ? ? ? re:run(Line, > ? ? ? ? ? ? ? ?"\(.*\)[ ][\\(]\(.*\)[\\)][ ][=][ ]\([0-9a-f]*\)\n", > ? ? ? ? ? ? ? ?[{capture,all_but_first,list}]), > ? ? {Type,File,Hash}. > > %%-------------------------------------------------------------------- > %% Function: read_lines/1 > %% Description: read in all the lines from a "properly formatted" > %% ? ?md5 output on MacOSX, returning a list with the tupples. > %%-------------------------------------------------------------------- > > read_lines(IOfd) -> > ? ? case ?file:read_line(IOfd) of > ? ? ? ? {ok,Line} -> > ? ? ? ? ? ? [process_line(Line)|read_lines(IOfd)]; > ? ? ? ? eof -> > ? ? ? ? ? ? [] > ? ? end. > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From phmander@REDACTED Mon Nov 23 14:59:30 2009 From: phmander@REDACTED (Peter-Henry Mander) Date: Mon, 23 Nov 2009 13:59:30 +0000 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <6a3ae47e0911230508x14ef24b6s17d83a6fb13f2a6e@mail.gmail.com> References: <200911202335.nAKNZPkB057859@pluto.hedeland.org> <4846.1258953002@snookles.snookles.com> <38e3b9f40911230319q24b894dex69135b8050308441@mail.gmail.com> <6a3ae47e0911230508x14ef24b6s17d83a6fb13f2a6e@mail.gmail.com> Message-ID: <38e3b9f40911230559r5077a2c6t67373a57f928952b@mail.gmail.com> Hi Robby, (Long time no speak. How've you been?) No, I carefully tried to manually run the commands to load the driver as they are in crypto_server.erl, and checked the paths used. So I'm confident that the correct crypto_drv.so is being loaded. I have another piece of the puzzle: gcc -64 -G -o ../priv/lib/i386-pc-solaris2.10/crypto_drv.so ../priv/obj/i386-pc -solaris2.10/crypto_drv.o /Software/openssl-0.9.8i/lib/libcrypto.a gcc: unrecognized option '-64' This is the same link command issued by the Makefile, regardless of whether the linker is GNU ld or Solaris ld. The configure script seems to assume that one size fits all. I'm going to try ./configure --without-gnu-ld --with-ld=/usr/ccs/bin/ld just to see if this hint is picked up by the configure script. Pete. On Mon, Nov 23, 2009 at 1:08 PM, Robert Raschke wrote: > Hi Pete, > > On Mon, Nov 23, 2009 at 11:19 AM, Peter-Henry Mander > wrote: >> >> Hi Scott, >> >> No, I don't think that the issue is to do with paths. The loader >> _does_ find the crypto_drv.so file, and fails to load it because >> "symbol (unknown): value 0xfffffd7ffd514fdf does not fit." I began to >> think that it was an AMD ABI "memory model" issue, where a 64 bit >> offset could not fit into a 32 bit value of an address register >> relative memory access instruction. >> >> Per pointed out that there are symbols in the crypto_drv.so file which >> suggest that the linker is being invoked with the wrong options, >> resulting in what smells like a stand-alone executable file with a >> main() function, not a shared library object. >> >> I'm going to continue hacking the Makefiles to see if adding the >> suggested -shared option to the crypto_drv linker command makes a >> positive difference. >> > > And you are sure the right crypto_drv.so?is getting picked, yes? You don't > by any chance have another one on your system, which might be getting in the > way? Potentially stupid question, I know, but worth asking nonetheless. > > Robby > > From phmander@REDACTED Mon Nov 23 15:13:50 2009 From: phmander@REDACTED (Peter-Henry Mander) Date: Mon, 23 Nov 2009 14:13:50 +0000 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <38e3b9f40911230559r5077a2c6t67373a57f928952b@mail.gmail.com> References: <200911202335.nAKNZPkB057859@pluto.hedeland.org> <4846.1258953002@snookles.snookles.com> <38e3b9f40911230319q24b894dex69135b8050308441@mail.gmail.com> <6a3ae47e0911230508x14ef24b6s17d83a6fb13f2a6e@mail.gmail.com> <38e3b9f40911230559r5077a2c6t67373a57f928952b@mail.gmail.com> Message-ID: <38e3b9f40911230613v7d2ed736ub6ad8eae0775c0d7@mail.gmail.com> Well, this didn't work: LD=/usr/ccs/bin/ld ./configure --without-gnu-ld --with-ld=/usr/ccs/bin/ld Now I'm going to create a wrapper to gcc to replace the -64 with -m64 and -G with -shared. Pete. On Mon, Nov 23, 2009 at 1:59 PM, Peter-Henry Mander wrote: > I'm going to try ./configure --without-gnu-ld > --with-ld=/usr/ccs/bin/ld just to see if this hint is picked up by the > configure script. From mikpe@REDACTED Mon Nov 23 15:32:47 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Mon, 23 Nov 2009 15:32:47 +0100 Subject: [erlang-questions] ll_alloc reallocate crash In-Reply-To: References: Message-ID: <19210.40207.491057.303903@pilspetsen.it.uu.se> mliu writes: > We have a simple tcp server spawning many processes that manage > accepted connections. these connections are long-lived, with long > periods of idle waits (5 min). While stress-testing the server, we > came across a strange memory-related crash (dump below). At the time > of the crash, we have around 150k connections, and roughly 4 processes > per conn (~600K). Server has 8Gb memory. > > Hoping someone has a clue of where we should look, within erlang or > maybe os configuration. ... > Here's the crash dump: > -- > Slogan: ll_alloc: Cannot reallocate 2033104 bytes of memory (of type > "fd_status"). > System version: Erlang R13B01 (erts-5.7.2) [source] [smp:8:8] [rq:8] > [async-threads:0] [hipe] [kernel-poll:true] > Compiled: Wed Aug 26 15:33:27 2009 > Atoms: 9193 > =memory > total: 3396483592 > processes: 2538096436 > processes_used: 2538092308 > system: 858387156 > atom: 416385 > atom_used: 395052 > binary: 248350576 > code: 3168606 > ets: 209338444 ... I can't analyze crash dumps, but this does look like a normal out-of-memory situation, though whether you ran out of virtual address space or physical memory I can't tell. Things to try: - build the Erlang VM as a 64-bit executable, otherwise it will be limited to about 3GB of virtual address space - do you have swap on the machine? if not, maybe you should From erlang@REDACTED Mon Nov 23 16:16:02 2009 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 23 Nov 2009 16:16:02 +0100 Subject: user management code for a web site? Message-ID: <9b08084c0911230716ve985827q412b907c1ac24ae0@mail.gmail.com> I want to make a web site :-) One thing I need is "all the junk to do with user management" This includes: - signing up for a new account - "I forgot my passport" - quotas and accounting - rejecting "bots" that sign up for false accounts - being able to send mails to gmail/etc. - CAPTCHA integration etc. - etc. etc. etc. These needs are *generic* to many applications and have very little to do with the specific application concerned. Is there an Erlang applications that can do all of this? I'm not concerned with presentation issues, nor with which web framework to use. Nor with any backend issues of which data store is used as a persistent store for user data. I just need a generic interface. Is there such a thing in Erlang? Failing this is there such a thing in any other language that I might learn from? Cheers /Joe From erlang@REDACTED Mon Nov 23 16:39:18 2009 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 23 Nov 2009 16:39:18 +0100 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: <18a1db030911201602n24ee3bb4ob4838492120f84ee@mail.gmail.com> References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> <5c493e530911161651w41e46aecu823bd4d36236815f@mail.gmail.com> <18a1db030911201602n24ee3bb4ob4838492120f84ee@mail.gmail.com> Message-ID: <9b08084c0911230739h2ed78eb3g8a91ba9381175cab@mail.gmail.com> On Sat, Nov 21, 2009 at 1:02 AM, zabrane Mikael wrote: > Hi List ! > > While trying to learn how to write a simple TCP Web Server in Erlang which > only dump what it gets to stdout, I realize that time to time, the HTTP > requests get truncated when reaching the server. Whenever I read something like that I think - "fragmentation". If you write N bytes to a TCP socket, you will eventually be able to read N bytes from the socket but the bytes may or may not be delivered "all in one go". Since you have said {packet, 0} you'll just get whatever happened to be read. This is why you *must* write a re-entrant parser. First you collect data until you see "\r\n\r\n" - only then can you parse the header. Then you check for a content length header. If you find a content length header it will contain the content length (N). Then you collect *exactly* N bytes following the "\r\n\r\n". Otherwise you collect until the socket closes (there is also a chunked alternative which I will ignore) The code in http://www.sics.se/~joe/tutorials/web_server/http_driver.erl does this: If you don't do this your program will work sometimes - in the case where the incoming packets were not fragmented but it will fail mysteriously if the packets are fragmented. Forgetting about fragmentation is the "first basic" mistake that *everybody* makes when writing networking code - .... This mistake happens often when you deploy something. You test it locally on localhost it works. You test it live on the Internet - it fails. Why? Packets are not often fragmented on localhost - but very rare. The chance of fragmentation on the Internet is very high - even if you have good connection. aside: this is why one of the tcp options is {packet, N} - if you write a client AND a server in Erlang and BOTH use (say) {packet,4} then gen_tcp will silently reassemble fragmented packets behind the scenes before delivering them to the application program. This together with term_to_binary (and its inverse) and the bit syntax will save you many sleepless nights. > My main socket loop looks > like: > > ------------------------------------------- > -define(TCP_OPTIONS,[binary, {packet, 0}, {active, false}, {reuseaddr, > true}]). > ... > loop_recv(Socket) > ? case gen_tcp:recv(Socket, 0) of > ? ? ? ?{ok, BinData} -> > ? ? ? ? ? ? %% here, I'm assuming that all the HTTP request (Headers + > Body) is in "BinData". Hope I'm right. No No No - programs should not depend upon Hope. This assumption is wrong. Hint - print out the packet lengths, so you can see the pack lengths .. /Joe > ? ? ? ? ? ? io:format("BinData: ~p~n", []), > ? ? ? ? ? ? ok; > ? ? ? ?NotOK -> > ? ? ? ? ? ?error_logger:info_report([{"gen_tcp:recv/2", NotOK}]), > ? ? ? ? ? ?error > ? ?end. > ------------------------------------------- > > For some requests, the "io:format" prints a truncated data (in BinData): > > <<" > http://www.foo.tv/images/v30/LoaderV3.swf?loop=false&quality=high&request=357&HTTP/1.0\r\nHost: > www.foo.tv\r\nUser-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; > fr; rv:1.9.0.2) Gecko/2008090512 Firefox/3.0.2\r\nAccept: text/h">> > > As you can see, the request isn't complete "... Accept: text/h". > > Am I doing somthing wrong? How can I fix it please? > > Regards > Zabrane > > 2009/11/17 Tony Rogvall > >> Do not forget about {active, once} mode. >> {active,once} will receive one message (depends on buffer size etc) >> the it will switch to passive mode. To get the next message you use >> inet:setopts(Socket, [{active,once}]) to activate it again. This mode >> enables >> a selective receive at the same time as it enables flow control. >> >> /Tony >> >> >> On 17 nov 2009, at 01.51, Ngoc Dao wrote: >> >> >> From inet's doc: >> > >> http://www1.erlang.org/documentation/doc-4.9.1/lib/kernel-2.4.1/doc/html/inet.html >> > >> > If the active option is true, which is the default, everything >> > received from the socket will be sent as messages to the receiving >> > process. If the active option is set to false (passive mode), the >> > process must explicitly receive incoming data by calling >> > gen_tcp:recv/N or gen_udp:recv/N ?(depending on the type of socket). >> > Note: Passive mode provides flow control; the other side will not be >> > able send faster than the receiver can read. Active mode provides no >> > flow control; a fast sender could easily overflow the receiver with >> > incoming messages. Use active mode only if your high-level protocol >> > provides its own flow control (for instance, acknowledging received >> > messages) or the amount of data exchanged is small. >> > >> > >> > On Tue, Nov 17, 2009 at 2:59 AM, ERLANG wrote: >> >> Hi Chandru ! >> >> >> >> That's fix my problem. Thanks. >> >> While googling a bit, I found two ways to read from the Socket: >> >> >> >> recv(Socket, Bin) -> >> >> ? ?receive >> >> ? ? ? ?{tcp, Socket, B} -> >> >> ? ? ? ? ? ?io:format(".", []), >> >> ? ? ? ? ? ?recv(Socket, concat_binary([Bin, B])); >> >> ? ? ? ?{tcp_closed, Socket} -> >> >> ? ? ? ? ? ?{ok, Bin}; >> >> ? ? ? ?Other -> >> >> ? ? ? ? ? ?{error, {socket, Other}} >> >> ? ? ? ?after >> >> ? ? ? ? ? ??TIMEOUT -> >> >> ? ? ? ? ? ?{error, {socket, timeout}} >> >> ? ?end. >> >> >> >> % version 2 with "gen_tcp:recv" >> >> recv2(Socket, Bin) -> >> >> ? ?case gen_tcp:recv(Socket, 0, ?TIMEOUT) of >> >> ? ? ? ? {ok, B} -> >> >> ? ? ? ? ? ? io:format(".", []), >> >> ? ? ? ? ? ? recv(Socket, concat_binary([Bin, B])); >> >> ? ? ? ? {error, closed} -> >> >> ? ? ? ? ? ? {ok, Bin}; >> >> ? ? ? ?{error, timeout} -> >> >> ? ? ? ? ? ? {error, {socket, timeout}}; >> >> ? ? ? ? Other -> >> >> ? ? ? ? ? ? {error, {socket, Other}} >> >> ? ? end. >> >> >> >> >> >> Which one is the best in my case (see below: fetch.erl)? >> >> >> >> Regards >> >> Zabrane >> >> >> >> Le 16 nov. 09 ? 18:53, Chandru a ?crit : >> >> >> >>> You are expecting the server to indicate end of response by closing the >> >>> connection, but because you specify HTTP/1.1 in the request, the server >> is >> >>> holding up your connection, and you are timing out. Try replacing >> HTTP/1.1 >> >>> with HTTP/1.0 in your request, or parse the response to detect end of >> >>> response. >> >>> >> >>> cheers >> >>> Chandru >> >>> >> >>> 2009/11/16 zabrane Mikael >> >>> >> >>>> Hi List ! >> >>>> >> >>>> New to Erlang, I'm trying to implement a simple URL fetcher. >> >>>> Here's my code (please, feel free to correct it if you find any bug or >> >>>> know >> >>>> a better approach): >> >>>> >> >>>> >> >>>> >> >>>> >> 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8---- >> >>>> -module(fetch). >> >>>> >> >>>> -export([url/1]). >> >>>> >> >>>> -define(TIMEOUT, ? ?7000). >> >>>> -define(TCP_OPTS, ? [binary, {packet, raw}, {nodelay, true}, >> >>>> ? ? ? ? ? ? ? ? ? {active, true}]). >> >>>> >> >>>> url(Url) -> >> >>>> ?{ok, _Tag, Host, Port} = split_url(Url), >> >>>> >> >>>> ?Hdrs = [], >> >>>> ?Request = ["GET ", Url, " HTTP/1.1\r\n", Hdrs, "\r\n\r\n"], >> >>>> >> >>>> ?case catch gen_tcp:connect(Host, Port, ?TCP_OPTS) of >> >>>> ? ?{'EXIT', Why} -> >> >>>> ? ? ? ? ?{error, {socket_exit, Why}}; >> >>>> ? ? ?{error, Why} -> >> >>>> ? ? ? ? ?{error, {socket_error, Why}}; >> >>>> ? ? ?{ok, Socket} -> >> >>>> ? ? ? ? ?gen_tcp:send(Socket, list_to_binary(Request)), >> >>>> ? ? ? ? ?recv(Socket, list_to_binary([])) >> >>>> ?end. >> >>>> >> >>>> recv(Socket, Bin) -> >> >>>> ?receive >> >>>> ? ? ?{tcp, Socket, B} -> >> >>>> ? ? ? ? ?io:format(".", []), >> >>>> ? ? ? ? ?recv(Socket, concat_binary([Bin, B])); >> >>>> ? ? ?{tcp_closed, Socket} -> >> >>>> ? ? ? ? ?{ok, Bin}; >> >>>> ? ? ?Other -> >> >>>> ? ? ? ? ?{error, {socket, Other}} >> >>>> after >> >>>> ??TIMEOUT -> >> >>>> ? ? ? ? ?{error, {socket, timeout}} >> >>>> ?end. >> >>>> >> >>>> >> >>>> split_url([$h,$t,$t,$p,$:,$/,$/|T]) -> ?split_url(http, T); >> >>>> split_url(_X) ? ? ? ? ? ? ? ? ? ? ? -> ?{error, split_url}. >> >>>> >> >>>> split_url(Tag, X) -> >> >>>> ?case string:chr(X, $:) of >> >>>> ? ? ?0 -> >> >>>> ? ? ? ? ?Port = 80, >> >>>> ? ? ? ? ?case string:chr(X,$/) of >> >>>> ? ? ? ? ? ? ?0 -> >> >>>> ? ? ? ? ? ? ? ? ?{ok, Tag, X, Port}; >> >>>> ? ? ? ? ? ? ?N -> >> >>>> ? ? ? ? ? ? ? ? ?Site = string:substr(X,1,N-1), >> >>>> ? ? ? ? ? ? ? ? ?{ok, Tag, Site, Port} >> >>>> ? ? ? ? ?end; >> >>>> ? ? ?N1 -> >> >>>> ? ? ? ? ?case string:chr(X,$/) of >> >>>> ? ? ? ? ? ? ?0 -> >> >>>> ? ? ? ? ? ? ? ? ?error; >> >>>> ? ? ? ? ? ? ?N2 -> >> >>>> ? ? ? ? ? ? ? ? ?PortStr = string:substr(X,N1+1, N2-N1-1), >> >>>> ? ? ? ? ? ? ? ? ?case catch list_to_integer(PortStr) of >> >>>> ? ? ? ? ? ? ? ? ? ? ?{'EXIT', _} -> >> >>>> ? ? ? ? ? ? ? ? ? ? ? ? ?{error, port_number}; >> >>>> ? ? ? ? ? ? ? ? ? ? ?Port -> >> >>>> ? ? ? ? ? ? ? ? ? ? ? ? ?Site = string:substr(X,1,N1-1), >> >>>> ? ? ? ? ? ? ? ? ? ? ? ? ?{ok, Tag, Site, Port} >> >>>> ? ? ? ? ? ? ? ? ?end >> >>>> ? ? ? ? ?end >> >>>> ?end. >> >>>> >> >>>> >> >>>> >> >>>> >> 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8------ >> >>>> >> >>>> When testing it, the receiving socket gets very very slow: >> >>>> $ erl >> >>>> 1> c(fetch). >> >>>> 2> Bin = fetch:url("http://www.google.com"). >> >>>> ......{error,{socket,timeout}} >> >>>> >> >>>> Am I missing something? >> >>>> What I like to get at the end is a very fast fetcher. Any hint? >> >>>> >> >>>> Regards >> >>>> Zabrane >> >>>> >> >> >> >> >> >> ________________________________________________________________ >> >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> >> erlang-questions (at) erlang.org >> >> >> >> >> > >> > ________________________________________________________________ >> > erlang-questions mailing list. See http://www.erlang.org/faq.html >> > erlang-questions (at) erlang.org >> > >> >> > From roberto.aloi@REDACTED Mon Nov 23 16:48:28 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Mon, 23 Nov 2009 15:48:28 +0000 (GMT) Subject: [erlang-questions] user management code for a web site? In-Reply-To: <6259594.55171258991088009.JavaMail.root@zimbra> Message-ID: <13744342.55191258991308441.JavaMail.root@zimbra> Hi Joe, > I want to make a web site :-) lol > One thing I need is "all the junk to do with user management" Fair enough > This includes: > > - signing up for a new account > - "I forgot my passport" > - quotas and accounting > - rejecting "bots" that sign up for false accounts > - being able to send mails to gmail/etc. > - CAPTCHA integration etc. > - etc. etc. etc. > > > These needs are *generic* to many applications and have very little > to > do with the > specific application concerned. Is there an Erlang applications that > can do all of this? > I'm not concerned with presentation issues, nor with which web > framework to use. > Nor with any backend issues of which data store is used as a > persistent store > for user data. > > I just need a generic interface. Is there such a thing in Erlang? The Erlang Web framework offers an e_auth component. The idea is to have an e_auth interface layer and some specific implementations. As far as I know they support dets tables and mnesia. They also have a LDAP implementation, even if I'm not sure they ever completed it. More information here: http://wiki.erlang-web.org/Tutorial/Step5_Authentication?highlight=(e\_auth) About the rest of your needs, I already have 80% of what you asked, but the code is not released yet. This will hopefully happen in the next couple of weeks. I'm still missing CAPTCHAs. I was thinking about implementing an Erlang Web component for this: http://www.captcha.net/ I also would like to integrate my stuff with esmtp (so far I'm using my own mailer): http://github.com/archaelus/esmtp Let me know if you want to share some of this work. > Failing this is there such a thing in any other language that I might > learn from? There are no languages other than Erlang and Music. Maybe Sicilian. Regards, Roberto Aloi Erlang Training and Consulting Ltd. http://www.erlang-consulting.com Twitter: @prof3ta Blog: http://aloiroberto.wordpress.com From olivier.sambourg@REDACTED Mon Nov 23 16:55:51 2009 From: olivier.sambourg@REDACTED (Olivier Sambourg) Date: Mon, 23 Nov 2009 16:55:51 +0100 Subject: [erlang-questions] user management code for a web site? In-Reply-To: <13744342.55191258991308441.JavaMail.root@zimbra> References: <6259594.55171258991088009.JavaMail.root@zimbra> <13744342.55191258991308441.JavaMail.root@zimbra> Message-ID: Hi, There's also the new Erlang CMS, Zotonic, which was announced here a couple of days ago: http://zotonic.com/ Looks promising, but I haven't got the chance to give it a try yet. Olivier On Mon, Nov 23, 2009 at 4:48 PM, Roberto Aloi < roberto.aloi@REDACTED> wrote: > Hi Joe, > > > I want to make a web site :-) > lol > > > One thing I need is "all the junk to do with user management" > Fair enough > > > This includes: > > > > - signing up for a new account > > - "I forgot my passport" > > - quotas and accounting > > - rejecting "bots" that sign up for false accounts > > - being able to send mails to gmail/etc. > > - CAPTCHA integration etc. > > - etc. etc. etc. > > > > > > These needs are *generic* to many applications and have very little > > to > > do with the > > specific application concerned. Is there an Erlang applications that > > can do all of this? > > I'm not concerned with presentation issues, nor with which web > > framework to use. > > Nor with any backend issues of which data store is used as a > > persistent store > > for user data. > > > > I just need a generic interface. Is there such a thing in Erlang? > > The Erlang Web framework offers an e_auth component. The idea is to have an > e_auth interface layer and some specific implementations. > As far as I know they support dets tables and mnesia. They also have a LDAP > implementation, even if I'm not sure they ever completed it. > More information here: > > http://wiki.erlang-web.org/Tutorial/Step5_Authentication?highlight=(e > \_auth) > > About the rest of your needs, I already have 80% of what you asked, but the > code is not released yet. This will hopefully happen in the next couple of > weeks. > > I'm still missing CAPTCHAs. I was thinking about implementing an Erlang Web > component for this: > http://www.captcha.net/ > > I also would like to integrate my stuff with esmtp (so far I'm using my own > mailer): > > http://github.com/archaelus/esmtp > > Let me know if you want to share some of this work. > > > Failing this is there such a thing in any other language that I might > > learn from? > There are no languages other than Erlang and Music. Maybe Sicilian. > > Regards, > > Roberto Aloi > Erlang Training and Consulting Ltd. > http://www.erlang-consulting.com > Twitter: @prof3ta > Blog: http://aloiroberto.wordpress.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From phmander@REDACTED Mon Nov 23 16:58:53 2009 From: phmander@REDACTED (Peter-Henry Mander) Date: Mon, 23 Nov 2009 15:58:53 +0000 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <38e3b9f40911230613v7d2ed736ub6ad8eae0775c0d7@mail.gmail.com> References: <200911202335.nAKNZPkB057859@pluto.hedeland.org> <4846.1258953002@snookles.snookles.com> <38e3b9f40911230319q24b894dex69135b8050308441@mail.gmail.com> <6a3ae47e0911230508x14ef24b6s17d83a6fb13f2a6e@mail.gmail.com> <38e3b9f40911230559r5077a2c6t67373a57f928952b@mail.gmail.com> <38e3b9f40911230613v7d2ed736ub6ad8eae0775c0d7@mail.gmail.com> Message-ID: <38e3b9f40911230758h59f94ed0n3786966cfeb35313@mail.gmail.com> The wrapper script detects the presence of -64 and -G flags, and substitutes the Solaris 'ld' for any occurances of 'gcc' where these flags exist: Original: /Software/Tools-64/gcc \ -64 -G \ -o ../priv/lib/i386-pc-solaris2.10/crypto_drv.so \ ../priv/obj/i386-pc-solaris2.10/crypto_drv.o \ /Software/openssl-0.9.8i/lib/libcrypto.a Modified: /usr/ccs/bin/ld \ -64 -G \ -o ../priv/lib/i386-pc-solaris2.10/crypto_drv.so \ ../priv/obj/i386-pc-solaris2.10/crypto_drv.o \ /Software/openssl-0.9.8i/lib/libcrypto.a The build completes without error, but the original problem remains: > ~/R13B03/bin/erl Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [kernel-poll:false] Eshell V5.7.4 (abort with ^G) 1> application:start(crypto). {error,{shutdown,{crypto_app,start,[normal,[]]}}} =ERROR REPORT==== 23-Nov-2009::15:39:29 === Unable to load crypto_drv. Failed with error: "ld.so.1: beam.smp: fatal: relocation error: R_AMD64_32: file /Software/R13B03/lib/erlang/lib/crypto-1.6.3/priv/lib/crypto_drv.so: symbol (unknown): value 0xfffffd7ffd6582d0 does not fit" OpenSSL might not be installed on this system. 2> =INFO REPORT==== 23-Nov-2009::15:39:29 === application: crypto exited: {shutdown,{crypto_app,start,[normal,[]]}} type: temporary Going through the motions manually: 3> PrivDir = code:priv_dir(crypto). "/Software/R13B03/lib/erlang/lib/crypto-1.6.3/priv" 4> LibDir1 = filename:join([PrivDir, "lib"]). "/Software/R13B03/lib/erlang/lib/crypto-1.6.3/priv/lib" 5> erl_ddll:load_driver(LibDir1, crypto_drv). {error,{open_error,-11}} 6> erl_ddll:format_error({open_error,-11}). "ld.so.1: beam.smp: fatal: relocation error: R_AMD64_32: file /Software/R13B03/lib/erlang/lib/crypto-1.6.3/priv/lib/crypto_drv.so: symbol (unknown): value 0xfffffd7ffd7582d0 does not fit" 677> /bin/file /Software/R13B03/lib/erlang/lib/crypto-1.6.3/priv/lib/crypto_drv.so /Software/R13B03/lib/erlang/lib/crypto-1.6.3/priv/lib/crypto_drv.so: ELF 64-bit LSB dynamic lib AMD64 Version 1, dynamically linked, not stripped 678> /bin/file /Software/R13B03/lib/erlang/erts-5.7.4/bin/beam /Software/R13B03/lib/erlang/erts-5.7.4/bin/beam: ELF 64-bit LSB executable AMD64 Version 1, dynamically linked, not stripped 679> /bin/file /Software/R13B03/lib/erlang/erts-5.7.4/bin/beam.smp /Software/R13B03/lib/erlang/erts-5.7.4/bin/beam.smp: ELF 64-bit LSB executable AMD64 Version 1, dynamically linked, not stripped 682> /usr/xpg4/bin/nm /Software/R13B03/lib/erlang/lib/crypto-1.6.3/priv/lib/crypto_drv.so | sed -n '/UNDEF/p' ( This does not show any references to 'main' nor 'exit' as pointer out by Per. This time it looks as if the linker did the correct thing. ) So, I do suspect that the configure script is getting confused by which linker is suitable to use, but also the "relocation error: R_AMD64_32 ... symbol (unknown): value 0xfffffd7ffd7582d0 does not fit" is not strictly to do with the way crypto_drv.so is linked, but something to do with the point raised by Wallentin Dahlberg on Friday: The AMD ABI small/medium/large memory model. Is it at all possible to regress the crypto_drv.so 'C' source back to the one in R12B? Is the API compatible? It may have bugs that are mended in R13B, but it didn't seem to suffer from the R_AMD64_32 error. Could this be due to changes in the way pointer arithmetic is done in crypto_drv.c version 1.5.3 vs. version 1.6.3? Pete. On Mon, Nov 23, 2009 at 2:13 PM, Peter-Henry Mander wrote: > Well, this didn't work: > LD=/usr/ccs/bin/ld ./configure --without-gnu-ld --with-ld=/usr/ccs/bin/ld > > Now I'm going to create a wrapper to gcc to replace the -64 with -m64 > and -G with -shared. > > Pete. > > On Mon, Nov 23, 2009 at 1:59 PM, Peter-Henry Mander > wrote: > >> I'm going to try ./configure --without-gnu-ld >> --with-ld=/usr/ccs/bin/ld just to see if this hint is picked up by the >> configure script. > From jerome.sautret@REDACTED Mon Nov 23 17:29:07 2009 From: jerome.sautret@REDACTED (=?ISO-8859-1?Q?J=E9r=F4me_Sautret?=) Date: Mon, 23 Nov 2009 17:29:07 +0100 Subject: [ANN] ejabberd 2.1.0 released Message-ID: <8397da240911230829g6960d6cfxd6ef18e2ebe06486@mail.gmail.com> ejabberd 2.1.0 is finally released, and it comes with an enormous list of 314 new features, improvements and bugfixes. It has been 21 months since the last major ejabberd release 2.0.0, and 8 months since 2.0.5. Very brief list of additions: * Erlang/OTP R13 support * HTTP-Bind (BOSH) optimization and clean-up * LDAPS support * MUC rooms can require a CAPTCHA to join the room * PubSub: major improvements. Update to XEP version 1.12 * PEP and Caps: many improvements * Run commands with ejabberdctl or other frontends (XML-RPC, HTTP) * STUN server * XEPs like: XMPP Ping, Roster Versioning, Import/Export Format Check the Release Notes for a more complete list of changes: http://www.process-one.net/en/ejabberd/release_notes/release_note_ejabberd_2.1.0 If you upgrade from a previous ejabberd version, read carefully the release notes, because there were several changes in the installation path and the configuration options. The list of solved tickets since 2.0.0 is available on ProcessOne bug tracker: http://redir.process-one.net/ejabberd-2.1.0 ejabberd 2.1.0 is available as source code package and binary installers for Linux 32 bits, 64 bits, Mac OS X Intel, and Windows: http://www.process-one.net/en/ejabberd/downloads Acknowledgments: A big thank you to everyone who helped in the ejabberd project during the last months. This list is extracted from ejabberd's commit log between version 2.0.0 and 2.1.0: Alexander Tsvyashchenko, Alexey Shchepin, Anastasia Gornostaeva, Andreas Ntaflos, Andreas van Cranenburgh, Andrzej Smyk, Andy Harb, Andy Skelton, Anton Podavalov, Apag0r, Attila Vangel, Badlop, Brian Cully, Carlos E. Lopez, Chris Newcombe, Christoph Anton Mitterer, Christophe Romain, Christopher Dupont, Clochix, Dariusz Markowicz, Dominges, Doruk Fisek, Eric Cestari, Etan Reisner, Evgeniy Khramtsov, Fabrice Colliot, Felix Geyer, Gabriele Stilli, Geoff Cant, George Hazan, Glenn Sieb, Graham Whitted, Gustaf Alstr?mer, Igor Goryachev, Jack Moffitt, Jacob Vorreuter, James Iakovos Mandelis, Jan Kusanagi, Janusz Dziemidowicz, Jeffrey Rogiers, Jon Bendtsen, Jonathan Auer, Jonathan Schleifer, Juan Pablo Carlino, Justin Boffemmyer, Justin Kirby, J?r?me Sautret, Kevin Crosbie, Konstantin Khomoutov, Luca Brivio, Luk?? Pol?vka, Magnus Henoch, Marcin Owsiany, Marek Be?ka, Martin Langhoff, Massimiliano Mirra, Mats Cronqvist, Matthew Reilly, Maxim Ryazanov, Michal Schmidt, Micka?l R?mond, Myers Carpenter, Neustradamus, Nicolas V?rit?, Nikolaus Polak, Oleg Palij, Ot?vio Fernandes, Pablo Polvorin, Pablo Saratxaga, Paul Guyot, Pedro Melo, Philipp Hancke, Ruslan Rakhmanin, Sergei Golovan, Shelley Shyan, Smart2128, Spike, Stefan Strigler, Stephan Maka, Stian B. Barmen, Thomas, Thomas Baden, Thore Alstromer, Tsukasa Hamano, Vincent Barat, Viq, Zbyszek ???kiewski, and Zhan Caibao. From zabrane3@REDACTED Mon Nov 23 17:55:56 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 23 Nov 2009 17:55:56 +0100 Subject: [erlang-questions] gen_tcp very slow to fetch data In-Reply-To: <9b08084c0911230739h2ed78eb3g8a91ba9381175cab@mail.gmail.com> References: <18a1db030911160817t3fff6715mc50772a6450a3a6c@mail.gmail.com> <739B7E6C-D7A7-4267-9E09-5E3E33A16FC6@gmail.com> <5c493e530911161651w41e46aecu823bd4d36236815f@mail.gmail.com> <18a1db030911201602n24ee3bb4ob4838492120f84ee@mail.gmail.com> <9b08084c0911230739h2ed78eb3g8a91ba9381175cab@mail.gmail.com> Message-ID: <18a1db030911230855l14e1d3c0xda9fc573b27ae834@mail.gmail.com> Thanks for the lesson guys ! Regards Zabrane 2009/11/23 Joe Armstrong > On Sat, Nov 21, 2009 at 1:02 AM, zabrane Mikael > wrote: > > Hi List ! > > > > While trying to learn how to write a simple TCP Web Server in Erlang > which > > only dump what it gets to stdout, I realize that time to time, the HTTP > > requests get truncated when reaching the server. > > Whenever I read something like that I think - "fragmentation". > > If you write N bytes to a TCP socket, you will eventually be able to > read N bytes from the socket > but the bytes may or may not be delivered "all in one go". Since you > have said {packet, 0} you'll just > get whatever happened to be read. This is why you *must* write a > re-entrant parser. > > First you collect data until you see "\r\n\r\n" - only then can you > parse the header. > Then you check for a content length header. If you find a content > length header it will contain the > content length (N). Then you collect *exactly* N bytes following the > "\r\n\r\n". Otherwise you collect > until the socket closes (there is also a chunked alternative which I > will ignore) > > The code in http://www.sics.se/~joe/tutorials/web_server/http_driver.erl > does this: > > If you don't do this your program will work sometimes - in the case > where the incoming packets were not > fragmented but it will fail mysteriously if the packets are fragmented. > > Forgetting about fragmentation is the "first basic" mistake that > *everybody* makes when writing > networking code - .... > > This mistake happens often when you deploy something. You test it > locally on localhost it works. > You test it live on the Internet - it fails. > > Why? Packets are not often fragmented on localhost - but very rare. > The chance of fragmentation on the Internet is very high - even if you > have good connection. > > aside: this is why one of the tcp options is {packet, N} - if you > write a client AND a server in Erlang > and BOTH use (say) {packet,4} then gen_tcp will silently reassemble > fragmented packets behind the scenes > before delivering them to the application program. > > This together with term_to_binary (and its inverse) and the bit syntax > will save you many sleepless nights. > > > > My main socket loop looks > > like: > > > > ------------------------------------------- > > -define(TCP_OPTIONS,[binary, {packet, 0}, {active, false}, {reuseaddr, > > true}]). > > ... > > loop_recv(Socket) > > case gen_tcp:recv(Socket, 0) of > > {ok, BinData} -> > > %% here, I'm assuming that all the HTTP request (Headers + > > Body) is in "BinData". Hope I'm right. > > No No No - programs should not depend upon Hope. This assumption is wrong. > > Hint - print out the packet lengths, so you can see the pack lengths .. > > /Joe > > > > io:format("BinData: ~p~n", []), > > ok; > > NotOK -> > > error_logger:info_report([{"gen_tcp:recv/2", NotOK}]), > > error > > end. > > ------------------------------------------- > > > > For some requests, the "io:format" prints a truncated data (in BinData): > > > > <<" > > > http://www.foo.tv/images/v30/LoaderV3.swf?loop=false&quality=high&request=357&HTTP/1.0 > \r\nHost: > > www.foo.tv\r\nUser-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X > 10.5; > > fr; rv:1.9.0.2) Gecko/2008090512 Firefox/3.0.2\r\nAccept: text/h">> > > > > As you can see, the request isn't complete "... Accept: text/h". > > > > Am I doing somthing wrong? How can I fix it please? > > > > Regards > > Zabrane > > > > 2009/11/17 Tony Rogvall > > > >> Do not forget about {active, once} mode. > >> {active,once} will receive one message (depends on buffer size etc) > >> the it will switch to passive mode. To get the next message you use > >> inet:setopts(Socket, [{active,once}]) to activate it again. This mode > >> enables > >> a selective receive at the same time as it enables flow control. > >> > >> /Tony > >> > >> > >> On 17 nov 2009, at 01.51, Ngoc Dao wrote: > >> > >> >> From inet's doc: > >> > > >> > http://www1.erlang.org/documentation/doc-4.9.1/lib/kernel-2.4.1/doc/html/inet.html > >> > > >> > If the active option is true, which is the default, everything > >> > received from the socket will be sent as messages to the receiving > >> > process. If the active option is set to false (passive mode), the > >> > process must explicitly receive incoming data by calling > >> > gen_tcp:recv/N or gen_udp:recv/N (depending on the type of socket). > >> > Note: Passive mode provides flow control; the other side will not be > >> > able send faster than the receiver can read. Active mode provides no > >> > flow control; a fast sender could easily overflow the receiver with > >> > incoming messages. Use active mode only if your high-level protocol > >> > provides its own flow control (for instance, acknowledging received > >> > messages) or the amount of data exchanged is small. > >> > > >> > > >> > On Tue, Nov 17, 2009 at 2:59 AM, ERLANG wrote: > >> >> Hi Chandru ! > >> >> > >> >> That's fix my problem. Thanks. > >> >> While googling a bit, I found two ways to read from the Socket: > >> >> > >> >> recv(Socket, Bin) -> > >> >> receive > >> >> {tcp, Socket, B} -> > >> >> io:format(".", []), > >> >> recv(Socket, concat_binary([Bin, B])); > >> >> {tcp_closed, Socket} -> > >> >> {ok, Bin}; > >> >> Other -> > >> >> {error, {socket, Other}} > >> >> after > >> >> ?TIMEOUT -> > >> >> {error, {socket, timeout}} > >> >> end. > >> >> > >> >> % version 2 with "gen_tcp:recv" > >> >> recv2(Socket, Bin) -> > >> >> case gen_tcp:recv(Socket, 0, ?TIMEOUT) of > >> >> {ok, B} -> > >> >> io:format(".", []), > >> >> recv(Socket, concat_binary([Bin, B])); > >> >> {error, closed} -> > >> >> {ok, Bin}; > >> >> {error, timeout} -> > >> >> {error, {socket, timeout}}; > >> >> Other -> > >> >> {error, {socket, Other}} > >> >> end. > >> >> > >> >> > >> >> Which one is the best in my case (see below: fetch.erl)? > >> >> > >> >> Regards > >> >> Zabrane > >> >> > >> >> Le 16 nov. 09 ? 18:53, Chandru a ?crit : > >> >> > >> >>> You are expecting the server to indicate end of response by closing > the > >> >>> connection, but because you specify HTTP/1.1 in the request, the > server > >> is > >> >>> holding up your connection, and you are timing out. Try replacing > >> HTTP/1.1 > >> >>> with HTTP/1.0 in your request, or parse the response to detect end > of > >> >>> response. > >> >>> > >> >>> cheers > >> >>> Chandru > >> >>> > >> >>> 2009/11/16 zabrane Mikael > >> >>> > >> >>>> Hi List ! > >> >>>> > >> >>>> New to Erlang, I'm trying to implement a simple URL fetcher. > >> >>>> Here's my code (please, feel free to correct it if you find any bug > or > >> >>>> know > >> >>>> a better approach): > >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> > 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8---- > >> >>>> -module(fetch). > >> >>>> > >> >>>> -export([url/1]). > >> >>>> > >> >>>> -define(TIMEOUT, 7000). > >> >>>> -define(TCP_OPTS, [binary, {packet, raw}, {nodelay, true}, > >> >>>> {active, true}]). > >> >>>> > >> >>>> url(Url) -> > >> >>>> {ok, _Tag, Host, Port} = split_url(Url), > >> >>>> > >> >>>> Hdrs = [], > >> >>>> Request = ["GET ", Url, " HTTP/1.1\r\n", Hdrs, "\r\n\r\n"], > >> >>>> > >> >>>> case catch gen_tcp:connect(Host, Port, ?TCP_OPTS) of > >> >>>> {'EXIT', Why} -> > >> >>>> {error, {socket_exit, Why}}; > >> >>>> {error, Why} -> > >> >>>> {error, {socket_error, Why}}; > >> >>>> {ok, Socket} -> > >> >>>> gen_tcp:send(Socket, list_to_binary(Request)), > >> >>>> recv(Socket, list_to_binary([])) > >> >>>> end. > >> >>>> > >> >>>> recv(Socket, Bin) -> > >> >>>> receive > >> >>>> {tcp, Socket, B} -> > >> >>>> io:format(".", []), > >> >>>> recv(Socket, concat_binary([Bin, B])); > >> >>>> {tcp_closed, Socket} -> > >> >>>> {ok, Bin}; > >> >>>> Other -> > >> >>>> {error, {socket, Other}} > >> >>>> after > >> >>>> ?TIMEOUT -> > >> >>>> {error, {socket, timeout}} > >> >>>> end. > >> >>>> > >> >>>> > >> >>>> split_url([$h,$t,$t,$p,$:,$/,$/|T]) -> split_url(http, T); > >> >>>> split_url(_X) -> {error, split_url}. > >> >>>> > >> >>>> split_url(Tag, X) -> > >> >>>> case string:chr(X, $:) of > >> >>>> 0 -> > >> >>>> Port = 80, > >> >>>> case string:chr(X,$/) of > >> >>>> 0 -> > >> >>>> {ok, Tag, X, Port}; > >> >>>> N -> > >> >>>> Site = string:substr(X,1,N-1), > >> >>>> {ok, Tag, Site, Port} > >> >>>> end; > >> >>>> N1 -> > >> >>>> case string:chr(X,$/) of > >> >>>> 0 -> > >> >>>> error; > >> >>>> N2 -> > >> >>>> PortStr = string:substr(X,N1+1, N2-N1-1), > >> >>>> case catch list_to_integer(PortStr) of > >> >>>> {'EXIT', _} -> > >> >>>> {error, port_number}; > >> >>>> Port -> > >> >>>> Site = string:substr(X,1,N1-1), > >> >>>> {ok, Tag, Site, Port} > >> >>>> end > >> >>>> end > >> >>>> end. > >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> > 8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8-----8------ > >> >>>> > >> >>>> When testing it, the receiving socket gets very very slow: > >> >>>> $ erl > >> >>>> 1> c(fetch). > >> >>>> 2> Bin = fetch:url("http://www.google.com"). > >> >>>> ......{error,{socket,timeout}} > >> >>>> > >> >>>> Am I missing something? > >> >>>> What I like to get at the end is a very fast fetcher. Any hint? > >> >>>> > >> >>>> Regards > >> >>>> Zabrane > >> >>>> > >> >> > >> >> > >> >> ________________________________________________________________ > >> >> erlang-questions mailing list. See http://www.erlang.org/faq.html > >> >> erlang-questions (at) erlang.org > >> >> > >> >> > >> > > >> > ________________________________________________________________ > >> > erlang-questions mailing list. See http://www.erlang.org/faq.html > >> > erlang-questions (at) erlang.org > >> > > >> > >> > > > From aniko@REDACTED Mon Nov 23 19:32:31 2009 From: aniko@REDACTED (Aniko Nagyne Vig) Date: Mon, 23 Nov 2009 18:32:31 +0000 (GMT) Subject: Take ProTest Tools Survey and win Erlang Programming book! In-Reply-To: <21457382.55841259001082299.JavaMail.root@zimbra> Message-ID: <19964578.55861259001151847.JavaMail.root@zimbra> It has been two years since ProTest (http://www.protest-project.eu) - an EU funded research project on property based testing and test driven development - was established. We are preparing for the next phase, and in doing so need YOUR feedback! Have you heard about Exago, McErlang, Onviso, PULSE, QuickCheck or Wrangler? Are you using any of these tools? By answering these and other questions, you will help influence the research and dissemination strategies over the next two years, ensuring that we are right on track to solving your problems and increasing your productivity. Filling the survey should not take longer than 15 minutes. Its results will be presented at Erlang conferences and workshops and made public on the ProTest website (http://www.protest-project.eu). What's more, all participants are entered into a FREE PRIZE DRAW to win one of three 'Erlang Programming' books signed by the authors. So please give us 15 minutes of your time and take our survey before the 14th of December! It is available at http://surveys.erlang-consulting.com/index.php?sid=87283&lang=en in the traditional web-based form. You can also take it via IM - just add: survey@REDACTED to your google talk / Jabber buddies and enter "start" (The latter solution is powered by Erlang!!). If you have any queries, do not hesitate to contact me on aniko@REDACTED We appreciate your time and willingness to help us! Best Regards, Aniko Nagyne Vig From mryufeng@REDACTED Mon Nov 23 19:38:58 2009 From: mryufeng@REDACTED (Feng Yu) Date: Tue, 24 Nov 2009 02:38:58 +0800 Subject: OTP release tutorial examples missing Message-ID: <549b206a0911231038q458cedb6neb9b4e30152e2953@mail.gmail.com> hi, A lot of the tutorials seem to have their files missing. For example, in the OTP release tutorial: http://www.trapexit.org/OTP_Release_Handling_Tutorial All the file links lead to a page that says there is no content there. Could anybody upload the example? ?? ????????????????? http://blog.yufeng.info From hokan.stenholm@REDACTED Mon Nov 23 21:19:46 2009 From: hokan.stenholm@REDACTED (=?UTF-8?B?SMOla2FuIFN0ZW5ob2xt?=) Date: Mon, 23 Nov 2009 21:19:46 +0100 Subject: [erlang-questions] OTP release tutorial examples missing In-Reply-To: <549b206a0911231038q458cedb6neb9b4e30152e2953@mail.gmail.com> References: <549b206a0911231038q458cedb6neb9b4e30152e2953@mail.gmail.com> Message-ID: <4B0AEE62.6010908@bredband.net> Feng Yu wrote: > hi, > > A lot of the tutorials seem to have their files > missing. For example, in the OTP release tutorial: > > http://www.trapexit.org/OTP_Release_Handling_Tutorial > The link works for me (from Sweden 2009-11-23 21:18 local time), there is plenty of content on the page for me. > All the file links lead to a page that says there > is no content there. > > Could anybody upload the example? > > ?? ????????????????? > http://blog.yufeng.info > > From per@REDACTED Mon Nov 23 21:46:46 2009 From: per@REDACTED (Per Hedeland) Date: Mon, 23 Nov 2009 21:46:46 +0100 (CET) Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <38e3b9f40911230758h59f94ed0n3786966cfeb35313@mail.gmail.com> Message-ID: <200911232046.nANKkkdh067394@pluto.hedeland.org> Peter-Henry Mander wrote: > >The wrapper script detects the presence of -64 and -G flags, and >substitutes the Solaris 'ld' for any occurances of 'gcc' where these >flags exist: > >Original: /Software/Tools-64/gcc \ > -64 -G \ > -o ../priv/lib/i386-pc-solaris2.10/crypto_drv.so \ > ../priv/obj/i386-pc-solaris2.10/crypto_drv.o \ > /Software/openssl-0.9.8i/lib/libcrypto.a >Modified: /usr/ccs/bin/ld \ > -64 -G \ > -o ../priv/lib/i386-pc-solaris2.10/crypto_drv.so \ > ../priv/obj/i386-pc-solaris2.10/crypto_drv.o \ > /Software/openssl-0.9.8i/lib/libcrypto.a > >The build completes without error, but the original problem remains: >> ~/R13B03/bin/erl >Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] [rq:8] >[async-threads:0] [kernel-poll:false] Perhaps you need to take a step back here - what does file .../priv/obj/i386-pc-solaris2.10/crypto_drv.o say? From my reading of the Solaris 'ld' man page, -64 shouldn't be needed - but it might possibly "mask" the mistake of trying to build a 64-bit shared object from a 32-bit "unshared" object. --Per From peter@REDACTED Mon Nov 23 21:57:50 2009 From: peter@REDACTED (Peter Sabaini) Date: Mon, 23 Nov 2009 21:57:50 +0100 Subject: [erlang-questions] massive distribution In-Reply-To: References: Message-ID: <1259009870.3358.11.camel@gram.lan> On Fri, 2009-11-20 at 11:17 +0100, Roberto Ostinelli wrote: > dear all, > > we are using erlang for cloud computing applications, and i would like > to know if it's possible to take advantage of the processing power of, > let's say, 50,000 cores [i.e. from 6,000 to 12,000 machines, therefore > erlang nodes]. i'm interested in knowing if someone has the > information on how many interconnected nodes erlang has been proven > able to support. If you're talking about the built-in Erlang distribution mechanisms -- FWIW: in a quick test I did a while ago I had trouble keeping stable connections for more than ~80 connected nodes. I must admit I was a bit surprised as the "Efficiency Guide" seems to imply that the number should be much higher (?). Possibly the number of known (but not connected) nodes can be higher than this. HTH, peter. > sorry if this came along already somewhere, i could not find relevant info. > > thank you, > > r. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From attila.r.nohl@REDACTED Mon Nov 23 22:47:13 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 23 Nov 2009 22:47:13 +0100 Subject: [erlang-questions] OTP release tutorial examples missing In-Reply-To: <4B0AEE62.6010908@bredband.net> References: <549b206a0911231038q458cedb6neb9b4e30152e2953@mail.gmail.com> <4B0AEE62.6010908@bredband.net> Message-ID: <401d3ba30911231347l4eebac39w474acce01ba692db@mail.gmail.com> 2009/11/23, H?kan Stenholm : > Feng Yu wrote: >> hi, >> >> A lot of the tutorials seem to have their files >> missing. For example, in the OTP release tutorial: >> >> http://www.trapexit.org/OTP_Release_Handling_Tutorial >> > The link works for me (from Sweden 2009-11-23 21:18 local time), there > is plenty of content on the page for me. I believe the problematic links are the http://www.trapexit.org/Upload/release_handling_tutorial/lib/base-1.0/src/base_server.erl, http://www.trapexit.org/Upload/release_handling_tutorial/lib/base-1.0/src/base_app.erl , etc. which shows "There is currently no text in this page, you can search for this page title in other pages or edit this page." From russell.brown@REDACTED Mon Nov 23 22:47:22 2009 From: russell.brown@REDACTED (Russell Brown) Date: Mon, 23 Nov 2009 21:47:22 +0000 Subject: Cover and nested src trees Message-ID: Hi, I keep me src code in a hierarchy of directories like src - pages - elements - data - mnesia - couchdb - utils and so on. I use cover (for coverage). I use cover:compile_directory for each directory in my src tree and then call cover:analyse_to_file for each element in the result. Only the modules in the root src directory don't return {error,no_source_code_found}. Is there a way to let cover know where a modules src code is? Cover's documentation says: - 'If the module was cover compiled from the .beam file, i.e. using compile_beam/1 or compile_beam_directory/0,1, it is assumed that the source code can be found in the same directory as the .beam file, or in ../src relative to that directory. If no source code is found, , {error,no_source_code_found} is returned." But any module in question wasn't. It was cover compiled with cover:compile_directory("/my/nested/src/dir/") and yet the same return value results. Many thanks in advance if you can help me figure out how to get analyse_to_file for modules belwo my src root directory. Cheers Russell From ok@REDACTED Mon Nov 23 23:00:03 2009 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 24 Nov 2009 11:00:03 +1300 Subject: Idle question Message-ID: Suppose Erlang were available on BlueFern (the BlueGene machine at the University of Canterbury, see http://www.bluefern.canterbury.ac.nz/ ). What would you use it for? What science or technology would it advance? From aniko@REDACTED Mon Nov 23 23:08:46 2009 From: aniko@REDACTED (Aniko Nagyne Vig) Date: Mon, 23 Nov 2009 22:08:46 +0000 (GMT) Subject: [erlang-questions] OTP release tutorial examples missing In-Reply-To: <3959234.55981259013952832.JavaMail.root@zimbra> Message-ID: <21675032.56001259014126537.JavaMail.root@zimbra> Hi, Yes, you are right. The links for the files are broken. We will look at the issue and keep you updated. Best Regards, Aniko Nagyne Vig Web: ----- Original Message ----- From: "Attila Rajmund Nohl" To: "Erlang Users' List" Sent: Monday, 23 November, 2009 21:47:13 GMT +00:00 GMT Britain, Ireland, Portugal Subject: Re: [erlang-questions] OTP release tutorial examples missing 2009/11/23, H?kan Stenholm : > Feng Yu wrote: >> hi, >> >> A lot of the tutorials seem to have their files >> missing. For example, in the OTP release tutorial: >> >> http://www.trapexit.org/OTP_Release_Handling_Tutorial >> > The link works for me (from Sweden 2009-11-23 21:18 local time), there > is plenty of content on the page for me. I believe the problematic links are the http://www.trapexit.org/Upload/release_handling_tutorial/lib/base-1.0/src/base_server.erl, http://www.trapexit.org/Upload/release_handling_tutorial/lib/base-1.0/src/base_app.erl , etc. which shows "There is currently no text in this page, you can search for this page title in other pages or edit this page." ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From vik@REDACTED Mon Nov 23 23:13:40 2009 From: vik@REDACTED (Vik Olliver) Date: Tue, 24 Nov 2009 11:13:40 +1300 Subject: [erlang-questions] Idle question In-Reply-To: References: Message-ID: <4B0B0914.9080008@catalyst.net.nz> On 24/11/09 Richard O'Keefe wrote: > Suppose Erlang were available on BlueFern > (the BlueGene machine at the University of Canterbury, > see http://www.bluefern.canterbury.ac.nz/ > ). What would you use it for? What science or > technology would it advance? Protein & DNA packing - I guess that's what is was for, right? But I'd aim for creating nanomachine components rather than purely biological structures. Once they're designed, they'll need simulating too. Nanoscale RepRap anyone? Vik :v) From hokan.stenholm@REDACTED Mon Nov 23 23:14:09 2009 From: hokan.stenholm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Stenholm?=) Date: Mon, 23 Nov 2009 23:14:09 +0100 Subject: [erlang-questions] OTP release tutorial examples missing In-Reply-To: <401d3ba30911231347l4eebac39w474acce01ba692db@mail.gmail.com> References: <549b206a0911231038q458cedb6neb9b4e30152e2953@mail.gmail.com> <4B0AEE62.6010908@bredband.net> <401d3ba30911231347l4eebac39w474acce01ba692db@mail.gmail.com> Message-ID: <4B0B0931.5060601@bredband.net> Attila Rajmund Nohl wrote: > 2009/11/23, H?kan Stenholm : > >> Feng Yu wrote: >> >>> hi, >>> >>> A lot of the tutorials seem to have their files >>> missing. For example, in the OTP release tutorial: >>> >>> http://www.trapexit.org/OTP_Release_Handling_Tutorial >>> >>> >> The link works for me (from Sweden 2009-11-23 21:18 local time), there >> is plenty of content on the page for me. >> > > I believe the problematic links are the > http://www.trapexit.org/Upload/release_handling_tutorial/lib/base-1.0/src/base_server.erl, > http://www.trapexit.org/Upload/release_handling_tutorial/lib/base-1.0/src/base_app.erl > , etc. which shows "There is currently no text in this page, you can > search for this page title in other pages or edit this page." > Your right, the links to the various .erl, .rel, .script, .app ... files all appear to be broken, I thought it was the actual tutorial pages that where missing. > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ok@REDACTED Tue Nov 24 03:48:31 2009 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 24 Nov 2009 15:48:31 +1300 Subject: [erlang-questions] Idle question In-Reply-To: <4B0B0914.9080008@catalyst.net.nz> References: <4B0B0914.9080008@catalyst.net.nz> Message-ID: On Nov 24, 2009, at 11:13 AM, Vik Olliver wrote: > On 24/11/09 Richard O'Keefe wrote: >> Suppose Erlang were available on BlueFern >> (the BlueGene machine at the University of Canterbury, >> see http://www.bluefern.canterbury.ac.nz/ >> ). What would you use it for? What science or >> technology would it advance? > > Protein & DNA packing - I guess that's what is was for, right? But I'd > aim for creating nanomachine components rather than purely biological > structures. The page http://www.bluefern.canterbury.ac.nz/projects/index.shtml shows what BlueFern is currently being used for. I note two nanotechnology projects there already. (But no protein or DNA packing.) What would make >>> Erlang <<< a good language for Protein and DNA packing or computational nanotechnology or birthweight prediction &c rather than Fortran or C++ with ESSL and MPI or Stata or even R? It struck me that here's a large machine on the same island as me, 160 cores for the p575 cluster, 4096 cores for the BlueGene, and to be perfectly frank, most of the projects look like things where Fortran 95 would be the programming language of choice, or possibly Ada. There's one project where the main need wasn't so much the number of cores as the amount of main memory. One should not expect Erlang to do _everything_, but this was making me reflect on the difference between highly *concurrent* computing and high *performance* computing. From vik@REDACTED Tue Nov 24 03:54:57 2009 From: vik@REDACTED (Vik Olliver) Date: Tue, 24 Nov 2009 15:54:57 +1300 Subject: [erlang-questions] Idle question In-Reply-To: References: <4B0B0914.9080008@catalyst.net.nz> Message-ID: <4B0B4B01.9080703@catalyst.net.nz> Richard O'Keefe wrote: > What would make >>> Erlang <<< a good language for Protein and DNA > packing or computational nanotechnology or birthweight prediction > &c rather than Fortran or C++ with ESSL and MPI or Stata or even R? I've not done it you understand, but it seems to me like Erlang would be good for calculating the component part interactions for large molecules. Vik :v) From mjtruog@REDACTED Tue Nov 24 05:43:37 2009 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 23 Nov 2009 20:43:37 -0800 Subject: [erlang-questions] Idle question In-Reply-To: References: Message-ID: <4B0B6479.30606@gmail.com> I dunno, but you could use Cloudi to do it :-) http://cloudi.org/ Richard O'Keefe wrote: > Suppose Erlang were available on BlueFern > (the BlueGene machine at the University of Canterbury, > see http://www.bluefern.canterbury.ac.nz/ > ). What would you use it for? What science or > technology would it advance? > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From leap@REDACTED Tue Nov 24 08:25:01 2009 From: leap@REDACTED (Michael Turner) Date: Tue, 24 Nov 2009 07:25:01 +0000 Subject: Erlang "object-oriented" after all? Message-ID: I ran across a very interesting exchange with Alan Kay, who most likely coined the term "object-oriented programming." http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/doc_kay_oop_en If you'll excuse some (only slightly tendentious) editing of his remarks: ---- "I thought of objects being like biological cells and/or individual computers on a network, only able to communicate with messages . . . . I wanted to get rid of data. . . ." "I realized that the cell/whole-computer metaphor would get rid of data, and that "<-" [assignment operator in smalltalk] would be just another message token . . . ." "I didn't like the way Simula I or Simula 67 did inheritance . . . . So I decided to leave out inheritance as a built-in feature until I understood it better . . . . "My original experiments with this architecture were done using a model I adapted from . . . rather LISP-like [languages] but with a more conventional readable syntax. . . ." "Another big influence at this time was Carl Hewitt's PLANNER (which has never gotten the recognition it deserves, given how well and how earlier it was able to anticipate Prolog) . . . ." "I'm not against types, but I don't know of any type systems that aren't a complete pain, so I still like dynamic typing.) . . . ." "The people who liked objects as non-data were smaller in number, and . . . pretty much all of this group were . . . . were involved in one way or another with the design of ARPAnet->Internet in which the basic unit of computation was a whole computer. But just to show how stubbornly an idea can hang on, all through the seventies and eighties, there were many people who tried to get by with "Remote Procedure Call" instead of thinking about objects and messages. . . ." "OOP to me means only messaging, local retention and protection and hiding of state-process, and extreme late-binding of all things . . . ." ---- So perhaps it would be forgiveable to describe Erlang as not just concurrent, functional, fault-tolerant, soft-real-time, and strongly influenced by both academic logic programming and real-world networking problems, but also as "object-oriented in the original (and perhaps best) sense of the term." At least on the theory that any major buzzword pile-up should be salvaged with a rhetorically graceful crescendo. Besides, what's that old saying again? "Those who don't learn from history are condemned to be repeatedly over-awed by those who can fake it"? ;-) -michael turner From mmzeeman@REDACTED Tue Nov 24 09:47:05 2009 From: mmzeeman@REDACTED (Maas-Maarten Zeeman) Date: Tue, 24 Nov 2009 09:47:05 +0100 Subject: Possible hostname handling bug in node handshake procedure? Message-ID: Today I stumbled upon some strange behavior when using net_adm:ping. I was trying to connect two machines, buka and Edo on the same lan, but it just would get them to talk to each other. Net_adm:ping worked in one way only. Normal ping worked both ways, double checked if there where firewall rules. I even looked at the network packets, which I could see arriving at both ends. Then it slowly dawned on me. One of the hosts has a hostname with a capital in it. so I tried to the exact name. It worked... Here is what happens. (a@REDACTED)1> net_adm:ping(b@REDACTED). **long wait** pang (a@REDACTED)2> net_adm:ping(b@REDACTED). pong In both situations the network packets arrive at b@REDACTED To me it looks like there is a case sensitivity problem with the hostname handling in the handshake procedure. (a@REDACTED)51> net_adm:ping(a@REDACTED). pong (a@REDACTED)52> net_adm:ping(a@REDACTED). pang Shouldn't both calls return pong as hostnames should normally be treated as case insensitive strings? Kind regards, Maas-Maarten Zeeman From erlang@REDACTED Tue Nov 24 10:06:28 2009 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 24 Nov 2009 10:06:28 +0100 Subject: [erlang-questions] Erlang "object-oriented" after all? In-Reply-To: References: Message-ID: <9b08084c0911240106w4d946ffaod6fecaca4576983c@mail.gmail.com> Yes. It's difficult to say if Erlang is OO - since there is no widely accepted definition of what OO is or means. When we wrote the first Erlang book we looked up many definitions of OO but there was a wide variation in what different languages regarded as comprising OO'ness. I now believe the following to be central to the notion of OO. - Isolated concurrent things - Communication through message passing - Polymorphism All the other stuff (inheritance, private/public methods, ....) has nothing to do with OO. isolation is essential - if my set of objects can be damaged by your set of objects then we just can't build reliable systems. Isolation implies pure (copying) message passing and concurrency. Things must be concurrent - otherwise how could we make a "clock" object. Polymorphism is essential in the messaging interface to abstracting out the internal details of an object. Pid ! printYouself cold tell any objects to print itself. All I/O objects should respond to read/write/... messages etc. By these criteria there are actually no OO languages (nobody can do *complete* isolation) though Erlang is nearer to being OO than many other languages. And most so-called OO languages are not OO. One problem with saying "everything is an object" is that it doesn't allow us to talk about things which are not objects. So ask questions like "is time an object?" is "code replacement an object?" is "is scalability an object?" you won't get a sensible reply. The erlang answers are easier: "Q: What is code replacement?" "A: A protocol - Pid ! {newCode, F} tells the process Pid to start executing new code F" and so on.... For convenience things like code replacement are built into the OTP behaviours - which basically just enforce several low-level protocols. /joe On Tue, Nov 24, 2009 at 8:25 AM, Michael Turner wrote: > > I ran across a very interesting exchange with Alan Kay, who most likely > coined the term "object-oriented programming." > > ?http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/doc_kay_oop_en > > If you'll excuse some (only slightly tendentious) editing of his remarks: > > ---- > "I thought of objects being like biological cells and/or individual > computers on a network, only able to communicate with messages . . . . I > wanted to get rid of data. . . ." > > "I realized that the cell/whole-computer metaphor would get rid of data, > and that "<-" [assignment operator in smalltalk] would be just another > message token . . . ." > > "I didn't like the way Simula I or Simula 67 did inheritance . . . . So > I decided to leave out inheritance as a built-in feature until I > understood it better . . . . > > "My original experiments with this architecture were done using a model > I adapted from . . . rather LISP-like [languages] but with a more > conventional readable syntax. ?. . ." > > "Another big influence at this time was Carl Hewitt's PLANNER (which > has never gotten the recognition it deserves, given how well and how > earlier it was able to anticipate Prolog) . . . ." > > "I'm not against types, but I don't know of any type systems that > aren't a complete pain, so I still like dynamic typing.) . . . ." > > "The people who liked objects as non-data were smaller in number, and . > ?. . ?pretty much all of this group were . . . . were involved in one > way or another with the design of ARPAnet->Internet in which the basic > unit of computation was a whole computer. But just to show how > stubbornly an idea can hang on, all through the seventies and eighties, > there were many people who tried to get by with "Remote Procedure > Call" instead of thinking about objects and messages. . . ." > > "OOP to me means only messaging, local retention and protection and > hiding of state-process, and extreme late-binding of all things . . . ." > ---- > > So perhaps it would be forgiveable to describe Erlang as not just > concurrent, functional, fault-tolerant, soft-real-time, and strongly > influenced by both academic logic programming and real-world networking > problems, but also as "object-oriented in the original (and perhaps > best) sense of the term." ?At least on the theory that any major > buzzword pile-up should be salvaged with a rhetorically graceful > crescendo. > > Besides, what's that old saying again? ?"Those who don't learn from > history are condemned to be repeatedly over-awed by those who can fake > it"? > > ;-) > > -michael turner > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From erlang@REDACTED Tue Nov 24 10:14:05 2009 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 24 Nov 2009 10:14:05 +0100 Subject: [erlang-questions] Idle question In-Reply-To: References: Message-ID: <9b08084c0911240114ofcc9a33l9ab7b226fcc6e42e@mail.gmail.com> On Mon, Nov 23, 2009 at 11:00 PM, Richard O'Keefe wrote: > Suppose Erlang were available on BlueFern > (the BlueGene machine at the University of Canterbury, > see http://www.bluefern.canterbury.ac.nz/ > ). ?What would you use it for? ?What science or > technology would it advance? I'd write a general purpose management layer (middleware) for distributing computations over the cluster. If you want to do heavy numerical computations, you'll do them in an imperative language. I'd use Erlang as the glue for glueing togther components and for monitoring the system as a whole. I'd make an infrastructure that is language neutral, but based on communicating components and let Erlang manage the components - start them and stop them, move them, give them resources etc. I would allow the components to compete and evolve so the best component for the job could be found. I'd use something like UBF to constrain the messaging between components. /Joe > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From vladdu55@REDACTED Tue Nov 24 10:25:37 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 24 Nov 2009 10:25:37 +0100 Subject: [erlang-questions] Erlang "object-oriented" after all? In-Reply-To: <9b08084c0911240106w4d946ffaod6fecaca4576983c@mail.gmail.com> References: <9b08084c0911240106w4d946ffaod6fecaca4576983c@mail.gmail.com> Message-ID: <95be1d3b0911240125h1916ab40s8c9cc9b8acc47463@mail.gmail.com> On Tue, Nov 24, 2009 at 10:06, Joe Armstrong wrote: > It's difficult to ?say if Erlang is OO - since there is no widely > accepted definition of what OO is or means. > > I now believe the following to be central to the notion of OO. > > ? ? ?- Isolated concurrent things > ? ? ?- Communication through message passing > ? ? ?- Polymorphism > > All the other stuff (inheritance, private/public methods, ....) has > nothing to do with OO. Hi, True. I feel though that there is something missing before this OO-ness really becomes useful. And that is a way to specify the contract that one expects a certain object to fulfill. An interface or a protocol has to be possible to be referred to by name. Today that name would be the name of a module, but it feels like module names get a too overloaded meaning. I don't have an alternative suggestion yet, though. best regards, Vlad From bengt.kleberg@REDACTED Tue Nov 24 10:26:46 2009 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 24 Nov 2009 10:26:46 +0100 Subject: [erlang-questions] Possible hostname handling bug in node handshake procedure? In-Reply-To: References: Message-ID: <1259054807.4920.12.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, What are your ideas about both parts of the node name? Should they be case insensitive or just the host name part? Consider the node aA@REDACTED It might be confusing to have net_adm:ping(aA@REDACTED) => pong but then get net_adm:ping(aa@REDACTED) => pang So we probably should have case insensitivity here. But then we have to explain why aA =:= aa is false in all the rest of the code. Perhaps this is a problem only for me. bengt On Tue, 2009-11-24 at 09:47 +0100, Maas-Maarten Zeeman wrote: > Today I stumbled upon some strange behavior when using net_adm:ping. > I was trying to connect two machines, buka and Edo on the same lan, > but it just would get them to talk to each other. Net_adm:ping worked > in one way only. Normal ping worked both ways, double checked if there > where firewall rules. I even looked at the network packets, which I > could see arriving at both ends. Then it slowly dawned on me. One of > the hosts has a hostname with a capital in it. so I tried to the exact > name. It worked... > > Here is what happens. > > (a@REDACTED)1> net_adm:ping(b@REDACTED). > **long wait** > pang > (a@REDACTED)2> net_adm:ping(b@REDACTED). > pong > > In both situations the network packets arrive at b@REDACTED To me it looks > like there is a case sensitivity problem with the hostname handling in > the handshake procedure. > > (a@REDACTED)51> net_adm:ping(a@REDACTED). > pong > (a@REDACTED)52> net_adm:ping(a@REDACTED). > pang > > Shouldn't both calls return pong as hostnames should normally be > treated as case insensitive strings? > > Kind regards, > > Maas-Maarten Zeeman > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From tamas.nagy@REDACTED Tue Nov 24 10:59:07 2009 From: tamas.nagy@REDACTED (Tamas Nagy) Date: Tue, 24 Nov 2009 09:59:07 +0000 (GMT) Subject: [erlang-questions] Compile error on Mac OS X 10.6 In-Reply-To: Message-ID: <17612280.56431259056747721.JavaMail.root@zimbra> Hi Jarrod, If you do not mind the bleeding edge I would higly recommend you give homebrew a try (http://github.com/mxcl/homebrew). It does not try to build everything and the kitchen sink when building erlang (like macports does). Regards, Tamas ----- "Jarrod Roberson" wrote: > On Sun, Nov 22, 2009 at 8:06 PM, Herman Chan > wrote: > > > yes, I have XCode 3.1 installed. In fact, the error message seems > to > > suggested that it can't link with crt1.10.5.o file, which is under > a > > folder named /Developer/SDKs/MacOSX10.5.sdk/usr/lib. Is my > terminal > > wired up to look into /Develoepr/SDKs/MacOSX1.5.sdk/usr/lib for > > library? > > > > Herman > > > > if you are compling for 10.6 you should have Xcode 3.2.1 which is the > latest. > You can get the update from Apple Developers Connection. -- Tamas Nagy Erlang Training & Consulting http://www.erlang-consulting.com From hvjunk@REDACTED Tue Nov 24 11:25:37 2009 From: hvjunk@REDACTED (Hendrik Visage) Date: Tue, 24 Nov 2009 12:25:37 +0200 Subject: [erlang-questions] Re: list size 'causing VM "problems" In-Reply-To: <9a06d836-736f-4468-96ed-e7cf0d41547c@p19g2000vbq.googlegroups.com> References: <9a06d836-736f-4468-96ed-e7cf0d41547c@p19g2000vbq.googlegroups.com> Message-ID: On 11/23/09, Steve Davis wrote: > Sounds very likely you are 64 bit which explains the 2 Gig, MacOSX 64bit erlang yes... perhaps I need to recompile to 32bit while testing/playing with this one... > but I > think that... > > {ok,Line} -> > [process_line(Line)|read_lines(IOfd)]; > > ...would maybe not be "tail recursive" so the entire stack is being > run hence chewing up 7G or more? What happens if you change the > read_lines function to use an accumulator for the result? e.g... Tries this, same trouble type of troubles in the erl shell. The symptoms: the: List=read_lines(FD). executes, doesn't *appear* to be using lots of VM space, and it outputs the partial (being the shell it only prints a part of the lines with the [...]...] stuff. Then it *hangs* at that, not showing the prompt (this is inside Aquamacs's erlang shell mode). This is then where it appears the system goes "west", as the memory utilization (as measured/shown by the MacOSX Activity Monitor) starts to grow and grow. with the last test (using the Accumulator as below) it grew to 7G at which point I killed the beam.smp process. > read_lines(IOfd) -> > read_lines(IOfd, []). > > read_lines(IOfd, Acc) -> > case file:read_line(IOfd) of > {ok, Line} -> > read_lines(IOfd, [process_line(Line) | Acc]); > eof -> > lists:reverse(Acc) > end. > > > > On Nov 23, 4:09 am, Hendrik Visage wrote: >> Hi there, >> >> Yes, I know this code is not yet optimal (I'm still learning :), but >> it begs a few questions I'd like to understand from the VM etc. >> >> 1) I've run it fine with a small subset, but once I've loaded the 930k >> lines file, the VM sucks up a lot of RAM/Virtualmemory. Like a burst >> of about 2G (I have a 4G MacBookPro) and then once it returned in the >> erl shell, the VM starts to go balistic and consumes >7G of >> virtualmemory ;( >> Q1: why did the VM exhibit this behaviour? the garbage collector going >> bad/mad?? >> >> 2) I will push the data into an ETS of sorts, as I'll try to find >> duplicate files, but were thinking of an initial pull into a list, en >> then fron there do the tests etc. The idea might be to pull in one >> disk, and then compare it to another removal disk's files. >> Q2: Should I rather do this straight into an ETS/DETS? >> Q3: Should I preferably start to consider DETS 'cause of the size?? >> Q4: will Mnesia help in this case? >> >> %%-------------------------------------------------------------------- >> %% Function: process_line/1 >> %% Description: take a properly formated line, and parse it, and >> %% returns the tuple {Type,File,Hash} >> %% Line: "MD5 (/.file) = d41d8cd98f00b204e9800998ecf8427e" >> %% Nore some might be SHA1 in future. >> %%-------------------------------------------------------------------- >> process_line(Line) -> >> {match,[Type,File,Hash]}= >> re:run(Line, >> "\(.*\)[ ][\\(]\(.*\)[\\)][ ][=][ ]\([0-9a-f]*\)\n", >> [{capture,all_but_first,list}]), >> {Type,File,Hash}. >> >> %%-------------------------------------------------------------------- >> %% Function: read_lines/1 >> %% Description: read in all the lines from a "properly formatted" >> %% md5 output on MacOSX, returning a list with the tupples. >> %%-------------------------------------------------------------------- >> >> read_lines(IOfd) -> >> case file:read_line(IOfd) of >> {ok,Line} -> >> [process_line(Line)|read_lines(IOfd)]; >> eof -> >> [] >> end. >> >> ________________________________________________________________ >> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html >> erlang-questions (at) erlang.org > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From olivier.boudeville@REDACTED Tue Nov 24 11:08:05 2009 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Tue, 24 Nov 2009 11:08:05 +0100 Subject: RE [erlang-questions] Idle question In-Reply-To: Message-ID: Hi, We are currently making use of Erlang for some kind of distributed discrete time-stepped simulations, and we were actually considering the port of an Erlang VM on a Bluegene/P for 2010. We are less in need for floating-point number-crunching abilities than for a massive parallel execution that could take advantage of the vast amount of total RAM that the Bluegene provides: for the large-scale simulations we aim, with the Bluegene we might get an edge over classical clusters (if still able to rely on a low-latency network). I believe that Erlang is very relevant to develop (somewhat complex) distributed algorithms, involving message reordering and massive concurrency. This is true for time-stepped algorithms, and I guess still truer for event-driven conservative or optimistic simulations, where deadlock detection/avoidance and distributed rollback mechanisms would be surely easier to develop in Erlang than, say, C or C++. So there would be definitively an interest in such a port, and the outlook of any joint effort/shared needs could help me promote the porting effort internally. On the technical side, the port of the Erlang VM to a Bluegene node could be less complex than expected at first glance: unless I am mistaken, the PowerPC architecture is supported by Erlang, a minimal GNU/Linux environment should be available on each node, and the lack of multithread support should not be a problem (thanks to green threads). The major issue would probably be the network, knowing that by default not TCP/IP stack is available on the Bluegene, as I understand. Options could be to switch to a Bluegene-native carrier, or to make use of MPI for internode-communications, or maybe to rely on any experimental TCP/IP support that may already exist. Finally, according to http://www.ece.iastate.edu/~crb002/cnr.html Ruby was already ported successfully to the Bluegene; is there anything that Ruby can do and that Erlang cannot? Best regards, Olivier Boudeville. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 ok@REDACTED Envoy? par : erlang-questions@REDACTED 23/11/2009 23:02 A erlang-questions@REDACTED cc Objet [erlang-questions] Idle question Suppose Erlang were available on BlueFern (the BlueGene machine at the University of Canterbury, see http://www.bluefern.canterbury.ac.nz/ ). What would you use it for? What science or technology would it advance? ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. From phmander@REDACTED Tue Nov 24 12:03:42 2009 From: phmander@REDACTED (Peter-Henry Mander) Date: Tue, 24 Nov 2009 11:03:42 +0000 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <200911232046.nANKkkdh067394@pluto.hedeland.org> References: <38e3b9f40911230758h59f94ed0n3786966cfeb35313@mail.gmail.com> <200911232046.nANKkkdh067394@pluto.hedeland.org> Message-ID: <38e3b9f40911240303n2db44650hcb0b00887633d185@mail.gmail.com> $> /bin/file ./lib/crypto/priv/obj/i386-pc-solaris2.10/crypto_drv.o ./lib/crypto/priv/obj/i386-pc-solaris2.10/crypto_drv.o: ELF 64-bit LSB relocatable AMD64 Version 1 The gcc.wrapper script is a belts-and-braces device to prevent any 32 bit stuff from seeping through. It may be excessive, but it is a guarantee. I'm going to experiment with a different machine, a Core2Quad with 4Gb of RAM. The 4Gb falls within the 32 bit address range, not the 36 bits required by the 64Gb we have installed in the original machine. I have a nagging suspicion that the I'm observing a Solaris10 loader bug (suggested by Wallentin Dahlberg), that will only be triggered when more than 4Gb RAM is available. I'm betting that the Intel EM64T is equivalent to AMD64, and that it behaves exactly the same. To prove the point, I shall try to plug an extra 4Gb RAM (or more, if I can afford it) to see if the Core2Quad changes behaviour when more than 32 bits addressing range is available. Pete. On Mon, Nov 23, 2009 at 8:46 PM, Per Hedeland wrote: > Perhaps you need to take a step back here - what does > > ?file .../priv/obj/i386-pc-solaris2.10/crypto_drv.o > > say? From my reading of the Solaris 'ld' man page, -64 shouldn't be > needed - but it might possibly "mask" the mistake of trying to build a > 64-bit shared object from a 32-bit "unshared" object. > > --Per > From igorrs@REDACTED Tue Nov 24 15:23:33 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Tue, 24 Nov 2009 12:23:33 -0200 Subject: The 2 GB limit Message-ID: Hi. Is there any plan (or work in progress) for removing the 2 GB size limit of dets files? And, while this is not accomplished, do you think it could be easier to try to raise that limit to 4 GB? Assuming the issue is related to 32-bit addressing, I'm guessing (just guessing) that dealing with 4 GB files should not be difficult. Am I wrong? Why? Thanks. Igor. -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From masklinn@REDACTED Tue Nov 24 15:40:02 2009 From: masklinn@REDACTED (Masklinn) Date: Tue, 24 Nov 2009 15:40:02 +0100 Subject: [erlang-questions] The 2 GB limit In-Reply-To: References: Message-ID: <55832A75-D8F4-4300-9F9B-1FD829D71174@masklinn.net> On 24 Nov 2009, at 15:23 , Igor Ribeiro Sucupira wrote: > Is there any plan (or work in progress) for removing the 2 GB size > limit of dets files? > > And, while this is not accomplished, do you think it could be easier > to try to raise that limit to 4 GB? Assuming the issue is related to > 32-bit addressing, I'm guessing (just guessing) that dealing with 4 GB > files should not be difficult. Am I wrong? Why? Well if the dets files are merely mmapped when read, given Windows puts the kernel data in the application's address space by default the available memory really is only 2GB (can go to ~3GB using the right boot switch, which may or may not make the system less stable). Even under OSX I'm not sure the memory available to the application process reaches the full 4GB. From v@REDACTED Tue Nov 24 15:47:40 2009 From: v@REDACTED (Valentin Micic) Date: Tue, 24 Nov 2009 16:47:40 +0200 Subject: [erlang-questions] The 2 GB limit In-Reply-To: Message-ID: <20091124144747.570833D0D4A@mail.pharos-avantgard.com> I must express my skepticism regarding dets performance when a file reaches such a level of "data density" (assuming that 2 GB translates to at least a few million records). As much as I believe that reading from such a big file may be relatively fast, I think that updating the data would be disproportionately slower. In my experience, the only way to maintain a good performance with sizable data sets (say, 50-150 million records) in dets is to distribute data over a number of dets files (similar to what mnesia fragmentation does). So, if my skepticism is justifiable, this is not a question of how difficult it is to increase a dets file size limit, but would there be any point in doing it? V/ -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Igor Ribeiro Sucupira Sent: 24 November 2009 04:24 PM To: erlang-questions Subject: [erlang-questions] The 2 GB limit Hi. Is there any plan (or work in progress) for removing the 2 GB size limit of dets files? And, while this is not accomplished, do you think it could be easier to try to raise that limit to 4 GB? Assuming the issue is related to 32-bit addressing, I'm guessing (just guessing) that dealing with 4 GB files should not be difficult. Am I wrong? Why? Thanks. Igor. -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From thijsterlouw@REDACTED Tue Nov 24 16:00:09 2009 From: thijsterlouw@REDACTED (Thijs) Date: Tue, 24 Nov 2009 07:00:09 -0800 (PST) Subject: Fastest pseudo-random number-generator: erlang:statistics(io) ? Message-ID: Today I needed a faster alternative for erlang:now(). It turns out that using erlang:now() is quite slow: it requires a system call. I decided to test some alternatives. My requirements are: speed, speed, speed :) I will use it in a server that is quite busy, so I figured I could use various system metrics to generate random numbers. I use these pseudo-random numbers to do load-balancing, log requests at random etc. benchmark results : erlang:now() 288.090 erlang:statistics(io) 2.730.853 erlang:statistics(wall_clock) 280.396 erlang:statistics(runtime) 510.765 erlang:statistics(reductions) 471.688 erlang:statistics(context_switches) 493.851 random:uniform() 870.261 This test is spawning 100 concurrent processes, each executing 10000 calls to each function. Two notes: 1. Note that you can only use this function if your server is actually performing (lots) of IO 2. random:uniform() needs to be seeded for real use (each process starts the same), I didn't do that. Based on these tests, erlang:statistics(io) is the clear winner. I will use it now for picking random elements from lists and replacing pg2:get_closest_pid(Name). Does anyone have any faster suggestions than erlang:statistics(io) ? From ulf.wiger@REDACTED Tue Nov 24 16:06:29 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 24 Nov 2009 16:06:29 +0100 Subject: [erlang-questions] The 2 GB limit In-Reply-To: References: Message-ID: <4B0BF675.2050708@erlang-consulting.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Igor Ribeiro Sucupira wrote: > Hi. > > Is there any plan (or work in progress) for removing the 2 GB size > limit of dets files? > > And, while this is not accomplished, do you think it could be easier > to try to raise that limit to 4 GB? Assuming the issue is related to > 32-bit addressing, I'm guessing (just guessing) that dealing with 4 GB > files should not be difficult. Am I wrong? Why? Rather than re-writing dets, I would go with something like Tokyo Tyrant, build on a suitable erlang wrapper (like http://github.com/mallipeddi/tora), and perhaps put something like my sext library on top (http://svn.ulf.wiger.net/sext/trunk/sext/examples/tt_proto.erl). To match dets, I'd like to see some administrative code around the creation and opening of Tyrant tables, including the ability to "repair" the data from the transaction log. Seems doable, though. http://1978th.net/tokyotyrant/ BR, Ulf W -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAksL9nUACgkQtqqFieqzed1v7wCfUf9bgDU+JfD8Gey51lvR7fxG 3iIAn0J5+8NpHH7DkLTgRlRuUKWomkXO =C66x -----END PGP SIGNATURE----- From igorrs@REDACTED Tue Nov 24 16:01:34 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Tue, 24 Nov 2009 13:01:34 -0200 Subject: [erlang-questions] The 2 GB limit In-Reply-To: <20091124144747.570833D0D4A@mail.pharos-avantgard.com> References: <20091124144747.570833D0D4A@mail.pharos-avantgard.com> Message-ID: On Tue, Nov 24, 2009 at 12:47 PM, Valentin Micic wrote: > I must express my skepticism regarding dets performance when a file reaches > such a level of "data density" Hello. Since I don't know very much about the inner workings of dets files, maybe I am confused by the terminology: wouldn't data density be inversely proportional to file size and directly proportional to data size? Thank you. Igor. > (assuming that 2 GB translates to at least a > few million records). As much as I believe that reading from such a big file > may be relatively fast, I think that updating the data would be > disproportionately slower. In my experience, the only way to maintain a good > performance with sizable data sets (say, 50-150 million records) in dets is > to distribute data over a number of dets files (similar to what mnesia > fragmentation does). > So, if my skepticism is justifiable, this is not a question of how difficult > it is to increase a dets file size limit, but would there be any point in > doing it? > > V/ > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Igor Ribeiro Sucupira > Sent: 24 November 2009 04:24 PM > To: erlang-questions > Subject: [erlang-questions] The 2 GB limit > > Hi. > > Is there any plan (or work in progress) for removing the 2 GB size > limit of dets files? > > And, while this is not accomplished, do you think it could be easier > to try to raise that limit to 4 GB? Assuming the issue is related to > 32-bit addressing, I'm guessing (just guessing) that dealing with 4 GB > files should not be difficult. Am I wrong? Why? > > Thanks. > Igor. > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From dizzyd@REDACTED Tue Nov 24 16:20:32 2009 From: dizzyd@REDACTED (Dave Smith) Date: Tue, 24 Nov 2009 08:20:32 -0700 Subject: [erlang-questions] Fastest pseudo-random number-generator: erlang:statistics(io) ? In-Reply-To: References: Message-ID: On Tue, Nov 24, 2009 at 8:00 AM, Thijs wrote: > benchmark results : > > erlang:now() ? ?288.090 > erlang:statistics(io) ? 2.730.853 > erlang:statistics(wall_clock) ? 280.396 > erlang:statistics(runtime) ? ? ?510.765 > erlang:statistics(reductions) ? 471.688 > erlang:statistics(context_switches) ? ? 493.851 > random:uniform() ? ? ? ?870.261 Off the top of my head, I have to wonder if using statistics(io) is going to yield sufficient pseudo-randomness for load distribution purposes. If you consider that most IO is likely to be a block size modulo 8 (for efficiency) it seems like you might get a skewed distribution. A simpler (and perhaps faster) approach would be to just maintain a counter that gets atomically incremented on each call -- a port driver or ETS table would suffice and have similar amounts of contention as the call to statistics. My $0.02...and you know how weak the dollar has been these days. :) D. From thijsterlouw@REDACTED Tue Nov 24 16:35:10 2009 From: thijsterlouw@REDACTED (Thijs) Date: Tue, 24 Nov 2009 07:35:10 -0800 (PST) Subject: Fastest pseudo-random number-generator: erlang:statistics(io) ? In-Reply-To: References: Message-ID: On Nov 24, 11:20?pm, Dave Smith wrote: > Off the top of my head, I have to wonder if using statistics(io) is > going to yield sufficient pseudo-randomness for load distribution > purposes. If you consider that most IO is likely to be a block size > modulo 8 (for efficiency) it seems like you might get a skewed > distribution. A simpler (and perhaps faster) approach would be to just > maintain a counter that gets atomically incremented on each call -- a > port driver or ETS table would suffice and have similar amounts of > contention as the call to statistics. Maintaining a counter in ETS would be (much) slower, so that is unfortunately no option for me. I have tried some simple randomness tests for this io-function in the running server and the distribution looks sufficiently random. I have many processes sending and receiving data (via ports), so that shouldn't be a problem. Someone else told me that erlang:now() blocks the scheduler shortly, so that is something I definitely want to avoid. From gleber.p@REDACTED Tue Nov 24 16:39:36 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Tue, 24 Nov 2009 16:39:36 +0100 Subject: [erlang-questions] Re: Fastest pseudo-random number-generator: erlang:statistics(io) ? In-Reply-To: References: Message-ID: <14f0e3620911240739n576e938cx1cecdbf919102924@mail.gmail.com> > Someone else told me > that erlang:now() blocks the scheduler shortly, so that is something I > definitely want to avoid. Yes, since erlang:now() is guaranteed to return ascending numbers on each call regardless of number of cores VM is running one. It has to do some synchronization to meet this requirement. From kiszl@REDACTED Tue Nov 24 17:10:19 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Tue, 24 Nov 2009 17:10:19 +0100 (CET) Subject: [erlang-questions] Re: Fastest pseudo-random number-generator: erlang:statistics(io) ? In-Reply-To: References: Message-ID: <6791.194.88.55.211.1259079019.squirrel@localhost> > On Nov 24, 11:20?pm, Dave Smith wrote: >> Off the top of my head, I have to wonder if using statistics(io) is >> going to yield sufficient pseudo-randomness for load distribution >> purposes. If you consider that most IO is likely to be a block size >> modulo 8 (for efficiency) it seems like you might get a skewed >> distribution. A simpler (and perhaps faster) approach would be to just >> maintain a counter that gets atomically incremented on each call -- a >> port driver or ETS table would suffice and have similar amounts of >> contention as the call to statistics. > > Maintaining a counter in ETS would be (much) slower, so that is > unfortunately no option for me. I have tried some simple randomness > tests for this io-function in the running server and the distribution > looks sufficiently random. I have many processes sending and receiving > data (via ports), so that shouldn't be a problem. Someone else told me > that erlang:now() blocks the scheduler shortly, so that is something I > definitely want to avoid. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > How about process_info(self(), reductions). ? Regards, Zoltan. From v@REDACTED Tue Nov 24 17:49:13 2009 From: v@REDACTED (Valentin Micic) Date: Tue, 24 Nov 2009 18:49:13 +0200 Subject: [erlang-questions] The 2 GB limit In-Reply-To: Message-ID: <20091124164919.B8B643D0D4A@mail.pharos-avantgard.com> If you assume data size (as in record size) to be constant, then file size is directly proportional to a number of the records. OTOH, if you take a file size as a constant (as in 2GB), then number of records is inversely proportional to the data size (as in record size). I've used the term "data density" (for the lack of the better word) to express a level of distribution of data among dets slots -- higher the number of records per slot, higher the data density. In this context, the value for data density is very much in a function of hashing algorithm used to map a particular key value to a given slot. Assuming that hashing algorithm maps things (relatively) evenly, thus the "data density" ends up being function of number of records in dets file. What I was trying to tell you is that updates on dets that contains high number of records (and hence more records per dets slot, therefore higher "data density") may be quite processing intensive, as probability of not being able to host the resulting data within the memory allocated to the particular slot is higher with a higher "data density". With this in mind, I've been asking if increasing dets file beyond 2GB would make any sense (because bigger file inevitably leads to a higher "data density") -- it is far healthier to distribute the data across multiple dets files and keep the "data density" per file at lower level for the same amount of data. My apologies for the confusion. V/ -----Original Message----- From: Igor Ribeiro Sucupira [mailto:igorrs@REDACTED] Sent: 24 November 2009 05:02 PM To: Valentin Micic Cc: erlang-questions Subject: Re: [erlang-questions] The 2 GB limit On Tue, Nov 24, 2009 at 12:47 PM, Valentin Micic wrote: > I must express my skepticism regarding dets performance when a file reaches > such a level of "data density" Hello. Since I don't know very much about the inner workings of dets files, maybe I am confused by the terminology: wouldn't data density be inversely proportional to file size and directly proportional to data size? Thank you. Igor. > (assuming that 2 GB translates to at least a > few million records). As much as I believe that reading from such a big file > may be relatively fast, I think that updating the data would be > disproportionately slower. In my experience, the only way to maintain a good > performance with sizable data sets (say, 50-150 million records) in dets is > to distribute data over a number of dets files (similar to what mnesia > fragmentation does). > So, if my skepticism is justifiable, this is not a question of how difficult > it is to increase a dets file size limit, but would there be any point in > doing it? > > V/ > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Igor Ribeiro Sucupira > Sent: 24 November 2009 04:24 PM > To: erlang-questions > Subject: [erlang-questions] The 2 GB limit > > Hi. > > Is there any plan (or work in progress) for removing the 2 GB size > limit of dets files? > > And, while this is not accomplished, do you think it could be easier > to try to raise that limit to 4 GB? Assuming the issue is related to > 32-bit addressing, I'm guessing (just guessing) that dealing with 4 GB > files should not be difficult. Am I wrong? Why? > > Thanks. > Igor. > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From phmander@REDACTED Tue Nov 24 18:25:49 2009 From: phmander@REDACTED (Peter-Henry Mander) Date: Tue, 24 Nov 2009 17:25:49 +0000 Subject: [erlang-questions] Erlang crypto_drv in R13B02/03 fails to load on amd64-Solaris2.10 In-Reply-To: <200911232046.nANKkkdh067394@pluto.hedeland.org> References: <38e3b9f40911230758h59f94ed0n3786966cfeb35313@mail.gmail.com> <200911232046.nANKkkdh067394@pluto.hedeland.org> Message-ID: <38e3b9f40911240925w29a4563ld62a829ae3c9d919@mail.gmail.com> Minor update: To be absolutely sure, I verified that the same build environment works for R12B-5, which it does. The crypto_drv.so loads in R12B-5 on AMD64. On Mon, Nov 23, 2009 at 8:46 PM, Per Hedeland wrote: > Perhaps you need to take a step back here - what does > > ?file .../priv/obj/i386-pc-solaris2.10/crypto_drv.o > > say? From my reading of the Solaris 'ld' man page, -64 shouldn't be > needed - but it might possibly "mask" the mistake of trying to build a > 64-bit shared object from a 32-bit "unshared" object. > > --Per > From jarrod@REDACTED Tue Nov 24 19:20:40 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Tue, 24 Nov 2009 13:20:40 -0500 Subject: [erlang-questions] The 2 GB limit In-Reply-To: <20091124164919.B8B643D0D4A@mail.pharos-avantgard.com> References: <20091124164919.B8B643D0D4A@mail.pharos-avantgard.com> Message-ID: Actually an interesting thing to look into for LARGE structured storage is HDF5. I have used it in a couple of projects where I needed to be able to store arbitrary sized files and file groups with metadata about them. I have used it from Python, Java and Objective-C. It shouldn't be to hard to "port" dets to use something like this library to store data in. From igorrs@REDACTED Tue Nov 24 19:23:39 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Tue, 24 Nov 2009 16:23:39 -0200 Subject: [erlang-questions] The 2 GB limit In-Reply-To: <20091124164919.B8B643D0D4A@mail.pharos-avantgard.com> References: <20091124164919.B8B643D0D4A@mail.pharos-avantgard.com> Message-ID: Hello, Valentin. I took a quick look at dets_v9.erl and I understand what you mean: as you insert more records, you will have more and more data in each slot. But, from the documentation: http://erlang.org/doc/man/dets.html It seems you can tune the number of slots (see options: min_no_slots and max_no_slots). Today you have to create enough fragments so that none of them will approach the 2 GB limit. If that limit was raised to, say, 4 GB, you could in advance create your dets files with more slots, thus being able to store more data without losing performance (and using less fragments). I have never tried to fine tune the number of slots in dets files (just seen it in the docs), so maybe what I am saying causes other performance problems (they do say something about it in the documentation, but I'll have to see the source code to understand it). Thank you. Igor. On Tue, Nov 24, 2009 at 2:49 PM, Valentin Micic wrote: > If you assume data size (as in record size) to be constant, then file size > is directly proportional to a number of the records. > > OTOH, if you take a file size as a constant (as in 2GB), then number of > records is inversely proportional to the data size (as in record size). > > I've used the term "data density" (for the lack of the better word) to > express a level of distribution of data among dets slots -- higher the > number of records per slot, higher the data density. In this context, the > value for data density is very much in a function of hashing algorithm used > to map a particular key value to a given slot. Assuming that hashing > algorithm maps things (relatively) evenly, thus the "data density" ends up > being function of number of records in dets file. > > What I was trying to tell you is that updates on dets that contains high > number of records (and hence more records per dets slot, therefore higher > "data density") may be quite processing intensive, as probability of not > being able to host the resulting data within the memory allocated to the > particular slot is higher with a higher "data density". > > With this in mind, I've been asking if increasing dets file beyond 2GB would > make any sense (because bigger file inevitably leads to a higher "data > density") -- it is far healthier to distribute the data across multiple dets > files and keep the "data density" per file at lower level for the same > amount of data. > > My apologies for the confusion. > > V/ > > > -----Original Message----- > From: Igor Ribeiro Sucupira [mailto:igorrs@REDACTED] > Sent: 24 November 2009 05:02 PM > To: Valentin Micic > Cc: erlang-questions > Subject: Re: [erlang-questions] The 2 GB limit > > On Tue, Nov 24, 2009 at 12:47 PM, Valentin Micic > wrote: >> I must express my skepticism regarding dets performance when a file > reaches >> such a level of "data density" > > Hello. > > Since I don't know very much about the inner workings of dets files, > maybe I am confused by the terminology: wouldn't data density be > inversely proportional to file size and directly proportional to data > size? > > Thank you. > Igor. > >> (assuming that 2 GB translates to at least a >> few million records). As much as I believe that reading from such a big > file >> may be relatively fast, I think that updating the data would be >> disproportionately slower. In my experience, the only way to maintain a > good >> performance with sizable data sets (say, 50-150 million records) in dets > is >> to distribute data over a number of dets files (similar to what mnesia >> fragmentation does). >> So, if my skepticism is justifiable, this is not a question of how > difficult >> it is to increase a dets file size limit, but would there be any point in >> doing it? >> >> V/ >> >> -----Original Message----- >> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On >> Behalf Of Igor Ribeiro Sucupira >> Sent: 24 November 2009 04:24 PM >> To: erlang-questions >> Subject: [erlang-questions] The 2 GB limit >> >> Hi. >> >> Is there any plan (or work in progress) for removing the 2 GB size >> limit of dets files? >> >> And, while this is not accomplished, do you think it could be easier >> to try to raise that limit to 4 GB? Assuming the issue is related to >> 32-bit addressing, I'm guessing (just guessing) that dealing with 4 GB >> files should not be difficult. Am I wrong? Why? >> >> Thanks. >> Igor. >> >> -- >> "The secret of joy in work is contained in one word - excellence. To >> know how to do something well is to enjoy it." - Pearl S. Buck. >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From raould@REDACTED Tue Nov 24 19:36:52 2009 From: raould@REDACTED (Raoul Duke) Date: Tue, 24 Nov 2009 10:36:52 -0800 Subject: [erlang-questions] Erlang "object-oriented" after all? In-Reply-To: <95be1d3b0911240125h1916ab40s8c9cc9b8acc47463@mail.gmail.com> References: <9b08084c0911240106w4d946ffaod6fecaca4576983c@mail.gmail.com> <95be1d3b0911240125h1916ab40s8c9cc9b8acc47463@mail.gmail.com> Message-ID: <91a2ba3e0911241036p31a95058yffc7886591602ae1@mail.gmail.com> > Today that name would be > the name of a module, but it feels like module names get a too > overloaded meaning. I don't have an alternative suggestion yet, > though. perhaps take a look at Clojure http://blog.higher-order.net/2009/11/23/clojure-circuit-breaker/ (requires javascript on to see the code samples) From tony@REDACTED Tue Nov 24 19:46:44 2009 From: tony@REDACTED (Tony Arcieri) Date: Tue, 24 Nov 2009 11:46:44 -0700 Subject: [erlang-questions] Erlang "object-oriented" after all? In-Reply-To: References: Message-ID: On Tue, Nov 24, 2009 at 12:25 AM, Michael Turner wrote: > "I thought of objects being like biological cells and/or individual > computers on a network, only able to communicate with messages . . . . I > wanted to get rid of data. . . ." You'll find this quote in my presentations on Reia: http://www.unlimitednovelty.com/2009/05/reia-presentations.html In this regard, I think Erlang is closer to Alan Kay's conception of an OO language than most "OO" languages today. Such is the legacy of C++. The "only able to communicate with messages" concept is not upheld in many of these languages, as objects can "communicate" through destructive mutations of shared state rather than messages. -- Tony Arcieri Medioh/Nagravision From raould@REDACTED Tue Nov 24 19:54:36 2009 From: raould@REDACTED (Raoul Duke) Date: Tue, 24 Nov 2009 10:54:36 -0800 Subject: [erlang-questions] Erlang "object-oriented" after all? In-Reply-To: References: Message-ID: <91a2ba3e0911241054m2ef660a2n7b6db42c58318c03@mail.gmail.com> > Such is the legacy of C++. The > "only able to communicate with messages" concept is not upheld in many of seems like Objective-C is a little more really OOish. not surprising given its history. From kagato@REDACTED Tue Nov 24 19:59:49 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Tue, 24 Nov 2009 10:59:49 -0800 Subject: [erlang-questions] Fastest pseudo-random number-generator: erlang:statistics(io) ? In-Reply-To: References: Message-ID: <40930F41-0E0C-484D-B7CC-7EF80DEF06B8@souja.net> Is it too slow to run the result through erlang:phash/2? This has the added bonus that phash2/2 (not hash or phash, but phash2) has a range argument, so you wouldn't have to scale the number down to list size. It also wouldn't be a problem on a system doing very little I/O, as long as it has done some IO. Of course, you could always toss a counter from the process dictionary (and maybe the PID) into a tuple with it, and poof "unique hash". I haven't benchmarked it, but I'd imagine it shouldn't be too much slower, right? For that matter, what about erlang:process_info(self(),reductions)? Maybe it would be faster since it's only process-local data? On Nov 24, 2009, at 7:20 AM, Dave Smith wrote: > On Tue, Nov 24, 2009 at 8:00 AM, Thijs wrote: > >> benchmark results : >> >> erlang:now() 288.090 >> erlang:statistics(io) 2.730.853 >> erlang:statistics(wall_clock) 280.396 >> erlang:statistics(runtime) 510.765 >> erlang:statistics(reductions) 471.688 >> erlang:statistics(context_switches) 493.851 >> random:uniform() 870.261 > > Off the top of my head, I have to wonder if using statistics(io) is > going to yield sufficient pseudo-randomness for load distribution > purposes. If you consider that most IO is likely to be a block size > modulo 8 (for efficiency) it seems like you might get a skewed > distribution. A simpler (and perhaps faster) approach would be to just > maintain a counter that gets atomically incremented on each call -- a > port driver or ETS table would suffice and have similar amounts of > contention as the call to statistics. > > My $0.02...and you know how weak the dollar has been these days. :) > > D. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From b@REDACTED Tue Nov 24 20:18:03 2009 From: b@REDACTED (Benjamin Black) Date: Tue, 24 Nov 2009 11:18:03 -0800 Subject: public_key and CSRs? Message-ID: <4B0C316B.8010202@b3k.us> The documentation for public_key 0.3 seems incomplete and, looking at the source, there is quite a bit of functionality buried in it. I am trying to implement a certificate authority and I can load and push around keys and CSRs, but I don't see an obvious way to sign a CSR to generate a new certificate. Could someone point me at the documentation or a code sample for this? Thanks, b -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 550 bytes Desc: OpenPGP digital signature URL: From v@REDACTED Tue Nov 24 21:15:48 2009 From: v@REDACTED (Valentin Micic) Date: Tue, 24 Nov 2009 22:15:48 +0200 Subject: [erlang-questions] The 2 GB limit In-Reply-To: Message-ID: <20091124201600.CD2023D0D4A@mail.pharos-avantgard.com> Yes, you can tune a number of slots -- I did try it once, but for whatever reason it did not create any difference in my case (when the table grew to around 400,000 records, performance deteriorated rapidly when inserting or updating). It could be that the problem was elsewhere and that I did not persevere in optimizing it -- it was so much easier to settle for mensia fragmentation, which combined with an appropriate +A value made substantial performance difference with disc_only_copy tables (dets). I will be very glad if you arrive to a different conclusion regarding min_no_slot testing. V/ -----Original Message----- From: Igor Ribeiro Sucupira [mailto:igorrs@REDACTED] Sent: 24 November 2009 08:24 PM To: Valentin Micic Cc: erlang-questions Subject: Re: [erlang-questions] The 2 GB limit Hello, Valentin. I took a quick look at dets_v9.erl and I understand what you mean: as you insert more records, you will have more and more data in each slot. But, from the documentation: http://erlang.org/doc/man/dets.html It seems you can tune the number of slots (see options: min_no_slots and max_no_slots). Today you have to create enough fragments so that none of them will approach the 2 GB limit. If that limit was raised to, say, 4 GB, you could in advance create your dets files with more slots, thus being able to store more data without losing performance (and using less fragments). I have never tried to fine tune the number of slots in dets files (just seen it in the docs), so maybe what I am saying causes other performance problems (they do say something about it in the documentation, but I'll have to see the source code to understand it). Thank you. Igor. On Tue, Nov 24, 2009 at 2:49 PM, Valentin Micic wrote: > If you assume data size (as in record size) to be constant, then file size > is directly proportional to a number of the records. > > OTOH, if you take a file size as a constant (as in 2GB), then number of > records is inversely proportional to the data size (as in record size). > > I've used the term "data density" (for the lack of the better word) to > express a level of distribution of data among dets slots -- higher the > number of records per slot, higher the data density. In this context, the > value for data density is very much in a function of hashing algorithm used > to map a particular key value to a given slot. Assuming that hashing > algorithm maps things (relatively) evenly, thus the "data density" ends up > being function of number of records in dets file. > > What I was trying to tell you is that updates on dets that contains high > number of records (and hence more records per dets slot, therefore higher > "data density") may be quite processing intensive, as probability of not > being able to host the resulting data within the memory allocated to the > particular slot is higher with a higher "data density". > > With this in mind, I've been asking if increasing dets file beyond 2GB would > make any sense (because bigger file inevitably leads to a higher "data > density") -- it is far healthier to distribute the data across multiple dets > files and keep the "data density" per file at lower level for the same > amount of data. > > My apologies for the confusion. > > V/ > > > -----Original Message----- > From: Igor Ribeiro Sucupira [mailto:igorrs@REDACTED] > Sent: 24 November 2009 05:02 PM > To: Valentin Micic > Cc: erlang-questions > Subject: Re: [erlang-questions] The 2 GB limit > > On Tue, Nov 24, 2009 at 12:47 PM, Valentin Micic > wrote: >> I must express my skepticism regarding dets performance when a file > reaches >> such a level of "data density" > > Hello. > > Since I don't know very much about the inner workings of dets files, > maybe I am confused by the terminology: wouldn't data density be > inversely proportional to file size and directly proportional to data > size? > > Thank you. > Igor. > >> (assuming that 2 GB translates to at least a >> few million records). As much as I believe that reading from such a big > file >> may be relatively fast, I think that updating the data would be >> disproportionately slower. In my experience, the only way to maintain a > good >> performance with sizable data sets (say, 50-150 million records) in dets > is >> to distribute data over a number of dets files (similar to what mnesia >> fragmentation does). >> So, if my skepticism is justifiable, this is not a question of how > difficult >> it is to increase a dets file size limit, but would there be any point in >> doing it? >> >> V/ >> >> -----Original Message----- >> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On >> Behalf Of Igor Ribeiro Sucupira >> Sent: 24 November 2009 04:24 PM >> To: erlang-questions >> Subject: [erlang-questions] The 2 GB limit >> >> Hi. >> >> Is there any plan (or work in progress) for removing the 2 GB size >> limit of dets files? >> >> And, while this is not accomplished, do you think it could be easier >> to try to raise that limit to 4 GB? Assuming the issue is related to >> 32-bit addressing, I'm guessing (just guessing) that dealing with 4 GB >> files should not be difficult. Am I wrong? Why? >> >> Thanks. >> Igor. >> >> -- >> "The secret of joy in work is contained in one word - excellence. To >> know how to do something well is to enjoy it." - Pearl S. Buck. >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From jarrod@REDACTED Tue Nov 24 21:18:06 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Tue, 24 Nov 2009 15:18:06 -0500 Subject: [erlang-questions] How to match on a nested record against a dict In-Reply-To: <4B0A49CC.90307@erlang-consulting.com> References: <4B0A49CC.90307@erlang-consulting.com> Message-ID: Thanks for the example code, it is getting me past the "how" and onto the "why" of logic in Erlang, I have it all working except this one part. I keep getting this exception. I have used your example and come up with this. start() -> S=open({224,0,0,251},5353), Pid=spawn(?MODULE,receiver,[dict:store("_see._tcp.local",[],dict:new())]), gen_udp:controlling_process(S,Pid), {S,Pid}. receiver(Sub) -> receive {udp, _Socket, _IP, _InPortNo, Packet} -> process_dnsrec(Sub,inet_dns:decode(Packet)), receiver(Sub); stop -> true; AnythingElse -> io:format("RECEIVED: ~p~n",[AnythingElse]), receiver(Sub) end. process_dnsrec(_Sub,{error,E}) -> io:format("Error: ~p~n", [E]); process_dnsrec(Sub,{ok,#dns_rec{anlist=Responses}}) -> process_dnsrec1(Sub,Responses). process_dnsrec1(_,[]) -> ok; process_dnsrec1(Sub,[#dns_rr{domain=Dom}|Rest]) -> case dict:find(Dom,Sub) of [Result] -> io:format("Interesting domain ~p=~p~n",[Dom,Result]); error -> %% do nothing for non-interesting domains ok end, process_dnsrec1(Sub,Rest). and here is the Error I am getting Erlang R13B02 (erts-5.7.3) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.7.3 (abort with ^G) 1> zeroconf:start(). {#Port<0.442>,<0.34.0>} 2> =ERROR REPORT==== 24-Nov-2009::15:16:40 === Error in process <0.34.0> with exit value: {{case_clause,{ok,[]}},[{zeroconf,process_dnsrec1,2},{zeroconf,receiver,1}]} From jarrod@REDACTED Tue Nov 24 22:25:57 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Tue, 24 Nov 2009 16:25:57 -0500 Subject: [erlang-questions] How to match on a nested record against a dict In-Reply-To: References: <4B0A49CC.90307@erlang-consulting.com> Message-ID: On Tue, Nov 24, 2009 at 3:18 PM, Jarrod Roberson wrote: > > process_dnsrec1(Sub,[#dns_rr{domain=Dom}|Rest]) -> > case dict:find(Dom,Sub) of > [Result] -> > io:format("Interesting domain ~p=~p~n",[Dom,Result]); > thanks for reading I figured it out by trial and error. I replaced the [Result] with {ok,Result} and it started working as expected From ok@REDACTED Tue Nov 24 23:01:51 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 25 Nov 2009 11:01:51 +1300 Subject: [erlang-questions] Erlang "object-oriented" after all? In-Reply-To: <9b08084c0911240106w4d946ffaod6fecaca4576983c@mail.gmail.com> References: <9b08084c0911240106w4d946ffaod6fecaca4576983c@mail.gmail.com> Message-ID: <40F3F1FA-0302-40CE-A8C9-A2BDEDC0999F@cs.otago.ac.nz> On Nov 24, 2009, at 10:06 PM, Joe Armstrong wrote: > One problem with saying "everything is an object" is that it doesn't > allow us to talk about things which are not > objects. So ask questions like "is time an object?" is "code > replacement an object?" is "is scalability an object?" > you won't get a sensible reply. But "everything is an object" doesn't mean that. It means that "whatever can be the value of a variable is an object". If "time" or "code replacement" or "scalability" could be the value of a variable in some language, then it would make sense, in that language, to ask if they were objects, but not otherwise. In Smalltalk, for example, when you look at 2 squared + 2 is an object + #squared is an object + "tell somebody to square itself" is an object + the source code for Integer>>squared is an object + the history of changes to that source code is an object + the executable code for Integer>>squared is an object + the compiler is an object and can be asked to install new code + the execution context when the method runs is an object - but "execution" per se cannot be the value of a variable and is not an object and "compiling" and "installing" are not objects. They are processes in time that take place when certain objects are asked to do certain things. > > > The erlang answers are easier: > > "Q: What is code replacement?" > "A: A protocol - Pid ! {newCode, F} tells the process Pid to start > executing new code F" > > and so on.... That "A" is the wrong category to be an answer to that "Q". "A" answers Q': How do I make code replacement happen? A : A protocol - Pid ! {newCode, F} tells the process Pid to start executing new code F % but almost no processes understand that protocol From bernie@REDACTED Wed Nov 25 00:20:43 2009 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 25 Nov 2009 10:20:43 +1100 Subject: [erlang-questions] Possible hostname handling bug in node handshake procedure? In-Reply-To: References: Message-ID: <4B0C6A4B.9040704@m5net.com> Maas-Maarten Zeeman wrote: > (a@REDACTED)1> net_adm:ping(b@REDACTED). > **long wait** > pang > (a@REDACTED)2> net_adm:ping(b@REDACTED). > pong > > In both situations the network packets arrive at b@REDACTED To me it looks > like there is a case sensitivity problem with the hostname handling in > the handshake procedure. Remember, though: What you're passing to ping() is not a host name, it's a /node/ name. A node name is an erlang atom and is therefore case sensitive. It's certainly derived from a hostname, and might seem at first like it should be case insensitive as a result, but as Bengt noted that would mean implying atom equality where they're not actually equal. Cheers, Bernard From bernie@REDACTED Wed Nov 25 00:37:28 2009 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 25 Nov 2009 10:37:28 +1100 Subject: Record typing for dialyzer Message-ID: <4B0C6E38.50409@m5net.com> Hi list, We have a lot of auto-generated record types that we use to allow our (newer, awesomer) erlang/mnesia code to interact with our (older, less awesome, but we're stuck with it for now) C++/mysql code. It recently occurred to me that it would be a simple matter to add type specifiers to the records to give dialyzer (which we love to death) some extra info to work with. And for the most part it was. So we now have nice auto-generated records of the form: -record( person{ id :: integer(), name :: string(), is_awesome :: boolean() }). which is great (and already helped us find several bugs - have I mentioned we love dialyzer?). We struck a problem, however, when we used one of our records in a select statement thus: Match = #person{name="Bernie", id='$1', _='_'}, mnesia:select(person, [{Match, [], [{{'$1'}}]}]), At this point, dialyzer quite correctly points out that '$1' isn't an integer and '_' isn't a boolean and so we've got no business assigning them to fields typed as such. My solution, then was to add an extra type definition: -type search_spec() :: '_' | '$1' | '$2' | '$3' | '$4' | '$5' | '$6' | '$7' | '$8' | '$9'. and change all the record definitions to something like: -record( person{ id :: integer() | search_spec(), name :: string() | search_spec(), is_awesome :: boolean() | search_spec() }). Now this solves the problem, but it really looks ugly and seems like kind of a hack and I can't help but feel like there must be some more elegant solution. But I don't know what it is :) Any input (even if it's only "nope, there's no better way to do it") would be greatly appreciated. Cheers, Bernard From steven.charles.davis@REDACTED Wed Nov 25 01:18:12 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 24 Nov 2009 18:18:12 -0600 Subject: [erlang-questions] Re: list size 'causing VM "problems" In-Reply-To: References: <9a06d836-736f-4468-96ed-e7cf0d41547c@p19g2000vbq.googlegroups.com> Message-ID: <4B0C77C4.6050601@gmail.com> Hendrik Visage wrote: > the: > List=read_lines(FD). > executes, doesn't *appear* to be using lots of VM space, and it > outputs the partial (being the shell it only prints a part of the > lines with the [...]...] stuff. This appears to be an improvement! > Then it *hangs* at that, not showing the prompt (this is inside > Aquamacs's erlang shell mode). This is then where it appears the > system goes "west", as the memory utilization (as measured/shown by > the MacOSX Activity Monitor) starts to grow and grow. with the last > test (using the Accumulator as below) it grew to 7G at which point I > killed the beam.smp process. Next I'd suspect your regex is causing issues. Try some logging to figure out what lines are causing issues with the regex expression you defined (and perhaps incorporate Ulf's suggestion to use binaries). If you don't find a solution there, it could well be that you'll have to push the results out to ETS as Ulf suggested (and my experience is that Ulf is generally right about such things - not to mention the fact that he has many more years of experience with OTP than I do). /s From paul-trapexit@REDACTED Wed Nov 25 01:21:56 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Tue, 24 Nov 2009 16:21:56 -0800 (PST) Subject: [erlang-questions] Record typing for dialyzer In-Reply-To: <4B0C6E38.50409@m5net.com> References: <4B0C6E38.50409@m5net.com> Message-ID: I had the same problem. I ended up keeping the type signature in the declaration strict, and using setelement/3 to construct the type-violating version of the record that I pass to mnesia:select/2. -- p On Wed, 25 Nov 2009, Bernard Duggan wrote: > Hi list, > We have a lot of auto-generated record types that we use to allow > our (newer, awesomer) erlang/mnesia code to interact with our (older, > less awesome, but we're stuck with it for now) C++/mysql code. It > recently occurred to me that it would be a simple matter to add type > specifiers to the records to give dialyzer (which we love to death) some > extra info to work with. And for the most part it was. So we now have > nice auto-generated records of the form: > > -record( person{ > id :: integer(), > name :: string(), > is_awesome :: boolean() > }). > > which is great (and already helped us find several bugs - have I > mentioned we love dialyzer?). We struck a problem, however, when we > used one of our records in a select statement thus: > > Match = #person{name="Bernie", id='$1', _='_'}, > mnesia:select(person, [{Match, [], [{{'$1'}}]}]), > > At this point, dialyzer quite correctly points out that '$1' isn't an > integer and '_' isn't a boolean and so we've got no business assigning > them to fields typed as such. > > My solution, then was to add an extra type definition: > > -type search_spec() :: '_' | '$1' | '$2' | '$3' | '$4' | '$5' | '$6' | > '$7' | '$8' | '$9'. > > and change all the record definitions to something like: > > -record( person{ > id :: integer() | search_spec(), > name :: string() | search_spec(), > is_awesome :: boolean() | search_spec() > }). > > Now this solves the problem, but it really looks ugly and seems like > kind of a hack and I can't help but feel like there must be some more > elegant solution. But I don't know what it is :) Any input (even if > it's only "nope, there's no better way to do it") would be greatly > appreciated. > > Cheers, > > Bernard > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From bernie@REDACTED Wed Nov 25 01:51:41 2009 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 25 Nov 2009 11:51:41 +1100 Subject: [erlang-questions] Record typing for dialyzer In-Reply-To: References: <4B0C6E38.50409@m5net.com> Message-ID: <4B0C7F9D.5040207@m5net.com> That was another other option I toyed with - I ended up coming down on the side of keeping the "code" as un-hacked as possible and just messing with the headers and auto-generated stuff. My reasoning being that I didn't want others in the team to have to re-solve the problem every time they wanted to use a select() statement. Not that I'm saying it's not a perfectly valid solution, just explaining why I didn't choose it :) Cheers, B Paul Mineiro wrote: > I had the same problem. > > I ended up keeping the type signature in the declaration strict, and using > setelement/3 to construct the type-violating version of the record that I > pass to mnesia:select/2. > > -- p > > On Wed, 25 Nov 2009, Bernard Duggan wrote: > > >> Hi list, >> We have a lot of auto-generated record types that we use to allow >> our (newer, awesomer) erlang/mnesia code to interact with our (older, >> less awesome, but we're stuck with it for now) C++/mysql code. It >> recently occurred to me that it would be a simple matter to add type >> specifiers to the records to give dialyzer (which we love to death) some >> extra info to work with. And for the most part it was. So we now have >> nice auto-generated records of the form: >> >> -record( person{ >> id :: integer(), >> name :: string(), >> is_awesome :: boolean() >> }). >> >> which is great (and already helped us find several bugs - have I >> mentioned we love dialyzer?). We struck a problem, however, when we >> used one of our records in a select statement thus: >> >> Match = #person{name="Bernie", id='$1', _='_'}, >> mnesia:select(person, [{Match, [], [{{'$1'}}]}]), >> >> At this point, dialyzer quite correctly points out that '$1' isn't an >> integer and '_' isn't a boolean and so we've got no business assigning >> them to fields typed as such. >> >> My solution, then was to add an extra type definition: >> >> -type search_spec() :: '_' | '$1' | '$2' | '$3' | '$4' | '$5' | '$6' | >> '$7' | '$8' | '$9'. >> >> and change all the record definitions to something like: >> >> -record( person{ >> id :: integer() | search_spec(), >> name :: string() | search_spec(), >> is_awesome :: boolean() | search_spec() >> }). >> >> Now this solves the problem, but it really looks ugly and seems like >> kind of a hack and I can't help but feel like there must be some more >> elegant solution. But I don't know what it is :) Any input (even if >> it's only "nope, there's no better way to do it") would be greatly >> appreciated. >> >> Cheers, >> >> Bernard >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From thijsterlouw@REDACTED Wed Nov 25 02:33:00 2009 From: thijsterlouw@REDACTED (Thijs) Date: Tue, 24 Nov 2009 17:33:00 -0800 (PST) Subject: Fastest pseudo-random number-generator: erlang:statistics(io) ? In-Reply-To: <40930F41-0E0C-484D-B7CC-7EF80DEF06B8@souja.net> References: <40930F41-0E0C-484D-B7CC-7EF80DEF06B8@souja.net> Message-ID: On Nov 25, 2:59?am, Jayson Vantuyl wrote: > Is it too slow to run the result through erlang:phash/2? I tested it, erlang:phash2(erlang:statistics(io), 10) it still is fast, but significantly slower than without. You are right that it immediately scales the result, so i tested with scaling the erlang:statistics(io) as well. I think the randomness (for my case) will not improve a lot, but I will keep this function in mind. On Nov 25, 12:10 am, "Zoltan Lajos Kis" wrote: > How about process_info(self(), reductions). ? This one is also very fast (only a bit slower than statistics(io)), but I am concerned about the randomness of this function (at least in my case). I have many short-lived processes and it's likely that the number of reductions for each process might be quite similar if they follow similar code paths? new benchmark results: erlang_now req/sec 296667.7 stats_io req/sec 2638898.4 stats_wall_clock req/sec 289306.0 stats_runtime req/sec 522258.4 stats_reductions req/sec 484499.9 stats_cont_switch req/sec 491190.3 random_uniform req/sec 1109517.1 process_info_reduc req/sec 2410724.8 phash2_io req/sec 1773493.0 stats_io_scaled req/sec 2357073.0 tested like this: --------------------------- test_random_speed(ParCount, SeqCount) -> FunErlangNow = fun() -> erlang:now() end, FunIO = fun() -> erlang:statistics(io) end, FunWallClock = fun() -> erlang:statistics(wall_clock) end, FunRuntime = fun() -> erlang:statistics(runtime) end, FunReductions = fun() -> erlang:statistics(reductions) end, FunContextSwitches = fun() -> erlang:statistics(context_switches) end, FunRandomUniform = fun() -> random:uniform() end, FunProcessInfoRed = fun() -> erlang:process_info(self(), reductions) end, FunPhashIO = fun() -> erlang:phash2(erlang:statistics(io), 10) end, FunIOScaled = fun() -> {{input, Input}, {output, Output}} = erlang:statistics(io), (Input+Output) rem 10 end, test_fun_x_times_parallel(erlang_now, FunErlangNow, ParCount, SeqCount), test_fun_x_times_parallel(stats_io, FunIO, ParCount, SeqCount), test_fun_x_times_parallel(stats_wall_clock, FunWallClock, ParCount, SeqCount), test_fun_x_times_parallel(stats_runtime, FunRuntime, ParCount, SeqCount), test_fun_x_times_parallel(stats_reductions, FunReductions, ParCount, SeqCount), test_fun_x_times_parallel(stats_cont_switch, FunContextSwitches, ParCount, SeqCount), test_fun_x_times_parallel(random_uniform, FunRandomUniform, ParCount, SeqCount), test_fun_x_times_parallel(process_info_reduc, FunProcessInfoRed, ParCount, SeqCount), test_fun_x_times_parallel(phash2_io, FunPhashIO, ParCount, SeqCount), test_fun_x_times_parallel(stats_io_scaled, FunIOScaled, ParCount, SeqCount), ok. test_fun_x_times_parallel(Name, Fun, ParCount, SeqCount) -> Tick = now(), app_util:pmap( fun(_) -> lists:foreach(fun(_) -> Fun() end, lists:seq(1, SeqCount)) end, lists:seq(1, ParCount), [], 250000), Tock = now(), AvgTime = timer:now_diff(Tock, Tick)/1000, Speed = ParCount*SeqCount*1000/AvgTime, io:format("~20w req/sec ~10.1f ~n", [Name, Speed]), Speed. pmap(Fun, List, Nodes, Timeout) -> SpawnFun = case length(Nodes) of 0 -> fun spawn/1; Length -> NextNode = fun() -> lists:nth(random:uniform(Length), Nodes) end, fun(F) -> spawn(NextNode(), F) end end, Parent = self(), Pids = [ SpawnFun(fun() -> Parent ! {self(), (catch Fun(Elem))} end) || Elem <- List ], [ receive {Pid, Val} -> Val after Timeout -> exit(timeout) end || Pid <- Pids ]. From kostis@REDACTED Wed Nov 25 08:44:06 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 25 Nov 2009 09:44:06 +0200 Subject: [erlang-questions] Record typing for dialyzer In-Reply-To: <4B0C6E38.50409@m5net.com> References: <4B0C6E38.50409@m5net.com> Message-ID: <4B0CE046.5060402@cs.ntua.gr> Bernard Duggan wrote: > Hi list, > We have a lot of auto-generated record types that we use to allow > our (newer, awesomer) erlang/mnesia code to interact with our (older, > less awesome, but we're stuck with it for now) C++/mysql code. It > recently occurred to me that it would be a simple matter to add type > specifiers to the records to give dialyzer (which we love to death) some > extra info to work with. And for the most part it was. So we now have > nice auto-generated records of the form: > > -record( person{ > id :: integer(), > name :: string(), > is_awesome :: boolean() > }). > > which is great (and already helped us find several bugs - have I > mentioned we love dialyzer?). We struck a problem, however, when we > used one of our records in a select statement thus: > > Match = #person{name="Bernie", id='$1', _='_'}, > mnesia:select(person, [{Match, [], [{{'$1'}}]}]), > > At this point, dialyzer quite correctly points out that '$1' isn't an > integer and '_' isn't a boolean and so we've got no business assigning > them to fields typed as such. > > My solution, then was to add an extra type definition: > > -type search_spec() :: '_' | '$1' | '$2' | '$3' | '$4' | '$5' | '$6' | > '$7' | '$8' | '$9'. > > and change all the record definitions to something like: > > -record( person{ > id :: integer() | search_spec(), > name :: string() | search_spec(), > is_awesome :: boolean() | search_spec() > }). > > Now this solves the problem, but it really looks ugly and seems like > kind of a hack and I can't help but feel like there must be some more > elegant solution. But I don't know what it is :) Any input (even if > it's only "nope, there's no better way to do it") would be greatly > appreciated. I am not aware of any cleaner way of doing what you want to do, so what you've done is how I would personally also do it. The only thing I would recommend is to try to add these declarations as sparsely as possible, meaning try to add them only to the record fields that are used in match specifications in your program, not everywhere. The problem is that you (and dialyzer) are up against a really bad design decision in the language here, namely the use of "special" atoms to represent variables in match specifications. I sincerely hope that this is fixed by Erlang providing an alternative way of writing match specifications that does not lie about types. Thanks for your thumbs up for dialyzer! Kostis From erlang@REDACTED Wed Nov 25 10:56:37 2009 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 25 Nov 2009 10:56:37 +0100 Subject: A Generic API for controlling software components Message-ID: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> I would like to start a discussion about the generic control of software components. Let me start by describing the problem and a possible solution. I'd welcome feedback on this. What's the problem? To start with I'll use the term "component" rather than the word "application" since the word application has specific meaning in the OTP context. To me a component is a black-box offering some functionality that I want to use in my project. Examples of components are couchDB, mochiweb, rabbitMQ, ejabberd, yaws etc. All these different components have chosen Ad Hock ways of controlling them. The way that I control couchDB and yaws is completely different. The way I upgrade the code is completely different. What I would like to do is manage all the components in a uniform manner. Much of this design is inspired by how Mac OS-X manages applications. By manage I mean start and stop the component, upgrade the code, change the behavior of the component at run time and so on. Here's a suggestion for a set of rules for managed components: Draft 1 - 25 Nov 2009 Rule1: All components are unpacked into the same top-level directory (default $HOME/eComponent) and have the extension .ec (Erlang component) Example: Imagine I have installed mochiweb ejabberd and couchDB then after installation I should see the following: $pwd /home/joe/eComponents $ls mochiweb.ec ejabberd.ec couchDB.ec Rule2: A normal user should *never* have to examine any of the files *inside* the component. To do so causes "abstraction leakage" - I want to consider each of these components as black boxes. Once installed the component should be installed in a read-only disk area. Rule3: The component should not break if relocated to a different top-level directory. A simple "mv" command should suffice So no hard-wired paths or links please. Rule4: All components C must have a file called $HOME/eComponents/C.ec/Preferences.pl The extension .pl means the file contains a property list. Here is an example: {codePath, "/bin"}. {expiryDate, {2009,12,24}}. {icon, "/images/myIcon.png"} {version,1} {myKey1, ...} The keys codePath, version, and expiryDate are obligatory Why do we need code path? - so that Erlang can find a module called C.erl Expiry date has a "time to live for the component" Version is used for local configuration data (see later) Version numbers should start at one and be increased by one for each new release. Rule5: Local configuration data Local configuration data must not be stored under the eComponents directory - (you can't remember we said the component is in a read-only disk area - see rule2) Local data for the component C must be stored in the directory $HOME/eLibrary/ComponentName/Vsn Thus local preferences for ejabberd version 1 would be stored in the file $HOME/eLibrary/ejabberd/1/Preferences.pl Rule6: Code upgrade We should upgrade an component C when it's expiry date has been reached. To update an component we delete the entire component under $HOME/eComponents/C.ec we install the new component and run the command: C:install(). Data that is to be carried over between different versions of the component is stored in $HOME/eLibrary/C/V/... Rule7: management All components C must provide a module in the file C_control.erl - so ejabberd provides ejabberd_control.erl The management API is as follows: C:start_link() -> Pid create a controller process for the component. Pid has the following protocol [see note 1 for notation] PL is a property list (list of {Key::atom(), Value::any()} Pid !! {start, PL} => ok | {error, Why} Cold start the component. PL is a property list describing how the component should behave Pid !! {stop, PL} => ok Stop the component Pid !! {modify, PL} => ok | {error, Why} Change the behavior of the component Pid !! info => PL Pid !! {info, [Keys]} => [Values] Pid !! suspendLocal => ok Pid !! resumeLocal => ok Suspend the component data can be written to $HOME/eLibrary/ComponentName/Vsn/... Pid !! suspendRemote => <> Pid !! {resumeRemote, <>} => ok This ton suspend an component on one machine and resume it on another Notes: [1] Notation Pid !! M => A | B | C ... Is my notation for for a remote procedure call This means if you send Pid the message Pid ! {self(), M} you will eventually be send one of the messages {Pid,A} or {Pid,B}, ... /Joe From frank@REDACTED Wed Nov 25 11:55:39 2009 From: frank@REDACTED (Frank Mueller) Date: Wed, 25 Nov 2009 11:55:39 +0100 (CET) Subject: [erlang-questions] A Generic API for controlling software components In-Reply-To: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> References: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> Message-ID: <8bca773cc03a3c8e10050c702f30116c.squirrel@alpha.mworld.biz> Hi Joe, your idea sounds really good to me, even if I would like some changes. > Draft 1 - 25 Nov 2009 > > Rule1: All components are unpacked into the same top-level > directory (default $HOME/eComponent) and have the extension > .ec (Erlang component) > > Example: Imagine I have installed mochiweb ejabberd and couchDB > then after installation I should see the following: > > $pwd > /home/joe/eComponents > $ls > mochiweb.ec ejabberd.ec couchDB.ec It's OK to have a default value for the directory, but I would like to have a defined variable named $ECOMPONENT_HOME so that I can easily change it. Rule 2 and 3 full ack. > Rule4: All components C must have a file called > > $HOME/eComponents/C.ec/Preferences.pl > > The extension .pl means the file contains a property > list. Here is an example: > > {codePath, "/bin"}. > {expiryDate, {2009,12,24}}. > {icon, "/images/myIcon.png"} > {version,1} > {myKey1, ...} > > The keys codePath, version, and expiryDate are obligatory > > Why do we need code path? - so that Erlang can find > a module called C.erl > > Expiry date has a "time to live for the component" > Version is used for local configuration data (see later) > Version numbers should start at one and be increased by one > for each new release. Here I would like to have a consistent naming style. Until today most parts of Erlang don't use camel case. I like it in Smalltalk, and I like the consistency inside Scheme, and until today I like the consistency in Erlang. It's not like Python, where you can see which libs are from the Smalltalk or Java side, from the .NET side or the C side. Awful. So it should be called preferences.pl with code_path, expiry_date, and so on. Additionally I would like nested preferences. My Tideland CEL celcfg module handles this well. > Rule5: Local configuration data > > Local configuration data must not be stored under the > eComponents directory - (you can't remember we said the > component is in a read-only disk area - see rule2) > > Local data for the component C must be stored > in the directory > > $HOME/eLibrary/ComponentName/Vsn > > Thus local preferences for ejabberd version 1 > would be stored in the file > > $HOME/eLibrary/ejabberd/1/Preferences.pl Ack, but see comments above about env variable and naming. Rule 6 and 7 again ack. Thx for this idea, Joe. I would like to make my Tideland EAS conforming to such a standard. mue -- ** ** Frank Mueller / Oldenburg / Germany ** From erlang@REDACTED Wed Nov 25 12:54:16 2009 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 25 Nov 2009 12:54:16 +0100 Subject: [erlang-questions] A Generic API for controlling software components In-Reply-To: <8bca773cc03a3c8e10050c702f30116c.squirrel@alpha.mworld.biz> References: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> <8bca773cc03a3c8e10050c702f30116c.squirrel@alpha.mworld.biz> Message-ID: <9b08084c0911250354o408ac896p87a2a85cc612365d@mail.gmail.com> On Wed, Nov 25, 2009 at 11:55 AM, Frank Mueller wrote: > Hi Joe, > > your idea sounds really good to me, even if I would like some changes. > >> ? ? Draft 1 - 25 Nov 2009 >> >> ? ? Rule1: All components are unpacked into the same top-level >> ? ? ? ? ? ?directory (default $HOME/eComponent) and have the extension >> ? ? ? ? ? ?.ec (Erlang component) >> >> ? ? ? ? ? ?Example: Imagine I have installed mochiweb ejabberd and couchDB >> ? ? ? ? ?then after installation I should see the following: >> >> ? ? ? ? ?$pwd >> ? ? ? ? ? ?/home/joe/eComponents >> ? ? ? ? ? ?$ls >> ? ? ? ? ? ?mochiweb.ec ejabberd.ec couchDB.ec > > It's OK to have a default value for the directory, but I would like to have a defined variable > named $ECOMPONENT_HOME so that I can easily change it. Yes > > Rule 2 and 3 full ack. > >> ? ? Rule4: All components C must have a file called >> >> ? ? ? ? ? ?$HOME/eComponents/C.ec/Preferences.pl >> >> ? ? ? ? ?The extension .pl means the file contains a property >> ? ? ? ? ? ?list. Here is an example: >> >> ? ? ? ? ? ?{codePath, "/bin"}. >> ? ? ? ? ? ?{expiryDate, {2009,12,24}}. >> ? ? ? ? ?{icon, "/images/myIcon.png"} >> ? ? ? ? ?{version,1} >> ? ? ? ? ? ?{myKey1, ...} >> >> ? ? ? ? ? ?The keys codePath, version, and expiryDate are obligatory >> >> ? ? ? ? ? ?Why do we need code path? - so that Erlang can find >> ? ? ? ? ? ?a module called C.erl >> >> ? ? ? ? ? ?Expiry date has a "time to live for the component" >> ? ? ? ? ? ?Version is used for local configuration data (see later) >> ? ? ? ? ? ?Version numbers should start at one and be increased by one >> ? ? ? ? ? ?for each new release. > > Here I would like to have a consistent naming style. Until today most parts of Erlang don't use > camel case. I like it in Smalltalk, and I like the consistency inside Scheme, and until today I > like the consistency in Erlang. It's not like Python, where you can see which libs are from the > Smalltalk or Java side, from the .NET side or the C side. Awful. So it should be called > preferences.pl with code_path, expiry_date, and so on. Additionally I would like nested > preferences. My Tideland CEL celcfg module handles this well. I use names_like_this for function and module names but namesLikeThis for atoms in code >> ? ? Rule5: Local configuration data >> >> ? ? ? ? ? ?Local configuration data must not be stored under the >> ? ? ? ? ? ?eComponents directory - (you can't remember we said the >> ? ? ? ? ? ?component is in a read-only disk area - see rule2) >> >> ? ? ? ? ? ?Local data for the component C must be stored >> ? ? ? ? ? ?in the directory >> >> ? ? ? ? ? ?$HOME/eLibrary/ComponentName/Vsn >> >> ? ? ? ? ?Thus local preferences for ejabberd version 1 >> ? ? ? ? ?would be stored in the file >> >> ? ? ? ? ?$HOME/eLibrary/ejabberd/1/Preferences.pl > > Ack, but see comments above about env variable and naming. > > Rule 6 and 7 again ack. > > Thx for this idea, Joe. I would like to make my Tideland EAS conforming to such a standard. Great - I'm going to write a generic manager to support this. But I'd like some discussion in the list first. > > mue > > -- > ** > ** Frank Mueller / Oldenburg / Germany > ** > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ttmrichter@REDACTED Wed Nov 25 13:34:03 2009 From: ttmrichter@REDACTED (Michael Richter) Date: Wed, 25 Nov 2009 20:34:03 +0800 Subject: [erlang-questions] A Generic API for controlling software components In-Reply-To: <9b08084c0911250354o408ac896p87a2a85cc612365d@mail.gmail.com> References: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> <8bca773cc03a3c8e10050c702f30116c.squirrel@alpha.mworld.biz> <9b08084c0911250354o408ac896p87a2a85cc612365d@mail.gmail.com> Message-ID: 2009/11/25 Joe Armstrong > >> Rule4: All components C must have a file called > >> > >> $HOME/eComponents/C.ec/Preferences.pl > >> > >> The extension .pl means the file contains a property > >> list. Here is an example: > >> > >> {codePath, "/bin"}. > >> {expiryDate, {2009,12,24}}. > >> {icon, "/images/myIcon.png"} > >> {version,1} > >> {myKey1, ...} > >> > >> The keys codePath, version, and expiryDate are obligatory > >> > >> Why do we need code path? - so that Erlang can find > >> a module called C.erl > >> > >> Expiry date has a "time to live for the component" > >> Version is used for local configuration data (see later) > >> Version numbers should start at one and be increased by one > >> for each new release. > > > > Here I would like to have a consistent naming style. Until today most > parts of Erlang don't use > > camel case. I like it in Smalltalk, and I like the consistency inside > Scheme, and until today I > > like the consistency in Erlang. It's not like Python, where you can see > which libs are from the > > Smalltalk or Java side, from the .NET side or the C side. Awful. So it > should be called > > preferences.pl with code_path, expiry_date, and so on. Additionally I > would like nested > > preferences. My Tideland CEL celcfg module handles this well. > > I use names_like_this for function and module names but namesLikeThis > for atoms in code > The idea is sound. The execution is suboptimal here. The ending *.pl* is embedded in the minds of a very large number of people as a perl script. Choose perhaps a longer, more descriptive extension like, say, .proplist or .propl or .plist or whatever? From kagato@REDACTED Wed Nov 25 13:40:21 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Wed, 25 Nov 2009 04:40:21 -0800 Subject: [erlang-questions] A Generic API for controlling software components In-Reply-To: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> References: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> Message-ID: First, let me take a stab at describing the goal (or what I think is and should be the goal) of components. It seems that you're trying to address: 1. Unifying execution / configuration conventions across popular Erlang projects. 2. Putting some sort of management conventions around deployments of said products. So, I don't think you're going to get a lot of traction on #2 unless it makes deployment easier. Right now, those programs all have ad-hoc conventions precisely because Erlang's deployment conventions, while rock solid, are nearly impossible to use outside of an embedded environment. If you really want to make things better (and I'm sure you do), here's what I suggest doing. Don't create yet-another-packaging-abstraction (i.e. components). Applications are perfectly fine for this. Really. Address the issues that make it hard to deploy Erlang applications. Specifically, applications should probably be deployed as a set of modules in a second LIBDIR, that shadows the existing one, thereby reducing the number of duplicates of the entire environment (which is a current problem with Erlang's release stuff). I already do this, and it's great. Address issues that make deploying applications hard, in general. The CLI itch is a good one to scratch. Make it trivial to write a CLI event handler. Make it a normal gen_event. Maybe require a key in the app config (maybe {cli_handler,Module}) or perhaps just have a convention (something like an atom of the form '_cli'). For that matter, make a distribution container that is more like a gem/egg. I think this is really want you want. Installing an Erlang application should be as easy as commands something like: # epkg install rabbitmq ...downloads, builds, and installs rabbitmq code somewhere sane... # epkg deploy rabbitmq mydeploy ...generates a "deployment" somewhere, basically just config and state data, mydeploy is used to generate nodenames (i.e. mydeploy_rabbitmq_main, mydeploy_rabbitmq_helper, etc)... # ectl mydeploy start ...starts nodes specified for the deploy... # ectl mydeploy flush_cache ...since flush_cache is not implemented by ectl, it should be passed through to an event handler... Obviously there's a lot to go on under the covers, but anything more complex than that is, at best, a well engineered user-experience failure. More importantly, I think the above captures what you should be doing. All of that said, here are my comments on the proposal, so far (most of which I've already stated above): > What I would like to do is manage all the components in a uniform manner. > Much of this design is inspired by how Mac OS-X manages applications. Why not just use the application framework, again? It's 90% there for this. Is there anything about an application that makes it unsuitable (i.e. smaller than) a component? I would just add this to an application. > By manage I mean start and stop the component, upgrade the code, > change the behavior of the component at run time and so on. I think you're thinking in terms of actual user-level management, right? Basically, this is to unify things like ejabberdctl and rabbitmqctl. I think that's a fantastic idea. Looking forward, I'd suggest taking a hard look at Ruby gems and Python distutils/setuputils/eggs. I really think that it would make a lot of sense to take the existing Erlang release system and use it to make something not unlike a gem or egg (which it already kind of does, but at a whole-Erlang-system level, not a component level). As for the command line, it would be extremely handy to be able to have a convention where the nodename of a certain component was known, and there was a command line wrapper to send it commands. Perhaps, have the component run a gen_event, and have "erlctl [args...]" generate an event in the node that the component runs in. Extending edoc to generate commandline documentation would be killer here, too. > Here's a suggestion for a set of rules for managed components: > > Draft 1 - 25 Nov 2009 > > Rule1: All components are unpacked into the same top-level > directory (default $HOME/eComponent) and have the extension > .ec (Erlang component) > > Example: Imagine I have installed mochiweb ejabberd and couchDB > then after installation I should see the following: > > $pwd > /home/joe/eComponents > $ls > mochiweb.ec ejabberd.ec couchDB.ec I don't think eComponents, as a non-hidden top-level directory is going to thrill the aesthetic sense of most Unix users. I would humbly suggest something more like $HOME/.erlang/components/. Or, better yet, a system directory (probably just in the Erlang code root) and a user directory ($HOME/.erlang/lib). > Rule2: A normal user should *never* have to examine any of the > files *inside* the component. To do so causes > "abstraction leakage" - I want to consider each of these > components as black boxes. Once installed the component > should be installed in a read-only disk area. Certainly. > > Rule3: The component should not break if relocated to a different > top-level directory. A simple "mv" command should suffice > So no hard-wired paths or links please. Certainly. > Rule4: All components C must have a file called > > $HOME/eComponents/C.ec/Preferences.pl > > The extension .pl means the file contains a property > list. Here is an example: .pl is bad mojo, as it's universally accepts to be Perl. How about, .epl (i.e. Erlang Property List). > > {codePath, "/bin"}. > {expiryDate, {2009,12,24}}. > {icon, "/images/myIcon.png"} > {version,1} > {myKey1, ...} > > The keys codePath, version, and expiryDate are obligatory > > Why do we need code path? - so that Erlang can find > a module called C.erl If the code path is relative, what is gained by having it specifiable? Why not just have a convention that it's ebin (or maybe have component/lib be added as an additional LIBDIR). > Expiry date has a "time to live for the component" > Version is used for local configuration data (see later) > Version numbers should start at one and be increased by one > for each new release. The expiry time thing is confusing to me. Is this automatically done by some program, or is it metadata for a person? > Rule5: Local configuration data > > Local configuration data must not be stored under the > eComponents directory - (you can't remember we said the > component is in a read-only disk area - see rule2) > > Local data for the component C must be stored > in the directory > > $HOME/eLibrary/ComponentName/Vsn > > Thus local preferences for ejabberd version 1 > would be stored in the file > > $HOME/eLibrary/ejabberd/1/Preferences.pl Again, I humbly suggest more traditional Unix pathnames. How about $HOME/.erlang/library///prefs.epl? > Rule6: Code upgrade > > We should upgrade an component C when it's expiry date has > been reached. To update an component we delete the entire > component under $HOME/eComponents/C.ec we install the > new component and run the command: C:install(). > > Data that is to be carried over between different versions > of the component is stored in $HOME/eLibrary/C/V/... Is this done by the component framework? Is this really a good idea to be done automatically? Automatically deployed upgrades have generally been associated with tears and gnashing of teeth in my experience. > Rule7: management > > All components C must provide a module in the file > C_control.erl - so ejabberd provides ejabberd_control.erl > > The management API is as follows: > > C:start_link() -> Pid > > create a controller process for the component. > > Pid has the following protocol [see note 1 for notation] > PL is a property list (list of {Key::atom(), Value::any()} > > Pid !! {start, PL} => ok | {error, Why} > > Cold start the component. PL is a property > list describing how the component should behave > > Pid !! {stop, PL} => ok > > Stop the component > > Pid !! {modify, PL} => ok | {error, Why} > Change the behavior of the component > > Pid !! info => PL > Pid !! {info, [Keys]} => [Values] > > Pid !! suspendLocal => ok > Pid !! resumeLocal => ok > > Suspend the component data can be written to > $HOME/eLibrary/ComponentName/Vsn/... > > Pid !! suspendRemote => <> > Pid !! {resumeRemote, <>} => ok > > This ton suspend an component on one machine > and resume it on another This looks a lot like it could just use the existing application framework. I don't think that most of this would even require breaking backwards-compatibility. -- Jayson Vantuyl kagato@REDACTED From frank@REDACTED Wed Nov 25 13:41:16 2009 From: frank@REDACTED (Frank Mueller) Date: Wed, 25 Nov 2009 13:41:16 +0100 (CET) Subject: [erlang-questions] A Generic API for controlling software components In-Reply-To: References: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> <8bca773cc03a3c8e10050c702f30116c.squirrel@alpha.mworld.biz> <9b08084c0911250354o408ac896p87a2a85cc612365d@mail.gmail.com> Message-ID: > The idea is sound. The execution is suboptimal here. The ending *.pl* is > embedded in the minds of a very large number of people as a perl script. > Choose perhaps a longer, more descriptive extension like, say, .proplist or > .propl or .plist or whatever? .plist is used in OS X. So I would vite for .proplist. mue -- ** ** Frank Mueller / Oldenburg / Germany ** From andrewmmc@REDACTED Wed Nov 25 14:07:21 2009 From: andrewmmc@REDACTED (andrew mmc) Date: Wed, 25 Nov 2009 14:07:21 +0100 Subject: mnesia:transform_table Message-ID: Hello, I needed to change the structure of an mnesia table. I followed the example in the documentation: -record(old, {key, val}). -record(new, {key, val, extra}). Transformer = fun(X) when record(X, old) -> #new{key = X#old.key, val = X#old.val, extra = 42} end, {atomic, ok} = mnesia:transform_table(foo, Transformer, record_info(fields, new), new) Substituting the record names and field mappings as per my requirement. The transformation function works fine when I test it on a record of the old type, outputting the new record ok. However, now when I try to read from the new table, mnesia reports no_exists, and the old table still exists but reads as empty... Where has my data gone?! Any help much appreciated! Regards, Andrew From erlang@REDACTED Wed Nov 25 14:12:55 2009 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 25 Nov 2009 14:12:55 +0100 Subject: [erlang-questions] A Generic API for controlling software components In-Reply-To: References: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> Message-ID: <9b08084c0911250512p1b80ee84w96f840ab6007e844@mail.gmail.com> On Wed, Nov 25, 2009 at 1:40 PM, Jayson Vantuyl wrote: > First, let me take a stab at describing the goal (or what I think is and should be the goal) of components. > > It seems that you're trying to address: > > 1. ?Unifying execution / configuration conventions across popular Erlang projects. > 2. ?Putting some sort of management conventions around deployments of said products. > > So, I don't think you're going to get a lot of traction on #2 unless it makes deployment easier. ?Right now, those programs all have ad-hoc conventions precisely because Erlang's deployment conventions, while rock solid, are nearly impossible to use outside of an embedded environment. > > If you really want to make things better (and I'm sure you do), here's what I suggest doing. > > Don't create yet-another-packaging-abstraction (i.e. components). ?Applications are perfectly fine for this. ?Really. > I guess you mean OTP applications? - If so then I disagree. To start with the directory structure within an OTP application is fixed which I don't really like. Also there is no construction rule that says you are not allowed to write anything in the application directory tree. I'd like to put the entire component in a read-only area of disk for safety reasons (once an initial install has been performed). Some people want to put components into read-only memory for embedded devices - it also makes it easier to scan things for security reasons. > Address the issues that make it hard to deploy Erlang applications. ?Specifically, applications should probably be deployed as a set of modules in a second LIBDIR, that shadows the existing one, thereby reducing the number of duplicates of the entire environment (which is a current problem with Erlang's release stuff). ?I already do this, and it's great. I have a differnt proposal for reducing duplicates - this is a different problem. Yes to deploying Erlang components *and upgrading them* > > Address issues that make deploying applications hard, in general. ?The CLI itch is a good one to scratch. ?Make it trivial to write a CLI event handler. ?Make it a normal gen_event. ?Maybe require a key in the app config (maybe {cli_handler,Module}) or perhaps just have a convention (something like an atom of the form '_cli'). ?For that matter, make a distribution container that is more like a gem/egg. > > I think this is really want you want. ?Installing an Erlang application should be as easy as commands something like: > > # epkg install rabbitmq > ...downloads, builds, and installs rabbitmq code somewhere sane... > # epkg deploy rabbitmq mydeploy > ...generates a "deployment" somewhere, basically just config and state data, mydeploy is used to generate nodenames (i.e. mydeploy_rabbitmq_main, mydeploy_rabbitmq_helper, etc)... > # ectl mydeploy start > ...starts nodes specified for the deploy... > # ectl mydeploy flush_cache > ...since flush_cache is not implemented by ectl, it should be passed through to an event handler... Sure. epkg etc. will be just a thin shell script layer over the API I suggested. The CLI can be written as a thin layer over the API as well. > > Obviously there's a lot to go on under the covers, but anything more complex than that is, at best, a well engineered user-experience failure. ?More importantly, I think the above captures what you should be doing. > > All of that said, here are my comments on the proposal, so far (most of which I've already stated above): > >> What I would like to do is manage all the components in a uniform manner. >> Much of this design is inspired by how Mac OS-X manages applications. > Why not just use the application framework, again? ?It's 90% there for this. because 90% is not enough > Is there anything about an application that makes it unsuitable (i.e. smaller than) a component? ?I would just add this to an application. > >> By manage I mean start and stop the component, upgrade the code, >> change the behavior of the component at run time and so on. > I think you're thinking in terms of actual user-level management, right? ?Basically, this is to unify things like ejabberdctl and rabbitmqctl. ?I think that's a fantastic idea. Of course. I just want to download yaws, ejabberd etc. drop them into my eComponent Directory and that's it - they are installed and runnable. > > Looking forward, I'd suggest taking a hard look at Ruby gems and Python distutils/setuputils/eggs. ?I really think that it would make a lot of sense to take the existing Erlang release system and use it to make something not unlike a gem or egg (which it already kind of does, but at a whole-Erlang-system level, not a component level). > > As for the command line, it would be extremely handy to be able to have a convention where the nodename of a certain component was known, and there was a command line wrapper to send it commands. ?Perhaps, have the component run a gen_event, and have "erlctl [args...]" generate an event in the node that the component runs in. ?Extending edoc to generate commandline documentation would be killer here, too. > >> Here's a suggestion for a set of rules for managed components: >> >> ? ?Draft 1 - 25 Nov 2009 >> >> ? ?Rule1: All components are unpacked into the same top-level >> ? ? ? ? ? directory (default $HOME/eComponent) and have the extension >> ? ? ? ? ? .ec (Erlang component) >> >> ? ? ? ? ? Example: Imagine I have installed mochiweb ejabberd and couchDB >> ? ? ? ? ?then after installation I should see the following: >> >> ? ? ? ? ?$pwd >> ? ? ? ? ? /home/joe/eComponents >> ? ? ? ? ? $ls >> ? ? ? ? ? mochiweb.ec ejabberd.ec couchDB.ec > I don't think eComponents, as a non-hidden top-level directory is going to thrill the aesthetic sense of most Unix users. ?I would humbly suggest something more like $HOME/.erlang/components/. ?Or, better yet, a system directory (probably just in the Erlang code root) and a user directory ($HOME/.erlang/lib). Ok, I didn't fully specify this - fine to change the details here or use an environment variable to point to the directory. > >> ? ?Rule2: A normal user should *never* have to examine any of the >> ? ? ? ? ? files *inside* the component. To do so causes >> ? ? ? ? ? "abstraction leakage" - I want to consider each of these >> ? ? ? ? ? components as black boxes. Once installed the component >> ? ? ? ? ?should be installed in a read-only disk area. > Certainly. This is the bit that is not enforced in OTP applications - I think users are even encouraged to use the /priv directory in an otp application > >> >> ? ?Rule3: The component should not break if relocated to a different >> ? ? ? ? ? top-level directory. A simple "mv" command should suffice >> ? ? ? ? ?So no hard-wired paths or links please. > Certainly. > >> ? ?Rule4: All components C must have a file called >> >> ? ? ? ? ? $HOME/eComponents/C.ec/Preferences.pl >> >> ? ? ? ? ?The extension .pl means the file contains a property >> ? ? ? ? ? list. Here is an example: > .pl is bad mojo, as it's universally accepts to be Perl. ?How about, .epl (i.e. Erlang Property List). Ok .eplist then? > >> >> ? ? ? ? ? {codePath, "/bin"}. >> ? ? ? ? ? {expiryDate, {2009,12,24}}. >> ? ? ? ? ?{icon, "/images/myIcon.png"} >> ? ? ? ? ?{version,1} >> ? ? ? ? ? {myKey1, ...} >> >> ? ? ? ? ? The keys codePath, version, and expiryDate are obligatory >> >> ? ? ? ? ? Why do we need code path? - so that Erlang can find >> ? ? ? ? ? a module called C.erl > If the code path is relative, what is gained by having it specifiable? ?Why not just have a convention that it's ebin (or maybe have component/lib be added as an additional LIBDIR). Because I might not want to follow that particular convention. For simple applications I usually put everything in one directory. Actually this could be dropped, the system could scan the filesystem to find the code paths the first time you run the program. directory > >> ? ? ? ? ? Expiry date has a "time to live for the component" >> ? ? ? ? ? Version is used for local configuration data (see later) >> ? ? ? ? ? Version numbers should start at one and be increased by one >> ? ? ? ? ? for each new release. > The expiry time thing is confusing to me. ?Is this automatically done by some program, or is it metadata for a person? It's a hint to tell the package manager how often to check for updates - I havn't thought through all the details it could be something else: {checkForUpdates, everyTimeYouStart | dayly | hourly | weekly | {after,Year,Month,Day} | {every, 10, minutes}} etc. The designer of the program knows what is a sensible value here > >> ? ?Rule5: Local configuration data >> >> ? ? ? ? ? Local configuration data must not be stored under the >> ? ? ? ? ? eComponents directory - (you can't remember we said the >> ? ? ? ? ? component is in a read-only disk area - see rule2) >> >> ? ? ? ? ? Local data for the component C must be stored >> ? ? ? ? ? in the directory >> >> ? ? ? ? ? $HOME/eLibrary/ComponentName/Vsn >> >> ? ? ? ? ?Thus local preferences for ejabberd version 1 >> ? ? ? ? ?would be stored in the file >> >> ? ? ? ? ?$HOME/eLibrary/ejabberd/1/Preferences.pl > Again, I humbly suggest more traditional Unix pathnames. ?How about $HOME/.erlang/library///prefs.epl? No worries ... > >> ? ? Rule6: Code upgrade >> >> ? ? ? ? ? We should upgrade an component C when it's expiry date has >> ? ? ? ? ? been reached. To update an component we delete the entire >> ? ? ? ? ? component under $HOME/eComponents/C.ec we install the >> ? ? ? ? ? new component and run the command: C:install(). >> >> ? ? ? ? ? Data that is to be carried over between different versions >> ? ? ? ? ? of the component is stored in $HOME/eLibrary/C/V/... > Is this done by the component framework? ?Is this really a good idea to be done automatically? ?Automatically deployed upgrades have generally been associated with tears and gnashing of teeth in my experience. I guess the component framework would use the check for updates or expiry date or whatever to decide whether to do a check. The check would return a list of improvements and the user would decide whether or not to install them. I guess we should also be able to roll back a version (which is why I have eLibrary/ComponentName/VSN/data .. tags. > >> ? ? Rule7: management >> >> ? ? ? ? All components C must provide a module in the file >> ? ? ? ? C_control.erl - so ejabberd provides ejabberd_control.erl >> >> ? ? ? ? The management API is as follows: >> >> ? ? ? ? C:start_link() -> Pid >> >> ? ? ? ? ? ?create a controller process for the component. >> >> ? ? ? ? Pid has the following protocol [see note 1 for notation] >> ? ? ? ? PL is a property list (list of {Key::atom(), Value::any()} >> >> ? ? ? ? ?Pid !! {start, PL} => ok | {error, Why} >> >> ? ? ? ? ? ? Cold start the component. PL is a property >> ? ? ? ? ? ? ?list describing how the component should behave >> >> ? ? ? ? Pid !! {stop, PL} => ok >> >> ? ? ? ? ? ? Stop the component >> >> ? ? ? ? Pid !! {modify, PL} => ok | {error, Why} >> ? ? ? ? ? ? Change the behavior of the component >> >> ? ? ? ? Pid !! info => PL >> ? ? ? ? Pid !! {info, [Keys]} => [Values] >> >> ? ? ? ? Pid !! suspendLocal => ok >> ? ? ? ? Pid !! resumeLocal => ok >> >> ? ? ? ? ? ? ? Suspend the component data can be written to >> ? ? ? ? ? ? ? $HOME/eLibrary/ComponentName/Vsn/... >> >> ? ? ? ? ?Pid !! suspendRemote => <> >> ? ? ? ? ?Pid !! {resumeRemote, <>} => ok >> >> ? ? ? ? ? ? ? ?This ton suspend an component on one machine >> ? ? ? ? ? ? ? ?and resume it on another > > This looks a lot like it could just use the existing application framework. ?I don't think that most of this would even require breaking backwards-compatibility. It is pretty similar /Joe > > > -- > Jayson Vantuyl > kagato@REDACTED > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From hawk.mattsson@REDACTED Wed Nov 25 14:40:21 2009 From: hawk.mattsson@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Wed, 25 Nov 2009 14:40:21 +0100 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: References: Message-ID: <922d05850911250540x6e63e2b3vd7c0fb8573339937@mail.gmail.com> It looks like you have done the right thing. Perhaps you have simplified your code too much in order to post it on the list? I tried to reproduce your error, but it worked for me. See the printouts and the test program below. /H?kan 16> foo:go(foo). Before: [{old,7,49}, {old,6,36}, {old,4,16}, {old,1,1}, {old,9,81}, {old,10,100}, {old,8,64}, {old,2,4}, {old,3,9}, {old,5,25}] Before: [{new,7,49,42}, {new,6,36,42}, {new,4,16,42}, {new,1,1,42}, {new,9,81,42}, {new,10,100,42}, {new,8,64,42}, {new,2,4,42}, {new,3,9,42}, {new,5,25,42}] ok -module(foo). -compile([export_all]). -record(old, {key, val}). -record(new, {key, val, extra}). go(Tab) -> mnesia:delete_table(Tab), {atomic, ok} = mnesia:create_table(Tab, [{attributes, record_info(fields, old)}, {record_name, old}]), [mnesia:dirty_write(Tab, #old{key = K, val = K*K}) || K <- lists:seq(1,10)], io:format("Before: ~p\n", [ets:tab2list(Tab)]), Transformer = fun(#old{key = Key, val = Val}) -> #new{key = Key, val = Val, extra = 42} end, {atomic, ok} = mnesia:transform_table(Tab, Transformer, record_info(fields, new), new), io:format("Before: ~p\n", [ets:tab2list(Tab)]). On Wed, Nov 25, 2009 at 2:07 PM, andrew mmc wrote: > Hello, > > I needed to change the structure of an mnesia table. I followed the example > in the documentation: > > -record(old, {key, val}). > -record(new, {key, val, extra}). > > Transformer = > fun(X) when record(X, old) -> > #new{key = X#old.key, > val = X#old.val, > extra = 42} > end, > {atomic, ok} = mnesia:transform_table(foo, Transformer, > record_info(fields, new), > new) > > Substituting the record names and field mappings as per my > requirement. The transformation function works fine when I test it on > a record of the old type, outputting the new record ok. > > However, now when I try to read from the new table, mnesia reports > no_exists, and the old table still exists but reads as empty... Where > has my data gone?! > > Any help much appreciated! > > Regards, > > > Andrew On Wed, Nov 25, 2009 at 2:07 PM, andrew mmc wrote: > Hello, > > I needed to change the structure of an mnesia table. ?I followed the example > in the documentation: > > -record(old, {key, val}). > -record(new, {key, val, extra}). > > Transformer = > ? fun(X) when record(X, old) -> > ? ? ?#new{key = X#old.key, > ? ? ? ? ? val = X#old.val, > ? ? ? ? ? extra = 42} > ? end, > {atomic, ok} = mnesia:transform_table(foo, Transformer, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?record_info(fields, new), > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?new) > > Substituting the record names and field mappings as per my > requirement. The transformation function works fine when I test it on > a record of the old type, outputting the new record ok. > > However, now when I try to read from the new table, mnesia reports > no_exists, and the old table still exists but reads as empty... ?Where > has my data gone?! > > Any help much appreciated! > > Regards, > > > Andrew From andrewmmc@REDACTED Wed Nov 25 14:47:39 2009 From: andrewmmc@REDACTED (andrew mmc) Date: Wed, 25 Nov 2009 14:47:39 +0100 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: <922d05850911250540x6e63e2b3vd7c0fb8573339937@mail.gmail.com> References: <922d05850911250540x6e63e2b3vd7c0fb8573339937@mail.gmail.com> Message-ID: Hi H?kan, Thanks for your answer... Actually I see that it has worked - I expected the table name to change with the record name, but the name is the same as before. I got an empty table because I used dirty_match_object without specifying the 'old' table name. So I guess the next question, is: how do I change the table name!? Thanks, Andrew 2009/11/25 H?kan Mattsson > It looks like you have done the right thing. Perhaps you have > simplified your code too much in order to post it on the list? > > I tried to reproduce your error, but it worked for me. > See the printouts and the test program below. > > /H?kan > > 16> foo:go(foo). > Before: [{old,7,49}, > {old,6,36}, > {old,4,16}, > {old,1,1}, > {old,9,81}, > {old,10,100}, > {old,8,64}, > {old,2,4}, > {old,3,9}, > {old,5,25}] > Before: [{new,7,49,42}, > {new,6,36,42}, > {new,4,16,42}, > {new,1,1,42}, > {new,9,81,42}, > {new,10,100,42}, > {new,8,64,42}, > {new,2,4,42}, > {new,3,9,42}, > {new,5,25,42}] > ok > > > -module(foo). > -compile([export_all]). > > -record(old, {key, val}). > -record(new, {key, val, extra}). > > go(Tab) -> > mnesia:delete_table(Tab), > {atomic, ok} = mnesia:create_table(Tab, [{attributes, > record_info(fields, old)}, > {record_name, old}]), > [mnesia:dirty_write(Tab, #old{key = K, val = K*K}) || K <- > lists:seq(1,10)], > io:format("Before: ~p\n", [ets:tab2list(Tab)]), > Transformer = > fun(#old{key = Key, val = Val}) -> > #new{key = Key, val = Val, extra = 42} > end, > {atomic, ok} = mnesia:transform_table(Tab, Transformer, > record_info(fields, new), > new), > io:format("Before: ~p\n", [ets:tab2list(Tab)]). > > > > On Wed, Nov 25, 2009 at 2:07 PM, andrew mmc wrote: > > Hello, > > > > I needed to change the structure of an mnesia table. I followed the > example > > in the documentation: > > > > -record(old, {key, val}). > > -record(new, {key, val, extra}). > > > > Transformer = > > fun(X) when record(X, old) -> > > #new{key = X#old.key, > > val = X#old.val, > > extra = 42} > > end, > > {atomic, ok} = mnesia:transform_table(foo, Transformer, > > record_info(fields, new), > > new) > > > > Substituting the record names and field mappings as per my > > requirement. The transformation function works fine when I test it on > > a record of the old type, outputting the new record ok. > > > > However, now when I try to read from the new table, mnesia reports > > no_exists, and the old table still exists but reads as empty... Where > > has my data gone?! > > > > Any help much appreciated! > > > > Regards, > > > > > > Andrew > > On Wed, Nov 25, 2009 at 2:07 PM, andrew mmc wrote: > > Hello, > > > > I needed to change the structure of an mnesia table. I followed the > example > > in the documentation: > > > > -record(old, {key, val}). > > -record(new, {key, val, extra}). > > > > Transformer = > > fun(X) when record(X, old) -> > > #new{key = X#old.key, > > val = X#old.val, > > extra = 42} > > end, > > {atomic, ok} = mnesia:transform_table(foo, Transformer, > > record_info(fields, new), > > new) > > > > Substituting the record names and field mappings as per my > > requirement. The transformation function works fine when I test it on > > a record of the old type, outputting the new record ok. > > > > However, now when I try to read from the new table, mnesia reports > > no_exists, and the old table still exists but reads as empty... Where > > has my data gone?! > > > > Any help much appreciated! > > > > Regards, > > > > > > Andrew > From bgustavsson@REDACTED Wed Nov 25 14:49:50 2009 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 25 Nov 2009 14:49:50 +0100 Subject: Erlang/OTP R13B03 has been released Message-ID: <6672d0160911250549t18e7416cre4e953d51ed83d0f@mail.gmail.com> Bug fix release : otp_src_R13B03 Build date : 2009-11-23 This is R13B03, the third maintenance release for the R13B major release. You can find the README file for the release at http://www.erlang.org/download/otp_src_R13B03.readme The source distribution and binary distribution for Windows can be downloaded from http://www.erlang.org/download/otp_src_R13B03.tar.gz http://www.erlang.org/download/otp_win32_R13B03.exe The distribution can also be downloaded using the BitTorrent protocol. Use the following torrent files to download the source distribution and binary distribution for Windows: http://www.erlang.org/download/otp_src_R13B03.tar.gz.torrent http://www.erlang.org/download/otp_win32_R13B03.exe.torrent Note: To unpack the TAR archive you need a GNU TAR compatible program. For installation instructions please read the README file that is part of the distribution. The on-line documentation can be found at: http://www.erlang.org/doc/ You can also download the complete HTML documentation or the Unix manual files http://www.erlang.org/download/otp_doc_html_R13B03.tar.gz http://www.erlang.org/download/otp_doc_man_R13B03.tar.gz We also want to thank those that sent us patches, suggestions and bug reports, The OTP Team -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From hakan@REDACTED Wed Nov 25 14:56:42 2009 From: hakan@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Wed, 25 Nov 2009 14:56:42 +0100 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: References: <922d05850911250540x6e63e2b3vd7c0fb8573339937@mail.gmail.com> Message-ID: <922d05850911250556r7f7293aev6cf245196c16a8e9@mail.gmail.com> 2009/11/25 andrew mmc : > So I guess the next question, is: how do I change the table name!? It is not supported. As a last resort you can do it off-line by creating a backup, use mnesia:traverse_backup/8 to patch the backup and then restore the database from the backup. But if you would do that you could transform the records off-line also. /H?kan From andrewmmc@REDACTED Wed Nov 25 15:00:59 2009 From: andrewmmc@REDACTED (andrew mmc) Date: Wed, 25 Nov 2009 15:00:59 +0100 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: <922d05850911250556r7f7293aev6cf245196c16a8e9@mail.gmail.com> References: <922d05850911250540x6e63e2b3vd7c0fb8573339937@mail.gmail.com> <922d05850911250556r7f7293aev6cf245196c16a8e9@mail.gmail.com> Message-ID: That would be the reason I couldn't find it anywhere in the documentation then! I'll just live with the old table name, it seems easier! Thanks very much, Andrew 2009/11/25 H?kan Mattsson > 2009/11/25 andrew mmc : > > > So I guess the next question, is: how do I change the table name!? > > It is not supported. As a last resort you can do it off-line by creating a > backup, use mnesia:traverse_backup/8 to patch the backup and then > restore the database from the backup. But if you would do that you > could transform the records off-line also. > > /H?kan > From bgustavsson@REDACTED Wed Nov 25 15:01:24 2009 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 25 Nov 2009 15:01:24 +0100 Subject: Erlang/OTP now at Github Message-ID: <6672d0160911250601uad36706l570e036472d5d905@mail.gmail.com> The official git repository for Erlang/OTP can now be found at Github: http://github.com/erlang/otp Build instructions can be found at: http://wiki.github.com/erlang/otp We plan to add more wiki pages with additional information in the near future. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From freza@REDACTED Wed Nov 25 15:14:06 2009 From: freza@REDACTED (Jachym Holecek) Date: Wed, 25 Nov 2009 15:14:06 +0100 Subject: [erlang-questions] Erlang "object-oriented" after all? In-Reply-To: References: Message-ID: <20091125141406.GA1548@hanele> # Michael Turner 2009-11-24: > > I ran across a very interesting exchange with Alan Kay, who most likely > coined the term "object-oriented programming." > > http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/doc_kay_oop_en > > If you'll excuse some (only slightly tendentious) editing of his remarks: > > ---- > "I thought of objects being like biological cells and/or individual > computers on a network, only able to communicate with messages . . . . No, you didn't. If you did, you would certainly have noticed that individual cells/computers live concurrently and communicate asynchronously, which is not how your objects behave. :-) SCNR, I still remember the disappointment of discovering (ages back) that Objective-C objects didn't really fulfill the "they behave like real-world objects" promise given by the (otherwise pretty good) NeXT book because you couldn't "run objects in parallel" which sounded like something that should obviously be possible. Regards, -- Jachym From juanjo@REDACTED Wed Nov 25 15:20:14 2009 From: juanjo@REDACTED (Juan Jose Comellas) Date: Wed, 25 Nov 2009 11:20:14 -0300 Subject: [erlang-questions] Erlang/OTP now at Github In-Reply-To: <6672d0160911250601uad36706l570e036472d5d905@mail.gmail.com> References: <6672d0160911250601uad36706l570e036472d5d905@mail.gmail.com> Message-ID: <1c3be50f0911250620h22aa000ch5c089c4c20175a95@mail.gmail.com> Why was the commit history discarded? Wouldn't it be better to at least preserve the history corresponding to the public releases? Maybe you could use Matthew Foemmel's repository[1] in Github as a base. [1] http://github.com/mfoemmel/erlang-otp/ 2009/11/25 Bj?rn Gustavsson > > The official git repository for Erlang/OTP can now be found at Github: > > http://github.com/erlang/otp > > Build instructions can be found at: > > http://wiki.github.com/erlang/otp > > We plan to add more wiki pages with additional information in the > near future. > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From wael.ebrahim@REDACTED Wed Nov 25 15:21:27 2009 From: wael.ebrahim@REDACTED (Wael Ebrahim) Date: Wed, 25 Nov 2009 15:21:27 +0100 Subject: Question about erlang.log Message-ID: <4D95FB39404AA04CA998E7F63B1A2E5F03C3EAD6@esealmw110.eemea.ericsson.se> I have a question about Erlang log. I have a target system and I want to increase the size and number of log files named erlang.log.1 ....erlang.log.5 in the directory ROOT_DIR/log/ The default values are 5 files each one is 100K , but I had a very big crash that was repeated multiple times causing these files to be overwritten, and I couldn't reach its start. I know I can use sasl for this kind of monitoring, but it doesn't log the user input on the erlang shell as the erlang.log.x does. Also sasl doesn't log any output except from standard applications, or error_logger. Any ideas about increasing size of these files "erlang.log.x" ??? Thanks in advance, Wael Ebrahim From zambal@REDACTED Wed Nov 25 15:30:37 2009 From: zambal@REDACTED (zambal) Date: Wed, 25 Nov 2009 06:30:37 -0800 (PST) Subject: Erlang/OTP R13B03 has been released In-Reply-To: <6672d0160911250549t18e7416cre4e953d51ed83d0f@mail.gmail.com> References: <6672d0160911250549t18e7416cre4e953d51ed83d0f@mail.gmail.com> Message-ID: <27fc1683-3b9d-4d39-bb38-1424cd9dc47d@j14g2000yqm.googlegroups.com> Hi, Thanks a lot for another OTP release! I just wanted to say that the links to the torrent files are not correct. The correct links should be: http://www.erlang.org/download/torrents/otp_src_R13B03.tar.gz.torrent http://www.erlang.org/download/torrents/otp_win32_R13B03.exe.torrent - vincent From bgustavsson@REDACTED Wed Nov 25 17:00:46 2009 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 25 Nov 2009 17:00:46 +0100 Subject: [erlang-questions] Erlang/OTP now at Github In-Reply-To: <1c3be50f0911250620h22aa000ch5c089c4c20175a95@mail.gmail.com> References: <6672d0160911250601uad36706l570e036472d5d905@mail.gmail.com> <1c3be50f0911250620h22aa000ch5c089c4c20175a95@mail.gmail.com> Message-ID: <6672d0160911250800t2ed7b724m4c445ff5d99d1306@mail.gmail.com> On Wed, Nov 25, 2009 at 3:20 PM, Juan Jose Comellas wrote: > Why was the commit history discarded? Wouldn't it be better to at > least preserve the history corresponding to the public releases? Maybe > you could use Matthew Foemmel's repository[1] in Github as a base. Including the history would require approximately 300 Mb. However, we will probably provide an *optional* Erlang/OTP history that can be added using the new replace mechanism in git 1.6.5. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From rob@REDACTED Wed Nov 25 18:24:29 2009 From: rob@REDACTED (Rob Charlton) Date: Wed, 25 Nov 2009 17:24:29 +0000 Subject: [erlang-questions] Question about erlang.log In-Reply-To: <4D95FB39404AA04CA998E7F63B1A2E5F03C3EAD6@esealmw110.eemea.ericsson.se> References: <4D95FB39404AA04CA998E7F63B1A2E5F03C3EAD6@esealmw110.eemea.ericsson.se> Message-ID: <4B0D684D.3070807@erlang-consulting.com> Wael Ebrahim wrote: > I have a question about Erlang log. > > I have a target system and I want to increase the size and number of log > files named erlang.log.1 ....erlang.log.5 in the directory ROOT_DIR/log/ > > > The default values are 5 files each one is 100K , but I had a very big > crash that was repeated multiple times causing these files to be > overwritten, and I couldn't reach its start. > > I know I can use sasl for this kind of monitoring, but it doesn't log > the user input on the erlang shell as the erlang.log.x does. > Also sasl doesn't log any output except from standard applications, or > error_logger. > > Any ideas about increasing size of these files "erlang.log.x" ??? > > Thanks in advance, > > Wael Ebrahim > > > Before starting run_erl, you can set these environment variables: export RUN_ERL_LOG_GENERATIONS=10 export RUN_ERL_LOG_MAXSIZE=1000000 see http://www.erlang.org/doc/man/run_erl.html for more details Cheers Rob -- Erlang Training and Consulting Ltd www.erlang-consulting.com From kagato@REDACTED Wed Nov 25 22:50:03 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Wed, 25 Nov 2009 13:50:03 -0800 Subject: [erlang-questions] A Generic API for controlling software components In-Reply-To: <9b08084c0911250512p1b80ee84w96f840ab6007e844@mail.gmail.com> References: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> <9b08084c0911250512p1b80ee84w96f840ab6007e844@mail.gmail.com> Message-ID: <4081E7BC-939A-4E3C-A992-52848A6BE12C@souja.net> > I guess you mean OTP applications? - If so then I disagree. To start > with the directory structure within > an OTP application is fixed which I don't really like. Also there is It's fine that you don't like it, however, what use case justifies it. "Because I want to" is not really a reason. If you don't want to do it for a reason, it's probably bad design. No offense, really. Here's what I don't want. I don't want each component to have radically different layouts that require a ton of work to adapt to. I want contributing to projects and deploying them to be easy. Conventions solve this. Unless you have a problem that "rigid directory structures" prevent you from solving, that's just not good enough. > no construction rule > that says you are not allowed to write anything in the application > directory tree. I'd like to put the > entire component in a read-only area of disk for safety reasons (once > an initial install has been performed). This is a good idea. Separate the mutable data from code. This can be done perfectly well by convention, though. > I have a differnt proposal for reducing duplicates - this is a > different problem. If the problem is that deployment and installation is a hassle, I beg to differ. Part of the problem is that the core Erlang developers are often changing internal modules, so it is natural that their deployment system duplicates the entire system. For most of us, we just need to overlay our little modules on top of the giant Erlang environment. That's why I've been suggesting an overlay approach. > Yes to deploying Erlang components *and upgrading them* Applications are in modules and have versions. The upgrade case for Erlang applications is pretty much solid. Now, the way that releases do it (mutating an existing installation) is not exactly ideal, and I'd love to make adapt it to something smarter. > Sure. epkg etc. will be just a thin shell script layer over the API I suggested. > The CLI can be written as a thin layer over the API as well. I would welcome that. I think that a gen_event is a good model for handling runtime CLI functions. > because 90% is not enough Perhaps I wasn't clear. I was suggesting you start from 90% done instead of start from 0% done. Also, components and applications have enough conceptual overlap that it would only serve to be confusing. If you want to deprecate applications, that's fine, but understand that two packaging / distribution formats is a horrible idea. It took forever in Python to unify eggs and setuptools. > Of course. I just want to download yaws, ejabberd etc. drop them into > my eComponent Directory > and that's it - they are installed and runnable. No, you don't. :) A list of steps is not what you want. You want "installing components to be easy". Downloading a file and dropping it in can't run any code, can't make any decisions custom to your system, and can't autogenerate configuration. All of these things are great strengths of "gem install" and "easy_install". The last thing you want is to have a dozen different component downloads. For example, assume that RabbitMQ gets a bunch of C components that need building for three OSes. Now assume that it gets a special federated mode that requires different configuration. Next assume that it develops a multi-hosted version for hosting providers. If each combination has to be packaged separately (i.e. because we can't build the C extensions or mutate the configuration on install) then we end up with 12 (3*2*2) packages! Being devil's advocate here, let's say we do it your way and have "install code" that runs when you start it the first time we do a deployment. Even with this code, it has very weird behavior. The first time startup may take forever or fail if the extension fails to build. Now we have to keep track of that process and recover gracefully from it. It would have been far better to have had the "install" or "deploy" step fail. Basically, I don't see any value to the extra step. It should be "install" then "deploy", and install should be smart. If "install" is downloading a file and working with directories it's not smart. Ironically, it's also not really that beneficial to anyone in terms of user experience. Power users won't mind a single command in the shell. Non-power users would rather click an icon (moving files around for them is scary). It lands in a middle-ground that is probably less populated than you would think. > Ok, I didn't fully specify this - fine to change the details here or > use an environment variable to point > to the directory. I think unobtrusive default + environmental variable is good here. > This is the bit that is not enforced in OTP applications - I think > users are even encouraged to use the /priv > directory in an otp application Is there something about this that is hard to retrofit into applications? The great thing is that it's all abstracted already--just change what comes back from code:priv_dir/1. If people are doing it properly, their data should go where you want it to go. I'd really recommend having a cascading/overlaying/shadowing setup. Have everything search the deploy location, user location, and system location in that order. Mutable data never is found outside of the deploy. > Ok .eplist then? That sounds good. > Because I might not want to follow that particular convention. For > simple applications I usually put everything in one directory. > Actually this could be dropped, the system could scan the filesystem > to find the code paths > the first time you run the program. > directory One of the most misunderstood things about the Ruby community is their intolerance. Don't get me wrong, some of them are intolerant bigots, just like you get anywhere. However, a lot of the intolerance comes from a desire to have solid conventions. If you have to write applications all day long, knobs are only helpful if they're invisible. Every required parameter and every choice about where to put a directory is extra time before I have a working project and extra trouble when I come back to it six months later. Having the ability to override a convention is fine, but it should have a default and it should require no configuration. The scanning the filesystem part is a good one. One of the ugly things about Python, Java, Ruby, and Erlang is management of the code path. Java has it down with a set of mostly ugly conventions. Ruby still does a lot of manhandling (mostly generating stub files that know where the real code is). Python has, in my opinion, the best solution. Specifically, in the "system" directory, you can drop .pth files that point to additional code paths. This is fabulous in that it helps package managers (i.e. dpkg, rpm) plug in code in a manageable way. If Erlang were to have, for example, .epth files in /usr/lib/erlang/bin, that would make an awesome hook for distribution managers. Adding a .erlang directory (in the $HOME) with the same semantics would make it easy for users. It would be nice to move the cookie and hosts file into that directory, but I understand that might be asking too much too soon. > It's a hint to tell the package manager how often to check for updates > - I havn't thought through all the details > it could be something else: > > {checkForUpdates, everyTimeYouStart | dayly | hourly | weekly | > {after,Year,Month,Day} | {every, 10, minutes}} > > etc. Most system administration tasks are done rarely (i.e. running updates with the package manager, which this is a hint for). In this case, everybody else just downloads the current package list and compares it locally. I don't think that hints are all that useful in comparison. They can only generate false positives (expired but no update) and false negatives (not expired, but new update is out there). > The designer of the program knows what is a sensible value here Upgrades can trash data. It is not the designer's data. I don't think it takes more than blowing away the data in your RabbitMQ or losing your ejabberd user database to see why this could be extraordinarily bad on a large scale. Bugs happen. Best to let the administrator be in the loop. >> Again, I humbly suggest more traditional Unix pathnames. How about $HOME/.erlang/library///prefs.epl? > > No worries ... Thank you. > I guess the component framework would use the check for updates or > expiry date or whatever > to decide whether to do a check. The check would return a list of > improvements and the user would decide > whether or not to install them. Almost everything under the sun has an update command, but never does anything automatically. I really think it would be prudent. If you want to do "automatic updates", I would at least suggest that you have it turned off by default, and have it turned on by a project, not the user. The rationale is that the user of a component probably doesn't know enough about how the data is stored to keep it safe, but that the component developer should know. > I guess we should also be able to roll back a version (which is why I > have eLibrary/ComponentName/VSN/data .. > tags. Definitely. This is another reason that I suggested having a separate "deploy" step. You no longer have to worry about upgrading an installation. Each "version" is another deployment. In this scenario, you can specify "migrations" to upgrade/downgrade the data, but the code is managed as a separate entity. > It is pretty similar One is better than two. :) -- Jayson Vantuyl kagato@REDACTED From olivier.boudeville@REDACTED Wed Nov 25 22:49:11 2009 From: olivier.boudeville@REDACTED (Olivier Boudeville) Date: Wed, 25 Nov 2009 22:49:11 +0100 Subject: [erlang-questions] Erlang "object-oriented" after all? In-Reply-To: <20091125141406.GA1548@hanele> References: <20091125141406.GA1548@hanele> Message-ID: <4B0DA657.4040009@online.fr> Hi, This is surely a shameless plug, but, in terms of OOP with Erlang, one might give WOOPER a try (http://ceylan.sourceforge.net/main/documentation/wooper/), as it basically provides most OOP constructs in order to rely on concurrent instances in pure Erlang. Best regards, Olivier. Jachym Holecek a ?crit : > # Michael Turner 2009-11-24: > >> I ran across a very interesting exchange with Alan Kay, who most likely >> coined the term "object-oriented programming." >> >> http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/doc_kay_oop_en >> >> If you'll excuse some (only slightly tendentious) editing of his remarks: >> >> ---- >> "I thought of objects being like biological cells and/or individual >> computers on a network, only able to communicate with messages . . . . >> > > No, you didn't. If you did, you would certainly have noticed that individual > cells/computers live concurrently and communicate asynchronously, which is not > how your objects behave. > > :-) > > SCNR, I still remember the disappointment of discovering (ages back) that > Objective-C objects didn't really fulfill the "they behave like real-world > objects" promise given by the (otherwise pretty good) NeXT book because > you couldn't "run objects in parallel" which sounded like something that > should obviously be possible. > > Regards, > -- Jachym > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > > From ok@REDACTED Wed Nov 25 22:59:35 2009 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 26 Nov 2009 10:59:35 +1300 Subject: [erlang-questions] A Generic API for controlling software components In-Reply-To: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> References: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> Message-ID: <6509658C-B3B1-4A61-A291-246AEEDBCD87@cs.otago.ac.nz> On Nov 25, 2009, at 10:56 PM, Joe Armstrong wrote: > Rule4: All components C must have a file called > > $HOME/eComponents/C.ec/Preferences.pl > > The extension .pl means the file contains a property > list. Here is an example: We already have .pl meaning Prolog and .pl meaning Perl. Since the property list uses Erlang syntax, how about .epl (Erlang Property List)? > > Rule6: Code upgrade > > We should upgrade an component C when it's expiry date has > been reached. To update an component we delete the entire > component under $HOME/eComponents/C.ec we install the > new component and run the command: C:install(). What happens if the expiry date is reached and there _isn't_ any new version, or the site the new version should be obtained from is unreachable? > There are many kinds of components. For example, - there should be exactly one instance - there should be at most one instance - there can be up to N instances to share work - there should be one instance per client ... It isn't clear to me what range of component types this proposal is meant to cover. From steven.charles.davis@REDACTED Thu Nov 26 00:40:03 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 25 Nov 2009 15:40:03 -0800 (PST) Subject: Erlang/OTP now at Github In-Reply-To: <6672d0160911250601uad36706l570e036472d5d905@mail.gmail.com> References: <6672d0160911250601uad36706l570e036472d5d905@mail.gmail.com> Message-ID: <57580379-8e5f-4904-9380-d9951c59cd8d@m38g2000yqd.googlegroups.com> An event of significance -> mojombo has featured this on the GitHub main blog :) http://github.com/blog/554-erlang-otp-now-on-github /s From g@REDACTED Thu Nov 26 01:06:44 2009 From: g@REDACTED (Garrett Smith) Date: Wed, 25 Nov 2009 18:06:44 -0600 Subject: [erlang-questions] A Generic API for controlling software components In-Reply-To: <4081E7BC-939A-4E3C-A992-52848A6BE12C@souja.net> References: <9b08084c0911250156j4eff22e1wfd3cb655b01c5332@mail.gmail.com> <9b08084c0911250512p1b80ee84w96f840ab6007e844@mail.gmail.com> <4081E7BC-939A-4E3C-A992-52848A6BE12C@souja.net> Message-ID: On Wed, Nov 25, 2009 at 3:50 PM, Jayson Vantuyl wrote: > If you want to deprecate applications, that's fine, but understand that two packaging / distribution formats is a horrible idea. ?It took forever in Python to unify eggs and setuptools. I'd heartily second this. Python still doesn't have a unified packaging schema/toolset -- distutils and setuptools are still both in mainstream use, and it's a total pain. I've done a fair amount of work in package management systems in the commercial/proprietary work. I've never seen a packaging scheme that didn't have some pretty annoying warts. This tends to lead to package manager proliferation as it's very tempting to start anew on this common problem. Unfortunately, what initially looks like not-a-wheel is probably just another wheel. Unfortunately, I can only hand wave as I don't have a deep understanding of Erlang's current application/release scheme (though I have managed to used it :) I do think there's a reasonable burden to demonstrate why Erlang's application abstraction can't be evolved to support Python Egg or Ruby Gem like features. If the application scheme doesn't work, could application be morphed into components, or packages (also a common term for this)? Is there something from Faxien that gets us closer to Joe's goal? http://code.google.com/p/faxien/ Garrett From igorrs@REDACTED Thu Nov 26 01:09:28 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Wed, 25 Nov 2009 22:09:28 -0200 Subject: [erlang-questions] How to match on a nested record against a dict In-Reply-To: References: <4B0A49CC.90307@erlang-consulting.com> Message-ID: If you don't need the values that you store in the dict (in your example, you store [] and never use it), you can use a sets instead of a dict. Usage is a little bit simpler: ------------------------- 1> D = dict:store("_see._tcp.local", [], dict:new()). {dict,1,16,16,8,80,48, {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, {{[],[],[],[], [["_see._tcp.local"]], [],[],[],[],[],[],[],[],[],[],[]}}} 2> dict:find("_see._tcp.local", D). {ok,[]} 3> S = sets:add_element("_see._tcp.local", sets:new()). {set,1,16,16,8,80,48, {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, {{[],[],[],[], ["_see._tcp.local"], [],[],[],[],[],[],[],[],[],[],[]}}} 4> sets:is_element("_see._tcp.local", S). true ------------------------- See also gb_sets and ordsets. Igor. On Tue, Nov 24, 2009 at 7:25 PM, Jarrod Roberson wrote: > On Tue, Nov 24, 2009 at 3:18 PM, Jarrod Roberson wrote: > >> >> process_dnsrec1(Sub,[#dns_rr{domain=Dom}|Rest]) -> >> ? case dict:find(Dom,Sub) of >> ? ? ? [Result] -> >> ? ? ? ? ? io:format("Interesting domain ~p=~p~n",[Dom,Result]); >> > > thanks for reading I figured it out by trial and error. > I replaced the [Result] with {ok,Result} and it started working as expected -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From kenji.rikitake@REDACTED Thu Nov 26 03:29:00 2009 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Thu, 26 Nov 2009 11:29:00 +0900 Subject: Erlang R13B03 *EXPERIMENTAL* FreeBSD port Message-ID: <20091126022900.GA69974@k2r.org> FreeBSD *EXPERIMENTAL* (unofficial) port for R13B03 available at http://www.ne.jp/asahi/bdx/info/software/erlang-r13b03,1-20091126-1.tar.gz KNOWN issues: * NO JAVA test yet (WITHOUT_JAVA=yes set in the Makefile) * Makefile.man has been put dupes of snmp*.3 in MAN3/MAN6/MAN7 (necessary to remove errors on "make package") * r13b02_1,1 fix (of Steve Vinocki's erlang.el) hasn't been merged I've sent this port to Giacomo Olgeni (the official port maintainer) so I hope he'll use mine as a feedback source. Kenji Rikitake From thijsterlouw@REDACTED Thu Nov 26 04:57:40 2009 From: thijsterlouw@REDACTED (Thijs) Date: Wed, 25 Nov 2009 19:57:40 -0800 (PST) Subject: Fastest pseudo-random number-generator: erlang:statistics(io) ? In-Reply-To: References: <40930F41-0E0C-484D-B7CC-7EF80DEF06B8@souja.net> Message-ID: <8e431a91-5d39-4f0c-b105-32e6410151ae@g22g2000prf.googlegroups.com> The new nif (native implemented functions) provide a great alternative to generate random numbers and timestamps, much faster than the Erlang alternatives! The speed is about 1/2 to 1/3 of statistics(io), but still much faster than erlang:now(). I simply used time(0) for the nif_now() and gettimeofday(&tv, NULL) for the nif_random() functions. nif's are a great addition to the language! From bgustavsson@REDACTED Thu Nov 26 06:52:36 2009 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 26 Nov 2009 06:52:36 +0100 Subject: [erlang-questions] Erlang/OTP now at Github In-Reply-To: <1c3be50f0911250620h22aa000ch5c089c4c20175a95@mail.gmail.com> References: <6672d0160911250601uad36706l570e036472d5d905@mail.gmail.com> <1c3be50f0911250620h22aa000ch5c089c4c20175a95@mail.gmail.com> Message-ID: <6672d0160911252152t6a72d3ebj33de436abe4f9251@mail.gmail.com> On Wed, Nov 25, 2009 at 3:20 PM, Juan Jose Comellas wrote: > Why was the commit history discarded? Wouldn't it be better to at > least preserve the history corresponding to the public releases? Maybe > you could use Matthew Foemmel's repository[1] in Github as a base. > > [1] http://github.com/mfoemmel/erlang-otp/ I have added a wiki page explaining how you can prepend mfoemmel's history to our history by using two simple commands: http://wiki.github.com/erlang/otp/extending-the-history-of-erlangotp -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From kenji.rikitake@REDACTED Thu Nov 26 07:14:30 2009 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Thu, 26 Nov 2009 15:14:30 +0900 Subject: [erlang-questions] Fastest pseudo-random number-generator: erlang:statistics(io) ? In-Reply-To: References: Message-ID: <20091126061430.GA73856@k2r.org> Writing a linked-in driver with SFMT will give you much faster solution. Or probably in NIF? SIMD-oriented Fast Mersenne Twister (SFMT) http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/index.html (C code available) Kenji Rikitake In the message dated Tue, Nov 24, 2009 at 06:59:45AM -0800, Thijs writes: > Today I needed a faster alternative for erlang:now(). It turns out > that using erlang:now() is quite slow: it requires a system call. I > decided to test some alternatives. My requirements are: speed, speed, > speed :) I will use it in a server that is quite busy, so I figured I > could use various system metrics to generate random numbers. I use > these pseudo-random numbers to do load-balancing, log requests at > random etc. (rest snapped) From tony@REDACTED Thu Nov 26 07:19:01 2009 From: tony@REDACTED (Tony Arcieri) Date: Wed, 25 Nov 2009 23:19:01 -0700 Subject: Reia: moving closer to Erlang Message-ID: Reia is a language with Ruby-like syntax on top of the Erlang VM: http://reia-lang.org/ It's been undergoing a bit of a transition, abandoning many of the features which differentiate it from Erlang and focusing on merely providing a cleaner syntax with destructive assignment: http://www.unlimitednovelty.com/2009/11/new-reia-now-without-rainbow-farting.html This is inspired to a large degree by Robert Virding's Lisp Flavoured Erlang (LFE), which does something similar. Future versions of the language will focus on additional features which are not present in the Erlang language itself, Until then, Reia will focus on attaining complete feature parity with Erlang and reaching a production-quality implementation. -- Tony Arcieri Medioh/Nagravision From max.lapshin@REDACTED Thu Nov 26 21:34:46 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 26 Nov 2009 23:34:46 +0300 Subject: Which postgresql driver Message-ID: There are several of postgresql drivers and I cannot select which is supported currently. It seems, that this one is best for now: http://github.com/joshuamiller/epgsql-utf8, epgsql is used in zootonic and this fork claims to support utf8 properly (haven't look at code, to get out, what it means). So, which are working? From ngocdaothanh@REDACTED Fri Nov 27 01:52:31 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 27 Nov 2009 09:52:31 +0900 Subject: [erlang-questions] Which postgresql driver In-Reply-To: References: Message-ID: <5c493e530911261652y60260565w8bb6fbc35021094d@mail.gmail.com> I think the latest version of epgsql (epgsql-utf8 vs 4 months ago) is in its official repository: http://bitbucket.org/will/epgsql/overview/ In my experience it is the best and it supports UTF8 properly. I have used it in a project together with epgsql_pool: http://www.erlangatwork.com/2009/01/erlang-and-postgresql-redux.html On Fri, Nov 27, 2009 at 5:34 AM, Max Lapshin wrote: > There are several of postgresql drivers and I cannot select which is > supported currently. > It seems, that this one is best for now: > http://github.com/joshuamiller/epgsql-utf8, epgsql is used in zootonic > and this fork claims to support utf8 properly (haven't look at code, > to get out, what it means). > > So, which are working? From max.lapshin@REDACTED Fri Nov 27 06:40:06 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 27 Nov 2009 08:40:06 +0300 Subject: [erlang-questions] Which postgresql driver In-Reply-To: <5c493e530911261652y60260565w8bb6fbc35021094d@mail.gmail.com> References: <5c493e530911261652y60260565w8bb6fbc35021094d@mail.gmail.com> Message-ID: Thank You. From thijsterlouw@REDACTED Fri Nov 27 08:50:28 2009 From: thijsterlouw@REDACTED (Thijs) Date: Thu, 26 Nov 2009 23:50:28 -0800 (PST) Subject: R13B03 linkedin driver incompatible with R13B01 Message-ID: <025c17f8-edf6-4156-965d-9a69713c993f@u18g2000pro.googlegroups.com> Today I tried to use a linkedin driver that was compiled on a Linux system with R13B03 on another machine that is still running R13B01. Unfortunately I got this error: driver_incorrect_version : "Driver compiled with incorrect version of erl_driver.h" I have no problems running R13B01 drivers on R13B03. I saw in the release notes (OTP-8205) that erl_driver.h has been updated, but I didn't expect this would be incompatible. Might be good to point this out explicitly. From puzza007@REDACTED Fri Nov 27 10:31:42 2009 From: puzza007@REDACTED (Paul Oliver) Date: Fri, 27 Nov 2009 09:31:42 +0000 Subject: ODBC crashes on stored procedure result sets Message-ID: Hi, Has anyone observed problems with stored procedures that return result-sets? It seems (at least with unixodbc and the informix ODBC driver) that SQLDescribeCol returns a name_len < 0 which then causes a crash on calling SQLBindCol. Since I'm not expected these columns to be named I've fixed it by doing: if(!sql_success(SQLDescribeCol(statement_handle(state), (SQLSMALLINT)(i+1), name, sizeof(name), &name_len, &sql_type, &size, &dec_digits, &nullable))) DO_EXIT(EXIT_DESC); /* Don't let the driver tell us that the name_len is negative, it can crash SQLBindCol below */ if (name_len < 0) name_len = 0; The patch is available here http://github.com/puzza007/otp/tree/stored_proc_colname_has_bad_length. Does anyone have any advice? Cheers, Paul. From koops.j@REDACTED Fri Nov 27 11:04:46 2009 From: koops.j@REDACTED (Jeroen Koops) Date: Fri, 27 Nov 2009 11:04:46 +0100 Subject: [JOB] Looking for an Erlang/OTP developer in Singapore Message-ID: <331a9abb0911270204h55f1a600tdcecbc9a8d8b393b@mail.gmail.com> Texprezzo, a privately owned startup based in the Netherlands, is extending its activities to South East Asia and looking for a full-time Erlang/OTP developer for its Singapore office, to be opened in Q1 2010. What we ask: - Several years of professional programming experience. - At least one year experience in Erlang/OTP development. - Ability to work unsupervised - good problem-solving skills. - Good communication skills in English, written and verbally. - Legally able to work in Singapore. Knowledge of any of the following is a plus: - HTML, Javascript, PHP, Linux, Solaris and Java - Telecom-related technologies, especially relating to SMS What we offer: - Opportunity to be among the first employees in a startup - Competitive renumeration package - Earn a living doing what you love doing: programming Erlang! Note: This is a permanent position, at this moment we are not interested in contractors/freelancers. To apply, please send a CV to jeroen.koops@REDACTED You can also contact me for more information at the same address, or by telephone on +31-20-3120528. *Company information: The Texprezzo Group was founded in 2008 by a team of experienced telecom professionals with a wide variety of backgrounds with experience in various areas related to mobile messaging. Their goal was to create new value with the existing messaging infrastructures as currently deployed by mobile operators today, thus stimulating a new wave of revenues and end user experience. The main investors behind Texprezzo Group BV have a long track record in establishing and creating new technology businesses. An example of which is the recently sold mBalance Group, the leading SMS Routing infrastructure company.* From dmitriid@REDACTED Fri Nov 27 11:22:11 2009 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Fri, 27 Nov 2009 12:22:11 +0200 Subject: exmpp joining groupchats Message-ID: <925F7B92-7882-4891-B0AF-6AC41C045B14@gmail.com> How can my bot/client writen in XMPP join a conference/groupchat? From Mike.French@REDACTED Fri Nov 27 13:23:18 2009 From: Mike.French@REDACTED (French, Mike) Date: Fri, 27 Nov 2009 12:23:18 -0000 Subject: [erlang-questions] Idle question Message-ID: Idle Thoughts of an Idle Fellow (and Erlang novice) .... > -----Original Message----- > > On 24/11/09 Richard O'Keefe wrote: > >> Suppose Erlang were available on BlueFern > >> (the BlueGene machine at the University of Canterbury, > >> see http://www.bluefern.canterbury.ac.nz/ ). > >> What would you use it for? > >> What science or technology would it advance? ..... Erlang should have a good headstart for parallel processing because of the usual suspects: share nothing process isolation, message passing and asynchronicity (as Jung would have called it, had he been an Erlang hacker - Jungerl ?) But Erlang is also bad for HPC because of: - no dense numerical data structures with pointer arithmetic: vectors, matrices, multidimensional arrays - memory overhead for lists, hence also for strings, so no dense character data structure with pointer arithmetic (implement string processing over binaries ?!) - no mutable in-place (zero-copy) data structures even when isolated in a single process thread of control - message copy overhead even when local (except large binaries) - no control over process locality ? So I conclude that dense numerical computation in Erlang will always be slower than C, C++, Fortran, ... Fortress ? I also conclude that if Erlang is just used for coordination of compute processes implemented in other languages, then extra copies required by Erlang message passing will always be slower than just using MPI directly between the compute processes ? So what is Erlang good for in the HPC arena ? A suggestion would be to avoid problems that need tight computational inner loops over multi-dimensional arrays, and look for problems requiring: - complex decisions (conditionals in your loops) - across uncorrelated pathways (no advantage for dense structures) - over arbitrary graphs of data (communicating processes are graphs) A place to start would be to look at the 13 Dwarfs (should be Dwarves!) which are patterns of computation extracted from common HPC problems: http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-183.pdf Then follow the related architectural design process outlined here: http://parlab.eecs.berkeley.edu/sites/all/parlab/files/Patterns-Keutzer.pdf (best overall summary slide is 23/75) and mark Erlang on its strengths and weaknesses as you make decisions about HPC design patterns. When you come across ... task decomposition, arbitrary task graph, event-based, pipe and filter, graph algorithms, dynamic programming, map-reduce, graph traversal, finite state machines, divide & conquer, backtrack branch & bound, task parallelism, discrete event, pipeline, master/worker, CSP, message passing .... then Erlang has potential. Notice how Erlang is better 'above the line' in the Productivity Layer rather than 'below the line' in the Efficiency Layer. Of course, it would also be fun to speculate how Erlang could be improved with dense multidimensional arrays, zero copy local messages, mutation BIFs (like HiPE hacks), SIMD map and fold, etc. ... but maybe it wouldn't be Erlang any more. Regards, Mike Thales UK Ltd (Wells) DISCLAIMER: The information contained in this e-mail is confidential. It may also be legally privileged. It is intended only for the stated addressee(s) and access to it by any other person is unauthorised. If you are not an addressee, you must not disclose, copy, circulate or in any other way use or rely on the information contained in this e-mail. Such unauthorised use may be unlawful. We may monitor all e-mail communications through our networks. If you have received this e-mail in error, please inform us immediately on sender's telephone number above and delete it and all copies from your system. We accept no responsibility for changes to any e-mail which occur after it has been sent. Attachments to this e-mail may contain software viruses which could damage your system. We therefore recommend you virus-check all attachments before opening. Thales UK Ltd. Registered Office: 2 Dashwood Lang Road, The Bourne Business Park, Addlestone, Weybridge, Surrey KT15 2NX Registered in England No. 868273 From vladdu55@REDACTED Fri Nov 27 14:23:13 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 27 Nov 2009 14:23:13 +0100 Subject: online documentation Message-ID: <95be1d3b0911270523m7e78892dg5e80aeaf875e3829@mail.gmail.com> Hi! I really like the looks of the new documentation. There is a small thing that was suggested before and that would help a lot: a search field (hopefully with autocompletion, too). Typing "li:keys" to go to lists:keysearch would be soooo sweet!... I realize now that generating this is probably the job of the docbuilder application, which would mean that I can probably submit a patch myself (if I get to it). Will check that (but if someone else has faster fingers, I won't hold a grudge :-). BTW, a related question: I see that the issue tracker at github is being used. What is the current policy for reporting bugs? It would be nice if there was only a single place instead of three (some people like me do post bug reports here instead of erlang-bugs anyway), for example if the github tracker could send email notifications to erlang-bugs. best regards, Vlad From zabrane3@REDACTED Fri Nov 27 15:11:56 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Fri, 27 Nov 2009 15:11:56 +0100 Subject: Read from a compressed "GZIP" file Message-ID: <18a1db030911270611y4165743q6cd3d7bb455077f4@mail.gmail.com> Hi List, Is there a way to read from a GZIPped file (example please)? I found a mode called "compressed" in the "file" module, but failed to understand how it works: $ erl 1> {ok, FH} = file:open("foo.gz", [read, compressed]). {ok,<0.42.0>} 2> {ok, Bin} = file:read(FH, 8). ** exception error: no match of right hand side value {error,terminated} Why am I faling to read 8 bytes from it? How can I read the whole file in one go? Regards Zabrane From kenneth.lundin@REDACTED Fri Nov 27 15:15:03 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Fri, 27 Nov 2009 15:15:03 +0100 Subject: [erlang-questions] online documentation In-Reply-To: <95be1d3b0911270523m7e78892dg5e80aeaf875e3829@mail.gmail.com> References: <95be1d3b0911270523m7e78892dg5e80aeaf875e3829@mail.gmail.com> Message-ID: Hi Vlad, A search facility will come , we will create an index for this purpose as part of the docbuilding process. No need to patch the docbuilder application. /Kenneth Erlang/OTP Ericsson On Fri, Nov 27, 2009 at 2:23 PM, Vlad Dumitrescu wrote: > Hi! > > I really like the looks of the new documentation. > > There is a small thing that was suggested before and that would help a > lot: a search field (hopefully with autocompletion, too). Typing > "li:keys" to go to lists:keysearch would be soooo sweet!... > > I realize now that generating this is probably the job of the > docbuilder application, which would mean that I can probably submit a > patch myself (if I get to it). Will check that (but if someone else > has faster fingers, I won't hold a grudge :-). > > BTW, a related question: I see that the issue tracker at github is > being used. What is the current policy for reporting bugs? It would be > nice if there was only a single place instead of three (some people > like me do post bug reports here instead of erlang-bugs anyway), for > example if the github tracker could send email notifications to > erlang-bugs. > > best regards, > Vlad > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From kenji.rikitake@REDACTED Fri Nov 27 16:04:38 2009 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Sat, 28 Nov 2009 00:04:38 +0900 Subject: FreeBSD Erlang R13B03 port (lang/erlang) now officially available In-Reply-To: <20091126022900.GA69974@k2r.org> References: <20091126022900.GA69974@k2r.org> Message-ID: <20091127150438.GA99418@k2r.org> FreeBSD official port lang/erlang for R13B03 now available. (Thanks to Giacomo Olgeni!) http://www.freebsd.org/cgi/cvsweb.cgi/ports/lang/erlang/ http://www.freshports.org/lang/erlang/ Kenji Rikitake From vinayakapawar@REDACTED Fri Nov 27 16:26:04 2009 From: vinayakapawar@REDACTED (Vinayak Pawar) Date: Fri, 27 Nov 2009 20:56:04 +0530 Subject: [erlang-questions] Read from a compressed "GZIP" file In-Reply-To: <18a1db030911270611y4165743q6cd3d7bb455077f4@mail.gmail.com> References: <18a1db030911270611y4165743q6cd3d7bb455077f4@mail.gmail.com> Message-ID: <23237d040911270726w5bc813ecn45ffed48aab4d688@mail.gmail.com> Z = zlib:open(), zlib:inflateInit(Z, 31), % hopefully everything fits in memory % if file size is big, you can do this read and inflate in chunks CompressedBin = file:read_file(Filename), Uncompressed = zlib:inflate(Z, CompressedBin), zlib:inflateEnd(Z), zlib:close(Z). Hope this helps! Thanks, Vinayak On Fri, Nov 27, 2009 at 7:41 PM, zabrane Mikael wrote: > Hi List, > > Is there a way to read from a GZIPped file (example please)? > I found a mode called "compressed" in the "file" module, but failed to > understand how it works: > > $ erl > 1> {ok, FH} = file:open("foo.gz", [read, compressed]). > {ok,<0.42.0>} > > 2> {ok, Bin} = file:read(FH, 8). > ** exception error: no match of right hand side value {error,terminated} > > Why am I faling to read 8 bytes from it? > How can I read the whole file in one go? > > Regards > Zabrane > From m.nacos@REDACTED Fri Nov 27 16:33:07 2009 From: m.nacos@REDACTED (Michael Nacos) Date: Fri, 27 Nov 2009 15:33:07 +0000 Subject: [erlang-questions] Which postgresql driver Message-ID: <407fa4640911270733x6460462fj1957affb1a8b03be@mail.gmail.com> FYI I have just released a libpq wrapper (postgresql libs required) which supports utf8 in a true Erlang way :-) strings are lists of bytes etc. http://github.com/mnacos/epg no clever stuff, it's just an interface for issuing SQL queries and retrieving results. Character encoding is pretty much handled by postgresql. because it uses libpq it supports pretty much everything postgres allows you to do (e.g. COPY) I am thinking about adding support for async queries (libpq supports that) cheers, Michael PS my Erlang experience is limited, so please don't flame. Suggestions/ improvements are, of course, welcome. From yan.beijing.china@REDACTED Fri Nov 27 16:47:12 2009 From: yan.beijing.china@REDACTED (Yan Yan) Date: Fri, 27 Nov 2009 07:47:12 -0800 Subject: [erlang-questions] online documentation References: <95be1d3b0911270523m7e78892dg5e80aeaf875e3829@mail.gmail.com>, Message-ID: <200911270747099066231@gmail.com> Great news! Looking forward. Yan Yan 2009-11-27 From: Kenneth Lundin Time: 2009-11-27 06:16:53 To: Vlad Dumitrescu Fw: erlang-questions Subject: Re: [erlang-questions] online documentation Hi Vlad, A search facility will come , we will create an index for this purpose as part of the docbuilding process. No need to patch the docbuilder application. /Kenneth Erlang/OTP Ericsson On Fri, Nov 27, 2009 at 2:23 PM, Vlad Dumitrescu wrote: > Hi! > > I really like the looks of the new documentation. > > There is a small thing that was suggested before and that would help a > lot: a search field (hopefully with autocompletion, too). Typing > "li:keys" to go to lists:keysearch would be soooo sweet!... > > I realize now that generating this is probably the job of the > docbuilder application, which would mean that I can probably submit a > patch myself (if I get to it). Will check that (but if someone else > has faster fingers, I won't hold a grudge :-). > > BTW, a related question: I see that the issue tracker at github is > being used. What is the current policy for reporting bugs? It would be > nice if there was only a single place instead of three (some people > like me do post bug reports here instead of erlang-bugs anyway), for > example if the github tracker could send email notifications to > erlang-bugs. > > best regards, > Vlad > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From zabrane3@REDACTED Fri Nov 27 18:38:34 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Fri, 27 Nov 2009 18:38:34 +0100 Subject: [erlang-questions] Read from a compressed "GZIP" file In-Reply-To: <23237d040911270726w5bc813ecn45ffed48aab4d688@mail.gmail.com> References: <18a1db030911270611y4165743q6cd3d7bb455077f4@mail.gmail.com> <23237d040911270726w5bc813ecn45ffed48aab4d688@mail.gmail.com> Message-ID: <18a1db030911270938r35560b03ub56539a28632f7be@mail.gmail.com> Thanks very much Vinayak! Exactly what I needed 2009/11/27 Vinayak Pawar > > Z = zlib:open(), > zlib:inflateInit(Z, 31), > > % hopefully everything fits in memory > % if file size is big, you can do this read and inflate in chunks > > CompressedBin = file:read_file(Filename), > > Uncompressed = zlib:inflate(Z, CompressedBin), > > zlib:inflateEnd(Z), > zlib:close(Z). > > Hope this helps! > > Thanks, > Vinayak > > > On Fri, Nov 27, 2009 at 7:41 PM, zabrane Mikael wrote: > >> Hi List, >> >> Is there a way to read from a GZIPped file (example please)? >> I found a mode called "compressed" in the "file" module, but failed to >> understand how it works: >> >> $ erl >> 1> {ok, FH} = file:open("foo.gz", [read, compressed]). >> {ok,<0.42.0>} >> >> 2> {ok, Bin} = file:read(FH, 8). >> ** exception error: no match of right hand side value {error,terminated} >> >> Why am I faling to read 8 bytes from it? >> How can I read the whole file in one go? >> >> Regards >> Zabrane >> > > From igorrs@REDACTED Fri Nov 27 23:08:04 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Fri, 27 Nov 2009 20:08:04 -0200 Subject: [erlang-questions] The 2 GB limit In-Reply-To: <4B0BF675.2050708@erlang-consulting.com> References: <4B0BF675.2050708@erlang-consulting.com> Message-ID: On Tue, Nov 24, 2009 at 1:06 PM, Ulf Wiger wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Igor Ribeiro Sucupira wrote: >> Hi. >> >> Is there any plan (or work in progress) for removing the 2 GB size >> limit of dets files? >> >> And, while this is not accomplished, do you think it could be easier >> to try to raise that limit to 4 GB? Assuming the issue is related to >> 32-bit addressing, I'm guessing (just guessing) that dealing with 4 GB >> files should not be difficult. Am I wrong? Why? > > Rather than re-writing dets, I would go with something like Tokyo > Tyrant, build on a suitable erlang wrapper (like Hi. I'm running Mnesia with fragmented tables, over a pool of servers. Considering the amount of records to be inserted and the size limits of dets, I will need a lot of fragments for each table and there seems to be a big performance penalty on using many Mnesia fragments (even when they don't have so much data). The more plausible replacement, then, would be something like tcerl with mnesiaex, but I've read somewhere that mnesiaex is still experimental. I'm dealing with data that affects millions of users, in my current project, so I have to be very careful. That was the reason for wanting to raise the limits of dets tables. Thank you. Igor. > http://github.com/mallipeddi/tora), and perhaps put something like > my sext library on top > (http://svn.ulf.wiger.net/sext/trunk/sext/examples/tt_proto.erl). > > To match dets, I'd like to see some administrative code around the > creation and opening of Tyrant tables, including the ability to > "repair" the data from the transaction log. Seems doable, though. > > http://1978th.net/tokyotyrant/ > > BR, > Ulf W > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iEYEARECAAYFAksL9nUACgkQtqqFieqzed1v7wCfUf9bgDU+JfD8Gey51lvR7fxG > 3iIAn0J5+8NpHH7DkLTgRlRuUKWomkXO > =C66x > -----END PGP SIGNATURE----- -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From ulf.wiger@REDACTED Sat Nov 28 00:51:32 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 28 Nov 2009 00:51:32 +0100 Subject: [erlang-questions] The 2 GB limit In-Reply-To: References: <4B0BF675.2050708@erlang-consulting.com> Message-ID: <4B106604.6070304@erlang-consulting.com> Mnesiaex is not supported by OTP. As far as I know, it is stable. There are some things to consider... - If you put significantly more than 2GB into each table, mnesia's method of synchronizing replicas (copying the entire table over the network) may start to hurt. - Tokyo Cabinet has slightly different recovery characteristics than dets. This may make a difference in some crash recovery situations. That is, TC gets its speed partly from making use of memory-mapped IO, so more data is likely to get lost in some failure situations. TC is supposedly also more sensitive to corrupted files than dets, since it doesn't 'repair' the file like dets does. I don't think there are major problems with performance when using many fragments. You should perhaps do some measurements first? It can be a problem that you need to have enough fragments that you are absolutely sure never to reach 2 GB in any fragment, since mnesia has no way of dealing with dets refusing to insert more records. BTW, if you use disc_copies (not disc_only_copies), dets is not used for the persistent storage, and there is no 2 GB limit per table (only the restriction that the table must fit in RAM). BR, Ulf W Igor Ribeiro Sucupira wrote: > On Tue, Nov 24, 2009 at 1:06 PM, Ulf Wiger > wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Igor Ribeiro Sucupira wrote: >>> Hi. >>> >>> Is there any plan (or work in progress) for removing the 2 GB size >>> limit of dets files? >>> >>> And, while this is not accomplished, do you think it could be easier >>> to try to raise that limit to 4 GB? Assuming the issue is related to >>> 32-bit addressing, I'm guessing (just guessing) that dealing with 4 GB >>> files should not be difficult. Am I wrong? Why? >> Rather than re-writing dets, I would go with something like Tokyo >> Tyrant, build on a suitable erlang wrapper (like > > Hi. > > I'm running Mnesia with fragmented tables, over a pool of servers. > Considering the amount of records to be inserted and the size limits > of dets, I will need a lot of fragments for each table and there seems > to be a big performance penalty on using many Mnesia fragments (even > when they don't have so much data). > > The more plausible replacement, then, would be something like tcerl > with mnesiaex, but I've read somewhere that mnesiaex is still > experimental. > I'm dealing with data that affects millions of users, in my current > project, so I have to be very careful. That was the reason for wanting > to raise the limits of dets tables. > > Thank you. > Igor. > >> http://github.com/mallipeddi/tora), and perhaps put something like >> my sext library on top >> (http://svn.ulf.wiger.net/sext/trunk/sext/examples/tt_proto.erl). >> >> To match dets, I'd like to see some administrative code around the >> creation and opening of Tyrant tables, including the ability to >> "repair" the data from the transaction log. Seems doable, though. >> >> http://1978th.net/tokyotyrant/ >> >> BR, >> Ulf W >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1.4.9 (GNU/Linux) >> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org >> >> iEYEARECAAYFAksL9nUACgkQtqqFieqzed1v7wCfUf9bgDU+JfD8Gey51lvR7fxG >> 3iIAn0J5+8NpHH7DkLTgRlRuUKWomkXO >> =C66x >> -----END PGP SIGNATURE----- > > -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From igorrs@REDACTED Sat Nov 28 03:44:40 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Sat, 28 Nov 2009 00:44:40 -0200 Subject: [erlang-questions] The 2 GB limit In-Reply-To: <4B106604.6070304@erlang-consulting.com> References: <4B0BF675.2050708@erlang-consulting.com> <4B106604.6070304@erlang-consulting.com> Message-ID: On Fri, Nov 27, 2009 at 9:51 PM, Ulf Wiger wrote: > > I don't think there are major problems with performance > when using many fragments. You should perhaps do some > measurements first? Sorry I was not clear enough. The performance penalty I observed was more related to the number of fragments PER SERVER. If I insert a total of 10 GB of data on 500 initially empty fragments, distributed over 2 servers, and then repeat the experiment with 1000 initially empty fragments (without changing the number of servers nor the total amount of data), the insertion rate will be smaller (I don't have the data here, but I think it was something around 20% smaller on my experiments). Other people in my company have conducted similar experiments (for completely independent projects) and have arrived at the same general conclusion: try to keep the number of fragments per server as small as possible. > It can be a problem that you need to have enough > fragments that you are absolutely sure never to > reach 2 GB in any fragment, since mnesia has no > way of dealing with dets refusing to insert more > records. I'm aware of that. I need to create the pool with enough fragments that I will not need to add more in the future (adding fragments in that situation will certainly be a very heavy operation), but, at the same time, I can't just "start with 100 thousand fragments" (:-)), as I thought in the beginning. That is an important trade-off you have to consider when deciding to use Mnesia's fragmented tables in disc_only_copies. > BTW, if you use disc_copies (not disc_only_copies), > dets is not used for the persistent storage, and > there is no 2 GB limit per table (only the restriction > that the table must fit in RAM). I've read about that. But having to keep all my data in RAM would increase so much the cost of the project that it would be much wiser to give up Mnesia. Thank you. Igor. > BR, > Ulf W > > > Igor Ribeiro Sucupira wrote: >> >> On Tue, Nov 24, 2009 at 1:06 PM, Ulf Wiger >> wrote: >>> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Igor Ribeiro Sucupira wrote: >>>> >>>> Hi. >>>> >>>> Is there any plan (or work in progress) for removing the 2 GB size >>>> limit of dets files? >>>> >>>> And, while this is not accomplished, do you think it could be easier >>>> to try to raise that limit to 4 GB? Assuming the issue is related to >>>> 32-bit addressing, I'm guessing (just guessing) that dealing with 4 GB >>>> files should not be difficult. Am I wrong? Why? >>> >>> Rather than re-writing dets, I would go with something like Tokyo >>> Tyrant, build on a suitable erlang wrapper (like >> >> Hi. >> >> I'm running Mnesia with fragmented tables, over a pool of servers. >> Considering the amount of records to be inserted and the size limits >> of dets, I will need a lot of fragments for each table and there seems >> to be a big performance penalty on using many Mnesia fragments (even >> when they don't have so much data). >> >> The more plausible replacement, then, would be something like tcerl >> with mnesiaex, but I've read somewhere that mnesiaex is still >> experimental. >> I'm dealing with data that affects millions of users, in my current >> project, so I have to be very careful. That was the reason for wanting >> to raise the limits of dets tables. >> >> Thank you. >> Igor. >> >>> http://github.com/mallipeddi/tora), and perhaps put something like >>> my sext library on top >>> (http://svn.ulf.wiger.net/sext/trunk/sext/examples/tt_proto.erl). >>> >>> To match dets, I'd like to see some administrative code around the >>> creation and opening of Tyrant tables, including the ability to >>> "repair" the data from the transaction log. Seems doable, though. >>> >>> http://1978th.net/tokyotyrant/ >>> >>> BR, >>> Ulf W >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v1.4.9 (GNU/Linux) >>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org >>> >>> iEYEARECAAYFAksL9nUACgkQtqqFieqzed1v7wCfUf9bgDU+JfD8Gey51lvR7fxG >>> 3iIAn0J5+8NpHH7DkLTgRlRuUKWomkXO >>> =C66x >>> -----END PGP SIGNATURE----- >> >> > > > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From ulf.wiger@REDACTED Sat Nov 28 09:31:39 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 28 Nov 2009 09:31:39 +0100 Subject: [erlang-questions] The 2 GB limit In-Reply-To: References: <4B0BF675.2050708@erlang-consulting.com> <4B106604.6070304@erlang-consulting.com> Message-ID: <4B10DFEB.9030005@erlang-consulting.com> Igor Ribeiro Sucupira wrote: > > Sorry I was not clear enough. The performance penalty I observed was > more related to the number of fragments PER SERVER. > If I insert a total of 10 GB of data on 500 initially empty fragments, > distributed over 2 servers, and then repeat the experiment with 1000 > initially empty fragments (without changing the number of servers nor > the total amount of data), the insertion rate will be smaller (I don't > have the data here, but I think it was something around 20% smaller on > my experiments). Oh, 20% seems pretty good to me. :) Adding lots of local fragments ought to affect both the hit ratio of the disk cache and the amount of disk seeks, as both the locality and overhead per object get worse. But whether that's an acceptable penalty for something that's essentially a workaround is of course application specific. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ttmrichter@REDACTED Sat Nov 28 15:09:36 2009 From: ttmrichter@REDACTED (Michael Richter) Date: Sat, 28 Nov 2009 22:09:36 +0800 Subject: filelib:is_* is missing something necessary. Message-ID: I found out that the file type predicates in *filelib* (*is_file/1*, * is_dir/1*, *is_regular/1*) have a little quirk that makes them not that useful for my purposes. Specifically they treat symbolic links to files as files and symbolic links to directories as directories instead of flagging each respectively as symlinks. It seems to me that they're implemented in terms of *read_file_info/1*instead of *read_link_info/1* and I was wondering if it might not be possible to make one of the following changes, depending on what people in the community found most useful (I'm easy either way, personally): 1. Change the return values such that, say, *is_file/1* returns *true* if the named file is a straight file, but returns *symlink* if it is a symbolic link *to a file*. (In any other situation -- directory, symlink to directory, special file, etc. -- it would still return *false*.) 2. Add a predicate *is_symlink/1* so that I can first check if something's a directory and then follow through with a check if it's a symbolic link to one. Currently I've just rewritten the predicates for my code, but in general I prefer to use system libraries wherever possible so I'd really like to see one of the two above approaches (or perhaps a third, superior one) written in. My predicate for *is_symlink/1* looks like this: is_symlink(Name) -> case file:read_link_info(Name) of {ok, FileInfo} -> FileInfo#file_info.type =:= symlink end. It's not a particularly Earth-shattering change for option 2. Option 1 could look something like this only without the fugliness of my quick hack.: is_regular(Name) -> case file:read_file_info(Name) of {ok, #file_info{type = regular}} -> case file:read_link_info(Name) of {ok, #file_info{type = symlink}} -> symlink; _ -> true end; _ -> false end. From bgustavsson@REDACTED Sat Nov 28 15:59:41 2009 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sat, 28 Nov 2009 15:59:41 +0100 Subject: What's cooking in erlang/otp Message-ID: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> This is the first in a series of emails that will be sent approximately twice a week to erlang-patches@REDACTED This first email is also sent to erlang-questions. If you don't already subscribe to erlang-patches, and want to be informed about what new features and bug fixes that may make it into the next release, or to take part in the discussion, make sure that you subscribe to erlang-patches. Below you'll find a list of the proposed updates that have currently been merged to the 'pu' branch in the erlang/otp repository at github. To avoid having patches falling to the floor, as have unfortunately frequently happened in the past, we will now either merge proposed updates that are not obviously wrong or inappropriate to the 'pu' branch, or we will tell you what the problem is. Typically, one of those things should happen within a workday. There some is information about patch submitting on the wiki pages for erlang/otp at github (http://wiki.github.com/erlang/otp) and we will continue to update and improve that information. Here is the list: * ks/dialyzer-wx-fix (2009-11-28) 1 commit: - dialyzer: make -wx always start the SMP emulator Will soon be graduated to ccase/r13b04. * sv/sys_get_status (2009-11-27) 1 commit: - This change restores the ability for gen_server and gen_fsm callback modules to format their own state for display under the sys:get_status/0 call. This ability is extremely useful for new behavior modules based on gen_server or gen_fsm, so that they can display their status in a more meaningful way than just dumping the state record. It is also generally useful for applications wanting to display their gen_server or gen_fsm callback module state in something other than the default manner. Steve is working on documentation and test cases. * bg/on_load-types (2009-11-27) 1 commit: - Add type information for on_load Confirmed by Kostis that it is OK, so it will soon be graduated to ccase/r13b04_dev. * egil/readme-debug-emulator (2009-11-27) 2 commits: - Fix grammar - added runtime system debug build notes Will soon be graduated to ccase/r13b04. * bd/http_chunk_fix (2009-11-27) 1 commit: - http_chunk data handling/passing improvement. * ng/readme (2009-11-26) 2 commits: - Adding back the copyright markers. - Cleaning up some small formatting issues in the README. Will soon be graduated to ccase/r13b04_dev. * po/odbc-update-delete-no-rows (2009-11-26) 1 commit: - SQL_NO_DATA and SQLSTATE 00000 indicate success for updates/deletes that affect no rows * egil/misc-egd (2009-11-26) 3 commits: - re-added egd color specs and removed commented code - added test cases for egd_primitives - egd speed optimizations and code reduction -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From kiszl@REDACTED Sat Nov 28 16:29:55 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sat, 28 Nov 2009 16:29:55 +0100 Subject: [erlang-questions] filelib:is_* is missing something necessary. In-Reply-To: References: Message-ID: <4B1141F3.2090605@tmit.bme.hu> Michael Richter wrote: > I found out that the file type predicates in *filelib* (*is_file/1*, * > is_dir/1*, *is_regular/1*) have a little quirk that makes them not that > useful for my purposes. Specifically they treat symbolic links to files as > files and symbolic links to directories as directories instead of flagging > each respectively as symlinks. > > It seems to me that they're implemented in terms of > *read_file_info/1*instead of > *read_link_info/1* and I was wondering if it might not be possible to make > one of the following changes, depending on what people in the community > found most useful (I'm easy either way, personally): > > 1. Change the return values such that, say, *is_file/1* returns *true* if > the named file is a straight file, but returns *symlink* if it is a > symbolic link *to a file*. (In any other situation -- directory, symlink > to directory, special file, etc. -- it would still return *false*.) > 2. Add a predicate *is_symlink/1* so that I can first check if > something's a directory and then follow through with a check if it's a > symbolic link to one. > > Currently I've just rewritten the predicates for my code, but in general I > prefer to use system libraries wherever possible so I'd really like to see > one of the two above approaches (or perhaps a third, superior one) written > in. My predicate for *is_symlink/1* looks like this: > > is_symlink(Name) -> > case file:read_link_info(Name) of > {ok, FileInfo} -> FileInfo#file_info.type =:= symlink > end. > > It's not a particularly Earth-shattering change for option 2. Option 1 > could look something like this only without the fugliness of my quick hack.: > > is_regular(Name) -> > case file:read_file_info(Name) of > {ok, #file_info{type = regular}} -> > case file:read_link_info(Name) of > {ok, #file_info{type = symlink}} -> symlink; > _ -> true > end; > _ -> false > end. > > An is_xyz function returning anything other than 'true' or 'false' is certainly against the "least astonishment" principle, so I would not take that road... From the.ajarn@REDACTED Sat Nov 28 16:54:56 2009 From: the.ajarn@REDACTED (Brentley Jones) Date: Sat, 28 Nov 2009 09:54:56 -0600 Subject: [erlang-patches] What's cooking in erlang/otp In-Reply-To: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> References: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> Message-ID: I'm really liking the changes in the public-ness of Erlang as of R13B03; github, proposed updates (pu branch), and now this. All I have to say is keep it up! On a related note though, what invoked this change? - Brentley Jones On Nov 28, 2009, at 8:59 AM, Bj?rn Gustavsson wrote: > This is the first in a series of emails that will be sent > approximately twice a week to erlang-patches@REDACTED > > This first email is also sent to erlang-questions. If you > don't already subscribe to erlang-patches, and want to > be informed about what new features and bug fixes that > may make it into the next release, or to take part in the > discussion, make sure that you subscribe to erlang-patches. > > Below you'll find a list of the proposed updates that > have currently been merged to the 'pu' branch in the > erlang/otp repository at github. > > To avoid having patches falling to the floor, as have > unfortunately frequently happened in the past, we will now > either merge proposed updates that are not > obviously wrong or inappropriate to the 'pu' branch, or > we will tell you what the problem is. Typically, one of > those things should happen within a workday. > > There some is information about patch submitting on the > wiki pages for erlang/otp at github > (http://wiki.github.com/erlang/otp) and we will continue > to update and improve that information. > > Here is the list: > > * ks/dialyzer-wx-fix (2009-11-28) 1 commit: > - dialyzer: make -wx always start the SMP emulator > > Will soon be graduated to ccase/r13b04. > > * sv/sys_get_status (2009-11-27) 1 commit: > - This change restores the ability for gen_server and gen_fsm callback > modules to format their own state for display under the > sys:get_status/0 call. This ability is extremely useful for new > behavior modules based on gen_server or gen_fsm, so that they can > display their status in a more meaningful way than just dumping the > state record. It is also generally useful for applications wanting to > display their gen_server or gen_fsm callback module state in something > other than the default manner. > > Steve is working on documentation and test cases. > > * bg/on_load-types (2009-11-27) 1 commit: > - Add type information for on_load > > Confirmed by Kostis that it is OK, so it will soon be graduated to > ccase/r13b04_dev. > > * egil/readme-debug-emulator (2009-11-27) 2 commits: > - Fix grammar > - added runtime system debug build notes > > Will soon be graduated to ccase/r13b04. > > * bd/http_chunk_fix (2009-11-27) 1 commit: > - http_chunk data handling/passing improvement. > > * ng/readme (2009-11-26) 2 commits: > - Adding back the copyright markers. > - Cleaning up some small formatting issues in the README. > > Will soon be graduated to ccase/r13b04_dev. > > * po/odbc-update-delete-no-rows (2009-11-26) 1 commit: > - SQL_NO_DATA and SQLSTATE 00000 indicate success for updates/deletes > that affect no rows > > * egil/misc-egd (2009-11-26) 3 commits: > - re-added egd color specs and removed commented code > - added test cases for egd_primitives > - egd speed optimizations and code reduction > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-patches mailing list. See http://www.erlang.org/faq.html > erlang-patches (at) erlang.org > From dizzyd@REDACTED Sat Nov 28 17:40:59 2009 From: dizzyd@REDACTED (Dave Smith) Date: Sat, 28 Nov 2009 09:40:59 -0700 Subject: [erlang-questions] Re: [erlang-patches] What's cooking in erlang/otp In-Reply-To: References: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> Message-ID: 2009/11/28 Brentley Jones : > I'm really liking the changes in the public-ness of Erlang as of R13B03; github, proposed updates (pu branch), and now this. All I have to say is keep it up! +1. Thanks, Erlang/OTP team for taking this step -- I'm sure it entails more work. However, it's great to see the incremental changes and get a feel for how the next release is shaping up earlier in the process. D. From yan.beijing.china@REDACTED Sat Nov 28 19:58:46 2009 From: yan.beijing.china@REDACTED (Yan Yan) Date: Sun, 29 Nov 2009 02:58:46 +0800 Subject: A small question about TCP Socket of Erlang Message-ID: <33581d00911281058o74e081e0o33c334a1e11b963c@mail.gmail.com> Hi list, ( It is my first time of socket programming in Erlang, while my former experiences were all with Java and C++, so if there is any misunderstanding about the concept "socket", it is definitely mine :-) ) For description convenience, I changed some variable's names of "socket_examples.erl" on page 245~246 of Joe's book "Programming Erlang". Here is: For server, step 1: {ok, ListenSocket} = gen_tcp:listen(2345, ...). step 2: {ok, SocketOnServer} = gen_tcp:accept(ListenSocket). step 3: loop(SocketOnServer) to receive {tcp, SocketOnServer, BinData}. step 4: etc... For client, step a: {ok, SocketOnClient} = gen_tcp(Host, 2345, ...). step b: gen_tcp:send(SocketOnClient, BinData). step c: receive response (as the pattern of {tcp, SocketOnClient, BinData} ) from server. step d: etc... plus: In erlang doc, find gen_tcp:connect/3 or 4, we'll find Packets can be sent to the returned socket Socket using send/2. Packets sent from the peer are delivered as messages: {tcp, Socket, Data} Then the question is: After client's step b, a message as {tcp, SocketOnClient, Data} will be sent to server, but the server is in its step 3 looping to receive {tcp, SocketOnServer, BinData}. Why is there a match? SocketOnClient =:= SocketOnServer? Do server and client share the same socket after connection is set? ( Similar is why, after server send response message to client, is there a match in client's step c... ) I tried to read the source code but failed to get the answer. Shouldn't socket be the encapsulation of destination and transmission details only? ( So sockets in erlang contain details of both ends?) Thanks for your patience! Sincerely, Yan From joe@REDACTED Sat Nov 28 20:15:35 2009 From: joe@REDACTED (Joe Williams) Date: Sat, 28 Nov 2009 11:15:35 -0800 Subject: [erlang-questions] What's cooking in erlang/otp In-Reply-To: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> References: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> Message-ID: <4B1176D7.1040708@joetify.com> Awesome, like the others I am excited to see more openness in the development process. Thanks! -Joe On 11/28/09 6:59 AM, Bj?rn Gustavsson wrote: > This is the first in a series of emails that will be sent > approximately twice a week to erlang-patches@REDACTED > > This first email is also sent to erlang-questions. If you > don't already subscribe to erlang-patches, and want to > be informed about what new features and bug fixes that > may make it into the next release, or to take part in the > discussion, make sure that you subscribe to erlang-patches. > > Below you'll find a list of the proposed updates that > have currently been merged to the 'pu' branch in the > erlang/otp repository at github. > > To avoid having patches falling to the floor, as have > unfortunately frequently happened in the past, we will now > either merge proposed updates that are not > obviously wrong or inappropriate to the 'pu' branch, or > we will tell you what the problem is. Typically, one of > those things should happen within a workday. > > There some is information about patch submitting on the > wiki pages for erlang/otp at github > (http://wiki.github.com/erlang/otp) and we will continue > to update and improve that information. > > Here is the list: > > * ks/dialyzer-wx-fix (2009-11-28) 1 commit: > - dialyzer: make -wx always start the SMP emulator > > Will soon be graduated to ccase/r13b04. > > * sv/sys_get_status (2009-11-27) 1 commit: > - This change restores the ability for gen_server and gen_fsm callback > modules to format their own state for display under the > sys:get_status/0 call. This ability is extremely useful for new > behavior modules based on gen_server or gen_fsm, so that they can > display their status in a more meaningful way than just dumping the > state record. It is also generally useful for applications wanting to > display their gen_server or gen_fsm callback module state in something > other than the default manner. > > Steve is working on documentation and test cases. > > * bg/on_load-types (2009-11-27) 1 commit: > - Add type information for on_load > > Confirmed by Kostis that it is OK, so it will soon be graduated to > ccase/r13b04_dev. > > * egil/readme-debug-emulator (2009-11-27) 2 commits: > - Fix grammar > - added runtime system debug build notes > > Will soon be graduated to ccase/r13b04. > > * bd/http_chunk_fix (2009-11-27) 1 commit: > - http_chunk data handling/passing improvement. > > * ng/readme (2009-11-26) 2 commits: > - Adding back the copyright markers. > - Cleaning up some small formatting issues in the README. > > Will soon be graduated to ccase/r13b04_dev. > > * po/odbc-update-delete-no-rows (2009-11-26) 1 commit: > - SQL_NO_DATA and SQLSTATE 00000 indicate success for updates/deletes > that affect no rows > > * egil/misc-egd (2009-11-26) 3 commits: > - re-added egd color specs and removed commented code > - added test cases for egd_primitives > - egd speed optimizations and code reduction > > -- Name: Joseph A. Williams Email: joe@REDACTED Blog: http://www.joeandmotorboat.com/ From tony@REDACTED Sat Nov 28 21:09:35 2009 From: tony@REDACTED (Tony Arcieri) Date: Sat, 28 Nov 2009 13:09:35 -0700 Subject: What happens if the code server crashes? Message-ID: I'm working on my own language implementation on top of Erlang, and have something akin to my own code server (which proxies to the Erlang code server). I'm curious what the proper thing to do is if my code server crashes, so I thought I could take a lesson from how Erlang handles it. What does Erlang do when a bug in the code server is encountered? Does it bring down the entire VM? Is the state of all loaded modules persisted somewhere so the code server can restart and recover somehow? -- Tony Arcieri Medioh/Nagravision From max.lapshin@REDACTED Sat Nov 28 22:51:34 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 29 Nov 2009 00:51:34 +0300 Subject: Erlang & Runit Message-ID: Hi. Our production infrastructure is migrating to runit daemon. http://smarden.org/runit/ For making program a daemon, I must just write common program and make symlink from its executable to /etc/service/my_cool_service/run All its stdout can be stored in provided path with all features of logrotating. It is very convenient, especially for ruby scripts and non-daemonizable programs. But when I tried to launch erlang server through runit, I've experienced problems: it just fails after all application load. Have anybody tried to do it? From bgustavsson@REDACTED Sat Nov 28 23:07:33 2009 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sat, 28 Nov 2009 23:07:33 +0100 Subject: [erlang-patches] What's cooking in erlang/otp In-Reply-To: References: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> Message-ID: <6672d0160911281407u5738ffd4teb9fc2b5566baf8f@mail.gmail.com> 2009/11/28 Brentley Jones : > I'm really liking the changes in the public-ness of Erlang as of R13B03; github, proposed updates (pu branch), and now this. All I have to say is keep it up! Thanks! > On a related note though, what invoked this change? We have wanted to be more open for a while now, but still keep enough control to maintain OTP's reputation for stability and reliability. We were not sure how to achieve both goals at the same time. Subversion did not seem to be the answer. Git and github seem to be the answer. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bgustavsson@REDACTED Sat Nov 28 23:15:00 2009 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sat, 28 Nov 2009 23:15:00 +0100 Subject: [erlang-questions] Re: [erlang-patches] What's cooking in erlang/otp In-Reply-To: References: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> Message-ID: <6672d0160911281415t5fed633eh19e7057e9231a175@mail.gmail.com> 2009/11/28 Dave Smith : > 2009/11/28 Brentley Jones : >> I'm really liking the changes in the public-ness of Erlang as of R13B03; github, proposed updates (pu branch), and now this. All I have to say is keep it up! > > +1. Thanks, Erlang/OTP team for taking this step -- I'm sure it > entails more work. Yes, it will certainly be more work in the short term until we have worked out all procedures and written all scripts, but I think that we will be more productive in the long term. > However, it's great to see the incremental changes > and get a feel for how the next release is shaping up earlier in the > process. Yes, that will probably lower the barrier for sending in patches. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From tony@REDACTED Sun Nov 29 00:53:01 2009 From: tony@REDACTED (Tony Arcieri) Date: Sat, 28 Nov 2009 16:53:01 -0700 Subject: Why can't I use variables to denote members of a record? Message-ID: I ran into a case where it would really be helpful to do: Rec#some_record{Member = Value} And the Erlang preprocessor bitches: field 'Member' is not an atom or _ in record some_record I see no reason why this case can't be handled by the Erlang preprocessor with a case expression. Specifically this can be exploded into: case Member of member1 -> Rec#some_record{member1 = Value}; member2 -> Rec#some_record{member2 = Value}; ... memberN -> Rec#some_record{memberN = Value} end Then just do the normal preprocessor transforms for those records. You can handle multiple Member=Value assignments by folding over all of them with the same case logic. Am I the first one in the history of Erlang to desire this behaviour? As is I'm manually exploding it into a case expression myself, which means whenever I want to add a member to the original record I also have to update my case statement, and the list of the members is spelled out twice. For a bit more clarification on what I'm doing, I'm building a record from a list which contains both individual atoms and 2-tuples. The behavior of the individual atoms in the list is special cased, but the behavior when a 2-tuple is encountered is to treat them as {Member, Value} and set Member to Value accordingly. -- Tony Arcieri Medioh/Nagravision From ngocdaothanh@REDACTED Sun Nov 29 01:02:02 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Sun, 29 Nov 2009 09:02:02 +0900 Subject: [erlang-questions] A small question about TCP Socket of Erlang In-Reply-To: <33581d00911281058o74e081e0o33c334a1e11b963c@mail.gmail.com> References: <33581d00911281058o74e081e0o33c334a1e11b963c@mail.gmail.com> Message-ID: <5c493e530911281602t1fdbb71eh81d57e9cd60b21c2@mail.gmail.com> > Why is there a match? SocketOnClient =:= SocketOnServer? Do server and client share the same socket after connection is set? No. SocketOnServer in step 3 is for matching with SocketOnServer in step 2. SocketOnServer and SocketOnClient are just values and independent. To see their values, use io:format("~p~n", [SocketOnServer]) and io:format("~p~n", [SocketOnClient]). I think you're confused because of the resemblance of the code for server and the code for client. But actually this is the beauty of the design of the API. On Sun, Nov 29, 2009 at 3:58 AM, Yan Yan wrote: > Hi list, > > ( It is my first time of socket programming in Erlang, while my former > experiences were all with Java and C++, so if there is any misunderstanding > about the concept "socket", it is definitely mine :-) ?) > > For description convenience, I changed some variable's names of > "socket_examples.erl" on page 245~246 of Joe's book "Programming Erlang". > > Here is: > > For server, > step 1: ?{ok, ListenSocket} = gen_tcp:listen(2345, ...). > step 2: ?{ok, SocketOnServer} = gen_tcp:accept(ListenSocket). > step 3: ?loop(SocketOnServer) to receive {tcp, SocketOnServer, BinData}. > step 4: ?etc... > > For client, > step a: ?{ok, SocketOnClient} = gen_tcp(Host, 2345, ...). > step b: ?gen_tcp:send(SocketOnClient, BinData). > step c: ?receive response (as the pattern of {tcp, SocketOnClient, BinData} > ) from server. > step d: ?etc... > > plus: > In erlang doc, find gen_tcp:connect/3 or 4, we'll find > > ?Packets can be sent to the returned socket Socket using send/2. Packets > sent from the peer are delivered as messages: > ?{tcp, Socket, Data} > > > Then the question is: > After client's step b, a message as {tcp, SocketOnClient, Data} will be sent > to server, but the server is in its step 3 looping to receive {tcp, > SocketOnServer, BinData}. > Why is there a match? SocketOnClient =:= SocketOnServer? Do server and > client share the same socket after connection is set? > ( Similar is why, after server send response message to client, is there a > match in client's step c... ) > > I tried to read the source code but failed to get the answer. Shouldn't > socket be the encapsulation of destination and transmission details only? ( > So sockets in erlang contain details of both ends?) > > Thanks for your patience! > > Sincerely, > > Yan From the.ajarn@REDACTED Sun Nov 29 01:39:22 2009 From: the.ajarn@REDACTED (Brentley Jones) Date: Sat, 28 Nov 2009 18:39:22 -0600 Subject: Why can't I use variables to denote members of a record? In-Reply-To: References: Message-ID: If I remember correctly, all previous requests for this type of feature ended with someone saying "use setelement". Though, I also would like this type of functionality. The problem is that Erlang doesn't know what Member is equal to until runtime, and in that case it needs the case statement like you mention. Maybe that is a good trade off. - Brentley Jones On Nov 28, 2009, at 5:53 PM, Tony Arcieri wrote: > I ran into a case where it would really be helpful to do: > > Rec#some_record{Member = Value} > > And the Erlang preprocessor bitches: > > field 'Member' is not an atom or _ in record some_record > > I see no reason why this case can't be handled by the Erlang preprocessor > with a case expression. Specifically this can be exploded into: > > case Member of > member1 -> Rec#some_record{member1 = Value}; > member2 -> Rec#some_record{member2 = Value}; > ... > memberN -> Rec#some_record{memberN = Value} > end > > Then just do the normal preprocessor transforms for those records. You can > handle multiple Member=Value assignments by folding over all of them with > the same case logic. > > Am I the first one in the history of Erlang to desire this behaviour? As is > I'm manually exploding it into a case expression myself, which means > whenever I want to add a member to the original record I also have to update > my case statement, and the list of the members is spelled out twice. > > For a bit more clarification on what I'm doing, I'm building a record from a > list which contains both individual atoms and 2-tuples. The behavior of the > individual atoms in the list is special cased, but the behavior when a > 2-tuple is encountered is to treat them as {Member, Value} and set Member to > Value accordingly. > > -- > Tony Arcieri > Medioh/Nagravision From yan.beijing.china@REDACTED Sun Nov 29 04:15:32 2009 From: yan.beijing.china@REDACTED (=?utf-8?B?WWFuIFlhbg==?=) Date: Sat, 28 Nov 2009 19:15:32 -0800 Subject: =?utf-8?B?UmU6IFtlcmxhbmctcXVlc3Rpb25zXSBBIHNtYWxsIHF1ZXN0aW9uIGFib3V0IFRDUCBTb2NrZXQgb2YgRXJsYW5n?= References: <33581d00911281058o74e081e0o33c334a1e11b963c@mail.gmail.com> Message-ID: <200911281915297657811@gmail.com> Hi Ngoc, Thanks for your reply. But I am still confused on the message passed between ends. Is the message passed from client to server {tcp, SocketOnClient, BinData}, while server to client {tcp, SocketOnServer, BinData}? I am quite confused why there is a match within receive...end block of both ends, since SocketOnClient and SocketOnClient do not match. Sincerely, Yan 2009-11-28 From: Ngoc Dao Time: 2009-11-28 16:02:34 To: erlang-questions Fw: Subject: Re: [erlang-questions] A small question about TCP Socket of Erlang > Why is there a match? SocketOnClient =:= SocketOnServer? Do server and client share the same socket after connection is set? No. SocketOnServer in step 3 is for matching with SocketOnServer in step 2. SocketOnServer and SocketOnClient are just values and independent. To see their values, use io:format("~p~n", [SocketOnServer]) and io:format("~p~n", [SocketOnClient]). I think you're confused because of the resemblance of the code for server and the code for client. But actually this is the beauty of the design of the API. ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From ngocdaothanh@REDACTED Sun Nov 29 05:06:48 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Sun, 29 Nov 2009 13:06:48 +0900 Subject: [erlang-questions] A small question about TCP Socket of Erlang In-Reply-To: <200911281915297657811@gmail.com> References: <33581d00911281058o74e081e0o33c334a1e11b963c@mail.gmail.com> <200911281915297657811@gmail.com> Message-ID: <5c493e530911282006s1bc1195ev2dc50ad37f84fa66@mail.gmail.com> I see, the your confusion is you have "receive", but you don't know where the messages come from. You think messages received at the server are sent from the client and and vice versa, but actually they are not. Try a simpler experiment: * Run only your Erlang server, do no run your Erlang client. * Use "telnet" program as a client to connect to your Erlang server. You will see that messages received at the server are sent from the server itself, from an underlying process spawned by the API library. On Sun, Nov 29, 2009 at 12:15 PM, Yan Yan wrote: > Hi Ngoc, > > Thanks for your reply. But I am still confused on the message passed between > ends. > > Is the message passed from client to server {tcp, SocketOnClient, BinData}, > while server to client {tcp, SocketOnServer, BinData}? I am quite confused > why there is a match within receive...end block?of both ends, since > SocketOnClient and SocketOnClient do not match. > > Sincerely, > > Yan > 2009-11-28 > ________________________________ > From: Ngoc Dao > Time: ?2009-11-28?16:02:34 > To: erlang-questions > Fw: > Subject: Re: [erlang-questions] A small question about TCP Socket of Erlang > >>?Why?is?there?a?match??SocketOnClient?=:=?SocketOnServer??Do?server?and?client?share?the?same?socket?after?connection?is?set? > No.?SocketOnServer?in?step?3?is?for?matching?with?SocketOnServer?in > step?2.?SocketOnServer?and?SocketOnClient?are?just?values?and > independent.?To?see?their?values,?use?io:format("~p~n", > [SocketOnServer])?and?io:format("~p~n",?[SocketOnClient]). > I?think?you're?confused?because?of?the?resemblance?of?the?code?for > server?and?the?code?for?client.?But?actually?this?is?the?beauty?of?the > design?of?the?API. > ________________________________________________________________ > erlang-questions?mailing?list.?See?http://www.erlang.org/faq.html > erlang-questions?(at)?erlang.org From als@REDACTED Sun Nov 29 06:19:08 2009 From: als@REDACTED (Anthony Shipman) Date: Sun, 29 Nov 2009 16:19:08 +1100 Subject: [erlang-questions] Re: [erlang-patches] What's cooking in erlang/otp In-Reply-To: <6672d0160911281407u5738ffd4teb9fc2b5566baf8f@mail.gmail.com> References: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> <6672d0160911281407u5738ffd4teb9fc2b5566baf8f@mail.gmail.com> Message-ID: <200911291619.09306.als@iinet.net.au> On Sun, 29 Nov 2009 09:07:33 am Bj?rn Gustavsson wrote: > We have wanted to be more open for a while now, but still > keep enough control to maintain OTP's reputation for stability > and reliability. We were not sure how to achieve both goals > at the same time. > > Subversion did not seem to be the answer. > > Git and github seem to be the answer. How does git and github do it whereas subversion fails? -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From ttmrichter@REDACTED Sun Nov 29 07:06:37 2009 From: ttmrichter@REDACTED (Michael Richter) Date: Sun, 29 Nov 2009 14:06:37 +0800 Subject: [erlang-questions] Re: [erlang-patches] What's cooking in erlang/otp In-Reply-To: <200911291619.09306.als@iinet.net.au> References: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> <6672d0160911281407u5738ffd4teb9fc2b5566baf8f@mail.gmail.com> <200911291619.09306.als@iinet.net.au> Message-ID: 2009/11/29 Anthony Shipman > On Sun, 29 Nov 2009 09:07:33 am Bj?rn Gustavsson wrote: > > We have wanted to be more open for a while now, but still > > keep enough control to maintain OTP's reputation for stability > > and reliability. We were not sure how to achieve both goals > > at the same time. > > > Subversion did not seem to be the answer. > > > Git and github seem to be the answer. > > How does git and github do it whereas subversion fails? > Much easier branching and merging would be my guess. From ajuttner.list@REDACTED Sun Nov 29 07:52:55 2009 From: ajuttner.list@REDACTED (Alpar Juttner) Date: Sun, 29 Nov 2009 07:52:55 +0100 Subject: [erlang-questions] Re: [erlang-patches] What's cooking in erlang/otp In-Reply-To: <200911291619.09306.als@iinet.net.au> References: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> <6672d0160911281407u5738ffd4teb9fc2b5566baf8f@mail.gmail.com> <200911291619.09306.als@iinet.net.au> Message-ID: <1259477575.3325.10.camel@piko.site> On Sun, 2009-11-29 at 16:19 +1100, Anthony Shipman wrote: > On Sun, 29 Nov 2009 09:07:33 am Bj?rn Gustavsson wrote: > > We have wanted to be more open for a while now, but still > > keep enough control to maintain OTP's reputation for stability > > and reliability. We were not sure how to achieve both goals > > at the same time. > > > > Subversion did not seem to be the answer. > > > > Git and github seem to be the answer. > > How does git and github do it whereas subversion fails? Subversion is a centralized system, which divides the people into two categories - you either have a write access to the repository, then you can do everything without any control, or you don't have, then you can't directly contribute to the repository. In distributed RCS like git, the process of creating/commiting a changeset is separated from putting it into a repository (branch). Therefore you can make any kind of changes, your commits will appear under your name, but they can be reviewed before getting into the official repository. Regards, Alpar From yan.beijing.china@REDACTED Sun Nov 29 07:56:53 2009 From: yan.beijing.china@REDACTED (=?utf-8?B?WWFuIFlhbg==?=) Date: Sat, 28 Nov 2009 22:56:53 -0800 Subject: =?utf-8?B?UmU6IFtlcmxhbmctcXVlc3Rpb25zXSBBIHNtYWxsIHF1ZXN0aW9uIGFib3V0IFRDUCBTb2NrZXQgb2YgRXJsYW5n?= References: <33581d00911281058o74e081e0o33c334a1e11b963c@mail.gmail.com>, <200911281915297657811@gmail.com> Message-ID: <200911282256509681436@gmail.com> Cool, Ngoc Thanks a lot. Now I completely understand :-) Perfect! Such code is much more elegant and concise than socket in Java. Sincerely, Yan 2009-11-28 From: Ngoc Dao Time: 2009-11-28 20:07:30 To: erlang-questions Fw: Subject: Re: [erlang-questions] A small question about TCP Socket of Erlang I see, the your confusion is you have "receive", but you don't know where the messages come from. You think messages received at the server are sent from the client and and vice versa, but actually they are not. Try a simpler experiment: * Run only your Erlang server, do no run your Erlang client. * Use "telnet" program as a client to connect to your Erlang server. You will see that messages received at the server are sent from the server itself, from an underlying process spawned by the API library. On Sun, Nov 29, 2009 at 12:15 PM, Yan Yan wrote: > Hi Ngoc, > > Thanks for your reply. But I am still confused on the message passed between > ends. > > Is the message passed from client to server {tcp, SocketOnClient, BinData}, > while server to client {tcp, SocketOnServer, BinData}? I am quite confused > why there is a match within receive...end block of both ends, since > SocketOnClient and SocketOnClient do not match. > > Sincerely, > > Yan > 2009-11-28 > ________________________________ > From: Ngoc Dao > Time: 2009-11-28 16:02:34 > To: erlang-questions > Fw: > Subject: Re: [erlang-questions] A small question about TCP Socket of Erlang > >> Why is there a match? SocketOnClient =:= SocketOnServer? Do server and client share the same socket after connection is set? > No. SocketOnServer in step 3 is for matching with SocketOnServer in > step 2. SocketOnServer and SocketOnClient are just values and > independent. To see their values, use io:format("~p~n", > [SocketOnServer]) and io:format("~p~n", [SocketOnClient]). > I think you're confused because of the resemblance of the code for > server and the code for client. But actually this is the beauty of the > design of the API. > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From bgustavsson@REDACTED Sun Nov 29 09:00:48 2009 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sun, 29 Nov 2009 09:00:48 +0100 Subject: [erlang-questions] Re: [erlang-patches] What's cooking in erlang/otp In-Reply-To: <200911291619.09306.als@iinet.net.au> References: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> <6672d0160911281407u5738ffd4teb9fc2b5566baf8f@mail.gmail.com> <200911291619.09306.als@iinet.net.au> Message-ID: <6672d0160911290000y5d2946bfyb5533129e2ad1623@mail.gmail.com> 2009/11/29 Anthony Shipman : > > How does git and github do it whereas subversion fails? Michael Richter and Alp?r J?ttner emails explain why *git* does it. I'll just add that because of *github*, we didn't even have to set up a server with our own public git repository. That's one headache less. If we would have used Subversion (or any centralized VCS), we would have had to set up our own server and then worry about how it would scale when the numbers of users grew. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From als@REDACTED Sun Nov 29 09:28:42 2009 From: als@REDACTED (Anthony Shipman) Date: Sun, 29 Nov 2009 19:28:42 +1100 Subject: [erlang-questions] Re: [erlang-patches] What's cooking in erlang/otp In-Reply-To: <1259477575.3325.10.camel@piko.site> References: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> <200911291619.09306.als@iinet.net.au> <1259477575.3325.10.camel@piko.site> Message-ID: <200911291928.42654.als@iinet.net.au> On Sun, 29 Nov 2009 05:52:55 pm Alpar Juttner wrote: > In distributed RCS like git, the process of creating/commiting a > changeset is separated from putting it into a repository (branch). > Therefore you can make any kind of changes, your commits will appear > under your name, but they can be reviewed before getting into the > official repository. > > Regards, > Alpar Do you mean that people commit in repositories in their own computers and then later someone can pull the changes into the official repository? -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From ulf.wiger@REDACTED Sun Nov 29 10:51:51 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 29 Nov 2009 10:51:51 +0100 Subject: [erlang-questions] Why can't I use variables to denote members of a record? In-Reply-To: References: Message-ID: <4B124437.9070808@erlang-consulting.com> Tony Arcieri wrote: > I ran into a case where it would really be helpful to do: > > Rec#some_record{Member = Value} > > And the Erlang preprocessor bitches: > >[...] > Am I the first one in the history of Erlang to desire this behaviour? Not at all. It has in fact been debated quite a lot over the years. I once wrote a parse_transform called exprecs that offers a compromise by letting you 'export' a record, thereby auto- generating accessor functions for that record. http://forum.trapexit.org/viewtopic.php?p=21790#21790 I'm guessing that this won't entirely satisfy your needs, but if so, perhaps you can use it as a starting point for some similar transformation? BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From alex.arnon@REDACTED Sun Nov 29 11:56:02 2009 From: alex.arnon@REDACTED (Alex Arnon) Date: Sun, 29 Nov 2009 12:56:02 +0200 Subject: [erlang-questions] FreeBSD Erlang R13B03 port (lang/erlang) now officially available In-Reply-To: <20091127150438.GA99418@k2r.org> References: <20091126022900.GA69974@k2r.org> <20091127150438.GA99418@k2r.org> Message-ID: <944da41d0911290256h7b152385mfb7e7aa5d50c0d0c@mail.gmail.com> Any chance of an OpenBSD port as well? :) On Fri, Nov 27, 2009 at 5:04 PM, Kenji Rikitake wrote: > FreeBSD official port lang/erlang for R13B03 now available. > (Thanks to Giacomo Olgeni!) > > http://www.freebsd.org/cgi/cvsweb.cgi/ports/lang/erlang/ > http://www.freshports.org/lang/erlang/ > > Kenji Rikitake > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From puzza007@REDACTED Sun Nov 29 13:46:11 2009 From: puzza007@REDACTED (Paul Oliver) Date: Sun, 29 Nov 2009 12:46:11 +0000 Subject: [erlang-questions] Re: [erlang-patches] What's cooking in erlang/otp In-Reply-To: <200911291928.42654.als@iinet.net.au> References: <6672d0160911280659l679af131g4ee2c443b79f71f7@mail.gmail.com> <200911291619.09306.als@iinet.net.au> <1259477575.3325.10.camel@piko.site> <200911291928.42654.als@iinet.net.au> Message-ID: On Sun, Nov 29, 2009 at 8:28 AM, Anthony Shipman wrote: > On Sun, 29 Nov 2009 05:52:55 pm Alpar Juttner wrote: >> In distributed RCS like git, the process of creating/commiting a >> changeset is separated from putting it into a repository (branch). >> Therefore you can make any kind of changes, your commits will appear >> under your name, but they can be reviewed before getting into the >> official repository. >> >> Regards, >> Alpar > > Do you mean that people commit in repositories in their own computers and then > later someone can pull the changes into the official repository? > > -- > Anthony Shipman ? ? ? ? ? ? ? ? ? ?Mamas don't let your babies > als@REDACTED ? ? ? ? ? ? ? ? ? grow up to be outsourced. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > I found this Google tech talk on Git illuminating http://www.youtube.com/watch?v=8dhZ9BXQgc4 Cheers, Paul. From leap@REDACTED Sun Nov 29 14:24:05 2009 From: leap@REDACTED (Michael Turner) Date: Sun, 29 Nov 2009 13:24:05 +0000 Subject: [erlang-questions] Why can't I use variables to denote members of a record? In-Reply-To: <4B124437.9070808@erlang-consulting.com> Message-ID: Ulf - The link from your trapexit entry into the erlang mail archives http://forum.trapexit.org/viewtopic.php?p=21790#21790 seems to be dead. Direct search in mail archives at erlang.org on "exprecs" turns up messages that link to this same entry at trapexit.org, except for the first message you wrote on the subject of exprecs http://www.erlang.org/cgi-bin/ezmlm-cgi/4/20050 which says that the exprecs.erl code is attached. But I don't see how to get that attachment from the archives. -michael turner On 11/29/2009, "Ulf Wiger" wrote: >Tony Arcieri wrote: >> I ran into a case where it would really be helpful to do: >> >> Rec#some_record{Member = Value} >> >> And the Erlang preprocessor bitches: >> >>[...] >> Am I the first one in the history of Erlang to desire this behaviour? > >Not at all. It has in fact been debated quite a lot over >the years. > >I once wrote a parse_transform called exprecs that offers >a compromise by letting you 'export' a record, thereby auto- >generating accessor functions for that record. > >http://forum.trapexit.org/viewtopic.php?p=21790#21790 > >I'm guessing that this won't entirely satisfy your needs, >but if so, perhaps you can use it as a starting point for >some similar transformation? > >BR, >Ulf W >-- >Ulf Wiger >CTO, Erlang Training & Consulting Ltd >http://www.erlang-consulting.com > >________________________________________________________________ >erlang-questions mailing list. See http://www.erlang.org/faq.html >erlang-questions (at) erlang.org > > From leap@REDACTED Sun Nov 29 14:45:03 2009 From: leap@REDACTED (Michael Turner) Date: Sun, 29 Nov 2009 13:45:03 +0000 Subject: [erlang-questions] Why can't I use variables to denote members of a record? In-Reply-To: Message-ID: <9icPASGV.1259502303.4028570.leap@gol.com> Oh, wait, I found a version: http://code.google.com/p/svnarchive/source/browse/trunk/util/src/exprecs.erl?spec=svn18&r=18 -michael turner On 11/29/2009, "Michael Turner" wrote: >Ulf - > >The link from your trapexit entry into the erlang mail archives > > http://forum.trapexit.org/viewtopic.php?p=21790#21790 > >seems to be dead. > >Direct search in mail archives at erlang.org on "exprecs" turns up >messages that link to this same entry at trapexit.org, except for the >first message you wrote on the subject of exprecs > > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/20050 > >which says that the exprecs.erl code is attached. But I don't see how >to get that attachment from the archives. > >-michael turner > > >On 11/29/2009, "Ulf Wiger" wrote: > >>Tony Arcieri wrote: >>> I ran into a case where it would really be helpful to do: >>> >>> Rec#some_record{Member = Value} >>> >>> And the Erlang preprocessor bitches: >>> >>>[...] >>> Am I the first one in the history of Erlang to desire this behaviour? >> >>Not at all. It has in fact been debated quite a lot over >>the years. >> >>I once wrote a parse_transform called exprecs that offers >>a compromise by letting you 'export' a record, thereby auto- >>generating accessor functions for that record. >> >>http://forum.trapexit.org/viewtopic.php?p=21790#21790 >> >>I'm guessing that this won't entirely satisfy your needs, >>but if so, perhaps you can use it as a starting point for >>some similar transformation? >> >>BR, >>Ulf W >>-- >>Ulf Wiger >>CTO, Erlang Training & Consulting Ltd >>http://www.erlang-consulting.com >> >>________________________________________________________________ >>erlang-questions mailing list. See http://www.erlang.org/faq.html >>erlang-questions (at) erlang.org >> >> > >________________________________________________________________ >erlang-questions mailing list. See http://www.erlang.org/faq.html >erlang-questions (at) erlang.org > > From erlang@REDACTED Sun Nov 29 14:56:36 2009 From: erlang@REDACTED (Erik Reitsma) Date: Sun, 29 Nov 2009 14:56:36 +0100 Subject: Stopping spawned process Message-ID: <4B127D94.3030008@ernovation.com> Hi all, When I spawn an external process using open_port/2, and then I stop the Erlang VM, the spawned process is not terminated. For example, when I do the following the xterm remains. Erlang R13B01 (erts-5.7.2) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) 1> open_port({spawn_executable,"/usr/bin/xterm"},[]). #Port<0.424> 2> User switch command --> q Is there a way to have the spawned processes terminated when the Erlang VM terminates? Regards, *Erik. From max.lapshin@REDACTED Sun Nov 29 15:00:51 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 29 Nov 2009 17:00:51 +0300 Subject: [erlang-questions] Stopping spawned process In-Reply-To: <4B127D94.3030008@ernovation.com> References: <4B127D94.3030008@ernovation.com> Message-ID: There is no guaranteed way to kill child process in common unix =( Maybe another process watcher. From ulf.wiger@REDACTED Sun Nov 29 15:01:43 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 29 Nov 2009 15:01:43 +0100 Subject: [erlang-questions] Why can't I use variables to denote members of a record? In-Reply-To: <9icPASGV.1259502303.4028570.leap@gol.com> References: <9icPASGV.1259502303.4028570.leap@gol.com> Message-ID: <4B127EC7.4040208@erlang-consulting.com> Strange - for me, the attachment to exprecs v 0.3 works: http://forum.trapexit.org/viewtopic.php?p=21790#21790 BR, Ulf W Michael Turner wrote: > Oh, wait, I found a version: > > > http://code.google.com/p/svnarchive/source/browse/trunk/util/src/exprecs.erl?spec=svn18&r=18 > > > -michael turner > > > On 11/29/2009, "Michael Turner" wrote: > >> Ulf - >> >> The link from your trapexit entry into the erlang mail archives >> >> http://forum.trapexit.org/viewtopic.php?p=21790#21790 >> >> seems to be dead. >> >> Direct search in mail archives at erlang.org on "exprecs" turns up >> messages that link to this same entry at trapexit.org, except for >> the first message you wrote on the subject of exprecs >> >> http://www.erlang.org/cgi-bin/ezmlm-cgi/4/20050 >> >> which says that the exprecs.erl code is attached. But I don't see >> how to get that attachment from the archives. >> >> -michael turner >> >> >> On 11/29/2009, "Ulf Wiger" wrote: >> >> >>> Tony Arcieri wrote: >>>> I ran into a case where it would really be helpful to do: >>>> >>>> Rec#some_record{Member = Value} >>>> >>>> And the Erlang preprocessor bitches: >>>> >>>> [...] Am I the first one in the history of Erlang to desire >>>> this behaviour? >>> Not at all. It has in fact been debated quite a lot over the >>> years. >>> >>> I once wrote a parse_transform called exprecs that offers a >>> compromise by letting you 'export' a record, thereby auto- >>> generating accessor functions for that record. >>> >>> http://forum.trapexit.org/viewtopic.php?p=21790#21790 >>> >>> I'm guessing that this won't entirely satisfy your needs, but if >>> so, perhaps you can use it as a starting point for some similar >>> transformation? >>> >>> BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd >>> http://www.erlang-consulting.com >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ulf.wiger@REDACTED Sun Nov 29 15:03:44 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 29 Nov 2009 15:03:44 +0100 Subject: [erlang-questions] Why can't I use variables to denote members of a record? In-Reply-To: <4B127EC7.4040208@erlang-consulting.com> References: <9icPASGV.1259502303.4028570.leap@gol.com> <4B127EC7.4040208@erlang-consulting.com> Message-ID: <4B127F40.5030007@erlang-consulting.com> Ulf Wiger wrote: > > Strange - for me, the attachment to exprecs v 0.3 > works: > > http://forum.trapexit.org/viewtopic.php?p=21790#21790 Oh, sorry - it's of course 0.4 that is the newer version, but that one works too. :) BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From erlang@REDACTED Sun Nov 29 15:31:15 2009 From: erlang@REDACTED (Erik Reitsma) Date: Sun, 29 Nov 2009 15:31:15 +0100 Subject: [erlang-questions] Stopping spawned process In-Reply-To: References: <4B127D94.3030008@ernovation.com> Message-ID: <4B1285B3.5040308@ernovation.com> I checked what happens with other languages. In Python the behavior is similar to in Erlang, when I use subprocess.Popen: the child process stays alive when the Python process terminates. In Java, however, the child process is killed. Probably a list of child processes is maintained, and the children are sent a kill signal when the JVM is killed. In Erlang I do not get direct access to the PID of the child, neither do I get the opportunity to do anything when Erlang terminates (or do I?). Another process watcher would be a bit heavy for my purpose: a wrapper around mplayer, as a client to a DVB-S media server. Max Lapshin wrote: > There is no guaranteed way to kill child process in common unix =( > Maybe another process watcher. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From max.lapshin@REDACTED Sun Nov 29 15:39:20 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 29 Nov 2009 17:39:20 +0300 Subject: [erlang-questions] Stopping spawned process In-Reply-To: <4B1285B3.5040308@ernovation.com> References: <4B127D94.3030008@ernovation.com> <4B1285B3.5040308@ernovation.com> Message-ID: Yes, Erlang is poor in handling forked processes. It is hard to kill spawned process, impossible to get its stderr, impossible to chain processes via pipe(); However, there is still no warranty even with Java that child process will be killed. From leap@REDACTED Sun Nov 29 16:27:32 2009 From: leap@REDACTED (Michael Turner) Date: Sun, 29 Nov 2009 15:27:32 +0000 Subject: [erlang-questions] Why can't I use variables to denote members of a record? In-Reply-To: <4B127F40.5030007@erlang-consulting.com> Message-ID: Sorry, I didn't scroll down far enough to see that there was a download available at the mentioned trapexit entry. I'm used to seeing some indication of attachments in the headers for messages. And the erlang.org mail archive didn't have a link for the relevant attachment. -michael turner On 11/29/2009, "Ulf Wiger" wrote: >Ulf Wiger wrote: >> >> Strange - for me, the attachment to exprecs v 0.3 >> works: >> >> http://forum.trapexit.org/viewtopic.php?p=21790#21790 > >Oh, sorry - it's of course 0.4 that is the newer version, >but that one works too. :) > >BR, >Ulf W > >-- >Ulf Wiger >CTO, Erlang Training & Consulting Ltd >http://www.erlang-consulting.com > >________________________________________________________________ >erlang-questions mailing list. See http://www.erlang.org/faq.html >erlang-questions (at) erlang.org > > From rvirding@REDACTED Sun Nov 29 17:24:06 2009 From: rvirding@REDACTED (Robert Virding) Date: Sun, 29 Nov 2009 17:24:06 +0100 Subject: [erlang-questions] Stopping spawned process In-Reply-To: References: <4B127D94.3030008@ernovation.com> <4B1285B3.5040308@ernovation.com> Message-ID: <3dbc6d1c0911290824h2f4ee30el8d1e904b28e5c817@mail.gmail.com> When Erlang dies the stdin of the spawned process will be closed. If the spawned process checks its stdin it should be able to detect that Erlang has died (or the port has been closed). Robert 2009/11/29 Max Lapshin > Yes, Erlang is poor in handling forked processes. > It is hard to kill spawned process, impossible to get its stderr, > impossible to chain > processes via pipe(); > > > However, there is still no warranty even with Java that child process > will be killed. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From andrewmmc@REDACTED Sun Nov 29 17:59:34 2009 From: andrewmmc@REDACTED (andrew mmc) Date: Sun, 29 Nov 2009 17:59:34 +0100 Subject: [erlang-questions] Stopping spawned process In-Reply-To: <4B1285B3.5040308@ernovation.com> References: <4B127D94.3030008@ernovation.com> <4B1285B3.5040308@ernovation.com> Message-ID: Hello, What about taking the following steps: 1. spawn the child process using a bash/whatever script which writes the ID of the process to a file in a specific directory and then runs mplayer; 2. launch your application also using a script, so that when the VM dies or your application closes, the script then checks that directory for PID files then kills them one by one. It is clunky but should get the job done reliably. Andrew On Sun, Nov 29, 2009 at 3:31 PM, Erik Reitsma wrote: > I checked what happens with other languages. In Python the behavior is > similar to in Erlang, when I use subprocess.Popen: the child process stays > alive when the Python process terminates. In Java, however, the child > process is killed. Probably a list of child processes is maintained, and the > children are sent a kill signal when the JVM is killed. In Erlang I do not > get direct access to the PID of the child, neither do I get the opportunity > to do anything when Erlang terminates (or do I?). > > Another process watcher would be a bit heavy for my purpose: a wrapper > around mplayer, as a client to a DVB-S media server. > > > Max Lapshin wrote: > >> There is no guaranteed way to kill child process in common unix =( >> Maybe another process watcher. >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From dmitriid@REDACTED Sun Nov 29 20:43:43 2009 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Sun, 29 Nov 2009 21:43:43 +0200 Subject: exmpp joining groupchats In-Reply-To: <925F7B92-7882-4891-B0AF-6AC41C045B14@gmail.com> References: <925F7B92-7882-4891-B0AF-6AC41C045B14@gmail.com> Message-ID: <871917BF-A163-41D1-BB96-DCAF52A60423@gmail.com> Answering my own question :) Yes, it can. Send a presence, see http://xmpp.org/extensions/xep-0045.html > How can my bot/client writen in XMPP join a conference/groupchat? > > From ok@REDACTED Mon Nov 30 02:47:57 2009 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 30 Nov 2009 14:47:57 +1300 Subject: [erlang-questions] Re: Why can't I use variables to denote members of a record? In-Reply-To: References: Message-ID: The real-data-structure alternative to #records that Joe Armstrong and I independently proposed would handle this issue neatly. > From g@REDACTED Mon Nov 30 07:55:26 2009 From: g@REDACTED (Garrett Smith) Date: Mon, 30 Nov 2009 00:55:26 -0600 Subject: [erlang-questions] Stopping spawned process In-Reply-To: <3dbc6d1c0911290824h2f4ee30el8d1e904b28e5c817@mail.gmail.com> References: <4B127D94.3030008@ernovation.com> <4B1285B3.5040308@ernovation.com> <3dbc6d1c0911290824h2f4ee30el8d1e904b28e5c817@mail.gmail.com> Message-ID: On Sun, Nov 29, 2009 at 10:24 AM, Robert Virding wrote: > When Erlang dies the stdin of the spawned process will be closed. If the > spawned process checks its stdin it should be able to detect that Erlang has > died (or the port has been closed). +1 When running applications as ports, I specify a command line option to tell the app to run in a special "Erlang port" mode, which watches stdin and shuts down when it closes. It's hard to control the child process otherwise. Garrett From ulf.wiger@REDACTED Mon Nov 30 12:55:25 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 30 Nov 2009 12:55:25 +0100 Subject: [erlang-questions] OTP release tutorial examples missing In-Reply-To: <4B0B0931.5060601@bredband.net> References: <549b206a0911231038q458cedb6neb9b4e30152e2953@mail.gmail.com> <4B0AEE62.6010908@bredband.net> <401d3ba30911231347l4eebac39w474acce01ba692db@mail.gmail.com> <4B0B0931.5060601@bredband.net> Message-ID: <4B13B2AD.3030409@erlang-consulting.com> H?kan Stenholm wrote: > Attila Rajmund Nohl wrote: >> 2009/11/23, H?kan Stenholm : >> >>> Feng Yu wrote: >>> >>>> hi, >>>> >>>> A lot of the tutorials seem to have their files >>>> missing. For example, in the OTP release tutorial: >>>> >>>> http://www.trapexit.org/OTP_Release_Handling_Tutorial There is a general problem with the upload area, which is being worked on, but I have also fixed the Release Handling tutorial by storing the entire example in a subversion repository and pointing the links to it. http://www.trapexit.org/OTP_Release_Handling_Tutorial (I also made a very small addition, showing how to build the boot script using erlc, when having different lib roots that need setting.) BR, Ulf W >>>> >>>> >>> The link works for me (from Sweden 2009-11-23 21:18 local time), there >>> is plenty of content on the page for me. >>> >> >> I believe the problematic links are the >> http://www.trapexit.org/Upload/release_handling_tutorial/lib/base-1.0/src/base_server.erl, >> >> http://www.trapexit.org/Upload/release_handling_tutorial/lib/base-1.0/src/base_app.erl >> >> , etc. which shows "There is currently no text in this page, you can >> search for this page title in other pages or edit this page." >> > > Your right, the links to the various .erl, .rel, .script, .app ... files > all appear to be broken, I thought it was the actual tutorial pages that > where missing. > >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ulf.wiger@REDACTED Mon Nov 30 15:24:17 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 30 Nov 2009 15:24:17 +0100 Subject: deprecate or rewrite mnesia:transaction/1 Message-ID: <4B13D591.1030202@erlang-consulting.com> It has irked me for some time that everyone keeps using mnesia:transaction/1 instead of mnesia:activity/2. The main problem with mnesia:transaction/1 is that it can't be extended by changing activity modules. This means e.g. that if you outgrow your disc_only tables, what would have been a fairly simple operation to switch to fragmented tables (adding an environment variable and making the necessary schema changes) suddenly becomes a rewrite. The problem is that while mnesia:activity/2 and mnesia:transaction/1 call the same internal function, mnesia:transaction/6, in the former case, the environment variable -mnesia access_module is checked, while in the latter case, the access module is hard-coded to mnesia.erl. Would it be a terrible violation to change mnesia:transaction/1 so that it also respects the access_module variable? I have come to regard it as a misdirected case of backward compatibility, or perhaps even an oversight that it doesn't already. If there are strong reasons for keeping mnesia:transaction/1 as it is, I vote for having it deprecated in favour of mnesia:activity/2, or perhaps a shorter version: atomic(F) -> transaction(F, _Args = [], _Retries = infinity, mnesia_monitor:get_env(access_module)) ...or something like that. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From the.ajarn@REDACTED Mon Nov 30 15:29:47 2009 From: the.ajarn@REDACTED (Brentley Jones) Date: Mon, 30 Nov 2009 08:29:47 -0600 Subject: [erlang-questions] deprecate or rewrite mnesia:transaction/1 In-Reply-To: <4B13D591.1030202@erlang-consulting.com> References: <4B13D591.1030202@erlang-consulting.com> Message-ID: <39C8BA71-D03A-4F5B-8856-B816905F9BF0@gmail.com> Having read Joe's latest book on Erlang, I didn't even know about mnesia:activity. I thought transaction was the only way to go. With that said, I think that transaction should be changed, because I'm pretty sure there are others out there like me that use it out of habit. - Brentley Jones On Nov 30, 2009, at 8:24 AM, Ulf Wiger wrote: > > It has irked me for some time that everyone keeps using > mnesia:transaction/1 instead of mnesia:activity/2. > > The main problem with mnesia:transaction/1 is that it can't > be extended by changing activity modules. This means e.g. that > if you outgrow your disc_only tables, what would have been a > fairly simple operation to switch to fragmented tables (adding > an environment variable and making the necessary schema changes) > suddenly becomes a rewrite. > > The problem is that while mnesia:activity/2 and mnesia:transaction/1 > call the same internal function, mnesia:transaction/6, in the > former case, the environment variable -mnesia access_module is > checked, while in the latter case, the access module is hard-coded > to mnesia.erl. > > Would it be a terrible violation to change mnesia:transaction/1 > so that it also respects the access_module variable? I have come > to regard it as a misdirected case of backward compatibility, > or perhaps even an oversight that it doesn't already. > > If there are strong reasons for keeping mnesia:transaction/1 as > it is, I vote for having it deprecated in favour of mnesia:activity/2, > or perhaps a shorter version: > > atomic(F) -> transaction(F, _Args = [], _Retries = infinity, > mnesia_monitor:get_env(access_module)) > > ...or something like that. > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From johan.garpendahl@REDACTED Mon Nov 30 15:51:37 2009 From: johan.garpendahl@REDACTED (Johan Garpendahl) Date: Mon, 30 Nov 2009 15:51:37 +0100 Subject: IC examples Message-ID: <38E78E0735C49D448494F368109B801501B5C487@esealmw114.eemea.ericsson.se> I'm trying to compile and run the IC examples, specifically the c-server, but I'm having problems. A minor problem is that the installation looks slightly different than it did when the examples were written. I can easily rewrite the Makefile to handle that. Another problem is whether the init function in server.c should have 3 or 4 arguments. I've tried to work around that, but it still does not work. There are problems to /* let epmd know we are here */ as far as I could see. Has anyone been more successful in runinng the IC examples? Regards, Johan From peter.szilagyi@REDACTED Mon Nov 30 17:43:31 2009 From: peter.szilagyi@REDACTED (=?utf-8?Q?P=C3=A9ter_Szil=C3=A1gyi?=) Date: Mon, 30 Nov 2009 16:43:31 +0000 (GMT) Subject: Trapexit Open Source Erlang Project Crawler In-Reply-To: <10956402.70801259598936310.JavaMail.root@zimbra> Message-ID: <29504253.70901259599411348.JavaMail.root@zimbra> Hello everyone, We are proud to announce the Trapexit project crawler: a web crawler to go through popular open source hosting sites (GitHub, BitBucket, SuorceForge and GoogleCode) and index the Erlang components/projects found. It tries to collect a lot of generic information about the projects (name, description, owners, home page, project page, language breakdown, rss/atom feeds, etc), and present it in one big searchable index. As an extra, in case a project has forks (even if at multiple hosting sites), the crawler tries to identify them (and the original project) and display them accordingly on the index page. Also to make the whole tool more useful for the community, we've implemented a rating system, through which anyone with a Trapexit user can rate a component based on a multitude of criteria. You can access the tool at the Trapexit website: http://projects.trapexit.org . Have a go and let us have your feedback (preferably on the Trapexit forums), and if you have a trapexit account, rate the applications you use. Should you find an error or a missing open source Erlang component we did not find, then please contact us so that we can investigate. In case you're interested, the components used to create the crawler and the interface were: lhttpc (for the web page downloading), CouchDB as a storage engine for both the crawler and the website, couchbeam as the database interface and the Nitrogen Web Framework as the front-end for the website. Hope you like it, Peter PS: Sometimes the server has a hiccup and things slow down for a bit. We're still trying to figure out the reason behind this. --- P?ter Szil?gyi Erlang Training & Consulting Ltd. http://www.erlang-consulting.com From glpunzi@REDACTED Mon Nov 30 17:58:02 2009 From: glpunzi@REDACTED (Giuseppe Luigi Punzi) Date: Mon, 30 Nov 2009 17:58:02 +0100 Subject: Erlang on Windows from USB Message-ID: <1259600282.2790.101.camel@info01.REGNA> Hi all, There are some way, to use erlang from an USB? I, and a friend, we have a machine where we can't do installations. We need a "portable" erlang to develop. I installed on a WinXP box. Copied the erlang entire folder to my USB, and tried to run in from other WinXP machine. I updated the paths in "erl.5.7.4\bin\erl.ini" but, when try to run "erl.5.7.4\bin\erl.exe" gives me an error: "G:\Desarrollo\erl5.7.4\bin\erl.exe The aplication couldn't be started because it's configuration is incorrect. Reinstall the application could solve the problem" In the ini file, I updated the paths with double slash "\\". Cheers, and thanks. From glpunzi@REDACTED Mon Nov 30 18:00:54 2009 From: glpunzi@REDACTED (Giuseppe Luigi Punzi) Date: Mon, 30 Nov 2009 18:00:54 +0100 Subject: Presentation Message-ID: <1259600455.2790.102.camel@info01.REGNA> Hi all, As I do in all maillint list, I would like to present me. My name is Giuseppe, and I reasearching a little about erlang, to do some projects (server, and desktop applications) I have in mind. Thanks in advance for your help ;) From fabio.mazanatti@REDACTED Mon Nov 30 18:08:12 2009 From: fabio.mazanatti@REDACTED (Fabio Mazanatti) Date: Mon, 30 Nov 2009 15:08:12 -0200 Subject: [erlang-questions] Erlang on Windows from USB In-Reply-To: <1259600282.2790.101.camel@info01.REGNA> References: <1259600282.2790.101.camel@info01.REGNA> Message-ID: <535117ab0911300908i7b890b6g36f63164de5d79aa@mail.gmail.com> Hi Giuseppe, why didn't you install it directly on your pendrive, instead of copying an installation from your hardrive? I'm not aware of Erlang's install package for Windows, but I guess you can point the installation to any drive you want (that's standard Windows behaviour)... Cheers, Fabio Mazanatti On Mon, Nov 30, 2009 at 2:58 PM, Giuseppe Luigi Punzi < glpunzi@REDACTED> wrote: > Hi all, > > There are some way, to use erlang from an USB? I, and a friend, we have > a machine where we can't do installations. We need a "portable" erlang > to develop. > > I installed on a WinXP box. Copied the erlang entire folder to my USB, > and tried to run in from other WinXP machine. I updated the paths in > "erl.5.7.4\bin\erl.ini" but, when try to run "erl.5.7.4\bin\erl.exe" > gives me an error: > > "G:\Desarrollo\erl5.7.4\bin\erl.exe > The aplication couldn't be started because it's configuration is > incorrect. Reinstall the application could solve the problem" > > In the ini file, I updated the paths with double slash "\\". > > Cheers, and thanks. > From mevans@REDACTED Mon Nov 30 18:27:27 2009 From: mevans@REDACTED (Evans, Matthew) Date: Mon, 30 Nov 2009 12:27:27 -0500 Subject: Concurrent processes on multi-core platforms with lots of chatter Message-ID: Hi, I've been running messaging tests on R13B02, using both 8 core Intel and 8 core CAVIUM processors. The tests involve two or more processes that do nothing more than sit in a loop exchanging messages as fast as they can. These tests are, of course, not realistic (as in real applications do more than sit in a tight loop sending messages), so my findings will likely not apply to a real deployment. First the good news: When running tests that do more than just message passing the SMP features of R13B02 are leaps and bounds over R12B05 that I was running previously. What I have however noticed is that in a pure messaging test (lots of messages, in a tight loop) we appear to run into caching issues where messages are sent between processes that happen to be scheduled on different cores. This got me into thinking about a future enhancement to the Erlang VM: Process affinity. In this mode two or more processes that have a lot of IPC chatter would be associated into a group and executed on the same core. If the scheduler needed to move one process to another core - they would all be relocated. Although this grouping of processes could be done automatically by the VM I believe the decision making overhead would be too great, and it would likely make some poor choices as to what processes should be grouped together. Rather I would leave it to the developer to make these decisions, perhaps with a library similar to pg2. For example, library process affinity (paf) could have the functions: paf:create(Name,[Opts]) -> ok, {error, Reason} paf:join(Name,Pid,[Opts]) -> ok, {error, Reason} paf:leave(Name,Pid) -> ok paf:members(Name) -> MemberList An affinity group would be created with options for specifying the maximum size of the group (to ensure we don't have all processes on one core), a default membership time within a group (to ensure we don't unnecessarily keep a process in the group when there is no longer a need) and maybe an option to allow the group to be split over different cores if the group size reaches a certain threshold. A process would join the group with paf:join/3, and would be a member for the default duration (with options here to override the settings specified in paf:create). If the group is full the request is rejected (or maybe queued). After a period of time the process is removed from the group and a message {paf_leave, Pid} is sent to the process that issued the paf:join command. If needed the process could be re-joined at that time with another paf:join call. Any takers? R14B01 perhaps ;-) Thanks Matt From glpunzi@REDACTED Mon Nov 30 18:31:13 2009 From: glpunzi@REDACTED (Giuseppe Luigi Punzi) Date: Mon, 30 Nov 2009 18:31:13 +0100 Subject: Erlang on Windows from USB] Message-ID: <1259602274.2790.105.camel@info01.REGNA> I forgot the CC, sorry -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded message was scrubbed... From: Giuseppe Luigi Punzi Subject: Re: [erlang-questions] Erlang on Windows from USB Date: Mon, 30 Nov 2009 18:27:53 +0100 Size: 4403 URL: From rtrlists@REDACTED Mon Nov 30 18:35:00 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 30 Nov 2009 17:35:00 +0000 Subject: [erlang-questions] Erlang on Windows from USB In-Reply-To: <1259600282.2790.101.camel@info01.REGNA> References: <1259600282.2790.101.camel@info01.REGNA> Message-ID: <6a3ae47e0911300935q54e4e0fax720c794d36388cc3@mail.gmail.com> On Mon, Nov 30, 2009 at 4:58 PM, Giuseppe Luigi Punzi < glpunzi@REDACTED> wrote: > Hi all, > > There are some way, to use erlang from an USB? I, and a friend, we have > a machine where we can't do installations. We need a "portable" erlang > to develop. > > I installed on a WinXP box. Copied the erlang entire folder to my USB, > and tried to run in from other WinXP machine. I updated the paths in > "erl.5.7.4\bin\erl.ini" but, when try to run "erl.5.7.4\bin\erl.exe" > gives me an error: > > "G:\Desarrollo\erl5.7.4\bin\erl.exe > The aplication couldn't be started because it's configuration is > incorrect. Reinstall the application could solve the problem" > > In the ini file, I updated the paths with double slash "\\". > > Cheers, and thanks. > In the bin folder of your Erlang installation on Windows, you will find an erl.ini file. This contains some absolute paths that need to point to the correct locations of your installation root and the erts bin folder within that. As far as I am aware, the paths must be absolute (starting with drive letter and initial backslash) and all folder separators must be escaped backslashes (\\). You could make yourself a small batch file to start Erlang after having written over the erl.ini file. For example (using R12B, in R13B the erts version'll be different): setlocal set ROOT=%~dsp0% set ERTSPATH=%ROOT%erts-5.6.5\bin set ERLINI=%ERTSPATH%\erl.ini echo [erlang] >%ERLINI% echo Bindir=%ERTSPATH:\=\\% >>%ERLINI% echo Progname=erl >>%ERLINI% echo Rootdir=%ROOT:\=\\% >>%ERLINI% %ERTSPATH%\erl endlocal Put this into your Erlang root on the USB drive and you should be able to run it from there. (BTW, this batch file ignores the top level bin folder and uses the erts one directly.) And then adapt to pass in whatever options you need, set a better PATH to include OpenSSL, etc. Getting all this correct with erlsrv.exe for running Erlang as a Windows Service is a lot harder! Hope this helps, Robby PS The magic %~dsp0% is explained here: http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/percent.mspx From erlangy@REDACTED Mon Nov 30 18:36:14 2009 From: erlangy@REDACTED (erlangy@REDACTED) Date: Mon, 30 Nov 2009 09:36:14 -0800 Subject: [erlang-questions] Erlang on Windows from USB In-Reply-To: <535117ab0911300908i7b890b6g36f63164de5d79aa@mail.gmail.com> References: <1259600282.2790.101.camel@info01.REGNA> <535117ab0911300908i7b890b6g36f63164de5d79aa@mail.gmail.com> Message-ID: <20091130173614.GC11530@delora.autosys.us> I agree to install directly to pendrive from MS windows executable installer. When you invoke, use drive and path, for example F:\erl5.7.4\bin\werl.exe ... Maybe the fullpath will work with your existing copy. ~Michael On Mon, Nov 30, 2009 at 03:08:12PM -0200, Fabio Mazanatti wrote: > Hi Giuseppe, > > why didn't you install it directly on your pendrive, instead of copying an > installation from your hardrive? I'm not aware of Erlang's install package > for Windows, but I guess you can point the installation to any drive you > want (that's standard Windows behaviour)... > > > Cheers, > Fabio Mazanatti > > > On Mon, Nov 30, 2009 at 2:58 PM, Giuseppe Luigi Punzi < > glpunzi@REDACTED> wrote: > > > Hi all, > > > > There are some way, to use erlang from an USB? I, and a friend, we have > > a machine where we can't do installations. We need a "portable" erlang > > to develop. > > > > I installed on a WinXP box. Copied the erlang entire folder to my USB, > > and tried to run in from other WinXP machine. I updated the paths in > > "erl.5.7.4\bin\erl.ini" but, when try to run "erl.5.7.4\bin\erl.exe" > > gives me an error: > > > > "G:\Desarrollo\erl5.7.4\bin\erl.exe > > The aplication couldn't be started because it's configuration is > > incorrect. Reinstall the application could solve the problem" > > > > In the ini file, I updated the paths with double slash "\\". > > > > Cheers, and thanks. > > From glpunzi@REDACTED Mon Nov 30 18:38:25 2009 From: glpunzi@REDACTED (Giuseppe Luigi Punzi) Date: Mon, 30 Nov 2009 18:38:25 +0100 Subject: [erlang-questions] Erlang on Windows from USB In-Reply-To: <6a3ae47e0911300935q54e4e0fax720c794d36388cc3@mail.gmail.com> References: <1259600282.2790.101.camel@info01.REGNA> <6a3ae47e0911300935q54e4e0fax720c794d36388cc3@mail.gmail.com> Message-ID: <1259602706.2790.109.camel@info01.REGNA> > In the bin folder of your Erlang installation on Windows, you will > find an erl.ini file. This contains some absolute paths that need to > point to the correct locations of your installation root and the erts > bin folder within that. Yes, I changed it. [erlang] Bindir=G:\\Desarrollo\\erl5.7.4\\erts-5.7.4\\bin Progname=erl Rootdir=G:\\Desarrollo\\erl5.7.4 > > As far as I am aware, the paths must be absolute (starting with drive > letter and initial backslash) and all folder separators must be > escaped backslashes (\\). Did it.. > > You could make yourself a small batch file to start Erlang after > having written over the erl.ini file. For example (using R12B, in R13B > the erts version'll be different): > > setlocal > set ROOT=%~dsp0% > set ERTSPATH=%ROOT%erts-5.6.5\bin > set ERLINI=%ERTSPATH%\erl.ini > echo [erlang] >%ERLINI% > echo Bindir=%ERTSPATH:\=\\% >>%ERLINI% > echo Progname=erl >>%ERLINI% > echo Rootdir=%ROOT:\=\\% >>%ERLINI% > %ERTSPATH%\erl > endlocal > > Put this into your Erlang root on the USB drive and you should be able > to run it from there. (BTW, this batch file ignores the top level bin > folder and uses the erts one directly.) > > And then adapt to pass in whatever options you need, set a better PATH > to include OpenSSL, etc. > > Getting all this correct with erlsrv.exe for running Erlang as a > Windows Service is a lot harder! > > Hope this helps, > Robby > > PS The magic %~dsp0% is explained here: > http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/percent.mspx Oh, ok. I will try this. Thanks ;) From dima@REDACTED Mon Nov 30 18:43:32 2009 From: dima@REDACTED (Dmitry Vasiliev) Date: Mon, 30 Nov 2009 20:43:32 +0300 Subject: [erlang-questions] Trapexit Open Source Erlang Project Crawler In-Reply-To: <29504253.70901259599411348.JavaMail.root@zimbra> References: <29504253.70901259599411348.JavaMail.root@zimbra> Message-ID: <4B140444.90404@hlabs.spb.ru> P?ter Szil?gyi wrote: > We are proud to announce the Trapexit project crawler: a web crawler > to go through popular open source hosting sites (GitHub, BitBucket, > SuorceForge and GoogleCode) and index the Erlang components/projects > found. Will you plan to add Launchpad? -- Dmitry Vasiliev http://hlabs.spb.ru http://twitter.com/hdima From rtrlists@REDACTED Mon Nov 30 18:44:58 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 30 Nov 2009 17:44:58 +0000 Subject: [erlang-questions] Erlang on Windows from USB In-Reply-To: <1259602706.2790.109.camel@info01.REGNA> References: <1259600282.2790.101.camel@info01.REGNA> <6a3ae47e0911300935q54e4e0fax720c794d36388cc3@mail.gmail.com> <1259602706.2790.109.camel@info01.REGNA> Message-ID: <6a3ae47e0911300944u37d79973s8c570b47c853d0b7@mail.gmail.com> On Mon, Nov 30, 2009 at 5:38 PM, Giuseppe Luigi Punzi < glpunzi@REDACTED> wrote: > > In the bin folder of your Erlang installation on Windows, you will find an > erl.ini file. This contains some absolute paths that need to point to the > correct locations of your installation root and the erts bin folder within > that. > > > Yes, I changed it. > > [erlang] > Bindir=G:\\Desarrollo\\erl5.7.4\\erts-5.7.4\\bin > Progname=erl > Rootdir=G:\\Desarrollo\\erl5.7.4 > > > Ah, it's been so long since I looked at all of this. There's actually TWO erl.ini files! One in the top level bin, and another in the erts/bin folder. Methinks that probably they have to agree if you're invoking from the top level bin folder. I tend to ignore the top level bin folder and invoke the exe's in the erts folder instead. But I also pass in explicit config and boot options, so your mileage may vary. Robby PS There's also some registry entries, but I've never quite understood what they are for. From zabrane3@REDACTED Mon Nov 30 18:54:38 2009 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 30 Nov 2009 18:54:38 +0100 Subject: [erlang-questions] Trapexit Open Source Erlang Project Crawler In-Reply-To: <4B140444.90404@hlabs.spb.ru> References: <29504253.70901259599411348.JavaMail.root@zimbra> <4B140444.90404@hlabs.spb.ru> Message-ID: <18a1db030911300954s60d0b885pa653844f44960d72@mail.gmail.com> Hi P?ter ! Excellent intiative. We are proud to announce the Trapexit project crawler: a web crawler >> to go through popular open source hosting sites > > Is the crawler itself "open-source"? Regards Zabrane From peter.szilagyi@REDACTED Mon Nov 30 18:59:40 2009 From: peter.szilagyi@REDACTED (=?utf-8?Q?P=C3=A9ter_Szil=C3=A1gyi?=) Date: Mon, 30 Nov 2009 17:59:40 +0000 (GMT) Subject: [erlang-questions] Trapexit Open Source Erlang Project Crawler In-Reply-To: <4B140444.90404@hlabs.spb.ru> Message-ID: <23138651.71011259603980696.JavaMail.root@zimbra> Hello, Currently there aren't any plans on extending the crawler. If there is need for it we could consider it, but after a quick look through, I've found only 21 Erlang projects on Launchpad, many of which are already hosted elsewhere too. Should that number rise I'd be willing to extend it, but for a handful of projects I don't think the benefits outweigh the "costs" (development + maintenance time). Still if you have a very good reason why I should implement it, I'm looking forward to hearing it :). Have a great day, Peter ----- Original Message ----- From: "Dmitry Vasiliev" To: "P?ter Szil?gyi" Cc: "Erlang Questions Mailinglist" Sent: Monday, November 30, 2009 5:43:32 PM GMT +00:00 GMT Britain, Ireland, Portugal Subject: Re: [erlang-questions] Trapexit Open Source Erlang Project Crawler P?ter Szil?gyi wrote: > We are proud to announce the Trapexit project crawler: a web crawler > to go through popular open source hosting sites (GitHub, BitBucket, > SuorceForge and GoogleCode) and index the Erlang components/projects > found. Will you plan to add Launchpad? -- Dmitry Vasiliev http://hlabs.spb.ru http://twitter.com/hdima From peter.szilagyi@REDACTED Mon Nov 30 19:04:31 2009 From: peter.szilagyi@REDACTED (=?utf-8?Q?P=C3=A9ter_Szil=C3=A1gyi?=) Date: Mon, 30 Nov 2009 18:04:31 +0000 (GMT) Subject: [erlang-questions] Trapexit Open Source Erlang Project Crawler In-Reply-To: <9519737.71061259604037992.JavaMail.root@zimbra> Message-ID: <15539877.71081259604271845.JavaMail.root@zimbra> Hi, Glad you like it :). The crawler is heavily specialized to crawl the specifically mentioned hosting sites and to collect very specific infos (to keep the whole thing simple). Although it does collect infos about every component and only displays (and regularly updates) the Erlang ones, since it's not a generic crawler, you couldn't really use it for anything else. For this reason we chose not to make is open source. Have a nice day, Peter ----- Original Message ----- From: "zabrane Mikael" To: "Dmitry Vasiliev" Cc: "P?ter Szil?gyi" , "Erlang Questions Mailinglist" Sent: Monday, November 30, 2009 5:54:38 PM GMT +00:00 GMT Britain, Ireland, Portugal Subject: Re: [erlang-questions] Trapexit Open Source Erlang Project Crawler Hi P?ter ! Excellent intiative. We are proud to announce the Trapexit project crawler: a web crawler to go through popular open source hosting sites Is the crawler itself "open-source"? Regards Zabrane From erlang@REDACTED Mon Nov 30 19:17:55 2009 From: erlang@REDACTED (Erik Reitsma) Date: Mon, 30 Nov 2009 19:17:55 +0100 Subject: [erlang-questions] Stopping spawned process In-Reply-To: References: <4B127D94.3030008@ernovation.com> <4B1285B3.5040308@ernovation.com> <3dbc6d1c0911290824h2f4ee30el8d1e904b28e5c817@mail.gmail.com> Message-ID: <4B140C53.10403@ernovation.com> I would prefer not to modify mplayer, so I found a way around the fact that mplayer does not terminate when its stdin is closed. I wrap the call to mplayer in a small script as follows: #!/bin/sh (cat && kill 0) | mplayer $* This way I can still communicate with mplayer through stdin, (through the cat process). When stdin of cat is closed, it terminates. Then "kill 0" will terminate the shell and all its children (i.e. the mplayer process). Only is I manyally kill the shell process, the mplayer process will not be killed when I terminate the Erlang VM, but I can live with that... Thanks all for pointing me in this direction. Hopefully this solution will be helpful to others, or improved by others. *Erik. Garrett Smith wrote: > On Sun, Nov 29, 2009 at 10:24 AM, Robert Virding wrote: > >> When Erlang dies the stdin of the spawned process will be closed. If the >> spawned process checks its stdin it should be able to detect that Erlang has >> died (or the port has been closed). >> > > +1 > > When running applications as ports, I specify a command line option to > tell the app to run in a special "Erlang port" mode, which watches > stdin and shuts down when it closes. It's hard to control the child > process otherwise. > > Garrett > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From glpunzi@REDACTED Mon Nov 30 19:29:15 2009 From: glpunzi@REDACTED (Giuseppe Luigi Punzi) Date: Mon, 30 Nov 2009 19:29:15 +0100 Subject: [erlang-questions] Erlang on Windows from USB In-Reply-To: <20091130173614.GC11530@delora.autosys.us> References: <1259600282.2790.101.camel@info01.REGNA> <535117ab0911300908i7b890b6g36f63164de5d79aa@mail.gmail.com> <20091130173614.GC11530@delora.autosys.us> Message-ID: <1259605755.2790.118.camel@info01.REGNA> El lun, 30-11-2009 a las 09:36 -0800, erlangy@REDACTED escribi?: > I agree to install directly to pendrive from MS windows executable > installer. > > When you invoke, use drive and path, for example > > F:\erl5.7.4\bin\werl.exe ... > > Maybe the fullpath will work with your existing copy. > > ~Michael Hi, I suppose, install to pendrive directly is unnecesary. I don't know what installer exactly does, but, with reg entries, or some configuration in folders, once I take the USB and connect to other PC, the letter can change, and the reg entries are not there. I will try with a batch file, and invoking fullpath. My friend and me are researching with erlang for some web projects, and, we still learning at the moment. This question, It's not very important for me, in all my computers I can use erlang without problems, but my colleague, works at Spanish Army, and, computers are restringed there. For this, he can't install anything, and needs to get it working from USB, to develop from his work. From kiszl@REDACTED Mon Nov 30 19:35:45 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Mon, 30 Nov 2009 19:35:45 +0100 Subject: [erlang-questions] Concurrent processes on multi-core platforms with lots of chatter In-Reply-To: References: Message-ID: <4B141081.60201@tmit.bme.hu> Evans, Matthew wrote: > Hi, > > I've been running messaging tests on R13B02, using both 8 core Intel and 8 core CAVIUM processors. The tests involve two or more processes that do nothing more than sit in a loop exchanging messages as fast as they can. These tests are, of course, not realistic (as in real applications do more than sit in a tight loop sending messages), so my findings will likely not apply to a real deployment. > > First the good news: When running tests that do more than just message passing the SMP features of R13B02 are leaps and bounds over R12B05 that I was running previously. What I have however noticed is that in a pure messaging test (lots of messages, in a tight loop) we appear to run into caching issues where messages are sent between processes that happen to be scheduled on different cores. This got me into thinking about a future enhancement to the Erlang VM: Process affinity. > > In this mode two or more processes that have a lot of IPC chatter would be associated into a group and executed on the same core. If the scheduler needed to move one process to another core - they would all be relocated. > > Although this grouping of processes could be done automatically by the VM I believe the decision making overhead would be too great, and it would likely make some poor choices as to what processes should be grouped together. Rather I would leave it to the developer to make these decisions, perhaps with a library similar to pg2. > > For example, library process affinity (paf) could have the functions: > > paf:create(Name,[Opts]) -> ok, {error, Reason} > paf:join(Name,Pid,[Opts]) -> ok, {error, Reason} > paf:leave(Name,Pid) -> ok > paf:members(Name) -> MemberList > > An affinity group would be created with options for specifying the maximum size of the group (to ensure we don't have all processes on one core), a default membership time within a group (to ensure we don't unnecessarily keep a process in the group when there is no longer a need) and maybe an option to allow the group to be split over different cores if the group size reaches a certain threshold. > > A process would join the group with paf:join/3, and would be a member for the default duration (with options here to override the settings specified in paf:create). If the group is full the request is rejected (or maybe queued). After a period of time the process is removed from the group and a message {paf_leave, Pid} is sent to the process that issued the paf:join command. If needed the process could be re-joined at that time with another paf:join call. > > Any takers? R14B01 perhaps ;-) > > Thanks > > Matt > > Hi Matt, I think the "optimal" number of affinity groups and process bindings to these groups would pretty much depend on the number of schedulers, and in turn on the number of cores. Probably also depends on the CPU architecture. Doesn't this road lead to giving up the high-levelness of Erlang, where we produce hardware-dependent code? I don't want to end up writing inline BEAM code in the future... :) Also, having such a rigid grouping is not a good idea as it completely ignores the workload of the member processes. You might force a bunch of heavy-weight processes onto the same core, losing more than you gain. Anyway, in my opinion it would be a better approach if there was no grouping at all: we would only define the affinity of a process to another. This would result in a directed, fully-connected graph on all processes. Default edge label could be 1, so you only need to increase it when needed, something like process_flag(affinity, {Pid, 42}). You could also define some sort of workload-coefficient, put it on the vertices, and use a force-based alignment algorithm to assign processes to schedulers :) Regards, Zoltan. From dima@REDACTED Mon Nov 30 20:14:19 2009 From: dima@REDACTED (Dmitry Vasiliev) Date: Mon, 30 Nov 2009 22:14:19 +0300 Subject: [erlang-questions] Trapexit Open Source Erlang Project Crawler In-Reply-To: <23138651.71011259603980696.JavaMail.root@zimbra> References: <23138651.71011259603980696.JavaMail.root@zimbra> Message-ID: <4B14198B.1080009@hlabs.spb.ru> P?ter Szil?gyi wrote: > Currently there aren't any plans on extending the crawler. If there > is need for it we could consider it, but after a quick look through, > I've found only 21 Erlang projects on Launchpad, many of which are > already hosted elsewhere too. Should that number rise I'd be willing > to extend it, but for a handful of projects I don't think the > benefits outweigh the "costs" (development + maintenance time). > > Still if you have a very good reason why I should implement it, I'm > looking forward to hearing it :). The main reason that Launchpad is the only hosting site which support Bazaar DVCS (http://bazaar-vcs.org/en/). Currently Bazaar users feel yourself orphaned in the Erlang world. :-) Maybe it's still a good idea to release the crawler as open source if you need help with development/maintenance? Or you can just open some "plugin API" so for example I can create an Erlang module(s) which implements crawler logic for Launchpad. -- Dmitry Vasiliev http://hlabs.spb.ru http://twitter.com/hdima From peter.szilagyi@REDACTED Mon Nov 30 20:19:02 2009 From: peter.szilagyi@REDACTED (=?utf-8?Q?P=C3=A9ter_Szil=C3=A1gyi?=) Date: Mon, 30 Nov 2009 19:19:02 +0000 (GMT) Subject: [erlang-questions] Trapexit Open Source Erlang Project Crawler In-Reply-To: <4B14198B.1080009@hlabs.spb.ru> Message-ID: <21446323.71181259608742994.JavaMail.root@zimbra> Actually SourceForge also has Bazaar... I remember because it was a pain to add :P ----- Original Message ----- From: "Dmitry Vasiliev" To: "P?ter Szil?gyi" Cc: "Erlang Questions Mailinglist" Sent: Monday, November 30, 2009 7:14:19 PM GMT +00:00 GMT Britain, Ireland, Portugal Subject: Re: [erlang-questions] Trapexit Open Source Erlang Project Crawler P?ter Szil?gyi wrote: > Currently there aren't any plans on extending the crawler. If there > is need for it we could consider it, but after a quick look through, > I've found only 21 Erlang projects on Launchpad, many of which are > already hosted elsewhere too. Should that number rise I'd be willing > to extend it, but for a handful of projects I don't think the > benefits outweigh the "costs" (development + maintenance time). > > Still if you have a very good reason why I should implement it, I'm > looking forward to hearing it :). The main reason that Launchpad is the only hosting site which support Bazaar DVCS (http://bazaar-vcs.org/en/). Currently Bazaar users feel yourself orphaned in the Erlang world. :-) Maybe it's still a good idea to release the crawler as open source if you need help with development/maintenance? Or you can just open some "plugin API" so for example I can create an Erlang module(s) which implements crawler logic for Launchpad. -- Dmitry Vasiliev http://hlabs.spb.ru http://twitter.com/hdima From wallentin.dahlberg@REDACTED Mon Nov 30 20:45:54 2009 From: wallentin.dahlberg@REDACTED (Wallentin Dahlberg) Date: Mon, 30 Nov 2009 20:45:54 +0100 Subject: [erlang-questions] Concurrent processes on multi-core platforms with lots of chatter In-Reply-To: References: Message-ID: 2009/11/30 Evans, Matthew > First the good news: When running tests that do more than just message > passing the SMP features of R13B02 are leaps and bounds over R12B05 that I > was running previously. What I have however noticed is that in a pure > messaging test (lots of messages, in a tight loop) we appear to run into > caching issues where messages are sent between processes that happen to be > scheduled on different cores. This got me into thinking about a future > enhancement to the Erlang VM: Process affinity. > > In this mode two or more processes that have a lot of IPC chatter would be > associated into a group and executed on the same core. If the scheduler > needed to move one process to another core - they would all be relocated. > > Although this grouping of processes could be done automatically by the VM I > believe the decision making overhead would be too great, and it would likely > make some poor choices as to what processes should be grouped together. > Rather I would leave it to the developer to make these decisions, perhaps > with a library similar to pg2. > Process/scheduler affinity has been discussed at length before and there are several schools of thought on the matter. It is true that accounting and statistics gathering in the runtime has some overhead. However, this should be handled by the scheduler. The user (the erlang developer) might not have all the information at hand and if he doesn't he has to collect them which also costs. It would be easier for the system to schedule the process right. The scheduler still has collect other information and make decisions on memory models, memory distance, number of processing units, process to process message affinity, load balancing and other characteristics. It would be reasonable easy to model a scheduler algorithm after these charateristics and we can make it much more dynamic in the runtime system. Ideally the user shouldn't be concerned about the hardware he is running on. Thats my take on the situation anyway. Regards, Bj?rn-Egil From dima@REDACTED Mon Nov 30 21:01:11 2009 From: dima@REDACTED (Dmitry Vasiliev) Date: Mon, 30 Nov 2009 23:01:11 +0300 Subject: [erlang-questions] Trapexit Open Source Erlang Project Crawler In-Reply-To: <21446323.71181259608742994.JavaMail.root@zimbra> References: <21446323.71181259608742994.JavaMail.root@zimbra> Message-ID: <4B142487.4040002@hlabs.spb.ru> P?ter Szil?gyi wrote: > Actually SourceForge also has Bazaar... I remember because it was a pain to add :P You're right. SourceForge supports Bazaar 1.10 (well, it's little bit outdated version). Ok, it seems better to use Git for new Erlang projects. :-) -- Dmitry Vasiliev http://hlabs.spb.ru http://twitter.com/hdima From mevans@REDACTED Mon Nov 30 21:53:15 2009 From: mevans@REDACTED (Evans, Matthew) Date: Mon, 30 Nov 2009 15:53:15 -0500 Subject: [erlang-questions] Concurrent processes on multi-core platforms with lots of chatter In-Reply-To: References: Message-ID: Hi, I actually agree with both you and Zoltan. I absolutely don't want the developer to have any understanding of CPU architecture. But, at the same time I'm not 100% convinced that the runtime should know what an application is doing or needs to do (other than in the way it does today). This is one reason why I thought of something along the lines of a process group (Zoltan's idea has merits too). By that I mean the developer knows that process A, B, C and D are going to be involved in a large amount of IPC for the next few seconds (or longer, or shorter). The developer shouldn't care if it is single core or multi-core. All they are saying to the runtime is "please ensure that these processes are on the same scheduler for the next X micro seconds since they are going to be communicating a lot". Now the runtime could ignore that request if it thought things could run faster if they were on different cores. I think my point is that you are correct in that developers should not care what the CPU topology is, but at the same time you don't want the VM making bad choices too. Interesting discussion.... ________________________________ From: Wallentin Dahlberg [mailto:wallentin.dahlberg@REDACTED] Sent: Monday, November 30, 2009 2:46 PM To: Evans, Matthew Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Concurrent processes on multi-core platforms with lots of chatter 2009/11/30 Evans, Matthew > First the good news: When running tests that do more than just message passing the SMP features of R13B02 are leaps and bounds over R12B05 that I was running previously. What I have however noticed is that in a pure messaging test (lots of messages, in a tight loop) we appear to run into caching issues where messages are sent between processes that happen to be scheduled on different cores. This got me into thinking about a future enhancement to the Erlang VM: Process affinity. In this mode two or more processes that have a lot of IPC chatter would be associated into a group and executed on the same core. If the scheduler needed to move one process to another core - they would all be relocated. Although this grouping of processes could be done automatically by the VM I believe the decision making overhead would be too great, and it would likely make some poor choices as to what processes should be grouped together. Rather I would leave it to the developer to make these decisions, perhaps with a library similar to pg2. Process/scheduler affinity has been discussed at length before and there are several schools of thought on the matter. It is true that accounting and statistics gathering in the runtime has some overhead. However, this should be handled by the scheduler. The user (the erlang developer) might not have all the information at hand and if he doesn't he has to collect them which also costs. It would be easier for the system to schedule the process right. The scheduler still has collect other information and make decisions on memory models, memory distance, number of processing units, process to process message affinity, load balancing and other characteristics. It would be reasonable easy to model a scheduler algorithm after these charateristics and we can make it much more dynamic in the runtime system. Ideally the user shouldn't be concerned about the hardware he is running on. Thats my take on the situation anyway. Regards, Bj?rn-Egil From kenneth.lundin@REDACTED Mon Nov 30 22:02:08 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 30 Nov 2009 22:02:08 +0100 Subject: [erlang-questions] Erlang on Windows from USB In-Reply-To: <1259600282.2790.101.camel@info01.REGNA> References: <1259600282.2790.101.camel@info01.REGNA> Message-ID: Hi, >From Erlang/OTP R13B02 the erl.ini file in the erl.5.7.x\bin directory is no longer necessary. If erl.ini exist it is used but if you remove it you can drop the file tree erl.5.7.x anywhere in the file system. I suggest that you install Erlang on an USB and then delete or rename the erl.ini file. After having done that you don't need any extra scripts, just start werl.exe or erl.exe no matter what drive -letter your USB is mounted to. /Kenneth, Erlang/OTP Ericsson On Mon, Nov 30, 2009 at 5:58 PM, Giuseppe Luigi Punzi wrote: > Hi all, > > There are some way, to use erlang from an USB? I, and a friend, we have > a machine where we can't do installations. We need a "portable" erlang > to develop. > > I installed on a WinXP box. Copied the erlang entire folder to my USB, > and tried to run in from other WinXP machine. I updated the paths in > "erl.5.7.4\bin\erl.ini" but, when try to run "erl.5.7.4\bin\erl.exe" > gives me an error: > > "G:\Desarrollo\erl5.7.4\bin\erl.exe > The aplication couldn't be started because it's configuration is > incorrect. Reinstall the application could solve the problem" > > In the ini file, I updated the paths with double slash "\\". > > Cheers, and thanks. > From glpunzi@REDACTED Mon Nov 30 22:05:37 2009 From: glpunzi@REDACTED (Giuseppe Luigi Punzi Ruiz) Date: Mon, 30 Nov 2009 22:05:37 +0100 Subject: [erlang-questions] Erlang on Windows from USB In-Reply-To: References: <1259600282.2790.101.camel@info01.REGNA> Message-ID: <3EFCF5FE-3335-47D0-B28F-ADB91A38E9DA@lordzealon.com> Hi Keneth, Thanks a lot, I will try tomorrow. El 30/11/2009, a las 22:02, Kenneth Lundin escribi?: > Hi, > > From Erlang/OTP R13B02 the erl.ini file in the erl.5.7.x\bin directory > is no longer necessary. > If erl.ini exist it is used but if you remove it you can drop the file > tree erl.5.7.x anywhere in the file system. > > I suggest that you install Erlang on an USB and then delete or rename > the erl.ini file. > After having done that you don't need any extra scripts, just start > werl.exe or erl.exe no matter what > drive -letter your USB is mounted to. > > /Kenneth, Erlang/OTP Ericsson > > On Mon, Nov 30, 2009 at 5:58 PM, Giuseppe Luigi Punzi > wrote: >> Hi all, >> >> There are some way, to use erlang from an USB? I, and a friend, we >> have >> a machine where we can't do installations. We need a "portable" >> erlang >> to develop. >> >> I installed on a WinXP box. Copied the erlang entire folder to my >> USB, >> and tried to run in from other WinXP machine. I updated the paths in >> "erl.5.7.4\bin\erl.ini" but, when try to run "erl.5.7.4\bin\erl.exe" >> gives me an error: >> >> "G:\Desarrollo\erl5.7.4\bin\erl.exe >> The aplication couldn't be started because it's configuration is >> incorrect. Reinstall the application could solve the problem" >> >> In the ini file, I updated the paths with double slash "\\". >> >> Cheers, and thanks. >> Giuseppe Luigi Punzi Ruiz Blog: http://www.lordzealon.com Twitter & Skype & GoogleTalk accounts: glpunzi From jarrod@REDACTED Mon Nov 30 22:06:32 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Mon, 30 Nov 2009 16:06:32 -0500 Subject: Processing a List into a Dict Message-ID: I have the following list ["txtvers=1","userid=3A6524D4-E31C-491D-94DD-555883B1600A","name=Jarrod Roberson","version=2"] I want to create a Dict where the left side of the = is the key and the right side is the value. Preferably where the key is an atom. Using the following list comprehension I get this. KVL = [string:tokens(T,"=") || T <- TXT]. [["txtvers","1"], ["userid","3A6524D4-E31C-491D-94DD-555883B1600A"], ["name","Jarrod Roberson"], ["version","2"]] what I am struggling with now is how to convert the nested lists into tuples so I can send them into a list of tuples where I can send them into dict:from_list what I want is something like this [{txtvers,"1"}, {userid,"3A6524D4-E31C-491D-94DD-555883B1600A"}, {name,"Jarrod Roberson"}, {version,"2"}] I know there has to be a concise way to do this but I just can't get my head around it. From gleber.p@REDACTED Mon Nov 30 22:12:57 2009 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 30 Nov 2009 22:12:57 +0100 Subject: [erlang-questions] Processing a List into a Dict In-Reply-To: References: Message-ID: <14f0e3620911301312l3cb237des619017f64b4095ff@mail.gmail.com> 1> X = ["txtvers=1","userid=3A6524D4-E31C-491D-94DD-555883B1600A","name=Jarrod Roberson","version=2"]. 2> dict:from_list([list_to_tuple(string:tokens(T,"=")) || T <- X]). {dict,4,16,16,8,80,48, {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, {{[],[],[],[], [["name",74,97,114,114,111,100,10,82,111,98,101,114|...]], [],[], [["txtvers",49]], [],[],[], [["userid",51,65,54,53,50|...]], [], [["version",50]], [],[]}}} On Mon, Nov 30, 2009 at 22:06, Jarrod Roberson wrote: > I have the following list > > ["txtvers=1","userid=3A6524D4-E31C-491D-94DD-555883B1600A","name=Jarrod > Roberson","version=2"] > > I want to create a Dict where the left side of the = is the key and the > right side is the value. > Preferably where the key is an atom. > > Using the following list comprehension I get this. > > ?KVL = [string:tokens(T,"=") || T <- TXT]. > > [["txtvers","1"], > ["userid","3A6524D4-E31C-491D-94DD-555883B1600A"], ["name","Jarrod > Roberson"], ["version","2"]] > > what I am struggling with now is how to convert the nested lists into tuples > so I can send them into a list of tuples > where I can send them into dict:from_list > > what I want is something like this > > [{txtvers,"1"}, {userid,"3A6524D4-E31C-491D-94DD-555883B1600A"}, > {name,"Jarrod Roberson"}, {version,"2"}] > > I know there has to be a concise way to do this but I just can't get my head > around it. > From dima@REDACTED Mon Nov 30 22:15:31 2009 From: dima@REDACTED (Dmitry Vasiliev) Date: Tue, 01 Dec 2009 00:15:31 +0300 Subject: [erlang-questions] Processing a List into a Dict In-Reply-To: References: Message-ID: <4B1435F3.2000107@hlabs.spb.ru> Jarrod Roberson wrote: > what I want is something like this > > [{txtvers,"1"}, {userid,"3A6524D4-E31C-491D-94DD-555883B1600A"}, > {name,"Jarrod Roberson"}, {version,"2"}] > > I know there has to be a concise way to do this but I just can't get my head > around it. It's simple: [list_to_tuple(string:tokens(T,"=")) || T <- TXT]. -- Dmitry Vasiliev http://hlabs.spb.ru http://twitter.com/hdima From the.ajarn@REDACTED Mon Nov 30 22:17:30 2009 From: the.ajarn@REDACTED (Brentley Jones) Date: Mon, 30 Nov 2009 15:17:30 -0600 Subject: [erlang-questions] Processing a List into a Dict In-Reply-To: References: Message-ID: <407911BF-D015-4759-B75C-DEBCD63B9473@gmail.com> list_to_tuple in erlang module. - Brentley Jones On Nov 30, 2009, at 3:06 PM, Jarrod Roberson wrote: > I have the following list > > ["txtvers=1","userid=3A6524D4- > E31C-491D-94DD-555883B1600A","name=Jarrod > Roberson","version=2"] > > I want to create a Dict where the left side of the = is the key and > the > right side is the value. > Preferably where the key is an atom. > > Using the following list comprehension I get this. > > KVL = [string:tokens(T,"=") || T <- TXT]. > > [["txtvers","1"], > ["userid","3A6524D4-E31C-491D-94DD-555883B1600A"], ["name","Jarrod > Roberson"], ["version","2"]] > > what I am struggling with now is how to convert the nested lists > into tuples > so I can send them into a list of tuples > where I can send them into dict:from_list > > what I want is something like this > > [{txtvers,"1"}, {userid,"3A6524D4-E31C-491D-94DD-555883B1600A"}, > {name,"Jarrod Roberson"}, {version,"2"}] > > I know there has to be a concise way to do this but I just can't get > my head > around it. From seancribbs@REDACTED Mon Nov 30 22:17:59 2009 From: seancribbs@REDACTED (Sean Cribbs) Date: Mon, 30 Nov 2009 16:17:59 -0500 Subject: [erlang-questions] Processing a List into a Dict In-Reply-To: References: Message-ID: <4B143687.1090207@gmail.com> Why not do it like so: [list_to_tuple(string:tokens(T,"=")) || T <- TXT ]. Returns: [{"txtvers","1"}, {"userid","3A6524D4-E31C-491D-94DD-555883B1600A"}, {"name","Jarrod Roberson"}, {"version","2"}] Sean Jarrod Roberson wrote: > I have the following list > > ["txtvers=1","userid=3A6524D4-E31C-491D-94DD-555883B1600A","name=Jarrod > Roberson","version=2"] > > I want to create a Dict where the left side of the = is the key and the > right side is the value. > Preferably where the key is an atom. > > Using the following list comprehension I get this. > > KVL = [string:tokens(T,"=") || T <- TXT]. > > [["txtvers","1"], > ["userid","3A6524D4-E31C-491D-94DD-555883B1600A"], ["name","Jarrod > Roberson"], ["version","2"]] > > what I am struggling with now is how to convert the nested lists into tuples > so I can send them into a list of tuples > where I can send them into dict:from_list > > what I want is something like this > > [{txtvers,"1"}, {userid,"3A6524D4-E31C-491D-94DD-555883B1600A"}, > {name,"Jarrod Roberson"}, {version,"2"}] > > I know there has to be a concise way to do this but I just can't get my head > around it. > > From jarrod@REDACTED Mon Nov 30 22:22:23 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Mon, 30 Nov 2009 16:22:23 -0500 Subject: [erlang-questions] Processing a List into a Dict In-Reply-To: <4B1435F3.2000107@hlabs.spb.ru> References: <4B1435F3.2000107@hlabs.spb.ru> Message-ID: Thanks for the responses, that got me on the right track, here is what I came up with. TXT = ["txtvers=1","userid=3A6524D4-E31C-491D-94DD-555883B1600A","name=Jarrod Roberson","version=2"]. A = [ string:tokens(KV,"=") || KV <- TXT]. [["txtvers","1"],["userid","3A6524D4-E31C-491D-94DD-555883B1600A"],["name","Jarrod Roberson"],["version","2"]] D = [{list_to_atom(K),V} || [K|[V|_]] <- A]. [{txtvers,"1"},{userid,"3A6524D4-E31C-491D-94DD-555883B1600A"},{name,"Jarrod Roberson"},{version,"2"}] DT = dict:from_list(D). {dict,4,16,16,8,80,48, {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, {{[],[],[], [[txtvers,49]], [[userid,51,65,54,53,50,52,68,52,45,69,51,49|...]], [[name,74,97,114,114,111,100,32,82,111,98,101|...]], [],[],[],[],[],[],[],[], [[version,50]], []}}} dict:find(name,DT). {ok,"Jarrod Roberson"} this gets me the keys as atoms instead of "strings" From kiszl@REDACTED Mon Nov 30 22:24:05 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Mon, 30 Nov 2009 22:24:05 +0100 Subject: [erlang-questions] Processing a List into a Dict In-Reply-To: References: Message-ID: <4B1437F5.2070501@tmit.bme.hu> Jarrod Roberson wrote: > I have the following list > > ["txtvers=1","userid=3A6524D4-E31C-491D-94DD-555883B1600A","name=Jarrod > Roberson","version=2"] > > I want to create a Dict where the left side of the = is the key and the > right side is the value. > Preferably where the key is an atom. > > Using the following list comprehension I get this. > > KVL = [string:tokens(T,"=") || T <- TXT]. > > [["txtvers","1"], > ["userid","3A6524D4-E31C-491D-94DD-555883B1600A"], ["name","Jarrod > Roberson"], ["version","2"]] > > what I am struggling with now is how to convert the nested lists into tuples > so I can send them into a list of tuples > where I can send them into dict:from_list > > what I want is something like this > > [{txtvers,"1"}, {userid,"3A6524D4-E31C-491D-94DD-555883B1600A"}, > {name,"Jarrod Roberson"}, {version,"2"}] > > I know there has to be a concise way to do this but I just can't get my head > around it. > > KVL = [begin [K,V]=string:tokens(T,"="), {list_to_atom(K), V} end || T <- L]. If this feels ugly, you can move the begin-end part to a separate list. If you only need this list for creating a dictionary, you can also foldl on your original list: lists:foldl(fun(T, D) -> [K,V] = string:tokens(T, "="), dict:store(list_to_atom(K), V, D) end, dict:new(), L). Regards, oltan. From jarrod@REDACTED Mon Nov 30 22:26:08 2009 From: jarrod@REDACTED (Jarrod Roberson) Date: Mon, 30 Nov 2009 16:26:08 -0500 Subject: [erlang-questions] Processing a List into a Dict In-Reply-To: <4B1437F5.2070501@tmit.bme.hu> References: <4B1437F5.2070501@tmit.bme.hu> Message-ID: On Mon, Nov 30, 2009 at 4:24 PM, Zoltan Lajos Kis wrote: > KVL = [begin [K,V]=string:tokens(T,"="), {list_to_atom(K), V} end || T <- > L]. > If this feels ugly, you can move the begin-end part to a separate list. > > If you only need this list for creating a dictionary, you can also foldl on > your original list: > > lists:foldl(fun(T, D) -> > [K,V] = string:tokens(T, "="), > dict:store(list_to_atom(K), V, D) > end, > dict:new(), > L). > > Regards, > oltan. > > thanks this is even more compact than what I came up with, I knew there was some fancy one line way of doing this. From wallentin.dahlberg@REDACTED Mon Nov 30 22:49:46 2009 From: wallentin.dahlberg@REDACTED (Wallentin Dahlberg) Date: Mon, 30 Nov 2009 22:49:46 +0100 Subject: [erlang-questions] Concurrent processes on multi-core platforms with lots of chatter In-Reply-To: References: Message-ID: 2009/11/30 Evans, Matthew > This is one reason why I thought of something along the lines of a process > group (Zoltan?s idea has merits too). By that I mean the developer knows > that process A, B, C and D are going to be involved in a large amount of IPC > for the next few seconds (or longer, or shorter). The developer shouldn?t > care if it is single core or multi-core. All they are saying to the runtime > is ?please ensure that these processes are on the same scheduler for the > next X micro seconds since they are going to be communicating a lot?. Now > the runtime could ignore that request if it thought things could run faster > if they were on different cores. > Yes, the runtime system can only make decisions by observing the past, it knows nothing about the future. When sudden shifts in processes messaging patterns occurs, or completly new processes with new patterns emerges, the scheduler won't know anything about how to handle it and probably won't schedule those processes optimal. To do it reasonable we have to assume that messaging patterns converge to some steadystate in the system. We have to do some thinking on this to see if it is a reasonable assumption. The point is the system should be able to learn the best scheduling strategy given enough time. > > > I think my point is that you are correct in that developers should not care > what the CPU topology is, but at the same time you don?t want the VM making > bad choices too. > We could help the scheduler along by letting the user define some abstractions. Something that scheduler can look at if it feels uncertain, like process groups. Or perhaps it can take advantage of some of the design patterns and behaviours already in OTP? It will be alot of tweaking for finding the right balance between good enough scheduling, information, decision calculation and overall cost. But messaging patterns are only one part of the problem. When I think about this scheduling problem I often think about it like Zoltan describes it, a force graph. It is just dead simple that way. A second order diff.eq. like nature would have designed it. The forces involved to each scheduler are just rules that has to obeyed, each term is a criteria, message passing patterns, cpu topology and the cost of moving, memory hierachy, priorities, load differences, user defined hints, etc and then just do the math each scheduling iteration. Now, if we just could make accounting and statistics collecting cost free. I have to revisit this with Rickard. Regards, Bj?rn-Egil ------------------------------ > > *From:* Wallentin Dahlberg [mailto:wallentin.dahlberg@REDACTED] > *Sent:* Monday, November 30, 2009 2:46 PM > *To:* Evans, Matthew > *Cc:* erlang-questions@REDACTED > *Subject:* Re: [erlang-questions] Concurrent processes on multi-core > platforms with lots of chatter > > > > 2009/11/30 Evans, Matthew > > First the good news: When running tests that do more than just message > passing the SMP features of R13B02 are leaps and bounds over R12B05 that I > was running previously. What I have however noticed is that in a pure > messaging test (lots of messages, in a tight loop) we appear to run into > caching issues where messages are sent between processes that happen to be > scheduled on different cores. This got me into thinking about a future > enhancement to the Erlang VM: Process affinity. > > In this mode two or more processes that have a lot of IPC chatter would be > associated into a group and executed on the same core. If the scheduler > needed to move one process to another core - they would all be relocated. > > Although this grouping of processes could be done automatically by the VM I > believe the decision making overhead would be too great, and it would likely > make some poor choices as to what processes should be grouped together. > Rather I would leave it to the developer to make these decisions, perhaps > with a library similar to pg2. > > > Process/scheduler affinity has been discussed at length before and there > are several schools of thought on the matter. > > It is true that accounting and statistics gathering in the runtime has some > overhead. However, this should be handled by the scheduler. The user (the > erlang developer) might not have all the information at hand and if he > doesn't he has to collect them which also costs. It would be easier for the > system to schedule the process right. The scheduler still has collect other > information and make decisions on memory models, memory distance, number of > processing units, process to process message affinity, load balancing and > other characteristics. It would be reasonable easy to model a scheduler > algorithm after these charateristics and we can make it much more dynamic in > the runtime system. > > Ideally the user shouldn't be concerned about the hardware he is running > on. Thats my take on the situation anyway. > > Regards, > Bj?rn-Egil > From zambal@REDACTED Mon Nov 30 23:08:46 2009 From: zambal@REDACTED (zambal) Date: Mon, 30 Nov 2009 14:08:46 -0800 (PST) Subject: Erlang on Windows from USB In-Reply-To: <1259600282.2790.101.camel@info01.REGNA> References: <1259600282.2790.101.camel@info01.REGNA> Message-ID: <9dcf5345-093f-49be-93e7-14d69412c4b9@e23g2000yqd.googlegroups.com> On 30 nov, 17:58, Giuseppe Luigi Punzi wrote: > "G:\Desarrollo\erl5.7.4\bin\erl.exe > The aplication couldn't be started because it's configuration is > incorrect. Reinstall the application could solve the problem" My first guess would be missing Visual C++ run-time dll's. I'm pretty sure Erlang is dynamically linked to these files (because the Erlang setup automatically runs the VC++ redistributable installer) and if these files are missing, I know from experience that exactly this error message is generated. If you can't install the VC++ redistributable installer on the target machine, the solution would be to manually copy the needed dll's in to the same folder as erl.exe and werl.exe. I can't easily check which files these are but I think they are all named like mfc*80.dll. good luck, vincent From mihai@REDACTED Mon Nov 30 23:10:27 2009 From: mihai@REDACTED (Mihai Balea) Date: Mon, 30 Nov 2009 17:10:27 -0500 Subject: [erlang-questions] Processing a List into a Dict In-Reply-To: References: <4B1435F3.2000107@hlabs.spb.ru> Message-ID: On Nov 30, 2009, at 4:22 PM, Jarrod Roberson wrote: > Thanks for the responses, that got me on the right track, here is what I > came up with. > > TXT = > ["txtvers=1","userid=3A6524D4-E31C-491D-94DD-555883B1600A","name=Jarrod > Roberson","version=2"]. > > A = [ string:tokens(KV,"=") || KV <- TXT]. > [["txtvers","1"],["userid","3A6524D4-E31C-491D-94DD-555883B1600A"],["name","Jarrod > Roberson"],["version","2"]] > > D = [{list_to_atom(K),V} || [K|[V|_]] <- A]. > [{txtvers,"1"},{userid,"3A6524D4-E31C-491D-94DD-555883B1600A"},{name,"Jarrod > Roberson"},{version,"2"}] > > DT = dict:from_list(D). > {dict,4,16,16,8,80,48, > {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, > {{[],[],[], > [[txtvers,49]], > [[userid,51,65,54,53,50,52,68,52,45,69,51,49|...]], > [[name,74,97,114,114,111,100,32,82,111,98,101|...]], > [],[],[],[],[],[],[],[], > [[version,50]], > []}}} > > dict:find(name,DT). > {ok,"Jarrod Roberson"} > > this gets me the keys as atoms instead of "strings" list_to_atom/1 is not something you want to use lightly. The reason is that atom space is not garbage collected, so this is a recipe for out of memory conditions if your application is long running and/or processes a lot of data. If you really need them as atoms and the set of things you want to convert is limited and known beforehand you might want to look into list_to_existing_atom/1 Mihai