From jwatte@REDACTED Mon Aug 1 00:57:01 2011 From: jwatte@REDACTED (Jon Watte) Date: Sun, 31 Jul 2011 15:57:01 -0700 Subject: [erlang-questions] Backpressure In-Reply-To: <5201CD92-7925-4C2E-BEC2-DCB1A1129D7F@erlang-solutions.com> References: <5201CD92-7925-4C2E-BEC2-DCB1A1129D7F@erlang-solutions.com> Message-ID: Is there a good counter or other way of instrumenting this send queue, btw? (I'm using R13B3 but am starting to feel like I have to upgrade at some point...) Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Thu, Jul 28, 2011 at 1:10 PM, Ulf Wiger wrote: > > On 28 Jul 2011, at 20:16, Jay Nelson wrote: > > >> Ulf Wiger wrote: > > > >> The problems with Distributed Erlang are related to a heavy-handed > backpressure solution, where processes trying to send to the dist_port are > simply suspended if the output queue exceeds a given threshold. > > > > Does this only apply to distributed erlang? What about punishment of > processes communicating within a node? I've seen senders using more time in > local comms*. > > Only Distributed Erlang. Local message passing does come with a form of > backpressure too, as senders are punished with extra reductions if the > receiver's message queue is big. I think this feature should go away, as it > makes the message send more complex from a scalability point of view, but I > know others like it? One can probably reduce locking by allowing the queue > length value to be somewhat old, for example. > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Mon Aug 1 01:00:46 2011 From: jwatte@REDACTED (Jon Watte) Date: Sun, 31 Jul 2011 16:00:46 -0700 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> Message-ID: > One problem is that it's hard to do detailed profiling on complex > real-world applications. The issues limiting scalability might well be > wholly unrelated to core VM aspects such as GC, scheduling and message > passing. In the first SMP experiments with Ericsson's Telephony Gateway > Controller, the big bottleneck was the big lock protecting the ports and > linked-in drivers. > Speaking of which: what *is* best practice for profiling bottlenecks in a big Erlang system/application? I've read the Erlang and OTP books, but they don't talk about it, and trapexit.org or other references are also vexingly vague on this subject. Sincerely, jw -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Mon Aug 1 01:31:21 2011 From: james@REDACTED (James Aimonetti) Date: Sun, 31 Jul 2011 16:31:21 -0700 Subject: [erlang-questions] Hidden Memory Hog Message-ID: <4E35E5C9.90108@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 List, I'm at my wits end (they're short, no doubt, but still). I have a VM running with several OTP applications that we've written. We see, over the course of the day, memory consumption go up until its taking ~30% system memory and 300-500MB of swap. Iterating over all processes and checking their total_heap_size, no single process was greater than ~1 MB, with ~300 processes runnning. Forcing garbage collection on all processes released maybe 15MB of system memory. erlang:memory() lists total memory around 18-20MB; ets tables were all minimal, no dets tables. No processes has a message_queue_len > 0. No processes that we maintain use the process dictionary. I started a second VM, migrating our OTP apps to the second VM, one at a time, hoping that stopping one would cause a release of lots of memory on the first VM. No such luck; it released a couple more MBs but still a large presence and no change in swap usage. I then stopped everything I could until only stdlib and kernel were running on the first VM, but to no avail. Each application has 1-3 long running gen_servers that listen on an AMQP queue and spawn worker processes to handle messages. They carry almost no state. Some of the handlers will read binary blobs, open a socket, wait for a connection, then transmit the blobs, closing the socket and dying afterwards (they timeout after 5 minutes if no connection is attempted, and are in a supervision tree so I know there aren't a mass of them lying around). The blobs are between 20KB and 2MB. The rest of the workers are similarly ephemeral. What other options are there to find where memory has been allocated? We use binaries almost exclusively for strings, don't construct large lists. I've poured over the efficiency guide looking for ideas of where we've gone astray but am coming up blank so far. The system is idle; the only processes should be (and are, as far as I can tell) the gen_servers meant to run for a long time. Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:8] [kernel-poll:true] Any ideas are welcome, James - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJONeXJAAoJENc77s1OYoGgx+YIALQzjHIEgfTMUcoYzB9zJ6yd tGT/J/CccADk/ErTGkpMMSG1PYPa8jzlaIaPw8tOtzKnGWb6iFQGahsPNexKwIKM t9TwSELRBQeF9SMKBKh/tD1DzIMT47oBpf8uUYTfXy3Ww/ofDEhZ1vKNNkefb3ub TENEEThDP5UC0L43HYwWp82yIKMSXJvmQUG/2WVvVF7oYXrTd2EPOXtn38+/th5u pi+j6zFfYaAVZ4ZBijBNJOcePZK7vyw7h/mWQOP+8r5wkETexxaafeX677WIskYu QMGqp6aABfDl4W4rmFArR3MY47pO0SHjd9MH4JgkhQsbU6OvLFPryr7unq4qQek= =2WoE -----END PGP SIGNATURE----- From ok@REDACTED Mon Aug 1 02:05:09 2011 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 1 Aug 2011 12:05:09 +1200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> Message-ID: On 29/07/2011, at 9:30 PM, Tim Watson wrote: > On 29 July 2011 00:45, Richard O'Keefe wrote: >> One of the things criticised in the blog entry that we've been responding to was >> that >> {ok,Foo} = bar(...), >> {ok,Foo} = ugh(...) >> is too easy to write (when you really meant, say, Foo0, Foo1). >> >> This is a well defined part of the language, and it would not be a good idea to >> ban it. But how about an optional style warning (and we really need >> -warn(on | off, [warning...]) >> directives) whenever a bound variable appears in a pattern on the left of "="? > > We would need to be able to set that warning with a very localised > scope though. What you are intending in such a match might be > *exactly* what the code says - call bar/1 to get a Foo and then call > ugh/2 and match (assert) that the resulting tuple contains exactly the > same Foo. Things like a session id, (ets) table id and so on are > probably examples of this. I won't comment on whether this is good API > design or not. There is another way to express that intention that does not require the use of this anti-pattern. crash_if_different(X, X) -> ok. ... -> ... {ok,Bar_Foo} = bar(...), {ok,Ugh_Foo} = ugh(...), crash_if_different(Bar_Foo, Ugh_Foo), ... > A -warn(on | off, [Warning]) sounds lovely, but would only work at > module level (or as some option passed in to the compiler) Where did that restriction come from? Not from anything *I* wrote. I was imitating Prolog style warnings, which take effect from the point where they are asserted to the point where they are next set. For example, :- style_check(+charset). p(f?b?r). :- style_check(-charset). p(fa?ade). gets a non-portable characters warning for the first clause of p/1 but not for the second clause. > so how > would I do (or override) this for an individual line of code or for a > single function? Switch the warning on (or off) just before the function, then switch it off (or on) again just after. As for single lines of code, crash_if_different/2 means there is no use case for such an override. > What about function level attributes? That might get > the scope at which the switch is applied tight enough to be useful: > > -pragma(no_warnings, [bound_variable_in_match]). This is exactly what I proposed, just spelled differently. I don't think the spelling of the style check directives matters much, and have no objection to using -pragma here. > fwibble_ardvarks() -> > {ok,Foo} = bar(...), > {ok,Foo} = ugh(...), > %% etc.... > Foo. From mjtruog@REDACTED Mon Aug 1 02:22:19 2011 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 31 Jul 2011 17:22:19 -0700 Subject: [erlang-questions] Hidden Memory Hog In-Reply-To: <4E35E5C9.90108@2600hz.com> References: <4E35E5C9.90108@2600hz.com> Message-ID: <4E35F1BB.20206@gmail.com> It sounds like a port driver or NIF leaking memory within the Erlang VM's OS process. I would suggest using instrument:memory_data/0 or instrument:store_memory_data/1 to make sure. On 07/31/2011 04:31 PM, James Aimonetti wrote: > List, > > I'm at my wits end (they're short, no doubt, but still). I have a VM > running with several OTP applications that we've written. We see, over > the course of the day, memory consumption go up until its taking ~30% > system memory and 300-500MB of swap. Iterating over all processes and > checking their total_heap_size, no single process was greater than ~1 > MB, with ~300 processes runnning. Forcing garbage collection on all > processes released maybe 15MB of system memory. > > erlang:memory() lists total memory around 18-20MB; ets tables were all > minimal, no dets tables. No processes has a message_queue_len > 0. No > processes that we maintain use the process dictionary. > > I started a second VM, migrating our OTP apps to the second VM, one at a > time, hoping that stopping one would cause a release of lots of memory > on the first VM. No such luck; it released a couple more MBs but still a > large presence and no change in swap usage. I then stopped everything I > could until only stdlib and kernel were running on the first VM, but to > no avail. > > Each application has 1-3 long running gen_servers that listen on an AMQP > queue and spawn worker processes to handle messages. They carry almost > no state. Some of the handlers will read binary blobs, open a socket, > wait for a connection, then transmit the blobs, closing the socket and > dying afterwards (they timeout after 5 minutes if no connection is > attempted, and are in a supervision tree so I know there aren't a mass > of them lying around). The blobs are between 20KB and 2MB. The rest of > the workers are similarly ephemeral. > > What other options are there to find where memory has been allocated? We > use binaries almost exclusively for strings, don't construct large > lists. I've poured over the efficiency guide looking for ideas of where > we've gone astray but am coming up blank so far. > > The system is idle; the only processes should be (and are, as far as I > can tell) the gen_servers meant to run for a long time. > > Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:8] [kernel-poll:true] > > Any ideas are welcome, > > James > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From cyplo@REDACTED Mon Aug 1 10:17:34 2011 From: cyplo@REDACTED (=?UTF-8?Q?Cyryl_P=C5=82otnicki=2DChudyk?=) Date: Mon, 1 Aug 2011 10:17:34 +0200 Subject: [erlang-questions] Hidden Memory Hog In-Reply-To: <4E35F1BB.20206@gmail.com> References: <4E35E5C9.90108@2600hz.com> <4E35F1BB.20206@gmail.com> Message-ID: Maybe you're generating lots of unique atoms , these are not GCed On Mon, Aug 1, 2011 at 2:22 AM, Michael Truog wrote: > It sounds like a port driver or NIF leaking memory within the Erlang VM's OS process. ?I would suggest using instrument:memory_data/0 or instrument:store_memory_data/1 to make sure. > > On 07/31/2011 04:31 PM, James Aimonetti wrote: >> List, >> >> I'm at my wits end (they're short, no doubt, but still). I have a VM >> running with several OTP applications that we've written. We see, over >> the course of the day, memory consumption go up until its taking ~30% >> system memory and 300-500MB of swap. Iterating over all processes and >> checking their total_heap_size, no single process was greater than ~1 >> MB, with ~300 processes runnning. Forcing garbage collection on all >> processes released maybe 15MB of system memory. >> >> erlang:memory() lists total memory around 18-20MB; ets tables were all >> minimal, no dets tables. No processes has a message_queue_len > 0. No >> processes that we maintain use the process dictionary. >> >> I started a second VM, migrating our OTP apps to the second VM, one at a >> time, hoping that stopping one would cause a release of lots of memory >> on the first VM. No such luck; it released a couple more MBs but still a >> large presence and no change in swap usage. I then stopped everything I >> could until only stdlib and kernel were running on the first VM, but to >> no avail. >> >> Each application has 1-3 long running gen_servers that listen on an AMQP >> queue and spawn worker processes to handle messages. They carry almost >> no state. Some of the handlers will read binary blobs, open a socket, >> wait for a connection, then transmit the blobs, closing the socket and >> dying afterwards (they timeout after 5 minutes if no connection is >> attempted, and are in a supervision tree so I know there aren't a mass >> of them lying around). The blobs are between 20KB and 2MB. The rest of >> the workers are similarly ephemeral. >> >> What other options are there to find where memory has been allocated? We >> use binaries almost exclusively for strings, don't construct large >> lists. I've poured over the efficiency guide looking for ideas of where >> we've gone astray but am coming up blank so far. >> >> The system is idle; the only processes should be (and are, as far as I >> can tell) the gen_servers meant to run for a long time. >> >> Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] >> [async-threads:8] [kernel-poll:true] >> >> Any ideas are welcome, >> >> James >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- cyryl From alex.arnon@REDACTED Mon Aug 1 10:20:32 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Mon, 1 Aug 2011 11:20:32 +0300 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> Message-ID: On Sun, Jul 31, 2011 at 10:10 PM, Tim Watson wrote: > > > > I agree with the sentiment. Java will indeed usually get the least > friction > > for any implementation, which for the Enterprise-minded is usually reason > > enough to use it for everything. > > However, in many places one could find instances where building at least > a > > prototype in Erlang might be the Right Thing. Several back-end or even > > front-end services I have built I would certainly have knocked up in > Erlang > > very rapidly, just as an idea to Show The Boss or even as a real > deployment > > candidate. In all these instances, the database was either Oracle or > MS-SQL, > > with reasonably low DB access performance expectations. > > Now, like you said above, just dropping into another VM for whatever > service > > is going to "look bad". Certainly if you've really built it from scratch > for > > your pet project, but what if we were talking about something that's been > > battle-proven? That maybe speaks Enterprise lingo? Maybe put a nice face > on > > it, for instance - add a RESTful JSON interface or something, as a > > diversion? :) Like you said, the Enterprise is often willing to take a > lot > > of crap just to have something that looks "standard" - and deploying a > Lean > > JDBC Proxy can go down relatively easily. > > > > I've written a few real production systems in Erlang, but they were > all OSS applications and it was the right fit. Even in a telco, most > BSS applications tend not to be written in Erlang. I'm not saying > that's right or good - I personally would rather build something in > Erlang than Java - but there we have it. > > > > > Right. > > Something that I'd use the JDBC Proxy Abomination suggested above is a > > fallback. In that case, you're not expecting blazing performance - > however, > > you'll have a stable backend to bang your API out on. Gradually adding > clean > > Erlang backends can come later. > > > > Yeah I get that it will work ok, it just feels a bit weird not to use > existing native implementations that are known to work. ESL have a > postgres driver which I'm assuming (?) is used in production > applications. The database support for Zotonic is also very solid in > my experience. As I said, a guy at work wrote a driver based Oracle > back end and it's absolutely rock solid in production - the app quite > data access centric and has never once failed in the last 3 years > whilst it deals with a reasonable load (avg. few thousand > regular/daily users with peak times dipping into five figures). > > I agree, it does feel unnatural. However, in this case: 1. It's rather simple, easy to make consistent and stable. 2. It enables us to deliver immediate value to the customer... HELP I'VE GOT ENTERPRISITIS!!! But seriously: 2. It makes any and all RDBMS's available, immediately. 3. It's a fallback. > > > > I think that's spot on. > > Neither should they change this approach. > > If you want to leap forward in language/tool technology in the > Enterprise, > > there's always stuff like Scala (which is an excellent step up from Java, > > and starting to get some Traction). > > > > Like I said, we use Erlang at work where it's deemed appropriate. > Nobody there thinks Erlang "is a ghetto" but they do see it as niche > and the fact that they pay sub contractors a lot more for it than bog > standard Java applications is something they think about a lot. Having > said that IMHO the fact that these Erlang based systems never seem to > go wrong is testament to what I call the "you get what you pay for" > effect combined with "some tools are better than others". Just my > opinion though. > > > > > AFAICT, the existing APIs are similar to a severely cut-down JDBC API. So > > nothing very surprising there - I'm guessing most (if not all) have been > > built on a per-need basis. > > That's why I'd like to start by fronting a well established API - like > the ESL postgres driver - just to see what's already there and build > on it in the general case. > > OK. > > I agree that one should try to veer as much away from heavily-stateful > APIs, > > and this can probably be avoided. > > Yes - just avoiding exposing internal data structures by providing a > handle is probably enough to do this. The various modules can actually > return whatever (real state if required) but the API should just hide > this by exporting opaque types only: > > -opaque connection_handle :: term(). > -export_type([connection_handle/0]). > > > However, since we ARE dealing with RDBMS's, some of the flavour of using > > them is bound to come through the API - connection pooling, transactions > and > > error handling/reporting should definitely be part of any API that > expects > > to be widely useful. I'd also add an Erlang-y flavour to some operations, > > like streaming of result sets, to work with the platform's grain and not > so > > much against it. > > > > Yes all these things (pooling, transactions, etc) are necessary. I > maintain that you can provide them to the user without the > implementation details leaking though. > > I also agree with the notion of having operations that are Erlang-y. > Not sure what the best thing to do with streaming result sets would > be, but I suspect looking at the existing implementations would > clarify what people expect to see. For sure you don't always want to > have to wait until the entire result set has been processed before you > can do anything. I do think having operations that convert result sets > using some transformation fun/function would be useful though - you > might think of there as the Erlang equivalent of the Spring Framework > JdbcTemplate ResultSetConverter or RowMapper interfaces. Except in > Erlag you just a function, plain and simple. > > Right. One beneficial side effect of streaming result sets is that it enables processing enormous sets without blowing up your process/node. Transformation/map/reduce/whichever is probably a good starting point. > Is your JDBC thing open source? I might have a stab at this just for > fun - maybe fronting your library and the ESL postgres API as the > first cut. > > It was a prototype, got to the usable-but-hell-no-you're-not-seeing-this-pile stage. :) I don't think I've even updated my main repo with the last changes in a few months. Basically what I did was: 1. register a global controller (edbc_master). This is optional, meant to be used for general monitoring of the connection processes. 2. Each connection is an {Erlang process, JNode} pair, linked to its creator/user and registered with 'edbc_master'. The connection is started with a configuration such as: { { proxy_dir, "c:/dev/edbc/java/deploy" }, { driver_class, "com.oracle.xxx.yyy.OracleDriver" }, { hostname, "localhost" }, { port, 4444 }, ... other connection-stringy stuff } 3. The Controller process (the linked Erlang process) gets a "unique" node name for the JNode from the edbc_master, and spawns it. 4. A basic handshake with X-second timeout between the Controller process and a counterpart on the JNode (registered as 'edbc_proxy' on the JNode I think) is done, including adding monitors on success. 5. From this point on, operations are forwarded via the Controller to the JNode, which executes them, packages up the results and sends them back. What I've done is somewhat crude - a process per connection etc., no proper support for most datatypes - but it was just a couple of days' work to get a simple SELECT/UPDATE/INSERT working. And it would work on Oracle, MSSql, Postgres, MySQL, Sybase, what have you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Mon Aug 1 10:39:38 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 1 Aug 2011 10:39:38 +0200 Subject: [erlang-questions] Hidden Memory Hog In-Reply-To: <4E35E5C9.90108@2600hz.com> References: <4E35E5C9.90108@2600hz.com> Message-ID: What does erlang:memory() show? Binaries, processes or ets uses that much memory? 2011/8/1, James Aimonetti : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > List, > > I'm at my wits end (they're short, no doubt, but still). I have a VM > running with several OTP applications that we've written. We see, over > the course of the day, memory consumption go up until its taking ~30% > system memory and 300-500MB of swap. Iterating over all processes and > checking their total_heap_size, no single process was greater than ~1 > MB, with ~300 processes runnning. Forcing garbage collection on all > processes released maybe 15MB of system memory. > > erlang:memory() lists total memory around 18-20MB; ets tables were all > minimal, no dets tables. No processes has a message_queue_len > 0. No > processes that we maintain use the process dictionary. > > I started a second VM, migrating our OTP apps to the second VM, one at a > time, hoping that stopping one would cause a release of lots of memory > on the first VM. No such luck; it released a couple more MBs but still a > large presence and no change in swap usage. I then stopped everything I > could until only stdlib and kernel were running on the first VM, but to > no avail. > > Each application has 1-3 long running gen_servers that listen on an AMQP > queue and spawn worker processes to handle messages. They carry almost > no state. Some of the handlers will read binary blobs, open a socket, > wait for a connection, then transmit the blobs, closing the socket and > dying afterwards (they timeout after 5 minutes if no connection is > attempted, and are in a supervision tree so I know there aren't a mass > of them lying around). The blobs are between 20KB and 2MB. The rest of > the workers are similarly ephemeral. > > What other options are there to find where memory has been allocated? We > use binaries almost exclusively for strings, don't construct large > lists. I've poured over the efficiency guide looking for ideas of where > we've gone astray but am coming up blank so far. > > The system is idle; the only processes should be (and are, as far as I > can tell) the gen_servers meant to run for a long time. > > Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:8] [kernel-poll:true] > > Any ideas are welcome, > > James > > - -- > James Aimonetti > Distributed Systems Engineer / DJ MC_ > > 2600hz | http://2600hz.com > sip:james@REDACTED > tel: 415.886.7905 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQEcBAEBAgAGBQJONeXJAAoJENc77s1OYoGgx+YIALQzjHIEgfTMUcoYzB9zJ6yd > tGT/J/CccADk/ErTGkpMMSG1PYPa8jzlaIaPw8tOtzKnGWb6iFQGahsPNexKwIKM > t9TwSELRBQeF9SMKBKh/tD1DzIMT47oBpf8uUYTfXy3Ww/ofDEhZ1vKNNkefb3ub > TENEEThDP5UC0L43HYwWp82yIKMSXJvmQUG/2WVvVF7oYXrTd2EPOXtn38+/th5u > pi+j6zFfYaAVZ4ZBijBNJOcePZK7vyw7h/mWQOP+8r5wkETexxaafeX677WIskYu > QMGqp6aABfDl4W4rmFArR3MY47pO0SHjd9MH4JgkhQsbU6OvLFPryr7unq4qQek= > =2WoE > -----END PGP SIGNATURE----- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From joelr1@REDACTED Mon Aug 1 12:11:12 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 1 Aug 2011 11:11:12 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> Message-ID: <079C51C9-E6C6-40FB-99F0-2D8DBA0EB358@gmail.com> What about a new thread on this subject? On Aug 1, 2011, at 12:00 AM, Jon Watte wrote: > Speaking of which: what *is* best practice for profiling bottlenecks in a big Erlang system/application? > I've read the Erlang and OTP books, but they don't talk about it, and trapexit.org or other references are also vexingly vague on this subject. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From watson.timothy@REDACTED Mon Aug 1 12:42:20 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 1 Aug 2011 11:42:20 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <079C51C9-E6C6-40FB-99F0-2D8DBA0EB358@gmail.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <079C51C9-E6C6-40FB-99F0-2D8DBA0EB358@gmail.com> Message-ID: Good point Joel - will do. On 1 August 2011 11:11, Joel Reymont wrote: > What about a new thread on this subject? > > On Aug 1, 2011, at 12:00 AM, Jon Watte wrote: > >> Speaking of which: what *is* best practice for profiling bottlenecks in a big Erlang system/application? >> I've read the Erlang and OTP books, but they don't talk about it, and trapexit.org or other references are also vexingly vague on this subject. > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From watson.timothy@REDACTED Mon Aug 1 12:47:37 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 1 Aug 2011 11:47:37 +0100 Subject: [erlang-questions] Erlang DataBase Connectivity (was trouble with erlang or erlang is a ghetto) Message-ID: On 1 August 2011 09:20, Alex Arnon wrote: > > I agree, it does feel unnatural. However, in this case: > 1. It's rather simple, easy to make consistent and stable. > 2. It enables us to deliver immediate value to the customer... HELP I'VE GOT > ENTERPRISITIS!!! Ok fine, but I need to see your code in order to know how to plug it into any API we dream up. :) > But seriously: > 2. It makes any and all RDBMS's available, immediately. > 3. It's a fallback. > Fine let's do it. > > Right. > One beneficial side effect of streaming result sets is that it enables > processing enormous sets without blowing up your process/node. > Transformation/map/reduce/whichever is probably a good starting point. > I agree that streaming result sets is useful and we'll put something in for that. I also want to do a basic "transform the result set using this function" API - let's have both. >> >> Is your JDBC thing open source? I might have a stab at this just for >> fun - maybe fronting your library and the ESL postgres API as the >> first cut. >> > > It was a prototype, got to the > usable-but-hell-no-you're-not-seeing-this-pile stage. :) > I don't think I've even updated my main repo with the last changes in a few > months. > > Basically what I did was: > 1. register a global controller (edbc_master). This is optional, meant to be > used for general monitoring of the connection processes. > 2. Each connection is an {Erlang process, JNode} pair, linked to its > creator/user and registered with 'edbc_master'. > ??? The connection is started with a configuration such as: > ??? { { proxy_dir, "c:/dev/edbc/java/deploy" }, > ????? { driver_class, "com.oracle.xxx.yyy.OracleDriver" }, > ????? { hostname, "localhost" }, > ????? { port, 4444 }, > ????? ... > ????? other connection-stringy stuff > ??? } > 3. The Controller process (the linked Erlang process) gets a "unique" node > name for the JNode from the edbc_master, and spawns it. > 4. A basic handshake with X-second timeout between the Controller process > and a counterpart on the JNode (registered as 'edbc_proxy' on the JNode I > think) is done, including adding monitors on success. > 5. From this point on, operations are forwarded via the Controller to the > JNode, which executes them, packages up the results and sends them back. > > What I've done is somewhat crude - a process per connection etc., no proper > support for most datatypes - but it was just a couple of days' work to get a > simple SELECT/UPDATE/INSERT working. And it would work on Oracle, MSSql, > Postgres, MySQL, Sybase, what have you. > > Yeah I'm terribly well versed in JDBC and have a vague inkling of how JNodes work, so that's all fine. Like I said, let's glue together a basic API and plug it into your thing and one other popular API - perhaps the OTP odbc application would actually be the best thing alongside yours - and then we'll go from there. Once we have something working we can come back here and ask the community for input. From alex.arnon@REDACTED Mon Aug 1 12:55:40 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Mon, 1 Aug 2011 13:55:40 +0300 Subject: [erlang-questions] Erlang DataBase Connectivity (was trouble with erlang or erlang is a ghetto) In-Reply-To: References: Message-ID: Oh no, you mean, like, to do REAL work? :) I'll try to find the latest version - my main server has something old and barely usable. Wouldn't you like to spec a basic API first? That could be used one-to-one on the JNode proxy, for simplicity, as a thin wrapper. On Mon, Aug 1, 2011 at 1:47 PM, Tim Watson wrote: > On 1 August 2011 09:20, Alex Arnon wrote: > > > > I agree, it does feel unnatural. However, in this case: > > 1. It's rather simple, easy to make consistent and stable. > > 2. It enables us to deliver immediate value to the customer... HELP I'VE > GOT > > ENTERPRISITIS!!! > > Ok fine, but I need to see your code in order to know how to plug it > into any API we dream up. :) > > > But seriously: > > 2. It makes any and all RDBMS's available, immediately. > > 3. It's a fallback. > > > > Fine let's do it. > > > > > Right. > > One beneficial side effect of streaming result sets is that it enables > > processing enormous sets without blowing up your process/node. > > Transformation/map/reduce/whichever is probably a good starting point. > > > > I agree that streaming result sets is useful and we'll put something > in for that. I also want to do a basic "transform the result set using > this function" API - let's have both. > > >> > >> Is your JDBC thing open source? I might have a stab at this just for > >> fun - maybe fronting your library and the ESL postgres API as the > >> first cut. > >> > > > > It was a prototype, got to the > > usable-but-hell-no-you're-not-seeing-this-pile stage. :) > > I don't think I've even updated my main repo with the last changes in a > few > > months. > > > > Basically what I did was: > > 1. register a global controller (edbc_master). This is optional, meant to > be > > used for general monitoring of the connection processes. > > 2. Each connection is an {Erlang process, JNode} pair, linked to its > > creator/user and registered with 'edbc_master'. > > The connection is started with a configuration such as: > > { { proxy_dir, "c:/dev/edbc/java/deploy" }, > > { driver_class, "com.oracle.xxx.yyy.OracleDriver" }, > > { hostname, "localhost" }, > > { port, 4444 }, > > ... > > other connection-stringy stuff > > } > > 3. The Controller process (the linked Erlang process) gets a "unique" > node > > name for the JNode from the edbc_master, and spawns it. > > 4. A basic handshake with X-second timeout between the Controller process > > and a counterpart on the JNode (registered as 'edbc_proxy' on the JNode I > > think) is done, including adding monitors on success. > > 5. From this point on, operations are forwarded via the Controller to the > > JNode, which executes them, packages up the results and sends them back. > > > > What I've done is somewhat crude - a process per connection etc., no > proper > > support for most datatypes - but it was just a couple of days' work to > get a > > simple SELECT/UPDATE/INSERT working. And it would work on Oracle, MSSql, > > Postgres, MySQL, Sybase, what have you. > > > > > > Yeah I'm terribly well versed in JDBC and have a vague inkling of how > JNodes work, so that's all fine. Like I said, let's glue together a > basic API and plug it into your thing and one other popular API - > perhaps the OTP odbc application would actually be the best thing > alongside yours - and then we'll go from there. Once we have something > working we can come back here and ask the community for input. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon Aug 1 12:56:26 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 1 Aug 2011 11:56:26 +0100 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> Message-ID: On 1 August 2011 01:05, Richard O'Keefe wrote: >> A -warn(on | off, [Warning]) sounds lovely, but would only work at >> module level (or as some option passed in to the compiler) > > Where did that restriction come from? > Not from anything *I* wrote. > I was imitating Prolog style warnings, which take effect from the > point where they are asserted to the point where they are next set. > For example, > ? ? ? ?:- style_check(+charset). > ? ? ? ?p(f?b?r). > ? ? ? ?:- style_check(-charset). > ? ? ? ?p(fa?ade). > gets a non-portable characters warning for the first clause of p/1 > but not for the second clause. > Um, that's lovely but we're talking about Erlang and it doesn't support that syntax. >> so how >> would I do (or override) this for an individual line of code or for a >> single function? > > Switch the warning on (or off) just before the function, > then switch it off (or on) again just after. > Now you're going to have to exercise your patience with me here Mr O'Keefe, as I've lost you. How do I turn warnings on/off for individual functions? Was your proposal to introduce the `-warn(....)` syntax as an attribute that can be applied to a function and if so, is adding annotations to functions a new feature? I'm not aware this works today and that's why I chimed in. > As for single lines of code, crash_if_different/2 means there is no > use case for such an override. > Yes I can see that now. >> What about function level attributes? That might get >> the scope at which the switch is applied tight enough to be useful: >> >> -pragma(no_warnings, [bound_variable_in_match]). > > This is exactly what I proposed, just spelled differently. > I don't think the spelling of the style check directives matters much, > and have no objection to using -pragma here. > Agreed - I like `pragma` but likewise I'm not really fussed. My point was that AFAIK you can't put annotations on functions today and this would be a new feature - with the annotation info preserved at runtime in the generated beam if possible. From watson.timothy@REDACTED Mon Aug 1 13:00:15 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 1 Aug 2011 12:00:15 +0100 Subject: [erlang-questions] Erlang DataBase Connectivity (was trouble with erlang or erlang is a ghetto) In-Reply-To: References: Message-ID: On 1 August 2011 11:55, Alex Arnon wrote: > Oh no, you mean, like, to do REAL work? :) > I'll try to find the latest version - my main server has something old and > barely usable. > > Wouldn't you like to spec a basic API first? That could be used one-to-one > on the JNode proxy, for simplicity, as a thin wrapper. > Yes ok will do. I'll post back with a github URL once I have something in place. From mihai@REDACTED Mon Aug 1 13:47:51 2011 From: mihai@REDACTED (Mihai Balea) Date: Mon, 1 Aug 2011 07:47:51 -0400 Subject: [erlang-questions] Gproc in a dynamic cluster In-Reply-To: <6CD0A133-6F6D-408C-ACA3-B5237C6A67CC@erlang-solutions.com> References: <3E83ACB7-9B81-4812-AF4A-F153D2CFA3F8@hates.ms> <6CD0A133-6F6D-408C-ACA3-B5237C6A67CC@erlang-solutions.com> Message-ID: <36ADA8AF-80F2-4BA1-A655-AD1F6AEE56DA@hates.ms> That is correct, however, I was wondering if there's a way to make gproc aware of nodes that are added at a later time. On Jul 31, 2011, at 4:51 PM, Tino Breddin wrote: > Hi, > > When using 'all' gproc_dist passes the list of nodes in the current cluster to gen_leader. Therefor in both cases you are only starting gproc_dist with 1 candidate node since you only connect them later. > > T > > On Jul 29, 2011, at 11:19 PM, Mihai Balea wrote: > >> Hi all, >> >> Here's a scenario: >> - start Node1 >> - start gproc in global mode on node 1 (via {gproc, [{gproc_dist, all}]} in app.config) >> - start Node2 >> - start gproc in global mode on node 2 (same way) >> - connect the two nodes in a cluster, via net_adm:ping/1 >> >> The two nodes see each other, as expected, but the two instances of gproc don't seem to be aware of each other's presence - global registrations done on one node are not visible on the other side. >> >> Am I missing something? Is there a way to make gproc aware of new nodes in a cluster, either automatically or manually? >> >> Thanks in advance, >> Mihai >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > From mihai@REDACTED Mon Aug 1 13:55:47 2011 From: mihai@REDACTED (Mihai Balea) Date: Mon, 1 Aug 2011 07:55:47 -0400 Subject: [erlang-questions] Hidden Memory Hog In-Reply-To: <4E35E5C9.90108@2600hz.com> References: <4E35E5C9.90108@2600hz.com> Message-ID: On Jul 31, 2011, at 7:31 PM, James Aimonetti wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > List, > > I'm at my wits end (they're short, no doubt, but still). I have a VM > running with several OTP applications that we've written. We see, over > the course of the day, memory consumption go up until its taking ~30% > system memory and 300-500MB of swap. Iterating over all processes and > checking their total_heap_size, no single process was greater than ~1 > MB, with ~300 processes runnning. Forcing garbage collection on all > processes released maybe 15MB of system memory. > > erlang:memory() lists total memory around 18-20MB; ets tables were all > minimal, no dets tables. No processes has a message_queue_len > 0. No > processes that we maintain use the process dictionary. > > I started a second VM, migrating our OTP apps to the second VM, one at a > time, hoping that stopping one would cause a release of lots of memory > on the first VM. No such luck; it released a couple more MBs but still a > large presence and no change in swap usage. I then stopped everything I > could until only stdlib and kernel were running on the first VM, but to > no avail. > > Each application has 1-3 long running gen_servers that listen on an AMQP > queue and spawn worker processes to handle messages. They carry almost > no state. Some of the handlers will read binary blobs, open a socket, > wait for a connection, then transmit the blobs, closing the socket and > dying afterwards (they timeout after 5 minutes if no connection is > attempted, and are in a supervision tree so I know there aren't a mass > of them lying around). The blobs are between 20KB and 2MB. The rest of > the workers are similarly ephemeral. > > What other options are there to find where memory has been allocated? We > use binaries almost exclusively for strings, don't construct large > lists. I've poured over the efficiency guide looking for ideas of where > we've gone astray but am coming up blank so far. > > The system is idle; the only processes should be (and are, as far as I > can tell) the gen_servers meant to run for a long time. > > Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:8] [kernel-poll:true] > > Any ideas are welcome, Check the output of erlang:memory/0 Binaries, ets tables, atoms all use separate heaps that are different than your regular process heaps. Look for any of those that appears particularly large. Mihai From marthinus.beneke@REDACTED Mon Aug 1 15:02:41 2011 From: marthinus.beneke@REDACTED (Marthinus Beneke) Date: Mon, 01 Aug 2011 15:02:41 +0200 Subject: [erlang-questions] Multiple Erlang installations into same directory Message-ID: <4E36A3F1.5050301@cellfind.co.za> Hi guys I have a question regarding multiple Erlang installations on Linux. Is it possible to install multiple releases into the same prefix (/usr/local/lib/) and still use the different versions independently? For example erts-5.6.5, erts-5.7.5 and erts-5.8.4? And would I have to change anything in the applications to support this? It would seem that the installation is built to support this since each erts version has it's own folder. Thanks in advance. -- Marthinus From carlo.bertoldi@REDACTED Mon Aug 1 15:04:01 2011 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Mon, 01 Aug 2011 15:04:01 +0200 Subject: [erlang-questions] Multiple Erlang installations into same directory In-Reply-To: <4E36A3F1.5050301@cellfind.co.za> References: <4E36A3F1.5050301@cellfind.co.za> Message-ID: <4E36A441.1050602@ubiquity.it> On 01/08/2011 15:02, Marthinus Beneke wrote: > Hi guys > > I have a question regarding multiple Erlang installations on Linux. > > Is it possible to install multiple releases into the same prefix > (/usr/local/lib/) and still use the different versions independently? > For example erts-5.6.5, erts-5.7.5 and erts-5.8.4? > And would I have to change anything in the applications to support this? > > It would seem that the installation is built to support this since each > erts version has it's own folder. > > Thanks in advance. > Hey Marthinus, https://github.com/evax/kerl this tool is just what you're looking for. Cheers, Carlo From kunthar@REDACTED Mon Aug 1 15:04:06 2011 From: kunthar@REDACTED (Kunthar) Date: Mon, 1 Aug 2011 16:04:06 +0300 Subject: [erlang-questions] Multiple Erlang installations into same directory In-Reply-To: <4E36A3F1.5050301@cellfind.co.za> References: <4E36A3F1.5050301@cellfind.co.za> Message-ID: https://github.com/evax/kerl On Mon, Aug 1, 2011 at 4:02 PM, Marthinus Beneke wrote: > Hi guys > > I have a question regarding multiple Erlang installations on Linux. > > Is it possible to install multiple releases into the same prefix > (/usr/local/lib/) and still use the different versions independently? > For example erts-5.6.5, erts-5.7.5 and erts-5.8.4? > And would I have to change anything in the applications to support this? > > It would seem that the installation is built to support this since each erts > version has it's own folder. > > Thanks in advance. > > -- > Marthinus > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- BR, \|/ Kunthar From andrew.kondratovich@REDACTED Mon Aug 1 15:41:51 2011 From: andrew.kondratovich@REDACTED (Andrew Kondratovich) Date: Mon, 1 Aug 2011 16:41:51 +0300 Subject: [erlang-questions] compile:file In-Reply-To: References: Message-ID: Hi. Docs says, that compile:file("m.erl", ['S']) produces a listing of the assembler code in the file m.S. Also, docs says, that compile:file("m.S", [asm]) compiles the code from input file, which is expected to be assembler code. So, these two steps identically to simple compile:file("m.erl"). But, why I can't to compile code from assembler, when it has operations with floating points. For example: ============================================================ -module(m). -compile(export_all). do(A) -> A + 0.5. ============================================================ This code produces assembler code: ============================================================ {module, m}. %% version = 0 {exports, [{do,1},{module_info,0},{module_info,1}]}. {attributes, []}. {labels, 7}. {function, do, 1, 2}. {label,1}. {func_info,{atom,m},{atom,do},1}. {label,2}. {test_heap,{alloc,[{words,0},{floats,1}]},1}. {fconv,{x,0},{fr,0}}. {fmove,{float,0.5},{fr,1}}. fclearerror. {bif,fadd,{f,0},[{fr,0},{fr,1}],{fr,0}}. {fcheckerror,{f,0}}. {fmove,{fr,0},{x,0}}. return. {function, module_info, 0, 4}. {label,3}. {func_info,{atom,m},{atom,module_info},0}. {label,4}. {move,{atom,m},{x,0}}. {call_ext_only,1,{extfunc,erlang,get_module_info,1}}. {function, module_info, 1, 6}. {label,5}. {func_info,{atom,m},{atom,module_info},1}. {label,6}. {move,{x,0},{x,1}}. {move,{atom,m},{x,0}}. {call_ext_only,2,{extfunc,erlang,get_module_info,2}}. ============================================================ When I try to compile: ============================================================ 40> compile:file("m.erl", ['S']). {ok,m} 41> compile:file("m.S", [asm]). Function: do/1 error 42> ============================================================ win xp erl -version Erlang (SMP,ASYNC_THREADS) (BEAM) emulator version 5.8.3 /Andrew From ulf.wiger@REDACTED Mon Aug 1 16:00:49 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 1 Aug 2011 16:00:49 +0200 Subject: [erlang-questions] Gproc in a dynamic cluster In-Reply-To: <36ADA8AF-80F2-4BA1-A655-AD1F6AEE56DA@hates.ms> References: <3E83ACB7-9B81-4812-AF4A-F153D2CFA3F8@hates.ms> <6CD0A133-6F6D-408C-ACA3-B5237C6A67CC@erlang-solutions.com> <36ADA8AF-80F2-4BA1-A655-AD1F6AEE56DA@hates.ms> Message-ID: If you are using the Abecciu fork of gen_leader (http://github.com/abecciu/gen_leader_revival.git), you can start gproc with an explicit list of nodes. When adding a new candidate, you need to add it to the 'candidates' option via gproc, and also name one of the existing (and running) candidates as "seed node", using {seed, SeedNode}. This is undocumented, but the algorithm is sketched here: https://gist.github.com/499825 The gen_leader leader instance will trigger a candidate list update for all candidates, and the gproc dictionary will be automatically synched. This could all be baked into the 'all' option somehow. I'll think about that. BR, Ulf W On 1 Aug 2011, at 13:47, Mihai Balea wrote: > That is correct, however, I was wondering if there's a way to make gproc aware of nodes that are added at a later time. > > On Jul 31, 2011, at 4:51 PM, Tino Breddin wrote: > >> Hi, >> >> When using 'all' gproc_dist passes the list of nodes in the current cluster to gen_leader. Therefor in both cases you are only starting gproc_dist with 1 candidate node since you only connect them later. >> >> T >> >> On Jul 29, 2011, at 11:19 PM, Mihai Balea wrote: >> >>> Hi all, >>> >>> Here's a scenario: >>> - start Node1 >>> - start gproc in global mode on node 1 (via {gproc, [{gproc_dist, all}]} in app.config) >>> - start Node2 >>> - start gproc in global mode on node 2 (same way) >>> - connect the two nodes in a cluster, via net_adm:ping/1 >>> >>> The two nodes see each other, as expected, but the two instances of gproc don't seem to be aware of each other's presence - global registrations done on one node are not visible on the other side. >>> >>> Am I missing something? Is there a way to make gproc aware of new nodes in a cluster, either automatically or manually? >>> >>> Thanks in advance, >>> Mihai >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Mon Aug 1 16:02:14 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 1 Aug 2011 15:02:14 +0100 Subject: [erlang-questions] badarg, function_exported Message-ID: <270021A7-4967-4FE2-920D-9D1C0CE232BE@gmail.com> What is this telling me? Is a function not exported and if so which function? Is it game_logic_stack:game? Thanks, Joel --- (<0.526.0>) error_logger ! {notify, {error_report,<0.479.0>, {<0.526.0>,crash_report, [[{pid,<0.526.0>}, {registered_name,[]}, {error_info, {error,badarg, [{erlang,function_exported, [{game_logic_stack,game, {texas,none,none,none,false,false,0,0, -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Mon Aug 1 16:15:58 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 1 Aug 2011 15:15:58 +0100 Subject: [erlang-questions] badarg, function_exported In-Reply-To: <270021A7-4967-4FE2-920D-9D1C0CE232BE@gmail.com> References: <270021A7-4967-4FE2-920D-9D1C0CE232BE@gmail.com> Message-ID: It always helps to ask! The problem here is that erlang:function_exported is getting a bad argument. On Aug 1, 2011, at 3:02 PM, Joel Reymont wrote: > What is this telling me? > > Is a function not exported and if so which function? > > Is it game_logic_stack:game? > > Thanks, Joel > > --- > (<0.526.0>) error_logger ! {notify, > {error_report,<0.479.0>, > {<0.526.0>,crash_report, > [[{pid,<0.526.0>}, > {registered_name,[]}, > {error_info, > {error,badarg, > [{erlang,function_exported, > [{game_logic_stack,game, > {texas,none,none,none,false,false,0,0, -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From sam@REDACTED Mon Aug 1 16:29:14 2011 From: sam@REDACTED (Sam Elliott) Date: Mon, 1 Aug 2011 15:29:14 +0100 Subject: [erlang-questions] badarg, function_exported In-Reply-To: References: <270021A7-4967-4FE2-920D-9D1C0CE232BE@gmail.com> Message-ID: Have you defined your own behaviour, but not used atoms and integers in the list of tuples like so? It looks like the arity argument you are giving it is all messed up, it should be an integer not a tuple/record. behaviour_info(callbacks) -> [{init,1}, ... ] It might also be worth checking where you define any modules to look for callbacks in, such as when you start a gen_server. I expect if that isn't an atom, erlang will complain. Sam -- Sam Elliott sam@REDACTED -- On Mon, Aug 1, 2011 at 3:15 PM, Joel Reymont wrote: > It always helps to ask! > > The problem here is that erlang:function_exported is getting a bad > argument. > > On Aug 1, 2011, at 3:02 PM, Joel Reymont wrote: > > > What is this telling me? > > > > Is a function not exported and if so which function? > > > > Is it game_logic_stack:game? > > > > Thanks, Joel > > > > --- > > (<0.526.0>) error_logger ! {notify, > > {error_report,<0.479.0>, > > {<0.526.0>,crash_report, > > [[{pid,<0.526.0>}, > > {registered_name,[]}, > > {error_info, > > {error,badarg, > > [{erlang,function_exported, > > [{game_logic_stack,game, > > > {texas,none,none,none,false,false,0,0, > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrea.rosa@REDACTED Mon Aug 1 16:40:24 2011 From: andrea.rosa@REDACTED (Andrea Rosa) Date: Mon, 01 Aug 2011 15:40:24 +0100 Subject: [erlang-questions] New_ssl module and CRL support Message-ID: <1312209624.25314.53.camel@vicbristol6.hpl.hp.com> Hi I am working with a software (rabbitmq) which use the new_ssl module to implement SSL connections, everything goes well with the exception of managing revoked certification during the handshake, in this case the server accepts a revoked certificate. Is there a way for fixing that? In your plans is there the implementation of a new module for supporting CRL? Cheers -- Andrea From joelr1@REDACTED Mon Aug 1 17:15:44 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 1 Aug 2011 16:15:44 +0100 Subject: [erlang-questions] common test error handler Message-ID: <8EDAA3D2-17B2-4C2F-B730-F24FE32CD517@gmail.com> How do yo make sure that bits created in init_per_testcase are cleaned up even if the test itself fails? It appears that end_per_testcase is not called in case of error. Am I right? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Mon Aug 1 17:30:44 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 1 Aug 2011 16:30:44 +0100 Subject: [erlang-questions] common test error handler In-Reply-To: <8EDAA3D2-17B2-4C2F-B730-F24FE32CD517@gmail.com> References: <8EDAA3D2-17B2-4C2F-B730-F24FE32CD517@gmail.com> Message-ID: <87FBBD10-DD7C-43F6-A4FA-758B8D829A37@gmail.com> To be more precise, it appears that end_per_testcase is not called on a time trap, e.g. suite() -> [{timetrap, {seconds, 60}}]. Is this expected behavior? On Aug 1, 2011, at 4:15 PM, Joel Reymont wrote: > How do yo make sure that bits created in init_per_testcase are cleaned up even if the test itself fails? > > It appears that end_per_testcase is not called in case of error. Am I right? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From james@REDACTED Mon Aug 1 17:55:33 2011 From: james@REDACTED (James Aimonetti) Date: Mon, 1 Aug 2011 08:55:33 -0700 Subject: [erlang-questions] Hidden Memory Hog In-Reply-To: References: <4E35E5C9.90108@2600hz.com> Message-ID: <4E36CC75.8020409@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 No, no outliers. 1> erlang:memory(). [{total,29393336}, {processes,18646408}, {processes_used,18560824}, {system,10746928}, {atom,658865}, {atom_used,652857}, {binary,1059104}, {code,6608997}, {ets,434848}] Unless I'm reading that output incorrectly? On 08/01/2011 01:39 AM, Attila Rajmund Nohl wrote: > What does erlang:memory() show? Binaries, processes or ets uses that > much memory? > > 2011/8/1, James Aimonetti : > List, > > I'm at my wits end (they're short, no doubt, but still). I have a VM > running with several OTP applications that we've written. We see, over > the course of the day, memory consumption go up until its taking ~30% > system memory and 300-500MB of swap. Iterating over all processes and > checking their total_heap_size, no single process was greater than ~1 > MB, with ~300 processes runnning. Forcing garbage collection on all > processes released maybe 15MB of system memory. > > erlang:memory() lists total memory around 18-20MB; ets tables were all > minimal, no dets tables. No processes has a message_queue_len > 0. No > processes that we maintain use the process dictionary. > > I started a second VM, migrating our OTP apps to the second VM, one at a > time, hoping that stopping one would cause a release of lots of memory > on the first VM. No such luck; it released a couple more MBs but still a > large presence and no change in swap usage. I then stopped everything I > could until only stdlib and kernel were running on the first VM, but to > no avail. > > Each application has 1-3 long running gen_servers that listen on an AMQP > queue and spawn worker processes to handle messages. They carry almost > no state. Some of the handlers will read binary blobs, open a socket, > wait for a connection, then transmit the blobs, closing the socket and > dying afterwards (they timeout after 5 minutes if no connection is > attempted, and are in a supervision tree so I know there aren't a mass > of them lying around). The blobs are between 20KB and 2MB. The rest of > the workers are similarly ephemeral. > > What other options are there to find where memory has been allocated? We > use binaries almost exclusively for strings, don't construct large > lists. I've poured over the efficiency guide looking for ideas of where > we've gone astray but am coming up blank so far. > > The system is idle; the only processes should be (and are, as far as I > can tell) the gen_servers meant to run for a long time. > > Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:8] [kernel-poll:true] > > Any ideas are welcome, > > James > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJONsx1AAoJENc77s1OYoGgviIH/jhqGwz+Y3UoJcxQTAABqiZK Ho8o7+TIMKbPqWsnsNChA8S8ERJGiL8e+seuAYWvTo9Qw9tVgaOCgj8A6wOxeLJ1 SWp2DmjJAkunAZz2g58d02agiykZcAIiOSUZxhY4Z7phjmqLAz5C9UQs4Ih5mCSA qpeza+zht3pRqpO+Uo8Vz3OAMK5aRD7GiI7iC6n5nwP2ZBFBHMNaLx1JFNcXq9dF jGFdcuoFyZeKnHV40DxKuUcCx9pwyjUr5E4Xq+0gPqdmVEJRuOacwvYk5nlAT+BH xKCPmvjpyTrxenBYexpuYgDIBWWpESfNOohoKtXSYd/vTAS/W32f0oLPKraS3I0= =yE1n -----END PGP SIGNATURE----- From joelr1@REDACTED Mon Aug 1 18:36:00 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 1 Aug 2011 17:36:00 +0100 Subject: [erlang-questions] my best erlang "foot in mouth" Message-ID: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> gen_server:cast(stop, Pid). Anybody else? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From mihai@REDACTED Mon Aug 1 19:32:27 2011 From: mihai@REDACTED (Mihai Balea) Date: Mon, 1 Aug 2011 13:32:27 -0400 Subject: [erlang-questions] Hidden Memory Hog In-Reply-To: <4E36CC75.8020409@2600hz.com> References: <4E35E5C9.90108@2600hz.com> <4E36CC75.8020409@2600hz.com> Message-ID: <723FDC86-0FC6-48EE-B012-05C57F9ECF25@hates.ms> That looks pretty benign. I suspect the OS reported memory usage for the beam process is vastly higher, correct? Do you run any native code? NIFs, linked-in drivers? Maybe there's a leak in there. On Aug 1, 2011, at 11:55 AM, James Aimonetti wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > No, no outliers. > > 1> erlang:memory(). > [{total,29393336}, > {processes,18646408}, > {processes_used,18560824}, > {system,10746928}, > {atom,658865}, > {atom_used,652857}, > {binary,1059104}, > {code,6608997}, > {ets,434848}] > > Unless I'm reading that output incorrectly? > > On 08/01/2011 01:39 AM, Attila Rajmund Nohl wrote: >> What does erlang:memory() show? Binaries, processes or ets uses that >> much memory? >> >> 2011/8/1, James Aimonetti : >> List, >> >> I'm at my wits end (they're short, no doubt, but still). I have a VM >> running with several OTP applications that we've written. We see, over >> the course of the day, memory consumption go up until its taking ~30% >> system memory and 300-500MB of swap. Iterating over all processes and >> checking their total_heap_size, no single process was greater than ~1 >> MB, with ~300 processes runnning. Forcing garbage collection on all >> processes released maybe 15MB of system memory. >> >> erlang:memory() lists total memory around 18-20MB; ets tables were all >> minimal, no dets tables. No processes has a message_queue_len > 0. No >> processes that we maintain use the process dictionary. >> >> I started a second VM, migrating our OTP apps to the second VM, one at a >> time, hoping that stopping one would cause a release of lots of memory >> on the first VM. No such luck; it released a couple more MBs but still a >> large presence and no change in swap usage. I then stopped everything I >> could until only stdlib and kernel were running on the first VM, but to >> no avail. >> >> Each application has 1-3 long running gen_servers that listen on an AMQP >> queue and spawn worker processes to handle messages. They carry almost >> no state. Some of the handlers will read binary blobs, open a socket, >> wait for a connection, then transmit the blobs, closing the socket and >> dying afterwards (they timeout after 5 minutes if no connection is >> attempted, and are in a supervision tree so I know there aren't a mass >> of them lying around). The blobs are between 20KB and 2MB. The rest of >> the workers are similarly ephemeral. >> >> What other options are there to find where memory has been allocated? We >> use binaries almost exclusively for strings, don't construct large >> lists. I've poured over the efficiency guide looking for ideas of where >> we've gone astray but am coming up blank so far. >> >> The system is idle; the only processes should be (and are, as far as I >> can tell) the gen_servers meant to run for a long time. >> >> Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] >> [async-threads:8] [kernel-poll:true] >> >> Any ideas are welcome, >> >> James >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > - -- > James Aimonetti > Distributed Systems Engineer / DJ MC_ > > 2600hz | http://2600hz.com > sip:james@REDACTED > tel: 415.886.7905 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQEcBAEBAgAGBQJONsx1AAoJENc77s1OYoGgviIH/jhqGwz+Y3UoJcxQTAABqiZK > Ho8o7+TIMKbPqWsnsNChA8S8ERJGiL8e+seuAYWvTo9Qw9tVgaOCgj8A6wOxeLJ1 > SWp2DmjJAkunAZz2g58d02agiykZcAIiOSUZxhY4Z7phjmqLAz5C9UQs4Ih5mCSA > qpeza+zht3pRqpO+Uo8Vz3OAMK5aRD7GiI7iC6n5nwP2ZBFBHMNaLx1JFNcXq9dF > jGFdcuoFyZeKnHV40DxKuUcCx9pwyjUr5E4Xq+0gPqdmVEJRuOacwvYk5nlAT+BH > xKCPmvjpyTrxenBYexpuYgDIBWWpESfNOohoKtXSYd/vTAS/W32f0oLPKraS3I0= > =yE1n > -----END PGP SIGNATURE----- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From antoine.koener@REDACTED Mon Aug 1 20:00:21 2011 From: antoine.koener@REDACTED (Antoine Koener) Date: Mon, 1 Aug 2011 20:00:21 +0200 Subject: [erlang-questions] Erlang DataBase Connectivity (was trouble with erlang or erlang is a ghetto) In-Reply-To: References: Message-ID: Maybe i'm too late but i think that readers of this new thread may be interested in this : http://sqlrelay.sourceforge.net/ On 1 ao?t 2011, at 13:00, Tim Watson wrote: > On 1 August 2011 11:55, Alex Arnon wrote: >> Oh no, you mean, like, to do REAL work? :) >> I'll try to find the latest version - my main server has something old and >> barely usable. >> >> Wouldn't you like to spec a basic API first? That could be used one-to-one >> on the JNode proxy, for simplicity, as a thin wrapper. >> > > Yes ok will do. I'll post back with a github URL once I have something in place. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Mon Aug 1 20:08:08 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 1 Aug 2011 19:08:08 +0100 Subject: [erlang-questions] Erlang DataBase Connectivity (was trouble with erlang or erlang is a ghetto) In-Reply-To: References: Message-ID: Yes I've come across that before. Didn't work on a mac last I checked, so useful in production but annoying in development. :) On 1 August 2011 19:00, Antoine Koener wrote: > Maybe i'm too late but i think that readers of this new thread may be interested in this : > > http://sqlrelay.sourceforge.net/ > > > On 1 ao?t 2011, at 13:00, Tim Watson wrote: > >> On 1 August 2011 11:55, Alex Arnon wrote: >>> Oh no, you mean, like, to do REAL work? :) >>> I'll try to find the latest version - my main server has something old and >>> barely usable. >>> >>> Wouldn't you like to spec a basic API first? That could be used one-to-one >>> on the JNode proxy, for simplicity, as a thin wrapper. >>> >> >> Yes ok will do. I'll post back with a github URL once I have something in place. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From erlang@REDACTED Mon Aug 1 20:37:51 2011 From: erlang@REDACTED (Martijn Rijkeboer) Date: Mon, 1 Aug 2011 20:37:51 +0200 Subject: [erlang-questions] [ANN] euuid 1.0.0 - an erlang UUID implementation Message-ID: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> Hello, As an exercise for learning Erlang I wrote an Universally Unique IDentifier (UUID) implementation as defined in RFC 4122. This module implements the UUID versions 1, 3, 4 and 5. It also implements a custom time based UUID that is based on version 1 but has a different layout to allow easy sorting on the creation date. This module is available as open-source under the MIT licence for anyone to use. You can take a look at the code on my github repository: - https://github.com/mrijkeboer/euuid Since this is my first Erlang program please let me know what you think. I'm open to suggestions and enhancements. Best regards, Martijn Rijkeboer From mjtruog@REDACTED Mon Aug 1 21:42:40 2011 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 01 Aug 2011 12:42:40 -0700 Subject: [erlang-questions] [ANN] euuid 1.0.0 - an erlang UUID implementation In-Reply-To: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> References: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> Message-ID: <4E3701B0.3090904@gmail.com> There is simpler/faster code that serves the same purpose (rfc uuids, versions 1,3,4,5) under a similar license (BSD) here: https://github.com/okeuday/uuid Getting the real MAC address is possible in Erlang, though undocumented. I include the Erlang pid integers within the node id for version 1 UUIDs and keep the version 1 state cacheable for quicker usage. On 08/01/2011 11:37 AM, Martijn Rijkeboer wrote: > Hello, > > As an exercise for learning Erlang I wrote an Universally Unique > IDentifier (UUID) implementation as defined in RFC 4122. This module > implements the UUID versions 1, 3, 4 and 5. It also implements a custom > time based UUID that is based on version 1 but has a different layout to > allow easy sorting on the creation date. This module is available as > open-source under the MIT licence for anyone to use. > > You can take a look at the code on my github repository: > - https://github.com/mrijkeboer/euuid > > Since this is my first Erlang program please let me know what you think. > I'm open to suggestions and enhancements. > > Best regards, > > > Martijn Rijkeboer > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From alex.arnon@REDACTED Mon Aug 1 21:48:22 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Mon, 1 Aug 2011 22:48:22 +0300 Subject: [erlang-questions] Erlang DataBase Connectivity (was trouble with erlang or erlang is a ghetto) In-Reply-To: References: Message-ID: This is an interesting piece of work - what kind of traction does it have? On Mon, Aug 1, 2011 at 9:08 PM, Tim Watson wrote: > Yes I've come across that before. Didn't work on a mac last I checked, > so useful in production but annoying in development. :) > > On 1 August 2011 19:00, Antoine Koener wrote: > > Maybe i'm too late but i think that readers of this new thread may be > interested in this : > > > > http://sqlrelay.sourceforge.net/ > > > > > > On 1 ao?t 2011, at 13:00, Tim Watson wrote: > > > >> On 1 August 2011 11:55, Alex Arnon wrote: > >>> Oh no, you mean, like, to do REAL work? :) > >>> I'll try to find the latest version - my main server has something old > and > >>> barely usable. > >>> > >>> Wouldn't you like to spec a basic API first? That could be used > one-to-one > >>> on the JNode proxy, for simplicity, as a thin wrapper. > >>> > >> > >> Yes ok will do. I'll post back with a github URL once I have something > in place. > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gordon@REDACTED Tue Aug 2 00:26:09 2011 From: gordon@REDACTED (Gordon Guthrie) Date: Mon, 1 Aug 2011 23:26:09 +0100 Subject: [erlang-questions] Erlang At The Edinburgh Festival - Sun 28th August Message-ID: There is a new festival at the Edinburgh Festivals which focuses on technology - the Turing Festival: http://www.turingfestival.com/ It is a curated festival with an organisational team who decided on what's hot and what's not. But like all good Edinburgh Festivals it has an uncurated fringe - A Bit On The Side @ Turing As part of A Bit On The Side @ Turing I am organising a mini-Erlang Factory with Erlang Solutions. The confirmed speakers so far are: * keynote from Robert Virding * Mobile Couchbase - a talk about CouchDb on Android by Dale Harvey of erldocs.com fame * IronMaster - an open source platform manager for the cloud by Sam Elliott There is still room for a couple of more talks so if you are an Erlang bug who will be in or around the Edinburgh Festival on Sunday 28th August, or your boss wants you to plug the product and you fancy 'happening' to be at the worlds biggest arts, culture and comedy festival whilst doing your duty, get in touch. More details here: http://abitontheside.turingfestival.com/63443486 Cheers Gordon -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From nick.sfx.1@REDACTED Tue Aug 2 00:53:25 2011 From: nick.sfx.1@REDACTED (Nick S) Date: Tue, 2 Aug 2011 01:53:25 +0300 Subject: [erlang-questions] Developing killer / open source apps Message-ID: I have just finished implementing a "professional" Erlang server, and with time and enthusiasm in hand, I would like to start working on something personal. I must say, it has been really fun writing apps in OTP and see it processing millions of messages without ever crashing...! Would appreciate some new ideas, that would be useful to community :) - XMPP, AMQP etc are already in market! - Any audio/video processing layer? - Any telecom related...? - Some standard protocol implementation? - Something new for next generation...? (Buzz words... cloud computing etc..) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngocdaothanh@REDACTED Tue Aug 2 01:15:04 2011 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Tue, 2 Aug 2011 08:15:04 +0900 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: A package manager that supports repositories? On Tue, Aug 2, 2011 at 7:53 AM, Nick S wrote: > I have just finished implementing a "professional" Erlang server, and with > time and enthusiasm in hand, I would like to start working on something > personal. > I must say, it has been really fun writing apps in OTP and see it processing > millions of messages without ever crashing...! > > Would appreciate some new ideas, that would be useful to community :) > > - XMPP, AMQP etc are already in market! > > - Any audio/video processing layer? > > - Any telecom related...? > > - Some standard protocol implementation? > > - Something new for next generation...? (Buzz words... cloud computing > etc..) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From gnoblin@REDACTED Tue Aug 2 01:37:23 2011 From: gnoblin@REDACTED (Slav Pankratov) Date: Tue, 2 Aug 2011 02:37:23 +0300 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: something related to social games? [?] 2011/8/2 Nick S > I have just finished implementing a "professional" Erlang server, and with > time and enthusiasm in hand, I would like to start working on something > personal. > I must say, it has been really fun writing apps in OTP and see it > processing millions of messages without ever crashing...! > > Would appreciate some new ideas, that would be useful to community :) > > - XMPP, AMQP etc are already in market! > > - Any audio/video processing layer? > > - Any telecom related...? > > - Some standard protocol implementation? > > - Something new for next generation...? (Buzz words... cloud computing > etc..) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 330.gif Type: image/gif Size: 96 bytes Desc: not available URL: From olivier.girondel@REDACTED Tue Aug 2 02:09:22 2011 From: olivier.girondel@REDACTED (Olivier Girondel) Date: Tue, 2 Aug 2011 02:09:22 +0200 Subject: [erlang-questions] [ANN] euuid 1.0.0 - an erlang UUID implementation In-Reply-To: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> References: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> Message-ID: On Mon, Aug 1, 2011 at 8:37 PM, Martijn Rijkeboer wrote: > Hello, > > As an exercise for learning Erlang I wrote an Universally Unique > IDentifier (UUID) implementation as defined in RFC 4122. This module > implements the UUID versions 1, 3, 4 and 5. It also implements a custom > time based UUID that is based on version 1 but has a different layout to > allow easy sorting on the creation date. This module is available as > open-source under the MIT licence for anyone to use. > > You can take a look at the code on my github repository: > - https://github.com/mrijkeboer/euuid > > Since this is my first Erlang program please let me know what you think. > I'm open to suggestions and enhancements. Hello Martijn, Well, UUIDs seem to be: 1. A good exercise 2. Implemented over and over again (I'll just quote mine, but a good way to check your code would be to have a look on older implementations) 3. A good candidate for inclusion somewhere in a library upstream :) My (own and old) 2 cents http://erlang.biniou.org/erluuid/ Best regards, -- Olivier / http://biniou.net From dale@REDACTED Tue Aug 2 03:11:33 2011 From: dale@REDACTED (Dale Harvey) Date: Tue, 2 Aug 2011 02:11:33 +0100 Subject: [erlang-questions] my best erlang "foot in mouth" In-Reply-To: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> References: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> Message-ID: Writing a new module -module(http). that particular one wont screw up on the current otp release, but its fairly easy to write one that will if you dont use your own prefix, and it causes the hardest to diagnose errors around. On 1 August 2011 17:36, Joel Reymont wrote: > gen_server:cast(stop, Pid). > > Anybody else? > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethstone@REDACTED Tue Aug 2 03:16:49 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Mon, 1 Aug 2011 20:16:49 -0500 Subject: [erlang-questions] my best erlang "foot in mouth" In-Reply-To: References: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> Message-ID: list_to_atom on increasing sequence numbers will make for a good crash... Kenny On Mon, Aug 1, 2011 at 8:11 PM, Dale Harvey wrote: > Writing a new module > > -module(http). > > that particular one wont screw up on the current otp release, but its > fairly easy to write one that will if you dont use your own prefix, and it > causes the hardest to diagnose errors around. > > > On 1 August 2011 17:36, Joel Reymont wrote: > >> gen_server:cast(stop, Pid). >> >> Anybody else? >> >> -------------------------------------------------------------------------- >> - for hire: mac osx device driver ninja, kernel extensions and usb drivers >> ---------------------+------------+--------------------------------------- >> http://wagerlabs.com | @wagerlabs | >> http://www.linkedin.com/in/joelreymont >> ---------------------+------------+--------------------------------------- >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From overminddl1@REDACTED Tue Aug 2 04:55:36 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Mon, 1 Aug 2011 20:55:36 -0600 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: Message-ID: On Fri, Jul 29, 2011 at 12:17 PM, Icarus Alive wrote: > On Fri, Jul 29, 2011 at 7:43 PM, OvermindDL1 wrote: >> For note, my background is also 15+ years of C++, and although it and Erlang >> are different, C++ templates are very much similar to Erlangs syntax (and >> dang near identical to Haskell), so if you were a heavy template writer like >> I am then think of it that way. I jumped into Erlang and picked it up very >> quickly. > > @OvermindDL1, very glad to hear that. Not familiar with Haskell, but > C++ templates are a familiar territory. > >> Oh, and I use Nitrogen, mostly as I like to generate pages in code and theme >> and layout using css; I try to use as little straight html as possible, but >> using html directly is something Zotonic if good at if you prefer that. >> >> I can help with your C++ to Erlang mental conversions if you want. > > That would be really a great help. However, if you (or anyone else > here) had a chance to compare Nitrogen & Zotonic on the developer > efficiency and learning-curve aspects, would be good to hear. Also, > how does it compare in performance terms, i.e. programmatic HTML > generation in Erlang code, versus template driven HTML generation > using a dedicated templating engine (if I understood it correctly). I have not yet used Zotonic myself, but from what I gather they do about the same things but in different ways. It seems Nitrogen likes to generate web pages in code, although it can use templates. It seems Zotonic likes to generate web pages in templates, although it can use code. Zotonic seems to have a few more features then Nitrogen, but are easily made in Nitrogen thanks to its pure-code extension system. I chose Nitrogen because I prefer code to generate pages, my pages are mostly informational server stuff, so I do not need an html kid to do those things for me, but if you had one, maybe Zotonic? Both are quite fast, even the templates (from my understanding) are compiled to Erlang compiled beam files so there should not be any real difference. On Sat, Jul 30, 2011 at 10:51 AM, Jon Watte wrote: > The "web" pages I've written in Erlang are mostly just status and management > REST interfaces for a system whose main goal is something else, but what > I've found so far: > > If you want a web interface where you take the request once headers are > parsed and the request decoded, use mochiweb. > > If you want an industrial-strength HTTP protocol server stack (things like > content type negotiation, etc) for a REST-only implementation, use > webmachine. Exactly. Nitrogen/Zotonic are good for making the more direct 'website' kind of interface, but for making web services, something raw like mochiweb/cowboy or webmachine (webmachine excels at REST more then anything else) are better. From ngocdaothanh@REDACTED Tue Aug 2 06:57:43 2011 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Tue, 2 Aug 2011 13:57:43 +0900 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: Is there a tool for Erlang which is similar to gem for Ruby or npm for Node.js? I guess the main reason why libraries of Ruby and Node.js have grown so fast in just a few years is because they have gem and npm. On Tue, Aug 2, 2011 at 1:28 PM, Max Lapshin wrote: > On Tue, Aug 2, 2011 at 3:15 AM, Ngoc Dao wrote: >> A package manager that supports repositories? >> > > It is dull and dumb infrastructure that has nothing to do with erlang itself. > > > Erlang is really great at messaging, video delivery, online gaming. > From fabian.krol@REDACTED Tue Aug 2 06:59:59 2011 From: fabian.krol@REDACTED (Fabian Krol) Date: Tue, 2 Aug 2011 06:59:59 +0200 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: Yes, there is agner. http://erlagner.org/ -- Fabian Kr?l +48 794 746861 www.fabiankrol.com On Tue, Aug 2, 2011 at 6:57 AM, Ngoc Dao wrote: > Is there a tool for Erlang which is similar to gem for Ruby or npm for Node.js? > > I guess the main reason why libraries of Ruby and Node.js have grown > so fast in just a few years is because they have gem and npm. > > > On Tue, Aug 2, 2011 at 1:28 PM, Max Lapshin wrote: >> On Tue, Aug 2, 2011 at 3:15 AM, Ngoc Dao wrote: >>> A package manager that supports repositories? >>> >> >> It is dull and dumb infrastructure that has nothing to do with erlang itself. >> >> >> Erlang is really great at messaging, video delivery, online gaming. >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ngocdaothanh@REDACTED Tue Aug 2 07:09:03 2011 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Tue, 2 Aug 2011 14:09:03 +0900 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: Thank you for the link. This is great! On Tue, Aug 2, 2011 at 1:59 PM, Fabian Krol wrote: > Yes, there is agner. > http://erlagner.org/ > > > -- > Fabian Kr?l > +48 794 746861 > www.fabiankrol.com > > > > > On Tue, Aug 2, 2011 at 6:57 AM, Ngoc Dao wrote: >> Is there a tool for Erlang which is similar to gem for Ruby or npm for Node.js? >> >> I guess the main reason why libraries of Ruby and Node.js have grown >> so fast in just a few years is because they have gem and npm. >> >> >> On Tue, Aug 2, 2011 at 1:28 PM, Max Lapshin wrote: >>> On Tue, Aug 2, 2011 at 3:15 AM, Ngoc Dao wrote: >>>> A package manager that supports repositories? >>>> >>> >>> It is dull and dumb infrastructure that has nothing to do with erlang itself. >>> >>> >>> Erlang is really great at messaging, video delivery, online gaming. >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From ok@REDACTED Tue Aug 2 07:59:50 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 2 Aug 2011 17:59:50 +1200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> Message-ID: <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> The basic misunderstanding here is that I was describing one desirable instance of a style warnings facility that does not yet exist. I was not supposing or implying that any means of selectively {en,dis}abling warning exists. That was not the focus of my attention. I assumed for the purpose of argument something like the $SET ... $RESET $POP facility found in 40-year-old Burroughs compilers for Algol, Fortran, COBOL, BASIC et al or the similar age (*$+I*) (*$-R*) check switches found in many Pascal compilers, not any sort of function annotation scheme. That's not to say that a function annotation scheme (perhaps not entirely unlike the one found in Ciao Prolog) might not be a good thing, just that at this stage of the conversation, it's more useful to focus on 1. Do we want lint-like checks for Erlang at all? 2. Is matching against an already-bound variable a check we want? 3. What are some other checks we might have a use for? The SmallLint checks found in the Refactoring Browser for Smalltalk contain many examples of such checks, most of which don't apply to Erlang, although being able to check for consistency in the spelling of function and variable names is rather nice. Here's one that makes sense for Erlang: if G1 -> S1, S ; G2 -> S2, S ... end could have been if G1 -> S1 ; G2 -> S2 ... end, S From S.J.Thompson@REDACTED Tue Aug 2 09:12:51 2011 From: S.J.Thompson@REDACTED (Simon Thompson) Date: Tue, 2 Aug 2011 09:12:51 +0200 Subject: [erlang-questions] IFL 2011 - final call for papers Message-ID: <953BE57D-E851-4F5B-91BA-E752DAB7771F@kent.ac.uk> Erlang papers very welcome ... CALL FOR PAPERS 23rd Symposium on Implementation and Application of Functional Languages (IFL 2011) October 3-5, 2011 University of Kansas Lawrence, Kansas, USA http://www.ittc.ku.edu/ifl2011 The Symposium returns to the US this year, hosted by the University of Kansas, in scenic Lawrence, Kansas. Lawrence is a lively college town less than an hour from Kansas City and the Kansas City International Airport (MCI). The symposium dates are October 3-5, 2011. The goal of the IFL symposia is to bring together researchers actively engaged in the implementation and application of functional and function-based programming languages. IFL 2011 will be a venue for researchers to present and discuss new ideas and concepts, work in progress, and publication-ripe results related to the implementation and application of functional languages and function-based programming. Following the IFL tradition, IFL 2011 will use a post-symposium review process to produce formal proceedings which will be published by Springer Verlag in the Lecture Notes in Computer Science series. All participants in IFL 2011 are invited to submit either a draft paper or an extended abstract describing work to be presented at the symposium. At no time may work submitted to IFL be simultaneously submitted to other venues. Here we follow the ACM Sigplan republication policy: http://www.sigplan.org/republicationpolicy.htm. The submissions will be screened by the program committee chair to make sure they are within the scope of IFL, and will appear in the draft proceedings distributed at the symposium. Submissions appearing in the draft proceedings are not peer-reviewed publications. After the symposium, authors will be given the opportunity to incorporate the feedback from discussions at the symposium and will be invited to submit a revised full article for the formal review process. These revised submissions will be reviewed by the program committee using prevailing academic standards to select the best articles, which will appear in the formal proceedings. INVITED SPEAKER Bryan O'Sullivan, author of Real World Haskell, and co-founder of MailRank, Inc. TOPICS IFL welcomes submissions describing practical and theoretical work as well as submissions describing applications and tools. If you are not sure that your work is appropriate for IFL 2011, please contact the PC chair at andygill@REDACTED Topics of interest include, but are not limited to: language concepts type checking contracts compilation techniques staged compilation runtime function specialization runtime code generation partial evaluation (abstract) interpretation generic programming techniques automatic program generation array processing concurrent/parallel programming concurrent/parallel program execution functional programming and embedded systems functional programming and web applications functional programming and security novel memory management techniques runtime profiling and performance measurements debugging and tracing virtual/abstract machine architectures validation and verification of functional programs tools and programming techniques industrial applications of functional programming PAPER SUBMISSIONS Prospective authors are encouraged to submit papers or extended abstracts to be published in the draft proceedings and to present them at the symposium. All contributions must be written in English, conform to the Springer-Verlag LNCS series format and not exceed 16 pages. The draft proceedings will appear as a technical report of the Department of Electrical Engineering and Computer Science of the University of Kansas. (We are more liberal with the draft proceedings, also accepting (for example) longer papers or SIGPLAN 2 column 12 page papers. For other formats contact the chair. For the consideration for the final proceedings, the 16 page LNCS format is required.) PETER LANDIN PRIZE The Peter Landin Prize is awarded to the best paper presented at the symposium every year. The honored article is selected by the program committee based on the submissions received for the formal review process. The prize carries a cash award equivalent to 150 Euros. IMPORTANT DATES Presentation submission deadline August 15th, 2011 Notification of acceptance: August 19th, 2011 Early registration deadline: September 2nd, 2011 IFL 2011 Symposium: October 3-5, 2011 Submission for (post) review process: November 30th, 2011 Notification Accept/Reject: January 30th, 2012 Camera ready version: March 9th, 2012 PROGRAM COMMITTEE Torben Amtoft, Kansas State University Francesco Cesarini, Erlang Solutions Ltd Olaf Chitil, University of Kent Eelco Dolstra, Delft University of Technology Martin Erwig, Oregon State University Andy Gill, University of Kansas (Chair) Alwyn Goodloe, NASA Jurriaan Hage, Utrecht University Kevin Hammond, University of St. Andrews Bill Harrison, University of Missouri Ralf Hinze, Oxford University James Hook, Portland State University Garrin Kimmell, University of Iowa Andres L??h, Well-Typed LLP Rita Loogen, Philipps-University Marburg Neil Mitchell, Standard Chartered Rex Page, Oklahoma University Rinus Plasmeijer, Radboud University Nijmegen Sven-Bodo Scholz, University of Hertfordshire Mary Sheeran, Chalmers Satnam Singh, Microsoft Research Walid Taha, Halmstad University Simon Thompson, University of Kent Geoffrey Washburn, LogicBlox Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt From desired.mta@REDACTED Tue Aug 2 10:27:52 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Tue, 2 Aug 2011 11:27:52 +0300 Subject: [erlang-questions] Erlang de/serializer and data type validator Message-ID: <20110802082752.GA21594@jakstys.lt> Hello, I have a data structure validator. It should check if variable has a pre-defined data structure. The validator takes Erlang data type (in this example, #'bp.message'{}) and checks if: 1) to is either {participant | bpid, binary() | undefined} 2) module either se_port_v2 or se_transfer_v2 Part of validator schema: schema('bp.message') -> {record, 'bp.message', [ {to, schema('bp.message.destination')} {module, schema('bp.message.module')}, ]}; schema('bp.message.destination') -> {tuple, {choice, ['participant', 'bpid']}, {choice, ['binary', undefined]}}; schema('bp.message.module') -> {choice, [ se_assign_v2, se_release_v2, ]}; Two questions: 1) I feel I could do the same with erlang -type. How do I check the data structure complies? 2) I need serializer and deserializer to JSON, XML or any other common data format. Maybe there is such a thing in the wild? Serialized structure must not hold Erlang data type information as I have to share it with third parties. It should distinguish between atom and binary from schema (as well as list from tuple..). Serialized message to JSON would look like this: { "bp.message" : { "to" : [participant, "yadda"], "module" : "se_assign_v2" } } If I will have to write one, how do I recursively crawl type definition information? Motiejus From ulf.wiger@REDACTED Tue Aug 2 11:23:16 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 2 Aug 2011 11:23:16 +0200 Subject: [erlang-questions] Erlang de/serializer and data type validator In-Reply-To: <20110802082752.GA21594@jakstys.lt> References: <20110802082752.GA21594@jakstys.lt> Message-ID: <61DEDBFF-8B0A-48A4-A101-1E6C59568600@erlang-solutions.com> Hi Motiejus, Perhaps you should look into Joe Norton's work with UBF? UBF is a type system for describing contracts between services. http://www.sics.se/~joe/ubf/site/home.html (for original documentation) https://github.com/norton/ubf (for the current stuff) There is a UBF-to-JSON converter. There is also one for Thrift, and Joe Norton has combined UBF, UBF-JSON and QuickCheck to do validation of web services. I don't think UBF to XML would be particularly difficult. There may be some more work that needs to be done before it fits your problem. OTOH, you might get a lot of other benefits going this route. BR, Ulf W On 2 Aug 2011, at 10:27, Motiejus Jak?tys wrote: > Hello, > > I have a data structure validator. It should check if variable has > a pre-defined data structure. > > The validator takes Erlang data type (in this example, #'bp.message'{}) > and checks if: > 1) to is either {participant | bpid, binary() | undefined} > 2) module either se_port_v2 or se_transfer_v2 > > Part of validator schema: > > schema('bp.message') -> > {record, 'bp.message', [ > {to, schema('bp.message.destination')} > {module, schema('bp.message.module')}, > ]}; > > schema('bp.message.destination') -> > {tuple, {choice, ['participant', 'bpid']}, {choice, ['binary', undefined]}}; > > schema('bp.message.module') -> > {choice, [ > se_assign_v2, > se_release_v2, > ]}; > > Two questions: > 1) I feel I could do the same with erlang -type. How do I check the data > structure complies? > 2) I need serializer and deserializer to JSON, XML or any other common > data format. Maybe there is such a thing in the wild? > > Serialized structure must not hold Erlang data type information as I > have to share it with third parties. It should distinguish between atom > and binary from schema (as well as list from tuple..). Serialized message > to JSON would look like this: > > { "bp.message" : { > "to" : [participant, "yadda"], > "module" : "se_assign_v2" > } > } > > If I will have to write one, how do I recursively crawl type definition > information? > > Motiejus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Tue Aug 2 13:07:25 2011 From: mihai@REDACTED (Mihai Balea) Date: Tue, 2 Aug 2011 07:07:25 -0400 Subject: [erlang-questions] my best erlang "foot in mouth" In-Reply-To: References: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> Message-ID: <295220F8-A1F2-49B8-A985-45C298F9C843@hates.ms> Forgetting a comma between strings in a list, like ["foo" "bar"] Made for a _very_ insidious and hard to find bug. Mihai On Aug 1, 2011, at 9:16 PM, Kenny Stone wrote: > list_to_atom on increasing sequence numbers will make for a good crash... > > Kenny > > On Mon, Aug 1, 2011 at 8:11 PM, Dale Harvey wrote: > Writing a new module > > -module(http). > > that particular one wont screw up on the current otp release, but its fairly easy to write one that will if you dont use your own prefix, and it causes the hardest to diagnose errors around. > > > On 1 August 2011 17:36, Joel Reymont wrote: > gen_server:cast(stop, Pid). > > Anybody else? > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Tue Aug 2 15:08:57 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 2 Aug 2011 15:08:57 +0200 Subject: [erlang-questions] Gproc in a dynamic cluster In-Reply-To: References: <3E83ACB7-9B81-4812-AF4A-F153D2CFA3F8@hates.ms> <6CD0A133-6F6D-408C-ACA3-B5237C6A67CC@erlang-solutions.com> <36ADA8AF-80F2-4BA1-A655-AD1F6AEE56DA@hates.ms> Message-ID: <8A4A4005-CD8E-4CA5-A3C0-E1436C992CD2@erlang-solutions.com> On 1 Aug 2011, at 16:00, Ulf Wiger wrote: > This could all be baked into the 'all' option somehow. I'll think about that. Turns out that adding the {bcast_type, all} option did the trick. In the HEAD version of http://github.com/esl/gproc, dynamic addition of nodes seems to work as expected when you use the option -gproc gproc_dist all. Let me know if it doesn't work for you. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From kennethstone@REDACTED Tue Aug 2 15:25:51 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Tue, 2 Aug 2011 08:25:51 -0500 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: Message-ID: Nitrogen has a very cool new(ish) feature where you can use both nitrogen and webmachine in the same project pretty easily. Kenny On Mon, Aug 1, 2011 at 9:55 PM, OvermindDL1 wrote: > On Fri, Jul 29, 2011 at 12:17 PM, Icarus Alive > wrote: > > On Fri, Jul 29, 2011 at 7:43 PM, OvermindDL1 > wrote: > >> For note, my background is also 15+ years of C++, and although it and > Erlang > >> are different, C++ templates are very much similar to Erlangs syntax > (and > >> dang near identical to Haskell), so if you were a heavy template writer > like > >> I am then think of it that way. I jumped into Erlang and picked it up > very > >> quickly. > > > > @OvermindDL1, very glad to hear that. Not familiar with Haskell, but > > C++ templates are a familiar territory. > > > >> Oh, and I use Nitrogen, mostly as I like to generate pages in code and > theme > >> and layout using css; I try to use as little straight html as possible, > but > >> using html directly is something Zotonic if good at if you prefer that. > >> > >> I can help with your C++ to Erlang mental conversions if you want. > > > > That would be really a great help. However, if you (or anyone else > > here) had a chance to compare Nitrogen & Zotonic on the developer > > efficiency and learning-curve aspects, would be good to hear. Also, > > how does it compare in performance terms, i.e. programmatic HTML > > generation in Erlang code, versus template driven HTML generation > > using a dedicated templating engine (if I understood it correctly). > > I have not yet used Zotonic myself, but from what I gather they do > about the same things but in different ways. > > It seems Nitrogen likes to generate web pages in code, although it can > use templates. > It seems Zotonic likes to generate web pages in templates, although it > can use code. > Zotonic seems to have a few more features then Nitrogen, but are > easily made in Nitrogen thanks to its pure-code extension system. > > I chose Nitrogen because I prefer code to generate pages, my pages are > mostly informational server stuff, so I do not need an html kid to do > those things for me, but if you had one, maybe Zotonic? > > Both are quite fast, even the templates (from my understanding) are > compiled to Erlang compiled beam files so there should not be any real > difference. > > > On Sat, Jul 30, 2011 at 10:51 AM, Jon Watte wrote: > > The "web" pages I've written in Erlang are mostly just status and > management > > REST interfaces for a system whose main goal is something else, but what > > I've found so far: > > > > If you want a web interface where you take the request once headers are > > parsed and the request decoded, use mochiweb. > > > > If you want an industrial-strength HTTP protocol server stack (things > like > > content type negotiation, etc) for a REST-only implementation, use > > webmachine. > > Exactly. Nitrogen/Zotonic are good for making the more direct > 'website' kind of interface, but for making web services, something > raw like mochiweb/cowboy or webmachine (webmachine excels at REST more > then anything else) are better. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Tue Aug 2 16:32:00 2011 From: mihai@REDACTED (Mihai Balea) Date: Tue, 2 Aug 2011 10:32:00 -0400 Subject: [erlang-questions] Gproc in a dynamic cluster In-Reply-To: <8A4A4005-CD8E-4CA5-A3C0-E1436C992CD2@erlang-solutions.com> References: <3E83ACB7-9B81-4812-AF4A-F153D2CFA3F8@hates.ms> <6CD0A133-6F6D-408C-ACA3-B5237C6A67CC@erlang-solutions.com> <36ADA8AF-80F2-4BA1-A655-AD1F6AEE56DA@hates.ms> <8A4A4005-CD8E-4CA5-A3C0-E1436C992CD2@erlang-solutions.com> Message-ID: <52519FFE-29AA-4866-8F7D-A494EFCCBC14@hates.ms> On Aug 2, 2011, at 9:08 AM, Ulf Wiger wrote: > > On 1 Aug 2011, at 16:00, Ulf Wiger wrote: > >> This could all be baked into the 'all' option somehow. I'll think about that. > > Turns out that adding the {bcast_type, all} option did the trick. > > In the HEAD version of http://github.com/esl/gproc, dynamic addition of nodes seems > to work as expected when you use the option -gproc gproc_dist all. > > Let me know if it doesn't work for you. Still no automatic workage, but I wasn't really expecting that. Is there a way to manually prod gproc into joining another nodes after startup? The way i have it set up, nodes boot with gproc booting in global mode as a top level app. Then nodes find each other using an out of band mechanism and use net_adm:ping to connect. By glancing at gen_leader code, it appears there's no way for gen_leader to automatically be aware of newly connected nodes, so I figure there might be a way to indicate it manually. Thanks, Mihai From skruger@REDACTED Tue Aug 2 16:43:12 2011 From: skruger@REDACTED (Shaun Kruger) Date: Tue, 02 Aug 2011 08:43:12 -0600 (MDT) Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: Message-ID: I could really use some contributions for my proxy server/load balancer project. :) https://github.com/skruger/Surrogate I recently modified it to allow for writing multiple protocol handlers (not just HTTP proxy) and I'm working on the admin interface finally. I know it's a shameless plug, but the project really is cool. Though, I might just think that because I wrote it. :) Shaun ----- Original Message ----- > From: "Nick S" > To: erlang-questions@REDACTED > Sent: Monday, August 1, 2011 4:53:25 PM > Subject: [erlang-questions] Developing killer / open source apps > > > I have just finished implementing a "professional" Erlang server, and > with time and enthusiasm in hand, I would like to start working on > something personal. > I must say, it has been really fun writing apps in OTP and see it > processing millions of messages without ever crashing...! > > Would appreciate some new ideas, that would be useful to community :) > > - XMPP, AMQP etc are already in market! > > - Any audio/video processing layer? > > - Any telecom related...? > > - Some standard protocol implementation? > > - Something new for next generation...? (Buzz words... cloud > computing etc..) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf.wiger@REDACTED Tue Aug 2 16:43:30 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 2 Aug 2011 16:43:30 +0200 Subject: [erlang-questions] Gproc in a dynamic cluster In-Reply-To: <52519FFE-29AA-4866-8F7D-A494EFCCBC14@hates.ms> References: <3E83ACB7-9B81-4812-AF4A-F153D2CFA3F8@hates.ms> <6CD0A133-6F6D-408C-ACA3-B5237C6A67CC@erlang-solutions.com> <36ADA8AF-80F2-4BA1-A655-AD1F6AEE56DA@hates.ms> <8A4A4005-CD8E-4CA5-A3C0-E1436C992CD2@erlang-solutions.com> <52519FFE-29AA-4866-8F7D-A494EFCCBC14@hates.ms> Message-ID: <668C7836-B388-4BCE-9170-011CADB451C3@erlang-solutions.com> On 2 Aug 2011, at 16:32, Mihai Balea wrote: > > On Aug 2, 2011, at 9:08 AM, Ulf Wiger wrote: > >> >> On 1 Aug 2011, at 16:00, Ulf Wiger wrote: >> >>> This could all be baked into the 'all' option somehow. I'll think about that. >> >> Turns out that adding the {bcast_type, all} option did the trick. >> >> In the HEAD version of http://github.com/esl/gproc, dynamic addition of nodes seems >> to work as expected when you use the option -gproc gproc_dist all. >> >> Let me know if it doesn't work for you. > > Still no automatic workage, but I wasn't really expecting that. What exactly did you try? > Is there a way to manually prod gproc into joining another nodes after startup? The way i have it set up, nodes boot with gproc booting in global mode as a top level app. Then nodes find each other using an out of band mechanism and use net_adm:ping to connect. By glancing at gen_leader code, it appears there's no way for gen_leader to automatically be aware of newly connected nodes, so I figure there might be a way to indicate it manually. What I tried, and what worked for me, was: - starting node n1 with (-gproc gproc_dist all), and doing application:start(gproc) - starting node n2 the same way (making sure to first ping n1) - registering a global resource at n2, verifying that it was replicated to n1 - starting n3 - verifying that the global resource was visible on n3 afterwards Basically, when a new instance of gproc joins the cluster, the current leader will broadcast an updated candidate list. Note that the nodes must be connected before starting gproc. Gproc_dist will simply use the nodes() list, so if the nodes haven't connected first, it won't work. Note also that gen_leader (and thus gproc) has no facility for resolving netsplits. This is a reason why you need to do it this way; otherwise, the procedure can't be easily distinguished from a netsplit. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From mihai@REDACTED Tue Aug 2 17:00:16 2011 From: mihai@REDACTED (Mihai Balea) Date: Tue, 2 Aug 2011 11:00:16 -0400 Subject: [erlang-questions] Gproc in a dynamic cluster In-Reply-To: <668C7836-B388-4BCE-9170-011CADB451C3@erlang-solutions.com> References: <3E83ACB7-9B81-4812-AF4A-F153D2CFA3F8@hates.ms> <6CD0A133-6F6D-408C-ACA3-B5237C6A67CC@erlang-solutions.com> <36ADA8AF-80F2-4BA1-A655-AD1F6AEE56DA@hates.ms> <8A4A4005-CD8E-4CA5-A3C0-E1436C992CD2@erlang-solutions.com> <52519FFE-29AA-4866-8F7D-A494EFCCBC14@hates.ms> <668C7836-B388-4BCE-9170-011CADB451C3@erlang-solutions.com> Message-ID: <09E08DBF-B4EE-4B34-9F1E-14EC7579C636@hates.ms> On Aug 2, 2011, at 10:43 AM, Ulf Wiger wrote: > > On 2 Aug 2011, at 16:32, Mihai Balea wrote: > >> >> On Aug 2, 2011, at 9:08 AM, Ulf Wiger wrote: >> >>> >>> On 1 Aug 2011, at 16:00, Ulf Wiger wrote: >>> >>>> This could all be baked into the 'all' option somehow. I'll think about that. >>> >>> Turns out that adding the {bcast_type, all} option did the trick. >>> >>> In the HEAD version of http://github.com/esl/gproc, dynamic addition of nodes seems >>> to work as expected when you use the option -gproc gproc_dist all. >>> >>> Let me know if it doesn't work for you. >> >> Still no automatic workage, but I wasn't really expecting that. > > What exactly did you try? > >> Is there a way to manually prod gproc into joining another nodes after startup? The way i have it set up, nodes boot with gproc booting in global mode as a top level app. Then nodes find each other using an out of band mechanism and use net_adm:ping to connect. By glancing at gen_leader code, it appears there's no way for gen_leader to automatically be aware of newly connected nodes, so I figure there might be a way to indicate it manually. > > What I tried, and what worked for me, was: > > - starting node n1 with (-gproc gproc_dist all), and doing application:start(gproc) > - starting node n2 the same way (making sure to first ping n1) > - registering a global resource at n2, verifying that it was replicated to n1 > - starting n3 > - verifying that the global resource was visible on n3 afterwards > > Basically, when a new instance of gproc joins the cluster, the current leader will broadcast an updated candidate list. > > Note that the nodes must be connected before starting gproc. Gproc_dist will simply use the nodes() list, so if the nodes haven't connected first, it won't work. > > Note also that gen_leader (and thus gproc) has no facility for resolving netsplits. This is a reason why you need to do it this way; otherwise, the procedure can't be easily distinguished from a netsplit. That explains it. I'm starting gproc before connecting the nodes. I could delay gproc's start until the cluster is connected, but I suspect it would be easier to just switch to the built in global registry. Still, generally speaking, what would it take to add netsplit handling to gproc? It would be nice to be able to use some of gproc advanced features. Besides, I believe the initial intention for gproc was to be a replacement for global, and it cannot fill that role until it handles netsplits. Anyway, thank you so much for your assistance. Best regards, Mihai From roberto@REDACTED Tue Aug 2 19:20:49 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 2 Aug 2011 19:20:49 +0200 Subject: [erlang-questions] yet another Erlang WebSocket implementation In-Reply-To: References: Message-ID: hi andy, many code portions of this project of yours have been integrated into misultin 0.8.1-dev. i've added behaviours to support various websocket implementations. please feel free to check it out and comment. https://github.com/ostinelli/misultin/tree/dev r. 2011/7/29 Andy W. Song > Hi, > > Not to reinvent the wheel, just as an exercise for learning Erlang. > > Here you go. Copied some ideas > from this project as well as > misultin and cowboy. > > It is not complete right now. Doesn't support segment yet, which I hope is > not used very often. And it should be fairly easy to replace the current > implementation in misultin. > > Enjoy. > > Andy > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Tue Aug 2 21:11:35 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 2 Aug 2011 21:11:35 +0200 Subject: [erlang-questions] Gproc in a dynamic cluster In-Reply-To: <09E08DBF-B4EE-4B34-9F1E-14EC7579C636@hates.ms> References: <3E83ACB7-9B81-4812-AF4A-F153D2CFA3F8@hates.ms> <6CD0A133-6F6D-408C-ACA3-B5237C6A67CC@erlang-solutions.com> <36ADA8AF-80F2-4BA1-A655-AD1F6AEE56DA@hates.ms> <8A4A4005-CD8E-4CA5-A3C0-E1436C992CD2@erlang-solutions.com> <52519FFE-29AA-4866-8F7D-A494EFCCBC14@hates.ms> <668C7836-B388-4BCE-9170-011CADB451C3@erlang-solutions.com> <09E08DBF-B4EE-4B34-9F1E-14EC7579C636@hates.ms> Message-ID: On 2 Aug 2011, at 17:00, Mihai Balea wrote: > Still, generally speaking, what would it take to add netsplit handling to gproc? It would be nice to be able to use some of gproc advanced features. Besides, I believe the initial intention for gproc was to be a replacement for global, and it cannot fill that role until it handles netsplits. The first step is to make gen_leader detect netsplits. I'm working on it, but no promises. ;-) BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From watson.timothy@REDACTED Tue Aug 2 22:10:49 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 2 Aug 2011 21:10:49 +0100 Subject: [erlang-questions] Erlang de/serializer and data type validator In-Reply-To: <61DEDBFF-8B0A-48A4-A101-1E6C59568600@erlang-solutions.com> References: <20110802082752.GA21594@jakstys.lt> <61DEDBFF-8B0A-48A4-A101-1E6C59568600@erlang-solutions.com> Message-ID: http://piqi.org/ might also be worth a look. 2011/8/2 Ulf Wiger : > > Hi Motiejus, > Perhaps you should look into Joe Norton's work with UBF? > UBF is a type system for describing contracts between services. > http://www.sics.se/~joe/ubf/site/home.html?(for original documentation) > https://github.com/norton/ubf?(for the current stuff) > There is a UBF-to-JSON converter. There is also one for Thrift, and Joe > Norton has combined UBF, UBF-JSON and QuickCheck to do validation of web > services. > I don't think UBF to XML would be particularly difficult. > There may be some more work that needs to be done before it fits your > problem. OTOH, you might get a lot of other benefits going this route. > BR, > Ulf W > On 2 Aug 2011, at 10:27, Motiejus Jak?tys wrote: > > Hello, > > I have a data structure validator. It should check if variable has > a pre-defined data structure. > > The validator takes Erlang data type (in this example, #'bp.message'{}) > and checks if: > 1) to is either {participant | bpid, binary() | undefined} > 2) module either se_port_v2 or se_transfer_v2 > > Part of validator schema: > > schema('bp.message') -> > ???{record, 'bp.message', [ > ???????{to, schema('bp.message.destination')} > ???????{module, schema('bp.message.module')}, > ???]}; > > schema('bp.message.destination') -> > ???{tuple, {choice, ['participant', 'bpid']}, {choice, ['binary', > undefined]}}; > > schema('bp.message.module') -> > ???{choice, [ > ???????se_assign_v2, > ???????se_release_v2, > ???]}; > > Two questions: > 1) I feel I could do the same with erlang -type. How do I check the data > structure complies? > 2) I need serializer and deserializer to JSON, XML or any other common > data format. Maybe there is such a thing in the wild? > > Serialized structure must not hold Erlang data type information as I > have to share it with third parties. It should distinguish between atom > and binary from schema (as well as list from tuple..). Serialized message > to JSON would look like this: > > { "bp.message" : { > ???"to" : [participant, "yadda"], > ???"module" : "se_assign_v2" > ????} > } > > If I will have to write one, how do I recursively crawl type definition > information? > > Motiejus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger,?CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From essen@REDACTED Tue Aug 2 22:32:11 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 02 Aug 2011 22:32:11 +0200 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: <4E385ECB.6000705@dev-extend.eu> On 08/02/2011 12:53 AM, Nick S wrote: > - Any audio/video processing layer? This. Do we have any good audio output library? Using something like OpenAL or any other big standardized library. But I suspect that's more NIF work than Erlang work. -- Lo?c Hoguin Dev:Extend From sam@REDACTED Wed Aug 3 02:52:55 2011 From: sam@REDACTED (Sam Bobroff) Date: Wed, 3 Aug 2011 10:52:55 +1000 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> Message-ID: Hello all, I'll put in my $0.02 on this because I've just been bitten by a bug that would have been caught by this new warning. I'll stick to the point: [snip] > 2. Is matching against an already-bound variable a check we want? > [snip] Yes, because it protects against a kind of bug that is otherwise very easy to introduce in Erlang: Quite often when debugging I want to add a new line or two to a function, to examine or check an intermediate value. e.g.: ... Result = function_call(), validate(Result), ... The problem is that I now have to scan down the whole rest of the function just in case Result has already been used because, if it has, then it's going to compile fine and cause a run time error! It's such a simple scenario that it seems very annoying that Erlang can't help me to avoid it. The "cost" of not being able to do a "backwards" assignment with the newly bound variable on the right doesn't seem like a cost at all: I find code that is that way round, or mixes assignments on the left and right (!) to be unnecessarily confusing. I'd be happy if this was required by the language but as this is the real world I'd be happy with a warning I could turn on. Sam. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sam@REDACTED Wed Aug 3 03:13:03 2011 From: sam@REDACTED (Sam Bobroff) Date: Wed, 3 Aug 2011 11:13:03 +1000 Subject: [erlang-questions] Incorrect supervisor behaviour and crash when mixing temporary and permanent children Message-ID: Hello Erlangers, I've recently tracked down a bug in some code that seems to be caused by a problem with the supervisor module. What seems to be happening is that when a temporary child has been added to a one_for_all supervisor using start_child, if a permanent child of that supervisor exits then if that permanent child exits, the supervisor attempts to restart the temporary child (which seems counter to the documentation). In addition the MFA for the child has been replaced with undefined and start attempt crashes, causing the supervisor to continue attempting restarts until it reaches it's restart intensity and shuts down. This does not happen if the temporary child is added to the supervisor as part of the supervisor's initial spec. In this case the temporary child is restart but it's MFA is present so it is able to restart it successfully, although it still seems contrary to the documentation that the temporary child is restarted at all, although it depends on which section of the documentation you regard as more important (from the supervisor documentation): one_for_all - if one child process terminates and should be restarted, all other child processes are terminated and then all child processes are restarted. This is quite clear: "all child processes are restarted". But later: Restart defines when a terminated child process should be restarted. A permanent child process should always be restarted, a temporary child process should never be restarted and a transient child process should be restarted only if it terminates abnormally, i.e. with another exit reason than normal. Again clear but conflicting: "a temporary child process should never be restarted". I found a similar issue mentioned in a bug report for R14B02, here: http://erlang.org/pipermail/erlang-bugs/2011-March/002273.html But in that case it seemed necessary to call restart_child() which makes it much less of a problem. I'm testing on R14B03 and I've produced a small piece of code to replicate the problem: --- begin --- -module(bug). -behaviour(supervisor). -export([test_one/0, test_two/0, spec/2, init/1, main/1]). test_one() -> application:start(sasl), supervisor:start_link({local, sup}, ?MODULE, [spec(foo, permanent)]), supervisor:start_child(sup, spec(bar, temporary)), foo ! die. test_two() -> application:start(sasl), supervisor:start_link({local, sup}, ?MODULE, [spec(foo, permanent), spec(bar, temporary)]), foo ! die. spec(Name, Type) -> { Name, {proc_lib, start_link, [?MODULE, main, [Name]]}, Type, 3000, worker, [bug] }. init(Children) -> {ok, {{one_for_all, 3, 10000}, Children}}. main(Name) -> register(Name, self()), proc_lib:init_ack({ok, self()}), receive die -> ok end. --- end --- Running test_one from the shell produces this output on my system: $ erl Erlang R14B03 (erts-5.8.4) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) 1> bug:test_one(). ** exception exit: shutdown 2> [snip SASL startup] =PROGRESS REPORT==== 3-Aug-2011::11:06:40 === supervisor: {local,sup} started: [{pid,<0.44.0>}, {name,foo}, {mfargs,{proc_lib,start_link,[bug,main,[foo]]}}, {restart_type,permanent}, {shutdown,3000}, {child_type,worker}] =PROGRESS REPORT==== 3-Aug-2011::11:06:40 === supervisor: {local,sup} started: [{pid,<0.45.0>}, {name,bar}, {mfargs,{proc_lib,start_link,[bug,main,[bar]]}}, {restart_type,temporary}, {shutdown,3000}, {child_type,worker}] =SUPERVISOR REPORT==== 3-Aug-2011::11:06:40 === Supervisor: {local,sup} Context: child_terminated Reason: normal Offender: [{pid,<0.44.0>}, {name,foo}, {mfargs,{proc_lib,start_link,[bug,main,[foo]]}}, {restart_type,permanent}, {shutdown,3000}, {child_type,worker}] =PROGRESS REPORT==== 3-Aug-2011::11:06:40 === supervisor: {local,sup} started: [{pid,<0.46.0>}, {name,foo}, {mfargs,{proc_lib,start_link,[bug,main,[foo]]}}, {restart_type,permanent}, {shutdown,3000}, {child_type,worker}] =SUPERVISOR REPORT==== 3-Aug-2011::11:06:40 === Supervisor: {local,sup} Context: start_error Reason: {'EXIT', {badarg, [{erlang,apply,[proc_lib,start_link,undefined]}, {supervisor,do_start_child,2}, {supervisor,start_children,3}, {supervisor,restart,3}, {supervisor,handle_info,2}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]}} Offender: [{pid,undefined}, {name,bar}, {mfargs,{proc_lib,start_link,undefined}}, {restart_type,temporary}, {shutdown,3000}, {child_type,worker}] [supervisor loops until it shuts down] Running test_two() shows it restarting the temporary child: 2> bug:test_two(). die =PROGRESS REPORT==== 3-Aug-2011::11:09:23 === supervisor: {local,sup} started: [{pid,<0.52.0>}, {name,foo}, {mfargs,{proc_lib,start_link,[bug,main,[foo]]}}, {restart_type,permanent}, {shutdown,3000}, {child_type,worker}] 3> =PROGRESS REPORT==== 3-Aug-2011::11:09:23 === supervisor: {local,sup} started: [{pid,<0.53.0>}, {name,bar}, {mfargs,{proc_lib,start_link,[bug,main,[bar]]}}, {restart_type,temporary}, {shutdown,3000}, {child_type,worker}] =SUPERVISOR REPORT==== 3-Aug-2011::11:09:23 === Supervisor: {local,sup} Context: child_terminated Reason: normal Offender: [{pid,<0.52.0>}, {name,foo}, {mfargs,{proc_lib,start_link,[bug,main,[foo]]}}, {restart_type,permanent}, {shutdown,3000}, {child_type,worker}] =PROGRESS REPORT==== 3-Aug-2011::11:09:23 === supervisor: {local,sup} started: [{pid,<0.54.0>}, {name,foo}, {mfargs,{proc_lib,start_link,[bug,main,[foo]]}}, {restart_type,permanent}, {shutdown,3000}, {child_type,worker}] =PROGRESS REPORT==== 3-Aug-2011::11:09:23 === supervisor: {local,sup} started: [{pid,<0.55.0>}, {name,bar}, {mfargs,{proc_lib,start_link,[bug,main,[bar]]}}, {restart_type,temporary}, {shutdown,3000}, {child_type,worker}] Am I doing something wrong or is this actually a bug (or bugs)? Peace, Sam. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wsongcn@REDACTED Wed Aug 3 06:28:28 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Wed, 3 Aug 2011 12:28:28 +0800 Subject: [erlang-questions] Strange Binary behavior Message-ID: As below: 12> A = "abc". "abc" 13> <>. ** exception error: bad argument 14> <<"abc", "CDE">>. <<"abcCDE">> Any idea? Regards Andy -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeffm@REDACTED Wed Aug 3 06:37:32 2011 From: jeffm@REDACTED (jm) Date: Wed, 03 Aug 2011 14:37:32 +1000 Subject: [erlang-questions] Strange Binary behavior In-Reply-To: References: Message-ID: <4E38D08C.3000403@ghostgun.com> On 3/08/11 2:28 PM, Andy W. Song wrote: > As below: > Seems to be expecting a type specifier. try A = "abc", B = list_to_binary(A), <>. The manual states that <<"abc">> is short hand for <<$a, $b, $c>> which suggests there is an exception when the list is in place. see http://www.erlang.org/doc/reference_manual/expressions.html#id77409 Jeff. From wsong.cn@REDACTED Wed Aug 3 07:21:13 2011 From: wsong.cn@REDACTED (Wenqiang Song) Date: Wed, 3 Aug 2011 13:21:13 +0800 Subject: [erlang-questions] Strange Binary behavior In-Reply-To: <4E38D08C.3000403@ghostgun.com> References: <4E38D08C.3000403@ghostgun.com> Message-ID: Thanks. <<"abc">> is just a syntactic sugar. I missed it. On Wed, Aug 3, 2011 at 12:37 PM, jm wrote: > > On 3/08/11 2:28 PM, Andy W. Song wrote: > >> As below: >> >> Seems to be expecting a type specifier. try > > A = "abc", > B = list_to_binary(A), > <>. > > > The manual states that <<"abc">> is short hand for <<$a, $b, $c>> which > suggests there is an exception when the list is in place. > > see http://www.erlang.org/doc/**reference_manual/expressions.** > html#id77409 > > Jeff. > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From icarus.alive@REDACTED Wed Aug 3 07:31:06 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Wed, 3 Aug 2011 11:01:06 +0530 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: <9158FB12-40A7-4276-BD18-C7CACF7690B7@worrell.nl> Message-ID: On Wed, Aug 3, 2011 at 3:54 AM, Benson Wong wrote: > Hi, > My background is a web developer (~15 years), done PHP for about 11 years. > So I know it pretty well. > Just starting to learn Erlang. Done lots of javascript (jQuery, node.js, > etc). So a PHP developer, in the Erlang pool here :) > I think an imperative language like PHP would be a lot easier (and more > agile) to build the web app in than Erlang. I would choose erlang if I need > long lived connections (web sockets, long-poll, etc). Frameworks like Django > (Python)/Rails (ruby) would also make life easier. > If you want scalability/reliability/performance, add more PHP processes / > servers. > PHP is shared nothing, so it is easy to scale. Just add more. Using Nginx + > PHP-FPM. Solves a lot of slow request issues. > Put them behind an HTTP load balancer, HAProxy is a very popular choice. > Performance is usually database bound than it is app server bound. > On Fri, Jul 29, 2011 at 6:16 AM, Icarus Alive > wrote: >> >> Would appreciate if my thread wasn't hijacked :-)... although gotta >> agree that our scenarios are quite similar. >> >> On Fri, Jul 29, 2011 at 6:31 PM, Banibrata Dutta >> wrote: >> > Thanks @Marc, for taking time to reply this this mail. >> > On Fri, Jul 29, 2011 at 5:38 PM, Marc Worrell wrote: >> >> >> >> It mostly depends on what kind of web development you want to do. >> >> >> >> Is it more of a mobile application than a publishing web site? >> >> In my case, content characteristics are: >> 1. Always originating from single source >> 2. Always consumed by single user >> 3. Content has short lifetime - once consumed, will be archived >> 4. Each user has several hundreds of content display to her in "newest >> first" fashion >> >> >> >> >> Does is have specific authentication schemes? >> >> Etc. Etc. >> > >> > Honestly, haven't given this enough thought. One of the requirements is >> > to >> > ensure privacy of user data, and that of the interaction. I'm yet to >> > figure >> > out the "how" part. > > Security/Privacy, this is more of an operation and application design than > it is a Erlang/OTP vs PHP/Ruby/Python+framework. > Generally you'd want to: > * firewalls, no SSH access, web servers are only web servers, ie: only port > 443 open. > * use cookie based sessions. Don't use HTTP basic authentication. > * Server has controls all access based on the cookie. > * Use HTTPS. Always. > * There are lots of other techniques: > -- use bcrypt for passwords > -- sessions can be limited to one IP address at a time > -- sessions have short expire times (controlled on the server) > -- etc. > >> >> Well in my case it has to be some strong authentication. Storage, >> archival and access has to conform to HIPAA compliance. So we know we >> are dealing with EMR type content. Hi Benson, Really appreciate your descriptive & informative response, especially so for the part on security. My inclination towards Erlang stems from the fact that I was intending to do much of the backend in Erlang as well. The backend does lot of other things than just bridging between FE client request/responses and a DB, s.a. converting content format, managing archival policies, do some data analysis for reporting etc. Maybe some of that via ports. I wish I'd seen your mail few days back, because having tried both Nitrogen and Zotonic, which are I am sure excellent products, I think I've found peace in settling for ChicagoBoss. I found it lot easier to work with, as a beginner, maybe because the FP guts are almost hidden away from sight, and much of the developer/user coding is done in an imperative fashion. Also, I found it's documentation (although quite light) to be simple & just-enough to get me started and get me going. However, I might have to come back to PHP unless I am able to train my team on Erlang. Being a beginner, I find it hard to imagine being a good teacher / coach for rest of the team. Thanks again, everyone, for responding on this thread. Icarus. From wsongcn@REDACTED Wed Aug 3 07:47:18 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Wed, 3 Aug 2011 13:47:18 +0800 Subject: [erlang-questions] yet another Erlang WebSocket implementation In-Reply-To: References: Message-ID: Good to hear that. Just tested that my client can talk to the new hybi-10 version of misultin. I found two things interesting: 1. In the handshake I use "Upgrade: WebSocket", while you use "Upgrade: websocket". Obviously yours is correct because your version is inline with the latest spec. Mine is old. The change happened between hybi-03 and hybi-04. I didn't find out why. 2. Opcode 0x1 and 0x2. Is it only a hint for how to interpret the content of the frame while doesn't affect the content body at all just as MIME in HTTP? Might do more test later. Thanks Andy On Wed, Aug 3, 2011 at 1:20 AM, Roberto Ostinelli wrote: > hi andy, > > many code portions of this project of yours have been integrated into > misultin 0.8.1-dev. i've added behaviours to support various websocket > implementations. > > please feel free to check it out and comment. > > https://github.com/ostinelli/misultin/tree/dev > > r. > > > 2011/7/29 Andy W. Song > >> Hi, >> >> Not to reinvent the wheel, just as an exercise for learning Erlang. >> >> Here you go. Copied some ideas >> from this project as well >> as misultin and cowboy. >> >> It is not complete right now. Doesn't support segment yet, which I hope is >> not used very often. And it should be fairly easy to replace the current >> implementation in misultin. >> >> Enjoy. >> >> Andy >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Wed Aug 3 08:32:19 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Wed, 3 Aug 2011 09:32:19 +0300 Subject: [erlang-questions] Erlang de/serializer and data type validator In-Reply-To: References: <20110802082752.GA21594@jakstys.lt> <61DEDBFF-8B0A-48A4-A101-1E6C59568600@erlang-solutions.com> Message-ID: <20110803063219.GC22458@jakstys.lt> On Tue, Aug 02, 2011 at 09:10:49PM +0100, Tim Watson wrote: > http://piqi.org/ might also be worth a look. Looks good. Actually, if I understand correctly, it's exactly what I`m looking for. A thing that allows me to translate XML/JSON documents to native Erlang data types according to schema and serialize back. I cannot force our clients to use piqi, we either have to use JSON or XML. > 2011/8/2 Ulf Wiger : > > > > Hi Motiejus, > > Perhaps you should look into Joe Norton's work with UBF? > > UBF is a type system for describing contracts between services. > > http://www.sics.se/~joe/ubf/site/home.html?(for original documentation) > > https://github.com/norton/ubf?(for the current stuff) > > There is a UBF-to-JSON converter. There is also one for Thrift, and Joe > > Norton has combined UBF, UBF-JSON and QuickCheck to do validation of web > > services. > > I don't think UBF to XML would be particularly difficult. > > There may be some more work that needs to be done before it fits your > > problem. OTOH, you might get a lot of other benefits going this route. > > BR, > > Ulf W > > On 2 Aug 2011, at 10:27, Motiejus Jak?tys wrote: > > After reading all I could find about UBF, it looks too big (and incomplete) for me. If I worked with other UBF services (my clients), it would be just fine -- we exchange UBF data and everybody's happy. What is more, I could define a strict protocol and give message validation to UBF. However, we already have message formats and transport implemented. All I needed is UBF(b) with conversion to XML (like Ulf said, it wouldn't be difficult). But UBF in my scenario is an overkill. Motiejus From desired.mta@REDACTED Wed Aug 3 08:49:32 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Wed, 3 Aug 2011 09:49:32 +0300 Subject: [erlang-questions] Erlang de/serializer and data type validator In-Reply-To: <20110803063219.GC22458@jakstys.lt> References: <20110802082752.GA21594@jakstys.lt> <61DEDBFF-8B0A-48A4-A101-1E6C59568600@erlang-solutions.com> <20110803063219.GC22458@jakstys.lt> Message-ID: <20110803064932.GA23559@jakstys.lt> On Wed, Aug 03, 2011 at 09:32:19AM +0300, Motiejus Jak?tys wrote: > On Tue, Aug 02, 2011 at 09:10:49PM +0100, Tim Watson wrote: > > http://piqi.org/ might also be worth a look. > > Looks good. Actually, if I understand correctly, it's exactly what I`m > looking for. A thing that allows me to translate XML/JSON documents to > native Erlang data types according to schema and serialize back. > On the other hand, it is really heavy in terms of dependencies.. On debian: apt-get install ocaml camlp4-extra ocaml-findlib gcc m4 make sed Is there anything lighter? Ideally, it should be written in Erlang and C, as my required feature set is pretty short. Thanks Motiejus From watson.timothy@REDACTED Wed Aug 3 11:39:30 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 3 Aug 2011 10:39:30 +0100 Subject: [erlang-questions] Erlang de/serializer and data type validator In-Reply-To: <20110803063219.GC22458@jakstys.lt> References: <20110802082752.GA21594@jakstys.lt> <61DEDBFF-8B0A-48A4-A101-1E6C59568600@erlang-solutions.com> <20110803063219.GC22458@jakstys.lt> Message-ID: 2011/8/3 Motiejus Jak?tys : > On Tue, Aug 02, 2011 at 09:10:49PM +0100, Tim Watson wrote: >> http://piqi.org/ might also be worth a look. > > Looks good. Actually, if I understand correctly, it's exactly what I`m > looking for. A thing that allows me to translate XML/JSON documents to > native Erlang data types according to schema and serialize back. > > I cannot force our clients to use piqi, we either have to use JSON or > XML. > It doesn't force you to use piqi, but like you say it's better to hide this kind of detail from clients. From watson.timothy@REDACTED Wed Aug 3 11:44:54 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 3 Aug 2011 10:44:54 +0100 Subject: [erlang-questions] Erlang de/serializer and data type validator In-Reply-To: <20110803064932.GA23559@jakstys.lt> References: <20110802082752.GA21594@jakstys.lt> <61DEDBFF-8B0A-48A4-A101-1E6C59568600@erlang-solutions.com> <20110803063219.GC22458@jakstys.lt> <20110803064932.GA23559@jakstys.lt> Message-ID: > > Is there anything lighter? Ideally, it should be written in Erlang and > C, as my required feature set is pretty short. > I don't know - I use OCaml anyway so I didn't notice the overhead. :) I did think about doing this a while back (https://github.com/n0gg1n/lingo) but never took it anywhere. If you're working with XML, then https://github.com/willemdj/erlsom supports generating records from an XSD schema, validating XML documents against and translation between native Erlang (record) types and XML. It also seems reasonably fast in my experience. For JSON I don't know what you'd use as there isn't really a definitive schema language for JSON not a definitive transformation language to describe converting JSON to/from XML etc. There are zillions of libraries for working with JSON in Erlang but from an API point of view I like https://github.com/talentdeficit/jsx or mochijson2. I have no idea if the NIF based bindings on github are any faster/better as I haven't used them. Cheers, Tim From watson.timothy@REDACTED Wed Aug 3 11:56:02 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 3 Aug 2011 10:56:02 +0100 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: On 2 August 2011 00:15, Ngoc Dao wrote: > A package manager that supports repositories? > Really? Another one? It would be nice if the existing ones (agner, sutro, epm, cean 1.0/2.0, erlware) would come to some agreement about something (anything?) let alone have another of these thrust upon us. Having said that, if there *was* an Erlang equivalent of `cabal install` - which there really isn't BTW - then that would be really nice. But good grief it's a challenge to get right. I wish you good luck if you're going to try this. :) From watson.timothy@REDACTED Wed Aug 3 11:57:23 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 3 Aug 2011 10:57:23 +0100 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: On 2 August 2011 15:43, Shaun Kruger wrote: > I could really use some contributions for my proxy server/load balancer project. :) > > https://github.com/skruger/Surrogate > > I recently modified it to allow for writing multiple protocol handlers (not just HTTP proxy) and I'm working on the admin interface finally. ?I know it's a shameless plug, but the project really is cool. ?Though, I might just think that because I wrote it. :) > > Shaun No, it really is a pretty cool project. :) From watson.timothy@REDACTED Wed Aug 3 12:07:26 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 3 Aug 2011 11:07:26 +0100 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: On 1 August 2011 23:53, Nick S wrote: > I have just finished implementing a "professional" Erlang server, and with > time and enthusiasm in hand, I would like to start working on something > personal. > I must say, it has been really fun writing apps in OTP and see it processing > millions of messages without ever crashing...! > > Would appreciate some new ideas, that would be useful to community :) > > - XMPP, AMQP etc are already in market! > > - Any audio/video processing layer? http://code.google.com/p/erlyvideo/ has pretty much cornered this now. > > - Any telecom related...? > I have some interesting ideas here if you're up for learning megaco and have access to kit (incl. network) for testing purposes. > - Some standard protocol implementation? > There is a need for term (de)serialisation and data format translation between common formats (XML, JSON, CSV, etc) which would be useful. See this thread - http://erlang.org/pipermail/erlang-questions/2011-August/060405.html - read to the end. It would also be very nice to have a standard database connectivity API - see this thread for a conversation that's currently going on in this space: http://erlang.org/pipermail/erlang-questions/2011-July/060347.html. > - Something new for next generation...? (Buzz words... cloud computing > etc..) There is already CloudI on github. If you fancy implementing http://dataconstellation.com/ActiveFacts/index.shtml in Erlang then I'll join in, especially if you decide to extend the mapping capabilities to non-relational data storage engines. :) I'm not sure it's really a buzz-word, but the community is pretty desperate to get better support for profiling and diagnostic tooling. See this thread - http://erlang.org/pipermail/erlang-questions/2011-July/060241.html - there are *amazing* tools in OTP already, but support for working with clustered/grid applications and chopping up the stats would be really useful. There are projects starting to do this (my own, which I've stopped working on for the time being and a couple of others) but nobody seems to have *nailed it* yet. From andrea.rosa@REDACTED Wed Aug 3 12:37:10 2011 From: andrea.rosa@REDACTED (Rosa, Andrea) Date: Wed, 3 Aug 2011 11:37:10 +0100 Subject: [erlang-questions] New_ssl module and CRL support In-Reply-To: <1312209624.25314.53.camel@vicbristol6.hpl.hp.com> References: <1312209624.25314.53.camel@vicbristol6.hpl.hp.com> Message-ID: <8097A4E725D78E41A8B862ACE60E963945EF692CEC@GVW1355EXB.americas.hpqcorp.net> Hi Any news about that? Someone is so kind to give information about future plan for supporting CRL? Someone is facing with the same issue and can give me information about some workaround? Best regards -- Andrea -----Original Message----- Sent: 01 August 2011 15:40 To: erlang-questions@REDACTED Subject: [erlang-questions] New_ssl module and CRL support Hi I am working with a software (rabbitmq) which use the new_ssl module to implement SSL connections, everything goes well with the exception of managing revoked certification during the handshake, in this case the server accepts a revoked certificate. Is there a way for fixing that? In your plans is there the implementation of a new module for supporting CRL? Cheers -- Andrea _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From ivan@REDACTED Wed Aug 3 12:43:26 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 03 Aug 2011 11:43:26 +0100 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: <4E39264E.1000003@llaisdy.com> On 03/08/2011 11:07, Tim Watson wrote: > On 1 August 2011 23:53, Nick S wrote: >> - Any audio/video processing layer? > > http://code.google.com/p/erlyvideo/ has pretty much cornered this now. n.b.: now seems to be at https://github.com/erlyvideo Ivan -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From als@REDACTED Wed Aug 3 13:05:26 2011 From: als@REDACTED (Anthony Shipman) Date: Wed, 3 Aug 2011 21:05:26 +1000 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> Message-ID: <201108032105.26594.als@iinet.net.au> On Wed, 3 Aug 2011 10:52:55 am Sam Bobroff wrote: > Quite often when debugging I want to add a new line or two to a function, > to examine or check an intermediate value. e.g.: > > ?... > ?Result = function_call(), > ?validate(Result), I find a show function useful show(Name, Value) -> io:format("~s ~p\n", [Name, Value]), Value. noshow(_, V) -> V. validate(show("result", function_call())). -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From jodie.burch@REDACTED Wed Aug 3 13:32:56 2011 From: jodie.burch@REDACTED (Jodie Burch) Date: Wed, 03 Aug 2011 12:32:56 +0100 Subject: [erlang-questions] Erlang User Conference 2011 Message-ID: Hi Don?t forget to submit your talk for the Erlang User Conference 2011, Stockholm! The deadline is 31st August! University: 31st October ? 2nd November Conference: 3rd November Tutorials: 4th November To Submit your talk or tutorial proposal go to: http://www.erlang-factory.com/conference/ErlangUserConference2011 Registration will be opening on the 15th August with an Early Bird Rate. We look forward to seeing you there! Thanks Jodie -------------- next part -------------- An HTML attachment was scrubbed... URL: From fred.hebert@REDACTED Wed Aug 3 14:21:02 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Wed, 3 Aug 2011 08:21:02 -0400 Subject: [erlang-questions] Incorrect supervisor behaviour and crash when mixing temporary and permanent children In-Reply-To: References: Message-ID: On 2011-08-02, at 21:13 PM, Sam Bobroff wrote: > Hello Erlangers, > > I've recently tracked down a bug in some code that seems to be caused by a problem with the supervisor module. What seems to be happening is that when a temporary child has been added to a one_for_all supervisor using start_child, if a permanent child of that supervisor exits then if that permanent child exits, the supervisor attempts to restart the temporary child (which seems counter to the documentation). That one is fine. The 'one_for_all' is about each process supervised. In that case, the permanent child means 'when I crash, get the one_for_all strategy'. That strategy is applied no matter what the other kinds of children are. On the other hand, in that same supervisor, if the temporary child dies, then the permanent one won't be restarted because that restart strategy means 'in my case, I'm not that vital and you can ignore me dying'. As far as I know, this is expected behaviour. > In addition the MFA for the child has been replaced with undefined and start attempt crashes, causing the supervisor to continue attempting restarts until it reaches it's restart intensity and shuts down. > > This does not happen if the temporary child is added to the supervisor as part of the supervisor's initial spec. In this case the temporary child is restart but it's MFA is present so it is able to restart it successfully, although it still seems contrary to the documentation that the temporary child is restarted at all, although it depends on which section of the documentation you regard as more important (from the supervisor documentation): Ah, that one gets trickier. If I take a look at the source, the issue seems to be in the supervisor's 'save_child' function. This one has this comment: %% Note we do not want to save the parameter list for temporary processes as %% they will not be restarted, and hence we do not need this information. %% Especially for dynamic children to simple_one_for_one supervisors %% it could become very costly as it is not uncommon to spawn %% very many such processes. So the arguments that you pass to 'proc_lib:start_link' get dropped. This check DOES NOT happen when the Child Spec is part of the initial supervisor specification. One issue here is that we have two different ways of starting temporary children. The real issue however, seems to be that we assume that temporary processes will NOT be restarted, while the supervisor does attempt to do so. > > > one_for_all - if one child process terminates and should be restarted, all other child processes are terminated and then all child processes are restarted. > > This is quite clear: "all child processes are restarted". But later: > > > Restart defines when a terminated child process should be restarted. A permanent child process should always be restarted, a temporary child process should never be restarted and a transient child process should be restarted only if it terminates abnormally, i.e. with another exit reason than normal. > > > > Again clear but conflicting: "a temporary child process should never be restarted". These rules are about the processes themselves as far as I know. It's well possible to expect a transient process to be restarted if it depends on a permanent one that crashed. I'm not sure I get why this would not be the case with a temporary child. > > I found a similar issue mentioned in a bug report for R14B02, here: > > http://erlang.org/pipermail/erlang-bugs/2011-March/002273.html This is an interesting entry because it does suggest a fix that's still there and helps with the issue, but only when the process that crashed was the one we were looking for. The function now looks like this: del_child(Name, [Ch|Chs]) when Ch#child.name =:= Name, Ch#child.restart_type =:= temporary -> Chs; del_child(Name, [Ch|Chs]) when Ch#child.name =:= Name -> [Ch#child{pid = undefined} | Chs]; del_child(Pid, [Ch|Chs]) when Ch#child.pid =:= Pid, Ch#child.restart_type =:= temporary -> Chs; del_child(Pid, [Ch|Chs]) when Ch#child.pid =:= Pid -> [Ch#child{pid = undefined} | Chs]; del_child(Name, [Ch|Chs]) -> [Ch|del_child(Name, Chs)]; del_child(_, []) -> []. The issue there is that we're still giving temporary children in the list of processes to be restarted if they were not the ones to crash (Name/Pid). Then the sequent calls to 'terminate_children' and 'do_terminate_children' as part of the restart procedure also never clear these up, and the 'start_children' function doesn't do it either. In normal cases, it seems the supervisor will use state_del_child as a function to do it, which doesn't happen here (we're using del_child). --- If anyone from the OTP team is looking, it'd be nice to have a confirm/deny on that explanation. I've tried a small patch, but haven't tested it in the large or recompiled anything: 737a738,740 > terminate_children([Child = #child{restart_type=temporary} | Children], SupName, Res) -> > do_terminate(Child, SupName), > terminate_children(Children, SupName, Res); Just adding these lines to R14B03 seems to resolve the issue by avoiding to restart the child in the first place. I can make a standard patch and bug submission if required/appreciated. -- Fred H?bert http://www.erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From pguyot@REDACTED Wed Aug 3 14:31:53 2011 From: pguyot@REDACTED (Paul Guyot) Date: Wed, 3 Aug 2011 14:31:53 +0200 Subject: [erlang-questions] New_ssl module and CRL support In-Reply-To: References: Message-ID: <0D5BAAE2-5CAA-4110-AB6C-D10C1FDDFFBE@kallisys.net> > Message: 2 > Date: Wed, 3 Aug 2011 11:37:10 +0100 > From: "Rosa, Andrea" > To: "erlang-questions@REDACTED" > Subject: Re: [erlang-questions] New_ssl module and CRL support > Message-ID: > <8097A4E725D78E41A8B862ACE60E963945EF692CEC@REDACTED> > > Content-Type: text/plain; charset="us-ascii" > > Hi > Any news about that? Someone is so kind to give information about future plan for supporting CRL? > Someone is facing with the same issue and can give me information about some workaround? > Best regards As a workaround, you can use the verify_fun option. Such a function would check that the peer or the CA certificates do not belong to a given list of (revoked) certificates. Paul -- Semiocast http://semiocast.com/ +33.183627948 - 20 rue Lacaze, 75014 Paris From fred.hebert@REDACTED Wed Aug 3 15:02:57 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Wed, 3 Aug 2011 09:02:57 -0400 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> <5451A797-C158-43B5-B37F-F50583 456F72@cs.otago.ac.nz> Message-ID: <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> On 2011-08-02, at 01:59 AM, Richard O'Keefe wrote: > 2. Is matching against an already-bound variable a check we want? To me, matching against an already-bound variable is is a valid assertion, as much as 'ok = function_call()' might be, and as implicit as '{ok, Pid}' or '{error, already_started}' might be. Matching on already-bound variables is part of my standard code when trying to crash early when possible, and also part of many basic test suites that simply do pattern matching here and there. To me it's as basic as using the same variable twice in a single pattern (f([A,A,B,B|_]) when A =/= B -> ...), or something similar with records. I would likely not use the check at all, and if it were to be added, would prefer it to be a compiler argument (which could be enabled in -compile(...).) I foresee little use of such warnings for myself and would dislike to see it becoming a default setting. -- Fred H?bert http://www.erlang-solutions.com From mihai@REDACTED Wed Aug 3 15:16:04 2011 From: mihai@REDACTED (Mihai Balea) Date: Wed, 3 Aug 2011 09:16:04 -0400 Subject: [erlang-questions] Gproc in a dynamic cluster In-Reply-To: References: <3E83ACB7-9B81-4812-AF4A-F153D2CFA3F8@hates.ms> <6CD0A133-6F6D-408C-ACA3-B5237C6A67CC@erlang-solutions.com> <36ADA8AF-80F2-4BA1-A655-AD1F6AEE56DA@hates.ms> <8A4A4005-CD8E-4CA5-A3C0-E1436C992CD2@erlang-solutions.com> <52519FFE-29AA-4866-8F7D-A494EFCCBC14@hates.ms> <668C7836-B388-4BCE-9170-011CADB451C3@erlang-solutions.com> <09E08DBF-B4EE-4B34-9F1E-14EC7579C636@hates.ms> Message-ID: <2420B22E-4A07-44FA-8339-1661327BCD37@hates.ms> On Aug 2, 2011, at 3:11 PM, Ulf Wiger wrote: > > On 2 Aug 2011, at 17:00, Mihai Balea wrote: > >> Still, generally speaking, what would it take to add netsplit handling to gproc? It would be nice to be able to use some of gproc advanced features. Besides, I believe the initial intention for gproc was to be a replacement for global, and it cannot fill that role until it handles netsplits. > > The first step is to make gen_leader detect netsplits. I suspect netsplit detection is not the main issue here, but rather what happens after the split is resolved and the dictionaries must be reconciled. Mihai From erlang@REDACTED Wed Aug 3 15:37:50 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 3 Aug 2011 15:37:50 +0200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> Message-ID: Actually I like things as they are. My eye is trained to see bound and free occurrences of a variable. On the other hand it would be *excellent* if the color coding in emacs (whatever) was changed to reflect the binding status of a variable. If variables in the LHS of an equals were colored green if they were unbound and red if bound then one would see at a glance what was intended. (( assuming you're not red-green color blind that is )) - or set the variables in a different type face. /Joe 2011/8/3 Fr?d?ric Trottier-H?bert : > On 2011-08-02, at 01:59 AM, Richard O'Keefe wrote: > >> 2. Is matching against an already-bound variable a check we want? > > To me, matching against an already-bound variable is is a valid assertion, as much as 'ok = function_call()' might be, and as implicit as '{ok, Pid}' or '{error, already_started}' might be. Matching on already-bound variables is part of my standard code when trying to crash early when possible, and also part of many basic test suites that simply do pattern matching here and there. To me it's as basic as using the same variable twice in a single pattern (f([A,A,B,B|_]) when A =/= B -> ...), or something similar with records. > > I would likely not use the check at all, and if it were to be added, would prefer it to be a compiler argument (which could be enabled in -compile(...).) I foresee little use of such warnings for myself and would dislike to see it becoming a default setting. > > -- > Fred H?bert > http://www.erlang-solutions.com > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From bighostkim@REDACTED Wed Aug 3 15:48:02 2011 From: bighostkim@REDACTED (Allen Kim) Date: Wed, 3 Aug 2011 09:48:02 -0400 Subject: [erlang-questions] relup restart_new_emulator does not work for me Message-ID: Hi, I want to upgrade erlang otp from R13B04 to R43B03 I used this start_erl.data and relup(recommended from [tracking@REDACTED releases]$ cat new_start_erl.data 5.8.4 2.0.0 [tracking@REDACTED releases]$ cat current/relup {"2.0.0", [{"1.9.0", [], [restart_new_emulator]}], [{"1.9.0", [], [restart_new_emulator]}] }. when I run command release_handler:install_release("2.0.0") I see the node shutting down messages, but I don't see a new node start up.(looks like heart is not starting up a new node). My HEART_COMMAND is coreect, since I was able to kill the node and I see it starting up again. The reason why I said restart_new_emulator not working is I could reboot the node with command init:reboot(). Any help would be appreciated. Allen Kim -------------- next part -------------- An HTML attachment was scrubbed... URL: From bgustavsson@REDACTED Wed Aug 3 16:08:39 2011 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 3 Aug 2011 16:08:39 +0200 Subject: [erlang-questions] compile:file In-Reply-To: References: Message-ID: On Mon, Aug 1, 2011 at 3:41 PM, Andrew Kondratovich wrote: > Hi. > Docs says, that compile:file("m.erl", ['S']) produces a listing of the > assembler code in the file m.S. > Also, docs says, that compile:file("m.S", [asm]) compiles the code > from input file, which is expected to be assembler code. > > So, these two steps identically to simple compile:file("m.erl"). Yes, except that optimizations are run twice, and that causes problems in this case. > But, why I can't to compile code from assembler, when it has > operations with floating points. For example: The floating point optimization pass introduces instructions that the earlier live optimization pass does not handle. That only becomes a problem if the optimizations are run twice. Use the no_postopt option to prevent assembly language optimizations from being run twice: compile:file("m.S", [asm,no_postopt]). Note that there are test cases (e.g. guard_SUITE:tuple_size/1) that depend on optimizations being run even when the input file is assembly code, so I cannot simply change the 'asm' options not to run the optimizations passes. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From ulf.wiger@REDACTED Wed Aug 3 16:41:26 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 3 Aug 2011 16:41:26 +0200 Subject: [erlang-questions] Gproc in a dynamic cluster In-Reply-To: <2420B22E-4A07-44FA-8339-1661327BCD37@hates.ms> References: <3E83ACB7-9B81-4812-AF4A-F153D2CFA3F8@hates.ms> <6CD0A133-6F6D-408C-ACA3-B5237C6A67CC@erlang-solutions.com> <36ADA8AF-80F2-4BA1-A655-AD1F6AEE56DA@hates.ms> <8A4A4005-CD8E-4CA5-A3C0-E1436C992CD2@erlang-solutions.com> <52519FFE-29AA-4866-8F7D-A494EFCCBC14@hates.ms> <668C7836-B388-4BCE-9170-011CADB451C3@erlang-solutions.com> <09E08DBF-B4EE-4B34-9F1E-14EC7579C636@hates.ms> <2420B22E-4A07-44FA-8339-1661327BCD37@hates.ms> Message-ID: <710AE51B-EBCD-4560-AE06-267E8B428EB8@erlang-solutions.com> On 3 Aug 2011, at 15:16, Mihai Balea wrote: > > On Aug 2, 2011, at 3:11 PM, Ulf Wiger wrote: > >> >> On 2 Aug 2011, at 17:00, Mihai Balea wrote: >> >>> Still, generally speaking, what would it take to add netsplit handling to gproc? It would be nice to be able to use some of gproc advanced features. Besides, I believe the initial intention for gproc was to be a replacement for global, and it cannot fill that role until it handles netsplits. >> >> The first step is to make gen_leader detect netsplits. > > I suspect netsplit detection is not the main issue here, but rather what happens after the split is resolved and the dictionaries must be reconciled. Well, sure, but if gen_leader (as is currently the case) doesn't even detect netsplits and does something sane at the leader election layer, there really isn't much gproc can do (at least not much that could be called "elegant"). But naturally, once gen_leader has detected a netsplit and ordered all candidates into some safe mode, it needs to give the callback module a chance to reconcile the state. The approach would be similar to my old 'locker' contribution (http://forum.trapexit.org/viewtopic.php?p=30186#30186), but perhaps gproc could do with some vector clocks as well? I usually borrow that stuff from the riak codebase. :) BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasl_erlang@REDACTED Wed Aug 3 17:05:57 2011 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 3 Aug 2011 08:05:57 -0700 (PDT) Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> Message-ID: <1312383957.26732.YahooMailNeo@web111412.mail.gq1.yahoo.com> >From: Joe Armstrong ... >Actually I like things as they are. My eye is trained to see bound and >free occurrences of a variable. > >On the other hand it would be *excellent* if the color coding in emacs >(whatever) was changed to reflect the >binding status of a variable. > >If variables in the LHS of an equals were colored green if they were >unbound and red >if bound then one would see at a glance what was intended. > >(( assuming you're not red-green color blind that is )) - or set the >variables in a different type face. This raises the question of whether the order of matching variables is well-defined? I'm thinking of things like (order inside pattern): f(A) -> {X, X} = A, X. or (order of argument matching): same([X|Xs], [X|Ys]) -> same(Xs, Ys); same([], []) -> true; same(_, _) -> false. I seem to recall that Mike Williams wanted a fully determined evaluation order long ago, for the Erlang spec. Was this implemented? And did it extend to pattern matching? Best, Thomas From carlsson.richard@REDACTED Wed Aug 3 18:17:39 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 03 Aug 2011 18:17:39 +0200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <1312383957.26732.YahooMailNeo@web111412.mail.gq1.yahoo.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> <1312383957.26732.YahooMailNeo@web111412.mail.gq1.yahoo.com> Message-ID: <4E3974A3.6070104@gmail.com> On 08/03/2011 05:05 PM, Thomas Lindgren wrote: > I seem to recall that Mike Williams wanted a fully determined > evaluation order long ago, for the Erlang spec. Was this implemented? Yes. Arguments to functions and constructors are always evaluated left-to-right. This linearization is done in the transformation to Core Erlang, which doesn't in itself define an evaluation order for function arguments - the order from the Erlang source code is expressed on the Core level using let-bindings. (Run "erlc +to_core foo.erl" and look at the resulting file foo.core to see the explicit evaluation order.) But I don't know if this rule has been formally documented anywhere after the work on the formal Erlang specification got indefinitely suspended. For reference, here's an old exchange between me, Robert, and Ulf: http://erlang.org/pipermail/erlang-questions/2008-October/039170.html (I also recall that Barklund had originally suggested that arguments of list constructors should be evaluated right-to-left as an exception, probably in order to reduce the number of temporaries needed while constructing a list like [f(), g(), h()], but this was voted down since it didn't make much sense to a human being reading the code.) > And did it extend to pattern matching? No, there was never any discussion about that. If the match succeeds then it would succeed no matter what the order, and if a you get a 'badmatch' error, the entire matched term is blamed, not just the first subterm that doesn't match, so I can't see that there would be any observable difference depending on the order of matched subterms. /Richard From overminddl1@REDACTED Wed Aug 3 18:20:59 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Wed, 3 Aug 2011 10:20:59 -0600 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: <9158FB12-40A7-4276-BD18-C7CACF7690B7@worrell.nl> Message-ID: I am curious, how is ChicagoBoss compared to Nitrogen? On Aug 2, 2011 11:31 PM, "Icarus Alive" wrote: > On Wed, Aug 3, 2011 at 3:54 AM, Benson Wong wrote: >> Hi, >> My background is a web developer (~15 years), done PHP for about 11 years. >> So I know it pretty well. >> Just starting to learn Erlang. Done lots of javascript (jQuery, node.js, >> etc). So a PHP developer, in the Erlang pool here :) >> I think an imperative language like PHP would be a lot easier (and more >> agile) to build the web app in than Erlang. I would choose erlang if I need >> long lived connections (web sockets, long-poll, etc). Frameworks like Django >> (Python)/Rails (ruby) would also make life easier. >> If you want scalability/reliability/performance, add more PHP processes / >> servers. >> PHP is shared nothing, so it is easy to scale. Just add more. Using Nginx + >> PHP-FPM. Solves a lot of slow request issues. >> Put them behind an HTTP load balancer, HAProxy is a very popular choice. >> Performance is usually database bound than it is app server bound. >> On Fri, Jul 29, 2011 at 6:16 AM, Icarus Alive >> wrote: >>> >>> Would appreciate if my thread wasn't hijacked :-)... although gotta >>> agree that our scenarios are quite similar. >>> >>> On Fri, Jul 29, 2011 at 6:31 PM, Banibrata Dutta >>> wrote: >>> > Thanks @Marc, for taking time to reply this this mail. >>> > On Fri, Jul 29, 2011 at 5:38 PM, Marc Worrell wrote: >>> >> >>> >> It mostly depends on what kind of web development you want to do. >>> >> >>> >> Is it more of a mobile application than a publishing web site? >>> >>> In my case, content characteristics are: >>> 1. Always originating from single source >>> 2. Always consumed by single user >>> 3. Content has short lifetime - once consumed, will be archived >>> 4. Each user has several hundreds of content display to her in "newest >>> first" fashion >>> >>> >> >>> >> Does is have specific authentication schemes? >>> >> Etc. Etc. >>> > >>> > Honestly, haven't given this enough thought. One of the requirements is >>> > to >>> > ensure privacy of user data, and that of the interaction. I'm yet to >>> > figure >>> > out the "how" part. >> >> Security/Privacy, this is more of an operation and application design than >> it is a Erlang/OTP vs PHP/Ruby/Python+framework. >> Generally you'd want to: >> * firewalls, no SSH access, web servers are only web servers, ie: only port >> 443 open. >> * use cookie based sessions. Don't use HTTP basic authentication. >> * Server has controls all access based on the cookie. >> * Use HTTPS. Always. >> * There are lots of other techniques: >> -- use bcrypt for passwords >> -- sessions can be limited to one IP address at a time >> -- sessions have short expire times (controlled on the server) >> -- etc. >> >>> >>> Well in my case it has to be some strong authentication. Storage, >>> archival and access has to conform to HIPAA compliance. So we know we >>> are dealing with EMR type content. > > Hi Benson, > > Really appreciate your descriptive & informative response, especially > so for the part on security. > > My inclination towards Erlang stems from the fact that I was intending > to do much of the backend in Erlang as well. The backend does lot of > other things than just bridging between FE client request/responses > and a DB, s.a. converting content format, managing archival policies, > do some data analysis for reporting etc. Maybe some of that via ports. > > I wish I'd seen your mail few days back, because having tried both > Nitrogen and Zotonic, which are I am sure excellent products, I think > I've found peace in settling for ChicagoBoss. I found it lot easier to > work with, as a beginner, maybe because the FP guts are almost hidden > away from sight, and much of the developer/user coding is done in an > imperative fashion. Also, I found it's documentation (although quite > light) to be simple & just-enough to get me started and get me going. > > However, I might have to come back to PHP unless I am able to train my > team on Erlang. Being a beginner, I find it hard to imagine being a > good teacher / coach for rest of the team. > > Thanks again, everyone, for responding on this thread. > > Icarus. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From overminddl1@REDACTED Wed Aug 3 18:38:41 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Wed, 3 Aug 2011 10:38:41 -0600 Subject: [erlang-questions] Incorrect supervisor behaviour and crash when mixing temporary and permanent children In-Reply-To: References: Message-ID: >From my understanding, permanent and temporary processes should be under different supervisors, generally the temp supervisor under the perm supervisor. On Aug 3, 2011 6:21 AM, "Fr?d?ric Trottier-H?bert" < fred.hebert@REDACTED> wrote: > > > On 2011-08-02, at 21:13 PM, Sam Bobroff wrote: > >> Hello Erlangers, >> >> I've recently tracked down a bug in some code that seems to be caused by a problem with the supervisor module. What seems to be happening is that when a temporary child has been added to a one_for_all supervisor using start_child, if a permanent child of that supervisor exits then if that permanent child exits, the supervisor attempts to restart the temporary child (which seems counter to the documentation). > > That one is fine. The 'one_for_all' is about each process supervised. In that case, the permanent child means 'when I crash, get the one_for_all strategy'. That strategy is applied no matter what the other kinds of children are. On the other hand, in that same supervisor, if the temporary child dies, then the permanent one won't be restarted because that restart strategy means 'in my case, I'm not that vital and you can ignore me dying'. As far as I know, this is expected behaviour. > > >> In addition the MFA for the child has been replaced with undefined and start attempt crashes, causing the supervisor to continue attempting restarts until it reaches it's restart intensity and shuts down. >> >> This does not happen if the temporary child is added to the supervisor as part of the supervisor's initial spec. In this case the temporary child is restart but it's MFA is present so it is able to restart it successfully, although it still seems contrary to the documentation that the temporary child is restarted at all, although it depends on which section of the documentation you regard as more important (from the supervisor documentation): > > Ah, that one gets trickier. If I take a look at the source, the issue seems to be in the supervisor's 'save_child' function. This one has this comment: > > %% Note we do not want to save the parameter list for temporary processes as > %% they will not be restarted, and hence we do not need this information. > %% Especially for dynamic children to simple_one_for_one supervisors > %% it could become very costly as it is not uncommon to spawn > %% very many such processes. > > So the arguments that you pass to 'proc_lib:start_link' get dropped. This check DOES NOT happen when the Child Spec is part of the initial supervisor specification. > > One issue here is that we have two different ways of starting temporary children. The real issue however, seems to be that we assume that temporary processes will NOT be restarted, while the supervisor does attempt to do so. > >> >> >> one_for_all - if one child process terminates and should be restarted, all other child processes are terminated and then all child processes are restarted. >> >> This is quite clear: "all child processes are restarted". But later: >> >> >> Restart defines when a terminated child process should be restarted. A permanent child process should always be restarted, a temporary child process should never be restarted and a transient child process should be restarted only if it terminates abnormally, i.e. with another exit reason than normal. >> >> >> >> Again clear but conflicting: "a temporary child process should never be restarted". > > These rules are about the processes themselves as far as I know. It's well possible to expect a transient process to be restarted if it depends on a permanent one that crashed. I'm not sure I get why this would not be the case with a temporary child. >> >> I found a similar issue mentioned in a bug report for R14B02, here: >> >> http://erlang.org/pipermail/erlang-bugs/2011-March/002273.html > > This is an interesting entry because it does suggest a fix that's still there and helps with the issue, but only when the process that crashed was the one we were looking for. The function now looks like this: > > del_child(Name, [Ch|Chs]) when Ch#child.name =:= Name, Ch#child.restart_type =:= temporary -> > Chs; > del_child(Name, [Ch|Chs]) when Ch#child.name =:= Name -> > [Ch#child{pid = undefined} | Chs]; > del_child(Pid, [Ch|Chs]) when Ch#child.pid =:= Pid, Ch#child.restart_type =:= temporary -> > Chs; > del_child(Pid, [Ch|Chs]) when Ch#child.pid =:= Pid -> > [Ch#child{pid = undefined} | Chs]; > del_child(Name, [Ch|Chs]) -> > [Ch|del_child(Name, Chs)]; > del_child(_, []) -> > []. > > The issue there is that we're still giving temporary children in the list of processes to be restarted if they were not the ones to crash (Name/Pid). Then the sequent calls to 'terminate_children' and 'do_terminate_children' as part of the restart procedure also never clear these up, and the 'start_children' function doesn't do it either. > > In normal cases, it seems the supervisor will use state_del_child as a function to do it, which doesn't happen here (we're using del_child). > --- > > If anyone from the OTP team is looking, it'd be nice to have a confirm/deny on that explanation. I've tried a small patch, but haven't tested it in the large or recompiled anything: > > 737a738,740 >> terminate_children([Child = #child{restart_type=temporary} | Children], SupName, Res) -> >> do_terminate(Child, SupName), >> terminate_children(Children, SupName, Res); > > Just adding these lines to R14B03 seems to resolve the issue by avoiding to restart the child in the first place. > I can make a standard patch and bug submission if required/appreciated. > > -- > Fred H?bert > http://www.erlang-solutions.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fred.hebert@REDACTED Wed Aug 3 18:42:25 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Wed, 3 Aug 2011 12:42:25 -0400 Subject: [erlang-questions] Incorrect supervisor behaviour and crash when mixing temporary and permanent children In-Reply-To: References: Message-ID: <93FA1F29-A7A1-48E3-9608-51FFB6409582@erlang-solutions.com> This depends on the relationship they have. There's no one-size-fits-all approach there, but I would say that it is likely that you want short-lived permanent task usually higher up in the tree hierarchy than short-lived tasks. However, there is nothing keeping you from putting them under the same spot for some reason. In any case, temporary processes should never be restarted (especially given the supervisor module doesn't track the Args anymore). I've submitted a patch for the behaviour here: http://erlang.org/pipermail/erlang-patches/2011-August/002248.html -- Fred H?bert http://www.erlang-solutions.com On 2011-08-03, at 12:38 PM, OvermindDL1 wrote: > From my understanding, permanent and temporary processes should be under different supervisors, generally the temp supervisor under the perm supervisor. > > On Aug 3, 2011 6:21 AM, "Fr?d?ric Trottier-H?bert" wrote: > > > > > > On 2011-08-02, at 21:13 PM, Sam Bobroff wrote: > > > >> Hello Erlangers, > >> > >> I've recently tracked down a bug in some code that seems to be caused by a problem with the supervisor module. What seems to be happening is that when a temporary child has been added to a one_for_all supervisor using start_child, if a permanent child of that supervisor exits then if that permanent child exits, the supervisor attempts to restart the temporary child (which seems counter to the documentation). > > > > That one is fine. The 'one_for_all' is about each process supervised. In that case, the permanent child means 'when I crash, get the one_for_all strategy'. That strategy is applied no matter what the other kinds of children are. On the other hand, in that same supervisor, if the temporary child dies, then the permanent one won't be restarted because that restart strategy means 'in my case, I'm not that vital and you can ignore me dying'. As far as I know, this is expected behaviour. > > > > > >> In addition the MFA for the child has been replaced with undefined and start attempt crashes, causing the supervisor to continue attempting restarts until it reaches it's restart intensity and shuts down. > >> > >> This does not happen if the temporary child is added to the supervisor as part of the supervisor's initial spec. In this case the temporary child is restart but it's MFA is present so it is able to restart it successfully, although it still seems contrary to the documentation that the temporary child is restarted at all, although it depends on which section of the documentation you regard as more important (from the supervisor documentation): > > > > Ah, that one gets trickier. If I take a look at the source, the issue seems to be in the supervisor's 'save_child' function. This one has this comment: > > > > %% Note we do not want to save the parameter list for temporary processes as > > %% they will not be restarted, and hence we do not need this information. > > %% Especially for dynamic children to simple_one_for_one supervisors > > %% it could become very costly as it is not uncommon to spawn > > %% very many such processes. > > > > So the arguments that you pass to 'proc_lib:start_link' get dropped. This check DOES NOT happen when the Child Spec is part of the initial supervisor specification. > > > > One issue here is that we have two different ways of starting temporary children. The real issue however, seems to be that we assume that temporary processes will NOT be restarted, while the supervisor does attempt to do so. > > > >> > >> > >> one_for_all - if one child process terminates and should be restarted, all other child processes are terminated and then all child processes are restarted. > >> > >> This is quite clear: "all child processes are restarted". But later: > >> > >> > >> Restart defines when a terminated child process should be restarted. A permanent child process should always be restarted, a temporary child process should never be restarted and a transient child process should be restarted only if it terminates abnormally, i.e. with another exit reason than normal. > >> > >> > >> > >> Again clear but conflicting: "a temporary child process should never be restarted". > > > > These rules are about the processes themselves as far as I know. It's well possible to expect a transient process to be restarted if it depends on a permanent one that crashed. I'm not sure I get why this would not be the case with a temporary child. > >> > >> I found a similar issue mentioned in a bug report for R14B02, here: > >> > >> http://erlang.org/pipermail/erlang-bugs/2011-March/002273.html > > > > This is an interesting entry because it does suggest a fix that's still there and helps with the issue, but only when the process that crashed was the one we were looking for. The function now looks like this: > > > > del_child(Name, [Ch|Chs]) when Ch#child.name =:= Name, Ch#child.restart_type =:= temporary -> > > Chs; > > del_child(Name, [Ch|Chs]) when Ch#child.name =:= Name -> > > [Ch#child{pid = undefined} | Chs]; > > del_child(Pid, [Ch|Chs]) when Ch#child.pid =:= Pid, Ch#child.restart_type =:= temporary -> > > Chs; > > del_child(Pid, [Ch|Chs]) when Ch#child.pid =:= Pid -> > > [Ch#child{pid = undefined} | Chs]; > > del_child(Name, [Ch|Chs]) -> > > [Ch|del_child(Name, Chs)]; > > del_child(_, []) -> > > []. > > > > The issue there is that we're still giving temporary children in the list of processes to be restarted if they were not the ones to crash (Name/Pid). Then the sequent calls to 'terminate_children' and 'do_terminate_children' as part of the restart procedure also never clear these up, and the 'start_children' function doesn't do it either. > > > > In normal cases, it seems the supervisor will use state_del_child as a function to do it, which doesn't happen here (we're using del_child). > > --- > > > > If anyone from the OTP team is looking, it'd be nice to have a confirm/deny on that explanation. I've tried a small patch, but haven't tested it in the large or recompiled anything: > > > > 737a738,740 > >> terminate_children([Child = #child{restart_type=temporary} | Children], SupName, Res) -> > >> do_terminate(Child, SupName), > >> terminate_children(Children, SupName, Res); > > > > Just adding these lines to R14B03 seems to resolve the issue by avoiding to restart the child in the first place. > > I can make a standard patch and bug submission if required/appreciated. > > > > -- > > Fred H?bert > > http://www.erlang-solutions.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Wed Aug 3 19:05:26 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 3 Aug 2011 19:05:26 +0200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <4E3974A3.6070104@gmail.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> <1312383957.26732.YahooMailNeo@web111412.mail.gq1.yahoo.com> <4E3974A3.6070104@gmail .com> Message-ID: According to the Barklund spec (page 62), it said that "a compiler must only accept a program if for any evaluation order, there will not be an applied occurrence of an unbound variable". "For example, in a context where X is unbound, the expression (X=8) + X should give a compile-time error." (And it does, too. I checked). BR, Ulf W On 3 Aug 2011, at 18:17, Richard Carlsson wrote: >> And did it extend to pattern matching? > > No, there was never any discussion about that. If the match succeeds then it would succeed no matter what the order, and if a you get a 'badmatch' error, the entire matched term is blamed, not just the first subterm that doesn't match, so I can't see that there would be any observable difference depending on the order of matched subterms. Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From icarus.alive@REDACTED Wed Aug 3 20:00:35 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Wed, 3 Aug 2011 23:30:35 +0530 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: <9158FB12-40A7-4276-BD18-C7CACF7690B7@worrell.nl> Message-ID: Sure Benson, will do so, although I've got a really long way to go. On Wed, Aug 3, 2011 at 9:17 PM, Benson Wong wrote: > Hi Icarus, > I would appreciate it if you would share your experience with this when you > get deeper into Erlang, Chicago Boss web development. I'm just learning the > other side of the coin so highly interested in how it is. In particular, how > to structure a larger erlang system that coordinates front end / backend > jobs as you've mentioned. > Just for you benefit, with the traditional approach (PHP, Ruby, etc), > background jobs are usually done with tools like: > - Gearman > - beanstalkd (my preference) > - cron (archiving / periodic) > Sometimes I like to hack together my own simple event loop daemon using: > - PHP > - daemontools > - beanstalkd > The web server PHP jams a job into beanstalkd, the daemonized PHP pops the > job off the stack, does it, asks for another one. You can scale this out > over several machines quite easily. > Ben. > > On Tue, Aug 2, 2011 at 10:31 PM, Icarus Alive > wrote: >> >> Hi Benson, >> >> Really appreciate your descriptive & informative response, especially >> so for the part on security. >> >> My inclination towards Erlang stems from the fact that I was intending >> to do much of the backend in Erlang as well. The backend does lot of >> other things than just bridging between FE client request/responses >> and a DB, s.a. converting content format, managing archival policies, >> do some data analysis for reporting etc. Maybe some of that via ports. >> >> I wish I'd seen your mail few days back, because having tried both >> Nitrogen and Zotonic, which are I am sure excellent products, I think >> I've found peace in settling for ChicagoBoss. I found it lot easier to >> work with, as a beginner, maybe because the FP guts are almost hidden >> away from sight, and much of the developer/user coding is done in an >> imperative fashion. Also, I found it's documentation (although quite >> light) to be simple & just-enough to get me started and get me going. >> >> However, I might have to come back to PHP unless I am able to train my >> team on Erlang. Being a beginner, I find it hard to imagine being a >> good teacher / coach for rest of the team. >> >> Thanks again, everyone, for responding on this thread. >> >> Icarus. > > From james@REDACTED Wed Aug 3 20:35:31 2011 From: james@REDACTED (James Aimonetti) Date: Wed, 3 Aug 2011 11:35:31 -0700 Subject: [erlang-questions] Hidden Memory Hog In-Reply-To: <723FDC86-0FC6-48EE-B012-05C57F9ECF25@hates.ms> References: <4E35E5C9.90108@2600hz.com> <4E36CC75.8020409@2600hz.com> <723FDC86-0FC6-48EE-B012-05C57F9ECF25@hates.ms> Message-ID: <4E3994F3.1070407@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 So I started a VM with the +Mim true flag and wondered if folks can help narrow down where I should be looking for possible offenders? Using this in the shell: f(), {_,Mem}= instrument:descr(instrument:memory_data()), Sorted = lists:reverse(lists:keysort(3,Mem)). [{heap,140578906112088,41084720,<0.5051.56>}, {old_heap,140578980069464,32867800,<0.5051.56>}, {old_heap,140579035463768,10770152,<0.8797.57>}, {heap,140578978496600,1571344,<0.8797.57>}, {heap,140579077124184,971144,<0.25.0>}, {old_heap,140579095326808,971144,<0.455.0>}, {old_heap,140579078697048,971144,<0.25.0>}, {old_heap,140579090284632,600200,<0.404.0>}, {timer_wheel,140579109355368,524288,undefined}, {port_tab,140579107569760,466944,undefined}, {code,140579086084496,326762,<0.25.0>}, {scheduler_data,140579110159688,287680,undefined}, {proc_tab,140579109892896,262144,undefined}, {code,140579070881888,251787,<0.25.0>}, {code,140579106199176,239377,<0.0.0>}, {old_heap,140579074121920,229256,<0.396.0>}, {code,140579082245832,193718,<0.25.0>}, {code,140579082439600,187723,<0.25.0>}, {code,140579075601296,183984,<0.25.0>}, {code,140579085875512,183888,<0.25.0>}, |...] The first four lines are the shell and a process spawned by the shell. If I run: lists:foldl(fun({_,_,M,_},Acc) -> Acc + M end, 0, Sorted). 112277446 Is that number valid? Should I be looking at lines with type heap or old_heap only? The memory output from /proc/PID/status: VmPeak: 728864 kB VmSize: 335188 kB VmLck: 0 kB VmHWM: 421760 kB VmRSS: 250352 kB VmData: 306400 kB VmStk: 136 kB VmExe: 1800 kB VmLib: 3748 kB VmPTE: 680 kB VmSwap: 1520 kB Any direction for where to continue searching? Thanks for the help so far, James On 08/01/2011 10:32 AM, Mihai Balea wrote: > That looks pretty benign. I suspect the OS reported memory usage for the beam process is vastly higher, correct? > Do you run any native code? NIFs, linked-in drivers? Maybe there's a leak in there. > > On Aug 1, 2011, at 11:55 AM, James Aimonetti wrote: > > No, no outliers. > > 1> erlang:memory(). > [{total,29393336}, > {processes,18646408}, > {processes_used,18560824}, > {system,10746928}, > {atom,658865}, > {atom_used,652857}, > {binary,1059104}, > {code,6608997}, > {ets,434848}] > > Unless I'm reading that output incorrectly? > > On 08/01/2011 01:39 AM, Attila Rajmund Nohl wrote: >>>> What does erlang:memory() show? Binaries, processes or ets uses that >>>> much memory? >>>> >>>> 2011/8/1, James Aimonetti : >>>> List, >>>> >>>> I'm at my wits end (they're short, no doubt, but still). I have a VM >>>> running with several OTP applications that we've written. We see, over >>>> the course of the day, memory consumption go up until its taking ~30% >>>> system memory and 300-500MB of swap. Iterating over all processes and >>>> checking their total_heap_size, no single process was greater than ~1 >>>> MB, with ~300 processes runnning. Forcing garbage collection on all >>>> processes released maybe 15MB of system memory. >>>> >>>> erlang:memory() lists total memory around 18-20MB; ets tables were all >>>> minimal, no dets tables. No processes has a message_queue_len > 0. No >>>> processes that we maintain use the process dictionary. >>>> >>>> I started a second VM, migrating our OTP apps to the second VM, one at a >>>> time, hoping that stopping one would cause a release of lots of memory >>>> on the first VM. No such luck; it released a couple more MBs but still a >>>> large presence and no change in swap usage. I then stopped everything I >>>> could until only stdlib and kernel were running on the first VM, but to >>>> no avail. >>>> >>>> Each application has 1-3 long running gen_servers that listen on an AMQP >>>> queue and spawn worker processes to handle messages. They carry almost >>>> no state. Some of the handlers will read binary blobs, open a socket, >>>> wait for a connection, then transmit the blobs, closing the socket and >>>> dying afterwards (they timeout after 5 minutes if no connection is >>>> attempted, and are in a supervision tree so I know there aren't a mass >>>> of them lying around). The blobs are between 20KB and 2MB. The rest of >>>> the workers are similarly ephemeral. >>>> >>>> What other options are there to find where memory has been allocated? We >>>> use binaries almost exclusively for strings, don't construct large >>>> lists. I've poured over the efficiency guide looking for ideas of where >>>> we've gone astray but am coming up blank so far. >>>> >>>> The system is idle; the only processes should be (and are, as far as I >>>> can tell) the gen_servers meant to run for a long time. >>>> >>>> Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] >>>> [async-threads:8] [kernel-poll:true] >>>> >>>> Any ideas are welcome, >>>> >>>> James >>>> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOOZTzAAoJENc77s1OYoGgNSQH+QGcD5WDBSJp+z6RCjAqcRS+ TM0fxxLolnWN9wdeje/v91ZA+G+usz/LbR0LhctwD7gJk29Ni56CO2cNP1osw5/c VWMSvxllDqn6ZmqAGl+h0lpYH9ZLWiZNZ1f4BjihHLjyglhdAKdRYfbQSzZsilQA Zu2AcfBvSQ564ByOEnU4ZEqrVaY9+EnI7WOjfd9HMHG2GBPgpDJauUGWfF21Qglf w7FESB/Gv6TMUHgdBEpBXmi1A1XgJFlQh8kK0xz6rNEp7GbHHrzk9Nl/644HzwGJ jsMpmBf6r+NXDwlR4Brd7eIdoIcnJ7DFIGEfUHFS73zkFAGxQdOoDCDGgnTSwSc= =jp4S -----END PGP SIGNATURE----- From icarus.alive@REDACTED Wed Aug 3 20:40:20 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Thu, 4 Aug 2011 00:10:20 +0530 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: <9158FB12-40A7-4276-BD18-C7CACF7690B7@worrell.nl> Message-ID: On Wed, Aug 3, 2011 at 9:50 PM, OvermindDL1 wrote: > > I am curious, how is ChicagoBoss compared to Nitrogen? Haven't experienced Nitrogen long enough to make a good objective comparison. Just that there was something about Nitrogen Records, that seemed -- verbose and intimidating. Also, found it somewhat difficult to imagine how that syntax maps to the generated HTML. On the contrary, I found ErlyDTL very simple, once I saw the CB video-tutorials on youtube. Also the MVC constructs were a bit easier for me to follow in case of ChicagoBoss. The admin-interface was dead-simple. Finally, I found the tutorial for ChicagoBoss to be short, to-the-point, very simple/lucid and achieved something concrete I understood. The documentation get me functional quite quickly, and in fact, there wasn't very much to read. While it might take a lot more to become an expert, I felt that I could get up-n-running (from zero) with CB much sooner. > On Aug 2, 2011 11:31 PM, "Icarus Alive" wrote: >> On Wed, Aug 3, 2011 at 3:54 AM, Benson Wong wrote: >>> Hi, >>> My background is a web developer (~15 years), done PHP for about 11 >>> years. >>> So I know it pretty well. >>> Just starting to learn Erlang. Done lots of javascript (jQuery, node.js, >>> etc). So a PHP developer, in the Erlang pool here :) >>> I think an imperative language like PHP would be a lot easier (and more >>> agile) to build the web app in than Erlang. I would choose erlang if I >>> need >>> long lived connections (web sockets, long-poll, etc). Frameworks like >>> Django >>> (Python)/Rails (ruby) would also make life easier. >>> If you want scalability/reliability/performance, add more PHP processes / >>> servers. >>> PHP is shared nothing, so it is easy to scale. Just add more. Using Nginx >>> + >>> PHP-FPM. Solves a lot of slow request issues. >>> Put them behind an HTTP load balancer, HAProxy is a very popular choice. >>> Performance is usually database bound than it is app server bound. >>> On Fri, Jul 29, 2011 at 6:16 AM, Icarus Alive >>> wrote: >>>> >>>> Would appreciate if my thread wasn't hijacked :-)... although gotta >>>> agree that our scenarios are quite similar. >>>> >>>> On Fri, Jul 29, 2011 at 6:31 PM, Banibrata Dutta >>>> wrote: >>>> > Thanks @Marc, for taking time to reply this this mail. >>>> > On Fri, Jul 29, 2011 at 5:38 PM, Marc Worrell wrote: >>>> >> >>>> >> It mostly depends on what kind of web development you want to do. >>>> >> >>>> >> Is it more of a mobile application than a publishing web site? >>>> >>>> In my case, content characteristics are: >>>> 1. Always originating from single source >>>> 2. Always consumed by single user >>>> 3. Content has short lifetime - once consumed, will be archived >>>> 4. Each user has several hundreds of content display to her in "newest >>>> first" fashion >>>> >>>> >> >>>> >> Does is have specific authentication schemes? >>>> >> Etc. Etc. >>>> > >>>> > Honestly, haven't given this enough thought. One of the requirements >>>> > is >>>> > to >>>> > ensure privacy of user data, and that of the interaction. I'm yet to >>>> > figure >>>> > out the "how" part. >>> >>> Security/Privacy, this is more of an operation and application design >>> than >>> it is a Erlang/OTP vs PHP/Ruby/Python+framework. >>> Generally you'd want to: >>> * firewalls, no SSH access, web servers are only web servers, ie: only >>> port >>> 443 open. >>> * use cookie based sessions. Don't use HTTP basic authentication. >>> * Server has controls all access based on the cookie. >>> * Use HTTPS. Always. >>> * There are lots of other techniques: >>> -- use bcrypt for passwords >>> -- sessions can be limited to one IP address at a time >>> -- sessions have short expire times (controlled on the server) >>> -- etc. >>> >>>> >>>> Well in my case it has to be some strong authentication. Storage, >>>> archival and access has to conform to HIPAA compliance. So we know we >>>> are dealing with EMR type content. >> >> Hi Benson, >> >> Really appreciate your descriptive & informative response, especially >> so for the part on security. >> >> My inclination towards Erlang stems from the fact that I was intending >> to do much of the backend in Erlang as well. The backend does lot of >> other things than just bridging between FE client request/responses >> and a DB, s.a. converting content format, managing archival policies, >> do some data analysis for reporting etc. Maybe some of that via ports. >> >> I wish I'd seen your mail few days back, because having tried both >> Nitrogen and Zotonic, which are I am sure excellent products, I think >> I've found peace in settling for ChicagoBoss. I found it lot easier to >> work with, as a beginner, maybe because the FP guts are almost hidden >> away from sight, and much of the developer/user coding is done in an >> imperative fashion. Also, I found it's documentation (although quite >> light) to be simple & just-enough to get me started and get me going. >> >> However, I might have to come back to PHP unless I am able to train my >> team on Erlang. Being a beginner, I find it hard to imagine being a >> good teacher / coach for rest of the team. >> >> Thanks again, everyone, for responding on this thread. >> From alavrik@REDACTED Wed Aug 3 21:45:01 2011 From: alavrik@REDACTED (Anton Lavrik) Date: Wed, 3 Aug 2011 14:45:01 -0500 Subject: [erlang-questions] Erlang de/serializer and data type validator In-Reply-To: <20110803064932.GA23559@jakstys.lt> References: <20110802082752.GA21594@jakstys.lt> <61DEDBFF-8B0A-48A4-A101-1E6C59568600@erlang-solutions.com> <20110803063219.GC22458@jakstys.lt> <20110803064932.GA23559@jakstys.lt> Message-ID: 2011/8/3 Motiejus Jak?tys > On Wed, Aug 03, 2011 at 09:32:19AM +0300, Motiejus Jak?tys wrote: > > On Tue, Aug 02, 2011 at 09:10:49PM +0100, Tim Watson wrote: > > > http://piqi.org/ might also be worth a look. > > > > Looks good. Actually, if I understand correctly, it's exactly what I`m > > looking for. A thing that allows me to translate XML/JSON documents to > > native Erlang data types according to schema and serialize back. > > > On the other hand, it is really heavy in terms of dependencies.. > On debian: > apt-get install ocaml camlp4-extra ocaml-findlib gcc m4 make sed > > Is there anything lighter? Ideally, it should be written in Erlang and > C, as my required feature set is pretty short. > Those dependencies are needed only for building Piqi. The build produces two natively compiled self-contained executables that don't have any runtime dependencies. If you use rebar, Piqi has a rebar package which comes with the precompiled binaries for popular platforms. It is available at: https://github.com/alavrik/piqi-erlang This way, you won't need to deal with dependencies at all. (I'm the author of Piqi. Catching up on mail after vacation.) Anton -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Aug 3 22:19:41 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 3 Aug 2011 13:19:41 -0700 (PDT) Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: <29781133.2149.1312402781522.JavaMail.geo-discussion-forums@yqiw2> http://code.google.com/p/erlyvideo repository wasn't updated for years. All development now is going in http://github.com/erlyvideo and private repositories. Currently I have not only frame repacking layer for codecs h264,aac,mp3,mpeg4,vp6,h263,pcm,g.711,g.726 and such protocols as mpeg-ts, rtmp, rtsp/rtp, sip, but also some device handling features and transcoding modules. And I can say you: ffmpeg is much better when it is in erlang =) We are adding now generic libavcodec support, but such things like libavformat at very badly written in C. Erlang fits this task much better. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rapsey@REDACTED Wed Aug 3 22:35:58 2011 From: rapsey@REDACTED (Rapsey) Date: Wed, 3 Aug 2011 22:35:58 +0200 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: <9158FB12-40A7-4276-BD18-C7CACF7690B7@worrell.nl> Message-ID: Yeah I never liked nitrogen records either. I don't know how others build sites, but at my company the client side steps are always: photoshop file -> create html/css -> convert to templates and write javascript Using django templates makes complete sense when you're starting from an html file. It does not take that much work at all and requires very little programming knowledge. If you change anything, it's not difficult to put the changes in the template, because they are so similar. If we used nitrogen, it would take significantly more work and the person doing it, would need to learn something completely unnatural if he did not know erlang already. Sergej On Wed, Aug 3, 2011 at 8:40 PM, Icarus Alive wrote: > On Wed, Aug 3, 2011 at 9:50 PM, OvermindDL1 wrote: > > > > I am curious, how is ChicagoBoss compared to Nitrogen? > > Haven't experienced Nitrogen long enough to make a good objective > comparison. Just that there was something about Nitrogen Records, that > seemed -- verbose and intimidating. Also, found it somewhat difficult > to imagine how that syntax maps to the generated HTML. On the > contrary, I found ErlyDTL very simple, once I saw the CB > video-tutorials on youtube. Also the MVC constructs were a bit easier > for me to follow in case of ChicagoBoss. The admin-interface was > dead-simple. Finally, I found the tutorial for ChicagoBoss to be > short, to-the-point, very simple/lucid and achieved something concrete > I understood. The documentation get me functional quite quickly, and > in fact, there wasn't very much to read. While it might take a lot > more to become an expert, I felt that I could get up-n-running (from > zero) with CB much sooner. > > > On Aug 2, 2011 11:31 PM, "Icarus Alive" wrote: > >> On Wed, Aug 3, 2011 at 3:54 AM, Benson Wong > wrote: > >>> Hi, > >>> My background is a web developer (~15 years), done PHP for about 11 > >>> years. > >>> So I know it pretty well. > >>> Just starting to learn Erlang. Done lots of javascript (jQuery, > node.js, > >>> etc). So a PHP developer, in the Erlang pool here :) > >>> I think an imperative language like PHP would be a lot easier (and more > >>> agile) to build the web app in than Erlang. I would choose erlang if I > >>> need > >>> long lived connections (web sockets, long-poll, etc). Frameworks like > >>> Django > >>> (Python)/Rails (ruby) would also make life easier. > >>> If you want scalability/reliability/performance, add more PHP processes > / > >>> servers. > >>> PHP is shared nothing, so it is easy to scale. Just add more. Using > Nginx > >>> + > >>> PHP-FPM. Solves a lot of slow request issues. > >>> Put them behind an HTTP load balancer, HAProxy is a very popular > choice. > >>> Performance is usually database bound than it is app server bound. > >>> On Fri, Jul 29, 2011 at 6:16 AM, Icarus Alive > >>> wrote: > >>>> > >>>> Would appreciate if my thread wasn't hijacked :-)... although gotta > >>>> agree that our scenarios are quite similar. > >>>> > >>>> On Fri, Jul 29, 2011 at 6:31 PM, Banibrata Dutta > >>>> wrote: > >>>> > Thanks @Marc, for taking time to reply this this mail. > >>>> > On Fri, Jul 29, 2011 at 5:38 PM, Marc Worrell > wrote: > >>>> >> > >>>> >> It mostly depends on what kind of web development you want to do. > >>>> >> > >>>> >> Is it more of a mobile application than a publishing web site? > >>>> > >>>> In my case, content characteristics are: > >>>> 1. Always originating from single source > >>>> 2. Always consumed by single user > >>>> 3. Content has short lifetime - once consumed, will be archived > >>>> 4. Each user has several hundreds of content display to her in "newest > >>>> first" fashion > >>>> > >>>> >> > >>>> >> Does is have specific authentication schemes? > >>>> >> Etc. Etc. > >>>> > > >>>> > Honestly, haven't given this enough thought. One of the requirements > >>>> > is > >>>> > to > >>>> > ensure privacy of user data, and that of the interaction. I'm yet to > >>>> > figure > >>>> > out the "how" part. > >>> > >>> Security/Privacy, this is more of an operation and application design > >>> than > >>> it is a Erlang/OTP vs PHP/Ruby/Python+framework. > >>> Generally you'd want to: > >>> * firewalls, no SSH access, web servers are only web servers, ie: only > >>> port > >>> 443 open. > >>> * use cookie based sessions. Don't use HTTP basic authentication. > >>> * Server has controls all access based on the cookie. > >>> * Use HTTPS. Always. > >>> * There are lots of other techniques: > >>> -- use bcrypt for passwords > >>> -- sessions can be limited to one IP address at a time > >>> -- sessions have short expire times (controlled on the server) > >>> -- etc. > >>> > >>>> > >>>> Well in my case it has to be some strong authentication. Storage, > >>>> archival and access has to conform to HIPAA compliance. So we know we > >>>> are dealing with EMR type content. > >> > >> Hi Benson, > >> > >> Really appreciate your descriptive & informative response, especially > >> so for the part on security. > >> > >> My inclination towards Erlang stems from the fact that I was intending > >> to do much of the backend in Erlang as well. The backend does lot of > >> other things than just bridging between FE client request/responses > >> and a DB, s.a. converting content format, managing archival policies, > >> do some data analysis for reporting etc. Maybe some of that via ports. > >> > >> I wish I'd seen your mail few days back, because having tried both > >> Nitrogen and Zotonic, which are I am sure excellent products, I think > >> I've found peace in settling for ChicagoBoss. I found it lot easier to > >> work with, as a beginner, maybe because the FP guts are almost hidden > >> away from sight, and much of the developer/user coding is done in an > >> imperative fashion. Also, I found it's documentation (although quite > >> light) to be simple & just-enough to get me started and get me going. > >> > >> However, I might have to come back to PHP unless I am able to train my > >> team on Erlang. Being a beginner, I find it hard to imagine being a > >> good teacher / coach for rest of the team. > >> > >> Thanks again, everyone, for responding on this thread. > >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve@REDACTED Wed Aug 3 22:44:40 2011 From: steve@REDACTED (Steve Strong) Date: Wed, 3 Aug 2011 22:44:40 +0200 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: <29781133.2149.1312402781522.JavaMail.geo-discussion-forums@yqiw2> References: <29781133.2149.1312402781522.JavaMail.geo-discussion-forums@yqiw2> Message-ID: <93FADF05-1B85-4183-811A-12B6A7ADA575@srstrong.com> What's the chances of erlyvideo and the related libraries also being licensed with something slightly more permissive than GPL, such as LGPL or similar; we deal with a number of corporate clients where use of GPL is pretty much forbidden, so are having to replicate a number of areas of erylvideo functionality. If the licensing was different, we could instead spend such efforts contributing back to erlyvideo? Cheers, Steve On 3 Aug 2011, at 22:19, Max Lapshin wrote: > http://code.google.com/p/erlyvideo repository wasn't updated for years. > > All development now is going in http://github.com/erlyvideo and private repositories. > > Currently I have not only frame repacking layer for codecs h264,aac,mp3,mpeg4,vp6,h263,pcm,g.711,g.726 and such protocols as > mpeg-ts, rtmp, rtsp/rtp, sip, but also some device handling features and transcoding modules. > > And I can say you: ffmpeg is much better when it is in erlang =) We are adding now generic libavcodec support, but such things > like libavformat at very badly written in C. Erlang fits this task much better. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Aug 3 22:48:12 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 03 Aug 2011 22:48:12 +0200 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: <9158FB12-40A7-4276-BD18-C7CACF7690B7@worrell.nl> Message-ID: <4E39B40C.6030405@dev-extend.eu> On 08/03/2011 10:35 PM, Rapsey wrote: > Yeah I never liked nitrogen records either. I don't know how others > build sites, but at my company the client side steps are always: > photoshop file -> create html/css -> convert to templates and write > javascript Same here and everywhere I worked, whichever the language. In fact I wouldn't recommend doing otherwise for any sizeable project. There's people who know their photoshop, others who know their HTML/CSS/JS, so it's definitely worth using their skills directly. There's in fact many other reasons that make building the interface directly in HTML/CSS/JS worthwhile. But I guess I also see the appeal in people wanting to link client and server-side code to build complicated interfaces. I do not know Nitrogen enough to cast a judgement on it, though. One thing to keep in mind however, if you go the HTML/CSS/JS route, is that the URIs you use for serving your HTML can also serve JSON for the AJAX calls and also serve as a public API for your website, all in one without much extra code required. > Using django templates makes complete sense when you're starting from an > html file. It does not take that much work at all and requires very > little programming knowledge. If you change anything, it's not difficult > to put the changes in the template, because they are so similar. > If we used nitrogen, it would take significantly more work and the > person doing it, would need to learn something completely unnatural if > he did not know erlang already. Django is simple enough that the HTML/CSS people can make their own changes directly in the templates without breaking anything (or not too often at least). -- Lo?c Hoguin Dev:Extend From max.lapshin@REDACTED Wed Aug 3 23:08:06 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 4 Aug 2011 01:08:06 +0400 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: <93FADF05-1B85-4183-811A-12B6A7ADA575@srstrong.com> References: <29781133.2149.1312402781522.JavaMail.geo-discussion-forums@yqiw2> <93FADF05-1B85-4183-811A-12B6A7ADA575@srstrong.com> Message-ID: On Thu, Aug 4, 2011 at 12:44 AM, Steve Strong wrote: > What's the chances of erlyvideo and the related libraries also being > licensed with something slightly more permissive than GPL, such as LGPL or > similar; we deal with a number of corporate clients where use of GPL is > pretty much forbidden, so are having to replicate a number of areas of > erylvideo functionality. ?If the licensing was different, we could instead > spend such efforts contributing back to erlyvideo? > Cheers, > Steve I'm open to communications on this point. While erlyvideo was licensed as MIT I have received only 10 lines of code from external contributors. Only 10 lines during 3 years. Now, when erlyvideo is GPL it is easier for me to get money and erlyvideo becomes better and better. Sorry, but I don't believe in opensource contributors to streaming servers. I'm watching at situation in other streaming servers (rtmpd and red5) and see, that there is the same situation: no contributors, but many users. So, currently I have zero reasons to switch licenses, because for 3 years there were absolutely no qualified contributors willing to send their code, but there were guys, that wished to sell packed erlyvideo without sharing me any profit. Yet, there are commercial clients, that don't want to distribute their source code and I find ways to work with them and I share my code under non-GPL. Everybody is in win from this: we get profit, community gets better opensourced code. While there are no substitutions for libraries, that I've written for erlyvideo, I don't see any reasons to change anything. BSD is good for infrastructure code. For example, libreadline gets problems from GPL license because we have to use horrible libedit. Erlyvideo libraries aren't distributable outside erlyvideo, so there is even no reason to make them LGPL. From steve@REDACTED Wed Aug 3 23:20:01 2011 From: steve@REDACTED (Steve Strong) Date: Wed, 3 Aug 2011 23:20:01 +0200 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: <29781133.2149.1312402781522.JavaMail.geo-discussion-forums@yqiw2> <93FADF05-1B85-4183-811A-12B6A7ADA575@srstrong.com> Message-ID: <45E3A753-CD43-45A8-831A-DF7294F3B6A5@srstrong.com> Given your previous experience when licensed as MIT, I can fully understand your position - 10 lines in 3 years is a pretty poor return :( A real shame (from my perspective), but entirely understandable. Cheers, Steve On 3 Aug 2011, at 23:08, Max Lapshin wrote: > On Thu, Aug 4, 2011 at 12:44 AM, Steve Strong wrote: >> What's the chances of erlyvideo and the related libraries also being >> licensed with something slightly more permissive than GPL, such as LGPL or >> similar; we deal with a number of corporate clients where use of GPL is >> pretty much forbidden, so are having to replicate a number of areas of >> erylvideo functionality. If the licensing was different, we could instead >> spend such efforts contributing back to erlyvideo? >> Cheers, >> Steve > > > I'm open to communications on this point. > > While erlyvideo was licensed as MIT I have received only 10 lines of > code from external contributors. Only 10 lines during 3 years. > > Now, when erlyvideo is GPL it is easier for me to get money and > erlyvideo becomes better and better. Sorry, but I don't believe in > opensource > contributors to streaming servers. I'm watching at situation in other > streaming servers (rtmpd and red5) and see, that there is the same > situation: > no contributors, but many users. > > So, currently I have zero reasons to switch licenses, because for 3 > years there were absolutely no qualified contributors willing to send > their code, > but there were guys, that wished to sell packed erlyvideo without > sharing me any profit. > > Yet, there are commercial clients, that don't want to distribute their > source code and I find ways to work with them and I share my code > under non-GPL. > Everybody is in win from this: we get profit, community gets better > opensourced code. > > While there are no substitutions for libraries, that I've written for > erlyvideo, I don't see any reasons to change anything. > BSD is good for infrastructure code. For example, libreadline gets > problems from GPL license because we have to use horrible libedit. > > Erlyvideo libraries aren't distributable outside erlyvideo, so there > is even no reason to make them LGPL. From carlsson.richard@REDACTED Wed Aug 3 23:47:13 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 03 Aug 2011 23:47:13 +0200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> <1312383957.26732.YahooMailNeo@web111412.mail.gq1.yahoo.com> <4E3974A3.6070104@gmail .com> Message-ID: <4E39C1E1.7070301@gmail.com> On 08/03/2011 07:05 PM, Ulf Wiger wrote: > > According to the Barklund spec (page 62), it said that "a compiler > must only accept a program if for any evaluation order, there will > not be an applied occurrence of an unbound variable". > > "For example, in a context where X is unbound, the expression (X=8) + > X should give a compile-time error." > > (And it does, too. I checked). That's a slightly different thing - a requirement that any path that reaches a use of a variable must first pass through a binding of that variable. For instance: f(Y) -> case test(Y) of true -> X = 1; false -> ok end, g(X). is not a valid program, because if the false path is taken, X is not defined. The quote from the spec can be seen as a general design principle, which in practice is implemented by the scoping rules: For the case expression, a variable binding is only propagated out of a case if it is guaranteed to be bound in all clauses. For a function call (A+B is really a call to erlang:'+'(A,B)), all arguments are evaluated in the same ingoing environment , so even though the first argument to + creates a binding of X, this does not affect the environment for evaluating the second argument, in which X is not yet bound, so in both these examples the scoping rules forbid them. Hence, the scoping rules partially determine the order of evaluation of the subexpressions, for the programs that pass this check (and assuming there is not a bug in the lower levels of the compiler), but on top of this, Erlang defines a strict left-to-right evaluation order for function arguments that has nothing to do with scoping. E.g., {io:format("hello"), io:format("ulf")} should always print "helloulf" and never "ulfhello". An interesting twist on the example (X=8) + X is that if *both* arguments create a binding of the same variable, as in: (X=8) + (X=8) then Erlang will first evaluate the subexpressions, then *check* that the results match so that a single agreed value of X can be exported out from the whole expression, and only in that case will it call + on the results. You can experiment with the following to see what I mean: f(Y) -> (X=Y) + (X=8). (and no, it doesn't matter if the match against Y is on the right or left of the plus sign). f(8) will return 16, and f(Y) for any other Y will cause a badmatch. (I'm glad this worked - for a moment I was afraid that someone had forgotten to implement the check.) /Richard From carlsson.richard@REDACTED Thu Aug 4 00:05:20 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 04 Aug 2011 00:05:20 +0200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> <1312383957.26732.YahooMailNeo@web111412.mail.gq1.yahoo.com> <4E3974A3.6070104@gmail .com> Message-ID: <4E39C620.8020507@gmail.com> On 08/03/2011 07:05 PM, Ulf Wiger wrote: > On 3 Aug 2011, at 18:17, Richard Carlsson wrote: > >>> And did it extend to pattern matching? >> >> No, there was never any discussion about that. If the match >> succeeds then it would succeed no matter what the order, and if a >> you get a 'badmatch' error, the entire matched term is blamed, not >> just the first subterm that doesn't match, so I can't see that >> there would be any observable difference depending on the order of >> matched subterms. I forgot to clarify this. As I said in the previous mail, for an expression such as {f(), g()}, Erlang defines the evaluation order to be f() first, then g(). But in the case of pattern matching: {P1, P2} = SomeExpression (where P1 and P2 can be unbound variables, bound variables, or any other patterns large or small, including where P1 and P2 are exactly the same) - if the match works, then the resulting bindings (if any) from the pattern as a whole must be the same regardless of the order you matched the subpatterns against the value of SomeExpression. And if the match fails, then you just see a {badmatch, SomeExpression} error which doesn't reveal in what part of SomeExpression the match first discovered that the value couldn't be matched against the pattern, even though there could be more than one place, as in {1,2,3}={3,2,1}. Hence, whether the matching tries P1 or P2 first is an internal implementation detail. /Richard From robert.virding@REDACTED Thu Aug 4 00:17:21 2011 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 03 Aug 2011 23:17:21 +0100 (BST) Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: Message-ID: I am assuming that the check only applies for variables in matches with '=' and not in case/receive patterns. In that case I would not mind having such a warning turnable on/off at will. I seldom test for values in '=' matches, only in case/receive. The emacs colouring is difficult given how erlang handles variables and evaluation ordering etc. Robert ----- Original Message ----- > Actually I like things as they are. My eye is trained to see bound > and > free occurrences of a variable. > > On the other hand it would be *excellent* if the color coding in > emacs > (whatever) was changed to reflect the > binding status of a variable. > > If variables in the LHS of an equals were colored green if they were > unbound and red > if bound then one would see at a glance what was intended. > > (( assuming you're not red-green color blind that is )) - or set the > variables in a different type face. > > > /Joe > > > > 2011/8/3 Fr?d?ric Trottier-H?bert : > > On 2011-08-02, at 01:59 AM, Richard O'Keefe wrote: > > > >> 2. Is matching against an already-bound variable a check we want? > > > > To me, matching against an already-bound variable is is a valid > > assertion, as much as 'ok = function_call()' might be, and as > > implicit as '{ok, Pid}' or '{error, already_started}' might be. > > Matching on already-bound variables is part of my standard code > > when trying to crash early when possible, and also part of many > > basic test suites that simply do pattern matching here and there. > > To me it's as basic as using the same variable twice in a single > > pattern (f([A,A,B,B|_]) when A =/= B -> ...), or something similar > > with records. > > > > I would likely not use the check at all, and if it were to be > > added, would prefer it to be a compiler argument (which could be > > enabled in -compile(...).) I foresee little use of such warnings > > for myself and would dislike to see it becoming a default setting. > > > > -- > > Fred H?bert > > http://www.erlang-solutions.com > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From robert.virding@REDACTED Thu Aug 4 00:24:13 2011 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 03 Aug 2011 23:24:13 +0100 (BST) Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <4E39C1E1.7070301@gmail.com> Message-ID: <736aece5-b7e2-400d-b3c4-33ce38ec28d7@localhost> I must say that I find this a little strange. If the evaluation order of subexpressions is defined, which it is, then I would have expected that the whole effect of a subexpression would be visible in "later" subexpressions. Hence you could do things like: (X=8) + X foo(X = bar(42), X) but not X + (X=8) foo(X, X = bar(42)) because of the evaluation order. Although I suppose it does make it easier to see the effects of variable bindings if the are delayed. I will have to go back an re-read the spec. It should really be updated. Robert ----- Original Message ----- > On 08/03/2011 07:05 PM, Ulf Wiger wrote: > > > > According to the Barklund spec (page 62), it said that "a compiler > > must only accept a program if for any evaluation order, there will > > not be an applied occurrence of an unbound variable". > > > > "For example, in a context where X is unbound, the expression (X=8) > > + > > X should give a compile-time error." > > > > (And it does, too. I checked). > > That's a slightly different thing - a requirement that any path that > reaches a use of a variable must first pass through a binding of that > variable. For instance: > > f(Y) -> > case test(Y) of > true -> X = 1; > false -> ok > end, > g(X). > > is not a valid program, because if the false path is taken, X is not > defined. The quote from the spec can be seen as a general design > principle, which in practice is implemented by the scoping rules: For > the case expression, a variable binding is only propagated out of a > case > if it is guaranteed to be bound in all clauses. For a function call > (A+B > is really a call to erlang:'+'(A,B)), all arguments are evaluated in > the > same ingoing environment , so even though the first argument to + > creates a binding of X, this does not affect the environment for > evaluating the second argument, in which X is not yet bound, so in > both > these examples the scoping rules forbid them. > > Hence, the scoping rules partially determine the order of evaluation > of > the subexpressions, for the programs that pass this check (and > assuming > there is not a bug in the lower levels of the compiler), but on top > of > this, Erlang defines a strict left-to-right evaluation order for > function arguments that has nothing to do with scoping. E.g., > > {io:format("hello"), io:format("ulf")} > > should always print "helloulf" and never "ulfhello". > > An interesting twist on the example (X=8) + X is that if *both* > arguments create a binding of the same variable, as in: > > (X=8) + (X=8) > > then Erlang will first evaluate the subexpressions, then *check* that > the results match so that a single agreed value of X can be exported > out > from the whole expression, and only in that case will it call + on > the > results. You can experiment with the following to see what I mean: > > f(Y) -> > (X=Y) + (X=8). > > (and no, it doesn't matter if the match against Y is on the right or > left of the plus sign). f(8) will return 16, and f(Y) for any other Y > will cause a badmatch. (I'm glad this worked - for a moment I was > afraid > that someone had forgotten to implement the check.) > > /Richard > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From carlsson.richard@REDACTED Thu Aug 4 00:39:39 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 04 Aug 2011 00:39:39 +0200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <736aece5-b7e2-400d-b3c4-33ce38ec28d7@localhost> References: <736aece5-b7e2-400d-b3c4-33ce38ec28d7@localhost> Message-ID: <4E39CE2B.4050701@gmail.com> On 08/04/2011 12:24 AM, Robert Virding wrote: > I must say that I find this a little strange. If the evaluation order > of subexpressions is defined, which it is, then I would have expected > that the whole effect of a subexpression would be visible in "later" > subexpressions. Hence you could do things like: > > (X=8) + X > foo(X = bar(42), X) > > but not > > X + (X=8) > foo(X, X = bar(42)) > > because of the evaluation order. You could, yes, but it's not a good idea. For example, a refactoring that switches the order of arguments to a function would have to consider any bindings in calling occurrences and lift them out to preserve the evaluation order *and* the bindings: foo(f(), X=bar(), g(X)) must become V1 = f() X = bar(42), V2 = g(X), foo(V1, V2, X) or possibly foo(f(), begin X=bar(42), g(X) end, X) if you want to switch the order of the last two arguments. The rule that all arguments are evaluated in the same environment makes the world a happier place. If I want Basic, I know where I can get it. ;-) /Richard From ok@REDACTED Thu Aug 4 01:18:05 2011 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 4 Aug 2011 11:18:05 +1200 Subject: [erlang-questions] Erlang de/serializer and data type validator In-Reply-To: References: <20110802082752.GA21594@jakstys.lt> <61DEDBFF-8B0A-48A4-A101-1E6C59568600@erlang-solutions.com> <20110803063219.GC22458@jakstys.lt> <20110803064932.GA23559@jakstys.lt> Message-ID: <080A2468-9C83-4997-8E9D-030F6F7CB445@cs.otago.ac.nz> On 3/08/2011, at 9:44 PM, Tim Watson wrote: > For JSON I > don't know what you'd use as there isn't really a definitive schema > language for JSON not a definitive transformation language to describe > converting JSON to/from XML etc. I suppose it all depends on what "is" is, and what "definitive" defines. http://json-schema.org/ Implementations exist for Javascript, Java, PHP, Python, Ruby, and some others. The RFC is still only a draft, so I suppose you could say it "isn't" yet. I find it sufficiently horrifying. (As opposed to XML Schemas, which are excessively horrifying.) Converting JSON to XML is dead simple. Step 1. Define a universal JSON-in-XML representation. N S e1 ... en v1 ... vn where each element of an object has a key="ki" attribute. (A program that builds a DOM or writes a stream of this sort from any given JSON value is really quite trivial.) Step 2. Use any XML->XML transformation language (such as XSLT) that you please. Thanks to the universal mapping, if the translator accepts its input using SAX or DOM, the the universal representation need never actually exist. From max.lapshin@REDACTED Thu Aug 4 08:50:35 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 4 Aug 2011 10:50:35 +0400 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: <45E3A753-CD43-45A8-831A-DF7294F3B6A5@srstrong.com> References: <29781133.2149.1312402781522.JavaMail.geo-discussion-forums@yqiw2> <93FADF05-1B85-4183-811A-12B6A7ADA575@srstrong.com> <45E3A753-CD43-45A8-831A-DF7294F3B6A5@srstrong.com> Message-ID: On Thu, Aug 4, 2011 at 1:20 AM, Steve Strong wrote: > Given your previous experience when licensed as MIT, I can fully understand your position - 10 lines in 3 years is a pretty poor return :( ?A real shame (from my perspective), but entirely understandable. > Yes. I spend 16 hours per day on erlyvideo code, but this is why I release it under GPL. I'm not working on anybody and nobody invests in BSD license, so this is the same situation as with libx264: no substitutions in erlang world (and frankly speaking even in Java, red5 is not an issue) and I'm ready to release it closed source for commercial users. From caox@REDACTED Thu Aug 4 08:56:23 2011 From: caox@REDACTED (caox) Date: Thu, 4 Aug 2011 14:56:23 +0800 Subject: [erlang-questions] ranged lookup on ordered_set ETS Message-ID: <76FD639B-FEA8-4BEB-93A6-54F0E9849796@lightpole.net> Hi According to ETS reference, records in ordered_set ETS are stored in the Erlang Term order of their key. So, is there a efficient way to do a ranged lookup of key on this kind of ETS without scanning the whole table. For example, 100 < Key <105. I guess ets:select/2 doesn't fit for the job. BR From rexxe98@REDACTED Thu Aug 4 09:01:30 2011 From: rexxe98@REDACTED (Andrew Berman) Date: Thu, 4 Aug 2011 00:01:30 -0700 Subject: [erlang-questions] Best approach to creating a process/connection pool Message-ID: I'm fairly new to Erlang and I am trying to write a process pool which essentially is a connection pool to a database. I have designed the application like this: The main pool gen_server has a queue of PIDs of gen_fsm's. Each gen_fsm calls spawn_links a 3rd party gen_server and carries the state of the connection. So the 3rd party gen_server and the gen_fsm should always live and die together. I currently have set it up such that gen_fsm calls proc_lib:spawn_link() to start the 3rd party gen_server, but I'm wondering how this factors into the supervision tree. I currently have two supervisors: Sup1 -> one_for_all -> 2 children: pool gen_server and Sup2 Sup2 -> simple_one_for_one -> 1 child: gen_fsm So my question is: is it ok to use spawn_link instead of somehow trying to put the 3rd party gen_server into the supervision tree? If it should go into the supervision tree, how would I organize the tree then such that the gen_fsm and 3rd party gen_server are reliant on each other? Thanks for any help! From huss01@REDACTED Thu Aug 4 09:13:21 2011 From: huss01@REDACTED (=?ISO-8859-1?Q?H=E5kan_Huss?=) Date: Thu, 4 Aug 2011 09:13:21 +0200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: Message-ID: On Thu, Aug 4, 2011 at 00:17, Robert Virding wrote: > I am assuming that the check only applies for variables in matches with '=' and not in case/receive patterns. In that case I would not mind having such a warning turnable on/off at will. I seldom test for values in '=' matches, only in case/receive. > There is one area of code where I use matching of bound variables with = a lot, and that is in ct test cases. A typical example is: ... Alarms0 = get_current_alarm_list(), Alarms0 = get_current_alarm_list(), Alarms1 = get_current_alarm_list(), verify_alarm_raised(Alarms1, Alarms0, ), Alarms0 = get_current_alarm_list(), ... While ROK's crash_if_different could be used here, my personal opinion is that the test code as written expresses the test better and does not require that I introduce a bunch of new variables to hold old values. In normal code, however, I use this very sparingly. Then again, I seldom experience this as a problem. Thinking back, most of the times I've been bitten by this is probably in case clauses, especially when adding code. Typically, when using canonical variables like Pid, Reason or Result without checking that the code isn't in a branch which already uses it. And since I agree with Robert that case and receive should not cause a warning, I don't really feel the need for it. Regards, /H?kan > The emacs colouring is difficult given how erlang handles variables and evaluation ordering etc. > > Robert > > ----- Original Message ----- >> Actually I like things as they are. My eye is trained to see bound >> and >> free occurrences of a variable. >> >> On the other hand it would be *excellent* if the color coding in >> emacs >> (whatever) was changed to reflect the >> binding status of a variable. >> >> If variables in the LHS of an equals were colored green if they were >> unbound and red >> if bound then one would see at a glance what was intended. >> >> (( assuming you're not red-green color blind that is )) - or set the >> variables in a different type face. >> >> >> /Joe >> >> >> >> 2011/8/3 Fr?d?ric Trottier-H?bert : >> > On 2011-08-02, at 01:59 AM, Richard O'Keefe wrote: >> > >> >> 2. Is matching against an already-bound variable a check we want? >> > >> > To me, matching against an already-bound variable is is a valid >> > assertion, as much as 'ok = function_call()' might be, and as >> > implicit as '{ok, Pid}' or '{error, already_started}' might be. >> > Matching on already-bound variables is part of my standard code >> > when trying to crash early when possible, and also part of many >> > basic test suites that simply do pattern matching here and there. >> > To me it's as basic as using the same variable twice in a single >> > pattern (f([A,A,B,B|_]) when A =/= B -> ...), or something similar >> > with records. >> > >> > I would likely not use the check at all, and if it were to be >> > added, would prefer it to be a compiler argument (which could be >> > enabled in -compile(...).) I foresee little use of such warnings >> > for myself and would dislike to see it becoming a default setting. >> > >> > -- >> > Fred H?bert >> > http://www.erlang-solutions.com >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf.wiger@REDACTED Thu Aug 4 09:48:32 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 4 Aug 2011 09:48:32 +0200 Subject: [erlang-questions] ranged lookup on ordered_set ETS In-Reply-To: <76FD639B-FEA8-4BEB-93A6-54F0E9849796@lightpole.net> References: <76FD639B-FEA8-4BEB-93A6-54F0E9849796@lightpole.net> Message-ID: <2B688541-2AFB-488E-A2E0-3EDFEB68B70A@erlang-solutions.com> On 4 Aug 2011, at 08:56, caox wrote: > Hi > According to ETS reference, records in ordered_set ETS are stored in the Erlang Term order of their key. So, is there a efficient way to do a ranged lookup of key on this kind of ETS without scanning the whole table. For example, 100 < Key <105. > I guess ets:select/2 doesn't fit for the job. If you arrange the key so that you can call ets:select/2 with the first part of the key bound, then it will only scan the subset that matches the bound part. E.g. if you have a table where the key is {Document, Chapter}, and call ets:select(Docs, [{{{Document,'$1'}, '$2'}, [], [{{'$1','$2'}}]}]). Your search time will be proportional to the number of chapters in the book, modulo the normal O factor of ordered_set tables. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From desired.mta@REDACTED Thu Aug 4 09:50:48 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Thu, 4 Aug 2011 10:50:48 +0300 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> References: <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> Message-ID: <20110804075048.GA6197@jakstys.lt> On Fri, Jul 29, 2011 at 11:45:21AM +1200, Richard O'Keefe wrote: > One of the things criticised in the blog entry that we've been responding to was > that > {ok,Foo} = bar(...), > {ok,Foo} = ugh(...) > is too easy to write (when you really meant, say, Foo0, Foo1). I usually remember all variable names in a function I am working on, because functions are small (and easy to skim through in 0.3 sec). Don't write functions with more than 15 to 20 lines of code. Split large function into several smaller ones. Don't solve the problem by writing long lines, remember? http://www.erlang.se/doc/programming_rules.shtml#REF32141 This advice solved this problem for me. I re-use bound variables for crash early reason, therefore would not like for anything to change in the current behaviour. Motiejus From caox@REDACTED Thu Aug 4 10:18:22 2011 From: caox@REDACTED (caox) Date: Thu, 4 Aug 2011 16:18:22 +0800 Subject: [erlang-questions] ranged lookup on ordered_set ETS In-Reply-To: <2B688541-2AFB-488E-A2E0-3EDFEB68B70A@erlang-solutions.com> References: <76FD639B-FEA8-4BEB-93A6-54F0E9849796@lightpole.net> <2B688541-2AFB-488E-A2E0-3EDFEB68B70A@erlang-solutions.com> Message-ID: <1A463321-8AD4-468F-A142-0B47DED9CCE7@lightpole.net> Thanks. You mean in the case 100 > On 4 Aug 2011, at 08:56, caox wrote: > >> Hi >> According to ETS reference, records in ordered_set ETS are stored in the Erlang Term order of their key. So, is there a efficient way to do a ranged lookup of key on this kind of ETS without scanning the whole table. For example, 100 < Key <105. >> I guess ets:select/2 doesn't fit for the job. > > If you arrange the key so that you can call ets:select/2 with the first part of the key bound, then it will only scan the subset that matches the bound part. > > E.g. if you have a table where the key is {Document, Chapter}, and call > > ets:select(Docs, [{{{Document,'$1'}, '$2'}, [], [{{'$1','$2'}}]}]). > > Your search time will be proportional to the number of chapters in the book, modulo the normal O factor of ordered_set tables. > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > From tuxofil@REDACTED Thu Aug 4 11:55:05 2011 From: tuxofil@REDACTED (Alex Morarash) Date: Thu, 4 Aug 2011 12:55:05 +0300 Subject: [erlang-questions] Mistake in 'timer:now_diff/2' documentation Message-ID: Hi all! I have noticed some diversion in R14B03 doc for timer module. In R14B02: > now_diff(T2, T1) -> Tdiff > Types: > T1 = T2 = {MegaSecs, Secs, MicroSecs} > Tdiff = MegaSecs = Secs = MicroSecs = integer() > Calculates the time difference Tdiff = T2 - T1 in microseconds, where T1 and T2 probably are timestamp tuples returned from erlang:now/0. In R14B03: > now_diff(T1, T2) -> Tdiff > Types: > T1 = T2 = calendar:t_now() > Tdiff = integer() > In microseconds > Calculates the time difference Tdiff = T2 - T1 in microseconds, where T1 and T2 probably are timestamp tuples returned from erlang:now/0. According to R14B03 documentation, expression > timer:now_diff({0, 0, 1}, {0, 0, 2}). must return 1, but it returns -1. Regards, Aleksey Morarash, e-mail: tuxofil at gmail.com From watson.timothy@REDACTED Thu Aug 4 11:55:23 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 4 Aug 2011 10:55:23 +0100 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: Message-ID: > In normal code, however, I use this very sparingly. Then again, I > seldom experience this as a problem. Thinking back, most of the times > I've been bitten by this is probably in case clauses, especially when > adding code. Typically, when using canonical variables like Pid, > Reason or Result without checking that the code isn't in a branch > which already uses it. And since I agree with Robert that case and > receive should not cause a warning, I don't really feel the need for > it. > I suspect most experienced Erlang developers don't get bitten by this. Going back to the blog post that was the genesis of the original thread, the person in question (whose name I've forgotten) was complaining about this because the match fails at runtime. I think it's just an example of someone whose not used to single assignment and finds that they don't like it in practise. I would qualify that statement by pointing out the poster also went on to actually suggest that single assignment is bad. I don't agree but each to their own. From watson.timothy@REDACTED Thu Aug 4 12:22:06 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 4 Aug 2011 11:22:06 +0100 Subject: [erlang-questions] Best approach to creating a process/connection pool In-Reply-To: References: Message-ID: Take a look at the sources for http://zotonic.com/ as they use a decent connection pool. On 4 August 2011 08:01, Andrew Berman wrote: > I'm fairly new to Erlang and I am trying to write a process pool which > essentially is a connection pool to a database. ?I have designed the > application like this: ?The main pool gen_server has a queue of PIDs > of gen_fsm's. ?Each gen_fsm calls spawn_links a 3rd party gen_server and carries > the state of the connection. ?So the 3rd party gen_server and the > gen_fsm should always live and die together. ?I currently have set it > up such that gen_fsm calls proc_lib:spawn_link() to start the 3rd > party gen_server, but I'm wondering how this factors into the > supervision tree. ?I currently have two supervisors: > > Sup1 -> one_for_all -> 2 children: pool gen_server and Sup2 > Sup2 -> simple_one_for_one -> 1 child: gen_fsm > > So my question is: is it ok to use spawn_link instead of somehow > trying to put the 3rd party gen_server into the supervision tree? ?If > it should go into the supervision tree, how would I organize the tree > then such that the gen_fsm and 3rd party gen_server are reliant on > each other? > > Thanks for any help! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From nick@REDACTED Thu Aug 4 13:01:13 2011 From: nick@REDACTED (Niclas Eklund) Date: Thu, 4 Aug 2011 13:01:13 +0200 Subject: [erlang-questions] Mistake in 'timer:now_diff/2' documentation In-Reply-To: References: Message-ID: Hello! Thanks for reporting this! Should be now_diff(T2, T1) for R14B03 as well. Best regards, Niclas @ Erlang/OTP On Thu, 4 Aug 2011, Alex Morarash wrote: > Hi all! > > I have noticed some diversion in R14B03 doc for timer module. > > In R14B02: > >> now_diff(T2, T1) -> Tdiff >> Types: >> T1 = T2 = {MegaSecs, Secs, MicroSecs} >> Tdiff = MegaSecs = Secs = MicroSecs = integer() >> Calculates the time difference Tdiff = T2 - T1 in microseconds, where T1 and T2 probably are timestamp tuples returned from erlang:now/0. > > In R14B03: > >> now_diff(T1, T2) -> Tdiff >> Types: >> T1 = T2 = calendar:t_now() >> Tdiff = integer() >> In microseconds >> Calculates the time difference Tdiff = T2 - T1 in microseconds, where T1 and T2 probably are timestamp tuples returned from erlang:now/0. > > According to R14B03 documentation, expression >> timer:now_diff({0, 0, 1}, {0, 0, 2}). > must return 1, but it returns -1. > > Regards, > Aleksey Morarash, > e-mail: tuxofil at gmail.com > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf.wiger@REDACTED Thu Aug 4 14:07:59 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 4 Aug 2011 14:07:59 +0200 Subject: [erlang-questions] ranged lookup on ordered_set ETS In-Reply-To: <1A463321-8AD4-468F-A142-0B47DED9CCE7@lightpole.net> References: <76FD639B-FEA8-4BEB-93A6-54F0E9849796@lightpole.net> <2B688541-2AFB-488E-A2E0-3EDFEB68B70A@erlang-solutions.com> <1A463321-8AD4-468F-A142-0B47DED9CCE7@lightpole.net> Message-ID: <557BDC5E-E1B7-4F97-BED8-608CC3A86C79@erlang-solutions.com> Yeah, unfortunately, it is not smart enough to know from guard expressions that it could ignore an entire subtree, so a pattern like [{{{'$1', '_'}, '_'}, [{'and', {'>', '$1', 9999}, {'<', '$1', 10001}}], ['$_']}] will always scan the entire table. It seems as if this would be possible to fix, but that means digging into the PAM (Patrik's Abstract Machine), and I gather even Patrik is reluctant to do that. ;-) BR, Ulf W On 4 Aug 2011, at 10:18, caox wrote: > Thanks. > > You mean in the case 100 {100, Extra}, {106, Extra}, {110, Extra} or sth like that. And then match_spec like [{100, '$1'}, [], []} can be applied to do the match job. > > So the ordered_set table is only for the case that the order of search result is significant, but not for ranged searching. > > ? 2011-8-4???3:48? Ulf Wiger ??? > >> >> On 4 Aug 2011, at 08:56, caox wrote: >> >>> Hi >>> According to ETS reference, records in ordered_set ETS are stored in the Erlang Term order of their key. So, is there a efficient way to do a ranged lookup of key on this kind of ETS without scanning the whole table. For example, 100 < Key <105. >>> I guess ets:select/2 doesn't fit for the job. >> >> If you arrange the key so that you can call ets:select/2 with the first part of the key bound, then it will only scan the subset that matches the bound part. >> >> E.g. if you have a table where the key is {Document, Chapter}, and call >> >> ets:select(Docs, [{{{Document,'$1'}, '$2'}, [], [{{'$1','$2'}}]}]). >> >> Your search time will be proportional to the number of chapters in the book, modulo the normal O factor of ordered_set tables. >> >> BR, >> Ulf W >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From karol.skocik@REDACTED Thu Aug 4 15:46:06 2011 From: karol.skocik@REDACTED (karol skocik) Date: Thu, 4 Aug 2011 15:46:06 +0200 Subject: [erlang-questions] mnesia:subscribe on remote table Message-ID: Hi, I have just found out, that I can't mnesia:subscribe to table which does not have a replica on the caller's node. I am getting: (slave1@REDACTED)3> mnesia:subscribe({table, 'waiting_task.type.1', simple}). {error,{not_active_local,'waiting_task.type.1'}} Or, am I mistaken, is it possible? How? I couldn't find this restriction in the manual. Thanks, Karol From alavrik@REDACTED Thu Aug 4 15:59:23 2011 From: alavrik@REDACTED (Anton Lavrik) Date: Thu, 4 Aug 2011 08:59:23 -0500 Subject: [erlang-questions] Best approach to creating a process/connection pool In-Reply-To: References: Message-ID: You may want to take a look at Poolboy, it looks pretty good: https://github.com/devinus/poolboy It includes an example of pgsql connection pool. Anton On Thu, Aug 4, 2011 at 2:01 AM, Andrew Berman wrote: > I'm fairly new to Erlang and I am trying to write a process pool which > essentially is a connection pool to a database. I have designed the > application like this: The main pool gen_server has a queue of PIDs > of gen_fsm's. Each gen_fsm calls spawn_links a 3rd party gen_server and > carries > the state of the connection. So the 3rd party gen_server and the > gen_fsm should always live and die together. I currently have set it > up such that gen_fsm calls proc_lib:spawn_link() to start the 3rd > party gen_server, but I'm wondering how this factors into the > supervision tree. I currently have two supervisors: > > Sup1 -> one_for_all -> 2 children: pool gen_server and Sup2 > Sup2 -> simple_one_for_one -> 1 child: gen_fsm > > So my question is: is it ok to use spawn_link instead of somehow > trying to put the 3rd party gen_server into the supervision tree? If > it should go into the supervision tree, how would I organize the tree > then such that the gen_fsm and 3rd party gen_server are reliant on > each other? > > Thanks for any help! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hm@REDACTED Thu Aug 4 16:23:10 2011 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Thu, 4 Aug 2011 16:23:10 +0200 Subject: [erlang-questions] mnesia:subscribe on remote table In-Reply-To: References: Message-ID: Hi! mnesia:subscribe/1 allows you only to subscribe on *local* events in Mnesia, even though the manual does not spell out that restriction. Table events are generated when the local replica is updated. /H?kan On Thu, Aug 4, 2011 at 3:46 PM, karol skocik wrote: > Hi, > ?I have just found out, that I can't mnesia:subscribe to table which > does not have a replica on the caller's node. > I am getting: > > (slave1@REDACTED)3> mnesia:subscribe({table, 'waiting_task.type.1', simple}). > {error,{not_active_local,'waiting_task.type.1'}} > > Or, am I mistaken, is it possible? How? I couldn't find this > restriction in the manual. > > Thanks, > ?Karol From robert.virding@REDACTED Thu Aug 4 17:37:20 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 04 Aug 2011 16:37:20 +0100 (BST) Subject: [erlang-questions] ranged lookup on ordered_set ETS In-Reply-To: <557BDC5E-E1B7-4F97-BED8-608CC3A86C79@erlang-solutions.com> Message-ID: <2757b696-b236-4dd0-afc4-5e59fbd5695d@localhost> Yes, using ets:select would be the most beautiful way, but as Ulf says giving the key as a "variable" will scan the whole table. Another way of doing it is to use ets:next. If your key range LowerLimit < Key < UpperLimit you can step through the keys using ets:next. Something like: K1 = ets:next(Table, LowerLimit), [V1] = ets:lookup(Table, K1), K2 = ets:next(Table, K1), [V2] = ets:lookup(Table, K2), ... until you hit the UpperLimit. In a nice loop of course. This works *even if LowerLimit is not a key in the table*. It means that all lookups will be fast as you use the key and you never scan the table, but you have to do the looping yourself which will slow it down. If the table is big it should be better. Robert ----- Original Message ----- > > Yeah, unfortunately, it is not smart enough to know from guard > expressions that it could ignore an entire subtree, so a pattern > like [{{{'$1', '_'}, '_'}, [{'and', {'>', '$1', 9999}, {'<', '$1', > 10001}}], ['$_']}] will always scan the entire table. > > It seems as if this would be possible to fix, but that means digging > into the PAM (Patrik's Abstract Machine), and I gather even Patrik > is reluctant to do that. ;-) > > BR, > Ulf W > > On 4 Aug 2011, at 10:18, caox wrote: > > > Thanks. > > > > You mean in the case 100 > format of > > {100, Extra}, {106, Extra}, {110, Extra} or sth like that. And then > > match_spec like [{100, '$1'}, [], []} can be applied to do the > > match job. > > > > So the ordered_set table is only for the case that the order of > > search result is significant, but not for ranged searching. > > > > ? 2011-8-4???3:48? Ulf Wiger ??? > > > >> > >> On 4 Aug 2011, at 08:56, caox wrote: > >> > >>> Hi > >>> According to ETS reference, records in ordered_set ETS are > >>> stored in the Erlang Term order of their key. So, is there a > >>> efficient way to do a ranged lookup of key on this kind of ETS > >>> without scanning the whole table. For example, 100 < Key > >>> <105. > >>> I guess ets:select/2 doesn't fit for the job. > >> > >> If you arrange the key so that you can call ets:select/2 with the > >> first part of the key bound, then it will only scan the subset > >> that matches the bound part. > >> > >> E.g. if you have a table where the key is {Document, Chapter}, and > >> call > >> > >> ets:select(Docs, [{{{Document,'$1'}, '$2'}, [], > >> [{{'$1','$2'}}]}]). > >> > >> Your search time will be proportional to the number of chapters in > >> the book, modulo the normal O factor of ordered_set tables. > >> > >> BR, > >> Ulf W > >> > >> Ulf Wiger, CTO, Erlang Solutions, Ltd. > >> http://erlang-solutions.com > >> > >> > >> > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From rexxe98@REDACTED Thu Aug 4 18:35:47 2011 From: rexxe98@REDACTED (Andrew Berman) Date: Thu, 4 Aug 2011 09:35:47 -0700 Subject: [erlang-questions] Best approach to creating a process/connection pool In-Reply-To: References: Message-ID: Thanks Tim. So it looks like they are using erlang:monitor and demonitor, so is that the best approach in an OTP application? How does it figure into the supervision tree? On Thu, Aug 4, 2011 at 3:22 AM, Tim Watson wrote: > Take a look at the sources for http://zotonic.com/ as they use a > decent connection pool. > > On 4 August 2011 08:01, Andrew Berman wrote: >> I'm fairly new to Erlang and I am trying to write a process pool which >> essentially is a connection pool to a database. ?I have designed the >> application like this: ?The main pool gen_server has a queue of PIDs >> of gen_fsm's. ?Each gen_fsm calls spawn_links a 3rd party gen_server and carries >> the state of the connection. ?So the 3rd party gen_server and the >> gen_fsm should always live and die together. ?I currently have set it >> up such that gen_fsm calls proc_lib:spawn_link() to start the 3rd >> party gen_server, but I'm wondering how this factors into the >> supervision tree. ?I currently have two supervisors: >> >> Sup1 -> one_for_all -> 2 children: pool gen_server and Sup2 >> Sup2 -> simple_one_for_one -> 1 child: gen_fsm >> >> So my question is: is it ok to use spawn_link instead of somehow >> trying to put the 3rd party gen_server into the supervision tree? ?If >> it should go into the supervision tree, how would I organize the tree >> then such that the gen_fsm and 3rd party gen_server are reliant on >> each other? >> >> Thanks for any help! >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From hynek@REDACTED Thu Aug 4 19:39:42 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Thu, 4 Aug 2011 19:39:42 +0200 Subject: [erlang-questions] Best approach to creating a process/connection pool In-Reply-To: References: Message-ID: Using OTP supervisor tree is intended only for control of restarting application parts. If you want anything else you have to use links or monitors unless you write your own supervisor which is not recommended way. On Thu, Aug 4, 2011 at 6:35 PM, Andrew Berman wrote: > Thanks Tim. ?So it looks like they are using erlang:monitor and > demonitor, so is that the best approach in an OTP application? ?How > does it figure into the supervision tree? > > > > On Thu, Aug 4, 2011 at 3:22 AM, Tim Watson wrote: >> Take a look at the sources for http://zotonic.com/ as they use a >> decent connection pool. >> >> On 4 August 2011 08:01, Andrew Berman wrote: >>> I'm fairly new to Erlang and I am trying to write a process pool which >>> essentially is a connection pool to a database. ?I have designed the >>> application like this: ?The main pool gen_server has a queue of PIDs >>> of gen_fsm's. ?Each gen_fsm calls spawn_links a 3rd party gen_server and carries >>> the state of the connection. ?So the 3rd party gen_server and the >>> gen_fsm should always live and die together. ?I currently have set it >>> up such that gen_fsm calls proc_lib:spawn_link() to start the 3rd >>> party gen_server, but I'm wondering how this factors into the >>> supervision tree. ?I currently have two supervisors: >>> >>> Sup1 -> one_for_all -> 2 children: pool gen_server and Sup2 >>> Sup2 -> simple_one_for_one -> 1 child: gen_fsm >>> >>> So my question is: is it ok to use spawn_link instead of somehow >>> trying to put the 3rd party gen_server into the supervision tree? ?If >>> it should go into the supervision tree, how would I organize the tree >>> then such that the gen_fsm and 3rd party gen_server are reliant on >>> each other? >>> >>> Thanks for any help! >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From fred.hebert@REDACTED Thu Aug 4 19:42:32 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Thu, 4 Aug 2011 13:42:32 -0400 Subject: [erlang-questions] Best approach to creating a process/connection pool In-Reply-To: References: Message-ID: Technically, the OTP supervision tree is also used for application upgrades (appups & relups). When sending messages to do the broad reloading of code, the code in charge of that just traverses the VM's supervision trees asking the supervisors what kind of modules they have and whether they're workers and supervisors (that's why the child specs contain details about workers, supervisors, [Modules] (and dynamic sending special messages to figure out the same info). If you want to use appups/relups, you have to use the OTP supervision trees otherwise you will have processes left behind to be purged rather than upgraded. -- Fred H?bert http://www.erlang-solutions.com On 2011-08-04, at 13:39 PM, Hynek Vychodil wrote: > Using OTP supervisor tree is intended only for control of restarting > application parts. If you want anything else you have to use links or > monitors unless you write your own supervisor which is not recommended > way. > > On Thu, Aug 4, 2011 at 6:35 PM, Andrew Berman wrote: >> Thanks Tim. So it looks like they are using erlang:monitor and >> demonitor, so is that the best approach in an OTP application? How >> does it figure into the supervision tree? >> >> >> >> On Thu, Aug 4, 2011 at 3:22 AM, Tim Watson wrote: >>> Take a look at the sources for http://zotonic.com/ as they use a >>> decent connection pool. >>> >>> On 4 August 2011 08:01, Andrew Berman wrote: >>>> I'm fairly new to Erlang and I am trying to write a process pool which >>>> essentially is a connection pool to a database. I have designed the >>>> application like this: The main pool gen_server has a queue of PIDs >>>> of gen_fsm's. Each gen_fsm calls spawn_links a 3rd party gen_server and carries >>>> the state of the connection. So the 3rd party gen_server and the >>>> gen_fsm should always live and die together. I currently have set it >>>> up such that gen_fsm calls proc_lib:spawn_link() to start the 3rd >>>> party gen_server, but I'm wondering how this factors into the >>>> supervision tree. I currently have two supervisors: >>>> >>>> Sup1 -> one_for_all -> 2 children: pool gen_server and Sup2 >>>> Sup2 -> simple_one_for_one -> 1 child: gen_fsm >>>> >>>> So my question is: is it ok to use spawn_link instead of somehow >>>> trying to put the 3rd party gen_server into the supervision tree? If >>>> it should go into the supervision tree, how would I organize the tree >>>> then such that the gen_fsm and 3rd party gen_server are reliant on >>>> each other? >>>> >>>> Thanks for any help! >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rexxe98@REDACTED Thu Aug 4 23:20:28 2011 From: rexxe98@REDACTED (Andrew Berman) Date: Thu, 4 Aug 2011 14:20:28 -0700 Subject: [erlang-questions] Best approach to creating a process/connection pool In-Reply-To: References: Message-ID: Awesome! This makes sense. Thank you for the clarification on this. 2011/8/4 Fr?d?ric Trottier-H?bert : > Technically, the OTP supervision tree is also used for application upgrades (appups & relups). When sending messages to do the broad reloading of code, the code in charge of that just traverses the VM's supervision trees asking the supervisors what kind of modules they have and whether they're workers and supervisors (that's why the child specs contain details about workers, supervisors, [Modules] (and dynamic sending special messages to figure out the same info). > > If you want to use appups/relups, you have to use the OTP supervision trees otherwise you will have processes left behind to be purged rather than upgraded. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-08-04, at 13:39 PM, Hynek Vychodil wrote: > >> Using OTP supervisor tree is intended only for control of restarting >> application parts. If you want anything else you have to use links or >> monitors unless you write your own supervisor which is not recommended >> way. >> >> On Thu, Aug 4, 2011 at 6:35 PM, Andrew Berman wrote: >>> Thanks Tim. ?So it looks like they are using erlang:monitor and >>> demonitor, so is that the best approach in an OTP application? ?How >>> does it figure into the supervision tree? >>> >>> >>> >>> On Thu, Aug 4, 2011 at 3:22 AM, Tim Watson wrote: >>>> Take a look at the sources for http://zotonic.com/ as they use a >>>> decent connection pool. >>>> >>>> On 4 August 2011 08:01, Andrew Berman wrote: >>>>> I'm fairly new to Erlang and I am trying to write a process pool which >>>>> essentially is a connection pool to a database. ?I have designed the >>>>> application like this: ?The main pool gen_server has a queue of PIDs >>>>> of gen_fsm's. ?Each gen_fsm calls spawn_links a 3rd party gen_server and carries >>>>> the state of the connection. ?So the 3rd party gen_server and the >>>>> gen_fsm should always live and die together. ?I currently have set it >>>>> up such that gen_fsm calls proc_lib:spawn_link() to start the 3rd >>>>> party gen_server, but I'm wondering how this factors into the >>>>> supervision tree. ?I currently have two supervisors: >>>>> >>>>> Sup1 -> one_for_all -> 2 children: pool gen_server and Sup2 >>>>> Sup2 -> simple_one_for_one -> 1 child: gen_fsm >>>>> >>>>> So my question is: is it ok to use spawn_link instead of somehow >>>>> trying to put the 3rd party gen_server into the supervision tree? ?If >>>>> it should go into the supervision tree, how would I organize the tree >>>>> then such that the gen_fsm and 3rd party gen_server are reliant on >>>>> each other? >>>>> >>>>> Thanks for any help! >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> >> -- >> --Hynek (Pichi) Vychodil >> >> Analyze your data in minutes. Share your insights instantly. Thrill >> your boss. ?Be a data hero! >> Try GoodData now for free: www.gooddata.com >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From ok@REDACTED Fri Aug 5 00:29:55 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 5 Aug 2011 10:29:55 +1200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: Message-ID: On 4/08/2011, at 9:55 PM, Tim Watson wrote: > I suspect most experienced Erlang developers don't get bitten by this. > Going back to the blog post that was the genesis of the original > thread, the person in question (whose name I've forgotten) was > complaining about this because the match fails at runtime. I think > it's just an example of someone whose not used to single assignment > and finds that they don't like it in practise. I would qualify that > statement by pointing out the poster also went on to actually suggest > that single assignment is bad. I don't agree but each to their own. The person in question was Tony Arcieri, who built Reia (see http://reia-lang.org/). There are three things that seem obvious about Tony Arcieri: (1) He likes Ruby. (Nobody who didn't would want to build something like Reia.) (2) He knows rather a lot about Erlang. (Nobody who didn't *could* build something like Reia.) (3) He is a skilled programmer. (Nobody who wasn't could get something like Reia working and out there -- as long as directories don't include dots in their names...) It is important to understand the distinction that Tony Arcieri correctly makes between (A) single assignment in the *semantics* (data are immutable), and (B) single assignment in the *syntax* (variables can be assigned only once). The point he makes is that you can have (A), which he seems to be happy with, without requiring (B), which he is not: you can treat re*assignment* as re*binding*. Thus in X = f(), X = g(X), X = h(X) each occurrence of X on the left would be a new binding, equivalent to X0 = f(), X1 = f(X0), X2 = h(X1) but without requiring the programmer to keep track of numeric suffixes. There's a classic paper showing that the imperative programming language Euclid can actually be viewed as a functional programming language using such renaming tricks (and taking advantage of the fact that procedure interfaces had to specify _all_ the variables that could be used (extra inputs) and set (extra outputs) and the aliasing rules that meant that in-place mutation could not be detected). This _does_ include loops. A language treating assignment as rebinding could look more comfortable to an ex-imperative programmer without actually sacrificing any of the virtues of Erlang. It's not a change that could be made to Erlang by now, but it certainly would not need a new VM, and it would not introduce any failure modes that are not now present. I strongly criticised a recent suggestion of Tony Arcieri's, but while I personally do not feel a need for conventional-seeming variables in Erlang code, there is much good sense in his position on single assignment. I From jws@REDACTED Fri Aug 5 04:22:14 2011 From: jws@REDACTED (Jeff Schultz) Date: Fri, 5 Aug 2011 12:22:14 +1000 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <4E39C1E1.7070301@gmail.com> References: <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> <1312383957.26732.YahooMailNeo@web111412.mail.gq1.yahoo.com> <4E3974A3.6070104@gmail.com> <4E39C1E1.7070301@gmail.com> Message-ID: <20110805022214.GA29400@mulga.csse.unimelb.edu.au> On Wed, Aug 03, 2011 at 11:47:13PM +0200, Richard Carlsson wrote: >> According to the Barklund spec (page 62), it said that "a compiler >> must only accept a program if for any evaluation order, there will >> not be an applied occurrence of an unbound variable". >> >> "For example, in a context where X is unbound, the expression (X=8) + >> X should give a compile-time error." >> >> (And it does, too. I checked). Thanks for pointing that out. I'd have missed that one. > An interesting twist on the example (X=8) + X is that if *both* arguments > create a binding of the same variable, as in: > > (X=8) + (X=8) > > then Erlang will first evaluate the subexpressions, then *check* that the > results match so that a single agreed value of X can be exported out from > the whole expression, and only in that case will it call + on the results. > You can experiment with the following to see what I mean: The difference between the two cases seems a bit weird to me. I would have expected that the meaning of a call f(Expr1, Expr2) (I'll write it as |f(Expr1, Expr2)|) where the Expr are other Erlang expressions should be exactly the same as |T1 = Expr1|, |T2 = Expr2|, f(T1, T2) where the T are new variables and |.| is applied recursively. In this case, |(X = 8) + X| means X = 8, T1 = X, T2 = X, T1 + T2, which is okay, and |X + (X = 8)| means T1 = X, X = 8, T2 = X, T1 + T2, which is a compile-time error. While I appreciate the intent of Barklund's rule above, I don't think it plays well with Erlang's explicit left-to-right order of evaluation. Erlang really could do with a concise, agreed, current specification, preferrably executable. Jeff Schultz From ok@REDACTED Fri Aug 5 04:55:38 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 5 Aug 2011 14:55:38 +1200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <20110804075048.GA6197@jakstys.lt> References: <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> <20110804075048.GA6197@jakstys.lt> Message-ID: <4C122826-7B2B-4E85-A16B-AC3FC411B05D@cs.otago.ac.nz> On 4/08/2011, at 7:50 PM, Motiejus Jak?tys wrote: > On Fri, Jul 29, 2011 at 11:45:21AM +1200, Richard O'Keefe wrote: >> One of the things criticised in the blog entry that we've been responding to was >> that >> {ok,Foo} = bar(...), >> {ok,Foo} = ugh(...) >> is too easy to write (when you really meant, say, Foo0, Foo1). Please remember, people, that this wasn't *my* complaint, but a complaint in the original "Erlang is a Ghetto" blog. I was wondering whether we could do anything useful to help people who _do_ tend to make that kind of mistake. > > I usually remember all variable names in a function I am working on, > because functions are small (and easy to skim through in 0.3 sec). I don't make that kind of mistake much myself. However, I wouldn't take that "functions are small" for granted. > > Don't write functions with more than 15 to 20 lines of code. I took the Erlang/OTP R12B-5 *.erl files and ran them through a filter that removes comments. Blank lines were not counted. A line beginning with ['a-z] was taken as beginning a clause, and a line ending with [.] was taken as ending a function. > summary(clauses.per.function) Min. 1st Qu. Median Mean 3rd Qu. Max. 1.000 1.000 1.000 2.297 2.000 495.000 > summary(lines.per.clause) Min. 1st Qu. Median Mean 3rd Qu. Max. 1.000 2.000 2.000 3.998 4.000 797.000 > summary(lines.per.function) Min. 1st Qu. Median Mean 3rd Qu. Max. 1.000 2.000 5.000 9.184 10.000 2414.000 So yes, the mean is somewhere around 10. However, 8.8% of functions in R12B-5 were over 20 lines, and 2.7% were over 40 lines. 0.6% were over 100 lines. The "whopper" at 2,414 lines is admittedly extreme; the next smallest function is "only" 1,072 lines. > Split large > function into several smaller ones. Don't solve the problem by writing > long lines, remember? > http://www.erlang.se/doc/programming_rules.shtml#REF32141 Oh yes, line length. > summary(columns.per.nonempty.line) Min. 1st Qu. Median Mean 3rd Qu. Max. 1.00 24.00 39.00 40.07 54.00 1515.00 If you look at the source code of Reia, which I have, you may come away with the same impression I did, which is that Tony Arcieri writes pretty clean code that is no hardship to read. He tends to spread things out vertically more than I do, but I have to say that it's tidy and readable. The complaint about unintentionally writing a variable name more than once did NOT come from someone who doesn't know how to write clean code. You will also notice pretty quickly that some functions just *can't* be kept below 21 lines: if you have more than 20 cases to deal with, you are going to need more than 20 lines to do it. And a compiler runs into that kind of problem quite often. Here are the summaries for Reia: > summary(clauses.per.function) Min. 1st Qu. Median Mean 3rd Qu. Max. 1.000 1.000 1.000 2.542 2.000 105.000 > summary(lines.per.clause) Min. 1st Qu. Median Mean 3rd Qu. Max. 1.000 2.000 2.000 3.666 3.000 35.000 > summary(lines.per.function) Min. 1st Qu. Median Mean 3rd Qu. Max. 1.000 2.000 5.000 9.316 8.000 229.000 Since the scope of variables is a clause, not a function, the size we should be concerned with is clause size, and we see that each and every one of the clauses in Reia fits well within an edit window on a modern machine. (The big function is one which has, for good reason, lots of clauses; the individual clauses are not particularly large.) You may recall a recent thread in which I was not particularly kind to an Erlang "enhancement" suggestion of Arcieri's. I think that was a bad idea, but nobody looking at his code could accuse him of ignorance of Erlang or incompetence as a programmer. If he says that he runs into unintentionally-repeated-variable-names often enough to be a nuisance, then this is a problem that a *GOOD* programmer can have in Erlang, even if you or I do not. I gave my second lecture about static checking this morning, and spent some time explaining the importance of turning off inappropriate warnings so that you can see the real problems. I gave this example from my Smalltalk-to-C compiler's run-time library: #define ik_bitShift(x, y) \ (AREINTS(x, y) && -y < ASINT(INT_BITS) \ ? (Word)((Int)(x) >> (int)(((-y) >> 2)) &~ (ASINT(1)-1)) \ : k_bitShift(x, y)) and the code that gets generated t4 = ik_bitShift(l1 /*a*/, ASINT(24)); producing warnings like "foobar.c", line 229490: warning: shift count negative or too big: >> 1073741800 from Sun cc, gcc-4.5, and clang. These compilers are smart enough to see that the shift count is out of range, but so dumb that they pay no attention whatsoever to the preceding test which ensures that the shift in question can never under any circumstances be executed. I get hundreds of lines of worse-than-useless warnings, which I cannot switch off, because these compiler writers were so arrogantly cock-sure that THEY would never be mistaken about what was probably wrong. For efficiency the test should probably be special-cased by the Smalltalk compiler rather than the C compiler, but I want to make that decision when it rises to the top of the TODO list naturally, not because half-smart compilers make my life a misery if I don't. I really don't want to get in the way of programmers who know what they are doing and don't have a particular problem, but that's why I suggested being able to switch off warnings selectively. From watson.timothy@REDACTED Fri Aug 5 08:25:49 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 5 Aug 2011 07:25:49 +0100 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: Message-ID: On 4 August 2011 23:29, Richard O'Keefe wrote: > The person in question was Tony Arcieri, who built Reia Wasn't paying attention to the details. Although I haven't used Reia, I am aware of it and it is indeed very a impressive thing to have built. > > It is important to understand the distinction that Tony Arcieri > correctly makes between > (A) single assignment in the *semantics* (data are immutable), and > (B) single assignment in the *syntax* (variables can be assigned > ? ?only once). Thanks for clearing that up - I confess I didn't notice the distinction but on second reading you're quite right. > The point he makes is that you can have (A), which he seems to be > happy with, without requiring (B), which he is not: you can treat > re*assignment* as re*binding*. ?Thus in > > ? ? ? ?X = f(), > ? ? ? ?X = g(X), > ? ? ? ?X = h(X) > I must confess I find that very confusing. I have written plenty of imperative code over the years but it just feels wrong to keep reassigning like that even in Ruby. Given that Ruby is Object Oriented as well as imperative, I'd expect to see something more like `f.g.h` perhaps with parens in later versions where they seem to be required once again. If the code is not doing anything with these intermediate variables, why type them in the first place? After all, `h(f(f()))` is actually more concise than the alternative, and rubyists do seem to like conciseness (although I'm being a little flippant, as this doesn't mean they all prefer inlining). When I can make function/method parameters and/or local variables immutable, readonly, `final`, or whatever syntax is available to me, I do. I find this delegates checking for a common class of errors to the compiler. In Erlang I might use `{ok, X} = f(), {still_ok, X} = g(X)` as a deliberate check from time to time. From desired.mta@REDACTED Fri Aug 5 08:35:31 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Fri, 5 Aug 2011 09:35:31 +0300 Subject: [erlang-questions] exprecs installation failure Message-ID: <20110805063531.GA20905@jakstys.lt> Hello list, I encountered this problem and cannot solve it: http://erlang.org/pipermail/erlang-questions/2009-January/040926.html My compilation fails like this: $ erlc exprecs.erl ./exprecs.erl:249: Warning: regexp:split/2: the regexp module is deprecated (will be removed in R15A); use the re module instead $ ls -lh -rw-r--r-- 1 motiejus motiejus 14K 2011-08-05 09:31 exprecs.beam -rwxr-xr-x 1 motiejus motiejus 22K 2011-08-05 09:04 exprecs.erl -rw-r--r-- 1 motiejus motiejus 114 2011-08-05 09:30 struct_deserialize.erl $ erlc -pa . struct_deserialize.erl ./struct_deserialize.erl:none: error in parse transform 'exprecs': {undef, [{parse_trans,top, [#Fun, [{attribute,1,file, {"./struct_deserialize.erl",1}}, {attribute,1,module, struct_deserialize}, {attribute,4,record, {a, [{record_field,4,{atom,4,b}}, {record_field,4,{atom,4,c}}, {record_field,4,{atom,4,d}}]}}, {attribute,5,export_records,[a]}, {eof,6}], [report_warnings,report_errors, {cwd,"/tmp/project/myapp/src"}, {outdir,"/tmp/project/myapp/src"}]]}, {compile, '-foldl_transform/2-anonymous-2-',2}, {compile,foldl_transform,2}, {compile, '-internal_comp/4-anonymous-1-',2}, {compile,fold_comp,3}, {compile,internal_comp,4}, {compile,internal,3}]} $ cat struct_deserialize.erl -module(struct_deserialize). -compile({parse_transform, exprecs}). -record(a, {b, c, d}). -export_records([a]). motiejus:/tmp/project/myapp/src$ sha1sum exprecs.erl e1a7992c3ee9fcf15675bef351c05f29efcf8201 exprecs.erl exprecs.erl is from revision 24 from here: http://svn.ulf.wiger.net/parse_trans/trunk/parse_trans/examples/exprecs.erl Tried different compile parameters, with rebar and without it. Any help appreciated. Motiejus From desired.mta@REDACTED Fri Aug 5 08:44:27 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Fri, 5 Aug 2011 09:44:27 +0300 Subject: [erlang-questions] exprecs installation failure In-Reply-To: <20110805063531.GA20905@jakstys.lt> References: <20110805063531.GA20905@jakstys.lt> Message-ID: <20110805064427.GA22539@jakstys.lt> Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Motiejus On Fri, Aug 05, 2011 at 09:35:31AM +0300, Motiejus Jak?tys wrote: > Hello list, > > I encountered this problem and cannot solve it: > http://erlang.org/pipermail/erlang-questions/2009-January/040926.html > > My compilation fails like this: > $ erlc exprecs.erl > ./exprecs.erl:249: Warning: regexp:split/2: the regexp module is deprecated (will be removed in R15A); use the re module instead > $ ls -lh > -rw-r--r-- 1 motiejus motiejus 14K 2011-08-05 09:31 exprecs.beam > -rwxr-xr-x 1 motiejus motiejus 22K 2011-08-05 09:04 exprecs.erl > -rw-r--r-- 1 motiejus motiejus 114 2011-08-05 09:30 struct_deserialize.erl > $ erlc -pa . struct_deserialize.erl > ./struct_deserialize.erl:none: error in parse transform 'exprecs': {undef, > [{parse_trans,top, > [#Fun, > [{attribute,1,file, > {"./struct_deserialize.erl",1}}, > {attribute,1,module, > struct_deserialize}, > {attribute,4,record, > {a, > [{record_field,4,{atom,4,b}}, > {record_field,4,{atom,4,c}}, > {record_field,4,{atom,4,d}}]}}, > {attribute,5,export_records,[a]}, > {eof,6}], > [report_warnings,report_errors, > {cwd,"/tmp/project/myapp/src"}, > {outdir,"/tmp/project/myapp/src"}]]}, > {compile, > '-foldl_transform/2-anonymous-2-',2}, > {compile,foldl_transform,2}, > {compile, > '-internal_comp/4-anonymous-1-',2}, > {compile,fold_comp,3}, > {compile,internal_comp,4}, > {compile,internal,3}]} > $ cat struct_deserialize.erl > -module(struct_deserialize). > -compile({parse_transform, exprecs}). > > -record(a, {b, c, d}). > -export_records([a]). > motiejus:/tmp/project/myapp/src$ sha1sum exprecs.erl > e1a7992c3ee9fcf15675bef351c05f29efcf8201 exprecs.erl > > exprecs.erl is from revision 24 from here: > http://svn.ulf.wiger.net/parse_trans/trunk/parse_trans/examples/exprecs.erl > > Tried different compile parameters, with rebar and without it. > Any help appreciated. > > Motiejus From ulf.wiger@REDACTED Fri Aug 5 08:47:02 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 5 Aug 2011 08:47:02 +0200 Subject: [erlang-questions] exprecs installation failure In-Reply-To: <20110805063531.GA20905@jakstys.lt> References: <20110805063531.GA20905@jakstys.lt> Message-ID: <88CEB4F9-CCC6-49A0-ABD5-6431B3D04511@erlang-solutions.com> Hi Motiejus, What the error report says is that it can't find the parse_trans module, which exprecs relies on. Your "-pa ." option seems to take care of finding exprecs.beam, but not parse_trans.beam. In later versions of parse_trans, exprecs is in parse_trans/src, and you can even set your environment to find the parse_trans modules via ERL_LIBS. https://github.com/esl/parse_trans/tree/master/src BR, Ulf W On 5 Aug 2011, at 08:35, Motiejus Jak?tys wrote: > Hello list, > > I encountered this problem and cannot solve it: > http://erlang.org/pipermail/erlang-questions/2009-January/040926.html > > My compilation fails like this: > $ erlc exprecs.erl > ./exprecs.erl:249: Warning: regexp:split/2: the regexp module is deprecated (will be removed in R15A); use the re module instead > $ ls -lh > -rw-r--r-- 1 motiejus motiejus 14K 2011-08-05 09:31 exprecs.beam > -rwxr-xr-x 1 motiejus motiejus 22K 2011-08-05 09:04 exprecs.erl > -rw-r--r-- 1 motiejus motiejus 114 2011-08-05 09:30 struct_deserialize.erl > $ erlc -pa . struct_deserialize.erl > ./struct_deserialize.erl:none: error in parse transform 'exprecs': {undef, > [{parse_trans,top, > [#Fun, > [{attribute,1,file, > {"./struct_deserialize.erl",1}}, > {attribute,1,module, > struct_deserialize}, > {attribute,4,record, > {a, > [{record_field,4,{atom,4,b}}, > {record_field,4,{atom,4,c}}, > {record_field,4,{atom,4,d}}]}}, > {attribute,5,export_records,[a]}, > {eof,6}], > [report_warnings,report_errors, > {cwd,"/tmp/project/myapp/src"}, > {outdir,"/tmp/project/myapp/src"}]]}, > {compile, > '-foldl_transform/2-anonymous-2-',2}, > {compile,foldl_transform,2}, > {compile, > '-internal_comp/4-anonymous-1-',2}, > {compile,fold_comp,3}, > {compile,internal_comp,4}, > {compile,internal,3}]} > $ cat struct_deserialize.erl > -module(struct_deserialize). > -compile({parse_transform, exprecs}). > > -record(a, {b, c, d}). > -export_records([a]). > motiejus:/tmp/project/myapp/src$ sha1sum exprecs.erl > e1a7992c3ee9fcf15675bef351c05f29efcf8201 exprecs.erl > > exprecs.erl is from revision 24 from here: > http://svn.ulf.wiger.net/parse_trans/trunk/parse_trans/examples/exprecs.erl > > Tried different compile parameters, with rebar and without it. > Any help appreciated. > > Motiejus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Fri Aug 5 08:55:12 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Fri, 5 Aug 2011 09:55:12 +0300 Subject: [erlang-questions] exprecs installation failure In-Reply-To: <88CEB4F9-CCC6-49A0-ABD5-6431B3D04511@erlang-solutions.com> References: <20110805063531.GA20905@jakstys.lt> <88CEB4F9-CCC6-49A0-ABD5-6431B3D04511@erlang-solutions.com> Message-ID: <20110805065512.GA23359@jakstys.lt> On Fri, Aug 05, 2011 at 08:47:02AM +0200, Ulf Wiger wrote: > Hi Motiejus, > > What the error report says is that it can't find the parse_trans > module, which exprecs relies on. Your "-pa ." option seems to take > care of finding exprecs.beam, but not parse_trans.beam. > > In later versions of parse_trans, exprecs is in parse_trans/src, and > you can even set your environment to find the parse_trans modules via > ERL_LIBS. > > https://github.com/esl/parse_trans/tree/master/src > Adding parse_trans to dependencies helped. I didn't realize exprecs depends on parse_trans. Thanks! Motiejus From flaboy.cn@REDACTED Fri Aug 5 10:36:58 2011 From: flaboy.cn@REDACTED (Wanglei) Date: Fri, 5 Aug 2011 16:36:58 +0800 Subject: [erlang-questions] why auto completion not working in "erl -remsh" Message-ID: <319FA7D6-601C-4462-B589-1DC7E6DFEBD2@gmail.com> in a "-pa ebin -remsh" shell I can use test:aaa(), but can't type test:a[tab] to auto completion -> test:aaa( The auto completion only show system-libs modules "code:rehash()." can't reslove this problem -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4227 bytes Desc: not available URL: From carlsson.richard@REDACTED Fri Aug 5 10:47:29 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 05 Aug 2011 10:47:29 +0200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <20110805022214.GA29400@mulga.csse.unimelb.edu.au> References: <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> <1312383957.26732.YahooMailNeo@web111412.mail.gq1.yahoo.com> <4E3974A3.6070104@gmail.com> <4E39C1E1.7070301@gmail.com> <20110805022214.GA29400@mulga.csse.unimelb.edu.au> Message-ID: <4E3BAE21.2080903@gmail.com> On 08/05/2011 04:22 AM, Jeff Schultz wrote: > The difference between the two cases seems a bit weird to me. I would > have expected that the meaning of a call f(Expr1, Expr2) (I'll write > it as |f(Expr1, Expr2)|) where the Expr are other Erlang expressions > should be exactly the same as > > |T1 = Expr1|, > |T2 = Expr2|, > f(T1, T2) > > where the T are new variables and |.| is applied recursively. > > In this case, |(X = 8) + X| means X = 8, T1 = X, T2 = X, T1 + T2, > which is okay, and |X + (X = 8)| means T1 = X, X = 8, T2 = X, T1 + T2, > which is a compile-time error. > > While I appreciate the intent of Barklund's rule above, I don't think > it plays well with Erlang's explicit left-to-right order of > evaluation. I think you have misunderstood something - or maybe I'm missing something in your reasoning. Barklund's stated rule means that |(X = 8) + X| is just as invalid in Erlang as |X + (X = 8)|, because even though it will work in a left-to-right evaluation order (which is ultimately the order in which the arguments _will_ be executed), it will not work in _any_ evaluation order. The compiler therefore rejects it. The nice consequence is that you can always naively reorder the arguments of a function or operator call (in a valid program), because there's no possibility that one affects the variable bindings expected of another. Hence, the rule actually plays very _well_ with the explicit evaluation order: it says that even though it's tempting to think it would be good to allow this kind of variable propagation since the evaluation order is strictly left-to-right anyway, that's not a good idea, because it actually makes programs harder to reason about and harder to refactor without introducing bugs. The general linearization of f(Expr1, Expr2) can be expressed as: %% *if* Expr1 and Expr2 don't depend on each other's bindings: T1 = Expr1, T2 = Expr2, %% for all X,Y,... exported from both Expr1 and Expr2: X1 = X2, Y1 = Y2, ... % ensure they are the same f(T1, T2) (of course, the check of bindings only happens in extremely rare cases like my example f(X=8, X=8), so there's normally no overhead for this). Of course, side effects in argument expressions are a different matter. An expression like f(P ! foo, P ! bar) will _always_ cause foo to arrive before bar at P, because the evaluation order is fixed. And naively reordering arguments with side effects in them could easily screw up your program - which is why having side effects in arguments is bad style. /Richard From andrew.kondratovich@REDACTED Fri Aug 5 11:32:33 2011 From: andrew.kondratovich@REDACTED (Andrew Kondratovich) Date: Fri, 5 Aug 2011 12:32:33 +0300 Subject: [erlang-questions] about syntax tools Message-ID: Hello. How can i add some syntax expressions in language without editing source grammatics ? For example, i want to add something like list generator: [1..6] % generates [1,2,3,4,5,6] With macros, i can do something like: -define(Seq(A, B), lists:seq(A, lists:max([A, B]))). and then ?Seq(A, B). in code. Can i add my own syntax rules by syntax tools or something like that ? From carlsson.richard@REDACTED Fri Aug 5 11:38:37 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 05 Aug 2011 11:38:37 +0200 Subject: [erlang-questions] about syntax tools In-Reply-To: References: Message-ID: <4E3BBA1D.2010409@gmail.com> On 08/05/2011 11:32 AM, Andrew Kondratovich wrote: > Hello. > > How can i add some syntax expressions in language without editing > source grammatics ? > > For example, i want to add something like list generator: > [1..6] % generates [1,2,3,4,5,6] > > Can i add my own syntax rules by syntax tools or something like that ? No, you can't modify the grammar. You can however use parse transforms to change the meaning of some expressions, e.g. like mnesia query list comprehensions change the meaning of calls to qlc:q(...). See http://www.erlang.org/doc/man/compile.html for some more info. (You can use syntax_tools to help you write parse transforms, though.) /Richard From vasdeveloper@REDACTED Fri Aug 5 12:06:42 2011 From: vasdeveloper@REDACTED (Kannan) Date: Fri, 5 Aug 2011 15:36:42 +0530 Subject: [erlang-questions] IP v6 Message-ID: Hi List, I would like to know if any of you are using or heard of using Erlang in an IP v6 environment. We are planning some server migrations to IP v6 and before we start the development of a prototype, I would like to know your comments please. Regards, Kannan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Fri Aug 5 12:09:16 2011 From: vasdeveloper@REDACTED (Kannan) Date: Fri, 5 Aug 2011 15:39:16 +0530 Subject: [erlang-questions] IP v6 In-Reply-To: References: Message-ID: We will be using the latest version of Erlang/OTP and Redhat Enterprise Linux. On Fri, Aug 5, 2011 at 3:36 PM, Kannan wrote: > Hi List, > > I would like to know if any of you are using or heard of using Erlang in an > IP v6 environment. We are planning some server migrations to IP v6 and > before we start the development of a prototype, I would like to know your > comments please. > > Regards, > Kannan. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Fri Aug 5 12:22:47 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 5 Aug 2011 12:22:47 +0200 Subject: [erlang-questions] IP v6 In-Reply-To: References: Message-ID: Hello! What protocols (or applications) do you plan to use? We had issues with SNMP, SSL, CORBA, HTTP, FTP - I think SSH actually worked. Our issues have been fixed, but you'll probably need the development version of Erlang/OTP (or wait until OTP R14B04), because some of these fixes were made after R14B03 was released. 2011/8/5, Kannan : > We will be using the latest version of Erlang/OTP and Redhat Enterprise > Linux. > > > On Fri, Aug 5, 2011 at 3:36 PM, Kannan wrote: > >> Hi List, >> >> I would like to know if any of you are using or heard of using Erlang in >> an >> IP v6 environment. We are planning some server migrations to IP v6 and >> before we start the development of a prototype, I would like to know your >> comments please. >> >> Regards, >> Kannan. >> > From marc@REDACTED Fri Aug 5 14:12:10 2011 From: marc@REDACTED (Marc Worrell) Date: Fri, 5 Aug 2011 14:12:10 +0200 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: <29781133.2149.1312402781522.JavaMail.geo-discussion-forums@yqiw2> <93FADF05-1B85-4183-811A-12B6A7ADA575@srstrong.com> Message-ID: <4F8B3A97-B8D3-4641-A425-1E6030545EB5@worrell.nl> On 3 aug 2011, at 23:08, Max Lapshin wrote: > While erlyvideo was licensed as MIT I have received only 10 lines of > code from external contributors. Only 10 lines during 3 years. This is indeed a terrible return for all your efforts. With Zotonic we are more lucky, receiving many patches, complete modules, translations and documentation from the community. In your case I completely understand the GPL license. I already assumed that the erlyvideo business was not possible without this license. - Marc -------------- next part -------------- An HTML attachment was scrubbed... URL: From ttom.kelly@REDACTED Fri Aug 5 14:15:04 2011 From: ttom.kelly@REDACTED (tom kelly) Date: Fri, 5 Aug 2011 13:15:04 +0100 Subject: [erlang-questions] why auto completion not working in "erl -remsh" In-Reply-To: <319FA7D6-601C-4462-B589-1DC7E6DFEBD2@gmail.com> References: <319FA7D6-601C-4462-B589-1DC7E6DFEBD2@gmail.com> Message-ID: Hi Wangel, I think the answer here is that the module "test" is not loaded in the local node, only in the remote node you've attached to. You've added the path to its .beam but all the calls to it are executed in the remote shell, not the local. Because of this it never gets loaded in the local shell and you can't auto complete its functions. //Tom On Fri, Aug 5, 2011 at 9:36 AM, Wanglei wrote: > in a "-pa ebin -remsh" shell > > I can use test:aaa(), > but can't type test:a[tab] to auto completion -> test:aaa( > > The auto completion only show system-libs modules > > "code:rehash()." can't reslove this problem > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.kondratovich@REDACTED Fri Aug 5 14:30:17 2011 From: andrew.kondratovich@REDACTED (Andrew Kondratovich) Date: Fri, 5 Aug 2011 05:30:17 -0700 (PDT) Subject: [erlang-questions] about syntax tools In-Reply-To: <4E3BBA1D.2010409@gmail.com> References: <4E3BBA1D.2010409@gmail.com> Message-ID: <11525585.1243.1312547417183.JavaMail.geo-discussion-forums@yqic12> I can get tokens with erl_scan:string(Code), then i can modify it and get forms with erl_parse:parse_form(Modified_tokens). Then i can do compile:forms(Forms). I can set -compile(parse_transform, Module). But parse_transform allow to modify forms. Are there any way to modify tokens while compiling ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Fri Aug 5 15:30:34 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 05 Aug 2011 15:30:34 +0200 Subject: [erlang-questions] about syntax tools In-Reply-To: <11525585.1243.1312547417183.JavaMail.geo-discussion-forums@yqic12> References: <4E3BBA1D.2010409@gmail.com> <11525585.1243.1312547417183.JavaMail.geo-discussion-forums@yqic12> Message-ID: <4E3BF07A.7050703@gmail.com> On 08/05/2011 02:30 PM, Andrew Kondratovich wrote: > I can set -compile(parse_transform, Module). > But parse_transform allow to modify forms. Are there any way to modify > tokens while compiling ? No. /Richard From michael.santos@REDACTED Fri Aug 5 15:50:04 2011 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 5 Aug 2011 09:50:04 -0400 Subject: [erlang-questions] IP v6 In-Reply-To: References: Message-ID: <20110805135004.GA28788@ecn.lan> On Fri, Aug 05, 2011 at 12:22:47PM +0200, Attila Rajmund Nohl wrote: > Hello! > > What protocols (or applications) do you plan to use? We had issues > with SNMP, SSL, CORBA, HTTP, FTP - I think SSH actually worked. Our > issues have been fixed, but you'll probably need the development > version of Erlang/OTP (or wait until OTP R14B04), because some of > these fixes were made after R14B03 was released. Distribution is a bit broken too (IPv6 node registration with epmd isn't enabled). After a small change to epmd, distribution over IPv6 worked ok for me. > 2011/8/5, Kannan : > > We will be using the latest version of Erlang/OTP and Redhat Enterprise > > Linux. > > > > > > On Fri, Aug 5, 2011 at 3:36 PM, Kannan wrote: > > > >> Hi List, > >> > >> I would like to know if any of you are using or heard of using Erlang in > >> an > >> IP v6 environment. We are planning some server migrations to IP v6 and > >> before we start the development of a prototype, I would like to know your > >> comments please. > >> > >> Regards, > >> Kannan. > >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From robert.virding@REDACTED Fri Aug 5 15:56:41 2011 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 05 Aug 2011 14:56:41 +0100 (BST) Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <4E39CE2B.4050701@gmail.com> Message-ID: <0f13520c-4e5a-47d3-a275-ec106a664a60@localhost> ----- Original Message ----- > On 08/04/2011 12:24 AM, Robert Virding wrote: > > I must say that I find this a little strange. If the evaluation > > order > > of subexpressions is defined, which it is, then I would have > > expected > > that the whole effect of a subexpression would be visible in > > "later" > > subexpressions. Hence you could do things like: > > > > (X=8) + X > > foo(X = bar(42), X) > > > > but not > > > > X + (X=8) > > foo(X, X = bar(42)) > > > > because of the evaluation order. > > You could, yes, but it's not a good idea. For example, a refactoring > that switches the order of arguments to a function would have to > consider any bindings in calling occurrences and lift them out to > preserve the evaluation order *and* the bindings: > > foo(f(), X=bar(), g(X)) > > must become > > V1 = f() > X = bar(42), > V2 = g(X), > foo(V1, V2, X) > > or possibly > > foo(f(), begin X=bar(42), g(X) end, X) > > if you want to switch the order of the last two arguments. This wouldn't help as you still have an order dependency. I don't have any problems with refactoring tools, I never use them. :-) Being old-school I prefer my code exactly as it is written, if I had wanted it in another way I would have written it in another way. I view {X,Y} = foo(A) as *one* expression so think it is more logical to see all the effects of it after it has been evaluated, even the variable bindings. It is not a big issue as most people, including me, seldom write code where this would be an issue, although not handling bindings in this way could make some code a bit more verbose. > The rule that all arguments are evaluated in the same environment > makes > the world a happier place. If I want Basic, I know where I can get > it. ;-) Yes, write my own. The next language on the Erlang VM, Ebasic, for real programming. Robert From max.lapshin@REDACTED Fri Aug 5 17:09:53 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 5 Aug 2011 19:09:53 +0400 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: <4F8B3A97-B8D3-4641-A425-1E6030545EB5@worrell.nl> References: <29781133.2149.1312402781522.JavaMail.geo-discussion-forums@yqiw2> <93FADF05-1B85-4183-811A-12B6A7ADA575@srstrong.com> <4F8B3A97-B8D3-4641-A425-1E6030545EB5@worrell.nl> Message-ID: On Fri, Aug 5, 2011 at 4:12 PM, Marc Worrell wrote: > > This is indeed a terrible return for all your efforts. > With Zotonic we are more lucky, receiving many patches, complete modules, > translations and documentation from the community. > In your case I completely understand the GPL license. ?I already assumed > that the erlyvideo business was not possible without this license. > > Frankly speaking, the only practical reason for me to keep erlyvideo opensource is to have marketing advantage before competitor: my clients have their developers, that can edit erlyvideo sources and add their own plugins. From vasdeveloper@REDACTED Fri Aug 5 18:00:16 2011 From: vasdeveloper@REDACTED (Kannan) Date: Fri, 5 Aug 2011 21:30:16 +0530 Subject: [erlang-questions] IP v6 In-Reply-To: References: Message-ID: We are using protocols related to telecommunication signaling. Btw, if transport protocols fit in with network layer, there should not be any problems with application layer protocols, I guess. Almost all what you have mentioned are application layer protocols. I would like to know more about it form you guys. The first two mails themselves are discouraging, I would also like the advice from Erlang core developers. Thanks. On Fri, Aug 5, 2011 at 3:52 PM, Attila Rajmund Nohl wrote: > Hello! > > What protocols (or applications) do you plan to use? We had issues > with SNMP, SSL, CORBA, HTTP, FTP - I think SSH actually worked. Our > issues have been fixed, but you'll probably need the development > version of Erlang/OTP (or wait until OTP R14B04), because some of > these fixes were made after R14B03 was released. > > 2011/8/5, Kannan : > > We will be using the latest version of Erlang/OTP and Redhat Enterprise > > Linux. > > > > > > On Fri, Aug 5, 2011 at 3:36 PM, Kannan wrote: > > > >> Hi List, > >> > >> I would like to know if any of you are using or heard of using Erlang in > >> an > >> IP v6 environment. We are planning some server migrations to IP v6 and > >> before we start the development of a prototype, I would like to know > your > >> comments please. > >> > >> Regards, > >> Kannan. > >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Fri Aug 5 21:04:27 2011 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Fri, 5 Aug 2011 14:04:27 -0500 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: How about interfacing Erlang to the Qt UI library? I don't mean a convential binding like wxErlang or PyQt, but components to facilitate message passing between Erlang and a Qt port program or NIF thread running a Qt message loop. Qt has relatively recently introduced QML, which is basically a fancy GUI designer that lets you incorporate javascript. The application development model is that the GUI is written in QML/javascript, and the underlying application logic is in C++ (very Web 2.0-ish). Now instead of writing C++, you have your QML/javascript talk to the "ErQt" object which passes messages to/from Erlang, and Erlang implements the application logic. Conceivably one could create a single Qt port executable, driver, or NIF library that could run any number of QML/javascript programs. Creating a GUI for Erlang would involve using the native Qt tools for authoring QML and javascript, and implementing your comm protocol on both the javascript and Erlang sides. Your QML/javascript GUI would run on the standard Qt port exectable...no additional C++ compiling required. Now QML is currently only available for mobile devices, but the Qt folks have stated that they will port it to the desktop and this will become the preferred way to develop desktop Qt apps. This project would likely involve more C++ than Erlang, so I'm not sure how you feel about that. ;) Cheers, Dan. On Mon, Aug 1, 2011 at 5:53 PM, Nick S wrote: > I have just finished implementing a "professional" Erlang server, and with > time and enthusiasm in hand, I would like to start working on something > personal. > I must say, it has been really fun writing apps in OTP and see it > processing millions of messages without ever crashing...! > > Would appreciate some new ideas, that would be useful to community :) > > - XMPP, AMQP etc are already in market! > > - Any audio/video processing layer? > > - Any telecom related...? > > - Some standard protocol implementation? > > - Something new for next generation...? (Buzz words... cloud computing > etc..) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From overminddl1@REDACTED Fri Aug 5 22:11:19 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Fri, 5 Aug 2011 14:11:19 -0600 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: <4E39B40C.6030405@dev-extend.eu> References: <9158FB12-40A7-4276-BD18-C7CACF7690B7@worrell.nl> <4E39B40C.6030405@dev-extend.eu> Message-ID: On Wed, Aug 3, 2011 at 2:48 PM, Lo?c Hoguin wrote: > On 08/03/2011 10:35 PM, Rapsey wrote: >> Yeah I never liked nitrogen records either. I don't know how others >> build sites, but at my company the client side steps are always: >> photoshop file -> create html/css -> convert to templates and write >> javascript > > Same here and everywhere I worked, whichever the language. In fact I > wouldn't recommend doing otherwise for any sizeable project. There's > people who know their photoshop, others who know their HTML/CSS/JS, so > it's definitely worth using their skills directly. > > There's in fact many other reasons that make building the interface > directly in HTML/CSS/JS worthwhile. But I guess I also see the appeal in > people wanting to link client and server-side code to build complicated > interfaces. I do not know Nitrogen enough to cast a judgement on it, though. > > One thing to keep in mind however, if you go the HTML/CSS/JS route, is > that the URIs you use for serving your HTML can also serve JSON for the > AJAX calls and also serve as a public API for your website, all in one > without much extra code required. > >> Using django templates makes complete sense when you're starting from an >> html file. It does not take that much work at all and requires very >> little programming knowledge. If you change anything, it's not difficult >> to put the changes in the template, because they are so similar. >> If we used nitrogen, it would take significantly more work and the >> person doing it, would need to learn something completely unnatural if >> he did not know erlang already. > > Django is simple enough that the HTML/CSS people can make their own > changes directly in the templates without breaking anything (or not too > often at least). All the opposite for me, I have no html/css people, so I do it all myself, the more can be generated in code the better, I hate HTML itself, hence I like Nitrogen. Oh if only Nitrogen had a Wt-like Widget interface. ;-) From watson.timothy@REDACTED Fri Aug 5 22:26:12 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 5 Aug 2011 21:26:12 +0100 Subject: [erlang-questions] about syntax tools In-Reply-To: <4E3BF07A.7050703@gmail.com> References: <4E3BBA1D.2010409@gmail.com> <11525585.1243.1312547417183.JavaMail.geo-discussion-forums@yqic12> <4E3BF07A.7050703@gmail.com> Message-ID: On 5 August 2011 14:30, Richard Carlsson wrote: > On 08/05/2011 02:30 PM, Andrew Kondratovich wrote: >> >> I can set -compile(parse_transform, Module). >> But parse_transform allow to modify forms. Are there any way to modify >> tokens while compiling ? > > No. > Shame, that would be kind of cool, but I suppose a bit unruly. You can replace the tokenize bit during compilation as seen in https://github.com/alavrik/erlson - it'll need custom build though. That project uses a rebar plugin to hook into pre_compile and replace the `erl_parse` module. Pretty simple actually, though obviously being non-standard it could serious mess with compilation of other/unintended code if you mess up what erl_parse is supposed to really do. From attila.r.nohl@REDACTED Fri Aug 5 22:59:29 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 5 Aug 2011 22:59:29 +0200 Subject: [erlang-questions] IP v6 In-Reply-To: References: Message-ID: Hello! >From the top of my head: we couldn't register IPv6 agents into the SNMP manager, the API simly didn't handle IPv6 addresses (and the transport protocol). With HTTP a peername function didn't expect IPv6 addresses and crashed. I don't remember the other problems exactly, but ther were similar in caliber. With the source we could solve them - it was just annoying. 2011/8/5, Kannan : > We are using protocols related to telecommunication signaling. Btw, if > transport protocols fit in with network layer, there should not be any > problems with application layer protocols, I guess. > > Almost all what you have mentioned are application layer protocols. I would > like to know more about it form you guys. > > The first two mails themselves are discouraging, I would also like the > advice from Erlang core developers. > > Thanks. > > > On Fri, Aug 5, 2011 at 3:52 PM, Attila Rajmund Nohl > wrote: > >> Hello! >> >> What protocols (or applications) do you plan to use? We had issues >> with SNMP, SSL, CORBA, HTTP, FTP - I think SSH actually worked. Our >> issues have been fixed, but you'll probably need the development >> version of Erlang/OTP (or wait until OTP R14B04), because some of >> these fixes were made after R14B03 was released. >> >> 2011/8/5, Kannan : >> > We will be using the latest version of Erlang/OTP and Redhat Enterprise >> > Linux. >> > >> > >> > On Fri, Aug 5, 2011 at 3:36 PM, Kannan wrote: >> > >> >> Hi List, >> >> >> >> I would like to know if any of you are using or heard of using Erlang >> >> in >> >> an >> >> IP v6 environment. We are planning some server migrations to IP v6 and >> >> before we start the development of a prototype, I would like to know >> your >> >> comments please. >> >> >> >> Regards, >> >> Kannan. >> >> >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From bob@REDACTED Fri Aug 5 23:26:16 2011 From: bob@REDACTED (Bob Cowdery) Date: Fri, 05 Aug 2011 22:26:16 +0100 Subject: [erlang-questions] Interleave binaries Message-ID: <4E3C5FF8.50709@bobcowdery.plus.com> Hi I'm wanting to interleave two binaries. Bin1 = <<0,1>>, Bin2 = <<2,3>>, << <> || <> <= Bin1, <> <= Bin2 >>. <<0,1,2,3>> Bin1 = <<0,1,2,3>>, Bin2 = <<4,5,6,7>>, << <> || <> <= Bin1, <> <= Bin2 >>. <<0,1,4,5,0,1,6,7,2,3,4,5,2,3,6,7>> whereas I want: <<0,1,4,5,2,3,6,7>> I know what it does is what the documentation says but is there a way to get a pure interleave efficiently. Perhaps there is a filter that would work or a way to reduce the binary after merging without stepping through the whole thing. Regards Bob From kenneth.lundin@REDACTED Sat Aug 6 00:28:52 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Sat, 6 Aug 2011 00:28:52 +0200 Subject: [erlang-questions] New_ssl module and CRL support In-Reply-To: <8097A4E725D78E41A8B862ACE60E963945EF692CEC@GVW1355EXB.americas.hpqcorp.net> References: <1312209624.25314.53.camel@vicbristol6.hpl.hp.com> <8097A4E725D78E41A8B862ACE60E963945EF692CEC@GVW1355EXB.americas.hpqcorp.net> Message-ID: It is vacation time in Sweden , you can expect better answers on this in a couple of weeks when most people are back. I would say that it is in our plans to support CRL but I can't promise exactly when it will be there. There are workarounds as you may read in another response. /Kenneth, Erlang/OTP Ericsson. On Wed, Aug 3, 2011 at 12:37 PM, Rosa, Andrea wrote: > Hi > Any news about that? Someone is so kind to give information about future plan for supporting CRL? > Someone is facing with the same issue and can give me information about some workaround? > Best regards > -- > Andrea > > -----Original Message----- > > Sent: 01 August 2011 15:40 > To: erlang-questions@REDACTED > Subject: [erlang-questions] New_ssl module and CRL support > > Hi > > I am working with a software (rabbitmq) which use the new_ssl module to > implement SSL connections, everything goes well with the exception of > managing revoked certification during the handshake, in this case the > server accepts a revoked certificate. > Is there a way for fixing that? > In your plans is there the implementation of a new module for supporting > CRL? > > Cheers > -- > Andrea > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jwatte@REDACTED Sat Aug 6 01:21:59 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 5 Aug 2011 16:21:59 -0700 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: In my opinion, there exists exactly one user interface library: HTML5. That library comes with the CSS3 way of skinning, and the Canvas way of doing procedural drawing, and the JavaScript way of adding UI interaction. We've used Firfox as our UI toolkit for years (using C++/Python as the app languages), and it works great. These days, better choices probably are Berkelium or Chromium or maybe plain WebKit. A binding between JavaScript and Erlang, and/or between the HTML DOM and Erlang messages, would be pretty cool for user-facing things. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Fri, Aug 5, 2011 at 12:04 PM, Daniel Goertzen wrote: > How about interfacing Erlang to the Qt UI library? > > I don't mean a convential binding like wxErlang or PyQt, but components to > facilitate message passing between Erlang and a Qt port program or NIF > thread running a Qt message loop. > > Qt has relatively recently introduced QML, which is basically a fancy GUI > designer that lets you incorporate javascript. The application development > model is that the GUI is written in QML/javascript, and the underlying > application logic is in C++ (very Web 2.0-ish). Now instead of writing C++, > you have your QML/javascript talk to the "ErQt" object which passes messages > to/from Erlang, and Erlang implements the application logic. > > Conceivably one could create a single Qt port executable, driver, or NIF > library that could run any number of QML/javascript programs. Creating a > GUI for Erlang would involve using the native Qt tools for authoring QML and > javascript, and implementing your comm protocol on both the javascript and > Erlang sides. Your QML/javascript GUI would run on the standard Qt port > exectable...no additional C++ compiling required. > > Now QML is currently only available for mobile devices, but the Qt folks > have stated that they will port it to the desktop and this will become the > preferred way to develop desktop Qt apps. > > This project would likely involve more C++ than Erlang, so I'm not sure how > you feel about that. ;) > > Cheers, > Dan. > > > On Mon, Aug 1, 2011 at 5:53 PM, Nick S wrote: > >> I have just finished implementing a "professional" Erlang server, and with >> time and enthusiasm in hand, I would like to start working on something >> personal. >> I must say, it has been really fun writing apps in OTP and see it >> processing millions of messages without ever crashing...! >> >> Would appreciate some new ideas, that would be useful to community :) >> >> - XMPP, AMQP etc are already in market! >> >> - Any audio/video processing layer? >> >> - Any telecom related...? >> >> - Some standard protocol implementation? >> >> - Something new for next generation...? (Buzz words... cloud computing >> etc..) >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Sat Aug 6 06:40:12 2011 From: comptekki@REDACTED (Wes James) Date: Fri, 5 Aug 2011 22:40:12 -0600 Subject: [erlang-questions] Interleave binaries In-Reply-To: <4E3C5FF8.50709@bobcowdery.plus.com> References: <4E3C5FF8.50709@bobcowdery.plus.com> Message-ID: Hi, On Fri, Aug 5, 2011 at 3:26 PM, Bob Cowdery wrote: > Hi > > I'm wanting to interleave two binaries. > > Bin1 = <<0,1>>, > Bin2 = <<2,3>>, > << <> || <> <= Bin1, <> <= Bin2 >>. > <<0,1,2,3>> > > Bin1 = <<0,1,2,3>>, > Bin2 = <<4,5,6,7>>, > << <> || <> <= Bin1, <> <= Bin2 >>. > <<0,1,4,5,0,1,6,7,2,3,4,5,2,3,6,7>> > > whereas I want: > <<0,1,4,5,2,3,6,7>> I don't know if this will suffice: ------------------------------------------------- % interleave module -module(ilm). -export([il/2]). % interleave function il(<>, <>) -> Rest=il(RestA, RestB), <>; il(<<>>, <<>>) -> <<>>. ----------------------------------------------- Erlang R14B02 (erts-5.8.3) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.3 (abort with ^G) 1> c(ilm). {ok,ilm} 2> ilm:il(<<0,1,2,3>>, <<4,5,6,7>>). <<0,1,4,5,2,3,6,7>> ---------------------------------------------- -wes From hynek@REDACTED Sat Aug 6 10:06:36 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Sat, 6 Aug 2011 10:06:36 +0200 Subject: [erlang-questions] Interleave binaries In-Reply-To: References: <4E3C5FF8.50709@bobcowdery.plus.com> Message-ID: It would not be so much efficient because it would make stack for every two bytes of input. Much more efficient will be il(A, B) when is_binary(A), is_binary(B), size(A) =:= size(B) -> il(A, B, <<>>). il(<>, <>, Acc) -> ? ?Rest=il(RestA, RestB), ? ?il(RestA, RestB, <>); il(<<>>, <<>>, Acc) -> Acc. See http://www.erlang.org/doc/efficiency_guide/binaryhandling.html#id64529 for more info about accumulator efficiency trick. On Sat, Aug 6, 2011 at 6:40 AM, Wes James wrote: > Hi, > > On Fri, Aug 5, 2011 at 3:26 PM, Bob Cowdery wrote: >> Hi >> >> I'm wanting to interleave two binaries. >> >> Bin1 = <<0,1>>, >> Bin2 = <<2,3>>, >> << <> || <> <= Bin1, <> <= Bin2 >>. >> <<0,1,2,3>> >> >> Bin1 = <<0,1,2,3>>, >> Bin2 = <<4,5,6,7>>, >> << <> || <> <= Bin1, <> <= Bin2 >>. >> <<0,1,4,5,0,1,6,7,2,3,4,5,2,3,6,7>> >> >> whereas I want: >> <<0,1,4,5,2,3,6,7>> > > > > I don't know if this will suffice: > > ------------------------------------------------- > > % interleave module > > -module(ilm). > > -export([il/2]). > > % interleave function > > il(<>, <>) -> > ? ?Rest=il(RestA, RestB), > ? ?<>; > > il(<<>>, <<>>) -> <<>>. > > ----------------------------------------------- > > Erlang R14B02 (erts-5.8.3) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.8.3 ?(abort with ^G) > 1> c(ilm). > {ok,ilm} > 2> ilm:il(<<0,1,2,3>>, <<4,5,6,7>>). > <<0,1,4,5,2,3,6,7>> > > ---------------------------------------------- > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From robert.virding@REDACTED Sat Aug 6 14:59:33 2011 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 06 Aug 2011 13:59:33 +0100 (BST) Subject: [erlang-questions] about syntax tools In-Reply-To: Message-ID: <114bf997-393b-4a6a-914d-d14b08d42403@localhost> There is another way if you want to be able to dynamically add new syntax easily, and that is to use LFE (Lisp Flavoured Erlang) which does have a proper lisp macro package which lets you do things like this. Of course you need to do a "slight" syntax change first. :-) Robert ----- Original Message ----- > On 5 August 2011 14:30, Richard Carlsson > wrote: > > On 08/05/2011 02:30 PM, Andrew Kondratovich wrote: > >> > >> I can set -compile(parse_transform, Module). > >> But parse_transform allow to modify forms. Are there any way to > >> modify > >> tokens while compiling ? > > > > No. > > > > Shame, that would be kind of cool, but I suppose a bit unruly. You > can > replace the tokenize bit during compilation as seen in > https://github.com/alavrik/erlson - it'll need custom build though. > That project uses a rebar plugin to hook into pre_compile and replace > the `erl_parse` module. Pretty simple actually, though obviously > being > non-standard it could serious mess with compilation of > other/unintended code if you mess up what erl_parse is supposed to > really do. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From robert.virding@REDACTED Sat Aug 6 18:55:47 2011 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 06 Aug 2011 17:55:47 +0100 (BST) Subject: [erlang-questions] Erlang to Python Message-ID: Saw this on reddit. Any comments? http://code.mixpanel.com/2011/08/05/how-and-why-we-switched-from-erlang-to-python/ From wmacgyver@REDACTED Sat Aug 6 19:12:04 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Sat, 6 Aug 2011 13:12:04 -0400 Subject: [erlang-questions] Erlang to Python In-Reply-To: References: Message-ID: <3E1BCECC-A60D-4117-B0B6-DBFC31F1788A@gmail.com> At first read, it sounds like the core problem is they are python experts, and don't have a lot of erlang expertise in house. Thus finding python code more maintainable. I don't think anyone would disagree that python has much large community and talent pool to draw from. The only point in the post I wondered about is the JSON handling bit. Has anyone benchmarked mochijson/mochijson2 against well known fast performer like Jackson for java or aeson for Haskell? On Aug 6, 2011, at 12:55 PM, Robert Virding wrote: > Saw this on reddit. Any comments? > > http://code.mixpanel.com/2011/08/05/how-and-why-we-switched-from-erlang-to-python/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Sat Aug 6 19:12:39 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sun, 7 Aug 2011 02:12:39 +0900 Subject: [erlang-questions] Erlang to Python In-Reply-To: References: Message-ID: Part of their problem might have been architectural: "In Erlang, the right way to do this is to spawn off a separate set of actors to manage each data structure and message pass with them to save and retrieve data. Our code was not set up this way at all, and it was clearly crippled by being haphazardly implemented in a functional style." I'd suggest a friendly request to mixpanel to open-source their now-disused Erlang code - it might be instructive both about how their code could have been better and about how Erlang might need to improve. I wonder ... what seems to be the performance problem of serializing to JSON might actually be a latency problem - if the managing process for some data wakes up every now and then, with medium-to-low priority, and re-serializes if necessary, to be ready to respond to such requests, it might make all the difference in the world. OK, it might make *some* of the difference in the world. -michael turner On Sun, Aug 7, 2011 at 1:55 AM, Robert Virding wrote: > Saw this on reddit. Any comments? > > http://code.mixpanel.com/2011/08/05/how-and-why-we-switched-from-erlang-to-python/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From andrea.rosa@REDACTED Sat Aug 6 19:08:35 2011 From: andrea.rosa@REDACTED (Rosa, Andrea) Date: Sat, 6 Aug 2011 18:08:35 +0100 Subject: [erlang-questions] New_ssl module and CRL support In-Reply-To: References: <1312209624.25314.53.camel@vicbristol6.hpl.hp.com> <8097A4E725D78E41A8B862ACE60E963945EF692CEC@GVW1355EXB.americas.hpqcorp.net> Message-ID: <8097A4E725D78E41A8B862ACE60E963945EF82D50F@GVW1355EXB.americas.hpqcorp.net> Hi Thank you for your kindly reply, I wait the end of August to see if your colleagues have further information. Kind regards -- Andrea Sent: 05 August 2011 23:29 > To: Rosa, Andrea > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] New_ssl module and CRL support > > It is vacation time in Sweden , you can expect better answers on this > in a couple of weeks when most people are back. > > I would say that it is in our plans to support CRL but I can't promise > exactly when it will be there. There are workarounds > as you may read in another response. > > /Kenneth, Erlang/OTP Ericsson. > > On Wed, Aug 3, 2011 at 12:37 PM, Rosa, Andrea > wrote: > > Hi > > Any news about that? Someone is so kind to give information about > future plan for supporting CRL? > > Someone is facing with the same issue and can give me information > about some workaround? > > Best regards > > -- > > Andrea > > > > -----Original Message----- > > > > Sent: 01 August 2011 15:40 > > To: erlang-questions@REDACTED > > Subject: [erlang-questions] New_ssl module and CRL support > > > > Hi > > > > I am working with a software (rabbitmq) which use the new_ssl module > to > > implement SSL connections, everything goes well with the exception of > > managing revoked certification during the handshake, in this case the > > server accepts a revoked certificate. > > Is there a way for fixing that? > > In your plans is there the implementation of a new module for > supporting > > CRL? > > > > Cheers > > -- > > Andrea > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > From peter@REDACTED Sat Aug 6 19:14:26 2011 From: peter@REDACTED (Peter Sabaini) Date: Sat, 06 Aug 2011 19:14:26 +0200 Subject: [erlang-questions] Erlang to Python In-Reply-To: References: Message-ID: <4E3D7672.8010809@sabaini.at> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 08/06/2011 06:55 PM, Robert Virding wrote: > Saw this on reddit. Any comments? > > http://code.mixpanel.com/2011/08/05/how-and-why-we-switched-from-erlang-to-python/ Also on HN: http://news.ycombinator.com/item?id=2852415 Basically, they had no (or no good) Erlang people and rewrote a small app in Python looks like. Nothing to see here IMHO... > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk49dnIACgkQafAcTZYKVBxFwQCg3T0G3UgkeQEPdJ99WJcIwt3m vTEAniIF9ApAjp39mYmWDZ/+mrJCotDw =ZKMl -----END PGP SIGNATURE----- From jwatte@REDACTED Sat Aug 6 22:55:12 2011 From: jwatte@REDACTED (Jon Watte) Date: Sat, 6 Aug 2011 13:55:12 -0700 Subject: [erlang-questions] Erlang to Python In-Reply-To: References: Message-ID: "We didn't know what we were doing in language A, so we re-wrote it in language B." However, it IS very easy to run into unexpected performance problems in Erlang. To wit: we just profiled a slowdown in our application into the supervisor module. Turns out, it uses a dict for its children, which ends up generating lots of garbage (sort of n-squared) when adding/removing child processes a lot to a supervisor with lots of children. We're going to not use a supervisor for that case. Which brings me back to a previous discussion: Most scripting languages have convenient hash table classes. Python has dict. JavaScript has Object. Erlang has dict, which isn't really O(1) for large sets, and gb_trees(), which is asymptotically O(log n), and ets, which is close to O(1) -- and is quite heavyweight. I don't think it's possible to build an O(1) hash table on top of plain erlang data structures (as each time you modify an element, you have to re-create the top-level tuple). However, I imagine that with a C type, you could build one, that could keep item generation counts or similar to keep the amount of modified memory to a minimum. The question is: can it be done without the overhead of ETS? Has it been done? Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Sat, Aug 6, 2011 at 9:55 AM, Robert Virding < robert.virding@REDACTED> wrote: > Saw this on reddit. Any comments? > > > http://code.mixpanel.com/2011/08/05/how-and-why-we-switched-from-erlang-to-python/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sun Aug 7 01:48:06 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 7 Aug 2011 00:48:06 +0100 Subject: [erlang-questions] about syntax tools In-Reply-To: <114bf997-393b-4a6a-914d-d14b08d42403@localhost> References: <114bf997-393b-4a6a-914d-d14b08d42403@localhost> Message-ID: On 6 August 2011 13:59, Robert Virding wrote: > There is another way if you want to be able to dynamically add new syntax easily, and that is to use LFE (Lisp Flavoured Erlang) which does have a proper lisp macro package which lets you do things like this. Of course you need to do a "slight" syntax change first. :-) LOL only slight!? LFE is pretty awesome, but I must confess I've not spent much quality time with Lisp so all I know of the macro system is that *lots* of people tell me it's the bees knees. From michael.eugene.turner@REDACTED Sun Aug 7 08:51:42 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sun, 7 Aug 2011 15:51:42 +0900 Subject: [erlang-questions] Erlang to Python In-Reply-To: References: Message-ID: I wrote: "the performance problem of serializing to JSON" Oops. JSON *de*-coding seems to be issue. But wasn't CouchDB the beneficiary of some NIF JSON parser 10x faster than mochijson? -michael turner On Sun, Aug 7, 2011 at 2:12 AM, Michael Turner wrote: > Part of their problem might have been architectural: > > "In Erlang, the right way to do this is to spawn off a separate set of > actors to manage each data structure and message pass with them to > save and retrieve data. Our code was not set up this way at all, and > it was clearly crippled by being haphazardly implemented in a > functional style." > > I'd suggest a friendly request to mixpanel to open-source their > now-disused Erlang code - it might be instructive both about how their > code could have been better and about how Erlang might need to > improve. > > I wonder ... what seems to be the performance problem of serializing > to JSON might actually be a latency problem - if the managing process > for some data wakes up every now and then, with medium-to-low > priority, and re-serializes if necessary, to be ready to respond to > such requests, it might make all the difference in the world. OK, it > might make *some* of the difference in the world. > > -michael turner > > > > On Sun, Aug 7, 2011 at 1:55 AM, Robert Virding > wrote: >> Saw this on reddit. Any comments? >> >> http://code.mixpanel.com/2011/08/05/how-and-why-we-switched-from-erlang-to-python/ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From wmacgyver@REDACTED Sun Aug 7 09:02:18 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Sun, 7 Aug 2011 03:02:18 -0400 Subject: [erlang-questions] Erlang to Python In-Reply-To: References: Message-ID: <5716B852-025B-485D-BDE1-33392A600ED4@gmail.com> It's based on eep0018 https://issues.apache.org/jira/browse/COUCHDB-1118 The author created https://github.com/davisp/jiffy On Aug 7, 2011, at 2:51 AM, Michael Turner wrote: > I wrote: "the performance problem of serializing to JSON" > > Oops. JSON *de*-coding seems to be issue. > > But wasn't CouchDB the beneficiary of some NIF JSON parser 10x faster > than mochijson? > > -michael turner > > > > On Sun, Aug 7, 2011 at 2:12 AM, Michael Turner > wrote: >> Part of their problem might have been architectural: >> >> "In Erlang, the right way to do this is to spawn off a separate set of >> actors to manage each data structure and message pass with them to >> save and retrieve data. Our code was not set up this way at all, and >> it was clearly crippled by being haphazardly implemented in a >> functional style." >> >> I'd suggest a friendly request to mixpanel to open-source their >> now-disused Erlang code - it might be instructive both about how their >> code could have been better and about how Erlang might need to >> improve. >> >> I wonder ... what seems to be the performance problem of serializing >> to JSON might actually be a latency problem - if the managing process >> for some data wakes up every now and then, with medium-to-low >> priority, and re-serializes if necessary, to be ready to respond to >> such requests, it might make all the difference in the world. OK, it >> might make *some* of the difference in the world. >> >> -michael turner >> >> >> >> On Sun, Aug 7, 2011 at 1:55 AM, Robert Virding >> wrote: >>> Saw this on reddit. Any comments? >>> >>> http://code.mixpanel.com/2011/08/05/how-and-why-we-switched-from-erlang-to-python/ >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From paul.joseph.davis@REDACTED Sun Aug 7 10:04:20 2011 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Sun, 7 Aug 2011 03:04:20 -0500 Subject: [erlang-questions] Erlang to Python In-Reply-To: <5716B852-025B-485D-BDE1-33392A600ED4@gmail.com> References: <5716B852-025B-485D-BDE1-33392A600ED4@gmail.com> Message-ID: On Sun, Aug 7, 2011 at 2:02 AM, Wilson MacGyver wrote: > It's based on eep0018 > > https://issues.apache.org/jira/browse/COUCHDB-1118 > > The author created https://github.com/davisp/jiffy > Loosely based on EEP0018 with modifications over time due to CouchDB drifting and what the NIF API allows. I have a simple test [1] for benchmarking mochijson2 and jiffy together. I swear I had jsx in there but I'm not seeing it as a dep just now. Either way, I'd recommend either jsx or jiffy if you're looking for a JSON parser at the moment. jsx is pure erlang, handles more character encodings, does streaming encoding/decoding and follows EEP0018 more closely then Jiffy. Jiffy follows mochijson2 more closely, only supports UTF-8 encoded JSON, but its fairly speedy. I'd be interested in seeing how Jiffy compares against different languages. I've been meaning to tear out the NIF code and implement a Python module with it to compare against simplejson but haven't gotten around to it. If anyone wants help porting it to a different language I'd be more than happy to field questions and pitch in. There's also a previous implementation that for historical reasons is called eep0018 but in reality it diverges from the EEP spec nearly as far as Jiffy [2]. It's based on YAJL and is pretty close in terms of speed to Jiffy but has issues when dealing with bignums and other weird corners of JSON. This project is deprecated in favor of Jiffy. [1] https://github.com/davisp/erljson_bench [2] https://github.com/davisp/eep0018 > On Aug 7, 2011, at 2:51 AM, Michael Turner wrote: > >> I wrote: "the performance problem of serializing to JSON" >> >> Oops. JSON *de*-coding seems to be issue. >> >> But wasn't CouchDB the beneficiary of some NIF JSON parser 10x faster >> than mochijson? >> >> -michael turner >> >> >> >> On Sun, Aug 7, 2011 at 2:12 AM, Michael Turner >> wrote: >>> Part of their problem might have been architectural: >>> >>> "In Erlang, the right way to do this is to spawn off a separate set of >>> actors to manage each data structure and message pass with them to >>> save and retrieve data. Our code was not set up this way at all, and >>> it was clearly crippled by being haphazardly implemented in a >>> functional style." >>> >>> I'd suggest a friendly request to mixpanel to open-source their >>> now-disused Erlang code - it might be instructive both about how their >>> code could have been better and about how Erlang might need to >>> improve. >>> >>> I wonder ... what seems to be the performance problem of serializing >>> to JSON might actually be a latency problem - if the managing process >>> for some data wakes up every now and then, with medium-to-low >>> priority, and re-serializes if necessary, to be ready to respond to >>> such requests, it might make all the difference in the world. OK, it >>> might make *some* of the difference in the world. >>> >>> -michael turner >>> >>> >>> >>> On Sun, Aug 7, 2011 at 1:55 AM, Robert Virding >>> wrote: >>>> Saw this on reddit. Any comments? >>>> >>>> http://code.mixpanel.com/2011/08/05/how-and-why-we-switched-from-erlang-to-python/ >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From bg5sbk@REDACTED Sun Aug 7 10:05:33 2011 From: bg5sbk@REDACTED (=?gbk?B?tO+07w==?=) Date: Sun, 7 Aug 2011 16:05:33 +0800 Subject: [erlang-questions] prim_file:translate_response/2 take very long time almost suspend Message-ID: Hi. In my game project, I use ETS mapping mysql and use two process sync the ETS table's changes to mysql. One process generate SQL and write to a log file and then send SQL to another process to execute. In recent days, I got a problem. After some unknow thing happen (look like always happen on busy node). The SQL log process run into very very slow and no recovery. When this happen, I must disconnect all players and suspend the SQL log process and copy the messages to SQL execute process, after all action sync, restart the node. :( And when this happen, I always can get: {current_function,{prim_file,translate_response,2}} by: process_info(SqlLogProc, current_function). I think something make the "translate_response" take very very long time so the log file write very very slow. The log file open by: file:open(FileName, [append, raw, {delayed_write, 1024 * 1000 * 10, 6000}]). and write by: file:write(File, SQL). I use R14B3 on server. Is it a bug of erlang's prim_file module? or what can I optimize? I have 350+ game server node and add 15 ~ 20 node every day, this problem happens on 1 ~ 2 node every day, and make me crazy I need sleep :( -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmcknight@REDACTED Sun Aug 7 18:36:06 2011 From: cmcknight@REDACTED (cmcknight@REDACTED) Date: Sun, 7 Aug 2011 09:36:06 -0700 Subject: [erlang-questions] Erlang to Python In-Reply-To: References: Message-ID: <37A8850A-34CB-4A06-BB2B-4B5471AD1606@pheonic.com> I read it (pardon the pun) as "we know Python, we love Python, we don't know or want to learn Erlang because it's not Python". Wait until they have to deal with fault-tolerance and distribution. At that point, they'll wind up being like a lot of web applications that believe that downtime is an acceptable characteristic. IMHO. On Aug 6, 2011, at 11:51 PM, erlang-questions-request@REDACTED wrote: > Subject: [erlang-questions] Erlang to Python > Message-ID: > Content-Type: text/plain; charset=utf-8 > > Saw this on reddit. Any comments? > > http://code.mixpanel.com/2011/08/05/how-and-why-we-switched-from-erlang-to-python/ Chuck McKnight cmcknight@REDACTED "It's cold outside, there's no kind of atmosphere, I'm all alone, more or less, let me fly far away from here, fun, fun, fun, in the sun, sun, sun" --- Theme Song, Red Dwarf -------------- next part -------------- An HTML attachment was scrubbed... URL: From williamstw@REDACTED Sun Aug 7 18:50:31 2011 From: williamstw@REDACTED (Tim Williams) Date: Sun, 07 Aug 2011 16:50:31 -0000 Subject: [erlang-questions] Gxffhthdhchchfhdydgsfhchcjchxjx Message-ID: <11B683F4-3AAA-47AE-BF09-58E7B6CD49A7@gmail.com> Dhdjjffjfffjfjghfnfjdjfjdjjdyeyehrhffjhjfugfj Sent from my iPod From g@REDACTED Sun Aug 7 21:24:45 2011 From: g@REDACTED (Garrett Smith) Date: Sun, 7 Aug 2011 14:24:45 -0500 Subject: [erlang-questions] Extension for escript files Message-ID: Is there an official (or leaning?) on the correct extension for Erlang escript files? I've seen .escript and .es used. Garrett From g@REDACTED Mon Aug 8 01:24:22 2011 From: g@REDACTED (Garrett Smith) Date: Sun, 7 Aug 2011 18:24:22 -0500 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: Message-ID: On Thu, Aug 4, 2011 at 5:29 PM, Richard O'Keefe wrote: > > The person in question was Tony Arcieri, who built Reia > (see http://reia-lang.org/). ?There are three things that seem > obvious about Tony Arcieri: > (1) He likes Ruby. ?(Nobody who didn't would want to build > ? ?something like Reia.) > (2) He knows rather a lot about Erlang. ?(Nobody who didn't > ? ?*could* build something like Reia.) > (3) He is a skilled programmer. ?(Nobody who wasn't could > ? ?get something like Reia working and out there -- as long > ? ?as directories don't include dots in their names...) I reread his post, largely due to your generous (and dignified) representation of his position in this thread. The crux of his argument is this: "This is the type of error you only encounter at runtime. It can lay undetected in your codebase, unless you're writing tests." As if other dynamically typed languages (Ruby) excel in catching "errors" at compile time. This isn't even a straw man. It's straw. His post, for whatever reason, is an emotional outburst and largely unconstructive. At least that's my reading of it, twice. If your average number of lines per function is < 10, honestly, this is *really* a problem? Nonsense. Garrett From g@REDACTED Mon Aug 8 01:41:25 2011 From: g@REDACTED (Garrett Smith) Date: Sun, 7 Aug 2011 18:41:25 -0500 Subject: [erlang-questions] [ANN] euuid 1.0.0 - an erlang UUID implementation In-Reply-To: References: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> Message-ID: On Mon, Aug 1, 2011 at 7:09 PM, Olivier Girondel wrote: > On Mon, Aug 1, 2011 at 8:37 PM, Martijn Rijkeboer wrote: >> Hello, >> >> As an exercise for learning Erlang I wrote an Universally Unique >> IDentifier (UUID) implementation as defined in RFC 4122. > Well, UUIDs seem to be: > > 1. A good exercise > 2. Implemented over and over again (I'll just quote mine, but a good way to > check your code would be to have a look on older implementations) > 3. A good candidate for inclusion somewhere in a library upstream :) +1 I wonder, at least two to three times a week, why foobar module isn't a part of the OTP distribution. Admittedly, I'm not sure how that happens. Off the top of my head, a few things that I hate looking elsewhere for: gproc, json, bin_to_hex, uuid, urllib (URL utils ala mochiweb_util), getopt Garrett From abawcom@REDACTED Mon Aug 8 02:08:43 2011 From: abawcom@REDACTED (Aaron Bawcom) Date: Sun, 7 Aug 2011 20:08:43 -0400 Subject: [erlang-questions] ETS: binary matching in match specifications Message-ID: There doesn't seem to be any type of binary data type matching in a match specification besides '==' or '/='. To compare apples to apples and to exclude OS/hardware variances the following test shows the differences in using a 4 segment binary as a key (128bits total) and a 4-tuple of integers as a key. The segments of the binary and the tuple values for the lookups are randomly generated so as to eliminate any type of issue there. The methodologies in the 2 algorithms are exactly the same save the form of the key. Based on this data the speed of binary based lookups are somewhat superior than tuple based lookups. The problem is that we have a need to do certain types of ets matching on the binary keys (3 segments are _ whereas one segment is an int()). The question is this: Why are there no other ets match conditions than '==' and '/='? Is it just a development time/complexity issue or is there a more fundamental reason such as reducing the use of match specifications in favor of another approach. Or is it simply convention that this type of problem is best solved using the tuple approach? Our issue is that 99.999% of the time we need the speed of the binary '==' match and the rest of the time we need an incredibly simple match on a portion of the binary. And the memory cost of storing an alternate structure for that small use case is not acceptable. It would be great if even 'band' could operate on a binary type in a match specification. Thoughts? % Col1 == Number of elements in table % Col2 == Memory size of table % Col3 == Average ets:member/sec for 10M lookups 1> test:do_ets_test(). % Using 4 segment 128bit Binary for Key lookup 4, 362, 7369196 19, 557, 6476683 79, 1337, 6476683 319, 4457, 6345177 1279, 16937, 5882352 3> test:do_ets_test(). % Using 4 element tuple of integers for Key lookup 4, 358, 6345177 19, 538, 5773672 79, 1258, 5777007 319, 4138, 5724098 1279, 15658, 5479452 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jws@REDACTED Mon Aug 8 02:38:27 2011 From: jws@REDACTED (Jeff Schultz) Date: Mon, 8 Aug 2011 10:38:27 +1000 Subject: [erlang-questions] ETS: binary matching in match specifications In-Reply-To: References: Message-ID: <20110808003827.GA19424@mulga.csse.unimelb.edu.au> On Sun, Aug 07, 2011 at 08:08:43PM -0400, Aaron Bawcom wrote: > % Col1 == Number of elements in table > % Col2 == Memory size of table > % Col3 == Average ets:member/sec for 10M lookups > 1> test:do_ets_test(). % Using 4 segment 128bit Binary for Key lookup > 1279, 16937, 5882352 > 3> test:do_ets_test(). % Using 4 element tuple of integers for Key lookup > 1279, 15658, 5479452 There's something missing in these memory figures. 128b is 16 byte. 16937/1279 = 13.24, 15658/1279 = 12.24. Doesn't look like these figures include the keys. (Hmm, 16937-15658 = 1279. Looks like the reported memory use is exactly 1 byte / element more for the binary key version.) In any case, the performance difference you show between the two versions isn't large. Does it matter that much? Jeff Schultz From jws@REDACTED Mon Aug 8 03:33:36 2011 From: jws@REDACTED (Jeff Schultz) Date: Mon, 8 Aug 2011 11:33:36 +1000 Subject: [erlang-questions] Interleave binaries In-Reply-To: <4E3C5FF8.50709@bobcowdery.plus.com> References: <4E3C5FF8.50709@bobcowdery.plus.com> Message-ID: <20110808013336.GA24728@mulga.csse.unimelb.edu.au> On Fri, Aug 05, 2011 at 10:26:16PM +0100, Bob Cowdery wrote: > I'm wanting to interleave two binaries. > > Bin1 = <<0,1>>, > Bin2 = <<2,3>>, > << <> || <> <= Bin1, <> <= Bin2 >>. > <<0,1,2,3>> > > Bin1 = <<0,1,2,3>>, > Bin2 = <<4,5,6,7>>, > << <> || <> <= Bin1, <> <= Bin2 >>. > <<0,1,4,5,0,1,6,7,2,3,4,5,2,3,6,7>> > > whereas I want: > <<0,1,4,5,2,3,6,7>> > > I know what it does is what the documentation says but is there a way to > get a pure interleave efficiently. Perhaps there is a filter that would > work or a way to reduce the binary after merging without stepping > through the whole thing. I'd go via an intermediate list: 25> Bin1 = <<0,1,2,3>>, Bin2 = <<4,5,6,7>>. <<4,5,6,7>> 26> list_to_binary( [[binary:part(Bin1, I, 2), binary:part(Bin2, I, 2)] || I <- lists:seq(0, byte_size(Bin1) - 1, 2)]). <<0,1,4,5,2,3,6,7>> Makes O(N) garbage. If you want a bit string comprehension, try 27> << <> || I <- lists:seq(0, byte_size(Bin1) - 1, 2), P1 <- [binary:part(Bin1, I, 2)], P2 <- [binary:part(Bin2, I, 2)]>>. <<0,1,4,5,2,3,6,7>> (I may be missing some syntax trick here. Ugly, isn't it.) I have no idea how efficient that is. The naive implementation would be O(N^2). Even a clever one would make O(N) garbage, though unlike the list comprehension, it might not hang on to all of it at once. Jeff Schultz From ok@REDACTED Mon Aug 8 03:58:18 2011 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 8 Aug 2011 13:58:18 +1200 Subject: [erlang-questions] Declarative configuration languages? Message-ID: <5897AB6F-0A17-45FB-9E07-4BF4E9424AC2@cs.otago.ac.nz> A colleague just asked me I'm trying to kick off some work into middleware for configuration of large-scale, distributed systems. Have you come across any elegant, declarative configuration languages used for this sort of job? I've found a couple of papers, but I've never _used_ any of the systems. Any recommendations? From abawcom@REDACTED Mon Aug 8 04:12:15 2011 From: abawcom@REDACTED (Aaron Bawcom) Date: Sun, 7 Aug 2011 22:12:15 -0400 Subject: [erlang-questions] Declarative configuration languages? In-Reply-To: <5897AB6F-0A17-45FB-9E07-4BF4E9424AC2@cs.otago.ac.nz> References: <5897AB6F-0A17-45FB-9E07-4BF4E9424AC2@cs.otago.ac.nz> Message-ID: Puppet and chef are noteworthy in this area. rPath has an open-source implementation of app-config called iConfig but from what I understand it is not backed by a language. -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Richard O'Keefe Sent: Sunday, August 07, 2011 9:58 PM To: erlang-questions Questions; haskell-cafe Cafe Subject: [erlang-questions] Declarative configuration languages? A colleague just asked me I'm trying to kick off some work into middleware for configuration of large-scale, distributed systems. Have you come across any elegant, declarative configuration languages used for this sort of job? I've found a couple of papers, but I've never _used_ any of the systems. Any recommendations? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Mon Aug 8 04:13:15 2011 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 8 Aug 2011 14:13:15 +1200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: Message-ID: On 8/08/2011, at 11:24 AM, Garrett Smith wrote: > I reread his post, largely due to your generous (and dignified) > representation of his position in this thread. Well, given how I jumped all over his proposal to adopt Ruby-style "blocks" into Erlang, I thought it was time to shake the venom out of the pen... > The crux of his argument is this: > > "This is the type of error you only encounter at runtime. It can lay > undetected in your codebase, unless you're writing tests." > > As if other dynamically typed languages (Ruby) excel in catching > "errors" at compile time. My point here was that this kind of error *can* be caught at compile time. I have a Smalltalk to C compiler (it does not handle exceptions yet, so it's not really ready to release) and it's actually quite surprising how much you *can* detect at compile time. For example, I have a rule language that lets me write rules like these: rule: 'isNil is sacred' defines: #isNil -> is: Object | is: UndefinedObject rule: '= and hash must be consistent (A)' defines: #= -> defines: #hash rule: '= and hash must be consistent (B)' defines: #hash -> defines: #= rule: 'network addresses are values' kindOf: NetworkAddress -> isAbstract | isValue rule: 'streams are NOT values' kindOf: InputStream | kindOf: OutputStream -> isAbstract | isMutable This is not traditional in Smalltalk, but it's surprising the number of mistakes they've caught. The rule language is compiled into AWK by an AWK script, which was rather fun. There are a couple of hundred rules and should be more. Of course, now I need a checker for the checking language! One of the rules above had a typo I just noticed when pasting it in... From abawcom@REDACTED Mon Aug 8 04:22:50 2011 From: abawcom@REDACTED (Aaron Bawcom) Date: Sun, 7 Aug 2011 22:22:50 -0400 Subject: [erlang-questions] ETS: binary matching in match specifications In-Reply-To: <20110808003827.GA19424@mulga.csse.unimelb.edu.au> References: <20110808003827.GA19424@mulga.csse.unimelb.edu.au> Message-ID: 1) For a lot of the binary keys the first 32 bits are zero which I believe may be optimized out from a memory perspective. I've seen this in other tests. But again, any savings there as far as memory size are visible in the numbers for both tests. 2) The memory sizes are what ets:info(tid()) returns 3) Yes an approximate 10% drop in speed matters. Do you know the answer to the question? Why aren't there any binary pattern match conditions available in match specs? Is the answer simply "no one has needed them"? -----Original Message----- From: Jeff Schultz [mailto:jws@REDACTED] Sent: Sunday, August 07, 2011 8:38 PM To: Aaron Bawcom Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] ETS: binary matching in match specifications On Sun, Aug 07, 2011 at 08:08:43PM -0400, Aaron Bawcom wrote: > % Col1 == Number of elements in table > % Col2 == Memory size of table > % Col3 == Average ets:member/sec for 10M lookups > 1> test:do_ets_test(). % Using 4 segment 128bit Binary for Key lookup > 1279, 16937, 5882352 > 3> test:do_ets_test(). % Using 4 element tuple of integers for Key > 3> lookup > 1279, 15658, 5479452 There's something missing in these memory figures. 128b is 16 byte. 16937/1279 = 13.24, 15658/1279 = 12.24. Doesn't look like these figures include the keys. (Hmm, 16937-15658 = 1279. Looks like the reported memory use is exactly 1 byte / element more for the binary key version.) In any case, the performance difference you show between the two versions isn't large. Does it matter that much? Jeff Schultz From jws@REDACTED Mon Aug 8 09:33:18 2011 From: jws@REDACTED (Jeff Schultz) Date: Mon, 8 Aug 2011 17:33:18 +1000 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <4E3BAE21.2080903@gmail.com> References: <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> <1312383957.26732.YahooMailNeo@web111412.mail.gq1.yahoo.com> <4E3974A3.6070104@gmail.com> <4E39C1E1.7070301@gmail.com> <20110805022214.GA29400@mulga.csse.unimelb.edu.au> <4E3BAE21.2080903@gmail.com> Message-ID: <20110808073318.GA11363@mulga.csse.unimelb.edu.au> On Fri, Aug 05, 2011 at 10:47:29AM +0200, Richard Carlsson wrote: > On 08/05/2011 04:22 AM, Jeff Schultz wrote: >> While I appreciate the intent of Barklund's rule above, I don't think >> it plays well with Erlang's explicit left-to-right order of >> evaluation. > I think you have misunderstood something - or maybe I'm missing something > in your reasoning. Barklund's stated rule means that |(X = 8) + X| is just > as invalid in Erlang as |X + (X = 8)|, because even though it will work in > a left-to-right evaluation order (which is ultimately the order in which > the arguments _will_ be executed), it will not work in _any_ evaluation > order. The compiler therefore rejects it. > The nice consequence is that you can always naively reorder the arguments > of a function or operator call (in a valid program), because there's no > possibility that one affects the variable bindings expected of another. My point was that in a strict language with side-effects and an explicit evaluation order, the rule doesn't do anything useful. It doesn't, for example, identify any code where the program will "go wrong" that won't be picked up by the use-before-definition check anyway. Jeff Schultz From max.lapshin@REDACTED Mon Aug 8 10:18:44 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 8 Aug 2011 12:18:44 +0400 Subject: [erlang-questions] Erlang to Python In-Reply-To: References: <5716B852-025B-485D-BDE1-33392A600ED4@gmail.com> Message-ID: That post is just a way to receive some traffic on provocative title. I remember the same silly posts like "why we switched back to PHP from ruby on rails", that were written only to make some PR From bob@REDACTED Mon Aug 8 10:26:14 2011 From: bob@REDACTED (Bob Cowdery) Date: Mon, 08 Aug 2011 09:26:14 +0100 Subject: [erlang-questions] Interleave binaries In-Reply-To: <20110808013336.GA24728@mulga.csse.unimelb.edu.au> References: <4E3C5FF8.50709@bobcowdery.plus.com> <20110808013336.GA24728@mulga.csse.unimelb.edu.au> Message-ID: <4E3F9DA6.8010100@bobcowdery.plus.com> Jeff Thanks very much for the suggestions. I will give them a whirl. Bob On 08/08/2011 02:33, Jeff Schultz wrote: > On Fri, Aug 05, 2011 at 10:26:16PM +0100, Bob Cowdery wrote: >> I'm wanting to interleave two binaries. >> >> Bin1 = <<0,1>>, >> Bin2 = <<2,3>>, >> << <> || <> <= Bin1, <> <= Bin2 >>. >> <<0,1,2,3>> >> >> Bin1 = <<0,1,2,3>>, >> Bin2 = <<4,5,6,7>>, >> << <> || <> <= Bin1, <> <= Bin2 >>. >> <<0,1,4,5,0,1,6,7,2,3,4,5,2,3,6,7>> >> >> whereas I want: >> <<0,1,4,5,2,3,6,7>> >> >> I know what it does is what the documentation says but is there a way to >> get a pure interleave efficiently. Perhaps there is a filter that would >> work or a way to reduce the binary after merging without stepping >> through the whole thing. > I'd go via an intermediate list: > > 25> Bin1 = <<0,1,2,3>>, Bin2 = <<4,5,6,7>>. > <<4,5,6,7>> > 26> list_to_binary( > [[binary:part(Bin1, I, 2), binary:part(Bin2, I, 2)] > || I <- lists:seq(0, byte_size(Bin1) - 1, 2)]). > <<0,1,4,5,2,3,6,7>> > > Makes O(N) garbage. > > > If you want a bit string comprehension, try > > 27> << <> > || I <- lists:seq(0, byte_size(Bin1) - 1, 2), > P1 <- [binary:part(Bin1, I, 2)], > P2 <- [binary:part(Bin2, I, 2)]>>. > <<0,1,4,5,2,3,6,7>> > > (I may be missing some syntax trick here. Ugly, isn't it.) > > I have no idea how efficient that is. The naive implementation would > be O(N^2). Even a clever one would make O(N) garbage, though unlike > the list comprehension, it might not hang on to all of it at once. > > > Jeff Schultz From carlsson.richard@REDACTED Mon Aug 8 11:10:51 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 08 Aug 2011 11:10:51 +0200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <20110808073318.GA11363@mulga.csse.unimelb.edu.au> References: <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> <1312383957.26732.YahooMailNeo@web111412.mail.gq1.yahoo.com> <4E3974A3.6070104@gmail.com> <4E39C1E1.7070301@gmail.com> <20110805022214.GA29400@mulga.csse.unimelb.edu.au> <4E3BAE21.2080903@gmail.com> <20110808073318.GA11363@mulga.csse.unimelb.edu.au> Message-ID: <4E3FA81B.7010401@gmail.com> On 08/08/2011 09:33 AM, Jeff Schultz wrote: > On Fri, Aug 05, 2011 at 10:47:29AM +0200, Richard Carlsson wrote: >> On 08/05/2011 04:22 AM, Jeff Schultz wrote: >>> While I appreciate the intent of Barklund's rule above, I don't think >>> it plays well with Erlang's explicit left-to-right order of >>> evaluation. > >> I think you have misunderstood something - or maybe I'm missing something >> in your reasoning. Barklund's stated rule means that |(X = 8) + X| is just >> as invalid in Erlang as |X + (X = 8)|, because even though it will work in >> a left-to-right evaluation order (which is ultimately the order in which >> the arguments _will_ be executed), it will not work in _any_ evaluation >> order. The compiler therefore rejects it. > >> The nice consequence is that you can always naively reorder the arguments >> of a function or operator call (in a valid program), because there's no >> possibility that one affects the variable bindings expected of another. > > My point was that in a strict language with side-effects and an > explicit evaluation order, the rule doesn't do anything useful. It > doesn't, for example, identify any code where the program will "go > wrong" that won't be picked up by the use-before-definition check > anyway. As I said, the rule should be read as a general design principle for Erlang - it says that there has to *be* a check, one way or another (and at compile time), for potentially uninitialized uses - regardless of argument evaluation order. Otherwise, it's not Erlang. You could make a nonstandard dialect of Erlang that uses right-to-left evaluation order instead of left-to-right, and it would still be mostly the same language, but if you break the overall rule, then your language is no longer even Erlang-like. For comparison, C is a "strict language with side effects" and lax evaluation order - but even if you fixed the evaluation order, there would be no rule that said that a C program containing a use of an uninitialized variable is not a legal program - it's just that its behaviour is undefined if you execute such a path at runtime. Finally, it's best to keep the concepts of side effects and variable bindings separate. In a language like Erlang, variable bindings have nothing to do with side effects. /Richard From hynek@REDACTED Mon Aug 8 13:34:43 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Mon, 8 Aug 2011 13:34:43 +0200 Subject: [erlang-questions] Interleave binaries In-Reply-To: References: <4E3C5FF8.50709@bobcowdery.plus.com> Message-ID: Sorry, I have accidentally forgot remove one line from code. The correct version should be: il(A, B) when is_binary(A), is_binary(B), size(A) =:= size(B) -> il(A, B, <<>>). il(<>, <>, Acc) -> ? ?il(RestA, RestB, <>); il(<<>>, <<>>, Acc) -> Acc. You can experiment with some loop unrolling like inserting this clause on begin of il/3 and experiment with length of this unrolling il(<>, <>, Acc) -> il(RestA, RestB, <>); if it is not fast enough for you then try write NIF. On Sat, Aug 6, 2011 at 10:06 AM, Hynek Vychodil wrote: > It would not be so much efficient because it would make stack for > every two bytes of input. Much more efficient will be > > il(A, B) when is_binary(A), is_binary(B), size(A) =:= size(B) -> il(A, B, <<>>). > > il(<>, <>, Acc) -> > ? ?Rest=il(RestA, RestB), > ? ?il(RestA, RestB, <>); > il(<<>>, <<>>, Acc) -> Acc. > > See http://www.erlang.org/doc/efficiency_guide/binaryhandling.html#id64529 > for more info about accumulator efficiency trick. > > > On Sat, Aug 6, 2011 at 6:40 AM, Wes James wrote: >> Hi, >> >> On Fri, Aug 5, 2011 at 3:26 PM, Bob Cowdery wrote: >>> Hi >>> >>> I'm wanting to interleave two binaries. >>> >>> Bin1 = <<0,1>>, >>> Bin2 = <<2,3>>, >>> << <> || <> <= Bin1, <> <= Bin2 >>. >>> <<0,1,2,3>> >>> >>> Bin1 = <<0,1,2,3>>, >>> Bin2 = <<4,5,6,7>>, >>> << <> || <> <= Bin1, <> <= Bin2 >>. >>> <<0,1,4,5,0,1,6,7,2,3,4,5,2,3,6,7>> >>> >>> whereas I want: >>> <<0,1,4,5,2,3,6,7>> >> >> >> >> I don't know if this will suffice: >> >> ------------------------------------------------- >> >> % interleave module >> >> -module(ilm). >> >> -export([il/2]). >> >> % interleave function >> >> il(<>, <>) -> >> ? ?Rest=il(RestA, RestB), >> ? ?<>; >> >> il(<<>>, <<>>) -> <<>>. >> >> ----------------------------------------------- >> >> Erlang R14B02 (erts-5.8.3) [source] [smp:2:2] [rq:2] [async-threads:0] >> [hipe] [kernel-poll:false] >> >> Eshell V5.8.3 ?(abort with ^G) >> 1> c(ilm). >> {ok,ilm} >> 2> ilm:il(<<0,1,2,3>>, <<4,5,6,7>>). >> <<0,1,4,5,2,3,6,7>> >> >> ---------------------------------------------- >> >> -wes >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss.? Be a data hero! > Try GoodData now for free: www.gooddata.com > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From joelr1@REDACTED Mon Aug 8 14:17:09 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 8 Aug 2011 13:17:09 +0100 Subject: [erlang-questions] gen_fsm:send_event_after posts events in reverse order when timeout is zero Message-ID: I'm processing TCP events in handle_info of my gen_fsm and posting them for processing with gen_fsm:send_event_after, using a timeout of 0, e.g. handle_info({tcp, _Socket, Bin}, State, Bot = #bot{ cbk = Cbk, brain = Brain }) -> case pp:read(Bin) of none -> {next_state, State, Bot}; Event -> Cbk:process(Brain, Event), gen_fsm:send_event_after(0, Event), {next_state, State, Bot} end; It works as expected and the current state receives the event, except the order of events is reversed and I receive the last event first. Is this a bug or expected behavior? I'm using a workaround for the time being: ({?MODULE, State})(Event, Bot) This feels like a hack, though. Any suggestions for improvement? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From juanjo@REDACTED Mon Aug 8 17:40:20 2011 From: juanjo@REDACTED (Juan Jose Comellas) Date: Mon, 8 Aug 2011 12:40:20 -0300 Subject: [erlang-questions] [ANN] euuid 1.0.0 - an erlang UUID implementation In-Reply-To: References: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> Message-ID: Is there a formal procedure to submit new modules to OTP? I am the author of one implementation of getopt that some people are using and I'd like to submit it to be included in OTP. Are there any conditions that have to be met besides those of licensing and how useful it is? Juanjo On Sun, Aug 7, 2011 at 8:41 PM, Garrett Smith wrote: > On Mon, Aug 1, 2011 at 7:09 PM, Olivier Girondel > wrote: > > On Mon, Aug 1, 2011 at 8:37 PM, Martijn Rijkeboer > wrote: > >> Hello, > >> > >> As an exercise for learning Erlang I wrote an Universally Unique > >> IDentifier (UUID) implementation as defined in RFC 4122. > > Well, UUIDs seem to be: > > > > 1. A good exercise > > 2. Implemented over and over again (I'll just quote mine, but a good way > to > > check your code would be to have a look on older implementations) > > 3. A good candidate for inclusion somewhere in a library upstream :) > > +1 > > I wonder, at least two to three times a week, why foobar module isn't > a part of the OTP distribution. > > Admittedly, I'm not sure how that happens. > > Off the top of my head, a few things that I hate looking elsewhere for: > > gproc, json, bin_to_hex, uuid, urllib (URL utils ala mochiweb_util), getopt > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon Aug 8 21:23:11 2011 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 8 Aug 2011 21:23:11 +0200 Subject: [erlang-questions] Interleave binaries In-Reply-To: <4E3C5FF8.50709@bobcowdery.plus.com> References: <4E3C5FF8.50709@bobcowdery.plus.com> Message-ID: How about i(A,B) -> i(A,B,[]). i(<>,<>,L) -> i(AT, BT, [B2,B1,A2,A1|L]); i(<<>>,<<>>, L) -> list_to_binary(lists:reverse(L)). The good old "build-it-backwards and then reverse it " trick O(N) and tail-recursive. /Joe On Fri, Aug 5, 2011 at 11:26 PM, Bob Cowdery wrote: > Hi > > I'm wanting to interleave two binaries. > > Bin1 = <<0,1>>, > Bin2 = <<2,3>>, > << <> || <> <= Bin1, <> <= Bin2 >>. > <<0,1,2,3>> > > Bin1 = <<0,1,2,3>>, > Bin2 = <<4,5,6,7>>, > << <> || <> <= Bin1, <> <= Bin2 >>. > <<0,1,4,5,0,1,6,7,2,3,4,5,2,3,6,7>> > > whereas I want: > <<0,1,4,5,2,3,6,7>> > > I know what it does is what the documentation says but is there a way to > get a pure interleave efficiently. Perhaps there is a filter that would > work or a way to reduce the binary after merging without stepping > through the whole thing. > > Regards > Bob > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From erlang@REDACTED Mon Aug 8 21:23:11 2011 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 8 Aug 2011 21:23:11 +0200 Subject: [erlang-questions] Interleave binaries In-Reply-To: <4E3C5FF8.50709@bobcowdery.plus.com> References: <4E3C5FF8.50709@bobcowdery.plus.com> Message-ID: How about i(A,B) -> i(A,B,[]). i(<>,<>,L) -> i(AT, BT, [B2,B1,A2,A1|L]); i(<<>>,<<>>, L) -> list_to_binary(lists:reverse(L)). The good old "build-it-backwards and then reverse it " trick O(N) and tail-recursive. /Joe On Fri, Aug 5, 2011 at 11:26 PM, Bob Cowdery wrote: > Hi > > I'm wanting to interleave two binaries. > > Bin1 = <<0,1>>, > Bin2 = <<2,3>>, > << <> || <> <= Bin1, <> <= Bin2 >>. > <<0,1,2,3>> > > Bin1 = <<0,1,2,3>>, > Bin2 = <<4,5,6,7>>, > << <> || <> <= Bin1, <> <= Bin2 >>. > <<0,1,4,5,0,1,6,7,2,3,4,5,2,3,6,7>> > > whereas I want: > <<0,1,4,5,2,3,6,7>> > > I know what it does is what the documentation says but is there a way to > get a pure interleave efficiently. Perhaps there is a filter that would > work or a way to reduce the binary after merging without stepping > through the whole thing. > > Regards > Bob > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From joel.meyer@REDACTED Mon Aug 8 21:44:42 2011 From: joel.meyer@REDACTED (Joel Meyer) Date: Mon, 8 Aug 2011 12:44:42 -0700 Subject: [erlang-questions] Best approach to creating a process/connection pool In-Reply-To: References: Message-ID: If the things you are pooling are gen_servers, gen_server_pool may work for you (https://github.com/joelpm/gen_server_pool). That's a slight modification on a version I've used in production to pool thrift and protobuff conns to back-ends. To start a pool you'd simply do gen_server_pool:startlink( ..., GenServerPoolArgs). instead of doing gen_server:start_link(...). You would use the returned PID in the same way. I haven't had time to test that version, though, so YMMV. Joel 2011/8/4 Andrew Berman > Awesome! This makes sense. Thank you for the clarification on this. > > 2011/8/4 Fr?d?ric Trottier-H?bert : > > Technically, the OTP supervision tree is also used for application > upgrades (appups & relups). When sending messages to do the broad reloading > of code, the code in charge of that just traverses the VM's supervision > trees asking the supervisors what kind of modules they have and whether > they're workers and supervisors (that's why the child specs contain details > about workers, supervisors, [Modules] (and dynamic sending special messages > to figure out the same info). > > > > If you want to use appups/relups, you have to use the OTP supervision > trees otherwise you will have processes left behind to be purged rather than > upgraded. > > > > -- > > Fred H?bert > > http://www.erlang-solutions.com > > > > > > > > On 2011-08-04, at 13:39 PM, Hynek Vychodil wrote: > > > >> Using OTP supervisor tree is intended only for control of restarting > >> application parts. If you want anything else you have to use links or > >> monitors unless you write your own supervisor which is not recommended > >> way. > >> > >> On Thu, Aug 4, 2011 at 6:35 PM, Andrew Berman > wrote: > >>> Thanks Tim. So it looks like they are using erlang:monitor and > >>> demonitor, so is that the best approach in an OTP application? How > >>> does it figure into the supervision tree? > >>> > >>> > >>> > >>> On Thu, Aug 4, 2011 at 3:22 AM, Tim Watson > wrote: > >>>> Take a look at the sources for http://zotonic.com/ as they use a > >>>> decent connection pool. > >>>> > >>>> On 4 August 2011 08:01, Andrew Berman wrote: > >>>>> I'm fairly new to Erlang and I am trying to write a process pool > which > >>>>> essentially is a connection pool to a database. I have designed the > >>>>> application like this: The main pool gen_server has a queue of PIDs > >>>>> of gen_fsm's. Each gen_fsm calls spawn_links a 3rd party gen_server > and carries > >>>>> the state of the connection. So the 3rd party gen_server and the > >>>>> gen_fsm should always live and die together. I currently have set it > >>>>> up such that gen_fsm calls proc_lib:spawn_link() to start the 3rd > >>>>> party gen_server, but I'm wondering how this factors into the > >>>>> supervision tree. I currently have two supervisors: > >>>>> > >>>>> Sup1 -> one_for_all -> 2 children: pool gen_server and Sup2 > >>>>> Sup2 -> simple_one_for_one -> 1 child: gen_fsm > >>>>> > >>>>> So my question is: is it ok to use spawn_link instead of somehow > >>>>> trying to put the 3rd party gen_server into the supervision tree? If > >>>>> it should go into the supervision tree, how would I organize the tree > >>>>> then such that the gen_fsm and 3rd party gen_server are reliant on > >>>>> each other? > >>>>> > >>>>> Thanks for any help! > >>>>> _______________________________________________ > >>>>> erlang-questions mailing list > >>>>> erlang-questions@REDACTED > >>>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>>> > >>>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >> > >> > >> > >> -- > >> --Hynek (Pichi) Vychodil > >> > >> Analyze your data in minutes. Share your insights instantly. Thrill > >> your boss. Be a data hero! > >> Try GoodData now for free: www.gooddata.com > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From garry@REDACTED Mon Aug 8 21:45:47 2011 From: garry@REDACTED (Garry Hodgson) Date: Mon, 08 Aug 2011 15:45:47 -0400 Subject: [erlang-questions] suggestions for erlang drawing toolkit? Message-ID: <4E403CEB.2030105@research.att.com> for some visualization work i'm doing i need to be able to generate images from basic line/ellipse/text kind of primitives. we've been using PIL (Python Imaging Library) for this, but it'd be nicer to do this in just erlang. ideally i'd like to be able to generate PNG and SVG output. does anyone have any suggestions? thanks -- Garry Hodgson Lead Member of Technical Staff AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From wallentin.dahlberg@REDACTED Tue Aug 9 00:19:29 2011 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Tue, 9 Aug 2011 00:19:29 +0200 Subject: [erlang-questions] suggestions for erlang drawing toolkit? In-Reply-To: <4E403CEB.2030105@research.att.com> References: <4E403CEB.2030105@research.att.com> Message-ID: Hi Garry, Have a look at egd in the percept application, that might suit your need. Egd lacks a lot of features but if you only need rudimentary functions like ellipses, rectangles, thin lines and text (with one small font) it could be what you are looking for. Regards, Bj?rn-Egil 2011/8/8 Garry Hodgson > for some visualization work i'm doing i need to be able to generate > images from basic line/ellipse/text kind of primitives. we've been > using PIL (Python Imaging Library) for this, but it'd be nicer to do > this in just erlang. ideally i'd like to be able to generate PNG and > SVG output. > > does anyone have any suggestions? > > thanks > > > -- > Garry Hodgson > Lead Member of Technical Staff > AT&T Chief Security Office (CSO) > > "This e-mail and any files transmitted with it are AT&T property, are > confidential, and are intended solely for the use of the individual or > entity to whom this e-mail is addressed. If you are not one of the named > recipient(s) or otherwise have reason to believe that you have received this > message in error, please notify the sender and delete this message > immediately from your computer. Any other use, retention, dissemination, > forwarding, printing, or copying of this e-mail is strictly prohibited." > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Tue Aug 9 08:27:36 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Tue, 9 Aug 2011 09:27:36 +0300 Subject: [erlang-questions] how to run PropEr tests from Eunit? In-Reply-To: References: <1308867174.83770.YahooMailRC@web25402.mail.ukl.yahoo.com> <93BD232D-C14F-4B16-80D2-96C24EE44BB3@erlang-solutions.com> <1308939214.73194.YahooMailRC@web25404.mail.ukl.yahoo.com> Message-ID: <20110809062736.GA7058@jakstys.lt> On Mon, Jul 25, 2011 at 08:59:22AM -0400, Fred Hebert wrote: > You can just change the process' group leader back to the 'user' process (or > any other process depending on your app) from within the test, and then set > it back to what it was before using erlang:group_leader/0 and > erlang:group_leader/2. > Could you elaborate more on this? I couldn't figure out how to create a new group leader (erlang manpage does not say much). An example would be very helpful. Thank you. Motiejus From ulf.wiger@REDACTED Tue Aug 9 08:32:04 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 9 Aug 2011 08:32:04 +0200 Subject: [erlang-questions] Best approach to creating a process/connection pool In-Reply-To: References: Message-ID: <6EC89159-B65F-43E5-A2DC-D0BB7ADBA143@erlang-solutions.com> The Jobs load regulation system supports a pool concept. Actually, it allows you to combine concepts in a number of ways to tune characteristics. Here is one example, with a 'producer pool' and a passive queue. The producer pool is defined with a fun that is called to top up the pool. The producers are free to do pretty much what they want, and jobs will monitor them to ensure they are replaced if they die. https://gist.github.com/778319 In this case, the producers fetch jobs from a passive queue. In Jobs, that's a queue where you have to explicitly dequeue them. Otherwise, you typically tell jobs at which rate it should draw from the queue. Rates and pool sizes can be made to adapt automatically to load, using load sampler plugins. Configuration/re-configuration can also be done at runtime through the Jobs API. http://github.com/esl/jobs BR, Ulf W On 8 Aug 2011, at 21:44, Joel Meyer wrote: > If the things you are pooling are gen_servers, gen_server_pool may work for you (https://github.com/joelpm/gen_server_pool). That's a slight modification on a version I've used in production to pool thrift and protobuff conns to back-ends. To start a pool you'd simply do gen_server_pool:startlink( ..., GenServerPoolArgs). instead of doing gen_server:start_link(...). You would use the returned PID in the same way. I haven't had time to test that version, though, so YMMV. > > Joel > > 2011/8/4 Andrew Berman > Awesome! This makes sense. Thank you for the clarification on this. > > 2011/8/4 Fr?d?ric Trottier-H?bert : > > Technically, the OTP supervision tree is also used for application upgrades (appups & relups). When sending messages to do the broad reloading of code, the code in charge of that just traverses the VM's supervision trees asking the supervisors what kind of modules they have and whether they're workers and supervisors (that's why the child specs contain details about workers, supervisors, [Modules] (and dynamic sending special messages to figure out the same info). > > > > If you want to use appups/relups, you have to use the OTP supervision trees otherwise you will have processes left behind to be purged rather than upgraded. > > > > -- > > Fred H?bert > > http://www.erlang-solutions.com > > > > > > > > On 2011-08-04, at 13:39 PM, Hynek Vychodil wrote: > > > >> Using OTP supervisor tree is intended only for control of restarting > >> application parts. If you want anything else you have to use links or > >> monitors unless you write your own supervisor which is not recommended > >> way. > >> > >> On Thu, Aug 4, 2011 at 6:35 PM, Andrew Berman wrote: > >>> Thanks Tim. So it looks like they are using erlang:monitor and > >>> demonitor, so is that the best approach in an OTP application? How > >>> does it figure into the supervision tree? > >>> > >>> > >>> > >>> On Thu, Aug 4, 2011 at 3:22 AM, Tim Watson wrote: > >>>> Take a look at the sources for http://zotonic.com/ as they use a > >>>> decent connection pool. > >>>> > >>>> On 4 August 2011 08:01, Andrew Berman wrote: > >>>>> I'm fairly new to Erlang and I am trying to write a process pool which > >>>>> essentially is a connection pool to a database. I have designed the > >>>>> application like this: The main pool gen_server has a queue of PIDs > >>>>> of gen_fsm's. Each gen_fsm calls spawn_links a 3rd party gen_server and carries > >>>>> the state of the connection. So the 3rd party gen_server and the > >>>>> gen_fsm should always live and die together. I currently have set it > >>>>> up such that gen_fsm calls proc_lib:spawn_link() to start the 3rd > >>>>> party gen_server, but I'm wondering how this factors into the > >>>>> supervision tree. I currently have two supervisors: > >>>>> > >>>>> Sup1 -> one_for_all -> 2 children: pool gen_server and Sup2 > >>>>> Sup2 -> simple_one_for_one -> 1 child: gen_fsm > >>>>> > >>>>> So my question is: is it ok to use spawn_link instead of somehow > >>>>> trying to put the 3rd party gen_server into the supervision tree? If > >>>>> it should go into the supervision tree, how would I organize the tree > >>>>> then such that the gen_fsm and 3rd party gen_server are reliant on > >>>>> each other? > >>>>> > >>>>> Thanks for any help! > >>>>> _______________________________________________ > >>>>> erlang-questions mailing list > >>>>> erlang-questions@REDACTED > >>>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>>> > >>>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >> > >> > >> > >> -- > >> --Hynek (Pichi) Vychodil > >> > >> Analyze your data in minutes. Share your insights instantly. Thrill > >> your boss. Be a data hero! > >> Try GoodData now for free: www.gooddata.com > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From djreid@REDACTED Tue Aug 9 08:58:38 2011 From: djreid@REDACTED (Darryn) Date: Tue, 9 Aug 2011 16:28:38 +0930 Subject: [erlang-questions] Apparently strange dict behaviour Message-ID: <201108091628.38567.djreid@aapt.net.au> Kind Sirs/Madams, I'm new to erlang, but have extensive prior experience with Haskell. I thought I'd play with a small version of a parallel map/fold operation, but have run into an annoying problem. The test example is just a parallel estimation by sampling of Pi. pmap(Input, MapFn) -> process_flag(trap_exit, true), spawn_workers(self(), MapFn, Input), M = length(Input), collect(M, dict:new()). spawn_workers(Pid, Fn, Pairs) -> lists:foreach(fun({K1, V1}) -> spawn_link(fun() -> worker(Pid, Fn, {K1, V1}) end) end, Pairs). worker(Pid, Fn, {K1, V1}) -> Pid ! Fn(K1, V1). collect(0, Dict) -> Dict; collect(N, Dict) -> receive {Key2, Val2} -> DictD = dict:append(Key2, Val2, Dict), %% <- problem: inserted Val2 integer apparently turns into a string?!! io:format("Collected result:{~p,~p}~n", [Key2, Val2]), io:format(" ->is_integer?~p~n", [is_integer(Val2)]), io:format(" ->~p~n", [dict:to_list(DictD)]), collect(N, DictD); {'EXIT', Who, Why} -> io:format("Worker ~p terminated with ~p~n",[Who, Why]), collect(N-1, Dict) end. samples(Index, N) -> random:seed(erlang:now()), {Index, do_samples(0, N, 0)}. doit() -> Results = pmap([{1,100}, {2,100}, {3, 100}], fun samples/2), dict:to_list(Results). The problem seems to lie in the collect function, where the collected Val2 integer (the io:format confirms it is an integer) suddenly becomes a string upon insertion into the dictionary. Here is example output to illustrate. 20> c(pisim). {ok,pisim} 21> pisim:doit(). Collected result:{1,88} ->is_integer?true ->[{1,"X"}] Worker <0.114.0> terminated with normal Collected result:{1,87} ->is_integer?true ->[{1,"XW"}] Worker <0.116.0> terminated with normal Collected result:{2,76} ->is_integer?true ->[{2,"L"},{1,"XW"}] Worker <0.115.0> terminated with normal [{2,"L"},{1,"XW"}] 22> c(pisim). {ok,pisim} 23> pisim:doit(). Worker <0.118.0> terminated with normal Collected result:{1,78} ->is_integer?true ->[{1,"N"}] Worker <0.123.0> terminated with normal Collected result:{2,83} ->is_integer?true ->[{2,"S"},{1,"N"}] Worker <0.124.0> terminated with normal [{2,"S"},{1,"N"}] I don't understand how the Val2 integer suddenly turns into a string upon being inserted into the dictionary; when I manually cut and paste the relevant bits of code into the shell I get dictionary values with integers as I intend. I wonder if anyone more experienced can see what I've done wrong? Thanks so much for any help! Darryn. From carlsson.richard@REDACTED Tue Aug 9 09:16:59 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 09 Aug 2011 09:16:59 +0200 Subject: [erlang-questions] suggestions for erlang drawing toolkit? In-Reply-To: <4E403CEB.2030105@research.att.com> References: <4E403CEB.2030105@research.att.com> Message-ID: <4E40DEEB.6070306@gmail.com> On 08/08/2011 09:45 PM, Garry Hodgson wrote: > for some visualization work i'm doing i need to be able to generate > images from basic line/ellipse/text kind of primitives. we've been > using PIL (Python Imaging Library) for this, but it'd be nicer to do > this in just erlang. ideally i'd like to be able to generate PNG and > SVG output. If one of the formats you want is SVG, you presumably don't wan't to generate a raster image as the master output. So why not generate SVG directly? Using xmerl and the "simple" xml representation, the following example: becomes: [{rect, [{width,"300"},{height,"100"}, {style,"fill:rgb(0,0,255);stroke-width:1;stroke:rgb(0,0,0)"}], [],}] Then export this via xmerl to an XML file "foo.svg", and if you want a bitmap, run os:cmd("convert foo.svg foo.png") where 'convert' is the ImageMagick convert command. Just install the imagemagick package if you don' have it already. /Richard From raimo+erlang-questions@REDACTED Tue Aug 9 09:39:14 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 9 Aug 2011 09:39:14 +0200 Subject: [erlang-questions] Apparently strange dict behaviour In-Reply-To: <201108091628.38567.djreid@aapt.net.au> References: <201108091628.38567.djreid@aapt.net.au> Message-ID: <20110809073914.GA15823@erix.ericsson.se> Hi. I am sorry I did not have time to read your post thoroughly, because I think you just ran into the feature of the shell (io:format("~p", ...)) being "intelligent" and prints lists of integers as strings if the integers can represent printable characters. There is no character type in Erlang, integers are used instead, and lists of integers instead of strings. (or binaries instead of strings but that is another story) dict:append/3 I think creates a list of the values, your values are integers, hence you get something that is mistaken for a string. But as I said I did not read thoroughly... / Raimo On Tue, Aug 09, 2011 at 04:28:38PM +0930, Darryn wrote: > Kind Sirs/Madams, > > I'm new to erlang, but have extensive prior experience with Haskell. I thought > I'd play with a small version of a parallel map/fold operation, but have run > into an annoying problem. The test example is just a parallel estimation by > sampling of Pi. > > pmap(Input, MapFn) -> > process_flag(trap_exit, true), > spawn_workers(self(), MapFn, Input), > M = length(Input), > collect(M, dict:new()). > > spawn_workers(Pid, Fn, Pairs) -> > lists:foreach(fun({K1, V1}) -> > spawn_link(fun() -> worker(Pid, Fn, {K1, V1}) end) end, Pairs). > > worker(Pid, Fn, {K1, V1}) -> > Pid ! Fn(K1, V1). > > collect(0, Dict) -> Dict; > collect(N, Dict) -> > receive > {Key2, Val2} -> > DictD = dict:append(Key2, Val2, Dict), > %% <- problem: inserted Val2 integer apparently turns into a string?!! > io:format("Collected result:{~p,~p}~n", [Key2, Val2]), > io:format(" ->is_integer?~p~n", [is_integer(Val2)]), > io:format(" ->~p~n", [dict:to_list(DictD)]), > collect(N, DictD); > {'EXIT', Who, Why} -> > io:format("Worker ~p terminated with ~p~n",[Who, Why]), > collect(N-1, Dict) > end. > > samples(Index, N) -> > random:seed(erlang:now()), > {Index, do_samples(0, N, 0)}. > > doit() -> > Results = pmap([{1,100}, {2,100}, {3, 100}], fun samples/2), > dict:to_list(Results). > > > The problem seems to lie in the collect function, where the collected Val2 > integer (the io:format confirms it is an integer) suddenly becomes a string > upon insertion into the dictionary. Here is example output to illustrate. > > 20> c(pisim). > {ok,pisim} > 21> pisim:doit(). > Collected result:{1,88} > ->is_integer?true > ->[{1,"X"}] > Worker <0.114.0> terminated with normal > Collected result:{1,87} > ->is_integer?true > ->[{1,"XW"}] > Worker <0.116.0> terminated with normal > Collected result:{2,76} > ->is_integer?true > ->[{2,"L"},{1,"XW"}] > Worker <0.115.0> terminated with normal > [{2,"L"},{1,"XW"}] > 22> c(pisim). > {ok,pisim} > 23> pisim:doit(). > Worker <0.118.0> terminated with normal > Collected result:{1,78} > ->is_integer?true > ->[{1,"N"}] > Worker <0.123.0> terminated with normal > Collected result:{2,83} > ->is_integer?true > ->[{2,"S"},{1,"N"}] > Worker <0.124.0> terminated with normal > [{2,"S"},{1,"N"}] > > I don't understand how the Val2 integer suddenly turns into a string upon > being inserted into the dictionary; when I manually cut and paste the relevant > bits of code into the shell I get dictionary values with integers as I intend. > > I wonder if anyone more experienced can see what I've done wrong? Thanks so > much for any help! > > Darryn. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From hynek@REDACTED Tue Aug 9 10:52:37 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Tue, 9 Aug 2011 10:52:37 +0200 Subject: [erlang-questions] Interleave binaries In-Reply-To: References: <4E3C5FF8.50709@bobcowdery.plus.com> Message-ID: Just for curiosity I measured performance of different codes during train travel and here are fastest implementations on mine notebook (model name : Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz): il_16(A, B) when is_binary(A), is_binary(B), size(A) =:= size(B) -> il_16(A, B, <<>>). il_16(<< A1:16, A2:16, A3:16, A4:16, A5:16, A6:16, A7:16, A8:16, A9:16, A10:16, A11:16, A12:16, A13:16, A14:16, A15:16, A16:16, RestA/binary>>, << B1:16, B2:16, B3:16, B4:16, B5:16, B6:16, B7:16, B8:16, B9:16, B10:16, B11:16, B12:16, B13:16, B14:16, B15:16, B16:16, RestB/binary>>, Acc) -> il_16(RestA, RestB, <>); il_16(<>, <>, Acc) -> il_16(RestA, RestB, <>); il_16(<<>>, <<>>, Acc) -> Acc. il_32(A, B) when is_binary(A), is_binary(B), size(A) =:= size(B) -> il_32(A, B, <<>>). il_32(<< A1:16, A2:16, A3:16, A4:16, A5:16, A6:16, A7:16, A8:16, A9:16, A10:16, A11:16, A12:16, A13:16, A14:16, A15:16, A16:16, A17:16, A18:16, A19:16, A20:16, A21:16, A22:16, A23:16, A24:16, A25:16, A26:16, A27:16, A28:16, A29:16, A30:16, A31:16, A32:16, RestA/binary>>, << B1:16, B2:16, B3:16, B4:16, B5:16, B6:16, B7:16, B8:16, B9:16, B10:16, B11:16, B12:16, B13:16, B14:16, B15:16, B16:16, B17:16, B18:16, B19:16, B20:16, B21:16, B22:16, B23:16, B24:16, B25:16, B26:16, B27:16, B28:16, B29:16, B30:16, B31:16, B32:16, RestB/binary>>, Acc) -> il_32(RestA, RestB, <>); il_32(<>, <>, Acc) -> il_32(RestA, RestB, <>); il_32(<<>>, <<>>, Acc) -> Acc. il_64(A, B) when is_binary(A), is_binary(B), size(A) =:= size(B) -> il_64(A, B, <<>>). il_64(<< A1:16, A2:16, A3:16, A4:16, A5:16, A6:16, A7:16, A8:16, A9:16, A10:16, A11:16, A12:16, A13:16, A14:16, A15:16, A16:16, A17:16, A18:16, A19:16, A20:16, A21:16, A22:16, A23:16, A24:16, A25:16, A26:16, A27:16, A28:16, A29:16, A30:16, A31:16, A32:16, A33:16, A34:16, A35:16, A36:16, A37:16, A38:16, A39:16, A40:16, A41:16, A42:16, A43:16, A44:16, A45:16, A46:16, A47:16, A48:16, A49:16, A50:16, A51:16, A52:16, A53:16, A54:16, A55:16, A56:16, A57:16, A58:16, A59:16, A60:16, A61:16, A62:16, A63:16, A64:16, RestA/binary>>, << B1:16, B2:16, B3:16, B4:16, B5:16, B6:16, B7:16, B8:16, B9:16, B10:16, B11:16, B12:16, B13:16, B14:16, B15:16, B16:16, B17:16, B18:16, B19:16, B20:16, B21:16, B22:16, B23:16, B24:16, B25:16, B26:16, B27:16, B28:16, B29:16, B30:16, B31:16, B32:16, B33:16, B34:16, B35:16, B36:16, B37:16, B38:16, B39:16, B40:16, B41:16, B42:16, B43:16, B44:16, B45:16, B46:16, B47:16, B48:16, B49:16, B50:16, B51:16, B52:16, B53:16, B54:16, B55:16, B56:16, B57:16, B58:16, B59:16, B60:16, B61:16, B62:16, B63:16, B64:16, RestB/binary>>, Acc) -> il_64(RestA, RestB, <>); il_64(<>, <>, Acc) -> il_64(RestA, RestB, <>); il_64(<<>>, <<>>, Acc) -> Acc. 1> c(interleave, [native]). {ok,interleave} 2> {ok, A} = file:read_file(uniprot_sprot.fasta). {ok,<<">sp|Q197F8|002R_IIV3 Uncharacterized protein 002R OS=Invertebrate iridescent virus 3 GN=IIV3-002R PE=4 SV=1\n"...>>} 3> {ok, B} = file:read_file(uniprot_sprot2.fasta). {ok,<<">sp|Q197F8|002R_IIV3 Uncharacterized protein 002R OS=Invertebrate iridescent virus 3 GN=IIV3-002R PE=4 SV=1\n"...>>} 6> [{M, 20*size(A)/lists:sum([element(1, timer:tc(interleave, M, [A,B])) || _<-lists:seq(1,10)])} || M<-[il_16, il_32, il_64]]. [{il_16,181.14907627840208}, {il_32,193.28636400539335}, {il_64,184.31082640385935}] 7> c(interleave). {ok,interleave} 8> [{M, 20*size(A)/lists:sum([element(1, timer:tc(interleave, M, [A,B])) || _<-lists:seq(1,10)])} || M<-[il_16, il_32, il_64]]. [{il_16,87.1408557475168}, {il_32,93.51646799197587}, {il_64,98.28698318757078}] 9> size(A). 242838416 So testing data size was 231MB and achieved performance is almost 200MB/s in native and 100MB/s in byte code which surprised me. All other so far published implementations are at least order of magnitude slower. So I would use il_32 implementation. On Mon, Aug 8, 2011 at 9:23 PM, Joe Armstrong wrote: > How about > > i(A,B) -> i(A,B,[]). > > i(<>,<>,L) -> > ? ?i(AT, BT, [B2,B1,A2,A1|L]); > i(<<>>,<<>>, L) -> > ? ?list_to_binary(lists:reverse(L)). > > The good old "build-it-backwards and then reverse it " trick > O(N) and tail-recursive. > > > /Joe > > On Fri, Aug 5, 2011 at 11:26 PM, Bob Cowdery wrote: >> Hi >> >> I'm wanting to interleave two binaries. >> >> Bin1 = <<0,1>>, >> Bin2 = <<2,3>>, >> << <> || <> <= Bin1, <> <= Bin2 >>. >> <<0,1,2,3>> >> >> Bin1 = <<0,1,2,3>>, >> Bin2 = <<4,5,6,7>>, >> << <> || <> <= Bin1, <> <= Bin2 >>. >> <<0,1,4,5,0,1,6,7,2,3,4,5,2,3,6,7>> >> >> whereas I want: >> <<0,1,4,5,2,3,6,7>> >> >> I know what it does is what the documentation says but is there a way to >> get a pure interleave efficiently. Perhaps there is a filter that would >> work or a way to reduce the binary after merging without stepping >> through the whole thing. >> >> Regards >> Bob >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From torben.lehoff@REDACTED Tue Aug 9 10:57:07 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Tue, 9 Aug 2011 10:57:07 +0200 Subject: [erlang-questions] suggestions for erlang drawing toolkit? In-Reply-To: <4E403CEB.2030105@research.att.com> References: <4E403CEB.2030105@research.att.com> Message-ID: Hi Garry, I have created call graphs and similar for my own code using graphviz. The dot language is very easy to work with and you can generate SVG using graphviz. You will have to live with the way graphviz decides to lay out your drawing, but in most cases it is actually okay and you can insert some control commands to guide it if it is not suiting your needs. Cheers, Torben On Mon, Aug 8, 2011 at 21:45, Garry Hodgson wrote: > for some visualization work i'm doing i need to be able to generate > images from basic line/ellipse/text kind of primitives. we've been > using PIL (Python Imaging Library) for this, but it'd be nicer to do > this in just erlang. ideally i'd like to be able to generate PNG and > SVG output. > > does anyone have any suggestions? > > thanks > > > -- > Garry Hodgson > Lead Member of Technical Staff > AT&T Chief Security Office (CSO) > > "This e-mail and any files transmitted with it are AT&T property, are > confidential, and are intended solely for the use of the individual or > entity to whom this e-mail is addressed. If you are not one of the named > recipient(s) or otherwise have reason to believe that you have received this > message in error, please notify the sender and delete this message > immediately from your computer. Any other use, retention, dissemination, > forwarding, printing, or copying of this e-mail is strictly prohibited." > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Tue Aug 9 11:48:46 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Tue, 9 Aug 2011 11:48:46 +0200 Subject: [erlang-questions] what does l/1 really do? In-Reply-To: References: Message-ID: Hi Daniel! I also have a more specific question. I have a whole bunch of gen_fsm > processes that I *believe* have a common supervisor but I won't swear to it. > If I want to upgrade the module, is there a way to institute the code change > right away across all those processes? > > Most of this has already been said in this thread, I'm just adding some details... If you do l/1 in the shell, the new version of your modules will be loaded and taken into use the next time a fully qualified call is made to this module. If your module implements an OTP behaviour, this will happen e.g. the next time a message is passed to the process. If the new version of the module includes e.g. a change of the state record, the process might crash (if you try to match the state passed from the behaviour against your new data structure). To avoid this, you must either make your module totally backwards compatible, or implement a code_change function and make sure this is called... preferably by using the release_handler and an appup script. If you have changed the interface of the module in a non-backwards compatible way, you must make sure that all other modules using this interface are also updated. Such dependencies can also be specified in an appup script. So, what does the release_handler do?? First, all appup scripts must be converted to one single relup script - systools:make_relup. When generating a relup, systools will first generate a directed graph of all the dependencies from the appup instructions. Based on this graph, the depending modules will be grouped together, and e.g. an update instruction for a module will be converted to a {suspend,DepMods} - load - code_change - {resume,DepMods} sequence in the relup. During upgrade, the release_handler will traverse the complete supervision tree and look at the module list in the child specs in order to find all processes that have any of the DepMods as their callback. Load and code_change instructions are then executed (both for the module in the original upgrade instruction, and for the modules it depends on, if such instructions exist). Finally, all processes are resumed. So, to answer the question - l/1 will do the job, but only for the simplest case where your update is totally backwards compatible and there are no dependencies towards other modules. Regards /siri -------------- next part -------------- An HTML attachment was scrubbed... URL: From fred.hebert@REDACTED Tue Aug 9 12:53:09 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Tue, 9 Aug 2011 06:53:09 -0400 Subject: [erlang-questions] how to run PropEr tests from Eunit? In-Reply-To: <20110809062736.GA7058@jakstys.lt> References: <1308867174.83770.YahooMailRC@web25402.mail.ukl.yahoo.com> <93BD232D-C14F-4B16-80D2-96C24EE44BB3@erlang-solutions.com> <1308939214.73194.YahooMailRC@web25404.mail.ukl.yahoo.com> <20110809062736.GA7058@jakstys.lt> Message-ID: The Group Leader is a special concept. In each process' metadata, there is an entry for a group leader. The group leader, at the moment, is a Pid that is in charge of handling IO requests using the IO protocol (http://www.erlang.org/doc/apps/stdlib/io_protocol.html). This group leader is the 'user' process by default (that's its registered name). When you send io requests to the user process, the output gets printed to the shell. By default, all processes get the 'user' group leader -- this is set through inheritance when spawning a process. The exception to that is when you're using OTP application. Then the Application Master (a process spawned by the application controller) sets itself as a middleman between the user process and your own processes. That lets OTP create process groups to shut down in case it's needed. In any case, applications like EUnit have decided to overwrite these default group leaders and make their own -- this lets them capture input the way they want to work with it, log it, or just hide it. This makes some macros such as 'assertCmdOutput(Text, CommandString)' work. To bypass this, the framework then gives macros like ?debugVal(Val) and ?debugMsg(Text) to the programmer. All these macros do is something like 'io:format(user, String, Args)'. That's because adding a first argument of this kind lets you redirect IO to an io-device (file descriptor, group leader, etc.). You can, however, use the following functions to just plainly overwrite the group leader back to something new: http://erldocs.com/R14B02/erts/erlang.html?i=0&search=group_leader#group_leader/0 http://erldocs.com/R14B02/erts/erlang.html?i=1&search=group_leader#group_leader/2 You could do something such as: run_proper_test_() -> EunitLeader = erlang:group_leader(), erlang:group_leader(whereis(user), self()), Res = proper:module(?MODULE), erlang:group_leader(EunitLeader, self()), ?_assertEqual([], Res). (untested). This would temporarily switch the group leader of the test for the time being before setting it back to whatever it was after. So for the time PropEr tests are running you get regular output, and then it gets back to Eunit-normal afterwards. Hope this helps. -- Fred H?bert http://www.erlang-solutions.com On 2011-08-09, at 02:27 AM, Motiejus Jak?tys wrote: > On Mon, Jul 25, 2011 at 08:59:22AM -0400, Fred Hebert wrote: >> You can just change the process' group leader back to the 'user' process (or >> any other process depending on your app) from within the test, and then set >> it back to what it was before using erlang:group_leader/0 and >> erlang:group_leader/2. >> > > Could you elaborate more on this? I couldn't figure out how to create a > new group leader (erlang manpage does not say much). An example would be > very helpful. > > Thank you. > > Motiejus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlangsiri@REDACTED Tue Aug 9 15:05:07 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Tue, 9 Aug 2011 15:05:07 +0200 Subject: [erlang-questions] relup restart_new_emulator does not work for me In-Reply-To: References: Message-ID: Hi Allen! Can you see an erl_crash.dump file after this? Can you start the 5.8.4 emulator manually? Regards /siri 2011/8/3 Allen Kim > Hi, > > I want to upgrade erlang otp from R13B04 to R43B03 > > I used this start_erl.data and relup(recommended from > > [tracking@REDACTED releases]$ cat new_start_erl.data > 5.8.4 2.0.0 > > [tracking@REDACTED releases]$ cat current/relup > {"2.0.0", > [{"1.9.0", > [], > [restart_new_emulator]}], > [{"1.9.0", > [], > [restart_new_emulator]}] > }. > > when I run command release_handler:install_release("2.0.0") > I see the node shutting down messages, but I don't see a new node start > up.(looks like heart is not starting up a new node). > > My HEART_COMMAND is coreect, since I was able to kill the node and I see it > starting up again. > > The reason why I said restart_new_emulator not working is I could reboot > the node with command init:reboot(). > > Any help would be appreciated. > > Allen Kim > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Tue Aug 9 16:12:17 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 09 Aug 2011 15:12:17 +0100 (BST) Subject: [erlang-questions] suggestions for erlang drawing toolkit? In-Reply-To: Message-ID: <2f70da6e-d362-47d4-90e8-757f64b23ed9@localhost> If you want some example code for generating DOT files look towards the end of leex.erl (scanner generator), out_dfa_graph/3, where there is some code to generate a DOT file showing the internal DFA as a graph. It is only a limited example but gives you some indication how it is done. Robert ----- Original Message ----- > Hi Garry, > I have created call graphs and similar for my own code using > graphviz. > The dot language is very easy to work with and you can generate SVG > using graphviz. > You will have to live with the way graphviz decides to lay out your > drawing, but in most cases it is actually okay and you can insert > some control commands to guide it if it is not suiting your needs. > Cheers, > Torben > On Mon, Aug 8, 2011 at 21:45, Garry Hodgson < garry@REDACTED > > wrote: > > for some visualization work i'm doing i need to be able to generate > > > images from basic line/ellipse/text kind of primitives. we've been > > > using PIL (Python Imaging Library) for this, but it'd be nicer to > > do > > > this in just erlang. ideally i'd like to be able to generate PNG > > and > > > SVG output. > > > does anyone have any suggestions? > > > thanks > > > -- > > > Garry Hodgson > > > Lead Member of Technical Staff > > > AT&T Chief Security Office (CSO) > > > "This e-mail and any files transmitted with it are AT&T property, > > are > > confidential, and are intended solely for the use of the individual > > or entity to whom this e-mail is addressed. If you are not one of > > the named recipient(s) or otherwise have reason to believe that you > > have received this message in error, please notify the sender and > > delete this message immediately from your computer. Any other use, > > retention, dissemination, forwarding, printing, or copying of this > > e-mail is strictly prohibited." > > > ______________________________ _________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/ listinfo/erlang-questions > > -- > http://www.linkedin.com/in/torbenhoffmann > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From allen.kim@REDACTED Tue Aug 9 16:14:31 2011 From: allen.kim@REDACTED (Allen Kim) Date: Tue, 9 Aug 2011 09:14:31 -0500 Subject: [erlang-questions] relup restart_new_emulator does not work for me In-Reply-To: Message-ID: Siri, No, I don't see erl_crash.dump, or I could not find it. Yes, I could start 5.8.4 emulator manually. The weird thing is after I start manually, then I run command, release_handler:install_release("2.0.0") It gives me an error, which I don't remember. Now, I decided to upgrade erlang version manually, that means previous plan: 1. run command release_handler:install_release("2.0.0") new plan: 1. update RELEASE file manually by changing 1.9.0 old and 2.0.0 permanent 2. run command init:reboot() The new plan seems working, but I was curious why the previous plan did not work. Hope this helps From: Siri Hansen > Date: Tue, 9 Aug 2011 08:05:07 -0500 To: Allen Kim > Cc: erlang-questions > Subject: Re: [erlang-questions] relup restart_new_emulator does not work for me Hi Allen! Can you see an erl_crash.dump file after this? Can you start the 5.8.4 emulator manually? Regards /siri 2011/8/3 Allen Kim > Hi, I want to upgrade erlang otp from R13B04 to R43B03 I used this start_erl.data and relup(recommended from [tracking@REDACTED releases]$ cat new_start_erl.data 5.8.4 2.0.0 [tracking@REDACTED releases]$ cat current/relup {"2.0.0", [{"1.9.0", [], [restart_new_emulator]}], [{"1.9.0", [], [restart_new_emulator]}] }. when I run command release_handler:install_release("2.0.0") I see the node shutting down messages, but I don't see a new node start up.(looks like heart is not starting up a new node). My HEART_COMMAND is coreect, since I was able to kill the node and I see it starting up again. The reason why I said restart_new_emulator not working is I could reboot the node with command init:reboot(). Any help would be appreciated. Allen Kim _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From maggio.vincenzo@REDACTED Tue Aug 9 16:16:39 2011 From: maggio.vincenzo@REDACTED (Vincenzo Maggio) Date: Tue, 9 Aug 2011 07:16:39 -0700 (PDT) Subject: [erlang-questions] my best erlang "foot in mouth" In-Reply-To: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> References: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> Message-ID: <1312899399819-3730042.post@n4.nabble.com> lists:seq(1:100). Erlang answers with "illegal expression" -> an afternoon of code review :( And I STILL continue to write this!!! -- View this message in context: http://erlang.2086793.n4.nabble.com/my-best-erlang-foot-in-mouth-tp3710188p3730042.html Sent from the Erlang Questions mailing list archive at Nabble.com. From crusso@REDACTED Tue Aug 9 17:24:30 2011 From: crusso@REDACTED (Claudio Russo) Date: Tue, 9 Aug 2011 15:24:30 +0000 Subject: [erlang-questions] Second CFP: PADL'12 - Practical Aspects of Declarative Languages 2012 Message-ID: <88D1F4047EA9A2468BCC6B743186880F6404A427@DB3EX14MBXC311.europe.corp.microsoft.com> [Apologies if you receive multiple copies.] Call for Papers =============== 14th International Symposium on Practical Aspects of Declarative Languages (PADL 2012) http://research.microsoft.com/~crusso/padl12 Philadelphia, Pennsylvania, USA, January 23-24, 2012 Co-located with ACM POPL'12 Abstract/Paper submission deadline: September 10th/17th, 2011 Conference Description ====================== Declarative languages build on sound theoretical bases to provide attractive frameworks for application development. These languages have been successfully applied to many different real-world situations, ranging from data base management to active networks to software engineering to decision support systems. New developments in theory and implementation have opened up new application areas. At the same time, applications of declarative languages to novel problems raise numerous interesting research issues. Well-known questions include designing for scalability, language extensions for application deployment, and programming environments. Thus, applications drive the progress in the theory and implementation of declarative systems, and benefit from this progress as well. PADL is a forum for researchers and practitioners to present original work emphasizing novel applications and implementation techniques for all forms of declarative concepts, including, functional, logic, constraints, etc. Topics of interest include, but are not limited to: * Innovative applications of declarative languages * Declarative domain-specific languages and applications * Practical applications of theoretical results * New language developments and their impact on applications * Declarative languages and Software Engineering * Evaluation of implementation techniques on practical applications * Practical experiences and industrial applications * Novel uses of declarative languages in the classroom * Practical extensions such as constraint-based, probabilistic, and reactive languages. PADL'12 welcomes new ideas and approaches pertaining to applications and implementation of declarative languages. In this occasion PADL is co-located, as traditionally, with ACM POPL, which will be held immediately following PADL, January 25-27. The symposium will be held in Philadelphia, Pennsylvania, USA. Important Dates and Submission Guidelines ========================================= Abstract Submission: September 10, 2011 Paper Submission: September 17, 2011 Notification: October 22, 2011 Camera-ready: November 5, 2011 Symposium: January 23-24, 2012 Authors should submit an electronic copy of the full paper in PDF using the Springer LNCS format. The submission will be done through EasyChair conference system. If electronic submission is impossible, please contact the program chairs for information on how to submit hard copies. All submissions must be original work written in English. Submissions must be unpublished and not submitted for publication elsewhere. Work that already appeared in unpublished or informally published workshops proceedings may be submitted. PADL'12 will accept both technical and application papers: * Technical papers must describe original, previously unpublished research results. Technical papers must not exceed 15 pages in Springer LNCS format. * Application papers are a mechanism to present important practical applications of declarative languages that occur in industry or in areas of research other than Computer Science. Application papers will be published in the Springer-Verlag conference proceedings, and will be presented in a separate session. Application papers are expected to describe complex and/or real-world applications that rely on an innovative use of declarative languages. Application descriptions, engineering solutions and real-world experiences (both positive and negative) are solicited. The limit for application papers is 6 pages in Springer LNCS format. Program Committee ================= Marcello Balduccini, Intelligent Systems Department, Kodak Research Labs Edwin Brady, University of St Andrews, Scotland Henning Christiansen, Roskilde University, Denmark Agostino Dovier, University of Udine, Italy Matthew Flatt, University of Utah, USA Gopal Gupta, University of Texas at Dallas, USA John Hughes, Chalmers University of Technology, Sweden; Quviq AB Gabriele Keller, University of New South Wales, Australia Lunjin Lu, Oakland University, USA Marc Pouzet, ?cole normale sup?rieure, France Ricardo Rocha, University of Porto, Portugal Andreas Rossberg, Google Germany GmbH, Germany Claudio Russo, Microsoft Research Cambridge, UK (co-chair) Kostis Sagonas, Uppsala Univeristy, Sweden Satnam Singh, Microsoft Research Cambridge, UK Zoltan Somogyi, University of Melbourne, Australia Eijiro Sumii, Tohoku University, Japan Terrance Swift, Universidade Nova de Lisboa, Portugal; Johns Hopkins University, USA Andrew Tolmach, Portland State University, USA Jan Wielemaker, University of Amsterdam, The Netherlands Roland Yap, National University of Singapore, Republic of Singapore Kwangkeun Yi, Seoul National University, Korea Neng-Fa Zhou, Brooklyn College, City University of New York, USA (co-chair) Contacts ======== For additional information about papers and submissions, please contact the Program Chairs: Claudio Russo Microsoft Research Cambridge,UK Email: crusso microsoft com Neng-Fa Zhou Brooklyn College, The City University of New York, USA Email: zhou sci brooklyn cuny edu With the Cooperation of ======================= The Association for Logic Programming (ALP) ACM SIGPLAN Microsoft Research, Cambridge =================================== From maggio.vincenzo@REDACTED Tue Aug 9 23:06:18 2011 From: maggio.vincenzo@REDACTED (Vincenzo Maggio) Date: Tue, 9 Aug 2011 14:06:18 -0700 (PDT) Subject: [erlang-questions] common test error handler In-Reply-To: <8EDAA3D2-17B2-4C2F-B730-F24FE32CD517@gmail.com> References: <8EDAA3D2-17B2-4C2F-B730-F24FE32CD517@gmail.com> Message-ID: <1312923978801-3731253.post@n4.nabble.com> Hello, I'm fairly new to Erlang, so I don't know yet test suite behaviour, but my here's my two cents: use a try/catch block. -- View this message in context: http://erlang.2086793.n4.nabble.com/common-test-error-handler-tp3709956p3731253.html Sent from the Erlang Questions mailing list archive at Nabble.com. From joelr1@REDACTED Tue Aug 9 23:50:14 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 9 Aug 2011 22:50:14 +0100 Subject: [erlang-questions] common test hiding errors? Message-ID: <5FA0CF59-7FB1-4310-A6B0-05604965A6EC@gmail.com> I'm using Common Test for OpenPoker integration tests. I just discovered that I'm calling a function that is not exported from a module. I only discovered this by inserting copious printouts into my code to figure out what part was crashing. Why and where did Common Test hide the 'undef'? It's nowhere to be found! -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From comptekki@REDACTED Wed Aug 10 00:01:21 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 9 Aug 2011 16:01:21 -0600 Subject: [erlang-questions] common test hiding errors? In-Reply-To: <5FA0CF59-7FB1-4310-A6B0-05604965A6EC@gmail.com> References: <5FA0CF59-7FB1-4310-A6B0-05604965A6EC@gmail.com> Message-ID: On Tue, Aug 9, 2011 at 3:50 PM, Joel Reymont wrote: > I'm using Common Test for OpenPoker integration tests. > > I just discovered that I'm calling a function that is not exported from a module. > > I only discovered this by inserting copious printouts into my code to figure out what part was crashing. > > Why and where did Common Test hide the 'undef'? > > It's nowhere to be found! find . -name "*.erl" -exec grep undef {} \; -print might help.... -wes From joelr1@REDACTED Wed Aug 10 00:03:52 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 9 Aug 2011 23:03:52 +0100 Subject: [erlang-questions] common test hiding errors? In-Reply-To: References: <5FA0CF59-7FB1-4310-A6B0-05604965A6EC@gmail.com> Message-ID: On Aug 9, 2011, at 11:01 PM, Wes James wrote: > find . -name "*.erl" -exec grep undef {} \; -print How exactly will searching for undef in my code help? The problem is not particular to undef, actually. I just exported the function and it crashes on something else but I still don't see the crash report. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Wed Aug 10 00:07:56 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 9 Aug 2011 23:07:56 +0100 Subject: [erlang-questions] common test hiding errors? In-Reply-To: References: <5FA0CF59-7FB1-4310-A6B0-05604965A6EC@gmail.com> Message-ID: The following is what I see in logs/raw.log. I know the game has crashed because the players are leaving and because showdown:winners did not return. =ERROR REPORT==== 9-Aug-2011::22:59:35 === sitgo.erl:60 <0.258.0> hands ranked(<0.258.0>) call showdown:winners([{hand,<0.259.0>,1,[259,771],0,10,none,0}],[{0,[]}]) =ERROR REPORT==== 9-Aug-2011::22:59:35 === observer.erl:201 <0.275.0> Game 1: LEAVE -> 5 (<0.258.0>) call showdown:winners([{hand,<0.259.0>,1,[259,771],0,10,none,0}],[{0,[]}],[]) (<0.258.0>) call showdown:'-winners/3-fun-0-'({hand,<0.259.0>,1,[259,771],0,10,none,0},[]) =ERROR REPORT==== 9-Aug-2011::22:59:35 === observer.erl:201 <0.275.0> Game 1: LEAVE -> 4 =ERROR REPORT==== 9-Aug-2011::22:59:35 === observer.erl:201 <0.275.0> Game 1: LEAVE -> 3 Invoking showdown:winners at the Erlang prompt gives me an error. Eshell V5.8.4 (abort with ^G) 1> showdown:winners([{hand,self(),1,[259,771],0,10,none,0}],[{0,[]}]). ** exception error: no function clause matching gb_trees:is_defined(<0.31.0>,[]) in function lists:'-filter/2-lc$^0/1-0-'/2 in call from showdown:winners/3 How do I make Common Test show me this error during testing? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From sam@REDACTED Wed Aug 10 01:04:27 2011 From: sam@REDACTED (Sam Elliott) Date: Wed, 10 Aug 2011 00:04:27 +0100 Subject: [erlang-questions] Erlang Web Libraries & Frameworks Message-ID: Hi erlang-questions, Recently I needed to compare a few Erlang web libraries and frameworks for a friend who was writing a simple internal API endpoint. He suggested I should publish the rundown for others, and I thought I'd also circulate it here. I'm relatively new to Erlang, so I have probably missed a few libraries. I'll try to keep it updated as people send me information, so don't hesitate to contact me if i've got something wrong. The article: http://lenary.co.uk/erlang/2011/08/erlang-web-libraries/ What are your thoughts? Sam -- Sam Elliott sam@REDACTED -- From rexxe98@REDACTED Wed Aug 10 04:15:47 2011 From: rexxe98@REDACTED (Andrew Berman) Date: Tue, 9 Aug 2011 19:15:47 -0700 Subject: [erlang-questions] Best approach to creating a process/connection pool In-Reply-To: <6EC89159-B65F-43E5-A2DC-D0BB7ADBA143@erlang-solutions.com> References: <6EC89159-B65F-43E5-A2DC-D0BB7ADBA143@erlang-solutions.com> Message-ID: Thanks Ulf. That seems to be way more complicated than what i need....I think. I actually created a pool application here: https://github.com/aberman/pooly Every pooling app I found or was suggested didn't really take into account certain factors that most connection pools use, such as idle timeout, maximum age of a connection, acquire increment, initial number of connections to start, etc. I wrote my pool with these factors in mind and made them configurable in an external file. It was written for Riak, but it can easily be adapted to support any gen_server. Whether I did things properly, I don't know, but it seems to work how I want it. Thanks for your help, Andrew On Mon, Aug 8, 2011 at 11:32 PM, Ulf Wiger wrote: > > The Jobs load regulation system supports a pool concept. Actually, it allows > you to combine concepts in a number of ways to tune characteristics. > Here is one example, with a 'producer pool' and a passive queue. The > producer pool is defined with a fun that is called to top up the pool. The > producers are free to do pretty much what they want, and jobs will monitor > them to ensure they are replaced if they die. > https://gist.github.com/778319 > In this case, the producers fetch jobs from a passive queue. In Jobs, that's > a queue where you have to explicitly dequeue them. Otherwise, you typically > tell jobs at which rate it should draw from the queue. > Rates and pool sizes can be made to adapt automatically to load, using load > sampler plugins. Configuration/re-configuration can also be done at runtime > through the Jobs API. > http://github.com/esl/jobs > BR, > Ulf W > > On 8 Aug 2011, at 21:44, Joel Meyer wrote: > > If the things you are pooling are gen_servers, gen_server_pool may work for > you (https://github.com/joelpm/gen_server_pool). That's a slight > modification on a version I've used in production to pool thrift and > protobuff conns to back-ends. To start a pool you'd simply do > gen_server_pool:startlink( ..., GenServerPoolArgs). instead of doing > gen_server:start_link(...). You would use the returned PID in the same way. > I haven't had time to test that version, though, so YMMV. > Joel > > 2011/8/4 Andrew Berman >> >> Awesome! ?This makes sense. ?Thank you for the clarification on this. >> >> 2011/8/4 Fr?d?ric Trottier-H?bert : >> > Technically, the OTP supervision tree is also used for application >> > upgrades (appups & relups). When sending messages to do the broad reloading >> > of code, the code in charge of that just traverses the VM's supervision >> > trees asking the supervisors what kind of modules they have and whether >> > they're workers and supervisors (that's why the child specs contain details >> > about workers, supervisors, [Modules] (and dynamic sending special messages >> > to figure out the same info). >> > >> > If you want to use appups/relups, you have to use the OTP supervision >> > trees otherwise you will have processes left behind to be purged rather than >> > upgraded. >> > >> > -- >> > Fred H?bert >> > http://www.erlang-solutions.com >> > >> > >> > >> > On 2011-08-04, at 13:39 PM, Hynek Vychodil wrote: >> > >> >> Using OTP supervisor tree is intended only for control of restarting >> >> application parts. If you want anything else you have to use links or >> >> monitors unless you write your own supervisor which is not recommended >> >> way. >> >> >> >> On Thu, Aug 4, 2011 at 6:35 PM, Andrew Berman >> >> wrote: >> >>> Thanks Tim. ?So it looks like they are using erlang:monitor and >> >>> demonitor, so is that the best approach in an OTP application? ?How >> >>> does it figure into the supervision tree? >> >>> >> >>> >> >>> >> >>> On Thu, Aug 4, 2011 at 3:22 AM, Tim Watson >> >>> wrote: >> >>>> Take a look at the sources for http://zotonic.com/ as they use a >> >>>> decent connection pool. >> >>>> >> >>>> On 4 August 2011 08:01, Andrew Berman wrote: >> >>>>> I'm fairly new to Erlang and I am trying to write a process pool >> >>>>> which >> >>>>> essentially is a connection pool to a database. ?I have designed the >> >>>>> application like this: ?The main pool gen_server has a queue of PIDs >> >>>>> of gen_fsm's. ?Each gen_fsm calls spawn_links a 3rd party gen_server >> >>>>> and carries >> >>>>> the state of the connection. ?So the 3rd party gen_server and the >> >>>>> gen_fsm should always live and die together. ?I currently have set >> >>>>> it >> >>>>> up such that gen_fsm calls proc_lib:spawn_link() to start the 3rd >> >>>>> party gen_server, but I'm wondering how this factors into the >> >>>>> supervision tree. ?I currently have two supervisors: >> >>>>> >> >>>>> Sup1 -> one_for_all -> 2 children: pool gen_server and Sup2 >> >>>>> Sup2 -> simple_one_for_one -> 1 child: gen_fsm >> >>>>> >> >>>>> So my question is: is it ok to use spawn_link instead of somehow >> >>>>> trying to put the 3rd party gen_server into the supervision tree? >> >>>>> ?If >> >>>>> it should go into the supervision tree, how would I organize the >> >>>>> tree >> >>>>> then such that the gen_fsm and 3rd party gen_server are reliant on >> >>>>> each other? >> >>>>> >> >>>>> Thanks for any help! >> >>>>> _______________________________________________ >> >>>>> erlang-questions mailing list >> >>>>> erlang-questions@REDACTED >> >>>>> http://erlang.org/mailman/listinfo/erlang-questions >> >>>>> >> >>>> >> >>> _______________________________________________ >> >>> erlang-questions mailing list >> >>> erlang-questions@REDACTED >> >>> http://erlang.org/mailman/listinfo/erlang-questions >> >>> >> >> >> >> >> >> >> >> -- >> >> --Hynek (Pichi) Vychodil >> >> >> >> Analyze your data in minutes. Share your insights instantly. Thrill >> >> your boss. ?Be a data hero! >> >> Try GoodData now for free: www.gooddata.com >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger,?CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > From ok@REDACTED Wed Aug 10 05:11:00 2011 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 10 Aug 2011 15:11:00 +1200 Subject: [erlang-questions] my best erlang "foot in mouth" In-Reply-To: <1312899399819-3730042.post@n4.nabble.com> References: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> <1312899399819-3730042.post@n4.nabble.com> Message-ID: <954D081E-0A9D-4AE6-9162-50F883489D5C@cs.otago.ac.nz> On 10/08/2011, at 2:16 AM, Vincenzo Maggio wrote: > lists:seq(1:100). > > Erlang answers with "illegal expression" -> an afternoon of code review :( > And I STILL continue to write this!!! Here's a one-line AWK script you can use to check for this. #!/bin/awk -f /[^a-zA-Z0-9_][0-9]+ *:/ { print FILENAME ":" FNR ": integer-colon error" } You can probably configure your editor to run this on a keystroke. Of course 1:100 is perfectly legal S, so it's not _that_ silly a mistake. From andrew@REDACTED Wed Aug 10 05:25:40 2011 From: andrew@REDACTED (Andrew Thompson) Date: Tue, 9 Aug 2011 23:25:40 -0400 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: References: Message-ID: <20110810032540.GD6037@hijacked.us> On Wed, Aug 10, 2011 at 12:04:27AM +0100, Sam Elliott wrote: > Hi erlang-questions, > > Recently I needed to compare a few Erlang web libraries and frameworks > for a friend who was writing a simple internal API endpoint. He > suggested I should publish the rundown for others, and I thought I'd > also circulate it here. I'm relatively new to Erlang, so I have > probably missed a few libraries. > > I'll try to keep it updated as people send me information, so don't > hesitate to contact me if i've got something wrong. > > The article: http://lenary.co.uk/erlang/2011/08/erlang-web-libraries/ > > What are your thoughts? > I find it interesting that you include mochiweb (an http toolkit) alongside with things like chicago boss (a real web framework), webmachine (a REST toolkit). It's also interesting you omit misultin, cowboy and zotonic from the list. Andrew From gumm@REDACTED Wed Aug 10 07:55:09 2011 From: gumm@REDACTED (Jesse Gumm) Date: Wed, 10 Aug 2011 00:55:09 -0500 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: References: Message-ID: Hi, While, for the most part, you're right about Nitrogen not being *aimed* at simple restful api development, it's not too hard to do. wf:q and wf:qs will read values from querystring and post variables, so you can easily pass around json and what not. Using a template is optional, since every page's entry point is the main() function. Further, if you'd rather not do querystrings, you can trivially do path parsing. IE: assuming you have a module called 'api', for the path /api/myinfo/123, wf:path_info() returns "myinfo/123". Between those and mochijson, building a simple api should be a breeze in nitrogen. -Jesse On Tue, Aug 9, 2011 at 6:04 PM, Sam Elliott wrote: > Hi erlang-questions, > > Recently I needed to compare a few Erlang web libraries and frameworks > for a friend who was writing a simple internal API endpoint. He > suggested I should publish the rundown for others, and I thought I'd > also circulate it here. I'm relatively new to Erlang, so I have > probably missed a few libraries. > > I'll try to keep it updated as people send me information, so don't > hesitate to contact me if i've got something wrong. > > The article: http://lenary.co.uk/erlang/2011/08/erlang-web-libraries/ > > What are your thoughts? > > Sam > > -- > Sam Elliott > sam@REDACTED > -- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jesse Gumm Sigma Star Systems 414.940.4866 gumm@REDACTED http://www.sigma-star.com From dmitrii@REDACTED Wed Aug 10 08:56:48 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Wed, 10 Aug 2011 09:56:48 +0300 Subject: [erlang-questions] my best erlang "foot in mouth" In-Reply-To: <954D081E-0A9D-4AE6-9162-50F883489D5C@cs.otago.ac.nz> References: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> <1312899399819-3730042.post@n4.nabble.com> <954D081E-0A9D-4AE6-9162-50F883489D5C@cs.otago.ac.nz> Message-ID: If we only could get a public repository for Wrangler rules[1] we could have these checks actually integrated with proper Erlang tools [1] See refactoring videos at http://www.cs.kent.ac.uk/projects/wrangler/Home.html and DIY refactoring at http://www.erlang-factory.com/conference/London2011/speakers/SimonThompson > > On 10/08/2011, at 2:16 AM, Vincenzo Maggio wrote: > >> lists:seq(1:100). >> >> Erlang answers with "illegal expression" -> an afternoon of code review :( >> And I STILL continue to write this!!! > > Here's a one-line AWK script you can use to check for this. > > #!/bin/awk -f > /[^a-zA-Z0-9_][0-9]+ *:/ { print FILENAME ":" FNR ": integer-colon error" } > > You can probably configure your editor to run this on a keystroke. > > Of course 1:100 is perfectly legal S, so it's not _that_ silly a mistake. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions =================================== Dmitrii Dimandt dmitrii@REDACTED ------------------------------------------------------------ Erlang in Russian http://erlanger.ru/ TurkeyTPS http://turkeytps.com/ ------------------------------------------------------------ LinkedIn: http://www.linkedin.com/in/dmitriid GitHub: https://github.com/dmitriid -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Wed Aug 10 09:50:47 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Wed, 10 Aug 2011 10:50:47 +0300 Subject: [erlang-questions] how to run PropEr tests from Eunit? In-Reply-To: References: <1308867174.83770.YahooMailRC@web25402.mail.ukl.yahoo.com> <93BD232D-C14F-4B16-80D2-96C24EE44BB3@erlang-solutions.com> <1308939214.73194.YahooMailRC@web25404.mail.ukl.yahoo.com> <20110809062736.GA7058@jakstys.lt> Message-ID: <20110810075047.GA12427@jakstys.lt> On Tue, Aug 09, 2011 at 06:53:09AM -0400, Fr?d?ric Trottier-H?bert wrote: > The Group Leader is a special concept. In each process' metadata, > there is an entry for a group leader. The group leader, at the moment, > is a Pid that is in charge of handling IO requests using the IO > protocol (http://www.erlang.org/doc/apps/stdlib/io_protocol.html). > This group leader is the 'user' process by default (that's its > registered name). When you send io requests to the user process, the > output gets printed to the shell. > > By default, all processes get the 'user' group leader -- this is set > through inheritance when spawning a process. The exception to that is > when you're using OTP application. Then the Application Master (a > process spawned by the application controller) sets itself as a > middleman between the user process and your own processes. That lets > OTP create process groups to shut down in case it's needed. > > In any case, applications like EUnit have decided to overwrite these > default group leaders and make their own -- this lets them capture > input the way they want to work with it, log it, or just hide it. This > makes some macros such as 'assertCmdOutput(Text, CommandString)' work. > To bypass this, the framework then gives macros like ?debugVal(Val) > and ?debugMsg(Text) to the programmer. All these macros do is > something like 'io:format(user, String, Args)'. > > That's because adding a first argument of this kind lets you redirect > IO to an io-device (file descriptor, group leader, etc.). You can, > however, use the following functions to just plainly overwrite the > group leader back to something new: > > http://erldocs.com/R14B02/erts/erlang.html?i=0&search=group_leader#group_leader/0 > http://erldocs.com/R14B02/erts/erlang.html?i=1&search=group_leader#group_leader/2 > > You could do something such as: > > run_proper_test_() -> EunitLeader = erlang:group_leader(), > erlang:group_leader(whereis(user), self()), Res = > proper:module(?MODULE), erlang:group_leader(EunitLeader, self()), > ?_assertEqual([], Res). > > (untested). This would temporarily switch the group leader of the test > for the time being before setting it back to whatever it was after. So > for the time PropEr tests are running you get regular output, and then > it gets back to Eunit-normal afterwards. > > Hope this helps. > > -- Fred H?bert http://www.erlang-solutions.com > Thank you. It worked great! Motiejus From essen@REDACTED Wed Aug 10 10:04:04 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 10 Aug 2011 10:04:04 +0200 Subject: [erlang-questions] common test hiding errors? In-Reply-To: References: <5FA0CF59-7FB1-4310-A6B0-05604965A6EC@gmail.com> Message-ID: <4E423B74.2010107@dev-extend.eu> On 08/10/2011 12:07 AM, Joel Reymont wrote: > The following is what I see in logs/raw.log. > > I know the game has crashed because the players are leaving and because showdown:winners did not return. > > =ERROR REPORT==== 9-Aug-2011::22:59:35 === > sitgo.erl:60 <0.258.0> hands ranked(<0.258.0>) call showdown:winners([{hand,<0.259.0>,1,[259,771],0,10,none,0}],[{0,[]}]) > > =ERROR REPORT==== 9-Aug-2011::22:59:35 === > observer.erl:201 <0.275.0> Game 1: LEAVE -> 5 > (<0.258.0>) call showdown:winners([{hand,<0.259.0>,1,[259,771],0,10,none,0}],[{0,[]}],[]) > (<0.258.0>) call showdown:'-winners/3-fun-0-'({hand,<0.259.0>,1,[259,771],0,10,none,0},[]) > > =ERROR REPORT==== 9-Aug-2011::22:59:35 === > observer.erl:201 <0.275.0> Game 1: LEAVE -> 4 > > =ERROR REPORT==== 9-Aug-2011::22:59:35 === > observer.erl:201 <0.275.0> Game 1: LEAVE -> 3 > > Invoking showdown:winners at the Erlang prompt gives me an error. > > Eshell V5.8.4 (abort with ^G) > 1> showdown:winners([{hand,self(),1,[259,771],0,10,none,0}],[{0,[]}]). > ** exception error: no function clause matching > gb_trees:is_defined(<0.31.0>,[]) > in function lists:'-filter/2-lc$^0/1-0-'/2 > in call from showdown:winners/3 > > How do I make Common Test show me this error during testing? Are you catching the exception perhaps? The error message you get seems custom so I'm guessing your code catches the exception and then prints the error message ct gives you. So the test doesn't crash and ct doesn't know about that error. -- Lo?c Hoguin Dev:Extend From lukas.larsson@REDACTED Wed Aug 10 10:09:36 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Wed, 10 Aug 2011 09:09:36 +0100 (BST) Subject: [erlang-questions] common test hiding errors? In-Reply-To: Message-ID: <9a786b50-d22b-4bce-893a-52e36263d1fa@localhost> Hello! Common Test detects any crashes which happen in the same process as the testcase is run in. If a crash happens in another process, it will not be detected unless you in your testcase check whether said process is alive or is linked to it. Could this be what is going wrong? If you're still having difficulties, could you post a minimal example of what you are trying to do? It's difficult to guess what could be the problem without an example to look at. Also if you want to have the SASL and error_logger output in your common test logs instead of redirecting them to a local file you might want to use the cth_log_redirect hook found here https://github.com/garazdawi/cth_tools. Lukas ----- Original Message ----- From: "Joel Reymont" To: "erlang-questions" Sent: Wednesday, 10 August, 2011 12:07:56 AM Subject: Re: [erlang-questions] common test hiding errors? The following is what I see in logs/raw.log. I know the game has crashed because the players are leaving and because showdown:winners did not return. =ERROR REPORT==== 9-Aug-2011::22:59:35 === sitgo.erl:60 <0.258.0> hands ranked(<0.258.0>) call showdown:winners([{hand,<0.259.0>,1,[259,771],0,10,none,0}],[{0,[]}]) =ERROR REPORT==== 9-Aug-2011::22:59:35 === observer.erl:201 <0.275.0> Game 1: LEAVE -> 5 (<0.258.0>) call showdown:winners([{hand,<0.259.0>,1,[259,771],0,10,none,0}],[{0,[]}],[]) (<0.258.0>) call showdown:'-winners/3-fun-0-'({hand,<0.259.0>,1,[259,771],0,10,none,0},[]) =ERROR REPORT==== 9-Aug-2011::22:59:35 === observer.erl:201 <0.275.0> Game 1: LEAVE -> 4 =ERROR REPORT==== 9-Aug-2011::22:59:35 === observer.erl:201 <0.275.0> Game 1: LEAVE -> 3 Invoking showdown:winners at the Erlang prompt gives me an error. Eshell V5.8.4 (abort with ^G) 1> showdown:winners([{hand,self(),1,[259,771],0,10,none,0}],[{0,[]}]). ** exception error: no function clause matching gb_trees:is_defined(<0.31.0>,[]) in function lists:'-filter/2-lc$^0/1-0-'/2 in call from showdown:winners/3 How do I make Common Test show me this error during testing? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From erlangsiri@REDACTED Wed Aug 10 10:18:32 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Wed, 10 Aug 2011 10:18:32 +0200 Subject: [erlang-questions] relup restart_new_emulator does not work for me In-Reply-To: References: Message-ID: Hi again Allen! When executing a restart_new_emulator instruction, the release_handler changes the heart command to " $ROOTDIR/releases/new_start_erl.data". The start program defaults to $ROOTDIR/bin/start, unless you have specifically set the 'start_prg' environment variable for the sasl application. If you haven't modified this start script (found in you erlang installation), it will do $ROOTDIR/bin/run_erl -daemon /tmp/ $ROOTDIR/log "exec $ROOTDIR/bin/start_erl $ROOTDIR $RELDIR $START_ERL_DATA" i.e. it will start erlang node using run_erl, and log to $ROOTDIR/log - but only if this directory exists. In the log directory you should find erlang.log.*, and run_erl.log - they might give you an indication of what the problem is. I hope this can be to some help in the investigation. Regards /siri 2011/8/9 Allen Kim > Siri, > > No, I don't see erl_crash.dump, or I could not find it. > Yes, I could start 5.8.4 emulator manually. > > The weird thing is after I start manually, then I run command, > release_handler:install_release("2.0.0") > It gives me an error, which I don't remember. > > Now, I decided to upgrade erlang version manually, that means > previous plan: > 1. run command release_handler:install_release("2.0.0") > > new plan: > 1. update RELEASE file manually by changing 1.9.0 old and 2.0.0 > permanent > 2. run command init:reboot() > > The new plan seems working, but I was curious why the previous plan did not > work. > > Hope this helps > > From: Siri Hansen > Date: Tue, 9 Aug 2011 08:05:07 -0500 > To: Allen Kim > Cc: erlang-questions > Subject: Re: [erlang-questions] relup restart_new_emulator does not work > for me > > Hi Allen! > > Can you see an erl_crash.dump file after this? > Can you start the 5.8.4 emulator manually? > > Regards > /siri > > 2011/8/3 Allen Kim > >> Hi, >> >> I want to upgrade erlang otp from R13B04 to R43B03 >> >> I used this start_erl.data and relup(recommended from >> >> [tracking@REDACTED releases]$ cat new_start_erl.data >> 5.8.4 2.0.0 >> >> [tracking@REDACTED releases]$ cat current/relup >> {"2.0.0", >> [{"1.9.0", >> [], >> [restart_new_emulator]}], >> [{"1.9.0", >> [], >> [restart_new_emulator]}] >> }. >> >> when I run command release_handler:install_release("2.0.0") >> I see the node shutting down messages, but I don't see a new node start >> up.(looks like heart is not starting up a new node). >> >> My HEART_COMMAND is coreect, since I was able to kill the node and I see >> it starting up again. >> >> The reason why I said restart_new_emulator not working is I could reboot >> the node with command init:reboot(). >> >> Any help would be appreciated. >> >> Allen Kim >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From s.j.thompson@REDACTED Wed Aug 10 10:11:54 2011 From: s.j.thompson@REDACTED (Simon Thompson) Date: Wed, 10 Aug 2011 09:11:54 +0100 Subject: [erlang-questions] PEPM 2012 Second call for papers Message-ID: <4376F06D-7910-42E1-A353-B32D52E9C532@kent.ac.uk> Please accept in our apologies for multiple postings of this announcement. ACM SIGPLAN 2012 Workshop on Partial Evaluation and Program Manipulation January 23-24, 2012. Philadelphia, Pennsylvania, USA (co-located with POPL'12) Call For Papers Paper submission deadline: Mon, October 10, 2011, 23:59, GMT http://www.program-transformation.org/PEPM12 The PEPM Symposium/Workshop series aims to bring together researchers and practitioners working in the broad area of program transformation, which spans from refactoring, partial evaluation, supercompilation, fusion and other metaprogramming to model-driven development, program analyses including termination, inductive programming, program generation and applications of machine learning and probabilistic search. PEPM focuses on techniques, supporting theory, tools, and applications of the analysis and manipulation of programs. Each technique or tool of program manipulation should have a clear, although perhaps informal, statement of desired properties, along with an argument how these properties could be achieved. Topics of interest for PEPM'12 include, but are not limited to: - Program and model manipulation techniques such as: supercompilation, partial evaluation, fusion, on-the-fly program adaptation, active libraries, program inversion, slicing, symbolic execution, refactoring, decompilation, and obfuscation. - Program analysis techniques that are used to drive program/model manipulation such as: abstract interpretation, termination checking, binding-time analysis, constraint solving, type systems, automated testing and test case generation. - Techniques that treat programs/models as data objects including metaprogramming, generative programming, embedded domain-specific languages, program synthesis by sketching and inductive programming, staged computation, and model-driven program generation and transformation. - Application of the above techniques including case studies of program manipulation in real-world (industrial, open-source) projects and software development processes, descriptions of robust tools capable of effectively handling realistic applications, benchmarking. Examples of application domains include legacy program understanding and transformation, DSL implementations, visual languages and end-user programming, scientific computing, middleware frameworks and infrastructure needed for distributed and web-based applications, resource-limited computation, and security. To maintain the dynamic and interactive nature of PEPM, we will continue the category of `short papers' for tool demonstrations and for presentations of exciting if not fully polished research, and of interesting academic, industrial and open-source applications that are new or unfamiliar. Student attendants with accepted papers can apply for a SIGPLAN PAC grant to help cover travel expenses and other support. All accepted papers, short papers included, will appear in formal proceedings published by ACM Press and will be included in the ACM Digital Library. Selected papers may later on be invited for a journal special issue dedicated to PEPM'12. Submission Categories and Guidelines Authors are strongly encouraged to consult the advice for authoring research papers and tool papers before submitting. The PC Chairs welcome any inquiries about the authoring advice. Regular research papers must not exceed 10 pages in ACM Proceedings style. Short papers are up to 4 pages in ACM Proceedings style. Authors of tool demonstration proposals are expected to present a live demonstration of the described tool at the workshop (tool papers should include an additional appendix of up to 6 extra pages giving the outline, screenshots, examples, etc. to indicate the content of the proposed live demo at the workshop). Important Dates - Paper submission: Mon, October 10, 2011, 23:59, GMT - Author notification: Tue, November 8, 2011 - Workshop: Mon-Tue, January 23-24, 2012 Invited Speakers - Markus Pueschel (ETH Zurich, Switzerland) - Martin Berger (University of Sussex, UK) Program Chairs - Oleg Kiselyov (Monterey, CA, USA) - Simon Thompson (University of Kent, UK) Program Committee Members - Emilie Balland (INRIA, France) - Ewen Denney (NASA Ames Research Center, USA) - Martin Erwig (Oregon State University, USA) - Sebastian Fischer (National Institute of Informatics, Japan) - Lidia Fuentes (Universidad de Malaga, Spain) - John Gallagher (Roskilde University, Denmark and IMDEA Software, Spain) - Dave Herman (Mozilla Research, USA) - Stefan Holdermans (Vector Fabrics, the Netherlands) - Christian Kaestner (University of Marburg, Germany) - Emanuel Kitzelmann (International Computer Science Institute, USA) - Andrei Klimov (Keldysh Institute of Applied Mathematics, Russian Academy of Sciences) - Shin-Cheng Mu (Academia Sinica, Taiwan) - Alberto Pardo (Universidad de la Repu'blica, Uruguay) - Kostis Sagonas (Uppsala University, Sweden and National Technical University of Athens, Greece) - Anthony M. Sloane (Macquarie University, Australia) - Armando Solar-Lezama (MIT, USA) - Aaron Stump (The University of Iowa, USA) - Kohei Suenaga (University of Kyoto, Japan) - Eric Van Wyk (University of Minnesota, USA) - Kwangkeun Yi (Seoul National University, Korea) Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt From joelr1@REDACTED Wed Aug 10 12:36:49 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 10 Aug 2011 11:36:49 +0100 Subject: [erlang-questions] common test hiding errors? In-Reply-To: <4E423B74.2010107@dev-extend.eu> References: <5FA0CF59-7FB1-4310-A6B0-05604965A6EC@gmail.com> <4E423B74.2010107@dev-extend.eu> Message-ID: <658C236D-2AB8-4938-86E4-C94BF3637A3D@gmail.com> On Aug 10, 2011, at 9:04 AM, Lo?c Hoguin wrote: > Are you catching the exception perhaps? No. > The error message you get seems > custom so I'm guessing your code catches the exception and then prints > the error message ct gives you. So the test doesn't crash and ct doesn't > know about that error. I don't get any errors, that's the point. The custom error message is from running at the Erlang prompt, the rest are debug printouts. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Wed Aug 10 12:39:23 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 10 Aug 2011 11:39:23 +0100 Subject: [erlang-questions] common test hiding errors? In-Reply-To: <9a786b50-d22b-4bce-893a-52e36263d1fa@localhost> References: <9a786b50-d22b-4bce-893a-52e36263d1fa@localhost> Message-ID: <54EA06CD-79B2-4B4A-BF26-DE48D2C5FE00@gmail.com> On Aug 10, 2011, at 9:09 AM, Lukas Larsson wrote: > Common Test detects any crashes which happen in the same process as the testcase is run in. If a crash happens in another process, it will not be detected unless you in your testcase check whether said process is alive or is linked to it. Could this be what is going wrong? Yes, that's what's going on. I'm cannot link to all the processes in the "system under testing", I think that's unrealistic. I do expect to be told when something goes wrong, though, get a printout in the log, etc. This is in line with what EUnit does, for example. > If you're still having difficulties, could you post a minimal example of what you are trying to do? It's difficult to guess what could be the problem without an example to look at. It's another process that's crashes. > Also if you want to have the SASL and error_logger output in your common test logs instead of redirecting them to a local file you might want to use the cth_log_redirect hook found here https://github.com/garazdawi/cth_tools. I'm perfectly fine with redirecting error_logger output to a local file but it seems that it shows up just fine in the CT logs. What's not showing up is error messages from other processes crashing, a completely unintuitive behavior, I think. Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From carlsson.richard@REDACTED Wed Aug 10 13:20:10 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 10 Aug 2011 13:20:10 +0200 Subject: [erlang-questions] Wrangler (Was: Re: my best erlang "foot in mouth") In-Reply-To: References: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> <1312899399819-3730042.post@n4.nabble.com> <954D081E-0A9D-4AE6-9162-50F883489D5C@cs.otago.ac.nz> Message-ID: <4E42696A.80709@gmail.com> On 08/10/2011 08:56 AM, Dmitrii Dimandt wrote: > If we only could get a public repository for Wrangler rules[1] we could > have these checks actually integrated with proper Erlang tools Wrangler can now be found here: https://github.com/RefactoringTools/wrangler It's now also much easier to compile and set up, and plays well with a modern Distel (which is found at https://github.com/massemanet/distel, if you want to get it separately, but Wrangler includes a recent version). /Richard From sam@REDACTED Wed Aug 10 14:21:47 2011 From: sam@REDACTED (Sam Elliott) Date: Wed, 10 Aug 2011 13:21:47 +0100 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: <20110810032540.GD6037@hijacked.us> References: <20110810032540.GD6037@hijacked.us> Message-ID: On Wed, Aug 10, 2011 at 4:25 AM, Andrew Thompson wrote: > On Wed, Aug 10, 2011 at 12:04:27AM +0100, Sam Elliott wrote: >> Hi erlang-questions, >> >> Recently I needed to compare a few Erlang web libraries and frameworks >> for a friend who was writing a simple internal API endpoint. He >> suggested I should publish the rundown for others, and I thought I'd >> also circulate it here. I'm relatively new to Erlang, so I have >> probably missed a few libraries. >> >> I'll try to keep it updated as people send me information, so don't >> hesitate to contact me if i've got something wrong. >> >> The article: http://lenary.co.uk/erlang/2011/08/erlang-web-libraries/ >> >> What are your thoughts? >> > > I find it interesting that you include mochiweb (an http toolkit) > alongside with things like chicago boss (a real web framework), > webmachine (a REST toolkit). I wanted the guide to be completely comprehensive, so that I have somewhere to look when I'm choosing the library/framework I use for future projects, and I hope that it would also help other people. > It's also interesting you omit misultin, cowboy and zotonic from the > list. I hadn't heard of misultin or cowboy until your email, and I wasn't sure how extensible Zotonic was. Now I've looked into it, I have updated the article. Sam > > Andrew > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From gordon@REDACTED Wed Aug 10 14:48:11 2011 From: gordon@REDACTED (Gordon Guthrie) Date: Wed, 10 Aug 2011 13:48:11 +0100 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: References: Message-ID: <66EB65CD-A806-4FA1-AFAC-140DAA64BF3B@hypernumbers.com> Sam Hypernumbers have just released a request signing library which is substantially based on Amazon's HMAC-SHA authentication schema. The aim is to provide a common, reusable format for request canonicalisation and signing for private/public key pair authentication. Ideally we would like to build up a set of client libraries that implement it. The code is structured so as to make it as easy as possible for developers of client libraries to build and test them against unit tests and to try and avoid the hell that is debugging against production systems not in your control. This library is now in the examples part of mochiweb. Gordon On 10 Aug 2011, at 00:04, Sam Elliott wrote: > Hi erlang-questions, > > Recently I needed to compare a few Erlang web libraries and frameworks > for a friend who was writing a simple internal API endpoint. He > suggested I should publish the rundown for others, and I thought I'd > also circulate it here. I'm relatively new to Erlang, so I have > probably missed a few libraries. > > I'll try to keep it updated as people send me information, so don't > hesitate to contact me if i've got something wrong. > > The article: http://lenary.co.uk/erlang/2011/08/erlang-web-libraries/ > > What are your thoughts? > > Sam > > -- > Sam Elliott > sam@REDACTED > -- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From s.j.thompson@REDACTED Wed Aug 10 15:04:25 2011 From: s.j.thompson@REDACTED (Simon Thompson) Date: Wed, 10 Aug 2011 14:04:25 +0100 Subject: [erlang-questions] Wrangler (Was: Re: my best erlang "foot in mouth") In-Reply-To: <4E42696A.80709@gmail.com> References: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> <1312899399819-3730042.post@n4.nabble.com> <954D081E-0A9D-4AE6-9162-50F883489D5C@cs.otago.ac.nz> <4E42696A.80709@gmail.com> Message-ID: Richard - thanks for nudging us to doing this. The Wrangler team plans a release in the next few weeks with stable versions of the scripting, and a repository for scripts. Simon and Huiqing On 10 Aug 2011, at 12:20, Richard Carlsson wrote: > On 08/10/2011 08:56 AM, Dmitrii Dimandt wrote: >> If we only could get a public repository for Wrangler rules[1] we could >> have these checks actually integrated with proper Erlang tools > > Wrangler can now be found here: > https://github.com/RefactoringTools/wrangler > > It's now also much easier to compile and set up, and plays well with a modern Distel (which is found at https://github.com/massemanet/distel, if you want to get it separately, but Wrangler includes a recent version). > > /Richard > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt From g@REDACTED Wed Aug 10 17:10:00 2011 From: g@REDACTED (Garrett Smith) Date: Wed, 10 Aug 2011 10:10:00 -0500 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: References: Message-ID: On Tue, Aug 9, 2011 at 6:04 PM, Sam Elliott wrote: > Hi erlang-questions, > > Recently I needed to compare a few Erlang web libraries and frameworks > for a friend who was writing a simple internal API endpoint. He > suggested I should publish the rundown for others, and I thought I'd > also circulate it here. I'm relatively new to Erlang, so I have > probably missed a few libraries. > > I'll try to keep it updated as people send me information, so don't > hesitate to contact me if i've got something wrong. > > The article: http://lenary.co.uk/erlang/2011/08/erlang-web-libraries/ I'm glad to see you included httpd in the list! This is usually left out of the "usual suspects" of Erlang web libraries/frameworks. httpd is the only (feature complete) Erlang framework I'm aware of that uses a traditional "middleware" pattern that lets you chain arbitrary request handlers together. This lets you add in functionality at configuration time like auth, aliases, static content support, whatever-you-like, etc. If you're familiar with Python's wsgi ecosystem, it's very similar. The problem I have with it -- apart from the fact that few people use it and it's a bit crufty -- is that the mod API is very arcane. To simplify writing httpd mods, I wrote this library: https://github.com/gar1t/modlib The advantage of httpd+modlib is that it leverages what's already in the standard Erlang distribution, which is outstanding, apart from the minor criticisms above. Its API uses a straight forward functional model for handling HTTP requests. Garrett From andrew.kondratovich@REDACTED Wed Aug 10 17:54:28 2011 From: andrew.kondratovich@REDACTED (Andrew Kondratovich) Date: Wed, 10 Aug 2011 08:54:28 -0700 (PDT) Subject: [erlang-questions] about syntax tools In-Reply-To: References: Message-ID: <25140878.233.1312991669033.JavaMail.geo-discussion-forums@yqic37> Looking through Erlang sources I have found out that there are many ways to parse and compile the code. In general way there is transformation like Code -> Tokens -> Forms ->... But functions for these transformations can be found in many modules: erl_scan, erl_parse, compile, epp, io - theese are only those that have seen. Functions from that modules often do the same work. What for such variety ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Wed Aug 10 18:08:47 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Wed, 10 Aug 2011 19:08:47 +0300 Subject: [erlang-questions] PropEr: specify custom types in ?SIZED() fun Message-ID: <20110810160847.GA14706@jakstys.lt> I want to specify my custom type depending on recursion level of generated input. Consider this simple example: -module(properworks). -include_lib("proper/include/proper.hrl"). -compile(export_all). -type recursive() :: binary() | [recursive()]. sized_struct(Size) when Size > 10 -> binary(); sized_struct(_Size) -> recursive(). prop_simple() -> ?FORALL(_Item, ?SIZED(Size, sized_struct(Size)), begin true end ). $ erlc -pz ~/nrms/deps/proper/ebin -I ~/nrms/deps/proper properworks.erl ./properworks.erl:10: function recursive/0 undefined How can I specify a custom data type for generation in sized_struct/1? Built-in types work (binary() as in this example), but not mine. I tried ?LAZY, union() wrappers, no success. P.S. does PropEr have a mailing list? I would hang there if it did. :) P.P.S thank you for creating PropEr. Amazing tool. Regards, Motiejus From gumm@REDACTED Wed Aug 10 18:35:04 2011 From: gumm@REDACTED (Jesse Gumm) Date: Wed, 10 Aug 2011 11:35:04 -0500 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: References: Message-ID: Garret said: > httpd is the only (feature complete) Erlang framework I'm aware of > that uses a traditional "middleware" pattern that lets you chain > arbitrary request handlers together. This lets you add in > functionality at configuration time like auth, aliases, static content > support, whatever-you-like, etc. Nitrogen does also support custom request handlers for various types (session, request routing, etc), so if you wanted to use mnesia or mysql for session tracking, doing so is reasonably easy. Then again, if the handler functionality provided by Nitrogen isn't extensible enough for you, because Nitrogen isn't the webserver itself, if it doesn't have a matching module, or a matching route path, it'll pass the buck to the webserver of your choice (mochiweb, yaws, inets, webmachine, and it looks like Rusty might have stealth added Misultin support, I just noticed the code in the simple_bridge now, I hadn't dug around in there for a while). And if you need to use a webserver not immediately supported, simple_bridge's request and response handlers should be easy enough to hack. "So it's got that going for it, which is nice." Maybe as a side project in the coming weeks, I'll put together a Chicago Boss or Cowboy bridge for Nitrogen just to learn that part of the system. -- Jesse Gumm Sigma Star Systems 414.940.4866 gumm@REDACTED http://www.sigma-star.com From carlsson.richard@REDACTED Wed Aug 10 19:59:40 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 10 Aug 2011 19:59:40 +0200 Subject: [erlang-questions] about syntax tools In-Reply-To: <25140878.233.1312991669033.JavaMail.geo-discussion-forums@yqic37> References: <25140878.233.1312991669033.JavaMail.geo-discussion-forums@yqic37> Message-ID: <4E42C70C.6030504@gmail.com> On 08/10/2011 05:54 PM, Andrew Kondratovich wrote: > Looking through Erlang sources I have found out that there are many ways > to parse and compile the code. In general way there is transformation > like Code -> Tokens -> Forms ->... > But functions for these transformations can be found in many modules: > erl_scan, erl_parse, compile, epp, io - theese are only those that have > seen. Functions from that modules often do the same work. > > What for such variety ? All of these modules and their functions exist for specific purposes. Can you point out any concrete examples of functions that really do the same work? /Richard From dmercer@REDACTED Wed Aug 10 22:49:50 2011 From: dmercer@REDACTED (David Mercer) Date: Wed, 10 Aug 2011 15:49:50 -0500 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: <02a101cc579f$0d5bfa50$2813eef0$@com> On Tuesday, August 02, 2011, Shaun Kruger wrote: > I could really use some contributions for my proxy server/load balancer > project. :) > > https://github.com/skruger/Surrogate > > I recently modified it to allow for writing multiple protocol handlers > (not just HTTP proxy) and I'm working on the admin interface finally. > I know it's a shameless plug, but the project really is cool. Though, > I might just think that because I wrote it. :) Is there any introduction to this project, a quick-start guide or something that briefly but specifically explains what it does/what it is intended to do? From robert.virding@REDACTED Thu Aug 11 01:21:48 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 11 Aug 2011 00:21:48 +0100 (BST) Subject: [erlang-questions] about syntax tools In-Reply-To: <4E42C70C.6030504@gmail.com> Message-ID: erl_scan - the erlang tokeniser which takes charatecters and converts them to tokens. erl_parse - the erlang parser which parses the tokens and build an AST (abstract syntax tree). compile - the top-level of the compiler which runs all its passes, the scanner and parser being the first ones. epp - the erlang pre-processor which uses erl_scan to generate tokens, does all the macro processing and then uses erl_parse to generate the ASTs io - the top-level interface to the io-system for both input and output There is no direct overlap in what these modules do, although some do call the others. So there is only one erlang tokeniser, one erlang parser, one erlang macro pre-processor, etc. Robert ----- Original Message ----- > On 08/10/2011 05:54 PM, Andrew Kondratovich wrote: > > Looking through Erlang sources I have found out that there are many > > ways > > to parse and compile the code. In general way there is > > transformation > > like Code -> Tokens -> Forms ->... > > But functions for these transformations can be found in many > > modules: > > erl_scan, erl_parse, compile, epp, io - theese are only those that > > have > > seen. Functions from that modules often do the same work. > > > > What for such variety ? > > All of these modules and their functions exist for specific purposes. > Can you point out any concrete examples of functions that really do > the > same work? > > /Richard > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From robert.virding@REDACTED Thu Aug 11 01:30:46 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 11 Aug 2011 00:30:46 +0100 (BST) Subject: [erlang-questions] Apparently strange dict behaviour In-Reply-To: <20110809073914.GA15823@erix.ericsson.se> Message-ID: <20ade780-026d-4178-a961-f6fb8208c317@localhost> Yes, dict:append/3 (and dict:append_list/3) assumes, and creates in the first call, the value is actually a list of values kept in the order they were added. If you mix dict:store/3 and dict:append/3 you have to be aware of this and preserve the data-structure. Robert ----- Original Message ----- > Hi. > > I am sorry I did not have time to read your post thoroughly, > because I think you just ran into the feature of the shell > (io:format("~p", ...)) being "intelligent" and prints lists > of integers as strings if the integers can represent printable > characters. > > There is no character type in Erlang, integers are used instead, > and lists of integers instead of strings. (or binaries instead > of strings but that is another story) > > dict:append/3 I think creates a list of the values, your values > are integers, hence you get something that is mistaken for a string. > > But as I said I did not read thoroughly... > > / Raimo > > > > On Tue, Aug 09, 2011 at 04:28:38PM +0930, Darryn wrote: > > Kind Sirs/Madams, > > > > I'm new to erlang, but have extensive prior experience with > > Haskell. I thought > > I'd play with a small version of a parallel map/fold operation, but > > have run > > into an annoying problem. The test example is just a parallel > > estimation by > > sampling of Pi. > > > > pmap(Input, MapFn) -> > > process_flag(trap_exit, true), > > spawn_workers(self(), MapFn, Input), > > M = length(Input), > > collect(M, dict:new()). > > > > spawn_workers(Pid, Fn, Pairs) -> > > lists:foreach(fun({K1, V1}) -> > > spawn_link(fun() -> worker(Pid, Fn, {K1, V1}) end) end, > > Pairs). > > > > worker(Pid, Fn, {K1, V1}) -> > > Pid ! Fn(K1, V1). > > > > collect(0, Dict) -> Dict; > > collect(N, Dict) -> > > receive > > {Key2, Val2} -> > > DictD = dict:append(Key2, Val2, Dict), > > %% <- problem: inserted Val2 integer apparently turns into a > > string?!! > > io:format("Collected result:{~p,~p}~n", [Key2, Val2]), > > io:format(" ->is_integer?~p~n", [is_integer(Val2)]), > > io:format(" ->~p~n", [dict:to_list(DictD)]), > > collect(N, DictD); > > {'EXIT', Who, Why} -> > > io:format("Worker ~p terminated with ~p~n",[Who, Why]), > > collect(N-1, Dict) > > end. > > > > samples(Index, N) -> > > random:seed(erlang:now()), > > {Index, do_samples(0, N, 0)}. > > > > doit() -> > > Results = pmap([{1,100}, {2,100}, {3, 100}], fun samples/2), > > dict:to_list(Results). > > > > > > The problem seems to lie in the collect function, where the > > collected Val2 > > integer (the io:format confirms it is an integer) suddenly becomes > > a string > > upon insertion into the dictionary. Here is example output to > > illustrate. > > > > 20> c(pisim). > > {ok,pisim} > > 21> pisim:doit(). > > Collected result:{1,88} > > ->is_integer?true > > ->[{1,"X"}] > > Worker <0.114.0> terminated with normal > > Collected result:{1,87} > > ->is_integer?true > > ->[{1,"XW"}] > > Worker <0.116.0> terminated with normal > > Collected result:{2,76} > > ->is_integer?true > > ->[{2,"L"},{1,"XW"}] > > Worker <0.115.0> terminated with normal > > [{2,"L"},{1,"XW"}] > > 22> c(pisim). > > {ok,pisim} > > 23> pisim:doit(). > > Worker <0.118.0> terminated with normal > > Collected result:{1,78} > > ->is_integer?true > > ->[{1,"N"}] > > Worker <0.123.0> terminated with normal > > Collected result:{2,83} > > ->is_integer?true > > ->[{2,"S"},{1,"N"}] > > Worker <0.124.0> terminated with normal > > [{2,"S"},{1,"N"}] > > > > I don't understand how the Val2 integer suddenly turns into a > > string upon > > being inserted into the dictionary; when I manually cut and paste > > the relevant > > bits of code into the shell I get dictionary values with integers > > as I intend. > > > > I wonder if anyone more experienced can see what I've done wrong? > > Thanks so > > much for any help! > > > > Darryn. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From joel.meyer@REDACTED Thu Aug 11 01:42:09 2011 From: joel.meyer@REDACTED (Joel Meyer) Date: Wed, 10 Aug 2011 16:42:09 -0700 Subject: [erlang-questions] Proper type for passing pointer to/from linked-in driver? Message-ID: What's the recommended way to pass a pointer between a linked-in driver and the beam? I can cast it to a uint64_t and use ErlDrvUInt64 (and do the reverse when sending it back to the driver), but maybe there's a better way? Thanks, Joel -------------- next part -------------- An HTML attachment was scrubbed... URL: From skruger@REDACTED Thu Aug 11 06:19:08 2011 From: skruger@REDACTED (Shaun Kruger) Date: Wed, 10 Aug 2011 22:19:08 -0600 (MDT) Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: <02a101cc579f$0d5bfa50$2813eef0$@com> Message-ID: Unfortunately I'm guilty of writing a cool, but under documented open source program. What Surrogate does is provide forward proxy (HTTP and SOCKS) or reverse proxy with load balancing. The HTTP proxy is meant to allow developers to create custom stream modification handlers. These handlers can change various configuration settings. For instance, mod_host_pool reads the hostname from the Host: header and routes the traffic to a different load balancer pool based on an mnesia lookup. I also implemented a raw proxy handler which allows me to load balance anything that does not need to be protocol aware. This may be useful for load balancing mysql database connections in some cases. Surrogate does a lot, but it is to be viewed as a platform for creating other things as is evidenced by all of the behaviours that I have specified (and need to better document). filter_stream - This is the behaviour for inspecting and modifying the HTTP proxy behaviour based on what data passes through. gen_balancer - This behaviour allows a developer to implement additional load balance policies. I only wrote a round robin balancer thus far, but I would like to build others like least connections or response time. healthcheck - This behaviour allows for creating modules that work with mod_balancer. healthchecks return status information about a host so it can be automatically removed from a pool. proxy_mod - This behaviour is used for automatically starting and stopping modules. This facility is not unlike gen_mod in ejabberd. proxy_protocol - I recently add this one to allow surrogate to implement multiple protocol handlers. Before I did this it was just an HTTP/HTTPS/SOCKS proxy. Now I have been able to add a raw handler, and an http_admin handler. There is a lot there. filter_stream was the first big thing I wanted to do with Surrogate because I thought it would be great if open source load balancers could integrate better with web applications. To me it is the difference between load balancing and application delivery and I wanted to have more options than F5 when it comes to that. gen_balancer and heathcheck flowed from that based on the fact that without balance pools that are smart enough to drop nodes then you won't get very far. Once I had those basics down I really wanted to add clustering. This is erlang after all and load balancing really does need to be redundant in any kind of serious environment. That lead me to start developing some clustering features that I eventually split into their own project: https://github.com/skruger/ClusterSupervisor Cluster supervisor is integrated through gen_cluster which starts the cluster supervisor application and subscribes to vip up and vip down notification from cluster supervisor. Some of this is still a bit clunky. Last I tested it things worked pretty OK, but this is clustering and clustering with service takeover on failure is hard by nature. Now that I've talked a lot about what is in Surrogate I guess I need to explain how to get started. That I'm not sure how to do. I think I suffer an affliction common to many developers; I'm so close to my code that I don't know what isn't obvious to others. That being said, I would love feedback about what I need to clarify in order to help others get started. I think I'm going to make myself sit down and write some step by step instructions for installing Surrogate and configuring a forward and reverse HTTP proxy. Check the wiki tomorrow, I'll see how far I can get on that tonight. Once I finish that I would appreciate some feedback about what the document is missing. Shaun ----- Original Message ----- > From: "David Mercer" > To: "Shaun Kruger" , "Nick S" > Cc: erlang-questions@REDACTED > Sent: Wednesday, August 10, 2011 2:49:50 PM > Subject: RE: [erlang-questions] Developing killer / open source apps > > On Tuesday, August 02, 2011, Shaun Kruger wrote: > > > I could really use some contributions for my proxy server/load > > balancer > > project. :) > > > > https://github.com/skruger/Surrogate > > > > I recently modified it to allow for writing multiple protocol > > handlers > > (not just HTTP proxy) and I'm working on the admin interface > > finally. > > I know it's a shameless plug, but the project really is cool. > > Though, > > I might just think that because I wrote it. :) > > Is there any introduction to this project, a quick-start guide or > something > that briefly but specifically explains what it does/what it is > intended to > do? > > From rapsey@REDACTED Thu Aug 11 07:34:39 2011 From: rapsey@REDACTED (Rapsey) Date: Thu, 11 Aug 2011 07:34:39 +0200 Subject: [erlang-questions] Proper type for passing pointer to/from linked-in driver? In-Reply-To: References: Message-ID: That's how I did it in my old driver. Then I switched to a NIF and life was so much easier with resources. You should think about using a NIF instead of a driver. Sergej On Thu, Aug 11, 2011 at 1:42 AM, Joel Meyer wrote: > What's the recommended way to pass a pointer between a linked-in driver and > the beam? I can cast it to a uint64_t and use ErlDrvUInt64 (and do the > reverse when sending it back to the driver), but maybe there's a better way? > > Thanks, > Joel > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Thu Aug 11 08:27:40 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Thu, 11 Aug 2011 09:27:40 +0300 Subject: [erlang-questions] PropEr: specify custom types in ?SIZED() fun In-Reply-To: <20110810160847.GA14706@jakstys.lt> References: <20110810160847.GA14706@jakstys.lt> Message-ID: <20110811062740.GA15380@jakstys.lt> I re-read my own post and it did not look polite enough. Sorry. Corrected version (only the first line...): Hello List, I want to specify my custom type depending on recursion level of generated input. Consider this simple example: -module(properworks). -include_lib("proper/include/proper.hrl"). -compile(export_all). -type recursive() :: binary() | [recursive()]. sized_struct(Size) when Size > 10 -> binary(); sized_struct(_Size) -> recursive(). prop_simple() -> ?FORALL(_Item, ?SIZED(Size, sized_struct(Size)), begin true end ). $ erlc -pz ~/nrms/deps/proper/ebin -I ~/nrms/deps/proper properworks.erl ./properworks.erl:10: function recursive/0 undefined How can I specify a custom data type for generation in sized_struct/1? Built-in types work (binary() as in this example), but not mine. I tried ?LAZY, union() wrappers, no success. P.S. does PropEr have a mailing list? I would hang there if it did. :) P.P.S thank you for creating PropEr. Amazing tool. Regards, Motiejus From Andrew.Cheese@REDACTED Thu Aug 11 08:29:40 2011 From: Andrew.Cheese@REDACTED (Andrew Cheese) Date: Thu, 11 Aug 2011 08:29:40 +0200 Subject: [erlang-questions] FW: Mailman privacy alert Message-ID: <010f01cc57f0$0eab32d0$2c019870$%cheese@huawei.com> Hi, I was trying to change my subscription to a daily digest (otherwise my inbox overflows :-) How do I do this ? Thanks Andy Andrew Cheese System Architecture Expert HUAWEI TECHNOLOGIES CO.,LTD. huawei_logo Address: Skalholtsgatan 11, 16440 Kista, Sweden www.huawei.com ---------------------------------------------------------------------------- --------------------------------------------------------- This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] Sent: den 11 augusti 2011 08:24 To: Andrew.Cheese@REDACTED Subject: Mailman privacy alert An attempt was made to subscribe your address to the mailing list erlang-questions@REDACTED You are already subscribed to this mailing list. Note that the list membership is not public, so it is possible that a bad person was trying to probe the list for its membership. This would be a privacy violation if we let them do this, but we didn't. If you submitted the subscription request and forgot that you were already subscribed to the list, then you can ignore this message. If you suspect that an attempt is being made to covertly discover whether you are a member of this list, and you are worried about your privacy, then feel free to send a message to the list administrator at erlang-questions-owner@REDACTED From Andrew.Cheese@REDACTED Thu Aug 11 08:31:00 2011 From: Andrew.Cheese@REDACTED (Andrew Cheese) Date: Thu, 11 Aug 2011 08:31:00 +0200 Subject: [erlang-questions] Recall: Mailman privacy alert Message-ID: Andrew Cheese would like to recall the message, "Mailman privacy alert". -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 1121 bytes Desc: not available URL: From gertvangool@REDACTED Thu Aug 11 08:32:03 2011 From: gertvangool@REDACTED (Gert Van Gool) Date: Thu, 11 Aug 2011 08:32:03 +0200 Subject: [erlang-questions] FW: Mailman privacy alert In-Reply-To: <010f01cc57f0$0eab32d0$2c019870$%cheese@huawei.com> References: <010f01cc57f0$0eab32d0$2c019870$%cheese@huawei.com> Message-ID: At the bottom of the [page](http://erlang.org/mailman/listinfo/erlang-questions) is a field with next to it a button "unsubscribe or edit options". Put your email in there and you should be able to update it. -- Gert Mobile: +32 498725202 Twitter: @gvangool Web: http://gertvangool.be On Thu, Aug 11, 2011 at 08:29, Andrew Cheese wrote: > Hi, > > I was trying to change my subscription to a daily digest (otherwise my inbox > overflows :-) > > How do I do this ? > > Thanks > Andy > > Andrew Cheese > System Architecture Expert > HUAWEI TECHNOLOGIES CO.,LTD. ?huawei_logo > > > Address: > Skalholtsgatan 11, > 16440 Kista, > Sweden > www.huawei.com > ---------------------------------------------------------------------------- > --------------------------------------------------------- > This e-mail and its attachments contain confidential information from > HUAWEI, which > is intended only for the person or entity whose address is listed above. Any > use of the > information contained herein in any way (including, but not limited to, > total or partial > disclosure, reproduction, or dissemination) by persons other than the > intended > recipient(s) is prohibited. If you receive this e-mail in error, please > notify the sender by > phone or email immediately and delete it! > > > -----Original Message----- > From: erlang-questions-bounces@REDACTED > [mailto:erlang-questions-bounces@REDACTED] > Sent: den 11 augusti 2011 08:24 > To: Andrew.Cheese@REDACTED > Subject: Mailman privacy alert > > An attempt was made to subscribe your address to the mailing list > erlang-questions@REDACTED ?You are already subscribed to this mailing > list. > > Note that the list membership is not public, so it is possible that a bad > person was trying to probe the list for its membership. ?This would be a > privacy violation if we let them do this, but we didn't. > > If you submitted the subscription request and forgot that you were already > subscribed to the list, then you can ignore this message. ?If you suspect > that > an attempt is being made to covertly discover whether you are a member of > this > list, and you are worried about your privacy, then feel free to send a > message > to the list administrator at erlang-questions-owner@REDACTED > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From carlsson.richard@REDACTED Thu Aug 11 09:41:11 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 11 Aug 2011 09:41:11 +0200 Subject: [erlang-questions] Apparently strange dict behaviour In-Reply-To: <20ade780-026d-4178-a961-f6fb8208c317@localhost> References: <20ade780-026d-4178-a961-f6fb8208c317@localhost> Message-ID: <4E438797.4080200@gmail.com> On 08/11/2011 01:30 AM, Robert Virding wrote: > Yes, dict:append/3 (and dict:append_list/3) assumes, and creates in > the first call, the value is actually a list of values kept in the > order they were added. Of course, it's a bad idea in general to accumulate things by appending to a list (as opposed to prepending), so except for quite short lists, I don't recommend using these functions. It's better to use the update/4 function to make an operator that accumulates by prepending, and then reverse the stored list when you look it up (if order is of interest). /Richard From caox@REDACTED Thu Aug 11 10:39:33 2011 From: caox@REDACTED (caox) Date: Thu, 11 Aug 2011 16:39:33 +0800 Subject: [erlang-questions] What is the difference between keep-alive and pipeline in intes http client Message-ID: Hi According to RFC2616, pipelining allows client to send multiple requests without waiting for each response. So I guess the difference between pipeline and normal keep alive is that the later should send request only after request on the connection has been responded. But the implementation of inets http client seems not same as my expectation. Both of pipeline and keep-alive send request directly without checking whether the earlier request has been responded. So I have missed something in the source code, or just misunderstood the RFC? What is the difference between keep-alive and pipeline? BR From erlang@REDACTED Thu Aug 11 11:01:07 2011 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 11 Aug 2011 11:01:07 +0200 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: References: Message-ID: On Wed, Aug 10, 2011 at 1:04 AM, Sam Elliott wrote: > Hi erlang-questions, > > Recently I needed to compare a few Erlang web libraries and frameworks > for a friend who was writing a simple internal API endpoint. He > suggested I should publish the rundown for others, and I thought I'd > also circulate it here. I'm relatively new to Erlang, so I have > probably missed a few libraries. > > I'll try to keep it updated as people send me information, so don't > hesitate to contact me if i've got something wrong. > > The article: http://lenary.co.uk/erlang/2011/08/erlang-web-libraries/ > > What are your thoughts? This is great. In addition to your summaries (which are excellent) I'd like to see a simple cross-framework API. Is what your friend is doing? For example, If I start off by using yaws and then change my mind later and decide to use mochiweb I'd like this to have minimal impact on my code. It should be a simple as changing the name of an interface module. (( ie a GoF adapter pattern )) To use yaws I'd like to write: -define(?ADAPTOR, yaws_adapter). -include("adaptor.hrl"). -import(?ADAPTOR, [my_callback/1, reply/1, start_web_server/0]). start() -> ?ADAPTER:start_web_server(), ?ADAPTER:my_callback(fun my_handler/1). my_handler(X) when is_record(X,'GET') -> ... ?ADAPTER:reply(#'RELY'{type=html, data=Binary}) If I wanted to use mochiweb I'd just have to change the module name of the adaptor. (this was the original reason for the import declaration in Erlang - to make changing the implementation of a feature simple :-) Exactly which routines the adapter would export and the records in the include file would need a little thought - but handling HTTP GET, POST, PUT, DELETE requests should be easy enough If all suppliers of web- frameworks could adopt a common adapter pattern - it would make life a lot easier. I realize that this would not allow us to use some of the finer facilities offered by an individual web-server, but it would be very useful for the simple cases of serving up files etc. One could have a common API then choose the implementation depending upon the requirements of the application. /Joe > > Sam > > -- > Sam Elliott > sam@REDACTED > -- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From andrew.kondratovich@REDACTED Thu Aug 11 11:26:33 2011 From: andrew.kondratovich@REDACTED (Andrew Kondratovich) Date: Thu, 11 Aug 2011 02:26:33 -0700 (PDT) Subject: [erlang-questions] about syntax tools In-Reply-To: References: <4E42C70C.6030504@gmail.com> Message-ID: <8978295.1326.1313054793471.JavaMail.geo-discussion-forums@yqgp20> Default compiler use epp:parse_file/3 to get forms, and then several passes for optimizations and compiling. epp:parse_file use epp:parse_form. epp:parse_form use epp:parse_erl_form to get forms. epp:parse_erl_form use epp:scan_erl_form to get tokens and then erl_parse:parse_form to convert tokens to forms. epp:scan_erl_form use scan_toks, which use io:scan_erl_form to get tokens from file (Epp handler). io:scan_erl_form use erl_scan:tokens to get tokens from code. Pff.. Now i see. -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Aug 11 11:30:32 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 11 Aug 2011 11:30:32 +0200 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: References: Message-ID: <4E43A138.30300@dev-extend.eu> On 08/11/2011 11:01 AM, Joe Armstrong wrote: > If all suppliers of web- frameworks could adopt a common adapter > pattern - it would make > life a lot easier. I realize that this would not allow us to use some > of the finer facilities > offered by an individual web-server, but it would be very useful for the simple > cases of serving up files etc. A lot of people choose the server based on those finer facilities. Adaptor projects are already used by everyone else. See below. > One could have a common API then choose the implementation depending upon > the requirements of the application. This is fine and all, but some servers use lists while others use binaries. The handler code is obviously different so some servers would require converting everything (headers, path, ...) to lists or binary before passing the data to your common handler. There's no clean solution there. Some projects exist, like simple_bridge for example, but AFAIK they only do lists-based servers for now. What's a good solution to this? -- Lo?c Hoguin Dev:Extend From sam@REDACTED Thu Aug 11 11:55:47 2011 From: sam@REDACTED (Sam Elliott) Date: Thu, 11 Aug 2011 10:55:47 +0100 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: References: Message-ID: On Thu, Aug 11, 2011 at 10:01 AM, Joe Armstrong wrote: > On Wed, Aug 10, 2011 at 1:04 AM, Sam Elliott wrote: >> Hi erlang-questions, >> >> Recently I needed to compare a few Erlang web libraries and frameworks >> for a friend who was writing a simple internal API endpoint. He >> suggested I should publish the rundown for others, and I thought I'd >> also circulate it here. I'm relatively new to Erlang, so I have >> probably missed a few libraries. >> >> I'll try to keep it updated as people send me information, so don't >> hesitate to contact me if i've got something wrong. >> >> The article: http://lenary.co.uk/erlang/2011/08/erlang-web-libraries/ >> >> What are your thoughts? > > This is great. In addition to your summaries (which are excellent) I'd > like to see > a simple cross-framework API. Is what your friend is doing? Originally my idea was just to provide a rundown so that it makes it easier to choose the correct framework for the job (which is all my friend wanted too), however, your idea for an adapter doesn't sound bad at all. I'm about to say more in reply to Lo?c's email. > For example, If I start off by using yaws and then change my mind > later and decide to use mochiweb > I'd like this to have minimal impact on my code. It should be a simple > as changing the name of > an interface module. (( ie a GoF adapter pattern )) > > To use yaws I'd like to write: > > ? ?-define(?ADAPTOR, yaws_adapter). > > ? ?-include("adaptor.hrl"). > > ? ?-import(?ADAPTOR, [my_callback/1, reply/1, start_web_server/0]). > > ? ?start() -> > ? ? ? ? ? ?ADAPTER:start_web_server(), > ? ? ? ? ? ?ADAPTER:my_callback(fun my_handler/1). > > ? ?my_handler(X) when is_record(X,'GET') -> > ? ? ? ? ? ... > ? ? ? ? ? ?ADAPTER:reply(#'RELY'{type=html, data=Binary}) > > > If I wanted to use mochiweb I'd just have to change the module name of > the adaptor. > (this was the original reason for the import declaration in Erlang - > to make changing the > implementation of a feature simple :-) > > Exactly which routines the adapter would export and the records in > the include file would need a little thought - but handling > HTTP GET, POST, PUT, DELETE requests should be easy enough > > If all suppliers of web- frameworks could adopt a common adapter > pattern - it would make > life a lot easier. I realize that this would not allow us to use some > of the finer facilities > offered by an individual web-server, but it would be very useful for the simple > cases of serving up files etc. > > One could have a common API then choose the implementation depending upon > the requirements of the application. > > ?/Joe > >> >> Sam >> >> -- >> Sam Elliott >> sam@REDACTED >> -- >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From sam@REDACTED Thu Aug 11 12:11:31 2011 From: sam@REDACTED (Sam Elliott) Date: Thu, 11 Aug 2011 11:11:31 +0100 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: <4E43A138.30300@dev-extend.eu> References: <4E43A138.30300@dev-extend.eu> Message-ID: On Thu, Aug 11, 2011 at 10:30 AM, Lo?c Hoguin wrote: > On 08/11/2011 11:01 AM, Joe Armstrong wrote: >> If all suppliers of web- frameworks could adopt a common adapter >> pattern - it would make >> life a lot easier. I realize that this would not allow us to use some >> of the finer facilities >> offered by an individual web-server, but it would be very useful for the simple >> cases of serving up files etc. > > A lot of people choose the server based on those finer facilities. > Adaptor projects are already used by everyone else. See below. I don't think that's much of an issue. People don't have to use the adapter, but where it is helpful, they have the possibility of using it. In a lot of cases, People don't care which library they are using, they just want to use one that they understand and can get started with quickly. > >> One could have a common API then choose the implementation depending upon >> the requirements of the application. > > This is fine and all, but some servers use lists while others use > binaries. The handler code is obviously different so some servers would > require converting everything (headers, path, ...) to lists or binary > before passing the data to your common handler. There's no clean > solution there. I think the solution is to choose some sensible defaults for the adapter API, then put effort into the conversion that the underlying implementation requires. > Some projects exist, like simple_bridge for example, but AFAIK they only > do lists-based servers for now. SimpleBridge (https://github.com/etnt/SimpleBridge) and EWGI (https://github.com/skarab/ewgi) seem to be quite close to being useful as an adapter, but there's still an annoying reliance on calling things like `simple_bridge:make_request(inets_response_bridge, Info)` - the abstraction only seems to wrap the request and response objects to provide a compatible interface. I, however, agree with Joe in that the abstraction needs to cover far more than that. I'll probably add SimpleBridge and EWGI to my list today. Sam > > What's a good solution to this? > > -- > Lo?c Hoguin > Dev:Extend > From erlang@REDACTED Thu Aug 11 12:16:39 2011 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 11 Aug 2011 12:16:39 +0200 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: <4E43A138.30300@dev-extend.eu> References: <4E43A138.30300@dev-extend.eu> Message-ID: On Thu, Aug 11, 2011 at 11:30 AM, Lo?c Hoguin wrote: > On 08/11/2011 11:01 AM, Joe Armstrong wrote: >> If all suppliers of web- frameworks could adopt a common adapter >> pattern - it would make >> life a lot easier. I realize that this would not allow us to use some >> of the finer facilities >> offered by an individual web-server, but it would be very useful for the simple >> cases of serving up files etc. > > A lot of people choose the server based on those finer facilities. > Adaptor projects are already used by everyone else. See below. > >> One could have a common API then choose the implementation depending upon >> the requirements of the application. > > This is fine and all, but some servers use lists while others use > binaries. The handler code is obviously different so some servers would > require converting everything (headers, path, ...) to lists or binary > before passing the data to your common handler. There's no clean > solution there. Right, that's why an adaptor is useful. I suspect the differences by accident rather than design. I would assume the "natural" way to transfer large blobs would be as binaries, and to communicate parsed attributes would be a Key-Value proplist of strings and so on. Having a well defined adaptor interface might have beneficial side effect of causeing the sofwtare at either side of the interface to migrate towards a set of common types in the interface. > > Some projects exist, like simple_bridge for example, but AFAIK they only > do lists-based servers for now. I hadn't heard of simple bridge until you mentioned it - it looks pretty much to be what I wanted. @sam - can you add simple_bridge to your list (its at https://github.com/etnt/SimpleBridge) I wonder how many other adaptor patterns there are: if simple_bridge is the web adaptor where that the adapters for key-val databases? (( aside -- it is a source of never ending frustration that the interfaces to ets and dets are different getting people to change ets or dets is very difficult since we don't want to break old code but an adapter would do the job )) > What's a good solution to this? I don't know - I suspect good interfaces emerge. It's often not apparent at the start of a project what the best API is. We can't abstract over several interfaces until we have several interfaces. Once the problem is well understood and we have several implemenations we can begin to generalize and abstract out what should have been the interface. Then we have to refactor our programs. (( This is the 99% hard-work part - software gets continually polished )) /Joe > -- > Lo?c Hoguin > Dev:Extend > From erlang@REDACTED Thu Aug 11 12:27:31 2011 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 11 Aug 2011 12:27:31 +0200 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: References: <4E43A138.30300@dev-extend.eu> Message-ID: On Thu, Aug 11, 2011 at 12:11 PM, Sam Elliott wrote: > On Thu, Aug 11, 2011 at 10:30 AM, Lo?c Hoguin wrote: >> On 08/11/2011 11:01 AM, Joe Armstrong wrote: >>> If all suppliers of web- frameworks could adopt a common adapter >>> pattern - it would make >>> life a lot easier. I realize that this would not allow us to use some >>> of the finer facilities >>> offered by an individual web-server, but it would be very useful for the simple >>> cases of serving up files etc. >> >> A lot of people choose the server based on those finer facilities. >> Adaptor projects are already used by everyone else. See below. > > I don't think that's much of an issue. People don't have to use the > adapter, but where it is helpful, they have the possibility of using > it. > > In a lot of cases, People don't care which library they are using, > they just want to use one that they understand and can get started > with quickly. I agree fully and for this reason I think the generic adapter should offer a simple API and come with a few simple get-you-started examples. It should not attempt to drill down into the finer details of the individual back-ends. I think development should proceed as follows: 1) start with the generic adapter this should be the easiest of all the interfaces to understand and get working 2) choose any backend they should all work out of the box with the generic adapter 3) if this is not good enough change the choice of backend the API is the same but the non-functional behaviors might be different some servers replicate, load-balance etc others don't 4) if this is not good enough use the backend native API A casual user would start at point 1) then progress down the scale only if and when they had problems. /Joe > >> >>> One could have a common API then choose the implementation depending upon >>> the requirements of the application. >> >> This is fine and all, but some servers use lists while others use >> binaries. The handler code is obviously different so some servers would >> require converting everything (headers, path, ...) to lists or binary >> before passing the data to your common handler. There's no clean >> solution there. > > I think the solution is to choose some sensible defaults for the > adapter API, then put effort into the conversion that the underlying > implementation requires. > >> Some projects exist, like simple_bridge for example, but AFAIK they only >> do lists-based servers for now. > > SimpleBridge (https://github.com/etnt/SimpleBridge) and EWGI > (https://github.com/skarab/ewgi) seem to be quite close to being > useful as an adapter, but there's still an annoying reliance on > calling things like `simple_bridge:make_request(inets_response_bridge, > Info)` - the abstraction only seems to wrap the request and response > objects to provide a compatible interface. I, however, agree with Joe > in that the abstraction needs to cover far more than that. > > I'll probably add SimpleBridge and EWGI to my list today. > > Sam > >> >> What's a good solution to this? >> >> -- >> Lo?c Hoguin >> Dev:Extend >> > From essen@REDACTED Thu Aug 11 12:50:05 2011 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Thu, 11 Aug 2011 12:50:05 +0200 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: References: <4E43A138.30300@dev-extend.eu> Message-ID: <4E43B3DD.80000@dev-extend.eu> On 08/11/2011 12:16 PM, Joe Armstrong wrote: > On Thu, Aug 11, 2011 at 11:30 AM, Lo?c Hoguin wrote: >> On 08/11/2011 11:01 AM, Joe Armstrong wrote: >>> If all suppliers of web- frameworks could adopt a common adapter >>> pattern - it would make >>> life a lot easier. I realize that this would not allow us to use some >>> of the finer facilities >>> offered by an individual web-server, but it would be very useful for the simple >>> cases of serving up files etc. >> >> A lot of people choose the server based on those finer facilities. >> Adaptor projects are already used by everyone else. See below. >> >>> One could have a common API then choose the implementation depending upon >>> the requirements of the application. >> >> This is fine and all, but some servers use lists while others use >> binaries. The handler code is obviously different so some servers would >> require converting everything (headers, path, ...) to lists or binary >> before passing the data to your common handler. There's no clean >> solution there. > > Right, that's why an adaptor is useful. I suspect the differences > by accident rather than design. I would assume the "natural" way to > transfer large blobs would be as binaries, and to communicate parsed > attributes would be a Key-Value proplist of strings and so on. Cowboy has been filling a demand for a fully binary server, so I don't really agree with that. It's been designed that way by design, naturally gains from that binary usage (Cowboy can require up to 2 times less memory for the same number of connections). The other API changes are also by design and actually OTPish rather than providing a fun to handle your request. It also uses only 1 process per request instead of 2 in other servers, again by design. You should take a look at it, I've only had positive feedback on the design so far but I'm always looking for ways to improve it, nothing's set in stone yet. So if you've got the time, that'd be much appreciated! > Having a well defined adaptor interface might have beneficial side effect > of causeing the sofwtare at either side of the interface to migrate > towards a set > of common types in the interface. I agree with that. -- Lo?c Hoguin Dev:Extend From torben.lehoff@REDACTED Thu Aug 11 14:40:09 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 11 Aug 2011 14:40:09 +0200 Subject: [erlang-questions] Link to funny movie Message-ID: Hi, A while back there was a link to some funny youtube video (cartoon) with synthetic voices where a a non-Erlanger wanted to learn if Erlang could solve all his problems since he had heard all these wonderful things about Erlang; the Erlanger has to let him down on his crazy desires and the non-Erlanger concludes that the Erlang community is weird/closed or something like that. I have tried to find it but I cannot get it again. Any links out there? Thanks in advance, Torben -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Thu Aug 11 15:14:37 2011 From: dmercer@REDACTED (David Mercer) Date: Thu, 11 Aug 2011 08:14:37 -0500 Subject: [erlang-questions] Link to funny movie In-Reply-To: References: Message-ID: <02cf01cc5828$a0300720$e0901560$@com> This? http://www.youtube.com/watch?v=xEJ1n13soWU From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Torben Hoffmann Sent: Thursday, August 11, 2011 7:40 AM To: erlang-questions Subject: [erlang-questions] Link to funny movie Hi, A while back there was a link to some funny youtube video (cartoon) with synthetic voices where a a non-Erlanger wanted to learn if Erlang could solve all his problems since he had heard all these wonderful things about Erlang; the Erlanger has to let him down on his crazy desires and the non-Erlanger concludes that the Erlang community is weird/closed or something like that. I have tried to find it but I cannot get it again. Any links out there? Thanks in advance, Torben -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Thu Aug 11 16:33:50 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Thu, 11 Aug 2011 10:33:50 -0400 Subject: [erlang-questions] What is the difference between keep-alive and pipeline in intes http client In-Reply-To: References: Message-ID: On Thu, Aug 11, 2011 at 4:39 AM, caox wrote: > Hi > > ? ? ? ?According to RFC2616, pipelining allows client to send multiple requests without waiting for each ?response. So I guess the difference between pipeline and normal keep alive is that the later should send request only after request on the connection has been responded. > ? ? ? ?But the implementation of inets http client seems not same as my expectation. Both of pipeline and keep-alive send request directly without checking whether the earlier request has been responded. > ? ? ? ?So I have missed something in the source code, or just misunderstood the RFC? What is the difference between keep-alive and pipeline? Prior to HTTP 1.1 a client established a connection, sent its request, got the response, and the connection was closed. Keep-alive allowed the client to specify that it wanted to keep the connection open for multiple request/response pairs. Under HTTP 1.1 keep-alive is unnecessary since connections are persistent by default. Pipelining allows a client to send a sequence of idempotent requests and requires the server to send a sequence of responses in the same order as the corresponding requests. It's a feature of HTTP 1.1. Without pipelining you have a series of request/response pairs; with pipelining you have a series of a sequence of idempotent requests, each sequence paired with a sequence of corresponding responses to those requests. --steve From gnoblin@REDACTED Thu Aug 11 17:39:03 2011 From: gnoblin@REDACTED (Slav Pankratov) Date: Thu, 11 Aug 2011 18:39:03 +0300 Subject: [erlang-questions] record to bson (via emongo:encode() ) Message-ID: Hello! I'd like to encode my erlang record to bson via emongo:encode. What would be the best way to do it? thanks, Slav -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Thu Aug 11 17:47:46 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Thu, 11 Aug 2011 18:47:46 +0300 Subject: [erlang-questions] user-specified data types in PropEr generator In-Reply-To: <20110811062740.GA15380@jakstys.lt> References: <20110810160847.GA14706@jakstys.lt> <20110811062740.GA15380@jakstys.lt> Message-ID: <20110811154746.GA21345@jakstys.lt> Hello again, This is a follow-up email from PropEr: specify custom types in ?SIZED() fun http://erlang.org/pipermail/erlang-questions/2011-August/060592.html Sorry for spamming about this, but I think now I found the most minimal example of what I am trying to achieve. Code: -module(properworks). -include_lib("proper/include/proper.hrl"). -type primitive() :: binary(). struct_gen(_Size) -> primitive(). prop_simple() -> ?FORALL(_Item, sized_struct(Size), begin true end ). $ erlc -pz ~/nrms/deps/proper/ebin -I ~/nrms/deps/proper properworks.erl ./properworks.erl:7: function primitive/0 undefined ./properworks.erl:4: Warning: type primitive() is unused ./properworks.erl:6: Warning: function struct_gen/1 is unused Any way to conditionally feed custom data types to PropEr generator? Thanks, Motiejus From rexxe98@REDACTED Thu Aug 11 19:33:57 2011 From: rexxe98@REDACTED (Andrew Berman) Date: Thu, 11 Aug 2011 10:33:57 -0700 Subject: [erlang-questions] record to bson (via emongo:encode() ) In-Reply-To: References: Message-ID: You should definitely look at exprecs here: https://github.com/uwiger/parse_trans It will make your life much, much easier when dealing with records and will make this task far easier to accomplish. On Thu, Aug 11, 2011 at 8:39 AM, Slav Pankratov wrote: > Hello! > > I'd like to encode my erlang record to bson via emongo:encode. > What would be the best way to do it? > > thanks, > Slav > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From rapsey@REDACTED Thu Aug 11 19:48:56 2011 From: rapsey@REDACTED (Rapsey) Date: Thu, 11 Aug 2011 19:48:56 +0200 Subject: [erlang-questions] record to bson (via emongo:encode() ) In-Reply-To: References: Message-ID: Erlmongo has record to bson serialization. Sergej On Thu, Aug 11, 2011 at 5:39 PM, Slav Pankratov wrote: > Hello! > > I'd like to encode my erlang record to bson via emongo:encode. > What would be the best way to do it? > > thanks, > Slav > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Thu Aug 11 21:37:26 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 11 Aug 2011 22:37:26 +0300 Subject: [erlang-questions] user-specified data types in PropEr generator In-Reply-To: <20110811154746.GA21345@jakstys.lt> References: <20110810160847.GA14706@jakstys.lt> <20110811062740.GA15380@jakstys.lt> <20110811154746.GA21345@jakstys.lt> Message-ID: <4E442F76.5050908@cs.ntua.gr> On 08/11/11 18:47, Motiejus Jak?tys wrote: > Hello again, > > This is a follow-up email from > PropEr: specify custom types in ?SIZED() fun > http://erlang.org/pipermail/erlang-questions/2011-August/060592.html > > Sorry for spamming about this, but I think now I found the most minimal > example of what I am trying to achieve. Code: Would you be so kind so as to wait till the mid of next week? It's vacation time for southern Europe and communication between PropEr developers in order to have a proper answer and/or a fix for this issue is a bit problematic till then. Kostis PS. Thanks for your comments about PropEr! We will also try to set up a mailing list for PropEr users some time later in August. From joel.meyer@REDACTED Fri Aug 12 00:22:04 2011 From: joel.meyer@REDACTED (Joel Meyer) Date: Thu, 11 Aug 2011 15:22:04 -0700 Subject: [erlang-questions] Proper type for passing pointer to/from linked-in driver? In-Reply-To: References: Message-ID: Thanks Sergej. I'll look at the NIF spec again. I'm making heavy use of the async thread pool to avoid blocking a scheduler thread, I don't recall seeing a similar pool available for NIFs, but I may have missed it. Joel On Wed, Aug 10, 2011 at 10:34 PM, Rapsey wrote: > That's how I did it in my old driver. Then I switched to a NIF and life was > so much easier with resources. You should think about using a NIF instead of > a driver. > > > Sergej > > On Thu, Aug 11, 2011 at 1:42 AM, Joel Meyer wrote: > >> What's the recommended way to pass a pointer between a linked-in driver >> and the beam? I can cast it to a uint64_t and use ErlDrvUInt64 (and do the >> reverse when sending it back to the driver), but maybe there's a better way? >> >> Thanks, >> Joel >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Fri Aug 12 01:53:40 2011 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 11 Aug 2011 16:53:40 -0700 (PDT) Subject: [erlang-questions] Databases and economics Message-ID: <7b64ed87-5eb5-4fbd-8029-e4790dd94739@b20g2000vbz.googlegroups.com> Totally off topic, but a very interesting approach... http://www.physorg.com/news/2011-08-economics-database.html ...not at all to do with erlang, but the sort of thing that I felt many in this audience may appreciate. /s From xuxb1979@REDACTED Fri Aug 12 07:02:07 2011 From: xuxb1979@REDACTED (Xiaobin Xu) Date: Fri, 12 Aug 2011 13:02:07 +0800 Subject: [erlang-questions] prim_file:translate_response/2 take very long time almost suspend In-Reply-To: References: Message-ID: Hi, First of all, is that necessary to log every SQL expression even if it may successfully commit to the database? I suppose you could benefit from "log only error" strategy. Although I don't really know prim_file thing, but from my personal experience, you may use "lagger" or "log4erl" or the log modules from "ejabbered", that may resolve your problem. 2011/8/7 ?? > Hi. > > In my game project, I use ETS mapping mysql and use two process sync the > ETS table's changes to mysql. > > One process generate SQL and write to a log file and then send SQL to > another process to execute. > > In recent days, I got a problem. After some unknow thing happen (look like > always happen on busy node). The SQL log process run into very very slow and > no recovery. > > When this happen, I must disconnect all players and suspend the SQL log > process and copy the messages to SQL execute process, after all action sync, > restart the node. :( > > And when this happen, I always can get: > > {current_function,{prim_file,translate_response,2}} > > by: > > process_info(SqlLogProc, current_function). > > I think something make the "translate_response" take very very long time so > the log file write very very slow. > > The log file open by: > > file:open(FileName, [append, raw, {delayed_write, 1024 * 1000 * 10, > 6000}]). > > and write by: > > file:write(File, SQL). > > I use R14B3 on server. > > Is it a bug of erlang's prim_file module? or what can I optimize? > > I have 350+ game server node and add 15 ~ 20 node every day, this problem > happens on 1 ~ 2 node every day, and make me crazy I need sleep :( > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Fri Aug 12 08:27:01 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Fri, 12 Aug 2011 09:27:01 +0300 Subject: [erlang-questions] user-specified data types in PropEr generator In-Reply-To: <4E442F76.5050908@cs.ntua.gr> References: <20110810160847.GA14706@jakstys.lt> <20110811062740.GA15380@jakstys.lt> <20110811154746.GA21345@jakstys.lt> <4E442F76.5050908@cs.ntua.gr> Message-ID: <20110812062701.GC22255@jakstys.lt> On Thu, Aug 11, 2011 at 10:37:26PM +0300, Kostis Sagonas wrote: > On 08/11/11 18:47, Motiejus Jak?tys wrote: > >Hello again, > > > >This is a follow-up email from > >PropEr: specify custom types in ?SIZED() fun > >http://erlang.org/pipermail/erlang-questions/2011-August/060592.html > > > >Sorry for spamming about this, but I think now I found the most minimal > >example of what I am trying to achieve. Code: > > Would you be so kind so as to wait till the mid of next week? > > It's vacation time for southern Europe and communication between > PropEr developers in order to have a proper answer and/or a fix for > this issue is a bit problematic till then. Of course I can wait, holiday is what I was suspecting from the beginning. :) It's not I am anxious; I just had more input to share. Thanks again Motiejus From sam@REDACTED Fri Aug 12 15:12:55 2011 From: sam@REDACTED (Sam Elliott) Date: Fri, 12 Aug 2011 14:12:55 +0100 Subject: [erlang-questions] Erlang Web Libraries & Frameworks In-Reply-To: References: <4E43A138.30300@dev-extend.eu> Message-ID: On Thu, Aug 11, 2011 at 11:16 AM, Joe Armstrong wrote: > On Thu, Aug 11, 2011 at 11:30 AM, Lo?c Hoguin wrote: >> On 08/11/2011 11:01 AM, Joe Armstrong wrote: >>> If all suppliers of web- frameworks could adopt a common adapter >>> pattern - it would make >>> life a lot easier. I realize that this would not allow us to use some >>> of the finer facilities >>> offered by an individual web-server, but it would be very useful for the simple >>> cases of serving up files etc. >> >> A lot of people choose the server based on those finer facilities. >> Adaptor projects are already used by everyone else. See below. >> >>> One could have a common API then choose the implementation depending upon >>> the requirements of the application. >> >> This is fine and all, but some servers use lists while others use >> binaries. The handler code is obviously different so some servers would >> require converting everything (headers, path, ...) to lists or binary >> before passing the data to your common handler. There's no clean >> solution there. > > Right, that's why an adaptor is useful. I suspect the differences > by accident rather than design. I would assume the "natural" way to > transfer large blobs would be as binaries, and to communicate parsed > attributes would be a Key-Value proplist of strings and so on. > > Having a well defined adaptor interface might have beneficial side effect > of causeing the sofwtare at either side of the interface to migrate > towards a set > of common types in the interface. > >> >> Some projects exist, like simple_bridge for example, but AFAIK they only >> do lists-based servers for now. > > I hadn't heard of simple bridge until you mentioned it - it looks > pretty much to be what I wanted. > > > @sam - can you add simple_bridge to your list (its at > https://github.com/etnt/SimpleBridge) I updated the article this morning. It turns out that Jesse had mentioned SimpleBridge above when talking about nitrogen. I'll take a longer look at SimpleBridge with my editor open. My feeling is that it doesn't quite wrap enough of the interfaces, so I might start a pull request. > I wonder how many other adaptor patterns there are: if simple_bridge > is the web adaptor > where that the adapters for key-val databases? I was also planning to write a rundown of Erlang JSON encoders - of which there aren't as many - perhaps an adapter would be a good idea here too? That said, their apis are easier as you're just calling modules, not starting processes. If you want me to do any more rundowns of other places where there are loads of competing libraries, just email me and I'll see what I can do. This one was originally written to scratch an itch and seems to actually be useful. Sam > (( aside -- it is a source of never ending frustration that the > interfaces to ets and dets are different > ? getting people to change ets or dets is very difficult since we > don't want to break old code > ? but an adapter would do the job )) > >> What's a good solution to this? > > I don't know - I suspect good interfaces emerge. It's often not > apparent at the start of a project what > the best API is. We can't abstract over several interfaces until we > have several interfaces. > Once the problem is well understood and we have several implemenations > we can begin to > generalize and abstract out what should have been the interface. Then > we have to refactor our programs. > (( This is the 99% hard-work part - software gets continually polished )) > > /Joe > > >> -- >> Lo?c Hoguin >> Dev:Extend >> > From max.lapshin@REDACTED Fri Aug 12 18:43:43 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 12 Aug 2011 20:43:43 +0400 Subject: [erlang-questions] Erlang driver for USB web camera Message-ID: Hi. I've opened on github two subprojects: http://github.com/erlyvideo/uvc ? this is driver for capturing web-camera under linux. http://github.com/erlyvideo/jpeg ? decodes jpeg to yuv. It uses very fast turbojpeg library to decode jpeg from camera in very fast manner. We are using these libraries to create stream video from web cameras to erlyvideo. From torben.lehoff@REDACTED Fri Aug 12 21:54:02 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 12 Aug 2011 21:54:02 +0200 Subject: [erlang-questions] Link to funny movie In-Reply-To: <02cf01cc5828$a0300720$e0901560$@com> References: <02cf01cc5828$a0300720$e0901560$@com> Message-ID: Alas, that is not the one - funny as it is it is not the one I was looking for. But thanks for the link anyway! Cheers, Torben On Thu, Aug 11, 2011 at 15:14, David Mercer wrote: > This? http://www.youtube.com/watch?v=xEJ1n13soWU**** > > ** ** > > ** ** > > *From:* erlang-questions-bounces@REDACTED [mailto: > erlang-questions-bounces@REDACTED] *On Behalf Of *Torben Hoffmann > *Sent:* Thursday, August 11, 2011 7:40 AM > *To:* erlang-questions > *Subject:* [erlang-questions] Link to funny movie**** > > ** ** > > Hi,**** > > ** ** > > A while back there was a link to some funny youtube video (cartoon) with > synthetic voices where a a non-Erlanger wanted to learn if Erlang could > solve all his problems since he had heard all these wonderful things about > Erlang; the Erlanger has to let him down on his crazy desires and the > non-Erlanger concludes that the Erlang community is weird/closed or > something like that.**** > > ** ** > > I have tried to find it but I cannot get it again.**** > > ** ** > > Any links out there?**** > > ** ** > > Thanks in advance,**** > > Torben**** > > -- > http://www.linkedin.com/in/torbenhoffmann**** > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Fri Aug 12 22:58:15 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 12 Aug 2011 22:58:15 +0200 Subject: [erlang-questions] Link to funny movie In-Reply-To: References: <02cf01cc5828$a0300720$e0901560$@com> Message-ID: BTW: the one I am looking for uses the same type of characters and voices. On Fri, Aug 12, 2011 at 21:54, Torben Hoffmann wrote: > Alas, that is not the one - funny as it is it is not the one I was looking > for. > > But thanks for the link anyway! > > Cheers, > Torben > > > On Thu, Aug 11, 2011 at 15:14, David Mercer wrote: > >> This? http://www.youtube.com/watch?v=xEJ1n13soWU**** >> >> ** ** >> >> ** ** >> >> *From:* erlang-questions-bounces@REDACTED [mailto: >> erlang-questions-bounces@REDACTED] *On Behalf Of *Torben Hoffmann >> *Sent:* Thursday, August 11, 2011 7:40 AM >> *To:* erlang-questions >> *Subject:* [erlang-questions] Link to funny movie**** >> >> ** ** >> >> Hi,**** >> >> ** ** >> >> A while back there was a link to some funny youtube video (cartoon) with >> synthetic voices where a a non-Erlanger wanted to learn if Erlang could >> solve all his problems since he had heard all these wonderful things about >> Erlang; the Erlanger has to let him down on his crazy desires and the >> non-Erlanger concludes that the Erlang community is weird/closed or >> something like that.**** >> >> ** ** >> >> I have tried to find it but I cannot get it again.**** >> >> ** ** >> >> Any links out there?**** >> >> ** ** >> >> Thanks in advance,**** >> >> Torben**** >> >> -- >> http://www.linkedin.com/in/torbenhoffmann**** >> > > > > -- > http://www.linkedin.com/in/torbenhoffmann > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From freeakk@REDACTED Sat Aug 13 11:32:52 2011 From: freeakk@REDACTED (Michael Uvarov) Date: Sat, 13 Aug 2011 13:32:52 +0400 Subject: [erlang-questions] Advanced math library Message-ID: Hi, Are there math libraries for Erlang besides the stdlib/math module? I search the realization of the statistics functions. -- Best regards, Uvarov Michael From igarai@REDACTED Sat Aug 13 11:58:02 2011 From: igarai@REDACTED (=?UTF-8?B?ScOxYWtpIEdhcmF5?=) Date: Sat, 13 Aug 2011 06:58:02 -0300 Subject: [erlang-questions] Advanced math library In-Reply-To: References: Message-ID: On Sat, Aug 13, 2011 at 06:32, Michael Uvarov wrote: > Hi, > > Are there math libraries for Erlang besides the stdlib/math module? I > search the realization of the statistics functions. Hello, you may find something of use at: http://scutil.com/ The author's (John Haugeland) home page has several interesting articles: http://fullof.bs/ good luck! > > -- > Best regards, > Uvarov Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- I?aki Garay. -----BEGIN GEEK CODE BLOCK----- Version: 3.1 GCS/GMU d- s:- a25 C++ UL+++ P--- E- W+ PS+++ PE- Y+ t++ 5+ tv- b++ DI+ D--- G e h! r y++ ------END GEEK CODE BLOCK------ From anders.nygren@REDACTED Sat Aug 13 15:47:26 2011 From: anders.nygren@REDACTED (Anders Nygren) Date: Sat, 13 Aug 2011 08:47:26 -0500 Subject: [erlang-questions] Advanced math library In-Reply-To: References: Message-ID: Hi Here is a small statistics module I have. /Anders On Sat, Aug 13, 2011 at 4:32 AM, Michael Uvarov wrote: > Hi, > > Are there math libraries for Erlang besides the stdlib/math module? I > search the realization of the statistics functions. > > -- > Best regards, > Uvarov Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- A non-text attachment was scrubbed... Name: statistics.erl Type: text/x-erlang Size: 9018 bytes Desc: not available URL: From jwatte@REDACTED Sat Aug 13 18:48:17 2011 From: jwatte@REDACTED (Jon Watte) Date: Sat, 13 Aug 2011 09:48:17 -0700 Subject: [erlang-questions] common test hiding errors? In-Reply-To: <54EA06CD-79B2-4B4A-BF26-DE48D2C5FE00@gmail.com> References: <9a786b50-d22b-4bce-893a-52e36263d1fa@localhost> <54EA06CD-79B2-4B4A-BF26-DE48D2C5FE00@gmail.com> Message-ID: Here's how I would view the situation: Erlang allows processes to crash -- "let it crash!" The question is: What behavior are you getting from the function under test? If it doesn't behave the way it's specified to, that should generate a test failure. Once you've detected that there is a test failure, you should be able to re-run the test manually (say, from command line) to debug it. Is your request really something like: "When a CommonTest fails, how can I do post-mortem debugging on the failing test case"? Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Wed, Aug 10, 2011 at 3:39 AM, Joel Reymont wrote: > > On Aug 10, 2011, at 9:09 AM, Lukas Larsson wrote: > > > Common Test detects any crashes which happen in the same process as the > testcase is run in. If a crash happens in another process, it will not be > detected unless you in your testcase check whether said process is alive or > is linked to it. Could this be what is going wrong? > > Yes, that's what's going on. I'm cannot link to all the processes in the > "system under testing", I think that's unrealistic. > > I do expect to be told when something goes wrong, though, get a printout in > the log, etc. > > This is in line with what EUnit does, for example. > > > If you're still having difficulties, could you post a minimal example of > what you are trying to do? It's difficult to guess what could be the problem > without an example to look at. > > It's another process that's crashes. > > > Also if you want to have the SASL and error_logger output in your common > test logs instead of redirecting them to a local file you might want to use > the cth_log_redirect hook found here > https://github.com/garazdawi/cth_tools. > > > I'm perfectly fine with redirecting error_logger output to a local file but > it seems that it shows up just fine in the CT logs. > > What's not showing up is error messages from other processes crashing, a > completely unintuitive behavior, I think. > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From luismarianoguerra@REDACTED Sun Aug 14 16:17:04 2011 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Sun, 14 Aug 2011 15:17:04 +0100 Subject: [erlang-questions] [semi-ot] introducing constants and include (that can include erlang code) in efene Message-ID: hi, I just wanted to show you two new inclusions on efene. from the name they may seem pretty simple but I think they have some interesting features on them I wanted to get some feedback before they are "official" features of efene. http://www.efenelang.org/2011/08/rfc-introducing-experimental-constants.html http://www.efenelang.org/2011/08/rfc-introducing-experimental-polyglot.html let me know what you think. From joelr1@REDACTED Sun Aug 14 17:29:52 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sun, 14 Aug 2011 16:29:52 +0100 Subject: [erlang-questions] common test hiding errors? In-Reply-To: References: <9a786b50-d22b-4bce-893a-52e36263d1fa@localhost> <54EA06CD-79B2-4B4A-BF26-DE48D2C5FE00@gmail.com> Message-ID: <8F2D4C33-333E-42AC-BCF0-39865EBE8A1D@gmail.com> On Aug 13, 2011, at 5:48 PM, Jon Watte wrote: > The question is: What behavior are you getting from the function under test? If it doesn't behave the way it's specified to, that should generate a test failure. I do get a test failure, although I'm not testing a function. I'm testing a system composed of various processes. > Once you've detected that there is a test failure, you should be able to re-run the test manually (say, from command line) to debug it. Will I get the error from the crashing process in the error log or in the terminal? It's a pointless exercise if I do not. > Is your request really something like: "When a CommonTest fails, how can I do post-mortem debugging on the failing test case"? Yes but I would be content with simply seeing the errors from crashing processes in the Common Test raw log file. I get the output from my invocation of error_log:error_report, etc. but I do not see errors from crashing processes. badarg? undef? Nowhere to be found and must be guessed. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From connorsml@REDACTED Sun Aug 14 19:01:43 2011 From: connorsml@REDACTED (Michael Connors) Date: Sun, 14 Aug 2011 19:01:43 +0200 Subject: [erlang-questions] status of mqtt4erl Message-ID: Hi, I have been having a look at the library mqtt4erl http://code.google.com/p/mqtt4erl/ which has not had a commit for the last couple of years. I am working on modifying it to handle the 3.1 version of the MQTT (so far I have added usernames and passwords, 3 more changes to go), however I did not get any response from the author on the email address listed in the README. Before I fork it, I just want to check here to see if the author is listening and would prefer to integrate the changes into his project. Thanks, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Sun Aug 14 22:01:12 2011 From: jwatte@REDACTED (Jon Watte) Date: Sun, 14 Aug 2011 13:01:12 -0700 Subject: [erlang-questions] common test hiding errors? In-Reply-To: <8F2D4C33-333E-42AC-BCF0-39865EBE8A1D@gmail.com> References: <9a786b50-d22b-4bce-893a-52e36263d1fa@localhost> <54EA06CD-79B2-4B4A-BF26-DE48D2C5FE00@gmail.com> <8F2D4C33-333E-42AC-BCF0-39865EBE8A1D@gmail.com> Message-ID: Will I get the error from the crashing process in the error log or in the terminal? If you run the code under test, separate from the common test framework, I would expect that you get the same output you'd get in production. That may or may not be good enough for you... This is what I like about eunit, btw: you can call the unit test function directly and it behaves just like in production. We ended up using eunit even for integration tests, with some small wrapper functions to make sure the necssary supervisors/applications are started/stopped. I imagine changing test frameworks may be expensive for you, though. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Sun, Aug 14, 2011 at 8:29 AM, Joel Reymont wrote: > On Aug 13, 2011, at 5:48 PM, Jon Watte wrote: > > > The question is: What behavior are you getting from the function under > test? If it doesn't behave the way it's specified to, that should generate a > test failure. > > > I do get a test failure, although I'm not testing a function. > > I'm testing a system composed of various processes. > > > Once you've detected that there is a test failure, you should be able to > re-run the test manually (say, from command line) to debug it. > > > Will I get the error from the crashing process in the error log or in the > terminal? > > It's a pointless exercise if I do not. > > > Is your request really something like: "When a CommonTest fails, how can > I do post-mortem debugging on the failing test case"? > > > Yes but I would be content with simply seeing the errors from crashing > processes in the Common Test raw log file. > > I get the output from my invocation of error_log:error_report, etc. but I > do not see errors from crashing processes. > > badarg? undef? Nowhere to be found and must be guessed. > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.melin@REDACTED Sun Aug 14 22:37:05 2011 From: per.melin@REDACTED (Per Melin) Date: Sun, 14 Aug 2011 22:37:05 +0200 Subject: [erlang-questions] common test hiding errors? In-Reply-To: References: <9a786b50-d22b-4bce-893a-52e36263d1fa@localhost> <54EA06CD-79B2-4B4A-BF26-DE48D2C5FE00@gmail.com> <8F2D4C33-333E-42AC-BCF0-39865EBE8A1D@gmail.com> Message-ID: On 14 aug 2011, at 22:01, Jon Watte wrote: > This is what I like about eunit, btw: you can call the unit test function directly and it behaves just like in production. We ended up using eunit even for integration tests, with some small wrapper functions to make sure the necssary supervisors/applications are started/stopped. I imagine changing test frameworks may be expensive for you, though. I guess that is my cue to mention a little library I've written that lets you run many (but not all) Common Test suites in EUnit instead, without changing them (more or less) at all. The minimum change required is to add a line to include a library header file. https://github.com/permelin/common_eunit My reason for creating this was that we use EUnit for integration tests, just like you, but at the same time I wanted the option to write some tests in the format of Common Test. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela@REDACTED Mon Aug 15 10:49:20 2011 From: ingela@REDACTED (Ingela Andin) Date: Mon, 15 Aug 2011 10:49:20 +0200 Subject: [erlang-questions] New_ssl module and CRL support In-Reply-To: <8097A4E725D78E41A8B862ACE60E963945EF82D50F@GVW1355EXB.americas.hpqcorp.net> References: <1312209624.25314.53.camel@vicbristol6.hpl.hp.com> <8097A4E725D78E41A8B862ACE60E963945EF692CEC@GVW1355EXB.americas.hpqcorp.net> <8097A4E725D78E41A8B862ACE60E963945EF82D50F@GVW1355EXB.americas.hpqcorp.net> Message-ID: Hi! I have implemented a module for CRL-checking in the public_key-application. I have not implemented the use of it in ssl yet, but it is the plan to include it for R15. We have a slight delay due the fact that the conformance tests certificates expired and the new test certificates was released on an other format, so we need new ASN-1 support to get the test suite up and running again. If you would be interested in beta testing the CRL-functionality please send me a mail privately. Regards Ingela Erlang/OTP team - Ericsson AB 2011/8/6 Rosa, Andrea : > Hi > > Thank you for your kindly reply, I wait the end of August to see if your colleagues have further information. > Kind regards > -- > Andrea > ?Sent: 05 August 2011 23:29 >> To: Rosa, Andrea >> Cc: erlang-questions@REDACTED >> Subject: Re: [erlang-questions] New_ssl module and CRL support >> >> It is vacation time in Sweden , you can expect better answers on this >> in a couple of weeks when most people are back. >> >> I would say that it is in our plans to support CRL but I can't promise >> exactly when it will be there. There are workarounds >> as you may read in another response. >> >> /Kenneth, Erlang/OTP Ericsson. >> >> On Wed, Aug 3, 2011 at 12:37 PM, Rosa, Andrea >> wrote: >> > Hi >> > Any news about that? Someone is so kind to give information about >> future plan for supporting CRL? >> > Someone is facing with the same issue and can give me information >> about some workaround? >> > Best regards >> > -- >> > Andrea >> > >> > -----Original Message----- >> > >> > Sent: 01 August 2011 15:40 >> > To: erlang-questions@REDACTED >> > Subject: [erlang-questions] New_ssl module and CRL support >> > >> > Hi >> > >> > I am working with a software (rabbitmq) which use the new_ssl module >> to >> > implement SSL connections, everything goes well with the exception of >> > managing revoked certification during the handshake, in this case the >> > server accepts a revoked certificate. >> > Is there a way for fixing that? >> > In your plans is there the implementation of a new module for >> supporting >> > CRL? >> > >> > Cheers >> > -- >> > Andrea >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From watson.timothy@REDACTED Tue Aug 16 04:58:03 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 16 Aug 2011 03:58:03 +0100 Subject: [erlang-questions] common test hiding errors? In-Reply-To: <8F2D4C33-333E-42AC-BCF0-39865EBE8A1D@gmail.com> References: <9a786b50-d22b-4bce-893a-52e36263d1fa@localhost> <54EA06CD-79B2-4B4A-BF26-DE48D2C5FE00@gmail.com> <8F2D4C33-333E-42AC-BCF0-39865EBE8A1D@gmail.com> Message-ID: On 14 August 2011 16:29, Joel Reymont wrote: > I'm testing a system composed of various processes. > >> Once you've detected that there is a test failure, you should be able to re-run the test manually (say, from command line) to debug it. > > Will I get the error from the crashing process in the error log or in the terminal? > > It's a pointless exercise if I do not. > Hi Joel. Usually I try to find a way of making failures detectable, so that my tests can do proper assertions about them. Sometimes I'll do this by noting a pid once a process starts up, and then after some prodding and poking around, I'll assert that it's still alive by sneakily asking its supervisor or looking for it with whereis or gproc or whatever. If I can't make sense of other peoples code and need to test it I might turn on tracing (in the init_per_testcase/2 function) and inspect that. I'm not really sure why you're not seeing crashes in the test log, but I think the offending code is in either test_server_ctl or the handler(s) it installs, but again memory fails me as to how this can be fudged back into working order. Sorry that's not much to go on - it's kinda late here. :) From ok@REDACTED Tue Aug 16 07:58:06 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 16 Aug 2011 17:58:06 +1200 Subject: [erlang-questions] [semi-ot] introducing constants and include (that can include erlang code) in efene In-Reply-To: References: Message-ID: <1FB1B4D4-1323-4D31-831B-65DEA8737189@cs.otago.ac.nz> On 15/08/2011, at 2:17 AM, Mariano Guerra wrote: > hi, I just wanted to show you two new inclusions on efene. The idea of defining a "constant" using an expression that involves variables that are not in scope at the point of definition and then having them bound at the point of use is, well, that kind of horrible mess is why Scheme has "hygienic" macros instead of classic Lisp ones. Look at "once" features in Eiffel for a nice model (it's just lazy evaluation, really). From luismarianoguerra@REDACTED Tue Aug 16 11:26:04 2011 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Tue, 16 Aug 2011 10:26:04 +0100 Subject: [erlang-questions] [semi-ot] introducing constants and include (that can include erlang code) in efene In-Reply-To: <1FB1B4D4-1323-4D31-831B-65DEA8737189@cs.otago.ac.nz> References: <1FB1B4D4-1323-4D31-831B-65DEA8737189@cs.otago.ac.nz> Message-ID: On Tue, Aug 16, 2011 at 6:58 AM, Richard O'Keefe wrote: > > On 15/08/2011, at 2:17 AM, Mariano Guerra wrote: > >> hi, I just wanted to show you two new inclusions on efene. > > The idea of defining a "constant" using an expression that involves > variables that are not in scope at the point of definition and then > having them bound at the point of use is, well, that kind of horrible > mess is why Scheme has "hygienic" macros instead of classic Lisp ones. yes, but I don't know a way to make hygienic macros in efene without doing some hacky thing, if someone want's to help me I would be glad to try :) > Look at "once" features in Eiffel for a nice model (it's just lazy > evaluation, really). I don't know a way to mutate the body of a function at runtime, what can be done right now in efene is that if the expression can be calculated at compile time, you can execute it then and just put the result in the bytecode: [mariano@REDACTED ~]$ cat demo.ifn biglist = fn () $[1000 .. 1010] @public run = fn () io.format("list ~p~n", [biglist()]) [mariano@REDACTED ~]$ fnc -t erl demo.ifn -module(demo). -export([run/0]). biglist() -> [1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008, 1009, 1010]. run() -> io:format("list ~p~n", [biglist()]). [mariano@REDACTED ~]$ fnc demo.ifn Compiling demo.ifn [mariano@REDACTED ~]$ fnc -r demo run list [1000,1001,1002,1003,1004,1005,1006,1007,1008,1009,1010] of course this is just a small set of what "once" can do, but to replace constants in some cases it's enough From jameschurchman@REDACTED Tue Aug 16 13:23:25 2011 From: jameschurchman@REDACTED (James Churchman) Date: Tue, 16 Aug 2011 12:23:25 +0100 Subject: [erlang-questions] ++ to select the head ( instead of just the tail ) Message-ID: <4191CF03-18C6-451E-8444-2271DC4615D4@gmail.com> The ++ operator is not just a righthand side operator, but can in fact be a lefthand side operator in as far as I can tell just 1 case : When you know the head you wish to remove and getting the tail : Eg in the shell: FullList = "123456". "123" ++ Tail = FullList. Tail wil now be "456" However using it to get the head is not supported : FullList = "123456". Head ++ "456" = FullList. gives : * 1: illegal pattern Understandably this second type involves traversing the entire list, where the first does not, but it would still be a nice addition to erlang. Was this not added as the first one expands out to simply [$1,$2,$3 | Tail] etc... but the second needs to expand to something more complex like an actual function the way list comprehensions do? James From jameschurchman@REDACTED Tue Aug 16 13:27:30 2011 From: jameschurchman@REDACTED (James Churchman) Date: Tue, 16 Aug 2011 12:27:30 +0100 Subject: [erlang-questions] switching erlang versions Message-ID: <26EB35B6-A1EA-4BD9-A856-ED3EF73054A3@gmail.com> Does a tool like this : https://github.com/sstephenson/rbenv/ Exist for erlang? James From gkmngrgn@REDACTED Tue Aug 16 13:30:11 2011 From: gkmngrgn@REDACTED (=?ISO-8859-1?Q?G=F6kmen_G=F6rgen?=) Date: Tue, 16 Aug 2011 14:30:11 +0300 Subject: [erlang-questions] switching erlang versions In-Reply-To: <26EB35B6-A1EA-4BD9-A856-ED3EF73054A3@gmail.com> References: <26EB35B6-A1EA-4BD9-A856-ED3EF73054A3@gmail.com> Message-ID: On 16 August 2011 14:27, James Churchman wrote: > Does a tool like this : > > https://github.com/sstephenson/rbenv/ > > Exist for erlang? maybe it's for you: https://github.com/evax/kerl -- gkmngrgn ~ http://gokmengorgen.net From jameschurchman@REDACTED Tue Aug 16 13:52:19 2011 From: jameschurchman@REDACTED (James Churchman) Date: Tue, 16 Aug 2011 12:52:19 +0100 Subject: [erlang-questions] switching erlang versions In-Reply-To: References: <26EB35B6-A1EA-4BD9-A856-ED3EF73054A3@gmail.com> Message-ID: <5FDC55E7-D029-4684-B074-80578FDB4776@gmail.com> that looks exactly what i was after! the list far outperforms my google-ing skills :-) On 16 Aug 2011, at 12:30, G?kmen G?rgen wrote: > On 16 August 2011 14:27, James Churchman wrote: >> Does a tool like this : >> >> https://github.com/sstephenson/rbenv/ >> >> Exist for erlang? > > maybe it's for you: https://github.com/evax/kerl > > -- > gkmngrgn ~ http://gokmengorgen.net From ngreco@REDACTED Tue Aug 16 14:43:20 2011 From: ngreco@REDACTED (Nahuel Greco) Date: Tue, 16 Aug 2011 09:43:20 -0300 Subject: [erlang-questions] [semi-ot] introducing constants and include (that can include erlang code) in efene In-Reply-To: References: <1FB1B4D4-1323-4D31-831B-65DEA8737189@cs.otago.ac.nz> Message-ID: On Tue, Aug 16, 2011 at 6:26 AM, Mariano Guerra wrote: > > I don't know a way to mutate the body of a function at runtime, what > can be done right now in efene is that if the expression can be > calculated at compile time, you can execute it then and just put the > result in the bytecode: Maybe smerl can be useful to you: http://code.google.com/p/erlyweb/source/browse/trunk/src/smerl/smerl.erl Saludos, Nahuel Greco. From joelr1@REDACTED Tue Aug 16 16:20:55 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 16 Aug 2011 15:20:55 +0100 Subject: [erlang-questions] omaha poker hand permutations Message-ID: <1887F572-AE36-4D5A-82D8-57F095789540@gmail.com> Suppose I have a list of 5 common cards (Board) and another list of 4 private cards (#hand.cards). I need to figure out all the possible poker hand rankings, but can only use 2 private cards and 3 common cards. What is the best way to compile a list of unique permutations here? Following is a dumb approach that gives me 721 non-unique permutations but is there a better way? Thanks, Joel --- omaha_collect_hands(_, [], Acc) -> Acc; omaha_collect_hands(Board, [H|T], Acc) -> Hands = [ H#hand{ cards = [A, B, C, D, E] } || A <- H#hand.cards, B <- H#hand.cards, A /= B, C <- Board, D <- Board, E <- Board, C /= D, C /= E, D /= E ], omaha_collect_hands(Board, T, prepend(Hands, Acc)). prepend([], Target) when is_list(Target) -> Target; prepend([H|Source], Target) when is_list(Source), is_list(Target) -> prepend(Source, [H|Target]). -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Tue Aug 16 16:27:43 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 16 Aug 2011 15:27:43 +0100 Subject: [erlang-questions] omaha poker hand permutations In-Reply-To: <1887F572-AE36-4D5A-82D8-57F095789540@gmail.com> References: <1887F572-AE36-4D5A-82D8-57F095789540@gmail.com> Message-ID: <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com> The problem can be reduced to the following: 5> L3 = ["AD", "AH", "AS", "QS"]. ["AD","AH","AS","QS"] 6> L4 = [ {A, B} || A <- L3, B <- L3, A /= B]. [{"AD","AH"}, {"AD","AS"}, {"AD","QS"}, {"AH","AD"}, {"AH","AS"}, {"AH","QS"}, {"AS","AD"}, {"AS","AH"}, {"AS","QS"}, {"QS","AD"}, {"QS","AH"}, {"QS","AS"}] Note the duplicate pairs of AD, AH and AH, AD. Can these be eliminated inside the list comprehension itself? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From demeshchuk@REDACTED Tue Aug 16 16:35:03 2011 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Tue, 16 Aug 2011 18:35:03 +0400 Subject: [erlang-questions] omaha poker hand permutations In-Reply-To: <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com> References: <1887F572-AE36-4D5A-82D8-57F095789540@gmail.com> <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com> Message-ID: Lis comprehensions are a combination of map and filter. But I think you need lists:foldl/3 here, or just a custom reducing function. So the answer is no. On Tue, Aug 16, 2011 at 6:27 PM, Joel Reymont wrote: > The problem can be reduced to the following: > > 5> L3 = ["AD", "AH", "AS", "QS"]. > ["AD","AH","AS","QS"] > > 6> L4 = [ {A, B} || A <- L3, B <- L3, A /= B]. > [{"AD","AH"}, > ?{"AD","AS"}, > ?{"AD","QS"}, > ?{"AH","AD"}, > ?{"AH","AS"}, > ?{"AH","QS"}, > ?{"AS","AD"}, > ?{"AS","AH"}, > ?{"AS","QS"}, > ?{"QS","AD"}, > ?{"QS","AH"}, > ?{"QS","AS"}] > > Note the duplicate pairs of AD, AH and AH, AD. > > Can these be eliminated inside the list comprehension itself? > > ? ? ? ?Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best regards, Dmitry Demeshchuk From maggio.vincenzo@REDACTED Tue Aug 16 16:36:52 2011 From: maggio.vincenzo@REDACTED (Vincenzo Maggio) Date: Tue, 16 Aug 2011 16:36:52 +0200 Subject: [erlang-questions] omaha poker hand permutations In-Reply-To: <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com> References: <1887F572-AE36-4D5A-82D8-57F095789540@gmail.com>, <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com> Message-ID: I don't think is possible because the filter you are willing to apply is valid only after you have all the occurences in the resulting list. Vincenzo > From: joelr1@REDACTED > Date: Tue, 16 Aug 2011 15:27:43 +0100 > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] omaha poker hand permutations > > The problem can be reduced to the following: > > 5> L3 = ["AD", "AH", "AS", "QS"]. > ["AD","AH","AS","QS"] > > 6> L4 = [ {A, B} || A <- L3, B <- L3, A /= B]. > [{"AD","AH"}, > {"AD","AS"}, > {"AD","QS"}, > {"AH","AD"}, > {"AH","AS"}, > {"AH","QS"}, > {"AS","AD"}, > {"AS","AH"}, > {"AS","QS"}, > {"QS","AD"}, > {"QS","AH"}, > {"QS","AS"}] > > Note the duplicate pairs of AD, AH and AH, AD. > > Can these be eliminated inside the list comprehension itself? > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc@REDACTED Tue Aug 16 16:38:43 2011 From: marc@REDACTED (Marc Worrell) Date: Tue, 16 Aug 2011 16:38:43 +0200 Subject: [erlang-questions] omaha poker hand permutations In-Reply-To: References: <1887F572-AE36-4D5A-82D8-57F095789540@gmail.com> <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com> Message-ID: <0FD127BB-E1B0-48F8-A534-AFD3843E8399@worrell.nl> Can't you just do: L4 = [ {A, B} || A <- L3, B <- L3, A < B]. This adds a strict ordering on the returned pairs. - Marc On 16 aug 2011, at 16:35, Dmitry Demeshchuk wrote: > Lis comprehensions are a combination of map and filter. But I think > you need lists:foldl/3 here, or just a custom reducing function. So > the answer is no. > > On Tue, Aug 16, 2011 at 6:27 PM, Joel Reymont wrote: >> The problem can be reduced to the following: >> >> 5> L3 = ["AD", "AH", "AS", "QS"]. >> ["AD","AH","AS","QS"] >> >> 6> L4 = [ {A, B} || A <- L3, B <- L3, A /= B]. >> [{"AD","AH"}, >> {"AD","AS"}, >> {"AD","QS"}, >> {"AH","AD"}, >> {"AH","AS"}, >> {"AH","QS"}, >> {"AS","AD"}, >> {"AS","AH"}, >> {"AS","QS"}, >> {"QS","AD"}, >> {"QS","AH"}, >> {"QS","AS"}] >> >> Note the duplicate pairs of AD, AH and AH, AD. >> >> Can these be eliminated inside the list comprehension itself? >> >> Thanks, Joel >> >> -------------------------------------------------------------------------- >> - for hire: mac osx device driver ninja, kernel extensions and usb drivers >> ---------------------+------------+--------------------------------------- >> http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont >> ---------------------+------------+--------------------------------------- >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Best regards, > Dmitry Demeshchuk > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From maggio.vincenzo@REDACTED Tue Aug 16 16:43:32 2011 From: maggio.vincenzo@REDACTED (Vincenzo Maggio) Date: Tue, 16 Aug 2011 16:43:32 +0200 Subject: [erlang-questions] omaha poker hand permutations In-Reply-To: <0FD127BB-E1B0-48F8-A534-AFD3843E8399@worrell.nl> References: <1887F572-AE36-4D5A-82D8-57F095789540@gmail.com>, <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com>, , <0FD127BB-E1B0-48F8-A534-AFD3843E8399@worrell.nl> Message-ID: It's right for this case, but IMHO difficult to read through and mantain. > From: marc@REDACTED > Date: Tue, 16 Aug 2011 16:38:43 +0200 > To: demeshchuk@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] omaha poker hand permutations > > Can't you just do: > > L4 = [ {A, B} || A <- L3, B <- L3, A < B]. > > This adds a strict ordering on the returned pairs. > > - Marc > > > > On 16 aug 2011, at 16:35, Dmitry Demeshchuk wrote: > > > Lis comprehensions are a combination of map and filter. But I think > > you need lists:foldl/3 here, or just a custom reducing function. So > > the answer is no. > > > > On Tue, Aug 16, 2011 at 6:27 PM, Joel Reymont wrote: > >> The problem can be reduced to the following: > >> > >> 5> L3 = ["AD", "AH", "AS", "QS"]. > >> ["AD","AH","AS","QS"] > >> > >> 6> L4 = [ {A, B} || A <- L3, B <- L3, A /= B]. > >> [{"AD","AH"}, > >> {"AD","AS"}, > >> {"AD","QS"}, > >> {"AH","AD"}, > >> {"AH","AS"}, > >> {"AH","QS"}, > >> {"AS","AD"}, > >> {"AS","AH"}, > >> {"AS","QS"}, > >> {"QS","AD"}, > >> {"QS","AH"}, > >> {"QS","AS"}] > >> > >> Note the duplicate pairs of AD, AH and AH, AD. > >> > >> Can these be eliminated inside the list comprehension itself? > >> > >> Thanks, Joel > >> > >> -------------------------------------------------------------------------- > >> - for hire: mac osx device driver ninja, kernel extensions and usb drivers > >> ---------------------+------------+--------------------------------------- > >> http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > >> ---------------------+------------+--------------------------------------- > >> > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > > > > > -- > > Best regards, > > Dmitry Demeshchuk > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Tue Aug 16 17:07:09 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 16 Aug 2011 17:07:09 +0200 Subject: [erlang-questions] omaha poker hand permutations In-Reply-To: <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com> References: <1887F572-AE36-4D5A-82D8-57F095789540@gmail.com> <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com> Message-ID: <20110816150709.GA29932@erix.ericsson.se> On Tue, Aug 16, 2011 at 03:27:43PM +0100, Joel Reymont wrote: > The problem can be reduced to the following: > > 5> L3 = ["AD", "AH", "AS", "QS"]. > ["AD","AH","AS","QS"] > > 6> L4 = [ {A, B} || A <- L3, B <- L3, A /= B]. > [{"AD","AH"}, > {"AD","AS"}, > {"AD","QS"}, > {"AH","AD"}, > {"AH","AS"}, > {"AH","QS"}, > {"AS","AD"}, > {"AS","AH"}, > {"AS","QS"}, > {"QS","AD"}, > {"QS","AH"}, > {"QS","AS"}] > > Note the duplicate pairs of AD, AH and AH, AD. > > Can these be eliminated inside the list comprehension itself? I guess no, since list comprehension permutation assumes order is significant so the pairs {"AD","AH"} and {"AH","AD"} are both generated. Comparisions can only be done between generated term components. In this case you better write it with functions, the outer chewing off one combining with the rest. Remember that list comprehensions are just high level list function notation. For example: perm([A|[_|_]=As]) -> perm(A, lists:reverse(As), perm(As)); perm([_|_]) -> []; perm([]) -> []. perm(A, [B|Bs], Acc) -> perm(A, Bs, [{A,B}|Acc]); perm(_, [], Acc) -> Acc. X> mod:perm(["AD", "AH", "AS", "QS"]). [{"AD","QS"}, {"AD","AS"}, {"AD","AH"}, {"AH","QS"}, {"AH","AS"}, {"AS","QS"}] > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From erlang@REDACTED Tue Aug 16 17:09:03 2011 From: erlang@REDACTED (erlang) Date: Tue, 16 Aug 2011 09:09:03 -0600 Subject: [erlang-questions] Process surviving disconnect Message-ID: I am completely new to erlang and diving into this brave new world. I got the following setup: - 2 Servers with fqdn usa.local and gca.local - 1 erlang node on each of them named alice@REDACTED and bob@REDACTED When I start Alice (`alice:start/0`) on alice@REDACTED it spawns linked Bob (`bob:start/1`) on gca.local. Both processing are trapping exits. When Alice dies of something, Bob gets notified and keeps on running. When Bob dies of something, Alice gets notified and keeps on running. When I cut the network connection, Alice gets notified that Bob has died of `noconnection` and process bob dies on bob@REDACTED I do not want this to happen. I want Bob to keep on running although it looses connection to Alice and at the same I want both threads to be notified if their partner dies (of normal, of badarith, of whatever). **My questions are:** - Has this something to do that I initially spawn Bob from the Alice node? - How can I make Bob to survive a connection loss? Thank you in advance Anchise ---------- Here goes the code: -module (alice). -compile (export_all). start () -> register (alice, spawn (fun init/0) ). stop () -> whereis (alice) ! stop. init () -> process_flag (trap_exit, true), Bob = spawn_link ('bob@REDACTED', bob, start, [self () ] ), loop (Bob). loop (Bob) -> receive stop -> ok; {'EXIT', Bob, Reason} -> io:format ("Bob died of ~p.~n", [Reason] ), loop (Bob); Msg -> io:format ("Alice received ~p.~n", [Msg] ), loop (Bob) end. ---------- -module (bob). -compile (export_all). start (Alice) -> process_flag (trap_exit, true), register (bob, self () ), loop (Alice). loop (Alice) -> receive stop -> ok; {'EXIT', Alice, Reason} -> io:format ("Alice died of ~p.~n", [Reason] ), loop (Alice); Msg -> io:format ("Bob received ~p.~n", [Msg] ), loop (Alice) after 5000 -> Alice ! "Hi, this Bob", loop (Alice) end. From joelr1@REDACTED Tue Aug 16 18:26:21 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 16 Aug 2011 17:26:21 +0100 Subject: [erlang-questions] omaha poker hand permutations In-Reply-To: <0FD127BB-E1B0-48F8-A534-AFD3843E8399@worrell.nl> References: <1887F572-AE36-4D5A-82D8-57F095789540@gmail.com> <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com> <0FD127BB-E1B0-48F8-A534-AFD3843E8399@worrell.nl> Message-ID: That works for me. Thank you folks! On Aug 16, 2011, at 3:38 PM, Marc Worrell wrote: > Can't you just do: > > L4 = [ {A, B} || A <- L3, B <- L3, A < B]. > > This adds a strict ordering on the returned pairs. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From torben.lehoff@REDACTED Tue Aug 16 20:32:23 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Tue, 16 Aug 2011 20:32:23 +0200 Subject: [erlang-questions] omaha poker hand permutations In-Reply-To: References: <1887F572-AE36-4D5A-82D8-57F095789540@gmail.com> <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com> <0FD127BB-E1B0-48F8-A534-AFD3843E8399@worrell.nl> Message-ID: Hi Joel, I wrote this one on the way home on the train: 1> Board = [a,b,c,d,e]. [a,b,c,d,e] 2> Hand = [1,2,3,4]. [1,2,3,4] 3> HandPairs = lists:usort( [lists:sort([A,B]) || A <- Hand, B <- Hand -- [A] ] ). [[1,2],[1,3],[1,4],[2,3],[2,4],[3,4]] 4> BoardSets = lists:usort( [lists:sort([A,B,C]) || A <- Board, B <- (Board -- [A]), C <- (Board -- [A,B]) ]). [[a,b,c], [a,b,d], [a,b,e], [a,c,d], [a,c,e], [a,d,e], [b,c,d], [b,c,e], [b,d,e], [c,d,e]] 5> Hands = [ HP ++ BS || HP <- HandPairs, BS <- BoardSets ]. [[1,2,a,b,c], [1,2,a,b,d], [1,2,a,b,e], [1,2,a,c,d], [1,2,a,c,e], [1,2,a,d,e], [1,2,b,c,d], [1,2,b,c,e], [1,2,b,d,e], ..... It might not be the most efficient solution, but if you put it aside in a function you can optimize it the day that it becomes the biggest bottleneck in your system. Cheers, Torben On Tue, Aug 16, 2011 at 18:26, Joel Reymont wrote: > That works for me. > > Thank you folks! > > On Aug 16, 2011, at 3:38 PM, Marc Worrell wrote: > > > Can't you just do: > > > > L4 = [ {A, B} || A <- L3, B <- L3, A < B]. > > > > This adds a strict ordering on the returned pairs. > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From nx@REDACTED Tue Aug 16 23:03:48 2011 From: nx@REDACTED (nx) Date: Tue, 16 Aug 2011 17:03:48 -0400 Subject: [erlang-questions] Link to funny movie In-Reply-To: References: <02cf01cc5828$a0300720$e0901560$@com> Message-ID: More of gar1t's xtranormal videos are here: http://www.xtranormal.com/profile/3201665/?listtype=SERIES&listid=14273053 There's also: http://www.xtranormal.com/watch/6995033/mongo-db-is-web-scale On Fri, Aug 12, 2011 at 4:58 PM, Torben Hoffmann wrote: > BTW: the one I am looking for uses the same type of characters and voices. > > On Fri, Aug 12, 2011 at 21:54, Torben Hoffmann > wrote: >> >> Alas, that is not the one - funny as it is it is not the one I was looking >> for. >> But thanks for the link anyway! >> Cheers, >> Torben >> >> On Thu, Aug 11, 2011 at 15:14, David Mercer wrote: >>> >>> This?? http://www.youtube.com/watch?v=xEJ1n13soWU >>> >>> >>> >>> >>> >>> From: erlang-questions-bounces@REDACTED >>> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Torben Hoffmann >>> Sent: Thursday, August 11, 2011 7:40 AM >>> To: erlang-questions >>> Subject: [erlang-questions] Link to funny movie >>> >>> >>> >>> Hi, >>> >>> >>> >>> A while back there was a link to some funny youtube video (cartoon) with >>> synthetic voices where a a non-Erlanger wanted to learn if Erlang could >>> solve all his problems since he had heard all these wonderful things about >>> Erlang; the Erlanger has to let him down on his crazy desires and the >>> non-Erlanger concludes that the Erlang community is weird/closed or >>> something like that. >>> >>> >>> >>> I have tried to find it but I cannot get it again. >>> >>> >>> >>> Any links out there? >>> >>> >>> >>> Thanks in advance, >>> >>> Torben >>> >>> -- >>> http://www.linkedin.com/in/torbenhoffmann >> >> >> -- >> http://www.linkedin.com/in/torbenhoffmann > > > > -- > http://www.linkedin.com/in/torbenhoffmann > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From oribrost@REDACTED Tue Aug 16 23:40:45 2011 From: oribrost@REDACTED (ori brost) Date: Wed, 17 Aug 2011 00:40:45 +0300 Subject: [erlang-questions] Handling non http in mochiweb Message-ID: I am using mochiweb for a server that may also get a TCP connction to which the client sends a simple string (without a newline, the string is not http). Mochiweb uses HTTP sockets and therefore fails to detect this (i dont even get http_error that i can easily get in mochiweb). How can I solve this? Ideally I wish to change mochiweb code to do setopt({packet, http_or_raw}) but this kind of thing does not exist. How would you recommend handling this? my current idea was to modify mochiweb and use erlang:decode_packet, is there a better approach? From ryanhuffman@REDACTED Tue Aug 16 23:45:03 2011 From: ryanhuffman@REDACTED (Ryan Huffman) Date: Tue, 16 Aug 2011 14:45:03 -0700 Subject: [erlang-questions] Handling non http in mochiweb In-Reply-To: References: Message-ID: Does it have to be the same port? You could use gen_tcp to listen to connections on a different port and handle the traffic however you would like. On Tue, Aug 16, 2011 at 2:40 PM, ori brost wrote: > I am using mochiweb for a server that may also get a TCP connction to > which the client sends a simple string (without a newline, the string > is not http). Mochiweb uses HTTP sockets and therefore fails to detect > this (i dont even get http_error that i can easily get in mochiweb). > How can I solve this? Ideally I wish to change mochiweb code to do > setopt({packet, http_or_raw}) but this kind of thing does not exist. > How would you recommend handling this? my current idea was to modify > mochiweb and use erlang:decode_packet, is there a better approach? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Wed Aug 17 00:00:11 2011 From: bob@REDACTED (Bob Ippolito) Date: Tue, 16 Aug 2011 15:00:11 -0700 Subject: [erlang-questions] Handling non http in mochiweb In-Reply-To: References: Message-ID: mochiweb isn't designed to handle this use case, if it doesn't look like HTTP then the connection is closed and it gets discarded. You would have to go around mochiweb_http for this purpose. I'd suggest using an alternate port, or making it look like HTTP. If I really wanted to do what you say you want to do, I would copy mochiweb_http.erl to some other name (e.g. sometimes_not_http.erl) and make the appropriate changes to loop/2 and request/2? then instead of adding mochiweb_http to your supervisor you'd add sometimes_not_http. It is not necessary or recommended to make modifications to mochiweb in-place. On Tue, Aug 16, 2011 at 2:40 PM, ori brost wrote: > I am using mochiweb for a server that may also get a TCP connction to > which the client sends a simple string (without a newline, the string > is not http). Mochiweb uses HTTP sockets and therefore fails to detect > this (i dont even get http_error that i can easily get in mochiweb). > How can I solve this? Ideally I wish to change mochiweb code to do > setopt({packet, http_or_raw}) but this kind of thing does not exist. > How would you recommend handling this? my current idea was to modify > mochiweb and use erlang:decode_packet, is there a better approach? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jameschurchman@REDACTED Wed Aug 17 01:55:37 2011 From: jameschurchman@REDACTED (James Churchman) Date: Wed, 17 Aug 2011 00:55:37 +0100 Subject: [erlang-questions] Link to funny movie In-Reply-To: References: <02cf01cc5828$a0300720$e0901560$@com> Message-ID: i think you mean the ( official / unofficial ? ) spwanfest video http://vimeo.com/22360540 :-) On 16 Aug 2011, at 22:03, nx wrote: > More of gar1t's xtranormal videos are here: > http://www.xtranormal.com/profile/3201665/?listtype=SERIES&listid=14273053 > > There's also: http://www.xtranormal.com/watch/6995033/mongo-db-is-web-scale > > On Fri, Aug 12, 2011 at 4:58 PM, Torben Hoffmann > wrote: >> BTW: the one I am looking for uses the same type of characters and voices. >> >> On Fri, Aug 12, 2011 at 21:54, Torben Hoffmann >> wrote: >>> >>> Alas, that is not the one - funny as it is it is not the one I was looking >>> for. >>> But thanks for the link anyway! >>> Cheers, >>> Torben >>> >>> On Thu, Aug 11, 2011 at 15:14, David Mercer wrote: >>>> >>>> This? http://www.youtube.com/watch?v=xEJ1n13soWU >>>> >>>> >>>> >>>> >>>> >>>> From: erlang-questions-bounces@REDACTED >>>> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Torben Hoffmann >>>> Sent: Thursday, August 11, 2011 7:40 AM >>>> To: erlang-questions >>>> Subject: [erlang-questions] Link to funny movie >>>> >>>> >>>> >>>> Hi, >>>> >>>> >>>> >>>> A while back there was a link to some funny youtube video (cartoon) with >>>> synthetic voices where a a non-Erlanger wanted to learn if Erlang could >>>> solve all his problems since he had heard all these wonderful things about >>>> Erlang; the Erlanger has to let him down on his crazy desires and the >>>> non-Erlanger concludes that the Erlang community is weird/closed or >>>> something like that. >>>> >>>> >>>> >>>> I have tried to find it but I cannot get it again. >>>> >>>> >>>> >>>> Any links out there? >>>> >>>> >>>> >>>> Thanks in advance, >>>> >>>> Torben >>>> >>>> -- >>>> http://www.linkedin.com/in/torbenhoffmann >>> >>> >>> -- >>> http://www.linkedin.com/in/torbenhoffmann >> >> >> >> -- >> http://www.linkedin.com/in/torbenhoffmann >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Wed Aug 17 03:01:24 2011 From: g@REDACTED (Garrett Smith) Date: Tue, 16 Aug 2011 20:01:24 -0500 Subject: [erlang-questions] Link to funny movie In-Reply-To: References: <02cf01cc5828$a0300720$e0901560$@com> Message-ID: This is a better source for gar1t videos: http://shardingdevnull.com/ Rumor has it there's an Erlang rap video in the works :) On Tue, Aug 16, 2011 at 4:03 PM, nx wrote: > More of gar1t's xtranormal videos are here: > http://www.xtranormal.com/profile/3201665/?listtype=SERIES&listid=14273053 > > There's also: http://www.xtranormal.com/watch/6995033/mongo-db-is-web-scale > > On Fri, Aug 12, 2011 at 4:58 PM, Torben Hoffmann > wrote: >> BTW: the one I am looking for uses the same type of characters and voices. >> >> On Fri, Aug 12, 2011 at 21:54, Torben Hoffmann >> wrote: >>> >>> Alas, that is not the one - funny as it is it is not the one I was looking >>> for. >>> But thanks for the link anyway! >>> Cheers, >>> Torben >>> >>> On Thu, Aug 11, 2011 at 15:14, David Mercer wrote: >>>> >>>> This?? http://www.youtube.com/watch?v=xEJ1n13soWU >>>> >>>> >>>> >>>> >>>> >>>> From: erlang-questions-bounces@REDACTED >>>> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Torben Hoffmann >>>> Sent: Thursday, August 11, 2011 7:40 AM >>>> To: erlang-questions >>>> Subject: [erlang-questions] Link to funny movie >>>> >>>> >>>> >>>> Hi, >>>> >>>> >>>> >>>> A while back there was a link to some funny youtube video (cartoon) with >>>> synthetic voices where a a non-Erlanger wanted to learn if Erlang could >>>> solve all his problems since he had heard all these wonderful things about >>>> Erlang; the Erlanger has to let him down on his crazy desires and the >>>> non-Erlanger concludes that the Erlang community is weird/closed or >>>> something like that. >>>> >>>> >>>> >>>> I have tried to find it but I cannot get it again. >>>> >>>> >>>> >>>> Any links out there? >>>> >>>> >>>> >>>> Thanks in advance, >>>> >>>> Torben >>>> >>>> -- >>>> http://www.linkedin.com/in/torbenhoffmann >>> >>> >>> -- >>> http://www.linkedin.com/in/torbenhoffmann >> >> >> >> -- >> http://www.linkedin.com/in/torbenhoffmann >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ok@REDACTED Wed Aug 17 06:31:19 2011 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 17 Aug 2011 16:31:19 +1200 Subject: [erlang-questions] [semi-ot] introducing constants and include (that can include erlang code) in efene In-Reply-To: References: <1FB1B4D4-1323-4D31-831B-65DEA8737189@cs.otago.ac.nz> Message-ID: <48C0F209-E366-4504-8A37-A4D1716B0309@cs.otago.ac.nz> On 17/08/2011, at 12:43 AM, Nahuel Greco wrote: > On Tue, Aug 16, 2011 at 6:26 AM, Mariano Guerra > wrote: >> >> I don't know a way to mutate the body of a function at runtime, what >> can be done right now in efene is that if the expression can be >> calculated at compile time, you can execute it then and just put the >> result in the bytecode: Concerning hygienic macros, the crucial thing is that identifiers in the expansion of a macro refer to those visible at its definition point, unless they come from arguments. This can be handled by renaming, which doesn't sound terribly hacky to me. Other functional programming communities have experienced some difficulty with evaluating expressions at compile time; the results can be very big, and they can be kept far longer than you really want them to be. From ok@REDACTED Wed Aug 17 06:34:23 2011 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 17 Aug 2011 16:34:23 +1200 Subject: [erlang-questions] omaha poker hand permutations In-Reply-To: <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com> References: <1887F572-AE36-4D5A-82D8-57F095789540@gmail.com> <43B80645-3CCF-4718-90FD-C41A8BFCC199@gmail.com> Message-ID: <1A96D11A-4256-4843-AA7E-770A055F9AA5@cs.otago.ac.nz> On 17/08/2011, at 2:27 AM, Joel Reymont wrote: > The problem can be reduced to the following: > > 5> L3 = ["AD", "AH", "AS", "QS"]. > ["AD","AH","AS","QS"] > > 6> L4 = [ {A, B} || A <- L3, B <- L3, A /= B]. > [{"AD","AH"}, > {"AD","AS"}, > {"AD","QS"}, > {"AH","AD"}, > {"AH","AS"}, > {"AH","QS"}, > {"AS","AD"}, > {"AS","AH"}, > {"AS","QS"}, > {"QS","AD"}, > {"QS","AH"}, > {"QS","AS"}] > > Note the duplicate pairs of AD, AH and AH, AD. > > Can these be eliminated inside the list comprehension itself? If you regard AD,AH and AH,AD as duplicates, then presumably the order of {A, B} in the pairs does not matter. 1> L3 = ["AD", "AH", "AS", "QH"]. ["AD","AH","AS","QH"] 2> L4 = [{A, B} || A <- L3, B <- L3, A < B]. [{"AD","AH"},{"AD","AS"},{"AD","QH"},{"AH","AS"},{"AH","QH"},{"AS","QH"}] From desired.mta@REDACTED Wed Aug 17 08:32:30 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Wed, 17 Aug 2011 09:32:30 +0300 Subject: [erlang-questions] Handling non http in mochiweb In-Reply-To: References: Message-ID: <20110817063230.GA27237@jakstys.lt> On Tue, Aug 16, 2011 at 02:45:03PM -0700, Ryan Huffman wrote: > Does it have to be the same port? You could use gen_tcp to listen to > connections on a different port and handle the traffic however you would > like. > Sometimes people are forced to use http proxy for http traffic (80 and 443) (an example is university of Glasgow), therefore you can't send arbitrary traffic through port 80 -- it just doesn't work. However, you can use port 443 for arbitrary TCP and be quite sure it will work anywhere. (I used openvpn over tcp 443) Motiejus From tony@REDACTED Wed Aug 17 10:47:29 2011 From: tony@REDACTED (Tony Rogvall) Date: Wed, 17 Aug 2011 10:47:29 +0200 Subject: [erlang-questions] Link to funny movie In-Reply-To: References: <02cf01cc5828$a0300720$e0901560$@com> Message-ID: <216D34D6-7404-4790-AE42-6A90D809F0FD@rogvall.se> This time it was even better :-) Love it. Thanks. /Tony On 17 aug 2011, at 01:55, James Churchman wrote: > i think you mean the ( official / unofficial ? ) spwanfest video http://vimeo.com/22360540 :-) > > On 16 Aug 2011, at 22:03, nx wrote: > >> More of gar1t's xtranormal videos are here: >> http://www.xtranormal.com/profile/3201665/?listtype=SERIES&listid=14273053 >> >> There's also: http://www.xtranormal.com/watch/6995033/mongo-db-is-web-scale >> >> On Fri, Aug 12, 2011 at 4:58 PM, Torben Hoffmann >> wrote: >>> BTW: the one I am looking for uses the same type of characters and voices. >>> >>> On Fri, Aug 12, 2011 at 21:54, Torben Hoffmann >>> wrote: >>>> >>>> Alas, that is not the one - funny as it is it is not the one I was looking >>>> for. >>>> But thanks for the link anyway! >>>> Cheers, >>>> Torben >>>> >>>> On Thu, Aug 11, 2011 at 15:14, David Mercer wrote: >>>>> >>>>> This? http://www.youtube.com/watch?v=xEJ1n13soWU >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> From: erlang-questions-bounces@REDACTED >>>>> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Torben Hoffmann >>>>> Sent: Thursday, August 11, 2011 7:40 AM >>>>> To: erlang-questions >>>>> Subject: [erlang-questions] Link to funny movie >>>>> >>>>> >>>>> >>>>> Hi, >>>>> >>>>> >>>>> >>>>> A while back there was a link to some funny youtube video (cartoon) with >>>>> synthetic voices where a a non-Erlanger wanted to learn if Erlang could >>>>> solve all his problems since he had heard all these wonderful things about >>>>> Erlang; the Erlanger has to let him down on his crazy desires and the >>>>> non-Erlanger concludes that the Erlang community is weird/closed or >>>>> something like that. >>>>> >>>>> >>>>> >>>>> I have tried to find it but I cannot get it again. >>>>> >>>>> >>>>> >>>>> Any links out there? >>>>> >>>>> >>>>> >>>>> Thanks in advance, >>>>> >>>>> Torben >>>>> >>>>> -- >>>>> http://www.linkedin.com/in/torbenhoffmann >>>> >>>> >>>> -- >>>> http://www.linkedin.com/in/torbenhoffmann >>> >>> >>> >>> -- >>> http://www.linkedin.com/in/torbenhoffmann >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Have run Make so many times I dunno what's installed anymore" -------------- next part -------------- An HTML attachment was scrubbed... URL: From maggio.vincenzo@REDACTED Wed Aug 17 12:47:02 2011 From: maggio.vincenzo@REDACTED (Vincenzo Maggio) Date: Wed, 17 Aug 2011 12:47:02 +0200 Subject: [erlang-questions] Process surviving disconnect In-Reply-To: References: Message-ID: Hello, without further info a debug is rather difficult. But let's try to at least start analysis of the problem: > - Has this something to do that I initially spawn Bob from the Alice > node? Absolutely not: this would hit the very foundation of Erlang, process referential transparency. When a process is started is a brand new, clean entity (indeed, default process heap space is always the same size!). > - How can I make Bob to survive a connection loss? Look above: it SHOULD survive. Can you please start SASL (application:start(sasl) from the shell) and see if shell log puts some further information? Vincenzo -------------- next part -------------- An HTML attachment was scrubbed... URL: From luismarianoguerra@REDACTED Wed Aug 17 13:09:02 2011 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Wed, 17 Aug 2011 12:09:02 +0100 Subject: [erlang-questions] [semi-ot] introducing constants and include (that can include erlang code) in efene In-Reply-To: <48C0F209-E366-4504-8A37-A4D1716B0309@cs.otago.ac.nz> References: <1FB1B4D4-1323-4D31-831B-65DEA8737189@cs.otago.ac.nz> <48C0F209-E366-4504-8A37-A4D1716B0309@cs.otago.ac.nz> Message-ID: On Wed, Aug 17, 2011 at 5:31 AM, Richard O'Keefe wrote: > > On 17/08/2011, at 12:43 AM, Nahuel Greco wrote: > >> On Tue, Aug 16, 2011 at 6:26 AM, Mariano Guerra >> wrote: >>> >>> I don't know a way to mutate the body of a function at runtime, what >>> can be done right now in efene is that if the expression can be >>> calculated at compile time, you can execute it then and just put the >>> result in the bytecode: > > Concerning hygienic macros, the crucial thing is that identifiers in > the expansion of a macro refer to those visible at its definition > point, unless they come from arguments. ?This can be handled by > renaming, which doesn't sound terribly hacky to me. do you think that something like collecting variable names from the outer contexts and checking if there are collision names and renaming the names that collide is the best way to do it? I may give it a try :) > Other functional programming communities have experienced some > difficulty with evaluating expressions at compile time; the results > can be very big, and they can be kept far longer than you really > want them to be. From torben.lehoff@REDACTED Wed Aug 17 13:09:03 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Wed, 17 Aug 2011 13:09:03 +0200 Subject: [erlang-questions] Link to funny movie In-Reply-To: References: <02cf01cc5828$a0300720$e0901560$@com> Message-ID: This was the one! Thank you very much!! Torben On Wed, Aug 17, 2011 at 01:55, James Churchman wrote: > i think you mean the ( official / unofficial ? ) spwanfest video > http://vimeo.com/22360540 :-) > > On 16 Aug 2011, at 22:03, nx wrote: > > > More of gar1t's xtranormal videos are here: > > > http://www.xtranormal.com/profile/3201665/?listtype=SERIES&listid=14273053 > > > > There's also: > http://www.xtranormal.com/watch/6995033/mongo-db-is-web-scale > > > > On Fri, Aug 12, 2011 at 4:58 PM, Torben Hoffmann > > wrote: > >> BTW: the one I am looking for uses the same type of characters and > voices. > >> > >> On Fri, Aug 12, 2011 at 21:54, Torben Hoffmann > > >> wrote: > >>> > >>> Alas, that is not the one - funny as it is it is not the one I was > looking > >>> for. > >>> But thanks for the link anyway! > >>> Cheers, > >>> Torben > >>> > >>> On Thu, Aug 11, 2011 at 15:14, David Mercer wrote: > >>>> > >>>> This? http://www.youtube.com/watch?v=xEJ1n13soWU > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> From: erlang-questions-bounces@REDACTED > >>>> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Torben > Hoffmann > >>>> Sent: Thursday, August 11, 2011 7:40 AM > >>>> To: erlang-questions > >>>> Subject: [erlang-questions] Link to funny movie > >>>> > >>>> > >>>> > >>>> Hi, > >>>> > >>>> > >>>> > >>>> A while back there was a link to some funny youtube video (cartoon) > with > >>>> synthetic voices where a a non-Erlanger wanted to learn if Erlang > could > >>>> solve all his problems since he had heard all these wonderful things > about > >>>> Erlang; the Erlanger has to let him down on his crazy desires and the > >>>> non-Erlanger concludes that the Erlang community is weird/closed or > >>>> something like that. > >>>> > >>>> > >>>> > >>>> I have tried to find it but I cannot get it again. > >>>> > >>>> > >>>> > >>>> Any links out there? > >>>> > >>>> > >>>> > >>>> Thanks in advance, > >>>> > >>>> Torben > >>>> > >>>> -- > >>>> http://www.linkedin.com/in/torbenhoffmann > >>> > >>> > >>> -- > >>> http://www.linkedin.com/in/torbenhoffmann > >> > >> > >> > >> -- > >> http://www.linkedin.com/in/torbenhoffmann > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Wed Aug 17 13:54:37 2011 From: jameschurchman@REDACTED (James Churchman) Date: Wed, 17 Aug 2011 12:54:37 +0100 Subject: [erlang-questions] Link to funny movie In-Reply-To: References: <02cf01cc5828$a0300720$e0901560$@com> Message-ID: <27D7650B-91F5-4167-83F9-45C219A5CCD9@gmail.com> no probs :-) tho i do like the fact your funny vids thread got 10 replies, my "++ to get the head" 'thread' got 0 :-( hehe james On 17 Aug 2011, at 12:09, Torben Hoffmann wrote: > This was the one! > > Thank you very much!! > Torben > > On Wed, Aug 17, 2011 at 01:55, James Churchman wrote: > i think you mean the ( official / unofficial ? ) spwanfest video http://vimeo.com/22360540 :-) > > On 16 Aug 2011, at 22:03, nx wrote: > > > More of gar1t's xtranormal videos are here: > > http://www.xtranormal.com/profile/3201665/?listtype=SERIES&listid=14273053 > > > > There's also: http://www.xtranormal.com/watch/6995033/mongo-db-is-web-scale > > > > On Fri, Aug 12, 2011 at 4:58 PM, Torben Hoffmann > > wrote: > >> BTW: the one I am looking for uses the same type of characters and voices. > >> > >> On Fri, Aug 12, 2011 at 21:54, Torben Hoffmann > >> wrote: > >>> > >>> Alas, that is not the one - funny as it is it is not the one I was looking > >>> for. > >>> But thanks for the link anyway! > >>> Cheers, > >>> Torben > >>> > >>> On Thu, Aug 11, 2011 at 15:14, David Mercer wrote: > >>>> > >>>> This? http://www.youtube.com/watch?v=xEJ1n13soWU > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> From: erlang-questions-bounces@REDACTED > >>>> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Torben Hoffmann > >>>> Sent: Thursday, August 11, 2011 7:40 AM > >>>> To: erlang-questions > >>>> Subject: [erlang-questions] Link to funny movie > >>>> > >>>> > >>>> > >>>> Hi, > >>>> > >>>> > >>>> > >>>> A while back there was a link to some funny youtube video (cartoon) with > >>>> synthetic voices where a a non-Erlanger wanted to learn if Erlang could > >>>> solve all his problems since he had heard all these wonderful things about > >>>> Erlang; the Erlanger has to let him down on his crazy desires and the > >>>> non-Erlanger concludes that the Erlang community is weird/closed or > >>>> something like that. > >>>> > >>>> > >>>> > >>>> I have tried to find it but I cannot get it again. > >>>> > >>>> > >>>> > >>>> Any links out there? > >>>> > >>>> > >>>> > >>>> Thanks in advance, > >>>> > >>>> Torben > >>>> > >>>> -- > >>>> http://www.linkedin.com/in/torbenhoffmann > >>> > >>> > >>> -- > >>> http://www.linkedin.com/in/torbenhoffmann > >> > >> > >> > >> -- > >> http://www.linkedin.com/in/torbenhoffmann > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Wed Aug 17 14:13:47 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Wed, 17 Aug 2011 15:13:47 +0300 Subject: [erlang-questions] Link to funny movie In-Reply-To: <27D7650B-91F5-4167-83F9-45C219A5CCD9@gmail.com> References: <02cf01cc5828$a0300720$e0901560$@com> <27D7650B-91F5-4167-83F9-45C219A5CCD9@gmail.com> Message-ID: <20110817121347.GA10866@jakstys.lt> On Wed, Aug 17, 2011 at 12:54:37PM +0100, James Churchman wrote: > no probs :-) tho i do like the fact your funny vids thread got 10 > replies, my "++ to get the head" 'thread' got 0 :-( hehe james It's holiday time. Holiday means vacation (therefore no "serious" responses to list) or watching videos (therefore funny responses to list). Have a nice holiday! Motiejus From freeakk@REDACTED Wed Aug 17 14:35:23 2011 From: freeakk@REDACTED (Michael Uvarov) Date: Wed, 17 Aug 2011 05:35:23 -0700 (PDT) Subject: [erlang-questions] my best erlang "foot in mouth" In-Reply-To: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> References: <061AF03E-3BC1-476F-AED8-655607C1472A@gmail.com> Message-ID: <22390371.1051.1313584523843.JavaMail.geo-discussion-forums@yqic12> -define(DBG(X,Y), ?DBG(X,Y)). in the header file. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bgustavsson@REDACTED Wed Aug 17 15:00:23 2011 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 17 Aug 2011 15:00:23 +0200 Subject: [erlang-questions] Unicode noncharacter inconsistency In-Reply-To: References: Message-ID: On Wed, Jul 27, 2011 at 4:06 PM, Alisdair Sullivan wrote: > The binary Unicode 'switches' /utf8, /utf16, etc raise an error when asked to encode u+fffe or u+ffff (which are non characters, making this a defensible and perhaps sensible position), but not when asked to encode u+1fffe, u+1ffff, u+2fffe, ..., u+10ffff. I considered submitting a bug, but as these are ok for internal implementations and are just forbidden during interchange, it's arguable this does adhere to the spec. > > Is there some rationale behind this inconsistency, or is it simply an oversight? An oversight. > The reserved non characters u+fdd0 - u+fdef should also probably behave as u+fffe/ffff, as they are in the same category. We think that it is better to allow u+fffe and u+ffff as noncharacters could be useful for internal processing. This change will make the implementation consistent with RFC3629. (Another advantage is that the conversion will be slightly faster since there is one test less to perform.) We will probably make this change in the R15 release. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From erlang@REDACTED Wed Aug 17 18:57:35 2011 From: erlang@REDACTED (Anchise Inzaghi) Date: Wed, 17 Aug 2011 10:57:35 -0600 Subject: [erlang-questions] Process surviving disconnect In-Reply-To: References: Message-ID: <88dce0568e954d217577e7ffdc55fcd8@mail.nimqas.com> Thank you very much Vincenzo. You affirmed my assertion that Bob should survive the disconnect. Nevertheless he dies. I will point out exactly what I do and maybe someone can spot the error in my code, my setup or my thinking and tell me what I am doing wrong. I start a node on gca.local: unroot@REDACTED:~$ erl -name 'bob@REDACTED' -setcookie 123 I start a node on usa.local: unroot@REDACTED:~$ erl -name 'alice@REDACTED' -setcookie 123 I start sasl on bob@REDACTED: (bob@REDACTED)1> application:start (sasl). I run alice:start/0 on alice@REDACTED: (alice@REDACTED)1> alice:start (). true I look for bob on bob@REDACTED and save its pid: (bob@REDACTED)2> whereis (bob). <0.65.0> (bob@REDACTED)3> Pid = whereis (bob). <0.65.0> I cut the network cable and wait a minute for the timeout. On alice@REDACTED I get the following output: =ERROR REPORT==== 17-Aug-2011::10:53:21 === ** Node 'bob@REDACTED' not responding ** ** Removing (timedout) connection ** Bob die of noconnection. Nice, Alice trapped Bob's death and reported it. I check for Alice: (alice@REDACTED)2> whereis (alice). <0.42.0> Alice is up and running. On bob@REDACTED I get the following output: =ERROR REPORT==== 17-Aug-2011::10:53:10 === ** Node 'alice@REDACTED' not responding ** ** Removing (timedout) connection ** But Bob is dead: (bob@REDACTED)4> whereis (bob). undefined (bob@REDACTED)5> is_process_alive (Pid). false I really do not understand what is happening. Thank you in advance Anchise Here goes the code I used: -module (alice). -compile (export_all). start () -> register (alice, spawn (fun init/0) ). stop () -> whereis (alice) ! stop. init () -> process_flag (trap_exit, true), Bob = spawn_link ('bob@REDACTED', bob, start, [self () ] ), loop (Bob). loop (Bob) -> receive stop -> ok; {'EXIT', Bob, Reason} -> io:format ("Bob died of ~p.~n", [Reason] ), loop (Bob); Msg -> io:format ("Alice received ~p.~n", [Msg] ), loop (Bob) end. -module (bob). -compile (export_all). start (Alice) -> process_flag (trap_exit, true), register (bob, self () ), loop (Alice). loop (Alice) -> receive stop -> ok; {'EXIT', Alice, Reason} -> io:format ("Alice died of ~p.~n", [Reason] ), loop (Alice); Msg -> io:format ("Bob received ~p.~n", [Msg] ), loop (Alice) end. On Wed, 17 Aug 2011 12:47:02 +0200, Vincenzo Maggio wrote: > Hello, > without further info a debug is rather difficult. > But let's try to at least start analysis of the problem: > >> - Has this something to do that I initially spawn Bob from the >> Alice >> node? > > Absolutely not: this would hit the very foundation of Erlang, process > referential transparency. When a process is started is a brand new, > clean entity (indeed, default process heap space is always the same > size!). > >> - How can I make Bob to survive a connection loss? > > Look above: it SHOULD survive. > > Can you please start SASL (application:start(sasl) from the shell) > and see if shell log puts some further information? > > Vincenzo From kenneth.lundin@REDACTED Wed Aug 17 19:09:03 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 17 Aug 2011 19:09:03 +0200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> Message-ID: We in the OTP team think this is a good idea but we can't say when we would have time to implement it. It should be quite simple to implement mostly involving the module erl_lint. A nice user contrib would make it possible to introduce quicker. The possibility to turn on and off per function is not so important , Can be added later if needed and would require some extra thought. /Kenneth Erlang/OTP Ericsson Den 29 jul 2011 01.45, "Richard O'Keefe" skrev: One of the things criticised in the blog entry that we've been responding to was that {ok,Foo} = bar(...), {ok,Foo} = ugh(...) is too easy to write (when you really meant, say, Foo0, Foo1). This is a well defined part of the language, and it would not be a good idea to ban it. But how about an optional style warning (and we really need -warn(on | off, [warning...]) directives) whenever a bound variable appears in a pattern on the left of "="? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Wed Aug 17 19:26:37 2011 From: francesco@REDACTED (Francesco Cesarini) Date: Wed, 17 Aug 2011 18:26:37 +0100 Subject: [erlang-questions] Registration for the 2011 Erlang User Conference Open Message-ID: <4E4BF9CD.6010401@erlang-solutions.com> Hi All, a note to say that we have opened the registration for the 2011 Erlang User Conference. The first 50 people to register will get the very early bird rate (There are 25 places remaining). We have outgrown the last venue, and are this year moving the conference to the Munchen Bryggeriet, just South of the Old Town. We are planing on running a day of talks on Thursday the 3rd of November, followed by a day of tutorials and a hackathon on Friday the 4th. For those of you interested in learning Erlang, there will be three training courses in the runup to the conference. More information as it becomes available is here: http://www.erlang-factory.com/conference/ErlangUserConference2011 To register, the link is https://www.erlang-factory.com/conference/ErlangUserConference2011/register We are planing three parallel tracks leaving you spoilt for choice. The talk submission is still open, so if you want to present or give a tutorial, submit it here: http://www.erlang-factory.com/conference/ErlangUserConference2011/submit_talk Dates for your calendar are: August 15^th , Registration opens to past attendees (First 50 delegates to register get the Very Early Bird Rate) August 16^th , Registration opens to all August 30^th , Talk Submission is closed September 10^th , Final program is announced October 15^th , Last Day for Early Bird Rate October 31^st , Start of Erlang University November 2nd, Guided evening walk around central Stockholm November 3^rd , Conference Talks + Evening Dinner November 4^th , Hackathon and Tutorials If you have any questions, let us know! Francesco -- Erlang Solutions Ltd. http://www.erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Wed Aug 17 20:02:57 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 17 Aug 2011 14:02:57 -0400 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> Message-ID: 2011/8/3 Fr?d?ric Trottier-H?bert : > On 2011-08-02, at 01:59 AM, Richard O'Keefe wrote: > >> 2. Is matching against an already-bound variable a check we want? > > To me, matching against an already-bound variable is is a valid assertion, as much as 'ok = function_call()' might be, and as implicit as '{ok, Pid}' or '{error, already_started}' might be. Matching on already-bound variables is part of my standard code when trying to crash early when possible, and also part of many basic test suites that simply do pattern matching here and there. To me it's as basic as using the same variable twice in a single pattern (f([A,A,B,B|_]) when A =/= B -> ...), or something similar with records. > > I would likely not use the check at all, and if it were to be added, would prefer it to be a compiler argument (which could be enabled in -compile(...).) I foresee little use of such warnings for myself and would dislike to see it becoming a default setting. I completely agree. I use matching this way quite a bit, especially in testing, and wouldn't want to see any warnings for it by default. --steve From carnildo@REDACTED Wed Aug 17 21:39:50 2011 From: carnildo@REDACTED (Mark Wagner) Date: Wed, 17 Aug 2011 12:39:50 -0700 Subject: [erlang-questions] ++ to select the head ( instead of just the tail ) In-Reply-To: <4191CF03-18C6-451E-8444-2271DC4615D4@gmail.com> References: <4191CF03-18C6-451E-8444-2271DC4615D4@gmail.com> Message-ID: On Tue, Aug 16, 2011 at 04:23, James Churchman wrote: > The ++ operator is not just a righthand side operator, but can in fact be a lefthand side operator in as far as I can tell just 1 case : > > When you know the head you wish to remove and getting the tail : > > Eg in the shell: > > FullList = "123456". > "123" ++ Tail = FullList. > Tail wil now be "456" > > However using it to get the head is not supported : > > FullList = "123456". > Head ++ ?"456" = FullList. > > gives : * 1: illegal pattern > > Understandably this second type involves traversing the entire list, where the first does not, but it would still be a nice addition to erlang. > > Was this not added as the first one expands out to simply [$1,$2,$3 | Tail] etc... but the second needs to expand to something more complex like an actual function the way list comprehensions do? > My understanding is that Erlang lists are single-linked lists. This means that finding the front element (and by extension, a fixed number of elements at the front of the list) is an easy operation that runs in constant time. Finding the tail element is easy and slow (traverse the list until you hit the end), but there's no good way to find "the element before the tail element". An iterative solution requires traversing the list again; a recursive solution would require processing after the recursion step, meaning it can't be tail-recursive and will require space proportionate to the size of the list (picture running 'Head ++ "The End"' on a million-byte novel). -- Mark Wagner From maggio.vincenzo@REDACTED Wed Aug 17 21:48:30 2011 From: maggio.vincenzo@REDACTED (Vincenzo Maggio) Date: Wed, 17 Aug 2011 21:48:30 +0200 Subject: [erlang-questions] Process surviving disconnect In-Reply-To: <88dce0568e954d217577e7ffdc55fcd8@mail.nimqas.com> References: , , <88dce0568e954d217577e7ffdc55fcd8@mail.nimqas.com> Message-ID: Hello, I think something is wrong here. > Bob die of noconnection. This is printed by > {'EXIT', Bob, Reason} -> > io:format ("Bob died of ~p.~n", [Reason] ), So alice is in fact receiving bob last death cry :D and process_flag translate it in a message instead of transmitting exit signal to alice; I think this is ok from the point of view of Alice, so the real problem is that bob is dying (I know it's mundane, but I learned not to make assumption). Mmm, well I don't know if having no more a connection between the process makes Erlang VM do some assumption of a virtual master node. Well, if you want my opinion, I think that you should file a bug on the Erlang bugs mailing list if no one comes up with a proper explanation. Even if what we're thinking is wrong and this is not a bug, in the past I had a problem on node lookup and they resolved it. These are my two cents, but if you can please let me know if there are further updates 'cause it's a really interesting problem. BTW, before filing a bug, could you please substitute spawn_link with spawn_monitor and remove the process_flag lines? It would be interesting to understand if either bob dies on its own or it's killed by no more being able to communicate with alice. Vincenzo > Date: Wed, 17 Aug 2011 10:57:35 -0600 > From: erlang@REDACTED > To: maggio.vincenzo@REDACTED > CC: erlang-questions@REDACTED > Subject: RE: [erlang-questions] Process surviving disconnect > > Thank you very much Vincenzo. You affirmed my assertion that Bob should > survive the disconnect. > Nevertheless he dies. > I will point out exactly what I do and maybe someone can spot the error > in my code, my setup or my thinking and tell me what I am doing wrong. > > I start a node on gca.local: > unroot@REDACTED:~$ erl -name 'bob@REDACTED' -setcookie 123 > > I start a node on usa.local: > unroot@REDACTED:~$ erl -name 'alice@REDACTED' -setcookie 123 > > I start sasl on bob@REDACTED: > (bob@REDACTED)1> application:start (sasl). > > I run alice:start/0 on alice@REDACTED: > (alice@REDACTED)1> alice:start (). > true > > I look for bob on bob@REDACTED and save its pid: > (bob@REDACTED)2> whereis (bob). > <0.65.0> > (bob@REDACTED)3> Pid = whereis (bob). > <0.65.0> > > I cut the network cable and wait a minute for the timeout. > > On alice@REDACTED I get the following output: > =ERROR REPORT==== 17-Aug-2011::10:53:21 === > ** Node 'bob@REDACTED' not responding ** > ** Removing (timedout) connection ** > Bob die of noconnection. > > Nice, Alice trapped Bob's death and reported it. I check for Alice: > (alice@REDACTED)2> whereis (alice). > <0.42.0> > > Alice is up and running. > > On bob@REDACTED I get the following output: > =ERROR REPORT==== 17-Aug-2011::10:53:10 === > ** Node 'alice@REDACTED' not responding ** > ** Removing (timedout) connection ** > > But Bob is dead: > (bob@REDACTED)4> whereis (bob). > undefined > (bob@REDACTED)5> is_process_alive (Pid). > false > > I really do not understand what is happening. > > > Thank you in advance > > Anchise > > Here goes the code I used: > > -module (alice). > -compile (export_all). > > start () -> register (alice, spawn (fun init/0) ). > > stop () -> whereis (alice) ! stop. > > init () -> > process_flag (trap_exit, true), > Bob = spawn_link ('bob@REDACTED', bob, start, [self () ] ), > loop (Bob). > > loop (Bob) -> > receive > stop -> ok; > {'EXIT', Bob, Reason} -> > io:format ("Bob died of ~p.~n", [Reason] ), > loop (Bob); > Msg -> > io:format ("Alice received ~p.~n", [Msg] ), > loop (Bob) > end. > > > -module (bob). > -compile (export_all). > > start (Alice) -> > process_flag (trap_exit, true), > register (bob, self () ), > loop (Alice). > > loop (Alice) -> > receive > stop -> ok; > {'EXIT', Alice, Reason} -> > io:format ("Alice died of ~p.~n", [Reason] ), > loop (Alice); > Msg -> > io:format ("Bob received ~p.~n", [Msg] ), > loop (Alice) > end. > > > On Wed, 17 Aug 2011 12:47:02 +0200, Vincenzo Maggio wrote: > > Hello, > > without further info a debug is rather difficult. > > But let's try to at least start analysis of the problem: > > > >> - Has this something to do that I initially spawn Bob from the > >> Alice > >> node? > > > > Absolutely not: this would hit the very foundation of Erlang, process > > referential transparency. When a process is started is a brand new, > > clean entity (indeed, default process heap space is always the same > > size!). > > > >> - How can I make Bob to survive a connection loss? > > > > Look above: it SHOULD survive. > > > > Can you please start SASL (application:start(sasl) from the shell) > > and see if shell log puts some further information? > > > > Vincenzo > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth.lundin@REDACTED Wed Aug 17 22:09:23 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 17 Aug 2011 22:09:23 +0200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> Message-ID: I agree that it is very common in test code to match against already bound variables. But in production code it is quite rare. The warning should definitely not be on by default. We will also run a check over all OTP libraries and possibly other production code as well to see how it turns out before we make any definite decisions. But as said we have not given the implementation any priority at all yet. /Kenneth 2011/8/17 Steve Vinoski : > 2011/8/3 Fr?d?ric Trottier-H?bert : >> On 2011-08-02, at 01:59 AM, Richard O'Keefe wrote: >> >>> 2. Is matching against an already-bound variable a check we want? >> >> To me, matching against an already-bound variable is is a valid assertion, as much as 'ok = function_call()' might be, and as implicit as '{ok, Pid}' or '{error, already_started}' might be. Matching on already-bound variables is part of my standard code when trying to crash early when possible, and also part of many basic test suites that simply do pattern matching here and there. To me it's as basic as using the same variable twice in a single pattern (f([A,A,B,B|_]) when A =/= B -> ...), or something similar with records. >> >> I would likely not use the check at all, and if it were to be added, would prefer it to be a compiler argument (which could be enabled in -compile(...).) I foresee little use of such warnings for myself and would dislike to see it becoming a default setting. > > I completely agree. I use matching this way quite a bit, especially in > testing, and wouldn't want to see any warnings for it by default. > > --steve > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Aug 17 22:13:37 2011 From: erlang@REDACTED (Anchise Inzaghi) Date: Wed, 17 Aug 2011 14:13:37 -0600 Subject: [erlang-questions] Process surviving disconnect In-Reply-To: References: , , <88dce0568e954d217577e7ffdc55fcd8@mail.nimqas.com> Message-ID: Thank you for your quick answer. > BTW, before filing a bug, could you please substitute spawn_link with > spawn_monitor and remove the process_flag lines? It would be > interesting to understand if either bob dies on its own or it's > killed > by no more being able to communicate with alice. I am not sure how to replace spawn_link with spawn_monitor, as neither spawn_monitor/1 nor spawn_monitor/3 take a node parameter. How do I do that or how else can I get some more detailed information about Bob's sudden passing the Styx. On Wed, 17 Aug 2011 21:48:30 +0200, Vincenzo Maggio wrote: > Hello, > I think something is wrong here. >> Bob die of noconnection. > > This is printed by >> {'EXIT', Bob, Reason} -> >> io:format ("Bob died of ~p.~n", [Reason] ), > > So alice is in fact receiving bob last death cry :D and process_flag > translate it in a message instead of transmitting exit signal to > alice; I think this is ok from the point of view of Alice, so the > real > problem is that bob is dying (I know it's mundane, but I learned not > to make assumption). > Mmm, well I don't know if having no more a connection between the > process makes Erlang VM do some assumption of a virtual master node. > Well, if you want my opinion, I think that you should file a bug on > the Erlang bugs mailing list if no one comes up with a proper > explanation. > Even if what we're thinking is wrong and this is not a bug, in the > past I had a problem on node lookup and they resolved it. > These are my two cents, but if you can please let me know if there > are further updates 'cause it's a really interesting problem. > > BTW, before filing a bug, could you please substitute spawn_link with > spawn_monitor and remove the process_flag lines? It would be > interesting to understand if either bob dies on its own or it's > killed > by no more being able to communicate with alice. > > Vincenzo > >> Date: Wed, 17 Aug 2011 10:57:35 -0600 >> From: erlang@REDACTED >> To: maggio.vincenzo@REDACTED >> CC: erlang-questions@REDACTED >> Subject: RE: [erlang-questions] Process surviving disconnect >> >> Thank you very much Vincenzo. You affirmed my assertion that Bob >> should >> survive the disconnect. >> Nevertheless he dies. >> I will point out exactly what I do and maybe someone can spot the >> error >> in my code, my setup or my thinking and tell me what I am doing >> wrong. >> >> I start a node on gca.local: >> unroot@REDACTED:~$ erl -name 'bob@REDACTED' -setcookie 123 >> >> I start a node on usa.local: >> unroot@REDACTED:~$ erl -name 'alice@REDACTED' -setcookie 123 >> >> I start sasl on bob@REDACTED: >> (bob@REDACTED)1> application:start (sasl). >> >> I run alice:start/0 on alice@REDACTED: >> (alice@REDACTED)1> alice:start (). >> true >> >> I look for bob on bob@REDACTED and save its pid: >> (bob@REDACTED)2> whereis (bob). >> <0.65.0> >> (bob@REDACTED)3> Pid = whereis (bob). >> <0.65.0> >> >> I cut the network cable and wait a minute for the timeout. >> >> On alice@REDACTED I get the following output: >> =ERROR REPORT==== 17-Aug-2011::10:53:21 === >> ** Node 'bob@REDACTED' not responding ** >> ** Removing (timedout) connection ** >> Bob die of noconnection. >> >> Nice, Alice trapped Bob's death and reported it. I check for Alice: >> (alice@REDACTED)2> whereis (alice). >> <0.42.0> >> >> Alice is up and running. >> >> On bob@REDACTED I get the following output: >> =ERROR REPORT==== 17-Aug-2011::10:53:10 === >> ** Node 'alice@REDACTED' not responding ** >> ** Removing (timedout) connection ** >> >> But Bob is dead: >> (bob@REDACTED)4> whereis (bob). >> undefined >> (bob@REDACTED)5> is_process_alive (Pid). >> false >> >> I really do not understand what is happening. >> >> >> Thank you in advance >> >> Anchise >> >> Here goes the code I used: >> >> -module (alice). >> -compile (export_all). >> >> start () -> register (alice, spawn (fun init/0) ). >> >> stop () -> whereis (alice) ! stop. >> >> init () -> >> process_flag (trap_exit, true), >> Bob = spawn_link ('bob@REDACTED', bob, start, [self () ] ), >> loop (Bob). >> >> loop (Bob) -> >> receive >> stop -> ok; >> {'EXIT', Bob, Reason} -> >> io:format ("Bob died of ~p.~n", [Reason] ), >> loop (Bob); >> Msg -> >> io:format ("Alice received ~p.~n", [Msg] ), >> loop (Bob) >> end. >> >> >> -module (bob). >> -compile (export_all). >> >> start (Alice) -> >> process_flag (trap_exit, true), >> register (bob, self () ), >> loop (Alice). >> >> loop (Alice) -> >> receive >> stop -> ok; >> {'EXIT', Alice, Reason} -> >> io:format ("Alice died of ~p.~n", [Reason] ), >> loop (Alice); >> Msg -> >> io:format ("Bob received ~p.~n", [Msg] ), >> loop (Alice) >> end. >> >> >> On Wed, 17 Aug 2011 12:47:02 +0200, Vincenzo Maggio wrote: >> > Hello, >> > without further info a debug is rather difficult. >> > But let's try to at least start analysis of the problem: >> > >> >> - Has this something to do that I initially spawn Bob from the >> >> Alice >> >> node? >> > >> > Absolutely not: this would hit the very foundation of Erlang, >> process >> > referential transparency. When a process is started is a brand >> new, >> > clean entity (indeed, default process heap space is always the >> same >> > size!). >> > >> >> - How can I make Bob to survive a connection loss? >> > >> > Look above: it SHOULD survive. >> > >> > Can you please start SASL (application:start(sasl) from the shell) >> > and see if shell log puts some further information? >> > >> > Vincenzo >> From maggio.vincenzo@REDACTED Wed Aug 17 22:23:06 2011 From: maggio.vincenzo@REDACTED (Vincenzo Maggio) Date: Wed, 17 Aug 2011 22:23:06 +0200 Subject: [erlang-questions] Process surviving disconnect In-Reply-To: References: , , <88dce0568e954d217577e7ffdc55fcd8@mail.nimqas.com>, , Message-ID: Use monitor(process, PID_TO_BE_MONITORED) after spawn. BTW not having two PCs I simply tested your code killing alice and guess what? bob survives! :) > Date: Wed, 17 Aug 2011 14:13:37 -0600 > From: erlang@REDACTED > To: maggio.vincenzo@REDACTED > CC: erlang-questions@REDACTED > Subject: RE: [erlang-questions] Process surviving disconnect > > Thank you for your quick answer. > > > BTW, before filing a bug, could you please substitute spawn_link with > > spawn_monitor and remove the process_flag lines? It would be > > interesting to understand if either bob dies on its own or it's > > killed > > by no more being able to communicate with alice. > > I am not sure how to replace spawn_link with spawn_monitor, as neither > spawn_monitor/1 nor spawn_monitor/3 take a node parameter. > How do I do that or how else can I get some more detailed information > about Bob's sudden passing the Styx. > > > > On Wed, 17 Aug 2011 21:48:30 +0200, Vincenzo Maggio wrote: > > Hello, > > I think something is wrong here. > >> Bob die of noconnection. > > > > This is printed by > >> {'EXIT', Bob, Reason} -> > >> io:format ("Bob died of ~p.~n", [Reason] ), > > > > So alice is in fact receiving bob last death cry :D and process_flag > > translate it in a message instead of transmitting exit signal to > > alice; I think this is ok from the point of view of Alice, so the > > real > > problem is that bob is dying (I know it's mundane, but I learned not > > to make assumption). > > Mmm, well I don't know if having no more a connection between the > > process makes Erlang VM do some assumption of a virtual master node. > > Well, if you want my opinion, I think that you should file a bug on > > the Erlang bugs mailing list if no one comes up with a proper > > explanation. > > Even if what we're thinking is wrong and this is not a bug, in the > > past I had a problem on node lookup and they resolved it. > > These are my two cents, but if you can please let me know if there > > are further updates 'cause it's a really interesting problem. > > > > BTW, before filing a bug, could you please substitute spawn_link with > > spawn_monitor and remove the process_flag lines? It would be > > interesting to understand if either bob dies on its own or it's > > killed > > by no more being able to communicate with alice. > > > > Vincenzo > > > >> Date: Wed, 17 Aug 2011 10:57:35 -0600 > >> From: erlang@REDACTED > >> To: maggio.vincenzo@REDACTED > >> CC: erlang-questions@REDACTED > >> Subject: RE: [erlang-questions] Process surviving disconnect > >> > >> Thank you very much Vincenzo. You affirmed my assertion that Bob > >> should > >> survive the disconnect. > >> Nevertheless he dies. > >> I will point out exactly what I do and maybe someone can spot the > >> error > >> in my code, my setup or my thinking and tell me what I am doing > >> wrong. > >> > >> I start a node on gca.local: > >> unroot@REDACTED:~$ erl -name 'bob@REDACTED' -setcookie 123 > >> > >> I start a node on usa.local: > >> unroot@REDACTED:~$ erl -name 'alice@REDACTED' -setcookie 123 > >> > >> I start sasl on bob@REDACTED: > >> (bob@REDACTED)1> application:start (sasl). > >> > >> I run alice:start/0 on alice@REDACTED: > >> (alice@REDACTED)1> alice:start (). > >> true > >> > >> I look for bob on bob@REDACTED and save its pid: > >> (bob@REDACTED)2> whereis (bob). > >> <0.65.0> > >> (bob@REDACTED)3> Pid = whereis (bob). > >> <0.65.0> > >> > >> I cut the network cable and wait a minute for the timeout. > >> > >> On alice@REDACTED I get the following output: > >> =ERROR REPORT==== 17-Aug-2011::10:53:21 === > >> ** Node 'bob@REDACTED' not responding ** > >> ** Removing (timedout) connection ** > >> Bob die of noconnection. > >> > >> Nice, Alice trapped Bob's death and reported it. I check for Alice: > >> (alice@REDACTED)2> whereis (alice). > >> <0.42.0> > >> > >> Alice is up and running. > >> > >> On bob@REDACTED I get the following output: > >> =ERROR REPORT==== 17-Aug-2011::10:53:10 === > >> ** Node 'alice@REDACTED' not responding ** > >> ** Removing (timedout) connection ** > >> > >> But Bob is dead: > >> (bob@REDACTED)4> whereis (bob). > >> undefined > >> (bob@REDACTED)5> is_process_alive (Pid). > >> false > >> > >> I really do not understand what is happening. > >> > >> > >> Thank you in advance > >> > >> Anchise > >> > >> Here goes the code I used: > >> > >> -module (alice). > >> -compile (export_all). > >> > >> start () -> register (alice, spawn (fun init/0) ). > >> > >> stop () -> whereis (alice) ! stop. > >> > >> init () -> > >> process_flag (trap_exit, true), > >> Bob = spawn_link ('bob@REDACTED', bob, start, [self () ] ), > >> loop (Bob). > >> > >> loop (Bob) -> > >> receive > >> stop -> ok; > >> {'EXIT', Bob, Reason} -> > >> io:format ("Bob died of ~p.~n", [Reason] ), > >> loop (Bob); > >> Msg -> > >> io:format ("Alice received ~p.~n", [Msg] ), > >> loop (Bob) > >> end. > >> > >> > >> -module (bob). > >> -compile (export_all). > >> > >> start (Alice) -> > >> process_flag (trap_exit, true), > >> register (bob, self () ), > >> loop (Alice). > >> > >> loop (Alice) -> > >> receive > >> stop -> ok; > >> {'EXIT', Alice, Reason} -> > >> io:format ("Alice died of ~p.~n", [Reason] ), > >> loop (Alice); > >> Msg -> > >> io:format ("Bob received ~p.~n", [Msg] ), > >> loop (Alice) > >> end. > >> > >> > >> On Wed, 17 Aug 2011 12:47:02 +0200, Vincenzo Maggio wrote: > >> > Hello, > >> > without further info a debug is rather difficult. > >> > But let's try to at least start analysis of the problem: > >> > > >> >> - Has this something to do that I initially spawn Bob from the > >> >> Alice > >> >> node? > >> > > >> > Absolutely not: this would hit the very foundation of Erlang, > >> process > >> > referential transparency. When a process is started is a brand > >> new, > >> > clean entity (indeed, default process heap space is always the > >> same > >> > size!). > >> > > >> >> - How can I make Bob to survive a connection loss? > >> > > >> > Look above: it SHOULD survive. > >> > > >> > Can you please start SASL (application:start(sasl) from the shell) > >> > and see if shell log puts some further information? > >> > > >> > Vincenzo > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Aug 17 22:49:29 2011 From: erlang@REDACTED (Anchise Inzaghi) Date: Wed, 17 Aug 2011 14:49:29 -0600 Subject: [erlang-questions] Process surviving disconnect In-Reply-To: References: , , <88dce0568e954d217577e7ffdc55fcd8@mail.nimqas.com>, , Message-ID: OK, I changed the spawn_link to spawn and subsequent monitor as you suggest: exactely the same behaviour. Alice terminates gracefully -> Bob gets informed and is alive. Alice terminates ungracefully -> Bob gets informed and is alive. I kill Alice's node -> Bob gets informed and is alive. I disconnect Alice from the network -> Bob joins his ancestors. So I will wait some days more here on erlang-questions and if nothing shows up here, I will file a bug and hopefully will be proven wrong. On Wed, 17 Aug 2011 22:23:06 +0200, Vincenzo Maggio wrote: > Use monitor(process, PID_TO_BE_MONITORED) after spawn. > BTW not having two PCs I simply tested your code killing alice and > guess what? bob survives! :) > >> Date: Wed, 17 Aug 2011 14:13:37 -0600 >> From: erlang@REDACTED >> To: maggio.vincenzo@REDACTED >> CC: erlang-questions@REDACTED >> Subject: RE: [erlang-questions] Process surviving disconnect >> >> Thank you for your quick answer. >> >> > BTW, before filing a bug, could you please substitute spawn_link >> with >> > spawn_monitor and remove the process_flag lines? It would be >> > interesting to understand if either bob dies on its own or it's >> > killed >> > by no more being able to communicate with alice. >> >> I am not sure how to replace spawn_link with spawn_monitor, as >> neither >> spawn_monitor/1 nor spawn_monitor/3 take a node parameter. >> How do I do that or how else can I get some more detailed >> information >> about Bob's sudden passing the Styx. >> >> >> >> On Wed, 17 Aug 2011 21:48:30 +0200, Vincenzo Maggio wrote: >> > Hello, >> > I think something is wrong here. >> >> Bob die of noconnection. >> > >> > This is printed by >> >> {'EXIT', Bob, Reason} -> >> >> io:format ("Bob died of ~p.~n", [Reason] ), >> > >> > So alice is in fact receiving bob last death cry :D and >> process_flag >> > translate it in a message instead of transmitting exit signal to >> > alice; I think this is ok from the point of view of Alice, so the >> > real >> > problem is that bob is dying (I know it's mundane, but I learned >> not >> > to make assumption). >> > Mmm, well I don't know if having no more a connection between the >> > process makes Erlang VM do some assumption of a virtual master >> node. >> > Well, if you want my opinion, I think that you should file a bug >> on >> > the Erlang bugs mailing list if no one comes up with a proper >> > explanation. >> > Even if what we're thinking is wrong and this is not a bug, in the >> > past I had a problem on node lookup and they resolved it. >> > These are my two cents, but if you can please let me know if there >> > are further updates 'cause it's a really interesting problem. >> > >> > BTW, before filing a bug, could you please substitute spawn_link >> with >> > spawn_monitor and remove the process_flag lines? It would be >> > interesting to understand if either bob dies on its own or it's >> > killed >> > by no more being able to communicate with alice. >> > >> > Vincenzo >> > >> >> Date: Wed, 17 Aug 2011 10:57:35 -0600 >> >> From: erlang@REDACTED >> >> To: maggio.vincenzo@REDACTED >> >> CC: erlang-questions@REDACTED >> >> Subject: RE: [erlang-questions] Process surviving disconnect >> >> >> >> Thank you very much Vincenzo. You affirmed my assertion that Bob >> >> should >> >> survive the disconnect. >> >> Nevertheless he dies. >> >> I will point out exactly what I do and maybe someone can spot >> the >> >> error >> >> in my code, my setup or my thinking and tell me what I am doing >> >> wrong. >> >> >> >> I start a node on gca.local: >> >> unroot@REDACTED:~$ erl -name 'bob@REDACTED' -setcookie 123 >> >> >> >> I start a node on usa.local: >> >> unroot@REDACTED:~$ erl -name 'alice@REDACTED' -setcookie 123 >> >> >> >> I start sasl on bob@REDACTED: >> >> (bob@REDACTED)1> application:start (sasl). >> >> >> >> I run alice:start/0 on alice@REDACTED: >> >> (alice@REDACTED)1> alice:start (). >> >> true >> >> >> >> I look for bob on bob@REDACTED and save its pid: >> >> (bob@REDACTED)2> whereis (bob). >> >> <0.65.0> >> >> (bob@REDACTED)3> Pid = whereis (bob). >> >> <0.65.0> >> >> >> >> I cut the network cable and wait a minute for the timeout. >> >> >> >> On alice@REDACTED I get the following output: >> >> =ERROR REPORT==== 17-Aug-2011::10:53:21 === >> >> ** Node 'bob@REDACTED' not responding ** >> >> ** Removing (timedout) connection ** >> >> Bob die of noconnection. >> >> >> >> Nice, Alice trapped Bob's death and reported it. I check for >> Alice: >> >> (alice@REDACTED)2> whereis (alice). >> >> <0.42.0> >> >> >> >> Alice is up and running. >> >> >> >> On bob@REDACTED I get the following output: >> >> =ERROR REPORT==== 17-Aug-2011::10:53:10 === >> >> ** Node 'alice@REDACTED' not responding ** >> >> ** Removing (timedout) connection ** >> >> >> >> But Bob is dead: >> >> (bob@REDACTED)4> whereis (bob). >> >> undefined >> >> (bob@REDACTED)5> is_process_alive (Pid). >> >> false >> >> >> >> I really do not understand what is happening. >> >> >> >> >> >> Thank you in advance >> >> >> >> Anchise >> >> >> >> Here goes the code I used: >> >> >> >> -module (alice). >> >> -compile (export_all). >> >> >> >> start () -> register (alice, spawn (fun init/0) ). >> >> >> >> stop () -> whereis (alice) ! stop. >> >> >> >> init () -> >> >> process_flag (trap_exit, true), >> >> Bob = spawn_link ('bob@REDACTED', bob, start, [self () ] ), >> >> loop (Bob). >> >> >> >> loop (Bob) -> >> >> receive >> >> stop -> ok; >> >> {'EXIT', Bob, Reason} -> >> >> io:format ("Bob died of ~p.~n", [Reason] ), >> >> loop (Bob); >> >> Msg -> >> >> io:format ("Alice received ~p.~n", [Msg] ), >> >> loop (Bob) >> >> end. >> >> >> >> >> >> -module (bob). >> >> -compile (export_all). >> >> >> >> start (Alice) -> >> >> process_flag (trap_exit, true), >> >> register (bob, self () ), >> >> loop (Alice). >> >> >> >> loop (Alice) -> >> >> receive >> >> stop -> ok; >> >> {'EXIT', Alice, Reason} -> >> >> io:format ("Alice died of ~p.~n", [Reason] ), >> >> loop (Alice); >> >> Msg -> >> >> io:format ("Bob received ~p.~n", [Msg] ), >> >> loop (Alice) >> >> end. >> >> >> >> >> >> On Wed, 17 Aug 2011 12:47:02 +0200, Vincenzo Maggio wrote: >> >> > Hello, >> >> > without further info a debug is rather difficult. >> >> > But let's try to at least start analysis of the problem: >> >> > >> >> >> - Has this something to do that I initially spawn Bob from >> the >> >> >> Alice >> >> >> node? >> >> > >> >> > Absolutely not: this would hit the very foundation of Erlang, >> >> process >> >> > referential transparency. When a process is started is a brand >> >> new, >> >> > clean entity (indeed, default process heap space is always the >> >> same >> >> > size!). >> >> > >> >> >> - How can I make Bob to survive a connection loss? >> >> > >> >> > Look above: it SHOULD survive. >> >> > >> >> > Can you please start SASL (application:start(sasl) from the >> shell) >> >> > and see if shell log puts some further information? >> >> > >> >> > Vincenzo >> >> >> From maggio.vincenzo@REDACTED Wed Aug 17 22:56:20 2011 From: maggio.vincenzo@REDACTED (Vincenzo Maggio) Date: Wed, 17 Aug 2011 22:56:20 +0200 Subject: [erlang-questions] Process surviving disconnect In-Reply-To: References: , , <88dce0568e954d217577e7ffdc55fcd8@mail.nimqas.com>, , , , Message-ID: Absolutely, maybe it's us but I notice that you, as me, has tried all the possibilities and in fact in any case but on link disconnection bob (and we think is right, BTW!!!) remains alive and in good health. I hope someone shows up with a clever explanation, if not it's perfectly understable to file a bug. > I disconnect Alice from the network -> Bob joins his ancestors. Ahahah, may it rest in peace with the ones he loved!!! > Date: Wed, 17 Aug 2011 14:49:29 -0600 > From: erlang@REDACTED > To: maggio.vincenzo@REDACTED > CC: erlang-questions@REDACTED > Subject: RE: [erlang-questions] Process surviving disconnect > > OK, I changed the spawn_link to spawn and subsequent monitor as you > suggest: exactely the same behaviour. > > Alice terminates gracefully -> Bob gets informed and is alive. > Alice terminates ungracefully -> Bob gets informed and is alive. > I kill Alice's node -> Bob gets informed and is alive. > I disconnect Alice from the network -> Bob joins his ancestors. > > So I will wait some days more here on erlang-questions and if nothing > shows up here, I will file a bug and hopefully will be proven wrong. > > On Wed, 17 Aug 2011 22:23:06 +0200, Vincenzo Maggio wrote: > > Use monitor(process, PID_TO_BE_MONITORED) after spawn. > > BTW not having two PCs I simply tested your code killing alice and > > guess what? bob survives! :) > > > >> Date: Wed, 17 Aug 2011 14:13:37 -0600 > >> From: erlang@REDACTED > >> To: maggio.vincenzo@REDACTED > >> CC: erlang-questions@REDACTED > >> Subject: RE: [erlang-questions] Process surviving disconnect > >> > >> Thank you for your quick answer. > >> > >> > BTW, before filing a bug, could you please substitute spawn_link > >> with > >> > spawn_monitor and remove the process_flag lines? It would be > >> > interesting to understand if either bob dies on its own or it's > >> > killed > >> > by no more being able to communicate with alice. > >> > >> I am not sure how to replace spawn_link with spawn_monitor, as > >> neither > >> spawn_monitor/1 nor spawn_monitor/3 take a node parameter. > >> How do I do that or how else can I get some more detailed > >> information > >> about Bob's sudden passing the Styx. > >> > >> > >> > >> On Wed, 17 Aug 2011 21:48:30 +0200, Vincenzo Maggio wrote: > >> > Hello, > >> > I think something is wrong here. > >> >> Bob die of noconnection. > >> > > >> > This is printed by > >> >> {'EXIT', Bob, Reason} -> > >> >> io:format ("Bob died of ~p.~n", [Reason] ), > >> > > >> > So alice is in fact receiving bob last death cry :D and > >> process_flag > >> > translate it in a message instead of transmitting exit signal to > >> > alice; I think this is ok from the point of view of Alice, so the > >> > real > >> > problem is that bob is dying (I know it's mundane, but I learned > >> not > >> > to make assumption). > >> > Mmm, well I don't know if having no more a connection between the > >> > process makes Erlang VM do some assumption of a virtual master > >> node. > >> > Well, if you want my opinion, I think that you should file a bug > >> on > >> > the Erlang bugs mailing list if no one comes up with a proper > >> > explanation. > >> > Even if what we're thinking is wrong and this is not a bug, in the > >> > past I had a problem on node lookup and they resolved it. > >> > These are my two cents, but if you can please let me know if there > >> > are further updates 'cause it's a really interesting problem. > >> > > >> > BTW, before filing a bug, could you please substitute spawn_link > >> with > >> > spawn_monitor and remove the process_flag lines? It would be > >> > interesting to understand if either bob dies on its own or it's > >> > killed > >> > by no more being able to communicate with alice. > >> > > >> > Vincenzo > >> > > >> >> Date: Wed, 17 Aug 2011 10:57:35 -0600 > >> >> From: erlang@REDACTED > >> >> To: maggio.vincenzo@REDACTED > >> >> CC: erlang-questions@REDACTED > >> >> Subject: RE: [erlang-questions] Process surviving disconnect > >> >> > >> >> Thank you very much Vincenzo. You affirmed my assertion that Bob > >> >> should > >> >> survive the disconnect. > >> >> Nevertheless he dies. > >> >> I will point out exactly what I do and maybe someone can spot > >> the > >> >> error > >> >> in my code, my setup or my thinking and tell me what I am doing > >> >> wrong. > >> >> > >> >> I start a node on gca.local: > >> >> unroot@REDACTED:~$ erl -name 'bob@REDACTED' -setcookie 123 > >> >> > >> >> I start a node on usa.local: > >> >> unroot@REDACTED:~$ erl -name 'alice@REDACTED' -setcookie 123 > >> >> > >> >> I start sasl on bob@REDACTED: > >> >> (bob@REDACTED)1> application:start (sasl). > >> >> > >> >> I run alice:start/0 on alice@REDACTED: > >> >> (alice@REDACTED)1> alice:start (). > >> >> true > >> >> > >> >> I look for bob on bob@REDACTED and save its pid: > >> >> (bob@REDACTED)2> whereis (bob). > >> >> <0.65.0> > >> >> (bob@REDACTED)3> Pid = whereis (bob). > >> >> <0.65.0> > >> >> > >> >> I cut the network cable and wait a minute for the timeout. > >> >> > >> >> On alice@REDACTED I get the following output: > >> >> =ERROR REPORT==== 17-Aug-2011::10:53:21 === > >> >> ** Node 'bob@REDACTED' not responding ** > >> >> ** Removing (timedout) connection ** > >> >> Bob die of noconnection. > >> >> > >> >> Nice, Alice trapped Bob's death and reported it. I check for > >> Alice: > >> >> (alice@REDACTED)2> whereis (alice). > >> >> <0.42.0> > >> >> > >> >> Alice is up and running. > >> >> > >> >> On bob@REDACTED I get the following output: > >> >> =ERROR REPORT==== 17-Aug-2011::10:53:10 === > >> >> ** Node 'alice@REDACTED' not responding ** > >> >> ** Removing (timedout) connection ** > >> >> > >> >> But Bob is dead: > >> >> (bob@REDACTED)4> whereis (bob). > >> >> undefined > >> >> (bob@REDACTED)5> is_process_alive (Pid). > >> >> false > >> >> > >> >> I really do not understand what is happening. > >> >> > >> >> > >> >> Thank you in advance > >> >> > >> >> Anchise > >> >> > >> >> Here goes the code I used: > >> >> > >> >> -module (alice). > >> >> -compile (export_all). > >> >> > >> >> start () -> register (alice, spawn (fun init/0) ). > >> >> > >> >> stop () -> whereis (alice) ! stop. > >> >> > >> >> init () -> > >> >> process_flag (trap_exit, true), > >> >> Bob = spawn_link ('bob@REDACTED', bob, start, [self () ] ), > >> >> loop (Bob). > >> >> > >> >> loop (Bob) -> > >> >> receive > >> >> stop -> ok; > >> >> {'EXIT', Bob, Reason} -> > >> >> io:format ("Bob died of ~p.~n", [Reason] ), > >> >> loop (Bob); > >> >> Msg -> > >> >> io:format ("Alice received ~p.~n", [Msg] ), > >> >> loop (Bob) > >> >> end. > >> >> > >> >> > >> >> -module (bob). > >> >> -compile (export_all). > >> >> > >> >> start (Alice) -> > >> >> process_flag (trap_exit, true), > >> >> register (bob, self () ), > >> >> loop (Alice). > >> >> > >> >> loop (Alice) -> > >> >> receive > >> >> stop -> ok; > >> >> {'EXIT', Alice, Reason} -> > >> >> io:format ("Alice died of ~p.~n", [Reason] ), > >> >> loop (Alice); > >> >> Msg -> > >> >> io:format ("Bob received ~p.~n", [Msg] ), > >> >> loop (Alice) > >> >> end. > >> >> > >> >> > >> >> On Wed, 17 Aug 2011 12:47:02 +0200, Vincenzo Maggio wrote: > >> >> > Hello, > >> >> > without further info a debug is rather difficult. > >> >> > But let's try to at least start analysis of the problem: > >> >> > > >> >> >> - Has this something to do that I initially spawn Bob from > >> the > >> >> >> Alice > >> >> >> node? > >> >> > > >> >> > Absolutely not: this would hit the very foundation of Erlang, > >> >> process > >> >> > referential transparency. When a process is started is a brand > >> >> new, > >> >> > clean entity (indeed, default process heap space is always the > >> >> same > >> >> > size!). > >> >> > > >> >> >> - How can I make Bob to survive a connection loss? > >> >> > > >> >> > Look above: it SHOULD survive. > >> >> > > >> >> > Can you please start SASL (application:start(sasl) from the > >> shell) > >> >> > and see if shell log puts some further information? > >> >> > > >> >> > Vincenzo > >> >> > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.adams@REDACTED Wed Aug 17 23:27:26 2011 From: christian.adams@REDACTED (Adams Christian) Date: Wed, 17 Aug 2011 23:27:26 +0200 Subject: [erlang-questions] ++ to select the head ( instead of just the tail ) In-Reply-To: <4191CF03-18C6-451E-8444-2271DC4615D4@gmail.com> References: <4191CF03-18C6-451E-8444-2271DC4615D4@gmail.com> Message-ID: <41CE140B-71F1-4AED-9848-CBE0B1E36A7F@sol-venetus.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Maybe dumb .. but why don't you just RL = lists:reverse(List). "654" ++ RTail = RL. Head = lists:reverse(RTail). Just my 2c .. CA Am 16.08.2011 um 13:23 schrieb James Churchman: > The ++ operator is not just a righthand side operator, but can in fact be a lefthand side operator in as far as I can tell just 1 case : > > When you know the head you wish to remove and getting the tail : > > Eg in the shell: > > FullList = "123456". > "123" ++ Tail = FullList. > Tail wil now be "456" > > However using it to get the head is not supported : > > FullList = "123456". > Head ++ "456" = FullList. > > gives : * 1: illegal pattern > > Understandably this second type involves traversing the entire list, where the first does not, but it would still be a nice addition to erlang. > > Was this not added as the first one expands out to simply [$1,$2,$3 | Tail] etc... but the second needs to expand to something more complex like an actual function the way list comprehensions do? > > James > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions CHRISTIAN ADAMS Software Systems Engineer Fon: +49 651 99554792 christian.adams@REDACTED SOL VENETUS Software GmbH Sebastian-Kneipp-Str. 2 - 86482 Aystetten - Germany Fon: +49 170 7762480 Fax: +49 821 4865786 Gesch?ftsf?hrer: Benjamin Hilbig Sitz der Gesellschaft: Aystetten Amtsgericht Augsburg, HRB 24559 USt-IdNr.: DE267014985 -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAk5MMj4ACgkQr81gVylJyzFG1ACeI47hG24NJC2/5xRm+ZQw2ura AqwAoOL6Ch259AymGgGTVh5p2Q513dxS =OQ1H -----END PGP SIGNATURE----- From ok@REDACTED Thu Aug 18 03:56:35 2011 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 18 Aug 2011 13:56:35 +1200 Subject: [erlang-questions] [semi-ot] introducing constants and include (that can include erlang code) in efene In-Reply-To: References: <1FB1B4D4-1323-4D31-831B-65DEA8737189@cs.otago.ac.nz> <48C0F209-E366-4504-8A37-A4D1716B0309@cs.otago.ac.nz> Message-ID: <730624DF-6F52-4EBD-B8CD-00F9C89C78FA@cs.otago.ac.nz> On 17/08/2011, at 11:09 PM, Mariano Guerra wrote: >> Concerning hygienic macros, the crucial thing is that identifiers in >> the expansion of a macro refer to those visible at its definition >> point, unless they come from arguments. This can be handled by >> renaming, which doesn't sound terribly hacky to me. > > do you think that something like collecting variable names from the > outer contexts and checking if there are collision names and renaming > the names that collide is the best way to do it? Yes. From erlang@REDACTED Thu Aug 18 05:25:37 2011 From: erlang@REDACTED (Anchise Inzaghi) Date: Wed, 17 Aug 2011 21:25:37 -0600 Subject: [erlang-questions] Process surviving disconnect In-Reply-To: References: , , <88dce0568e954d217577e7ffdc55fcd8@mail.nimqas.com>, , , , Message-ID: <406facfc6c11cc7c163371f961a2be82@mail.nimqas.com> Mystery solved: no erlang bug but a brain bug of mine. When I cut the network connection, Bob tries to write "Alice died of noconnection" to Alice's console which obviously is unreachable. So loop crashEs and Bob dies. On Wed, 17 Aug 2011 22:56:20 +0200, Vincenzo Maggio wrote: > Absolutely, maybe it's us but I notice that you, as me, has tried all > the possibilities and in fact in any case but on link disconnection > bob (and we think is right, BTW!!!) remains alive and in good health. > I hope someone shows up with a clever explanation, if not it's > perfectly understable to file a bug. > >> I disconnect Alice from the network -> Bob joins his ancestors. > Ahahah, may it rest in peace with the ones he loved!!! > > >> Date: Wed, 17 Aug 2011 14:49:29 -0600 >> From: erlang@REDACTED >> To: maggio.vincenzo@REDACTED >> CC: erlang-questions@REDACTED >> Subject: RE: [erlang-questions] Process surviving disconnect >> >> OK, I changed the spawn_link to spawn and subsequent monitor as you >> suggest: exactely the same behaviour. >> >> Alice terminates gracefully -> Bob gets informed and is alive. >> Alice terminates ungracefully -> Bob gets informed and is alive. >> I kill Alice's node -> Bob gets informed and is alive. >> I disconnect Alice from the network -> Bob joins his ancestors. >> >> So I will wait some days more here on erlang-questions and if >> nothing >> shows up here, I will file a bug and hopefully will be proven >> wrong. >> >> On Wed, 17 Aug 2011 22:23:06 +0200, Vincenzo Maggio wrote: >> > Use monitor(process, PID_TO_BE_MONITORED) after spawn. >> > BTW not having two PCs I simply tested your code killing alice and >> > guess what? bob survives! :) >> > >> >> Date: Wed, 17 Aug 2011 14:13:37 -0600 >> >> From: erlang@REDACTED >> >> To: maggio.vincenzo@REDACTED >> >> CC: erlang-questions@REDACTED >> >> Subject: RE: [erlang-questions] Process surviving disconnect >> >> >> >> Thank you for your quick answer. >> >> >> >> > BTW, before filing a bug, could you please substitute >> spawn_link >> >> with >> >> > spawn_monitor and remove the process_flag lines? It would be >> >> > interesting to understand if either bob dies on its own or it's >> >> > killed >> >> > by no more being able to communicate with alice. >> >> >> >> I am not sure how to replace spawn_link with spawn_monitor, as >> >> neither >> >> spawn_monitor/1 nor spawn_monitor/3 take a node parameter. >> >> How do I do that or how else can I get some more detailed >> >> information >> >> about Bob's sudden passing the Styx. >> >> >> >> >> >> >> >> On Wed, 17 Aug 2011 21:48:30 +0200, Vincenzo Maggio wrote: >> >> > Hello, >> >> > I think something is wrong here. >> >> >> Bob die of noconnection. >> >> > >> >> > This is printed by >> >> >> {'EXIT', Bob, Reason} -> >> >> >> io:format ("Bob died of ~p.~n", [Reason] ), >> >> > >> >> > So alice is in fact receiving bob last death cry :D and >> >> process_flag >> >> > translate it in a message instead of transmitting exit signal >> to >> >> > alice; I think this is ok from the point of view of Alice, so >> the >> >> > real >> >> > problem is that bob is dying (I know it's mundane, but I >> learned >> >> not >> >> > to make assumption). >> >> > Mmm, well I don't know if having no more a connection between >> the >> >> > process makes Erlang VM do some assumption of a virtual master >> >> node. >> >> > Well, if you want my opinion, I think that you should file a >> bug >> >> on >> >> > the Erlang bugs mailing list if no one comes up with a proper >> >> > explanation. >> >> > Even if what we're thinking is wrong and this is not a bug, in >> the >> >> > past I had a problem on node lookup and they resolved it. >> >> > These are my two cents, but if you can please let me know if >> there >> >> > are further updates 'cause it's a really interesting problem. >> >> > >> >> > BTW, before filing a bug, could you please substitute >> spawn_link >> >> with >> >> > spawn_monitor and remove the process_flag lines? It would be >> >> > interesting to understand if either bob dies on its own or it's >> >> > killed >> >> > by no more being able to communicate with alice. >> >> > >> >> > Vincenzo >> >> > >> >> >> Date: Wed, 17 Aug 2011 10:57:35 -0600 >> >> >> From: erlang@REDACTED >> >> >> To: maggio.vincenzo@REDACTED >> >> >> CC: erlang-questions@REDACTED >> >> >> Subject: RE: [erlang-questions] Process surviving disconnect >> >> >> >> >> >> Thank you very much Vincenzo. You affirmed my assertion that >> Bob >> >> >> should >> >> >> survive the disconnect. >> >> >> Nevertheless he dies. >> >> >> I will point out exactly what I do and maybe someone can spot >> >> the >> >> >> error >> >> >> in my code, my setup or my thinking and tell me what I am >> doing >> >> >> wrong. >> >> >> >> >> >> I start a node on gca.local: >> >> >> unroot@REDACTED:~$ erl -name 'bob@REDACTED' -setcookie 123 >> >> >> >> >> >> I start a node on usa.local: >> >> >> unroot@REDACTED:~$ erl -name 'alice@REDACTED' -setcookie 123 >> >> >> >> >> >> I start sasl on bob@REDACTED: >> >> >> (bob@REDACTED)1> application:start (sasl). >> >> >> >> >> >> I run alice:start/0 on alice@REDACTED: >> >> >> (alice@REDACTED)1> alice:start (). >> >> >> true >> >> >> >> >> >> I look for bob on bob@REDACTED and save its pid: >> >> >> (bob@REDACTED)2> whereis (bob). >> >> >> <0.65.0> >> >> >> (bob@REDACTED)3> Pid = whereis (bob). >> >> >> <0.65.0> >> >> >> >> >> >> I cut the network cable and wait a minute for the timeout. >> >> >> >> >> >> On alice@REDACTED I get the following output: >> >> >> =ERROR REPORT==== 17-Aug-2011::10:53:21 === >> >> >> ** Node 'bob@REDACTED' not responding ** >> >> >> ** Removing (timedout) connection ** >> >> >> Bob die of noconnection. >> >> >> >> >> >> Nice, Alice trapped Bob's death and reported it. I check for >> >> Alice: >> >> >> (alice@REDACTED)2> whereis (alice). >> >> >> <0.42.0> >> >> >> >> >> >> Alice is up and running. >> >> >> >> >> >> On bob@REDACTED I get the following output: >> >> >> =ERROR REPORT==== 17-Aug-2011::10:53:10 === >> >> >> ** Node 'alice@REDACTED' not responding ** >> >> >> ** Removing (timedout) connection ** >> >> >> >> >> >> But Bob is dead: >> >> >> (bob@REDACTED)4> whereis (bob). >> >> >> undefined >> >> >> (bob@REDACTED)5> is_process_alive (Pid). >> >> >> false >> >> >> >> >> >> I really do not understand what is happening. >> >> >> >> >> >> >> >> >> Thank you in advance >> >> >> >> >> >> Anchise >> >> >> >> >> >> Here goes the code I used: >> >> >> >> >> >> -module (alice). >> >> >> -compile (export_all). >> >> >> >> >> >> start () -> register (alice, spawn (fun init/0) ). >> >> >> >> >> >> stop () -> whereis (alice) ! stop. >> >> >> >> >> >> init () -> >> >> >> process_flag (trap_exit, true), >> >> >> Bob = spawn_link ('bob@REDACTED', bob, start, [self () ] ), >> >> >> loop (Bob). >> >> >> >> >> >> loop (Bob) -> >> >> >> receive >> >> >> stop -> ok; >> >> >> {'EXIT', Bob, Reason} -> >> >> >> io:format ("Bob died of ~p.~n", [Reason] ), >> >> >> loop (Bob); >> >> >> Msg -> >> >> >> io:format ("Alice received ~p.~n", [Msg] ), >> >> >> loop (Bob) >> >> >> end. >> >> >> >> >> >> >> >> >> -module (bob). >> >> >> -compile (export_all). >> >> >> >> >> >> start (Alice) -> >> >> >> process_flag (trap_exit, true), >> >> >> register (bob, self () ), >> >> >> loop (Alice). >> >> >> >> >> >> loop (Alice) -> >> >> >> receive >> >> >> stop -> ok; >> >> >> {'EXIT', Alice, Reason} -> >> >> >> io:format ("Alice died of ~p.~n", [Reason] ), >> >> >> loop (Alice); >> >> >> Msg -> >> >> >> io:format ("Bob received ~p.~n", [Msg] ), >> >> >> loop (Alice) >> >> >> end. >> >> >> >> >> >> >> >> >> On Wed, 17 Aug 2011 12:47:02 +0200, Vincenzo Maggio wrote: >> >> >> > Hello, >> >> >> > without further info a debug is rather difficult. >> >> >> > But let's try to at least start analysis of the problem: >> >> >> > >> >> >> >> - Has this something to do that I initially spawn Bob >> from >> >> the >> >> >> >> Alice >> >> >> >> node? >> >> >> > >> >> >> > Absolutely not: this would hit the very foundation of >> Erlang, >> >> >> process >> >> >> > referential transparency. When a process is started is a >> brand >> >> >> new, >> >> >> > clean entity (indeed, default process heap space is always >> the >> >> >> same >> >> >> > size!). >> >> >> > >> >> >> >> - How can I make Bob to survive a connection loss? >> >> >> > >> >> >> > Look above: it SHOULD survive. >> >> >> > >> >> >> > Can you please start SASL (application:start(sasl) from the >> >> shell) >> >> >> > and see if shell log puts some further information? >> >> >> > >> >> >> > Vincenzo >> >> >> >> >> >> From smith.winston.101@REDACTED Thu Aug 18 06:22:11 2011 From: smith.winston.101@REDACTED (Winston Smith) Date: Thu, 18 Aug 2011 00:22:11 -0400 Subject: [erlang-questions] inet_gethost fails with enomem on AVR32 (R14B03) Message-ID: I am trying to update my AVR32 system (an Atmel NGW100 development board) to use OTP R14B03 and a reltool (via "rebar generate") based release -- previous releases were hand-built. However, I'm having problems getting the node to start up. I have built R14B03 using Atmel's buildroot-v3.0.0 for NGW100 and aside from a couple of issues (avr32-linux-gcc internal errors due to -O3, fixed by specifying -O and a missing implementation of finite() in uclibc, fixed by passing -Dfinite=__finite in CFLAGS), I have been able to fire up the erlang prompt via the erl command. I am using the erl-xcomp-avr32-atmel-linux-gnu.conf xcomp file I supplied a while back and is included in the otp releases. Using the "reltool via rebar generate" to create an AVR32 based release of my app, I first ran into issues with the following error on the NGW100 platform: # bin/mynode console Exec: /home/avr32/mynode/erts-5.8.4/bin/erlexec -boot /home/avr32/mynode/releases/1/mynode -mode embedded -config /home/avr32/mynode/etc/app.config -args_file /home/avr32/mynode/etc/vm.args -- console Root: /home/avr32/mynode (no error logger present) error: "Error in process <0.2.0> with exit value: {badarg,[{erl_prim_loader,check_file_result,3},{init,patch_dir,2},{init,'-patch_path/2-lc$^0/1-0-',2},{init,eval_script,8},{init,do_boot,3}]} " {"init terminating in do_boot",{badarg,[{erl_prim_loader,check_file_result,3},{init,patch_dir,2},{init,'-patch_path/2-lc$^0/1-0-',2},{init,eval_script,8},{init,do_boot,3}]}} Crash dump was written to: erl_crash.dump init terminating in do_boot () I couldn't figure out this error (or the "no error logger present" since sasl is included in the release), but I was able to work around this by commenting out "+K true" and "+A 5" in the vm.args file, as follows: ## Enable kernel poll and a few async threads ## +K true ## +A 5 So now, I get a bit further, but it fails trying to run inet_gethost with an enomem error which I'm now stuck at: # bin/mynode console Exec: /home/avr32/mynode/erts-5.8.4/bin/erlexec -boot /home/avr32/mynode/releases/1/mynode -mode embedded -config /home/avr32/mynode/etc/app.config -args_file /home/avr32/mynode/etc/vm.args -- console Root: /home/avr32/mynode {error_logger,{{2011,8,17},{23,43,58}},"~s~n",["Error in process <0.16.0> with exit value: {enomem,[{erlang,open_port,[{spawn,\"inet_gethost 4 \"},[{packet,4},eof,binary]]},{inet_gethost_native,do_open_port,2},{inet_gethost_native,run_once,0}]}\n"]} Segmentation fault A wild guess would be that there's some kind of problem with the packet sizes being passed into inet_gethost from inet_gethost_native.erl is causing a memory allocation error ... I suppose it's also possible that my AVR32 system just doesn't have enough memory to run everything (the NGW100 seems to have about ~30MB of memory available when running Linux), but I'm not sure this is the case; I think there's a xcomp bug here. So my question(s) are: 1) Has anyone seen anything like this when cross compiling erlang? (or otherwise got any ideas!) 2) How do I build otp with debug mode enabled? In inet_gethost.c, I found the ERL_INET_GETHOST_DEBUG environment variable, but setting it to say 5 doesn't seem to do anything! 3) Is there a way to configure or disable the use of inet_gethost (i.e. is it even necessary?) I had to hack rebar to add a new reltool.config setting "root_dir" which is passed into reltool:eval_target_spec() as the 2nd argument, this lets me point "rebar generate" at the cross compiled OTP instance for AVR32. I'm still working on adding something to allow rebar to cross compile the port driver! It's tricky trying to debug this on an embedded platform, so any help is greatly appreciated! Thanks. Winston. From bengt.kleberg@REDACTED Thu Aug 18 09:25:57 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 18 Aug 2011 09:25:57 +0200 Subject: [erlang-questions] [semi-ot] introducing constants and include (that can include erlang code) in efene In-Reply-To: <730624DF-6F52-4EBD-B8CD-00F9C89C78FA@cs.otago.ac.nz> References: <1FB1B4D4-1323-4D31-831B-65DEA8737189@cs.otago.ac.nz> <48C0F209-E366-4504-8A37-A4D1716B0309@cs.otago.ac.nz> <730624DF-6F52-4EBD-B8CD-00F9C89C78FA@cs.otago.ac.nz> Message-ID: <1313652357.4576.8.camel@seasc1137> Greetings, In at least one case (http://www.vitanuova.com/inferno/papers/sh.html, "Lexical binding") the solution was to add a random string to all variables. This is not 100% safe but seemed to work. bengt On Thu, 2011-08-18 at 03:56 +0200, Richard O'Keefe wrote: > On 17/08/2011, at 11:09 PM, Mariano Guerra wrote: > >> Concerning hygienic macros, the crucial thing is that identifiers in > >> the expansion of a macro refer to those visible at its definition > >> point, unless they come from arguments. This can be handled by > >> renaming, which doesn't sound terribly hacky to me. > > > > do you think that something like collecting variable names from the > > outer contexts and checking if there are collision names and renaming > > the names that collide is the best way to do it? > > Yes. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bengt.kleberg@REDACTED Thu Aug 18 09:33:36 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 18 Aug 2011 09:33:36 +0200 Subject: [erlang-questions] ot: David May, parallel processing pioneer Message-ID: <1313652816.4576.11.camel@seasc1137> Greetings, I worked with Occam and the Transputer around 1986-87: http://www.reghardware.com/2011/08/18/heroes_of_tech_david_may/ bengt From raimo+erlang-questions@REDACTED Thu Aug 18 09:48:36 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 18 Aug 2011 09:48:36 +0200 Subject: [erlang-questions] ++ to select the head ( instead of just the tail ) In-Reply-To: <41CE140B-71F1-4AED-9848-CBE0B1E36A7F@sol-venetus.de> References: <4191CF03-18C6-451E-8444-2271DC4615D4@gmail.com> <41CE140B-71F1-4AED-9848-CBE0B1E36A7F@sol-venetus.de> Message-ID: <20110818074836.GA10094@erix.ericsson.se> On Wed, Aug 17, 2011 at 11:27:26PM +0200, Adams Christian wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Maybe dumb .. but why don't you just > > RL = lists:reverse(List). > "654" ++ RTail = RL. > Head = lists:reverse(RTail). > > Just my 2c .. That is how you do it, but if you have a huge List you will have to watch out since you build RTail just to become garbage. A solution that not builds the result in reverse as garbage would have to be a two-pass solution. First finding if there is a match and the resulting Head length. Then building the list forwards with non-tail recursion. That solution assumes that the stack for non-tail recursion grows exactly as much as the heap grows when building one result list element, and that the heap and stack shares memory block. This is the case for the current BEAM implementation but not for e.g Erjang. These issues are most likely why "654"++RTail is allowed since that is just syntactical sugar for [$6,$4,$5|RTail], really easy to define and implement, while Head++"456" is not allowed in a pattern since the implementation is if not non-trivial certainly inefficient. The only O(N) operation I know of allowed in a pattern match (guard) is length(List) and it should not get any partners in misery. I think the OP asked chiefly for symmetry reasons, and the answer might be that "654"++RTail and Head++"456" may look symmetrical but require very different implementations. > > CA > > Am 16.08.2011 um 13:23 schrieb James Churchman: > > > The ++ operator is not just a righthand side operator, but can in fact be a lefthand side operator in as far as I can tell just 1 case : > > > > When you know the head you wish to remove and getting the tail : > > > > Eg in the shell: > > > > FullList = "123456". > > "123" ++ Tail = FullList. > > Tail wil now be "456" > > > > However using it to get the head is not supported : > > > > FullList = "123456". > > Head ++ "456" = FullList. > > > > gives : * 1: illegal pattern > > > > Understandably this second type involves traversing the entire list, where the first does not, but it would still be a nice addition to erlang. > > > > Was this not added as the first one expands out to simply [$1,$2,$3 | Tail] etc... but the second needs to expand to something more complex like an actual function the way list comprehensions do? > > > > James > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > CHRISTIAN ADAMS > Software Systems Engineer > Fon: +49 651 99554792 > christian.adams@REDACTED > > SOL VENETUS Software GmbH > Sebastian-Kneipp-Str. 2 - 86482 Aystetten - Germany > Fon: +49 170 7762480 > Fax: +49 821 4865786 > Gesch?ftsf?hrer: Benjamin Hilbig > Sitz der Gesellschaft: Aystetten > Amtsgericht Augsburg, HRB 24559 > USt-IdNr.: DE267014985 > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) > Comment: GPGTools - http://gpgtools.org > > iEYEARECAAYFAk5MMj4ACgkQr81gVylJyzFG1ACeI47hG24NJC2/5xRm+ZQw2ura > AqwAoOL6Ch259AymGgGTVh5p2Q513dxS > =OQ1H > -----END PGP SIGNATURE----- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From alex.arnon@REDACTED Thu Aug 18 09:57:40 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Thu, 18 Aug 2011 10:57:40 +0300 Subject: [erlang-questions] ot: David May, parallel processing pioneer In-Reply-To: <1313652816.4576.11.camel@seasc1137> References: <1313652816.4576.11.camel@seasc1137> Message-ID: That was very nice, thank you! On Thu, Aug 18, 2011 at 10:33 AM, Bengt Kleberg wrote: > Greetings, > > I worked with Occam and the Transputer around 1986-87: > http://www.reghardware.com/2011/08/18/heroes_of_tech_david_may/ > > > bengt > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From luismarianoguerra@REDACTED Thu Aug 18 10:51:29 2011 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Thu, 18 Aug 2011 09:51:29 +0100 Subject: [erlang-questions] [semi-ot] introducing constants and include (that can include erlang code) in efene In-Reply-To: <1313652357.4576.8.camel@seasc1137> References: <1FB1B4D4-1323-4D31-831B-65DEA8737189@cs.otago.ac.nz> <48C0F209-E366-4504-8A37-A4D1716B0309@cs.otago.ac.nz> <730624DF-6F52-4EBD-B8CD-00F9C89C78FA@cs.otago.ac.nz> <1313652357.4576.8.camel@seasc1137> Message-ID: On Thu, Aug 18, 2011 at 8:25 AM, Bengt Kleberg wrote: > Greetings, > > In at least one case (http://www.vitanuova.com/inferno/papers/sh.html, > "Lexical binding") the solution was to add a random string to all > variables. This is not 100% safe but seemed to work. I thought about that solution, but I don't like to get really weird random names on error messages :) From bzamaere@REDACTED Thu Aug 18 14:55:00 2011 From: bzamaere@REDACTED (Bruce Zamaere) Date: Thu, 18 Aug 2011 14:55:00 +0200 Subject: [erlang-questions] SCTP server the OTP way Message-ID: Hi, I'm pretty new to OTP and gen_sctp. I found Serge's great tutorial on writing a Non-blocking TCP using OTP Principles. I'm wondering if there are best practices for building SCTP applications using OTP principles. I am trying to write an SCTP server that can accept many (1,000s) of persistent connections. Each of these should be able to communicate with the server over a number of SCTP streams (e.g. one stream for control and others for different types of data). Can anyone on the list give me pointers to how best to achieve this using OTP principles? /Bruce. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maggio.vincenzo@REDACTED Thu Aug 18 15:19:05 2011 From: maggio.vincenzo@REDACTED (Vincenzo Maggio) Date: Thu, 18 Aug 2011 15:19:05 +0200 Subject: [erlang-questions] Process surviving disconnect In-Reply-To: <406facfc6c11cc7c163371f961a2be82@mail.nimqas.com> References: , , <88dce0568e954d217577e7ffdc55fcd8@mail.nimqas.com>, , , , , , <406facfc6c11cc7c163371f961a2be82@mail.nimqas.com> Message-ID: Mmm, sorry but I don't grasp it: shouldn't bob write 'Alice died of no connection' on bob's node, which is in fact well alive? > Date: Wed, 17 Aug 2011 21:25:37 -0600 > From: erlang@REDACTED > To: maggio.vincenzo@REDACTED > CC: erlang-questions@REDACTED > Subject: RE: [erlang-questions] Process surviving disconnect > > Mystery solved: no erlang bug but a brain bug of mine. > > When I cut the network connection, Bob tries to write "Alice died of > noconnection" to Alice's console which obviously is unreachable. So loop > crashEs and Bob dies. > > On Wed, 17 Aug 2011 22:56:20 +0200, Vincenzo Maggio wrote: > > Absolutely, maybe it's us but I notice that you, as me, has tried all > > the possibilities and in fact in any case but on link disconnection > > bob (and we think is right, BTW!!!) remains alive and in good health. > > I hope someone shows up with a clever explanation, if not it's > > perfectly understable to file a bug. > > > >> I disconnect Alice from the network -> Bob joins his ancestors. > > Ahahah, may it rest in peace with the ones he loved!!! > > > > > >> Date: Wed, 17 Aug 2011 14:49:29 -0600 > >> From: erlang@REDACTED > >> To: maggio.vincenzo@REDACTED > >> CC: erlang-questions@REDACTED > >> Subject: RE: [erlang-questions] Process surviving disconnect > >> > >> OK, I changed the spawn_link to spawn and subsequent monitor as you > >> suggest: exactely the same behaviour. > >> > >> Alice terminates gracefully -> Bob gets informed and is alive. > >> Alice terminates ungracefully -> Bob gets informed and is alive. > >> I kill Alice's node -> Bob gets informed and is alive. > >> I disconnect Alice from the network -> Bob joins his ancestors. > >> > >> So I will wait some days more here on erlang-questions and if > >> nothing > >> shows up here, I will file a bug and hopefully will be proven > >> wrong. > >> > >> On Wed, 17 Aug 2011 22:23:06 +0200, Vincenzo Maggio wrote: > >> > Use monitor(process, PID_TO_BE_MONITORED) after spawn. > >> > BTW not having two PCs I simply tested your code killing alice and > >> > guess what? bob survives! :) > >> > > >> >> Date: Wed, 17 Aug 2011 14:13:37 -0600 > >> >> From: erlang@REDACTED > >> >> To: maggio.vincenzo@REDACTED > >> >> CC: erlang-questions@REDACTED > >> >> Subject: RE: [erlang-questions] Process surviving disconnect > >> >> > >> >> Thank you for your quick answer. > >> >> > >> >> > BTW, before filing a bug, could you please substitute > >> spawn_link > >> >> with > >> >> > spawn_monitor and remove the process_flag lines? It would be > >> >> > interesting to understand if either bob dies on its own or it's > >> >> > killed > >> >> > by no more being able to communicate with alice. > >> >> > >> >> I am not sure how to replace spawn_link with spawn_monitor, as > >> >> neither > >> >> spawn_monitor/1 nor spawn_monitor/3 take a node parameter. > >> >> How do I do that or how else can I get some more detailed > >> >> information > >> >> about Bob's sudden passing the Styx. > >> >> > >> >> > >> >> > >> >> On Wed, 17 Aug 2011 21:48:30 +0200, Vincenzo Maggio wrote: > >> >> > Hello, > >> >> > I think something is wrong here. > >> >> >> Bob die of noconnection. > >> >> > > >> >> > This is printed by > >> >> >> {'EXIT', Bob, Reason} -> > >> >> >> io:format ("Bob died of ~p.~n", [Reason] ), > >> >> > > >> >> > So alice is in fact receiving bob last death cry :D and > >> >> process_flag > >> >> > translate it in a message instead of transmitting exit signal > >> to > >> >> > alice; I think this is ok from the point of view of Alice, so > >> the > >> >> > real > >> >> > problem is that bob is dying (I know it's mundane, but I > >> learned > >> >> not > >> >> > to make assumption). > >> >> > Mmm, well I don't know if having no more a connection between > >> the > >> >> > process makes Erlang VM do some assumption of a virtual master > >> >> node. > >> >> > Well, if you want my opinion, I think that you should file a > >> bug > >> >> on > >> >> > the Erlang bugs mailing list if no one comes up with a proper > >> >> > explanation. > >> >> > Even if what we're thinking is wrong and this is not a bug, in > >> the > >> >> > past I had a problem on node lookup and they resolved it. > >> >> > These are my two cents, but if you can please let me know if > >> there > >> >> > are further updates 'cause it's a really interesting problem. > >> >> > > >> >> > BTW, before filing a bug, could you please substitute > >> spawn_link > >> >> with > >> >> > spawn_monitor and remove the process_flag lines? It would be > >> >> > interesting to understand if either bob dies on its own or it's > >> >> > killed > >> >> > by no more being able to communicate with alice. > >> >> > > >> >> > Vincenzo > >> >> > > >> >> >> Date: Wed, 17 Aug 2011 10:57:35 -0600 > >> >> >> From: erlang@REDACTED > >> >> >> To: maggio.vincenzo@REDACTED > >> >> >> CC: erlang-questions@REDACTED > >> >> >> Subject: RE: [erlang-questions] Process surviving disconnect > >> >> >> > >> >> >> Thank you very much Vincenzo. You affirmed my assertion that > >> Bob > >> >> >> should > >> >> >> survive the disconnect. > >> >> >> Nevertheless he dies. > >> >> >> I will point out exactly what I do and maybe someone can spot > >> >> the > >> >> >> error > >> >> >> in my code, my setup or my thinking and tell me what I am > >> doing > >> >> >> wrong. > >> >> >> > >> >> >> I start a node on gca.local: > >> >> >> unroot@REDACTED:~$ erl -name 'bob@REDACTED' -setcookie 123 > >> >> >> > >> >> >> I start a node on usa.local: > >> >> >> unroot@REDACTED:~$ erl -name 'alice@REDACTED' -setcookie 123 > >> >> >> > >> >> >> I start sasl on bob@REDACTED: > >> >> >> (bob@REDACTED)1> application:start (sasl). > >> >> >> > >> >> >> I run alice:start/0 on alice@REDACTED: > >> >> >> (alice@REDACTED)1> alice:start (). > >> >> >> true > >> >> >> > >> >> >> I look for bob on bob@REDACTED and save its pid: > >> >> >> (bob@REDACTED)2> whereis (bob). > >> >> >> <0.65.0> > >> >> >> (bob@REDACTED)3> Pid = whereis (bob). > >> >> >> <0.65.0> > >> >> >> > >> >> >> I cut the network cable and wait a minute for the timeout. > >> >> >> > >> >> >> On alice@REDACTED I get the following output: > >> >> >> =ERROR REPORT==== 17-Aug-2011::10:53:21 === > >> >> >> ** Node 'bob@REDACTED' not responding ** > >> >> >> ** Removing (timedout) connection ** > >> >> >> Bob die of noconnection. > >> >> >> > >> >> >> Nice, Alice trapped Bob's death and reported it. I check for > >> >> Alice: > >> >> >> (alice@REDACTED)2> whereis (alice). > >> >> >> <0.42.0> > >> >> >> > >> >> >> Alice is up and running. > >> >> >> > >> >> >> On bob@REDACTED I get the following output: > >> >> >> =ERROR REPORT==== 17-Aug-2011::10:53:10 === > >> >> >> ** Node 'alice@REDACTED' not responding ** > >> >> >> ** Removing (timedout) connection ** > >> >> >> > >> >> >> But Bob is dead: > >> >> >> (bob@REDACTED)4> whereis (bob). > >> >> >> undefined > >> >> >> (bob@REDACTED)5> is_process_alive (Pid). > >> >> >> false > >> >> >> > >> >> >> I really do not understand what is happening. > >> >> >> > >> >> >> > >> >> >> Thank you in advance > >> >> >> > >> >> >> Anchise > >> >> >> > >> >> >> Here goes the code I used: > >> >> >> > >> >> >> -module (alice). > >> >> >> -compile (export_all). > >> >> >> > >> >> >> start () -> register (alice, spawn (fun init/0) ). > >> >> >> > >> >> >> stop () -> whereis (alice) ! stop. > >> >> >> > >> >> >> init () -> > >> >> >> process_flag (trap_exit, true), > >> >> >> Bob = spawn_link ('bob@REDACTED', bob, start, [self () ] ), > >> >> >> loop (Bob). > >> >> >> > >> >> >> loop (Bob) -> > >> >> >> receive > >> >> >> stop -> ok; > >> >> >> {'EXIT', Bob, Reason} -> > >> >> >> io:format ("Bob died of ~p.~n", [Reason] ), > >> >> >> loop (Bob); > >> >> >> Msg -> > >> >> >> io:format ("Alice received ~p.~n", [Msg] ), > >> >> >> loop (Bob) > >> >> >> end. > >> >> >> > >> >> >> > >> >> >> -module (bob). > >> >> >> -compile (export_all). > >> >> >> > >> >> >> start (Alice) -> > >> >> >> process_flag (trap_exit, true), > >> >> >> register (bob, self () ), > >> >> >> loop (Alice). > >> >> >> > >> >> >> loop (Alice) -> > >> >> >> receive > >> >> >> stop -> ok; > >> >> >> {'EXIT', Alice, Reason} -> > >> >> >> io:format ("Alice died of ~p.~n", [Reason] ), > >> >> >> loop (Alice); > >> >> >> Msg -> > >> >> >> io:format ("Bob received ~p.~n", [Msg] ), > >> >> >> loop (Alice) > >> >> >> end. > >> >> >> > >> >> >> > >> >> >> On Wed, 17 Aug 2011 12:47:02 +0200, Vincenzo Maggio wrote: > >> >> >> > Hello, > >> >> >> > without further info a debug is rather difficult. > >> >> >> > But let's try to at least start analysis of the problem: > >> >> >> > > >> >> >> >> - Has this something to do that I initially spawn Bob > >> from > >> >> the > >> >> >> >> Alice > >> >> >> >> node? > >> >> >> > > >> >> >> > Absolutely not: this would hit the very foundation of > >> Erlang, > >> >> >> process > >> >> >> > referential transparency. When a process is started is a > >> brand > >> >> >> new, > >> >> >> > clean entity (indeed, default process heap space is always > >> the > >> >> >> same > >> >> >> > size!). > >> >> >> > > >> >> >> >> - How can I make Bob to survive a connection loss? > >> >> >> > > >> >> >> > Look above: it SHOULD survive. > >> >> >> > > >> >> >> > Can you please start SASL (application:start(sasl) from the > >> >> shell) > >> >> >> > and see if shell log puts some further information? > >> >> >> > > >> >> >> > Vincenzo > >> >> >> > >> >> > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From oribrost@REDACTED Thu Aug 18 17:14:53 2011 From: oribrost@REDACTED (ori brost) Date: Thu, 18 Aug 2011 18:14:53 +0300 Subject: [erlang-questions] Handling non http in mochiweb In-Reply-To: <20110817063230.GA27237@jakstys.lt> References: <20110817063230.GA27237@jakstys.lt> Message-ID: I will describe my exact problem, maybe there are other solutions i did not think of. Our program provides a websocket service over a port. We wish to allow people without a websocket supporting browser to use it so we use a flash object to do websocket when the browser can't. In order for this to work, the flash object needs to get a flash policy file. Flash forces the file to be in one of two places: - port 843 (flash hard coded) - the port of the ws service Note that the flash protocol is NOT HTTP based. Originally we worked with 843, but we then decided to put out server behind an Amazon ELB. Amazon ELB does not allow port forwarding for ports below 1024 which are not 80 or 443, so our only choice was to implement the flash policy file server in the same port via a patch to mochiweb (https://github.com/nivertech/mochiweb/tree/ori_flash_170811). We also thought of putting the websocket service behind an haproxy instead of ELB, but ELB is better because it is also hardware based unlike haproxy which is only software. Do you see any other solution for this? 2011/8/17 Motiejus Jak?tys : > On Tue, Aug 16, 2011 at 02:45:03PM -0700, Ryan Huffman wrote: >> Does it have to be the same port? ?You could use gen_tcp to listen to >> connections on a different port and handle the traffic however you would >> like. >> > > Sometimes people are forced to use http proxy for http traffic (80 and > 443) (an example is university of Glasgow), therefore you can't send > arbitrary traffic through port 80 -- it just doesn't work. > > However, you can use port 443 for arbitrary TCP and be quite sure it > will work anywhere. (I used openvpn over tcp 443) > > Motiejus > From steve@REDACTED Thu Aug 18 17:29:00 2011 From: steve@REDACTED (Steve Strong) Date: Thu, 18 Aug 2011 17:29:00 +0200 Subject: [erlang-questions] Network partitions and Quorom Message-ID: <213D0AB9-AE20-43C1-AC1D-EC98567EE6E4@srstrong.com> Hi, After a lot of googling, I haven't found a clear answer to something that I'd have thought was a solved problem in the Erlang community - I've got an application that needs to run on one and only one of N potential hosts, where N is fixed. I'm hoping that there's a standard library I can use for this, but I've not found one yet. Currently, we are using a simple distributed app as built into the Erlang release but I don't believe this will handle network partitions in the way that I need (i.e., if node A becomes partitioned from B & C, I could end up with my app running on both A and one of B & C). Use of gen_leader is a possibility, although I'm unclear how (if) it handles network partitions at the moment - if it does have a concept of quorum then it will be ideal, but a quick google and read of the code left me unsure if it handled partitions or not. Any advice on either gen_leader or other options would be much appreciated. Cheers, Steve From g@REDACTED Thu Aug 18 17:29:11 2011 From: g@REDACTED (Garrett Smith) Date: Thu, 18 Aug 2011 10:29:11 -0500 Subject: [erlang-questions] ot: David May, parallel processing pioneer In-Reply-To: <1313652816.4576.11.camel@seasc1137> References: <1313652816.4576.11.camel@seasc1137> Message-ID: On Thu, Aug 18, 2011 at 2:33 AM, Bengt Kleberg wrote: > > I worked with Occam and the Transputer around 1986-87: > http://www.reghardware.com/2011/08/18/heroes_of_tech_david_may/ > Very interesting indeed. It's fascinating how the economics of CPUs and RAM shaped software design. Thanks for the great history! Garrett From magnus.klaar@REDACTED Thu Aug 18 17:48:41 2011 From: magnus.klaar@REDACTED (Magnus Klaar) Date: Thu, 18 Aug 2011 17:48:41 +0200 Subject: [erlang-questions] Handling non http in mochiweb In-Reply-To: References: <20110817063230.GA27237@jakstys.lt> Message-ID: 2011/8/18 ori brost > I will describe my exact problem, maybe there are other solutions i > did not think of. > Our program provides a websocket service over a port. We wish to allow > people without a websocket supporting browser to use it so we use a > flash object to do websocket when the browser can't. In order for this > to work, the flash object needs to get a flash policy file. Flash > forces the file to be in one of two places: > - port 843 (flash hard coded) > - the port of the ws service > Note that the flash protocol is NOT HTTP based. > Originally we worked with 843, but we then decided to put out server > behind an Amazon ELB. Amazon ELB does not allow port forwarding for > ports below 1024 which are not 80 or 443, so our only choice was to > implement the flash policy file server in the same port via a patch to > mochiweb (https://github.com/nivertech/mochiweb/tree/ori_flash_170811). > We also thought of putting the websocket service behind an haproxy > instead of ELB, but ELB is better because it is also hardware based > unlike haproxy which is only software. > > Do you see any other solution for this? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Hi! This is a shameless plug for cowboy, putting this aside, I think it you would end up with a solution similar to this if you were to patch mochiweb suit this usecase. I wrote a blog post on getting started using cowboy for non-http protocols yesterday because of a question on IRC wrt. to finding a reusable acceptor/handler pool for connection oriented servers. You will find it here: http://mklaar.posterous.com/accepting-and-handling-tcp-connections-with-c Now, you would be able to write a protocol module such as flash_policy_or_http_protocol that read N bytes from the socket, if they match an HTTP method, you can call the cowboy_http_protocol:init/4 and handle it as an HTTP request. If they match what you expect to find at the head of a flash policy request you can call flash_policy_protocol:init/4. The only change I anticipate that you would have to make to cowboy itself would be to add an option for injecting an intial buffer into the cowboy_http_protocol state to ensure that erlang_decode:packet doesn't start reading in the middle of the input. (I failed at the internet, I'm sorry if you receive duplicates of this email) MVH Magnus Klaar -------------- next part -------------- An HTML attachment was scrubbed... URL: From norton@REDACTED Thu Aug 18 17:54:05 2011 From: norton@REDACTED (Joseph Norton) Date: Fri, 19 Aug 2011 00:54:05 +0900 Subject: [erlang-questions] Network partitions and Quorom In-Reply-To: <213D0AB9-AE20-43C1-AC1D-EC98567EE6E4@srstrong.com> References: <213D0AB9-AE20-43C1-AC1D-EC98567EE6E4@srstrong.com> Message-ID: <12916D95-E1C1-4637-B0EE-054306BED549@lovely.email.ne.jp> There is an Erlang application bundled as a part of Hibari called partition-detector on GitHub. https://github.com/hibari/partition-detector There is some documentation for this particular application embedded in this document on GitHub. http://hibari.github.com/hibari-doc/hibari-sysadmin-guide.en.html#partition-detector This application was originally developed and deployed for Mnesia-based systems. If this looks helpful to you, contact me off list for further details. thanks, Joseph Norton norton@REDACTED On Aug 19, 2011, at 12:29 AM, Steve Strong wrote: > Hi, > > After a lot of googling, I haven't found a clear answer to something that I'd have thought was a solved problem in the Erlang community - I've got an application that needs to run on one and only one of N potential hosts, where N is fixed. I'm hoping that there's a standard library I can use for this, but I've not found one yet. > > Currently, we are using a simple distributed app as built into the Erlang release but I don't believe this will handle network partitions in the way that I need (i.e., if node A becomes partitioned from B & C, I could end up with my app running on both A and one of B & C). > > Use of gen_leader is a possibility, although I'm unclear how (if) it handles network partitions at the moment - if it does have a concept of quorum then it will be ideal, but a quick google and read of the code left me unsure if it handled partitions or not. > > Any advice on either gen_leader or other options would be much appreciated. > > Cheers, > > Steve > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bob@REDACTED Thu Aug 18 17:54:47 2011 From: bob@REDACTED (Bob Ippolito) Date: Thu, 18 Aug 2011 08:54:47 -0700 Subject: [erlang-questions] Handling non http in mochiweb In-Reply-To: References: <20110817063230.GA27237@jakstys.lt> Message-ID: On Thu, Aug 18, 2011 at 8:48 AM, Magnus Klaar wrote: > 2011/8/18 ori brost >> >> I will describe my exact problem, maybe there are other solutions i >> did not think of. >> Our program provides a websocket service over a port. We wish to allow >> people without a websocket supporting browser to use it so we use a >> flash object to do websocket when the browser can't. In order for this >> to work, the flash object needs to get a flash policy file. Flash >> forces the file to be in one of two places: >> - port 843 (flash hard coded) >> - the port of the ws service >> Note that the flash protocol is NOT HTTP based. >> Originally we worked with 843, but we then decided to put out server >> behind an Amazon ELB. Amazon ELB does not allow port forwarding for >> ports below 1024 which are not 80 or 443, so our only choice was to >> implement the flash policy file server in the same port via a patch to >> mochiweb (https://github.com/nivertech/mochiweb/tree/ori_flash_170811). >> We also thought of putting the websocket service behind an haproxy >> instead of ELB, but ELB is better because it is also hardware based >> unlike haproxy which is only software. >> >> Do you see any other solution for this? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > Hi! > This is a shameless plug for cowboy, putting this aside, I think it you > would end up with a solution similar to this if you were to patch mochiweb > suit this usecase. > I wrote a blog post on getting started using cowboy for non-http protocols > yesterday because of a question on IRC wrt. to finding a reusable > acceptor/handler pool for connection oriented servers. You will find it > here:?http://mklaar.posterous.com/accepting-and-handling-tcp-connections-with-c > Now, you would be able to write a protocol module such as > flash_policy_or_http_protocol that read N bytes from the socket, if they > match an HTTP method, you can call the cowboy_http_protocol:init/4 and > handle it as an HTTP request. If they match what you expect to find at the > head of a flash policy request you can call flash_policy_protocol:init/4. > The only change I anticipate that you would have to make to cowboy itself > would be to add an option for injecting an intial buffer into the > cowboy_http_protocol state to ensure that erlang_decode:packet doesn't start > reading in the middle of the input. > (I failed at the internet, I'm sorry if you receive duplicates of this > email) FWIW, mochiweb also has a reusable acceptor/handler pool implementation with an unsurprising name: mochiweb_socket_server. mochiweb_http is just a handler that sits on top of this facility. In either case, if you need to stuff something back onto the socket buffer, there's an undocumented gen_tcp:unrecv/2 that would work if you can't find an elegant and supported way to do it. -bob From magnus.klaar@REDACTED Thu Aug 18 18:23:03 2011 From: magnus.klaar@REDACTED (Magnus Klaar) Date: Thu, 18 Aug 2011 18:23:03 +0200 Subject: [erlang-questions] Handling non http in mochiweb In-Reply-To: References: <20110817063230.GA27237@jakstys.lt> Message-ID: On Thu, Aug 18, 2011 at 5:54 PM, Bob Ippolito wrote: > On Thu, Aug 18, 2011 at 8:48 AM, Magnus Klaar > wrote: > > 2011/8/18 ori brost > >> > >> I will describe my exact problem, maybe there are other solutions i > >> did not think of. > >> Our program provides a websocket service over a port. We wish to allow > >> people without a websocket supporting browser to use it so we use a > >> flash object to do websocket when the browser can't. In order for this > >> to work, the flash object needs to get a flash policy file. Flash > >> forces the file to be in one of two places: > >> - port 843 (flash hard coded) > >> - the port of the ws service > >> Note that the flash protocol is NOT HTTP based. > >> Originally we worked with 843, but we then decided to put out server > >> behind an Amazon ELB. Amazon ELB does not allow port forwarding for > >> ports below 1024 which are not 80 or 443, so our only choice was to > >> implement the flash policy file server in the same port via a patch to > >> mochiweb (https://github.com/nivertech/mochiweb/tree/ori_flash_170811). > >> We also thought of putting the websocket service behind an haproxy > >> instead of ELB, but ELB is better because it is also hardware based > >> unlike haproxy which is only software. > >> > >> Do you see any other solution for this? > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > Hi! > > This is a shameless plug for cowboy, putting this aside, I think it you > > would end up with a solution similar to this if you were to patch > mochiweb > > suit this usecase. > > I wrote a blog post on getting started using cowboy for non-http > protocols > > yesterday because of a question on IRC wrt. to finding a reusable > > acceptor/handler pool for connection oriented servers. You will find it > > here: > http://mklaar.posterous.com/accepting-and-handling-tcp-connections-with-c > > Now, you would be able to write a protocol module such as > > flash_policy_or_http_protocol that read N bytes from the socket, if they > > match an HTTP method, you can call the cowboy_http_protocol:init/4 and > > handle it as an HTTP request. If they match what you expect to find at > the > > head of a flash policy request you can call flash_policy_protocol:init/4. > > The only change I anticipate that you would have to make to cowboy itself > > would be to add an option for injecting an intial buffer into the > > cowboy_http_protocol state to ensure that erlang_decode:packet doesn't > start > > reading in the middle of the input. > > (I failed at the internet, I'm sorry if you receive duplicates of this > > email) > > FWIW, mochiweb also has a reusable acceptor/handler pool > implementation with an unsurprising name: mochiweb_socket_server. > mochiweb_http is just a handler that sits on top of this facility. > > In either case, if you need to stuff something back onto the socket > buffer, there's an undocumented gen_tcp:unrecv/2 that would work if > you can't find an elegant and supported way to do it. > > -bob > Hi! Nice, that solves this problem then. I only took a quick look at mochiweb_socket_server and mochiweb_http, but it looks like there is nothing stopping you from using the same module structure as I suggested earlier. Replace Protocol:init/2 in my post with Protocol:loop/2 and you're done. Making changes to either one of the flash_policy or mochiweb-http module will land you with at least one module that is "bloated" or hard to reuse. MVH Magnus Klaar -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew@REDACTED Thu Aug 18 18:55:42 2011 From: andrew@REDACTED (Andrew Thompson) Date: Thu, 18 Aug 2011 12:55:42 -0400 Subject: [erlang-questions] Network partitions and Quorom In-Reply-To: <213D0AB9-AE20-43C1-AC1D-EC98567EE6E4@srstrong.com> References: <213D0AB9-AE20-43C1-AC1D-EC98567EE6E4@srstrong.com> Message-ID: <20110818165542.GA31180@hijacked.us> On Thu, Aug 18, 2011 at 05:29:00PM +0200, Steve Strong wrote: > Hi, > > After a lot of googling, I haven't found a clear answer to something that I'd have thought was a solved problem in the Erlang community - I've got an application that needs to run on one and only one of N potential hosts, where N is fixed. I'm hoping that there's a standard library I can use for this, but I've not found one yet. > > Currently, we are using a simple distributed app as built into the Erlang release but I don't believe this will handle network partitions in the way that I need (i.e., if node A becomes partitioned from B & C, I could end up with my app running on both A and one of B & C). > > Use of gen_leader is a possibility, although I'm unclear how (if) it handles network partitions at the moment - if it does have a concept of quorum then it will be ideal, but a quick google and read of the code left me unsure if it handled partitions or not. > > Any advice on either gen_leader or other options would be much appreciated. > Basho recently ran into some partition tolerance issues with gen_leader and I spent a week figuring out the cause(s). I have several patches for gen_leader which make it *much* more resiliant in the case of partitioned networks. I've been meaning to publish the changes when I have a chance, let me know if you're interested. Andrew From erlang@REDACTED Thu Aug 18 19:36:37 2011 From: erlang@REDACTED (Anchise Inzaghi) Date: Thu, 18 Aug 2011 11:36:37 -0600 Subject: [erlang-questions] Process surviving disconnect In-Reply-To: References: , , <88dce0568e954d217577e7ffdc55fcd8@mail.nimqas.com>, , , , , , <406facfc6c11cc7c163371f961a2be82@mail.nimqas.com> Message-ID: <40c7dba97e4dc9457cd261a2708ee31f@mail.nimqas.com> By default a process writes to the shell which started it. Everything works as intended and expected if I change bob.erl DELETE: io:format ("Alice died of ~p.~n", [Reason] ), REPLACE: io:format (user, "Alice died of ~p.~n", [Reason] ), This way, Bob writes to his own shell and all is well. On Thu, 18 Aug 2011 15:19:05 +0200, Vincenzo Maggio wrote: > Mmm, sorry but I don't grasp it: shouldn't bob write 'Alice died of > no connection' on bob's node, which is in fact well alive? > >> Date: Wed, 17 Aug 2011 21:25:37 -0600 >> From: erlang@REDACTED >> To: maggio.vincenzo@REDACTED >> CC: erlang-questions@REDACTED >> Subject: RE: [erlang-questions] Process surviving disconnect >> >> Mystery solved: no erlang bug but a brain bug of mine. >> >> When I cut the network connection, Bob tries to write "Alice died >> of >> noconnection" to Alice's console which obviously is unreachable. So >> loop >> crashEs and Bob dies. >> >> On Wed, 17 Aug 2011 22:56:20 +0200, Vincenzo Maggio wrote: >> > Absolutely, maybe it's us but I notice that you, as me, has tried >> all >> > the possibilities and in fact in any case but on link >> disconnection >> > bob (and we think is right, BTW!!!) remains alive and in good >> health. >> > I hope someone shows up with a clever explanation, if not it's >> > perfectly understable to file a bug. >> > >> >> I disconnect Alice from the network -> Bob joins his ancestors. >> > Ahahah, may it rest in peace with the ones he loved!!! >> > >> > >> >> Date: Wed, 17 Aug 2011 14:49:29 -0600 >> >> From: erlang@REDACTED >> >> To: maggio.vincenzo@REDACTED >> >> CC: erlang-questions@REDACTED >> >> Subject: RE: [erlang-questions] Process surviving disconnect >> >> >> >> OK, I changed the spawn_link to spawn and subsequent monitor as >> you >> >> suggest: exactely the same behaviour. >> >> >> >> Alice terminates gracefully -> Bob gets informed and is alive. >> >> Alice terminates ungracefully -> Bob gets informed and is alive. >> >> I kill Alice's node -> Bob gets informed and is alive. >> >> I disconnect Alice from the network -> Bob joins his ancestors. >> >> >> >> So I will wait some days more here on erlang-questions and if >> >> nothing >> >> shows up here, I will file a bug and hopefully will be proven >> >> wrong. >> >> >> >> On Wed, 17 Aug 2011 22:23:06 +0200, Vincenzo Maggio wrote: >> >> > Use monitor(process, PID_TO_BE_MONITORED) after spawn. >> >> > BTW not having two PCs I simply tested your code killing alice >> and >> >> > guess what? bob survives! :) >> >> > >> >> >> Date: Wed, 17 Aug 2011 14:13:37 -0600 >> >> >> From: erlang@REDACTED >> >> >> To: maggio.vincenzo@REDACTED >> >> >> CC: erlang-questions@REDACTED >> >> >> Subject: RE: [erlang-questions] Process surviving disconnect >> >> >> >> >> >> Thank you for your quick answer. >> >> >> >> >> >> > BTW, before filing a bug, could you please substitute >> >> spawn_link >> >> >> with >> >> >> > spawn_monitor and remove the process_flag lines? It would be >> >> >> > interesting to understand if either bob dies on its own or >> it's >> >> >> > killed >> >> >> > by no more being able to communicate with alice. >> >> >> >> >> >> I am not sure how to replace spawn_link with spawn_monitor, >> as >> >> >> neither >> >> >> spawn_monitor/1 nor spawn_monitor/3 take a node parameter. >> >> >> How do I do that or how else can I get some more detailed >> >> >> information >> >> >> about Bob's sudden passing the Styx. >> >> >> >> >> >> >> >> >> >> >> >> On Wed, 17 Aug 2011 21:48:30 +0200, Vincenzo Maggio wrote: >> >> >> > Hello, >> >> >> > I think something is wrong here. >> >> >> >> Bob die of noconnection. >> >> >> > >> >> >> > This is printed by >> >> >> >> {'EXIT', Bob, Reason} -> >> >> >> >> io:format ("Bob died of ~p.~n", [Reason] ), >> >> >> > >> >> >> > So alice is in fact receiving bob last death cry :D and >> >> >> process_flag >> >> >> > translate it in a message instead of transmitting exit >> signal >> >> to >> >> >> > alice; I think this is ok from the point of view of Alice, >> so >> >> the >> >> >> > real >> >> >> > problem is that bob is dying (I know it's mundane, but I >> >> learned >> >> >> not >> >> >> > to make assumption). >> >> >> > Mmm, well I don't know if having no more a connection >> between >> >> the >> >> >> > process makes Erlang VM do some assumption of a virtual >> master >> >> >> node. >> >> >> > Well, if you want my opinion, I think that you should file a >> >> bug >> >> >> on >> >> >> > the Erlang bugs mailing list if no one comes up with a >> proper >> >> >> > explanation. >> >> >> > Even if what we're thinking is wrong and this is not a bug, >> in >> >> the >> >> >> > past I had a problem on node lookup and they resolved it. >> >> >> > These are my two cents, but if you can please let me know if >> >> there >> >> >> > are further updates 'cause it's a really interesting >> problem. >> >> >> > >> >> >> > BTW, before filing a bug, could you please substitute >> >> spawn_link >> >> >> with >> >> >> > spawn_monitor and remove the process_flag lines? It would be >> >> >> > interesting to understand if either bob dies on its own or >> it's >> >> >> > killed >> >> >> > by no more being able to communicate with alice. >> >> >> > >> >> >> > Vincenzo >> >> >> > >> >> >> >> Date: Wed, 17 Aug 2011 10:57:35 -0600 >> >> >> >> From: erlang@REDACTED >> >> >> >> To: maggio.vincenzo@REDACTED >> >> >> >> CC: erlang-questions@REDACTED >> >> >> >> Subject: RE: [erlang-questions] Process surviving >> disconnect >> >> >> >> >> >> >> >> Thank you very much Vincenzo. You affirmed my assertion >> that >> >> Bob >> >> >> >> should >> >> >> >> survive the disconnect. >> >> >> >> Nevertheless he dies. >> >> >> >> I will point out exactly what I do and maybe someone can >> spot >> >> >> the >> >> >> >> error >> >> >> >> in my code, my setup or my thinking and tell me what I am >> >> doing >> >> >> >> wrong. >> >> >> >> >> >> >> >> I start a node on gca.local: >> >> >> >> unroot@REDACTED:~$ erl -name 'bob@REDACTED' -setcookie 123 >> >> >> >> >> >> >> >> I start a node on usa.local: >> >> >> >> unroot@REDACTED:~$ erl -name 'alice@REDACTED' -setcookie 123 >> >> >> >> >> >> >> >> I start sasl on bob@REDACTED: >> >> >> >> (bob@REDACTED)1> application:start (sasl). >> >> >> >> >> >> >> >> I run alice:start/0 on alice@REDACTED: >> >> >> >> (alice@REDACTED)1> alice:start (). >> >> >> >> true >> >> >> >> >> >> >> >> I look for bob on bob@REDACTED and save its pid: >> >> >> >> (bob@REDACTED)2> whereis (bob). >> >> >> >> <0.65.0> >> >> >> >> (bob@REDACTED)3> Pid = whereis (bob). >> >> >> >> <0.65.0> >> >> >> >> >> >> >> >> I cut the network cable and wait a minute for the timeout. >> >> >> >> >> >> >> >> On alice@REDACTED I get the following output: >> >> >> >> =ERROR REPORT==== 17-Aug-2011::10:53:21 === >> >> >> >> ** Node 'bob@REDACTED' not responding ** >> >> >> >> ** Removing (timedout) connection ** >> >> >> >> Bob die of noconnection. >> >> >> >> >> >> >> >> Nice, Alice trapped Bob's death and reported it. I check >> for >> >> >> Alice: >> >> >> >> (alice@REDACTED)2> whereis (alice). >> >> >> >> <0.42.0> >> >> >> >> >> >> >> >> Alice is up and running. >> >> >> >> >> >> >> >> On bob@REDACTED I get the following output: >> >> >> >> =ERROR REPORT==== 17-Aug-2011::10:53:10 === >> >> >> >> ** Node 'alice@REDACTED' not responding ** >> >> >> >> ** Removing (timedout) connection ** >> >> >> >> >> >> >> >> But Bob is dead: >> >> >> >> (bob@REDACTED)4> whereis (bob). >> >> >> >> undefined >> >> >> >> (bob@REDACTED)5> is_process_alive (Pid). >> >> >> >> false >> >> >> >> >> >> >> >> I really do not understand what is happening. >> >> >> >> >> >> >> >> >> >> >> >> Thank you in advance >> >> >> >> >> >> >> >> Anchise >> >> >> >> >> >> >> >> Here goes the code I used: >> >> >> >> >> >> >> >> -module (alice). >> >> >> >> -compile (export_all). >> >> >> >> >> >> >> >> start () -> register (alice, spawn (fun init/0) ). >> >> >> >> >> >> >> >> stop () -> whereis (alice) ! stop. >> >> >> >> >> >> >> >> init () -> >> >> >> >> process_flag (trap_exit, true), >> >> >> >> Bob = spawn_link ('bob@REDACTED', bob, start, [self () ] >> ), >> >> >> >> loop (Bob). >> >> >> >> >> >> >> >> loop (Bob) -> >> >> >> >> receive >> >> >> >> stop -> ok; >> >> >> >> {'EXIT', Bob, Reason} -> >> >> >> >> io:format ("Bob died of ~p.~n", [Reason] ), >> >> >> >> loop (Bob); >> >> >> >> Msg -> >> >> >> >> io:format ("Alice received ~p.~n", [Msg] ), >> >> >> >> loop (Bob) >> >> >> >> end. >> >> >> >> >> >> >> >> >> >> >> >> -module (bob). >> >> >> >> -compile (export_all). >> >> >> >> >> >> >> >> start (Alice) -> >> >> >> >> process_flag (trap_exit, true), >> >> >> >> register (bob, self () ), >> >> >> >> loop (Alice). >> >> >> >> >> >> >> >> loop (Alice) -> >> >> >> >> receive >> >> >> >> stop -> ok; >> >> >> >> {'EXIT', Alice, Reason} -> >> >> >> >> io:format ("Alice died of ~p.~n", [Reason] ), >> >> >> >> loop (Alice); >> >> >> >> Msg -> >> >> >> >> io:format ("Bob received ~p.~n", [Msg] ), >> >> >> >> loop (Alice) >> >> >> >> end. >> >> >> >> >> >> >> >> >> >> >> >> On Wed, 17 Aug 2011 12:47:02 +0200, Vincenzo Maggio wrote: >> >> >> >> > Hello, >> >> >> >> > without further info a debug is rather difficult. >> >> >> >> > But let's try to at least start analysis of the problem: >> >> >> >> > >> >> >> >> >> - Has this something to do that I initially spawn Bob >> >> from >> >> >> the >> >> >> >> >> Alice >> >> >> >> >> node? >> >> >> >> > >> >> >> >> > Absolutely not: this would hit the very foundation of >> >> Erlang, >> >> >> >> process >> >> >> >> > referential transparency. When a process is started is a >> >> brand >> >> >> >> new, >> >> >> >> > clean entity (indeed, default process heap space is >> always >> >> the >> >> >> >> same >> >> >> >> > size!). >> >> >> >> > >> >> >> >> >> - How can I make Bob to survive a connection loss? >> >> >> >> > >> >> >> >> > Look above: it SHOULD survive. >> >> >> >> > >> >> >> >> > Can you please start SASL (application:start(sasl) from >> the >> >> >> shell) >> >> >> >> > and see if shell log puts some further information? >> >> >> >> > >> >> >> >> > Vincenzo >> >> >> >> >> >> >> >> >> >> From erlang@REDACTED Thu Aug 18 20:01:00 2011 From: erlang@REDACTED (Anchise Inzaghi) Date: Thu, 18 Aug 2011 12:01:00 -0600 Subject: [erlang-questions] OTP Design Message-ID: I am quite new to erlang and now on the first step to my first application in our actual production environment. I surely want to make use of the OTP framework. I have played a bit with it, read the documentation and Joe Armstrong's book. I am still unsure about the design. I hope you can help me out and give me some precious input. The setup is the following: 1 DB-Server somewhere in the internet, 1 Comm-Server in our local LAN with static public IP and static LAN IP, n Comm-Slaves in our local LAN with static LAN IPs and each with a number of different devices attached. Comm-Server and Comm-Slaves suffer from frequent blackouts and power surges, so they fall down and reboot quite a lot. The LAN is stable, the internet connection (of our LAN, not of the DB-Server) is the worst connection you can possibly imagine. Each Comm-Salve has one process for each of its comm devices attached which receive data from these devices. When it receives data I want them to pass this data to the Comm-Server, the Comm-Server then writes them to the persistence layer (postgres), does some preprocessing and then sends it to the central DB-Server. If there is no internet connection, it should wait until he sees the DB-Server again and send the buffered data then. The DB-Server, Comm-Server and Comm-Salves I imagined would be of the behaviout gen_server. The Comm-Server starts up with an ini-File indicating which Node on which Slave controlles which devices, and start up the corresponding Comm-Slaves on the indicated nodes, passing them the information of which devices to controll. The Comm-Server will also have some interface for adding and removing devices (i.e. processes receiving data from devices). If a Comm-Slave process dies, I need to restart it one-on-one. I thought about a supervisor on the Comm-Server to supervise all the Salves and here comes my first problem: **As I read the documentation, there is no way to start_link a process on a different node, or am I missing something?** **What supervision strategy would you suggest?** I also dived a bit into distributed application docu, but I have the impression that it is more aimed at Takeover and Failover mechanisms. I fear, these won't help me much, as each slave can only run on the machine where its actual hardware device is connected. **Please prove me wrong and show me the way how to use ditributed app?** I will be thankful for any indications how to sensefully use OTP with this setup. Thanks a lot in advance. Anchise Inzaghi From maggio.vincenzo@REDACTED Thu Aug 18 20:05:49 2011 From: maggio.vincenzo@REDACTED (Vincenzo Maggio) Date: Thu, 18 Aug 2011 20:05:49 +0200 Subject: [erlang-questions] Process surviving disconnect In-Reply-To: <40c7dba97e4dc9457cd261a2708ee31f@mail.nimqas.com> References: , , <88dce0568e954d217577e7ffdc55fcd8@mail.nimqas.com>, , , , , , <406facfc6c11cc7c163371f961a2be82@mail.nimqas.com>, , <40c7dba97e4dc9457cd261a2708ee31f@mail.nimqas.com> Message-ID: Thanks i didn't know of io:format behaviour and 'user' process to redirect I/O! > Date: Thu, 18 Aug 2011 11:36:37 -0600 > From: erlang@REDACTED > To: maggio.vincenzo@REDACTED > CC: erlang-questions@REDACTED > Subject: RE: [erlang-questions] Process surviving disconnect > > By default a process writes to the shell which started it. > > Everything works as intended and expected if I change bob.erl > > DELETE: io:format ("Alice died of ~p.~n", [Reason] ), > REPLACE: io:format (user, "Alice died of ~p.~n", [Reason] ), > > This way, Bob writes to his own shell and all is well. > > On Thu, 18 Aug 2011 15:19:05 +0200, Vincenzo Maggio wrote: > > Mmm, sorry but I don't grasp it: shouldn't bob write 'Alice died of > > no connection' on bob's node, which is in fact well alive? > > > >> Date: Wed, 17 Aug 2011 21:25:37 -0600 > >> From: erlang@REDACTED > >> To: maggio.vincenzo@REDACTED > >> CC: erlang-questions@REDACTED > >> Subject: RE: [erlang-questions] Process surviving disconnect > >> > >> Mystery solved: no erlang bug but a brain bug of mine. > >> > >> When I cut the network connection, Bob tries to write "Alice died > >> of > >> noconnection" to Alice's console which obviously is unreachable. So > >> loop > >> crashEs and Bob dies. > >> > >> On Wed, 17 Aug 2011 22:56:20 +0200, Vincenzo Maggio wrote: > >> > Absolutely, maybe it's us but I notice that you, as me, has tried > >> all > >> > the possibilities and in fact in any case but on link > >> disconnection > >> > bob (and we think is right, BTW!!!) remains alive and in good > >> health. > >> > I hope someone shows up with a clever explanation, if not it's > >> > perfectly understable to file a bug. > >> > > >> >> I disconnect Alice from the network -> Bob joins his ancestors. > >> > Ahahah, may it rest in peace with the ones he loved!!! > >> > > >> > > >> >> Date: Wed, 17 Aug 2011 14:49:29 -0600 > >> >> From: erlang@REDACTED > >> >> To: maggio.vincenzo@REDACTED > >> >> CC: erlang-questions@REDACTED > >> >> Subject: RE: [erlang-questions] Process surviving disconnect > >> >> > >> >> OK, I changed the spawn_link to spawn and subsequent monitor as > >> you > >> >> suggest: exactely the same behaviour. > >> >> > >> >> Alice terminates gracefully -> Bob gets informed and is alive. > >> >> Alice terminates ungracefully -> Bob gets informed and is alive. > >> >> I kill Alice's node -> Bob gets informed and is alive. > >> >> I disconnect Alice from the network -> Bob joins his ancestors. > >> >> > >> >> So I will wait some days more here on erlang-questions and if > >> >> nothing > >> >> shows up here, I will file a bug and hopefully will be proven > >> >> wrong. > >> >> > >> >> On Wed, 17 Aug 2011 22:23:06 +0200, Vincenzo Maggio wrote: > >> >> > Use monitor(process, PID_TO_BE_MONITORED) after spawn. > >> >> > BTW not having two PCs I simply tested your code killing alice > >> and > >> >> > guess what? bob survives! :) > >> >> > > >> >> >> Date: Wed, 17 Aug 2011 14:13:37 -0600 > >> >> >> From: erlang@REDACTED > >> >> >> To: maggio.vincenzo@REDACTED > >> >> >> CC: erlang-questions@REDACTED > >> >> >> Subject: RE: [erlang-questions] Process surviving disconnect > >> >> >> > >> >> >> Thank you for your quick answer. > >> >> >> > >> >> >> > BTW, before filing a bug, could you please substitute > >> >> spawn_link > >> >> >> with > >> >> >> > spawn_monitor and remove the process_flag lines? It would be > >> >> >> > interesting to understand if either bob dies on its own or > >> it's > >> >> >> > killed > >> >> >> > by no more being able to communicate with alice. > >> >> >> > >> >> >> I am not sure how to replace spawn_link with spawn_monitor, > >> as > >> >> >> neither > >> >> >> spawn_monitor/1 nor spawn_monitor/3 take a node parameter. > >> >> >> How do I do that or how else can I get some more detailed > >> >> >> information > >> >> >> about Bob's sudden passing the Styx. > >> >> >> > >> >> >> > >> >> >> > >> >> >> On Wed, 17 Aug 2011 21:48:30 +0200, Vincenzo Maggio wrote: > >> >> >> > Hello, > >> >> >> > I think something is wrong here. > >> >> >> >> Bob die of noconnection. > >> >> >> > > >> >> >> > This is printed by > >> >> >> >> {'EXIT', Bob, Reason} -> > >> >> >> >> io:format ("Bob died of ~p.~n", [Reason] ), > >> >> >> > > >> >> >> > So alice is in fact receiving bob last death cry :D and > >> >> >> process_flag > >> >> >> > translate it in a message instead of transmitting exit > >> signal > >> >> to > >> >> >> > alice; I think this is ok from the point of view of Alice, > >> so > >> >> the > >> >> >> > real > >> >> >> > problem is that bob is dying (I know it's mundane, but I > >> >> learned > >> >> >> not > >> >> >> > to make assumption). > >> >> >> > Mmm, well I don't know if having no more a connection > >> between > >> >> the > >> >> >> > process makes Erlang VM do some assumption of a virtual > >> master > >> >> >> node. > >> >> >> > Well, if you want my opinion, I think that you should file a > >> >> bug > >> >> >> on > >> >> >> > the Erlang bugs mailing list if no one comes up with a > >> proper > >> >> >> > explanation. > >> >> >> > Even if what we're thinking is wrong and this is not a bug, > >> in > >> >> the > >> >> >> > past I had a problem on node lookup and they resolved it. > >> >> >> > These are my two cents, but if you can please let me know if > >> >> there > >> >> >> > are further updates 'cause it's a really interesting > >> problem. > >> >> >> > > >> >> >> > BTW, before filing a bug, could you please substitute > >> >> spawn_link > >> >> >> with > >> >> >> > spawn_monitor and remove the process_flag lines? It would be > >> >> >> > interesting to understand if either bob dies on its own or > >> it's > >> >> >> > killed > >> >> >> > by no more being able to communicate with alice. > >> >> >> > > >> >> >> > Vincenzo > >> >> >> > > >> >> >> >> Date: Wed, 17 Aug 2011 10:57:35 -0600 > >> >> >> >> From: erlang@REDACTED > >> >> >> >> To: maggio.vincenzo@REDACTED > >> >> >> >> CC: erlang-questions@REDACTED > >> >> >> >> Subject: RE: [erlang-questions] Process surviving > >> disconnect > >> >> >> >> > >> >> >> >> Thank you very much Vincenzo. You affirmed my assertion > >> that > >> >> Bob > >> >> >> >> should > >> >> >> >> survive the disconnect. > >> >> >> >> Nevertheless he dies. > >> >> >> >> I will point out exactly what I do and maybe someone can > >> spot > >> >> >> the > >> >> >> >> error > >> >> >> >> in my code, my setup or my thinking and tell me what I am > >> >> doing > >> >> >> >> wrong. > >> >> >> >> > >> >> >> >> I start a node on gca.local: > >> >> >> >> unroot@REDACTED:~$ erl -name 'bob@REDACTED' -setcookie 123 > >> >> >> >> > >> >> >> >> I start a node on usa.local: > >> >> >> >> unroot@REDACTED:~$ erl -name 'alice@REDACTED' -setcookie 123 > >> >> >> >> > >> >> >> >> I start sasl on bob@REDACTED: > >> >> >> >> (bob@REDACTED)1> application:start (sasl). > >> >> >> >> > >> >> >> >> I run alice:start/0 on alice@REDACTED: > >> >> >> >> (alice@REDACTED)1> alice:start (). > >> >> >> >> true > >> >> >> >> > >> >> >> >> I look for bob on bob@REDACTED and save its pid: > >> >> >> >> (bob@REDACTED)2> whereis (bob). > >> >> >> >> <0.65.0> > >> >> >> >> (bob@REDACTED)3> Pid = whereis (bob). > >> >> >> >> <0.65.0> > >> >> >> >> > >> >> >> >> I cut the network cable and wait a minute for the timeout. > >> >> >> >> > >> >> >> >> On alice@REDACTED I get the following output: > >> >> >> >> =ERROR REPORT==== 17-Aug-2011::10:53:21 === > >> >> >> >> ** Node 'bob@REDACTED' not responding ** > >> >> >> >> ** Removing (timedout) connection ** > >> >> >> >> Bob die of noconnection. > >> >> >> >> > >> >> >> >> Nice, Alice trapped Bob's death and reported it. I check > >> for > >> >> >> Alice: > >> >> >> >> (alice@REDACTED)2> whereis (alice). > >> >> >> >> <0.42.0> > >> >> >> >> > >> >> >> >> Alice is up and running. > >> >> >> >> > >> >> >> >> On bob@REDACTED I get the following output: > >> >> >> >> =ERROR REPORT==== 17-Aug-2011::10:53:10 === > >> >> >> >> ** Node 'alice@REDACTED' not responding ** > >> >> >> >> ** Removing (timedout) connection ** > >> >> >> >> > >> >> >> >> But Bob is dead: > >> >> >> >> (bob@REDACTED)4> whereis (bob). > >> >> >> >> undefined > >> >> >> >> (bob@REDACTED)5> is_process_alive (Pid). > >> >> >> >> false > >> >> >> >> > >> >> >> >> I really do not understand what is happening. > >> >> >> >> > >> >> >> >> > >> >> >> >> Thank you in advance > >> >> >> >> > >> >> >> >> Anchise > >> >> >> >> > >> >> >> >> Here goes the code I used: > >> >> >> >> > >> >> >> >> -module (alice). > >> >> >> >> -compile (export_all). > >> >> >> >> > >> >> >> >> start () -> register (alice, spawn (fun init/0) ). > >> >> >> >> > >> >> >> >> stop () -> whereis (alice) ! stop. > >> >> >> >> > >> >> >> >> init () -> > >> >> >> >> process_flag (trap_exit, true), > >> >> >> >> Bob = spawn_link ('bob@REDACTED', bob, start, [self () ] > >> ), > >> >> >> >> loop (Bob). > >> >> >> >> > >> >> >> >> loop (Bob) -> > >> >> >> >> receive > >> >> >> >> stop -> ok; > >> >> >> >> {'EXIT', Bob, Reason} -> > >> >> >> >> io:format ("Bob died of ~p.~n", [Reason] ), > >> >> >> >> loop (Bob); > >> >> >> >> Msg -> > >> >> >> >> io:format ("Alice received ~p.~n", [Msg] ), > >> >> >> >> loop (Bob) > >> >> >> >> end. > >> >> >> >> > >> >> >> >> > >> >> >> >> -module (bob). > >> >> >> >> -compile (export_all). > >> >> >> >> > >> >> >> >> start (Alice) -> > >> >> >> >> process_flag (trap_exit, true), > >> >> >> >> register (bob, self () ), > >> >> >> >> loop (Alice). > >> >> >> >> > >> >> >> >> loop (Alice) -> > >> >> >> >> receive > >> >> >> >> stop -> ok; > >> >> >> >> {'EXIT', Alice, Reason} -> > >> >> >> >> io:format ("Alice died of ~p.~n", [Reason] ), > >> >> >> >> loop (Alice); > >> >> >> >> Msg -> > >> >> >> >> io:format ("Bob received ~p.~n", [Msg] ), > >> >> >> >> loop (Alice) > >> >> >> >> end. > >> >> >> >> > >> >> >> >> > >> >> >> >> On Wed, 17 Aug 2011 12:47:02 +0200, Vincenzo Maggio wrote: > >> >> >> >> > Hello, > >> >> >> >> > without further info a debug is rather difficult. > >> >> >> >> > But let's try to at least start analysis of the problem: > >> >> >> >> > > >> >> >> >> >> - Has this something to do that I initially spawn Bob > >> >> from > >> >> >> the > >> >> >> >> >> Alice > >> >> >> >> >> node? > >> >> >> >> > > >> >> >> >> > Absolutely not: this would hit the very foundation of > >> >> Erlang, > >> >> >> >> process > >> >> >> >> > referential transparency. When a process is started is a > >> >> brand > >> >> >> >> new, > >> >> >> >> > clean entity (indeed, default process heap space is > >> always > >> >> the > >> >> >> >> same > >> >> >> >> > size!). > >> >> >> >> > > >> >> >> >> >> - How can I make Bob to survive a connection loss? > >> >> >> >> > > >> >> >> >> > Look above: it SHOULD survive. > >> >> >> >> > > >> >> >> >> > Can you please start SASL (application:start(sasl) from > >> the > >> >> >> shell) > >> >> >> >> > and see if shell log puts some further information? > >> >> >> >> > > >> >> >> >> > Vincenzo > >> >> >> >> > >> >> >> > >> >> > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Thu Aug 18 20:44:32 2011 From: g@REDACTED (Garrett Smith) Date: Thu, 18 Aug 2011 13:44:32 -0500 Subject: [erlang-questions] OTP Design In-Reply-To: References: Message-ID: On Thu, Aug 18, 2011 at 1:01 PM, Anchise Inzaghi wrote: > **As I read the documentation, there is no way to start_link a process on a > different node, or am I missing something?** http://www.erlang.org/doc/man/erlang.html#spawn_link-4 > **What supervision strategy would you suggest?** I personally would not supervise processes over a network connection, unless it was some sort of cluster where the interconnect was reliable. Apart from prototyping, I don't generally use distributed Erlang (i.e. starting named nodes) but prefer to use other messaging protocols (e.g. AMQP, 0MQ, REST, etc). In any case, I'd recommend that you treat each server as an island that explicitly handles "no response" cases from other servers rather than rely on OTP supervision + restart patterns. > I also dived a bit into distributed application docu, but I have the > impression that it is more aimed at Takeover and Failover mechanisms. I > fear, these won't help me much, as each slave can only run on the machine > where its actual hardware device is connected. You'd need to design "takeover" into your application. I'd start with just getting the basic master/slave framework in place and working. Your interim solution for recovering may simply be to automate server startup/recovery. Once you have experience with your specific problem set, you can gradually improve that SLA. > **Please prove me wrong and show me the way how to use ditributed app?** Distributed applications are hard. You might poke around the 0MQ community -- in particular read this: http://zguide.zeromq.org/ It's not Erlang specific, but it certainly presents a lot of concepts that you're going to want to at least think about. The Erlang bindings are solid with comprehensive examples: https://github.com/imatix/zguide/tree/master/examples/Erlang Garrett From kunthar@REDACTED Thu Aug 18 20:46:31 2011 From: kunthar@REDACTED (Kunthar) Date: Thu, 18 Aug 2011 21:46:31 +0300 Subject: [erlang-questions] OTP Design In-Reply-To: References: Message-ID: I strongly recommend to read Erlang and OTP in Action. http://www.manning.com/logan/ It would be needed to have a concrete idea. On Thu, Aug 18, 2011 at 9:01 PM, Anchise Inzaghi wrote: > I am quite new to erlang and now on the first step to my first application > in our actual production environment. I surely want to make use of the OTP > framework. I have played a bit with it, read the documentation and Joe > Armstrong's book. I am still unsure about the design. I hope you can help me > out and give me some precious input. > > The setup is the following: > > 1 DB-Server somewhere in the internet, > 1 Comm-Server in our local LAN with static public IP and static LAN IP, > n Comm-Slaves in our local LAN with static LAN IPs and each with a number of > different devices attached. > > Comm-Server and Comm-Slaves suffer from frequent blackouts and power surges, > so they fall down and reboot quite a lot. > The LAN is stable, the internet connection (of our LAN, not of the > DB-Server) is the worst connection you can possibly imagine. > > Each Comm-Salve has one process for each of its comm devices attached which > receive data from these devices. When it receives data I want them to pass > this data to the Comm-Server, the Comm-Server then writes them to the > persistence layer (postgres), does some preprocessing and then sends it to > the central DB-Server. If there is no internet connection, it should wait > until he sees the DB-Server again and send the buffered data then. > > The DB-Server, Comm-Server and Comm-Salves I imagined would be of the > behaviout gen_server. The Comm-Server starts up with an ini-File indicating > which Node on which Slave controlles which devices, and start up the > corresponding Comm-Slaves on the indicated nodes, passing them the > information of which devices to controll. > The Comm-Server will also have some interface for adding and removing > devices (i.e. processes receiving data from devices). > > If a Comm-Slave process dies, I need to restart it one-on-one. I thought > about a supervisor on the Comm-Server to supervise all the Salves and here > comes my first problem: > > **As I read the documentation, there is no way to start_link a process on a > different node, or am I missing something?** > **What supervision strategy would you suggest?** > > I also dived a bit into distributed application docu, but I have the > impression that it is more aimed at Takeover and Failover mechanisms. I > fear, these won't help me much, as each slave can only run on the machine > where its actual hardware device is connected. > > **Please prove me wrong and show me the way how to use ditributed app?** > > I will be thankful for any indications how to sensefully use OTP with this > setup. > > Thanks a lot in advance. > > Anchise Inzaghi > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- BR, \|/ Kunthar From nx@REDACTED Thu Aug 18 21:45:26 2011 From: nx@REDACTED (nx) Date: Thu, 18 Aug 2011 15:45:26 -0400 Subject: [erlang-questions] OTP Design In-Reply-To: References: Message-ID: STOMP is another messaging option: http://stomp.github.com/ On Thu, Aug 18, 2011 at 2:46 PM, Kunthar wrote: > I strongly recommend to read Erlang and OTP in Action. > http://www.manning.com/logan/ > > It would be needed to have a concrete idea. > > > On Thu, Aug 18, 2011 at 9:01 PM, Anchise Inzaghi wrote: >> I am quite new to erlang and now on the first step to my first application >> in our actual production environment. I surely want to make use of the OTP >> framework. I have played a bit with it, read the documentation and Joe >> Armstrong's book. I am still unsure about the design. I hope you can help me >> out and give me some precious input. >> >> The setup is the following: >> >> 1 DB-Server somewhere in the internet, >> 1 Comm-Server in our local LAN with static public IP and static LAN IP, >> n Comm-Slaves in our local LAN with static LAN IPs and each with a number of >> different devices attached. >> >> Comm-Server and Comm-Slaves suffer from frequent blackouts and power surges, >> so they fall down and reboot quite a lot. >> The LAN is stable, the internet connection (of our LAN, not of the >> DB-Server) is the worst connection you can possibly imagine. >> >> Each Comm-Salve has one process for each of its comm devices attached which >> receive data from these devices. When it receives data I want them to pass >> this data to the Comm-Server, the Comm-Server then writes them to the >> persistence layer (postgres), does some preprocessing and then sends it to >> the central DB-Server. If there is no internet connection, it should wait >> until he sees the DB-Server again and send the buffered data then. >> >> The DB-Server, Comm-Server and Comm-Salves I imagined would be of the >> behaviout gen_server. The Comm-Server starts up with an ini-File indicating >> which Node on which Slave controlles which devices, and start up the >> corresponding Comm-Slaves on the indicated nodes, passing them the >> information of which devices to controll. >> The Comm-Server will also have some interface for adding and removing >> devices (i.e. processes receiving data from devices). >> >> If a Comm-Slave process dies, I need to restart it one-on-one. I thought >> about a supervisor on the Comm-Server to supervise all the Salves and here >> comes my first problem: >> >> **As I read the documentation, there is no way to start_link a process on a >> different node, or am I missing something?** >> **What supervision strategy would you suggest?** >> >> I also dived a bit into distributed application docu, but I have the >> impression that it is more aimed at Takeover and Failover mechanisms. I >> fear, these won't help me much, as each slave can only run on the machine >> where its actual hardware device is connected. >> >> **Please prove me wrong and show me the way how to use ditributed app?** >> >> I will be thankful for any indications how to sensefully use OTP with this >> setup. >> >> Thanks a lot in advance. >> >> Anchise Inzaghi >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > BR, > \|/ Kunthar > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From dmercer@REDACTED Thu Aug 18 22:08:44 2011 From: dmercer@REDACTED (David Mercer) Date: Thu, 18 Aug 2011 15:08:44 -0500 Subject: [erlang-questions] OTP Design In-Reply-To: References: Message-ID: <009c01cc5de2$a33d7e10$e9b87a30$@com> On Thursday, August 18, 2011, Garrett Smith wrote: > Apart from prototyping, I don't generally use distributed Erlang (i.e. > starting named nodes) but prefer to use other messaging protocols > (e.g. AMQP, 0MQ, REST, etc). Do you have any thoughts on 0MQ vs. AMQP? Cheers, DBM From g@REDACTED Thu Aug 18 23:09:08 2011 From: g@REDACTED (Garrett Smith) Date: Thu, 18 Aug 2011 16:09:08 -0500 Subject: [erlang-questions] OTP Design In-Reply-To: <009c01cc5de2$a33d7e10$e9b87a30$@com> References: <009c01cc5de2$a33d7e10$e9b87a30$@com> Message-ID: On Thu, Aug 18, 2011 at 3:08 PM, David Mercer wrote: > On Thursday, August 18, 2011, Garrett Smith wrote: > >> Apart from prototyping, I don't generally use distributed Erlang (i.e. >> starting named nodes) but prefer to use other messaging protocols >> (e.g. AMQP, 0MQ, REST, etc). > > Do you have any thoughts on 0MQ vs. AMQP? They're pretty different. 0MQ is more of a toolkit for building your own messaging systems. For centralized broker models, AMQP is great. RabbitMQ is very solid. And of course it's Erlang. I have nothing but good things to say about it. My problem with AMQP is that it tends to become a central architectural feature -- similar to the big, centralized relational databases. All of a sudden, you're feeding and caring for the broker -- and if it goes down, it takes a lot with it. AMQP is also a complicated protocol, which tends to cause problems for people implementing it. So, while there are quality servers and clients for it -- be prepared for hard-to-fix problems along the way. 0MQ is a counter point to this: support for federated and flexible topologies and simple protocol. There is a NIF based Erlang binding for it, which is excellent. You might find this interested, if you haven't seen it already: http://www.zeromq.org/whitepapers:brokerless Garrett From robert.virding@REDACTED Thu Aug 18 23:35:17 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 18 Aug 2011 22:35:17 +0100 (BST) Subject: [erlang-questions] Process surviving disconnect In-Reply-To: <40c7dba97e4dc9457cd261a2708ee31f@mail.nimqas.com> Message-ID: Not quite. By default when no explicit io device is given a process does io, including io:format, to/from its group_leader. When a process is spawned it will inherit its group_leader from the spawning process, this irrespective on which node the group_leader and process are. So when you spawn a process from a shell it will use that shell's group_leader which in this case was on the same node as the shell. Giving the io device 'user' to io:format it will use a default io server on the same node. That it was started from a shell is irrelevant here. Robert ----- Original Message ----- > By default a process writes to the shell which started it. > > Everything works as intended and expected if I change bob.erl > > DELETE: io:format ("Alice died of ~p.~n", [Reason] ), > REPLACE: io:format (user, "Alice died of ~p.~n", [Reason] ), > > This way, Bob writes to his own shell and all is well. > > On Thu, 18 Aug 2011 15:19:05 +0200, Vincenzo Maggio wrote: > > Mmm, sorry but I don't grasp it: shouldn't bob write 'Alice died of > > no connection' on bob's node, which is in fact well alive? > > > >> Date: Wed, 17 Aug 2011 21:25:37 -0600 > >> From: erlang@REDACTED > >> To: maggio.vincenzo@REDACTED > >> CC: erlang-questions@REDACTED > >> Subject: RE: [erlang-questions] Process surviving disconnect > >> > >> Mystery solved: no erlang bug but a brain bug of mine. > >> > >> When I cut the network connection, Bob tries to write "Alice died > >> of > >> noconnection" to Alice's console which obviously is unreachable. > >> So > >> loop > >> crashEs and Bob dies. > >> > >> On Wed, 17 Aug 2011 22:56:20 +0200, Vincenzo Maggio wrote: > >> > Absolutely, maybe it's us but I notice that you, as me, has > >> > tried > >> all > >> > the possibilities and in fact in any case but on link > >> disconnection > >> > bob (and we think is right, BTW!!!) remains alive and in good > >> health. > >> > I hope someone shows up with a clever explanation, if not it's > >> > perfectly understable to file a bug. > >> > > >> >> I disconnect Alice from the network -> Bob joins his > >> >> ancestors. > >> > Ahahah, may it rest in peace with the ones he loved!!! > >> > > >> > > >> >> Date: Wed, 17 Aug 2011 14:49:29 -0600 > >> >> From: erlang@REDACTED > >> >> To: maggio.vincenzo@REDACTED > >> >> CC: erlang-questions@REDACTED > >> >> Subject: RE: [erlang-questions] Process surviving disconnect > >> >> > >> >> OK, I changed the spawn_link to spawn and subsequent monitor > >> >> as > >> you > >> >> suggest: exactely the same behaviour. > >> >> > >> >> Alice terminates gracefully -> Bob gets informed and is alive. > >> >> Alice terminates ungracefully -> Bob gets informed and is > >> >> alive. > >> >> I kill Alice's node -> Bob gets informed and is alive. > >> >> I disconnect Alice from the network -> Bob joins his > >> >> ancestors. > >> >> > >> >> So I will wait some days more here on erlang-questions and if > >> >> nothing > >> >> shows up here, I will file a bug and hopefully will be proven > >> >> wrong. > >> >> > >> >> On Wed, 17 Aug 2011 22:23:06 +0200, Vincenzo Maggio wrote: > >> >> > Use monitor(process, PID_TO_BE_MONITORED) after spawn. > >> >> > BTW not having two PCs I simply tested your code killing > >> >> > alice > >> and > >> >> > guess what? bob survives! :) > >> >> > > >> >> >> Date: Wed, 17 Aug 2011 14:13:37 -0600 > >> >> >> From: erlang@REDACTED > >> >> >> To: maggio.vincenzo@REDACTED > >> >> >> CC: erlang-questions@REDACTED > >> >> >> Subject: RE: [erlang-questions] Process surviving disconnect > >> >> >> > >> >> >> Thank you for your quick answer. > >> >> >> > >> >> >> > BTW, before filing a bug, could you please substitute > >> >> spawn_link > >> >> >> with > >> >> >> > spawn_monitor and remove the process_flag lines? It would > >> >> >> > be > >> >> >> > interesting to understand if either bob dies on its own or > >> it's > >> >> >> > killed > >> >> >> > by no more being able to communicate with alice. > >> >> >> > >> >> >> I am not sure how to replace spawn_link with spawn_monitor, > >> as > >> >> >> neither > >> >> >> spawn_monitor/1 nor spawn_monitor/3 take a node parameter. > >> >> >> How do I do that or how else can I get some more detailed > >> >> >> information > >> >> >> about Bob's sudden passing the Styx. > >> >> >> > >> >> >> > >> >> >> > >> >> >> On Wed, 17 Aug 2011 21:48:30 +0200, Vincenzo Maggio wrote: > >> >> >> > Hello, > >> >> >> > I think something is wrong here. > >> >> >> >> Bob die of noconnection. > >> >> >> > > >> >> >> > This is printed by > >> >> >> >> {'EXIT', Bob, Reason} -> > >> >> >> >> io:format ("Bob died of ~p.~n", [Reason] ), > >> >> >> > > >> >> >> > So alice is in fact receiving bob last death cry :D and > >> >> >> process_flag > >> >> >> > translate it in a message instead of transmitting exit > >> signal > >> >> to > >> >> >> > alice; I think this is ok from the point of view of Alice, > >> so > >> >> the > >> >> >> > real > >> >> >> > problem is that bob is dying (I know it's mundane, but I > >> >> learned > >> >> >> not > >> >> >> > to make assumption). > >> >> >> > Mmm, well I don't know if having no more a connection > >> between > >> >> the > >> >> >> > process makes Erlang VM do some assumption of a virtual > >> master > >> >> >> node. > >> >> >> > Well, if you want my opinion, I think that you should file > >> >> >> > a > >> >> bug > >> >> >> on > >> >> >> > the Erlang bugs mailing list if no one comes up with a > >> proper > >> >> >> > explanation. > >> >> >> > Even if what we're thinking is wrong and this is not a > >> >> >> > bug, > >> in > >> >> the > >> >> >> > past I had a problem on node lookup and they resolved it. > >> >> >> > These are my two cents, but if you can please let me know > >> >> >> > if > >> >> there > >> >> >> > are further updates 'cause it's a really interesting > >> problem. > >> >> >> > > >> >> >> > BTW, before filing a bug, could you please substitute > >> >> spawn_link > >> >> >> with > >> >> >> > spawn_monitor and remove the process_flag lines? It would > >> >> >> > be > >> >> >> > interesting to understand if either bob dies on its own or > >> it's > >> >> >> > killed > >> >> >> > by no more being able to communicate with alice. > >> >> >> > > >> >> >> > Vincenzo > >> >> >> > > >> >> >> >> Date: Wed, 17 Aug 2011 10:57:35 -0600 > >> >> >> >> From: erlang@REDACTED > >> >> >> >> To: maggio.vincenzo@REDACTED > >> >> >> >> CC: erlang-questions@REDACTED > >> >> >> >> Subject: RE: [erlang-questions] Process surviving > >> disconnect > >> >> >> >> > >> >> >> >> Thank you very much Vincenzo. You affirmed my assertion > >> that > >> >> Bob > >> >> >> >> should > >> >> >> >> survive the disconnect. > >> >> >> >> Nevertheless he dies. > >> >> >> >> I will point out exactly what I do and maybe someone can > >> spot > >> >> >> the > >> >> >> >> error > >> >> >> >> in my code, my setup or my thinking and tell me what I > >> >> >> >> am > >> >> doing > >> >> >> >> wrong. > >> >> >> >> > >> >> >> >> I start a node on gca.local: > >> >> >> >> unroot@REDACTED:~$ erl -name 'bob@REDACTED' -setcookie 123 > >> >> >> >> > >> >> >> >> I start a node on usa.local: > >> >> >> >> unroot@REDACTED:~$ erl -name 'alice@REDACTED' -setcookie 123 > >> >> >> >> > >> >> >> >> I start sasl on bob@REDACTED: > >> >> >> >> (bob@REDACTED)1> application:start (sasl). > >> >> >> >> > >> >> >> >> I run alice:start/0 on alice@REDACTED: > >> >> >> >> (alice@REDACTED)1> alice:start (). > >> >> >> >> true > >> >> >> >> > >> >> >> >> I look for bob on bob@REDACTED and save its pid: > >> >> >> >> (bob@REDACTED)2> whereis (bob). > >> >> >> >> <0.65.0> > >> >> >> >> (bob@REDACTED)3> Pid = whereis (bob). > >> >> >> >> <0.65.0> > >> >> >> >> > >> >> >> >> I cut the network cable and wait a minute for the > >> >> >> >> timeout. > >> >> >> >> > >> >> >> >> On alice@REDACTED I get the following output: > >> >> >> >> =ERROR REPORT==== 17-Aug-2011::10:53:21 === > >> >> >> >> ** Node 'bob@REDACTED' not responding ** > >> >> >> >> ** Removing (timedout) connection ** > >> >> >> >> Bob die of noconnection. > >> >> >> >> > >> >> >> >> Nice, Alice trapped Bob's death and reported it. I check > >> for > >> >> >> Alice: > >> >> >> >> (alice@REDACTED)2> whereis (alice). > >> >> >> >> <0.42.0> > >> >> >> >> > >> >> >> >> Alice is up and running. > >> >> >> >> > >> >> >> >> On bob@REDACTED I get the following output: > >> >> >> >> =ERROR REPORT==== 17-Aug-2011::10:53:10 === > >> >> >> >> ** Node 'alice@REDACTED' not responding ** > >> >> >> >> ** Removing (timedout) connection ** > >> >> >> >> > >> >> >> >> But Bob is dead: > >> >> >> >> (bob@REDACTED)4> whereis (bob). > >> >> >> >> undefined > >> >> >> >> (bob@REDACTED)5> is_process_alive (Pid). > >> >> >> >> false > >> >> >> >> > >> >> >> >> I really do not understand what is happening. > >> >> >> >> > >> >> >> >> > >> >> >> >> Thank you in advance > >> >> >> >> > >> >> >> >> Anchise > >> >> >> >> > >> >> >> >> Here goes the code I used: > >> >> >> >> > >> >> >> >> -module (alice). > >> >> >> >> -compile (export_all). > >> >> >> >> > >> >> >> >> start () -> register (alice, spawn (fun init/0) ). > >> >> >> >> > >> >> >> >> stop () -> whereis (alice) ! stop. > >> >> >> >> > >> >> >> >> init () -> > >> >> >> >> process_flag (trap_exit, true), > >> >> >> >> Bob = spawn_link ('bob@REDACTED', bob, start, [self () > >> >> >> >> ] > >> ), > >> >> >> >> loop (Bob). > >> >> >> >> > >> >> >> >> loop (Bob) -> > >> >> >> >> receive > >> >> >> >> stop -> ok; > >> >> >> >> {'EXIT', Bob, Reason} -> > >> >> >> >> io:format ("Bob died of ~p.~n", [Reason] ), > >> >> >> >> loop (Bob); > >> >> >> >> Msg -> > >> >> >> >> io:format ("Alice received ~p.~n", [Msg] ), > >> >> >> >> loop (Bob) > >> >> >> >> end. > >> >> >> >> > >> >> >> >> > >> >> >> >> -module (bob). > >> >> >> >> -compile (export_all). > >> >> >> >> > >> >> >> >> start (Alice) -> > >> >> >> >> process_flag (trap_exit, true), > >> >> >> >> register (bob, self () ), > >> >> >> >> loop (Alice). > >> >> >> >> > >> >> >> >> loop (Alice) -> > >> >> >> >> receive > >> >> >> >> stop -> ok; > >> >> >> >> {'EXIT', Alice, Reason} -> > >> >> >> >> io:format ("Alice died of ~p.~n", [Reason] ), > >> >> >> >> loop (Alice); > >> >> >> >> Msg -> > >> >> >> >> io:format ("Bob received ~p.~n", [Msg] ), > >> >> >> >> loop (Alice) > >> >> >> >> end. > >> >> >> >> > >> >> >> >> > >> >> >> >> On Wed, 17 Aug 2011 12:47:02 +0200, Vincenzo Maggio > >> >> >> >> wrote: > >> >> >> >> > Hello, > >> >> >> >> > without further info a debug is rather difficult. > >> >> >> >> > But let's try to at least start analysis of the > >> >> >> >> > problem: > >> >> >> >> > > >> >> >> >> >> - Has this something to do that I initially spawn > >> >> >> >> >> Bob > >> >> from > >> >> >> the > >> >> >> >> >> Alice > >> >> >> >> >> node? > >> >> >> >> > > >> >> >> >> > Absolutely not: this would hit the very foundation of > >> >> Erlang, > >> >> >> >> process > >> >> >> >> > referential transparency. When a process is started is > >> >> >> >> > a > >> >> brand > >> >> >> >> new, > >> >> >> >> > clean entity (indeed, default process heap space is > >> always > >> >> the > >> >> >> >> same > >> >> >> >> > size!). > >> >> >> >> > > >> >> >> >> >> - How can I make Bob to survive a connection loss? > >> >> >> >> > > >> >> >> >> > Look above: it SHOULD survive. > >> >> >> >> > > >> >> >> >> > Can you please start SASL (application:start(sasl) from > >> the > >> >> >> shell) > >> >> >> >> > and see if shell log puts some further information? > >> >> >> >> > > >> >> >> >> > Vincenzo > >> >> >> >> > >> >> >> > >> >> > >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From andrew@REDACTED Fri Aug 19 00:39:55 2011 From: andrew@REDACTED (Andrew Thompson) Date: Thu, 18 Aug 2011 18:39:55 -0400 Subject: [erlang-questions] Network partitions and Quorom In-Reply-To: <20110818165542.GA31180@hijacked.us> References: <213D0AB9-AE20-43C1-AC1D-EC98567EE6E4@srstrong.com> <20110818165542.GA31180@hijacked.us> Message-ID: <20110818223955.GC31180@hijacked.us> Alright, as a number of people have contacted me off-list about this, here's a github branch containing the changes that I forward-ported to the lastest version of gen_leader_revival on github (archaelus's branch). https://github.com/Vagabond/gen_leader_revival/tree/netsplit-tolerance There's 4 commits, which fix a combination of issues we found at Basho when trying to test gen_leader in netsplit conditions. Andrew From jeraymond@REDACTED Fri Aug 19 01:26:21 2011 From: jeraymond@REDACTED (Jeremy Raymond) Date: Thu, 18 Aug 2011 19:26:21 -0400 Subject: [erlang-questions] Cross compiling no syslog Message-ID: Hi, I'm cross compiling to a system without syslog (no syslog.h). The build fails in a few places (run_erl.c, epmd.c, epmd_init.h). These files have ifdefs for NO_SYSLOG to conditionally compile without syslog support. The configure process detects syslog and defines HAVE_SYSLOG_H if syslog is detected. If I define NO_SYSLOG myself things compile fine. Are these files checking the wrong define (should be checking HAVE_SYSLOG_H) or is there someone to configure in NO_SYSLOG? - Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitrii@REDACTED Fri Aug 19 09:02:59 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Fri, 19 Aug 2011 10:02:59 +0300 Subject: [erlang-questions] =?windows-1252?q?ux_=97_a_Unicode_library?= Message-ID: <19DA319E-99F0-4830-815E-A6CA3420A20E@dmitriid.com> Michael Uvarov, a member of the Russian Erlang community has developed a library to deal with Unicode: https://github.com/freeakk/ux Features: It works with Unicode Characters Database and contains all the functions to do that, contained in ux_string: - case transforms: to_lower, to_upper - normalization: to_nfc, to_nfd, ... - works with grapheme clusters according to UTR29, http://unicode.org/reports/tr29/ - length, ... - works with symbol types: types/1 UCA (Collation) has also been implemented in ux_uca. This module provides string sorting and comparison. It also provides a function to search for a substring: - search - sort - compare - sort_key - generates a sequence which can be then compared against a binary Symbol data is stored in ETS and generated on first call. You can try it by running ./start-dev.sh From overminddl1@REDACTED Fri Aug 19 09:11:38 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Fri, 19 Aug 2011 01:11:38 -0600 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> <5451A797-C158-43B5-B37F-F50583456F72@cs.otago.ac.nz> <30ECDB3E-4A5C-4EDF-8542-B9A485BDC1BA@erlang-solutions.com> Message-ID: Not so rare in my production code for note, I use it heavily in a few of my math oriented modules and would not want it on by default. I have personally not yet been bit by the bug that the warning would prevent in my few years in erlang, although I have other functional backgrounds. On Aug 17, 2011 2:09 PM, "Kenneth Lundin" wrote: > I agree that it is very common in test code to match against already bound > variables. > But in production code it is quite rare. > The warning should definitely not be on by default. > We will also run a check over all OTP libraries and possibly other > production code > as well to see how it turns out before we make any definite decisions. > > But as said we have not given the implementation any priority at all yet. > > /Kenneth > > 2011/8/17 Steve Vinoski : >> 2011/8/3 Fr?d?ric Trottier-H?bert : >>> On 2011-08-02, at 01:59 AM, Richard O'Keefe wrote: >>> >>>> 2. Is matching against an already-bound variable a check we want? >>> >>> To me, matching against an already-bound variable is is a valid > assertion, as much as 'ok = function_call()' might be, and as implicit as > '{ok, Pid}' or '{error, already_started}' might be. Matching on > already-bound variables is part of my standard code when trying to crash > early when possible, and also part of many basic test suites that simply do > pattern matching here and there. To me it's as basic as using the same > variable twice in a single pattern (f([A,A,B,B|_]) when A =/= B -> ...), or > something similar with records. >>> >>> I would likely not use the check at all, and if it were to be added, > would prefer it to be a compiler argument (which could be enabled in > -compile(...).) I foresee little use of such warnings for myself and would > dislike to see it becoming a default setting. >> >> I completely agree. I use matching this way quite a bit, especially in >> testing, and wouldn't want to see any warnings for it by default. >> >> --steve >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From fred.hebert@REDACTED Fri Aug 19 13:25:18 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Fri, 19 Aug 2011 07:25:18 -0400 Subject: [erlang-questions] =?windows-1252?q?ux_=97_a_Unicode_library?= In-Reply-To: <19DA319E-99F0-4830-815E-A6CA3420A20E@dmitriid.com> References: <19DA319E-99F0-4830-815E-A6CA3420A20E@dmitriid.com> Message-ID: <1F6E8DDB-B5A6-4404-AC66-EF18D2808A17@erlang-solutions.com> I tried this library because I needed it for some work and ended up finding it very useful, although pretty confusing on many points. If it would be possible to get information on this I might consider using it more: - Why does it need to download mochiweb to deal with UTF stuff? I don't see the relationship with a web server and it worries me that versions could clash or that I could break it by using different stuff; - I don't get why every time I start the app I get "** /usr/local/agner/packages/rebar-@REDACTED/ebin/rebar_utils.beam hides /usr/local/agner/packages/rebar-@REDACTED/ebin/rebar_utils.beam ... ** Found 82 name clashes in code paths ". I figure the app checks its dependencies, but does it on a level way to broad or something. Not sure why it does this, especially when I figure rebar and agner aren't necessary for ux to work once compiled. - It makes some sense to load items as it goes, but it does make the library a bit more unpredictable. I also see a bunch of servers in there, but I assume they have to do with loading the data and whatnot? I'm somewhat afraid they might be serialising a bunch of calls, but I'm not really in a position to criticise the library given I don't have one to offer in exchange. In any case, it's good news to see a library that does handle multiple codepoints as single graphemes (so that [16#0065, 16#0301] and [233] both equal to "?", a string of length 1) as I've had issues with that in some projects that could not calculate lengths right. -- Fred H?bert http://www.erlang-solutions.com On 2011-08-19, at 03:02 AM, Dmitrii Dimandt wrote: > Michael Uvarov, a member of the Russian Erlang community has developed a library to deal with Unicode: https://github.com/freeakk/ux > > Features: > > It works with Unicode Characters Database and contains all the functions to do that, contained in ux_string: > - case transforms: to_lower, to_upper > - normalization: to_nfc, to_nfd, ... > - works with grapheme clusters according to UTR29, http://unicode.org/reports/tr29/ > - length, ... > - works with symbol types: types/1 > > UCA (Collation) has also been implemented in ux_uca. This module provides string sorting and comparison. It also provides a function to search for a substring: > > - search > - sort > - compare > - sort_key - generates a sequence which can be then compared against a binary > > Symbol data is stored in ETS and generated on first call. You can try it by running ./start-dev.sh > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From freeakk@REDACTED Fri Aug 19 18:22:02 2011 From: freeakk@REDACTED (Michael Uvarov) Date: Fri, 19 Aug 2011 20:22:02 +0400 Subject: [erlang-questions] =?utf-8?q?ux_=E2=80=94_a_Unicode_library?= In-Reply-To: <1F6E8DDB-B5A6-4404-AC66-EF18D2808A17@erlang-solutions.com> References: <19DA319E-99F0-4830-815E-A6CA3420A20E@dmitriid.com> <1F6E8DDB-B5A6-4404-AC66-EF18D2808A17@erlang-solutions.com> Message-ID: Hello, About deps. I use module reloader from mochiweb. These dependencies can be removed, which I did on github. About unpredictability. It's very interesting. The first version of the library converted data to erlang code and loaded it. It was fast, but I had the huge module. Then, there are two versions of the DUCET file. It stores sorting settings. The first version is standard, the second is for the CLDR (although it is not yet implemented). That's why the library needs two copies (or more) of the data of the same type. In new realization, different processes are able to use different settings. Disadvantages of loading on the fly: eunit needs the timeout directive. Also, I don't know how to say to eunit to load ux before running tests. So, when TESTS is defined, library behaves a little differently, because ux_unidata_filelist is not running. Does anybody khow how to ask Eunit run application ux before the tests? There is no serialization: data stores in the ETS, the parser returns a function for each data type that knows how to get data and what to do if the ETS table was killed. In order to avoid unnecessary requests to other processes, these functions are cached in the dictionary process. From patrick@REDACTED Fri Aug 19 21:09:38 2011 From: patrick@REDACTED (Patrick J Newton) Date: Fri, 19 Aug 2011 15:09:38 -0400 Subject: [erlang-questions] Building OTP on Mac OSX Lion Message-ID: <17795A86-D479-4B17-A979-50FB3E9DE270@linguisticsciences.com> I'm trying to build R14B03 on mac osx line but am running into difficulties that I have been unable to work around. During make, there are failures during build of hipe_icode_type.erl. Any advice would be appreciated. Thanks. >sw_vers ProductName: Mac OS X ProductVersion: 10.7 BuildVersion: 11A511 >gcc -v Using built-in specs. Target: i686-apple-darwin11 Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/src/configure --disable-checking --enable-werror --prefix=/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1 Thread model: posix gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) >CFLAGS=-O0 ./configure --enable-darwin-64bit ? >make ? erlc -W +debug_info +warn_unused_import +warn_missing_spec +warn_untyped_record -o../ebin hipe_icode_type.erl ./hipe_icode_type.erl:62: redefining macro 'debug' ./hipe_icode_type.erl:1414: function true/2 undefined ./hipe_icode_type.erl:1427: function true/2 undefined ./../flow/cfg.hrl:34: Warning: record cfg_info has field(s) without type information make[3]: *** [../ebin/hipe_icode_type.beam] Error 1 make[2]: *** [opt] Error 2 make[1]: *** [opt] Error 2 make: *** [secondary_bootstrap_build] Error 2 From ixmatus@REDACTED Fri Aug 19 21:17:25 2011 From: ixmatus@REDACTED (Parnell Springmeyer) Date: Fri, 19 Aug 2011 12:17:25 -0700 Subject: [erlang-questions] Building OTP on Mac OSX Lion In-Reply-To: <17795A86-D479-4B17-A979-50FB3E9DE270@linguisticsciences.com> (Patrick J. Newton's message of "Fri, 19 Aug 2011 15:09:38 -0400") References: <17795A86-D479-4B17-A979-50FB3E9DE270@linguisticsciences.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I had trouble building on Lion using the GCC-LLVM binaries, I just deferred to the plain GCC binaries (using the CC,CXX, etc... flags). Not sure if that is your problem though since your error is happening with hipe and mine was a different build error! Patrick J Newton writes: > I'm trying to build R14B03 on mac osx line but am running into difficulties that I have been unable to work around. During make, there are failures during build of hipe_icode_type.erl. Any advice would be appreciated. Thanks. > >>sw_vers > ProductName: Mac OS X > ProductVersion: 10.7 > BuildVersion: 11A511 > >>gcc -v > Using built-in specs. > Target: i686-apple-darwin11 > Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/src/configure --disable-checking --enable-werror --prefix=/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1 > Thread model: posix > gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) > >>CFLAGS=-O0 ./configure --enable-darwin-64bit > > ? > >>make > > ? > > erlc -W +debug_info +warn_unused_import +warn_missing_spec +warn_untyped_record -o../ebin hipe_icode_type.erl > ./hipe_icode_type.erl:62: redefining macro 'debug' > ./hipe_icode_type.erl:1414: function true/2 undefined > ./hipe_icode_type.erl:1427: function true/2 undefined > ./../flow/cfg.hrl:34: Warning: record cfg_info has field(s) without type information > make[3]: *** [../ebin/hipe_icode_type.beam] Error 1 > make[2]: *** [opt] Error 2 > make[1]: *** [opt] Error 2 > make: *** [secondary_bootstrap_build] Error 2 > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > - -- Parnell "ixmatus" Springmeyer (http://ixmat.us) -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJOTrbKAAoJEPvtlbpI1POLWKsIAKlHqoZe3OZ092uv+t61LODu ukScsa09p6mWE01638vDBW5B4h2d48dvU1n90gsqr9E1dkALieuI0CI5/8aH0jkC +O7z7bSefWF/HyxEGsBDCJiLvKTkudfS8pOqrheRWyR9MudNFeZS51W5Nw48bhUS 52qYU7JmY3mNJFzcar9FaKDgEpmlb/LVXVvK1a48LadwfPGAb0Lqdl9R3ZR3hxJu lyU4UxUwLOgjaMN7IPcRroxPg98KSEaS7X3Ichzr32b6hc1L3VK75PGAaLIrEibZ k6fMiZBbHmO0nFs/PsExlWqYb0AbfJfnhqb89xochTEnjK+j081521y8amuwukc= =G7rW -----END PGP SIGNATURE----- From frgo@REDACTED Fri Aug 19 21:22:03 2011 From: frgo@REDACTED (Frank Goenninger) Date: Fri, 19 Aug 2011 21:22:03 +0200 Subject: [erlang-questions] Building OTP on Mac OSX Lion In-Reply-To: References: <17795A86-D479-4B17-A979-50FB3E9DE270@linguisticsciences.com> Message-ID: FWIW, I run the following script to build Erlang/OTP -X-X-X- INSTALLDIR=/opt/langtools/erlang SRCDIR=/opt/langtools/packages/erlang/src/otp_src_R14B03 PWD=`pwd` PATH=$PATH:. cd $SRCDIR OPTS="--enable-smp-support" OPTS=$OPTS" --enable-sctp" OPTS=$OPTS" --enable-hipe --enable-native-libs" OPTS=$OPTS" --enable-shared-zlib" OPTS=$OPTS" --enable-darwin-64bit" OPTS=$OPTS" --enable-threads" OPTS=$OPTS" --prefix=$INSTALLDIR" echo "*** Running configure with the following options:" echo echo $OPTS echo echo "*** ..." echo "Press RETURN to continue !" frgo@REDACTED /opt/langtools/packages/erlang/src $ sudo more run-configure INSTALLDIR=/opt/langtools/erlang SRCDIR=/opt/langtools/packages/erlang/src/otp_src_R14B03 PWD=`pwd` PATH=$PATH:. cd $SRCDIR OPTS="--enable-smp-support" OPTS=$OPTS" --enable-sctp" OPTS=$OPTS" --enable-hipe --enable-native-libs" OPTS=$OPTS" --enable-shared-zlib" OPTS=$OPTS" --enable-darwin-64bit" OPTS=$OPTS" --enable-threads" OPTS=$OPTS" --prefix=$INSTALLDIR" echo "*** Running configure with the following options:" echo echo $OPTS echo echo "*** ..." echo "Press RETURN to continue !" read configure $OPTS make && sudo make install cd $PWD -X-X-X- HTH, Frank Am 19.08.2011 um 21:17 schrieb Parnell Springmeyer: > * PGP Signed by an unknown key > > I had trouble building on Lion using the GCC-LLVM binaries, I just > deferred to the plain GCC binaries (using the CC,CXX, etc... flags). > > Not sure if that is your problem though since your error is happening > with hipe and mine was a different build error! > > Patrick J Newton writes: > >> I'm trying to build R14B03 on mac osx line but am running into difficulties that I have been unable to work around. During make, there are failures during build of hipe_icode_type.erl. Any advice would be appreciated. Thanks. >> >>> sw_vers >> ProductName: Mac OS X >> ProductVersion: 10.7 >> BuildVersion: 11A511 >> >>> gcc -v >> Using built-in specs. >> Target: i686-apple-darwin11 >> Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/src/configure --disable-checking --enable-werror --prefix=/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1 >> Thread model: posix >> gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) >> >>> CFLAGS=-O0 ./configure --enable-darwin-64bit >> >> ? >> >>> make >> >> ? >> >> erlc -W +debug_info +warn_unused_import +warn_missing_spec +warn_untyped_record -o../ebin hipe_icode_type.erl >> ./hipe_icode_type.erl:62: redefining macro 'debug' >> ./hipe_icode_type.erl:1414: function true/2 undefined >> ./hipe_icode_type.erl:1427: function true/2 undefined >> ./../flow/cfg.hrl:34: Warning: record cfg_info has field(s) without type information >> make[3]: *** [../ebin/hipe_icode_type.beam] Error 1 >> make[2]: *** [opt] Error 2 >> make[1]: *** [opt] Error 2 >> make: *** [secondary_bootstrap_build] Error 2 >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -- > Parnell "ixmatus" Springmeyer (http://ixmat.us) > > * Unknown Key > * 0x48D4F38B(L) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From frgo@REDACTED Fri Aug 19 21:23:51 2011 From: frgo@REDACTED (Frank Goenninger) Date: Fri, 19 Aug 2011 21:23:51 +0200 Subject: [erlang-questions] Building OTP on Mac OSX Lion In-Reply-To: References: <17795A86-D479-4B17-A979-50FB3E9DE270@linguisticsciences.com> Message-ID: <8685EF8F-D339-41BE-BC6C-3C5858714A53@me.com> OOPS ... That should be: -X-X-X- INSTALLDIR=/opt/langtools/erlang SRCDIR=/opt/langtools/packages/erlang/src/otp_src_R14B03 PWD=`pwd` PATH=$PATH:. cd $SRCDIR OPTS="--enable-smp-support" OPTS=$OPTS" --enable-sctp" OPTS=$OPTS" --enable-hipe --enable-native-libs" OPTS=$OPTS" --enable-shared-zlib" OPTS=$OPTS" --enable-darwin-64bit" OPTS=$OPTS" --enable-threads" OPTS=$OPTS" --prefix=$INSTALLDIR" echo "*** Running configure with the following options:" echo echo $OPTS echo echo "*** ..." echo "Press RETURN to continue !" read configure $OPTS make && sudo make install cd $PWD -X-X-X- ;-) Frank Am 19.08.2011 um 21:22 schrieb Frank Goenninger: > FWIW, I run the following script to build Erlang/OTP > > -X-X-X- > INSTALLDIR=/opt/langtools/erlang > SRCDIR=/opt/langtools/packages/erlang/src/otp_src_R14B03 > > PWD=`pwd` > PATH=$PATH:. > > cd $SRCDIR > > OPTS="--enable-smp-support" > OPTS=$OPTS" --enable-sctp" > OPTS=$OPTS" --enable-hipe --enable-native-libs" > OPTS=$OPTS" --enable-shared-zlib" > OPTS=$OPTS" --enable-darwin-64bit" > OPTS=$OPTS" --enable-threads" > OPTS=$OPTS" --prefix=$INSTALLDIR" > > echo "*** Running configure with the following options:" > echo > echo $OPTS > echo > echo "*** ..." > > echo "Press RETURN to continue !" > frgo@REDACTED /opt/langtools/packages/erlang/src > $ sudo more run-configure > INSTALLDIR=/opt/langtools/erlang > SRCDIR=/opt/langtools/packages/erlang/src/otp_src_R14B03 > > PWD=`pwd` > PATH=$PATH:. > > cd $SRCDIR > > OPTS="--enable-smp-support" > OPTS=$OPTS" --enable-sctp" > OPTS=$OPTS" --enable-hipe --enable-native-libs" > OPTS=$OPTS" --enable-shared-zlib" > OPTS=$OPTS" --enable-darwin-64bit" > OPTS=$OPTS" --enable-threads" > OPTS=$OPTS" --prefix=$INSTALLDIR" > > echo "*** Running configure with the following options:" > echo > echo $OPTS > echo > echo "*** ..." > > echo "Press RETURN to continue !" > read > > > configure $OPTS > make && sudo make install > cd $PWD > > -X-X-X- > > HTH, > > Frank > > Am 19.08.2011 um 21:17 schrieb Parnell Springmeyer: > >>> Old Signed by an unknown key >> >> I had trouble building on Lion using the GCC-LLVM binaries, I just >> deferred to the plain GCC binaries (using the CC,CXX, etc... flags). >> >> Not sure if that is your problem though since your error is happening >> with hipe and mine was a different build error! >> >> Patrick J Newton writes: >> >>> I'm trying to build R14B03 on mac osx line but am running into difficulties that I have been unable to work around. During make, there are failures during build of hipe_icode_type.erl. Any advice would be appreciated. Thanks. >>> >>>> sw_vers >>> ProductName: Mac OS X >>> ProductVersion: 10.7 >>> BuildVersion: 11A511 >>> >>>> gcc -v >>> Using built-in specs. >>> Target: i686-apple-darwin11 >>> Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/src/configure --disable-checking --enable-werror --prefix=/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1 >>> Thread model: posix >>> gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) >>> >>>> CFLAGS=-O0 ./configure --enable-darwin-64bit >>> >>> ? >>> >>>> make >>> >>> ? >>> >>> erlc -W +debug_info +warn_unused_import +warn_missing_spec +warn_untyped_record -o../ebin hipe_icode_type.erl >>> ./hipe_icode_type.erl:62: redefining macro 'debug' >>> ./hipe_icode_type.erl:1414: function true/2 undefined >>> ./hipe_icode_type.erl:1427: function true/2 undefined >>> ./../flow/cfg.hrl:34: Warning: record cfg_info has field(s) without type information >>> make[3]: *** [../ebin/hipe_icode_type.beam] Error 1 >>> make[2]: *** [opt] Error 2 >>> make[1]: *** [opt] Error 2 >>> make: *** [secondary_bootstrap_build] Error 2 >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> -- >> Parnell "ixmatus" Springmeyer (http://ixmat.us) >> >> * Unknown Key >> * 0x48D4F38B(L) >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From dizzyd@REDACTED Fri Aug 19 22:45:06 2011 From: dizzyd@REDACTED (Dave Smith) Date: Fri, 19 Aug 2011 14:45:06 -0600 Subject: [erlang-questions] Building OTP on Mac OSX Lion In-Reply-To: <17795A86-D479-4B17-A979-50FB3E9DE270@linguisticsciences.com> References: <17795A86-D479-4B17-A979-50FB3E9DE270@linguisticsciences.com> Message-ID: If CFLAGS=-O0, it should work. Make certain that you are starting from a fresh unpack (tar -xzf), as the Erlang source doesn't cleanup stuff properly between compiles with different flags/configure settings. D. On Fri, Aug 19, 2011 at 1:09 PM, Patrick J Newton wrote: > I'm trying to build R14B03 on mac osx line but am running into difficulties that I have been unable to work around. During make, there are failures during build of hipe_icode_type.erl. Any advice would be appreciated. Thanks. > >>sw_vers > ProductName: ? ?Mac OS X > ProductVersion: 10.7 > BuildVersion: ? 11A511 > >>gcc -v > Using built-in specs. > Target: i686-apple-darwin11 > Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/src/configure --disable-checking --enable-werror --prefix=/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1 > Thread model: posix > gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) > >>CFLAGS=-O0 ./configure --enable-darwin-64bit > > ? > >>make > > ? > > erlc -W ?+debug_info +warn_unused_import +warn_missing_spec +warn_untyped_record -o../ebin hipe_icode_type.erl > ./hipe_icode_type.erl:62: redefining macro 'debug' > ./hipe_icode_type.erl:1414: function true/2 undefined > ./hipe_icode_type.erl:1427: function true/2 undefined > ./../flow/cfg.hrl:34: Warning: record cfg_info has field(s) without type information > make[3]: *** [../ebin/hipe_icode_type.beam] Error 1 > make[2]: *** [opt] Error 2 > make[1]: *** [opt] Error 2 > make: *** [secondary_bootstrap_build] Error 2 > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From patrick@REDACTED Fri Aug 19 22:50:19 2011 From: patrick@REDACTED (Patrick J Newton) Date: Fri, 19 Aug 2011 16:50:19 -0400 Subject: [erlang-questions] Building OTP on Mac OSX Lion In-Reply-To: References: <17795A86-D479-4B17-A979-50FB3E9DE270@linguisticsciences.com> Message-ID: Right, I've ensured a clean environment. I've even attempted the build from the tips of dev from the OTP git repository. Same result. On Aug 19, 2011, at 4:45 PM, Dave Smith wrote: > If CFLAGS=-O0, it should work. Make certain that you are starting from > a fresh unpack (tar -xzf), as the Erlang source doesn't cleanup stuff > properly between compiles with different flags/configure settings. > > D. > > On Fri, Aug 19, 2011 at 1:09 PM, Patrick J Newton > wrote: >> I'm trying to build R14B03 on mac osx line but am running into difficulties that I have been unable to work around. During make, there are failures during build of hipe_icode_type.erl. Any advice would be appreciated. Thanks. >> >>> sw_vers >> ProductName: Mac OS X >> ProductVersion: 10.7 >> BuildVersion: 11A511 >> >>> gcc -v >> Using built-in specs. >> Target: i686-apple-darwin11 >> Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/src/configure --disable-checking --enable-werror --prefix=/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1 >> Thread model: posix >> gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) >> >>> CFLAGS=-O0 ./configure --enable-darwin-64bit >> >> ? >> >>> make >> >> ? >> >> erlc -W +debug_info +warn_unused_import +warn_missing_spec +warn_untyped_record -o../ebin hipe_icode_type.erl >> ./hipe_icode_type.erl:62: redefining macro 'debug' >> ./hipe_icode_type.erl:1414: function true/2 undefined >> ./hipe_icode_type.erl:1427: function true/2 undefined >> ./../flow/cfg.hrl:34: Warning: record cfg_info has field(s) without type information >> make[3]: *** [../ebin/hipe_icode_type.beam] Error 1 >> make[2]: *** [opt] Error 2 >> make[1]: *** [opt] Error 2 >> make: *** [secondary_bootstrap_build] Error 2 >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From patrick@REDACTED Fri Aug 19 23:42:42 2011 From: patrick@REDACTED (Patrick J Newton) Date: Fri, 19 Aug 2011 17:42:42 -0400 Subject: [erlang-questions] Building OTP on Mac OSX Lion In-Reply-To: References: <17795A86-D479-4B17-A979-50FB3E9DE270@linguisticsciences.com> Message-ID: <41BC7E76-B68B-4C24-9D14-8A1BC3EFBD43@linguisticsciences.com> I managed to solve this issue by simply commenting out the re-definition of 'debug' and the two instances of usage of ?debug in hipe_icode_type.erl. Maybe this is a machine specific issue but I would be happy to help diagnose if anyone should choose to follow up. Thanks to all who replied, your help is truly appreciated. On Aug 19, 2011, at 4:50 PM, Patrick J Newton wrote: > Right, I've ensured a clean environment. I've even attempted the build from the tips of dev from the OTP git repository. Same result. > > On Aug 19, 2011, at 4:45 PM, Dave Smith wrote: > >> If CFLAGS=-O0, it should work. Make certain that you are starting from >> a fresh unpack (tar -xzf), as the Erlang source doesn't cleanup stuff >> properly between compiles with different flags/configure settings. >> >> D. >> >> On Fri, Aug 19, 2011 at 1:09 PM, Patrick J Newton >> wrote: >>> I'm trying to build R14B03 on mac osx line but am running into difficulties that I have been unable to work around. During make, there are failures during build of hipe_icode_type.erl. Any advice would be appreciated. Thanks. >>> >>>> sw_vers >>> ProductName: Mac OS X >>> ProductVersion: 10.7 >>> BuildVersion: 11A511 >>> >>>> gcc -v >>> Using built-in specs. >>> Target: i686-apple-darwin11 >>> Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/src/configure --disable-checking --enable-werror --prefix=/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1 >>> Thread model: posix >>> gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) >>> >>>> CFLAGS=-O0 ./configure --enable-darwin-64bit >>> >>> ? >>> >>>> make >>> >>> ? >>> >>> erlc -W +debug_info +warn_unused_import +warn_missing_spec +warn_untyped_record -o../ebin hipe_icode_type.erl >>> ./hipe_icode_type.erl:62: redefining macro 'debug' >>> ./hipe_icode_type.erl:1414: function true/2 undefined >>> ./hipe_icode_type.erl:1427: function true/2 undefined >>> ./../flow/cfg.hrl:34: Warning: record cfg_info has field(s) without type information >>> make[3]: *** [../ebin/hipe_icode_type.beam] Error 1 >>> make[2]: *** [opt] Error 2 >>> make[1]: *** [opt] Error 2 >>> make: *** [secondary_bootstrap_build] Error 2 >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From n.oxyde@REDACTED Sat Aug 20 11:10:04 2011 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sat, 20 Aug 2011 11:10:04 +0200 Subject: [erlang-questions] =?windows-1252?q?ux_=97_a_Unicode_library?= In-Reply-To: References: <19DA319E-99F0-4830-815E-A6CA3420A20E@dmitriid.com> <1F6E8DDB-B5A6-4404-AC66-EF18D2808A17@erlang-solutions.com> Message-ID: <5680533C-D26C-4E49-9513-73BC008AC095@gmail.com> Hi, Out of curiosity, what was wrong with having a huge module? Regards, Le 19 ao?t 2011 ? 18:22, Michael Uvarov a ?crit : > Hello, > > About deps. > I use module reloader from mochiweb. These dependencies can be > removed, which I did on github. > > About unpredictability. > It's very interesting. > > The first version of the library converted data to erlang code and > loaded it. It was fast, but I had the huge module. > > Then, there are two versions of the DUCET file. It stores sorting > settings. The first version is standard, the second is for the CLDR > (although it is not yet implemented). That's why the library needs two > copies (or more) of the data of the same type. > > In new realization, different processes are able to use different settings. > > Disadvantages of loading on the fly: > eunit needs the timeout directive. Also, I don't know how to say to > eunit to load ux before running tests. So, when TESTS is defined, > library behaves a little differently, because ux_unidata_filelist is > not running. > > Does anybody khow how to ask Eunit run application ux before the tests? > > There is no serialization: data stores in the ETS, the parser returns > a function for each data type that knows how to get data and what to > do if the ETS table was killed. > > In order to avoid unnecessary requests to other processes, these > functions are cached in the dictionary process. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Anthony Ramine / @nokusu Dev:Extend ? http://dev-extend.eu/ So as I pray, ?Unlimited Erlang Works? From nox@REDACTED Sat Aug 20 15:48:57 2011 From: nox@REDACTED (Anthony Ramine) Date: Sat, 20 Aug 2011 15:48:57 +0200 Subject: [erlang-questions] =?windows-1252?q?ux_=97_a_Unicode_library?= In-Reply-To: References: <19DA319E-99F0-4830-815E-A6CA3420A20E@dmitriid.com> <1F6E8DDB-B5A6-4404-AC66-EF18D2808A17@erlang-solutions.com> <5680533C-D26C-4E49-9513-73BC008AC095@gmail.com> Message-ID: <7E0BB7C2-E90D-410A-AF14-CCA9C2BDAED6@dev-extend.eu> What do you mean by not working? Do you mean dialyzing is too slow when working on a huge module? Le 20 ao?t 2011 ? 15:23, Michael Uvarov a ?crit : > Dialyzer and native compilation are not working with huge (few > megabytes) modules. -- Anthony Ramine / @nokusu Dev:Extend ? http://dev-extend.eu/ So as I pray, ?Unlimited Erlang Works? From kuleshovmail@REDACTED Sat Aug 20 18:53:56 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Sat, 20 Aug 2011 16:53:56 +0000 Subject: [erlang-questions] Pass message from C code to Erlang Message-ID: Hello, I try to send message from C code to erlang. I use EI Library. My C code: int main (int argc, char** argv) { erl_init (NULL, 0); extern const char *erl_thisnodename(void); extern short erl_thiscreation(void); #define SELF(fd) erl_mk_pid(erl_thisnodename(), fd, 0, erl_thiscreation()) ETERM *arr[2], *emsg; int sockfd, creation=1; arr[0] = SELF(sockfd); arr[1] = erl_mk_atom("ok"); emsg = erl_mk_tuple(arr, 2); erl_reg_send(sockfd, "my_server", emsg); erl_free_term(emsg); return (EXIT_SUCCESS); } And my erlang code: start() -> Pid = spawn(?MODULE, loop, []), register(my_server, Pid). loop() -> receive {ok} -> io:format("Ok received \n"), loop(); stop -> stop; _ -> io:format("Somethig else \n") end. stop() -> my_server ! stop. But when i run my C application, erlang application nothing received. What's wrong? Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kuleshovmail@REDACTED Sat Aug 20 19:00:36 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Sat, 20 Aug 2011 17:00:36 +0000 Subject: [erlang-questions] Pass message from C code to Erlang In-Reply-To: References: Message-ID: 2011/8/20 Alexander Kuleshov > Hello, > > I try to send message from C code to erlang. I use EI Library. > > My C code: > > int main (int argc, char** argv) > { > erl_init (NULL, 0); > > extern const char *erl_thisnodename(void); > extern short erl_thiscreation(void); > > #define SELF(fd) erl_mk_pid(erl_thisnodename(), fd, 0, > erl_thiscreation()) > > ETERM *arr[2], *emsg; > > int sockfd, creation=1; > > arr[0] = SELF(sockfd); > arr[1] = erl_mk_atom("ok"); > emsg = erl_mk_tuple(arr, 2); > > erl_reg_send(sockfd, "my_server", emsg); > erl_free_term(emsg); > > return (EXIT_SUCCESS); > } > > And my erlang code: > > start() -> > Pid = spawn(?MODULE, loop, []), > register(my_server, Pid). > > loop() -> > receive > {ok} -> > io:format("Ok received \n"), > loop(); > stop -> > stop; > _ -> > io:format("Somethig else \n") > end. > > stop() -> > my_server ! stop. > > But when i run my C application, erlang application nothing received. > What's wrong? > > Thank you. > > And i get error: [Thread debugging using libthread_db enabled] Program received signal SIGSEGV, Segmentation fault. 0x08055fac in ei_mutex_lock () -------------- next part -------------- An HTML attachment was scrubbed... URL: From freza@REDACTED Sat Aug 20 21:47:01 2011 From: freza@REDACTED (Jachym Holecek) Date: Sat, 20 Aug 2011 20:47:01 +0100 Subject: [erlang-questions] Pass message from C code to Erlang In-Reply-To: References: Message-ID: <20110820194701.GA898@hanele.lan> # Alexander Kuleshov 2011-08-20: > [... Code sample ommited ...] > > And i get error: Presumably this is your program crashing, not the Erlang node? > [Thread debugging using libthread_db enabled] > > Program received signal SIGSEGV, Segmentation fault. > 0x08055fac in ei_mutex_lock () > > [... This is where backtrace wasn't included so we can't really know ...] Let me guess: your C node isn't properly initialized (you don't call erl_connect_init()) so ei_sockets_lock is NULL and the library explodes as soon as you try to send a message (this tries to take the sockets lock). Even if this wasn't the case, your sockfd is uninitialized so things wouldn't work anyway :-). And judging from the EI files involved being placed under src/legacy/ directory, you may be using some old API. Not sure, I've only ever used the lowlevel encoding/decoding functions and ignored the C node parts altogether. Anyway, src/prog/erl_call.c in EI directory should be an illustrative example of how to write a C node. HTH, -- Jachym From max.lapshin@REDACTED Sun Aug 21 22:47:22 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 22 Aug 2011 00:47:22 +0400 Subject: [erlang-questions] CPU affinity and enif_thread_create Message-ID: I've created libx264 encoder in nif and started new thread for it. libx264 was started with option threads 4 and it really created 4 threads. Now I have problems. All created threads are bound to one CPU core and it is not enough. 100% busy and I have to drop frames. I have installed linux kernel with BFS (brainfuck scheduler), but it didn't change anything. So I have called sched_setaffinity by hands and libx264 threads spread across cores. And I have questions: does erlang VM makes some calls, that bind future created threads to current core? If no, that why doesn't erlang VM moved to other cores busy processes that had several hundred messages inbox? Maybe I should do some hinting like "bump_reductions" to tell VM to switch process to another scheduler? From freza@REDACTED Mon Aug 22 00:12:06 2011 From: freza@REDACTED (Jachym Holecek) Date: Sun, 21 Aug 2011 23:12:06 +0100 Subject: [erlang-questions] CPU affinity and enif_thread_create In-Reply-To: References: Message-ID: <20110821221206.GA29390@hanele.lan> # Max Lapshin 2011-08-21: > I've created libx264 encoder in nif and started new thread for it. > libx264 was started with option threads 4 and it really created 4 threads. > > Now I have problems. All created threads are bound to one CPU core and > it is not enough. 100% busy and I have to drop frames. > > I have installed linux kernel with BFS (brainfuck scheduler), but it > didn't change anything. > > So I have called sched_setaffinity by hands and libx264 threads spread > across cores. > > And I have questions: does erlang VM makes some calls, that bind > future created threads to current core? Nothing explicit in ethr_thr_create() (this ends up being called from the NIF API you're probably using) if I'm looking correctly, but it could perhaps be accidental? * Your thread is created from a scheduler thread. * Scheduler thread tries to be bound to some specific CPU. * On Linux affinity is hereditary[*]. * So your worker thread gets bound to the same CPU. Of course, OTP people will know for sure. BR, -- Jachym [*] Linux's pthread_setaffinity_np(3) says: A new thread created by pthread_create(3) inherits a copy of its creator's CPU affinity mask. The manpage further says this is implemented in terms sched_setaffinity(2) which does mention inheritance for fork()'d processes, but doesn't mention threads, so this hypothesis might be wrong -- it's sched_setaffinity() that's being used by Erlang, not its also nonstandard yet slightly more portable pthread cousing (for purely historical reasons I assume?). From max.lapshin@REDACTED Mon Aug 22 00:17:30 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 22 Aug 2011 02:17:30 +0400 Subject: [erlang-questions] CPU affinity and enif_thread_create In-Reply-To: <20110821221206.GA29390@hanele.lan> References: <20110821221206.GA29390@hanele.lan> Message-ID: Yes, I think that my newly created threads inherit scheduler affinity, so I need somehow to remove it. From max.lapshin@REDACTED Mon Aug 22 08:10:15 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 22 Aug 2011 10:10:15 +0400 Subject: [erlang-questions] Where is enif_keep_binary? Message-ID: I start enif_thread_create and pass binary to there: static ERL_NIF_TERM yuv_x264(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { .... enif_mutex_lock(x264->lock); enif_inspect_binary(env, argv[1], &x264->yuv); x264->has_yuv = 1; enif_cond_signal(x264->cond); enif_mutex_unlock(x264->lock); return enif_make_atom(env, "wait"); } and in thread I make: while(1) { while(!x264->has_yuv) { enif_cond_wait(x264->cond, x264->lock); } handle(x264->yuv); } Problem is that binary get expired. How should I add refcounter to binary? Copy it to other env? From nox@REDACTED Mon Aug 22 09:45:26 2011 From: nox@REDACTED (Anthony Ramine) Date: Mon, 22 Aug 2011 09:45:26 +0200 Subject: [erlang-questions] Where is enif_keep_binary? In-Reply-To: References: Message-ID: <81489483-1C5B-4009-9B38-E18FE517C773@dev-extend.eu> Hi, Are this binary big? If it is big enough, it won't be copied anyway and its refcounter will just be incremented. Regards, Le 22 ao?t 2011 ? 08:10, Max Lapshin a ?crit : > I start enif_thread_create and pass binary to there: > > > static ERL_NIF_TERM yuv_x264(ErlNifEnv* env, int argc, const > ERL_NIF_TERM argv[]) { > .... > enif_mutex_lock(x264->lock); > enif_inspect_binary(env, argv[1], &x264->yuv); > x264->has_yuv = 1; > enif_cond_signal(x264->cond); > enif_mutex_unlock(x264->lock); > return enif_make_atom(env, "wait"); > } > > and in thread I make: > > > while(1) { > while(!x264->has_yuv) { > enif_cond_wait(x264->cond, x264->lock); > } > > handle(x264->yuv); > } > > Problem is that binary get expired. How should I add refcounter to binary? > Copy it to other env? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Anthony Ramine / @nokusu Dev:Extend ? http://dev-extend.eu/ So as I pray, ?Unlimited Erlang Works? From max.lapshin@REDACTED Mon Aug 22 09:55:13 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 22 Aug 2011 11:55:13 +0400 Subject: [erlang-questions] Where is enif_keep_binary? In-Reply-To: <81489483-1C5B-4009-9B38-E18FE517C773@dev-extend.eu> References: <81489483-1C5B-4009-9B38-E18FE517C773@dev-extend.eu> Message-ID: On Mon, Aug 22, 2011 at 11:45 AM, Anthony Ramine wrote: > Hi, > > Are this binary big? If it is big enough, it won't be copied anyway and its refcounter will just be incremented. > enif_inspect_binary doesn't increment refcounter, so when thread starts to work with it, this binary is already freed. From max.lapshin@REDACTED Mon Aug 22 10:01:39 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 22 Aug 2011 12:01:39 +0400 Subject: [erlang-questions] Where is enif_keep_binary? In-Reply-To: References: Message-ID: On Mon, Aug 22, 2011 at 11:58 AM, Nikolas Bowe wrote: > see > http://erlang.2086793.n4.nabble.com/Processing-huge-binaries-in-NIFs-tp3430910p3432802.html > I've already implemented first way, but think that second way should be better. From max.lapshin@REDACTED Mon Aug 22 10:25:26 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 22 Aug 2011 12:25:26 +0400 Subject: [erlang-questions] Where is enif_keep_binary? In-Reply-To: References: Message-ID: On Mon, Aug 22, 2011 at 12:12 PM, Nikolas Bowe wrote: > Agreed. But at least its not a full copy of a large binary. Its an > acceptable interim solution for me. > I dont mind the extra allocation.?I just think enif_keep_binary would be a > more intuitive name. > I wonder how keep_binary would work for small binaries that are on a > processes heap. > I guess it would need to make a copy on the shared binary heap. > I see the problem. But there is erl_driver_keep_binary. I haven't looked at it yet. From francesco@REDACTED Mon Aug 22 10:45:08 2011 From: francesco@REDACTED (Francesco Cesarini) Date: Mon, 22 Aug 2011 09:45:08 +0100 Subject: [erlang-questions] CFP for IFL, 2011 In-Reply-To: <235FB1C3-EEA4-41C0-963F-3863FDD27A5A@ku.edu> References: <235FB1C3-EEA4-41C0-963F-3863FDD27A5A@ku.edu> Message-ID: <4E521714.3030006@erlang-solutions.com> The IFL deadline for submissions has been moved to the end of August. We already have a great program scheduled (to be posted shortly), and hope to get a few more interesting late additions because of this extension. CALL FOR PAPERS 23rd Symposium on Implementation and Application of Functional Languages (IFL 2011) October 3-5, 2011 University of Kansas Lawrence, Kansas, USA http://www.ittc.ku.edu/ifl2011 The Symposium returns to the US this year, hosted by the University of Kansas, in scenic Lawrence, Kansas. Lawrence is a lively college town less than an hour from Kansas City and the Kansas City International Airport (MCI). The symposium dates are October 3-5, 2011. The goal of the IFL symposia is to bring together researchers actively engaged in the implementation and application of functional and function-based programming languages. IFL 2011 will be a venue for researchers to present and discuss new ideas and concepts, work in progress, and publication-ripe results related to the implementation and application of functional languages and function-based programming. Following the IFL tradition, IFL 2011 will use a post-symposium review process to produce formal proceedings which will be published by Springer Verlag in the Lecture Notes in Computer Science series. All participants in IFL 2011 are invited to submit either a draft paper or an extended abstract describing work to be presented at the symposium. At no time may work submitted to IFL be simultaneously submitted to other venues. Here we follow the ACM Sigplan republication policy: http://www.sigplan.org/republicationpolicy.htm. The submissions will be screened by the program committee chair to make sure they are within the scope of IFL, and will appear in the draft proceedings distributed at the symposium. Submissions appearing in the draft proceedings are not peer-reviewed publications. After the symposium, authors will be given the opportunity to incorporate the feedback from discussions at the symposium and will be invited to submit a revised full article for the formal review process. These revised submissions will be reviewed by the program committee using prevailing academic standards to select the best articles, which will appear in the formal proceedings. INVITED SPEAKER Bryan O'Sullivan, author of Real World Haskell, and co-founder of MailRank, Inc. TOPICS IFL welcomes submissions describing practical and theoretical work as well as submissions describing applications and tools. If you are not sure that your work is appropriate for IFL 2011, please contact the PC chair at andygill@REDACTED Topics of interest include, but are not limited to: language concepts type checking contracts compilation techniques staged compilation runtime function specialization runtime code generation partial evaluation (abstract) interpretation generic programming techniques automatic program generation array processing concurrent/parallel programming concurrent/parallel program execution functional programming and embedded systems functional programming and web applications functional programming and security novel memory management techniques runtime profiling and performance measurements debugging and tracing virtual/abstract machine architectures validation and verification of functional programs tools and programming techniques industrial applications of functional programming PAPER SUBMISSIONS Prospective authors are encouraged to submit papers or extended abstracts to be published in the draft proceedings and to present them at the symposium. All contributions must be written in English, conform to the Springer-Verlag LNCS series format and not exceed 16 pages. The draft proceedings will appear as a technical report of the Department of Electrical Engineering and Computer Science of the University of Kansas. (We are more liberal with the draft proceedings, also accepting (for example) longer papers or SIGPLAN 2 column 12 page papers. For other formats contact the chair. For the consideration for the final proceedings, the 16 page LNCS format is required.) PETER LANDIN PRIZE The Peter Landin Prize is awarded to the best paper presented at the symposium every year. The honored article is selected by the program committee based on the submissions received for the formal review process. The prize carries a cash award equivalent to 150 Euros. IMPORTANT DATES Presentation submission deadline: (new) August 31st, 2011 Notification of acceptance: (new) Within 24 hours of submission Early registration deadline: (new) September 16th, 2011 IFL 2011 Symposium: October 3-5, 2011 Submission for (post) review process: November 30th, 2011 Notification Accept/Reject: January 30th, 2012 Camera ready version: March 9th, 2012 PROGRAM COMMITTEE Torben Amtoft, Kansas State University Francesco Cesarini, Erlang Solutions Ltd Olaf Chitil, University of Kent Eelco Dolstra, Delft University of Technology Martin Erwig, Oregon State University Andy Gill, University of Kansas (Chair) Alwyn Goodloe, NASA Jurriaan Hage, Utrecht University Kevin Hammond, University of St. Andrews Bill Harrison, University of Missouri Ralf Hinze, Oxford University James Hook, Portland State University Garrin Kimmell, University of Iowa Andres Loh, Well-Typed LLP Rita Loogen, Philipps-University Marburg Neil Mitchell, Standard Chartered Rex Page, Oklahoma University Rinus Plasmeijer, Radboud University Nijmegen Sven-Bodo Scholz, University of Hertfordshire Mary Sheeran, Chalmers Satnam Singh, Microsoft Research Walid Taha, Halmstad University Simon Thompson, University of Kent Geoffrey Washburn, LogicBlox ---- Andy Gill, Assistant Professor The University of Kansas, EECS Department Information and Telecommunication Technology Center http://www.ittc.ku.edu/csdl/fpg/Users/AndyGill From rickard@REDACTED Mon Aug 22 18:08:32 2011 From: rickard@REDACTED (Rickard Green) Date: Mon, 22 Aug 2011 18:08:32 +0200 Subject: [erlang-questions] CPU affinity and enif_thread_create In-Reply-To: References: Message-ID: enif_thread_create() and erl_drv_thread_create() will set the affinity of the newly created thread to the same as the affinity of the main thread (or you have found a bug). Binding of scheduler threads does not effect the affinity of the main thread. However, if you create the thread any other way (e.g. by calling pthread_create() directly), the newly created thread will inherit the affinity of the scheduler thread it happens to execute on (and it may also inherit other unwanted stuff). If you have changed the affinity of the emulator while it is running, you need to force a re-read of the cpu information by calling erlang:system_info(update_cpu_info) before it will fully take effect. What is the affinity of the emulator (main thread)? = os:getpid(). taskset -p (in os-shell) What are the affinities of the other individual threads? ps -Lp (in os-shell) will give you pids of individual threads (LWP column I think). taskset -p Which ERTS version do you use? Regards, Rickard Green, Erlang/OTP, Ericsson AB On Aug 21, 2011, at 10:47 PM, Max Lapshin wrote: > I've created libx264 encoder in nif and started new thread for it. > libx264 was started with option threads 4 and it really created 4 threads. > > Now I have problems. All created threads are bound to one CPU core and > it is not enough. 100% busy and I have to drop frames. > > I have installed linux kernel with BFS (brainfuck scheduler), but it > didn't change anything. > > So I have called sched_setaffinity by hands and libx264 threads spread > across cores. > > And I have questions: does erlang VM makes some calls, that bind > future created threads to current core? > > If no, that why doesn't erlang VM moved to other cores busy processes > that had several hundred messages inbox? > Maybe I should do some hinting like "bump_reductions" to tell VM to > switch process to another scheduler? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From moxford@REDACTED Mon Aug 22 20:31:44 2011 From: moxford@REDACTED (Mike Oxford) Date: Mon, 22 Aug 2011 11:31:44 -0700 Subject: [erlang-questions] BERT vs protobuf in the erlang world Message-ID: BERT is pretty much a "known good quantity" but protobuf is more efficient on the wire. Anyone have thoughts on which to go with? Also, along the BERT side, does anyone have recommendations on AS3 and Java implementations of the marshallers? The downside(s) to protobuf - hard IDLs and you have to build the project and generators. The downside(s) to BERT - less efficient on the wire (verbosity.) TIA! -mox From max.lapshin@REDACTED Mon Aug 22 21:19:13 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 22 Aug 2011 23:19:13 +0400 Subject: [erlang-questions] CPU affinity and enif_thread_create In-Reply-To: References: Message-ID: On Mon, Aug 22, 2011 at 8:08 PM, Rickard Green wrote: > enif_thread_create() and erl_drv_thread_create() will set the affinity of the newly created thread to the same as the affinity of the main thread (or you have found a bug). Binding of scheduler threads does not effect the affinity of the main thread. However, if you create the thread any other way (e.g. by calling pthread_create() directly), the newly created thread will inherit the affinity of the scheduler thread it happens to execute on (and it may also inherit other unwanted stuff). > I use R14B03 and I threads are started via pthread_create (inside of libx264). But after this happens very strange situation: htop shows that started threads doesn't have any affinity. If I set affinity, threads are spread across cores. Looks like erlang has nothing to do with it. From abawcom@REDACTED Tue Aug 23 00:17:08 2011 From: abawcom@REDACTED (Aaron Bawcom) Date: Mon, 22 Aug 2011 18:17:08 -0400 Subject: [erlang-questions] OTP Design In-Reply-To: References: <009c01cc5de2$a33d7e10$e9b87a30$@com> Message-ID: The new federation plugin ( http://www.rabbitmq.com/blog/2011/06/22/federation-plugin-preview-release/ ) for RabbitMQ definitely holds some promise to decentralizing AMQP and I've actually found the Erlang bindings for RabbitMQ are pretty easy and straightforward to use...if you like Erlang of course =) -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Garrett Smith Sent: Thursday, August 18, 2011 5:09 PM To: David Mercer Cc: erlang-questions@REDACTED; Anchise Inzaghi Subject: Re: [erlang-questions] OTP Design On Thu, Aug 18, 2011 at 3:08 PM, David Mercer wrote: > On Thursday, August 18, 2011, Garrett Smith wrote: > >> Apart from prototyping, I don't generally use distributed Erlang (i.e. >> starting named nodes) but prefer to use other messaging protocols >> (e.g. AMQP, 0MQ, REST, etc). > > Do you have any thoughts on 0MQ vs. AMQP? They're pretty different. 0MQ is more of a toolkit for building your own messaging systems. For centralized broker models, AMQP is great. RabbitMQ is very solid. And of course it's Erlang. I have nothing but good things to say about it. My problem with AMQP is that it tends to become a central architectural feature -- similar to the big, centralized relational databases. All of a sudden, you're feeding and caring for the broker -- and if it goes down, it takes a lot with it. AMQP is also a complicated protocol, which tends to cause problems for people implementing it. So, while there are quality servers and clients for it -- be prepared for hard-to-fix problems along the way. 0MQ is a counter point to this: support for federated and flexible topologies and simple protocol. There is a NIF based Erlang binding for it, which is excellent. You might find this interested, if you haven't seen it already: http://www.zeromq.org/whitepapers:brokerless Garrett _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From abawcom@REDACTED Tue Aug 23 00:23:02 2011 From: abawcom@REDACTED (Aaron Bawcom) Date: Mon, 22 Aug 2011 18:23:02 -0400 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: Message-ID: We went down the protobuf route for a while but ended up moving mostly everything to BERT in the end for performance reasons. Yes the byte_size is larger but the (de)serialization times offered a far bigger payoff from a performance perspective. -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Mike Oxford Sent: Monday, August 22, 2011 2:32 PM To: Erlang Questions Subject: [erlang-questions] BERT vs protobuf in the erlang world BERT is pretty much a "known good quantity" but protobuf is more efficient on the wire. Anyone have thoughts on which to go with? Also, along the BERT side, does anyone have recommendations on AS3 and Java implementations of the marshallers? The downside(s) to protobuf - hard IDLs and you have to build the project and generators. The downside(s) to BERT - less efficient on the wire (verbosity.) TIA! -mox _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Tue Aug 23 02:11:01 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 23 Aug 2011 02:11:01 +0200 Subject: [erlang-questions] Ensure if a function/arity is exported from a module or not? Message-ID: <46D68F67-B18D-442A-B43E-746D385150A6@gmail.com> Hi guys, What's the correct/best way to ensure that a module exports a certain function or not? Lets say: -module(foo). -export([main/1]). ... I need to know if module foo exports or not main/1 without having foo source code of course (only foo.beam is available). Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Tue Aug 23 02:17:20 2011 From: g@REDACTED (Garrett Smith) Date: Mon, 22 Aug 2011 19:17:20 -0500 Subject: [erlang-questions] Ensure if a function/arity is exported from a module or not? In-Reply-To: <46D68F67-B18D-442A-B43E-746D385150A6@gmail.com> References: <46D68F67-B18D-442A-B43E-746D385150A6@gmail.com> Message-ID: On Mon, Aug 22, 2011 at 7:11 PM, Zabrane Mickael wrote: > Hi guys, > What's the correct/best way to ensure that a module exports a certain > function or not? erlang:function_exported/3 From jwatte@REDACTED Tue Aug 23 02:57:06 2011 From: jwatte@REDACTED (Jon Watte) Date: Mon, 22 Aug 2011 17:57:06 -0700 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: Message-ID: We use protobuf; we wrote our own Erlang generator which is not super efficient, but better than what was available at the time. We use C++, Python, and PHP generators on other systems (we wrote the PHP generator ourselves, too). Sorry, no feedback on the Java and AS3 generators, but in the system as a whole, the protobufs work just as expected, including preservation of unknown fields, and because each 16-hardware-thread node (dual quad-core hyper-threaded Xeons) only has gigabit Ethernet, we can't really feed a single node faster than it can parse and process requests. Memory bandwidth is 50 GB/s or better; Ethernet TCP bandwidth is 110 MB/s :-) Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Mon, Aug 22, 2011 at 11:31 AM, Mike Oxford wrote: > BERT is pretty much a "known good quantity" but protobuf is more > efficient on the wire. > > Anyone have thoughts on which to go with? > > Also, along the BERT side, does anyone have recommendations on AS3 and > Java implementations of the marshallers? > > The downside(s) to protobuf - hard IDLs and you have to build the > project and generators. > The downside(s) to BERT - less efficient on the wire (verbosity.) > > TIA! > > -mox > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Tue Aug 23 07:23:38 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 23 Aug 2011 07:23:38 +0200 Subject: [erlang-questions] Ensure if a function/arity is exported from a module or not? In-Reply-To: References: <46D68F67-B18D-442A-B43E-746D385150A6@gmail.com> Message-ID: <4E240852-6984-45C4-8B0F-3DAD5BC3950E@gmail.com> Thanks a lot. On Aug 23, 2011, at 2:17 AM, Garrett Smith wrote: > erlang:function_exported/3 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rickard@REDACTED Tue Aug 23 11:51:10 2011 From: rickard@REDACTED (Rickard Green) Date: Tue, 23 Aug 2011 11:51:10 +0200 Subject: [erlang-questions] CPU affinity and enif_thread_create In-Reply-To: References: Message-ID: <4CF7E743-D38D-4669-86E4-49A73287145D@erlang.org> On Aug 22, 2011, at 9:19 PM, Max Lapshin wrote: > On Mon, Aug 22, 2011 at 8:08 PM, Rickard Green wrote: >> enif_thread_create() and erl_drv_thread_create() will set the affinity of the newly created thread to the same as the affinity of the main thread (or you have found a bug). Binding of scheduler threads does not effect the affinity of the main thread. However, if you create the thread any other way (e.g. by calling pthread_create() directly), the newly created thread will inherit the affinity of the scheduler thread it happens to execute on (and it may also inherit other unwanted stuff). >> > > I use R14B03 and I threads are started via pthread_create (inside of libx264). > In this case the new threads will inherit the affinity of the scheduler thread that calls this functionality. It is always preferred to create all threads using the functionality provided by ERTS. This might, however, not always be possible. You may fix affinity and other stuff you know about (by inspecting the emulator code) manually. This is, however, no good solution since this other stuff have and most likely will vary between releases also in the future. A better solution would be to create a new thread using enif_thread_create(), and call the functionality that creates new threads from this thread. By this, the newly created threads wont inherit scheduler specific stuff, but instead inherit settings that are appropriate for other threads in the system. > But after this happens very strange situation: htop shows that started > threads doesn't have any affinity. > How do you get htop to show affinity (I just installed it for the first time)? I only found the PROCESSOR column, but this doesn' t show affinity. I think it shows current processor assignment. > If I set affinity, threads are spread across cores. Looks like erlang > has nothing to do with it. Regards, Rickard Green, Erlang/OTP, Ericsson AB From vincent.dephily@REDACTED Tue Aug 23 12:03:29 2011 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Tue, 23 Aug 2011 12:03:29 +0200 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: Message-ID: <1682621.msH3m5tl1p@moltowork> On Monday 22 August 2011 11:31:44 Mike Oxford wrote: > BERT is pretty much a "known good quantity" but protobuf is more > efficient on the wire. > > Anyone have thoughts on which to go with? > > Also, along the BERT side, does anyone have recommendations on AS3 and > Java implementations of the marshallers? > > The downside(s) to protobuf - hard IDLs and you have to build the > project and generators. > The downside(s) to BERT - less efficient on the wire (verbosity.) Also take a look at bjson, and one of my favourite : msgpack. BERT is rather unique in supporting atoms, which *nice* and can reduce encoded size a lot (but that's less significant if you compress). For encoded size, I have yet to see anything beat UPER (asn1). A few things to consider : * Bench size and speed with a mockup of your data : different formats are better for different types of data. Add compression to your tests. * Think about format extensibility. It's easy enough with self-described formats like bert or msgpack, but gets more complicated with protobuf or asn1. * The initial hurdle of dealing with a grammar and generators is well paid off by having data validation and nicer api (mostly via #records{}) later on. * If you need to speak with the outside world, self-describing formats will make your life easyer. -- Vincent de Phily From max.lapshin@REDACTED Tue Aug 23 12:16:00 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 23 Aug 2011 14:16:00 +0400 Subject: [erlang-questions] CPU affinity and enif_thread_create In-Reply-To: <4CF7E743-D38D-4669-86E4-49A73287145D@erlang.org> References: <4CF7E743-D38D-4669-86E4-49A73287145D@erlang.org> Message-ID: > > How do you get htop to show affinity (I just installed it for the first time)? I only found the PROCESSOR column, but this doesn' t show affinity. I think it shows current processor assignment. > If to press "a", it claims to show affinity. But I'm not sure that it gives proper result. From rickard@REDACTED Tue Aug 23 14:43:02 2011 From: rickard@REDACTED (Rickard Green) Date: Tue, 23 Aug 2011 14:43:02 +0200 Subject: [erlang-questions] CPU affinity and enif_thread_create In-Reply-To: References: <4CF7E743-D38D-4669-86E4-49A73287145D@erlang.org> Message-ID: <81C0516A-EE04-496D-8CA5-DE398F3370BF@erlang.org> On Aug 23, 2011, at 12:16 PM, Max Lapshin wrote: >> >> How do you get htop to show affinity (I just installed it for the first time)? I only found the PROCESSOR column, but this doesn' t show affinity. I think it shows current processor assignment. >> > > If to press "a", it claims to show affinity. But I'm not sure that it > gives proper result. As I understand it, 'a' can only be used for setting affinity not reading it, but I could be wrong. Regards, Rickard Green, Erlang/OTP, Ericsson AB From bile@REDACTED Tue Aug 23 14:54:49 2011 From: bile@REDACTED (Antonio SJ Musumeci) Date: Tue, 23 Aug 2011 08:54:49 -0400 Subject: [erlang-questions] net_kernel:allow/1 doc mismatch? Message-ID: <4E53A319.4050508@landofbile.com> I realize this probably isn't a widely used feature but I may have a use for it and it seems the docs don't accurately represent the functionality and the actual functionality seems problematic. The docs indicate "Limits access to the specified set of nodes. Any access attempts made from (or to) nodes not in Nodes will be rejected." However, it actually concats the old list with the new: State#state{allowed = Allowed ++ Nodes}. Due to this there is no way to remove nodes which the docs imply is possible. Is it possible to have this changed to being State#state{allowed = Nodes} in a future release? Or if there is a fear it will break something for someone a new function to change it as defined? An accessor to that list would be useful too. From paul.james.barry@REDACTED Tue Aug 23 17:51:47 2011 From: paul.james.barry@REDACTED (Paul Barry) Date: Tue, 23 Aug 2011 16:51:47 +0100 Subject: [erlang-questions] Upcoming O'Reilly Webcast on Erlang Message-ID: Hi all. Just a quick announcement... I'll be presenting an on-line, live webcast for O'Reilly Media on Sept 22 (Thurs) at 6pm (Irish-time, GMT) entitled "An Introduction to Erlang (for Python programmers)". Sign-up is FREE, see the details here: http://oreillynet.com/pub/e/2022 I'd be delighted if you could come alone and critique, comment and/or cheer-lead as appropriate. :-) Paul. -- Paul Barry, w: http://paulbarry.itcarlow.ie, e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From jwatte@REDACTED Tue Aug 23 22:17:12 2011 From: jwatte@REDACTED (Jon Watte) Date: Tue, 23 Aug 2011 13:17:12 -0700 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <1682621.msH3m5tl1p@moltowork> References: <1682621.msH3m5tl1p@moltowork> Message-ID: Protobuf is fully self-described, and fully forwards/backwards compatible, in that it can carry through "unknown" members of a structure, including the semantic of the fields. ASN1 is generally not fully self-described, because it only describes type, not semantic, for each field. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Aug 23, 2011 at 3:03 AM, Vincent de Phily < vincent.dephily@REDACTED> wrote: > On Monday 22 August 2011 11:31:44 Mike Oxford wrote: > > BERT is pretty much a "known good quantity" but protobuf is more > > efficient on the wire. > > > > Anyone have thoughts on which to go with? > > > > Also, along the BERT side, does anyone have recommendations on AS3 and > > Java implementations of the marshallers? > > > > The downside(s) to protobuf - hard IDLs and you have to build the > > project and generators. > > The downside(s) to BERT - less efficient on the wire (verbosity.) > > Also take a look at bjson, and one of my favourite : msgpack. BERT is > rather > unique in supporting atoms, which *nice* and can reduce encoded size a lot > (but that's less significant if you compress). For encoded size, I have yet > to > see anything beat UPER (asn1). > > A few things to consider : > * Bench size and speed with a mockup of your data : different formats are > better for different types of data. Add compression to your tests. > * Think about format extensibility. It's easy enough with self-described > formats like bert or msgpack, but gets more complicated with protobuf or > asn1. > * The initial hurdle of dealing with a grammar and generators is well paid > off > by having data validation and nicer api (mostly via #records{}) later on. > * If you need to speak with the outside world, self-describing formats will > make your life easyer. > -- > Vincent de Phily > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Wed Aug 24 00:42:39 2011 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 23 Aug 2011 15:42:39 -0700 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <1682621.msH3m5tl1p@moltowork> Message-ID: <4E542CDF.5090208@gmail.com> I think a properly done ASN.1 file can be self-describing and can provide great documentation of a protocol. It gives you a lot of freedom and has been well tested within the Erlang distribution. The one good thing about protobufs is their handling of floating point makes more sense for general use (no embedded compatibility problems), it doesn't actually change the base of the number, internally, which would then alter the precision. On 08/23/2011 01:17 PM, Jon Watte wrote: > Protobuf is fully self-described, and fully forwards/backwards compatible, in that it can carry through "unknown" members of a structure, including the semantic of the fields. ASN1 is generally not fully self-described, because it only describes type, not semantic, for each field. > > Sincerely, > > jw > > -- > Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. > > > > On Tue, Aug 23, 2011 at 3:03 AM, Vincent de Phily > wrote: > > On Monday 22 August 2011 11:31:44 Mike Oxford wrote: > > BERT is pretty much a "known good quantity" but protobuf is more > > efficient on the wire. > > > > Anyone have thoughts on which to go with? > > > > Also, along the BERT side, does anyone have recommendations on AS3 and > > Java implementations of the marshallers? > > > > The downside(s) to protobuf - hard IDLs and you have to build the > > project and generators. > > The downside(s) to BERT - less efficient on the wire (verbosity.) > > Also take a look at bjson, and one of my favourite : msgpack. BERT is rather > unique in supporting atoms, which *nice* and can reduce encoded size a lot > (but that's less significant if you compress). For encoded size, I have yet to > see anything beat UPER (asn1). > > A few things to consider : > * Bench size and speed with a mockup of your data : different formats are > better for different types of data. Add compression to your tests. > * Think about format extensibility. It's easy enough with self-described > formats like bert or msgpack, but gets more complicated with protobuf or > asn1. > * The initial hurdle of dealing with a grammar and generators is well paid off > by having data validation and nicer api (mostly via #records{}) later on. > * If you need to speak with the outside world, self-describing formats will > make your life easyer. > -- > Vincent de Phily > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Wed Aug 24 01:22:46 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Tue, 23 Aug 2011 16:22:46 -0700 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <1682621.msH3m5tl1p@moltowork> Message-ID: <20110823232246.GB4977@alumni.caltech.edu> Protobuf is not what I would call self describing, if you have something like message foo { required int32 bar = 1; } message foo2 { optional int32 bar2 = 1; } and create an instance of each and then send it remotely, you can not distinguish between the 2 messages, unless the remote side know when one you mean to send. Here's an example in erlang Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) 1> rr(foo_pb). [foo,foo2] 2> foo_pb:encode(#foo{bar=42}). <<"\b*">> 3> foo_pb:encode(#foo2{bar2=42}). <<"\b*">> To me a self-describing protocol is something which also encoded the names of the messages and fields, so you could send it through to the other side and it would be unambiguous. An example of this is a system I wrote many years ago called lwes (www.lwes.org), which does not require code generation because messages are self describing. Now this does mean messages are longer, but sometimes that is okay. -Anthony On Tue, Aug 23, 2011 at 01:17:12PM -0700, Jon Watte wrote: > Protobuf is fully self-described, and fully forwards/backwards compatible, > in that it can carry through "unknown" members of a structure, including the > semantic of the fields. ASN1 is generally not fully self-described, because > it only describes type, not semantic, for each field. > > Sincerely, > > jw > > -- > Americans might object: there is no way we would sacrifice our living > standards for the benefit of people in the rest of the world. Nevertheless, > whether we get there willingly or not, we shall soon have lower consumption > rates, because our present rates are unsustainable. > > > > On Tue, Aug 23, 2011 at 3:03 AM, Vincent de Phily < > vincent.dephily@REDACTED> wrote: > > > On Monday 22 August 2011 11:31:44 Mike Oxford wrote: > > > BERT is pretty much a "known good quantity" but protobuf is more > > > efficient on the wire. > > > > > > Anyone have thoughts on which to go with? > > > > > > Also, along the BERT side, does anyone have recommendations on AS3 and > > > Java implementations of the marshallers? > > > > > > The downside(s) to protobuf - hard IDLs and you have to build the > > > project and generators. > > > The downside(s) to BERT - less efficient on the wire (verbosity.) > > > > Also take a look at bjson, and one of my favourite : msgpack. BERT is > > rather > > unique in supporting atoms, which *nice* and can reduce encoded size a lot > > (but that's less significant if you compress). For encoded size, I have yet > > to > > see anything beat UPER (asn1). > > > > A few things to consider : > > * Bench size and speed with a mockup of your data : different formats are > > better for different types of data. Add compression to your tests. > > * Think about format extensibility. It's easy enough with self-described > > formats like bert or msgpack, but gets more complicated with protobuf or > > asn1. > > * The initial hurdle of dealing with a grammar and generators is well paid > > off > > by having data validation and nicer api (mostly via #records{}) later on. > > * If you need to speak with the outside world, self-describing formats will > > make your life easyer. > > -- > > Vincent de Phily > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ------------------------------------------------------------------------ Anthony Molinaro From mark.geib@REDACTED Wed Aug 24 01:34:15 2011 From: mark.geib@REDACTED (Mark Geib) Date: Tue, 23 Aug 2011 17:34:15 -0600 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <20110823232246.GB4977@alumni.caltech.edu> References: <1682621.msH3m5tl1p@moltowork> <20110823232246.GB4977@alumni.caltech.edu> Message-ID: <4E5438F7.8050608@echostar.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 We almost always end up with a wrapper message def that includes a "type" enum field and a "message" bytes field that holds the actual encoded message of interest. However, in the implementation it's a bit ugly due to the dual encodes and decodes required, and it makes the messages a few bytes larger of course. Mark. On 08/23/2011 05:22 PM, Anthony Molinaro wrote: > Protobuf is not what I would call self describing, if you have > something like > > message foo { required int32 bar = 1; } > > message foo2 { optional int32 bar2 = 1; } > > and create an instance of each and then send it remotely, you can > not distinguish between the 2 messages, unless the remote side know > when one you mean to send. > > Here's an example in erlang > > Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.4 (abort with ^G) 1> rr(foo_pb). [foo,foo2] 2> > foo_pb:encode(#foo{bar=42}). <<"\b*">> 3> > foo_pb:encode(#foo2{bar2=42}). <<"\b*">> > > To me a self-describing protocol is something which also encoded > the names of the messages and fields, so you could send it through > to the other side and it would be unambiguous. An example of this > is a system I wrote many years ago called lwes (www.lwes.org), > which does not require code generation because messages are self > describing. > > Now this does mean messages are longer, but sometimes that is > okay. > > -Anthony > > On Tue, Aug 23, 2011 at 01:17:12PM -0700, Jon Watte wrote: >> Protobuf is fully self-described, and fully forwards/backwards >> compatible, in that it can carry through "unknown" members of a >> structure, including the semantic of the fields. ASN1 is >> generally not fully self-described, because it only describes >> type, not semantic, for each field. >> >> Sincerely, >> >> jw >> >> -- Americans might object: there is no way we would sacrifice our >> living standards for the benefit of people in the rest of the >> world. Nevertheless, whether we get there willingly or not, we >> shall soon have lower consumption rates, because our present >> rates are unsustainable. >> >> >> >> On Tue, Aug 23, 2011 at 3:03 AM, Vincent de Phily < >> vincent.dephily@REDACTED> wrote: >> >>> On Monday 22 August 2011 11:31:44 Mike Oxford wrote: >>>> BERT is pretty much a "known good quantity" but protobuf is >>>> more efficient on the wire. >>>> >>>> Anyone have thoughts on which to go with? >>>> >>>> Also, along the BERT side, does anyone have recommendations >>>> on AS3 and Java implementations of the marshallers? >>>> >>>> The downside(s) to protobuf - hard IDLs and you have to build >>>> the project and generators. The downside(s) to BERT - less >>>> efficient on the wire (verbosity.) >>> >>> Also take a look at bjson, and one of my favourite : msgpack. >>> BERT is rather unique in supporting atoms, which *nice* and can >>> reduce encoded size a lot (but that's less significant if you >>> compress). For encoded size, I have yet to see anything beat >>> UPER (asn1). >>> >>> A few things to consider : * Bench size and speed with a mockup >>> of your data : different formats are better for different types >>> of data. Add compression to your tests. * Think about format >>> extensibility. It's easy enough with self-described formats >>> like bert or msgpack, but gets more complicated with protobuf >>> or asn1. * The initial hurdle of dealing with a grammar and >>> generators is well paid off by having data validation and nicer >>> api (mostly via #records{}) later on. * If you need to speak >>> with the outside world, self-describing formats will make your >>> life easyer. -- Vincent de Phily >>> _______________________________________________ >>> erlang-questions mailing list erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> > >> _______________________________________________ erlang-questions >> mailing list erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > - -- Principal Engineer Cheyenne Software Engineering mark.geib@REDACTED / 35-215 PGP fingerprint:6DFC 389D 9796 0188 92E5 58F5 34C5 6B47 D091 76FD -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk5UOPcACgkQNMVrR9CRdv0+hACfXy1jEg0Lisbj5pPqFQt4tK8d be8AnRCDPLKzLZE1cpfsUp8LMUfv84A+ =WqFv -----END PGP SIGNATURE----- From steven.charles.davis@REDACTED Wed Aug 24 01:37:41 2011 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 23 Aug 2011 16:37:41 -0700 (PDT) Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: Message-ID: <749598ac-6b24-421f-9dce-d28bee5f5a34@f41g2000yqh.googlegroups.com> Why not term_to_binary() i.e. external erlang term format...? ...a genuine question looking for reasoning/insight from those more experienced. Additional - if not that, then what about Joe Armstrong's UBF? /s On Aug 22, 1:31?pm, Mike Oxford wrote: > BERT is pretty much a "known good quantity" but protobuf is more > efficient on the wire. From mark.geib@REDACTED Wed Aug 24 01:38:45 2011 From: mark.geib@REDACTED (Mark Geib) Date: Tue, 23 Aug 2011 17:38:45 -0600 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: Message-ID: <4E543A05.1000409@echostar.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Any chance of sharing your erlang generator for protobuf.? We have been using protobuf with erlang for a while and F-I-I-I-ghting with the generation of erlang code for our protobufs. Or, a pointer to a quality generator some one else knows of would be appreciated. Mark. On 08/22/2011 06:57 PM, Jon Watte wrote: > We use protobuf; we wrote our own Erlang generator which is not > super efficient, but better than what was available at the time. We > use C++, Python, and PHP generators on other systems (we wrote the > PHP generator ourselves, too). Sorry, no feedback on the Java and > AS3 generators, but in the system as a whole, the protobufs work > just as expected, including preservation of unknown fields, and > because each 16-hardware-thread node (dual quad-core hyper-threaded > Xeons) only has gigabit Ethernet, we can't really feed a single > node faster than it can parse and process requests. Memory > bandwidth is 50 GB/s or better; Ethernet TCP bandwidth is 110 MB/s > :-) > > Sincerely, > > jw > > > -- Americans might object: there is no way we would sacrifice our > living standards for the benefit of people in the rest of the > world. Nevertheless, whether we get there willingly or not, we > shall soon have lower consumption rates, because our present rates > are unsustainable. > > > > On Mon, Aug 22, 2011 at 11:31 AM, Mike Oxford > wrote: > > BERT is pretty much a "known good quantity" but protobuf is more > efficient on the wire. > > Anyone have thoughts on which to go with? > > Also, along the BERT side, does anyone have recommendations on AS3 > and Java implementations of the marshallers? > > The downside(s) to protobuf - hard IDLs and you have to build the > project and generators. The downside(s) to BERT - less efficient on > the wire (verbosity.) > > TIA! > > -mox _______________________________________________ > erlang-questions mailing list erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > - -- Principal Engineer Cheyenne Software Engineering mark.geib@REDACTED / 35-215 PGP fingerprint:6DFC 389D 9796 0188 92E5 58F5 34C5 6B47 D091 76FD -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk5UOgUACgkQNMVrR9CRdv3CtACfeMVSUHXNnBxMaRss1FInSMtR G7gAoIPtCg+C9n++vauXWoQgtzwCo4mU =ARk3 -----END PGP SIGNATURE----- From kennethstone@REDACTED Wed Aug 24 01:41:38 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Tue, 23 Aug 2011 18:41:38 -0500 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <4E543A05.1000409@echostar.com> References: <4E543A05.1000409@echostar.com> Message-ID: This should be very high quality for protobuf/erlang: https://github.com/basho/erlang_protobuffs On Tue, Aug 23, 2011 at 6:38 PM, Mark Geib wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Any chance of sharing your erlang generator for protobuf.? We have > been using protobuf with erlang for a while and F-I-I-I-ghting with > the generation of erlang code for our protobufs. Or, a pointer to a > quality generator some one else knows of would be appreciated. > > Mark. > > On 08/22/2011 06:57 PM, Jon Watte wrote: > > We use protobuf; we wrote our own Erlang generator which is not > > super efficient, but better than what was available at the time. We > > use C++, Python, and PHP generators on other systems (we wrote the > > PHP generator ourselves, too). Sorry, no feedback on the Java and > > AS3 generators, but in the system as a whole, the protobufs work > > just as expected, including preservation of unknown fields, and > > because each 16-hardware-thread node (dual quad-core hyper-threaded > > Xeons) only has gigabit Ethernet, we can't really feed a single > > node faster than it can parse and process requests. Memory > > bandwidth is 50 GB/s or better; Ethernet TCP bandwidth is 110 MB/s > > :-) > > > > Sincerely, > > > > jw > > > > > > -- Americans might object: there is no way we would sacrifice our > > living standards for the benefit of people in the rest of the > > world. Nevertheless, whether we get there willingly or not, we > > shall soon have lower consumption rates, because our present rates > > are unsustainable. > > > > > > > > On Mon, Aug 22, 2011 at 11:31 AM, Mike Oxford > > wrote: > > > > BERT is pretty much a "known good quantity" but protobuf is more > > efficient on the wire. > > > > Anyone have thoughts on which to go with? > > > > Also, along the BERT side, does anyone have recommendations on AS3 > > and Java implementations of the marshallers? > > > > The downside(s) to protobuf - hard IDLs and you have to build the > > project and generators. The downside(s) to BERT - less efficient on > > the wire (verbosity.) > > > > TIA! > > > > -mox _______________________________________________ > > erlang-questions mailing list erlang-questions@REDACTED > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > - -- > Principal Engineer > Cheyenne Software Engineering > mark.geib@REDACTED / 35-215 > > PGP fingerprint:6DFC 389D 9796 0188 92E5 58F5 34C5 6B47 D091 76FD > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAk5UOgUACgkQNMVrR9CRdv3CtACfeMVSUHXNnBxMaRss1FInSMtR > G7gAoIPtCg+C9n++vauXWoQgtzwCo4mU > =ARk3 > -----END PGP SIGNATURE----- > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.geib@REDACTED Wed Aug 24 01:48:02 2011 From: mark.geib@REDACTED (Mark Geib) Date: Tue, 23 Aug 2011 17:48:02 -0600 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <4E543A05.1000409@echostar.com> Message-ID: <4E543C32.1060906@echostar.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I have looked at that before, but the parsing of .proto files and generating erlang code is marked as alpha - do not use in production. Thanks, Mark. On 08/23/2011 05:41 PM, Kenny Stone wrote: > This should be very high quality for protobuf/erlang: > https://github.com/basho/erlang_protobuffs > > On Tue, Aug 23, 2011 at 6:38 PM, Mark Geib > wrote: > > > Any chance of sharing your erlang generator for protobuf.? We have > been using protobuf with erlang for a while and F-I-I-I-ghting > with the generation of erlang code for our protobufs. Or, a pointer > to a quality generator some one else knows of would be > appreciated. > > Mark. > > On 08/22/2011 06:57 PM, Jon Watte wrote: >> We use protobuf; we wrote our own Erlang generator which is not >> super efficient, but better than what was available at the > time. We >> use C++, Python, and PHP generators on other systems (we wrote >> the PHP generator ourselves, too). Sorry, no feedback on the Java >> and AS3 generators, but in the system as a whole, the protobufs >> work just as expected, including preservation of unknown fields, >> and because each 16-hardware-thread node (dual quad-core > hyper-threaded >> Xeons) only has gigabit Ethernet, we can't really feed a single >> node faster than it can parse and process requests. Memory >> bandwidth is 50 GB/s or better; Ethernet TCP bandwidth is 110 >> MB/s :-) > >> Sincerely, > >> jw > > >> -- Americans might object: there is no way we would sacrifice >> our living standards for the benefit of people in the rest of >> the world. Nevertheless, whether we get there willingly or not, >> we shall soon have lower consumption rates, because our present >> rates are unsustainable. > > > >> On Mon, Aug 22, 2011 at 11:31 AM, Mike Oxford > >> >> wrote: > >> BERT is pretty much a "known good quantity" but protobuf is more >> efficient on the wire. > >> Anyone have thoughts on which to go with? > >> Also, along the BERT side, does anyone have recommendations on >> AS3 and Java implementations of the marshallers? > >> The downside(s) to protobuf - hard IDLs and you have to build >> the project and generators. The downside(s) to BERT - less > efficient on >> the wire (verbosity.) > >> TIA! > >> -mox _______________________________________________ >> erlang-questions mailing list erlang-questions@REDACTED > >> > >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions - -- Principal Engineer Cheyenne Software Engineering mark.geib@REDACTED / 35-215 PGP fingerprint:6DFC 389D 9796 0188 92E5 58F5 34C5 6B47 D091 76FD -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk5UPDIACgkQNMVrR9CRdv03lwCeMzF1BCwLaXjs2bYxlH7eKTty 16IAniDWTCVpGKMdU0jiWmS3FpfoOFc4 =W+PE -----END PGP SIGNATURE----- From andrew@REDACTED Wed Aug 24 02:17:28 2011 From: andrew@REDACTED (Andrew Thompson) Date: Tue, 23 Aug 2011 20:17:28 -0400 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <4E543C32.1060906@echostar.com> References: <4E543A05.1000409@echostar.com> <4E543C32.1060906@echostar.com> Message-ID: <20110824001728.GA2315@hijacked.us> On Tue, Aug 23, 2011 at 05:48:02PM -0600, Mark Geib wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I have looked at that before, but the parsing of .proto files and > generating erlang code is marked as alpha - do not use in production. > I think this is a case of a stale README. Riak uses this library and we generate code from a .proto file without issues. Andrew From Mark.Geib@REDACTED Wed Aug 24 04:30:25 2011 From: Mark.Geib@REDACTED (Geib, Mark) Date: Tue, 23 Aug 2011 20:30:25 -0600 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <20110824001728.GA2315@hijacked.us> References: <4E543A05.1000409@echostar.com> <4E543C32.1060906@echostar.com> <20110824001728.GA2315@hijacked.us> Message-ID: Andrew, thanks for the info. I will certainly give it a try. Mark. Sent from my iPhone On Aug 23, 2011, at 6:17 PM, "Andrew Thompson" wrote: > On Tue, Aug 23, 2011 at 05:48:02PM -0600, Mark Geib wrote: >> >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> I have looked at that before, but the parsing of .proto files and >> generating erlang code is marked as alpha - do not use in production. >> > I think this is a case of a stale README. Riak uses this library and we > generate code from a .proto file without issues. > > Andrew > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From anthonym@REDACTED Wed Aug 24 04:47:28 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Tue, 23 Aug 2011 19:47:28 -0700 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <4E543A05.1000409@echostar.com> <4E543C32.1060906@echostar.com> <20110824001728.GA2315@hijacked.us> Message-ID: <20110824024728.GC4977@alumni.caltech.edu> One thing, I have a fork of erlang_protobuffs which fixes a minor issue with capitalized entries in the .proto file here https://github.com/openx/erlang_protobuffs It sort of hit me in production which sucked, but other than that I've been using one or another form of erlang_protobuffs for some time. I'll put a pull request into the basho version since it's probably useful to others. -Anthony On Tue, Aug 23, 2011 at 08:30:25PM -0600, Geib, Mark wrote: > Andrew, thanks for the info. I will certainly give it a try. > Mark. > > Sent from my iPhone > > On Aug 23, 2011, at 6:17 PM, "Andrew Thompson" wrote: > > > On Tue, Aug 23, 2011 at 05:48:02PM -0600, Mark Geib wrote: > >> > >> -----BEGIN PGP SIGNED MESSAGE----- > >> Hash: SHA1 > >> > >> I have looked at that before, but the parsing of .proto files and > >> generating erlang code is marked as alpha - do not use in production. > >> > > I think this is a case of a stale README. Riak uses this library and we > > generate code from a .proto file without issues. > > > > Andrew > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ------------------------------------------------------------------------ Anthony Molinaro From alavrik@REDACTED Wed Aug 24 05:25:43 2011 From: alavrik@REDACTED (Anton Lavrik) Date: Tue, 23 Aug 2011 22:25:43 -0500 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <4E543A05.1000409@echostar.com> References: <4E543A05.1000409@echostar.com> Message-ID: On Tue, Aug 23, 2011 at 6:38 PM, Mark Geib wrote: > Any chance of sharing your erlang generator for protobuf.? We have > been using protobuf with erlang for a while and F-I-I-I-ghting with > the generation of erlang code for our protobufs. Or, a pointer to a > quality generator some one else knows of would be appreciated. Piqi [1] generates pretty good Protocol Buffers (de)serialization Erlang code. It relies on native Protocol Buffers compiler for parsing .proto files and strives for 100% compatibility with Protocol Buffers. Piqi also supports custom Erlang names and has some other features making it more convenient for Erlang users. As far as I can tell, the other Protocol Buffers implementations [2] are less optimized, doesn't generate type specs (meaning it is not usable with Dialyzer) and doesn't handle some corner cases correctly. For instance, non-packed fields don't seem to be recognized by packed field deserializers. [1] http:/piqi.org/doc/erlang [2] https://github.com/basho/erlang_protobuffs and similar GitHub forks Anton From vincent.dephily@REDACTED Wed Aug 24 11:34:47 2011 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Wed, 24 Aug 2011 11:34:47 +0200 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <20110823232246.GB4977@alumni.caltech.edu> References: <20110823232246.GB4977@alumni.caltech.edu> Message-ID: <1974579.aqOgX6F3qA@moltowork> On Tuesday 23 August 2011 16:22:46 Anthony Molinaro wrote: > To me a self-describing protocol is something which also encoded the names > of the messages and fields, so you could send it through to the other side > and it would be unambiguous. An example of this is a system I wrote many > years ago called lwes (www.lwes.org), which does not require code generation > because messages are self describing. Yes, that's what I meant by self-describing. You can dump an unknown message and know not only that it's {42, "car"} but also that it's {id:42, class:"car"}. Protobuf takes carefull steps to enable grammar extensibility, but it is still not as flexible as a self-described format. You cannot switch a field to be optional, add a required field, reorder fields, or start sending a list of objects when you used to send a single one for example. One thing that irks me about protobuf is that it doesn't have proper "optional" fields: you'll always end up with a default value instead of "not present" and decoders won't tell you if they inferred the default value themselves. You can kludge around by using a sentinel value or a field repeated zero times but this is not the same thing. If you're going the grammar-described-format route, I'd go with asn1 instead of protobuf unless : * support for $OTHER_LANGUAGE is needed (despite being nearly as old as me, asn1 support isn't ubiquitous yet, and sometimes costs a fortune). * codec speed is paramount (I haven't benched one against the other for CPU usage, but I expect protobuf to have the edge) With asn1, you get a better grammar that supports : * DEFAULT vs OPTIONAL * CHOICE (like a protobuf enum, except values can be anything, not just integer) * checked ranges for everything, from integers to list size to character set (why waste a full byte when you're only using characters [a-z_] ?) * Neat tricks like 'NULL' and single-value ranges Extensibility of the BER variant is just as good/quirky as protobuf's, and if your bytes cost money you can spend less with the UPER variant. -- Vincent de Phily From hknight555@REDACTED Wed Aug 24 13:51:02 2011 From: hknight555@REDACTED (Hank Knight) Date: Wed, 24 Aug 2011 08:51:02 -0300 Subject: [erlang-questions] Erlang Scheduler for Recurring Events Message-ID: Hello, I need to scheduler a number of recurring events using Erlang. Some events need to be triggered every second, some every three seconds and some every minute. Any simple advise on how to do this or what tools to do? - Hank From zabrane3@REDACTED Wed Aug 24 13:56:03 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 24 Aug 2011 13:56:03 +0200 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: References: Message-ID: <2B8E5EE0-05DB-4C02-BA97-C8433EEBE868@gmail.com> Sounds like you need something like Unix "cron" : a. ecron: http://gitorious.org/ecron b. crone: http://catseye.tc/projects/crone/ Regards, Zabrane On Aug 24, 2011, at 1:51 PM, Hank Knight wrote: > Hello, > > I need to scheduler a number of recurring events using Erlang. Some > events need to be triggered every second, some every three seconds and > some every minute. Any simple advise on how to do this or what tools > to do? > > - Hank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vincent.dephily@REDACTED Wed Aug 24 14:56:02 2011 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Wed, 24 Aug 2011 14:56:02 +0200 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <749598ac-6b24-421f-9dce-d28bee5f5a34@f41g2000yqh.googlegroups.com> References: <749598ac-6b24-421f-9dce-d28bee5f5a34@f41g2000yqh.googlegroups.com> Message-ID: <8321958.OU6YYbbWVL@moltowork> On Tuesday 23 August 2011 16:37:41 Steve Davis wrote: > Why not term_to_binary() i.e. external erlang term format...? > > ...a genuine question looking for reasoning/insight from those more > experienced. BERT is exactly that, plus some checks to remove data types like erlang pids which should not exist in a language-agnostic format. BERT is faster than similar formats in the erlang VM, but that's an erlang-specific artefact. In the outside world, msgpack and bjson have more traction and more speed. -- Vincent de Phily From tomas.abrahamsson@REDACTED Wed Aug 24 15:09:13 2011 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Wed, 24 Aug 2011 15:09:13 +0200 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <4E543A05.1000409@echostar.com> References: <4E543A05.1000409@echostar.com> Message-ID: > Any chance of sharing your erlang generator for protobuf.? Here's yet another erlang generator for protobuf: https://github.com/tomas-abrahamsson/gpb (shameless self-plug :) ) We've used it at work for quite some time now. The parser handles most cases I've come across.[*] and the encoder/decoder is fully compatible with google protobuf as far as I know, also packed fields and negative enums. I originally wrote it because I needed support for messages in messages, which I couldn't find in any other protobuf library at that time. Since then, encoding/decoding performance has been a bit of an issue. There are also some QuickCheck properties here[**] for verifying encoding/decoding and also for compability with 'protoc' of Google protobuf. Works with QuickCheck-mini. Many thanks to Thomas Arts for writing a fair part of the generators for message definitions and messages! [*] The parser is a yecc/leex parser, so it gets confused if proto-reserved words such as int32 occur as for example a message name. I expect this to be pretty rare in practice. It correctly parses and handles the descriptor.proto file in protobuf. [**] https://github.com/tomas-abrahamsson/gpb-eqc BRs Tomas From steven.charles.davis@REDACTED Wed Aug 24 15:37:43 2011 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 24 Aug 2011 08:37:43 -0500 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <8321958.OU6YYbbWVL@moltowork> References: <749598ac-6b24-421f-9dce-d28bee5f5a34@f41g2000yqh.googlegroups.com> <8321958.OU6YYbbWVL@moltowork> Message-ID: <4E54FEA7.5030200@gmail.com> Short and insightful. Thank you, Vincent! I took the time to google/read the page at bert-rpc.org (and also scan over msgpack and bjson). WRT to the OP, and in my simple-minded world, protobuffs always seemed like a lot of hassle for not much gain; And if there is concern about the verbosity or sending big binaries why not use gzip? Maybe there's room for an extension to BERT/BERT-RPC to define the use of gzip compression? My 2c. /s On 8/24/2011 7:56 AM, Vincent de Phily wrote: > On Tuesday 23 August 2011 16:37:41 Steve Davis wrote: >> Why not term_to_binary() i.e. external erlang term format...? >> >> ...a genuine question looking for reasoning/insight from those more >> experienced. > > BERT is exactly that, plus some checks to remove data types like erlang pids > which should not exist in a language-agnostic format. BERT is faster than > similar formats in the erlang VM, but that's an erlang-specific artefact. In > the outside world, msgpack and bjson have more traction and more speed. From anders.nygren@REDACTED Wed Aug 24 15:54:01 2011 From: anders.nygren@REDACTED (Anders Nygren) Date: Wed, 24 Aug 2011 08:54:01 -0500 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: References: Message-ID: Hi I think that the simplest way is to just use timer:apply_interval or timer:send_interval from stdlib. /Anders On Wed, Aug 24, 2011 at 6:51 AM, Hank Knight wrote: > Hello, > > I need to scheduler a number of recurring events using Erlang. ?Some > events need to be triggered every second, some every three seconds and > some every minute. ?Any simple advise on how to do this or what tools > to do? > > - Hank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From g@REDACTED Wed Aug 24 15:59:08 2011 From: g@REDACTED (Garrett Smith) Date: Wed, 24 Aug 2011 08:59:08 -0500 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: References: Message-ID: On Wed, Aug 24, 2011 at 6:51 AM, Hank Knight wrote: > Hello, > > I need to scheduler a number of recurring events using Erlang. ?Some > events need to be triggered every second, some every three seconds and > some every minute. ?Any simple advise on how to do this or what tools > to do? A super simple approach is to use one of the 'interval' functions in the timer module. But this can "pile up" requests if your work takes longer than the interval. I personally don't like even the possibility of that, so I use erlang:send_after/3 from within the working process. Assuming you use a gen_server to perform the work, you can trigger subsequent events without floating if you save your original "start" time in state and use this algorithm to calculate value for Time in the send_after call: next_timer(#state{start=S, interval=I}) -> Now = timestamp(), ((Now - S) div I + 1) * I + S - Now. The "now" timestamp is epoch milliseconds (also used to calculate Start): timestamp() -> {M, S, U} = erlang:now(), M * 1000000000 + S * 1000 + U div 1000. This approach prevents the pileup problem -- if your work takes longer than interval, that period is simply skipped. Of course if you don't care about regular intervals, you can track your work time and subtract it from interval. Garrett From daniel@REDACTED Wed Aug 24 16:23:16 2011 From: daniel@REDACTED (Daniel Luna) Date: Wed, 24 Aug 2011 10:23:16 -0400 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: References: Message-ID: On 24 August 2011 09:59, Garrett Smith wrote: > On Wed, Aug 24, 2011 at 6:51 AM, Hank Knight wrote: >> Hello, >> >> I need to scheduler a number of recurring events using Erlang. ?Some >> events need to be triggered every second, some every three seconds and >> some every minute. ?Any simple advise on how to do this or what tools >> to do? [cut away a good solution to the problem] > Of course if you don't care about regular intervals, you can track > your work time and subtract it from interval. If you don't care about exact regular intervals the easiest thing would be to use the timeout feature of gen_server. Initialize your gen_server with {ok, State, 3000} for a three second wait to the next event. Add a handle_info(timeout, ...), and make sure that you set the timeout at every return point (i.e. {noreply, State, Timeout} for handle_info). This of course only works if the gen_server doesn't receive messages from elsewhere since then the timeout will never occur (it is a timeout after all). (If this method counts as good erlang coding style I leave to others to decide.) Cheers, Daniel From g@REDACTED Wed Aug 24 16:32:27 2011 From: g@REDACTED (Garrett Smith) Date: Wed, 24 Aug 2011 09:32:27 -0500 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: References: Message-ID: On Wed, Aug 24, 2011 at 9:23 AM, Daniel Luna wrote: > On 24 August 2011 09:59, Garrett Smith wrote: >> On Wed, Aug 24, 2011 at 6:51 AM, Hank Knight wrote: >>> Hello, >>> >>> I need to scheduler a number of recurring events using Erlang. ?Some >>> events need to be triggered every second, some every three seconds and >>> some every minute. ?Any simple advise on how to do this or what tools >>> to do? > [cut away a good solution to the problem] >> Of course if you don't care about regular intervals, you can track >> your work time and subtract it from interval. > > If you don't care about exact regular intervals the easiest thing > would be to use the timeout feature of gen_server. > > Initialize your gen_server with {ok, State, 3000} for a three second > wait to the next event. ?Add a handle_info(timeout, ...), and make > sure that you set the timeout at every return point (i.e. {noreply, > State, Timeout} for handle_info). > > This of course only works if the gen_server doesn't receive messages > from elsewhere since then the timeout will never occur (it is a > timeout after all). I'd avoid timeouts for this case. They're really designed for a different problem. Plus handling 'timeout' as a message is weird when you probably want something like 'do_some_work'. I was told that the timeout=0 reply from init/1 is has some implicit behavior that might be useful: it ensures that the handle_info('timout', State) is called before any other messages can be dispatched, which gives you a hook from init/1 that will run after the call to gen_server:start_link has returned. Feels hacky, to me -- but the only way to handle certain cases. Garrett From vincent.dephily@REDACTED Wed Aug 24 16:05:26 2011 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Wed, 24 Aug 2011 16:05:26 +0200 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <4E54FEA7.5030200@gmail.com> References: <8321958.OU6YYbbWVL@moltowork> <4E54FEA7.5030200@gmail.com> Message-ID: <2830522.EEyirigQHu@moltowork> On Wednesday 24 August 2011 08:37:43 Steve Davis wrote: > WRT to the OP, and in my simple-minded world, protobuffs always seemed > like a lot of hassle for not much gain; Grammar-described formats have a few advantages over self-described formats, such as validation, size, and speed. But self-describing formats can be great too. Depends on your use-case. > And if there is concern about > the verbosity or sending big binaries why not use gzip? Maybe there's > room for an extension to BERT/BERT-RPC to define the use of gzip > compression? I no fan of RPC frameworks, they tend to be either sinfully complicated or so simple that you might as well roll your own. Of course you should compress your data. But I dont think it should standardized in the wirelevel format. Maybe you want a compression algorythm that has a different speed/size tradeoff. Maybe you packets are so small or so tightly encoded (hello UPER !) that gziping them makes them bigger, not smaller. -- Vincent de Phily From lenartlad@REDACTED Wed Aug 24 16:51:50 2011 From: lenartlad@REDACTED (Ladislav Lenart) Date: Wed, 24 Aug 2011 16:51:50 +0200 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: References: Message-ID: <4E551006.5070704@volny.cz> Hello. > I was told that the timeout=0 reply from init/1 is has some implicit > behavior that might be useful: it ensures that the > handle_info('timout', State) is called before any other messages can > be dispatched, which gives you a hook from init/1 that will run after > the call to gen_server:start_link has returned. Feels hacky, to me -- > but the only way to handle certain cases. Just a side note to the special meaning of timeout=0 in init: the same applies to any message sent to self() from the init/1 function, because noone else knows my pid yet. Ladislav Lenart On 24.8.2011 16:32, Garrett Smith wrote: > On Wed, Aug 24, 2011 at 9:23 AM, Daniel Luna wrote: >> On 24 August 2011 09:59, Garrett Smith wrote: >>> On Wed, Aug 24, 2011 at 6:51 AM, Hank Knight wrote: >>>> Hello, >>>> >>>> I need to scheduler a number of recurring events using Erlang. Some >>>> events need to be triggered every second, some every three seconds and >>>> some every minute. Any simple advise on how to do this or what tools >>>> to do? >> [cut away a good solution to the problem] >>> Of course if you don't care about regular intervals, you can track >>> your work time and subtract it from interval. >> >> If you don't care about exact regular intervals the easiest thing >> would be to use the timeout feature of gen_server. >> >> Initialize your gen_server with {ok, State, 3000} for a three second >> wait to the next event. Add a handle_info(timeout, ...), and make >> sure that you set the timeout at every return point (i.e. {noreply, >> State, Timeout} for handle_info). >> >> This of course only works if the gen_server doesn't receive messages >> from elsewhere since then the timeout will never occur (it is a >> timeout after all). > > I'd avoid timeouts for this case. They're really designed for a > different problem. > > Plus handling 'timeout' as a message is weird when you probably want > something like 'do_some_work'. > > I was told that the timeout=0 reply from init/1 is has some implicit > behavior that might be useful: it ensures that the > handle_info('timout', State) is called before any other messages can > be dispatched, which gives you a hook from init/1 that will run after > the call to gen_server:start_link has returned. Feels hacky, to me -- > but the only way to handle certain cases. From magnus.klaar@REDACTED Wed Aug 24 17:49:59 2011 From: magnus.klaar@REDACTED (Magnus Klaar) Date: Wed, 24 Aug 2011 17:49:59 +0200 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: <4E551006.5070704@volny.cz> References: <4E551006.5070704@volny.cz> Message-ID: On Wed, Aug 24, 2011 at 4:51 PM, Ladislav Lenart wrote: > Hello. > > > > I was told that the timeout=0 reply from init/1 is has some implicit > > behavior that might be useful: it ensures that the > > handle_info('timout', State) is called before any other messages can > > be dispatched, which gives you a hook from init/1 that will run after > > the call to gen_server:start_link has returned. Feels hacky, to me -- > > but the only way to handle certain cases. > > Just a side note to the special meaning of timeout=0 in init: the same > applies to any message sent to self() from the init/1 function, because > noone else knows my pid yet. > > > Ladislav Lenart > > Hi! Returning a timeout of 0ms from the init/1 callback does not guarantee that a timeout message is "sent" to the handle_info function before any other messages are processed. If you for example registered the process in the init/1 callback, in the process registry or another facility that would make your process visible to other processes, you may return from the init/1 callback with a non-empty message queue. In this case these messages will be processed by your gen_server, you will not receive a timeout message after this. MVH Magnus Klaar -------------- next part -------------- An HTML attachment was scrubbed... URL: From gordon@REDACTED Wed Aug 24 18:17:50 2011 From: gordon@REDACTED (Gordon Guthrie) Date: Wed, 24 Aug 2011 17:17:50 +0100 Subject: [erlang-questions] This Sunday: Erlang Mini-Factory In Edinburgh Message-ID: Folks Just a final reminder that there will be an Erlang mini-factory in Edinburgh this Sunday. Speakers are: Robert Virding Dale Harvey of CouchOne talking about Erlang on Android Sam Elliott talking about IronMaster a platform management tool Jakub Chandla on user-defined functions in the Hypernumbers spreadsheet. The Mini-Factory is part of A Bit On The Side @ Turing: http://abitontheside.turingfestival.com/ which is the Fringe to the main Turing Festival: http://www.turingfestival.com/ Cheers Gordon -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 -------------- next part -------------- An HTML attachment was scrubbed... URL: From klacke@REDACTED Wed Aug 24 22:57:30 2011 From: klacke@REDACTED (Claes Wikstrom) Date: Wed, 24 Aug 2011 22:57:30 +0200 Subject: [erlang-questions] Tail-f Message-ID: <4E5565BA.10303@hyber.org> We're hiring like crazy, http://www.tail-f.com/company/careers We're an Erlang shop bigtime, most of the jobs are not directly Erlang related, but nevertheless ... /klacke From jwatte@REDACTED Thu Aug 25 00:15:57 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 24 Aug 2011 15:15:57 -0700 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <20110823232246.GB4977@alumni.caltech.edu> References: <1682621.msH3m5tl1p@moltowork> <20110823232246.GB4977@alumni.caltech.edu> Message-ID: You solve that in one of two ways: 1) Create a message that is the "PDU" that contains the union of everything that could be sent, or 2) Create a wrapper that is just "type" followed by "binary data that is a marshaled message." If you believe that the entirety of a protocol should be described by that protocol, then 1) is actually a fine solution. The preserve-and-forward semantic of unknown field ids means that forward compatibility is still possible. message MyProtocolPDU { optional SomeMessage some_message = 1; optional AnotherMessage other_message = 2; ... } So, encoding a MyProtocolPDU that contains a SomeMessage will end up being different than encoding a MyProtocolPDU that contains an AnotherMessage. At the high level, you have to know that there is only one member in each MyProtocolPDU; I find that to be quite acceptable. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Aug 23, 2011 at 4:22 PM, Anthony Molinaro < anthonym@REDACTED> wrote: > Protobuf is not what I would call self describing, if you have something > like > > message foo { > required int32 bar = 1; > } > > message foo2 { > optional int32 bar2 = 1; > } > > and create an instance of each and then send it remotely, you can not > distinguish between the 2 messages, unless the remote side know when > one you mean to send. > > Here's an example in erlang > > Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.8.4 (abort with ^G) > 1> rr(foo_pb). > [foo,foo2] > 2> foo_pb:encode(#foo{bar=42}). > <<"\b*">> > 3> foo_pb:encode(#foo2{bar2=42}). > <<"\b*">> > > To me a self-describing protocol is something which also encoded the names > of the messages and fields, so you could send it through to the other side > and it would be unambiguous. An example of this is a system I wrote many > years ago called lwes (www.lwes.org), which does not require code > generation > because messages are self describing. > > Now this does mean messages are longer, but sometimes that is okay. > > -Anthony > > On Tue, Aug 23, 2011 at 01:17:12PM -0700, Jon Watte wrote: > > Protobuf is fully self-described, and fully forwards/backwards > compatible, > > in that it can carry through "unknown" members of a structure, including > the > > semantic of the fields. ASN1 is generally not fully self-described, > because > > it only describes type, not semantic, for each field. > > > > Sincerely, > > > > jw > > > > -- > > Americans might object: there is no way we would sacrifice our living > > standards for the benefit of people in the rest of the world. > Nevertheless, > > whether we get there willingly or not, we shall soon have lower > consumption > > rates, because our present rates are unsustainable. > > > > > > > > On Tue, Aug 23, 2011 at 3:03 AM, Vincent de Phily < > > vincent.dephily@REDACTED> wrote: > > > > > On Monday 22 August 2011 11:31:44 Mike Oxford wrote: > > > > BERT is pretty much a "known good quantity" but protobuf is more > > > > efficient on the wire. > > > > > > > > Anyone have thoughts on which to go with? > > > > > > > > Also, along the BERT side, does anyone have recommendations on AS3 > and > > > > Java implementations of the marshallers? > > > > > > > > The downside(s) to protobuf - hard IDLs and you have to build the > > > > project and generators. > > > > The downside(s) to BERT - less efficient on the wire (verbosity.) > > > > > > Also take a look at bjson, and one of my favourite : msgpack. BERT is > > > rather > > > unique in supporting atoms, which *nice* and can reduce encoded size a > lot > > > (but that's less significant if you compress). For encoded size, I have > yet > > > to > > > see anything beat UPER (asn1). > > > > > > A few things to consider : > > > * Bench size and speed with a mockup of your data : different formats > are > > > better for different types of data. Add compression to your tests. > > > * Think about format extensibility. It's easy enough with > self-described > > > formats like bert or msgpack, but gets more complicated with protobuf > or > > > asn1. > > > * The initial hurdle of dealing with a grammar and generators is well > paid > > > off > > > by having data validation and nicer api (mostly via #records{}) later > on. > > > * If you need to speak with the outside world, self-describing formats > will > > > make your life easyer. > > > -- > > > Vincent de Phily > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > ------------------------------------------------------------------------ > Anthony Molinaro > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Thu Aug 25 00:17:20 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 24 Aug 2011 15:17:20 -0700 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: References: Message-ID: I second the use of timer:send_interval. Your gen_server can use this within init(), and will get timer messages sent to its handle_info() callback. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Wed, Aug 24, 2011 at 6:54 AM, Anders Nygren wrote: > Hi > I think that the simplest way is to just use timer:apply_interval or > timer:send_interval > from stdlib. > > /Anders > > On Wed, Aug 24, 2011 at 6:51 AM, Hank Knight wrote: > > Hello, > > > > I need to scheduler a number of recurring events using Erlang. Some > > events need to be triggered every second, some every three seconds and > > some every minute. Any simple advise on how to do this or what tools > > to do? > > > > - Hank > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Thu Aug 25 01:51:47 2011 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 24 Aug 2011 18:51:47 -0500 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <2830522.EEyirigQHu@moltowork> References: <8321958.OU6YYbbWVL@moltowork> <4E54FEA7.5030200@gmail.com> <2830522.EEyirigQHu@moltowork> Message-ID: <4E558E93.6020707@gmail.com> On 8/24/2011 9:05 AM, Vincent de Phily wrote: > On Wednesday 24 August 2011 08:37:43 Steve Davis wrote: >> WRT to the OP, and in my simple-minded world, protobuffs always seemed >> like a lot of hassle for not much gain; > > Grammar-described formats have a few advantages over self-described formats, > such as validation, size, and speed. But self-describing formats can be great > too. Depends on your use-case. > I think I would suggest that this is the same argument as "static versus dynamic typing". I would argue that in the cases where stronger checking is actually needed what's really missing is the core idea of a contract per UBF, rather than some standardized grammar. The distinction is narrow but the impact I think is quite different. >> And if there is concern about >> the verbosity or sending big binaries why not use gzip? Maybe there's >> room for an extension to BERT/BERT-RPC to define the use of gzip >> compression? > > I no fan of RPC frameworks, they tend to be either sinfully complicated or so > simple that you might as well roll your own. > > Of course you should compress your data. But I dont think it should > standardized in the wirelevel format. Maybe you want a compression algorythm > that has a different speed/size tradeoff. Maybe you packets are so small or so > tightly encoded (hello UPER !) that gziping them makes them bigger, not > smaller. Actually, I meant "an extension" for those situations, not as a requirement of the protocol... and gave gzip as the candidate in the spirit of keeping it all agnostic. /s From dialtone@REDACTED Thu Aug 25 02:04:21 2011 From: dialtone@REDACTED (Valentino Volonghi) Date: Wed, 24 Aug 2011 17:04:21 -0700 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <1682621.msH3m5tl1p@moltowork> <20110823232246.GB4977@alumni.caltech.edu> Message-ID: On 24 Aug 2011, at 15:15, Jon Watte wrote: > You solve that in one of two ways: > 1) Create a message that is the "PDU" that contains the union of everything that could be sent, > or > 2) Create a wrapper that is just "type" followed by "binary data that is a marshaled message." I've actually tested this against MessagePack. Original file is csv with base64 encoded fields and gzipped: 73554863 bytes Msgpack file with no base64 encoding and gzipped: 64226388 bytes protocolbuffers file no base64 with PDU and gzipped: 63470582 bytes The msgpack solution was faster to decode and encode though, and it's also significantly simpler code in both erlang and Python (which I tested). -- Valentino Volonghi http://www.adroll.com From watson.timothy@REDACTED Thu Aug 25 10:20:45 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 25 Aug 2011 09:20:45 +0100 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: References: Message-ID: On 24 August 2011 23:17, Jon Watte wrote: > I second the use of timer:send_interval. Your gen_server can use this within > init(), and will get ? timer messages sent to its handle_info() callback. > Sincerely, > jw I seem to remember reading comments on this list (which I'll have to dig out later) suggesting that the timer module doesn't perform that well compared to using the raw erlang:start_timer/3 API. Has this changed or was it mis-information to begin with? From kostis@REDACTED Thu Aug 25 10:32:25 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 25 Aug 2011 11:32:25 +0300 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: <4E551006.5070704@volny.cz> References: <4E551006.5070704@volny.cz> Message-ID: <4E560899.3050305@cs.ntua.gr> On 08/24/11 17:51, Ladislav Lenart wrote: > Hello. > > > I was told that the timeout=0 reply from init/1 is has some implicit > > behavior that might be useful: it ensures that the > > handle_info('timout', State) is called before any other messages can > > be dispatched, which gives you a hook from init/1 that will run after > > the call to gen_server:start_link has returned. Feels hacky, to me -- > > but the only way to handle certain cases. > > Just a side note to the special meaning of timeout=0 in init: the same > applies to any message sent to self() from the init/1 function, because > noone else knows my pid yet. Interesting... But would you care to elaborate a bit? What do you mean "noone else knows my pid yet" in an SMP environment with multiple schedulers and in the presence of the processes() function that can be called from anywhere? Am I missing something? Kostis From lenartlad@REDACTED Thu Aug 25 10:49:12 2011 From: lenartlad@REDACTED (Ladislav Lenart) Date: Thu, 25 Aug 2011 10:49:12 +0200 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: <4E560899.3050305@cs.ntua.gr> References: <4E551006.5070704@volny.cz> <4E560899.3050305@cs.ntua.gr> Message-ID: <4E560C88.8050003@volny.cz> On 25.8.2011 10:32, Kostis Sagonas wrote: > On 08/24/11 17:51, Ladislav Lenart wrote: >> Hello. >> >> > I was told that the timeout=0 reply from init/1 is has some implicit >> > behavior that might be useful: it ensures that the >> > handle_info('timout', State) is called before any other messages can >> > be dispatched, which gives you a hook from init/1 that will run after >> > the call to gen_server:start_link has returned. Feels hacky, to me -- >> > but the only way to handle certain cases. >> >> Just a side note to the special meaning of timeout=0 in init: the same >> applies to any message sent to self() from the init/1 function, because >> noone else knows my pid yet. > > Interesting... > > But would you care to elaborate a bit? What do you mean "noone else knows my pid yet" in an SMP environment with multiple schedulers and in the presence of the processes() function that can be called > from anywhere? Am I missing something? Nope, I've forgotten about processes() function but I consider it a debugging aid and not a part of normal application operation. I would not use (have never used) it in a production code. From this POV the only "allowed" way to get a pid of a process is to receive it from a message or as a return value of its start function. If the process itself registers to some global service in its init function, it has to do so after it has sent all required messages to self. Ladislav Lenart From matti.oinas@REDACTED Thu Aug 25 10:52:27 2011 From: matti.oinas@REDACTED (Matti Oinas) Date: Thu, 25 Aug 2011 11:52:27 +0300 Subject: [erlang-questions] OWASP codecs Message-ID: Hi, Is there an erlang implementation of OWASP ESAPI codecs? https://www.owasp.org/index.php/XSS_%28Cross_Site_Scripting%29_Prevention_Cheat_Sheet I'm considering to use erlang to generate HTML so I could drop one layer and one language (PHP) from the project. Matti From vincent.dephily@REDACTED Thu Aug 25 12:04:15 2011 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Thu, 25 Aug 2011 12:04:15 +0200 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <20110823232246.GB4977@alumni.caltech.edu> Message-ID: <2817450.46CDmI7xT0@moltowork> On Wednesday 24 August 2011 15:15:57 Jon Watte wrote: > You solve that in one of two ways: > 1) Create a message that is the "PDU" that contains the union of everything > that could be sent, > or > 2) Create a wrapper that is just "type" followed by "binary data that is a > marshaled message." > > If you believe that the entirety of a protocol should be described by that > protocol, then 1) is actually a fine solution. The preserve-and-forward > semantic of unknown field ids means that forward compatibility is still > possible. > > message MyProtocolPDU { > optional SomeMessage some_message = 1; > optional AnotherMessage other_message = 2; > ... > } > > So, encoding a MyProtocolPDU that contains a SomeMessage will end up being > different than encoding a MyProtocolPDU that contains an AnotherMessage. > At the high level, you have to know that there is only one member in each > MyProtocolPDU; I find that to be quite acceptable. Yup, they're reasonable solutions (as long as you planed ahead). 2) may be ugly and innefficient, but it has been used as a pragmatic solution in countless formats. 1) works as long as you control the sender or you have a well-defined policy when multiple fields are present, but I'd otherwise consider it a footgun. I really find it odd that protobuf doesn't provide a CHOICE construct. -- Vincent de Phily From jesper.louis.andersen@REDACTED Thu Aug 25 13:27:59 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 25 Aug 2011 13:27:59 +0200 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: References: Message-ID: On Wed, Aug 24, 2011 at 13:51, Hank Knight wrote: > Hello, > > I need to scheduler a number of recurring events using Erlang. ?Some > events need to be triggered every second, some every three seconds and > some every minute. ?Any simple advise on how to do this or what tools > to do? My solution would be to use erlang:send_after(..) to set up a timer and then set the timer again whenever it triggers. The "timer" module is also there, but it is considerably less effective if you end up with many timers (according to the performance guide - http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id58959). If you have lots and lots of these timers, and your naive approach hits performance problems, then perhaps a trick from operating system kernels can be tried. The idea is that you lag the timer. Rather than waiting exactly three seconds, you wait 3 seconds or more, until the next time slot trigger for the event. That means a 3 sec timer takes somewhere between 3 and 6 seconds and a 1 minute timer takes between 1 and 2 minutes. It allows you to bundle up large number of events on the same timer, which could be faster in some situations. On the other hand, Erlangs internal timer stuff is pretty fast already, so only begin playing with it when the basic implementation can't cope. -- J. From norton@REDACTED Thu Aug 25 13:39:43 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Thu, 25 Aug 2011 20:39:43 +0900 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: References: Message-ID: For reference and/or comparison purposes, this timer implementation might be helpful to you: https://github.com/hibari/gdss-brick/blob/master/src/brick_itimer.erl The OTP timer implementation didn't scale beyond several hundred timers. thanks, On 2011/08/25, at 20:27, Jesper Louis Andersen wrote: > On Wed, Aug 24, 2011 at 13:51, Hank Knight wrote: >> Hello, >> >> I need to scheduler a number of recurring events using Erlang. Some >> events need to be triggered every second, some every three seconds and >> some every minute. Any simple advise on how to do this or what tools >> to do? > > My solution would be to use erlang:send_after(..) to set up a timer > and then set the timer again whenever it triggers. The "timer" module > is also there, but it is considerably less effective if you end up > with many timers (according to the performance guide - > http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id58959). > > If you have lots and lots of these timers, and your naive approach > hits performance problems, then perhaps a trick from operating system > kernels can be tried. The idea is that you lag the timer. Rather than > waiting exactly three seconds, you wait 3 seconds or more, until the > next time slot trigger for the event. That means a 3 sec timer takes > somewhere between 3 and 6 seconds and a 1 minute timer takes between 1 > and 2 minutes. It allows you to bundle up large number of events on > the same timer, which could be faster in some situations. > > On the other hand, Erlangs internal timer stuff is pretty fast > already, so only begin playing with it when the basic implementation > can't cope. > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Joseph Wayne Norton norton@REDACTED From erlang@REDACTED Thu Aug 25 14:01:57 2011 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 25 Aug 2011 14:01:57 +0200 Subject: [erlang-questions] web frameworks - yet another question about ... Message-ID: I seem to have asked the same question in many ways ... I want to build a simple generic web server. By my way of thinking I should just have to write a single module, that exports a very simple interface. Suppose I define such a module, call this my_server. I'd like to write this as follows: -module(my_server). -export([request/6]) request(Who, Command, URI, Args, Headers, Data) -> ... {reply, Headers, Data1} Where for example, Who = "joe" Command = 'GET' | 'POST' | 'PUT' | ... URI = "/path/to/thing" Args = [{Key,Val}] (parsed part of URI , ie a parsed AAA=BBB&CCC=DDD& ... part of the query Headers and Data and the HTML headers and any other POST data To run the server - I'd just say: gen_http_server:start(Por, my_server) I want the generic bit to handle all the other nasty stuff that I don't want to know about. By this I mean user management. I want "Who" to be a fully authenticated user - or possibly the atom 'unknown' if the user is not authenticated. I wish to hide all the details of user management from the interface. Things like "I forgot my password" captchas to prevent abuse, preventing false account creation, black listing malicious IPS etc. I want all this part to be handled by the generic party of the server. I realize this is asking a lot and is fairly complicated - but all I want to do is to define the semantics of my web site. I don't care about templating languages and all that stuff. Given an parsed (or even unparsed) request generating a bundle of HTML or js or css is relatively trivial - the difficult bit (to me) seems to be all this attendant muck to authenticate users, prevent abuse, blacking abusers, take care of forgotten passwords etc. If people want to make yet another web framework then please make one that handles the user management part of the problem. In an ideal world I should just need three (ish) modules to make my web server. - a semantics module (as I described) - a user management module (missing) - a storage module (missing) With suitable abstract interfaces one should be able to pick and mix user management and backed storage - making different trade-offs between availability security and reliability which keeping the interfaces constant. has anybody any advice for how to make the user-management layer? /Joe From watson.timothy@REDACTED Thu Aug 25 14:06:07 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 25 Aug 2011 13:06:07 +0100 Subject: [erlang-questions] web frameworks - yet another question about ... In-Reply-To: References: Message-ID: > > If people want to make yet another web framework then please make > one that handles the user management part of the problem. > > In an ideal world I should just need three (ish) modules to make my web server. > > ? ?- a semantics module (as I described) > ? ?- a user management module (missing) > ? ?- a storage module (missing) > > With suitable abstract interfaces one should be able to pick and mix > user management > and backed storage - making different trade-offs between availability > security and reliability > which keeping the interfaces constant. > Having a common API would make life much simpler, especially when evaluating different options for web servers, approaches to AAA, persistence and so on. From max.lapshin@REDACTED Thu Aug 25 14:15:26 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 25 Aug 2011 16:15:26 +0400 Subject: [erlang-questions] web frameworks - yet another question about ... In-Reply-To: References: Message-ID: > > has anybody any advice for how to make the user-management layer? > Hi, Joe. This is a well known problem: each project has its own implementation of these tasks. Ruby on Rails have several major implementations, and I can tell about one of them (devise). It is splitted in two parts: authenticator and user interface. authenticator lives in callback modules before your my_server:request/6 and can translate cookies of http headers to your "Who" field. This authenticator can also use openid protocol or oauth, or anything else, including LDAP etc. user interface is basically copied to each program and includes view templates for login, password recovery and registration. It is a good idea to copy this architecture, because it works. With problems, but works. From tristan.sloughter@REDACTED Thu Aug 25 14:30:48 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Thu, 25 Aug 2011 07:30:48 -0500 Subject: [erlang-questions] web frameworks - yet another question about ... In-Reply-To: References: Message-ID: It has a ways to go but I think the framework I am working on as I develop a site is similar to what you will be looking for: https://github.com/claimstrade/maru It just requires you include maru_web in your app and you then can build off of the users and authentication that come with it. It is broke into apps for the web, models and db layer (and eventually security/ops) so they can be mixed and matched/replaced. In sys.config I define: {maru_web, [{webmachine_ip, any}, {webmachine_port, "8080"}, {host_dir, ""}, {dispatch_file, ""}]} So it is still webmachine resources but I'm working to simply the general cases, like the example you give, and resources that deal with GET/POST/PUT/DELETE for some model by json->record and record->json. It should grow fast and I have a number of things I still need to move out of the site to the general framework Maru. So its still very young, but if people have suggestions on what they'd like to see that they don't in other Erlang web frameworks please let me know! Right now it will grow based on what I find is generic as I build another production site with Webmachine. Tristan On Thu, Aug 25, 2011 at 7:01 AM, Joe Armstrong wrote: > I seem to have asked the same question in many ways ... > > I want to build a simple generic web server. > > By my way of thinking I should just have to write a single module, > that exports a very simple interface. > > Suppose I define such a module, call this my_server. > > I'd like to write this as follows: > > -module(my_server). > -export([request/6]) > > request(Who, Command, URI, Args, Headers, Data) -> > ... > {reply, Headers, Data1} > > Where for example, > > Who = "joe" > Command = 'GET' | 'POST' | 'PUT' | ... > URI = "/path/to/thing" > Args = [{Key,Val}] (parsed part of URI , ie a parsed > AAA=BBB&CCC=DDD& ... part of the query > Headers and Data and the HTML headers and any other POST data > > To run the server - I'd just say: > > gen_http_server:start(Por, my_server) > > I want the generic bit to handle all the other nasty stuff that I > don't want to know about. By this I mean user > management. I want "Who" to be a fully authenticated user - or > possibly the atom 'unknown' if the user is not authenticated. > > I wish to hide all the details of user management from the interface. > Things like "I forgot my password" > captchas to prevent abuse, preventing false account creation, black > listing malicious IPS etc. > > I want all this part to be handled by the generic party of the server. > > I realize this is asking a lot and is fairly complicated - but all I > want to do is to define > the semantics of my web site. I don't care about templating languages > and all that stuff. > Given an parsed (or even unparsed) request generating a bundle of HTML > or js or css > is relatively trivial - the difficult bit (to me) seems to be all this > attendant muck to > authenticate users, prevent abuse, blacking abusers, take care of > forgotten passwords etc. > > If people want to make yet another web framework then please make > one that handles the user management part of the problem. > > In an ideal world I should just need three (ish) modules to make my web > server. > > - a semantics module (as I described) > - a user management module (missing) > - a storage module (missing) > > With suitable abstract interfaces one should be able to pick and mix > user management > and backed storage - making different trade-offs between availability > security and reliability > which keeping the interfaces constant. > > has anybody any advice for how to make the user-management layer? > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vincent.dephily@REDACTED Thu Aug 25 15:08:01 2011 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Thu, 25 Aug 2011 15:08:01 +0200 Subject: [erlang-questions] TCP receive buffer: erlang size VS kernel size Message-ID: <1478234.he0o8qDZS3@moltowork> Hi list, while playing with the TCP receive buffer size, I noticed two strange things : 1) The size seen in inet:getopts(S, [recbuf]) is the double of the size set via gen_tcp:listen/2 (with some min/max values). * I assume that the discrepancy is between an in-VM buffer and the in-kernel buffer ? * Why is that 2x ratio necessary ? * The docs never mention that there are two different buffers, and which function reads/writes which buffer size. I only expected the kernel buffer, and I definitely expected 'recbuf' to represent the same object for inet:getopt as for gen_tcp:listen. 2) If I don't specify a recbuf in gen_tcp:listen/2, the kernel buffer (as seen in inet:getopts) is 87 KB while the VM buffer (as seen in the size of received packets) is 1460 B. * That's a much bigger difference ratio than previously seen. * The default value of 1460 bytes sounds too small for anything but the most ressource-constrained devices (my kernel agrees, but I admit having 4GB of memory). This is on linux 3.0.3, erlang R13B04 and R14B03. Here's a script for experimentation (using "nc -q1 $HOST $PORT < bigfile" on the client side): #!/usr/bin/env escript -mode(compile). main([Port, Active, Size, Timeout, Sleep, Buffer]) -> main(list_to_integer(Port), list_to_atom(Active), list_to_integer(Size), list_to_integer(Timeout), list_to_integer(Sleep), list_to_integer(Buffer)). main(Port, Active, Size, Timeout, Sleep, Buffer) -> {ok, Listen} = gen_tcp:listen(Port, [binary, {reuseaddr, true}, {active, Active}] ++ case Buffer > 0 of true -> [{recbuf, Buffer}]; false -> [] end), accept(Listen, Port, Active, Size, Timeout, Sleep). accept(Listen, Port, Active, Size, Timeout, Sleep) -> io:format("accepting connections on ~p~n", [Port]), {ok, Socket} = gen_tcp:accept(Listen), io:format("~w ~w~n", [element(1,lists:unzip(element(2,inet:getstat(Socket)))), element(2,inet:getopts(Socket, [recbuf]))]), loop(Socket, Active, Size, Timeout, Sleep), accept(Listen, Port, Active, Size, Timeout, Sleep). loop(Socket, Active, Size, Timeout, Sleep) -> io:format("~w\t", [stat(Socket)]), receive {tcp, Socket, Data1} -> case Active of true -> io:format("case1 ~p~n",[byte_size(Data1)]); once -> Sleep > 0 andalso timer:sleep(Sleep), case gen_tcp:recv(Socket, Size, Timeout) of {ok, Data2} -> io:format("case2 ~p ~p~n", [byte_size(Data1), byte_size(Data2)]); Other -> io:format("case3 ~p ~p~n", [byte_size(Data1), Other]) end, inet:setopts(Socket, [{active, once}]) end, loop(Socket, Active, Size, Timeout, Sleep); Other -> io:format("~p~n",[Other]), gen_tcp:close(Socket) after 1000 -> io:format("timeout~n",[]), gen_tcp:close(Socket) end. stat(Socket) -> case inet:getstat(Socket) of {ok, S} -> element(2,lists:unzip(S)); Other -> Other end. -- Vincent de Phily From simsek.burak@REDACTED Thu Aug 25 15:43:14 2011 From: simsek.burak@REDACTED (Simsek Burak) Date: Thu, 25 Aug 2011 15:43:14 +0200 Subject: [erlang-questions] Building plt Message-ID: Hello! I'm a newbie with erlang, i want to use dialyzer on Windows. I tried to use it but it required HOME enviroment. Then i used os:putenv/2 to declare HOME enviroment as my home path(like c:\Users\..). Then it gave "Could not read the PLT" error. So it wants an existing PLT. How can i create plt now? Thank you Regards, Burak -------------- next part -------------- An HTML attachment was scrubbed... URL: From Lennart.Ohman@REDACTED Thu Aug 25 16:14:09 2011 From: Lennart.Ohman@REDACTED (=?iso-8859-1?Q?Lennart_=D6hman?=) Date: Thu, 25 Aug 2011 16:14:09 +0200 Subject: [erlang-questions] Building plt In-Reply-To: References: Message-ID: Hi, first you are correct you need to set the HOME windows environment variable. As far as I know, you can set it to anything actually. Dialyzer just requires it to exist, if you specify all other paths. First you must build your own plt. ...\dialyzer -build_plt -output_plt C:/mydir/myplt.plt -apps kernel stdlib erts Then when you use it, simple specify -plt C:/mydir/myplt.plt Best Regards Lennart From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Simsek Burak Sent: den 25 augusti 2011 15:43 To: erlang-questions@REDACTED Subject: [erlang-questions] Building plt Hello! I'm a newbie with erlang, i want to use dialyzer on Windows. I tried to use it but it required HOME enviroment. Then i used os:putenv/2 to declare HOME enviroment as my home path(like c:\Users\..). Then it gave "Could not read the PLT" error. So it wants an existing PLT. How can i create plt now? Thank you Regards, Burak -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericbmerritt@REDACTED Thu Aug 25 16:43:26 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Thu, 25 Aug 2011 09:43:26 -0500 Subject: [erlang-questions] Erlang Scheduler for Recurring Events In-Reply-To: <2B8E5EE0-05DB-4C02-BA97-C8433EEBE868@gmail.com> References: <2B8E5EE0-05DB-4C02-BA97-C8433EEBE868@gmail.com> Message-ID: Another is erlcron: https://github.com/erlware/erlcron Not heavily used but has the important semantic that it allows you to test your system. That is you can fast-forward the time and your events will still fire in the virtual time they are supposed to. On Wed, Aug 24, 2011 at 6:56 AM, Zabrane Mickael wrote: > Sounds like you need something like Unix "cron" : > a. ecron:?http://gitorious.org/ecron > b. crone:?http://catseye.tc/projects/crone/ > Regards, > Zabrane > > On Aug 24, 2011, at 1:51 PM, Hank Knight wrote: > > Hello, > > I need to scheduler a number of recurring events using Erlang. ?Some > events need to be triggered every second, some every three seconds and > some every minute. ?Any simple advise on how to do this or what tools > to do? > > - Hank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From g@REDACTED Thu Aug 25 17:05:29 2011 From: g@REDACTED (Garrett Smith) Date: Thu, 25 Aug 2011 10:05:29 -0500 Subject: [erlang-questions] web frameworks - yet another question about ... In-Reply-To: References: Message-ID: On Thu, Aug 25, 2011 at 7:01 AM, Joe Armstrong wrote: > I seem to have asked the same question in many ways ... > > I want to build a simple generic web server. > > By my way of thinking I should just have to write a single module, > that exports a very simple interface. > > Suppose I define such a module, call this my_server. > > I'd like to write this as follows: > > ? ?-module(my_server). > ? ?-export([request/6]) > > ? ?request(Who, Command, URI, Args, Headers, Data) -> > ? ? ? ? ... > ? ? ? ? {reply, Headers, Data1} https://github.com/gar1t/modlib It's a light wrapper around the inets httpd server and uses semantics similar to what you're describing. Garrett From rtrlists@REDACTED Thu Aug 25 17:43:38 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 25 Aug 2011 16:43:38 +0100 Subject: [erlang-questions] How to kill a process started with open_port({spawn_executable, ... ? Message-ID: Hi, is it possible to explicitly kill an OS process that's been started with and open_port({spawn_executable, "executable"}, ...) call from within Erlang? This is on Windows for me just now, in case there is a way and it differs between OSes. Thanks, Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.joseph.davis@REDACTED Thu Aug 25 17:51:52 2011 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Thu, 25 Aug 2011 10:51:52 -0500 Subject: [erlang-questions] How to kill a process started with open_port({spawn_executable, ... ? In-Reply-To: References: Message-ID: I'm not sure if this is the cleanest approach or not, but in CouchDB we place a shim between Erlang and the OS process that reports a cmd that can be passed to os:cmd to execute the process. There's a version for both Unix and Windows. The Unix version in its entirety (minus the license): #! /bin/sh -e echo "kill -9 $$" exec $* https://github.com/apache/couchdb/tree/trunk/src/couchdb/priv/spawnkillable On Thu, Aug 25, 2011 at 10:43 AM, Robert Raschke wrote: > Hi, > > is it possible to explicitly kill an OS process that's been started with and > open_port({spawn_executable, "executable"}, ...) call from within Erlang? > > This is on Windows for me just now, in case there is a way and it differs > between OSes. > > Thanks, > Robby > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From rtrlists@REDACTED Thu Aug 25 18:30:29 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 25 Aug 2011 17:30:29 +0100 Subject: [erlang-questions] How to kill a process started with open_port({spawn_executable, ... ? In-Reply-To: References: Message-ID: On Thu, Aug 25, 2011 at 4:51 PM, Paul Davis wrote: > I'm not sure if this is the cleanest approach or not, but in CouchDB > we place a shim between Erlang and the OS process that reports a cmd > that can be passed to os:cmd to execute the process. There's a version > for both Unix and Windows. > > The Unix version in its entirety (minus the license): > > #! /bin/sh -e > echo "kill -9 $$" > exec $* > > https://github.com/apache/couchdb/tree/trunk/src/couchdb/priv/spawnkillable > > On Thu, Aug 25, 2011 at 10:43 AM, Robert Raschke > wrote: > > Hi, > > > > is it possible to explicitly kill an OS process that's been started with > and > > open_port({spawn_executable, "executable"}, ...) call from within Erlang? > > > > This is on Windows for me just now, in case there is a way and it differs > > between OSes. > > > > Thanks, > > Robby > Yeah, I was staring down that alleyway and thought, there must be something better. 'Twould be nice if we had a port_close/2, that could take an option to choose between killing or detaching from the OS process. But I guess that's tricky, 'cause ports are so versatile. Thanks for the link to the CouchDB approach, I might head down that route after all. Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Aug 25 18:44:35 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 25 Aug 2011 20:44:35 +0400 Subject: [erlang-questions] How to kill a process started with open_port({spawn_executable, ... ? In-Reply-To: References: Message-ID: > Yeah, I was staring down that alleyway and thought, there must be something > better. 'Twould be nice if we had a port_close/2, that could take an option > to choose between killing or detaching from the OS process. But I guess > that's tricky, 'cause ports are so versatile. > > Thanks for the link to the CouchDB approach, I might head down that route > after all. > It tried to add something like os:get_pid(Port) and os:kill(Pid, Signal) but failed to launch OTP test infrastructure From max.lapshin@REDACTED Thu Aug 25 20:56:03 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 25 Aug 2011 22:56:03 +0400 Subject: [erlang-questions] catch supervisor failure Message-ID: I need to fetch url once a minute. It is ok for remote server to fail to reply with this url several times, so I want to use supervisor mechanism for it: set 60 restarts in 5 seconds as a limit and if supervisor fails, it should be stopped for some time. After 10 minutes it should be restarted. Question is: should I use supervisors for this mechanism or I need to write my own failure tracker? From ericbmerritt@REDACTED Thu Aug 25 21:52:22 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Thu, 25 Aug 2011 14:52:22 -0500 Subject: [erlang-questions] catch supervisor failure In-Reply-To: References: Message-ID: No this is exactly what supervisors are for. I see no need at all to write your own supervisor. On Thu, Aug 25, 2011 at 1:56 PM, Max Lapshin wrote: > I need to fetch url once a minute. It is ok for remote server to fail > to reply with this url several times, so I want to use supervisor > mechanism for it: > > set 60 restarts in 5 seconds as a limit and if supervisor fails, it > should be stopped for some time. After 10 minutes it should be > restarted. > > Question is: should I use supervisors for this mechanism or I need to > write my own failure tracker? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From matthew.hillsborough@REDACTED Fri Aug 26 05:02:18 2011 From: matthew.hillsborough@REDACTED (Matthew Hillsborough) Date: Thu, 25 Aug 2011 23:02:18 -0400 Subject: [erlang-questions] How do I move this into a config file? Message-ID: Hi everyone, I have some hard coded defines in one of my modules that looks like the following: -ifdef(debug). -define(FOOBAR, "DEBUG_FOOBAR"). -else. -define(FOOBAR, "PROD_FOOBAR"). -endif. I want to be able to deploy this to multiple servers with a config file using rebar. So I suppose this is a two part question: - How does one move the above into an external configuration file and then read it back into the application? - I need to have multiple configurations as I will have different vms hitting different FOOBAR instances. I'm looking to copy these to the appropriate directory with either a bash script or rebar if possible. What's the best way of doing that? ServerA will have external_a.config, ServerB would have external_b.config. Ideally rebar or some third party script would copy that over to external_[x].config and the module that reads this only knows about external.config. I can ultimately get rid of the debug flag and just use an external_local.config file where debug is always enabled. I read about potentially being able to do this with vars.config, but I can't find any examples / documentation supporting it. I want to put this all into my build process and make sure each server has the right FOOBAR defined. I would love any suggestions anyone has, I'm sure you've all dealt with something similar when dealing with multiple environments. Thanks all! Matthew -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Aug 26 06:44:10 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 26 Aug 2011 08:44:10 +0400 Subject: [erlang-questions] catch supervisor failure In-Reply-To: References: Message-ID: On Thu, Aug 25, 2011 at 11:52 PM, Eric Merritt wrote: > No this is exactly what supervisors are for. I see no need at all to > write your own supervisor. > But how can I handle the fact, that supervisor failed? From bengt.kleberg@REDACTED Fri Aug 26 07:25:31 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 26 Aug 2011 07:25:31 +0200 Subject: [erlang-questions] catch supervisor failure In-Reply-To: References: Message-ID: <1314336331.4778.1.camel@seasc1137> Greetings, Supervisor tree. Ie, your supervisor has a supervisor. See http://www.erlang.org/doc/design_principles/des_princ.html bengt On Fri, 2011-08-26 at 06:44 +0200, Max Lapshin wrote: > On Thu, Aug 25, 2011 at 11:52 PM, Eric Merritt wrote: > > No this is exactly what supervisors are for. I see no need at all to > > write your own supervisor. > > > > But how can I handle the fact, that supervisor failed? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Fri Aug 26 07:42:05 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 26 Aug 2011 09:42:05 +0400 Subject: [erlang-questions] catch supervisor failure In-Reply-To: <1314336331.4778.1.camel@seasc1137> References: <1314336331.4778.1.camel@seasc1137> Message-ID: On Fri, Aug 26, 2011 at 9:25 AM, Bengt Kleberg wrote: > Greetings, > > Supervisor tree. Ie, your supervisor has a supervisor. > > See http://www.erlang.org/doc/design_principles/des_princ.html > And what next? worker has failed for 100 times during 3 seconds. It's supervisor is dead and is restarted by parent. This happens for 5 times during 15 seconds and then master supervisor dies with all application. I don't see any callback: "handle supervisor death" From bengt.kleberg@REDACTED Fri Aug 26 07:47:50 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 26 Aug 2011 07:47:50 +0200 Subject: [erlang-questions] catch supervisor failure In-Reply-To: References: <1314336331.4778.1.camel@seasc1137> Message-ID: <1314337670.4778.8.camel@seasc1137> Would it be possible to have a supervisor between the master supervisor and the one that dies? This new supervisor could handle the behavior you described before, but has gotten lost from this email. It was something about 1 minute, I think. bengt On Fri, 2011-08-26 at 07:42 +0200, Max Lapshin wrote: > On Fri, Aug 26, 2011 at 9:25 AM, Bengt Kleberg > wrote: > > Greetings, > > > > Supervisor tree. Ie, your supervisor has a supervisor. > > > > See http://www.erlang.org/doc/design_principles/des_princ.html > > > > And what next? > > worker has failed for 100 times during 3 seconds. It's supervisor is > dead and is restarted by parent. This happens for 5 times during 15 > seconds and then master supervisor dies with all application. > > I don't see any callback: "handle supervisor death" From max.lapshin@REDACTED Fri Aug 26 07:56:56 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 26 Aug 2011 09:56:56 +0400 Subject: [erlang-questions] catch supervisor failure In-Reply-To: <1314337670.4778.8.camel@seasc1137> References: <1314336331.4778.1.camel@seasc1137> <1314337670.4778.8.camel@seasc1137> Message-ID: On Fri, Aug 26, 2011 at 9:47 AM, Bengt Kleberg wrote: > Would it be possible to have a supervisor between the master supervisor > and the one that dies? > This new supervisor could handle the behavior you described before, but > has gotten lost from this email. It was something about 1 minute, I > think. > I can add as much supervisors as required. But I don't understand how to write my own callback, that will be called if some child of supervisor is dead. From steve@REDACTED Fri Aug 26 08:08:57 2011 From: steve@REDACTED (Steve Strong) Date: Fri, 26 Aug 2011 08:08:57 +0200 Subject: [erlang-questions] catch supervisor failure In-Reply-To: References: <1314336331.4778.1.camel@seasc1137> <1314337670.4778.8.camel@seasc1137> Message-ID: I'm not aware of any way to implement the "wait 10 minutes" logic in the standard supervisor pattern. The way we've handled something like this in the past is to have a separate gen_server running higher up the tree that the server (the url fetcher in your example) calls into at startup. This gen_server keeps track of how many initialisations there have been, and performs timer:sleep()s as appropriate - in our scenario, it was a simple exponential back off algorithm. Not particularly pretty, but it did drop in with almost no change to the rest of the system. Some sort of supervisor callback at termination time would be much more preferable, but I don't know of a mechanism for that - would be glad to hear other suggestions! Cheers, Steve -- Steve Strong Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Friday, 26 August 2011 at 07:56, Max Lapshin wrote: > On Fri, Aug 26, 2011 at 9:47 AM, Bengt Kleberg > wrote: > > Would it be possible to have a supervisor between the master supervisor > > and the one that dies? > > This new supervisor could handle the behavior you described before, but > > has gotten lost from this email. It was something about 1 minute, I > > think. > > I can add as much supervisors as required. But I don't understand how > to write my own callback, that will be called if some child > of supervisor is dead. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rapsey@REDACTED Fri Aug 26 09:17:55 2011 From: rapsey@REDACTED (Rapsey) Date: Fri, 26 Aug 2011 09:17:55 +0200 Subject: [erlang-questions] How do I move this into a config file? In-Reply-To: References: Message-ID: What we do is have a module that has all configurations for all setups. Then we have modules that just return the name of the current setup. The module that returns a specific name is placed on each server in that setup, the configuration file is the same on all servers across all setups. It works great with how we use erlang. Which typically involves updating servers daily, often more than that with small increments. Sergej On Fri, Aug 26, 2011 at 5:02 AM, Matthew Hillsborough < matthew.hillsborough@REDACTED> wrote: > Hi everyone, > > I have some hard coded defines in one of my modules that looks like the > following: > > -ifdef(debug). > > -define(FOOBAR, "DEBUG_FOOBAR"). > > -else. > > -define(FOOBAR, "PROD_FOOBAR"). > > -endif. > > I want to be able to deploy this to multiple servers with a config file > using rebar. So I suppose this is a two part question: > > - How does one move the above into an external configuration file and > then read it back into the application? > > > - I need to have multiple configurations as I will have different vms > hitting different FOOBAR instances. I'm looking to copy these to the > appropriate directory with either a bash script or rebar if possible. What's > the best way of doing that? ServerA will have external_a.config, ServerB > would have external_b.config. Ideally rebar or some third party script would > copy that over to external_[x].config and the module that reads this only > knows about external.config. I can ultimately get rid of the debug flag and > just use an external_local.config file where debug is always enabled. > > I read about potentially being able to do this with vars.config, but I > can't find any examples / documentation supporting it. I want to put this > all into my build process and make sure each server has the right FOOBAR > defined. I would love any suggestions anyone has, I'm sure you've all dealt > with something similar when dealing with multiple environments. > > Thanks all! > > Matthew > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Fri Aug 26 10:52:10 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 26 Aug 2011 10:52:10 +0200 Subject: [erlang-questions] catch supervisor failure In-Reply-To: References: Message-ID: <8E9481B9-B03E-47F5-BD90-FB182B4AF8B4@erlang-solutions.com> On 25 Aug 2011, at 20:56, Max Lapshin wrote: > I need to fetch url once a minute. It is ok for remote server to fail > to reply with this url several times, so I want to use supervisor > mechanism for it: > > set 60 restarts in 5 seconds as a limit and if supervisor fails, it > should be stopped for some time. After 10 minutes it should be > restarted. > > Question is: should I use supervisors for this mechanism or I need to > write my own failure tracker? This is a bit convoluted with standard supervisors, but one way to do it, is to have layers of supervisors with rest_for_one supervision. The request process can die and get restarted immediately; if it hits the limit, its supervisor is restarted, which will also restart a 'delay' process before the request process. Given that the start sequence is synchronous, the 'delay' process could wait during init/1, but this is considered bad form. A better solution might be for the request process to ask the delay process for permission. The next problem becomes to avoid waiting 10 minutes the first time. Common tricks is to create an ets table on initial start (have it owned by a higher supervisor), or repeat the same pattern in the layer above, so that the 'delay' process can ask another process if it's an initial start, or escalated restart. I once wrote a modified supervisor behaviour which could keep track of the number of restarts, including escalated restarts. I usually refer to it every once a year or so, but I wrote the code back in 2001, so I'm not sure I can find it anymore. OTOH, writing your own custom supervisor is not *that* hard. The main complication may be that the OTP supervisors also tie into the release handler and appmon. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From connorsml@REDACTED Fri Aug 26 15:25:45 2011 From: connorsml@REDACTED (Michael Connors) Date: Fri, 26 Aug 2011 15:25:45 +0200 Subject: [erlang-questions] Irish Erlang User Group Message-ID: Hi Everyone, A while back, a couple of us set up a Google group for Erlang programmers in Ireland. We now have 20 members, and we are hoping to have a first meeting in September. I just wanted to send out another message to see if there were any more Ireland based Erlang developers that we missed the first time. Here is the Google group if you would like to join. http://groups.google.com/group/erlang-ireland Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.james.barry@REDACTED Fri Aug 26 16:19:40 2011 From: paul.james.barry@REDACTED (Paul Barry) Date: Fri, 26 Aug 2011 15:19:40 +0100 Subject: [erlang-questions] Irish Erlang User Group In-Reply-To: References: Message-ID: On the back cover of the Cesarini/Thompson book it has the rather tantalizing snippet: "He has helped set up Erlang development centers in Ireland, the U.S., and the U.K.". I'd imagine if such a center exists in Ireland, we could be looking at a sizable group of developers. It would be interesting to know more about this... Paul. On 26 August 2011 14:25, Michael Connors wrote: > Hi Everyone, > A while back, a couple of us set up a Google group for Erlang programmers in > Ireland. We now have 20 members, and we are hoping to have a first meeting > in September. > I just wanted to send out another message to see if there were any more > Ireland based Erlang developers that we missed the first time. > Here is the Google group if you would like to join. > http://groups.google.com/group/erlang-ireland > Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Paul Barry, w: http://paulbarry.itcarlow.ie, e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From connorsml@REDACTED Fri Aug 26 16:46:16 2011 From: connorsml@REDACTED (Michael Connors) Date: Fri, 26 Aug 2011 16:46:16 +0200 Subject: [erlang-questions] Irish Erlang User Group In-Reply-To: References: Message-ID: > On the back cover of the Cesarini/Thompson book it has the rather > tantalizing snippet: "He has helped set up Erlang development centers > in Ireland, the U.S., and the U.K.". > > I'd imagine if such a center exists in Ireland, we could be looking at > a sizable group of developers. It would be interesting to know more > about this... > Something tells me, that if such a center exists/existed, it is/was in Athlone. Ericsson have a large operation there. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladimir.kirichenko@REDACTED Fri Aug 26 18:26:05 2011 From: vladimir.kirichenko@REDACTED (Volodymyr Kyrychenko) Date: Fri, 26 Aug 2011 19:26:05 +0300 Subject: [erlang-questions] standard_io out of memory crash Message-ID: <4E57C91D.9070309@gmail.com> code: -module(outofmem). -export([main/0]). main() -> receive X -> X end. $ cat really_big_file | erl -noshell -s outofmem main will consume all available memory and crash: Slogan: binary_alloc: Cannot allocate 65567 bytes of memory (of type "binary"). System version: Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:false] Compiled: Mon May 30 15:59:01 2011 Taints:. Actual problem was discovered while trying to process some 16GB file. If the processing code consumes lines from standard_io slower than erlang consumes pipe data - it will eat all memory and crash. -- Volodymyr Kyrychenko -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From paul.james.barry@REDACTED Fri Aug 26 18:41:39 2011 From: paul.james.barry@REDACTED (Paul Barry) Date: Fri, 26 Aug 2011 17:41:39 +0100 Subject: [erlang-questions] Irish Erlang User Group In-Reply-To: References: Message-ID: Well... not according to Ericsson's Athlone HR people... At a recent undergraduate presentation, I ask how much Erlang they do in the 700-person Ericsson Software Development Centre located in the heart of Ireland, and they looked at me as if I had two heads. They only do Java in Athlone - it's where their Operations Center client software is created/maintained. They went on to tell me that 60% of all of Ericcson's software development is done in Athlone. When I then suggested that the other 40% was Erlang and was done elsewhere (like in Sweden), the Ericsson people told me - rather unbelievably - that they had never heard of Erlang. They only wanted Java programmers. As you can imagine, that's when my conversation with them ended! ;-) Paul On 26 August 2011 15:46, Michael Connors wrote: > >> On the back cover of the Cesarini/Thompson book it has the rather >> tantalizing snippet: "He has helped set up Erlang development centers >> in Ireland, the U.S., and the U.K.". >> >> I'd imagine if such a center exists in Ireland, we could be looking at >> a sizable group of developers. ?It would be interesting to know more >> about this... > > Something tells me, that if such a center exists/existed, it is/was in > Athlone. Ericsson have a large operation there. > -- Paul Barry, w: http://paulbarry.itcarlow.ie, e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From gnoblin@REDACTED Sat Aug 27 02:27:35 2011 From: gnoblin@REDACTED (Slav Pankratov) Date: Sat, 27 Aug 2011 03:27:35 +0300 Subject: [erlang-questions] rudp & erlang Message-ID: Hello! Are there any mature "reliable udp" (rudp) frameworks available for erlang? thanks, Slav -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Sat Aug 27 05:43:30 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 26 Aug 2011 20:43:30 -0700 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: <2817450.46CDmI7xT0@moltowork> References: <20110823232246.GB4977@alumni.caltech.edu> <2817450.46CDmI7xT0@moltowork> Message-ID: 1) works as long as you control the sender or you have a well-defined policy when multiple fields are present, but I'd otherwise consider it a footgun. Your policy could easily be "there is only one field present at the top level" -- the next field encountered would simply be considered its own PDU. I really find it odd that protobuf doesn't provide a CHOICE construct. The argument is that making everything optional gives you the most of that, with no additional overhead. What you want is the additional rule that there is only ONE of those optionals, something which can be added at the application layer if it's needed. I'm not saying it wouldn't be nice if "switch" and PDU naming/segmenting were supported in protobuf -- that would be sweet! But we deal pretty well without it, and still find it better than the alternatives, especially for cross-language projects. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Thu, Aug 25, 2011 at 3:04 AM, Vincent de Phily < vincent.dephily@REDACTED> wrote: > On Wednesday 24 August 2011 15:15:57 Jon Watte wrote: > > You solve that in one of two ways: > > 1) Create a message that is the "PDU" that contains the union of > everything > > that could be sent, > > or > > 2) Create a wrapper that is just "type" followed by "binary data that is > a > > marshaled message." > > > > If you believe that the entirety of a protocol should be described by > that > > protocol, then 1) is actually a fine solution. The preserve-and-forward > > semantic of unknown field ids means that forward compatibility is still > > possible. > > > > message MyProtocolPDU { > > optional SomeMessage some_message = 1; > > optional AnotherMessage other_message = 2; > > ... > > } > > > > So, encoding a MyProtocolPDU that contains a SomeMessage will end up > being > > different than encoding a MyProtocolPDU that contains an AnotherMessage. > > At the high level, you have to know that there is only one member in each > > MyProtocolPDU; I find that to be quite acceptable. > > > Yup, they're reasonable solutions (as long as you planed ahead). 2) may be > ugly and innefficient, but it has been used as a pragmatic solution in > countless formats. 1) works as long as you control the sender or you have a > well-defined policy when multiple fields are present, but I'd otherwise > consider it a footgun. I really find it odd that protobuf doesn't provide a > CHOICE construct. > > -- > Vincent de Phily > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Sat Aug 27 09:10:43 2011 From: vances@REDACTED (Vance Shipley) Date: Sat, 27 Aug 2011 12:40:43 +0530 Subject: [erlang-questions] rudp & erlang In-Reply-To: References: Message-ID: <20110827071041.GC57466@rajeev.wavenet.lk> On Sat, Aug 27, 2011 at 03:27:35AM +0300, Slav Pankratov wrote: } Are there any mature "reliable udp" (rudp) frameworks available for erlang? Yes, the sctp application provides an API to the SCTP protocol which provides a reliable UDP service. -- -Vance From mabrek@REDACTED Sat Aug 27 10:22:09 2011 From: mabrek@REDACTED (Anton Lebedevich) Date: Sat, 27 Aug 2011 12:22:09 +0400 Subject: [erlang-questions] catch supervisor failure In-Reply-To: References: Message-ID: <4E58A931.7010609@gmail.com> On 08/25/2011 10:56 PM, Max Lapshin wrote: > I need to fetch url once a minute. It is ok for remote server to fail > to reply with this url several times, so I want to use supervisor > mechanism for it: > > set 60 restarts in 5 seconds as a limit and if supervisor fails, it > should be stopped for some time. After 10 minutes it should be > restarted. > > Question is: should I use supervisors for this mechanism or I need to > write my own failure tracker? Take a look at supervisor2 from rabbitmq-server, it seems that they implemented desired behaviour: %% 3) child specifications can contain, as the restart type, a tuple %% {permanent, Delay} | {transient, Delay} where Delay >= 0. The %% delay, in seconds, indicates what should happen if a child, upon %% being restarted, exceeds the MaxT and MaxR parameters. Thus, if %% a child exits, it is restarted as normal. If it exits %% sufficiently quickly and often to exceed the boundaries set by %% the MaxT and MaxR parameters, and a Delay is specified, then %% rather than stopping the supervisor, the supervisor instead %% continues and tries to start up the child again, Delay seconds %% later. Regards, Anton Lebedevich. From neumark.peter@REDACTED Sat Aug 27 11:57:33 2011 From: neumark.peter@REDACTED (Peter Neumark) Date: Sat, 27 Aug 2011 11:57:33 +0200 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <20110823232246.GB4977@alumni.caltech.edu> <2817450.46CDmI7xT0@moltowork> Message-ID: I'm surprised Apache Thrift wasn't mentioned yet in this thread. Thrift is inspired by protobuf, so it sports s similar (although more modular) design. It supports a wide range a languages, several protocols and transports (with the ability to add your own). The erlang lib + code generator code is actively developed and already works quite well. Peter On Sat, Aug 27, 2011 at 5:43 AM, Jon Watte wrote: > 1) works as long as you control the sender or you have a well-defined > policy when multiple fields are present, but I'd otherwise consider it a > footgun. > > > > Your policy could easily be "there is only one field present at the top > level" -- the next field encountered would simply be considered its own PDU. > > > I really find it odd that protobuf doesn't provide a CHOICE construct. > > > The argument is that making everything optional gives you the most of that, > with no additional overhead. What you want is the additional rule that there > is only ONE of those optionals, something which can be added at the > application layer if it's needed. > I'm not saying it wouldn't be nice if "switch" and PDU naming/segmenting > were supported in protobuf -- that would be sweet! But we deal pretty well > without it, and still find it better than the alternatives, especially for > cross-language projects. > > Sincerely, > > jw > > -- > Americans might object: there is no way we would sacrifice our living > standards for the benefit of people in the rest of the world. Nevertheless, > whether we get there willingly or not, we shall soon have lower consumption > rates, because our present rates are unsustainable. > > > > > On Thu, Aug 25, 2011 at 3:04 AM, Vincent de Phily < > vincent.dephily@REDACTED> wrote: > > >> On Wednesday 24 August 2011 15:15:57 Jon Watte wrote: >> > You solve that in one of two ways: >> > 1) Create a message that is the "PDU" that contains the union of >> everything >> > that could be sent, >> > or >> > 2) Create a wrapper that is just "type" followed by "binary data that is >> a >> > marshaled message." >> > >> > If you believe that the entirety of a protocol should be described by >> that >> > protocol, then 1) is actually a fine solution. The preserve-and-forward >> > semantic of unknown field ids means that forward compatibility is still >> > possible. >> > >> > message MyProtocolPDU { >> > optional SomeMessage some_message = 1; >> > optional AnotherMessage other_message = 2; >> > ... >> > } >> > >> > So, encoding a MyProtocolPDU that contains a SomeMessage will end up >> being >> > different than encoding a MyProtocolPDU that contains an AnotherMessage. >> > At the high level, you have to know that there is only one member in >> each >> > MyProtocolPDU; I find that to be quite acceptable. >> >> >> Yup, they're reasonable solutions (as long as you planed ahead). 2) may be >> ugly and innefficient, but it has been used as a pragmatic solution in >> countless formats. 1) works as long as you control the sender or you have >> a >> well-defined policy when multiple fields are present, but I'd otherwise >> consider it a footgun. I really find it odd that protobuf doesn't provide >> a >> CHOICE construct. >> >> -- >> Vincent de Phily >> >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wsong.cn@REDACTED Sat Aug 27 15:57:30 2011 From: wsong.cn@REDACTED (Wenqiang Song) Date: Sat, 27 Aug 2011 21:57:30 +0800 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <20110823232246.GB4977@alumni.caltech.edu> <2817450.46CDmI7xT0@moltowork> Message-ID: I second this. But the code generator directly generates synchronous socket server which is hard to embed into an existing asynchronous server. Andy On 8/27/11, Peter Neumark wrote: > I'm surprised Apache Thrift wasn't mentioned yet in this thread. > Thrift is inspired by protobuf, so it sports s similar (although more > modular) design. > It supports a wide range a languages, several protocols and transports (with > the ability to add your own). > The erlang lib + code generator code is actively developed and already works > quite well. > > Peter > > On Sat, Aug 27, 2011 at 5:43 AM, Jon Watte wrote: > >> 1) works as long as you control the sender or you have a well-defined >> policy when multiple fields are present, but I'd otherwise consider it a >> footgun. >> >> >> >> Your policy could easily be "there is only one field present at the top >> level" -- the next field encountered would simply be considered its own >> PDU. >> >> >> I really find it odd that protobuf doesn't provide a CHOICE construct. >> >> >> The argument is that making everything optional gives you the most of >> that, >> with no additional overhead. What you want is the additional rule that >> there >> is only ONE of those optionals, something which can be added at the >> application layer if it's needed. >> I'm not saying it wouldn't be nice if "switch" and PDU naming/segmenting >> were supported in protobuf -- that would be sweet! But we deal pretty well >> without it, and still find it better than the alternatives, especially for >> cross-language projects. >> >> Sincerely, >> >> jw >> >> -- >> Americans might object: there is no way we would sacrifice our living >> standards for the benefit of people in the rest of the world. >> Nevertheless, >> whether we get there willingly or not, we shall soon have lower >> consumption >> rates, because our present rates are unsustainable. >> >> >> >> >> On Thu, Aug 25, 2011 at 3:04 AM, Vincent de Phily < >> vincent.dephily@REDACTED> wrote: >> >> >>> On Wednesday 24 August 2011 15:15:57 Jon Watte wrote: >>> > You solve that in one of two ways: >>> > 1) Create a message that is the "PDU" that contains the union of >>> everything >>> > that could be sent, >>> > or >>> > 2) Create a wrapper that is just "type" followed by "binary data that >>> > is >>> a >>> > marshaled message." >>> > >>> > If you believe that the entirety of a protocol should be described by >>> that >>> > protocol, then 1) is actually a fine solution. The preserve-and-forward >>> > semantic of unknown field ids means that forward compatibility is still >>> > possible. >>> > >>> > message MyProtocolPDU { >>> > optional SomeMessage some_message = 1; >>> > optional AnotherMessage other_message = 2; >>> > ... >>> > } >>> > >>> > So, encoding a MyProtocolPDU that contains a SomeMessage will end up >>> being >>> > different than encoding a MyProtocolPDU that contains an >>> > AnotherMessage. >>> > At the high level, you have to know that there is only one member in >>> each >>> > MyProtocolPDU; I find that to be quite acceptable. >>> >>> >>> Yup, they're reasonable solutions (as long as you planed ahead). 2) may >>> be >>> ugly and innefficient, but it has been used as a pragmatic solution in >>> countless formats. 1) works as long as you control the sender or you have >>> a >>> well-defined policy when multiple fields are present, but I'd otherwise >>> consider it a footgun. I really find it odd that protobuf doesn't provide >>> a >>> CHOICE construct. >>> >>> -- >>> Vincent de Phily >>> >>> >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- Sent from my mobile device --------------------------------------------------------------- ???????????????????? ???????????????????? From neumark.peter@REDACTED Sat Aug 27 18:11:24 2011 From: neumark.peter@REDACTED (Peter Neumark) Date: Sat, 27 Aug 2011 18:11:24 +0200 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <20110823232246.GB4977@alumni.caltech.edu> <2817450.46CDmI7xT0@moltowork> Message-ID: It is true that the erlang socket server which ships with thrift is synchronous (it's a gen_server). This code is not generated, however. It is possible to create your own async server without touching the code generator. I know because I am writing a server for use with socket.io-erlang (which is async). Peter On Sat, Aug 27, 2011 at 3:57 PM, Wenqiang Song wrote: > I second this. But the code generator directly generates synchronous > socket server which is hard to embed into an existing asynchronous > server. > > Andy > > On 8/27/11, Peter Neumark wrote: > > I'm surprised Apache Thrift wasn't mentioned yet in this thread. > > Thrift is inspired by protobuf, so it sports s similar (although more > > modular) design. > > It supports a wide range a languages, several protocols and transports > (with > > the ability to add your own). > > The erlang lib + code generator code is actively developed and already > works > > quite well. > > > > Peter > > > > On Sat, Aug 27, 2011 at 5:43 AM, Jon Watte wrote: > > > >> 1) works as long as you control the sender or you have a well-defined > >> policy when multiple fields are present, but I'd otherwise consider it a > >> footgun. > >> > >> > >> > >> Your policy could easily be "there is only one field present at the top > >> level" -- the next field encountered would simply be considered its own > >> PDU. > >> > >> > >> I really find it odd that protobuf doesn't provide a CHOICE construct. > >> > >> > >> The argument is that making everything optional gives you the most of > >> that, > >> with no additional overhead. What you want is the additional rule that > >> there > >> is only ONE of those optionals, something which can be added at the > >> application layer if it's needed. > >> I'm not saying it wouldn't be nice if "switch" and PDU naming/segmenting > >> were supported in protobuf -- that would be sweet! But we deal pretty > well > >> without it, and still find it better than the alternatives, especially > for > >> cross-language projects. > >> > >> Sincerely, > >> > >> jw > >> > >> -- > >> Americans might object: there is no way we would sacrifice our living > >> standards for the benefit of people in the rest of the world. > >> Nevertheless, > >> whether we get there willingly or not, we shall soon have lower > >> consumption > >> rates, because our present rates are unsustainable. > >> > >> > >> > >> > >> On Thu, Aug 25, 2011 at 3:04 AM, Vincent de Phily < > >> vincent.dephily@REDACTED> wrote: > >> > >> > >>> On Wednesday 24 August 2011 15:15:57 Jon Watte wrote: > >>> > You solve that in one of two ways: > >>> > 1) Create a message that is the "PDU" that contains the union of > >>> everything > >>> > that could be sent, > >>> > or > >>> > 2) Create a wrapper that is just "type" followed by "binary data that > >>> > is > >>> a > >>> > marshaled message." > >>> > > >>> > If you believe that the entirety of a protocol should be described by > >>> that > >>> > protocol, then 1) is actually a fine solution. The > preserve-and-forward > >>> > semantic of unknown field ids means that forward compatibility is > still > >>> > possible. > >>> > > >>> > message MyProtocolPDU { > >>> > optional SomeMessage some_message = 1; > >>> > optional AnotherMessage other_message = 2; > >>> > ... > >>> > } > >>> > > >>> > So, encoding a MyProtocolPDU that contains a SomeMessage will end up > >>> being > >>> > different than encoding a MyProtocolPDU that contains an > >>> > AnotherMessage. > >>> > At the high level, you have to know that there is only one member in > >>> each > >>> > MyProtocolPDU; I find that to be quite acceptable. > >>> > >>> > >>> Yup, they're reasonable solutions (as long as you planed ahead). 2) may > >>> be > >>> ugly and innefficient, but it has been used as a pragmatic solution in > >>> countless formats. 1) works as long as you control the sender or you > have > >>> a > >>> well-defined policy when multiple fields are present, but I'd otherwise > >>> consider it a footgun. I really find it odd that protobuf doesn't > provide > >>> a > >>> CHOICE construct. > >>> > >>> -- > >>> Vincent de Phily > >>> > >>> > >>> > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > > > > -- > Sent from my mobile device > > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sat Aug 27 18:31:36 2011 From: g@REDACTED (Garrett Smith) Date: Sat, 27 Aug 2011 11:31:36 -0500 Subject: [erlang-questions] How do I move this into a config file? In-Reply-To: References: Message-ID: On Thu, Aug 25, 2011 at 10:02 PM, Matthew Hillsborough wrote: > Hi everyone, > > I have some hard coded defines in one of my modules that looks like the > following: > > -ifdef(debug). > > -define(FOOBAR, "DEBUG_FOOBAR"). > > -else. > > -define(FOOBAR, "PROD_FOOBAR"). > > -endif. > > I want to be able to deploy this to multiple servers with a config file > using rebar. So I suppose this is a two part question: > > How does one move the above into an external configuration file and then > read it back into the application? There's a canonical way to do this in Erlang that you can (should) use. It will help to do a little reading on "applications": http://www.erlang.org/doc/design_principles/applications.html In particular, look at "Configuring an Application". Also, check out the get_env and get_all_env functions in the application module: http://www.erlang.org/doc/man/application.html The gist: - Define you release specific configuration in a *.config file - When running erl, specify "-config FILE" as an option (note, do not include the .config suffix, but path is okay) - From your OTP application, use application:get_env/1 to read your config The only catch here that can be confusing (IMO) is application:get_env/1, which looks up values associated with the application that the process is running under. If process wasn't started by an application (application:start/1) you'll get undefined. Just an FYI that can throw you. Garrett From g@REDACTED Sat Aug 27 18:34:23 2011 From: g@REDACTED (Garrett Smith) Date: Sat, 27 Aug 2011 11:34:23 -0500 Subject: [erlang-questions] How do I move this into a config file? In-Reply-To: References: Message-ID: On Sat, Aug 27, 2011 at 11:31 AM, Garrett Smith wrote: > > If process wasn't > started by an application (application:start/1) you'll get undefined. Pedantic clarification: "started *by way of* an application" - i.e. started within the application's supervisory hierarchy. Garrett From jwatte@REDACTED Sat Aug 27 19:38:50 2011 From: jwatte@REDACTED (Jon Watte) Date: Sat, 27 Aug 2011 10:38:50 -0700 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <20110823232246.GB4977@alumni.caltech.edu> <2817450.46CDmI7xT0@moltowork> Message-ID: We looked at Thrift, back when it was Facebook's, and didn't like it because: 1) It's more of an RPC framework than we wanted 2) It's significantly less compact (about 40% more overhead for our use case) If that's not a problem -- or if you're specifically looking for a cross-platform RPC framework -- then Thrift can work very well! Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Sat, Aug 27, 2011 at 2:57 AM, Peter Neumark wrote: > I'm surprised Apache Thrift wasn't mentioned yet in this thread. > Thrift is inspired by protobuf, so it sports s similar (although more > modular) design. > It supports a wide range a languages, several protocols and transports > (with the ability to add your own). > The erlang lib + code generator code is actively developed and already > works quite well. > > Peter > > On Sat, Aug 27, 2011 at 5:43 AM, Jon Watte wrote: > >> 1) works as long as you control the sender or you have a well-defined >> policy when multiple fields are present, but I'd otherwise consider it a >> footgun. >> >> >> >> Your policy could easily be "there is only one field present at the top >> level" -- the next field encountered would simply be considered its own PDU. >> >> >> I really find it odd that protobuf doesn't provide a CHOICE construct. >> >> >> The argument is that making everything optional gives you the most of >> that, with no additional overhead. What you want is the additional rule that >> there is only ONE of those optionals, something which can be added at the >> application layer if it's needed. >> I'm not saying it wouldn't be nice if "switch" and PDU naming/segmenting >> were supported in protobuf -- that would be sweet! But we deal pretty well >> without it, and still find it better than the alternatives, especially for >> cross-language projects. >> >> Sincerely, >> >> jw >> >> -- >> Americans might object: there is no way we would sacrifice our living >> standards for the benefit of people in the rest of the world. Nevertheless, >> whether we get there willingly or not, we shall soon have lower consumption >> rates, because our present rates are unsustainable. >> >> >> >> >> On Thu, Aug 25, 2011 at 3:04 AM, Vincent de Phily < >> vincent.dephily@REDACTED> wrote: >> >> >>> On Wednesday 24 August 2011 15:15:57 Jon Watte wrote: >>> > You solve that in one of two ways: >>> > 1) Create a message that is the "PDU" that contains the union of >>> everything >>> > that could be sent, >>> > or >>> > 2) Create a wrapper that is just "type" followed by "binary data that >>> is a >>> > marshaled message." >>> > >>> > If you believe that the entirety of a protocol should be described by >>> that >>> > protocol, then 1) is actually a fine solution. The preserve-and-forward >>> > semantic of unknown field ids means that forward compatibility is still >>> > possible. >>> > >>> > message MyProtocolPDU { >>> > optional SomeMessage some_message = 1; >>> > optional AnotherMessage other_message = 2; >>> > ... >>> > } >>> > >>> > So, encoding a MyProtocolPDU that contains a SomeMessage will end up >>> being >>> > different than encoding a MyProtocolPDU that contains an >>> AnotherMessage. >>> > At the high level, you have to know that there is only one member in >>> each >>> > MyProtocolPDU; I find that to be quite acceptable. >>> >>> >>> Yup, they're reasonable solutions (as long as you planed ahead). 2) may >>> be >>> ugly and innefficient, but it has been used as a pragmatic solution in >>> countless formats. 1) works as long as you control the sender or you have >>> a >>> well-defined policy when multiple fields are present, but I'd otherwise >>> consider it a footgun. I really find it odd that protobuf doesn't provide >>> a >>> CHOICE construct. >>> >>> -- >>> Vincent de Phily >>> >>> >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From igouy2@REDACTED Sat Aug 27 20:17:59 2011 From: igouy2@REDACTED (Isaac Gouy) Date: Sat, 27 Aug 2011 11:17:59 -0700 (PDT) Subject: [erlang-questions] Please suggest why these repeated timings increase Message-ID: <1314469079.13480.YahooMailNeo@web65606.mail.ac4.yahoo.com> Here's a tiny (somewhat silly) program - a circle of processes, removed one at a time until there's only one left. I expect there's a simple explanation - I expect I've done something basic wrong - but repeatedly creating and reducing such a circle of processes seems to slowdown, and I'd like to understand why. There doesn't seem to be any accumulation of zombie processes, so I'm a little puzzled what else might be going on. $ /usr/local/src/otp_src_R14B02/bin/erlc josephus.erl $ /usr/local/src/otp_src_R14B02/bin/erl -noshell -run josephus main 943157 2219531 3490628 4814178 6334408 7419798 8615281 9778484 11364821 12440720 -module(josephus). -export([main/0,soldier/3,timedloop/1]). soldier( PreviousPid, NextPid, Id ) -> ?? receive ????? {MainPid,countoff,Kth,K} -> ???????? countoff(PreviousPid,NextPid,Id,MainPid,Kth,K); ???????? ????? {set_previous,Pid} -> ???????? soldier(Pid,NextPid,Id); ????? {set_next,Pid} -> ???????? soldier(PreviousPid,Pid,Id) ?? end. countoff( Previous, Next, Id, MainPid, Kth, K ) when Kth == K -> ?? MainPid ! {details,[Id,Previous,self(),Next]}, ?? if ????? Previous == self() -> ???????? MainPid ! {countoff_ack,Id}; ????? true -> ???????? Previous ! {set_next,Next}, ???????? Next ! {set_previous,Previous}, ???????? Next ! {MainPid,countoff,Kth,1} ?? end; countoff( Previous, Next, Id, MainPid, Kth, K ) ->??? ?? Next ! {MainPid,countoff,Kth,K+1}, ?? soldier(Previous,Next,Id). ring(N) -> ?? ring(N,1,nil,[]). ring(N,I,P,L) when I =< N -> ?? Pid = spawn(josephus,soldier,[P,nil,I]), ?? ring(N,I+1,Pid,[Pid|L]); ring(_,_,Last,L) ->??? ?? First = ring_set_next(nil,L),? ?? Last ! {set_next,First}, ?? First ! {set_previous,Last}, ?? First. ? ring_set_next(Last,[]) -> Last; ring_set_next(P,[Pid|T]) -> ?? Pid ! {set_next,P}, ?? ring_set_next(Pid,T). countoffSoldiers(N,Kth) -> ?? Soldier = ring(N), ?? %Soldier ! {self(),countoff,Kth,1}, % standard Josephus problem ?? Soldier ! {self(),countoff,Kth,Kth}, ?? receive {countoff_ack,I} -> I end. ? %----------------------------------------------------------------------------- timedloop(0) -> ok; timedloop(N) -> ??? countoffSoldiers(40,3), ??? timedloop(N-1). repeat(_,0) -> ok; repeat(N, Repeat) -> ??? {MicroSecs,I} = timer:tc(josephus,timedloop,[N]), ??? io:format("~w~n",[MicroSecs]), ??? repeat(N,Repeat-1). main() -> ??? repeat(1000,10), ??? halt(0). From igouy2@REDACTED Sun Aug 28 00:50:09 2011 From: igouy2@REDACTED (Isaac Gouy) Date: Sat, 27 Aug 2011 15:50:09 -0700 (PDT) Subject: [erlang-questions] Please suggest why these repeated timings increase In-Reply-To: <1314469079.13480.YahooMailNeo@web65606.mail.ac4.yahoo.com> References: <1314469079.13480.YahooMailNeo@web65606.mail.ac4.yahoo.com> Message-ID: <1314485409.10707.YahooMailNeo@web65604.mail.ac4.yahoo.com> I found my mistake ;-) I'd put this in to debug, and then removed the handler ?? MainPid ! {details,[Id,Previous,self(),Next]} so messages were just accumulating... ----- Forwarded Message ----- > From: Isaac Gouy > To: "erlang-questions@REDACTED" > Cc: > Sent: Saturday, August 27, 2011 11:17 AM > Subject: Please suggest why these repeated timings increase > > Here's a tiny (somewhat silly) program - a circle of processes, removed one > at a time until there's only one left. > > I expect there's a simple explanation - I expect I've done something > basic wrong - but repeatedly creating and reducing such a circle of processes > seems to slowdown, and I'd like to understand why. > > There doesn't seem to be any accumulation of zombie processes, so I'm a > little puzzled what else might be going on. > > > > $ /usr/local/src/otp_src_R14B02/bin/erlc josephus.erl > > $ /usr/local/src/otp_src_R14B02/bin/erl -noshell -run josephus main > > 943157 > 2219531 > 3490628 > 4814178 > 6334408 > 7419798 > 8615281 > 9778484 > 11364821 > 12440720 > > > -module(josephus). > -export([main/0,soldier/3,timedloop/1]). > > soldier( PreviousPid, NextPid, Id ) -> > ?? receive > ????? {MainPid,countoff,Kth,K} -> > ???????? countoff(PreviousPid,NextPid,Id,MainPid,Kth,K); > ???????? > ????? {set_previous,Pid} -> > ???????? soldier(Pid,NextPid,Id); > > ????? {set_next,Pid} -> > ???????? soldier(PreviousPid,Pid,Id) > ?? end. > > > countoff( Previous, Next, Id, MainPid, Kth, K ) when Kth == K -> > ?? MainPid ! {details,[Id,Previous,self(),Next]}, > ?? if > ????? Previous == self() -> > ???????? MainPid ! {countoff_ack,Id}; > > ????? true -> > ???????? Previous ! {set_next,Next}, > ???????? Next ! {set_previous,Previous}, > ???????? Next ! {MainPid,countoff,Kth,1} > ?? end; > > countoff( Previous, Next, Id, MainPid, Kth, K ) ->??? > ?? Next ! {MainPid,countoff,Kth,K+1}, > ?? soldier(Previous,Next,Id). > > > ring(N) -> > ?? ring(N,1,nil,[]). > > ring(N,I,P,L) when I =< N -> > ?? Pid = spawn(josephus,soldier,[P,nil,I]), > ?? ring(N,I+1,Pid,[Pid|L]); > > ring(_,_,Last,L) ->??? > ?? First = ring_set_next(nil,L),? > ?? Last ! {set_next,First}, > ?? First ! {set_previous,Last}, > ?? First. > ? > ring_set_next(Last,[]) -> Last; > ring_set_next(P,[Pid|T]) -> > ?? Pid ! {set_next,P}, > ?? ring_set_next(Pid,T). > > > countoffSoldiers(N,Kth) -> > ?? Soldier = ring(N), > ?? %Soldier ! {self(),countoff,Kth,1}, % standard Josephus problem > ?? Soldier ! {self(),countoff,Kth,Kth}, > ?? receive {countoff_ack,I} -> I end. > > ? > %----------------------------------------------------------------------------- > > timedloop(0) -> ok; > timedloop(N) -> > ??? countoffSoldiers(40,3), > ??? timedloop(N-1). > > repeat(_,0) -> ok; > repeat(N, Repeat) -> > ??? {MicroSecs,I} = timer:tc(josephus,timedloop,[N]), > ??? io:format("~w~n",[MicroSecs]), > ??? repeat(N,Repeat-1). > > main() -> > ??? repeat(1000,10), > ??? halt(0). > From wsong.cn@REDACTED Sun Aug 28 01:18:01 2011 From: wsong.cn@REDACTED (Wenqiang Song) Date: Sun, 28 Aug 2011 07:18:01 +0800 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <20110823232246.GB4977@alumni.caltech.edu> <2817450.46CDmI7xT0@moltowork> Message-ID: You are right, the socket server is in lib/erl. The whole Erlang library is tightly bound to synchronous model and hard to modify. Do you mind sharing your code? Thanks Andy On 8/28/11, Peter Neumark wrote: > It is true that the erlang socket server which ships with thrift is > synchronous (it's a gen_server). > This code is not generated, however. It is possible to create your own async > server without touching the code generator. > I know because I am writing a server for use with socket.io-erlang (which is > async). > > Peter > > On Sat, Aug 27, 2011 at 3:57 PM, Wenqiang Song wrote: > >> I second this. But the code generator directly generates synchronous >> socket server which is hard to embed into an existing asynchronous >> server. >> >> Andy >> >> On 8/27/11, Peter Neumark wrote: >> > I'm surprised Apache Thrift wasn't mentioned yet in this thread. >> > Thrift is inspired by protobuf, so it sports s similar (although more >> > modular) design. >> > It supports a wide range a languages, several protocols and transports >> (with >> > the ability to add your own). >> > The erlang lib + code generator code is actively developed and already >> works >> > quite well. >> > >> > Peter >> > >> > On Sat, Aug 27, 2011 at 5:43 AM, Jon Watte wrote: >> > >> >> 1) works as long as you control the sender or you have a well-defined >> >> policy when multiple fields are present, but I'd otherwise consider it >> >> a >> >> footgun. >> >> >> >> >> >> >> >> Your policy could easily be "there is only one field present at the top >> >> level" -- the next field encountered would simply be considered its own >> >> PDU. >> >> >> >> >> >> I really find it odd that protobuf doesn't provide a CHOICE construct. >> >> >> >> >> >> The argument is that making everything optional gives you the most of >> >> that, >> >> with no additional overhead. What you want is the additional rule that >> >> there >> >> is only ONE of those optionals, something which can be added at the >> >> application layer if it's needed. >> >> I'm not saying it wouldn't be nice if "switch" and PDU >> >> naming/segmenting >> >> were supported in protobuf -- that would be sweet! But we deal pretty >> well >> >> without it, and still find it better than the alternatives, especially >> for >> >> cross-language projects. >> >> >> >> Sincerely, >> >> >> >> jw >> >> >> >> -- >> >> Americans might object: there is no way we would sacrifice our living >> >> standards for the benefit of people in the rest of the world. >> >> Nevertheless, >> >> whether we get there willingly or not, we shall soon have lower >> >> consumption >> >> rates, because our present rates are unsustainable. >> >> >> >> >> >> >> >> >> >> On Thu, Aug 25, 2011 at 3:04 AM, Vincent de Phily < >> >> vincent.dephily@REDACTED> wrote: >> >> >> >> >> >>> On Wednesday 24 August 2011 15:15:57 Jon Watte wrote: >> >>> > You solve that in one of two ways: >> >>> > 1) Create a message that is the "PDU" that contains the union of >> >>> everything >> >>> > that could be sent, >> >>> > or >> >>> > 2) Create a wrapper that is just "type" followed by "binary data >> >>> > that >> >>> > is >> >>> a >> >>> > marshaled message." >> >>> > >> >>> > If you believe that the entirety of a protocol should be described >> >>> > by >> >>> that >> >>> > protocol, then 1) is actually a fine solution. The >> preserve-and-forward >> >>> > semantic of unknown field ids means that forward compatibility is >> still >> >>> > possible. >> >>> > >> >>> > message MyProtocolPDU { >> >>> > optional SomeMessage some_message = 1; >> >>> > optional AnotherMessage other_message = 2; >> >>> > ... >> >>> > } >> >>> > >> >>> > So, encoding a MyProtocolPDU that contains a SomeMessage will end up >> >>> being >> >>> > different than encoding a MyProtocolPDU that contains an >> >>> > AnotherMessage. >> >>> > At the high level, you have to know that there is only one member in >> >>> each >> >>> > MyProtocolPDU; I find that to be quite acceptable. >> >>> >> >>> >> >>> Yup, they're reasonable solutions (as long as you planed ahead). 2) >> >>> may >> >>> be >> >>> ugly and innefficient, but it has been used as a pragmatic solution in >> >>> countless formats. 1) works as long as you control the sender or you >> have >> >>> a >> >>> well-defined policy when multiple fields are present, but I'd >> >>> otherwise >> >>> consider it a footgun. I really find it odd that protobuf doesn't >> provide >> >>> a >> >>> CHOICE construct. >> >>> >> >>> -- >> >>> Vincent de Phily >> >>> >> >>> >> >>> >> >> >> >> >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> > >> >> -- >> Sent from my mobile device >> >> --------------------------------------------------------------- >> ???????????????????? >> ???????????????????? >> > -- Sent from my mobile device --------------------------------------------------------------- ???????????????????? ???????????????????? From roberto@REDACTED Sun Aug 28 11:55:31 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 28 Aug 2011 11:55:31 +0200 Subject: [erlang-questions] ets:tid() type Message-ID: Dear list, I'm defining a spec using the ets:tid() remote type. While everything goes well on R14xx, I've reports from users that modules do not compile on centos with R13B04 [not sure if the OS is relevant here]. The error they get is: $ make /usr/local/bin/erlc -W -I include -o ebin src/*.erl src/acceptor.erl:85: referring to built-in type tid as a remote type; please take out the module name make: *** [all] Error 1 I've read about some issues on tid() definition, so my question is: should I just adapt my specs to any() instead of ets:tid() so that the code compiles on previous versions? I can't obviously use tid() alone on R14xx as it is then an undefined type. Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Sun Aug 28 12:00:18 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sun, 28 Aug 2011 13:00:18 +0300 Subject: [erlang-questions] ets:tid() type In-Reply-To: References: Message-ID: <4E5A11B2.9000908@cs.ntua.gr> On 08/28/11 12:55, Roberto Ostinelli wrote: > Dear list, > > I'm defining a spec using the ets:tid() remote type. While everything > goes well on R14xx, I've reports from users that modules do not compile > on centos with R13B04 [not sure if the OS is relevant here]. > > The error they get is: > > $ make > /usr/local/bin/erlc -W -I include -o ebin src//.erl > src/acceptor.erl:85: referring to built-in type tid as a remote type; > please take out the module name > make: **/ [all] Error 1 > > I've read about some issues on tid() definition, so my question is: > should I just adapt my specs to any() instead of ets:tid() so that the > code compiles on previous versions? I can't obviously use tid() alone on > R14xx as it is then an undefined type. This type used to be a hard-coded builtin in R13 and became a type exported from the 'ets' module (where it clearly belongs) in R14. If you want your code to be compilable with different Erlang/OTP versions, the best solution is doing this as part of a configure option (see e.g. the code of Scalaris for a good example on how to do this). But I would really recommend to the users of R13 to switch to R14. There are many good reasons to do that. Kostis From roberto@REDACTED Sun Aug 28 12:46:10 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 28 Aug 2011 12:46:10 +0200 Subject: [erlang-questions] record with dynamic item name Message-ID: Hi list, is there any way to retrieve a record item whose identifier is in a variable? For instance: -record(state, { type = 1 }). These do not work: Type = type, Val = #state.Type. nor #state{Type = Val}. I have a routine which basically just needs to lookup record items and I'd like to avoid writing 20 time the same code just because Type changes. Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sun Aug 28 12:46:37 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 28 Aug 2011 12:46:37 +0200 Subject: [erlang-questions] ets:tid() type In-Reply-To: <4E5A11B2.9000908@cs.ntua.gr> References: <4E5A11B2.9000908@cs.ntua.gr> Message-ID: Thank you Kostis. Will do. Cheers, r. 2011/8/28 Kostis Sagonas > This type used to be a hard-coded builtin in R13 and became a type exported > from the 'ets' module (where it clearly belongs) in R14. > > If you want your code to be compilable with different Erlang/OTP versions, > the best solution is doing this as part of a configure option (see e.g. the > code of Scalaris for a good example on how to do this). > > But I would really recommend to the users of R13 to switch to R14. There > are many good reasons to do that. > > Kostis > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ttmrichter@REDACTED Sun Aug 28 13:40:52 2011 From: ttmrichter@REDACTED (Michael Richter) Date: Sun, 28 Aug 2011 14:40:52 +0300 Subject: [erlang-questions] record with dynamic item name In-Reply-To: References: Message-ID: On 28 August 2011 13:46, Roberto Ostinelli wrote: > is there any way to retrieve a record item whose identifier is in a > variable? For instance: > > You could probably quickly cook up a function or macros that does that using record_info as detailed in http://www.erlang.org/doc/reference_manual/records.html#id80708. -- "Perhaps people don't believe this, but throughout all of the discussions of entering China our focus has really been what's best for the Chinese people. It's not been about our revenue or profit or whatnot." --Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra. -------------- next part -------------- An HTML attachment was scrubbed... URL: From freza@REDACTED Sun Aug 28 20:13:11 2011 From: freza@REDACTED (Jachym Holecek) Date: Sun, 28 Aug 2011 19:13:11 +0100 Subject: [erlang-questions] record with dynamic item name In-Reply-To: References: Message-ID: <20110828181311.GA1619@hanele.lan> # Roberto Ostinelli 2011-08-28: > is there any way to retrieve a record item whose identifier is in a variable? > > For instance: > > -record(state, { > ??? type = 1 > }). > > These do not work: > > Type = type, > Val = #state.Type. > > nor > > #state{Type = Val}. If you're already passing Type to the common extraction routine, perhaps you could pass field position (#state.type) instead of field name (type) and use element/2 to extract it. If the names are fixed, but their positions vary you can use record_info/2 as Michael suggests. Alternatively you could establish the horribly fragile convention of fixing positions of "shared" fields among a family of records and wait for subtle bugs to emerge in live systems over time (well, it doesn't have to be this pessimistic, but...). :-) In the most general case, I would say you're using the wrong data structure for the job or your APIs need a slight redesign -- is the code where you need this publically available somewhere by any chance? BR, -- Jachym From alavrik@REDACTED Mon Aug 29 06:50:22 2011 From: alavrik@REDACTED (Anton Lavrik) Date: Sun, 28 Aug 2011 23:50:22 -0500 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax Message-ID: Hi list, It is my pleasure to announce Erlson - a proof of concept implementation of dynamic dictionary data type. It can be best described by examples: X = #{}, % create an empty dictionary D = #{foo = 1, bar = "abc", fum}, % associate fields 'foo' with 1, 'bar' with "abc" and 'fum' with 'true' 1 = D.foo, % access dictionary element D1 = D#{baz = #{fum = #{i = 0}}}, % add nested dictionaries to dictionary D 0 = D1.baz.fum.i, % access elements of the nested dictionary D2 = D1#{baz.fum.i = 100, baz.fum.j = "new nested value"}. % modify elements of the nested dictionary ... erlson:to_json(D2). % convert Erlson dictionary to JSON iolist() D = erlson:from_json(Json). % create Erlson dictionary from JSON iolist() ... D = erlson:from_proplist(L). % create Erlson dictionary from a proplist D = erlson:from_nested_proplist(L). % create nested Erlson dictionary from a nested proplist Erlson implementation is fully working and includes syntax support, runtime library and rebar packaging. The source code is available on GitHub: https://github.com/alavrik/erlson Internally, Erlson dictionaries are represented as orddicts, i.e. ordered lists of {atom(), any()} pairs. This way, they can be easily printed, manipulated and explored using conventional methods. For example, using the standard "orddict" and "proplists" libraries. It is also possible to define types for Erlson dictionaries and have them checked by Dialyzer. Erlson dictionary syntax can be used in both .erl modules and Erlang shell. Its implementation is based on a customized version of "erl_parse.yrl", which extends several grammar rules and overloads existing syntax elements used by Erlang records. Because there was no need to introduce new syntax elements, the implementation turned out to be quite simple. There is a catch, though. The current Erlson implementation is based on the original "erl_parse.yrl" from R14B03 release. This makes Erlson incompatible with other Erlang releases if they have a different version of "erl_parse.yrl". Although it would be easy to provide Erlson-enabled "erl_parse.yrl" version for every Erlang release, I was thinking about of a less hacky and more reliable solution. And this brings me to the question: Would it be useful if Erlson or some subset/superset of it became a part of the Erlang language? What do you think? Anton From rapsey@REDACTED Mon Aug 29 07:05:19 2011 From: rapsey@REDACTED (Rapsey) Date: Mon, 29 Aug 2011 07:05:19 +0200 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: References: Message-ID: Looks great. Glancing at the code, erlson:from_proplist if your proplist key is not an atom will not work, or will it? It would be nice if it automatically converted from lists and binaries. Sergej On Mon, Aug 29, 2011 at 6:50 AM, Anton Lavrik wrote: > Hi list, > > It is my pleasure to announce Erlson - a proof of concept > implementation of dynamic dictionary data type. It can be best > described by examples: > > X = #{}, % create an empty dictionary > > D = #{foo = 1, bar = "abc", fum}, % associate fields 'foo' with > 1, 'bar' with "abc" and 'fum' with 'true' > > 1 = D.foo, % access dictionary element > > D1 = D#{baz = #{fum = #{i = 0}}}, % add nested dictionaries to > dictionary D > > 0 = D1.baz.fum.i, % access elements of the nested dictionary > > D2 = D1#{baz.fum.i = 100, baz.fum.j = "new nested value"}. % > modify elements of the nested dictionary > ... > > erlson:to_json(D2). % convert Erlson dictionary to JSON iolist() > > D = erlson:from_json(Json). % create Erlson dictionary from JSON > iolist() > ... > > D = erlson:from_proplist(L). % create Erlson dictionary from a > proplist > > D = erlson:from_nested_proplist(L). % create nested Erlson > dictionary from a nested proplist > > > Erlson implementation is fully working and includes syntax support, > runtime library and rebar packaging. The source code is available on > GitHub: https://github.com/alavrik/erlson > > Internally, Erlson dictionaries are represented as orddicts, i.e. > ordered lists of {atom(), any()} pairs. This way, they can be easily > printed, manipulated and explored using conventional methods. For > example, using the standard "orddict" and "proplists" libraries. It is > also possible to define types for Erlson dictionaries and have them > checked by Dialyzer. > > Erlson dictionary syntax can be used in both .erl modules and Erlang > shell. Its implementation is based on a customized version of > "erl_parse.yrl", which extends several grammar rules and overloads > existing syntax elements used by Erlang records. Because there was no > need to introduce new syntax elements, the implementation turned out > to be quite simple. > > There is a catch, though. The current Erlson implementation is based > on the original "erl_parse.yrl" from R14B03 release. This makes Erlson > incompatible with other Erlang releases if they have a different > version of "erl_parse.yrl". > > Although it would be easy to provide Erlson-enabled "erl_parse.yrl" > version for every Erlang release, I was thinking about of a less hacky > and more reliable solution. And this brings me to the question: > > Would it be useful if Erlson or some subset/superset of it became a > part of the Erlang language? What do you think? > > > Anton > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alavrik@REDACTED Mon Aug 29 07:35:10 2011 From: alavrik@REDACTED (Anton Lavrik) Date: Mon, 29 Aug 2011 00:35:10 -0500 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: References: Message-ID: On Mon, Aug 29, 2011 at 12:05 AM, Rapsey wrote: > Looks great. Glancing at the code, erlson:from_proplist if your proplist key > is not an atom will not work, or will it? It would be nice if it > automatically converted from lists and binaries. Yes, erlson:from_proplist() accepts only atoms as keys. Otherwise, it would be inconsistent with property lists as defined by the "proplists" module. Here's a quote from the documentation: "Property lists are ordinary lists containing entries in the form of either tuples, whose first elements are keys used for lookup and insertion, or atoms, which work as shorthand for tuples {Atom, true} ." Anton From kostis@REDACTED Mon Aug 29 08:02:49 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 29 Aug 2011 09:02:49 +0300 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: References: Message-ID: <4E5B2B89.9010803@cs.ntua.gr> On 08/29/11 08:35, Anton Lavrik wrote: > On Mon, Aug 29, 2011 at 12:05 AM, Rapsey wrote: >> Looks great. Glancing at the code, erlson:from_proplist if your proplist key >> is not an atom will not work, or will it? It would be nice if it >> automatically converted from lists and binaries. > > Yes, erlson:from_proplist() accepts only atoms as keys. Otherwise, it > would be inconsistent with property lists as defined by the > "proplists" module. Here's a quote from the documentation: > > "Property lists are ordinary lists containing entries in the form of > either tuples, whose first elements are keys used for lookup and > insertion, or atoms, which work as shorthand for tuples {Atom, true}." I really do not see how the above quote implies that keys are atoms in proplists. My reading is that if tuples are used as properties, their keys can actually be any term. You may be right, but not because of this quote. Kostis From alavrik@REDACTED Mon Aug 29 08:08:48 2011 From: alavrik@REDACTED (Anton Lavrik) Date: Mon, 29 Aug 2011 01:08:48 -0500 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: <4E5B2B89.9010803@cs.ntua.gr> References: <4E5B2B89.9010803@cs.ntua.gr> Message-ID: On Mon, Aug 29, 2011 at 1:02 AM, Kostis Sagonas wrote: > On 08/29/11 08:35, Anton Lavrik wrote: >> >> On Mon, Aug 29, 2011 at 12:05 AM, Rapsey ?wrote: >>> >>> Looks great. Glancing at the code, erlson:from_proplist if your proplist >>> key >>> is not an atom will not work, or will it? It would be nice if it >>> automatically converted from lists and binaries. >> >> Yes, erlson:from_proplist() accepts only atoms as keys. Otherwise, it >> would be inconsistent with property lists as defined by the >> "proplists" module. Here's a quote from the documentation: >> >> "Property lists are ordinary lists containing entries in the form of >> either tuples, whose first elements are keys used for lookup and >> insertion, or atoms, which work as shorthand for tuples {Atom, true}." > > I really do not see how the above quote implies that keys are atoms in > proplists. ?My reading is that if tuples are used as properties, their keys > can actually be any term. ?You may be right, but not because of this quote. Kostis, thank you for the correction! I actually misread this sentence. Anton From ok@REDACTED Mon Aug 29 08:11:53 2011 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 29 Aug 2011 18:11:53 +1200 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: <4E5B2B89.9010803@cs.ntua.gr> References: <4E5B2B89.9010803@cs.ntua.gr> Message-ID: On 29/08/2011, at 6:02 PM, Kostis Sagonas wrote: > I really do not see how the above quote implies that keys are atoms in proplists. My reading is that if tuples are used as properties, their keys can actually be any term. You may be right, but not because of this quote. I read the proplist documentation the same way Kostis Sagonas does. Perhaps more importantly, we are told over and over again that Key = term() not that Key = atom(). And look at the code for lookup, edited for readability: lookup(Key, [P | Ps]) -> if is_atom(P), P =:= Key -> {Key, true} ; tuple_size(P) >= 1, element(1, P) =:= Key -> %% Note that Key does not have to be an atom in this case. P ; true -> lookup(Key, Ps) end; lookup(_Key, []) -> none. The comment there could hardly be more explicit. Of course, as explained at length in the proposals and on this mailing list, the ultra-lightweight "frames"/"proper structs" that Erlang needs ought to restrict *their* keys to being atoms. From ingham.k@REDACTED Mon Aug 29 08:33:33 2011 From: ingham.k@REDACTED (Igor Karymov) Date: Sun, 28 Aug 2011 23:33:33 -0700 (PDT) Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: References: Message-ID: <8ae239ce-5762-45c2-a0db-d6ac8086c0fc@gz5g2000vbb.googlegroups.com> If it can be used in pattern-mathing, it would be a very good alternative to the records. From ulf.wiger@REDACTED Mon Aug 29 09:16:52 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 29 Aug 2011 09:16:52 +0200 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: References: Message-ID: <1508A0A9-1E9F-4302-8698-B11E7854E922@erlang-solutions.com> On 29 Aug 2011, at 06:50, Anton Lavrik wrote: > Hi list, > > It is my pleasure to announce Erlson - a proof of concept > implementation of dynamic dictionary data type. This library was one of the entries in the Spawnfest competition, and received favourable mention from several of the judges (including me). I like the syntax - it feels like a good, minimal extension to the existing language. Personally, I would really like to see a special data type for this, as well as proper support for pattern matching and guards. I'm wary of yet another syntactic-sugar solution, overloading existing data types. This is in no way meant as criticism of the work so far - rather as a suggestion to grab the opportunity and implement full support for dynamic dictionaries (hashes, or whatever we want to call them). As always, there may be real, substantial objections that I have overlooked. If so, the good members of this list will surely present them. :) BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From desired.mta@REDACTED Mon Aug 29 09:30:01 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Mon, 29 Aug 2011 10:30:01 +0300 Subject: [erlang-questions] record with dynamic item name In-Reply-To: References: Message-ID: <20110829073001.GB2394@jakstys.lt> On Sun, Aug 28, 2011 at 12:46:10PM +0200, Roberto Ostinelli wrote: > Hi list, > > is there any way to retrieve a record item whose identifier is in a > variable? For instance: > > -record(state, { > type = 1 > }). > Ulf Wiger has done this job in parse_trans package: https://github.com/uwiger/parse_trans Documentation here: http://forum.trapexit.org/viewtopic.php?p=21790 Basically looks like this: -compile({parse_transform, exprecs}). -export_records(['r']). -record(r, {a, b}). imho(MyField) -> R = r{a = yadda, b = yabba}, '#get-'(MyField, R). main() -> imho(a). %% Returns yadda Motiejus From vincent.dephily@REDACTED Mon Aug 29 12:32:39 2011 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Mon, 29 Aug 2011 12:32:39 +0200 Subject: [erlang-questions] TCP receive buffer: erlang size VS kernel size In-Reply-To: <1478234.he0o8qDZS3@moltowork> References: <1478234.he0o8qDZS3@moltowork> Message-ID: <1574239.WNt6QSAJQI@moltowork> On Thursday 25 August 2011 15:08:01 Vincent de Phily wrote: > 1) The size seen in inet:getopts(S, [recbuf]) is the double of the size set > via gen_tcp:listen/2 (with some min/max values). > > * I assume that the discrepancy is between an in-VM buffer and the in-kernel > buffer ? > * Why is that 2x ratio necessary ? > * The docs never mention that there are two different buffers, and which > function reads/writes which buffer size. I only expected the kernel > buffer, and I definitely expected 'recbuf' to represent the same object for > inet:getopt as for gen_tcp:listen. After reading `man tcp` it appears that this 2x ratio is a kernel-side thing : "Note that TCP actually allocates twice the size of the buffer requested in the setsockopt(2) call, and so a succeeding getsockopt(2) call will not return the same size of buffer as requested in the setsockopt(2) call. TCP uses the extra space for administrative purposes and internal kernel structures, and the /proc file values reflect the larger sizes compared to the actual TCP windows." So no issue here, everything works "as expected" and there is no double- buffering in erlang. > 2) If I don't specify a recbuf in gen_tcp:listen/2, the kernel buffer (as > seen in inet:getopts) is 87 KB while the VM buffer (as seen in the size of > received packets) is 1460 B. > > * That's a much bigger difference ratio than previously seen. > * The default value of 1460 bytes sounds too small for anything but the most > ressource-constrained devices (my kernel agrees, but I admit having 4GB of > memory). I got the answer to that one reading erts/emulator/drivers/common/inet_drv.c : the default size of 1460 bytes for the read(2) buffer is hardcoded and is used if no buffer has been allocated yet and no read size had been specified by caller. I understand that the buffer needs to be preallocated and that erlang is sometimes used in restricted-memory environements, but as-is the behaviour is surprising and underperforming. Instead of receiving "whatever is is the socket right now", we get a fraction of it. Or if we recently requested a specified-big ammount, the next unspecified-size request will receive as much data as the previous request did. At the very least this value of 1460 bytes should be documented in the erlang docs ? Ideally, the read buffer size should be the same as the kernel-side buffer size. Can anybody share an opinion on this ? In the meantime I learned my lesson : never gen_tcp:listen/2 without specifying a recbuf :) -- Vincent de Phily From neumark.peter@REDACTED Mon Aug 29 13:40:26 2011 From: neumark.peter@REDACTED (Peter Neumark) Date: Mon, 29 Aug 2011 13:40:26 +0200 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <20110823232246.GB4977@alumni.caltech.edu> <2817450.46CDmI7xT0@moltowork> Message-ID: Unfortunately, I've just started writing the portion of the code which are would be interested in. I will push it to github as soon as it works. In the meantime you can check out what I have here: https://github.com/postlock/thrift-over-socketio Peter On Sun, Aug 28, 2011 at 1:18 AM, Wenqiang Song wrote: > You are right, the socket server is in lib/erl. The whole Erlang > library is tightly bound to synchronous model and hard to modify. Do > you mind sharing your code? > > Thanks > Andy > > On 8/28/11, Peter Neumark wrote: > > It is true that the erlang socket server which ships with thrift is > > synchronous (it's a gen_server). > > This code is not generated, however. It is possible to create your own > async > > server without touching the code generator. > > I know because I am writing a server for use with socket.io-erlang (which > is > > async). > > > > Peter > > > > On Sat, Aug 27, 2011 at 3:57 PM, Wenqiang Song > wrote: > > > >> I second this. But the code generator directly generates synchronous > >> socket server which is hard to embed into an existing asynchronous > >> server. > >> > >> Andy > >> > >> On 8/27/11, Peter Neumark wrote: > >> > I'm surprised Apache Thrift wasn't mentioned yet in this thread. > >> > Thrift is inspired by protobuf, so it sports s similar (although more > >> > modular) design. > >> > It supports a wide range a languages, several protocols and transports > >> (with > >> > the ability to add your own). > >> > The erlang lib + code generator code is actively developed and already > >> works > >> > quite well. > >> > > >> > Peter > >> > > >> > On Sat, Aug 27, 2011 at 5:43 AM, Jon Watte wrote: > >> > > >> >> 1) works as long as you control the sender or you have a well-defined > >> >> policy when multiple fields are present, but I'd otherwise consider > it > >> >> a > >> >> footgun. > >> >> > >> >> > >> >> > >> >> Your policy could easily be "there is only one field present at the > top > >> >> level" -- the next field encountered would simply be considered its > own > >> >> PDU. > >> >> > >> >> > >> >> I really find it odd that protobuf doesn't provide a CHOICE > construct. > >> >> > >> >> > >> >> The argument is that making everything optional gives you the most of > >> >> that, > >> >> with no additional overhead. What you want is the additional rule > that > >> >> there > >> >> is only ONE of those optionals, something which can be added at the > >> >> application layer if it's needed. > >> >> I'm not saying it wouldn't be nice if "switch" and PDU > >> >> naming/segmenting > >> >> were supported in protobuf -- that would be sweet! But we deal pretty > >> well > >> >> without it, and still find it better than the alternatives, > especially > >> for > >> >> cross-language projects. > >> >> > >> >> Sincerely, > >> >> > >> >> jw > >> >> > >> >> -- > >> >> Americans might object: there is no way we would sacrifice our living > >> >> standards for the benefit of people in the rest of the world. > >> >> Nevertheless, > >> >> whether we get there willingly or not, we shall soon have lower > >> >> consumption > >> >> rates, because our present rates are unsustainable. > >> >> > >> >> > >> >> > >> >> > >> >> On Thu, Aug 25, 2011 at 3:04 AM, Vincent de Phily < > >> >> vincent.dephily@REDACTED> wrote: > >> >> > >> >> > >> >>> On Wednesday 24 August 2011 15:15:57 Jon Watte wrote: > >> >>> > You solve that in one of two ways: > >> >>> > 1) Create a message that is the "PDU" that contains the union of > >> >>> everything > >> >>> > that could be sent, > >> >>> > or > >> >>> > 2) Create a wrapper that is just "type" followed by "binary data > >> >>> > that > >> >>> > is > >> >>> a > >> >>> > marshaled message." > >> >>> > > >> >>> > If you believe that the entirety of a protocol should be described > >> >>> > by > >> >>> that > >> >>> > protocol, then 1) is actually a fine solution. The > >> preserve-and-forward > >> >>> > semantic of unknown field ids means that forward compatibility is > >> still > >> >>> > possible. > >> >>> > > >> >>> > message MyProtocolPDU { > >> >>> > optional SomeMessage some_message = 1; > >> >>> > optional AnotherMessage other_message = 2; > >> >>> > ... > >> >>> > } > >> >>> > > >> >>> > So, encoding a MyProtocolPDU that contains a SomeMessage will end > up > >> >>> being > >> >>> > different than encoding a MyProtocolPDU that contains an > >> >>> > AnotherMessage. > >> >>> > At the high level, you have to know that there is only one member > in > >> >>> each > >> >>> > MyProtocolPDU; I find that to be quite acceptable. > >> >>> > >> >>> > >> >>> Yup, they're reasonable solutions (as long as you planed ahead). 2) > >> >>> may > >> >>> be > >> >>> ugly and innefficient, but it has been used as a pragmatic solution > in > >> >>> countless formats. 1) works as long as you control the sender or you > >> have > >> >>> a > >> >>> well-defined policy when multiple fields are present, but I'd > >> >>> otherwise > >> >>> consider it a footgun. I really find it odd that protobuf doesn't > >> provide > >> >>> a > >> >>> CHOICE construct. > >> >>> > >> >>> -- > >> >>> Vincent de Phily > >> >>> > >> >>> > >> >>> > >> >> > >> >> > >> >> _______________________________________________ > >> >> erlang-questions mailing list > >> >> erlang-questions@REDACTED > >> >> http://erlang.org/mailman/listinfo/erlang-questions > >> >> > >> >> > >> > > >> > >> -- > >> Sent from my mobile device > >> > >> --------------------------------------------------------------- > >> ???????????????????? > >> ???????????????????? > >> > > > > -- > Sent from my mobile device > > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From norton@REDACTED Mon Aug 29 14:08:04 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Mon, 29 Aug 2011 21:08:04 +0900 Subject: [erlang-questions] BERT vs protobuf in the erlang world In-Reply-To: References: <20110823232246.GB4977@alumni.caltech.edu> <2817450.46CDmI7xT0@moltowork> Message-ID: Not sure if this is repetitive or not but I posted a similar message earlier. http://erlang.2086793.n4.nabble.com/Erlang-with-Thrift-td3544946.html The UBF framework has a thrift format, erlang binary format, native ubf format, and other encoder/decoder implementations. The UBF framework supports synchronous and asynchronous models. If you need any help getting started, let me know. thanks, On 2011/08/29, at 20:40, Peter Neumark wrote: > Unfortunately, I've just started writing the portion of the code which are would be interested in. > I will push it to github as soon as it works. In the meantime you can check out what I have here: > https://github.com/postlock/thrift-over-socketio > > Peter > > On Sun, Aug 28, 2011 at 1:18 AM, Wenqiang Song wrote: > You are right, the socket server is in lib/erl. The whole Erlang > library is tightly bound to synchronous model and hard to modify. Do > you mind sharing your code? > > Thanks > Andy > > On 8/28/11, Peter Neumark wrote: > > It is true that the erlang socket server which ships with thrift is > > synchronous (it's a gen_server). > > This code is not generated, however. It is possible to create your own async > > server without touching the code generator. > > I know because I am writing a server for use with socket.io-erlang (which is > > async). > > > > Peter > > > > On Sat, Aug 27, 2011 at 3:57 PM, Wenqiang Song wrote: > > > >> I second this. But the code generator directly generates synchronous > >> socket server which is hard to embed into an existing asynchronous > >> server. > >> > >> Andy > >> > >> On 8/27/11, Peter Neumark wrote: > >> > I'm surprised Apache Thrift wasn't mentioned yet in this thread. > >> > Thrift is inspired by protobuf, so it sports s similar (although more > >> > modular) design. > >> > It supports a wide range a languages, several protocols and transports > >> (with > >> > the ability to add your own). > >> > The erlang lib + code generator code is actively developed and already > >> works > >> > quite well. > >> > > >> > Peter > >> > > >> > On Sat, Aug 27, 2011 at 5:43 AM, Jon Watte wrote: > >> > > >> >> 1) works as long as you control the sender or you have a well-defined > >> >> policy when multiple fields are present, but I'd otherwise consider it > >> >> a > >> >> footgun. > >> >> > >> >> > >> >> > >> >> Your policy could easily be "there is only one field present at the top > >> >> level" -- the next field encountered would simply be considered its own > >> >> PDU. > >> >> > >> >> > >> >> I really find it odd that protobuf doesn't provide a CHOICE construct. > >> >> > >> >> > >> >> The argument is that making everything optional gives you the most of > >> >> that, > >> >> with no additional overhead. What you want is the additional rule that > >> >> there > >> >> is only ONE of those optionals, something which can be added at the > >> >> application layer if it's needed. > >> >> I'm not saying it wouldn't be nice if "switch" and PDU > >> >> naming/segmenting > >> >> were supported in protobuf -- that would be sweet! But we deal pretty > >> well > >> >> without it, and still find it better than the alternatives, especially > >> for > >> >> cross-language projects. > >> >> > >> >> Sincerely, > >> >> > >> >> jw > >> >> > >> >> -- > >> >> Americans might object: there is no way we would sacrifice our living > >> >> standards for the benefit of people in the rest of the world. > >> >> Nevertheless, > >> >> whether we get there willingly or not, we shall soon have lower > >> >> consumption > >> >> rates, because our present rates are unsustainable. > >> >> > >> >> > >> >> > >> >> > >> >> On Thu, Aug 25, 2011 at 3:04 AM, Vincent de Phily < > >> >> vincent.dephily@REDACTED> wrote: > >> >> > >> >> > >> >>> On Wednesday 24 August 2011 15:15:57 Jon Watte wrote: > >> >>> > You solve that in one of two ways: > >> >>> > 1) Create a message that is the "PDU" that contains the union of > >> >>> everything > >> >>> > that could be sent, > >> >>> > or > >> >>> > 2) Create a wrapper that is just "type" followed by "binary data > >> >>> > that > >> >>> > is > >> >>> a > >> >>> > marshaled message." > >> >>> > > >> >>> > If you believe that the entirety of a protocol should be described > >> >>> > by > >> >>> that > >> >>> > protocol, then 1) is actually a fine solution. The > >> preserve-and-forward > >> >>> > semantic of unknown field ids means that forward compatibility is > >> still > >> >>> > possible. > >> >>> > > >> >>> > message MyProtocolPDU { > >> >>> > optional SomeMessage some_message = 1; > >> >>> > optional AnotherMessage other_message = 2; > >> >>> > ... > >> >>> > } > >> >>> > > >> >>> > So, encoding a MyProtocolPDU that contains a SomeMessage will end up > >> >>> being > >> >>> > different than encoding a MyProtocolPDU that contains an > >> >>> > AnotherMessage. > >> >>> > At the high level, you have to know that there is only one member in > >> >>> each > >> >>> > MyProtocolPDU; I find that to be quite acceptable. > >> >>> > >> >>> > >> >>> Yup, they're reasonable solutions (as long as you planed ahead). 2) > >> >>> may > >> >>> be > >> >>> ugly and innefficient, but it has been used as a pragmatic solution in > >> >>> countless formats. 1) works as long as you control the sender or you > >> have > >> >>> a > >> >>> well-defined policy when multiple fields are present, but I'd > >> >>> otherwise > >> >>> consider it a footgun. I really find it odd that protobuf doesn't > >> provide > >> >>> a > >> >>> CHOICE construct. > >> >>> > >> >>> -- > >> >>> Vincent de Phily > >> >>> > >> >>> > >> >>> > >> >> > >> >> > >> >> _______________________________________________ > >> >> erlang-questions mailing list > >> >> erlang-questions@REDACTED > >> >> http://erlang.org/mailman/listinfo/erlang-questions > >> >> > >> >> > >> > > >> > >> -- > >> Sent from my mobile device > >> > >> --------------------------------------------------------------- > >> ???????????????????? > >> ???????????????????? > >> > > > > -- > Sent from my mobile device > > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Joseph Wayne Norton norton@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.boudeville@REDACTED Mon Aug 29 16:58:14 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Mon, 29 Aug 2011 16:58:14 +0200 Subject: [erlang-questions] Dialyzer PLT & unknown Erlang/OTP functions Message-ID: Hi, I am using R14B03, and apparently cannot succeed in making good use of Dialyzer: I wanted first to generate a PLT for the full Erlang/OTP install and then use it against my codebase to perform more accurate checkings. So I generated a full PLT with a line like: (this is slightly edited as actually the building of the PLT is already integrated into the script we use to streamline our Erlang installations) ${prefix}/bin/dialyzer --build_plt -r ${prefix}/lib/erlang --output_plt ${prefix}/Erlang-R14B03.plt The operation seems to succeed and I end up some time later with a 4.4MB PLT file. However if I run it against a random module of us: ${prefix}/bin/dialyzer --plt ${prefix}/Erlang-R14B03.plt --output_plt ./common.plt src/utils/basic_utils.beam --build_plt It looks as if the previous PLT was not used: Creating PLT ./common.plt ... Unknown functions: calendar:datetime_to_gregorian_seconds/1 calendar:seconds_to_daystime/1 code:get_object_code/1 global:register_name/2 global:unregister_name/1 global:whereis_name/1 os:cmd/1 random:seed/0 random:seed/3 random:uniform/0 random:uniform/1 rpc:multicall/5 sets:from_list/1 sets:to_list/1 string:tokens/2 text_utils:string_to_integer/1 timer:sleep/1 done in 0m0.58s done (passed successfully) Most of these functions come from stdlib and al, so I would expect them to be in Erlang-R14B03.plt. And if I run, as an example (calendar being among the missing modules): ${prefix}/dialyzer --plt_info --plt ${prefix}/Erlang-R14B03.plt | grep calendar I have: "${prefix}/lib/erlang/lib/stdlib-1.17.4/ebin/calendar.beam", So apparently the two calendar functions should be known of this PLT. I must be making some mistake, but I tried different other approaches (ex: with '--apps erts kernel stdlib' instead of a wide '-r ') but with no luck. Are the installed BEAMs stripped in some way of debug_info/type specs, or anything like that ? Any help appreciated! Thanks in advance, Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Mon Aug 29 17:15:38 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 29 Aug 2011 17:15:38 +0200 Subject: [erlang-questions] xmlrpc library is alive and well Message-ID: <4E5BAD1A.2050601@gmail.com> This is just to announce that the XML-RPC library by Joakim Greben? is still being maintained (and has by now seen several years of hard use at Klarna). We have tracked down patches by others around the 'net and included them in a neat git repository in the hands of Tobbe T.: https://github.com/etnt/xmlrpc.git If you want to use xmlrpc without going through a full-fledged webserver, this can be useful. And if you have patches that we haven't already pulled in, please submit them. /Richard From john.hughes@REDACTED Mon Aug 29 16:51:11 2011 From: john.hughes@REDACTED (John Hughes) Date: Mon, 29 Aug 2011 16:51:11 +0200 Subject: [erlang-questions] QuickCheck lounge in Tokyo Message-ID: <66F111BF43164F90BDCD703215317F9D@quviqemachine> Quviq and Gemini Mobile will be organizing a QuickCheck Lounge in Tokyo on September 21st, with presentations of some of the current applications of QuickCheck. If you're in Tokyo at that time and interested in this kind of testing, please come! Details and registration here: http://geminitech.eventbrite.com/. Welcome! John Hughes From joelr1@REDACTED Mon Aug 29 17:27:16 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 29 Aug 2011 16:27:16 +0100 Subject: [erlang-questions] common test hiding errors? In-Reply-To: <4E5BAE8D.5010308@erix.ericsson.se> References: <9a786b50-d22b-4bce-893a-52e36263d1fa@localhost> <54EA06CD-79B2-4B4A-BF26-DE48D2C5FE00@gmail.com> <4E5BAE8D.5010308@erix.ericsson.se> Message-ID: <90E0FC89-3AA9-4733-93B1-85C71CFFCF0B@gmail.com> On Aug 29, 2011, at 4:21 PM, Peter Andersson wrote: > We will have a solution for this, probably in the R14B04 release > already. Is there a release date? > However, if you need a test case to terminate as a result of such an > error, you still need to define your own CT hook that adds an error > report handler (error_logger:add_report_handler/2) that tells the CT > hook to fail the test case as a result. Thanks Peter! -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Mon Aug 29 17:31:43 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 29 Aug 2011 16:31:43 +0100 Subject: [erlang-questions] distributed common test Message-ID: <59AADF44-733E-4E7D-AF27-D198EEE2CBBC@gmail.com> I'm trying to replace the distributed test harness I have in OpenPoker with Common Test. According to the CT docs, "The main idea is that CT based test programs connect to the target system(s) via standard O&M interfaces. CT provides implementations and wrappers of some of these O&M interfaces and will be extended with more interfaces later. There are a number of target independent interfaces supported in CT such as Generic Telnet, FTP etc. which can be specialized or used directly for controlling instruments, traffic generators etc." What are "standard O&M" interfaces? How do I add my own? Are there examples showing distributed CT in action? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Mon Aug 29 17:54:16 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 29 Aug 2011 16:54:16 +0100 Subject: [erlang-questions] common test: erlang rpc for blackbox testing Message-ID: <4FC238D6-848F-4AF2-9456-0BE704F63157@gmail.com> From the docs: "Common Test is also a very useful tool for white-box testing Erlang code since the test programs can call Erlang API functions directly. For black-box testing Erlang software, Erlang RPC as well as standard O&M interfaces can be used." How exactly do you use Erlang RPC for blackbox testing? How do you use the "standard O&M interfaces"? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From nox@REDACTED Mon Aug 29 18:25:20 2011 From: nox@REDACTED (Anthony Ramine) Date: Mon, 29 Aug 2011 18:25:20 +0200 Subject: [erlang-questions] xmlrpc library is alive and well In-Reply-To: <4E5BAD1A.2050601@gmail.com> References: <4E5BAD1A.2050601@gmail.com> Message-ID: <3FC09E31-D086-4FBD-AAFD-9D67B0FB531F@dev-extend.eu> Hi, This library uses xmerl so it should never be passed anything directly, as xmerl converts every latin1-compatible XML name to an atom, random names would lead to a big atom leak and the VM death. So please be aware that the XML documents given to it should be validated beforehand. Le 29 ao?t 2011 ? 17:15, Richard Carlsson a ?crit : > This is just to announce that the XML-RPC library by Joakim Greben? is still being maintained (and has by now seen several years of hard use at Klarna). We have tracked down patches by others around the 'net and included them in a neat git repository in the hands of Tobbe T.: > > https://github.com/etnt/xmlrpc.git > > If you want to use xmlrpc without going through a full-fledged webserver, this can be useful. And if you have patches that we haven't already pulled in, please submit them. > > /Richard > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Anthony Ramine / @nokusu Dev:Extend ? http://dev-extend.eu/ So as I pray, ?Unlimited Erlang Works? From ulf.wiger@REDACTED Mon Aug 29 19:00:06 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 29 Aug 2011 19:00:06 +0200 Subject: [erlang-questions] distributed common test In-Reply-To: <59AADF44-733E-4E7D-AF27-D198EEE2CBBC@gmail.com> References: <59AADF44-733E-4E7D-AF27-D198EEE2CBBC@gmail.com> Message-ID: On 29 Aug 2011, at 17:31, Joel Reymont wrote: > What are "standard O&M" interfaces? O&M = Operation and Maintenance. This is a term used widely in e.g. Telecoms to describe systems interfaces for configuration, alarms, monitoring, billing, etc. You can read this as CT being primarily designed for black-box testing where protocols like CORBA, SNMP, HTTP, FTP, NetConf, Telnet, ssh, etc. are used to configure and inspect the systems under test. CT has some helper libraries for some of these, but the pattern really is pretty simple. > How do I add my own? > > Are there examples showing distributed CT in action? Perhaps this presentation can be of some help: http://www.erlang-factory.com/upload/presentations/204/EUC2009-CommonTestPresentation.pdf BR, Ulf > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From aronisstav@REDACTED Mon Aug 29 19:43:13 2011 From: aronisstav@REDACTED (Stavros Aronis) Date: Mon, 29 Aug 2011 10:43:13 -0700 (PDT) Subject: [erlang-questions] Dialyzer PLT & unknown Erlang/OTP functions In-Reply-To: References: Message-ID: <27083565.3849.1314639793966.JavaMail.geo-discussion-forums@yqap15> This command: ${prefix}/bin/dialyzer --plt ${prefix}/Erlang-R14B03.plt --output_plt ./common.plt src/utils/basic_utils.beam --build_plt is wrong. You should not pass the --build_plt option again, as it will make Dialyzer generate everything from scratch (ignoring anything within the supplied PLT). To add some of your own code to the PLT use the --add_to_plt option instead, after you have the initial with the OTP modules. In this way Dialyzer will use the supplied PLT to generate a new one with all the old and new information. Stavros -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Mon Aug 29 20:44:43 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 29 Aug 2011 19:44:43 +0100 Subject: [erlang-questions] distributed common test In-Reply-To: References: <59AADF44-733E-4E7D-AF27-D198EEE2CBBC@gmail.com> Message-ID: <351A3C5A-2539-4918-8F52-334FFA11EAB1@gmail.com> On Aug 29, 2011, at 6:00 PM, Ulf Wiger wrote: > Perhaps this presentation can be of some help: > > http://www.erlang-factory.com/upload/presentations/204/EUC2009-CommonTestPresentation.pdf This is just the ticket. Thank you Uffe! One question? Page 34 (Using Common Test for Large Scale Testing ) mentions that test nodes can be added and removed dynamically. How is this done? ct_master API does not seem to include the ability to add nodes. Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From yrashk@REDACTED Tue Aug 30 01:01:10 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Mon, 29 Aug 2011 16:01:10 -0700 Subject: [erlang-questions] reltool puts all releases into releases/BootVsn Message-ID: Hi, I was wondering what was the motivation for putting all the releases specified in reltool.config into a single directory releases/, where BootVsn is a version of a release specified as boot_rel. I have a situation where I have multiple releases specified in a single reltool.config, and some of them have different versions (and can also be upgraded independently). I find it odd having releases with different versions sitting in a directory for a version of the boot_rel release. Shouldn't each version get its own directory? I can see how this change can be done in reltool, but I'd rather hold it off until I hear more feedback on this. The source code that does that is available here: https://github.com/erlang/otp/blob/dev/lib/reltool/src/reltool_target.erl#L774 Any insights, thoughts? Yurii. From juanjo@REDACTED Tue Aug 30 01:26:16 2011 From: juanjo@REDACTED (Juan Jose Comellas) Date: Mon, 29 Aug 2011 20:26:16 -0300 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: References: Message-ID: May I suggest switching the decoded JSON format to that used by ejson ( https://github.com/benoitc/ejson) and jiffy (https://github.com/davisp/jiffy) instead of the format used by mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem to be moving to the format I'm suggesting. Juanjo On Mon, Aug 29, 2011 at 1:50 AM, Anton Lavrik wrote: > Hi list, > > It is my pleasure to announce Erlson - a proof of concept > implementation of dynamic dictionary data type. It can be best > described by examples: > > X = #{}, % create an empty dictionary > > D = #{foo = 1, bar = "abc", fum}, % associate fields 'foo' with > 1, 'bar' with "abc" and 'fum' with 'true' > > 1 = D.foo, % access dictionary element > > D1 = D#{baz = #{fum = #{i = 0}}}, % add nested dictionaries to > dictionary D > > 0 = D1.baz.fum.i, % access elements of the nested dictionary > > D2 = D1#{baz.fum.i = 100, baz.fum.j = "new nested value"}. % > modify elements of the nested dictionary > ... > > erlson:to_json(D2). % convert Erlson dictionary to JSON iolist() > > D = erlson:from_json(Json). % create Erlson dictionary from JSON > iolist() > ... > > D = erlson:from_proplist(L). % create Erlson dictionary from a > proplist > > D = erlson:from_nested_proplist(L). % create nested Erlson > dictionary from a nested proplist > > > Erlson implementation is fully working and includes syntax support, > runtime library and rebar packaging. The source code is available on > GitHub: https://github.com/alavrik/erlson > > Internally, Erlson dictionaries are represented as orddicts, i.e. > ordered lists of {atom(), any()} pairs. This way, they can be easily > printed, manipulated and explored using conventional methods. For > example, using the standard "orddict" and "proplists" libraries. It is > also possible to define types for Erlson dictionaries and have them > checked by Dialyzer. > > Erlson dictionary syntax can be used in both .erl modules and Erlang > shell. Its implementation is based on a customized version of > "erl_parse.yrl", which extends several grammar rules and overloads > existing syntax elements used by Erlang records. Because there was no > need to introduce new syntax elements, the implementation turned out > to be quite simple. > > There is a catch, though. The current Erlson implementation is based > on the original "erl_parse.yrl" from R14B03 release. This makes Erlson > incompatible with other Erlang releases if they have a different > version of "erl_parse.yrl". > > Although it would be easy to provide Erlson-enabled "erl_parse.yrl" > version for every Erlang release, I was thinking about of a less hacky > and more reliable solution. And this brings me to the question: > > Would it be useful if Erlson or some subset/superset of it became a > part of the Erlang language? What do you think? > > > Anton > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jack@REDACTED Tue Aug 30 01:36:33 2011 From: jack@REDACTED (Jack Moffitt) Date: Mon, 29 Aug 2011 17:36:33 -0600 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: References: Message-ID: > May I suggest switching the decoded JSON format to that used by ejson > (https://github.com/benoitc/ejson) and jiffy > (https://github.com/davisp/jiffy) instead of the format used by > mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem > to be moving to the format I'm suggesting. A tuple of an array of tuples seems less than elegant Why not just a tuple of tuples? Is scanning through a tuple significantly slower than scanning an array? As far as formats go, this is a new one to me. Most projects I interact with seem to use either mochijson or mochijson2. I'd love for there to be a canonical one though. jack. From essen@REDACTED Tue Aug 30 01:53:23 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 30 Aug 2011 01:53:23 +0200 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: References: Message-ID: <4E5C2673.80805@dev-extend.eu> On 08/30/2011 01:36 AM, Jack Moffitt wrote: >> May I suggest switching the decoded JSON format to that used by ejson >> (https://github.com/benoitc/ejson) and jiffy >> (https://github.com/davisp/jiffy) instead of the format used by >> mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem >> to be moving to the format I'm suggesting. > > A tuple of an array of tuples seems less than elegant Why not just a > tuple of tuples? Is scanning through a tuple significantly slower than > scanning an array? This should explain the whys: http://www.erlang.org/eeps/eep-0018.html > As far as formats go, this is a new one to me. Most projects I > interact with seem to use either mochijson or mochijson2. I'd love for > there to be a canonical one though. All recent JSON projects follow that EEP as far as I know. If OTP gets JSON support then your code will be working with the OTP implementation. It's also less annoying to write than {struct, ...} because it's trying (and not failing on me yet) to be a 1:1 match with Erlang types or when not possible to give expected results. Personally I prefer jsx (https://github.com/talentdeficit/jsx) simply because it's not a NIF and thus doesn't impact the scheduler. But your needs and requirements may vary. -- Lo?c Hoguin Dev:Extend From bob@REDACTED Tue Aug 30 02:12:35 2011 From: bob@REDACTED (Bob Ippolito) Date: Mon, 29 Aug 2011 17:12:35 -0700 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: <4E5C2673.80805@dev-extend.eu> References: <4E5C2673.80805@dev-extend.eu> Message-ID: On Mon, Aug 29, 2011 at 4:53 PM, Lo?c Hoguin wrote: > On 08/30/2011 01:36 AM, Jack Moffitt wrote: >>> May I suggest switching the decoded JSON format to that used by ejson >>> (https://github.com/benoitc/ejson) and jiffy >>> (https://github.com/davisp/jiffy) instead of the format used by >>> mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem >>> to be moving to the format I'm suggesting. >> >> A tuple of an array of tuples seems less than elegant ?Why not just a >> tuple of tuples? Is scanning through a tuple significantly slower than >> scanning an array? > > This should explain the whys: > > http://www.erlang.org/eeps/eep-0018.html > >> As far as formats go, this is a new one to me. Most projects I >> interact with seem to use either mochijson or mochijson2. I'd love for >> there to be a canonical one though. > > All recent JSON projects follow that EEP as far as I know. If OTP gets > JSON support then your code will be working with the OTP implementation. > It's also less annoying to write than {struct, ...} because it's trying > (and not failing on me yet) to be a 1:1 match with Erlang types or when > not possible to give expected results. > > Personally I prefer jsx (https://github.com/talentdeficit/jsx) simply > because it's not a NIF and thus doesn't impact the scheduler. But your > needs and requirements may vary. It would be basically trivial to add {proplist()} support to mochijson2 as an addition to {struct, proplist()}. There's not much of a difference really, it's still wrapping a tuple around it but just without a tag. I'll have a look at that. -bob From james@REDACTED Tue Aug 30 03:05:18 2011 From: james@REDACTED (James Aimonetti) Date: Mon, 29 Aug 2011 18:05:18 -0700 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: References: <4E5C2673.80805@dev-extend.eu> Message-ID: <4E5C374E.3090604@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 08/29/2011 05:12 PM, Bob Ippolito wrote: > On Mon, Aug 29, 2011 at 4:53 PM, Lo?c Hoguin wrote: >> On 08/30/2011 01:36 AM, Jack Moffitt wrote: >>>> May I suggest switching the decoded JSON format to that used by ejson >>>> (https://github.com/benoitc/ejson) and jiffy >>>> (https://github.com/davisp/jiffy) instead of the format used by >>>> mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem >>>> to be moving to the format I'm suggesting. >>> >>> A tuple of an array of tuples seems less than elegant Why not just a >>> tuple of tuples? Is scanning through a tuple significantly slower than >>> scanning an array? >> >> This should explain the whys: >> >> http://www.erlang.org/eeps/eep-0018.html >> >>> As far as formats go, this is a new one to me. Most projects I >>> interact with seem to use either mochijson or mochijson2. I'd love for >>> there to be a canonical one though. >> >> All recent JSON projects follow that EEP as far as I know. If OTP gets >> JSON support then your code will be working with the OTP implementation. >> It's also less annoying to write than {struct, ...} because it's trying >> (and not failing on me yet) to be a 1:1 match with Erlang types or when >> not possible to give expected results. >> >> Personally I prefer jsx (https://github.com/talentdeficit/jsx) simply >> because it's not a NIF and thus doesn't impact the scheduler. But your >> needs and requirements may vary. > > It would be basically trivial to add {proplist()} support to > mochijson2 as an addition to {struct, proplist()}. There's not much of > a difference really, it's still wrapping a tuple around it but just > without a tag. I'll have a look at that. > > -bob > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions An old version of Couchbeam accomplished that conversion thusly, if that helps: json_encode(V) -> Handler = fun({L}) when is_list(L) -> {struct,L}; (Bad) -> exit({json_encode, {bad_term, Bad}}) end, (mochijson2:encoder([{handler, Handler}]))(V). json_decode(V) -> try (mochijson2:decoder([{object_hook, fun({struct,L}) -> {L} end}]))(V) catch _Type:_Error -> throw({invalid_json,V}) end. - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOXDdOAAoJENc77s1OYoGg4D8H/2j63QYpyFRymqz62rhmlquf 5jCf9NoMpzYfSsJuC9fJhcvOtbpdr/Cx+62Y3GRDi0UMntdzvS0IwNmJvqH9uCer WQ7gpPY+0tGxGfxDlMTnoBvgcfQM6bbjjMPFqN+dgDinVliAgwRhZpaJgd0hgvOm g0Xb6ngK+pvueJV0cfTBsTFmMgI89rxJMb4yUFc0V54XctQTsAZhaxmP429jt0mP e0YGQ0ZvAQaHM+CjRYuUlKHUrCUNmkXmgmjk54L9S0Ic5PZLpJMdpMWq6FWsMrIt D7klF7x8E2v3w0gCHsPRS7n/XPV0Sp0sEuMz75Kifs5nsAuNUmLX5vG77sXuZ5s= =QFs/ -----END PGP SIGNATURE----- From bob@REDACTED Tue Aug 30 03:19:34 2011 From: bob@REDACTED (Bob Ippolito) Date: Mon, 29 Aug 2011 18:19:34 -0700 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: <4E5C374E.3090604@2600hz.com> References: <4E5C2673.80805@dev-extend.eu> <4E5C374E.3090604@2600hz.com> Message-ID: On Mon, Aug 29, 2011 at 6:05 PM, James Aimonetti wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > On 08/29/2011 05:12 PM, Bob Ippolito wrote: >> On Mon, Aug 29, 2011 at 4:53 PM, Lo?c Hoguin wrote: >>> On 08/30/2011 01:36 AM, Jack Moffitt wrote: >>>>> May I suggest switching the decoded JSON format to that used by ejson >>>>> (https://github.com/benoitc/ejson) and jiffy >>>>> (https://github.com/davisp/jiffy) instead of the format used by >>>>> mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem >>>>> to be moving to the format I'm suggesting. >>>> >>>> A tuple of an array of tuples seems less than elegant ?Why not just a >>>> tuple of tuples? Is scanning through a tuple significantly slower than >>>> scanning an array? >>> >>> This should explain the whys: >>> >>> http://www.erlang.org/eeps/eep-0018.html >>> >>>> As far as formats go, this is a new one to me. Most projects I >>>> interact with seem to use either mochijson or mochijson2. I'd love for >>>> there to be a canonical one though. >>> >>> All recent JSON projects follow that EEP as far as I know. If OTP gets >>> JSON support then your code will be working with the OTP implementation. >>> It's also less annoying to write than {struct, ...} because it's trying >>> (and not failing on me yet) to be a 1:1 match with Erlang types or when >>> not possible to give expected results. >>> >>> Personally I prefer jsx (https://github.com/talentdeficit/jsx) simply >>> because it's not a NIF and thus doesn't impact the scheduler. But your >>> needs and requirements may vary. >> >> It would be basically trivial to add {proplist()} support to >> mochijson2 as an addition to {struct, proplist()}. There's not much of >> a difference really, it's still wrapping a tuple around it but just >> without a tag. I'll have a look at that. >> > An old version of Couchbeam accomplished that conversion thusly, if that > helps: > [snip code] Yeah, it was never hard to do it from the outside using object_hook on decode and handler on encode. It's now natively supported in mochiweb v2.1.0 without having to write any funs. https://github.com/mochi/mochiweb JSON = <<"{\"k\": \"v\"}">>. [{<<"k">>, <<"v">>}] =:= mochijson2:decode(JSON, [{format, proplist}]). {[{<<"k">>, <<"v">>}]} =:= mochijson2:decode(JSON, [{format, eep18}]). %% the default {struct, [{<<"k">>, <<"v">>}]} =:= mochijson2:decode(JSON, [{format, struct}]). These will also round-trip with the encoder without any surprises or additional options. I've also added support for the {} and {P} format to kvc. kvc is basically a simple query language for a host of proplist-like data structures. https://github.com/etrepum/kvc -bob From alavrik@REDACTED Tue Aug 30 04:13:22 2011 From: alavrik@REDACTED (Anton Lavrik) Date: Mon, 29 Aug 2011 21:13:22 -0500 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: References: Message-ID: On Mon, Aug 29, 2011 at 6:26 PM, Juan Jose Comellas wrote: > May I suggest switching the decoded JSON format to that used by ejson > (https://github.com/benoitc/ejson) and jiffy > (https://github.com/davisp/jiffy) instead of the format used by > mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem > to be moving to the format I'm suggesting. By the way, there's another JSON library: https://github.com/talentdeficit/jsx Somebody has asked me if I plan to add support for it. This is actually a very easy question. It takes less that 100 lines of code to add support for any of the above libraries/decoded formats. If somebody has a need for it and care to write the conversion code, I'll be happy to merge it into Erlson and extend the existing API to allow choosing a specific JSON codec. For those who might be interested, adding XML support would not be substantially harder. Mochijson2 will probably remain the default JSON library in Erlson. It is very stable, while statuses of the other libraries are not clear to me. At least I couldn't figure this out by looking at their project pages. Anton From olivier.boudeville@REDACTED Tue Aug 30 11:24:45 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Tue, 30 Aug 2011 11:24:45 +0200 Subject: [erlang-questions] Dialyzer PLT & unknown Erlang/OTP functions In-Reply-To: <27083565.3849.1314639793966.JavaMail.geo-discussion-forums@yqap15> Message-ID: Hello Stavros, Thanks again for your answer; indeed --add_to_plt was the way to go, it then worked like a charm. On a side note, making the PLT for the full Erlang distro exhibits the following remarks: """ eunit_test.erl:242: Call to missing or unexported function eunit_test:nonexisting_function/0 xrc.erl:124: Call to missing or unexported function wxXmlResource:reload/2 Unknown functions: wxWindows:'Destroy'/1 wx_core:quit/0 Unknown types: gen_sctp:assoc_id/0 gen_sctp:sctp_socket/0 """ Thanks again, Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 aronisstav@REDACTED Envoy? par : erlang-questions-bounces@REDACTED 29/08/2011 19:43 Veuillez r?pondre ? erlang-programming@REDACTED A erlang-programming@REDACTED cc erlang-questions@REDACTED Objet Re: [erlang-questions] Dialyzer PLT & unknown Erlang/OTP functions This command: ${prefix}/bin/dialyzer --plt ${prefix}/Erlang-R14B03.plt --output_plt ./common.plt src/utils/basic_utils.beam --build_plt is wrong. You should not pass the --build_plt option again, as it will make Dialyzer generate everything from scratch (ignoring anything within the supplied PLT). To add some of your own code to the PLT use the --add_to_plt option instead, after you have the initial with the OTP modules. In this way Dialyzer will use the supplied PLT to generate a new one with all the old and new information. Stavros_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Tue Aug 30 11:27:11 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Tue, 30 Aug 2011 11:27:11 +0200 Subject: [erlang-questions] reltool puts all releases into releases/BootVsn In-Reply-To: References: Message-ID: Yurii, I agree - this seems strange. Unfortunately I don't know the background for this. I can only guess that it was originally done this way in order to have the default start_clean.rel and start_sasl.rel in the same directory - these files do both have the same version - and that other use of the feature of specifying multiple rel files was not really considered. I can not see that it would be a big problem to correct this. One could of course argue that it will be backwards incompatible, but one could also see it as a bug... Anyway - a patch would be much appreciated. Thanks /siri@REDACTED 2011/8/30 Yurii Rashkovskii > Hi, > > I was wondering what was the motivation for putting all the releases > specified in reltool.config into a single directory > releases/, where BootVsn is a version of a release specified > as boot_rel. I have a situation where I have multiple releases > specified in a single reltool.config, and some of them have different > versions (and can also be upgraded independently). > > I find it odd having releases with different versions sitting in a > directory for a version of the boot_rel release. Shouldn't each > version get its own directory? I can see how this change can be done > in reltool, but I'd rather hold it off until I hear more feedback on > this. > > The source code that does that is available here: > > https://github.com/erlang/otp/blob/dev/lib/reltool/src/reltool_target.erl#L774 > > Any insights, thoughts? > > Yurii. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yrashk@REDACTED Tue Aug 30 11:33:10 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Tue, 30 Aug 2011 02:33:10 -0700 Subject: [erlang-questions] reltool puts all releases into releases/BootVsn In-Reply-To: References: Message-ID: Siri, Thanks for your feedback. I will prepare the patch shortly ? it will have the effect of the workaround I currently use: https://gist.github.com/58366bd0a757c8c9c2b1 ? hopefully I got it right. On Tue, Aug 30, 2011 at 2:27 AM, Siri Hansen wrote: > Yurii, > I agree - this seems strange. Unfortunately I don't know the background for > this. I can only guess that it was originally done this way in order to have > the default start_clean.rel and start_sasl.rel in the same directory - these > files do both have the same version - and that other use of the feature of > specifying multiple rel files was not really considered. > I can not see that it would be a big problem to correct this. One could of > course argue that it will be backwards incompatible, but one could also see > it as a bug... Anyway - a patch would be much appreciated. > Thanks > /siri@REDACTED > > 2011/8/30 Yurii Rashkovskii >> >> Hi, >> >> I was wondering what was the motivation for putting all the releases >> specified in reltool.config into a single directory >> releases/, where BootVsn is a version of a release specified >> as boot_rel. I have a situation where I have multiple releases >> specified in a single reltool.config, and some of them have different >> versions (and can also be upgraded independently). >> >> I find it odd having releases with different versions sitting in a >> directory for a version of the boot_rel release. Shouldn't each >> version get its own directory? I can see how this change can be done >> in reltool, but I'd rather hold it off until I hear more feedback on >> this. >> >> The source code that does that is available here: >> >> https://github.com/erlang/otp/blob/dev/lib/reltool/src/reltool_target.erl#L774 >> >> Any insights, thoughts? >> >> Yurii. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From erlangsiri@REDACTED Tue Aug 30 11:45:15 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Tue, 30 Aug 2011 11:45:15 +0200 Subject: [erlang-questions] reltool puts all releases into releases/BootVsn In-Reply-To: References: Message-ID: Great! Thanks! /siri 2011/8/30 Yurii Rashkovskii > Siri, > > Thanks for your feedback. I will prepare the patch shortly ? it will > have the effect of the workaround > I currently use: https://gist.github.com/58366bd0a757c8c9c2b1 ? > hopefully I got it right. > > On Tue, Aug 30, 2011 at 2:27 AM, Siri Hansen wrote: > > Yurii, > > I agree - this seems strange. Unfortunately I don't know the background > for > > this. I can only guess that it was originally done this way in order to > have > > the default start_clean.rel and start_sasl.rel in the same directory - > these > > files do both have the same version - and that other use of the feature > of > > specifying multiple rel files was not really considered. > > I can not see that it would be a big problem to correct this. One could > of > > course argue that it will be backwards incompatible, but one could also > see > > it as a bug... Anyway - a patch would be much appreciated. > > Thanks > > /siri@REDACTED > > > > 2011/8/30 Yurii Rashkovskii > >> > >> Hi, > >> > >> I was wondering what was the motivation for putting all the releases > >> specified in reltool.config into a single directory > >> releases/, where BootVsn is a version of a release specified > >> as boot_rel. I have a situation where I have multiple releases > >> specified in a single reltool.config, and some of them have different > >> versions (and can also be upgraded independently). > >> > >> I find it odd having releases with different versions sitting in a > >> directory for a version of the boot_rel release. Shouldn't each > >> version get its own directory? I can see how this change can be done > >> in reltool, but I'd rather hold it off until I hear more feedback on > >> this. > >> > >> The source code that does that is available here: > >> > >> > https://github.com/erlang/otp/blob/dev/lib/reltool/src/reltool_target.erl#L774 > >> > >> Any insights, thoughts? > >> > >> Yurii. > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simsek.burak@REDACTED Tue Aug 30 14:03:26 2011 From: simsek.burak@REDACTED (Simsek Burak) Date: Tue, 30 Aug 2011 14:03:26 +0200 Subject: [erlang-questions] Couldnt not get abstract code for:... Message-ID: Hello, When i try to analyze the folder that .beam files in, with dialyzer, it gives "could not get abstract code for /../....../.. Recompile with debug_info or analyze starting from source code" error. What should i do? Regards, Burak -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Tue Aug 30 14:21:01 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 30 Aug 2011 15:21:01 +0300 Subject: [erlang-questions] Dialyzer PLT & unknown Erlang/OTP functions In-Reply-To: References: Message-ID: <4E5CD5AD.7070003@cs.ntua.gr> On 08/30/11 12:24, Olivier BOUDEVILLE wrote: > > Hello Stavros, > > Thanks again for your answer; indeed --add_to_plt was the way to go, it > then worked like a charm. Hi Olivier, Let me add to the info that Stavros sent that --add_to_plt is a very convenient way to avoid building the PLT for the full Erlang disto in one go. One could start with a PLT for e.g. erts, kernel and stdlib and then incrementally add to it when dialyzer reports unknown functions on the code base they want to analyze. Also, note that adding parts of your code base to the PLT is typically something to do only when these parts are considered more or less stable (because if there are any changes to them, the up-to-date check will fail and the PLT will be re-built incrementally) and a "base" upon which the rest of the code is build. Adding them to the PLT just allows one to avoid having to specify them as part of the files that will be analyzed and for which discrepancies will be detected. Cheers, Kostis PS. As to the following, it is something for the OTP folks to look at: > On a side note, making the PLT for the full Erlang distro exhibits the > following remarks: > """ > eunit_test.erl:242: Call to missing or unexported function > eunit_test:nonexisting_function/0 > xrc.erl:124: Call to missing or unexported function wxXmlResource:reload/2 > Unknown functions: > wxWindows:'Destroy'/1 > wx_core:quit/0 > Unknown types: > gen_sctp:assoc_id/0 > gen_sctp:sctp_socket/0 From norton@REDACTED Tue Aug 30 14:22:40 2011 From: norton@REDACTED (Joseph Norton) Date: Tue, 30 Aug 2011 21:22:40 +0900 Subject: [erlang-questions] Couldnt not get abstract code for:... In-Reply-To: References: Message-ID: <1A0E41D2-42A2-4A35-9772-560EAD0D3C05@lovely.email.ne.jp> See here ... http://learnyousomeerlang.com/modules There are a whole lot of compilation flags existing to get more control over how a module is compiled. You can get a list of all of them in the Erlang documentation. The most common flags are: -debug_info Erlang tools such as debuggers, code coverage and static analysis tools will use the debug information of a module in order to do their work. Joseph Norton norton@REDACTED On Aug 30, 2011, at 9:03 PM, Simsek Burak wrote: > Hello, > > When i try to analyze the folder that .beam files in, with dialyzer, it gives "could not get abstract code for /../....../.. Recompile with debug_info or analyze starting from source code" error. > What should i do? > > Regards, > Burak > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Tue Aug 30 14:27:07 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 30 Aug 2011 15:27:07 +0300 Subject: [erlang-questions] Couldnt not get abstract code for:... In-Reply-To: References: Message-ID: <4E5CD71B.8040107@cs.ntua.gr> On 08/30/11 15:03, Simsek Burak wrote: > Hello, > When i try to analyze the folder that .beam files in, with dialyzer, it > gives "could not get abstract code for /../....../.. Recompile with > debug_info or analyze starting from source code" error. > What should i do? Hmm... let's see your options: 1. Re-create these .beam files by using the BEAM compiler option debug_info. (You specify it as +debug_info to erlc.) 2. Analyze the .erl files that correspond to these .beam files. Option #1 is typically easier (because you do not need to specify possible define values or include paths) but it's your choice really. Kostis From simsek.burak@REDACTED Tue Aug 30 14:51:24 2011 From: simsek.burak@REDACTED (Simsek Burak) Date: Tue, 30 Aug 2011 14:51:24 +0200 Subject: [erlang-questions] (no subject) Message-ID: Thank you for solutions :) Regards, Burak -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.boudeville@REDACTED Tue Aug 30 15:32:45 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Tue, 30 Aug 2011 15:32:45 +0200 Subject: [erlang-questions] Dialyzer PLT & unknown Erlang/OTP functions In-Reply-To: <4E5CD5AD.7070003@cs.ntua.gr> Message-ID: Hi Kostis, As for me I tend to prefer building everything at once, on a purely brute-force automated way, so that the same exact test-proofed process always applies (from one version to another, from a developer to another). What we are doing currently is generating "incremental" PLT for each level of our software stack (starting from Erlang of course). It seems to work satisfactorily. As our codebase, despite being small, had already too many files to be updated with appropriate type specs in one shot, we used an escript which instrumented first the existing source code with specs *as they were deduced by Dialyzer*. Then only, as time permits, we will fine-tune, file by file, the specs according to the developer's original intent rather than according to what Dialyzer inferred (which allows to have some insights and save some time). Later we will have to see how type spec information can be injected in BEAMs whose generation involved functions being defined thanks to a parse-transform. Not for today though! Thanks, Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 kostis@REDACTED Envoy? par : erlang-questions-bounces@REDACTED 30/08/2011 14:21 A erlang-questions@REDACTED cc Objet Re: [erlang-questions] Dialyzer PLT & unknown Erlang/OTP functions On 08/30/11 12:24, Olivier BOUDEVILLE wrote: > > Hello Stavros, > > Thanks again for your answer; indeed --add_to_plt was the way to go, it > then worked like a charm. Hi Olivier, Let me add to the info that Stavros sent that --add_to_plt is a very convenient way to avoid building the PLT for the full Erlang disto in one go. One could start with a PLT for e.g. erts, kernel and stdlib and then incrementally add to it when dialyzer reports unknown functions on the code base they want to analyze. Also, note that adding parts of your code base to the PLT is typically something to do only when these parts are considered more or less stable (because if there are any changes to them, the up-to-date check will fail and the PLT will be re-built incrementally) and a "base" upon which the rest of the code is build. Adding them to the PLT just allows one to avoid having to specify them as part of the files that will be analyzed and for which discrepancies will be detected. Cheers, Kostis PS. As to the following, it is something for the OTP folks to look at: > On a side note, making the PLT for the full Erlang distro exhibits the > following remarks: > """ > eunit_test.erl:242: Call to missing or unexported function > eunit_test:nonexisting_function/0 > xrc.erl:124: Call to missing or unexported function wxXmlResource:reload/2 > Unknown functions: > wxWindows:'Destroy'/1 > wx_core:quit/0 > Unknown types: > gen_sctp:assoc_id/0 > gen_sctp:sctp_socket/0 _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From agarwal1975@REDACTED Tue Aug 30 16:12:31 2011 From: agarwal1975@REDACTED (Ashish Agarwal) Date: Tue, 30 Aug 2011 10:12:31 -0400 Subject: [erlang-questions] [CUFP 2011] Birds of a Feather sessions In-Reply-To: References: Message-ID: ========================================================= Birds of a Feather sessions (BOFs) Commercial Users of Functional Programming Workshop (CUFP 2011) http://cufp.org/bofs-2011 Tokyo, Japan, September 22 - 23 ========================================================= Following last year's success, CUFP 2011 will again include Birds of a Feather sessions. BoFs provide a place for our community to gather informally at CUFP and reach consensus on matters of importance. Any CUFP attendee can propose a BoF session, and grab one of the spare rooms to hold a discussion. Attendance in the evening is open to all. BoF sessions facilitate ad-hoc discussions and provide a place to gather and start off the chat, before moving on during the evening to a local restaurant or pub. There is no deadline for proposing a session, but we encourage you to do so at the earliest possible time to help us with planning and to publicize your proposed discussion. Attendees are also free to show up even if you have not confirmed, but again earlier confirmation will help us with planning. Please see http://cufp.org/bofs-2011 for details on how to propose a BoF session or confirm attendance at one of them. Scheduled BoFs will be listed here as they are confirmed. General information on CUFP 2011 can be found at http://cufp.org/conference/2011. The conference is co-located with ICFP. You can register for CUFP at the main ICFP website http://www.icfpconference.org/icfp2011. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pekadan@REDACTED Tue Aug 30 16:22:06 2011 From: pekadan@REDACTED (Peter Andersson) Date: Tue, 30 Aug 2011 16:22:06 +0200 Subject: [erlang-questions] distributed common test In-Reply-To: <351A3C5A-2539-4918-8F52-334FFA11EAB1@gmail.com> References: <59AADF44-733E-4E7D-AF27-D198EEE2CBBC@gmail.com> <351A3C5A-2539-4918-8F52-334FFA11EAB1@gmail.com> Message-ID: Actually, the word "dynamically" is somewhat misleading. What it means is that you can add nodes not defined in the test specification (and remove nodes defined in the specification) at the time you *start the test run*. See ct_master:run/3. There is no support for adding or removing nodes when tests are already running. /Peter Ericsson AB, Erlang/OTP 2011/8/29 Joel Reymont : > > On Aug 29, 2011, at 6:00 PM, Ulf Wiger wrote: > >> Perhaps this presentation can be of some help: >> >> http://www.erlang-factory.com/upload/presentations/204/EUC2009-CommonTestPresentation.pdf > > This is just the ticket. Thank you Uffe! > > One question? Page 34 (Using Common Test for Large Scale Testing ) mentions that test nodes can be added and removed dynamically. > > How is this done? > > ct_master API does not seem to include the ability to add nodes. > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jodie.burch@REDACTED Tue Aug 30 17:55:27 2011 From: jodie.burch@REDACTED (Jodie Burch) Date: Tue, 30 Aug 2011 16:55:27 +0100 Subject: [erlang-questions] Talk Submission - Erlang User Conference 2011 Message-ID: Hi Today is the last day we are taking talk submissions for the Erlang User Conference 2011 on the 3rd November and for the tutorials on the 4th November. We welcome submissions from developers, system designers, testers, community leaders, inventors, CTOs, evangelists, students, researchers and entrepreneurs alike. Participants at the Erlang User Conference want to hear about how Erlang/OTP is used real-world scenarios, talks about tricks, tools and applications which increase productivity or enable developers to write better code. Submit your talk and find out more info here: http://www.erlang-factory.com/conference/ErlangUserConference2011 Thanks Jodie -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Tue Aug 30 20:07:35 2011 From: francesco@REDACTED (Francesco Cesarini) Date: Tue, 30 Aug 2011 19:07:35 +0100 Subject: [erlang-questions] Erlang User Conference Talk Submission Deadline Wednesday Message-ID: <4E5D26E7.2040101@erlang-solutions.com> Hi All, this is just a reminder that tomorrow, Wednesday August 31st, is the deadline to submit your talks and tutorials for the Erlang User Conference. We have a great program lined up, but there are still a few slots available. If you need to bounce ideas or discuss the suitability of a presentation over another, feel free to email me. I will pass on your queries to the program committee. More information on the conference and submitting a talk is here: http://www.erlang-factory.com/conference/ErlangUserConference2011/submit_talk Other dates for your calendar include: September 10^th , Final program is announced October 15^th , Last Day for Early Bird Rate October 31^st , Start of Erlang University November 2nd, Guided evening walk around central Stockholm November 3^rd , Conference Talks + Evening Dinner November 4^th , Hackathon and Tutorials Looking forward to it, Francesco -- Erlang Solutions Ltd. http://www.erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Tue Aug 30 20:12:13 2011 From: francesco@REDACTED (Francesco Cesarini) Date: Tue, 30 Aug 2011 19:12:13 +0100 Subject: [erlang-questions] Erlang User Conference Talk Submission Deadline Wednesday In-Reply-To: <4E5D26E7.2040101@erlang-solutions.com> References: <4E5D26E7.2040101@erlang-solutions.com> Message-ID: <4E5D27FD.30006@erlang-solutions.com> OOppsss. Apologies for sending the message out twice, hadn't realized Jodie had just sent it :-(. /F On 30/08/2011 19:07, Francesco Cesarini wrote: > Hi All, > > this is just a reminder that tomorrow, Wednesday August 31st, is the > deadline to submit your talks and tutorials for the Erlang User > Conference. We have a great program lined up, but there are still a > few slots available. If you need to bounce ideas or discuss the > suitability of a presentation over another, feel free to email me. I > will pass on your queries to the program committee. > > More information on the conference and submitting a talk is here: > > http://www.erlang-factory.com/conference/ErlangUserConference2011/submit_talk > > Other dates for your calendar include: > > September 10^th , Final program is announced > October 15^th , Last Day for Early Bird Rate > October 31^st , Start of Erlang University > November 2nd, Guided evening walk around central Stockholm > November 3^rd , Conference Talks + Evening Dinner > November 4^th , Hackathon and Tutorials > > Looking forward to it, > Francesco > -- > Erlang Solutions Ltd. > http://www.erlang-solutions.com > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Erlang Solutions Ltd. http://www.erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Tue Aug 30 21:36:24 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 30 Aug 2011 21:36:24 +0200 Subject: [erlang-questions] fd_server library Message-ID: <4E5D3BB8.5030902@gmail.com> The use of fd_server to open privileged ports has largely gone out of fashion, but if anyone still needs the Erlang fd_server library (originally by Sebastian Strollo), we have put up a git repository here with a bunch of minor patches on top of the old Jungerl version: https://github.com/richcarl/fd_server /Richard From ericbmerritt@REDACTED Tue Aug 30 23:03:07 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 30 Aug 2011 16:03:07 -0500 Subject: [erlang-questions] [ANN] Erlson - dynamic name-value dictionary data type and syntax In-Reply-To: <1508A0A9-1E9F-4302-8698-B11E7854E922@erlang-solutions.com> References: <1508A0A9-1E9F-4302-8698-B11E7854E922@erlang-solutions.com> Message-ID: Let me back up Ulf on this on. The project is great. I love direct declarative support for common types and dictionaries have been missing for a long time. That said its really not complete until you can use it inside matches and guards. If that gets implemented this goes from a pleasant solution that gets rid of some syntax on occasion to something that fundamentally changes the language for the better. On Mon, Aug 29, 2011 at 2:16 AM, Ulf Wiger wrote: > > On 29 Aug 2011, at 06:50, Anton Lavrik wrote: > >> Hi list, >> >> It is my pleasure to announce Erlson - a proof of concept >> implementation of dynamic dictionary data type. > > This library was one of the entries in the Spawnfest competition, and received favourable mention from several of the judges (including me). I like the syntax - it feels like a good, minimal extension to the existing language. > > Personally, I would really like to see a special data type for this, as well as proper support for pattern matching and guards. I'm wary of yet another syntactic-sugar solution, overloading existing data types. > > This is in no way meant as criticism of the work so far - rather as a suggestion to grab the opportunity and implement full support for dynamic dictionaries (hashes, or whatever we want to call them). > > As always, there may be real, substantial objections that I have overlooked. If so, the good members of this list will surely present them. :) > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From joelr1@REDACTED Tue Aug 30 23:43:32 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 30 Aug 2011 22:43:32 +0100 Subject: [erlang-questions] common test: do suites run in parallel? Message-ID: I'm thinking of generating thousands of suites from test data and then having CT run these. I definitely want to run these generated suites in parallel, though. Can I do this? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From nskoric@REDACTED Wed Aug 31 01:27:09 2011 From: nskoric@REDACTED (Nikola Skoric) Date: Wed, 31 Aug 2011 01:27:09 +0200 Subject: [erlang-questions] systools:make_tar/2 fails because of {filename, join, [[]]} Message-ID: Hello everybody, Here is my problem: nick@REDACTED:~/code/egon_server$ erl -pa ebin Erlang R13B03 (erts-5.7.4) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.4 (abort with ^G) 1> systools:make_script("es_rel-0.0.1", [{path, ["ebin"]}]). ok 2> systools:make_tar("es_rel-0.0.1", [{path, ["ebin"]},{erts, code:root_dir()}]). {{case_clause, {'EXIT', {function_clause, [{filename,join,[[]]}, {systools_make,add_appl,7}, {systools_make,'-add_applications/5-fun-0-',6}, {lists,foldl,3}, {systools_make,add_applications,5}, {systools_make,mk_tar,6}, {systools_make,mk_tar,5}, {systools_make,make_tar,2}]}}}, [{systools_make,'-add_applications/5-fun-0-',6}, {lists,foldl,3}, {systools_make,add_applications,5}, {systools_make,mk_tar,6}, {systools_make,mk_tar,5}, {systools_make,make_tar,2}, {erl_eval,do_apply,5}, {shell,exprs,6}]} error 3> Now, that is exactly the same problem as these two: http://erlang.2086793.n4.nabble.com/troubleshooting-systools-make-tar-td2117171.html http://erlang.org/pipermail/erlang-questions/2008-August/037463.html I tried things advised in those threads, but nothing helped. I'm trying to build a release of this project: https://github.com/egon-sim/egon_server. Does anybody have any hints where to start looking for error? Thanks in advance, Nikola -- "Strange women lying in ponds distributing swords is no basis for a system of government." From bernie@REDACTED Wed Aug 31 08:00:26 2011 From: bernie@REDACTED (Bernie Duggan) Date: Wed, 31 Aug 2011 16:00:26 +1000 Subject: [erlang-questions] Dialyzer and rpc:pmap Message-ID: Hi all (Hi dialyzer people especially :)) I have been having a heated debate with dialyzer over the return value of rpc:pmap/3. The following code demonstrates the issue: ---------------------------------------------- -module(dialyzer_test). -export([run/0, f/1]). run() -> A = [{a, a}, {b, b}], B = [X || {a, X} <- A], C = rpc:pmap({?MODULE, f}, [], A), D = [X || {a, X} <- C], {B, D}. f(X) -> X. ---------------------------------------------- Basically I have a list, A, and an rpc:pmap call to the identity function with A as the list - this quite correctly returns A (stored in C). Next, I run a filter list comprehension over A and C to grab just the tuples with 'a' as their first element. The code all works fine (returning {[a], [a]}) but dialyzer complains: dialyzer_test.erl:10: The pattern [{'a', X} | _] can never match the type ['timeout' | {'value',_}] Line 10 is the "D =..." line. It looks like the return value of rpc:pmap is being inferred incorrectly, but the -spec for it in rpc.erl gives the return type as [term()] which is what I'd expect. Any hints? Something obvious I've missed? Cheers, Bernard -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Wed Aug 31 09:50:37 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Wed, 31 Aug 2011 09:50:37 +0200 Subject: [erlang-questions] systools:make_tar/2 fails because of {filename, join, [[]]} In-Reply-To: References: Message-ID: Hi Nikola! A workaround for the problem is to give the path as "./ebin" instead of just "ebin". The root cause is here: 35> file:path_open(["ebin"],"a.app",[read]). {ok,<0.189.0>,"ebin/a.app"} 36> file:path_open(["./ebin"],"a.app",[read]). {ok,<0.191.0>,"./ebin/a.app"} The last element of the returned tuple is used for setting #application.dir ("ebin" vs "./ebin") - later on systools_make:appDir/1 tries to find the parent directory of this... and that's where it fails. I'll try to get a solution into R14B04. Regards /siri 2011/8/31 Nikola Skoric > Hello everybody, > > Here is my problem: > > nick@REDACTED:~/code/egon_server$ erl -pa ebin > Erlang R13B03 (erts-5.7.4) [source] [smp:4:4] [rq:4] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.7.4 (abort with ^G) > 1> systools:make_script("es_rel-0.0.1", [{path, ["ebin"]}]). > ok > 2> systools:make_tar("es_rel-0.0.1", [{path, ["ebin"]},{erts, > code:root_dir()}]). > {{case_clause, > {'EXIT', > {function_clause, > [{filename,join,[[]]}, > {systools_make,add_appl,7}, > {systools_make,'-add_applications/5-fun-0-',6}, > {lists,foldl,3}, > {systools_make,add_applications,5}, > {systools_make,mk_tar,6}, > {systools_make,mk_tar,5}, > {systools_make,make_tar,2}]}}}, > [{systools_make,'-add_applications/5-fun-0-',6}, > {lists,foldl,3}, > {systools_make,add_applications,5}, > {systools_make,mk_tar,6}, > {systools_make,mk_tar,5}, > {systools_make,make_tar,2}, > {erl_eval,do_apply,5}, > {shell,exprs,6}]} > error > 3> > > Now, that is exactly the same problem as these two: > > http://erlang.2086793.n4.nabble.com/troubleshooting-systools-make-tar-td2117171.html > http://erlang.org/pipermail/erlang-questions/2008-August/037463.html > > I tried things advised in those threads, but nothing helped. I'm > trying to build a release of this project: > https://github.com/egon-sim/egon_server. Does anybody have any hints > where to start looking for error? > > Thanks in advance, > Nikola > > -- > "Strange women lying in ponds distributing swords is no basis for a > system of government." > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Aug 31 10:35:29 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 31 Aug 2011 10:35:29 +0200 Subject: [erlang-questions] REST interface to mnesia Message-ID: I want a resident Erlang database that always runs on my machine. When I boot the system it's always running (like mySQL etc.) I want to be able to access it from any programming language. What I'd like 1) You install X X might be erlang + mnesia + REST interface to mnesia 2) X runs forever 3) You manage X create tables etc via a web interface or via REST commands or through a native-erlang interface 4) You populate the tables via REST commands or through a native-erlang interface /Joe From gustav.simonsson@REDACTED Wed Aug 31 10:45:32 2011 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Wed, 31 Aug 2011 09:45:32 +0100 (BST) Subject: [erlang-questions] Dialyzer and rpc:pmap In-Reply-To: Message-ID: In Erlang function specifications term() is equivalent with any() which contains the set of all Erlang terms, including the atom 'timeout'. Looking at the internal rpc function calls from pmap/3, we see that in the end do_yield/2 can return the atom timeout. However, since do_yield/2 is called as do_yield(Key, infinity) the receive clause in do_yield/2 shouldn't be able to return timeout in the case of calling pmap/3. Does dialyzer infer that the receive clause in do_yield will wait indefinitely for a matching message in this case, or does it think that a timeout can occur, and therefore include the timeout atom as a possible return value? Regards, Gustav Simonsson ----- Original Message ----- From: "Bernie Duggan" To: "Erlang Questions" Sent: Wednesday, August 31, 2011 8:00:26 AM Subject: [erlang-questions] Dialyzer and rpc:pmap Hi all (Hi dialyzer people especially :)) I have been having a heated debate with dialyzer over the return value of rpc:pmap/3. The following code demonstrates the issue: ---------------------------------------------- -module(dialyzer_test). -export([run/0, f/1]). run() -> A = [{a, a}, {b, b}], B = [X || {a, X} <- A], C = rpc:pmap({?MODULE, f}, [], A), D = [X || {a, X} <- C], {B, D}. f(X) -> X. ---------------------------------------------- Basically I have a list, A, and an rpc:pmap call to the identity function with A as the list - this quite correctly returns A (stored in C). Next, I run a filter list comprehension over A and C to grab just the tuples with 'a' as their first element. The code all works fine (returning {[a], [a]}) but dialyzer complains: dialyzer_test.erl:10: The pattern [{'a', X} | _] can never match the type ['timeout' | {'value',_}] Line 10 is the "D =..." line. It looks like the return value of rpc:pmap is being inferred incorrectly, but the -spec for it in rpc.erl gives the return type as [term()] which is what I'd expect. Any hints? Something obvious I've missed? Cheers, Bernard _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Wed Aug 31 10:46:10 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 31 Aug 2011 10:46:10 +0200 Subject: [erlang-questions] REST interface to mnesia In-Reply-To: References: Message-ID: <4E5DF4D2.4080108@dev-extend.eu> On 08/31/2011 10:35 AM, Joe Armstrong wrote: > I want a resident Erlang database that always runs on my machine. > When I boot the system it's always running (like mySQL etc.) > > I want to be able to access it from any programming language. > > What I'd like > > 1) You install X > X might be erlang + mnesia + REST interface to mnesia > 2) X runs forever > 3) You manage X create tables etc via a web interface > or via REST commands or through a native-erlang interface > 4) You populate the tables via REST commands > or through a native-erlang interface Riak has a REST interface. -- Lo?c Hoguin Dev:Extend From aronisstav@REDACTED Wed Aug 31 11:10:59 2011 From: aronisstav@REDACTED (Stavros Aronis) Date: Wed, 31 Aug 2011 02:10:59 -0700 (PDT) Subject: [erlang-questions] Dialyzer and rpc:pmap In-Reply-To: References: Message-ID: <30125980.875.1314781859848.JavaMail.geo-discussion-forums@yqgn17> No, the problem lies in the spec of rpc:yield/1 -spec yield(Key) -> {value, Val} | timeout when Key :: key(), Val :: (Res :: term()) | {badrpc, Reason :: term()}. yield(Key) when is_pid(Key) -> {value,R} = do_yield(Key, infinity), R. The return value is simply "Val" and not the union "{value, Val} | timeout". I guess somebody in the OTP team can fix that. Dialyzer has a few tricks up its sleeve for handling incoming messages as the ones that appear in do_yield, but these are still experimental! Stavros -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Wed Aug 31 11:13:30 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 31 Aug 2011 10:13:30 +0100 Subject: [erlang-questions] monitoring process queue length Message-ID: <1BB1AEBF-F40C-4DE0-8CD2-C5BA36778DC1@gmail.com> Is there a built-in facility in Erlang that allows me to monitor process queue length across the system and issue an alarm if a threshold is exceeded? Am I stuck with manually going through a process list and sampling queue length? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From kostis@REDACTED Wed Aug 31 11:32:31 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 31 Aug 2011 12:32:31 +0300 Subject: [erlang-questions] Dialyzer and rpc:pmap In-Reply-To: <30125980.875.1314781859848.JavaMail.geo-discussion-forums@yqgn17> References: <30125980.875.1314781859848.JavaMail.geo-discussion-forums@yqgn17> Message-ID: <4E5DFFAF.8050401@cs.ntua.gr> On 08/31/11 12:10, Stavros Aronis wrote: > No, the problem lies in the spec of rpc:yield/1 > > -spec yield(Key) -> {value, Val} | timeout when > Key :: key(), > Val :: (Res :: term()) | {badrpc, Reason :: term()}. > > yield(Key) when is_pid(Key) -> > {value,R} = do_yield(Key, infinity), > R. > > The return value is simply "Val" and not the union "{value, Val} | > timeout". I guess somebody in the OTP team can fix that. Stavros is right. This typo in the spec was introduced in R14B03 and it's an obvious copy and paste error from functions nb_yield/[1,2] and do_yield/2 which appear below yield/1 in the file. It should be fixed by the OTP team. While editing the rpc.erl file, and although this does not affect Dialyzer in any way, perhaps a better way of writing the type of Val in a situation like the above is without the "Res ::" part which does not add any information whatsoever, IMO. It just makes the spec more bulky. Kostis From ulf.wiger@REDACTED Wed Aug 31 11:33:37 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 31 Aug 2011 11:33:37 +0200 Subject: [erlang-questions] monitoring process queue length In-Reply-To: <1BB1AEBF-F40C-4DE0-8CD2-C5BA36778DC1@gmail.com> References: <1BB1AEBF-F40C-4DE0-8CD2-C5BA36778DC1@gmail.com> Message-ID: <80444532-108D-4B8E-9B01-E2E6218399F0@erlang-solutions.com> An indirect way to catch them is by using erlang:system_monitor(self(), [{large_heap, Sz}]) since the message queue is part of the heap. http://www.erlang.org/doc/man/erlang.html#system_monitor-2 BR, Ulf W On 31 Aug 2011, at 11:13, Joel Reymont wrote: > Is there a built-in facility in Erlang that allows me to monitor process queue length across the system and issue an alarm if a threshold is exceeded? > > Am I stuck with manually going through a process list and sampling queue length? > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanjo@REDACTED Wed Aug 31 14:12:57 2011 From: juanjo@REDACTED (Juan Jose Comellas) Date: Wed, 31 Aug 2011 09:12:57 -0300 Subject: [erlang-questions] REST interface to mnesia In-Reply-To: References: Message-ID: Both CouchDB and Riak could meet your requirements. In fact, if you're using a recent version of Ubuntu, CouchDB is already installed and works practically in the same way you're describing (Canonical uses it as the main database for the Ubuntu One service). The only issue I see is that neither of them have tables (in the relational sense). What you normally do is create databases and store JSON documents in them. Both have REST interfaces and allow you to query documents by ID but other than that, they way in which you work with them is very different. CouchDB allows you to build "views" (indexes where you select the documents that go into the index and the key that is used) through a map/reduce process. Riak allows you to link documents and use those links to relate them or you can run map/reduce jobs to query the database. Riak is now adding support for secondary indexes. Both have "extensions" for full-text search (couchdb-lucene; Riak Search). Riak runs distributed out of the box (as a Dynamo key/value store). CouchDB does multi-master replication very easily but you need to use BigCouch (a CouchDB "fork") to run it as a single database over several nodes Dynamo-style. I hope this helps. Juanjo On Wed, Aug 31, 2011 at 5:35 AM, Joe Armstrong wrote: > I want a resident Erlang database that always runs on my machine. > When I boot the system it's always running (like mySQL etc.) > > I want to be able to access it from any programming language. > > What I'd like > > 1) You install X > X might be erlang + mnesia + REST interface to mnesia > 2) X runs forever > 3) You manage X create tables etc via a web interface > or via REST commands or through a native-erlang interface > 4) You populate the tables via REST commands > or through a native-erlang interface > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Wed Aug 31 16:06:11 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 31 Aug 2011 15:06:11 +0100 Subject: [erlang-questions] capturing all traces in a running system Message-ID: <483A36B7-2A6F-4785-BEAA-1A8A60CDE229@gmail.com> Has anyone tried to capture all traces in a running system for later analysis? What kind of performance impact does trace collection have on a running system, particularly if traces are shipped over TCP to a non-Erlang server? It seems that it should be possible to reconstruct the behavior of a running system by logging what module started what process and by capturing all traces for offline analysis. Has this been tried before? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From kevin@REDACTED Wed Aug 31 16:09:35 2011 From: kevin@REDACTED (Kevin Smith) Date: Wed, 31 Aug 2011 10:09:35 -0400 Subject: [erlang-questions] REST interface to mnesia In-Reply-To: References: Message-ID: <698733B7-3FBC-418F-88A8-942660F7F11B@hypotheticalabs.com> How important is queryability? Would you want to use qlc, SQL, MapReduce, or something else to write the queries? --Kevin On Aug 31, 2011, at 4:35 AM, Joe Armstrong wrote: > I want a resident Erlang database that always runs on my machine. > When I boot the system it's always running (like mySQL etc.) > > I want to be able to access it from any programming language. > > What I'd like > > 1) You install X > X might be erlang + mnesia + REST interface to mnesia > 2) X runs forever > 3) You manage X create tables etc via a web interface > or via REST commands or through a native-erlang interface > 4) You populate the tables via REST commands > or through a native-erlang interface > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jsmilam@REDACTED Wed Aug 31 16:13:30 2011 From: jsmilam@REDACTED (Justin Milam) Date: Wed, 31 Aug 2011 10:13:30 -0400 Subject: [erlang-questions] ssl_esock leaking file descriptors Message-ID: I've started to notice a slow leak of file descriptors in the ssl_esock port. I'm running Erlang R14B and using SSL to encrypt traffic over the Erlang distribution protocol. The cluster has 10 nodes minimum with transient nodes joining and leaving the cluster regularly. From checking the ssl_esock process with lsof it appears to be slowly leaking file descriptors. The number of open file descriptors seems to increase after a node joins the cluster and then leaves. Eventually ssl_esock holds open enough file descriptors to hit the ulimit (currently 8192) in which case ssl_esock goes into an infinite loop using near 100% of one of the CPUs. I've been able to reproduce the issue by lowering the ulimit and continually connecting/disconnecting a remote shell to a local running node until the ulimit is reached. When ssl_esock is running in debug mode I see the following being logged continually: ==========LOOP============= MASKS SET FOR FD: 27 (read) 26 (read) 25 (read) 24 (read) 19 (read) 18 (read) 17 (read) 16 (read) 12 (read) 11 (read) 10 (read) 9 (read) 8 (read) 7 (read) 6 (read) CONNECTIONS: - DEFUNCT [0x8772978] (fd = 29) - DEFUNCT [0x86f9950] (fd = 28) - JOINED [0x875ae30] (origin = accept) (fd = 26, eof = 0, wq = 0, bp = 0) (proxyfd = 27, eof = 0, wq = 0, bp = 0) - JOINED [0x86fa970] (origin = accept) (fd = 24, eof = 0, wq = 0, bp = 0) (proxyfd = 25, eof = 0, wq = 0, bp = 0) - DEFUNCT [0x8733600] (fd = 21) - DEFUNCT [0x8732c38] (fd = 20) - JOINED [0x8733958] (origin = accept) (fd = 18, eof = 0, wq = 0, bp = 0) (proxyfd = 19, eof = 0, wq = 0, bp = 0) - JOINED [0x8734f78] (origin = accept) (fd = 16, eof = 0, wq = 0, bp = 0) (proxyfd = 17, eof = 0, wq = 0, bp = 0) - CONNECTED [0x87134a8] (fd = 15) - DEFUNCT [0x871f220] (fd = 13) - JOINED [0x87147d0] (origin = accept) (fd = 11, eof = 0, wq = 0, bp = 0) (proxyfd = 12, eof = 0, wq = 0, bp = 0) - JOINED [0x87083d0] (origin = connect) (fd = 9, eof = 0, wq = 0, bp = 0) (proxyfd = 10, eof = 0, wq = 0, bp = 0) - JOINED [0x86f29e8] (origin = connect) (fd = 7, eof = 0, wq = 0, bp = 0) (proxyfd = 8, eof = 0, wq = 0, bp = 0) - ACTIVE_LISTENING [0x86f2258] (fd = 6, acceptors = 1) Before poll/select: 15 descriptors (total 29) Error calling accept() accept error (proxy_listensock): emfile Has anyone else experienced such behavior? Thanks -justin -------------- next part -------------- An HTML attachment was scrubbed... URL: From gordon@REDACTED Wed Aug 31 16:35:50 2011 From: gordon@REDACTED (Gordon Guthrie) Date: Wed, 31 Aug 2011 15:35:50 +0100 Subject: [erlang-questions] ssl_esock leaking file descriptors In-Reply-To: References: Message-ID: We get an intermittent ssl_esock problem which I have never successful reproduced. It goes to 100% and the process needs to be manually killed. Richard Andrews also reported a problem with it going to 100% CPU in 2009: http://erlang.2086793.n4.nabble.com/ssl-esock-spinning-out-of-control-in-poll-td2117067.html He has a patch for that. It is on my 'long list' of things to fix but more frequent/reproducable ones allways get in the way. Gordon On 31 August 2011 15:13, Justin Milam wrote: > I've started to notice a slow leak of file descriptors in the ssl_esock > port. I'm running Erlang R14B and using SSL to encrypt traffic over the > Erlang distribution protocol. The cluster has 10 nodes minimum with > transient nodes joining and leaving the cluster regularly. From checking the > ssl_esock process with lsof it appears to be slowly leaking file > descriptors. The number of open file descriptors seems to increase after a > node joins the cluster and then leaves. Eventually ssl_esock holds open > enough file descriptors to hit the ulimit (currently 8192) in which case > ssl_esock goes into an infinite loop using near 100% of one of the CPUs. > > I've been able to reproduce the issue by lowering the ulimit and > continually connecting/disconnecting a remote shell to a local running node > until the ulimit is reached. When ssl_esock is running in debug mode I see > the following being logged continually: > > ==========LOOP============= > MASKS SET FOR FD: 27 (read) 26 (read) 25 (read) 24 (read) 19 (read) 18 > (read) 17 (read) 16 (read) 12 (read) 11 (read) 10 (read) 9 (read) 8 (read) 7 > (read) 6 (read) > CONNECTIONS: > - DEFUNCT [0x8772978] (fd = 29) > - DEFUNCT [0x86f9950] (fd = 28) > - JOINED [0x875ae30] (origin = accept) > (fd = 26, eof = 0, wq = 0, bp = 0) > (proxyfd = 27, eof = 0, wq = 0, bp = 0) > - JOINED [0x86fa970] (origin = accept) > (fd = 24, eof = 0, wq = 0, bp = 0) > (proxyfd = 25, eof = 0, wq = 0, bp = 0) > - DEFUNCT [0x8733600] (fd = 21) > - DEFUNCT [0x8732c38] (fd = 20) > - JOINED [0x8733958] (origin = accept) > (fd = 18, eof = 0, wq = 0, bp = 0) > (proxyfd = 19, eof = 0, wq = 0, bp = 0) > - JOINED [0x8734f78] (origin = accept) > (fd = 16, eof = 0, wq = 0, bp = 0) > (proxyfd = 17, eof = 0, wq = 0, bp = 0) > - CONNECTED [0x87134a8] (fd = 15) > - DEFUNCT [0x871f220] (fd = 13) > - JOINED [0x87147d0] (origin = accept) > (fd = 11, eof = 0, wq = 0, bp = 0) > (proxyfd = 12, eof = 0, wq = 0, bp = 0) > - JOINED [0x87083d0] (origin = connect) > (fd = 9, eof = 0, wq = 0, bp = 0) > (proxyfd = 10, eof = 0, wq = 0, bp = 0) > - JOINED [0x86f29e8] (origin = connect) > (fd = 7, eof = 0, wq = 0, bp = 0) > (proxyfd = 8, eof = 0, wq = 0, bp = 0) > - ACTIVE_LISTENING [0x86f2258] (fd = 6, acceptors = 1) > Before poll/select: 15 descriptors (total 29) > Error calling accept() > accept error (proxy_listensock): emfile > > Has anyone else experienced such behavior? > > Thanks > > -justin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 -------------- next part -------------- An HTML attachment was scrubbed... URL: From pmacgown@REDACTED Wed Aug 31 17:08:56 2011 From: pmacgown@REDACTED (pmacgown@REDACTED) Date: Wed, 31 Aug 2011 15:08:56 +0000 (UTC) Subject: [erlang-questions] Request to be removed from Erlang Questions In-Reply-To: Message-ID: <790487340.698709.1314803336707.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> Hi, I have decided to take a hiatus from this list and have requested to be taken off.? I did so from the web site but it didn't seem to take.? Please remove me from this list.? Thanks, Peter MacGown -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Aug 31 18:58:24 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 31 Aug 2011 18:58:24 +0200 Subject: [erlang-questions] REST interface to mnesia In-Reply-To: <698733B7-3FBC-418F-88A8-942660F7F11B@hypotheticalabs.com> References: <698733B7-3FBC-418F-88A8-942660F7F11B@hypotheticalabs.com> Message-ID: On Wed, Aug 31, 2011 at 4:09 PM, Kevin Smith wrote: > How important is queryability? Would you want to use qlc, SQL, MapReduce, or something else to write the queries? Not really. My order of priorities is: 1) Ease of install "working out of the box" and "a few files" is important I don't want lot's of files scattered all over the place Entire database in one blob is nice (like SQLite) should run forever on my machine without bothering me once installed 2) Easy management "it just works" - nice to have a web interface. 3) Low storage overhead - don't be too profligate with storage 4) Fast (ish) key-value storage. over 5K reads/writes/sec is fine 5) Crash resiliant 6) Can be controlled via a socket interface 7) indexing or secondary keys 8) full-text indexing 9) Relational queries less important - I'm kind of used to Key-Value queries and designing data structures round them /Joe > > --Kevin > On Aug 31, 2011, at 4:35 AM, Joe Armstrong wrote: > >> I want a resident Erlang database that always runs on my machine. >> When I boot the system it's always running (like mySQL etc.) >> >> I want to be able to access it from any programming language. >> >> What I'd like >> >> ? 1) You install X >> ? ? ? ?X might be erlang + mnesia + REST interface to mnesia >> ? 2) X runs forever >> ? 3) You manage X create tables etc via a web interface >> ? ? ? ?or via REST commands or through a native-erlang interface >> ? 4) You populate the tables via REST commands >> ? ? ? ?or through a native-erlang interface >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From erlang@REDACTED Wed Aug 31 19:07:54 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 31 Aug 2011 19:07:54 +0200 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet Message-ID: When I do > mnesia:stop(). I see this =INFO REPORT==== 31-Aug-2011::19:03:57 === application: mnesia exited: stopped type: temporary How can I get rid of this printout? I never want to see excess output if the system has correctly done what I told it to do. /Joe From juanjo@REDACTED Wed Aug 31 19:54:34 2011 From: juanjo@REDACTED (Juan Jose Comellas) Date: Wed, 31 Aug 2011 14:54:34 -0300 Subject: [erlang-questions] REST interface to mnesia In-Reply-To: References: <698733B7-3FBC-418F-88A8-942660F7F11B@hypotheticalabs.com> Message-ID: If you want something simple and restricted to a single node then I'd use CouchDB. It may not be very strong on items (3) and (4), but it does fulfill the other ones. If you end up using it I'd recommend reading CouchDB's performance guide here: http://guide.couchdb.org/draft/performance.html (hint: monotonic document IDs and bulk inserts when possible will improve the performance and used space a lot). On Wed, Aug 31, 2011 at 1:58 PM, Joe Armstrong wrote: > > On Wed, Aug 31, 2011 at 4:09 PM, Kevin Smith wrote: > > How important is queryability? Would you want to use qlc, SQL, MapReduce, or something else to write the queries? > > Not really. My order of priorities is: > > 1) Ease of install > ? ?"working out of the box" and "a few files" is important > ? ?I don't want lot's of files scattered all over the place > ? ?Entire database in one blob is nice (like SQLite) > ? ?should run forever on my machine without bothering me once installed > 2) Easy management > ? ?"it just works" - nice to have a web interface. > 3) Low storage overhead - don't be too profligate with storage > 4) Fast (ish) key-value storage. over 5K reads/writes/sec is fine > 5) Crash resiliant > 6) Can be controlled via a socket interface > 7) indexing or secondary keys > 8) full-text indexing > 9) Relational queries less important - I'm kind of used to Key-Value queries > ? ?and designing data structures round them > > /Joe > > > > > > --Kevin > > On Aug 31, 2011, at 4:35 AM, Joe Armstrong wrote: > > > >> I want a resident Erlang database that always runs on my machine. > >> When I boot the system it's always running (like mySQL etc.) > >> > >> I want to be able to access it from any programming language. > >> > >> What I'd like > >> > >> ? 1) You install X > >> ? ? ? ?X might be erlang + mnesia + REST interface to mnesia > >> ? 2) X runs forever > >> ? 3) You manage X create tables etc via a web interface > >> ? ? ? ?or via REST commands or through a native-erlang interface > >> ? 4) You populate the tables via REST commands > >> ? ? ? ?or through a native-erlang interface > >> > >> /Joe > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From roberto@REDACTED Wed Aug 31 20:18:33 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 31 Aug 2011 20:18:33 +0200 Subject: [erlang-questions] REST interface to mnesia In-Reply-To: References: <698733B7-3FBC-418F-88A8-942660F7F11B@hypotheticalabs.com> Message-ID: full-text indexing does come with a cost, sometimes related to write speed. depending on how you want to balance these factors, and if text-search is important (since it's your 8th point i'm not sure whether you can cross it out or not), you may consider: . riak with riak-search [as already suggested]; . solr (http://lucene.apache.org/solr/) both are fast, have a JSON interface, and should answer your needs. r. 2011/8/31 Joe Armstrong > On Wed, Aug 31, 2011 at 4:09 PM, Kevin Smith > wrote: > > How important is queryability? Would you want to use qlc, SQL, MapReduce, > or something else to write the queries? > > Not really. My order of priorities is: > > 1) Ease of install > "working out of the box" and "a few files" is important > I don't want lot's of files scattered all over the place > Entire database in one blob is nice (like SQLite) > should run forever on my machine without bothering me once installed > 2) Easy management > "it just works" - nice to have a web interface. > 3) Low storage overhead - don't be too profligate with storage > 4) Fast (ish) key-value storage. over 5K reads/writes/sec is fine > 5) Crash resiliant > 6) Can be controlled via a socket interface > 7) indexing or secondary keys > 8) full-text indexing > 9) Relational queries less important - I'm kind of used to Key-Value > queries > and designing data structures round them > > /Joe > -------------- next part -------------- An HTML attachment was scrubbed... URL: From demeshchuk@REDACTED Wed Aug 31 20:27:33 2011 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Wed, 31 Aug 2011 22:27:33 +0400 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet In-Reply-To: References: Message-ID: That's exactly what I want to do with rabbitmq erlang client as well :( On Wed, Aug 31, 2011 at 9:07 PM, Joe Armstrong wrote: > When I do > >> mnesia:stop(). > > I see this > > =INFO REPORT==== 31-Aug-2011::19:03:57 === > ? ?application: mnesia > ? ?exited: stopped > ? ?type: temporary > > How can I get rid of this printout? I never want to see excess output > if the system has correctly done what I told it to do. > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best regards, Dmitry Demeshchuk