From matan.levy@REDACTED Thu Oct 1 00:31:49 2009 From: matan.levy@REDACTED (Zohan) Date: Wed, 30 Sep 2009 15:31:49 -0700 (PDT) Subject: mneisa table fragmentation vs manually splitting In-Reply-To: <05840888-1633-4702-a3ce-839b038fe9a4@m20g2000vbp.googlegroups.com> References: <05840888-1633-4702-a3ce-839b038fe9a4@m20g2000vbp.googlegroups.com> Message-ID: <72ddfd5c-1ef8-43c6-8fe2-776d0debf056@c28g2000yqd.googlegroups.com> hi, For the manual split option you describe, why do you claim the application should know how to use a specific node to write to a specific key, in order to use master-slave duplicated database scheme? What are the master-slave relations you assume in the 10 machine cluster? BR, Z. On Sep 27, 8:40?am, vim wrote: > hi, > > i am working on a erlang based distributed architecture, at the end i > intend to use it on EC2. my application included a database, and since > i want to spread the load on the database, i thought of splitting the > table. > > the question is should i choose mnesia built-in feature 'fragmented > table' over manually splitting the database into clusters? > > i.e. suppose i have a simple key-value sets which i need to store in > my database. and suppose i have 1000 keys, and i did some measurements > and i want only 10 keys on each machine (for better performance) so i > need 100 machines. ?thus, if i understand correctly the > documentation, i have at least two options: > > 1. manual split: > i can set up clusters of 10 machines each. on each machine i will have > a node with a running mnesia, and on a such cluster all the mnesia > nodes will be connected. furthermore, i can use master-slave > duplicated database scheme, if my application will know how to use a > specific node to write to a specific key. i assume here that *all* the > nodes in my world (100) are erlang connected, and on each 10 machine > cluster there is a connected (shared) mnesia database. > > 2. table fragmentation: > i can connect all my 100 machines with single mnesia schema, and > fragment the table into 10 pieces, where each cluster of 10 machines > will share each piece. again, i can use the same master-slave scheme i > described above to access my database. > > am i correct with those assumptions? > should i prefer one method over the other? > > thanks for your help, > vim. > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From ckerr@REDACTED Thu Oct 1 00:36:55 2009 From: ckerr@REDACTED (Cameron Kerr) Date: Thu, 01 Oct 2009 11:36:55 +1300 Subject: Need parallel calls to gen_server:call, similar but different to gen_server:multi_call Message-ID: <4AC3DD87.6000309@cs.otago.ac.nz> Hello, I'm working on a project and have a need to send a gen_server:call to multiple gen_servers in parallel. I've looked at gen_server:multi_call, but if I understand it correctly, it is for working with multiple nodes, not multiple processes on the same node. I have a list of gen_server PIDs, so would like something that looks like: gen_server:parallel_call(Pid_list, Request) -> [Response] Is there something like this? If not, perhaps I could just send the response using the '!' operator and process it using gen_server's handle_info, then manually receive any responses... but this seems messy. Many thanks, Cameron Kerr From steven.charles.davis@REDACTED Thu Oct 1 01:33:28 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 30 Sep 2009 16:33:28 -0700 (PDT) Subject: Need parallel calls to gen_server:call, similar but different to gen_server:multi_call In-Reply-To: <4AC3DD87.6000309@cs.otago.ac.nz> References: <4AC3DD87.6000309@cs.otago.ac.nz> Message-ID: <3d1e6513-a8c5-48da-bc3c-e7e8e3da770b@j39g2000yqh.googlegroups.com> ...and the following is not appropriate to your needs? Responses = [gen_server:call(Pid,Request) || Pid <- PidList] /s On Sep 30, 5:36?pm, Cameron Kerr wrote: > Hello, I'm working on a project and have a need to send a > gen_server:call to multiple gen_servers in parallel. > > I've looked at gen_server:multi_call, but if I understand it correctly, > it is for working with multiple nodes, not multiple processes on the > same node. I have a list of gen_server PIDs, so would like something > that looks like: > > gen_server:parallel_call(Pid_list, Request) -> [Response] > > Is there something like this? If not, perhaps I could just send the > response using the '!' operator and process it using gen_server's > handle_info, then manually receive any responses... but this seems messy. > > Many thanks, > Cameron Kerr > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From mihai@REDACTED Thu Oct 1 03:07:34 2009 From: mihai@REDACTED (Mihai Balea) Date: Wed, 30 Sep 2009 21:07:34 -0400 Subject: [erlang-questions] OS X 10.5 build without WxWidgets In-Reply-To: <97619b170909301239v1490e713g4c1aa73b39b7564@mail.gmail.com> References: <18569D8E-0920-40F8-A005-A0005B566B05@miceda.org> <97619b170909301239v1490e713g4c1aa73b39b7564@mail.gmail.com> Message-ID: On Sep 30, 2009, at 3:39 PM, Rapsey wrote: > build erlang like this: > > ./configure ...your params > touch lib/wx/SKIP > make Wouldn't it be better to have the option to disable WxWidgets from configure? I imagine that many server-oriented deployments could do without it. Mihai From mihai@REDACTED Thu Oct 1 03:23:26 2009 From: mihai@REDACTED (Mihai Balea) Date: Wed, 30 Sep 2009 21:23:26 -0400 Subject: [erlang-questions] Re: Need parallel calls to gen_server:call, similar but different to gen_server:multi_call In-Reply-To: <3d1e6513-a8c5-48da-bc3c-e7e8e3da770b@j39g2000yqh.googlegroups.com> References: <4AC3DD87.6000309@cs.otago.ac.nz> <3d1e6513-a8c5-48da-bc3c-e7e8e3da770b@j39g2000yqh.googlegroups.com> Message-ID: <78765CB3-951D-4A27-B931-AC1FFA2E2A89@hates.ms> On Sep 30, 2009, at 7:33 PM, Steve Davis wrote: > ...and the following is not appropriate to your needs? > > Responses = [gen_server:call(Pid,Request) || Pid <- PidList] That would serialize the calls to the gen_server processes, not taking advantage of the parallel nature of erlang How about using rpc:pmap or rpc:parallel_eval? Mihai From rvirding@REDACTED Thu Oct 1 03:36:38 2009 From: rvirding@REDACTED (Robert Virding) Date: Thu, 1 Oct 2009 03:36:38 +0200 Subject: [erlang-questions] Re: Need parallel calls to gen_server:call, similar but different to gen_server:multi_call In-Reply-To: <78765CB3-951D-4A27-B931-AC1FFA2E2A89@hates.ms> References: <4AC3DD87.6000309@cs.otago.ac.nz> <3d1e6513-a8c5-48da-bc3c-e7e8e3da770b@j39g2000yqh.googlegroups.com> <78765CB3-951D-4A27-B931-AC1FFA2E2A89@hates.ms> Message-ID: <3dbc6d1c0909301836n75b94064g927de26da0caef1d@mail.gmail.com> The problem is getting the right message formats for gen_server, and they aren't really specified. One alternative would be to use gen_server:cast to send and then handle the replies explicitly yourself. Robert P.S. One long, never-resolved discussion Joe and I had for many years was whether you should have an API like is done in gen_XXX or to explicitly specify the message protocol. Having the message protocol would allow you to increase concurrency. 2009/10/1 Mihai Balea > > > On Sep 30, 2009, at 7:33 PM, Steve Davis wrote: > > ...and the following is not appropriate to your needs? >> >> Responses = [gen_server:call(Pid,Request) || Pid <- PidList] >> > > That would serialize the calls to the gen_server processes, not taking > advantage of the parallel nature of erlang > How about using rpc:pmap or rpc:parallel_eval? > > Mihai > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ckerr@REDACTED Thu Oct 1 03:01:24 2009 From: ckerr@REDACTED (Cameron Kerr) Date: Thu, 01 Oct 2009 14:01:24 +1300 Subject: [erlang-questions] Re: Need parallel calls to gen_server:call, similar but different to gen_server:multi_call In-Reply-To: <3d1e6513-a8c5-48da-bc3c-e7e8e3da770b@j39g2000yqh.googlegroups.com> References: <4AC3DD87.6000309@cs.otago.ac.nz> <3d1e6513-a8c5-48da-bc3c-e7e8e3da770b@j39g2000yqh.googlegroups.com> Message-ID: <4AC3FF64.2070909@cs.otago.ac.nz> No, I need something that runs in parallel. I've found rpc:pmap, and am currently trying to adapt it to my needs. Thanks. Steve Davis wrote: > ...and the following is not appropriate to your needs? > > Responses = [gen_server:call(Pid,Request) || Pid <- PidList] > > /s > > On Sep 30, 5:36 pm, Cameron Kerr wrote: > >> Hello, I'm working on a project and have a need to send a >> gen_server:call to multiple gen_servers in parallel. >> >> I've looked at gen_server:multi_call, but if I understand it correctly, >> it is for working with multiple nodes, not multiple processes on the >> same node. I have a list of gen_server PIDs, so would like something >> that looks like: >> >> gen_server:parallel_call(Pid_list, Request) -> [Response] >> >> Is there something like this? If not, perhaps I could just send the >> response using the '!' operator and process it using gen_server's >> handle_info, then manually receive any responses... but this seems messy. >> >> Many thanks, >> Cameron Kerr >> >> ________________________________________________________________ >> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From mihai@REDACTED Thu Oct 1 04:52:42 2009 From: mihai@REDACTED (Mihai Balea) Date: Wed, 30 Sep 2009 22:52:42 -0400 Subject: Message protocol vs. Function call API In-Reply-To: <3dbc6d1c0909301836n75b94064g927de26da0caef1d@mail.gmail.com> References: <4AC3DD87.6000309@cs.otago.ac.nz> <3d1e6513-a8c5-48da-bc3c-e7e8e3da770b@j39g2000yqh.googlegroups.com> <78765CB3-951D-4A27-B931-AC1FFA2E2A89@hates.ms> <3dbc6d1c0909301836n75b94064g927de26da0caef1d@mail.gmail.com> Message-ID: This came up on a different thread, but I believe it is an interesting subject on its own. On Sep 30, 2009, at 9:36 PM, Robert Virding wrote: > P.S. One long, never-resolved discussion Joe and I had for many > years was whether you should have an API like is done in gen_XXX or > to explicitly specify the message protocol. Having the message > protocol would allow you to increase concurrency. I've been mulling this question myself lately. Besides increased concurrency, a message protocol would also allow for other nifty things, like versioning. Function calls make synchronous APIs easier. So I was wondering what folks on this list think... besides the obvious "it depends on the situation". Mihai From erlang@REDACTED Thu Oct 1 09:39:27 2009 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 1 Oct 2009 09:39:27 +0200 Subject: [erlang-questions] Message protocol vs. Function call API In-Reply-To: References: <4AC3DD87.6000309@cs.otago.ac.nz> <3d1e6513-a8c5-48da-bc3c-e7e8e3da770b@j39g2000yqh.googlegroups.com> <78765CB3-951D-4A27-B931-AC1FFA2E2A89@hates.ms> <3dbc6d1c0909301836n75b94064g927de26da0caef1d@mail.gmail.com> Message-ID: <9b08084c0910010039r2f4bf171qfd00d4e3f11c689a@mail.gmail.com> Think of it as a protocol stack. Users Programs ------------------ API --------------- (A) Library ------------ Messaging interface ----- (B) Implementation The users program call functions in a module. These are defined in an API (A). The library which implements the API works together with a messaging interface (B). The top-level interface (A) is defined in terms of typed function calls. The messaging interface (B) is defined in terms of message flows. Now (A) is usually well defined and should be stable over many years. (B) might change with time. This has the advantage that we can change B as we learn more while not damaging applications that use A. There are some problems with this: Usually the interface (B) is not well specified and has to be inferred by reading the code. Also for efficiency reasons a user program might like to bypass (A) and talk to the (B) interface directly, the latter will cause problems if the (B) protocol is changed. This is exactly how the erlang system is structured. A user program doing io will call io:read(Pid, X) (ie a function according to the (A) interface) the implementation of io:read will send a {get_until,unicode,X,erl_scan,tokens,[1]} message across the (B) interface, and so on ... The (A) interface is well defined and stable. The (B) interface is not well documented (read the code Luke). The is not too bad. Usually only one person has to understand the (B) interface (ie the person who implements the library) - whereas large numbers of people have to understand the (A) interface. Taking a problem and partitioning it into (A) and (B) interfaces is not easy. You can start with (A) and "invent" the necessary (B) interface (this is top-down design). or you can start with B and invent A (bottom-up design). In the case where B is given we'd do bottom up design (for example B *is* TCP/IP or a given hardware interface like an SSD). For user convenience we'd like to start with A and derive B. Sometimes we design A with a set of specific set of properties and B is given - so here we just have to write some glue in the middle (what I call a middle man) - an example of this might be AMQP (the Advanced Message Queue Protocol). The designer of AMQP things "what would be a nice API to write messaging applications with?" - answer "reliable messaging with, security and notifications" - so (A) is given. (B) might be TCP/IP or UDP or whatever ... For some projects I start by defining (A) then invent (B), or use a given (B). others B is given and I figure out an (A) that seems appropriate. "It depends ..." :-) /Joe On Thu, Oct 1, 2009 at 4:52 AM, Mihai Balea wrote: > This came up on a different thread, but I believe it is an interesting > subject on its own. > > On Sep 30, 2009, at 9:36 PM, Robert Virding wrote: > >> P.S. One long, never-resolved discussion Joe and I had for many years was >> whether you should have an API like is done in gen_XXX or to explicitly >> specify the message protocol. Having the message protocol would allow you to >> increase concurrency. > > I've been mulling this question myself lately. Besides increased > concurrency, a message protocol would also allow for other nifty things, > like versioning. Function calls make synchronous APIs easier. So I was > wondering what folks on this list think... besides the obvious "it depends > on the situation". > > Mihai > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From max.lapshin@REDACTED Thu Oct 1 09:50:59 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 1 Oct 2009 11:50:59 +0400 Subject: Switching between http/non-http depending of first byte Message-ID: Hi. I want to implement an unusual feature: when server accepts socket, it must look at first byte. If it is 0x03, than I want to handle it by one handler (RTMP), if other, than HTTP handler. Problem is that I need only one byte and received packet should be passed to exact protocol handler. I understand, that I will not be able to use erlang setopt(http) feature, so question is the following: should I better use Yaws or MochiWeb as underlying webserver? From erlang@REDACTED Thu Oct 1 10:09:23 2009 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 1 Oct 2009 10:09:23 +0200 Subject: [erlang-questions] Re: Need parallel calls to gen_server:call, similar but different to gen_server:multi_call In-Reply-To: <3dbc6d1c0909301836n75b94064g927de26da0caef1d@mail.gmail.com> References: <4AC3DD87.6000309@cs.otago.ac.nz> <3d1e6513-a8c5-48da-bc3c-e7e8e3da770b@j39g2000yqh.googlegroups.com> <78765CB3-951D-4A27-B931-AC1FFA2E2A89@hates.ms> <3dbc6d1c0909301836n75b94064g927de26da0caef1d@mail.gmail.com> Message-ID: <9b08084c0910010109w4915096dg31b83e5b529f0d2c@mail.gmail.com> On Thu, Oct 1, 2009 at 3:36 AM, Robert Virding wrote: > The problem is getting the right message formats for gen_server, and they > aren't really specified. One alternative would be to use gen_server:cast to > send and then handle the replies explicitly yourself. > > Robert > > P.S. One long, never-resolved discussion Joe and I had for many years was > whether you should have an API like is done in gen_XXX or to explicitly > specify the message protocol. Having the message protocol would allow you to > increase concurrency. Even worse - I'm pretty sure the entire system is "over synchronized" - we made many interactions into "pure RPC" in order to avoid buffer overrun problems. ie if we have a producer and consumer, we have to make sure that the consumer runs faster than the producer - we solved this by explicitly syncing the producer and consumer. we write code like this all the time: rpc(Pid, X) -> Pid ! {self(), X}, receive {Pid, Reply} -> Reply end. The problem is that the client blocks until it receives a reply from the server. So there is no concurrency, either the client runs or the server. No problem in 1986 on a unicore, but today we could do better. We *do* need to sync up producers and consumers, because we can never know if a producer produces stuff faster than the consumer can consume it. There are a couple of ways to do this - in Erlang code we could require the sever to ack every (say) 10'th command - so a client could pump out ten requests before suspending and checking that the server was keeping up, this would increase concurrency since it's not necessary to sync up every RPC. Another idea might be to use something like TCP "backpressure" between erlang processes viewing causing a client to suspend if the inbox of a server was getting large - but this would introduce an entire class of new problems and create strange inter-dependencies. We should also probably have some abstractions for concurrent pipes. pipe([F1, F2, F3, ..., Fn]) in which spawn(F1) starts a process which sends messages to a process started by spawn(F2) ... which are consumed by a process that is started by spawn(F3) The trick would be to handle the queue sizes between the steps in the pipeline and deal with out of band data and exceptions. A naive implementation of pipe is easy but to avoid buffer overruns would be inefficient. (consider the case where F1 produces messages extremely quickly, but F2 is very slow :-) /Joe > > 2009/10/1 Mihai Balea > >> >> >> On Sep 30, 2009, at 7:33 PM, Steve Davis wrote: >> >> ?...and the following is not appropriate to your needs? >>> >>> Responses = [gen_server:call(Pid,Request) || Pid <- PidList] >>> >> >> That would serialize the calls to the gen_server processes, not taking >> advantage of the parallel nature of erlang >> How about using rpc:pmap or rpc:parallel_eval? >> >> Mihai >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > From rapsey@REDACTED Thu Oct 1 10:45:46 2009 From: rapsey@REDACTED (Rapsey) Date: Thu, 1 Oct 2009 10:45:46 +0200 Subject: [erlang-questions] Switching between http/non-http depending of first byte In-Reply-To: References: Message-ID: <97619b170910010145u3bd266ccuea5358469ad5c437@mail.gmail.com> I've thought about the same thing for my streaming server. But I didn't decide to do it that way. Are you sure you want to put RTMP and HTTP on port 80? It means you'll be running your Erlang server as root, which is not exactly safe. It also means load balancers/reverse proxies, software or hardware based are not going to work. This is why RTMPT was invented (http tunneling for RTMP). Also do you need a full blown web server like yaws or mochiweb? It may be easier to just implement the HTTP features you need manually. It's pretty easy to do in Erlang and It's what I did. Sergej On Thu, Oct 1, 2009 at 9:50 AM, Max Lapshin wrote: > Hi. I want to implement an unusual feature: when server accepts socket, > it must look at first byte. If it is 0x03, than I want to handle it by > one handler (RTMP), > if other, than HTTP handler. > > Problem is that I need only one byte and received packet should be > passed to exact protocol handler. > > I understand, that I will not be able to use erlang setopt(http) > feature, so question is the following: > > should I better use Yaws or MochiWeb as underlying webserver? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From max.lapshin@REDACTED Thu Oct 1 10:56:13 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 1 Oct 2009 12:56:13 +0400 Subject: [erlang-questions] Switching between http/non-http depending of first byte In-Reply-To: <97619b170910010145u3bd266ccuea5358469ad5c437@mail.gmail.com> References: <97619b170910010145u3bd266ccuea5358469ad5c437@mail.gmail.com> Message-ID: On Thu, Oct 1, 2009 at 12:45 PM, Rapsey wrote: > I've thought about the same thing for my streaming server. But I didn't > decide to do it that way. > Are you sure you want to put RTMP and HTTP on port 80? It means you'll be > running your Erlang server as root, which is not exactly safe. It also means > load balancers/reverse proxies, software or hardware based are not going to > work. No, I need just an ad-hoc solution for streaming server to tell via its http interface about its contents. I.e. GET http://erlyvideo.maxidoors.ru/videos.xml will be rewritten via nginx to GET http://erlyvideo.maxidoors.ru:1935/videos.xml > This is why RTMPT was invented (http tunneling for RTMP). RTMPT is quite another thing (and if possible, we try to avoid it). I need not to tunnel streaming in HTTP. > > Also do you need a full blown web server like yaws or mochiweb? It may be > easier to just implement the HTTP features you need manually. It's pretty > easy to do in Erlang and It's what I did. > But I think, it will be easier not to damage my brains and just take next port =) BTW: Your server is opensource or private one? My repository of erlyvideo is on http://github.com/maxlapshin/erlyvideo From vladdu55@REDACTED Thu Oct 1 11:02:10 2009 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 1 Oct 2009 11:02:10 +0200 Subject: [erlang-questions] Re: Need parallel calls to gen_server:call, similar but different to gen_server:multi_call In-Reply-To: <9b08084c0910010109w4915096dg31b83e5b529f0d2c@mail.gmail.com> References: <4AC3DD87.6000309@cs.otago.ac.nz> <3d1e6513-a8c5-48da-bc3c-e7e8e3da770b@j39g2000yqh.googlegroups.com> <78765CB3-951D-4A27-B931-AC1FFA2E2A89@hates.ms> <3dbc6d1c0909301836n75b94064g927de26da0caef1d@mail.gmail.com> <9b08084c0910010109w4915096dg31b83e5b529f0d2c@mail.gmail.com> Message-ID: <95be1d3b0910010202h6c112f32v5df85173ebb72560@mail.gmail.com> On Thu, Oct 1, 2009 at 10:09, Joe Armstrong wrote: > We should also probably have some abstractions for concurrent pipes. > > ? ?pipe([F1, F2, F3, ..., Fn]) > > in which spawn(F1) ?starts a process which sends messages to a process > started by > spawn(F2) ... which are consumed by a process that is started by spawn(F3) > > The trick would be to handle the queue sizes between the steps in the pipeline > and deal with out of band data and exceptions. > > A naive implementation of pipe is easy but to avoid buffer overruns > would be inefficient. > (consider the case where F1 produces messages extremely quickly, but > F2 is very slow :-) I wrote a (half-naive and probably over-engineered) implementation of pipes for some year ago. It does something similar to the backpressure mechanism Joe mentioned, by providing buffering and automatically suspending producers if consumer queue is full. The code is at http://code.google.com/p/erl-pipes/. The problem with the implementation is that a pipe is implemented as two processes: one is handling the flow control but messages go through it and there is a lot of unneeded copying of messages. If flow control would exist at VM-level, the throughput would be much higher. Code example with current API, counting the number of lines in a file containing "=" and the number of lines that don't contain it (cat lines, etc are predefined pipe segments, just like in the unix toolkit). This example might or might not work (but I think it does). P1 = pipes:pipe(cat, [FileName]), P2 = pipes:pipe(lines), P4 = pipes:pipe(grep, ["="]), P3 = pipes:pipe(count), P5 = pipes:pipe(count), pipes:connect([P1, P2, P4, P3]), pipes:connect({P4, nomatch}, P5), {WithEquals, WithoutEquals} = {pipes:get_result(P3), pipes:get_result(P5)}. A DSL for building pipes is called for, as the above is way too inconvenient. Maybe something similar to the Unix-ish {WithEquals, WithoutEquals} = pipes:build( " cat $FileName | lines | g: grep \"=\" | count > $1 ", " :g/nomatch | count > $2 " ). best regards, Vlad From rapsey@REDACTED Thu Oct 1 11:23:22 2009 From: rapsey@REDACTED (Rapsey) Date: Thu, 1 Oct 2009 11:23:22 +0200 Subject: [erlang-questions] Switching between http/non-http depending of first byte In-Reply-To: References: <97619b170910010145u3bd266ccuea5358469ad5c437@mail.gmail.com> Message-ID: <97619b170910010223g50aa6e20l15a0fdec2a55bf26@mail.gmail.com> If you did RTMP, implementing the HTTP features you need is literally childs play. My server is private. Pretty complete though, supports RTMP/RTMPT/RTMPE/RTMPTE, RTSP/RTP and HTTP for live web streaming (can also stream to iptv). Sergej On Thu, Oct 1, 2009 at 10:56 AM, Max Lapshin wrote: > On Thu, Oct 1, 2009 at 12:45 PM, Rapsey wrote: > > I've thought about the same thing for my streaming server. But I didn't > > decide to do it that way. > > Are you sure you want to put RTMP and HTTP on port 80? It means you'll be > > running your Erlang server as root, which is not exactly safe. It also > means > > load balancers/reverse proxies, software or hardware based are not going > to > > work. > > No, I need just an ad-hoc solution for streaming server to tell via > its http interface about > its contents. I.e. GET http://erlyvideo.maxidoors.ru/videos.xml will > be rewritten via nginx > to GET http://erlyvideo.maxidoors.ru:1935/videos.xml > > > > This is why RTMPT was invented (http tunneling for RTMP). > > RTMPT is quite another thing (and if possible, we try to avoid it). I > need not to tunnel > streaming in HTTP. > > > > > Also do you need a full blown web server like yaws or mochiweb? It may be > > easier to just implement the HTTP features you need manually. It's pretty > > easy to do in Erlang and It's what I did. > > > > But I think, it will be easier not to damage my brains and just take > next port =) > > BTW: Your server is opensource or private one? My repository of > erlyvideo is on http://github.com/maxlapshin/erlyvideo > From joelr1@REDACTED Thu Oct 1 12:45:55 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 11:45:55 +0100 Subject: trading systems Message-ID: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> Has anyone used Erlang to backtest trading systems? This basically involves loading price and timestamp pairs and iterating over them calculating various things like moving averages, correlation, etc. Off the top of my head, Erlang does not seem to be the most efficient platform for this because prices would need to be stuffed into ETS or copied around otherwise. The most efficient way to manage price data is to simply keep appending quotes to the end of an mmap-ed file. I would love to represent the mmap-ed memory as a binary and I think it may even be possible. It would require some tricky work at the driver level, though. What do you think? Thanks, Joel --- fastest mac firefox! http://wagerlabs.com From rtrlists@REDACTED Thu Oct 1 12:58:10 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 1 Oct 2009 11:58:10 +0100 Subject: [erlang-questions] trading systems In-Reply-To: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> Message-ID: <6a3ae47e0910010358m49596d42h549fc5585b77d4c0@mail.gmail.com> On Thu, Oct 1, 2009 at 11:45 AM, Joel Reymont wrote: > Has anyone used Erlang to backtest trading systems? > > This basically involves loading price and timestamp pairs > and iterating over them calculating various things like > moving averages, correlation, etc. > > Off the top of my head, Erlang does not seem to be the most > efficient platform for this because prices would need to be > stuffed into ETS or copied around otherwise. > > The most efficient way to manage price data is to simply keep > appending quotes to the end of an mmap-ed file. I would love > to represent the mmap-ed memory as a binary and I think it may > even be possible. It would require some tricky work at the > driver level, though. > > What do you think? > > Thanks, Joel > > You'll want a column based data store, K comes to mind (http://www.kx.com/). If you want something a little bit more open, look at J ( http://www.jsoftware.com/) and it's JDB effort. Staying with the one letter theme (must be an in joke of stats heads, methinks), there's also S and R. Good look searching for those ;-) Robby From rtrlists@REDACTED Thu Oct 1 12:59:46 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 1 Oct 2009 11:59:46 +0100 Subject: [erlang-questions] trading systems In-Reply-To: <6a3ae47e0910010358m49596d42h549fc5585b77d4c0@mail.gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <6a3ae47e0910010358m49596d42h549fc5585b77d4c0@mail.gmail.com> Message-ID: <6a3ae47e0910010359l37c7e400m93ac0f859553e86@mail.gmail.com> On Thu, Oct 1, 2009 at 11:58 AM, Robert Raschke wrote: > > On Thu, Oct 1, 2009 at 11:45 AM, Joel Reymont wrote: > >> Has anyone used Erlang to backtest trading systems? >> >> This basically involves loading price and timestamp pairs >> and iterating over them calculating various things like >> moving averages, correlation, etc. >> >> Off the top of my head, Erlang does not seem to be the most >> efficient platform for this because prices would need to be >> stuffed into ETS or copied around otherwise. >> >> The most efficient way to manage price data is to simply keep >> appending quotes to the end of an mmap-ed file. I would love >> to represent the mmap-ed memory as a binary and I think it may >> even be possible. It would require some tricky work at the >> driver level, though. >> >> What do you think? >> >> Thanks, Joel >> >> > You'll want a column based data store, K comes to mind (http://www.kx.com/ > ). > > If you want something a little bit more open, look at J ( > http://www.jsoftware.com/) and it's JDB effort. > > Staying with the one letter theme (must be an in joke of stats heads, > methinks), there's also S and R. Good look searching for those ;-) > > Robby > > Sorry, meant to say "have a look at ... for inspiration". Robby From joelr1@REDACTED Thu Oct 1 13:01:29 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 12:01:29 +0100 Subject: [erlang-questions] trading systems In-Reply-To: <6a3ae47e0910010358m49596d42h549fc5585b77d4c0@mail.gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <6a3ae47e0910010358m49596d42h549fc5585b77d4c0@mail.gmail.com> Message-ID: On Oct 1, 2009, at 11:58 AM, Robert Raschke wrote: > You'll want a column based data store, K comes to mind (http://www.kx.com/ > ). I have this one and actually know how to use it. > If you want something a little bit more open, look at J (http://www.jsoftware.com/ > ) and it's JDB effort. A nice free replacement. > Staying with the one letter theme (must be an in joke of stats > heads, methinks), there's also S and R. Good look searching for > those ;-) Searching for QuantMod is easier. That's a module that provides trading tools for R, including charting. Still, I'd like to know how this can be done with Erlang, as optimally as possible. And no, I don't think I want to use byte arrays, gb_trees or the array module ;). Thanks, Joel --- fastest mac firefox! http://wagerlabs.com From manprog@REDACTED Thu Oct 1 12:52:10 2009 From: manprog@REDACTED (Bernardo Alvez) Date: Thu, 01 Oct 2009 07:52:10 -0300 Subject: Better use of io:format and io_lib:format Message-ID: <4AC489DA.5080203@adinet.com.uy> Hi all, Which of the following is better to use in terms of performance and resources usage?: io:format("hello ~s", ["world"]). or io:format(<<"hello ~s">>, [<<"world">>]). Thanks, Bernardo From rtrlists@REDACTED Thu Oct 1 13:23:59 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 1 Oct 2009 12:23:59 +0100 Subject: [erlang-questions] trading systems In-Reply-To: References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <6a3ae47e0910010358m49596d42h549fc5585b77d4c0@mail.gmail.com> Message-ID: <6a3ae47e0910010423l38311086h4964e6ebaf9a540c@mail.gmail.com> On Thu, Oct 1, 2009 at 12:01 PM, Joel Reymont wrote: > > On Oct 1, 2009, at 11:58 AM, Robert Raschke wrote: > > You'll want a column based data store, K comes to mind ( >> http://www.kx.com/). >> > > I have this one and actually know how to use it. > > If you want something a little bit more open, look at J ( >> http://www.jsoftware.com/) and it's JDB effort. >> > > A nice free replacement. > > Staying with the one letter theme (must be an in joke of stats heads, >> methinks), there's also S and R. Good look searching for those ;-) >> > > Searching for QuantMod is easier. That's a module that provides trading > tools for R, including charting. > > Still, I'd like to know how this can be done with Erlang, as optimally as > possible. And no, I don't think I want to use byte arrays, gb_trees or the > array module ;). > > > Shooting from the hip, how about files containing Erlang binaries for fixed index ranges. So, per column have a series of files col0, col1000000, col2000000, ... If you're on 64bit, then mmapping whole columns without segmentation might be an option (that's how J does it). Each column contains fixed size data values. And then have an access layer with potential caching. Speedy things needed would be the loading of a file to memory and sequential access through the data. If live update is important, you'll want zero cost append. And over the Erlang binaries containing your fixed sized data you can implement map and fold operations. Easy. Robby From ttmrichter@REDACTED Thu Oct 1 14:55:29 2009 From: ttmrichter@REDACTED (Michael Richter) Date: Thu, 1 Oct 2009 20:55:29 +0800 Subject: [erlang-questions] Better use of io:format and io_lib:format In-Reply-To: <4AC489DA.5080203@adinet.com.uy> References: <4AC489DA.5080203@adinet.com.uy> Message-ID: You could try an experiment, maybe? File: "output": #! /usr/bin/env escript %% -mode(compile). main(["list"]) -> M1 = erlang:process_info(self(), memory), T1 = erlang:now(), list_loop(500), T2 = erlang:now(), M2 = erlang:process_info(self(), memory), io:format("~n~n~w , ~w , ~w~n~n", [M1, M2, timer:now_diff(T2,T1)]); main(["binary"]) -> M1 = erlang:process_info(self(), memory), T1 = erlang:now(), binary_loop(500), T2 = erlang:now(), M2 = erlang:process_info(self(), memory), io:format("~w , ~w , ~w~n", [M1, M2, timer:now_diff(T2,T1)]); main(_) -> io:format("Try again.~n"). list_loop(0) -> io:format("Done!~n~n"); list_loop(N) -> io:format("hello ~s", ["world"]), list_loop(N-1). binary_loop(0) -> io:format(<<"Done!~n~n">>); binary_loop(N) -> io:format(<<"hello ~s">>, [<<"world">>]), binary_loop(N-1). Result set from several runs of "clear ; ./output list": - {memory,54492} , {memory,33820} , 104308 - {memory,54492} , {memory,33820} , 103686 - {memory,54492} , {memory,33820} , 100892 - {memory,54492} , {memory,33820} , 87483 - {memory,54492} , {memory,33820} , 99644 Result set from several runs of "clear ; ./output binary": - {memory,44156} , {memory,33820} , 94222 - {memory,44156} , {memory,33820} , 110587 - {memory,44156} , {memory,33820} , 115329 - {memory,44156} , {memory,33820} , 111326 - {memory,44156} , {memory,33820} , 102580 I make no representation that my code is any good (indeed it's the opposite), but a brief eyeball of the results shows that binaries use up a little less memory and a little more CPU. I'm certain the other, better Erlang users can give you better benchmarking code to answer your question (but so can you, and your benchmarking code would reflect your intended use). 2009/10/1 Bernardo Alvez > Hi all, > > Which of the following is better to use in terms of performance and > resources usage?: > > io:format("hello ~s", ["world"]). > or > io:format(<<"hello ~s">>, [<<"world">>]). > > Thanks, > Bernardo > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From dmercer@REDACTED Thu Oct 1 16:14:53 2009 From: dmercer@REDACTED (David Mercer) Date: Thu, 1 Oct 2009 09:14:53 -0500 Subject: [erlang-questions] Re: Need parallel calls to gen_server:call, similar but different to gen_server:multi_call In-Reply-To: <3dbc6d1c0909301836n75b94064g927de26da0caef1d@mail.gmail.com> References: <4AC3DD87.6000309@cs.otago.ac.nz> <3d1e6513-a8c5-48da-bc3c-e7e8e3da770b@j39g2000yqh.googlegroups.com> <78765CB3-951D-4A27-B931-AC1FFA2E2A89@hates.ms> <3dbc6d1c0909301836n75b94064g927de26da0caef1d@mail.gmail.com> Message-ID: Robert Virding wrote: > One alternative would be to use gen_server:cast > to > send and then handle the replies explicitly yourself. Another alternative might be to enhance the gen_server API, adding in gen_server:async_call/2 which returns {Node, Mref}, and gen_server:sync/1 that takes the {Node, Mref} and returns the Reply that would have been received from gen_server:call/2. This shouldn't be too too difficult. In fact, you would probably then implement gen:call using a gen:async_call followed directly by a gen:sync. Of course, this doesn't give Cameron what he needs NOW... Cheers, David From daveb@REDACTED Thu Oct 1 16:17:41 2009 From: daveb@REDACTED (Dave Bryson) Date: Thu, 1 Oct 2009 09:17:41 -0500 Subject: Network communications fails with R13B02-1 on OS X Message-ID: <0A1C45D7-8BAA-4A08-BF32-7D237DE7DD84@miceda.org> Yesterday I upgraded to otp_13B02-1 from R12-5 on OS X 10.5.8. I notice today that any form of network communication between nodes seems to fail. Trying to start slaves on the same host (which I was able to do before the upgrade) fails with an {error,timeout}. Even doing a simple 'net:ping(node1@REDACTED)' on the same host between two nodes fails with a pang. I can run the same exact tests on Ubuntu with R13B02-1 and they pass with flying colors. Has anyone else noticed this problem on OS X? Any ideas what the problem may be? Thanks! Dave From joelr1@REDACTED Thu Oct 1 17:02:57 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 16:02:57 +0100 Subject: [erlang-questions] Network communications fails with R13B02-1 on OS X In-Reply-To: <0A1C45D7-8BAA-4A08-BF32-7D237DE7DD84@miceda.org> References: <0A1C45D7-8BAA-4A08-BF32-7D237DE7DD84@miceda.org> Message-ID: Dave, You haven't modified __DARWIN_FD_SETSIZE in sys/_structs.h, have you? What is your system ulimit -n setting? I know for a fact that if you change the system-wide FD_SETSIZE via _structs.h then epmd and ssl_esock stop working when ulimit -n is high enough. I used to start epmd manually but then resorted to compiling OTP with a high system FD_SETSIZE and then recompiling just ssl_esock and epmd with a low setting. I would bet epmd is the issue, try starting it manually in the non- demon mode and see what it says. On Oct 1, 2009, at 3:17 PM, Dave Bryson wrote: > Yesterday I upgraded to otp_13B02-1 from R12-5 on OS X 10.5.8. I > notice today that any form of network communication between nodes > seems to fail. Trying to start slaves on the same host (which I was > able to do before the upgrade) fails with an {error,timeout}. Even > doing a simple 'net:ping(node1@REDACTED)' on the same host between > two nodes fails with a pang. I can run the same exact tests on > Ubuntu with R13B02-1 and they pass with flying colors. Has anyone > else noticed this problem on OS X? Any ideas what the problem may be? > > Thanks! > Dave > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > --- fastest mac firefox! http://wagerlabs.com From kenneth.lundin@REDACTED Thu Oct 1 17:50:24 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Thu, 1 Oct 2009 17:50:24 +0200 Subject: [erlang-questions] trading systems In-Reply-To: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> Message-ID: Hi Joel, As usual you are jumping right into low level optimizations without explaining why that would be necessary:). It would be nice if you could explain what the system needs to do and after that why you think it is a problem to store prices in an ETS-table. I am sure there are other options as well. At least for me it is not obvious what performance that is needed for an application like this. And pure performance might not be the only reason to use Erlang. It can also be that it is so very simple to implement and maintain the solution and that is scales very well over multiple cores without extra work for the programmer. Why are you talking about mmapped files? Please explain for a novice in trading systems. /Kenneth On Thu, Oct 1, 2009 at 12:45 PM, Joel Reymont wrote: > Has anyone used Erlang to backtest trading systems? > > This basically involves loading price and timestamp pairs > and iterating over them calculating various things like > moving averages, correlation, etc. > > Off the top of my head, Erlang does not seem to be the most > efficient platform for this because prices would need to be > stuffed into ETS or copied around otherwise. > > The most efficient way to manage price data is to simply keep > appending quotes to the end of an mmap-ed file. I would love > to represent the mmap-ed memory as a binary and I think it may > even be possible. It would require some tricky work at the > driver level, though. > > What do you think? > > ? ? ? ?Thanks, Joel > > --- > fastest mac firefox! > http://wagerlabs.com > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From joelr1@REDACTED Thu Oct 1 18:04:12 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 17:04:12 +0100 Subject: [erlang-questions] trading systems In-Reply-To: References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> Message-ID: <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> Kenneth, On Oct 1, 2009, at 4:50 PM, Kenneth Lundin wrote: > It would be nice if you could explain what the system needs to do > and after that > why you think it is a problem to store prices in an ETS-table. > [...] > Why are you talking about mmapped files? Please explain for a novice > in trading systems. It's high-frequency trading systems I'm talking about. Where's Serge Aleynikov [1] when you need him? I'd like to process an incoming price quote, make a decision and submit a trade order with minimal latency. I know that this is normally done using C++ (or OCaml [1]) but I'm wondering how this can be achieved using Erlang. Minimal latency is usually achieved by hitting the CPU cache as frequently as possible and minimizing memory copying overhead. ETS copies memory and I suspect it's use of the cache is rather poor by virtue of being a "hash table" and pointing all over the place in memory. A simple array of floats or doubles groups data together in memory and is very cache friendly. I figure that an Erlang binary is the closest I can get to an array in Erlang. mmap-ing a file of floats or doubles and wrapping it in a binary should avoid unnecessary copying of data from disk to memory. I think operating on binaries representing mmap-ed files of price quotes is as close as you can get to low latency in Erlang. I know that premature optimization is the root of all evil but I'd rather have my teeth pulled than optimize Erlang. This is speaking from experience. I figure a much better way is to choose a concept that guarantees fast code from the start. Thanks, Joel [1] http://www.puppetmastertrading.com/blog/2009/07/08/the-other-interesting-thing-about-the-serge-aleynikov-story/ --- fastest mac firefox! http://wagerlabs.com From andrew@REDACTED Thu Oct 1 18:40:39 2009 From: andrew@REDACTED (Andrew Thompson) Date: Thu, 1 Oct 2009 12:40:39 -0400 Subject: [erlang-questions] trading systems In-Reply-To: <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> Message-ID: <20091001164039.GE18744@hijacked.us> On Thu, Oct 01, 2009 at 05:04:12PM +0100, Joel Reymont wrote: > Kenneth, > > On Oct 1, 2009, at 4:50 PM, Kenneth Lundin wrote: > > >It would be nice if you could explain what the system needs to do > >and after that > >why you think it is a problem to store prices in an ETS-table. > >[...] > >Why are you talking about mmapped files? Please explain for a novice > >in trading systems. > > It's high-frequency trading systems I'm talking about. Where's Serge > Aleynikov [1] when you need him? > > I'd like to process an incoming price quote, make a decision and > submit a trade order with minimal latency. I know that this is > normally done using C++ (or OCaml [1]) but I'm wondering how this can > be achieved using Erlang. > > Minimal latency is usually achieved by hitting the CPU cache as > frequently as possible and minimizing memory copying overhead. ETS > copies memory and I suspect it's use of the cache is rather poor by > virtue of being a "hash table" and pointing all over the place in > memory. > > A simple array of floats or doubles groups data together in memory and > is very cache friendly. I figure that an Erlang binary is the closest > I can get to an array in Erlang. mmap-ing a file of floats or doubles > and wrapping it in a binary should avoid unnecessary copying of data > from disk to memory. > > I think operating on binaries representing mmap-ed files of price > quotes is as close as you can get to low latency in Erlang. I know > that premature optimization is the root of all evil but I'd rather > have my teeth pulled than optimize Erlang. This is speaking from > experience. I figure a much better way is to choose a concept that > guarantees fast code from the start. > There's always the undocumented hipe bit and bytearrays (which are mutable) if you're willing to sacrifice everything on the altar of performance. http://erlang.org/pipermail/erlang-questions/2009-April/043460.html Andrew From pfisher@REDACTED Thu Oct 1 18:15:51 2009 From: pfisher@REDACTED (Paul Fisher) Date: Thu, 01 Oct 2009 11:15:51 -0500 Subject: [erlang-questions] trading systems In-Reply-To: <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> Message-ID: <4AC4D5B7.2040302@alertlogic.net> Food for thought: http://www.acm.org/ubiquity/volume_10/v10i3_hyde.html Joel Reymont wrote: > Kenneth, > > On Oct 1, 2009, at 4:50 PM, Kenneth Lundin wrote: > >> It would be nice if you could explain what the system needs to do >> and after that >> why you think it is a problem to store prices in an ETS-table. >> [...] >> Why are you talking about mmapped files? Please explain for a novice >> in trading systems. > > It's high-frequency trading systems I'm talking about. Where's Serge > Aleynikov [1] when you need him? > > I'd like to process an incoming price quote, make a decision and > submit a trade order with minimal latency. I know that this is > normally done using C++ (or OCaml [1]) but I'm wondering how this can > be achieved using Erlang. > > Minimal latency is usually achieved by hitting the CPU cache as > frequently as possible and minimizing memory copying overhead. ETS > copies memory and I suspect it's use of the cache is rather poor by > virtue of being a "hash table" and pointing all over the place in > memory. > > A simple array of floats or doubles groups data together in memory and > is very cache friendly. I figure that an Erlang binary is the closest > I can get to an array in Erlang. mmap-ing a file of floats or doubles > and wrapping it in a binary should avoid unnecessary copying of data > from disk to memory. > > I think operating on binaries representing mmap-ed files of price > quotes is as close as you can get to low latency in Erlang. I know > that premature optimization is the root of all evil but I'd rather > have my teeth pulled than optimize Erlang. This is speaking from > experience. I figure a much better way is to choose a concept that > guarantees fast code from the start. > > Thanks, Joel > > [1] http://www.puppetmastertrading.com/blog/2009/07/08/the-other-interesting-thing-about-the-serge-aleynikov-story/ > > --- > fastest mac firefox! > http://wagerlabs.com > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From joelr1@REDACTED Thu Oct 1 18:42:47 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 17:42:47 +0100 Subject: [erlang-questions] trading systems In-Reply-To: <20091001164039.GE18744@hijacked.us> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <20091001164039.GE18744@hijacked.us> Message-ID: <42E6741C-0A99-4F0C-B6D2-9FC50E20DDA8@gmail.com> On Oct 1, 2009, at 5:40 PM, Andrew Thompson wrote: > There's always the undocumented hipe bit and bytearrays (which are > mutable) if you're willing to sacrifice everything on the altar of > performance. You cannot store doubles in a bytearray from what I know. Also, you need to populate the byte array which involves loading data from disk into buffers, into erlang, etc. mmap for the win! --- fastest mac firefox! http://wagerlabs.com From andrew@REDACTED Thu Oct 1 18:43:19 2009 From: andrew@REDACTED (Andrew Thompson) Date: Thu, 1 Oct 2009 12:43:19 -0400 Subject: [erlang-questions] trading systems In-Reply-To: <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> Message-ID: <20091001164318.GA25338@hijacked.us> Crap, group replied instead of list replied... On Thu, Oct 01, 2009 at 05:04:12PM +0100, Joel Reymont wrote: > Kenneth, > > On Oct 1, 2009, at 4:50 PM, Kenneth Lundin wrote: > > >It would be nice if you could explain what the system needs to do > >and after that > >why you think it is a problem to store prices in an ETS-table. > >[...] > >Why are you talking about mmapped files? Please explain for a novice > >in trading systems. > > It's high-frequency trading systems I'm talking about. Where's Serge > Aleynikov [1] when you need him? > > I'd like to process an incoming price quote, make a decision and > submit a trade order with minimal latency. I know that this is > normally done using C++ (or OCaml [1]) but I'm wondering how this can > be achieved using Erlang. > > Minimal latency is usually achieved by hitting the CPU cache as > frequently as possible and minimizing memory copying overhead. ETS > copies memory and I suspect it's use of the cache is rather poor by > virtue of being a "hash table" and pointing all over the place in > memory. > > A simple array of floats or doubles groups data together in memory and > is very cache friendly. I figure that an Erlang binary is the closest > I can get to an array in Erlang. mmap-ing a file of floats or doubles > and wrapping it in a binary should avoid unnecessary copying of data > from disk to memory. > > I think operating on binaries representing mmap-ed files of price > quotes is as close as you can get to low latency in Erlang. I know > that premature optimization is the root of all evil but I'd rather > have my teeth pulled than optimize Erlang. This is speaking from > experience. I figure a much better way is to choose a concept that > guarantees fast code from the start. > There's always the undocumented hipe bit and bytearrays (which are mutable) if you're willing to sacrifice everything on the altar of performance. http://erlang.org/pipermail/erlang-questions/2009-April/043460.html Andrew From arturmatos78@REDACTED Thu Oct 1 19:06:28 2009 From: arturmatos78@REDACTED (Artur Matos) Date: Thu, 1 Oct 2009 18:06:28 +0100 Subject: trading systems Message-ID: Hi Joel, I tried to implement a backtesting/execution system in Erlang a few years ago, but eventually gave up and moved to Java. I was not trading intraday at that time (just using daily data) so performance was never much of an issue. My biggest problem with Erlang is that it was lacking essential libraries, and I guess this is still pretty much a problem today. Of the top of my head: - Converting dates between different timezones - I needed this because I trade in several exchanges around the world (NYSE, London, ...). You can kinda do this in Erlang by setting the TZ environment variable before calling date functions, but it is hacky and I could never put it to work. - Fixed point arithmetic, for things like money. You don't need that for share prices and such, but you will need it at least in a few places if you care for accuracy, like for computing PnLs and such. This is not so much of an issue when you are backtesting, but at least in my case I want stuff to be as accurate as possible when doing real trading. - Mathematical and scientific libraries. For things like statistical arbitrage you will eventually need to compute more things besides correlations, e.g. to do linear regressions and such. Erlang has no libraries for this. - FIX connectivity, or interfaces for standard brokers, like Interactive Brokers. - Graphical user interfaces. Implementing them in pure Erlang doesn't make much sense because it is just too limited (although now you have bindings for wx, so it might be better now). - Charts. Even if you are not doing technical analysis, you still need to generate simple charts like equity curves and such when backtesting. Most of these things can be implemented in Erlang and some of them are not even that hard, but it slows you down considerably. And of course you can always write parts of your application in other languages and interface them with Erlang, but that is quite cumbersome as well. I eventually went down that way - I implemented parts of my application in Java - but then I got to a point where it doesn't make much sense to use Erlang anymore. If you are using Java for the GUI, and Java for talking with the broker, and Java for generating reports, what are you using Erlang for? I could easily be much more productive in a language like Java that has libraries for everything, even if Java is such a pain to use. Of course, I would much prefer to use Erlang, but at least for this project the lack of libraries was really a problem. Things might be better now there is the new FFI interface and such, but for now I would just avoid Erlang for trading. Of course you might have a different set of requirements - this might not apply to you if you just want to implement a simple trading bot, and have another system in place for managing orders, executions, etc... Artur. On Oct 1, 11:45 am, Joel Reymont wrote: > Has anyone used Erlang to backtest trading systems? > > This basically involves loading price and timestamp pairs > and iterating over them calculating various things like > moving averages, correlation, etc. > > Off the top of my head, Erlang does not seem to be the most > efficient platform for this because prices would need to be > stuffed into ETS or copied around otherwise. > > The most efficient way to manage price data is to simply keep > appending quotes to the end of an mmap-ed file. I would love > to represent the mmap-ed memory as a binary and I think it may > even be possible. It would require some tricky work at the > driver level, though. > > What do you think? > > Thanks, Joel > > --- > fastest mac firefox!http://wagerlabs.com > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From joelr1@REDACTED Thu Oct 1 19:23:11 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 18:23:11 +0100 Subject: [erlang-questions] Re: trading systems In-Reply-To: References: Message-ID: <759F579A-5242-4CE3-A771-C478B810E212@gmail.com> On Oct 1, 2009, at 6:06 PM, Artur Matos wrote: > Things might be better now there is the new FFI interface and such, Is there a new FFI interface? If you are talking about loadable BIFs then I would very much like to make use of it. > Of course you might have a different set of requirements - this might > not apply to you if you just want to implement a simple trading bot, > and have another system in place for managing orders, executions, > etc... The platform I'm thinking about is rather simple. Consider EasyLanguage (EL) for example [1]. This is a Pascal-like language, the Basic of trading languages judging by popularity. It's the language used by TradeStation [2]. It's a neat and very capable language but using it requires you to babysit your desktop. You cannot put EL on the server for unattended but monitored trading. The language itself does not have heavy-duty statistics or anything of the sort. It does have a bunch of technical indicators like moving averages. In fact, most if not all the indicators are implemented in EL rather than built in. I have already written compilers from EL to C# in Haskell, OCaml and Lisp [3]. I plan to write yet another compiler, this time in and to Erlang. I then plan to sell this "black box" execution system to brokerage houses and enable them to offer a value-added service to their clients. Server-side (co-located?) execution does not need any graphics or charting, although a web interface would come in extremely handy. [1] http://lambda-the-ultimate.org/node/2201 [2] http://www.tradestation.com [3] http://groups.google.com/group/topdog/browse_thread/thread/d5fd8feac8e68331 --- fastest mac firefox! http://wagerlabs.com From daveb@REDACTED Thu Oct 1 20:00:34 2009 From: daveb@REDACTED (Dave Bryson) Date: Thu, 1 Oct 2009 13:00:34 -0500 Subject: Network communications fails with R13B02-1 on OS X References: <0A1C45D7-8BAA-4A08-BF32-7D237DE7DD84@miceda.org> Message-ID: <943D76C7-57F1-45FD-A7FA-F0EB6F517864@miceda.org> After more debugging I'm still baffled as to why two nodes cannot communicate on the same host. I started epmd with the debug flag and see the nodes are registering but the net:ping does not seem to make the PORT request to epmd: epmd: Thu Oct 1 12:51:25 2009: epmd running - daemon = 0 epmd: Thu Oct 1 12:51:31 2009: ** got ALIVE2_REQ epmd: Thu Oct 1 12:51:31 2009: registering 'n1:2', port 49353 epmd: Thu Oct 1 12:51:31 2009: type 77 proto 0 highvsn 5 lowvsn 5 epmd: Thu Oct 1 12:51:31 2009: ** sent ALIVE2_RESP for "n1" epmd: Thu Oct 1 12:51:37 2009: ** got ALIVE2_REQ epmd: Thu Oct 1 12:51:37 2009: registering 'n2:2', port 49355 epmd: Thu Oct 1 12:51:37 2009: type 77 proto 0 highvsn 5 lowvsn 5 epmd: Thu Oct 1 12:51:37 2009: ** sent ALIVE2_RESP for "n2" Note: epmd is NOT showing a PORT2_REQ from the net:ping(). I know the code I'm testing with is correct because it works on Ubuntu. Does anyone have ANY idea why this is happening?? Thanks in advance! Dave Begin forwarded message: > From: Dave Bryson > Date: October 1, 2009 9:17:41 AM CDT > To: erlang-questions@REDACTED > Subject: Network communications fails with R13B02-1 on OS X > > Yesterday I upgraded to otp_13B02-1 from R12-5 on OS X 10.5.8. I > notice today that any form of network communication between nodes > seems to fail. Trying to start slaves on the same host (which I was > able to do before the upgrade) fails with an {error,timeout}. Even > doing a simple 'net:ping(node1@REDACTED)' on the same host between > two nodes fails with a pang. I can run the same exact tests on > Ubuntu with R13B02-1 and they pass with flying colors. Has anyone > else noticed this problem on OS X? Any ideas what the problem may be? > > Thanks! > Dave From paul-trapexit@REDACTED Thu Oct 1 20:11:44 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Thu, 1 Oct 2009 11:11:44 -0700 (PDT) Subject: [erlang-questions] Re: trading systems In-Reply-To: <759F579A-5242-4CE3-A771-C478B810E212@gmail.com> References: <759F579A-5242-4CE3-A771-C478B810E212@gmail.com> Message-ID: On Thu, 1 Oct 2009, Joel Reymont wrote: > > On Oct 1, 2009, at 6:06 PM, Artur Matos wrote: > > > Things might be better now there is the new FFI interface and such, > > Is there a new FFI interface? If you are talking about loadable BIFs > then I would very much like to make use of it. While I'm also excited about the new FFI interface, I got really good results (i.e., low latency, high throughput) doing something analogous (interfacing with localmemcache, an mmap based db) from a linked-in driver using port_control and PORT_CONTROL_FLAG_BINARY. So I think you already have what you need to pull this off. -- p From daveb@REDACTED Thu Oct 1 20:27:55 2009 From: daveb@REDACTED (Dave Bryson) Date: Thu, 1 Oct 2009 13:27:55 -0500 Subject: Network communications fails with R13B02-1 on OS X In-Reply-To: <943D76C7-57F1-45FD-A7FA-F0EB6F517864@miceda.org> References: <0A1C45D7-8BAA-4A08-BF32-7D237DE7DD84@miceda.org> <943D76C7-57F1-45FD-A7FA-F0EB6F517864@miceda.org> Message-ID: <41B3BDF5-6183-4644-87DD-7D1EE9EEDDA5@miceda.org> Backing down to otp R 12B-5 the problem disappears and all works as expected. One thing I noticed: Each registering node on 12B-5 (epmd) has a different number AFTER the colon ":" epmd: Thu Oct 1 12:51:31 2009: registering 'n1:2', port 49370 ... epmd: Thu Oct 1 12:51:31 2009: registering 'n2:3', port 49372 Notice 'n1:2' and 'n2:3' Where as on otp R13 below, the number AFTER the colon ':' is the same: Notice below: 'n1:2' and 'n2:2' Could this be the issue? Dave On Oct 1, 2009, at 1:00 PM, Dave Bryson wrote: > After more debugging I'm still baffled as to why two nodes cannot > communicate on the same host. I started epmd with the debug flag > and see the nodes are registering but the net:ping does not seem to > make the PORT request to epmd: > > epmd: Thu Oct 1 12:51:25 2009: epmd running - daemon = 0 > epmd: Thu Oct 1 12:51:31 2009: ** got ALIVE2_REQ > epmd: Thu Oct 1 12:51:31 2009: registering 'n1:2', port 49353 > epmd: Thu Oct 1 12:51:31 2009: type 77 proto 0 highvsn 5 lowvsn 5 > epmd: Thu Oct 1 12:51:31 2009: ** sent ALIVE2_RESP for "n1" > epmd: Thu Oct 1 12:51:37 2009: ** got ALIVE2_REQ > epmd: Thu Oct 1 12:51:37 2009: registering 'n2:2', port 49355 > epmd: Thu Oct 1 12:51:37 2009: type 77 proto 0 highvsn 5 lowvsn 5 > epmd: Thu Oct 1 12:51:37 2009: ** sent ALIVE2_RESP for "n2" > > Note: epmd is NOT showing a PORT2_REQ from the net:ping(). I know > the code I'm testing with is correct because it works on Ubuntu. > > Does anyone have ANY idea why this is happening?? > > Thanks in advance! > Dave > > Begin forwarded message: > >> From: Dave Bryson >> Date: October 1, 2009 9:17:41 AM CDT >> To: erlang-questions@REDACTED >> Subject: Network communications fails with R13B02-1 on OS X >> >> Yesterday I upgraded to otp_13B02-1 from R12-5 on OS X 10.5.8. I >> notice today that any form of network communication between nodes >> seems to fail. Trying to start slaves on the same host (which I was >> able to do before the upgrade) fails with an {error,timeout}. Even >> doing a simple 'net:ping(node1@REDACTED)' on the same host between >> two nodes fails with a pang. I can run the same exact tests on >> Ubuntu with R13B02-1 and they pass with flying colors. Has anyone >> else noticed this problem on OS X? Any ideas what the problem may >> be? >> >> Thanks! >> Dave > From kenneth.lundin@REDACTED Thu Oct 1 20:43:35 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Thu, 1 Oct 2009 20:43:35 +0200 Subject: [erlang-questions] trading systems In-Reply-To: <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> Message-ID: Joel Please explain more. I have also heard about the latency requirements for trading systems but they are still very diffuse to me. I got the impression that latency is a big selling point for trading systems but in reality there might be other factors that are more limiting and makes the hunting for latency under a certain level unnecessary. Please explain more. A backtest system is not run in real time and there should be no absolute requirements on latency? Correct me if I am wrong. On Thu, Oct 1, 2009 at 6:04 PM, Joel Reymont wrote: > Kenneth, > > On Oct 1, 2009, at 4:50 PM, Kenneth Lundin wrote: > >> It would be nice if you could explain what the system needs to do and >> after that >> why you think it is a problem to store prices in an ETS-table. >> [...] >> Why are you talking about mmapped files? Please explain for a novice >> in trading systems. > > It's high-frequency trading systems I'm talking about. Where's Serge > Aleynikov [1] when you need him? > > I'd like to process an incoming price quote, make a decision and submit a > trade order with minimal latency. I know that this is normally done using > C++ (or OCaml [1]) but I'm wondering how this can be achieved using Erlang. > You have a server supporting some protocol (probably FIX) , the price quote arrives this way? The decision is taken at the server? The trade order is handled in the server or sent to another system? >From where , to where is the latency measured? >From your reasoning it seems that the encoding/decoding + communication latency is not part of the important latency? In that case why not? Anyway there must be som latency value that is just good enough and going below that is just unnecessary? /Kenneth > > > > From kagato@REDACTED Thu Oct 1 20:49:06 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 1 Oct 2009 11:49:06 -0700 Subject: [erlang-questions] trading systems In-Reply-To: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> Message-ID: I would really recommend starting out with a naive implementation. If anything it will give you something to use as a baseline for comparison and profiling will tell you how much you can eventually save with a more complex approach. Also, play with HIPE. It currently is a bit unstable for general use, but judicious HIPE compilation of key modules might be very helpful. Out of curiosity, what features of Erlang make you want to try this? It would seem that it's made more for fault-tolerance / scalability and less for the absolute lowest-latency. On Oct 1, 2009, at 3:45 AM, Joel Reymont wrote: > Has anyone used Erlang to backtest trading systems? > > This basically involves loading price and timestamp pairs > and iterating over them calculating various things like > moving averages, correlation, etc. > > Off the top of my head, Erlang does not seem to be the most > efficient platform for this because prices would need to be > stuffed into ETS or copied around otherwise. > > The most efficient way to manage price data is to simply keep > appending quotes to the end of an mmap-ed file. I would love > to represent the mmap-ed memory as a binary and I think it may > even be possible. It would require some tricky work at the > driver level, though. > > What do you think? > > Thanks, Joel > > --- > fastest mac firefox! > http://wagerlabs.com > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From joelr1@REDACTED Thu Oct 1 20:56:44 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 19:56:44 +0100 Subject: [erlang-questions] trading systems In-Reply-To: References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> Message-ID: <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> Kenneth, On Oct 1, 2009, at 7:43 PM, Kenneth Lundin wrote: > Please explain more. Feel free to ask pointed questions. > I got the impression that latency is a big selling point for > trading systems Correct. If you can react to the market faster than the next guy then you win. There's even a new kind of high-frequency trading now called latency arbitrage [1]. > but in reality there might be > other factors that are more limiting and makes the hunting for latency > under a certain level unnecessary. There may be other factors but people pull tricks like disabling RTTI in their C++ code, not using exceptions and moving as much as they can to the compilation stage with C++ templates. This is in addition to using cache-friendly and lock-free data structures. No need to put myself at a disadvantage from the start by picking an inefficient approach to price data management and processing. > A backtest system is not run in real time and there should be no > absolute requirements on latency? It's pointless to build one system for backtesting and another one for execution. I'd rather have a single system handle everything. > You have a server supporting some protocol (probably FIX) , the price > quote arrives this way? In my case it's gonna be via a C++ API, over the network. > The decision is taken at the server? Right. > The trade order is handled in the server or sent to another system? The order is sent to the broker via a C++ API, over the network. > From where , to where is the latency measured? The latency I'm talking about is measured from the moment the price quote is received by Erlang to the moment Erlang tells the C++ API to send the trade to the broker. There's also the network latency but it should be minimal for a server co-located at the exchange data center or close to it. > From your reasoning it seems that the encoding/decoding + > communication latency is not part of > the important latency? In that case why not? Encoding and decoding is part of the latency but network latency is not. > > Anyway there must be som latency value that is just good enough and > going below that is just unnecessary? No such thing :D [1]. It's a war out there over who can go down faster. People are embedding FIX adapters and trading strategies into FPGAs for microsecond latency [2]. Incidentally, the HPC Platform software is written in Lisp. It's mostly compilation and VHDL generation, though. Thanks, Joel [1] http://www.google.com/search?client=safari&rls=en&q=latency+arbitrage&ie=UTF-8&oe=UTF-8 [2] http://www.hpcplatform.com/ --- fastest mac firefox! http://wagerlabs.com From kagato@REDACTED Thu Oct 1 21:04:13 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 1 Oct 2009 12:04:13 -0700 Subject: [erlang-questions] trading systems In-Reply-To: References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> Message-ID: In trading systems, timing is everything. Basically, you are writing code that eventually figures into executing trades (sometimes automatically). In the automatic systems, you are literally racing other systems. This particular race is so competitive (and lucrative) that companies build datacenters closer to the trading house just so that they have less latency. It is not unheard of to even tweak the communications timing on their uplinks, use fibre just to shave off a few ms, or even embed assembly code. In extreme cases, people have systems that watch for large buys / sells, predict the follow-on effect it will have on the market, and try to wedge in their own order to capitalize on the inevitable (but lagging) price shifts. In any other industry, this level of optimization would be well into the "diminishing returns" area. However, financial markets operate at a big enough scale this sort of thing is still quite profitable. On Oct 1, 2009, at 11:43 AM, Kenneth Lundin wrote: > Joel > > Please explain more. > I have also heard about the latency requirements for trading systems > but they are still very diffuse > to me. I got the impression that latency is a big selling point for > trading systems but in reality there might be > other factors that are more limiting and makes the hunting for latency > under a certain level unnecessary. > > Please explain more. > > A backtest system is not run in real time and there should be no > absolute requirements on latency? > Correct me if I am wrong. > > > > On Thu, Oct 1, 2009 at 6:04 PM, Joel Reymont wrote: >> Kenneth, >> >> On Oct 1, 2009, at 4:50 PM, Kenneth Lundin wrote: >> >>> It would be nice if you could explain what the system needs to do >>> and >>> after that >>> why you think it is a problem to store prices in an ETS-table. >>> [...] >>> Why are you talking about mmapped files? Please explain for a novice >>> in trading systems. >> >> It's high-frequency trading systems I'm talking about. Where's Serge >> Aleynikov [1] when you need him? >> >> I'd like to process an incoming price quote, make a decision and >> submit a >> trade order with minimal latency. I know that this is normally done >> using >> C++ (or OCaml [1]) but I'm wondering how this can be achieved using >> Erlang. >> > You have a server supporting some protocol (probably FIX) , the price > quote arrives this way? > The decision is taken at the server? > The trade order is handled in the server or sent to another system? > From where , to where is the latency measured? > From your reasoning it seems that the encoding/decoding + > communication latency is not part of > the important latency? In that case why not? > > > Anyway there must be som latency value that is just good enough and > going below that is just unnecessary? > > > /Kenneth >> >> >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From kagato@REDACTED Thu Oct 1 21:07:15 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 1 Oct 2009 12:07:15 -0700 Subject: [erlang-questions] trading systems In-Reply-To: <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> Message-ID: > No need to put myself at a disadvantage from the start by picking an > inefficient approach to price data management and processing. Premature optimization *IS* the root of all evil. Specifically, because you make decisions on optimization in the absence of information. Processor cache is a very unpredictable beast in the face of multiple cores. Trying to predict how it behaves when other code is in the pipeline is unfortunately optimistic. Multicore systems inherently increase latency and decrease cache performance. Erlang is optimized for these. Any information you have regarding cache behavior is highly unlikely to be directly applicable. Test and profile, early and often. It's the only way to go. It is entirely possible (due to the design goals of Erlang) that you might not even be able to wring the speed you need out of it. Get something running, profile it, and fail fast if necessary. Putting 2000 lines of C before running a profiler is not a timesaver--even in the world of cutthroat performance. -- Jayson Vantuyl kagato@REDACTED From mjtruog@REDACTED Thu Oct 1 21:13:28 2009 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 01 Oct 2009 12:13:28 -0700 Subject: [erlang-questions] trading systems In-Reply-To: <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> Message-ID: <4AC4FF58.2090704@gmail.com> One thing you might want to consider before building the system is the cost of the data to actually do the type of trading you want to do. If you are trying to do low latency arbitrage type trading you might want to relocate to New York for obvious reasons. The data costs of a real-time feed with reliable data might be a bit bothersome and would dig into any profit. I think that after you figured out the constraints on the system due to network and data the design would take shape. The design could be as simple as using Tradestation libraries, but any slippage on order fills would be something you would discover after you finished creating the system (and that sort of discovery is negative). - Michael Joel Reymont wrote: > Kenneth, > > On Oct 1, 2009, at 7:43 PM, Kenneth Lundin wrote: > >> Please explain more. > > Feel free to ask pointed questions. > >> I got the impression that latency is a big selling point for >> trading systems > > Correct. If you can react to the market faster than the next guy > then you win. There's even a new kind of high-frequency trading now > called latency arbitrage [1]. > >> but in reality there might be >> other factors that are more limiting and makes the hunting for latency >> under a certain level unnecessary. > > There may be other factors but people pull tricks like disabling RTTI > in their C++ code, not using exceptions and moving as much as they can > to the compilation stage with C++ templates. This is in addition to using > cache-friendly and lock-free data structures. > > No need to put myself at a disadvantage from the start by picking an > inefficient approach to price data management and processing. > >> A backtest system is not run in real time and there should be no >> absolute requirements on latency? > > It's pointless to build one system for backtesting and another one > for execution. I'd rather have a single system handle everything. > >> You have a server supporting some protocol (probably FIX) , the price >> quote arrives this way? > > In my case it's gonna be via a C++ API, over the network. > >> The decision is taken at the server? > > Right. > >> The trade order is handled in the server or sent to another system? > > The order is sent to the broker via a C++ API, over the network. > >> From where , to where is the latency measured? > > The latency I'm talking about is measured from the moment the price > quote is received by Erlang to the moment Erlang tells the C++ API > to send the trade to the broker. > > There's also the network latency but it should be minimal for a > server co-located at the exchange data center or close to it. > >> From your reasoning it seems that the encoding/decoding + >> communication latency is not part of >> the important latency? In that case why not? > > Encoding and decoding is part of the latency but network latency is not. >> >> Anyway there must be som latency value that is just good enough and >> going below that is just unnecessary? > > No such thing :D [1]. It's a war out there over who can go down faster. > People are embedding FIX adapters and trading strategies into FPGAs > for microsecond latency [2]. Incidentally, the HPC Platform software > is written in Lisp. It's mostly compilation and VHDL generation, though. > > Thanks, Joel > > [1] > http://www.google.com/search?client=safari&rls=en&q=latency+arbitrage&ie=UTF-8&oe=UTF-8 > > [2] http://www.hpcplatform.com/ > > --- > fastest mac firefox! > http://wagerlabs.com > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From joelr1@REDACTED Thu Oct 1 21:32:33 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 20:32:33 +0100 Subject: [erlang-questions] trading systems In-Reply-To: References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> Message-ID: <45B769E4-C263-4288-8BDE-E3758B60868F@gmail.com> On Oct 1, 2009, at 7:49 PM, Jayson Vantuyl wrote: > I would really recommend starting out with a naive implementation. > If anything it will give you something to use as a baseline for > comparison and profiling will tell you how much you can eventually > save with a more complex approach. True. I think I'll still start by experimenting with mmap-ed binaries, or at least a binaries-based approach. These can always be mmap-ed later. > Also, play with HIPE. It currently is a bit unstable for general > use, but judicious HIPE compilation of key modules might be very > helpful. Yes, I may try porting HiPE to x86-64 when I have time to spare. > Out of curiosity, what features of Erlang make you want to try > this? It would seem that it's made more for fault-tolerance / > scalability and less for the absolute lowest-latency. Let's consider a few alternatives from the point of view of someone who is trying to built a platform for sale, me. The platform is to allow many users to host and run their trading strategies unattended, somewhere on a server co-located near the exchange. The platform has to have a web-based user and admin interface. I want to be able to compile strategies to machine code for fast execution. I also want some kind of typing to catch my errors and give me an additional layer of assurance. I can write the whole thing in Lisp. Lisp has a built-in compiler and good web servers. I can take an HTTP POST-ed strategy and compile it, making sure it will run fast. I have a free license to a commercial Lisp but it requires royalties. I don't think I want to go there and I don't want to spend money on a non-royalty commercial Lisp either. Neither Lisp will give me fault tolerance, scalability or typing. OCaml. I like the static typing, speed and ease of use (for me). I had the most pleasant experience of all writing and debugging my compiler in OCaml. OCaml does not have a built-in compiler, though. I will need to schedule jobs to compile strategies. The OCaml web server (Ocsigen) is not exactly proven and OCaml does not give me scalability of fault tolerance. C++. I like the static typing and Boost Spirit v2.1 seems like the ticket for writing the compiler. Performance can truly be squeezed here. I could build my engine into a web server (nginx?) and use LLVM or NanoJIT to compile strategies. I will surely spend a hell of a long time developing compared to other solutions. C++ does not give me scalability but I could use ZeroMQ or something similar. Fault tolerance will need to be ensured by making each trading strategy into a separate process. Erlang. There's a layer of typing with dialyzer. The web servers are good, there's scalability and fault tolerance. Building a compiler should be straightforward with leex and yecc. I can take HTTP POST-ed strategies and compile them. There's always HiPE for that ultimate performance boost. Performance may not be top-notch in the end but development should be fast and debugging straightforward. I can safely discard Lisp. I would like to use OCaml but I won't since I'll have to built a distribution infrastructure, bind to ZeroMQ for scalability and NanoJIT or LLVM for compilation to machine code. I don't really see the point. I see migrating to C++ as need arises, chunk by chunk. Erlang may end up as the message bus or I may swap it out entirely. This won't happen for a while or may never happen at all. Erlang just seems like the lesser of all evils or, perhaps, the golden middle. --- fastest mac firefox! http://wagerlabs.com From joelr1@REDACTED Thu Oct 1 21:39:36 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 20:39:36 +0100 Subject: [erlang-questions] trading systems In-Reply-To: References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> Message-ID: <50B67FF1-0EF1-4393-B0D4-875A17125D87@gmail.com> On Oct 1, 2009, at 8:07 PM, Jayson Vantuyl wrote: > Multicore systems inherently increase latency and decrease cache > performance. Erlang is optimized for these. Any information you > have regarding cache behavior is highly unlikely to be directly > applicable. Test and profile, early and often. It's the only way > to go. Erlang is often slow, even on multicore. Erlang is damn hard to optimize. I have spent literally months trying to understand the behavior and optimize OpenPoker and recently another system I build for a client. The latter system scales to hundreds of thousands of users on Amazon EC2 and makes use of up to 200 servers in simulation. Both projects need high performance. I still bear the scars and I'm still recovering. I feel that Erlang is a good start but I also feel that I must start the right way. After a good deal of time spent with Erlang, I'm very wary of "use the naive approach first" suggestions. It's all good in theory but does everyone making these suggestions have my amount of practice ;-). --- fastest mac firefox! http://wagerlabs.com From kagato@REDACTED Thu Oct 1 21:43:56 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 1 Oct 2009 12:43:56 -0700 Subject: [erlang-questions] trading systems In-Reply-To: <45B769E4-C263-4288-8BDE-E3758B60868F@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <45B769E4-C263-4288-8BDE-E3758B60868F@gmail.com> Message-ID: <475E92BA-FF6C-4AA5-ABAC-E0FEE99B9BCD@souja.net> > True. I think I'll still start by experimenting with mmap-ed > binaries, or at least a binaries-based approach. These can always be > mmap-ed later. Erlang assumes that binaries don't change. Note that it never modifies them, only makes new ones. Given that the compiler optimizes for this (or should), I would be careful about modifying binaries after they make it to Erlang. > Yes, I may try porting HiPE to x86-64 when I have time to spare. If you're very speed conscious, I would recommend running in 32-bit mode. With everything I've done, it seems faster to page a GB of data in/out of memory than it does to deal with the larger pointers-- especially in something pointer-heavy (like a dynamic language). > Let's consider a few alternatives from the point of view of someone > who is trying to built a platform for sale, me. The platform is to > allow many users to host and run their trading strategies > unattended, somewhere on a server co-located near the exchange. The > platform has to have a web-based user and admin interface. I want to > be able to compile strategies to machine code for fast execution. I > also want some kind of typing to catch my errors and give me an > additional layer of assurance. In my experience, typing really doesn't catch errors anymore. That said, dialyzer is wonderful. > I can write the whole thing in Lisp. Lisp has a built-in compiler > and good web servers. I can take an HTTP POST-ed strategy and > compile it, making sure it will run fast. I have a free license to a > commercial Lisp but it requires royalties. I don't think I want to > go there and I don't want to spend money on a non-royalty commercial > Lisp either. Neither Lisp will give me fault tolerance, scalability > or typing. Agreed. > OCaml. I like the static typing, speed and ease of use (for me). I > had the most pleasant experience of all writing and debugging my > compiler in OCaml. OCaml does not have a built-in compiler, though. > I will need to schedule jobs to compile strategies. The OCaml web > server (Ocsigen) is not exactly proven and OCaml does not give me > scalability of fault tolerance. Okay. Haskell might be another option. If you're going typed, it's the best, but it's probably suboptimal due to lack of libraries. > C++. I like the static typing and Boost Spirit v2.1 seems like the > ticket for writing the compiler. Performance can truly be squeezed > here. I could build my engine into a web server (nginx?) and use > LLVM or NanoJIT to compile strategies. I will surely spend a hell of > a long time developing compared to other solutions. C++ does not > give me scalability but I could use ZeroMQ or something similar. > Fault tolerance will need to be ensured by making each trading > strategy into a separate process. Okay. Also, avoid templates and try compiling for size. In cache- sensitive environments, I've found that -Os gets me better performance that -O99 just because of the smaller code size. > > Erlang. There's a layer of typing with dialyzer. The web servers are > good, there's scalability and fault tolerance. Building a compiler > should be straightforward with leex and yecc. I can take HTTP POST- > ed strategies and compile them. There's always HiPE for that > ultimate performance boost. Performance may not be top-notch in the > end but development should be fast and debugging straightforward. In particular, debugging of concurrent code can be very nice. http://souja.net/2009/04/making-sense-of-erlangs-event-tracer.html > I see migrating to C++ as need arises, chunk by chunk. Erlang may > end up as the message bus or I may swap it out entirely. This won't > happen for a while or may never happen at all. Erlang just seems > like the lesser of all evils or, perhaps, the golden middle. That makes sense. It sounds like the selling points are debuggableness, fault-tolerance, scalability, and concurrency. That all makes sense. Good luck on the latency front, though, as each of those traits increases latency. -- Jayson Vantuyl kagato@REDACTED From joelr1@REDACTED Thu Oct 1 21:47:03 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 20:47:03 +0100 Subject: [erlang-questions] trading systems In-Reply-To: <4AC4FF58.2090704@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> <4AC4FF58.2090704@gmail.com> Message-ID: <6E3E087C-AC8A-4A99-95C4-28D6BEE2C7FE@gmail.com> On Oct 1, 2009, at 8:13 PM, Michael Truog wrote: > One thing you might want to consider before building the system is the > cost of the data to actually do the type of trading you want to do. http://www.rithmic.com > If you are trying to do low latency arbitrage type trading you might > want > to relocate to New York for obvious reasons. There's no need to relocate to New York. I'm not going to run the strategies on my desktop and a server co-located in Chicago does not care that I'm in Tenerife. --- fastest mac firefox! http://wagerlabs.com From kagato@REDACTED Thu Oct 1 21:50:43 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 1 Oct 2009 12:50:43 -0700 Subject: [erlang-questions] trading systems In-Reply-To: <50B67FF1-0EF1-4393-B0D4-875A17125D87@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> <50B67FF1-0EF1-4393-B0D4-875A17125D87@gmail.com> Message-ID: > Erlang is often slow, even on multicore. Erlang is damn hard to > optimize. I have spent literally months trying to understand the > behavior and optimize OpenPoker and recently another system I build > for a client. The latter system scales to hundreds of thousands of > users on Amazon EC2 and makes use of up to 200 servers in simulation. Concurrent + latency-sensitive + distributed == damn hard to optimize in any language. That said, Erlang certainly takes a specific mindset to optimize. Are these built with Distributed Erlang? I find that it's rife with latency / weirdness. Working on an alternative to that is one of my pet projects, although I strongly doubt that I can make it as transparent as Distributed Erlang (a distribution plugin is HARD, but links and monitors require deep integration with the VM). > Both projects need high performance. I still bear the scars and I'm > still recovering. Please distinguish between low-latency, high-throughput, and both. "High performance" and "fast" are hard terms to work with. > I feel that Erlang is a good start but I also feel that I must start > the right way. After a good deal of time spent with Erlang, I'm very > wary of "use the naive approach first" suggestions. It's all good in > theory but does everyone making these suggestions have my amount of > practice ;-). In my experience, optimizing first has never been better. I always start out thinking so, but it's just not. -- Jayson Vantuyl kagato@REDACTED From raould@REDACTED Thu Oct 1 21:54:52 2009 From: raould@REDACTED (Raoul Duke) Date: Thu, 1 Oct 2009 12:54:52 -0700 Subject: [erlang-questions] trading systems In-Reply-To: References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> <50B67FF1-0EF1-4393-B0D4-875A17125D87@gmail.com> Message-ID: <91a2ba3e0910011254p14d7d820od93773d1f46d3b95@mail.gmail.com> >> I feel that Erlang is a good start but I also feel that I must start the >> right way. After a good deal of time spent with Erlang, I'm very wary of >> "use the naive approach first" suggestions. It's all good in theory but does >> everyone making these suggestions have my amount of practice ;-). > > In my experience, optimizing first has never been better. I always start > out thinking so, but it's just not. the wrong thing: premature optimization. the right thing: spending time on designing an appropriate architecture. ? sincerely. From joelr1@REDACTED Thu Oct 1 21:58:16 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 20:58:16 +0100 Subject: [erlang-questions] trading systems In-Reply-To: <475E92BA-FF6C-4AA5-ABAC-E0FEE99B9BCD@souja.net> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <45B769E4-C263-4288-8BDE-E3758B60868F@gmail.com> <475E92BA-FF6C-4AA5-ABAC-E0FEE99B9BCD@souja.net> Message-ID: <1D56639A-B9E7-43DC-8496-E622A4B8FC43@gmail.com> On Oct 1, 2009, at 8:43 PM, Jayson Vantuyl wrote: > Erlang assumes that binaries don't change. Note that it never > modifies them, only makes new ones. Given that the compiler > optimizes for this (or should), I would be careful about modifying > binaries after they make it to Erlang. An Erlang binary in the driver is an iovec structure or something like that. I assume I'll just create a new binary (structure) once I have new data. I'll have to experiment with this. > If you're very speed conscious, I would recommend running in 32-bit > mode. With everything I've done, it seems faster to page a GB of > data in/out of memory than it does to deal with the larger pointers-- > especially in something pointer-heavy (like a dynamic language). I'm assuming it's an Erlang-specific suggestion, right? http://www.kx.com flies with 64 bits, Mozilla's TraceMonkey Javascript engine is up to 20% faster with 64 bits on some benchmarks. There are more registers available for the compiler to use, etc. In short, I disagree since your suggestion is quite general and is not backed up by known use cases. > Okay. Haskell might be another option. If you're going typed, it's > the best, but it's probably suboptimal due to lack of libraries. I assume you have not used Haskell for heavy-duty projects but heard of it and maybe tinkered with it. I used Haskell for a networking project I extensively wrote about (google for Erlang vs Haskell). I switched to Erlang in the end and felt much better. Haskell is lazy which makes performance unpredictable. > Okay. Also, avoid templates Why? I interviewed with a hedge fund. They use templates extensively and it makes sense. Partial template specialization is just the ticket to optimize generic code at compile time. > and try compiling for size. In cache-sensitive environments, I've > found that -Os gets me better performance that -O99 just because of > the smaller code size. I agree with this. > In particular, debugging of concurrent code can be very nice. http://souja.net/2009/04/making-sense-of-erlangs-event-tracer.html Yes, thanks. I've been using tracing for years now. > That makes sense. It sounds like the selling points are > debuggableness, fault-tolerance, scalability, and concurrency. And faster time to market. > That all makes sense. Good luck on the latency front, though, as > each of those traits increases latency. Thanks. I'm not aiming to be the top dog but I would like to have a very strong showing. --- fastest mac firefox! http://wagerlabs.com From joelr1@REDACTED Thu Oct 1 22:02:13 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 21:02:13 +0100 Subject: [erlang-questions] trading systems In-Reply-To: References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> <50B67FF1-0EF1-4393-B0D4-875A17125D87@gmail.com> Message-ID: <32358EA6-7123-4A2E-AD33-FD6FEFDA077C@gmail.com> On Oct 1, 2009, at 8:50 PM, Jayson Vantuyl wrote: > Concurrent + latency-sensitive + distributed == damn hard to > optimize in any language. Not necessarily hard. It's perfectly fine to be concurrent at the process level, for example. The difficulty with Erlang is tracking the state of all the message queues at any given moment and just plain figuring out what's going on among all the different processes. I have resorted to various tricks apart from tracing, e.g. putting stuff into the process dictionary at various stages of process life for ease of fetching via process_info. > That said, Erlang certainly takes a specific mindset to optimize. > Are these built with Distributed Erlang? Distributed Erlang indeed. Very distributed. Check the recent netsplit thread for some of the weirdness. Go back to the "Ultimate Erlang Challenge" for more. --- fastest mac firefox! http://wagerlabs.com From max.lapshin@REDACTED Thu Oct 1 22:04:58 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 2 Oct 2009 00:04:58 +0400 Subject: [erlang-questions] Switching between http/non-http depending of first byte In-Reply-To: <46873482-569F-4BCF-90FE-55F5B689F3E0@rogvall.se> References: <46873482-569F-4BCF-90FE-55F5B689F3E0@rogvall.se> Message-ID: On Thu, Oct 1, 2009 at 11:54 PM, Tony Rogvall wrote: > Try using: > > gen_tcp:unrecv(S, Data). > Excelent! It's just what I was asking about. From joelr1@REDACTED Thu Oct 1 22:21:35 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 21:21:35 +0100 Subject: [erlang-questions] trading systems In-Reply-To: <91a2ba3e0910011254p14d7d820od93773d1f46d3b95@mail.gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> <50B67FF1-0EF1-4393-B0D4-875A17125D87@gmail.com> <91a2ba3e0910011254p14d7d820od93773d1f46d3b95@mail.gmail.com> Message-ID: On Oct 1, 2009, at 8:54 PM, Raoul Duke wrote: > the wrong thing: premature optimization. > > the right thing: spending time on designing an appropriate > architecture. I admit that I'm biased. My (contract) day job nowadays is optimizing Firefox performance. I spend days staring at profiles, I have become expert with DTrace and I'm trying to think of how I can carry all that knowledge back to Erlang. --- fastest mac firefox! http://wagerlabs.com From kagato@REDACTED Thu Oct 1 22:26:22 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 1 Oct 2009 13:26:22 -0700 Subject: [erlang-questions] trading systems In-Reply-To: <1D56639A-B9E7-43DC-8496-E622A4B8FC43@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <45B769E4-C263-4288-8BDE-E3758B60868F@gmail.com> <475E92BA-FF6C-4AA5-ABAC-E0FEE99B9BCD@souja.net> <1D56639A-B9E7-43DC-8496-E622A4B8FC43@gmail.com> Message-ID: <8412C42B-1589-4B85-9D77-7F8669307895@souja.net> > I'm assuming it's an Erlang-specific suggestion, right? http://www.kx.com > flies with 64 bits, Mozilla's TraceMonkey Javascript engine is up > to 20% faster with 64 bits on some benchmarks. There are more > registers available for the compiler to use, etc. In short, I > disagree since your suggestion is quite general and is not backed up > by known use cases. Actually, it's only mildly supported by Erlang, but heavily supported in Python and Ruby. Unless you're generating machine code, I've never seen 64-bit to be a big boost. While HIPE might make that a non- issue, I've generally observed that VMs don't benefit from 64-bit in that way. > I assume you have not used Haskell for heavy-duty projects but heard > of it and maybe tinkered with it. I used Haskell for a networking > project I extensively wrote about (google for Erlang vs Haskell). I > switched to Erlang in the end and felt much better. Haskell is lazy > which makes performance unpredictable. I've used it and loathed that lack of libraries. I was only pointing out that it's type system is cleaner. Just trying to round out the spectrum of options. > Why? I interviewed with a hedge fund. They use templates extensively > and it makes sense. Partial template specialization is just the > ticket to optimize generic code at compile time. Templates can drastically inflate code size and cause an extra indirection at function call time. I've always found that, for optimization, templates are middling, doing your own indirection in straight C is king--especially when you give branching hints. Of course, that might be premature optimization at this point. ;P > Yes, thanks. I've been using tracing for years now. I'll assume from your OpenPoker statement that you were doing tracing with Erlang. I've found the documentation a bit lacking, so I like to throw that example around to make sure that people can actually figure out how to use it. Let's just say that dbg:tpl is not exactly an obvious interface. > And faster time to market. Right on. -- Jayson Vantuyl kagato@REDACTED From joelr1@REDACTED Thu Oct 1 22:34:32 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 21:34:32 +0100 Subject: job: architecture review of a scalable erlang system Message-ID: <929FA521-5648-4D62-BE5B-F1B031FC07C2@gmail.com> I wrote a scalable Erlang system for my client. The system can handle hundreds of thousands of clients on EC2 and uses up to 200 instances in simulation. The system supports both socket and http connections and has certain latency requirements which are fully met by the socket side. The HTTP side of the system does not give me the latency required by my customer but we are in a disagreement over whether the architecture of the HTTP transport is causing this. I spent a couple of months tweaking the system and I don't think I can push HTTP performance any further. I need an expert review of the architecture and suggestions for performance improvement. The system is ~7K LOC including an extensive simulation harness and is written by me entirely in Erlang. This is a short-term paid job and only experts need apply. Please contact Neil Berkman at neil@REDACTED with a description of highly scalable Erlang code that you have written and optimized. Please copy me on your email message. Thanks, Joel --- fastest mac firefox! http://wagerlabs.com From kagato@REDACTED Thu Oct 1 22:36:54 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 1 Oct 2009 13:36:54 -0700 Subject: [erlang-questions] trading systems In-Reply-To: <32358EA6-7123-4A2E-AD33-FD6FEFDA077C@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> <50B67FF1-0EF1-4393-B0D4-875A17125D87@gmail.com> <32358EA6-7123-4A2E-AD33-FD6FEFDA077C@gmail.com> Message-ID: <9650A124-0251-4379-83F5-C57491EF4695@souja.net> > The difficulty with Erlang is tracking the state of all the message > queues at any given moment and just plain figuring out what's going > on among all the different processes. I have resorted to various > tricks apart from tracing, e.g. putting stuff into the process > dictionary at various stages of process life for ease of fetching > via process_info. Are you using vanilla tracing, sequential tracing, or both? Yes, the mailboxes are the number one headache for me. You can always get at the message queue with erlang:process_info(self(),messages). I have a check in most of my servers that, when I'm debugging, will dump the message queue to disk if it gets over a certain size. Helps me track down places that need to be synchronized to prevent run-ahead. >> That said, Erlang certainly takes a specific mindset to optimize. >> Are these built with Distributed Erlang? > > Distributed Erlang indeed. Very distributed. Check the recent > netsplit thread for some of the weirdness. Go back to the "Ultimate > Erlang Challenge" for more. Yeah. Since distribution modules are C, I doubt it's going to get fixed any time soon though. :( -- Jayson Vantuyl kagato@REDACTED From joelr1@REDACTED Thu Oct 1 22:41:56 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 21:41:56 +0100 Subject: tracing with erlang made easy Message-ID: <43CD9D82-4F02-4722-BEB0-91DF5AD9AB62@gmail.com> -module(t). -compile([export_all]). -include_lib("stdlib/include/ms_transform.hrl"). %%% Stop all tracing ut() -> dbg:ctp(). %%% Trace a module t(Mod) when is_atom(Mod) -> t([Mod]); %%% Trace Mod:Fun (any arity) t({Mod, Fun}) when is_atom(Mod), is_atom(Fun) -> t([{Mod, Fun}]); %%% Use a combination of Mod and {Mod, Fun} in a list t(What) -> dbg:tracer(), dbg:p(all, [call]), t1(What). t1([]) -> ok; t1([{M, F}|T]) -> dbg:tpl(M, F, dbg:fun2ms(fun(_) -> return_trace() end)), t1(T); t1([H|T]) -> dbg:tpl(H, dbg:fun2ms(fun(_) -> return_trace() end)), t1(T). %%% What processes are running these modules? mods2procs(Mods) -> [P || M <- Mods, P <- processes(), erlang:check_process_code(P, M)]. %%% What processes are running a given module %%% that is an implementation of gen_fsm, gen_server, etc. p(Mod) -> p(Mod, processes(), []). p(_, [], Acc) -> Acc; p(Mod, [H|T], Acc) -> {_, L} = process_info(H, dictionary), case lists:keyfind('$initial_call', 1, L) of {'$initial_call', {Mod, init, 1}} -> p(Mod, T, [H|Acc]); _ -> p(Mod, T, Acc) end. %%% Remote process info pi(P) when is_pid(P) -> rpc:call(node(P), erlang, process_info, [P]). --- fastest mac firefox! http://wagerlabs.com From joelr1@REDACTED Thu Oct 1 22:45:54 2009 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 1 Oct 2009 21:45:54 +0100 Subject: [erlang-questions] trading systems In-Reply-To: <9650A124-0251-4379-83F5-C57491EF4695@souja.net> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> <50B67FF1-0EF1-4393-B0D4-875A17125D87@gmail.com> <32358EA6-7123-4A2E-AD33-FD6FEFDA077C@gmail.com> <9650A124-0251-4379-83F5-C57491EF4695@souja.net> Message-ID: <32E5AB1D-C951-4EC9-8764-FBB64E782EF0@gmail.com> On Oct 1, 2009, at 9:36 PM, Jayson Vantuyl wrote: > Are you using vanilla tracing, sequential tracing, or both? Yes, > the mailboxes are the number one headache for me. I haven't tried sequential tracing, although I did consider it at one point. See the message I just posted to the list for my tracing helpers. > You can always get at the message queue with erlang:process_info(self > (),messages). I have a check in most of my servers that, when I'm > debugging, will dump the message queue to disk if it gets over a > certain size. Helps me track down places that need to be > synchronized to prevent run-ahead. You can also crash the system and then inspect the dump for the contents of the message queues, stacks, etc. It's not that difficult. --- fastest mac firefox! http://wagerlabs.com From tony@REDACTED Thu Oct 1 21:54:48 2009 From: tony@REDACTED (Tony Rogvall) Date: Thu, 1 Oct 2009 21:54:48 +0200 Subject: [erlang-questions] Switching between http/non-http depending of first byte In-Reply-To: References: Message-ID: <46873482-569F-4BCF-90FE-55F5B689F3E0@rogvall.se> Try using: gen_tcp:unrecv(S, Data). On accepting side you should have the listen socket in passive mode [{active,false},{packet,0}], then: {ok,S} = gen_tcp:accept(L). case gen_tcp:recv(S, 1) of {ok,[3]} -> gen_tcp:unrecv(S, [3]), rtmp_handler(S); {ok,H} -> gen_tcp:unrecv(S, H), %% switch to http protocol .. http_handler(S) Error -> ... end. /Tony On 1 okt 2009, at 09.50, Max Lapshin wrote: > Hi. I want to implement an unusual feature: when server accepts > socket, > it must look at first byte. If it is 0x03, than I want to handle it by > one handler (RTMP), > if other, than HTTP handler. > > Problem is that I need only one byte and received packet should be > passed to exact protocol handler. > > I understand, that I will not be able to use erlang setopt(http) > feature, so question is the following: > > should I better use Yaws or MochiWeb as underlying webserver? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From colm.dougan@REDACTED Thu Oct 1 23:20:27 2009 From: colm.dougan@REDACTED (Colm Dougan) Date: Thu, 1 Oct 2009 22:20:27 +0100 Subject: [erlang-questions] Switching between http/non-http depending of first byte In-Reply-To: <46873482-569F-4BCF-90FE-55F5B689F3E0@rogvall.se> References: <46873482-569F-4BCF-90FE-55F5B689F3E0@rogvall.se> Message-ID: <24d4f39c0910011420m617f6729i6a41b97efb332e6b@mail.gmail.com> On Thu, Oct 1, 2009 at 8:54 PM, Tony Rogvall wrote: > Try using: > > gen_tcp:unrecv(S, Data). unrecv() appears to be undocumented. Is there a good reason for that? Colm From manprog@REDACTED Fri Oct 2 00:12:49 2009 From: manprog@REDACTED (Bernardo Alvez) Date: Thu, 01 Oct 2009 19:12:49 -0300 Subject: [erlang-questions] Better use of io:format and io_lib:format In-Reply-To: References: <4AC489DA.5080203@adinet.com.uy> Message-ID: <4AC52961.9060504@adinet.com.uy> Thanks Michael! BR, Bernardo Michael Richter escribi?: > You could try an experiment, maybe? > > File: "output": > > #! /usr/bin/env escript > %% -mode(compile). > > main(["list"]) -> > M1 = erlang:process_info(self(), memory), > T1 = erlang:now(), > list_loop(500), > T2 = erlang:now(), > M2 = erlang:process_info(self(), memory), > io:format("~n~n~w , ~w , ~w~n~n", [M1, M2, timer:now_diff(T2,T1)]); > > main(["binary"]) -> > M1 = erlang:process_info(self(), memory), > T1 = erlang:now(), > binary_loop(500), > T2 = erlang:now(), > M2 = erlang:process_info(self(), memory), > io:format("~w , ~w , ~w~n", [M1, M2, timer:now_diff(T2,T1)]); > > main(_) -> > io:format("Try again.~n"). > > list_loop(0) -> > io:format("Done!~n~n"); > > list_loop(N) -> > io:format("hello ~s", ["world"]), > list_loop(N-1). > > binary_loop(0) -> > io:format(<<"Done!~n~n">>); > > binary_loop(N) -> > io:format(<<"hello ~s">>, [<<"world">>]), > binary_loop(N-1). > > Result set from several runs of "clear ; ./output list": > > - {memory,54492} , {memory,33820} , 104308 > - {memory,54492} , {memory,33820} , 103686 > - {memory,54492} , {memory,33820} , 100892 > - {memory,54492} , {memory,33820} , 87483 > - {memory,54492} , {memory,33820} , 99644 > > Result set from several runs of "clear ; ./output binary": > > - {memory,44156} , {memory,33820} , 94222 > - {memory,44156} , {memory,33820} , 110587 > - {memory,44156} , {memory,33820} , 115329 > - {memory,44156} , {memory,33820} , 111326 > - {memory,44156} , {memory,33820} , 102580 > > I make no representation that my code is any good (indeed it's the > opposite), but a brief eyeball of the results shows that binaries use up a > little less memory and a little more CPU. I'm certain the other, better > Erlang users can give you better benchmarking code to answer your question > (but so can you, and your benchmarking code would reflect your intended > use). > > 2009/10/1 Bernardo Alvez > > >> Hi all, >> >> Which of the following is better to use in terms of performance and >> resources usage?: >> >> io:format("hello ~s", ["world"]). >> or >> io:format(<<"hello ~s">>, [<<"world">>]). >> >> Thanks, >> Bernardo >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> > > From adrianob@REDACTED Fri Oct 2 05:02:15 2009 From: adrianob@REDACTED (Adriano Bonat) Date: Fri, 2 Oct 2009 00:02:15 -0300 Subject: Building R13B02-1 on Snow Leopard Message-ID: <6bb4c8cb0910012002y688b5665s8ce3d847c37ee990@mail.gmail.com> Hi guys, I've been trying for 2 days build R13B02-1 on Snow Leopard without success, it fails with some linker warnings like: ld: warning: in /opt/local/lib/libssl.dylib, file is not of required architecture ld: warning: in /opt/local/lib/libcrypto.dylib, file is not of required architecture and then several undefined symbols for OpenSSL in this case, in other tries I got also warnings for PCRE, Crypto, etc. You can find the complete configure and make here: http://dl.getdropbox.com/u/274701/configure_make_R13B02-1.txt Summarizing some informations which I judge important: adriano@REDACTED:~/Downloads/otp_src_R13B02-1$ uname -a Darwin planck.local 10.0.0 Darwin Kernel Version 10.0.0: Fri Jul 31 22:47:34 PDT 2009; root:xnu-1456.1.25~1/RELEASE_I386 i386 adriano@REDACTED:~/Downloads/otp_src_R13B02-1$ ./configure --enable-kernel-poll --enable-threads --enable-smp-support --enable-hipe --with-ssl=/opt/local --enable-m64-build adriano@REDACTED:~/Downloads/otp_src_R13B02-1$ file /opt/local/lib/libssl.dylib /opt/local/lib/libssl.dylib: Mach-O 64-bit dynamically linked shared library x86_64 The problem seems to be because my libraries are compiled as 64bit, and in the configure process it is defining the linker flags as 32bit, from here: checking for linker flags for loadable drivers... -m32 -bundle -flat_namespace -undefined suppress Any help to get it compiled is appreciated :) Thanks. -Adriano Bonat From jeffm@REDACTED Fri Oct 2 05:19:57 2009 From: jeffm@REDACTED (jm) Date: Fri, 02 Oct 2009 13:19:57 +1000 Subject: [erlang-questions] Building R13B02-1 on Snow Leopard In-Reply-To: <6bb4c8cb0910012002y688b5665s8ce3d847c37ee990@mail.gmail.com> References: <6bb4c8cb0910012002y688b5665s8ce3d847c37ee990@mail.gmail.com> Message-ID: <4AC5715D.9060005@ghostgun.com> Warning: blind leading the blind here. I notice that checking build system type... i386-apple-darwin10.0.0 checking host system type... i386-apple-darwin10.0.0 What happens if you change the build and systems systems with System types: --build=BUILD configure for building on BUILD [guessed] --host=HOST cross-compile to build programs to run on HOST [BUILD] Cant think of a better way of doing this at the moment, but take a look in /usr/libexec/gcc/ and see what other systems are listed. eg, on mine $ ls /usr/libexec/gcc/ darwin i686-apple-darwin8 powerpc-apple-darwin8 and just for the hell of it see what this produces $ gcc --dumpmachine powerpc-apple-darwin8-gcc-4.0.1: no input files Jeff. Adriano Bonat wrote: > Hi guys, > > I've been trying for 2 days build R13B02-1 on Snow Leopard without > success, it fails with some linker warnings like: > > From jeffm@REDACTED Fri Oct 2 05:24:03 2009 From: jeffm@REDACTED (jm) Date: Fri, 02 Oct 2009 13:24:03 +1000 Subject: [erlang-questions] Building R13B02-1 on Snow Leopard In-Reply-To: <4AC5715D.9060005@ghostgun.com> References: <6bb4c8cb0910012002y688b5665s8ce3d847c37ee990@mail.gmail.com> <4AC5715D.9060005@ghostgun.com> Message-ID: <4AC57253.9070403@ghostgun.com> There's also the --enable-darwin-64bit flag at least in R12B. I'm downloading R13B as I write this to check it. Jeff. jm wrote: > Warning: blind leading the blind here. > From adrianob@REDACTED Fri Oct 2 05:36:01 2009 From: adrianob@REDACTED (Adriano Bonat) Date: Fri, 2 Oct 2009 00:36:01 -0300 Subject: [erlang-questions] Building R13B02-1 on Snow Leopard In-Reply-To: <4AC5715D.9060005@ghostgun.com> References: <6bb4c8cb0910012002y688b5665s8ce3d847c37ee990@mail.gmail.com> <4AC5715D.9060005@ghostgun.com> Message-ID: <6bb4c8cb0910012036t8068714s7aa5647c7ae6efb4@mail.gmail.com> On Fri, Oct 2, 2009 at 12:19 AM, jm wrote: > Warning: blind leading the blind here. > > I notice that > > checking build system type... i386-apple-darwin10.0.0 > checking host system type... i386-apple-darwin10.0.0 > > What happens if you change the build and systems systems with > > System types: > ?--build=BUILD ? ? configure for building on BUILD [guessed] > ?--host=HOST ? ? ? cross-compile to build programs to run on HOST [BUILD] > Just tried appending --build=i686-apple-darwin10 but it continues to guess "-m32". > Cant think of a better way of doing this at the moment, but take a look in > /usr/libexec/gcc/ and see what other systems are listed. eg, on mine > > $ ls /usr/libexec/gcc/ > darwin ? ? ? ? ? ? ? ? ?i686-apple-darwin8 ? ? ?powerpc-apple-darwin8 adriano@REDACTED:~/Downloads/otp_src_R13B02-1$ ls /usr/libexec/gcc darwin i686-apple-darwin10 powerpc-apple-darwin10 > > and just for the hell of it see what this produces > > $ gcc --dumpmachine > powerpc-apple-darwin8-gcc-4.0.1: no input files > adriano@REDACTED:~/Downloads/otp_src_R13B02-1$ gcc --dumpmachine i686-apple-darwin10-gcc-4.2.1: no input files Just tried configure appending --enable-darwin-64bit: ./configure --enable-kernel-poll --enable-threads --enable-smp-support --enable-hipe --with-ssl=/opt/local --enable-m64-build --enable-darwin-64bit and it gives me: checking for linker flags for loadable drivers... -m64 -m64 -bundle -flat_namespace -undefined suppress Now let me `make` :) From adrianob@REDACTED Fri Oct 2 05:40:06 2009 From: adrianob@REDACTED (Adriano Bonat) Date: Fri, 2 Oct 2009 00:40:06 -0300 Subject: [erlang-questions] Building R13B02-1 on Snow Leopard In-Reply-To: <6bb4c8cb0910012036t8068714s7aa5647c7ae6efb4@mail.gmail.com> References: <6bb4c8cb0910012002y688b5665s8ce3d847c37ee990@mail.gmail.com> <4AC5715D.9060005@ghostgun.com> <6bb4c8cb0910012036t8068714s7aa5647c7ae6efb4@mail.gmail.com> Message-ID: <6bb4c8cb0910012040x15f261f3vb866497a07faa2ea@mail.gmail.com> On Fri, Oct 2, 2009 at 12:36 AM, Adriano Bonat wrote: > > Now let me `make` :) > Not yet. When generating "beam.smp": gcc -o /Users/adriano/Downloads/otp_src_R13B02-1/bin/i386-apple-darwin10.0.0/beam.smp ... ld: warning: in /Users/adriano/Downloads/otp_src_R13B02-1/erts/emulator/zlib/obj/i386-apple-darwin10.0.0/opt/libz.a, file is not of required architecture ld: warning: in /Users/adriano/Downloads/otp_src_R13B02-1/erts/emulator/pcre/obj/i386-apple-darwin10.0.0/opt/libepcre.a, file is not of required architecture Undefined symbols: "_deflate", referenced from: _erl_zlib_compress2 in erl_zlib.o _erts_gzdeflate_buffer in gzio.o [snip...] ld: symbol(s) not found collect2: ld returned 1 exit status make[3]: *** [/Users/adriano/Downloads/otp_src_R13B02-1/bin/i386-apple-darwin10.0.0/beam.smp] Error 1 make[2]: *** [opt] Error 2 make[1]: *** [smp] Error 2 make: *** [emulator] Error 2 From jeffm@REDACTED Fri Oct 2 05:48:32 2009 From: jeffm@REDACTED (jm) Date: Fri, 02 Oct 2009 13:48:32 +1000 Subject: [erlang-questions] Building R13B02-1 on Snow Leopard In-Reply-To: <6bb4c8cb0910012040x15f261f3vb866497a07faa2ea@mail.gmail.com> References: <6bb4c8cb0910012002y688b5665s8ce3d847c37ee990@mail.gmail.com> <4AC5715D.9060005@ghostgun.com> <6bb4c8cb0910012036t8068714s7aa5647c7ae6efb4@mail.gmail.com> <6bb4c8cb0910012040x15f261f3vb866497a07faa2ea@mail.gmail.com> Message-ID: <4AC57810.9080202@ghostgun.com> Adriano Bonat wrote: > On Fri, Oct 2, 2009 at 12:36 AM, Adriano Bonat wrote: > >> Now let me `make` :) >> >> > > Not yet. When generating "beam.smp": > > Unfortunately I don't have a matching architecture in front of me. I'll have to compile erlang on a recent model mac book shortly so I'll be interested in the fix myself. Jeff. From adrianob@REDACTED Fri Oct 2 06:01:23 2009 From: adrianob@REDACTED (Adriano Bonat) Date: Fri, 2 Oct 2009 01:01:23 -0300 Subject: [erlang-questions] Building R13B02-1 on Snow Leopard In-Reply-To: <4AC57810.9080202@ghostgun.com> References: <6bb4c8cb0910012002y688b5665s8ce3d847c37ee990@mail.gmail.com> <4AC5715D.9060005@ghostgun.com> <6bb4c8cb0910012036t8068714s7aa5647c7ae6efb4@mail.gmail.com> <6bb4c8cb0910012040x15f261f3vb866497a07faa2ea@mail.gmail.com> <4AC57810.9080202@ghostgun.com> Message-ID: <6bb4c8cb0910012101h3be8aaadl359961bddd497711@mail.gmail.com> Just got success using the options combined: ./configure --enable-kernel-poll --enable-threads --enable-smp-support --enable-hipe --with-ssl=/opt/local --enable-m64-build --enable-darwin-64bit --build=i686-apple-darwin10 Don't know why my system turned the process so complicated, maybe my upgrade to Snow Leopard isn't that OK? Anyway, thanks for sharing some ideas :) On Fri, Oct 2, 2009 at 12:48 AM, jm wrote: > > > Adriano Bonat wrote: >> >> On Fri, Oct 2, 2009 at 12:36 AM, Adriano Bonat wrote: >> >>> >>> Now let me `make` :) >>> >>> >> >> Not yet. When generating "beam.smp": >> >> > > Unfortunately I don't have a matching architecture in front of me. I'll have > to compile erlang on a recent model mac book shortly so I'll be interested > in the fix myself. > > > Jeff. > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ngocdaothanh@REDACTED Fri Oct 2 06:09:16 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 2 Oct 2009 13:09:16 +0900 Subject: [erlang-questions] Building R13B02-1 on Snow Leopard In-Reply-To: <6bb4c8cb0910012101h3be8aaadl359961bddd497711@mail.gmail.com> References: <6bb4c8cb0910012002y688b5665s8ce3d847c37ee990@mail.gmail.com> <4AC5715D.9060005@ghostgun.com> <6bb4c8cb0910012036t8068714s7aa5647c7ae6efb4@mail.gmail.com> <6bb4c8cb0910012040x15f261f3vb866497a07faa2ea@mail.gmail.com> <4AC57810.9080202@ghostgun.com> <6bb4c8cb0910012101h3be8aaadl359961bddd497711@mail.gmail.com> Message-ID: <5c493e530910012109m4dbc97dbt3a0da5c5a5ce9061@mail.gmail.com> Are you using MacPorts? I had problem like this before on Leopard. May be because libs of MacPorts (in /opt/local/lib) and libs of Leopard (in /usr/local/lib) are not compatible. My dirty solution is moving /usr/local/lib away and symlink /opt/local/lib there. On Fri, Oct 2, 2009 at 1:01 PM, Adriano Bonat wrote: > Just got success using the options combined: > ./configure --enable-kernel-poll --enable-threads --enable-smp-support > --enable-hipe --with-ssl=/opt/local --enable-m64-build > --enable-darwin-64bit --build=i686-apple-darwin10 > > Don't know why my system turned the process so complicated, maybe my > upgrade to Snow Leopard isn't that OK? > > Anyway, thanks for sharing some ideas :) > > > On Fri, Oct 2, 2009 at 12:48 AM, jm wrote: >> >> >> Adriano Bonat wrote: >>> >>> On Fri, Oct 2, 2009 at 12:36 AM, Adriano Bonat wrote: >>> >>>> >>>> Now let me `make` :) >>>> >>>> >>> >>> Not yet. When generating "beam.smp": >>> >>> >> >> Unfortunately I don't have a matching architecture in front of me. I'll have >> to compile erlang on a recent model mac book shortly so I'll be interested >> in the fix myself. >> >> >> Jeff. >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From kagato@REDACTED Fri Oct 2 06:42:54 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 1 Oct 2009 21:42:54 -0700 Subject: binary_to_term and Leaking Atoms Message-ID: <7336B504-11BE-4864-A0AF-DF731D608169@souja.net> I'm working on a network protocol. The easy way to do it is just use term_to_binary and binary_to_term. There is one problem, though. I don't necessarily trust the sender of these binaries. It seems that decoding them should be relatively safe, except for the potential for someone to leak atoms by creating a large number of atoms until I run out of memory. I presume this sort of thing is the reason that binary_to_existing_atom (and list_to_existing_atom) were created. A few questions: 1. Is there any facility to "safely" decode binaries? 2. If not, is it likely that there will be one added? Perhaps binary_to_term/2 (with a "existing_atoms_only" option)? 3. Are there any other dangers I should be aware of when using binary_to_term? For example, despite the fact that I don't intend to use them anywhere, are there any reasons to fear decoding fun's? Bonus question: Are binaries checksummed internally or when stored with dets? Thanks, -- Jayson Vantuyl kagato@REDACTED From max.lapshin@REDACTED Fri Oct 2 06:44:49 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 2 Oct 2009 08:44:49 +0400 Subject: [erlang-questions] Switching between http/non-http depending of first byte In-Reply-To: <24d4f39c0910011420m617f6729i6a41b97efb332e6b@mail.gmail.com> References: <46873482-569F-4BCF-90FE-55F5B689F3E0@rogvall.se> <24d4f39c0910011420m617f6729i6a41b97efb332e6b@mail.gmail.com> Message-ID: As I've already told, I refused from original idea, but in any case, such thing may be useful. There are some features in erlang, that are undocumented, but required for normal work, such as prim_inet:async_accept. From max.lapshin@REDACTED Fri Oct 2 06:46:40 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 2 Oct 2009 08:46:40 +0400 Subject: [erlang-questions] binary_to_term and Leaking Atoms In-Reply-To: <7336B504-11BE-4864-A0AF-DF731D608169@souja.net> References: <7336B504-11BE-4864-A0AF-DF731D608169@souja.net> Message-ID: There are erlang:binary_to_existing_atom/2, erlang:list_to_existing_atom/1 From kagato@REDACTED Fri Oct 2 06:48:11 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 1 Oct 2009 21:48:11 -0700 Subject: [erlang-questions] binary_to_term and Leaking Atoms In-Reply-To: References: <7336B504-11BE-4864-A0AF-DF731D608169@souja.net> Message-ID: <5D7992C1-09E6-4B52-8FEB-6851645FF1AB@souja.net> That works if I was just sending an atom as a binary, but I was hoping to send whole data structures using term_to_binary. Looks like there's no way to make that atom safe, then? On Oct 1, 2009, at 9:46 PM, Max Lapshin wrote: > There are erlang:binary_to_existing_atom/2, > erlang:list_to_existing_atom/1 -- Jayson Vantuyl kagato@REDACTED From max.lapshin@REDACTED Fri Oct 2 06:51:07 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 2 Oct 2009 08:51:07 +0400 Subject: [erlang-questions] binary_to_term and Leaking Atoms In-Reply-To: <5D7992C1-09E6-4B52-8FEB-6851645FF1AB@souja.net> References: <7336B504-11BE-4864-A0AF-DF731D608169@souja.net> <5D7992C1-09E6-4B52-8FEB-6851645FF1AB@souja.net> Message-ID: On Fri, Oct 2, 2009 at 8:48 AM, Jayson Vantuyl wrote: > That works if I was just sending an atom as a binary, but I was hoping to > send whole data structures using term_to_binary. ?Looks like there's no way > to make that atom safe, then? As for me, I don't see any ways except special driver or patch for such a thing. From ngocdaothanh@REDACTED Fri Oct 2 07:11:36 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 2 Oct 2009 14:11:36 +0900 Subject: characters_to_list vs characters_to_binary Message-ID: <5c493e530910012211m3353596fob3242417f4fe8192@mail.gmail.com> I am using Erlang R13B02-1 with Japanese data. The doc of unicode module says that their results are the same, only one in list and one in binary. But: unicode:characters_to_list([230,151,165,230,156,172,232,170,158,231,137,136]). => [230,151,165,230,156,172,232,170,158,231,137,136] unicode:characters_to_binary([230,151,165,230,156,172,232,170,158,231,137,136]). => <<195,166,194,151,194,165,195,166,194,156,194,172,195,168, 194,170,194,158,195,167,194,137,194,136>> Is there something wrong? From ckerr@REDACTED Fri Oct 2 07:54:40 2009 From: ckerr@REDACTED (Cameron Kerr) Date: Fri, 02 Oct 2009 18:54:40 +1300 Subject: [erlang-questions] Need parallel calls to gen_server:call, similar but different to gen_server:multi_call In-Reply-To: <4AC3DD87.6000309@cs.otago.ac.nz> References: <4AC3DD87.6000309@cs.otago.ac.nz> Message-ID: <4AC595A0.6090905@cs.otago.ac.nz> Thanks for your feedback everyone. This is how I ended up doing it. I'd appreciate any criticisms. If doing this long-term, I think I'd prefer to extend the gen_server API. Thanks to David Mercer for his guidance. %%%============================================================================ %%% Utility functions for dealing with gen_servers %%% %%% Author: Cameron Kerr %%% Date: 1 October 2009 %%%============================================================================ -module(gen_server_util). -include_lib("eunit/include/eunit.hrl"). -export([parallel_call/2]). %% @spec parallel_call([Server_ref()], Request) -> [Response] %% %% Makes multiple gen_server calls, in parallel. Responses are %% returned in the same order as they are found in the Servers list. %% %% FIXME: this is not robust if a call causes a process to fail. %% In this case, we may end up getting a badrpc on subsequent %% messages if the Servers list is not maintained. %% parallel_call(Servers, Request) -> rpc:pmap({gen_server, call}, [Request], Servers). parallel_call_test() -> Servers_params = [{one,400}, {two,100}, {three,500}, {four,200}], Servers_pids = [launch_responder(Response,Delay) || {Response,Delay} <- Servers_params], [one,two,three,four] = parallel_call(Servers_pids, ping), parallel_call(Servers_pids, shutdown). launch_responder(Response, Delay) -> {ok, Pid} = parallel_call_test_server:start_link(Response, Delay), Pid. %%%------------------------------------------------------------------- %%% File : parallel_call_test_server.erl %%% Author : %%% Description : Implements a simple server used for testing %%% gen_server_util:parallel_call/2 %%% %%% Created : 1 Oct 2009 by %%%------------------------------------------------------------------- -module(parallel_call_test_server). -include_lib("eunit/include/eunit.hrl"). -behaviour(gen_server). %% API -export([start_link/2]). %% gen_server callbacks -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). -record(state, {response, delay}). %%==================================================================== %% API %%==================================================================== %%-------------------------------------------------------------------- %% Function: start_link() -> {ok,Pid} | ignore | {error,Error} %% Description: Starts the server %%-------------------------------------------------------------------- start_link(Response, Delay) -> gen_server:start_link(?MODULE, [Response, Delay], []). %%==================================================================== %% gen_server callbacks %%==================================================================== %%-------------------------------------------------------------------- %% Function: init(Args) -> {ok, State} | %% {ok, State, Timeout} | %% ignore | %% {stop, Reason} %% Description: Initiates the server %%-------------------------------------------------------------------- init([Response, Delay]) -> {ok, #state{response = Response, delay = Delay}}. %%-------------------------------------------------------------------- %% Function: %% handle_call(Request, From, State) -> {reply, Reply, State} | %% {reply, Reply, State, Timeout} | %% {noreply, State} | %% {noreply, State, Timeout} | %% {stop, Reason, Reply, State} | %% {stop, Reason, State} %% Description: Handling call messages %%-------------------------------------------------------------------- handle_call(ping, _From, State) -> timer:sleep(State#state.delay), ?debugFmt("Server ~p responding to ping with response of ~p~n", [self(), State#state.response]), {reply, State#state.response, State}; handle_call(shutdown, _From, State) -> ?debugFmt("Server ~p (which responds with ~p) responding to shutdown request~n", [self(), State#state.response]), {stop, normal, ok, State}; handle_call(_Request, _From, State) -> Reply = ok, {reply, Reply, State}. %%-------------------------------------------------------------------- %% Function: handle_cast(Msg, State) -> {noreply, State} | %% {noreply, State, Timeout} | %% {stop, Reason, State} %% Description: Handling cast messages %%-------------------------------------------------------------------- handle_cast(_Msg, State) -> {noreply, State}. %%-------------------------------------------------------------------- %% Function: handle_info(Info, State) -> {noreply, State} | %% {noreply, State, Timeout} | %% {stop, Reason, State} %% Description: Handling all non call/cast messages %%-------------------------------------------------------------------- handle_info(_Info, State) -> {noreply, State}. %%-------------------------------------------------------------------- %% Function: terminate(Reason, State) -> void() %% Description: This function is called by a gen_server when it is about to %% terminate. It should be the opposite of Module:init/1 and do any necessary %% cleaning up. When it returns, the gen_server terminates with Reason. %% The return value is ignored. %%-------------------------------------------------------------------- terminate(_Reason, _State) -> ok. %%-------------------------------------------------------------------- %% Func: code_change(OldVsn, State, Extra) -> {ok, NewState} %% Description: Convert process state when code is changed %%-------------------------------------------------------------------- code_change(_OldVsn, State, _Extra) -> {ok, State}. %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- From adrianob@REDACTED Fri Oct 2 07:55:32 2009 From: adrianob@REDACTED (Adriano Bonat) Date: Fri, 2 Oct 2009 02:55:32 -0300 Subject: [erlang-questions] Building R13B02-1 on Snow Leopard In-Reply-To: <5c493e530910012109m4dbc97dbt3a0da5c5a5ce9061@mail.gmail.com> References: <6bb4c8cb0910012002y688b5665s8ce3d847c37ee990@mail.gmail.com> <4AC5715D.9060005@ghostgun.com> <6bb4c8cb0910012036t8068714s7aa5647c7ae6efb4@mail.gmail.com> <6bb4c8cb0910012040x15f261f3vb866497a07faa2ea@mail.gmail.com> <4AC57810.9080202@ghostgun.com> <6bb4c8cb0910012101h3be8aaadl359961bddd497711@mail.gmail.com> <5c493e530910012109m4dbc97dbt3a0da5c5a5ce9061@mail.gmail.com> Message-ID: <6bb4c8cb0910012255j778f669tf9338de408eafbda@mail.gmail.com> On Fri, Oct 2, 2009 at 1:09 AM, Ngoc Dao wrote: > Are you using MacPorts? Yes I'm. And I just did some tweaks (after several tries) to get a Portfile working for R13B02-1 :) > I had problem like this before on Leopard. May be because libs of > MacPorts (in /opt/local/lib) and libs of Leopard (in /usr/local/lib) > are not compatible. My dirty solution is moving /usr/local/lib away > and symlink /opt/local/lib there. Hm, honestly I don't know. Regards. From ajuttner.list@REDACTED Fri Oct 2 08:11:27 2009 From: ajuttner.list@REDACTED (Alpar Juttner) Date: Fri, 02 Oct 2009 08:11:27 +0200 Subject: [erlang-questions] trading systems In-Reply-To: <8412C42B-1589-4B85-9D77-7F8669307895@souja.net> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <45B769E4-C263-4288-8BDE-E3758B60868F@gmail.com> <475E92BA-FF6C-4AA5-ABAC-E0FEE99B9BCD@souja.net> <1D56639A-B9E7-43DC-8496-E622A4B8FC43@gmail.com> <8412C42B-1589-4B85-9D77-7F8669307895@souja.net> Message-ID: <1254463887.4966.157.camel@piko.site> > > Why? I interviewed with a hedge fund. They use templates extensively > > and it makes sense. Partial template specialization is just the > > ticket to optimize generic code at compile time. > Templates can drastically inflate code size It doesn't true today, due to inlining and optimization. Consider for example 'std::vector a;'. If you hane look at the header file you will see that a single statement like 'a[5]=12;' results in a tremendous amount of (nested) function calls. But the optimizer will reduce it to same machine code as the corresponding C expression would result in. > and cause an extra > indirection at function call time. I cannot even guess what do you mean by "extra indirection". Regards, Alpar From ngocdaothanh@REDACTED Fri Oct 2 08:33:57 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 2 Oct 2009 15:33:57 +0900 Subject: [erlang-questions] Switching between http/non-http depending of first byte In-Reply-To: References: <46873482-569F-4BCF-90FE-55F5B689F3E0@rogvall.se> <24d4f39c0910011420m617f6729i6a41b97efb332e6b@mail.gmail.com> Message-ID: <5c493e530910012333h41e8f635v410369083138dd45@mail.gmail.com> Smerl in ErlyWeb: http://code.google.com/p/erlyweb/source/browse/trunk/src/smerl/smerl.erl allows you to manipulate existing modules and functions just like in Ruby. You can use Smerl to monkeypatch Yaw's yaws_server module's aloop function. Example: a.erl ------ -module(a). -compile(export_all). f() -> 1. b.erl ------ -module(b). -compile(export_all). f_with_feature() -> a:f_without_feature() + 1. t.erl ----- -module(t). -compile(export_all). t() -> % Take out metamods of the 2 modules {ok, MA1} = smerl:for_module(a), {ok, MB1} = smerl:for_module(b), % Take out forms of the 2 functions {ok, FA1} = smerl:get_func(MA1, f, 0), {ok, FB1} = smerl:get_func(MB1, f_with_feature, 0), % Rename functions in the 2 forms FA2 = smerl:rename(FA1, f_without_feature), FB2 = smerl:rename(FB1, f), % Remove and add functions to module a MA2 = smerl:remove_func(MA1, f, 0), {ok, MA3} = smerl:add_func(MA2, FA2), {ok, MA4} = smerl:add_func(MA3, FB2), smerl:compile(MA4), io:format("~p~n", [a:f()]). Ngoc On Fri, Oct 2, 2009 at 1:44 PM, Max Lapshin wrote: > As I've already told, I refused from original idea, but in any case, > such thing may be useful. > There are some features in erlang, that are undocumented, but required > for normal work, > such as prim_inet:async_accept. From mjtruog@REDACTED Fri Oct 2 08:38:05 2009 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 01 Oct 2009 23:38:05 -0700 Subject: [erlang-questions] trading systems In-Reply-To: <1254463887.4966.157.camel@piko.site> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <45B769E4-C263-4288-8BDE-E3758B60868F@gmail.com> <475E92BA-FF6C-4AA5-ABAC-E0FEE99B9BCD@souja.net> <1D56639A-B9E7-43DC-8496-E622A4B8FC43@gmail.com> <8412C42B-1589-4B85-9D77-7F8669307895@souja.net> <1254463887.4966.157.camel@piko.site> Message-ID: <4AC59FCD.6050104@gmail.com> Alpar Juttner wrote: >>> Why? I interviewed with a hedge fund. They use templates extensively >>> and it makes sense. Partial template specialization is just the >>> ticket to optimize generic code at compile time. >>> >> Templates can drastically inflate code size >> > > It doesn't true today, due to inlining and optimization. Consider for > example 'std::vector a;'. If you hane look at the header file you > will see that a single statement like 'a[5]=12;' results in a tremendous > amount of (nested) function calls. But the optimizer will reduce it to > same machine code as the corresponding C expression would result in. > I agree that templates are generally fine now that we have compilers better than g++ 2.95, and that we are reaching the new C++ changes with tr1/C++0x or what will soon be C++1x. I think the main problems I have seen with templates inflating code size have related to static data/methods within the template class, so it is an issue with proper design of the system (not the language itself). From arun.suresh@REDACTED Fri Oct 2 09:02:30 2009 From: arun.suresh@REDACTED (Arun Suresh) Date: Fri, 2 Oct 2009 12:32:30 +0530 Subject: [erlang-questions] debugger not starting up Message-ID: <9f4a617f0910020002w3b078c2drb2ac6d84c0e59a4@mail.gmail.com> Hello I am currently using R13B01 on my Mac OSX 10.5.8. For some reason, I am not able to start the debugger.. I fire up erl and, if i do a debugger:start() or an im(), all i see is a white-box in the middle of the screen.. I was wondering if anybody else here has experienced anything similar.. and has found a solution to this.. -Arun From ngocdaothanh@REDACTED Fri Oct 2 09:21:07 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 2 Oct 2009 16:21:07 +0900 Subject: [erlang-questions] debugger not starting up In-Reply-To: <9f4a617f0910020002w3b078c2drb2ac6d84c0e59a4@mail.gmail.com> References: <9f4a617f0910020002w3b078c2drb2ac6d84c0e59a4@mail.gmail.com> Message-ID: <5c493e530910020021q131d39cj114cf3cce490e014@mail.gmail.com> Hi Suresh, You need wxWidgets: * Install wxWidgets (installing from MacPors is OK) * Reinstall Erlang (I recommend installing from source code) Ngoc On Fri, Oct 2, 2009 at 4:02 PM, Arun Suresh wrote: > Hello > > I am currently using R13B01 on my Mac OSX 10.5.8. > For some reason, I am not able to start the debugger.. > > I fire up erl and, if i do a debugger:start() or an im(), all i see is a > white-box in the middle of the screen.. > > I was wondering if anybody else here has experienced anything similar.. and > has found a solution to this.. > > -Arun > From raimo+erlang-questions@REDACTED Fri Oct 2 09:22:28 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 2 Oct 2009 09:22:28 +0200 Subject: [erlang-questions] tracing with erlang made easy In-Reply-To: <43CD9D82-4F02-4722-BEB0-91DF5AD9AB62@gmail.com> References: <43CD9D82-4F02-4722-BEB0-91DF5AD9AB62@gmail.com> Message-ID: <20091002072228.GA26018@erix.ericsson.se> I just want to point out a simplification possible from R13B; the built in match spec alias 'x' for the match spec [{'_',[],[{exception_trace}]}], now Joel used {return_trace} but I would recommend {exception_trace}. This makes manual tracing from the shell something like: %% Start tracing dbg:tracer(). dbg:p(all, c). % or maybe (self(), c) dbg:tpl(Mod, Fun, x). %% Stop tracing dbg:stop_clear(). See below... On Thu, Oct 01, 2009 at 09:41:56PM +0100, Joel Reymont wrote: > -module(t). > > -compile([export_all]). > > -include_lib("stdlib/include/ms_transform.hrl"). > > %%% Stop all tracing > > ut() -> > dbg:ctp(). > > %%% Trace a module > > t(Mod) > when is_atom(Mod) -> > t([Mod]); > > %%% Trace Mod:Fun (any arity) > > t({Mod, Fun}) > when is_atom(Mod), > is_atom(Fun) -> > t([{Mod, Fun}]); > > %%% Use a combination of Mod and {Mod, Fun} in a list > > t(What) -> > dbg:tracer(), > dbg:p(all, [call]), > t1(What). > > t1([]) -> > ok; > > t1([{M, F}|T]) -> > dbg:tpl(M, F, dbg:fun2ms(fun(_) -> return_trace() end)), dbg:tpl(M, F, x), > t1(T); > > t1([H|T]) -> > dbg:tpl(H, dbg:fun2ms(fun(_) -> return_trace() end)), %% If you can not remember the 'x', maybe this: dbg:tpl(M, F, exception_trace), > t1(T). > > %%% What processes are running these modules? > > mods2procs(Mods) -> > [P || M <- Mods, P <- processes(), erlang:check_process_code(P, > M)]. > > %%% What processes are running a given module > %%% that is an implementation of gen_fsm, gen_server, etc. > > p(Mod) -> > p(Mod, processes(), []). > > p(_, [], Acc) -> > Acc; > > p(Mod, [H|T], Acc) -> > {_, L} = process_info(H, dictionary), > case lists:keyfind('$initial_call', 1, L) of > {'$initial_call', {Mod, init, 1}} -> > p(Mod, T, [H|Acc]); > _ -> > p(Mod, T, Acc) > end. > > %%% Remote process info > > pi(P) > when is_pid(P) -> > rpc:call(node(P), erlang, process_info, [P]). > > --- > fastest mac firefox! > http://wagerlabs.com > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From adam@REDACTED Fri Oct 2 09:30:46 2009 From: adam@REDACTED (Adam Lindberg) Date: Fri, 2 Oct 2009 08:30:46 +0100 (BST) Subject: Variable 'Error' unsafe in 'try' Message-ID: <4635374.179021254468646193.JavaMail.root@zimbra> Hi! I'm working with some try-catch exceptions and have stumbled upon something which I think is a bit wierd. There was an explanation a year ago on the mailing list which explained why variables bound inside the try-catch expression were not safe to use when an exception got caught (http://erlang.org/pipermail/erlang-bugs/2008-November/001083.html). This is fully understandable. I've compacted my case to this minimal one: test() -> try ok catch _:Error -> Error end, try ok catch _:Error -> Error end. I get an error saying that the variable 'Error' is unsafe in the first line of the function (the first try-catch statement). With just one try-catch statement it works fine. Why is this? I was under the impression that stuff inside try-catch statements were block local, but that isn't the case? I can't access Error outside the try-catch statement so it shouldn't be a problem, should it? Cheers, Adam From raimo+erlang-questions@REDACTED Fri Oct 2 09:32:24 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 2 Oct 2009 09:32:24 +0200 Subject: [erlang-questions] characters_to_list vs characters_to_binary In-Reply-To: <5c493e530910012211m3353596fob3242417f4fe8192@mail.gmail.com> References: <5c493e530910012211m3353596fob3242417f4fe8192@mail.gmail.com> Message-ID: <20091002073224.GB26018@erix.ericsson.se> On Fri, Oct 02, 2009 at 02:11:36PM +0900, Ngoc Dao wrote: > I am using Erlang R13B02-1 with Japanese data. > > The doc of unicode module says that their results are the same, only > one in list and one in binary. But: > > unicode:characters_to_list([230,151,165,230,156,172,232,170,158,231,137,136]). > => [230,151,165,230,156,172,232,170,158,231,137,136] > > unicode:characters_to_binary([230,151,165,230,156,172,232,170,158,231,137,136]). > => <<195,166,194,151,194,165,195,166,194,156,194,172,195,168, > 194,170,194,158,195,167,194,137,194,136>> > > Is there something wrong? I think you misinterpreted the documentation. The results are the same i.e the same character sequence, but the binary result from unicode:characters_to_binary/1 is encoded in utf8 while unicode:characters_to_list/1 simply returns a list of unicode code points (character codes). unicode:characters_to_list(<<195,166,194,151,194,165,195,166, 194,156,194,172,195,168,194,170,194,158,195,167,194,137,194,136>>). => [230,151,165,230,156,172,232,170,158,231,137,136] The basic principle: binaries are encoded, lists just contain character codes (and (encoded) binaries). > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From rvirding@REDACTED Fri Oct 2 09:37:58 2009 From: rvirding@REDACTED (Robert Virding) Date: Fri, 2 Oct 2009 09:37:58 +0200 Subject: [erlang-questions] binary_to_term and Leaking Atoms In-Reply-To: <5D7992C1-09E6-4B52-8FEB-6851645FF1AB@souja.net> References: <7336B504-11BE-4864-A0AF-DF731D608169@souja.net> <5D7992C1-09E6-4B52-8FEB-6851645FF1AB@souja.net> Message-ID: <3dbc6d1c0910020037g68c85e20l8899c523d25a79d@mail.gmail.com> There is always the obvious solution: decode the binary yourself and use XXX_to_existing_atom to be safe. It won't be as fast but the binary encoding is relativly straight forward and shouldn't cause any problems. Robert 2009/10/2 Jayson Vantuyl > That works if I was just sending an atom as a binary, but I was hoping to > send whole data structures using term_to_binary. Looks like there's no way > to make that atom safe, then? > > On Oct 1, 2009, at 9:46 PM, Max Lapshin wrote: > > There are erlang:binary_to_existing_atom/2, >> erlang:list_to_existing_atom/1 >> > > > -- > Jayson Vantuyl > kagato@REDACTED > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From kagato@REDACTED Fri Oct 2 10:40:00 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 2 Oct 2009 01:40:00 -0700 Subject: [erlang-questions] Variable 'Error' unsafe in 'try' In-Reply-To: <4635374.179021254468646193.JavaMail.root@zimbra> References: <4635374.179021254468646193.JavaMail.root@zimbra> Message-ID: <990ADFAD-E68B-4CD0-BC3B-4D2E4F80DB6E@souja.net> The previous example (from the mailing list last year) doesn't exactly apply. In that case, test1 and test2 are different functions, so there's no danger of Error being bound in the second function. In that particular case, it has to do with the fact that a binding in an after clause isn't handled in a way that the compiler thinks is safe (even though it should be). In your case, since they are in the same function, the second Error is already potentially bound to the result of the first try and the second usage considers the use of Error in the catch clause to already be bound. Since the first try has an exit path that doesn't bind Error (namely, the successful path), it's unsafe use of the binding anyways. This works: > test() -> > try ok catch > _:Error0 -> Error0 > end, > try ok catch > _:Error1 -> Error1 > end. Within a function in Erlang, a binding NEVER goes "out-of-scope" or are in any way "unbound". Well, that's not entirely true, there are three times that I know of: 1. Variables defined in a fun() inside of a function go out of (lexical) scope at the end of the fun(). 2. Variables can be shadowed in a nested function head (effectively unbinding the parent variable). 3. Variables can be shadowed by bindings inside of a generator. Consider the following: > test(X) -> > fun(X) -> > [X || X <- X] > end. Mind you, this is unnatural code-that-should-not-be. On Oct 2, 2009, at 12:30 AM, Adam Lindberg wrote: > Hi! > > I'm working with some try-catch exceptions and have stumbled upon > something which I think is a bit wierd. There was an explanation a > year ago on the mailing list which explained why variables bound > inside the try-catch expression were not safe to use when an > exception got caught (http://erlang.org/pipermail/erlang-bugs/2008-November/001083.html > ). This is fully understandable. > > I've compacted my case to this minimal one: > > test() -> > try ok catch > _:Error -> Error > end, > try ok catch > _:Error -> Error > end. > > I get an error saying that the variable 'Error' is unsafe in the > first line of the function (the first try-catch statement). With > just one try-catch statement it works fine. > > Why is this? I was under the impression that stuff inside try-catch > statements were block local, but that isn't the case? I can't access > Error outside the try-catch statement so it shouldn't be a problem, > should it? > > Cheers, > Adam > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From yuriy.pasichnyk@REDACTED Fri Oct 2 11:31:26 2009 From: yuriy.pasichnyk@REDACTED (Yuriy Pasichnyk) Date: Fri, 2 Oct 2009 11:31:26 +0200 (CEST) Subject: Network communications fails with R13B02-1 on OS X In-Reply-To: <1254422616.20559.ezmlm@erlang.org> References: <1254422616.20559.ezmlm@erlang.org> Message-ID: <76cda584ccdb9f138ef831789d70f4f3.squirrel@www.and.com> Hi Dave, I had the same issue recently on Mac OS (cannot check right now what version of OTP it was) and it was caused by -name / -sname (Mac OS handles hostnames differently than Linux). Hope it help, Yuriy > Yesterday I upgraded to otp_13B02-1 from R12-5 on OS X 10.5.8. I > notice today that any form of network communication between nodes > seems to fail. Trying to start slaves on the same host (which I was > able to do before the upgrade) fails with an {error,timeout}. Even > doing a simple 'net:ping(node1@REDACTED)' on the same host between two > nodes fails with a pang. I can run the same exact tests on Ubuntu > with R13B02-1 and they pass with flying colors. Has anyone else > noticed this problem on OS X? Any ideas what the problem may be? > > Thanks! > Dave > > From ofer.affias@REDACTED Fri Oct 2 12:33:11 2009 From: ofer.affias@REDACTED (vim) Date: Fri, 2 Oct 2009 03:33:11 -0700 (PDT) Subject: mneisa table fragmentation vs manually splitting In-Reply-To: <72ddfd5c-1ef8-43c6-8fe2-776d0debf056@c28g2000yqd.googlegroups.com> References: <05840888-1633-4702-a3ce-839b038fe9a4@m20g2000vbp.googlegroups.com> <72ddfd5c-1ef8-43c6-8fe2-776d0debf056@c28g2000yqd.googlegroups.com> Message-ID: <2ae6cef3-57f2-45b5-aaff-0424618af112@o41g2000yqb.googlegroups.com> hi Z, thanks for your reply, > why do you claim the application should know how to use a specific node to write to a specific key? i need to know in which cluster resides the key i need. i.e. if i have partitioned my database to 100 clusters, in need to keep a map which tell me on which cluster i can found a specific key. this is not related to the master-slave scheme, even if i have just 1 machine on each cluster, i need to do the same. the master-slave scheme is a just a possible enhancement, so i can have hot-backup for each key-value. on the other hand, if i use mnesia fragmented database feature, mnesia (as far as i understand) should keep track on where each key resides, releasing me from keeping that key mapping. the disadvantage i fear is the up-scale limit of connecting many machines together without partitioning into clusters. from what i have read here and there, erlang shouldn't be connected together on more than few hundreds active machine. * if anyone has numbers of this practical limit, please advise * > What are the master-slave relations you assume in the 10 machine cluster? master-slave refers to a scheme where i connect all the mnesia instances on the relevant machines, so they are synced. but i choose split up my dataset, where for a given key i only do writes through specific machine/node. on that kind of scheme i get fault-tolerance because if the master (the writer) fails, i always have a backup (slave) with the most up to date data. and i can assign one of the slave to be a master if the master fails. furthermore, i don't get into many problems of multiple write, and syncing them, when you use master-master scheme (more than 1 writer) vim. On Oct 1, 12:31 am, Zohan wrote: > hi, > > For the manual split option you describe, why do you claim > the application should know how to use a specific node to write to a > specific key, > in order to use master-slave duplicated database scheme? > What are the master-slave relations you assume in the 10 machine > cluster? > > BR, > Z. > > On Sep 27, 8:40 am, vim wrote: > > > > > > > hi, > > > i am working on a erlang based distributed architecture, at the end i > > intend to use it on EC2. my application included a database, and since > > i want to spread the load on the database, i thought of splitting the > > table. > > > the question is should i choose mnesia built-in feature 'fragmented > > table' over manually splitting the database into clusters? > > > i.e. suppose i have a simple key-value sets which i need to store in > > my database. and suppose i have 1000 keys, and i did some measurements > > and i want only 10 keys on each machine (for better performance) so i > > need 100 machines. thus, if i understand correctly the > > documentation, i have at least two options: > > > 1. manual split: > > i can set up clusters of 10 machines each. on each machine i will have > > a node with a running mnesia, and on a such cluster all the mnesia > > nodes will be connected. furthermore, i can use master-slave > > duplicated database scheme, if my application will know how to use a > > specific node to write to a specific key. i assume here that *all* the > > nodes in my world (100) are erlang connected, and on each 10 machine > > cluster there is a connected (shared) mnesia database. > > > 2. table fragmentation: > > i can connect all my 100 machines with single mnesia schema, and > > fragment the table into 10 pieces, where each cluster of 10 machines > > will share each piece. again, i can use the same master-slave scheme i > > described above to access my database. > > > am i correct with those assumptions? > > should i prefer one method over the other? > > > thanks for your help, > > vim. > > > ________________________________________________________________ > > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From arun.suresh@REDACTED Fri Oct 2 14:34:25 2009 From: arun.suresh@REDACTED (Arun Suresh) Date: Fri, 2 Oct 2009 18:04:25 +0530 Subject: [erlang-questions] debugger not starting up In-Reply-To: <5c493e530910020021q131d39cj114cf3cce490e014@mail.gmail.com> References: <9f4a617f0910020002w3b078c2drb2ac6d84c0e59a4@mail.gmail.com> <5c493e530910020021q131d39cj114cf3cce490e014@mail.gmail.com> Message-ID: <9f4a617f0910020534h69ff2b71k3262dfa293f64548@mail.gmail.com> As of now... I have tried all of the following.. * reinstall erlang R13B01 and wxwidgets 2.8.10 via macports. * build elrang R13B01 from source AND wxwidgets 2.8.10 from source.. * build elrang R13B02-1 from source AND wxwidgets 2.8.10 from source.. In each case... > wx:new(). just results in a boring white rectangle in the middle of the screen. any suggestions anyone ? -Arun On Fri, Oct 2, 2009 at 12:51 PM, Ngoc Dao wrote: > Hi Suresh, > > You need wxWidgets: > * Install wxWidgets (installing from MacPors is OK) > * Reinstall Erlang (I recommend installing from source code) > > Ngoc > > > On Fri, Oct 2, 2009 at 4:02 PM, Arun Suresh wrote: > > Hello > > > > I am currently using R13B01 on my Mac OSX 10.5.8. > > For some reason, I am not able to start the debugger.. > > > > I fire up erl and, if i do a debugger:start() or an im(), all i see is a > > white-box in the middle of the screen.. > > > > I was wondering if anybody else here has experienced anything similar.. > and > > has found a solution to this.. > > > > -Arun > > > From roberto.aloi@REDACTED Fri Oct 2 14:48:29 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Fri, 02 Oct 2009 13:48:29 +0100 Subject: [erlang-questions] Better use of io:format and io_lib:format In-Reply-To: <4AC489DA.5080203@adinet.com.uy> References: <4AC489DA.5080203@adinet.com.uy> Message-ID: <4AC5F69D.8050404@erlang-consulting.com> Hi Bernardo, you might want to try something like this: -module(benchmark). -export([test/1, with_list/1, with_binary/1]). with_list(0) -> ok; with_list(N) -> io:format("hello ~s", ["world"]), with_list(N-1). with_binary(0) -> ok; with_binary(N) -> io:format(<<"hello ~s">>, [<<"world">>]), with_binary(N-1). test(Num) -> {TimeList, _} = timer:tc(?MODULE, with_list, [Num]), {TimeBinary, _} = timer:tc(?MODULE, with_binary, [Num]), {{with_list, TimeList}, {with_binary, TimeBinary}}. You will be able to see how performances degrade for strings by trying the test/1 function with different values. For example: >benchmark:test(1). {{with_list,32},{with_binary,19}} > benchmark:test(10). {{with_list,6253},{with_binary,262}} > benchmark:test(100). {{with_list,11329},{with_binary,5572}} It would be nice to try this with R12 and R13, for a comparison. You might also be interested in the function: memory(). Regards, Roberto Aloi Erlang Training and Consulting Ltd. http://erlang-consulting.com http://aloiroberto.wordpress.com Bernardo Alvez wrote: > Hi all, > > Which of the following is better to use in terms of performance and > resources usage?: > > io:format("hello ~s", ["world"]). > or > io:format(<<"hello ~s">>, [<<"world">>]). > > Thanks, > Bernardo > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ngocdaothanh@REDACTED Fri Oct 2 15:28:26 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 2 Oct 2009 22:28:26 +0900 Subject: [erlang-questions] debugger not starting up In-Reply-To: <9f4a617f0910020534h69ff2b71k3262dfa293f64548@mail.gmail.com> References: <9f4a617f0910020002w3b078c2drb2ac6d84c0e59a4@mail.gmail.com> <5c493e530910020021q131d39cj114cf3cce490e014@mail.gmail.com> <9f4a617f0910020534h69ff2b71k3262dfa293f64548@mail.gmail.com> Message-ID: <5c493e530910020628s4fc3071bx8fb4c6ecbe2e2133@mail.gmail.com> Just in case... you should install wxWidgets first, then Erlang. When compiling Erlang from source, you should check the output of ./configure command to see if it can find wxWidgets. On Fri, Oct 2, 2009 at 9:34 PM, Arun Suresh wrote: > As of now... I have tried all of the following.. > > * reinstall erlang R13B01 and wxwidgets 2.8.10 via macports. > * build elrang R13B01 from source AND wxwidgets 2.8.10 from source.. > * build elrang R13B02-1 from source AND wxwidgets 2.8.10 from source.. > > In each case... >> wx:new(). > just results > in a boring white rectangle in the middle of the screen. > > any suggestions anyone ? > > -Arun > > On Fri, Oct 2, 2009 at 12:51 PM, Ngoc Dao wrote: >> >> Hi Suresh, >> >> You need wxWidgets: >> * Install wxWidgets (installing from MacPors is OK) >> * Reinstall Erlang (I recommend installing from source code) >> >> Ngoc >> >> >> On Fri, Oct 2, 2009 at 4:02 PM, Arun Suresh wrote: >> > Hello >> > >> > I am currently using R13B01 on my Mac OSX 10.5.8. >> > For some reason, I am not able to start the debugger.. >> > >> > I fire up erl and, if i do a debugger:start() or an im(), all i see is a >> > white-box in the middle of the screen.. >> > >> > I was wondering if anybody else here has experienced anything similar.. >> > and >> > has found a solution to this.. >> > >> > -Arun >> > > > From victor.sovetov@REDACTED Fri Oct 2 18:48:28 2009 From: victor.sovetov@REDACTED (Viktor Sovietov) Date: Fri, 2 Oct 2009 09:48:28 -0700 (PDT) Subject: trading systems In-Reply-To: References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> <50B67FF1-0EF1-4393-B0D4-875A17125D87@gmail.com> Message-ID: <3c6f9578-c760-4fce-8ea9-14b113663b9c@37g2000yqm.googlegroups.com> Joel, Do you really think that software-based trading systems would survive in the near future? Honestly, specifically accelerated hardware solutions (most probably FPGA-based) should win the battle. This discussion reminded me what Marc Battyani (you surely know him as the Lisp expert) is doing right now in http://www.hpcplatform.com/ - it's the compiler for FPGA accelerators, mostly targeted to serve financial market's needs. Now it can look as a far perspective, but... it's so typical for IT techs - most of them looked unobvious or unproductive just a few years ago. Sincerely, --Victor From joelr1@REDACTED Fri Oct 2 19:17:17 2009 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 2 Oct 2009 18:17:17 +0100 Subject: [erlang-questions] Re: trading systems In-Reply-To: <3c6f9578-c760-4fce-8ea9-14b113663b9c@37g2000yqm.googlegroups.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> <50B67FF1-0EF1-4393-B0D4-875A17125D87@gmail.com> <3c6f9578-c760-4fce-8ea9-14b113663b9c@37g2000yqm.googlegroups.com> Message-ID: <5EF102BE-8558-4594-A46B-0592A6F7D647@gmail.com> On Oct 2, 2009, at 5:48 PM, Viktor Sovietov wrote: > Do you really think that software-based trading systems would survive > in the near future? Sure. Software can always be baked into hardware. > Honestly, specifically accelerated hardware > solutions (most probably FPGA-based) should win the battle. This > discussion reminded me what Marc Battyani I referred to Marc's solution in the beginning of this thread ;-). --- fastest mac firefox! http://wagerlabs.com From victor.sovetov@REDACTED Fri Oct 2 20:07:28 2009 From: victor.sovetov@REDACTED (Viktor Sovietov) Date: Fri, 2 Oct 2009 11:07:28 -0700 (PDT) Subject: trading systems In-Reply-To: <5EF102BE-8558-4594-A46B-0592A6F7D647@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> <50B67FF1-0EF1-4393-B0D4-875A17125D87@gmail.com> <3c6f9578-c760-4fce-8ea9-14b113663b9c@37g2000yqm.googlegroups.com> <5EF102BE-8558-4594-A46B-0592A6F7D647@gmail.com> Message-ID: > > Honestly, specifically accelerated hardware > > solutions (most probably FPGA-based) should win the battle. This > > discussion reminded me what ?Marc Battyani > > I referred to Marc's solution in the beginning of this thread ;-). Ah, sorry, sometimes I'm so inattentive :) But what I was going to say... Probably, low latency trading is too specific problem area for Erlang. Erlang can find a niche as the language for writing monitoring and management software (and there are no others can beat Erlang on this field) but not for implementing main functionality of such systems. It's like the situation with HPC. Almost all production software for parallel supercomputers is MPI- based, but Erlang is inapplicable in this area because it's great in MPI but not in calculations, and even great MPI model of Erlang is too expensive for HPC purposes. Sincerely, --Victor From andrewmmc@REDACTED Fri Oct 2 20:09:43 2009 From: andrewmmc@REDACTED (andrew mmc) Date: Fri, 2 Oct 2009 20:09:43 +0200 Subject: [erlang-questions] Re: trading systems In-Reply-To: <5EF102BE-8558-4594-A46B-0592A6F7D647@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> <06EF4170-CDE7-4483-B8FB-E1074CE3429D@gmail.com> <50B67FF1-0EF1-4393-B0D4-875A17125D87@gmail.com> <3c6f9578-c760-4fce-8ea9-14b113663b9c@37g2000yqm.googlegroups.com> <5EF102BE-8558-4594-A46B-0592A6F7D647@gmail.com> Message-ID: Is this a project for a big bank or something else? Have you got any idea of what level of performance you need? How close are you to the exchange, for example? There isn't much point in slaving over unit milliseconds when you are 400ms from the exchange, for example. Personally, unless you are doing this with one of the very big banks, I think you are fighting a losing battle. The big players (and the only people actually making money doing this) colocate their trading servers with the exchange servers - in the same room, with dedicated network connections. Unless you can compete with that level of funding and access, there are better alternatives to make money trading. On Fri, Oct 2, 2009 at 7:17 PM, Joel Reymont wrote: > > On Oct 2, 2009, at 5:48 PM, Viktor Sovietov wrote: > > Do you really think that software-based trading systems would survive >> in the near future? >> > > Sure. Software can always be baked into hardware. > > Honestly, specifically accelerated hardware >> solutions (most probably FPGA-based) should win the battle. This >> discussion reminded me what Marc Battyani >> > > I referred to Marc's solution in the beginning of this thread ;-). > > --- > fastest mac firefox! > http://wagerlabs.com > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From tony@REDACTED Fri Oct 2 23:12:55 2009 From: tony@REDACTED (Tony Arcieri) Date: Fri, 2 Oct 2009 15:12:55 -0600 Subject: [erlang-questions] binary_to_term and Leaking Atoms In-Reply-To: <7336B504-11BE-4864-A0AF-DF731D608169@souja.net> References: <7336B504-11BE-4864-A0AF-DF731D608169@souja.net> Message-ID: On Thu, Oct 1, 2009 at 10:42 PM, Jayson Vantuyl wrote: > I'm working on a network protocol. The easy way to do it is just use > term_to_binary and binary_to_term. There is one problem, though. > > I don't necessarily trust the sender of these binaries. It seems that > decoding them should be relatively safe, except for the potential for > someone to leak atoms by creating a large number of atoms until I run out of > memory. I presume this sort of thing is the reason that > binary_to_existing_atom (and list_to_existing_atom) were created. > > A few questions: > > 1. Is there any facility to "safely" decode binaries? > You might check out mojombo's BERT (Binary ERlang Term) serialization format: http://github.com/mojombo/ernie -- Tony Arcieri Medioh/Nagravision From ckerr@REDACTED Sat Oct 3 00:05:14 2009 From: ckerr@REDACTED (Cameron Kerr) Date: Sat, 03 Oct 2009 11:05:14 +1300 Subject: How to format a local timestamp? Message-ID: <4AC6791A.6080708@cs.otago.ac.nz> I must surely be missing something in the documentation, but I cannot find any easy way to generate a human-readable timestamp easily in Erlang. I would have expected something in the calendar module, but there is nothing that returns a string. I've hunted and I found something for RFC1123 format in httpd_util, but that is always in GMT, and I would much prefer something more familiar to the user (ie. localtime, with or without timezone). Surely there must be a standard function for doing something like this... until then, I'll just ~p calendar:local_time(). Thanks, Cameron From bob@REDACTED Sat Oct 3 00:25:36 2009 From: bob@REDACTED (Bob Ippolito) Date: Fri, 2 Oct 2009 15:25:36 -0700 Subject: [erlang-questions] How to format a local timestamp? In-Reply-To: <4AC6791A.6080708@cs.otago.ac.nz> References: <4AC6791A.6080708@cs.otago.ac.nz> Message-ID: <6a36e7290910021525xfc53826ida1e3977a3f514de@mail.gmail.com> Sadly there isn't, but you might be able to cut and paste something from one of these *11 different modules* that ship with Erlang: $ find . -name "*.erl"|xargs grep '^month(12)' ./common_test-1.4.2/src/ct_logs.erl:month(12) -> "Dec". ./common_test-1.4.2/src/ct_master_logs.erl:month(12) -> "Dec". ./ic-4.2.21/src/ic_pp.erl:month(12) -> "Dec". ./inets-5.1/src/httpd_util.erl:month(12) -> "Dec". ./sasl-2.1.6/src/sasl_report.erl:month(12) -> "Dec". ./snmp-4.13.3/src/compiler/snmpc_mib_to_hrl.erl:month(12) -> "Dec". ./stdlib-1.16.2/src/c.erl:month(12) -> "December". ./stdlib-1.16.2/src/erl_tar.erl:month(12) -> "Dec". ./stdlib-1.16.2/src/error_logger_file_h.erl:month(12) -> "Dec". ./stdlib-1.16.2/src/error_logger_tty_h.erl:month(12) -> "Dec". ./stdlib-1.16.2/src/zip.erl:month(12) -> "Dec". On Fri, Oct 2, 2009 at 3:05 PM, Cameron Kerr wrote: > I must surely be missing something in the documentation, but I cannot find > any easy way to generate a human-readable timestamp easily in Erlang. > > I would have expected something in the calendar module, but there is nothing > that returns a string. I've hunted and I found something for RFC1123 format > in httpd_util, but that is always in GMT, and I would much prefer something > more familiar to the user (ie. localtime, with or without timezone). > > Surely there must be a standard function for doing something like this... > until then, I'll just ~p ?calendar:local_time(). > > Thanks, > Cameron > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From roberto.aloi@REDACTED Sat Oct 3 01:06:14 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Sat, 3 Oct 2009 00:06:14 +0100 (BST) Subject: [erlang-questions] How to format a local timestamp? In-Reply-To: <4AC6791A.6080708@cs.otago.ac.nz> Message-ID: <16970208.180621254524774800.JavaMail.root@zimbra> Hi Cameron, this has been implemented within the Erlang Web Framework, available here: http://www.erlang-web.org/ Please, have a look to the wtype_datetime module. Documentation can be found here: http://wiki.erlang-web.org/Wparts/BasicTypes/datetime Regards, Roberto Aloi roberto.aloi@REDACTED http://www.erlang-consulting.com http://aloiroberto.wordpress.com ----- Messaggio originale ----- Da: "Cameron Kerr" A: "Erlang Questions" Inviato: Venerd?, 2 ottobre 2009 23:05:14 GMT +00:00 GMT Gran Bretagna, Irlanda, Portogallo Oggetto: [erlang-questions] How to format a local timestamp? I must surely be missing something in the documentation, but I cannot find any easy way to generate a human-readable timestamp easily in Erlang. I would have expected something in the calendar module, but there is nothing that returns a string. I've hunted and I found something for RFC1123 format in httpd_util, but that is always in GMT, and I would much prefer something more familiar to the user (ie. localtime, with or without timezone). Surely there must be a standard function for doing something like this... until then, I'll just ~p calendar:local_time(). Thanks, Cameron ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From colm.dougan@REDACTED Sat Oct 3 17:58:56 2009 From: colm.dougan@REDACTED (Colm Dougan) Date: Sat, 3 Oct 2009 16:58:56 +0100 Subject: [erlang-questions] How to format a local timestamp? In-Reply-To: <4AC6791A.6080708@cs.otago.ac.nz> References: <4AC6791A.6080708@cs.otago.ac.nz> Message-ID: <24d4f39c0910030858y7fd891c6k4e602617ea26c136@mail.gmail.com> On Fri, Oct 2, 2009 at 11:05 PM, Cameron Kerr wrote: > I must surely be missing something in the documentation, but I cannot find > any easy way to generate a human-readable timestamp easily in Erlang. erlydtl contains a self-contained date formatting module : http://erlydtl.googlecode.com/svn/trunk/src/erlydtl/erlydtl_dateformat.erl e.g. Eshell V5.6.5 (abort with ^G) 1> erlydtl_dateformat:format(erlang:localtime(), "Y-m-d H:i:s"). "2009-10-03 16:55:40" It implements most of the tags of the original django date formatting, (which itself is based on http://php.net/date) which is documented here : http://docs.djangoproject.com/en/dev/ref/templates/builtins/#now The erlang implementation lacks some of the time-zone formatting calls. Colm From saleyn@REDACTED Sun Oct 4 15:41:49 2009 From: saleyn@REDACTED (Serge Aleynikov) Date: Sun, 04 Oct 2009 09:41:49 -0400 Subject: [erlang-questions] trading systems In-Reply-To: <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> References: <748ECB13-0FB1-4978-9A0D-12928D052504@gmail.com> <59A1A46A-9F57-414B-888B-75F8A502E64D@gmail.com> Message-ID: <4AC8A61D.9040204@aleynikov.org> Joel Reymont wrote: > Kenneth, > > On Oct 1, 2009, at 4:50 PM, Kenneth Lundin wrote: > >> It would be nice if you could explain what the system needs to do and >> after that >> why you think it is a problem to store prices in an ETS-table. >> [...] >> Why are you talking about mmapped files? Please explain for a novice >> in trading systems. > > It's high-frequency trading systems I'm talking about. Where's Serge > Aleynikov [1] when you need him? Still here... > I'd like to process an incoming price quote, make a decision and submit > a trade order with minimal latency. I know that this is normally done > using C++ (or OCaml [1]) but I'm wondering how this can be achieved > using Erlang. Don't want to be too repetitive to answers already posted here, but what is your definition of "minimal latency"? If you are talking about microseconds, having much logic coded in Erlang will not help you. At the end you will end up with basic configuration/controlling functionality in Erlang navigating a low-level component coded in some low-level language (unless of course you are willing to invest time in writing some domain-specific language for the task, which might be an overkill given the scope of your work). > A simple array of floats or doubles groups data together in memory and > is very cache friendly. I figure that an Erlang binary is the closest I > can get to an array in Erlang. mmap-ing a file of floats or doubles and > wrapping it in a binary should avoid unnecessary copying of data from > disk to memory. Even if that component is fast, you'll still have a bytecode-related overhead on the Erlang side reading/writing data to this structure through copying. It is ok (and very convenient) for most applications, but for what you are talking about that overhead may be a show stopper. > I think operating on binaries representing mmap-ed files of price quotes > is as close as you can get to low latency in Erlang. I know that > premature optimization is the root of all evil but I'd rather have my > teeth pulled than optimize Erlang. This is speaking from experience. I > figure a much better way is to choose a concept that guarantees fast > code from the start. If you don't want to spend too much time coding C++, since you have already done good work of abstracting mnesia backend, why don't you create an Erlang driver for this database: www.fastdb.org. It's based on memory mapped files, implemented in C++, is relatively fast compared to other in-memory databases (about 250k lookups/s, 50k inserts/s), has time-series capabilities, simple C-API. I implemented FastDB interface in several languages, and generally it's good for systems that deal with latencies in sub-milliseconds range. It may suffice for you unless you need to stay in the low microseconds realm. My 2c. Serge From kiszl@REDACTED Sun Oct 4 21:26:28 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sun, 04 Oct 2009 21:26:28 +0200 Subject: Prebuilt PLTs for Dialyzer Message-ID: <4AC8F6E4.5010600@tmit.bme.hu> Hello. Would it be possible to distribute a prebuilt .dialyzer_plt file for the erlang/OTP releases (either packed or separately) that we could just download and use? It seems to take forever building these PLTs, even for the core "applications"... Thanks. -- Zoltan. From matan.levy@REDACTED Mon Oct 5 00:05:50 2009 From: matan.levy@REDACTED (Zohan) Date: Sun, 4 Oct 2009 15:05:50 -0700 (PDT) Subject: mneisa table fragmentation vs manually splitting In-Reply-To: <2ae6cef3-57f2-45b5-aaff-0424618af112@o41g2000yqb.googlegroups.com> References: <05840888-1633-4702-a3ce-839b038fe9a4@m20g2000vbp.googlegroups.com> <72ddfd5c-1ef8-43c6-8fe2-776d0debf056@c28g2000yqd.googlegroups.com> <2ae6cef3-57f2-45b5-aaff-0424618af112@o41g2000yqb.googlegroups.com> Message-ID: <7a66d4c4-0da7-424f-b474-e1d12c741c15@o35g2000vbi.googlegroups.com> I have no practical experience with using the mnesia 'fragmented table' but it makes sense to reuse it and not invent it. i wonder how it scales up, as you stated. Z. On Oct 2, 12:33?pm, vim wrote: > hi Z, thanks for your reply, > > > why do you claim the application should know how to use a specific node to write to a specific key? > > i need to know in which cluster resides the key i need. i.e. if i have > partitioned my database to 100 clusters, in need to keep a map which > tell me on which cluster i can found a specific key. > > this is not related to the master-slave scheme, even if i have just 1 > machine on each cluster, i need to do the same. the master-slave > scheme is a just a possible enhancement, so i can have hot-backup for > each key-value. > > on the other hand, if i use mnesia fragmented database feature, mnesia > (as far as i understand) should keep track on where each key resides, > releasing me from keeping that key mapping. > > the disadvantage i fear is the up-scale limit of connecting many > machines together without partitioning into clusters. from what i have > read here and there, erlang shouldn't be connected together on more > than few hundreds active machine. * if anyone has numbers of this > practical limit, please advise * > > > What are the master-slave relations you assume in the 10 machine cluster? > > master-slave refers to a scheme where i connect all the mnesia > instances on the relevant machines, so they are synced. but i choose > split up my dataset, where for a given key i only do writes through > specific machine/node. on that kind of scheme i get fault-tolerance > because if the master (the writer) fails, i always have a backup > (slave) with the most up to date data. and i can assign one of the > slave to be a master if the master fails. furthermore, i don't get > into many problems of multiple write, and syncing them, when you use > master-master scheme (more than 1 writer) > > vim. > > On Oct 1, 12:31 am, Zohan wrote: > > > > > hi, > > > For the manual split option you describe, why do you claim > > the application should know how to use a specific node to write to a > > specific key, > > in order to use master-slave duplicated database scheme? > > What are the master-slave relations you assume in the 10 machine > > cluster? > > > BR, > > Z. > > > On Sep 27, 8:40 am, vim wrote: > > > > hi, > > > > i am working on a erlang based distributed architecture, at the end i > > > intend to use it on EC2. my application included a database, and since > > > i want to spread the load on the database, i thought of splitting the > > > table. > > > > the question is should i choose mnesia built-in feature 'fragmented > > > table' over manually splitting the database into clusters? > > > > i.e. suppose i have a simple key-value sets which i need to store in > > > my database. and suppose i have 1000 keys, and i did some measurements > > > and i want only 10 keys on each machine (for better performance) so i > > > need 100 machines. ?thus, if i understand correctly the > > > documentation, i have at least two options: > > > > 1. manual split: > > > i can set up clusters of 10 machines each. on each machine i will have > > > a node with a running mnesia, and on a such cluster all the mnesia > > > nodes will be connected. furthermore, i can use master-slave > > > duplicated database scheme, if my application will know how to use a > > > specific node to write to a specific key. i assume here that *all* the > > > nodes in my world (100) are erlang connected, and on each 10 machine > > > cluster there is a connected (shared) mnesia database. > > > > 2. table fragmentation: > > > i can connect all my 100 machines with single mnesia schema, and > > > fragment the table into 10 pieces, where each cluster of 10 machines > > > will share each piece. again, i can use the same master-slave scheme i > > > described above to access my database. > > > > am i correct with those assumptions? > > > should i prefer one method over the other? > > > > thanks for your help, > > > vim. > > > > ________________________________________________________________ > > > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > > > erlang-questions (at) erlang.org > > > ________________________________________________________________ > > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From tuncer.ayaz@REDACTED Mon Oct 5 00:34:06 2009 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 5 Oct 2009 00:34:06 +0200 Subject: [erlang-questions] Memory Usage In-Reply-To: <20090929164832.GF22515@hijacked.us> References: <3C44D46958109E408F5848593B05A1D8C5DAF8F5@rrsmsx505.amr.corp.intel.com> <20090929164832.GF22515@hijacked.us> Message-ID: <4ac8254d0910041534q109af757tbd6212797d2e8c18@mail.gmail.com> On Tue, Sep 29, 2009 at 6:48 PM, Andrew Thompson wrote: > So yeah, binaries are a much better solution for any larger data, > they're just a little tricker to work with because the bitstring library > proposed by EEP9 doesn't seem to exist. Good question. Fredrik, any idea on EEP9's status? What happened to your patches? From kostis@REDACTED Mon Oct 5 10:10:03 2009 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 05 Oct 2009 11:10:03 +0300 Subject: [erlang-questions] Prebuilt PLTs for Dialyzer In-Reply-To: <4AC8F6E4.5010600@tmit.bme.hu> References: <4AC8F6E4.5010600@tmit.bme.hu> Message-ID: <4AC9A9DB.8030808@cs.ntua.gr> Zoltan Lajos Kis wrote: > Hello. > > Would it be possible to distribute a prebuilt .dialyzer_plt file for the > erlang/OTP releases (either packed or separately) that we could just > download and use? > It seems to take forever building these PLTs, even for the core > "applications"... I cannot answer the first question, but are you aware that the PLT can be built incrementally and you can add information on an as-needed basis? What you can do is the following. Start with a minimal PLT: one that contains information about say "kernel" and "stdlib" only dialyzer --build_plt -r $ERL_TOP/lib/kernel/ebin $ERL_TOP/lib/stdlib/ebin Indeed, it currently takes some time to build this PLT, but considerably shorter than "forever" ;-) Then analyze your code with this PLT. If dialyzer complains that you are missing some info from say a mnesia file, you can expand the PLT you just built with the missing information by issuing the command: dialyzer --add_to_plt -r $ERL_TOP/lib/mnesia/ebin As long as you do not modify any file in kernel, stdlib, or mnesia (or anything else that you have included in your PLT for that matter), the PLT will stay consistent and usable. Not an ideal solution, but this should be close enough to what you want. Kostis From roberto.aloi@REDACTED Mon Oct 5 11:19:56 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Mon, 05 Oct 2009 10:19:56 +0100 Subject: [erlang-questions] Prebuilt PLTs for Dialyzer In-Reply-To: <4AC8F6E4.5010600@tmit.bme.hu> References: <4AC8F6E4.5010600@tmit.bme.hu> Message-ID: <4AC9BA3C.3070904@erlang-consulting.com> Hi, just as a note, you might want to use the following: dialyzer --check_plt --plt my.plt when you change something to one of your applications. For more information about incremental usage of .plt see: http://www.erlang.org/doc/apps/dialyzer/dialyzer_chapter.html Regards, Roberto Aloi roberto.aloi@REDACTED http://www.erlang-consulting.com http://aloiroberto.wordpress.com Zoltan Lajos Kis wrote: > Hello. > > Would it be possible to distribute a prebuilt .dialyzer_plt file for > the erlang/OTP releases (either packed or separately) that we could > just download and use? > It seems to take forever building these PLTs, even for the core > "applications"... > > Thanks. > > -- > Zoltan. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From ngocdaothanh@REDACTED Mon Oct 5 11:31:49 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Mon, 5 Oct 2009 18:31:49 +0900 Subject: [erlang-questions] UTF8 and EDoc In-Reply-To: <4AC326DA.8020309@gmail.com> References: <5c493e530909300102x2d929ad5l9bb468d6eb390c55@mail.gmail.com> <4AC326DA.8020309@gmail.com> Message-ID: <5c493e530910050231m6a46f72fm1aecc3ba382e04b3@mail.gmail.com> This is my fix to make EDoc work with Japanese (R13B02-1, .erl and overview.edoc files are saved in UTF-8). I think it will work for other languages: 1. At edoc_lib:write_file/4 Change file:open(File) to file:open(File, [write, {encoding, utf8}]) This is better than my previous dirty hack. 2. At edoc_tags:parse_tags/5 Change case dict:fetch(Name, How) of text -> parse_tags(Ts, How, Env, Where, [T | Ts1]); to case dict:fetch(Name, How) of text -> Data = unicode:characters_to_list(list_to_binary(T#tag.data)), T2 = T#tag{data = Data}, parse_tags(Ts, How, Env, Where, [T2 | Ts1]); Regards, Ngoc On Wed, Sep 30, 2009 at 6:37 PM, Richard Carlsson wrote: > Ngoc Dao wrote: >> When I use EDoc library in Erlang R13B02-1 to create document with >> Japanese characters in the doc comments, there is error: > > Yes, this is a known problem. The short answer is that the input > encoding for Erlang source code is defined to be Latin-1. That is, > if you put things like Japanese or Russian characters in the > source files, you are breaking the rules to begin with. (If it's only > in comments, and using UTF-8, it will not prevent the compiler from > skipping the comments and compiling the program, but you can't > expect anything else to work.) > > What would be needed is something like a \u-escaping preprocessing > stage, as specified for Java. But then, the tools must also know > about \u escape sequences and turn them back into the proper code > point in UTF-8 or whatever. > > ? ?/Richard > From bgustavsson@REDACTED Mon Oct 5 15:54:28 2009 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 5 Oct 2009 15:54:28 +0200 Subject: [erlang-questions] Prebuilt PLTs for Dialyzer In-Reply-To: <4AC8F6E4.5010600@tmit.bme.hu> References: <4AC8F6E4.5010600@tmit.bme.hu> Message-ID: <6672d0160910050654t48ea812fq6f5333dfa72c2369@mail.gmail.com> On Sun, Oct 4, 2009 at 9:26 PM, Zoltan Lajos Kis wrote: > Hello. > > Would it be possible to distribute a prebuilt .dialyzer_plt file for the > erlang/OTP releases (either packed or separately) that we could just > download and use? > It seems to take forever building these PLTs, even for the core > "applications"... We will consider making PLTs available as a separate download. We don't want to have the PLTs as part of the distribution as it was in earlier releases, because that caused a lot of problem. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From caio.ariede@REDACTED Mon Oct 5 16:06:10 2009 From: caio.ariede@REDACTED (caio ariede) Date: Mon, 5 Oct 2009 11:06:10 -0300 Subject: Qt4 Binding Message-ID: <6a9ba5690910050706u60d53f3j351accb43f218803@mail.gmail.com> There is a QT binding for Erlang, or someone is working on this? Caio Ariede http://caioariede.com/ From S.J.Thompson@REDACTED Mon Oct 5 16:10:49 2009 From: S.J.Thompson@REDACTED (S.J.Thompson) Date: Mon, 5 Oct 2009 15:10:49 +0100 (BST) Subject: Erlang Workshop videos now available Message-ID: I am happy to announce that videos of all talks at this summer's Erlang Workshop have made available online: http://www.vimeo.com/album/129567 http://www.vimeo.com/user2191865/albums I'm sure you'll join me in thanking Malcolm Wallace for the time and effort he put into making this possible. Thank you Malcolm! Simon Thompson From tomas.abrahamsson@REDACTED Mon Oct 5 21:38:52 2009 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Mon, 5 Oct 2009 21:38:52 +0200 Subject: [erlang-questions] UTF8 and EDoc In-Reply-To: <5c493e530910050231m6a46f72fm1aecc3ba382e04b3@mail.gmail.com> References: <5c493e530909300102x2d929ad5l9bb468d6eb390c55@mail.gmail.com> <4AC326DA.8020309@gmail.com> <5c493e530910050231m6a46f72fm1aecc3ba382e04b3@mail.gmail.com> Message-ID: > Ngoc Dao wrote: >> When I use EDoc library in Erlang R13B02-1 to create document with >> Japanese characters in the doc comments, there is error: Richard Carlsson wrote: > Yes, this is a known problem. The short answer is that the input > encoding for Erlang source code is defined to be Latin-1. [...] > What would be needed is something like a \u-escaping preprocessing > stage, as specified for Java. But then, the tools must also know > about \u escape sequences and turn them back into the proper code > point in UTF-8 or whatever. An option could be to adopt the way it is done in Python: it (re)uses the editor's encoding declaration. If it finds the text -*- coding: utf-8 -*- or vim: set fileencoding=utf-8 : on the first or second line of the source file, then it sets the encoding for the entire source file accordingly. (It also understands unicode byte-order marks at the beginning of the file, which apparently makes life easier in editors on Windows.) See http://www.python.org/peps/pep-0263.html for details. An advantage with this scheme seems to be that it fits nicely with editors. They already know how to handle this. It would probably require the Erlang compiler, edoc, and other tools to be modified to know about source file encodings, though. I suppose that with the \u-escaping, existing tools would continue to work without modification, but it would be more work for the programmer to type the text in as \u-seqences, unless editors already know how to do such a transformation on the fly? If no such encoding declaration is found, Python assumes ASCII, but Erlang could maybe assume Latin-1. If Python finds non-ASCII characters in a file with no encoding declaration, then it spits out an error like this (wrapped for readability): prompt# python /tmp/x.py File "/tmp/x.py", line 3 SyntaxError: Non-ASCII character '\xe5' in file /tmp/x.py on line 3, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details prompt# cat /tmp/ #! /usr/bin/env python print '???' BRs Tomas From getsumit@REDACTED Mon Oct 5 21:40:45 2009 From: getsumit@REDACTED (Sumit Chatterjee) Date: Tue, 6 Oct 2009 01:10:45 +0530 Subject: [erlang-questions] UTF8 and EDoc In-Reply-To: References: <5c493e530909300102x2d929ad5l9bb468d6eb390c55@mail.gmail.com> <4AC326DA.8020309@gmail.com> <5c493e530910050231m6a46f72fm1aecc3ba382e04b3@mail.gmail.com> Message-ID: <9e3b41110910051240m1534f8fgc8449c20bcdcddd7@mail.gmail.com> how can i convert the erlang files to c source code? Thanks Sumit 2009/10/6 Tomas Abrahamsson > > Ngoc Dao wrote: > >> When I use EDoc library in Erlang R13B02-1 to create document with > >> Japanese characters in the doc comments, there is error: > > Richard Carlsson wrote: > > Yes, this is a known problem. The short answer is that the input > > encoding for Erlang source code is defined to be Latin-1. [...] > > What would be needed is something like a \u-escaping preprocessing > > stage, as specified for Java. But then, the tools must also know > > about \u escape sequences and turn them back into the proper code > > point in UTF-8 or whatever. > > An option could be to adopt the way it is done in Python: > it (re)uses the editor's encoding declaration. If it finds the text > -*- coding: utf-8 -*- or vim: set fileencoding=utf-8 : > on the first or second line of the source file, then it sets > the encoding for the entire source file accordingly. (It also > understands unicode byte-order marks at the beginning > of the file, which apparently makes life easier in editors > on Windows.) > > See http://www.python.org/peps/pep-0263.html for details. > > An advantage with this scheme seems to be that it fits nicely > with editors. They already know how to handle this. > > It would probably require the Erlang compiler, edoc, and other tools > to be modified to know about source file encodings, though. > > I suppose that with the \u-escaping, existing tools would continue > to work without modification, but it would be more work for the > programmer to type the text in as \u-seqences, unless editors > already know how to do such a transformation on the fly? > > If no such encoding declaration is found, Python assumes ASCII, > but Erlang could maybe assume Latin-1. If Python finds non-ASCII > characters in a file with no encoding declaration, then it spits > out an error like this (wrapped for readability): > > prompt# python /tmp/x.py > File "/tmp/x.py", line 3 > SyntaxError: Non-ASCII character '\xe5' in file /tmp/x.py on line 3, > but no encoding declared; see http://www.python.org/peps/pep-0263.html > for details > prompt# cat /tmp/ > #! /usr/bin/env python > > print '???' > > BRs > Tomas > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From thomas.elsgaard@REDACTED Mon Oct 5 22:02:35 2009 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Mon, 5 Oct 2009 22:02:35 +0200 Subject: How to automatically start erlang on embedded device Message-ID: Hi Guys I am trying to find the correct way to start erlang on an embedded device (openBSD). run_erl seems to be the way forward, but i have some problems understanding exactly how to use it. Does anybody have an good example of how to correctly start erlang automatically on a *NIX system? ///Thomas From rvirding@REDACTED Tue Oct 6 00:11:26 2009 From: rvirding@REDACTED (Robert Virding) Date: Tue, 6 Oct 2009 00:11:26 +0200 Subject: What are BIFs? Message-ID: <3dbc6d1c0910051511p133bd39dnb8cfc58434e9bf46@mail.gmail.com> There has always been much confusion about what a BIF really is, how they relate to the Erlang language. An example of this was a discussion earlier this year when someone wanted to have some more functions in lists coded in C and so become part of Erlang. Saying that they are functions coded in C just tell us how they are implemented not what they *are*. The source of this confusion is very old and exists in the earliest documentation, where a BIF was described as function "which could not be written in Erlang or written in C for efficiency". They were in some sense implicitly part of Erlang and considered to be in the module 'erlang'. Some could be used without specifying the module but there was no clear reason why some needed the module and others did not. A first proper attempt to define what a BIF is came when Jonas Barklund and I wrote the Erlang specification. There we proposed that a BIF was part of the Erlang language that did not have a special syntax but looked like a normal function call. Spawn, link and trapexit are just as much part of Erlang as ! and receive even though they use normal function syntax and semantics. We also decided that it was irrelevant in what language the function was written, what was important was the function itself. This is analogous to processes where the semantics are what is important. I think this definition is correct but calling them BIFs was wrong. Our BIF proposals were put on ice together with the Erlang specification. The problem still remains and has not become better - there is still much confusion as to what a BIF is, if being coded in C has anything to do with it, and if being coded in C means that the functions automatically becomes part of the language. BEAM and the compiler also handle BIFs in many different ways which does not always help. Currently BIF is used to describe at least three different things: - Functions which are part of Erlang, spawn and link are examples of this. - Functions which are not part of Erlang but which are part of a basic runtime system, the trace BIFs are an example of this. - Functions which have been coded in C for efficiency, for example lists:reverse. (Some might not agree with separating the first two) Moving the module 'erlang' from kernel to erts is a step in the right direction. As is putting the new unicode BIFs (?) in a separate module, because they are not part of the language just a library where some functions are coded in C. OK, so why worry about it? Does it really matter? I think it does (or else I wouldn't be writing this mail) for the following reasons: - We need a proper language specification and this would be a part of it. Some day I plan to resurrect the old Erlang spec and get it up to date and we have to start somewhere. - It would clarify what is part of the language and what is not. This would put various discussions about adding things to the implementation/language in their proper context. So coding a function in lists in C would just become an implementation detail where the issue would be whether it is worth the extra effort to do this, and not misplaced discussions about adding to the language. - I think it would allow you to do more with the FFI than is proposed today and still be consistent with the language. - And it would help people realise and understand what is what - unclarity and confusion are never good. Sorry this became a bit long, but I have managed to restrain myself. :-) Robert From ckerr@REDACTED Tue Oct 6 00:25:44 2009 From: ckerr@REDACTED (Cameron Kerr) Date: Tue, 06 Oct 2009 11:25:44 +1300 Subject: [erlang-questions] How to automatically start erlang on embedded device In-Reply-To: References: Message-ID: <4ACA7268.3090103@cs.otago.ac.nz> You could look at ejabberd as an example. The Embedded System User's Guide may be of use also, but I think you are definitely right: a worked example in the documentation is needed. A worked example of the care and maintenance of an Erlang environment is needed in the documentation; preferably one that doesn't require the user (administrator) to be aware that it is written in Erlang. Thomas Elsgaard wrote: > Hi Guys > > I am trying to find the correct way to start erlang on an embedded > device (openBSD). > > run_erl seems to be the way forward, but i have some problems > understanding exactly how to use it. > > Does anybody have an good example of how to correctly start erlang > automatically on a *NIX system? > > ///Thomas > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From ckerr@REDACTED Tue Oct 6 00:31:01 2009 From: ckerr@REDACTED (Cameron Kerr) Date: Tue, 06 Oct 2009 11:31:01 +1300 Subject: [erlang-questions] UTF8 and EDoc In-Reply-To: References: <5c493e530909300102x2d929ad5l9bb468d6eb390c55@mail.gmail.com> <4AC326DA.8020309@gmail.com> <5c493e530910050231m6a46f72fm1aecc3ba382e04b3@mail.gmail.com> Message-ID: <4ACA73A5.3070502@cs.otago.ac.nz> Tomas Abrahamsson wrote: > Richard Carlsson wrote: >> Yes, this is a known problem. The short answer is that the input >> encoding for Erlang source code is defined to be Latin-1. [...] >> What would be needed is something like a \u-escaping preprocessing >> stage, as specified for Java. But then, the tools must also know >> about \u escape sequences and turn them back into the proper code >> point in UTF-8 or whatever. >> > > An option could be to adopt the way it is done in Python: > it (re)uses the editor's encoding declaration. If it finds the text > -*- coding: utf-8 -*- or vim: set fileencoding=utf-8 : > There is already a way to indicate whether something is UTF-8 (or UTF-16BE or UTF-16LE for that matter), and that is a byte-order mark; although the BOM serves no useful byte-ordering semantic for UTF-8, it does also have the function of saying "hey, I'm UTF-8!", a message which numerous programs understand. From masklinn@REDACTED Tue Oct 6 01:17:22 2009 From: masklinn@REDACTED (Masklinn) Date: Tue, 6 Oct 2009 01:17:22 +0200 Subject: [erlang-questions] What are BIFs? In-Reply-To: <3dbc6d1c0910051511p133bd39dnb8cfc58434e9bf46@mail.gmail.com> References: <3dbc6d1c0910051511p133bd39dnb8cfc58434e9bf46@mail.gmail.com> Message-ID: On 6 Oct 2009, at 00:11 , Robert Virding wrote: > There has always been much confusion about what a BIF really is, how > they > relate to the Erlang language. An example of this was a discussion > earlier > this year when someone wanted to have some more functions in lists > coded in > C and so become part of Erlang. Saying that they are functions coded > in C > just tell us how they are implemented not what they *are*. You might want to check out how Python's documentation/spec handles the similar concept of "builtin functions" (which actually aren't all functions, some are type constructors but in python constructors and functions are used the same way so?). Python's builtin functions are core functions existing in the "default" namespace (and therefore always available without the need to prefix them with a namespace qualifier). From getsumit@REDACTED Tue Oct 6 05:30:05 2009 From: getsumit@REDACTED (Sumit Chatterjee) Date: Tue, 6 Oct 2009 09:00:05 +0530 Subject: BASIC DOUBT Message-ID: <9e3b41110910052030r78f1be88u548579adc1759e45@mail.gmail.com> how can i convert the erlang files to c source code? Thanks Sumit 2009/10/6 Cameron Kerr > Tomas Abrahamsson wrote: > >> Richard Carlsson wrote: >> >>> Yes, this is a known problem. The short answer is that the input >>> encoding for Erlang source code is defined to be Latin-1. [...] >>> What would be needed is something like a \u-escaping preprocessing >>> stage, as specified for Java. But then, the tools must also know >>> about \u escape sequences and turn them back into the proper code >>> point in UTF-8 or whatever. >>> >>> >> >> An option could be to adopt the way it is done in Python: >> it (re)uses the editor's encoding declaration. If it finds the text >> -*- coding: utf-8 -*- or vim: set fileencoding=utf-8 : >> >> > There is already a way to indicate whether something is UTF-8 (or UTF-16BE > or UTF-16LE for that matter), and that is a byte-order mark; although the > BOM serves no useful byte-ordering semantic for UTF-8, it does also have the > function of saying "hey, I'm UTF-8!", a message which numerous programs > understand. > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From getsumit@REDACTED Tue Oct 6 05:48:26 2009 From: getsumit@REDACTED (Sumit Chatterjee) Date: Tue, 6 Oct 2009 09:18:26 +0530 Subject: [erlang-questions] BASIC DOUBT In-Reply-To: <4ACABBC1.6040209@gmail.com> References: <9e3b41110910052030r78f1be88u548579adc1759e45@mail.gmail.com> <4ACABBC1.6040209@gmail.com> Message-ID: <9e3b41110910052048u18152e4fsd4699c277baf21f9@mail.gmail.com> Hi Michael, Thanks for your response. It's because we wont to make it work in an embedded environment based on C. Could you please help me with some other diretion to port the solution then? Thanks Sumit 2009/10/6 Michael Truog > Hi Sumit, > > I am not sure why you are asking this question. However, Erlang code > runs on a Virtual Machine. Code can run partially native when HIPE is > used. There does not seem to be a current project to convert Erlang > code to C code, since that would remove some of the benefits the > language currently has when ran on the Virtual Machine. > > - Michael > > Sumit Chatterjee wrote: > > how can i convert the erlang files to c source code? > > Thanks > > Sumit > > > > 2009/10/6 Cameron Kerr > > > > > >> Tomas Abrahamsson wrote: > >> > >> > >>> Richard Carlsson wrote: > >>> > >>> > >>>> Yes, this is a known problem. The short answer is that the input > >>>> encoding for Erlang source code is defined to be Latin-1. [...] > >>>> What would be needed is something like a \u-escaping preprocessing > >>>> stage, as specified for Java. But then, the tools must also know > >>>> about \u escape sequences and turn them back into the proper code > >>>> point in UTF-8 or whatever. > >>>> > >>>> > >>>> > >>> An option could be to adopt the way it is done in Python: > >>> it (re)uses the editor's encoding declaration. If it finds the text > >>> -*- coding: utf-8 -*- or vim: set fileencoding=utf-8 : > >>> > >>> > >>> > >> There is already a way to indicate whether something is UTF-8 (or > UTF-16BE > >> or UTF-16LE for that matter), and that is a byte-order mark; although > the > >> BOM serves no useful byte-ordering semantic for UTF-8, it does also have > the > >> function of saying "hey, I'm UTF-8!", a message which numerous programs > >> understand. > >> > >> > >> > >> ________________________________________________________________ > >> erlang-questions mailing list. See http://www.erlang.org/faq.html > >> erlang-questions (at) erlang.org > >> > >> > >> > > > > > > From mjtruog@REDACTED Tue Oct 6 05:38:41 2009 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 05 Oct 2009 20:38:41 -0700 Subject: [erlang-questions] BASIC DOUBT In-Reply-To: <9e3b41110910052030r78f1be88u548579adc1759e45@mail.gmail.com> References: <9e3b41110910052030r78f1be88u548579adc1759e45@mail.gmail.com> Message-ID: <4ACABBC1.6040209@gmail.com> Hi Sumit, I am not sure why you are asking this question. However, Erlang code runs on a Virtual Machine. Code can run partially native when HIPE is used. There does not seem to be a current project to convert Erlang code to C code, since that would remove some of the benefits the language currently has when ran on the Virtual Machine. - Michael Sumit Chatterjee wrote: > how can i convert the erlang files to c source code? > Thanks > Sumit > > 2009/10/6 Cameron Kerr > > >> Tomas Abrahamsson wrote: >> >> >>> Richard Carlsson wrote: >>> >>> >>>> Yes, this is a known problem. The short answer is that the input >>>> encoding for Erlang source code is defined to be Latin-1. [...] >>>> What would be needed is something like a \u-escaping preprocessing >>>> stage, as specified for Java. But then, the tools must also know >>>> about \u escape sequences and turn them back into the proper code >>>> point in UTF-8 or whatever. >>>> >>>> >>>> >>> An option could be to adopt the way it is done in Python: >>> it (re)uses the editor's encoding declaration. If it finds the text >>> -*- coding: utf-8 -*- or vim: set fileencoding=utf-8 : >>> >>> >>> >> There is already a way to indicate whether something is UTF-8 (or UTF-16BE >> or UTF-16LE for that matter), and that is a byte-order mark; although the >> BOM serves no useful byte-ordering semantic for UTF-8, it does also have the >> function of saying "hey, I'm UTF-8!", a message which numerous programs >> understand. >> >> >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> >> > > From jeffm@REDACTED Tue Oct 6 06:23:56 2009 From: jeffm@REDACTED (jm) Date: Tue, 06 Oct 2009 15:23:56 +1100 Subject: [erlang-questions] BASIC DOUBT In-Reply-To: <9e3b41110910052048u18152e4fsd4699c277baf21f9@mail.gmail.com> References: <9e3b41110910052030r78f1be88u548579adc1759e45@mail.gmail.com> <4ACABBC1.6040209@gmail.com> <9e3b41110910052048u18152e4fsd4699c277baf21f9@mail.gmail.com> Message-ID: <4ACAC65C.9020308@ghostgun.com> Can you give some indication of: * purpose (eg to control traffic lights) * constraints (eg platform, OS, speed, diskspace, ram, etc) Helps to have some back ground when framing an answer. Hope that makes sense I'm answering between doing other things. Jeff. Sumit Chatterjee wrote: > Hi Michael, > Thanks for your response. It's because we wont to make it work in an > embedded environment based on C. Could you please help me with some other > diretion to port the solution then? > > > From dave.pawson@REDACTED Tue Oct 6 06:30:32 2009 From: dave.pawson@REDACTED (Dave Pawson) Date: Tue, 6 Oct 2009 05:30:32 +0100 Subject: [erlang-questions] UTF8 and EDoc In-Reply-To: References: <5c493e530909300102x2d929ad5l9bb468d6eb390c55@mail.gmail.com> <4AC326DA.8020309@gmail.com> <5c493e530910050231m6a46f72fm1aecc3ba382e04b3@mail.gmail.com> Message-ID: <711a73df0910052130o29ffca30x7a1f391ac136ed72@mail.gmail.com> 2009/10/5 Tomas Abrahamsson : > An option could be to adopt the way it is done in Python: > it (re)uses the editor's encoding declaration. If it finds the text > ? -*- coding: utf-8 -*- ?or ?vim: set fileencoding=utf-8 : > on the first or second line of the source file, then it sets > the encoding for the entire source file accordingly. (It also > understands unicode byte-order marks at the beginning > of the file, which apparently makes life easier in editors > on Windows.) yuk! Not everyone editor has this information? If a text file needs to inform an app of its encoding, then either a) Enclose the encoding in the file (xml example encoding='utf-8') b) Be explicit when calling up the application. I also think a default encoding as a fallback is essential, utf-8 being the obvious one. The BOM (byte order mark) as the first character of a file has not been successful. > > See http://www.python.org/peps/pep-0263.html for details. > > An advantage with this scheme seems to be that it fits nicely > with editors. They already know how to handle this. Only if you use the 'right' editor surely? > > It would probably require the Erlang compiler, edoc, and other tools > to be modified to know about source file encodings, though. What of programmatically generated files? > > I suppose that with the \u-escaping, existing tools would continue > to work without modification, but it would be more work for the > programmer to type the text in as \u-seqences, unless editors > already know how to do such a transformation on the fly? Or mimic python even more? u"A utf-8 encoded string" and a unicode('another unicode string') a string operator and encoding function. > > If no such encoding declaration is found, Python assumes ASCII, > but Erlang could maybe assume Latin-1. Please move on to utf-8. Latin-1 is so restrictive.. regards -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk From mjtruog@REDACTED Tue Oct 6 06:32:20 2009 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 05 Oct 2009 21:32:20 -0700 Subject: [erlang-questions] BASIC DOUBT In-Reply-To: <9e3b41110910052048u18152e4fsd4699c277baf21f9@mail.gmail.com> References: <9e3b41110910052030r78f1be88u548579adc1759e45@mail.gmail.com> <4ACABBC1.6040209@gmail.com> <9e3b41110910052048u18152e4fsd4699c277baf21f9@mail.gmail.com> Message-ID: <4ACAC854.704@gmail.com> Hi Sumit, You probably want either an Erlang port driver, an Erlang port, or an Erlang cnode. These are the 3 main methods to communicate with C code. You probably want to read about them here: http://erlang.org/doc/tutorial/part_frame.html An Erlang port driver runs within the Erlang VM process, and can cause many problems with the system because of this. An Erlang port runs within its own process, spawned from a fork within the Erlang VM. An Erlang cnode can be an independent process that connects as a separate hidden node, talking to other Erlang nodes on a socket and speaking a protocol that uses Erlang terms or something else. - Michael Sumit Chatterjee wrote: > Hi Michael, > Thanks for your response. It's because we wont to make it work in > an embedded environment based on C. Could you please help me with some > other diretion to port the solution then? > > Thanks > Sumit > > 2009/10/6 Michael Truog > > > Hi Sumit, > > I am not sure why you are asking this question. However, Erlang code > runs on a Virtual Machine. Code can run partially native when HIPE is > used. There does not seem to be a current project to convert Erlang > code to C code, since that would remove some of the benefits the > language currently has when ran on the Virtual Machine. > > - Michael > > Sumit Chatterjee wrote: > > how can i convert the erlang files to c source code? > > Thanks > > Sumit > > > > 2009/10/6 Cameron Kerr > > > > > > >> Tomas Abrahamsson wrote: > >> > >> > >>> Richard Carlsson wrote: > >>> > >>> > >>>> Yes, this is a known problem. The short answer is that the input > >>>> encoding for Erlang source code is defined to be Latin-1. [...] > >>>> What would be needed is something like a \u-escaping > preprocessing > >>>> stage, as specified for Java. But then, the tools must also know > >>>> about \u escape sequences and turn them back into the proper code > >>>> point in UTF-8 or whatever. > >>>> > >>>> > >>>> > >>> An option could be to adopt the way it is done in Python: > >>> it (re)uses the editor's encoding declaration. If it finds the > text > >>> -*- coding: utf-8 -*- or vim: set fileencoding=utf-8 : > >>> > >>> > >>> > >> There is already a way to indicate whether something is UTF-8 > (or UTF-16BE > >> or UTF-16LE for that matter), and that is a byte-order mark; > although the > >> BOM serves no useful byte-ordering semantic for UTF-8, it does > also have the > >> function of saying "hey, I'm UTF-8!", a message which numerous > programs > >> understand. > >> > >> > >> > >> ________________________________________________________________ > >> erlang-questions mailing list. See http://www.erlang.org/faq.html > >> erlang-questions (at) erlang.org > >> > >> > >> > > > > > > From max.lapshin@REDACTED Tue Oct 6 06:45:27 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 6 Oct 2009 08:45:27 +0400 Subject: C Node and {'EXIT', Pid, noconnection} in Erlang Message-ID: Hi, everyone. I want to make an experiment: our library, written in ObjC can stream some kind of data and I want to make it stream data to Erlang front server. As library is build on top of libev, I had to reimplement whole erl_interface to be it asynchronous, also I've implemented direct mapping from many of erlang types to native ObjectiveC types (such as NSNumber, NSString, NSData, etc). Everything works ok, epmd register my C node (as hidden one), erlang connects, net_adm:ping works, because net_kernel process is implemented, rpc:call is handled, etc. >From erlang side it looks so: {ok, NetStream} = rpc:call(NetStreamNode, netstream, start, [Name], ?TIMEOUT), link(NetStream), ?D({"Netstream created", NetStream}), {ok, NetStream} In ObjC side all messages are received, object is allocated and subscribes to messages under proper Pid. However, before first message from ObjC is arrived, frontend receive message: {'EXIT', NetStream, noconnection} I've looked through all source of erlang, and I cannot clearly understand, what can provide such a message? In node flags, while establishing protocol, I do not mention monitor abilities? Maybe I miss some message, that should be replied as error? If it will help, I can dump all messages, received from erlang to C and back. From icfp.publicity@REDACTED Tue Oct 6 09:24:12 2009 From: icfp.publicity@REDACTED (Wouter Swierstra) Date: Tue, 6 Oct 2009 09:24:12 +0200 Subject: ICFP videos now available Message-ID: <53ff55480910060024p27619c52i937b5c039282dd72@mail.gmail.com> I am happy to announce that videos of all talks at ICFP and some of the associated workshops this year have made available online: http://www.vimeo.com/user2191865/albums I'm sure you'll join me in thanking Malcolm Wallace for the time and effort he put into making this possible. Thank you Malcolm! Wouter From torben.lehoff@REDACTED Tue Oct 6 10:47:27 2009 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Tue, 6 Oct 2009 10:47:27 +0200 Subject: [erlang-questions] Qt4 Binding In-Reply-To: <6a9ba5690910050706u60d53f3j351accb43f218803@mail.gmail.com> References: <6a9ba5690910050706u60d53f3j351accb43f218803@mail.gmail.com> Message-ID: There has been some work on a GTK binding, but I only know this from a google search. Erlang ships with wxErlang that provides a binding to the wx library - why is that not enough for you? (Just trying to understand the need behind the question). Cheers, Torben On Mon, Oct 5, 2009 at 16:06, caio ariede wrote: > There is a QT binding for Erlang, or someone is working on this? > > Caio Ariede > http://caioariede.com/ > -- http://www.linkedin.com/in/torbenhoffmann From max.lapshin@REDACTED Tue Oct 6 11:03:56 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 6 Oct 2009 13:03:56 +0400 Subject: C Node and {'EXIT', Pid, noconnection} in Erlang In-Reply-To: References: Message-ID: Here it my debug log of connection: Oct 6 12:59:24 Connection established with node ems@REDACTED from 127.0.0.1:51335 Oct 6 12:59:24 Header is: {6, #Pid, '', 'rex'} Oct 6 12:59:24 Received message: {'$gen_call', {#Pid, #Ref}, {'call', 'netstream', 'start', ( "ort-tm/1254744976" ), #Pid}} So, OjbC receive only one message from erlang, erlang log is: DEBUG ems_play:78 {"Netstream created",<5892.42.4>} DEBUG ems_fsm:402 {"Player died",<5892.42.4>,noconnection} However, ObjC is still sending messages to established connection. From max.lapshin@REDACTED Tue Oct 6 11:43:48 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 6 Oct 2009 13:43:48 +0400 Subject: C Node and {'EXIT', Pid, noconnection} in Erlang In-Reply-To: References: Message-ID: Problem disappeared when I remove link(NetStream) from erlang code. It is strange for me, because no LINK messages haven't been sent to ObjC node From cyril.romain@REDACTED Tue Oct 6 14:12:06 2009 From: cyril.romain@REDACTED (cyril romain) Date: Tue, 6 Oct 2009 14:12:06 +0200 Subject: [erlang-questions] Qt4 Binding In-Reply-To: <6a9ba5690910050706u60d53f3j351accb43f218803@mail.gmail.com> References: <6a9ba5690910050706u60d53f3j351accb43f218803@mail.gmail.com> Message-ID: On Mon, Oct 5, 2009 at 4:06 PM, caio ariede wrote: > There is a QT binding for Erlang, or someone is working on this? > There is no Qt binding that I know. But with erlua [1], you can use lqt [2] and therefore get a Qt binding for Erlang via Lua. Without any patch, I could this morning successfully compile lqt with Qt-4.5.2 and run the qt examples provided with lqt (i.e. various widgets such as push buttons, sliders, a webview, etc.) from Erlang! Hope it can help you and enlighten anyone willing to combine the power of Erlang with the power of Qt. Best regards, Cyril [1] an Erlang - Lua binding that I wrote some months ago: http://gitorious.org/erlua [2] an automatic Qt binding for Lua: http://code.google.com/p/lqt From caio.ariede@REDACTED Tue Oct 6 15:05:00 2009 From: caio.ariede@REDACTED (caio ariede) Date: Tue, 6 Oct 2009 10:05:00 -0300 Subject: [erlang-questions] Qt4 Binding In-Reply-To: References: <6a9ba5690910050706u60d53f3j351accb43f218803@mail.gmail.com> Message-ID: <6a9ba5690910060605j6968ac4au3a271ce1977b3432@mail.gmail.com> I'm just searching for Erlang toolkit bindings. There is no much activity (abandoned?) in erlgtk project... erlua looks very interesting. thank you cyril. ;) I think that the best option is even the wxErlang. Caio Ariede http://caioariede.com/ On Tue, Oct 6, 2009 at 9:12 AM, cyril romain wrote: > On Mon, Oct 5, 2009 at 4:06 PM, caio ariede wrote: > >> There is a QT binding for Erlang, or someone is working on this? >> > > > There is no Qt binding that I know. > > But with erlua [1], you can use lqt [2] and therefore get a Qt binding for > Erlang via Lua. > Without any patch, I could this morning successfully compile lqt with > Qt-4.5.2 and run the qt examples provided with lqt (i.e. various widgets > such as push buttons, sliders, a webview, etc.) from Erlang! > > Hope it can help you and enlighten anyone willing to combine the power of > Erlang with the power of Qt. > > Best regards, > > Cyril > > [1] an Erlang - Lua binding that I wrote some months ago: > http://gitorious.org/erlua > [2] an automatic Qt binding for Lua: http://code.google.com/p/lqt > > From rvirding@REDACTED Wed Oct 7 18:55:10 2009 From: rvirding@REDACTED (Robert Virding) Date: Wed, 7 Oct 2009 18:55:10 +0200 Subject: [erlang-questions] What are BIFs? In-Reply-To: References: <3dbc6d1c0910051511p133bd39dnb8cfc58434e9bf46@mail.gmail.com> Message-ID: <3dbc6d1c0910070955s4c1d9a7eu6406ca13fb6f1bff@mail.gmail.com> 2009/10/6 Masklinn > On 6 Oct 2009, at 00:11 , Robert Virding wrote: > >> There has always been much confusion about what a BIF really is, how they >> relate to the Erlang language. An example of this was a discussion earlier >> this year when someone wanted to have some more functions in lists coded >> in >> C and so become part of Erlang. Saying that they are functions coded in C >> just tell us how they are implemented not what they *are*. >> > > You might want to check out how Python's documentation/spec handles the > similar concept of "builtin functions" (which actually aren't all functions, > some are type constructors but in python constructors and functions are used > the same way so?). Python's builtin functions are core functions existing in > the "default" namespace (and therefore always available without the need to > prefix them with a namespace qualifier). Yes, I think the way it is done today in Erlang is ok: having them both explicitly in the module erlang and being able to use them without a module name. I think the problem is that the module erlang is a mix of "core" erlang language functions and run-time functions which aren't really part of the language. Then you have the added confusion of for some you don't need to qualify them with a module name and some you do. The solution is *NOT* to just remove the need for module name for all them!!! What I would like to do is make the module erlang only consist core erlang language functions which are part of the erlang language. These functions you can use without the explicit module name. All other functions, what I call run-time functions, should be moved out of erlang into another module/modules. This would have the benefit of clearly defining which functions are part of the language and have a module structure which supports that. N.B. This separates which are the core functions from how they are implemented, there is no requirement that they are implemented in C. Jonas Barklund and I were also working with the idea that their could be a number of core language modules, not just erlang, containing the core functions. It would certainly make the modules smaller and more managable but I don't know if it is necessary. Robert From amit.murthy@REDACTED Thu Oct 8 06:14:01 2009 From: amit.murthy@REDACTED (Amit Murthy) Date: Thu, 8 Oct 2009 09:44:01 +0530 Subject: wxErlang - wxLogNull question Message-ID: <4f5fdb630910072114m2137f163v466053684bcf2ef3@mail.gmail.com> Hi, I am using the wxImage module of wxErlang to resize some images in a back ground process. Now, if there is an invalid file name or the image binary is corrupt, wxImage throws a window popup message which I don't want. The popup is actually coming from wxWidgets as outlined here http://www.wxwidgets.org/docs/faqcmn.htm#lognull and here - http://docs.wxwidgets.org/stable/wx_wxlognull.html The solution is to call wxLogNull if we do not want the error popups. But wxErlang does not expose wxLogNull. Any particular reason why? Any other way to avoid the popup. A related question - Since I am only using wxImage to resize images and not really having any GUI displayed at all, is it possible to use wxErlang without an X server running ? Amit From dangud@REDACTED Thu Oct 8 09:25:31 2009 From: dangud@REDACTED (Dan Gudmundsson) Date: Thu, 8 Oct 2009 09:25:31 +0200 Subject: [erlang-questions] wxErlang - wxLogNull question In-Reply-To: <4f5fdb630910072114m2137f163v466053684bcf2ef3@mail.gmail.com> References: <4f5fdb630910072114m2137f163v466053684bcf2ef3@mail.gmail.com> Message-ID: <93df43b60910080025l51a217eanf7090cd4d8c1596@mail.gmail.com> On Thu, Oct 8, 2009 at 6:14 AM, Amit Murthy wrote: > Hi, > > I am using the wxImage module of wxErlang to resize some images in a back > ground process. Now, if there is an invalid file name or the image binary is > corrupt, wxImage throws a window popup message which I don't want. The popup > is actually coming from wxWidgets as outlined here > > http://www.wxwidgets.org/docs/faqcmn.htm#lognull > > and here - > http://docs.wxwidgets.org/stable/wx_wxlognull.html > > The solution is to call wxLogNull if we do not want the error popups. > > But wxErlang does not expose wxLogNull. Any particular reason why? Any other > way to avoid the popup. No reason I didn't know about it, send a patch :-) > > A related question - Since I am only using wxImage to resize images and not > really having any GUI displayed at all, is it possible to use wxErlang > without an X server running ? I don't know /Dan > > ?Amit > From clist@REDACTED Thu Oct 8 12:23:24 2009 From: clist@REDACTED (Angel) Date: Thu, 8 Oct 2009 12:23:24 +0200 Subject: [erlang-questions] wxErlang - wxLogNull question In-Reply-To: <4f5fdb630910072114m2137f163v466053684bcf2ef3@mail.gmail.com> References: <4f5fdb630910072114m2137f163v466053684bcf2ef3@mail.gmail.com> Message-ID: <200910081223.24970.clist@uah.es> El Jueves, 8 de Octubre de 2009 06:14:01 Amit Murthy escribi?: > Hi, > > I am using the wxImage module of wxErlang to resize some images in a back > ground process. Now, if there is an invalid file name or the image binary is > corrupt, wxImage throws a window popup message which I don't want. The popup > is actually coming from wxWidgets as outlined here > > http://www.wxwidgets.org/docs/faqcmn.htm#lognull > > and here - > http://docs.wxwidgets.org/stable/wx_wxlognull.html > > The solution is to call wxLogNull if we do not want the error popups. > > But wxErlang does not expose wxLogNull. Any particular reason why? Any other > way to avoid the popup. > > > A related question - Since I am only using wxImage to resize images and not > really having any GUI displayed at all, is it possible to use wxErlang > without an X server running ? > > Amit > Well , this seems to be the reason for having a separate wxexlang os process instead of a driver. If people begins to exercise more and more library funtions for porpouses other that display a GUI we can lose all the resilence we have in the VM. Just take this "or the image binary is corrup" and now how do you (1) "let other process fix the error" when we have a lot of code and subtle bugs buried inside a big erlang driver? Erlang driver should be streamlined versions of code that needs to be fast and low level. Whats the problem on being a port process for wxerlang? ?we need speed? Why? Clearly this library seem to be able to do many thing on its own and being able to load binaries, images and files without erlang code intervention seems a good point against the need for being a driver. Just throw another process , just throw another core to the problem. There is a lot of hype and fear simultaneously for erlang being the next Big Thing about concurrency and pushing big things into drivers does not help the concept. Will erlang became "big and bloated" like the linux kernel? As a newbie I can't claim seriously any evidence in one or the other direction, but for me the beauty of GS was mainly based on the port concept showing the world that great things can be made today from the pipes concept. I feel this is really the erlang spirit /Angel (1): http://www.erlang.org/pipermail/erlang-questions/2003-April/008647.html PS: Sorry for my english :-P -- Este correo no tiene dibujos. Las formas extra?as en la pantalla son letras. __________________________________________ Clist UAH a.k.a Angel __________________________________________ Evitar la programaci?n defensiva. Manual de Erlang From allan.merolla@REDACTED Thu Oct 8 13:03:43 2009 From: allan.merolla@REDACTED (Allan Merolla) Date: Thu, 8 Oct 2009 21:03:43 +1000 Subject: Mnesia 25MB database takes 15min to load!!! Message-ID: Hello All, I am currently using mnesia to store simple records consisting of {ID,Time,Value}. The purpose of the database is simple. High availability of large amounts of "simple data". I have inserted 500 thousand records and the DCD is 24mb. When I do a mnesia:stop() and start() it takes over 10min just to load the test data. I originally had planned to have this fragmented and have at least 200mb per fragment but at 10min load for 25mb the whole idea seems impossible now as it would take over 2hours to start a node. I find it hard to believe this is by design? Can anyone tell me what I am doing wrong? Thanks in advance folks. Cheers, Allan From roberto.aloi@REDACTED Thu Oct 8 13:12:34 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Thu, 08 Oct 2009 12:12:34 +0100 Subject: [erlang-questions] Mnesia 25MB database takes 15min to load!!! In-Reply-To: References: Message-ID: <4ACDC922.2000800@erlang-consulting.com> Hi Allan, you might want to have a look to this thread on Trapexit: http://www.trapexit.org/forum/viewtopic.php?p=44433 Regards, Roberto Aloi Erlang Training and Consulting Ltd. http://www.erlang-consulting.com http://aloiroberto.wordpress.com Allan Merolla wrote: > Hello All, > > I am currently using mnesia to store simple records consisting of {ID,Time,Value}. The purpose of the database is simple. High availability of large amounts of "simple data". I have inserted 500 thousand records and the DCD is 24mb. When I do a mnesia:stop() and start() it takes over 10min just to load the test data. I originally had planned to have this fragmented and have at least 200mb per fragment but at 10min load for 25mb the whole idea seems impossible now as it would take over 2hours to start a node. I find it hard to believe this is by design? Can anyone tell me what I am doing wrong? > > Thanks in advance folks. > > Cheers, > Allan > > > > From pfisher@REDACTED Thu Oct 8 14:32:49 2009 From: pfisher@REDACTED (Paul Fisher) Date: Thu, 08 Oct 2009 07:32:49 -0500 Subject: [erlang-questions] Mnesia 25MB database takes 15min to load!!! In-Reply-To: References: Message-ID: <4ACDDBF1.4080405@alertlogic.net> Allan Merolla wrote: > Hello All, > > I am currently using mnesia to store simple records consisting of {ID,Time,Value}. The purpose of the database is simple. High availability of large amounts of "simple data". I have inserted 500 thousand records and the DCD is 24mb. When I do a mnesia:stop() and start() it takes over 10min just to load the test data. I originally had planned to have this fragmented and have at least 200mb per fragment but at 10min load for 25mb the whole idea seems impossible now as it would take over 2hours to start a node. I find it hard to believe this is by design? Can anyone tell me what I am doing wrong? Do you have a secondary index defined on the table? I have seem similar behavior when a secondary index is defined, and have changed our designs to not use them at all. Without, disk_copies tables load at disk i/o speed. -- paul From dizzyd@REDACTED Thu Oct 8 19:23:43 2009 From: dizzyd@REDACTED (Dave Smith) Date: Thu, 8 Oct 2009 11:23:43 -0600 Subject: beam crasher: ets + write_concurrency Message-ID: Greetings, I believe that I have identified a VM crasher when using ETS with write concurrency enabled. Sample code is as follows: T = ets:new(?MODULE, [{write_concurrency, true}]), ets:insert_new(T, {foo, 1}), ets:match(T, {nosuchvalue, '$1'}). GDB stack trace shows: Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000008 [Switching to process 69425] 0x00000001000c894b in db_select_chunk_hash (p=0x101466398, tbl=0x1013b6eb0, pattern=4330115329, chunk_size=0, reverse=0, ret=0x102185080) at beam/erl_db_hash.c:1426 1426 if (current->hvalue != INVALID_HASH && (gdb) bt #0 0x00000001000c894b in db_select_chunk_hash (p=0x101466398, tbl=0x1013b6eb0, pattern=4330115329, chunk_size=0, reverse=0, ret=0x102185080) at beam/erl_db_hash.c:1426 #1 0x00000001000c85f2 in db_select_hash (p=0x101466398, tbl=0x1013b6eb0, pattern=4330115329, reverse=0, ret=0x102185080) at beam/erl_db_hash.c:1344 #2 0x00000001000b1c6f in ets_select_2 (A__p=0x101466398, A_1=262367, A_2=4330115329) at beam/erl_db.c:2136 #3 0x00000001000b1524 in ets_match_2 (A__p=0x101466398, A_1=262367, A_2=4335025986) at beam/erl_db.c:1940 #4 0x000000010011e5ed in process_main () at beam/beam_emu.c:2084 #5 0x000000010007ed86 in sched_thread_func (vesdp=0x1007812e8) at beam/erl_process.c:3015 #6 0x00000001001a3360 in thr_wrapper (vtwd=0x7fff5fbff0e0) at common/ethread.c:475 #7 0x00007fff819a2e19 in thread_start () (gdb) This crash was repeatable on the following platforms: OS X 10.6 (Snow Leopard), 64-bit and 32-bit VM OS X 10.5 (Leopard), 32-bit VM Debian Linux 2.6 x86_64, 64-bit VM Erlang versions tested: R13B01, R13B02-1 Thanks, D. From clist@REDACTED Thu Oct 8 21:15:51 2009 From: clist@REDACTED (Angel Alvarez) Date: Thu, 8 Oct 2009 21:15:51 +0200 Subject: [erlang-questions] beam crasher: ets + write_concurrency In-Reply-To: References: Message-ID: <200910082115.51923.clist@uah.es> This crash is repeatable, reproducced on OpenSUSE 10.3 this code crash the emulator on SMP mode.. -module(crash). -export([crash/0]). crash() -> T = ets:new(?MODULE, [{write_concurrency, true}]), ets:insert_new(T, {foo, 1}), ets:match(T, {nosuchvalue, '$1'}). sinosuke@REDACTED:~/Documents/Personal/Erlang/Code> erl Erlang R13B01 (erts-5.7.2) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) 1> crash:crash(). [] 2> 3> sinosuke@REDACTED:~/Documents/Personal/Erlang/Code> erl -smp on Erlang R13B01 (erts-5.7.2) [source] [smp:1:1] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) 1> crash:crash(). Violaci?n de segmento sinosuke@REDACTED:~/Documents/Personal/Erlang/Code> /Angel El Jueves, 8 de Octubre de 2009 Dave Smith escribi?: > Greetings, > > I believe that I have identified a VM crasher when using ETS with > write concurrency enabled. Sample code is as follows: > > T = ets:new(?MODULE, [{write_concurrency, true}]), > ets:insert_new(T, {foo, 1}), > ets:match(T, {nosuchvalue, '$1'}). > > GDB stack trace shows: > > Program received signal EXC_BAD_ACCESS, Could not access memory. > Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000008 > [Switching to process 69425] > 0x00000001000c894b in db_select_chunk_hash (p=0x101466398, > tbl=0x1013b6eb0, pattern=4330115329, chunk_size=0, reverse=0, > ret=0x102185080) at beam/erl_db_hash.c:1426 > 1426 if (current->hvalue != INVALID_HASH && > (gdb) bt > #0 0x00000001000c894b in db_select_chunk_hash (p=0x101466398, > tbl=0x1013b6eb0, pattern=4330115329, chunk_size=0, reverse=0, > ret=0x102185080) at beam/erl_db_hash.c:1426 > #1 0x00000001000c85f2 in db_select_hash (p=0x101466398, > tbl=0x1013b6eb0, pattern=4330115329, reverse=0, ret=0x102185080) at > beam/erl_db_hash.c:1344 > #2 0x00000001000b1c6f in ets_select_2 (A__p=0x101466398, A_1=262367, > A_2=4330115329) at beam/erl_db.c:2136 > #3 0x00000001000b1524 in ets_match_2 (A__p=0x101466398, A_1=262367, > A_2=4335025986) at beam/erl_db.c:1940 > #4 0x000000010011e5ed in process_main () at beam/beam_emu.c:2084 > #5 0x000000010007ed86 in sched_thread_func (vesdp=0x1007812e8) at > beam/erl_process.c:3015 > #6 0x00000001001a3360 in thr_wrapper (vtwd=0x7fff5fbff0e0) at > common/ethread.c:475 > #7 0x00007fff819a2e19 in thread_start () > (gdb) > > > This crash was repeatable on the following platforms: > > OS X 10.6 (Snow Leopard), 64-bit and 32-bit VM > OS X 10.5 (Leopard), 32-bit VM > Debian Linux 2.6 x86_64, 64-bit VM > > Erlang versions tested: R13B01, R13B02-1 > > Thanks, > > D. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. ->>----------------------------------------------- Clist UAH a.k.a Angel ---------------------------------[www.uah.es]-<<-- ...being the second biggest search engine in the world is good enough for us. Peter @ Pirate Bay. From john.erickson@REDACTED Thu Oct 8 19:41:17 2009 From: john.erickson@REDACTED (Erickson, John) Date: Thu, 8 Oct 2009 11:41:17 -0600 Subject: Passing Filehandles Between Processes Message-ID: <3C44D46958109E408F5848593B05A1D8C60DAEBF@rrsmsx505.amr.corp.intel.com> Hi, I have noticed that Erlang acts strangely when I pass a file handle from one process to another (for local threads). It seems that the port sends the data back to the process that opened the file rather than the current one using the handle. Is this a bug? Or is there a reason this cannot be done? The only workaround I have found is to close and reopen the file whenever I pass it to a new process. From max.lapshin@REDACTED Thu Oct 8 22:26:41 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Oct 2009 00:26:41 +0400 Subject: [erlang-questions] Passing Filehandles Between Processes In-Reply-To: <3C44D46958109E408F5848593B05A1D8C60DAEBF@rrsmsx505.amr.corp.intel.com> References: <3C44D46958109E408F5848593B05A1D8C60DAEBF@rrsmsx505.amr.corp.intel.com> Message-ID: On Thu, Oct 8, 2009 at 9:41 PM, Erickson, John wrote: > Hi, I have noticed that Erlang acts strangely when I pass a file handle from one process to another (for local threads). ?It seems that the port sends the data back to the process that opened the file rather than the current one using the handle. ?Is this a bug? ?Or is there a reason this cannot be done? ?The only workaround I have found is to close and reopen the file whenever I pass it to a new process. > Are you speaking about sockets? If yes, than look at gen_tcp:controlling_process From matthew@REDACTED Thu Oct 8 22:25:34 2009 From: matthew@REDACTED (Matthew Sackman) Date: Thu, 8 Oct 2009 21:25:34 +0100 Subject: [erlang-questions] Passing Filehandles Between Processes In-Reply-To: <3C44D46958109E408F5848593B05A1D8C60DAEBF@rrsmsx505.amr.corp.intel.com> References: <3C44D46958109E408F5848593B05A1D8C60DAEBF@rrsmsx505.amr.corp.intel.com> Message-ID: <20091008202534.GA8540@wellquite.org> On Thu, Oct 08, 2009 at 11:41:17AM -0600, Erickson, John wrote: > Hi, I have noticed that Erlang acts strangely when I pass a file handle from one process to another (for local threads). It seems that the port sends the data back to the process that opened the file rather than the current one using the handle. Is this a bug? Or is there a reason this cannot be done? The only workaround I have found is to close and reopen the file whenever I pass it to a new process. I've seen that too actually, I put it down to opening the fh raw, which is documented as meaning only the opening process can access the fh, but I didn't investigate further to see if it worked opening not raw. Matthew From max.lapshin@REDACTED Thu Oct 8 22:58:18 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Oct 2009 00:58:18 +0400 Subject: crash collectors (with emailing) Message-ID: There are many services in world of Ruby on Rails, that collect exceptions, raised on production servers and send them directly via email, or via buffering service (if there are thousands of similar errors, you will receive only one). It haven't yet seen anything the same for Erlang. I want to receive via email crash dumps, logged to error_logger. Am I right, that I will need to make an error_logger handler for this? From fritchie@REDACTED Thu Oct 8 23:26:21 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 08 Oct 2009 16:26:21 -0500 Subject: [erlang-questions] Mnesia 25MB database takes 15min to load!!! In-Reply-To: Message of "Thu, 08 Oct 2009 07:32:49 CDT." <4ACDDBF1.4080405@alertlogic.net> Message-ID: <67277.1255037181@snookles.snookles.com> Paul Fisher wrote: pf> Do you have a secondary index defined on the table? I have seem pf> similar behavior when a secondary index is defined, and have changed pf> our designs to not use them at all. Without, disk_copies tables pf> load at disk i/o speed. Mnesia secondary indexes are great, until you have a secondary index key that occurs in lots of records. Secondary keys are implemented using ETS 'bag'/'duplicate_bag' (I can never remember, sorry). Inserting the same key repeatedly into the secondary index ETS table will get you O(N^2) behavior. See this long thread for more details: http://erlang.org/pipermail/erlang-questions/2008-June/035745.html -Scott From nesrait@REDACTED Thu Oct 8 23:31:48 2009 From: nesrait@REDACTED (=?ISO-8859-1?Q?Davide_Marqu=EAs?=) Date: Thu, 8 Oct 2009 22:31:48 +0100 Subject: [erlang-questions] Passing Filehandles Between Processes In-Reply-To: <20091008202534.GA8540@wellquite.org> References: <3C44D46958109E408F5848593B05A1D8C60DAEBF@rrsmsx505.amr.corp.intel.com> <20091008202534.GA8540@wellquite.org> Message-ID: <523869a70910081431o6d868f79l3d412b075995d06f@mail.gmail.com> Hi there, I did some tests a while ago trying out various read combinations. Testing showed that: - files opened in raw mode get binded to the opening process and that process is the only one that can read/write from it - read attempts by other processes will block (although you can always route the read requests through the process that opened the file); - files opened in non-raw mode can be access by other processes without any problems; - finally, *the process that opened the file must be alive for read/write operations to succeed*. This last one seems like a good argument against trying to read/write outside of the "file owner" process. I'm attaching the code I used for testing in case someone is interested. Cheers, :Davide -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test_file.erl Type: application/octet-stream Size: 2605 bytes Desc: not available URL: From igorrs@REDACTED Fri Oct 9 01:12:29 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Thu, 8 Oct 2009 20:12:29 -0300 Subject: Ordered set with lower_bound Message-ID: Hi. I need some ordered set that allows me to "search and then iterate" efficiently. In other words, I need this functionality: http://www.cplusplus.com/reference/stl/set/lower_bound/ After looking at the documentation for gb_sets and ordsets, I have concluded that Erlang does not provide that functionality and I'll have to roll my own. Is it the right conclusion? :-) Thanks. Igor. From yogishb@REDACTED Fri Oct 9 00:32:17 2009 From: yogishb@REDACTED (Yogish Baliga) Date: Thu, 8 Oct 2009 15:32:17 -0700 (PDT) Subject: Huge binary usage Message-ID: <98409.67270.qm@web31808.mail.mud.yahoo.com> I have a erlang process which is running at 376MB Resident memory. erlang:memory() shows the following output. [{total,265245056}, {processes,56800140}, {processes_used,56777884}, {system,208444916}, {atom,662173}, {atom_used,636586}, {binary,178220728}, {code,4720224}, {ets,21754908}] Seems like most of the memory is used by "binary". Along with this, I did a sum of total memory used by all the processes running on the system. It comes to around 53MB (which is same as what is reported in processes_used). Question is how can I find what process(es) allocated the binaries and why they are not released? Thanx, -- baliga "Point of view is worth 80 IQ points" --Alan Kay http://dudefrommangalore.blogspot.com/ From paul-trapexit@REDACTED Fri Oct 9 01:32:32 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Thu, 8 Oct 2009 16:32:32 -0700 (PDT) Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: References: Message-ID: On Thu, 8 Oct 2009, Igor Ribeiro Sucupira wrote: > Hi. > > I need some ordered set that allows me to "search and then iterate" > efficiently. In other words, I need this functionality: > http://www.cplusplus.com/reference/stl/set/lower_bound/ > > After looking at the documentation for gb_sets and ordsets, I have > concluded that Erlang does not provide that functionality and I'll > have to roll my own. Is it the right conclusion? :-) ets:next/2 on an ordered_set table acts like that. "efficiently" ... well, it'll be O (log N) per call, you might have hoped for O (1) after the first O (log N) call. also you might not like the constant factors. benchmark to be sure. -- p p.z. perhaps ets:slot/2 is faster? From rvirding@REDACTED Fri Oct 9 01:34:24 2009 From: rvirding@REDACTED (Robert Virding) Date: Fri, 9 Oct 2009 01:34:24 +0200 Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: References: Message-ID: <3dbc6d1c0910081634k5c417e8byed019ad9f71fbdf0@mail.gmail.com> You will have to roll your own, though you can use either as a base from which to start. gb_sets is more efficient than ordsets except for small sets. Robert 2009/10/9 Igor Ribeiro Sucupira > Hi. > > I need some ordered set that allows me to "search and then iterate" > efficiently. In other words, I need this functionality: > http://www.cplusplus.com/reference/stl/set/lower_bound/ > > After looking at the documentation for gb_sets and ordsets, I have > concluded that Erlang does not provide that functionality and I'll > have to roll my own. Is it the right conclusion? :-) > > Thanks. > Igor. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From rvirding@REDACTED Fri Oct 9 01:36:49 2009 From: rvirding@REDACTED (Robert Virding) Date: Fri, 9 Oct 2009 01:36:49 +0200 Subject: [erlang-questions] Huge binary usage In-Reply-To: <98409.67270.qm@web31808.mail.mud.yahoo.com> References: <98409.67270.qm@web31808.mail.mud.yahoo.com> Message-ID: <3dbc6d1c0910081636l164b2211kea9e086d6c99d597@mail.gmail.com> Large binaries are kept in the normal process heaps but outside of them. This means that they are not collected in the same way as normal data and can hang around for a longer time, though they will be collected. Robert 2009/10/9 Yogish Baliga > I have a erlang process which is running at 376MB Resident memory. > erlang:memory() shows the following output. > > [{total,265245056}, > {processes,56800140}, > {processes_used,56777884}, > {system,208444916}, > {atom,662173}, > {atom_used,636586}, > {binary,178220728}, > {code,4720224}, > {ets,21754908}] > > Seems like most of the memory is used by "binary". > > Along with this, I did a sum of total memory used by all the processes > running on the system. It comes to around 53MB (which is same as what is > reported in processes_used). > > Question is how can I find what process(es) allocated the binaries and why > they are not released? > > Thanx, > -- baliga > > "Point of view is worth 80 IQ points" --Alan Kay > > http://dudefrommangalore.blogspot.com/ From rvirding@REDACTED Fri Oct 9 01:38:52 2009 From: rvirding@REDACTED (Robert Virding) Date: Fri, 9 Oct 2009 01:38:52 +0200 Subject: [erlang-questions] Huge binary usage In-Reply-To: <98409.67270.qm@web31808.mail.mud.yahoo.com> References: <98409.67270.qm@web31808.mail.mud.yahoo.com> Message-ID: <3dbc6d1c0910081638r42c50667u6c7d867fb02b3e3c@mail.gmail.com> Aren't kept in the normal process heaps I mean of course. From mjtruog@REDACTED Fri Oct 9 01:51:22 2009 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 08 Oct 2009 16:51:22 -0700 Subject: [erlang-questions] Huge binary usage In-Reply-To: <3dbc6d1c0910081636l164b2211kea9e086d6c99d597@mail.gmail.com> References: <98409.67270.qm@web31808.mail.mud.yahoo.com> <3dbc6d1c0910081636l164b2211kea9e086d6c99d597@mail.gmail.com> Message-ID: <4ACE7AFA.4070205@gmail.com> So you are probably generating binaries faster than the garbage collector is removing them (especially if you are holding onto their references). You can try to find the offending process with the instrument module (http://erlang.org/doc/man/instrument.html). If you want to be a bit harsh on a function to force binary garbage collection, as quick as possible, you can use code similar to what I have attached below. - Michael %%% -*- coding: utf-8; Mode: erlang; tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- %%% ex: set softtabstop=4 tabstop=4 shiftwidth=4 expandtab fileencoding=utf-8: %%% %%%------------------------------------------------------------------------ %%% @doc %%% ==Enforce immediate garbage collection on a function== %%% @end %%% %%% BSD LICENSE %%% %%% Copyright (c) 2009, Michael Truog %%% All rights reserved. %%% %%% Redistribution and use in source and binary forms, with or without %%% modification, are permitted provided that the following conditions are met: %%% %%% * Redistributions of source code must retain the above copyright %%% notice, this list of conditions and the following disclaimer. %%% * Redistributions in binary form must reproduce the above copyright %%% notice, this list of conditions and the following disclaimer in %%% the documentation and/or other materials provided with the %%% distribution. %%% * All advertising materials mentioning features or use of this %%% software must display the following acknowledgment: %%% This product includes software developed by Michael Truog %%% * The name of the author may not be used to endorse or promote %%% products derived from this software without specific prior %%% written permission %%% %%% THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND %%% CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, %%% INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES %%% OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE %%% DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR %%% CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, %%% SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, %%% BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR %%% SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS %%% INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, %%% WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING %%% NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE %%% OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH %%% DAMAGE. %%% %%% @author Michael Truog %%% @copyright 2009 Michael Truog %%% @version 0.0.4 {@date} {@time} %%%------------------------------------------------------------------------ -module(immediate_gc). -author('mjtruog [at] gmail (dot) com'). %% external interface -export([sync_fun/2, sync_fun/3, async_fun/2, async_fun/3]). %%%------------------------------------------------------------------------ %%% External interface functions %%%------------------------------------------------------------------------ %%------------------------------------------------------------------------- %% @doc %% ===Make a synchronous call to a function that will be garbage collected when the function returns.=== %% @end %%------------------------------------------------------------------------- -spec sync_fun(M :: atom(), F :: atom(), A :: list()) -> any(). sync_fun(M, F, A) when is_atom(M), is_atom(F), is_list(A) -> Parent = self(), Child = erlang:spawn_opt(fun() -> Parent ! {self(), erlang:apply(M, F, A)}, erlang:garbage_collect() end, [link, {fullsweep_after, 0}]), receive {Child, Result} -> Result end. %%------------------------------------------------------------------------- %% @doc %% ===Make a synchronous call to an anonymous function that will be garbage collected when the function returns.=== %% @end %%------------------------------------------------------------------------- -spec sync_fun(F :: fun(), A :: list()) -> any(). sync_fun(F, A) when is_function(F), is_list(A) -> Parent = self(), Child = erlang:spawn_opt(fun() -> Parent ! {self(), erlang:apply(F, A)}, erlang:garbage_collect() end, [link, {fullsweep_after, 0}]), receive {Child, Result} -> Result end. %%------------------------------------------------------------------------- %% @doc %% ===Make an asynchronous call to a function that will be garbage collected when the function returns.=== %% @end %%------------------------------------------------------------------------- -spec async_fun(M :: atom(), F :: atom(), A :: list()) -> pid(). async_fun(M, F, A) when is_atom(M), is_atom(F), is_list(A) -> erlang:spawn_opt(fun() -> erlang:apply(M, F, A), erlang:garbage_collect() end, [link, {fullsweep_after, 0}]). %%------------------------------------------------------------------------- %% @doc %% ===Make an asynchronous call to an anonymous function that will be garbage collected when the function returns.=== %% @end %%------------------------------------------------------------------------- -spec async_fun(F :: fun(), A :: list()) -> pid(). async_fun(F, A) when is_function(F), is_list(A) -> erlang:spawn_opt(fun() -> erlang:apply(F, A), erlang:garbage_collect() end, [link, {fullsweep_after, 0}]). Robert Virding wrote: > Large binaries aren't kept in the normal process heaps but outside of them. > This means that they are not collected in the same way as normal data and > can hang around for a longer time, though they will be collected. > > Robert > > 2009/10/9 Yogish Baliga > > >> I have a erlang process which is running at 376MB Resident memory. >> erlang:memory() shows the following output. >> >> [{total,265245056}, >> {processes,56800140}, >> {processes_used,56777884}, >> {system,208444916}, >> {atom,662173}, >> {atom_used,636586}, >> {binary,178220728}, >> {code,4720224}, >> {ets,21754908}] >> >> Seems like most of the memory is used by "binary". >> >> Along with this, I did a sum of total memory used by all the processes >> running on the system. It comes to around 53MB (which is same as what is >> reported in processes_used). >> >> Question is how can I find what process(es) allocated the binaries and why >> they are not released? >> >> Thanx, >> -- baliga >> >> "Point of view is worth 80 IQ points" --Alan Kay >> >> http://dudefrommangalore.blogspot.com/ >> > > From igorrs@REDACTED Fri Oct 9 02:05:10 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Thu, 8 Oct 2009 21:05:10 -0300 Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: References: Message-ID: Hello. Indeed, I "have hoped for O (1) after the first O (log N) call". But, thinking again at my case, I don't really need the iterator. Given a key K, I believe it will be enough to find the smallest key that is greater or equals K. So, ets:lookup/2, maybe followed by ets:next/2, would do. Thanks for the answers. Igor. On Thu, Oct 8, 2009 at 8:32 PM, Paul Mineiro wrote: > On Thu, 8 Oct 2009, Igor Ribeiro Sucupira wrote: > >> Hi. >> >> I need some ordered set that allows me to "search and then iterate" >> efficiently. In other words, I need this functionality: >> http://www.cplusplus.com/reference/stl/set/lower_bound/ >> >> After looking at the documentation for gb_sets and ordsets, I have >> concluded that Erlang does not provide that functionality and I'll >> have to roll my own. Is it the right conclusion? ?:-) > > ets:next/2 on an ordered_set table acts like that. ?"efficiently" ... > well, it'll be O (log N) per call, you might have hoped for O (1) after > the first O (log N) call. ?also you might not like the constant factors. > benchmark to be sure. > > -- p > > p.z. perhaps ets:slot/2 is faster? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From amit.murthy@REDACTED Fri Oct 9 02:18:01 2009 From: amit.murthy@REDACTED (Amit Murthy) Date: Fri, 9 Oct 2009 05:48:01 +0530 Subject: [erlang-questions] wxErlang - wxLogNull question In-Reply-To: <93df43b60910080025l51a217eanf7090cd4d8c1596@mail.gmail.com> References: <4f5fdb630910072114m2137f163v466053684bcf2ef3@mail.gmail.com> <93df43b60910080025l51a217eanf7090cd4d8c1596@mail.gmail.com> Message-ID: <4f5fdb630910081718s2cb99c6n80e2db9b06e36af9@mail.gmail.com> > > > > > But wxErlang does not expose wxLogNull. Any particular reason why? Any > other > > way to avoid the popup. > > No reason I didn't know about it, send a patch :-) > > OK. Will do once I understand how the driver works... Amit From allan.merolla@REDACTED Fri Oct 9 02:21:24 2009 From: allan.merolla@REDACTED (Allan Merolla) Date: Fri, 9 Oct 2009 10:21:24 +1000 Subject: : [erlang-questions] Mnesia 25MB database takes 15min to load In-Reply-To: <4ACDC922.2000800@erlang-consulting.com> References: <4ACDC922.2000800@erlang-consulting.com> Message-ID: Thank you very much guys. You were correct. I added a second index on time which was slowing things to a crawl on load. I removed the index and things were great again. I didn't really understand what was happening until I switched to a disc_only_copies table and saw that there was a second table called keyVal_3 the number 3 was generated by mnesia to represent the additional index I specified. I'm not sure why it generates this on load for the memory resident version of the table but that was definitely the cause. Now I'm trying to figure out what's better a disc_only_copies table with an additional index or disc_copies without. Appreciate your help with this and apologize for my noobiness =). Cheers Allan -----Original Message----- From: Roberto Aloi [mailto:roberto.aloi@REDACTED] Sent: Thursday, 8 October 2009 9:13 PM To: Allan Merolla Cc: 'erlang-questions@REDACTED' Subject: Re: [erlang-questions] Mnesia 25MB database takes 15min to load!!! Hi Allan, you might want to have a look to this thread on Trapexit: http://www.trapexit.org/forum/viewtopic.php?p=44433 Regards, Roberto Aloi Erlang Training and Consulting Ltd. http://www.erlang-consulting.com http://aloiroberto.wordpress.com Allan Merolla wrote: > Hello All, > > I am currently using mnesia to store simple records consisting of {ID,Time,Value}. The purpose of the database is simple. High availability of large amounts of "simple data". I have inserted 500 thousand records and the DCD is 24mb. When I do a mnesia:stop() and start() it takes over 10min just to load the test data. I originally had planned to have this fragmented and have at least 200mb per fragment but at 10min load for 25mb the whole idea seems impossible now as it would take over 2hours to start a node. I find it hard to believe this is by design? Can anyone tell me what I am doing wrong? > > Thanks in advance folks. > > Cheers, > Allan > > > > From amit.murthy@REDACTED Fri Oct 9 02:24:58 2009 From: amit.murthy@REDACTED (Amit Murthy) Date: Fri, 9 Oct 2009 05:54:58 +0530 Subject: [erlang-questions] wxErlang - wxLogNull question In-Reply-To: <200910081223.24970.clist@uah.es> References: <4f5fdb630910072114m2137f163v466053684bcf2ef3@mail.gmail.com> <200910081223.24970.clist@uah.es> Message-ID: <4f5fdb630910081724k67297e27l68f17be37f2f6ce4@mail.gmail.com> > > this seems to be the reason for having a separate wxexlang os process > instead of a driver. > If people begins to exercise more and more library funtions for porpouses > other that display a GUI > we can lose all the resilence we have in the VM. > > Just take this "or the image binary is corrup" and now how do you (1) "let > other process fix the error" when we > have a lot of code and subtle bugs buried inside a big erlang driver? > > Just started using wxErlang, but I did not notice the following : - create a wxImage object from a corrupted image file. - the getHeight() and getWidth() functions return 0, which is OK - but, if you execute some other image manipulation functions like convertAlphaToMask(), it results in a "segmentation fault" and crashes the entire VM. Amit From ok@REDACTED Fri Oct 9 04:34:12 2009 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 9 Oct 2009 15:34:12 +1300 Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: References: Message-ID: <21B2D38B-9831-4B9B-99A8-8B2771CE0BCA@cs.otago.ac.nz> On Oct 9, 2009, at 12:12 PM, Igor Ribeiro Sucupira wrote: > Hi. > > I need some ordered set that allows me to "search and then iterate" > efficiently. In other words, I need this functionality: > http://www.cplusplus.com/reference/stl/set/lower_bound/ What you want is a function something like iterator_beginning_at(Set, Element) -> Iterator The existing gb_sets *module* does not support that, but the *data structure* easily could. Look at the code for constructing and using iterators. reverse_iterator/1 and previous/1 are mine. tree --> {key,tree,tree} | 'nil'. iterator --> [tree|iterator] | []. The (sub)trees in an iterator are there for the sake of the key and _one_ of the subtrees. iterator({_Size, Tree}) -> iterator(Tree, []). iterator({_X, nil, _Right} = Tree, As) -> [Tree|As]; iterator({_X, Left, _Right} = Tree, As) -> iterator(Left, [Tree|As]); iterator(nil, As) -> As. next([{X, _Left, Right}|As]) -> {X, iterator(Right, As)}; next([]) -> none. reverse_iterator({_Size, Tree}) -> reverse_iterator(Tree, []). reverse_iterator({_X, _Left, nil} = Tree, As) -> [Tree|As]; reverse_iterator({_X, _Left, Right} = Tree, As) -> reverse_iterator(Right, [Tree|As]); reverse_iterator(nil, As) -> As. previous([{X, Left, _Right}|As]) -> {X, reverse_iterator(Left, As)}; previous([]) -> none. Perhaps the simplest way to do it would be to use iterator(remove_less(Set, Start)) or reverse_iterator(remove_greater(Set, Start)) where remove_less(Set, X) -> {Y \in Set | Y >= X} remove_greater(Set, X) -> {Y \in Set | X >= Y} However, since the size of a gb_set is stored only at the top, exporting these functions in the interface, returning GB sets, would require time linear in the result. It's OK to use them internally, though, as iterators don't need to know the size. So geq_iterator({_Size, Tree}, Min) -> iterator(remove_less(Tree, Min), []). leq_iterator({_Size, Tree}, Max) -> iterator(remove_greater(Tree, Max), []). range_iterator({_Size, Tree}, Min, Max) -> iterator(remove_less(remove_greater(Tree, Max), Min), []). geq_reverse_iterator({_Size, Tree}, Min) -> reverse_iterator(remove_less(Tree, Min), []). leq_reverse_iterator({_Size, Tree}, Max) -> reverse_iterator(remove_greater(Tree, Max), []). range_reverse_iterator({_Size, Tree}, Min, Max) -> reverse_iterator(remove_less(remove_greater(Tree, Max), Min), []). remove_less({Key,Left,Right}, Min) -> if Key < Min -> remove_less(Right, Min) ; Key > Min -> {Key, remove_less(Left, Min), Right} ; true -> {Key, nil, Right} end; remove_less(nil, _) -> nil. remove_greater({Key,Left,Right}, Max) -> if Key > Max -> remove_greater(Left, Max) ; Key < Max -> {Key, Left, remove_greater(Right, Max)} ; true -> {Key, Left, nil} end; remove_greater(nil, _) -> nil. I made a copy of gb_sets.erl called gb_sets2.erl, added this code to it, added an -export declaration -export([reverse_iterator/1, previous/1, geq_iterator/2, geq_reverse_iterator/2, leq_iterator/2, leq_reverse_iterator/2, range_iterator/3, range_reverse_iterator/3]). and put the stuff through some testing, and it seemed to work. This has been tested, BUT NOT THOROUGHLY TESTED. From ok@REDACTED Fri Oct 9 04:43:11 2009 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 9 Oct 2009 15:43:11 +1300 Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: References: Message-ID: <4D659E9E-B9CA-442F-A204-CD1AF1640461@cs.otago.ac.nz> On Oct 9, 2009, at 1:05 PM, Igor Ribeiro Sucupira wrote: > Hello. > > Indeed, I "have hoped for O (1) after the first O (log N) call". Just off the top of my head, gb_sets iterators give you O(1) *amortised* time per call to next/1. So the {geq,leq,range}_{reverse_,}iterator functions I sent in my previous message give you in O(log N) time an iterator which subsequently gives you O(1) *amortised* time per call. > But, thinking again at my case, I don't really need the iterator. > Given a key K, I believe it will be enough to find the smallest key > that is greater or equals K. The gb_set data structure, or any balanced tree, can do that in O(log N) time too. With the functions in my previous message, case gb_sets2:next(gb_sets2:geq_iterator(Set, K)) of {M, _} -> M is the smallest element of Set that is greater than or equal to K ; none -> there is no such M end will do the trick. You can of course do it more efficiently than that, but only by writing even more code inside the module. > > From juliadotter@REDACTED Fri Oct 9 06:13:59 2009 From: juliadotter@REDACTED (Nina Alex Juliadotter) Date: Fri, 9 Oct 2009 15:13:59 +1100 Subject: PDF reports from Mnesia: Erlguten and XMerL? In-Reply-To: <97814d820910081916vff16452kb9a817af7ff8d7cc@mail.gmail.com> References: <97814d820910081916vff16452kb9a817af7ff8d7cc@mail.gmail.com> Message-ID: <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> Hi all I'm writing an app with a Mnesia backend that needs to generate PDF files from the data and was wondering if anyone had any tips on how to go about doing this. I've come across Erlguten for PDF generation, which takes XML files as input. So, to use Mnesia data, I take it I could use XMerL to transform it into XML? The only trouble is that both these projects seem rather idle and alpha like, the last time I've seen any discussions was in 2004 and the source I found is from 2003. Is anyone generating PDF files in their Erlang app, and in that case what libraries are you using? Thanks heaps Nina From igorrs@REDACTED Fri Oct 9 07:49:24 2009 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Fri, 9 Oct 2009 02:49:24 -0300 Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: <21B2D38B-9831-4B9B-99A8-8B2771CE0BCA@cs.otago.ac.nz> References: <21B2D38B-9831-4B9B-99A8-8B2771CE0BCA@cs.otago.ac.nz> Message-ID: On Thu, Oct 8, 2009 at 11:34 PM, Richard O'Keefe wrote: > > On Oct 9, 2009, at 12:12 PM, Igor Ribeiro Sucupira wrote: > >> Hi. >> >> I need some ordered set that allows me to "search and then iterate" >> efficiently. In other words, I need this functionality: >> http://www.cplusplus.com/reference/stl/set/lower_bound/ > > What you want is a function something like > ? ? ? ?iterator_beginning_at(Set, Element) -> Iterator > > The existing gb_sets *module* does not support that, but the > *data structure* easily could. Hi, Richard. My question was actually "Do I need to implement it?". So, I was not questioning the capability of gb_sets to support that. But, thank you for the code. Although I realized I don't really need the iterators now, I'll certainly need them some day, since I happened to need STL's lower_bound a couple of times in the past. Also, I've had a glance at your code and it makes sense to me. :-) geq_iterator rearranges the tree to not contain the elements smaller than Min and then returns an iterator to that new tree. Looks OK. :-) Thanks. Igor. > Look at the code for constructing and using iterators. > reverse_iterator/1 and previous/1 are mine. > > tree --> {key,tree,tree} | 'nil'. > iterator --> [tree|iterator] | []. > The (sub)trees in an iterator are there for the sake of the key > and _one_ of the subtrees. > > iterator({_Size, Tree}) -> > ? ?iterator(Tree, []). > > iterator({_X, nil, _Right} = Tree, As) -> > ? ?[Tree|As]; > iterator({_X, Left, _Right} = Tree, As) -> > ? ?iterator(Left, [Tree|As]); > iterator(nil, As) -> > ? ?As. > > next([{X, _Left, Right}|As]) -> > ? ?{X, iterator(Right, As)}; > next([]) -> > ? ?none. > > reverse_iterator({_Size, Tree}) -> > ? ?reverse_iterator(Tree, []). > > reverse_iterator({_X, _Left, nil} = Tree, As) -> > ? ?[Tree|As]; > reverse_iterator({_X, _Left, Right} = Tree, As) -> > ? ?reverse_iterator(Right, [Tree|As]); > reverse_iterator(nil, As) -> > ? ?As. > > previous([{X, Left, _Right}|As]) -> > ? ?{X, reverse_iterator(Left, As)}; > previous([]) -> > ? ?none. > > Perhaps the simplest way to do it would be to use > ? ?iterator(remove_less(Set, Start)) > or ?reverse_iterator(remove_greater(Set, Start)) > where > > ? ?remove_less(Set, X) ? ?-> {Y \in Set | Y >= X} > ? ?remove_greater(Set, X) -> {Y \in Set | X >= Y} > > However, since the size of a gb_set is stored only at the top, > exporting these functions in the interface, returning GB sets, > would require time linear in the result. ?It's OK to use them > internally, though, as iterators don't need to know the size. > > So > > geq_iterator({_Size, Tree}, Min) -> > ? ?iterator(remove_less(Tree, Min), []). > > leq_iterator({_Size, Tree}, Max) -> > ? ?iterator(remove_greater(Tree, Max), []). > > range_iterator({_Size, Tree}, Min, Max) -> > ? iterator(remove_less(remove_greater(Tree, Max), Min), []). > > geq_reverse_iterator({_Size, Tree}, Min) -> > ? ?reverse_iterator(remove_less(Tree, Min), []). > > leq_reverse_iterator({_Size, Tree}, Max) -> > ? ?reverse_iterator(remove_greater(Tree, Max), []). > > range_reverse_iterator({_Size, Tree}, Min, Max) -> > ? ?reverse_iterator(remove_less(remove_greater(Tree, Max), Min), []). > > remove_less({Key,Left,Right}, Min) -> > ? ?if Key < Min -> remove_less(Right, Min) > ? ? ; Key > Min -> {Key, remove_less(Left, Min), Right} > ? ? ; true ? ? ?-> {Key, nil, Right} > ? ?end; > remove_less(nil, _) -> > ? ?nil. > > remove_greater({Key,Left,Right}, Max) -> > ? ?if Key > Max -> remove_greater(Left, Max) > ? ? ; Key < Max -> {Key, Left, remove_greater(Right, Max)} > ? ? ; true ? ? ?-> {Key, Left, nil} > ? ?end; > remove_greater(nil, _) -> > ? ?nil. > > > I made a copy of gb_sets.erl called gb_sets2.erl, > added this code to it, added an -export declaration > > -export([reverse_iterator/1, previous/1, > ? ? ? ? geq_iterator/2, ? ? geq_reverse_iterator/2, > ? ? ? ? leq_iterator/2, ? ? leq_reverse_iterator/2, > ? ? ? ? range_iterator/3, ? range_reverse_iterator/3]). > > and put the stuff through some testing, and it seemed to work. > This has been tested, BUT NOT THOROUGHLY TESTED. > > > > > > > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From max.lapshin@REDACTED Fri Oct 9 08:43:49 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Oct 2009 10:43:49 +0400 Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: References: <21B2D38B-9831-4B9B-99A8-8B2771CE0BCA@cs.otago.ac.nz> Message-ID: It seems, that I require the same functionality: list of frames is stored in memory and when user requests frame number N, I need the first keyframe, left to requested. They are stored in ets ordered set table (it becomes rather large), so I take all keyframes, left to requested and select first from them. Very, very non-optimal =( From dangud@REDACTED Fri Oct 9 09:47:41 2009 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 9 Oct 2009 09:47:41 +0200 Subject: [erlang-questions] wxErlang - wxLogNull question In-Reply-To: <4f5fdb630910081724k67297e27l68f17be37f2f6ce4@mail.gmail.com> References: <4f5fdb630910072114m2137f163v466053684bcf2ef3@mail.gmail.com> <200910081223.24970.clist@uah.es> <4f5fdb630910081724k67297e27l68f17be37f2f6ce4@mail.gmail.com> Message-ID: <93df43b60910090047q226a0fa9p6b010e412be076d9@mail.gmail.com> On Fri, Oct 9, 2009 at 2:24 AM, Amit Murthy wrote: >> >> this seems to be the reason for having a separate wxexlang os process >> instead of a driver. >> If people begins to exercise more and more library funtions for porpouses >> other that display a GUI >> we can lose all the resilence we have in the VM. >> >> Just take this "or the image binary is corrup" and now how do you (1) "let >> other process fix the error" when we >> have a lot of code and subtle bugs buried inside a big erlang driver? >> >> > OK. Will do once I understand how the driver works... That was a joke I'll fix it if I can find the time, but if you want to I'll gladly accept a patch. > Just started using wxErlang, but I did not notice the following : > > ?- create a wxImage object from a corrupted image file. > ?- the getHeight() and getWidth() functions return 0, which is OK > ?- but, if you execute some other image manipulation functions like > convertAlphaToMask(), > it results in a "segmentation fault" and crashes the entire VM. Yeah that is the problem with wxWidgets and a driver. It's very tough for me to workaround every way you can crash wxWidgets. Don't use it on your critical server directly, use the erlang-distribution and let the gui access the server from another node for critical systems. Using rpc or communicating with another node is easier in erlang than in any other language that I know. /Dan > > > ?Amit > From ulf.wiger@REDACTED Fri Oct 9 10:30:11 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 09 Oct 2009 10:30:11 +0200 Subject: [erlang-questions] PDF reports from Mnesia: Erlguten and XMerL? In-Reply-To: <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> References: <97814d820910081916vff16452kb9a817af7ff8d7cc@mail.gmail.com> <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> Message-ID: <4ACEF493.7040105@erlang-consulting.com> Nina Alex Juliadotter wrote: > > I've come across Erlguten for PDF generation, which takes XML files as > input. So, to use Mnesia data, I take it I could use XMerL to > transform it into XML? The only trouble is that both these projects > seem rather idle and alpha like, the last time I've seen any > discussions was in 2004 and the source I found is from 2003. Xmerl idle and alpha? I must have misunderstood this, or you have been looking at the jungerl version of xmerl. It has been a supported part of OTP for several years now. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ulf.wiger@REDACTED Fri Oct 9 11:24:01 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 09 Oct 2009 11:24:01 +0200 Subject: [erlang-questions] PDF reports from Mnesia: Erlguten and XMerL? In-Reply-To: <4ACEF694.60303@bigpond.net.au> References: <97814d820910081916vff16452kb9a817af7ff8d7cc@mail.gmail.com> <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> <4ACEF493.7040105@erlang-consulting.com> <4ACEF694.60303@bigpond.net.au> Message-ID: <4ACF0131.70305@erlang-consulting.com> Benjamin Tolputt wrote: > > That said, I am interested in the status of erlguten. I haven't heard of > it being used for years, yet highly-scalable PDF generation would be > pretty useful to have. The best hope in this regard ought to be http://code.google.com/p/erlguten/ It doesn't seem to have been updated much since it was uploaded to Google Code in 2007, but that's not to say that it isn't being used, of course. :) BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From dangud@REDACTED Fri Oct 9 11:45:13 2009 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 9 Oct 2009 11:45:13 +0200 Subject: [erlang-questions] PDF reports from Mnesia: Erlguten and XMerL? In-Reply-To: <4ACF0131.70305@erlang-consulting.com> References: <97814d820910081916vff16452kb9a817af7ff8d7cc@mail.gmail.com> <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> <4ACEF493.7040105@erlang-consulting.com> <4ACEF694.60303@bigpond.net.au> <4ACF0131.70305@erlang-consulting.com> Message-ID: <93df43b60910090245hca63fc1u36a5cdb4fb380597@mail.gmail.com> On Fri, Oct 9, 2009 at 11:24 AM, Ulf Wiger wrote: > Benjamin Tolputt wrote: >> >> That said, I am interested in the status of erlguten. I haven't heard of >> it being used for years, yet highly-scalable PDF generation would be >> pretty useful to have. > > The best hope in this regard ought to be > http://code.google.com/p/erlguten/ > > It doesn't seem to have been updated much since it was uploaded to > Google Code in 2007, but that's not to say that it isn't being > used, of course. :) Another hack would be to use a 'pdf' printer and use wx to render to it. atleast on my linux with gnome-printing such a device is available. Run wx:demo() and select canvas_paint, from there I invoke print select printer 'Create a pdf document'. I don't know it is possible to get access to it via wx interface without a user selecting it but it might be worth a try. /Dan > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From btolputt@REDACTED Fri Oct 9 10:38:44 2009 From: btolputt@REDACTED (Benjamin Tolputt) Date: Fri, 09 Oct 2009 19:38:44 +1100 Subject: [erlang-questions] PDF reports from Mnesia: Erlguten and XMerL? In-Reply-To: <4ACEF493.7040105@erlang-consulting.com> References: <97814d820910081916vff16452kb9a817af7ff8d7cc@mail.gmail.com> <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> <4ACEF493.7040105@erlang-consulting.com> Message-ID: <4ACEF694.60303@bigpond.net.au> Ulf Wiger wrote: > Nina Alex Juliadotter wrote: >> >> I've come across Erlguten for PDF generation, which takes XML files as >> input. So, to use Mnesia data, I take it I could use XMerL to >> transform it into XML? The only trouble is that both these projects >> seem rather idle and alpha like, the last time I've seen any >> discussions was in 2004 and the source I found is from 2003. > > Xmerl idle and alpha? I must have misunderstood this, or you have been > looking at the jungerl version of xmerl. It has been a supported > part of OTP for several years now. Agreed. As I understand it xmerl has been the "official" XML processing module for Erlang for some time. That said, I am interested in the status of erlguten. I haven't heard of it being used for years, yet highly-scalable PDF generation would be pretty useful to have. -- Regards, Benjamin Tolputt Analyst Programmer From roberto.aloi@REDACTED Fri Oct 9 12:05:30 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Fri, 09 Oct 2009 11:05:30 +0100 Subject: [erlang-questions] PDF reports from Mnesia: Erlguten and XMerL? In-Reply-To: <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> References: <97814d820910081916vff16452kb9a817af7ff8d7cc@mail.gmail.com> <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> Message-ID: <4ACF0AEA.8010000@erlang-consulting.com> Hi Nina, I had to automatically generate PDF around one year ago. After struggling a bit with Erlguten, I found it a bit too "alpha". My requirements were very different from yours, but this is what I did at the end. - I defined a new LaTeX template - I reused some code from the wpart_gen.erl module contained in Erlang Web to build a "file injector" (15 minutes job). - Then I was able to do the following: {ok, Title} = injector:inject("report.tex", [{"title", Title}]), Where report.tex was my .tex file, looking like: [...] \author{Roberto Aloi} \title{<% title %>} \begin{center} [...] - Then I used the latex2pdf to generate the pdf file. Again, this is far away from an optimal and/or efficient solution, but it worked fine and it was quick! Regards, Roberto Aloi Nina Alex Juliadotter wrote: > Hi all > > I'm writing an app with a Mnesia backend that needs to generate PDF > files from the data and was wondering if anyone had any tips on how to > go about doing this. > > I've come across Erlguten for PDF generation, which takes XML files as > input. So, to use Mnesia data, I take it I could use XMerL to > transform it into XML? The only trouble is that both these projects > seem rather idle and alpha like, the last time I've seen any > discussions was in 2004 and the source I found is from 2003. Is anyone > generating PDF files in their Erlang app, and in that case what > libraries are you using? > > Thanks heaps > Nina > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From nm@REDACTED Fri Oct 9 14:12:20 2009 From: nm@REDACTED (Gaspar Chilingarov) Date: Fri, 09 Oct 2009 17:12:20 +0500 Subject: [erlang-questions] PDF reports from Mnesia: Erlguten and XMerL? In-Reply-To: <4ACF0AEA.8010000@erlang-consulting.com> References: <97814d820910081916vff16452kb9a817af7ff8d7cc@mail.gmail.com> <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> <4ACF0AEA.8010000@erlang-consulting.com> Message-ID: <4ACF28A4.7010600@web.am> Hi Nina! I would like to share my experience. In my case there were a reports, which should be sent every month. I've used ready html2pdf converter (written in php) for a several reasons: - I had the template engine, which allows generation of HTML from HTML template file and deeply nested proplist structures (and it supports FOREACH and IF statements in templates - so they are completely flexible). - HTML templates are easily editable in any visual HTML editor - so I can pass all design work to another person. - html2pdf converter usually works faster than latex. - you can easily check your output before pdf generation and fix it or write automatic tests. The remaining part was exactly the same as below - spawning external command and so on and so on. /Gaspar PS. When generating command name and arguments - be VERY careful. Do not allow user inject any illegal symbol in that string, in other case it may lead to OS command injection by remote attacker. On 09/10/09 15:05, Roberto Aloi wrote: > Hi Nina, > > I had to automatically generate PDF around one year ago. After > struggling a bit with Erlguten, I found it a bit too "alpha". > My requirements were very different from yours, but this is what I did > at the end. > > - I defined a new LaTeX template > - I reused some code from the wpart_gen.erl module contained in Erlang > Web to build a "file injector" (15 minutes job). > - Then I was able to do the following: > > {ok, Title} = injector:inject("report.tex", [{"title", Title}]), > > Where report.tex was my .tex file, looking like: > > [...] > \author{Roberto Aloi} > \title{<% title %>} > \begin{center} > [...] > > - Then I used the latex2pdf to generate the pdf file. > > Again, this is far away from an optimal and/or efficient solution, but > it worked fine and it was quick! > > Regards, > > Roberto Aloi > > Nina Alex Juliadotter wrote: >> Hi all >> >> I'm writing an app with a Mnesia backend that needs to generate PDF >> files from the data and was wondering if anyone had any tips on how to >> go about doing this. >> >> I've come across Erlguten for PDF generation, which takes XML files as >> input. So, to use Mnesia data, I take it I could use XMerL to >> transform it into XML? The only trouble is that both these projects >> seem rather idle and alpha like, the last time I've seen any >> discussions was in 2004 and the source I found is from 2003. Is anyone >> generating PDF files in their Erlang app, and in that case what >> libraries are you using? >> >> Thanks heaps >> Nina >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Gaspar Chilingarov tel +37493 419763 (mobile - leave voice mail message) icq 63174784 skype://gasparch e mailto:nm@REDACTED mailto:gasparch@REDACTED w http://gasparchilingarov.com/ From kagato@REDACTED Fri Oct 9 16:11:57 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 9 Oct 2009 09:11:57 -0500 Subject: [erlang-questions] crash collectors (with emailing) In-Reply-To: References: Message-ID: <8E563965-36F0-4470-8272-5FDA142B60C6@souja.net> Yes. This would require an error_logger. It's a gen_event, so you'll want to brush up on the gen_event docs. If you want examples, the sasl TTY logger is the simplest (see OTP_ROOT/lib/sasl-X.X.X/src/ sasl_report_tty_h.erl). There's also log_mf_h (in OTP_ROOT/lib/stdlib- X.X.X/src/log_mf_h.erl). Making a logger is dead-simple. One other note, if you're doing Distributed Erlang, I believe that system-level errors (like node-down) may be logged on the node they occur, but remote processes may log to their group-leader (which may be on another node). So you may have weird issues getting those logs when network problems are involved. Good luck! On Oct 8, 2009, at 3:58 PM, Max Lapshin wrote: > There are many services in world of Ruby on Rails, that collect > exceptions, raised on production servers > and send them directly via email, or via buffering service (if there > are thousands of similar errors, you will receive only one). > > It haven't yet seen anything the same for Erlang. I want to receive > via email crash dumps, logged to error_logger. > Am I right, that I will need to make an error_logger handler for this? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- Jayson Vantuyl kagato@REDACTED From kagato@REDACTED Fri Oct 9 16:20:47 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 9 Oct 2009 09:20:47 -0500 Subject: [erlang-questions] : [erlang-questions] Mnesia 25MB database takes 15min to load In-Reply-To: References: <4ACDC922.2000800@erlang-consulting.com> Message-ID: <9EF7CECD-04F1-4AB1-AADA-84776403AE17@souja.net> > Now I'm trying to figure out what's better a disc_only_copies table > with an additional index or disc_copies without. This depends entirely on your dataset. Note that disc_only_copies doesn't support ordered_set type (at least, that's what the docs say). In almost all cases, disc_copies would probably be faster. > Appreciate your help with this and apologize for my noobiness =). This is certainly not a trivial problem. Mnesia is finicky even for the experienced. Good luck! -- Jayson Vantuyl kagato@REDACTED From christopher.faulet@REDACTED Fri Oct 9 16:31:44 2009 From: christopher.faulet@REDACTED (christopher faulet) Date: Fri, 09 Oct 2009 16:31:44 +0200 Subject: Epp Bugfix and macros overloading Message-ID: <4ACF4950.4070409@capflam.org> Hi all, Here are two patches for the module epp in R13B02-1. * The first one fixes a bug in the function scan_undef. The dict that helps to find circular macros is not correctly updated. Here is an example: ============================= -module(test). -export([test/0]). -define(FOO(X), ?BAR(X)). -define(BAR(X), ?FOO(X)). -undef(FOO). test() -> ?BAR(1). ============================= On the last line, the preprocessor should found that 'FOO' is undefined. But, with the current version, it finds a cycle between FOO and BAR. * The second patch includes the first one and adds support of multiple definitions for macros in the module epp (i.e. with the same name but with different arities). This feature wouldn't break any code (I hope so :) and might be usefull. Here is an simple example that uses it: ============================= -module(test). -export([test/0]). -define(MY_MACRO(), foo). -define(MY_MACRO, bar). -define(MY_MACRO(X,Y), {X, Y}). test() -> %% return [bar, foo, foo, {foo, bar}] [ ?MY_MACRO, %% use the 2nd def, replaced by bar ?MY_MACRO(), %% use the 1st def, replaced by foo ?MY_MACRO(foo), %% use the 2nd def, replaced by bar(foo) ?MY_MACRO(foo, bar) %% use the 3rd def, replaced by {foo,bar} ]. bar(X) -> X. ============================= In this patch, we follow these rules: - the overloading of predefined macros (?MODULE, ?LINE, ...) is forbidden - the directive '-undef' removes all definitions of a macro - when a macro is called, the preprocessor tries to match the definition with the right number of arguments. If it fails, it uses the constant definition, if there is one, else returns an error. - the circular definitions are checked following the previous rule. It would be interesting to add this feature. What do you think about it? Best regards, -- Christopher Faulet -------------- next part -------------- A non-text attachment was scrubbed... Name: bugfix-epp-R13B02-1.patch Type: text/x-patch Size: 940 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: overload-epp-R13B02-1.patch Type: text/x-patch Size: 10034 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From kagato@REDACTED Fri Oct 9 16:35:31 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 9 Oct 2009 09:35:31 -0500 Subject: [erlang-questions] PDF reports from Mnesia: Erlguten and XMerL? In-Reply-To: <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> References: <97814d820910081916vff16452kb9a817af7ff8d7cc@mail.gmail.com> <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> Message-ID: To throw one more hack on the pile, I've found Apache FOP (http://xmlgraphics.apache.org/fop/ ) to be pretty decent. Basically, you generate XSL-FO (originally the output format for XSLT). It can easily create PDFs reproducibly, and is arguably easier to deploy that TeX for that purpose. Its big wins are that it complies with a fairly unambiguous standard, is rock-solid, doesn't require a windowing system, and is maintained. About the only drawback is that the stable version only processes one document per invocation (which can be quite slow). It's pretty trivial to wrap it in some Java, if you have the stomach for it. Of course, you probably still don't enjoy generating XML in Erlang, but you might be able to write your templates in some sort of intermediate template language. If I was doing this today, I'd probably suggest Mustache (http://github.com/defunkt/mustache). Good luck! On Oct 8, 2009, at 11:13 PM, Nina Alex Juliadotter wrote: > Hi all > > I'm writing an app with a Mnesia backend that needs to generate PDF > files from the data and was wondering if anyone had any tips on how to > go about doing this. > > I've come across Erlguten for PDF generation, which takes XML files as > input. So, to use Mnesia data, I take it I could use XMerL to > transform it into XML? The only trouble is that both these projects > seem rather idle and alpha like, the last time I've seen any > discussions was in 2004 and the source I found is from 2003. Is anyone > generating PDF files in their Erlang app, and in that case what > libraries are you using? > > Thanks heaps > Nina > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- Jayson Vantuyl kagato@REDACTED From max.lapshin@REDACTED Fri Oct 9 16:49:32 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Oct 2009 18:49:32 +0400 Subject: [erlang-questions] PDF reports from Mnesia: Erlguten and XMerL? In-Reply-To: References: <97814d820910081916vff16452kb9a817af7ff8d7cc@mail.gmail.com> <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> Message-ID: Also, If you are not tighed to Erlang, I want advice to look at prawn ? tool on Ruby. It works for us. From taavi@REDACTED Fri Oct 9 19:04:57 2009 From: taavi@REDACTED (Taavi Talvik) Date: Fri, 9 Oct 2009 20:04:57 +0300 Subject: [erlang-questions] PDF reports from Mnesia: Erlguten and XMerL? In-Reply-To: <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> References: <97814d820910081916vff16452kb9a817af7ff8d7cc@mail.gmail.com> <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> Message-ID: If erlang is not hard requirement look at www.princexml.com best regards, taavi On Oct 9, 2009, at 7:13 AM, Nina Alex Juliadotter wrote: > Hi all > > I'm writing an app with a Mnesia backend that needs to generate PDF > files from the data and was wondering if anyone had any tips on how to > go about doing this. > > I've come across Erlguten for PDF generation, which takes XML files as > input. So, to use Mnesia data, I take it I could use XMerL to > transform it into XML? The only trouble is that both these projects > seem rather idle and alpha like, the last time I've seen any > discussions was in 2004 and the source I found is from 2003. Is anyone > generating PDF files in their Erlang app, and in that case what > libraries are you using? From fritchie@REDACTED Fri Oct 9 23:45:44 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Fri, 09 Oct 2009 16:45:44 -0500 Subject: [erlang-questions] Passing Filehandles Between Processes In-Reply-To: Message of "Thu\, 08 Oct 2009 22\:31\:48 BST." <523869a70910081431o6d868f79l3d412b075995d06f@mail.gmail.com> Message-ID: <39776.1255124744@snookles.snookles.com> Davide Marqu?s wrote: dm> - finally, *the process that opened the file must be alive for dm> read/write operations to succeed*. This behavior is pretty clearly documented in the R13B02's reference for the file:open/2 function. The process linking (for non-raw file handles) or port linking (for raw file handles) is an important way to avoid leaking file handles when processes crash. It doesn't prevent leaks if a process lives forever but never closes a file handle (or socket, for that matter), but 9/10ths of a pastrami sandwich is better than no sandwich. :-) -Scott From yogishb@REDACTED Sat Oct 10 02:26:11 2009 From: yogishb@REDACTED (Yogish Baliga) Date: Fri, 9 Oct 2009 17:26:11 -0700 (PDT) Subject: [erlang-questions] Huge binary usage In-Reply-To: <3dbc6d1c0910081636l164b2211kea9e086d6c99d597@mail.gmail.com> References: <98409.67270.qm@web31808.mail.mud.yahoo.com> <3dbc6d1c0910081636l164b2211kea9e086d6c99d597@mail.gmail.com> Message-ID: <367461.66057.qm@web31803.mail.mud.yahoo.com> If binary is not used, will that space is reused? Is it possible that the resident memory will grow to a point where VM will crash because of out-of-memory error? -- baliga ________________________________ From: Robert Virding To: Yogish Baliga Cc: Erlang Questions Sent: Thu, October 8, 2009 4:36:49 PM Subject: Re: [erlang-questions] Huge binary usage Large binaries are kept in the normal process heaps but outside of them. This means that they are not collected in the same way as normal data and can hang around for a longer time, though they will be collected. Robert 2009/10/9 Yogish Baliga > I have a erlang process which is running at 376MB Resident memory. > erlang:memory() shows the following output. > > [{total,265245056}, > {processes,56800140}, > {processes_used,56777884}, > {system,208444916}, > {atom,662173}, > {atom_used,636586}, > {binary,178220728}, > {code,4720224}, > {ets,21754908}] > > Seems like most of the memory is used by "binary". > > Along with this, I did a sum of total memory used by all the processes > running on the system. It comes to around 53MB (which is same as what is > reported in processes_used). > > Question is how can I find what process(es) allocated the binaries and why > they are not released? > > Thanx, > -- baliga > > "Point of view is worth 80 IQ points" --Alan Kay > > http://dudefrommangalore.blogspot.com/ From jeffm@REDACTED Sat Oct 10 03:54:48 2009 From: jeffm@REDACTED (jm) Date: Sat, 10 Oct 2009 12:54:48 +1100 Subject: writing an snmp client Message-ID: <4ACFE968.3020405@ghostgun.com> I'm looking in to writting an application which is an SNMP client. I've found two references that look to be worth read namely, http://www.trapexit.org/SNMP_Quick_Start http://www.erlang.org/doc/pdf/snmp.pdf Besides the manual is there anything I should read? Any examples I should look at? What I'm really looking for is a sort of check list or walk through for a minimal client. I tried setting up a snmp manager as described in the quick start (NB: I only followed the section Manager Config on as I don't need an agent) , but got 1> application:start(snmp). ok 2> snmpm:which_users(). ** exception error: bad argument in function ets:match/2 called as ets:match(snmpm_user_table,{user,'$1','_','_'}) in call from snmpm_config:which_users/0 3> snmpm:which_agents(). ** exception error: bad argument in function ets:match/2 called as ets:match(snmpm_agent_table,{{'$1','$2',user_id},'_'}) in call from snmpm_config:which_agents/1 Is there a general bit that's common to the both the agent and the manager that I would have missed by skipping to the manager section? Jeff. From kagato@REDACTED Sat Oct 10 08:54:53 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Sat, 10 Oct 2009 01:54:53 -0500 Subject: [erlang-questions] Passing Filehandles Between Processes In-Reply-To: <39776.1255124744@snookles.snookles.com> References: <39776.1255124744@snookles.snookles.com> Message-ID: <22AEC80E-1B17-4F92-8A25-524611F45ABA@souja.net> Are there any plans to add something like gen_tcp:controlling_process for files? I don't see any easy way to do it, looking at file.erl (and gen_tcp hands it off to some magic in inet). On Oct 9, 2009, at 4:45 PM, Scott Lystig Fritchie wrote: > Davide Marqu?s wrote: > > dm> - finally, *the process that opened the file must be alive for > dm> read/write operations to succeed*. > > This behavior is pretty clearly documented in the R13B02's reference > for > the file:open/2 function. The process linking (for non-raw file > handles) or port linking (for raw file handles) is an important way to > avoid leaking file handles when processes crash. It doesn't prevent > leaks if a process lives forever but never closes a file handle (or > socket, for that matter), but 9/10ths of a pastrami sandwich is better > than no sandwich. :-) > > -Scott > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- Jayson Vantuyl kagato@REDACTED From juanbackson@REDACTED Sat Oct 10 15:45:06 2009 From: juanbackson@REDACTED (Juan Backson) Date: Sat, 10 Oct 2009 21:45:06 +0800 Subject: newbie needing help getting started Message-ID: <27c25bc40910100645t2e2b4d81yfbbc85741723bb98@mail.gmail.com> Hi, I am following the tutorial trying to get started. I am getting error with the following sample code in ERL command line. 10> -module(math1). ** exception error: undefined shell command module/1 11> -export([factorial/1]). ** exception error: bad argument in an arithmetic expression in operator '/'/2 called as factorial / 1 Could someone help me out ? What is wrong ? jb From erlang@REDACTED Sat Oct 10 15:51:57 2009 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 10 Oct 2009 15:51:57 +0200 Subject: [erlang-questions] newbie needing help getting started In-Reply-To: <27c25bc40910100645t2e2b4d81yfbbc85741723bb98@mail.gmail.com> References: <27c25bc40910100645t2e2b4d81yfbbc85741723bb98@mail.gmail.com> Message-ID: <9b08084c0910100651j247116edicfe782501137d42e@mail.gmail.com> You cannot type -module(math1). directly into the shell. make a file (math.erl) containing the lines "-module(math1). ..." etc. then compile the module in the shell like this: > c(math1). This will compile the code in math1.erl, then you can run the functions in the shell > math1:funcname(...) etc. /Joe On Sat, Oct 10, 2009 at 3:45 PM, Juan Backson wrote: > Hi, > > I am following the tutorial trying to get started. ?I am getting error with > the following sample code in ERL command line. > > 10> -module(math1). > ** exception error: undefined shell command module/1 > 11> -export([factorial/1]). > ** exception error: bad argument in an arithmetic expression > ? ? in operator ?'/'/2 > ? ? ? ?called as factorial / 1 > > > Could someone help me out ? ?What is wrong ? > > jb > From juanbackson@REDACTED Sat Oct 10 15:57:09 2009 From: juanbackson@REDACTED (Juan Backson) Date: Sat, 10 Oct 2009 21:57:09 +0800 Subject: [erlang-questions] newbie needing help getting started In-Reply-To: <9b08084c0910100651j247116edicfe782501137d42e@mail.gmail.com> References: <27c25bc40910100645t2e2b4d81yfbbc85741723bb98@mail.gmail.com> <9b08084c0910100651j247116edicfe782501137d42e@mail.gmail.com> Message-ID: <27c25bc40910100657l6afe61aehe6849ea58cdb6c8d@mail.gmail.com> Hi Thank you so much for your help. I managed to compile the sample code, but I can't run it: [root@REDACTED erlang]# cat math1.erl -module(math1). -export([factorial/1]). factorial(0) -> 1; factorial(N) -> N * factorial(N-1). [root@REDACTED erlang]# ls math1.beam math1.erl [root@REDACTED erlang]# erl Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.3 (abort with ^G) 1> math1:factorial(0) 1> ; 1> math1:factorial(0); What am I missing? jb On Sat, Oct 10, 2009 at 9:51 PM, Joe Armstrong wrote: > You cannot type -module(math1). directly into the shell. > make a file (math.erl) containing the lines "-module(math1). ..." etc. > then compile the module in the shell like this: > > > c(math1). > > This will compile the code in math1.erl, then you can run the > functions in the shell > > math1:funcname(...) etc. > > /Joe > > > On Sat, Oct 10, 2009 at 3:45 PM, Juan Backson > wrote: > > Hi, > > > > I am following the tutorial trying to get started. I am getting error > with > > the following sample code in ERL command line. > > > > 10> -module(math1). > > ** exception error: undefined shell command module/1 > > 11> -export([factorial/1]). > > ** exception error: bad argument in an arithmetic expression > > in operator '/'/2 > > called as factorial / 1 > > > > > > Could someone help me out ? What is wrong ? > > > > jb > > > From kiszl@REDACTED Sat Oct 10 15:57:09 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sat, 10 Oct 2009 15:57:09 +0200 Subject: [erlang-questions] newbie needing help getting started In-Reply-To: <27c25bc40910100645t2e2b4d81yfbbc85741723bb98@mail.gmail.com> References: <27c25bc40910100645t2e2b4d81yfbbc85741723bb98@mail.gmail.com> Message-ID: <4AD092B5.6090603@tmit.bme.hu> Hi Juan, You need to copy that piece of code into a file named "math1.erl". Then in the erlang shell first you need to compile the code... 1> c(math1). {ok,math1} ...to be able to use the funcion: 2> math1:factorial(3). 6 Zoltan. PS: If the compile does not work, you need to go to the directory, where "math1.erl" is: 0> cd("C:/Zed/Erlang"). "C:/Zed/Erlang" ok Juan Backson wrote: > Hi, > > I am following the tutorial trying to get started. I am getting error with > the following sample code in ERL command line. > > 10> -module(math1). > ** exception error: undefined shell command module/1 > 11> -export([factorial/1]). > ** exception error: bad argument in an arithmetic expression > in operator '/'/2 > called as factorial / 1 > > > Could someone help me out ? What is wrong ? > > jb > > From rvirding@REDACTED Sat Oct 10 17:12:45 2009 From: rvirding@REDACTED (Robert Virding) Date: Sat, 10 Oct 2009 17:12:45 +0200 Subject: [erlang-questions] Huge binary usage In-Reply-To: <367461.66057.qm@web31803.mail.mud.yahoo.com> References: <98409.67270.qm@web31808.mail.mud.yahoo.com> <3dbc6d1c0910081636l164b2211kea9e086d6c99d597@mail.gmail.com> <367461.66057.qm@web31803.mail.mud.yahoo.com> Message-ID: <3dbc6d1c0910100812j4a65ab8cr4323b44bdbf8dbd6@mail.gmail.com> When a binary is no longer used then that space will be reclaimed. The problem is that since the binary is not local to a process it is more difficult to find out and be sure of when it is no longer in use. This means that the space may not be freed immediately for reuse and in more pathological cases where large binaries are allocated extremely quickly you may run out of space. Robert 2009/10/10 Yogish Baliga > If binary is not used, will that space is reused? Is it possible that the > resident memory will grow to a point where VM will crash because of > out-of-memory error? > > -- baliga > > ------------------------------ > *From:* Robert Virding > *To:* Yogish Baliga > *Cc:* Erlang Questions > *Sent:* Thu, October 8, 2009 4:36:49 PM > *Subject:* Re: [erlang-questions] Huge binary usage > > Large binaries are kept in the normal process heaps but outside of them. > This means that they are not collected in the same way as normal data and > can hang around for a longer time, though they will be collected. > > Robert > > 2009/10/9 Yogish Baliga > > > I have a erlang process which is running at 376MB Resident memory. > > erlang:memory() shows the following output. > > > > [{total,265245056}, > > {processes,56800140}, > > {processes_used,56777884}, > > {system,208444916}, > > {atom,662173}, > > {atom_used,636586}, > > {binary,178220728}, > > {code,4720224}, > > {ets,21754908}] > > > > Seems like most of the memory is used by "binary". > > > > Along with this, I did a sum of total memory used by all the processes > > running on the system. It comes to around 53MB (which is same as what is > > reported in processes_used). > > > > Question is how can I find what process(es) allocated the binaries and > why > > they are not released? > > > > Thanx, > > -- baliga > > > > "Point of view is worth 80 IQ points" --Alan Kay > > > > http://dudefrommangalore.blogspot.com/ > From francesco@REDACTED Sat Oct 10 17:37:49 2009 From: francesco@REDACTED (Francesco Cesarini (Erlang Training and Consulting)) Date: Sat, 10 Oct 2009 16:37:49 +0100 Subject: 2009 Erlang User Conference Update (Pls read if you are thinking of attending) Message-ID: <4AD0AA4D.9000107@erlang-consulting.com> Hi All, I wanted to let you have a quick update on the Erlang User Conference in case you are thinking of attending. Attendance is almost 50% over last year, with well over 200 attendees registered. We will be meeting with the owners of the Astoria Cinema in Stockholm next week to see if we can rent (and fit in) more chairs and increase the number of places. Until we have a definitive answer, we will have to close the registration as soon as we reach 220 delegates and open the standby list. If you know you want to attend, register ASAP. Early bird registration also ends next week (October 15th), and prices will go up. To register: http://www.erlang-factory.com/conference/ErlangUserConference2009/register There was an overwhelming number of talk submissions, and the program is now almost complete (One last talk will be added on Monday). You can see more about the programme here: http://www.erlang-factory.com/conference/ErlangUserConference2009/talks On the Friday after the conference, there will be two events. The first is a Hackathon hosted by Kreditor. The second is a set of tutorial on Erlang and Test Driven Development given by the tool inventors and implementors. We will be opening the registrations for them next Friday. Conference attendance for either is not mandatory. You are welcome to either, free of charge, even if you do not attend the conference. More information on the tutorials is available here: http://www.erlang-factory.com/conference/testingtutorialworkshop/talks Sponsors this year have been many and plentiful! A BIG thanks goes to Tail-f, QuviQ, Process-One, Klarna (Formerly known as Kreditor), Corelatus, Ericsson, Synapse, Mobile Arts Erlang Training & Consulting and Ericsson for their generosity. If anyone else wants to give back to the Erlang community and help sponsor the event allowing us to keep the costs as low as possible, get in contact with either me or Frank, the conference manager (frank @ erlang-consulting.com) You can help us by blogging, tweeting and talking about the conference. There is also an event on facebook you can register for. Get the word out. If you do, let us know and we will add your blog to the community page: http://www.erlang-factory.com/conference/ErlangUserConference2009/community I will be attending and I will be speaking banners will be available on Monday. If you have any questions or thoughts, feel free to drop me or anyone else involved a line. Looking forward to another great EUC in Stockholm this year! Hope to see you there. Francesco -- http://www.erlang-consulting.com From paul-trapexit@REDACTED Sat Oct 10 18:36:43 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Sat, 10 Oct 2009 09:36:43 -0700 (PDT) Subject: [erlang-questions] Passing Filehandles Between Processes In-Reply-To: <22AEC80E-1B17-4F92-8A25-524611F45ABA@souja.net> References: <39776.1255124744@snookles.snookles.com> <22AEC80E-1B17-4F92-8A25-524611F45ABA@souja.net> Message-ID: I had to do this as a hack once. I grabbed the port out of the record and called erlang:port_connect/2 on it. ------ #file_descriptor { module = prim_file, data = { Port, _ } } = Device, try erlang:port_connect (Port, Pid), ok catch error : badarg -> { error, badarg } end. ------ -- p On Sat, 10 Oct 2009, Jayson Vantuyl wrote: > Are there any plans to add something like gen_tcp:controlling_process > for files? I don't see any easy way to do it, looking at file.erl > (and gen_tcp hands it off to some magic in inet). > > On Oct 9, 2009, at 4:45 PM, Scott Lystig Fritchie wrote: > > > Davide Marqu?s wrote: > > > > dm> - finally, *the process that opened the file must be alive for > > dm> read/write operations to succeed*. > > > > This behavior is pretty clearly documented in the R13B02's reference > > for > > the file:open/2 function. The process linking (for non-raw file > > handles) or port linking (for raw file handles) is an important way to > > avoid leaking file handles when processes crash. It doesn't prevent > > leaks if a process lives forever but never closes a file handle (or > > socket, for that matter), but 9/10ths of a pastrami sandwich is better > > than no sandwich. :-) > > > > -Scott > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > -- > Jayson Vantuyl > kagato@REDACTED > > > > > > From roberto.aloi@REDACTED Sat Oct 10 19:14:55 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Sat, 10 Oct 2009 18:14:55 +0100 Subject: [erlang-questions] newbie needing help getting started In-Reply-To: <27c25bc40910100657l6afe61aehe6849ea58cdb6c8d@mail.gmail.com> References: <27c25bc40910100645t2e2b4d81yfbbc85741723bb98@mail.gmail.com> <9b08084c0910100651j247116edicfe782501137d42e@mail.gmail.com> <27c25bc40910100657l6afe61aehe6849ea58cdb6c8d@mail.gmail.com> Message-ID: <634A53E4-FBDB-46E8-9771-26811E2F4DA6@erlang-consulting.com> Just use a dot instead of a semicolon at the end of your commands. Roberto Aloi http://erlang-consulting.com On 10 Oct 2009, at 14:57, Juan Backson wrote: > Hi > > Thank you so much for your help. I managed to compile the sample > code, but > I can't run it: > > [root@REDACTED erlang]# cat math1.erl > -module(math1). > -export([factorial/1]). > factorial(0) -> 1; > factorial(N) -> N * factorial(N-1). > > [root@REDACTED erlang]# ls > math1.beam math1.erl > [root@REDACTED erlang]# erl > Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.7.3 (abort with ^G) > 1> math1:factorial(0) > 1> ; > 1> math1:factorial(0); > > > What am I missing? > > jb > > On Sat, Oct 10, 2009 at 9:51 PM, Joe Armstrong > wrote: > >> You cannot type -module(math1). directly into the shell. >> make a file (math.erl) containing the lines "-module(math1). ..." >> etc. >> then compile the module in the shell like this: >> >>> c(math1). >> >> This will compile the code in math1.erl, then you can run the >> functions in the shell >>> math1:funcname(...) etc. >> >> /Joe >> >> >> On Sat, Oct 10, 2009 at 3:45 PM, Juan Backson >> wrote: >>> Hi, >>> >>> I am following the tutorial trying to get started. I am getting >>> error >> with >>> the following sample code in ERL command line. >>> >>> 10> -module(math1). >>> ** exception error: undefined shell command module/1 >>> 11> -export([factorial/1]). >>> ** exception error: bad argument in an arithmetic expression >>> in operator '/'/2 >>> called as factorial / 1 >>> >>> >>> Could someone help me out ? What is wrong ? >>> >>> jb >>> >> From flw@REDACTED Sat Oct 10 21:49:09 2009 From: flw@REDACTED (Florian Waas) Date: Sat, 10 Oct 2009 12:49:09 -0700 Subject: Job opportunities at Greenplum In-Reply-To: Message-ID: Interesting job opportunities for Erlang developers: http://www.jobvite.com/Jobvite/Job.aspx?m=nRxYkfwq&j=oageVfwj Best regards, -fl. ----- Building the world's largest Data Warehouses at true petabyte-scale, Greenplum is committed to pioneering massively parallel data-intensive analytic processing. We are looking for talented individuals to join our team and explore and develop advanced technology as part of our Distributed Systems Engineering group. Greenplum Database is a massively parallel and distributed database system consisting of clusters of database servers. Coordination of independent systems, effective fault detection, and recovery strategies are just some of the areas that are critical for meeting requirements for high-availability and scalability for which our team is responsible. With the Enterprise Data Cloud Initiative Greenplum enables dynamic (re-)configuration of clusters, data-mart consolidation, and enterprise-wide data sharing pushing the envelope of scalability and availability for data management even further. Responsibilities: * participate in the research and design for state-of-the-art fault-tolerant management frameworks * own the design process, implementation, and verification of framework components leveraging standard software engineering methodologies * maintain and improve existing infrastructure components Requirements: * a passion for building great software extensive experience in developing highly scalable distributed systems * in-depth understanding of High Availability concepts * strong system development skills * good knowledge of parallel programming paradigms * advanced degree in CS, or equivalent * solid coding skills in Erlang/OTP * team player; capable of conducting independent research * result-driven, self-motivated, self-starter * not afraid to take on hard technical challenges * excellent communication skills Experience in any of the following fields is a definite plus: * database internals * experience with PostgreSQL/Greenplum Database * enterprise software development Greenplum is setting the pace in the database/datawarehousing space. We are growing rapidly and providing solutions to major companies in the industry. From erlang@REDACTED Sat Oct 10 21:59:19 2009 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 10 Oct 2009 21:59:19 +0200 Subject: [erlang-questions] newbie needing help getting started In-Reply-To: <27c25bc40910100657l6afe61aehe6849ea58cdb6c8d@mail.gmail.com> References: <27c25bc40910100645t2e2b4d81yfbbc85741723bb98@mail.gmail.com> <9b08084c0910100651j247116edicfe782501137d42e@mail.gmail.com> <27c25bc40910100657l6afe61aehe6849ea58cdb6c8d@mail.gmail.com> Message-ID: <9b08084c0910101259s625fd639nffac3a7e2c4df388@mail.gmail.com> I'm puzzled by your questions - you souldn't be having these kind of problems. You say "I'm following THE tutorial" - but there are several - which tutorial are you following? There are several tutorials on the net - if you type in *exactly* what they say from the beginning you shouldn't have the kind of problems you are running into. It would be helpful to know which so the author can fix the text to prevent the kind of problem you are experiencing. Two good places to start are http://www.erlang.org/download/getting_started-5.4.pdf http://learnyousomeerlang.com/starting-out#the-shell It's important not to "jump in at the middle" - because the stuff like "end every command with dot whitespace" is usually explained near the start of the tutorial and is easily missed. Have fun and don't give up quietly if you get stuck - ask the list! /Joe On Sat, Oct 10, 2009 at 3:57 PM, Juan Backson wrote: > Hi > > Thank you so much for your help.? I managed to compile the sample code, but > I can't run it: > > [root@REDACTED erlang]# cat math1.erl > -module(math1). > -export([factorial/1]). > factorial(0) -> 1; > factorial(N) -> N * factorial(N-1). > > [root@REDACTED erlang]# ls > math1.beam? math1.erl > [root@REDACTED erlang]# erl > Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.7.3? (abort with ^G) > 1> math1:factorial(0) > 1> ; > 1> math1:factorial(0); > > > What am I missing? > > jb > > On Sat, Oct 10, 2009 at 9:51 PM, Joe Armstrong wrote: >> >> You cannot type -module(math1). directly into the shell. >> make a file (math.erl) containing the lines "-module(math1). ..." etc. >> then compile the module in the shell like this: >> >> > c(math1). >> >> This will compile the code in math1.erl, then you can run the >> functions in the shell >> > math1:funcname(...) etc. >> >> /Joe >> >> >> On Sat, Oct 10, 2009 at 3:45 PM, Juan Backson >> wrote: >> > Hi, >> > >> > I am following the tutorial trying to get started. ?I am getting error >> > with >> > the following sample code in ERL command line. >> > >> > 10> -module(math1). >> > ** exception error: undefined shell command module/1 >> > 11> -export([factorial/1]). >> > ** exception error: bad argument in an arithmetic expression >> > ? ? in operator ?'/'/2 >> > ? ? ? ?called as factorial / 1 >> > >> > >> > Could someone help me out ? ?What is wrong ? >> > >> > jb >> > > > From flw@REDACTED Sun Oct 11 06:13:51 2009 From: flw@REDACTED (Florian Waas) Date: Sat, 10 Oct 2009 21:13:51 -0700 Subject: [erlang-questions] Job opportunities at Greenplum In-Reply-To: Message-ID: Correct link: http://www.greenplum.com/about-us/careers/ -fl. On 10/10/09 12:49 PM, "Florian Waas" wrote: Interesting job opportunities for Erlang developers: http://www.jobvite.com/Jobvite/Job.aspx?m=nRxYkfwq&j=oageVfw Best regards, -fl. ----- Building the world's largest Data Warehouses at true petabyte-scale, Greenplum is committed to pioneering massively parallel data-intensive analytic processing. We are looking for talented individuals to join our team and explore and develop advanced technology as part of our Distributed Systems Engineering group. Greenplum Database is a massively parallel and distributed database system consisting of clusters of database servers. Coordination of independent systems, effective fault detection, and recovery strategies are just some of the areas that are critical for meeting requirements for high-availability and scalability for which our team is responsible. With the Enterprise Data Cloud Initiative Greenplum enables dynamic (re-)configuration of clusters, data-mart consolidation, and enterprise-wide data sharing pushing the envelope of scalability and availability for data management even further. Responsibilities: * participate in the research and design for state-of-the-art fault-tolerant management frameworks * own the design process, implementation, and verification of framework components leveraging standard software engineering methodologies * maintain and improve existing infrastructure components Requirements: * a passion for building great software extensive experience in developing highly scalable distributed systems * in-depth understanding of High Availability concepts * strong system development skills * good knowledge of parallel programming paradigms * advanced degree in CS, or equivalent * solid coding skills in Erlang/OTP * team player; capable of conducting independent research * result-driven, self-motivated, self-starter * not afraid to take on hard technical challenges * excellent communication skills Experience in any of the following fields is a definite plus: * database internals * experience with PostgreSQL/Greenplum Database * enterprise software development Greenplum is setting the pace in the database/datawarehousing space. We are growing rapidly and providing solutions to major companies in the industry. From caluml@REDACTED Sun Oct 11 23:55:11 2009 From: caluml@REDACTED (Calum) Date: Sun, 11 Oct 2009 22:55:11 +0100 Subject: A less trusting Erlang. Message-ID: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> Hello all, I am an Erlang beginner, although I've been dipping in and out of it for a while. I'd like to use Erlang, and the built-in distributed meshing effect of it to build a sort of IRC-plus distributed storage/messaging system platform, of which each node would run the Erlang app. However, I'd like to use avoid using lib_chan if possible, and just use the native Erlang connectivity/distribution mechanisms. The big problem then is that Erlang is way too trusting for connecting to random people in the default if-you-know-my-cookie-you-can-do-whatever-you-like mode. The obvious thing (to me anyway) would be to limit the calls that remote nodes can make - Erlang Untrusting, if you will, or Paranoid Erlang. Would it be possible to patch the core libraries that "accept/receive" the spawn requests, and check them against a list of "allowed" functions. For example, if erl was run with the -paranoid parameter, it would only allow calls which are listed in .erlang.allowed to be made from remote nodes. Such a file might look like public:*, mystuff:another/1, etc etc. Would it require patching and recompiling of the source to Erlang, or could the "receive" function be "overwritten" from the shell/programatically? Is this approach one that could work? Are there sufficiently few places that could be restricted to make this solution secure? How much of a patch would this be, or are there too many places to be able to secure it with any degree of confidence? And even if these restrictions were in place, would it provide a high-enough level of security to prevent ne'er-do-wells doing things like drain resources, insert code somehow, etc? Or would it be too hard to prevent people passing in Funs, or doing other unexpected things? Hoping some Erlang heavyweights can provide some thoughts onto this approach, and give some feedback on this idea. C PS. Apologies if this functionality already exists somewhere. From jim.mccoy@REDACTED Mon Oct 12 06:21:51 2009 From: jim.mccoy@REDACTED (Jim McCoy) Date: Sun, 11 Oct 2009 21:21:51 -0700 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> Message-ID: The short answer is that you can't effectively prevent a node in a distributed erlang setup from doing just about anything it wants to another node. It just doesn't work that way. There was some effort a while back to try to deal with these issues (see the SSErl work and a long thread on the list from the summer of 2006 relating to E/Erlang integration) but nothing concrete has come of it. You are better off using a well-defined protocol at the trust boundaries and not trying to let untrusted users make calls directly into the trusted side of this boundary. jim On Sun, Oct 11, 2009 at 2:55 PM, Calum wrote: > [...] > I'd like to use Erlang, and the built-in distributed meshing effect of > it to build a sort of IRC-plus distributed storage/messaging system > platform, of which each node would run the Erlang app. > However, I'd like to use avoid using lib_chan if possible, and just > use the native Erlang connectivity/distribution mechanisms. > > The big problem then is that Erlang is way too trusting for connecting > to random people in the default > if-you-know-my-cookie-you-can-do-whatever-you-like mode. > [...] From colm.dougan@REDACTED Mon Oct 12 12:09:25 2009 From: colm.dougan@REDACTED (Colm Dougan) Date: Mon, 12 Oct 2009 11:09:25 +0100 Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: References: <21B2D38B-9831-4B9B-99A8-8B2771CE0BCA@cs.otago.ac.nz> Message-ID: <24d4f39c0910120309u1bd700a5j863454ce4493fa87@mail.gmail.com> On Fri, Oct 9, 2009 at 7:43 AM, Max Lapshin wrote: > It seems, that I require the same functionality: list of frames is > stored in memory and when user requests frame number N, > I need the first keyframe, left to requested. > > They are stored in ets ordered set table (it becomes rather large), so > I take all keyframes, left to requested and select first from them. > Very, very non-optimal =( If by "frame number N" you mean "the Nth object in the ordered set" then you can use ets:slot for this Colm From max.lapshin@REDACTED Mon Oct 12 12:26:55 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 12 Oct 2009 14:26:55 +0400 Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: <24d4f39c0910120309u1bd700a5j863454ce4493fa87@mail.gmail.com> References: <21B2D38B-9831-4B9B-99A8-8B2771CE0BCA@cs.otago.ac.nz> <24d4f39c0910120309u1bd700a5j863454ce4493fa87@mail.gmail.com> Message-ID: > If by "frame number N" you mean "the Nth object in the ordered set" > then you can use ets:slot for this > > Colm No,no, I was not too clear: I store milliseconds as id of frame and need something like following expression: seek(Time) -> "SELECT id FROM frames WHERE is_keyframe = 't' AND timestamp <= Time ORDER BY id DESC limit 1" Something like this. Binary tree will be ideal. From kamiseq@REDACTED Mon Oct 12 12:28:30 2009 From: kamiseq@REDACTED (kamiseq) Date: Mon, 12 Oct 2009 12:28:30 +0200 Subject: linking modules from other directories Message-ID: is there a way to direct erlang that specified module is in another directory so it can use it when compiling. let say my project is like proj /lib/ /inc /header.hrl /src /foo /efoo.erl (where module is defined as -module(foo.efoo) ) /baz /ebaz.erl (where module is defined as -module(baz.ebaz) ) /main.erl /other.erl util /util.erl so now, how to reference util.erl from main.erl. I havent tried that but it seems that I may reference efoo and ebaz from main.erl using package notation (ie foo.efoo:countTo5() ), but there is no way to reference other.erl from efoo.erl or ebaz.erl. OR should I just include file (like with header files) and then use it normally? util:fun()??? but it will include the other file in my module and all I want is point compiler where the module is. the other thing is that compiler can't catch calls to missing modules (in other dirs) it will throw {'module could not be loaded' only in runtime.......... is there something like class path for erlang?? -- pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From kamiseq@REDACTED Mon Oct 12 13:37:10 2009 From: kamiseq@REDACTED (kamiseq) Date: Mon, 12 Oct 2009 13:37:10 +0200 Subject: [erlang-questions] linking modules from other directories In-Reply-To: References: Message-ID: > > I'd recommend you make directory "proj/deps" and make symlink > "proj/deps/util". > ok and what is *symlink* ?? Then you can setup compilation options to compile "proj/deps/util/*" to > "proj/ebin". > > can you give me some small ex - I cant find anything useful on ggl. but then all my files will go into ebin dir?? -- pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From taavi@REDACTED Mon Oct 12 13:45:16 2009 From: taavi@REDACTED (Taavi Talvik) Date: Mon, 12 Oct 2009 14:45:16 +0300 Subject: [erlang-questions] linking modules from other directories In-Reply-To: References: Message-ID: <0458B01A-5410-4CB1-9BEA-604F511AE3DE@uninet.ee> On Oct 12, 2009, at 2:37 PM, kamiseq wrote: >> >> I'd recommend you make directory "proj/deps" and make symlink >> "proj/deps/util". >> > > ok and what is *symlink* ?? > > Then you can setup compilation options to compile "proj/deps/util/*" > to >> "proj/ebin". >> >> can you give me some small ex - I cant find anything useful on ggl. > > but then all my files will go into ebin dir?? man code(3) or http://www.erlang.org/doc/man/code.html At application start-up phase you can add directories to code server search path with code:add_path/1, code:add_patha/1 and similari function from code server. best regards, taavi From tamas.nagy@REDACTED Mon Oct 12 13:47:16 2009 From: tamas.nagy@REDACTED (Tamas Nagy) Date: Mon, 12 Oct 2009 12:47:16 +0100 Subject: [erlang-questions] linking modules from other directories In-Reply-To: References: Message-ID: Hi, The package application is unsupported in Erlang. That being said calls to other modules do not create compile time dependency so I would assume if all the modules are available in runtime system util:foo(..) will be fine. I would recommend reading up on what erlang deployment's usual layout is as the tools/system are implemented with that layout in mind.(http://www.erlang.org/doc/system_principles/part_frame.html ) This does not mean that you cannot create your own layout but it will be harder to make it work. Erlang has a code path which can be modified with code:add_patha/1, code:add_pathz/1 etc. or with the -pa, -pz switches given to the erl command etc. (http://www.erlang.org/doc/man/code.html) Regards, Tamas Tamas Nagy Erlang Training & Consulting http://www.erlang-consulting.com On 12 Oct 2009, at 11:28, kamiseq wrote: > is there a way to direct erlang that specified module is in another > directory so it can use it when compiling. > > let say my project is like > > proj > /lib/ > /inc > /header.hrl > /src > /foo > /efoo.erl (where module is defined as -module > (foo.efoo) ) > /baz > /ebaz.erl (where module is defined as -module > (baz.ebaz) ) > /main.erl > /other.erl > util > /util.erl > > so now, how to reference util.erl from main.erl. > I havent tried that but it seems that I may reference efoo and ebaz > from > main.erl using package notation (ie foo.efoo:countTo5() ), but there > is no > way to reference other.erl from efoo.erl or ebaz.erl. > > OR should I just include file (like with header files) and then use it > normally? util:fun()??? but it will include the other file in my > module and > all I want is point compiler where the module is. > > the other thing is that compiler can't catch calls to missing > modules (in > other dirs) it will throw {'module could not be loaded' only in > runtime.......... > > is there something like class path for erlang?? > -- > > pozdrawiam > Pawe? Kami?ski > > kamiseq@REDACTED > pkaminski.prv@REDACTED > ______________________ From D.Belyaev@REDACTED Mon Oct 12 13:49:46 2009 From: D.Belyaev@REDACTED (=?utf-8?B?0JHQtdC70Y/QtdCyINCU0LzQuNGC0YDQuNC5?=) Date: Mon, 12 Oct 2009 15:49:46 +0400 Subject: [erlang-questions] linking modules from other directories In-Reply-To: References: Message-ID: http://en.wikipedia.org/wiki/Symbolic_link This helps to one library to be used in different projects instead of copying it. Often I create ?proj/Emakefile? with something like this: {"src/*", [debug_info, {outdir, "ebin"}]}. {"deps/*/src/*", [debug_info, {outdir, "ebin"}]}. I start shell with current directory in project root. Then I have compile project with ?erl ?make? and start application with command like ?erl ?pa ebin?. Generally I have all erl files located in src without any subfolders. Dmitry From: kamiseq [mailto:kamiseq@REDACTED] Sent: Monday, October 12, 2009 3:37 PM Subject: Re: [erlang-questions] linking modules from other directories I'd recommend you make directory "proj/deps" and make symlink "proj/deps/util". ok and what is symlink ?? Then you can setup compilation options to compile "proj/deps/util/*" to "proj/ebin". can you give me some small ex - I cant find anything useful on ggl. but then all my files will go into ebin dir?? -- pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From mevans@REDACTED Mon Oct 12 17:31:25 2009 From: mevans@REDACTED (Evans, Matthew) Date: Mon, 12 Oct 2009 11:31:25 -0400 Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: References: <21B2D38B-9831-4B9B-99A8-8B2771CE0BCA@cs.otago.ac.nz> <24d4f39c0910120309u1bd700a5j863454ce4493fa87@mail.gmail.com> Message-ID: I faced a similar problem where I needed to search a database with a random index value, and have it return the "closest match". I initially used an mnesia table and used a mnesia:select with a guard to perform a query of the type: Match = #my_data{key = '$1', index = '$2'}, Guard = [{'>= ','$2',SomeValue}], Result = ['$1','$2'], mnesia:select(my_data,[{Match,Guard,[Result]}]), I soon realized that as my data set grew in size that the above method would be very inefficient. Instead I created a secondary index implemented using gb_trees. I would query this secondary index using a random value, and have the result of that value being an exact match to "index". To do this I had to modify the gb_tree module, with a new function that lets it return the "closest match" (i.e. as it iterates over the tree it will return the previous, or next node if it fails to find a match). Maybe you could do the same in your project, but have it return an iterator instead? Matt -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Max Lapshin Sent: Monday, October 12, 2009 6:27 AM To: Colm Dougan Cc: erlang-questions Subject: Re: [erlang-questions] Ordered set with lower_bound > If by "frame number N" you mean "the Nth object in the ordered set" > then you can use ets:slot for this > > Colm No,no, I was not too clear: I store milliseconds as id of frame and need something like following expression: seek(Time) -> "SELECT id FROM frames WHERE is_keyframe = 't' AND timestamp <= Time ORDER BY id DESC limit 1" Something like this. Binary tree will be ideal. ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From wenewboy@REDACTED Mon Oct 12 17:48:07 2009 From: wenewboy@REDACTED (wenew zhang) Date: Mon, 12 Oct 2009 08:48:07 -0700 Subject: [erlang-questions] MacOSX 10.5, odbc, MySQL Message-ID: <6E358551-C5A9-4207-94CD-7D76D358C72F@gmail.com> why no body answers this question? it's a bug or what's wrong with it? maybe it's very simple and i realy confused! http://erlang.org/pipermail/erlang-questions/2009-March/042389.html Eshell V5.7.3 (abort with ^G) 1> odbc:start(). ok 2> odbc:connect("";[]). * 1: syntax error before: ';' 2> odbc:connect("",[]). =ERROR REPORT==== 12-Oct-2009::07:26:45 === ODBC: received unexpected info: {tcp_closed,#Port<0.565>} =ERROR REPORT==== 12-Oct-2009::07:26:45 === ** Generic server <0.39.0> terminating ** Last message in was {#Port<0.563>,{exit_status,138}} ** When Server state == {state,#Port<0.563>, {<0.31.0>,#Ref<0.0.0.39>}, <0.31.0>,undefined,on,undefined,undefined,on, connecting,undefined,0, [#Port<0.561>,#Port<0.562>], #Port<0.564>,#Port<0.565>} ** Reason for termination == ** {port_exit,killed} {error,connection_closed} 3> From john.erickson@REDACTED Mon Oct 12 19:04:01 2009 From: john.erickson@REDACTED (Erickson, John) Date: Mon, 12 Oct 2009 11:04:01 -0600 Subject: License for example code in docs? Message-ID: <3C44D46958109E408F5848593B05A1D8C620CB3C@rrsmsx505.amr.corp.intel.com> Hello, I was wondering if there is any license on the code snippets in the Erlang documentation. Are these public domain, or do they fall under the EPL? I was unable to find any license info in the documentation. From andrew@REDACTED Mon Oct 12 19:47:18 2009 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 12 Oct 2009 13:47:18 -0400 Subject: [erlang-questions] MacOSX 10.5, odbc, MySQL In-Reply-To: <6E358551-C5A9-4207-94CD-7D76D358C72F@gmail.com> References: <6E358551-C5A9-4207-94CD-7D76D358C72F@gmail.com> Message-ID: <20091012174717.GG8073@hijacked.us> On Mon, Oct 12, 2009 at 08:48:07AM -0700, wenew zhang wrote: > why no body answers this question? > it's a bug or what's wrong with it? > maybe it's very simple and i realy confused! > > > http://erlang.org/pipermail/erlang-questions/2009-March/042389.html > > Eshell V5.7.3 (abort with ^G) > 1> odbc:start(). > ok > 2> odbc:connect("";[]). > * 1: syntax error before: ';' > 2> odbc:connect("",[]). > =ERROR REPORT==== 12-Oct-2009::07:26:45 === > ODBC: received unexpected info: {tcp_closed,#Port<0.565>} > =ERROR REPORT==== 12-Oct-2009::07:26:45 === > ** Generic server <0.39.0> terminating > ** Last message in was {#Port<0.563>,{exit_status,138}} > ** When Server state == {state,#Port<0.563>, > {<0.31.0>,#Ref<0.0.0.39>}, > > <0.31.0>,undefined,on,undefined,undefined,on, > connecting,undefined,0, > [#Port<0.561>,#Port<0.562>], > #Port<0.564>,#Port<0.565>} > ** Reason for termination == > ** {port_exit,killed} > {error,connection_closed} > 3> The first argument to odbc:connect has to be an ODBC DSN, it can't just be a blank string. Examples of DSNs can be found in the odbc user's guide. Have you configured ODBC at all? Andrew From paul-trapexit@REDACTED Mon Oct 12 20:23:56 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Mon, 12 Oct 2009 11:23:56 -0700 (PDT) Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: References: <21B2D38B-9831-4B9B-99A8-8B2771CE0BCA@cs.otago.ac.nz> <24d4f39c0910120309u1bd700a5j863454ce4493fa87@mail.gmail.com> Message-ID: On Mon, 12 Oct 2009, Paul Mineiro wrote: > On Mon, 12 Oct 2009, Max Lapshin wrote: > > > > If by "frame number N" you mean "the Nth object in the ordered set" > > > then you can use ets:slot for this > > > > > > Colm > > > > No,no, I was not too clear: I store milliseconds as id of frame and > > need something like following expression: > > > > seek(Time) -> > > "SELECT id FROM frames WHERE is_keyframe = 't' AND timestamp <= Time > > ORDER BY id DESC limit 1" > > > > Something like this. Binary tree will be ideal. > > If you had an ets ordered_set table with tuple primary key { is_keyframe, > timestamp } then you could call ets:lookup (Tab, { 't', Time }) to catch > the '=' case, and then ets:prev (Tab, { 't', Time }) for the '<' case. > You'd have to confirm the correct is_keyframe in the return value when > using ets:prev/2. > > If this primary key is not convenient, create a second table with entries > #secondary{ is_keyframe_and_timestamp :: { bool (), number () }, > original_primary_key :: any () } > and then do the join yourself. > > -- p > missed the "ORDER BY id DESC" part, but you can do that by adding id to the tuple at the end. another wrinkle, ets uses erlang term order, so for instance if id is integral and you want to probe with ets:prev/2 you should pick an id whose type is not actually used but which will cause it to sort above in erlang term order, e.g., the empty atom will sort higher than any integer. happily you can skip the ets:lookup/2 call in this case, but you'll have to verify that is_keyframe and timestamp are correct in the return value of ets:prev/2. -- p From paul-trapexit@REDACTED Mon Oct 12 20:36:37 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Mon, 12 Oct 2009 11:36:37 -0700 (PDT) Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: References: <21B2D38B-9831-4B9B-99A8-8B2771CE0BCA@cs.otago.ac.nz> <24d4f39c0910120309u1bd700a5j863454ce4493fa87@mail.gmail.com> Message-ID: On Mon, 12 Oct 2009, Paul Mineiro wrote: > On Mon, 12 Oct 2009, Paul Mineiro wrote: > > > On Mon, 12 Oct 2009, Max Lapshin wrote: > > > > > > If by "frame number N" you mean "the Nth object in the ordered set" > > > > then you can use ets:slot for this > > > > > > > > Colm > > > > > > No,no, I was not too clear: I store milliseconds as id of frame and > > > need something like following expression: > > > > > > seek(Time) -> > > > "SELECT id FROM frames WHERE is_keyframe = 't' AND timestamp <= Time > > > ORDER BY id DESC limit 1" > > > > > > Something like this. Binary tree will be ideal. > > > > If you had an ets ordered_set table with tuple primary key { is_keyframe, > > timestamp } then you could call ets:lookup (Tab, { 't', Time }) to catch > > the '=' case, and then ets:prev (Tab, { 't', Time }) for the '<' case. > > You'd have to confirm the correct is_keyframe in the return value when > > using ets:prev/2. > > > > If this primary key is not convenient, create a second table with entries > > #secondary{ is_keyframe_and_timestamp :: { bool (), number () }, > > original_primary_key :: any () } > > and then do the join yourself. > > > > -- p > > > > missed the "ORDER BY id DESC" part, but you can do that by adding id to > the tuple at the end. > > another wrinkle, ets uses erlang term order, so for instance if id is > integral and you want to probe with ets:prev/2 you should pick an id whose > type is not actually used but which will cause it to sort above in erlang > term order, e.g., the empty atom will sort higher than any integer. > happily you can skip the ets:lookup/2 call in this case, but you'll have > to verify that is_keyframe and timestamp are correct in the return value > of ets:prev/2. > > -- p Another thought: if the frames are being added in temporal order, you could leverage the min-stack trick. http://stackoverflow.com/questions/685060/design-a-stack-such-that-getminimum-should-be-o1 -- p From rob@REDACTED Mon Oct 12 20:34:02 2009 From: rob@REDACTED (Rob Charlton) Date: Mon, 12 Oct 2009 19:34:02 +0100 Subject: Socket connects to itself Message-ID: <4AD3769A.2020104@erlang-consulting.com> Hi, I encountered a bug recently, which I have now fixed, but I wanted to share it in case anyone else encounters it. I'm also curious to see if anyone can give a better explanation of what happens. I have a process, call it P, which uses gen_tcp:connect(Addr,Port, [binary,{packet,4}], 1000) to try and connect to a server on the local machine. Actually there are several of these processes and they try to connect to a list of different servers specified in the application's config file. If the connection times out, or fails for some reason, then an attempt is made to connect again in 1 second using erlang:send_after. Normally the system is configured in such a way that the list of servers matches those that are actually running (or will be running), so eventually all the P processes get connected and all is well. Sometimes, however, the system is incorrectly configured so that one of the servers is listed in the config file but is not actually running. In this case the process P will continually try and connect to it every second. This is where it gets interesting. I noticed in the logs one night that this process P which had tried for several hours to connect to a non-existent server on 127.0.0.1:2920 had actually succeeded! Even more exciting than that, it had connected to itself, so when it started trying to communicate with its peer it was very surprised to receive its own communications back again. Needless to say P crashed at this point. I was very curious to know how this could happen and to try and fix it! You could argue this isn't a bug because the system configuration was incorrectly specified. In this particular case though, the misconfiguration can happen easily and I would like P not to crash but instead to keep trying. I tried Googling for this and did find a couple of non-erlang sources suggesting it had happened to them in similar circumstances. It could be that this is a very timing sensitive 'feature' of TCP/IP. Can anyone enlighten me? I did manage to fix it, by testing whether inet:peername(Socket) =:= inet:sockname(Socket) when I get a connection. Very occasionally this evaluates to true, so in that case I close the socket and try again. Cheers Rob -- Erlang Training and Consulting Ltd www.erlang-consulting.com From caluml@REDACTED Mon Oct 12 20:39:07 2009 From: caluml@REDACTED (Calum) Date: Mon, 12 Oct 2009 19:39:07 +0100 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> Message-ID: <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> On Mon, Oct 12, 2009 at 5:21 AM, Jim McCoy wrote: > The short answer is that you can't (with Erlang as it currently is) > effectively prevent a node in a > distributed erlang setup from doing just about anything it wants to > another node. ?It just doesn't work that way. ?There was some effort a > while back to try to deal with these issues (see the SSErl work and a > long thread on the list from the summer of 2006 relating to E/Erlang > integration) but nothing concrete has come of it. I looked at SSErl, and seems as though it was an internal proof of concept. > You are better off using a well-defined protocol at the trust > boundaries and not trying to let untrusted users make calls directly > into the trusted side of this boundary. Which is all very fine, but then I'd end up duplicating (probably badly) all the nice node discovery, linking, rpc stuff that Erlang has got built in. If Erlang didn't allow anyone to run anything, it'd be perfect - I'm just curious as to whether it's fairly simple to limit it so. In my very simplistic mind *, I'm imagining a recv_spawn function squirrelled away in the source somewhere, which could have a few lines added to check that the function being requested matches a list (perhaps even as simple as being hardcoded to a module starting with the word public). C * I'm sure it's not as easy as this - but I'm wondering if anyone with knowledge of the internals of Erlang could tell me how many thousands of miles off I am. :) From jim.mccoy@REDACTED Mon Oct 12 21:16:07 2009 From: jim.mccoy@REDACTED (Jim McCoy) Date: Mon, 12 Oct 2009 12:16:07 -0700 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> Message-ID: The problem I am seeing with this is that it is not sufficient to just prevent a remote node from running an apply() or calling certain functions, you also have to prevent it from sending messages to places you care about. What happens to your system if my node connects and then starts spraying exit signals around? This is the hard part. In theory a pid() should be a relatively safe unguessable number, in practice it is trivial to guess and there are built-in facilities to make this task even easier. There are at least two parts to making Erlang "internally secure": restricting function calls across trust boundaries (as you have noted) and restricting message sending across these same boundaries. There are probably other components necessary for this task, but those two jump right out as hard problems to solve. jim From max.lapshin@REDACTED Mon Oct 12 21:21:54 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 12 Oct 2009 23:21:54 +0400 Subject: [erlang-questions] Ordered set with lower_bound In-Reply-To: References: <21B2D38B-9831-4B9B-99A8-8B2771CE0BCA@cs.otago.ac.nz> <24d4f39c0910120309u1bd700a5j863454ce4493fa87@mail.gmail.com> Message-ID: Thank You, I'll look at the trick. But I cannot make a direct lookup (=), because user requests Timestamp, that doesn't exist. From discipleofranok@REDACTED Mon Oct 12 21:46:01 2009 From: discipleofranok@REDACTED (Jacob Torrey) Date: Mon, 12 Oct 2009 15:46:01 -0400 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> Message-ID: <4AD38779.3040700@gmail.com> Could perhaps the pid generation in the VM be made more random, therefore you can only gain access to pids that you know the name to? This seems like an obvious (and pretty trivial) solution to obscure and increase the security of the Erlang system. Jacob Jim McCoy wrote: > The problem I am seeing with this is that it is not sufficient to just > prevent a remote node from running an apply() or calling certain > functions, you also have to prevent it from sending messages to places > you care about. What happens to your system if my node connects and > then starts spraying exit signals around? This is the hard part. In > theory a pid() should be a relatively safe unguessable number, in > practice it is trivial to guess and there are built-in facilities to > make this task even easier. There are at least two parts to making > Erlang "internally secure": restricting function calls across trust > boundaries (as you have noted) and restricting message sending across > these same boundaries. There are probably other components necessary > for this task, but those two jump right out as hard problems to solve. > > jim > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From kenneth.lundin@REDACTED Mon Oct 12 21:51:44 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 12 Oct 2009 21:51:44 +0200 Subject: [erlang-questions] License for example code in docs? In-Reply-To: <3C44D46958109E408F5848593B05A1D8C620CB3C@rrsmsx505.amr.corp.intel.com> References: <3C44D46958109E408F5848593B05A1D8C620CB3C@rrsmsx505.amr.corp.intel.com> Message-ID: The documentation is also under the Erlang Public License. You can see that in the license info in the xml source for the documentation (under doc/src in each application) We will try to make the license info more visible in the generated html and pdf as well. /Kenneth, Erlang/OTP Ericsson On Mon, Oct 12, 2009 at 7:04 PM, Erickson, John wrote: > Hello, I was wondering if there is any license on the code snippets in the Erlang documentation. ?Are these public domain, or do they fall under the EPL? ?I was unable to find any license info in the documentation. > > From rvirding@REDACTED Mon Oct 12 22:07:54 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 12 Oct 2009 22:07:54 +0200 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> Message-ID: <3dbc6d1c0910121307s7177b7d2m34016d74f9c28af8@mail.gmail.com> The trouble is is that there are no "special" processes in the system as such. Messages from the outside will be routed to a normal Erlang process which will then decide whether to ignore it or or treat it as a command and execute it. So by only exposing certain processes which are "safe" then your system is "safe". How do you mean recv_spawn? There is nothing builtin to the emulator which somehow takes a message and spawns a new process to handle it, it all done in Erlang. I can't remember where it all handled but I would start by checking out the module net_kernel which must be called to start networking. >From there you can probably work out how to restrict external access. Robert 2009/10/12 Calum > On Mon, Oct 12, 2009 at 5:21 AM, Jim McCoy wrote: > > The short answer is that you can't > > (with Erlang as it currently is) > > > effectively prevent a node in a > > distributed erlang setup from doing just about anything it wants to > > another node. It just doesn't work that way. There was some effort a > > while back to try to deal with these issues (see the SSErl work and a > > long thread on the list from the summer of 2006 relating to E/Erlang > > integration) but nothing concrete has come of it. > > I looked at SSErl, and seems as though it was an internal proof of concept. > > > You are better off using a well-defined protocol at the trust > > boundaries and not trying to let untrusted users make calls directly > > into the trusted side of this boundary. > > Which is all very fine, but then I'd end up duplicating (probably > badly) all the nice node discovery, linking, rpc stuff that Erlang has > got built in. > If Erlang didn't allow anyone to run anything, it'd be perfect - I'm > just curious as to whether it's fairly simple to limit it so. > > In my very simplistic mind *, I'm imagining a recv_spawn function > squirrelled away in the source somewhere, which could have a few lines > added to check that the function being requested matches a list > (perhaps even as simple as being hardcoded to a module starting with > the word public). > > > C > > * I'm sure it's not as easy as this - but I'm wondering if anyone with > knowledge of the internals of Erlang could tell me how many thousands > of miles off I am. :) > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From raould@REDACTED Mon Oct 12 23:12:39 2009 From: raould@REDACTED (Raoul Duke) Date: Mon, 12 Oct 2009 14:12:39 -0700 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <4AD38779.3040700@gmail.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> <4AD38779.3040700@gmail.com> Message-ID: <91a2ba3e0910121412h5b64fb99wcd73a65aed6070dc@mail.gmail.com> On Mon, Oct 12, 2009 at 12:46 PM, Jacob Torrey wrote: > Could perhaps the pid generation in the VM be made more random, > therefore you can only gain access to pids that you know the name to? > This seems like an obvious (and pretty trivial) solution to obscure and > increase the security of the Erlang system. swiss numbers? http://www.mail-archive.com/cryptography@REDACTED/msg02711.html From caluml@REDACTED Mon Oct 12 23:39:07 2009 From: caluml@REDACTED (Calum) Date: Mon, 12 Oct 2009 22:39:07 +0100 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <3dbc6d1c0910121307s7177b7d2m34016d74f9c28af8@mail.gmail.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> <3dbc6d1c0910121307s7177b7d2m34016d74f9c28af8@mail.gmail.com> Message-ID: <635498b70910121439v7c6adbbcs9291124157a524c3@mail.gmail.com> On Mon, Oct 12, 2009 at 9:07 PM, Robert Virding wrote: > How do you mean recv_spawn? There is nothing builtin to the emulator which > somehow takes a message and spawns a new process to handle it, it all done > in Erlang. I was thinking of the code that gets run on the remote node when you do something like 1> spawn('two@REDACTED', c, ls, []). or the proverbial 2> rpc:multicall(nodes(), os, cmd, ["rm / -rf"]). Something on the second node must receive that bunch of args - just a small check at that point then to make sure the second arg is "public" or similar - if not, it could then just refuse to pass that message on to the rest of the code that actually runs it and returns the result. Net result - in "paranoid mode" - you'd have to consciously expose any public methods in a module called public. From per@REDACTED Tue Oct 13 00:12:05 2009 From: per@REDACTED (Per Hedeland) Date: Tue, 13 Oct 2009 00:12:05 +0200 (CEST) Subject: [erlang-questions] Socket connects to itself In-Reply-To: <4AD3769A.2020104@erlang-consulting.com> Message-ID: <200910122212.n9CMC5pG080185@pluto.hedeland.org> Rob Charlton wrote: > >I noticed in the logs one night that this process P which had tried for >several hours to connect to a non-existent server on 127.0.0.1:2920 had >actually succeeded! Even more exciting than that, it had connected to >itself, so when it started trying to communicate with its peer it was >very surprised to receive its own communications back again. Needless to >say P crashed at this point. I was very curious to know how this could >happen and to try and fix it! You could argue this isn't a bug because >the system configuration was incorrectly specified. In this particular >case though, the misconfiguration can happen easily and I would like P >not to crash but instead to keep trying. > >I tried Googling for this and did find a couple of non-erlang sources >suggesting it had happened to them in similar circumstances. It could be >that this is a very timing sensitive 'feature' of TCP/IP. Can anyone >enlighten me? That's a nice one!:-) Whenever you call gen_tcp:connect(), the underlying OS connect() system call will first bind the socket to a local address and port, and then send the initial SYN packet using that address/port as source. It is quite possible that at some point it may happen to choose the unused port 2920 for that, and hence attempt a connect from 127.0.0.1:2920 to 127.0.0.1:2920. This is where the "feature" comes into play - you might think that this connection should fail, since there is no process listening on port 2920, but the TCP spec explicitly prescribes that "simultaneous initiation" should be handled - e.g. from section 2.7 of RFC 793: Two processes which issue active OPENs to each other at the same time will be correctly connected. This flexibility is critical for the support of distributed computing in which components act asynchronously with respect to each other. You can also see it in the famous state diagram in Figure 6 of RFC 793, as the "rcv SYN" transition from SYN_SENT to SYN_RCVD. Though of course it's a pretty safe bet that no protocol in serious use actually requires this "critical" functionality. --Per Hedeland From ulf.wiger@REDACTED Tue Oct 13 08:01:28 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 13 Oct 2009 08:01:28 +0200 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <4AD38779.3040700@gmail.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> <4AD38779.3040700@gmail.com> Message-ID: <4AD417B8.8000702@erlang-consulting.com> Jacob Torrey wrote: > Could perhaps the pid generation in the VM be made more random, > therefore you can only gain access to pids that you know the name to? > This seems like an obvious (and pretty trivial) solution to obscure and > increase the security of the Erlang system. You don't even need to bother with the pids: {rex,Node} ! {'$gen_cast',{cast,M,F,A,user}} ...will allow you to execute any function on Node. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From juliadotter@REDACTED Tue Oct 13 08:01:58 2009 From: juliadotter@REDACTED (Nina Alex Juliadotter) Date: Tue, 13 Oct 2009 17:01:58 +1100 Subject: [erlang-questions] PDF reports from Mnesia: Erlguten and XMerL? In-Reply-To: <4ACEEF7F.70200@gmail.com> References: <97814d820910081916vff16452kb9a817af7ff8d7cc@mail.gmail.com> <97814d820910082113s3faa74fbp70e9082cb5e66ab1@mail.gmail.com> <4ACEEF7F.70200@gmail.com> Message-ID: <97814d820910122301x49d2ced4w865bf8e57633e29@mail.gmail.com> Thanks all for your help on this topic! I've managed to get started with Erlguten thanks to the samples in the test directory, they are really god. Although some more formal documentation would be very welcome as well, in particular explaining the "galley", "con" and "tem" and "map" files. In the end I found I could just get the data out of mnesia straight into Erlguten, no XML required to my relief. Now all that's left is a reporting engine and some pretty diagrams :-) And a pointer to any other poor sod trying to build Erlguten: you'll need autoconf, but the versions 2.63 and 2.64 appear to have a bug causing the build to fail, so use version 2.60, that'll work. So if you're on Ubuntu like me, you can't just go apt-get autoconf, you have to get the tar file and install it manually. Nina On Fri, Oct 9, 2009 at 7:08 PM, Richard Carlsson wrote: > Nina Alex Juliadotter wrote: >> I've come across Erlguten for PDF generation, which takes XML files as >> input. So, to use Mnesia data, I take it I could use XMerL to >> transform it into XML? The only trouble is that both these projects >> seem rather idle and alpha like, the last time I've seen any >> discussions was in 2004 and the source I found is from 2003. Is anyone >> generating PDF files in their Erlang app, and in that case what >> libraries are you using? > > We are using erlguten for heavy duty work in daily production. > Just make sure you use the latest version, kept at google code: > > http://code.google.com/p/erlguten/ > > ? ?/Richard > From kenneth.lundin@REDACTED Tue Oct 13 08:45:18 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 13 Oct 2009 08:45:18 +0200 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <4AD417B8.8000702@erlang-consulting.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> <4AD38779.3040700@gmail.com> <4AD417B8.8000702@erlang-consulting.com> Message-ID: There has been plenty of thinking and prototyping in the area of "SAFE" Erlang and the solution is far from trivial. There are many aspects to consider. Have a look at the following Master theses: Safe Erlang. Gustaf Naeser. UU/CSD, 1997. http://www.erlang.se/publications/xjobb/0109-naeser.pdf Enhancing Security in Distributed Erlang by Integrating Access Control. Rickard Green. KTH, 2000. http://www.erlang.se/publications/xjobb/d95-rgr.thesis.2s.pdf Secure Distributed Communication in SafeErlang. Bertil Karlsson. KTH, 2000. http://www.erlang.se/publications/xjobb/Secure_Dist_Comm_in_SafeErlang.pdf We have not been able to prioritize work towards "SAFER" built in distribution mechanisms since the current implementation support local trusted networks well enough and that was the intended use from the beginning. This kind of distribution is still the dominating. It might whoever be the right time to do something in this area now. /Kenneth Erlang/OTP Ericsson On Tue, Oct 13, 2009 at 8:01 AM, Ulf Wiger wrote: > Jacob Torrey wrote: >> >> Could perhaps the pid generation in the VM be made more random, >> therefore you can only gain access to pids that you know the name to? >> This seems like an obvious (and pretty trivial) solution to obscure and >> increase the security of the Erlang system. > > You don't even need to bother with the pids: > > {rex,Node} ! {'$gen_cast',{cast,M,F,A,user}} > > ...will allow you to execute any function on Node. > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ngocdaothanh@REDACTED Tue Oct 13 09:38:13 2009 From: ngocdaothanh@REDACTED (ngocdaothanh) Date: Tue, 13 Oct 2009 00:38:13 -0700 (PDT) Subject: memsup:get_memory_usage() on leopard In-Reply-To: <4EE7B92A-09E9-4797-8D7C-0BDC999EA254@gmail.com> References: <4EE7B92A-09E9-4797-8D7C-0BDC999EA254@gmail.com> Message-ID: Thank you for this patch. As "{unix,darwin}" implies, with the original code, there is error starting os_mon on Leopard, but no problem on Linux. On Sep 25, 1:07 am, Joel Reymont wrote: > Does not work and prevents the os_mon app from starting. Here's a > patch... > > diff --git a/memsup.erl b/memsup.erl > index 4e4eff0..f9ad44e 100644 > --- a/memsup.erl > +++ b/memsup.erl > @@ -728,8 +728,10 @@ get_memory_usage({unix,darwin}) -> > io_lib:fread("Pages active:~d.", skip_to_eol(Str2)), > {ok, [Inactive],Str4} = > io_lib:fread("Pages inactive:~d.", skip_to_eol(Str3)), > + {ok, _,Str5} = > + io_lib:fread("Pages speculative:~d.", skip_to_eol(Str4)), > {ok, [Wired],_} = > - io_lib:fread("Pages wired down:~d.", skip_to_eol(Str4)), > + io_lib:fread("Pages wired down:~d.", skip_to_eol(Str5)), > NMemUsed = (Wired + Active + Inactive) * 4000, > NMemTotal = NMemUsed + Free * 4000, > {NMemUsed,NMemTotal}; > > --- > fastest mac firefox!http://wagerlabs.com > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From tony@REDACTED Tue Oct 13 08:53:19 2009 From: tony@REDACTED (Tony Rogvall) Date: Tue, 13 Oct 2009 08:53:19 +0200 Subject: [erlang-questions] Socket connects to itself In-Reply-To: <200910122212.n9CMC5pG080185@pluto.hedeland.org> References: <200910122212.n9CMC5pG080185@pluto.hedeland.org> Message-ID: <5D38F61E-89AE-4A1B-89B6-14290EBAADDC@rogvall.se> In C you could of course check the bound port before connecting. And I guess the same fix could be applied to prim_inet if that is what you want ;-) /Tony On 13 okt 2009, at 00.12, Per Hedeland wrote: > Rob Charlton wrote: >> >> I noticed in the logs one night that this process P which had tried >> for >> several hours to connect to a non-existent server on 127.0.0.1:2920 >> had >> actually succeeded! Even more exciting than that, it had connected to >> itself, so when it started trying to communicate with its peer it was >> very surprised to receive its own communications back again. >> Needless to >> say P crashed at this point. I was very curious to know how this >> could >> happen and to try and fix it! You could argue this isn't a bug >> because >> the system configuration was incorrectly specified. In this >> particular >> case though, the misconfiguration can happen easily and I would >> like P >> not to crash but instead to keep trying. >> >> I tried Googling for this and did find a couple of non-erlang sources >> suggesting it had happened to them in similar circumstances. It >> could be >> that this is a very timing sensitive 'feature' of TCP/IP. Can anyone >> enlighten me? > > That's a nice one!:-) Whenever you call gen_tcp:connect(), the > underlying OS connect() system call will first bind the socket to a > local address and port, and then send the initial SYN packet using > that > address/port as source. It is quite possible that at some point it may > happen to choose the unused port 2920 for that, and hence attempt a > connect from 127.0.0.1:2920 to 127.0.0.1:2920. > > This is where the "feature" comes into play - you might think that > this > connection should fail, since there is no process listening on port > 2920, but the TCP spec explicitly prescribes that "simultaneous > initiation" should be handled - e.g. from section 2.7 of RFC 793: > > Two processes which issue active OPENs to each > other at the same time will be correctly connected. This flexibility > is critical for the support of distributed computing in which > components act asynchronously with respect to each other. > > You can also see it in the famous state diagram in Figure 6 of RFC > 793, > as the "rcv SYN" transition from SYN_SENT to SYN_RCVD. Though of > course > it's a pretty safe bet that no protocol in serious use actually > requires > this "critical" functionality. > > --Per Hedeland > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From cbenac@REDACTED Tue Oct 13 10:20:14 2009 From: cbenac@REDACTED (Clara Benac Earle) Date: Tue, 13 Oct 2009 10:20:14 +0200 Subject: Erlang Workshop 2009 talks Message-ID: <4AD4383E.6070500@fi.upm.es> The Erlang Workshop talks and slides are available online. Please check: http://www.erlang-factory.com/conference/2009ErlangWorkshop/talks Cheers, Clara From per@REDACTED Tue Oct 13 12:37:07 2009 From: per@REDACTED (Per Hedeland) Date: Tue, 13 Oct 2009 12:37:07 +0200 (CEST) Subject: [erlang-questions] Socket connects to itself In-Reply-To: <5D38F61E-89AE-4A1B-89B6-14290EBAADDC@rogvall.se> Message-ID: <200910131037.n9DAb7Hd096390@pluto.hedeland.org> Tony Rogvall wrote: > >In C you could of course check the bound port before connecting. Actually no, you couldn't, since the bind happens implicitly as part of the connect() call. What you *could* do in C but not in Erlang would be to explicitly bind the socket before the connect, but of course that carries a whole can of worms that you really don't want to open for this case. >And I guess the same fix could be applied to prim_inet if that is what >you want ;-) Personally I don't care much:-), but I don't see any particular reason that Erlang should work differently from C in this (unusual) case. Maybe the TCP/IP stack "should" reject this particular case of "simultaneous initiation" since it is meaningless though - maybe some of them actually do. --Per From rob@REDACTED Tue Oct 13 13:14:02 2009 From: rob@REDACTED (Rob Charlton) Date: Tue, 13 Oct 2009 12:14:02 +0100 Subject: [erlang-questions] Socket connects to itself In-Reply-To: <200910131037.n9DAb7Hd096390@pluto.hedeland.org> References: <200910131037.n9DAb7Hd096390@pluto.hedeland.org> Message-ID: <4AD460FA.70007@erlang-consulting.com> Per Hedeland wrote: > Maybe the TCP/IP stack "should" reject this particular case of "simultaneous > initiation" since it is meaningless though - maybe some of them actually > do. > I should have said this happened in Linux - I don't know if the same occurs on Windows or Mac. Many thanks for the explanation. I had heard about simultaneous connect before, but I couldn't see why it would occur in this scenario until you explained it. BR Rob -- Erlang Training and Consulting Ltd www.erlang-consulting.com From tony@REDACTED Tue Oct 13 14:25:32 2009 From: tony@REDACTED (Tony Rogvall) Date: Tue, 13 Oct 2009 14:25:32 +0200 Subject: [erlang-questions] Socket connects to itself In-Reply-To: <200910131037.n9DAb7Hd096390@pluto.hedeland.org> References: <200910131037.n9DAb7Hd096390@pluto.hedeland.org> Message-ID: On 13 okt 2009, at 12.37, Per Hedeland wrote: > Tony Rogvall wrote: >> >> In C you could of course check the bound port before connecting. > > Actually no, you couldn't, since the bind happens implicitly as part > of > the connect() call. What you *could* do in C but not in Erlang would > be > to explicitly bind the socket before the connect, but of course that > carries a whole can of worms that you really don't want to open for > this > case. Hmm. I normally bind my ports before connecting ;-) Sometime you need to control the source address and source port. So that is also what Erlang does ;-) The connect defaults to bind to ifaddr = any and port = 0 This bind, I guess, is the operating system default so the feature/ problem reported may occur anyway. The check would probably be a bit hard to implement anyway. If connect is connecting to a local address and the socket is bound to any (0.0.0.0) address, the way to determine if this connect is an "invalid" connect, one must scan the interface addresses and probably the routing tables (and more) to see if this case will apply.? Is this the "can" you where talking about ;-) > >> And I guess the same fix could be applied to prim_inet if that is >> what >> you want ;-) > > Personally I don't care much:-), but I don't see any particular reason > that Erlang should work differently from C in this (unusual) case. > Maybe > the TCP/IP stack "should" reject this particular case of "simultaneous > initiation" since it is meaningless though - maybe some of them > actually > do. As said above it depends on who is writing the C code ;-) /Tony > > --Per > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From per@REDACTED Tue Oct 13 16:38:25 2009 From: per@REDACTED (Per Hedeland) Date: Tue, 13 Oct 2009 16:38:25 +0200 (CEST) Subject: [erlang-questions] Socket connects to itself In-Reply-To: Message-ID: <200910131438.n9DEcPuc000962@pluto.hedeland.org> Tony Rogvall wrote: > >On 13 okt 2009, at 12.37, Per Hedeland wrote: > >> Tony Rogvall wrote: >>> >>> In C you could of course check the bound port before connecting. >> >> Actually no, you couldn't, since the bind happens implicitly as part >> of >> the connect() call. What you *could* do in C but not in Erlang would >> be >> to explicitly bind the socket before the connect, but of course that >> carries a whole can of worms that you really don't want to open for >> this >> case. > >Hmm. I normally bind my ports before connecting ;-) You do such weird things...:-) > Sometime you >need to control the source address and source port. Yes, in some rare cases you do, but in 99.9% of them the kernel knows better than you, and you'll just risk messing things up that way - e.g. binding to an address that isn't reachable from the destination. Not to mention trying to bind to a "free" port different from the destination port, to avoid the problem here. > So that is also >what Erlang does ;-) The connect defaults to bind to ifaddr = any and >port = 0 Ah, it was a while since I last had the pleasure of digging through that code, my memory is fading... >This bind, I guess, is the operating system default so the feature/ >problem reported >may occur anyway. Indeed, such a bind should never be problematic, but is also just a waste of cycles. >The check would probably be a bit hard to implement anyway. If connect >is connecting to a local address and the socket is bound to any >(0.0.0.0) address, >the way to determine if this connect is an "invalid" connect, one must >scan >the interface addresses and probably the routing tables (and more) to >see >if this case will apply.? Well, you could take a simpler approach and just consider it invalid if the port was the same as the destination port. I'm not sure that there is a fool-proof way (for all implementations) to make sure that you don't get the same port again if you re-try, though... > Is this the "can" you where talking about ;-) No, rather the problems that you can run into if you try to bind to a *specific* address/port (unless you really need to), per above. --Per From tony@REDACTED Tue Oct 13 16:57:46 2009 From: tony@REDACTED (Tony Rogvall) Date: Tue, 13 Oct 2009 16:57:46 +0200 Subject: [erlang-questions] Socket connects to itself In-Reply-To: <200910131438.n9DEcPuc000962@pluto.hedeland.org> References: <200910131438.n9DEcPuc000962@pluto.hedeland.org> Message-ID: <6F1DEFF1-1B44-4584-9AAF-3CE21341F1DF@rogvall.se> On 13 okt 2009, at 16.38, Per Hedeland wrote: > Tony Rogvall wrote: >> >> On 13 okt 2009, at 12.37, Per Hedeland wrote: >> >>> Tony Rogvall wrote: >>>> >>>> In C you could of course check the bound port before connecting. >>> >>> Actually no, you couldn't, since the bind happens implicitly as part >>> of >>> the connect() call. What you *could* do in C but not in Erlang would >>> be >>> to explicitly bind the socket before the connect, but of course that >>> carries a whole can of worms that you really don't want to open for >>> this >>> case. >> >> Hmm. I normally bind my ports before connecting ;-) > > You do such weird things...:-) Talk about it ;-) I did some experiments (yes I am a bit bored right now). I tried some of the low levels calls for gen_tcp:connect to see what is going on: On my mac: {ok,S} = inet:open(-1,{0,0,0,0},0,[{active,true}],tcp,inet,inet_tcp). {ok,{_,Port}} = inet:sockname(S). prim_inet:connect(S, {127,0,0,1}, Port, infinity). Result is {error,einval} Same thing on my linux (ubuntu 9.04) / vmware (on my mac :-) {ok,S} = inet:open(-1,{0,0,0,0},0,[{active,true}],tcp,inet,inet_tcp). {ok,{_,Port}} = inet:sockname(S). prim_inet:connect(S, {127,0,0,1}, Port, infinity). Result is {ok,#Port<..>} I like this one ;-) No one is listening .... how about you? /Tony > >> Sometime you >> need to control the source address and source port. > > Yes, in some rare cases you do, but in 99.9% of them the kernel knows > better than you, and you'll just risk messing things up that way - > e.g. binding to an address that isn't reachable from the destination. > Not to mention trying to bind to a "free" port different from the > destination port, to avoid the problem here. > >> So that is also >> what Erlang does ;-) The connect defaults to bind to ifaddr = any and >> port = 0 > > Ah, it was a while since I last had the pleasure of digging through > that > code, my memory is fading... > >> This bind, I guess, is the operating system default so the feature/ >> problem reported >> may occur anyway. > > Indeed, such a bind should never be problematic, but is also just a > waste of cycles. > >> The check would probably be a bit hard to implement anyway. If >> connect >> is connecting to a local address and the socket is bound to any >> (0.0.0.0) address, >> the way to determine if this connect is an "invalid" connect, one >> must >> scan >> the interface addresses and probably the routing tables (and more) to >> see >> if this case will apply.? > > Well, you could take a simpler approach and just consider it invalid > if > the port was the same as the destination port. I'm not sure that there > is a fool-proof way (for all implementations) to make sure that you > don't get the same port again if you re-try, though... > >> Is this the "can" you where talking about ;-) > > No, rather the problems that you can run into if you try to bind to a > *specific* address/port (unless you really need to), per above. > > --Per > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From jim.mccoy@REDACTED Tue Oct 13 19:02:22 2009 From: jim.mccoy@REDACTED (Jim McCoy) Date: Tue, 13 Oct 2009 10:02:22 -0700 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> <4AD38779.3040700@gmail.com> <4AD417B8.8000702@erlang-consulting.com> Message-ID: As much as I would love to see some work go into making Erlang more "internally secure" I would probably prefer to see the effort directed at making it easier and less cumbersome to run distributed erlang across untrusted networks. Something as simple as a flag to make all node-to-node communication run over SSL and a few of the previously discussed tweaks to make epmd less susceptible to DoS attacks would probably deliver more short-term benefit to the community at large. jim On Mon, Oct 12, 2009 at 11:45 PM, Kenneth Lundin wrote: >[...] > We have not been able to prioritize work towards "SAFER" built in > distribution mechanisms > since the current implementation support local trusted networks well > enough and that > was the intended use from the beginning. This kind of distribution is still > the dominating. > > It might whoever be the right time to do something in this area now. > > > /Kenneth Erlang/OTP Ericsson > From andrey@REDACTED Tue Oct 13 18:28:25 2009 From: andrey@REDACTED (Andrey Tsirulev) Date: Tue, 13 Oct 2009 20:28:25 +0400 Subject: CPU load of TCP server Message-ID: Hello all, I'm exploring the possibility of using Erlang for my TCP service application (actually the game server). I've prepared test server and client applications. The test server application accepts client connections and sends 2 small (<1 Kb) packets per second to each client (and receives answers). I've met the following problems: 1) Kernel polling doesn't give any benefit with R13B02-1. 2) CPU load is too high. All the details are below. Here's my test server's `uname -a`: Linux source 2.6.29-gentoo-r5 #1 SMP Tue Aug 18 01:15:17 MSD 2009 x86_64 AMD Sempron(tm) Dual Core Processor 2200 AuthenticAMD GNU/Linux (I've made tests also with 2 other linux servers with different kernel versions and results were close). I've made server connection processes as simple and possible. I've tried up to 10000 concurrent connections. Test results didn't not show any visible difference between using multiple remote machines for client connections, one remote machine or localhost. I tried R13B02-1 and R12B-5 OTP versions. I found that memory usage grow is linear, as expected. But I came to the problem with CPU load. First of all, kernel polling didn't give any benefit for R13B02-1 (while erlang:system_info(kernel_poll) returned true and erl started with message [kernel-poll:true]). I've got about 55% of CPU usage with 4000 connections both with and without kernel polling enabled, while with R12B-5 I have about 26% of CPU usage with +Ktrue. I suspect a bug either in OTP or in gentoo ebuild (of course it's also quite possible that I'm doing something wrong or missed something in docs). The following is about R12B-5. I get about 6-7% of CPU load per every 1000 connections (about 60% CPU load for 10000 connections). I'm not sure if I should consider this as a good result or a bad one. Most of the articles on the same subject say that CPU load is negligible in their tests and they are fighting for memory only, so I expected I won't be CPU-limited too, but evidently I am. `top` says that about 50% of CPU load is userspace, 25% software interrupts, 20% system and 5% hardware interrupts (that's by eye, not very strict). I found that CPU load depends not as much on connection count but on transmitted packet count (ok, that's obviously the number of system calls). Thus if I send 4 packets per second, not 2, I should decrease the number of connections twice to preserve the same CPU load. CPU load does not depend on packet size. 1 byte or 1Kbyte - no visible difference. CPU usage is slightly less with active socket option enabled than with blocking recvs. CPU usage on the single windows client machine with 4000 connections spawned is on the same level as with the linux server handling these 4000 connections (while I expected linux to perform better). Switching Nagle on and off had no effect. I also tried to tune TCP stack with sysctl using advises found here and there but almost without any effect too. I've tried to trace with fprof and found that bottlenecks are 'send' operations (but I'm a relative novice to erlang so I'm not sure my usage of fprof was correct). Ok, that was expected too. I've read the 'why is gen_tcp:send slow?' thread but none of advises given there helped me. So the main question is: is the CPU usage of 7% per 1000 connections (or maybe better say 2000 packets per second) a good result? If no, what is the expected result? How can I improve my test application? Or maybe something in my story looks strange? I know that the possible optimization is decreasing the number of packets and keep it in mind. Here's the server connection process loop: loop(Socket) -> receive {tcp, Socket, _Packet} -> loop(Socket); {tcp_closed, Socket} -> normal; _ -> loop(Socket) after 500 -> gen_tcp:send(Socket,[?PACKET]), loop(Socket) end. Client loop has blocking recv and answers with send immediately. Thank you very much for your time. Sorry for too many words, I tried to provide all possible information. I will answer any question and appreciate any hint. Best regards, Andrey From rapsey@REDACTED Tue Oct 13 19:50:17 2009 From: rapsey@REDACTED (Rapsey) Date: Tue, 13 Oct 2009 19:50:17 +0200 Subject: [erlang-questions] CPU load of TCP server In-Reply-To: References: Message-ID: <97619b170910131050g2632f794q46b45ad3cd48f831@mail.gmail.com> Every time after gets executed a timer gets created (I presume). With 10k processes it probably makes a noticeable CPU impact. Sergej 2009/10/13 Andrey Tsirulev > Hello all, > > I'm exploring the possibility of using Erlang for my TCP service > application (actually the game server). I've prepared test server and client > applications. The test server application accepts client connections and > sends 2 small (<1 Kb) packets per second to each client (and receives > answers). > > I've met the following problems: > 1) Kernel polling doesn't give any benefit with R13B02-1. > 2) CPU load is too high. > > All the details are below. > > Here's my test server's `uname -a`: > Linux source 2.6.29-gentoo-r5 #1 SMP Tue Aug 18 01:15:17 MSD 2009 x86_64 > AMD Sempron(tm) Dual Core Processor 2200 AuthenticAMD GNU/Linux > (I've made tests also with 2 other linux servers with different kernel > versions and results were close). > > I've made server connection processes as simple and possible. I've tried up > to 10000 concurrent connections. > > Test results didn't not show any visible difference between using multiple > remote machines for client connections, one remote machine or localhost. > > I tried R13B02-1 and R12B-5 OTP versions. > > I found that memory usage grow is linear, as expected. But I came to the > problem with CPU load. > > First of all, kernel polling didn't give any benefit for R13B02-1 (while > erlang:system_info(kernel_poll) returned true and erl started with message > [kernel-poll:true]). I've got about 55% of CPU usage with 4000 connections > both with and without kernel polling enabled, while with R12B-5 I have about > 26% of CPU usage with +Ktrue. I suspect a bug either in OTP or in gentoo > ebuild (of course it's also quite possible that I'm doing something wrong or > missed something in docs). > > The following is about R12B-5. I get about 6-7% of CPU load per every 1000 > connections (about 60% CPU load for 10000 connections). I'm not sure if I > should consider this as a good result or a bad one. Most of the articles on > the same subject say that CPU load is negligible in their tests and they are > fighting for memory only, so I expected I won't be CPU-limited too, but > evidently I am. > > `top` says that about 50% of CPU load is userspace, 25% software > interrupts, 20% system and 5% hardware interrupts (that's by eye, not very > strict). > > I found that CPU load depends not as much on connection count but on > transmitted packet count (ok, that's obviously the number of system calls). > Thus if I send 4 packets per second, not 2, I should decrease the number of > connections twice to preserve the same CPU load. > > CPU load does not depend on packet size. 1 byte or 1Kbyte - no visible > difference. > > CPU usage is slightly less with active socket option enabled than with > blocking recvs. > > CPU usage on the single windows client machine with 4000 connections > spawned is on the same level as with the linux server handling these 4000 > connections (while I expected linux to perform better). > > Switching Nagle on and off had no effect. I also tried to tune TCP stack > with sysctl using advises found here and there but almost without any effect > too. > > I've tried to trace with fprof and found that bottlenecks are 'send' > operations (but I'm a relative novice to erlang so I'm not sure my usage of > fprof was correct). Ok, that was expected too. I've read the 'why is > gen_tcp:send slow?' thread but none of advises given there helped me. > > So the main question is: is the CPU usage of 7% per 1000 connections (or > maybe better say 2000 packets per second) a good result? If no, what is the > expected result? How can I improve my test application? Or maybe something > in my story looks strange? > > I know that the possible optimization is decreasing the number of packets > and keep it in mind. > > Here's the server connection process loop: > > loop(Socket) -> > receive > {tcp, Socket, _Packet} -> > loop(Socket); > {tcp_closed, Socket} -> > normal; > _ -> > loop(Socket) > after 500 -> > gen_tcp:send(Socket,[?PACKET]), > loop(Socket) > end. > > Client loop has blocking recv and answers with send immediately. > > Thank you very much for your time. Sorry for too many words, I tried to > provide all possible information. I will answer any question and appreciate > any hint. > > Best regards, > Andrey From garry@REDACTED Tue Oct 13 20:15:55 2009 From: garry@REDACTED (Garry Hodgson) Date: Tue, 13 Oct 2009 14:15:55 -0400 Subject: is inet:gethostbyname( IP ) correct? Message-ID: <4AD4C3DB.2000408@research.att.com> recently, a bug in my code caused us to pass a string representing a floating point number to inet:gethostbyname(). i would have expected it to return an error, but instead it returned an ip address, but one that made no sense to me: 1> inet:gethostbyname( '12.27' ). {ok,{hostent,"12.27",[],inet,4,[{12,0,0,27}]}} so my question is, is this behavior correct, and if so, what exactly does it mean that a lookup of '12.27' maps to '12.0.0.27'? thanks -- Garry Hodgson Lead Member of Technical Staff AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From andrey@REDACTED Tue Oct 13 20:34:05 2009 From: andrey@REDACTED (Andrey Tsirulev) Date: Tue, 13 Oct 2009 22:34:05 +0400 Subject: [erlang-questions] CPU load of TCP server References: <97619b170910131050g2632f794q46b45ad3cd48f831@mail.gmail.com> Message-ID: Hi Sergej, Thanks for the hint. I moved timer to client. Now I have about 5.5% of CPU usage per each 1000 connections. I still expect it should be less.. Andrey ----- Original Message ----- From: "Rapsey" To: Sent: Tuesday, October 13, 2009 9:50 PM Subject: Re: [erlang-questions] CPU load of TCP server > Every time after gets executed a timer gets created (I presume). With 10k > processes it probably makes a noticeable CPU impact. > > > Sergej > > 2009/10/13 Andrey Tsirulev > >> Hello all, >> >> I'm exploring the possibility of using Erlang for my TCP service >> application (actually the game server). I've prepared test server and >> client >> applications. The test server application accepts client connections and >> sends 2 small (<1 Kb) packets per second to each client (and receives >> answers). >> >> I've met the following problems: >> 1) Kernel polling doesn't give any benefit with R13B02-1. >> 2) CPU load is too high. >> >> All the details are below. >> >> Here's my test server's `uname -a`: >> Linux source 2.6.29-gentoo-r5 #1 SMP Tue Aug 18 01:15:17 MSD 2009 x86_64 >> AMD Sempron(tm) Dual Core Processor 2200 AuthenticAMD GNU/Linux >> (I've made tests also with 2 other linux servers with different kernel >> versions and results were close). >> >> I've made server connection processes as simple and possible. I've tried >> up >> to 10000 concurrent connections. >> >> Test results didn't not show any visible difference between using >> multiple >> remote machines for client connections, one remote machine or localhost. >> >> I tried R13B02-1 and R12B-5 OTP versions. >> >> I found that memory usage grow is linear, as expected. But I came to the >> problem with CPU load. >> >> First of all, kernel polling didn't give any benefit for R13B02-1 (while >> erlang:system_info(kernel_poll) returned true and erl started with >> message >> [kernel-poll:true]). I've got about 55% of CPU usage with 4000 >> connections >> both with and without kernel polling enabled, while with R12B-5 I have >> about >> 26% of CPU usage with +Ktrue. I suspect a bug either in OTP or in gentoo >> ebuild (of course it's also quite possible that I'm doing something wrong >> or >> missed something in docs). >> >> The following is about R12B-5. I get about 6-7% of CPU load per every >> 1000 >> connections (about 60% CPU load for 10000 connections). I'm not sure if I >> should consider this as a good result or a bad one. Most of the articles >> on >> the same subject say that CPU load is negligible in their tests and they >> are >> fighting for memory only, so I expected I won't be CPU-limited too, but >> evidently I am. >> >> `top` says that about 50% of CPU load is userspace, 25% software >> interrupts, 20% system and 5% hardware interrupts (that's by eye, not >> very >> strict). >> >> I found that CPU load depends not as much on connection count but on >> transmitted packet count (ok, that's obviously the number of system >> calls). >> Thus if I send 4 packets per second, not 2, I should decrease the number >> of >> connections twice to preserve the same CPU load. >> >> CPU load does not depend on packet size. 1 byte or 1Kbyte - no visible >> difference. >> >> CPU usage is slightly less with active socket option enabled than with >> blocking recvs. >> >> CPU usage on the single windows client machine with 4000 connections >> spawned is on the same level as with the linux server handling these 4000 >> connections (while I expected linux to perform better). >> >> Switching Nagle on and off had no effect. I also tried to tune TCP stack >> with sysctl using advises found here and there but almost without any >> effect >> too. >> >> I've tried to trace with fprof and found that bottlenecks are 'send' >> operations (but I'm a relative novice to erlang so I'm not sure my usage >> of >> fprof was correct). Ok, that was expected too. I've read the 'why is >> gen_tcp:send slow?' thread but none of advises given there helped me. >> >> So the main question is: is the CPU usage of 7% per 1000 connections (or >> maybe better say 2000 packets per second) a good result? If no, what is >> the >> expected result? How can I improve my test application? Or maybe >> something >> in my story looks strange? >> >> I know that the possible optimization is decreasing the number of packets >> and keep it in mind. >> >> Here's the server connection process loop: >> >> loop(Socket) -> >> receive >> {tcp, Socket, _Packet} -> >> loop(Socket); >> {tcp_closed, Socket} -> >> normal; >> _ -> >> loop(Socket) >> after 500 -> >> gen_tcp:send(Socket,[?PACKET]), >> loop(Socket) >> end. >> >> Client loop has blocking recv and answers with send immediately. >> >> Thank you very much for your time. Sorry for too many words, I tried to >> provide all possible information. I will answer any question and >> appreciate >> any hint. >> >> Best regards, >> Andrey > From caio.ariede@REDACTED Tue Oct 13 21:06:37 2009 From: caio.ariede@REDACTED (caio ariede) Date: Tue, 13 Oct 2009 16:06:37 -0300 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <4AD4C3DB.2000408@research.att.com> References: <4AD4C3DB.2000408@research.att.com> Message-ID: <6a9ba5690910131206v11245896kb8088aa9c335bf9f@mail.gmail.com> Interesting case. The result doesn't appear to be an issue, but a feature of the gethostbyname original implementation, in C code. You get the same result, testing with PHP: $ php -r 'var_dump(gethostbyname("12.27"));' string(9) "12.0.0.27" And other interesting results: $ php -r 'var_dump(gethostbyname("255.2.256"));' string(9) "255.2.1.0" But I can't see where it's really useful. Caio Ariede http://caioariede.com/ On Tue, Oct 13, 2009 at 3:15 PM, Garry Hodgson wrote: > recently, a bug in my code caused us to pass a string > representing a floating point number to inet:gethostbyname(). > i would have expected it to return an error, but instead it > returned an ip address, but one that made no sense to me: > > 1> inet:gethostbyname( '12.27' ). > {ok,{hostent,"12.27",[],inet,4,[{12,0,0,27}]}} > > so my question is, is this behavior correct, and if so, what > exactly does it mean that a lookup of '12.27' maps to '12.0.0.27'? > > thanks > > -- > Garry Hodgson > Lead Member of Technical Staff > AT&T Chief Security Office (CSO) > > "This e-mail and any files transmitted with it are AT&T property, are > confidential, and are intended solely for the use of the individual or > entity to whom this e-mail is addressed. If you are not one of the named > recipient(s) or otherwise have reason to believe that you have received this > message in error, please notify the sender and delete this message > immediately from your computer. Any other use, retention, dissemination, > forwarding, printing, or copying of this e-mail is strictly prohibited." > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From max.lapshin@REDACTED Tue Oct 13 22:03:03 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 14 Oct 2009 00:03:03 +0400 Subject: [erlang-questions] CPU load of TCP server In-Reply-To: References: <97619b170910131050g2632f794q46b45ad3cd48f831@mail.gmail.com> Message-ID: Andrew, You should test against real business logic, because syntetic tests don't show anything at all. While experimenting with video streaming server, I've got strange results: 5 users used the same CPU, like 30. So I advice to test not erlang virtual machine (which isn't fastest one), but erlang environment (which can be faster due to advanced binary usage and scalability possibilities). From per@REDACTED Tue Oct 13 23:05:31 2009 From: per@REDACTED (Per Hedeland) Date: Tue, 13 Oct 2009 23:05:31 +0200 (CEST) Subject: [erlang-questions] Socket connects to itself In-Reply-To: <6F1DEFF1-1B44-4584-9AAF-3CE21341F1DF@rogvall.se> Message-ID: <200910132105.n9DL5VR9008621@pluto.hedeland.org> Tony Rogvall wrote: > >I did some experiments (yes I am a bit bored right now). >I tried some of the low levels calls for gen_tcp:connect to see what >is going on: > >On my mac: >{ok,S} = inet:open(-1,{0,0,0,0},0,[{active,true}],tcp,inet,inet_tcp). >{ok,{_,Port}} = inet:sockname(S). >prim_inet:connect(S, {127,0,0,1}, Port, infinity). >Result is {error,einval} Interesting. >Same thing on my linux (ubuntu 9.04) / vmware (on my mac :-) > >{ok,S} = inet:open(-1,{0,0,0,0},0,[{active,true}],tcp,inet,inet_tcp). >{ok,{_,Port}} = inet:sockname(S). >prim_inet:connect(S, {127,0,0,1}, Port, infinity). >Result is {ok,#Port<..>} > >I like this one ;-) No one is listening .... how about you? Yes, this is what I elaborated on in my first message in the thread. The connect() sends a SYN, which loops back into the TCP input side, where it finds the socket in SYN_SENT state (obviously). Per the spec, SYN reception in SYN_SENT should cause transition to SYN_RCVD and the connection should be established. It works on FreeBSD too, so it must be The Right Thing(tm).:-) 1> {ok,S} = inet:open(-1,{0,0,0,0},0,[{active,true}],tcp,inet,inet_tcp). {ok,#Port<0.423>} 2> {ok,{_,Port}} = inet:sockname(S). {ok,{{0,0,0,0},59004}} 3> prim_inet:connect(S, {127,0,0,1}, Port, infinity). ok 4> gen_tcp:send(S, <<"foo">>). ok 5> flush(). Shell got {tcp,#Port<0.423>,"foo"} ok --Per From ahmed.nawras@REDACTED Wed Oct 14 01:09:33 2009 From: ahmed.nawras@REDACTED (Ahmed Ali) Date: Wed, 14 Oct 2009 03:09:33 +0400 Subject: SCTP PPID change Message-ID: Hi, I'm experminting with SCTP support in Erlang. I'm using Solaris 10 with Erlang R13B02. SCTP support is enabled during compiling Erlang. I'm having problem with changing ppid to 3 (M3UA). The first thing I did is to add sctp_default_send_param in gen_sctp:connect/4. However, I got the error below. I've tried to used gen_sctp:send/3 instead of send/4 as below but I get exactly the same error. I also tried to troubleshoot the issue and found out the error is generated from module prim_inet.erl in erts code. My guess is that type_value/3 function checks that the values of the supplied #sctp_sndrcvinfo{} are valid but appearntly, something is not correct in #sctp_sndrcvinfo{} record I'm sending. Without setting PPID, everything works and I could see traffic is sent and received correctly. Any help is appreciated. gen_sctp:send(S, #sctp_sndrcvinfo{stream=Stream, assoc_id=Assoc, ppid=3}, Data) {error, {type, {record,set, {sctp_sndrcvinfo, [uint16,0], [], [{bitenumlist, [{unordered,1},{addr_over,2},{abort,4},{eof,8}], uint16}, []], [uint32,0], [uint32,0], [uint32,0], [],[], [sctp_assoc_id,0]}, {sctp_sndrcvinfo,0,undefined,undefined,3,undefined,undefined, undefined,undefined, {sctp_assoc_change,comm_up,0,5,17,1}}, 10 } } } Best regards, Ahmed From yogishb@REDACTED Wed Oct 14 04:38:01 2009 From: yogishb@REDACTED (Yogish Baliga) Date: Tue, 13 Oct 2009 19:38:01 -0700 (PDT) Subject: Sending messages between erlang nodes (across firewall boundary) Message-ID: <65871.49596.qm@web31807.mail.mud.yahoo.com> Hello all, I am working on setting up the Erlang nodes across multiple data centers. I do not want to send the data across the data centers in clear text. Within the data centers, I did a simple setup for 2 Erlang nodes on 2 physical boxes. Captured the tcp traffic while sending the messages between these 2 nodes. Raw TCP packets do not show the text being sent in clear text. Is it because by default the messages are encrypted using the cookie or something? Do I need to specify the proto_dist as inet_ssl in order to setup the Erlang node grid across data center? Ideally I don't want to setup the SSL communication between nodes. If that is what is absolutely required in-order to make sure that the communication is encrypted, I will do that. Thanx, -- baliga "Point of view is worth 80 IQ points" --Alan Kay http://dudefrommangalore.blogspot.com/ From kagato@REDACTED Wed Oct 14 08:07:31 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Wed, 14 Oct 2009 01:07:31 -0500 Subject: [erlang-questions] newbie needing help getting started In-Reply-To: <634A53E4-FBDB-46E8-9771-26811E2F4DA6@erlang-consulting.com> References: <27c25bc40910100645t2e2b4d81yfbbc85741723bb98@mail.gmail.com> <9b08084c0910100651j247116edicfe782501137d42e@mail.gmail.com> <27c25bc40910100657l6afe61aehe6849ea58cdb6c8d@mail.gmail.com> <634A53E4-FBDB-46E8-9771-26811E2F4DA6@erlang-consulting.com> Message-ID: <9919866E-D828-4A4A-A2EB-C46FD5EB6E53@souja.net> Short Version: Compile the math1 module. Either use this in the shell: > c(math). Or compile it first using the erlc command. Or compile it using "erl -make" or "make:all()". There are a few options. In the shell and in Erlang .erl files, use periods, instead of semicolons, to separate top-level definitions. To see a complete, if empty, example of an Erlang build environment, you can use git to checkout my skeleton deployment directory. It's a work in progress, but it should be close. Long Version: When you run commands in the shell, the system will automatically load compiled modules in the path. Running: > math1:factoral(0). Will attempt to load math1.beam from the code path. To make this file from a .erl file, you must compile it. Erlang provides a command-line compiler (the erlc command), a shortcut to it within the shell (the 'c' command), and a full "make" system that can be used for more complex, automatic builds. There are a few different ways you can layout your code in directories, but for starters, just put things in your current directory, and it should find it. Finally, a bit on syntax. In Erlang, there is a sort of weird syntax based on punctuation. The way that I try to explain it: * the dot (.): This is used to finish a definition. * the comma (,): This is used to separate a series of actions that occur in order. * the semicolon (;): This is used to separate branches, where control could go through one of the branches (but only one) Consider the following: > test(X) when is_atom(X) -> > do_something(), % this is the first in a series of ordered > instructions, use a comma > do_something_else((), % another entry, with a comma > case X of % The case is made up of different branches > first_option -> something0(); % this is one branch > second_option -> % this is another branch, it might run instead > of the first one > something1(), % this is a series of two instructions within > the branch, note the comma > something2(); % note the semicolon, which indicates the next > line is another branch of the case > third_option -> 5 % no punctuation here, as this is the end of > the case > end, > something3(); % this semicolon indicates that there is another > function head to match > test(Y) when is_number(Y) -> > 42. % note the period, this is the end of the statement, defining > the function Hopefully that's not too confusing, but getting the punctuation right is important, if a bit difficult. Good luck. On Oct 10, 2009, at 12:14 PM, Roberto Aloi wrote: > Just use a dot instead of a semicolon at the end of your commands. > > Roberto Aloi > http://erlang-consulting.com > > On 10 Oct 2009, at 14:57, Juan Backson wrote: > >> Hi >> >> Thank you so much for your help. I managed to compile the sample >> code, but >> I can't run it: >> >> [root@REDACTED erlang]# cat math1.erl >> -module(math1). >> -export([factorial/1]). >> factorial(0) -> 1; >> factorial(N) -> N * factorial(N-1). >> >> [root@REDACTED erlang]# ls >> math1.beam math1.erl >> [root@REDACTED erlang]# erl >> Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:2:2] [rq:2] >> [async-threads:0] [hipe] [kernel-poll:false] >> >> Eshell V5.7.3 (abort with ^G) >> 1> math1:factorial(0) >> 1> ; >> 1> math1:factorial(0); >> >> >> What am I missing? >> >> jb >> >> On Sat, Oct 10, 2009 at 9:51 PM, Joe Armstrong >> wrote: >> >>> You cannot type -module(math1). directly into the shell. >>> make a file (math.erl) containing the lines "-module(math1). ..." >>> etc. >>> then compile the module in the shell like this: >>> >>>> c(math1). >>> >>> This will compile the code in math1.erl, then you can run the >>> functions in the shell >>>> math1:funcname(...) etc. >>> >>> /Joe >>> >>> >>> On Sat, Oct 10, 2009 at 3:45 PM, Juan Backson >>> >>> wrote: >>>> Hi, >>>> >>>> I am following the tutorial trying to get started. I am getting >>>> error >>> with >>>> the following sample code in ERL command line. >>>> >>>> 10> -module(math1). >>>> ** exception error: undefined shell command module/1 >>>> 11> -export([factorial/1]). >>>> ** exception error: bad argument in an arithmetic expression >>>> in operator '/'/2 >>>> called as factorial / 1 >>>> >>>> >>>> Could someone help me out ? What is wrong ? >>>> >>>> jb >>>> >>> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From v@REDACTED Wed Oct 14 07:39:42 2009 From: v@REDACTED (Valentin Micic) Date: Wed, 14 Oct 2009 07:39:42 +0200 Subject: [erlang-questions] CPU load of TCP server In-Reply-To: Message-ID: <20091014054002.073C13D0009@mail.pharos-avantgard.com> Hi Andrey, >From your code snippet I suspect that you've opened your socket with {active, true} flag, which may explain excessive CPU usage -- it is much cheaper to keep excess messages in a TCP buffer, rather than as bunch of messages waiting on a process queue (for example, messages in the TCP buffer have no effect on selective receive). Thus, changing your code to: loop(Socket) -> receive {tcp, Socket, _Packet} -> inet:setopts( Socket, [{active, once}] ), loop(Socket); {tcp_closed, Socket} -> normal; _ -> loop(Socket) after 500 -> gen_tcp:send(Socket,[?PACKET]), loop(Socket) end. Should save some CPU cycles... don't forget to open this socket with initial {active, once}. As for performance difference between R12 and R13 -- not sure, but I think there is more tax to be paid for pushing bunch of messages around to a scheduler with a right queue. V/ -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Andrey Tsirulev Sent: 13 October 2009 08:34 PM To: Rapsey; erlang-questions@REDACTED Subject: Re: [erlang-questions] CPU load of TCP server Hi Sergej, Thanks for the hint. I moved timer to client. Now I have about 5.5% of CPU usage per each 1000 connections. I still expect it should be less.. Andrey ----- Original Message ----- From: "Rapsey" To: Sent: Tuesday, October 13, 2009 9:50 PM Subject: Re: [erlang-questions] CPU load of TCP server > Every time after gets executed a timer gets created (I presume). With 10k > processes it probably makes a noticeable CPU impact. > > > Sergej > > 2009/10/13 Andrey Tsirulev > >> Hello all, >> >> I'm exploring the possibility of using Erlang for my TCP service >> application (actually the game server). I've prepared test server and >> client >> applications. The test server application accepts client connections and >> sends 2 small (<1 Kb) packets per second to each client (and receives >> answers). >> >> I've met the following problems: >> 1) Kernel polling doesn't give any benefit with R13B02-1. >> 2) CPU load is too high. >> >> All the details are below. >> >> Here's my test server's `uname -a`: >> Linux source 2.6.29-gentoo-r5 #1 SMP Tue Aug 18 01:15:17 MSD 2009 x86_64 >> AMD Sempron(tm) Dual Core Processor 2200 AuthenticAMD GNU/Linux >> (I've made tests also with 2 other linux servers with different kernel >> versions and results were close). >> >> I've made server connection processes as simple and possible. I've tried >> up >> to 10000 concurrent connections. >> >> Test results didn't not show any visible difference between using >> multiple >> remote machines for client connections, one remote machine or localhost. >> >> I tried R13B02-1 and R12B-5 OTP versions. >> >> I found that memory usage grow is linear, as expected. But I came to the >> problem with CPU load. >> >> First of all, kernel polling didn't give any benefit for R13B02-1 (while >> erlang:system_info(kernel_poll) returned true and erl started with >> message >> [kernel-poll:true]). I've got about 55% of CPU usage with 4000 >> connections >> both with and without kernel polling enabled, while with R12B-5 I have >> about >> 26% of CPU usage with +Ktrue. I suspect a bug either in OTP or in gentoo >> ebuild (of course it's also quite possible that I'm doing something wrong >> or >> missed something in docs). >> >> The following is about R12B-5. I get about 6-7% of CPU load per every >> 1000 >> connections (about 60% CPU load for 10000 connections). I'm not sure if I >> should consider this as a good result or a bad one. Most of the articles >> on >> the same subject say that CPU load is negligible in their tests and they >> are >> fighting for memory only, so I expected I won't be CPU-limited too, but >> evidently I am. >> >> `top` says that about 50% of CPU load is userspace, 25% software >> interrupts, 20% system and 5% hardware interrupts (that's by eye, not >> very >> strict). >> >> I found that CPU load depends not as much on connection count but on >> transmitted packet count (ok, that's obviously the number of system >> calls). >> Thus if I send 4 packets per second, not 2, I should decrease the number >> of >> connections twice to preserve the same CPU load. >> >> CPU load does not depend on packet size. 1 byte or 1Kbyte - no visible >> difference. >> >> CPU usage is slightly less with active socket option enabled than with >> blocking recvs. >> >> CPU usage on the single windows client machine with 4000 connections >> spawned is on the same level as with the linux server handling these 4000 >> connections (while I expected linux to perform better). >> >> Switching Nagle on and off had no effect. I also tried to tune TCP stack >> with sysctl using advises found here and there but almost without any >> effect >> too. >> >> I've tried to trace with fprof and found that bottlenecks are 'send' >> operations (but I'm a relative novice to erlang so I'm not sure my usage >> of >> fprof was correct). Ok, that was expected too. I've read the 'why is >> gen_tcp:send slow?' thread but none of advises given there helped me. >> >> So the main question is: is the CPU usage of 7% per 1000 connections (or >> maybe better say 2000 packets per second) a good result? If no, what is >> the >> expected result? How can I improve my test application? Or maybe >> something >> in my story looks strange? >> >> I know that the possible optimization is decreasing the number of packets >> and keep it in mind. >> >> Here's the server connection process loop: >> >> loop(Socket) -> >> receive >> {tcp, Socket, _Packet} -> >> loop(Socket); >> {tcp_closed, Socket} -> >> normal; >> _ -> >> loop(Socket) >> after 500 -> >> gen_tcp:send(Socket,[?PACKET]), >> loop(Socket) >> end. >> >> Client loop has blocking recv and answers with send immediately. >> >> Thank you very much for your time. Sorry for too many words, I tried to >> provide all possible information. I will answer any question and >> appreciate >> any hint. >> >> Best regards, >> Andrey > ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From kdronnqvist@REDACTED Wed Oct 14 08:31:40 2009 From: kdronnqvist@REDACTED (=?ISO-8859-1?Q?Daniel_R=F6nnqvist?=) Date: Wed, 14 Oct 2009 08:31:40 +0200 Subject: No bridge problem with nl80211 Message-ID: <65691fa00910132331j6c4bf849l20375bde883cd557@mail.gmail.com> Hello, I feel a bit stupid now =) The problem I specified in yesterdays post "Hostapd bridge problem using nl80211" is showing itself even when running without a bridge. So my questions are: * Is hostapd from git supposed to work with nl80211 (with a bcm4306 rev3)? * What does "deauthenticated due to local death" mean? * The only messages i see with tshark during the authentication process is "EAPOL Key" and some "LLC" messages coming from the client MAC. The hostapd log says EAPOL Key Timeout on handshake 1/4. Is this enough to guess what ballpark my problem is in? //Daniel From kagato@REDACTED Wed Oct 14 08:49:15 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Wed, 14 Oct 2009 01:49:15 -0500 Subject: [erlang-questions] newbie needing help getting started In-Reply-To: <9919866E-D828-4A4A-A2EB-C46FD5EB6E53@souja.net> References: <27c25bc40910100645t2e2b4d81yfbbc85741723bb98@mail.gmail.com> <9b08084c0910100651j247116edicfe782501137d42e@mail.gmail.com> <27c25bc40910100657l6afe61aehe6849ea58cdb6c8d@mail.gmail.com> <634A53E4-FBDB-46E8-9771-26811E2F4DA6@erlang-consulting.com> <9919866E-D828-4A4A-A2EB-C46FD5EB6E53@souja.net> Message-ID: <6C17CC84-62F1-4404-9F92-351FC8B9418A@souja.net> On Oct 14, 2009, at 1:07 AM, Jayson Vantuyl wrote: > To see a complete, if empty, example of an Erlang build environment, > you can use git to checkout my skeleton deployment directory. It's > a work in progress, but it should be close. Oops. A link would be good. See here: http://github.com/jvantuyl/erl-skel You can use git to clone a copy with "git clone git://github.com/ jvantuyl/erl-skel.git". Assuming you have git, that is. This directory layout is pretty much a working Erlang environment. I have included (hopefully) helpful scripts for working with it. Look in README.skel for details. -- Jayson Vantuyl kagato@REDACTED From roberto.aloi@REDACTED Wed Oct 14 11:34:53 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Wed, 14 Oct 2009 10:34:53 +0100 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <4AD4C3DB.2000408@research.att.com> References: <4AD4C3DB.2000408@research.att.com> Message-ID: <4AD59B3D.50206@erlang-consulting.com> This is due to the dot notation used in inet_addr(). Reading from the doc: Values specified using the dot notation take one of the following forms: */a/*.*/b/*.*/c/*.*/d/* */a/*.*/b/*.*/c/* */a/*.*/b/* */a/* Each of the four notation types are described below. * */a/*.*/b/*.*/c/*.*/d/* notation When four parts are specified, each is interpreted as a byte of data and assigned, from left to right, to the four bytes of an Internet address. * */a/*.*/b/*.*/c/* notation When a three-part address is specified, the last part is interpreted as a 16-bit quantity and placed in the right most two bytes of the network address. This makes the three-part address format convenient for specifying Class B network addresses as 128.*/net/*.*/host/*. * */a/*.*/b/* notation When a two-part address is supplied, the last part is interpreted as a 24-bit quantity and placed in the right most three bytes of the network address. This makes the two-part address format convenient for specifying Class A network addresses as */net/*.*/host/*. * */a/* notation When only one part is given, the value is stored directly in the network address without any byte rearrangement. All numbers supplied as parts in dot notation may be decimal, octal, or hexadecimal, as specified in the C language (that is, a leading 0x or 0X implies hexadecimal; otherwise, a leading 0 implies octal; otherwise, the number is interpreted as decimal). Sources: http://www.ietf.org/rfc/rfc3493.txt http://uw714doc.sco.com/en/man/html.3N/inet.3N.html Best regards, Roberto Aloi Erlang Training and Consulting Ltd. http://www.erlang-consulting.com http://aloiroberto.wordpress.com Garry Hodgson wrote: > recently, a bug in my code caused us to pass a string > representing a floating point number to inet:gethostbyname(). > i would have expected it to return an error, but instead it > returned an ip address, but one that made no sense to me: > > 1> inet:gethostbyname( '12.27' ). > {ok,{hostent,"12.27",[],inet,4,[{12,0,0,27}]}} > > so my question is, is this behavior correct, and if so, what > exactly does it mean that a lookup of '12.27' maps to '12.0.0.27'? > > thanks > From rtrlists@REDACTED Wed Oct 14 12:14:46 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 14 Oct 2009 11:14:46 +0100 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <6a9ba5690910131206v11245896kb8088aa9c335bf9f@mail.gmail.com> References: <4AD4C3DB.2000408@research.att.com> <6a9ba5690910131206v11245896kb8088aa9c335bf9f@mail.gmail.com> Message-ID: <6a3ae47e0910140314q1c8d1490x595a5bbfa994f342@mail.gmail.com> This is part of the IP address spec (not sure where to look to find it, though). I've seen addresses specified like this on and off in various locations on a wide variety of OSes, for example in a /etc/hosts file you can write: localhost 127.1 Robby On Tue, Oct 13, 2009 at 8:06 PM, caio ariede wrote: > Interesting case. > > The result doesn't appear to be an issue, but a feature of the > gethostbyname > original implementation, in C code. > > You get the same result, testing with PHP: > > $ php -r 'var_dump(gethostbyname("12.27"));' > string(9) "12.0.0.27" > > And other interesting results: > > $ php -r 'var_dump(gethostbyname("255.2.256"));' > string(9) "255.2.1.0" > > But I can't see where it's really useful. > > Caio Ariede > http://caioariede.com/ > > > On Tue, Oct 13, 2009 at 3:15 PM, Garry Hodgson >wrote: > > > recently, a bug in my code caused us to pass a string > > representing a floating point number to inet:gethostbyname(). > > i would have expected it to return an error, but instead it > > returned an ip address, but one that made no sense to me: > > > > 1> inet:gethostbyname( '12.27' ). > > {ok,{hostent,"12.27",[],inet,4,[{12,0,0,27}]}} > > > > so my question is, is this behavior correct, and if so, what > > exactly does it mean that a lookup of '12.27' maps to '12.0.0.27'? > > > > thanks > > > > -- > > Garry Hodgson > > Lead Member of Technical Staff > > AT&T Chief Security Office (CSO) > > > > "This e-mail and any files transmitted with it are AT&T property, are > > confidential, and are intended solely for the use of the individual or > > entity to whom this e-mail is addressed. If you are not one of the named > > recipient(s) or otherwise have reason to believe that you have received > this > > message in error, please notify the sender and delete this message > > immediately from your computer. Any other use, retention, dissemination, > > forwarding, printing, or copying of this e-mail is strictly prohibited." > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > From roberto.aloi@REDACTED Wed Oct 14 12:20:34 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Wed, 14 Oct 2009 11:20:34 +0100 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <6a3ae47e0910140314q1c8d1490x595a5bbfa994f342@mail.gmail.com> References: <4AD4C3DB.2000408@research.att.com> <6a9ba5690910131206v11245896kb8088aa9c335bf9f@mail.gmail.com> <6a3ae47e0910140314q1c8d1490x595a5bbfa994f342@mail.gmail.com> Message-ID: <4AD5A5F2.3020407@erlang-consulting.com> At the end of my last mail you have the references. Regards, Roberto Aloi Erlang Training and Consulting Ltd. http://www.erlang-consulting.com http://aloiroberto.wordpress.com Robert Raschke wrote: > This is part of the IP address spec (not sure where to look to find it, > though). I've seen addresses specified like this on and off in various > locations on a wide variety of OSes, for example in a /etc/hosts file you > can write: > > localhost 127.1 > > Robby > > On Tue, Oct 13, 2009 at 8:06 PM, caio ariede wrote: > > >> Interesting case. >> >> The result doesn't appear to be an issue, but a feature of the >> gethostbyname >> original implementation, in C code. >> >> You get the same result, testing with PHP: >> >> $ php -r 'var_dump(gethostbyname("12.27"));' >> string(9) "12.0.0.27" >> >> And other interesting results: >> >> $ php -r 'var_dump(gethostbyname("255.2.256"));' >> string(9) "255.2.1.0" >> >> But I can't see where it's really useful. >> >> Caio Ariede >> http://caioariede.com/ >> >> >> On Tue, Oct 13, 2009 at 3:15 PM, Garry Hodgson > >>> wrote: >>> >>> recently, a bug in my code caused us to pass a string >>> representing a floating point number to inet:gethostbyname(). >>> i would have expected it to return an error, but instead it >>> returned an ip address, but one that made no sense to me: >>> >>> 1> inet:gethostbyname( '12.27' ). >>> {ok,{hostent,"12.27",[],inet,4,[{12,0,0,27}]}} >>> >>> so my question is, is this behavior correct, and if so, what >>> exactly does it mean that a lookup of '12.27' maps to '12.0.0.27'? >>> >>> thanks >>> >>> -- >>> Garry Hodgson >>> Lead Member of Technical Staff >>> AT&T Chief Security Office (CSO) >>> >>> "This e-mail and any files transmitted with it are AT&T property, are >>> confidential, and are intended solely for the use of the individual or >>> entity to whom this e-mail is addressed. If you are not one of the named >>> recipient(s) or otherwise have reason to believe that you have received >>> >> this >> >>> message in error, please notify the sender and delete this message >>> immediately from your computer. Any other use, retention, dissemination, >>> forwarding, printing, or copying of this e-mail is strictly prohibited." >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >>> >>> > > From raimo+erlang-questions@REDACTED Wed Oct 14 12:33:59 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 14 Oct 2009 12:33:59 +0200 Subject: [erlang-questions] SCTP PPID change In-Reply-To: References: Message-ID: <20091014103359.GA29456@erix.ericsson.se> On Wed, Oct 14, 2009 at 03:09:33AM +0400, Ahmed Ali wrote: > Hi, > > I'm experminting with SCTP support in Erlang. I'm using Solaris 10 > with Erlang R13B02. SCTP support is enabled during compiling Erlang. > > I'm having problem with changing ppid to 3 (M3UA). The first thing I > did is to add sctp_default_send_param in gen_sctp:connect/4. However, > I got the error below. I've tried to used gen_sctp:send/3 instead of > send/4 as below but I get exactly the same error. I also tried to > troubleshoot the issue and found out the error is generated from > module prim_inet.erl in erts code. My guess is that type_value/3 > function checks that the values of the supplied #sctp_sndrcvinfo{} are > valid but appearntly, something is not correct in #sctp_sndrcvinfo{} > record I'm sending. > > Without setting PPID, everything works and I could see traffic is sent > and received correctly. Any help is appreciated. > > gen_sctp:send(S, #sctp_sndrcvinfo{stream=Stream, > assoc_id=Assoc, > ppid=3}, Data) I guess this is what you ment: gen_sctp:send(S, #sctp_sndrcvinfo{stream=Stream, assoc_id=Assoc#sctp_assoc_change.assoc_id, ppid=3}, Data) The shortcut of using #sctp_assoc_change{} as assoc_id in gen_sctp:send/4 is not appropriate for gen_sctp:send/3. Argument 2 has to be a #sctp_sndrcvinfo{} and #sctp_sndrcvinfo.assoc_id has to be an integer. Read inet_sctp.hrl. > > {error, > {type, > {record,set, > {sctp_sndrcvinfo, > [uint16,0], > [], > [{bitenumlist, > [{unordered,1},{addr_over,2},{abort,4},{eof,8}], > uint16}, > []], > [uint32,0], > [uint32,0], > [uint32,0], > [],[], ***> [sctp_assoc_id,0]}, > {sctp_sndrcvinfo,0,undefined,undefined,3,undefined,undefined, > undefined,undefined, ***> {sctp_assoc_change,comm_up,0,5,17,1}}, > 10 > } > } > } *** Type mismatch > > Best regards, > > Ahmed > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From ahmed.nawras@REDACTED Wed Oct 14 13:12:36 2009 From: ahmed.nawras@REDACTED (Ahmed Ali) Date: Wed, 14 Oct 2009 15:12:36 +0400 Subject: [erlang-questions] SCTP PPID change In-Reply-To: <20091014103359.GA29456@erix.ericsson.se> References: <20091014103359.GA29456@erix.ericsson.se> Message-ID: Hi Raimo, Thanks a lot for your help. The issue was due to assoc_id in #sctp_sndrcvinfo{} not being integer. Best regards, Ahmed On Wed, Oct 14, 2009 at 2:33 PM, Raimo Niskanen wrote: > On Wed, Oct 14, 2009 at 03:09:33AM +0400, Ahmed Ali wrote: >> Hi, >> >> I'm experminting with SCTP support in Erlang. I'm using Solaris 10 >> with Erlang R13B02. SCTP support is enabled during compiling Erlang. >> >> I'm having problem with changing ppid to 3 (M3UA). The first thing I >> did is to add sctp_default_send_param in gen_sctp:connect/4. However, >> I got the error below. I've tried to used gen_sctp:send/3 instead of >> send/4 as below but I get exactly the same error. I also tried to >> troubleshoot the issue and found out the error is generated from >> module prim_inet.erl in erts code. My guess is that type_value/3 >> function checks that the values of the supplied #sctp_sndrcvinfo{} are >> valid but appearntly, something is not correct in #sctp_sndrcvinfo{} >> record I'm sending. >> >> Without setting PPID, everything works and I could see traffic is sent >> and received correctly. Any help is appreciated. >> >> gen_sctp:send(S, #sctp_sndrcvinfo{stream=Stream, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? assoc_id=Assoc, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ppid=3}, Data) > I guess this is what you ment: > > gen_sctp:send(S, > ? ?#sctp_sndrcvinfo{stream=Stream, > ? ? ? ? ? ? ? ? ? ? assoc_id=Assoc#sctp_assoc_change.assoc_id, > ? ? ? ? ? ? ? ? ? ? ppid=3}, Data) > > The shortcut of using #sctp_assoc_change{} as assoc_id > in gen_sctp:send/4 is not appropriate for gen_sctp:send/3. > Argument 2 has to be a #sctp_sndrcvinfo{} and > #sctp_sndrcvinfo.assoc_id has to be an integer. > Read inet_sctp.hrl. > >> >> {error, >> ? ? ? {type, >> ? ? ? ? ? {record,set, >> ? ? ? ? ? ? ? {sctp_sndrcvinfo, >> ? ? ? ? ? ? ? ? ? [uint16,0], >> ? ? ? ? ? ? ? ? ? [], >> ? ? ? ? ? ? ? ? ? [{bitenumlist, >> ? ? ? ? ? ? ? ? ? ? ? ?[{unordered,1},{addr_over,2},{abort,4},{eof,8}], >> ? ? ? ? ? ? ? ? ? ? ? ?uint16}, >> ? ? ? ? ? ? ? ? ? ?[]], >> ? ? ? ? ? ? ? ? ? [uint32,0], >> ? ? ? ? ? ? ? ? ? [uint32,0], >> ? ? ? ? ? ? ? ? ? [uint32,0], >> ? ? ? ? ? ? ? ? ? [],[], > ***> ? ? ? ? ? ? ? ? ? [sctp_assoc_id,0]}, >> ? ? ? ? ? ? ? {sctp_sndrcvinfo,0,undefined,undefined,3,undefined,undefined, >> ? ? ? ? ? ? ? ? ? undefined,undefined, > ***> ? ? ? ? ? ? ? ? ? {sctp_assoc_change,comm_up,0,5,17,1}}, >> ? ? ? ? ? ? ? 10 >> ? ? ? ? } >> ? ? ? ?} >> } > > *** Type mismatch > >> >> Best regards, >> >> Ahmed >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > From andrey@REDACTED Wed Oct 14 14:07:08 2009 From: andrey@REDACTED (Andrey Tsirulev) Date: Wed, 14 Oct 2009 16:07:08 +0400 Subject: [erlang-questions] CPU load of TCP server References: <20091014054002.073C13D0009@mail.pharos-avantgard.com> Message-ID: <38EE6A55D40043A897EF9AB057BDB828@thehangedman> Hi Valentin, Right, but in this particular case it doesn't give any benefit as there's only one packet per 500ms for each process which is handled at once before the next packet comes, so tests showed no difference between these 2 cases (though I've got your point and will use {active, once} in production). Andrey ----- Original Message ----- From: "Valentin Micic" To: "'Andrey Tsirulev'" ; "'Rapsey'" ; Sent: Wednesday, October 14, 2009 9:39 AM Subject: RE: [erlang-questions] CPU load of TCP server > Hi Andrey, > > From your code snippet I suspect that you've opened your socket with > {active, true} flag, which may explain excessive CPU usage -- it is much > cheaper to keep excess messages in a TCP buffer, rather than as bunch of > messages waiting on a process queue (for example, messages in the TCP > buffer > have no effect on selective receive). > > Thus, changing your code to: > > loop(Socket) -> > receive > {tcp, Socket, _Packet} -> > inet:setopts( Socket, [{active, once}] ), > loop(Socket); > {tcp_closed, Socket} -> > normal; > _ -> > loop(Socket) > after 500 -> > gen_tcp:send(Socket,[?PACKET]), > loop(Socket) > end. > > Should save some CPU cycles... don't forget to open this socket with > initial > {active, once}. > > As for performance difference between R12 and R13 -- not sure, but I think > there is more tax to be paid for pushing bunch of messages around to a > scheduler with a right queue. > > V/ > > > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Andrey Tsirulev > Sent: 13 October 2009 08:34 PM > To: Rapsey; erlang-questions@REDACTED > Subject: Re: [erlang-questions] CPU load of TCP server > > Hi Sergej, > > Thanks for the hint. I moved timer to client. Now I have about 5.5% of CPU > usage per each 1000 connections. I still expect it should be less.. > > Andrey > > > ----- Original Message ----- > From: "Rapsey" > To: > Sent: Tuesday, October 13, 2009 9:50 PM > Subject: Re: [erlang-questions] CPU load of TCP server > > >> Every time after gets executed a timer gets created (I presume). With 10k >> processes it probably makes a noticeable CPU impact. >> >> >> Sergej >> >> 2009/10/13 Andrey Tsirulev >> >>> Hello all, >>> >>> I'm exploring the possibility of using Erlang for my TCP service >>> application (actually the game server). I've prepared test server and >>> client >>> applications. The test server application accepts client connections and >>> sends 2 small (<1 Kb) packets per second to each client (and receives >>> answers). >>> >>> I've met the following problems: >>> 1) Kernel polling doesn't give any benefit with R13B02-1. >>> 2) CPU load is too high. >>> >>> All the details are below. >>> >>> Here's my test server's `uname -a`: >>> Linux source 2.6.29-gentoo-r5 #1 SMP Tue Aug 18 01:15:17 MSD 2009 x86_64 >>> AMD Sempron(tm) Dual Core Processor 2200 AuthenticAMD GNU/Linux >>> (I've made tests also with 2 other linux servers with different kernel >>> versions and results were close). >>> >>> I've made server connection processes as simple and possible. I've tried >>> up >>> to 10000 concurrent connections. >>> >>> Test results didn't not show any visible difference between using >>> multiple >>> remote machines for client connections, one remote machine or localhost. >>> >>> I tried R13B02-1 and R12B-5 OTP versions. >>> >>> I found that memory usage grow is linear, as expected. But I came to the >>> problem with CPU load. >>> >>> First of all, kernel polling didn't give any benefit for R13B02-1 (while >>> erlang:system_info(kernel_poll) returned true and erl started with >>> message >>> [kernel-poll:true]). I've got about 55% of CPU usage with 4000 >>> connections >>> both with and without kernel polling enabled, while with R12B-5 I have >>> about >>> 26% of CPU usage with +Ktrue. I suspect a bug either in OTP or in gentoo >>> ebuild (of course it's also quite possible that I'm doing something >>> wrong > >>> or >>> missed something in docs). >>> >>> The following is about R12B-5. I get about 6-7% of CPU load per every >>> 1000 >>> connections (about 60% CPU load for 10000 connections). I'm not sure if >>> I >>> should consider this as a good result or a bad one. Most of the articles >>> on >>> the same subject say that CPU load is negligible in their tests and they >>> are >>> fighting for memory only, so I expected I won't be CPU-limited too, but >>> evidently I am. >>> >>> `top` says that about 50% of CPU load is userspace, 25% software >>> interrupts, 20% system and 5% hardware interrupts (that's by eye, not >>> very >>> strict). >>> >>> I found that CPU load depends not as much on connection count but on >>> transmitted packet count (ok, that's obviously the number of system >>> calls). >>> Thus if I send 4 packets per second, not 2, I should decrease the number >>> of >>> connections twice to preserve the same CPU load. >>> >>> CPU load does not depend on packet size. 1 byte or 1Kbyte - no visible >>> difference. >>> >>> CPU usage is slightly less with active socket option enabled than with >>> blocking recvs. >>> >>> CPU usage on the single windows client machine with 4000 connections >>> spawned is on the same level as with the linux server handling these >>> 4000 >>> connections (while I expected linux to perform better). >>> >>> Switching Nagle on and off had no effect. I also tried to tune TCP stack >>> with sysctl using advises found here and there but almost without any >>> effect >>> too. >>> >>> I've tried to trace with fprof and found that bottlenecks are 'send' >>> operations (but I'm a relative novice to erlang so I'm not sure my usage >>> of >>> fprof was correct). Ok, that was expected too. I've read the 'why is >>> gen_tcp:send slow?' thread but none of advises given there helped me. >>> >>> So the main question is: is the CPU usage of 7% per 1000 connections (or >>> maybe better say 2000 packets per second) a good result? If no, what is >>> the >>> expected result? How can I improve my test application? Or maybe >>> something >>> in my story looks strange? >>> >>> I know that the possible optimization is decreasing the number of >>> packets >>> and keep it in mind. >>> >>> Here's the server connection process loop: >>> >>> loop(Socket) -> >>> receive >>> {tcp, Socket, _Packet} -> >>> loop(Socket); >>> {tcp_closed, Socket} -> >>> normal; >>> _ -> >>> loop(Socket) >>> after 500 -> >>> gen_tcp:send(Socket,[?PACKET]), >>> loop(Socket) >>> end. >>> >>> Client loop has blocking recv and answers with send immediately. >>> >>> Thank you very much for your time. Sorry for too many words, I tried to >>> provide all possible information. I will answer any question and >>> appreciate >>> any hint. >>> >>> Best regards, >>> Andrey >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From andrey@REDACTED Wed Oct 14 14:15:53 2009 From: andrey@REDACTED (Andrey Tsirulev) Date: Wed, 14 Oct 2009 16:15:53 +0400 Subject: [erlang-questions] CPU load of TCP server References: <97619b170910131050g2632f794q46b45ad3cd48f831@mail.gmail.com> Message-ID: <366DCE9C08484F4BA2CFBA5989DB5382@thehangedman> Hello Max, Well, I cannot agree syntetic tests are so useless as they can show at least theoretical limit for my conditions (before I even started to implement real business logic). I'm not sure I understood the opposition of erlang virtual machine to erlang environment, could you explain it, please? Now I prepare a release and start erlang with a boot script, what is another approach? Andrey ----- Original Message ----- From: "Max Lapshin" To: "Andrey Tsirulev" Cc: "Rapsey" ; Sent: Wednesday, October 14, 2009 12:03 AM Subject: Re: [erlang-questions] CPU load of TCP server > Andrew, You should test against real business logic, because syntetic > tests don't show anything at all. > > While experimenting with video streaming server, I've got strange > results: 5 users used the same CPU, like 30. > So I advice to test not erlang virtual machine (which isn't fastest > one), but erlang environment (which can > be faster due to advanced binary usage and scalability possibilities). From sapan.gcet@REDACTED Wed Oct 14 15:09:12 2009 From: sapan.gcet@REDACTED (sapan shah) Date: Wed, 14 Oct 2009 18:39:12 +0530 Subject: Calling a Function with Same argument twice Message-ID: Hi, see the code below. %%%%%%%%%% -module(temp). -compile(export_all). start(X) -> put(test, 0), case X of 1 -> test(2, 2); 2-> test1(2) end, get(test). test(0,0) -> put(test, get(test) +1), {0,0}; test(X,Y) -> {element(1,test(X-1,Y-1)),element(2,test(X-1,Y-1))}. test1(0) -> put(test, get(test) +1), {0}; test1(X) -> {element(1,test1(X-1))}. %%%%%%% We call temp:start(1), the output is 4. The output should have been 1. Here, when start(1) is called, it intern calls {element(1, test(1,1), element(2, test(1,1)} The first call of test(1,1) should recursively evaluate test(1,1) while the second call should already use the value returned by the first call.. Is this a right thinking??? -- Regards, Sapan Shah From ulf.wiger@REDACTED Wed Oct 14 15:31:25 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 14 Oct 2009 15:31:25 +0200 Subject: [erlang-questions] Calling a Function with Same argument twice In-Reply-To: References: Message-ID: <4AD5D2AD.2030604@erlang-consulting.com> sapan shah wrote: > Hi, > > see the code below. > ... > > We call temp:start(1), the output is 4. The output should have been 1. No, 4 is right. > Here, when start(1) is called, it intern calls > {element(1, test(1,1), element(2, test(1,1)} test(2,2) results in two calls to test(1,1), which each result in two calls to test(0,0), each incrementing the 'test' object by 1. > > The first call of test(1,1) should recursively evaluate test(1,1) while the > second call should already use the value returned by the first call.. Is > this a right thinking??? I assume you're thinking that it would work like: test(0,0) -> put(test, get(test) +1), {0,0}; test(X,Y) -> Z = test(X-1, Y-1), {element(1, Z), element(2, Z)}. but here you make only one call and reuse the result. If the test/2 function werer purely functional (i.e. no side- effects), the two would be equivalent, but here you use get and put, so it really makes a difference how many times you explicitly call the function. Btw, since test/2 returns a two-tuple, the above is of course equivalent to test(0,0) -> put(test, get(test) +1), {0,0}; test(X,Y) -> test(X-1, Y-1). ...which is almost equivalent to test(X,X) when is_integer(X) -> put(test, get(test) +1), {0,0}. (It differs only slightly in how it fails on erroneous input). Think twice before using get/put. Also, why wrap the return values inside a 1-tuple? Remove the tuple, and you can remove some of the calls to element/2. But of course it was a contrived example, so I assume the real problem you're trying to solve is somewhat different? BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From info@REDACTED Wed Oct 14 15:34:12 2009 From: info@REDACTED (info) Date: Wed, 14 Oct 2009 15:34:12 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? References: <4AD4C3DB.2000408@research.att.com>, <6a9ba5690910131206v11245896kb8088aa9c335bf9f@mail.gmail.com>, <6a3ae47e0910140314q1c8d1490x595a5bbfa994f342@mail.gmail.com>, <4AD5A5F2.3020407@erlang-consulting.com> Message-ID: <200910141534121608248@its3.ch> For me this function always returns {error,timeout} and I didn't find the reason even with the help of Raimo Niskanen ! At the end of my last mail you have the references. Regards, Roberto Aloi Erlang Training and Consulting Ltd. http://www.erlang-consulting.com http://aloiroberto.wordpress.com Robert Raschke wrote: > This is part of the IP address spec (not sure where to look to find it, > though). I've seen addresses specified like this on and off in various > locations on a wide variety of OSes, for example in a /etc/hosts file you > can write: > > localhost 127.1 > > Robby > > On Tue, Oct 13, 2009 at 8:06 PM, caio ariede wrote: > > > > Interesting case. > > > > The result doesn't appear to be an issue, but a feature of the > > gethostbyname > > original implementation, in C code. > > > > You get the same result, testing with PHP: > > > > $ php -r 'var_dump(gethostbyname("12.27"));' > > string(9) "12.0.0.27" > > > > And other interesting results: > > > > $ php -r 'var_dump(gethostbyname("255.2.256"));' > > string(9) "255.2.1.0" > > > > But I can't see where it's really useful. > > > > Caio Ariede > > http://caioariede.com/ > > > > > > On Tue, Oct 13, 2009 at 3:15 PM, Garry Hodgson > > > > wrote: > > > > > > recently, a bug in my code caused us to pass a string > > > representing a floating point number to inet:gethostbyname(). > > > i would have expected it to return an error, but instead it > > > returned an ip address, but one that made no sense to me: > > > > > > 1 > inet:gethostbyname( '12.27' ). > > > {ok,{hostent,"12.27",[],inet,4,[{12,0,0,27}]}} > > > > > > so my question is, is this behavior correct, and if so, what > > > exactly does it mean that a lookup of '12.27' maps to '12.0.0.27'? > > > > > > thanks > > > > > > -- > > > Garry Hodgson > > > Lead Member of Technical Staff > > > AT&T Chief Security Office (CSO) > > > > > > "This e-mail and any files transmitted with it are AT&T property, are > > > confidential, and are intended solely for the use of the individual or > > > entity to whom this e-mail is addressed. If you are not one of the named > > > recipient(s) or otherwise have reason to believe that you have received > > > > > this > > > > > message in error, please notify the sender and delete this message > > > immediately from your computer. Any other use, retention, dissemination, > > > forwarding, printing, or copying of this e-mail is strictly prohibited." > > > > > > ________________________________________________________________ > > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > > erlang-questions (at) erlang.org > > > > > > > > > > > ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From dmercer@REDACTED Wed Oct 14 15:52:42 2009 From: dmercer@REDACTED (David Mercer) Date: Wed, 14 Oct 2009 08:52:42 -0500 Subject: [erlang-questions] Calling a Function with Same argument twice In-Reply-To: References: Message-ID: sapan shah wrote: > test(X,Y) -> > {element(1,test(X-1,Y-1)),element(2,test(X-1,Y-1))}. . . . > Here, when start(1) is called, it intern calls > {element(1, test(1,1), element(2, test(1,1)} > > The first call of test(1,1) should recursively evaluate test(1,1) while > the > second call should already use the value returned by the first call.. Is > this a right thinking??? No, test(X,Y) is evaluating test(X-1,Y-1) twice. If you wanted it to only evaluate it once, you could write: test(X,Y) -> Test = test(X-1,Y-1), {element(1,Test), element(2,Test)} Even in Haskell, you would have to evaluate test(X-1,Y-1) twice, since there are side-effects (the put call in test(0,0)), which would require it to be wrapped in a monad. (I'm not a Haskell expert, never having written a line of it in my life, so feel free to correct me if my understanding of the language is wrong.) Cheers, David > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of sapan shah > Sent: Wednesday, October 14, 2009 8:09 AM > To: erlang-questions > Subject: [erlang-questions] Calling a Function with Same argument twice > > Hi, > > see the code below. > %%%%%%%%%% > -module(temp). > -compile(export_all). > > > start(X) -> > put(test, 0), > case X of 1 -> test(2, 2); 2-> test1(2) end, > get(test). > > > test(0,0) -> > put(test, get(test) +1), > {0,0}; > > test(X,Y) -> > {element(1,test(X-1,Y-1)),element(2,test(X-1,Y-1))}. > > > test1(0) -> > put(test, get(test) +1), > {0}; > test1(X) -> > {element(1,test1(X-1))}. > %%%%%%% > > We call temp:start(1), the output is 4. The output should have been 1. > Here, when start(1) is called, it intern calls > {element(1, test(1,1), element(2, test(1,1)} > > The first call of test(1,1) should recursively evaluate test(1,1) while > the > second call should already use the value returned by the first call.. Is > this a right thinking??? > -- > Regards, > Sapan Shah From garry@REDACTED Wed Oct 14 16:03:22 2009 From: garry@REDACTED (Garry Hodgson) Date: Wed, 14 Oct 2009 10:03:22 -0400 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <4AD59B3D.50206@erlang-consulting.com> References: <4AD4C3DB.2000408@research.att.com> <4AD59B3D.50206@erlang-consulting.com> Message-ID: <4AD5DA2A.1010207@research.att.com> Roberto Aloi Roberto Aloi wrote: > This is due to the dot notation used in inet_addr(). > > Reading from the doc: > Values specified using the dot notation take one of the following forms: > > */a/*.*/b/*.*/c/*.*/d/* > */a/*.*/b/*.*/c/* > */a/*.*/b/* > */a/* thanks to roberto and all the others who clarified this for me. -- Garry Hodgson Lead Member of Technical Staff AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From rvirding@REDACTED Wed Oct 14 16:45:10 2009 From: rvirding@REDACTED (Robert Virding) Date: Wed, 14 Oct 2009 16:45:10 +0200 Subject: [erlang-questions] newbie needing help getting started In-Reply-To: <9919866E-D828-4A4A-A2EB-C46FD5EB6E53@souja.net> References: <27c25bc40910100645t2e2b4d81yfbbc85741723bb98@mail.gmail.com> <9b08084c0910100651j247116edicfe782501137d42e@mail.gmail.com> <27c25bc40910100657l6afe61aehe6849ea58cdb6c8d@mail.gmail.com> <634A53E4-FBDB-46E8-9771-26811E2F4DA6@erlang-consulting.com> <9919866E-D828-4A4A-A2EB-C46FD5EB6E53@souja.net> Message-ID: <3dbc6d1c0910140745i774b9ae5jedcfdc23a46a3bfe@mail.gmail.com> 2009/10/14 Jayson Vantuyl > > Finally, a bit on syntax. In Erlang, there is a sort of weird syntax based > on punctuation. The way that I try to explain it: > > * the dot (.): This is used to finish a definition. > * the comma (,): This is used to separate a series of actions that occur > in order. > * the semicolon (;): This is used to separate branches, where control could > go through one of the branches (but only one) > It's not weird, just different from C/Java. Look at (,) and (;) as sequential operators (which they originally were) and it becomes much clearer and simpler. (,) is first evaluate the lhs and then evaluate the rhs, while (;) is first try the lh clause and if that fails then try the rh clause. That is why there is no (;) after the last clause. So erlang has no terminators like C/Java (;). Look on the bright side, you don't have to wrap all blocks in begin ... end which you do in C/Java. Robert From discipleofranok@REDACTED Wed Oct 14 17:13:49 2009 From: discipleofranok@REDACTED (Jacob Torrey) Date: Wed, 14 Oct 2009 11:13:49 -0400 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <4AD417B8.8000702@erlang-consulting.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> <4AD38779.3040700@gmail.com> <4AD417B8.8000702@erlang-consulting.com> Message-ID: <4AD5EAAD.2000007@gmail.com> Ulf, If rex a required process? Is there any way to turn it off, or replace it with a module of my choosing? Could you please point me to a reference on rex, all I can find is a passing reference in erl_call. Jacob Ulf Wiger wrote: > Jacob Torrey wrote: >> Could perhaps the pid generation in the VM be made more random, >> therefore you can only gain access to pids that you know the name to? >> This seems like an obvious (and pretty trivial) solution to obscure and >> increase the security of the Erlang system. > > You don't even need to bother with the pids: > > {rex,Node} ! {'$gen_cast',{cast,M,F,A,user}} > > ...will allow you to execute any function on Node. > > BR, > Ulf W From ulf.wiger@REDACTED Wed Oct 14 17:30:58 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 14 Oct 2009 17:30:58 +0200 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <4AD5EAAD.2000007@gmail.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> <4AD38779.3040700@gmail.com> <4AD417B8.8000702@erlang-consulting.com> <4AD5EAAD.2000007@gmail.com> Message-ID: <4AD5EEB2.7030905@erlang-consulting.com> Jacob Torrey wrote: > Ulf, > If rex a required process? Is there any way to turn it off, or replace > it with a module of my choosing? Could you please point me to a > reference on rex, all I can find is a passing reference in erl_call. > > Jacob Rex is the registered name of the rpc server. The one way I know of to turn it off is to start erlang with erl -mode minimal Not really sure where -mode minimal is documented, though, but rpc is not all it turns off - you get no Distributed Erlang at all. So I don't think that solves your problem. You should see this as an example of how difficult it is to make Erlang safe if you allow remote processes to send messages. Since any message can be sent to any process, 'rex' is just the most obvious vulnerability to exploit. You are much better off setting up a narrow pipe and allowing only a small set of messages. Since most communication is wrapped inside function calls, it really isn't as limiting as you might suspect at first. BR, Ulf W > > Ulf Wiger wrote: >> Jacob Torrey wrote: >>> Could perhaps the pid generation in the VM be made more random, >>> therefore you can only gain access to pids that you know the name to? >>> This seems like an obvious (and pretty trivial) solution to obscure and >>> increase the security of the Erlang system. >> You don't even need to bother with the pids: >> >> {rex,Node} ! {'$gen_cast',{cast,M,F,A,user}} >> >> ...will allow you to execute any function on Node. >> >> BR, >> Ulf W > -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From max.lapshin@REDACTED Wed Oct 14 17:33:19 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 14 Oct 2009 19:33:19 +0400 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <4AD5EAAD.2000007@gmail.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> <4AD38779.3040700@gmail.com> <4AD417B8.8000702@erlang-consulting.com> <4AD5EAAD.2000007@gmail.com> Message-ID: Damn, it is impossible to establish security through obscurity. Just imagine how funny would be, if any open file in operation system has its unique file id and if you know this file descriptor, you can write there. Yes, Erlang lacks secure remoting, like any platform lacks. One of the easiest way to build it is HTTP-based service. From kamiseq@REDACTED Wed Oct 14 18:54:03 2009 From: kamiseq@REDACTED (kamiseq) Date: Wed, 14 Oct 2009 18:54:03 +0200 Subject: tcp connection with timeout Message-ID: hi, is there a way to discard any message from peer after set timeout occurs? A server should respond in given time or transmission should be repeated. I implemented small ex where one process is listening and responding in random time (let say 1 to 3 sek). and it is always accepting exactly one process during its life. other process is connecting via tcp and is sending msg. I set send_timeout flag for socket and a timer on receive after to let say 500ms and 1000ms so it should get timeout all the time. when I receive timeout I send special msg that server should repeat last response. all this is great but then I will receive two responses from server with timeouted and then repeated msg. in fact Im receiving all "lost" messages that were sent to my client by server. I could maybe add some ticket to the msg and take out any incoming messages that are not matching but this is impossible right now (afac) as I will break protocol. -- pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From yogishb@REDACTED Wed Oct 14 18:25:38 2009 From: yogishb@REDACTED (Yogish Baliga) Date: Wed, 14 Oct 2009 09:25:38 -0700 (PDT) Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> Message-ID: <474512.24919.qm@web31804.mail.mud.yahoo.com> We are using Erlang across data center boundary. We are using proto_dist as inet_ssl for communication between erlang nodes. Also using min and max port setup provided by inet to limit the communication on a specific set of ports. On each physical box, we have setup iptable based firewall. -- baliga "Point of view is worth 80 IQ points" --Alan Kay http://dudefrommangalore.blogspot.com/ ________________________________ From: Calum To: erlang-questions@REDACTED Sent: Sun, October 11, 2009 2:55:11 PM Subject: [erlang-questions] A less trusting Erlang. Hello all, I am an Erlang beginner, although I've been dipping in and out of it for a while. I'd like to use Erlang, and the built-in distributed meshing effect of it to build a sort of IRC-plus distributed storage/messaging system platform, of which each node would run the Erlang app. However, I'd like to use avoid using lib_chan if possible, and just use the native Erlang connectivity/distribution mechanisms. The big problem then is that Erlang is way too trusting for connecting to random people in the default if-you-know-my-cookie-you-can-do-whatever-you-like mode. The obvious thing (to me anyway) would be to limit the calls that remote nodes can make - Erlang Untrusting, if you will, or Paranoid Erlang. Would it be possible to patch the core libraries that "accept/receive" the spawn requests, and check them against a list of "allowed" functions. For example, if erl was run with the -paranoid parameter, it would only allow calls which are listed in .erlang.allowed to be made from remote nodes. Such a file might look like public:*, mystuff:another/1, etc etc. Would it require patching and recompiling of the source to Erlang, or could the "receive" function be "overwritten" from the shell/programatically? Is this approach one that could work? Are there sufficiently few places that could be restricted to make this solution secure? How much of a patch would this be, or are there too many places to be able to secure it with any degree of confidence? And even if these restrictions were in place, would it provide a high-enough level of security to prevent ne'er-do-wells doing things like drain resources, insert code somehow, etc? Or would it be too hard to prevent people passing in Funs, or doing other unexpected things? Hoping some Erlang heavyweights can provide some thoughts onto this approach, and give some feedback on this idea. C PS. Apologies if this functionality already exists somewhere. ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From hayeah@REDACTED Wed Oct 14 20:48:48 2009 From: hayeah@REDACTED (Howard Yeh) Date: Wed, 14 Oct 2009 11:48:48 -0700 Subject: Job Opportunity Working with AMQP Message-ID: Hi All, I hope it's ok to look for people here, Briefly, we have a distributed service architecture built on top of AMQP. It's a lot like gen_server, with the sequential application code of the services handled by Ruby modules, but the coordination & communication of the services using message-passing concurrency via AMQP, like Erlang. Even though it's Ruby, we've stolen a lot of ideas from Erlang : ) If that sounds interesting, send me an email at hayeah@REDACTED We are looking for contract workers who can commit 30~40hrs a week. Howard -- blog: www.metacircus.com From paul.joseph.davis@REDACTED Wed Oct 14 20:50:05 2009 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Wed, 14 Oct 2009 14:50:05 -0400 Subject: Erlang on EC2 - Filesystem errors? Message-ID: Hey list, I've got a bit of a head scratcher. The basic premise is trying to run Erlang on an EC2 sometimes results in errors like those pasted below. I've seen this type of error from two different CouchDB users and the only thing I can figure is that they're both using EC2. Granted lots of other people use Erlang on EC2 just fine so I haven't the slightest if that's even related. The only thing that's confusing me is how the VM even started if it's unable to read these files. My first thought was that some rogue backup process was changing file permissions as it did its thing but both users reported nothing in cron or had knowledge of anything of that nature running on their system. I've googled around looking for general filesystem issues on EC2 as well as things related specifically to Erlang. The only thing that looked remotely similar was very similar output that some rabbitmq users have reported. The most interesting of which is this thread [1] where the last post says that the guy could fix the errors just by running a 'find | grep sth' which seems odd. Anyone else experienced this before? Thank you, Paul J. Davis [1] http://www.ejabberd.im/node/3667 {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation error: eacces. Target: .. Function: read_file_info. Process: code_server."} {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation error: eacces. Target: ./user_sup.beam. Function: get_file. Process: code_server."} {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation error: eacces. Target: ./supervisor_bridge.beam. Function: get_file. Process: code_server."} {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation error: eacces. Target: ./user.beam. Function: get_file. Process: code_server."} {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation error: eacces. Target: ./kernel_config.beam. Function: get_file. Process: code_server."} {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation error: eacces. Target: ./queue.beam. Function: get_file. Process: code_server."} =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: .. Function: read_file_info. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./user_sup.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./supervisor_bridge.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./user.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./kernel_config.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./queue.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./error_logger_tty_h.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./io_lib.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./io_lib_format.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./io_lib_pretty.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./io.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./c.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./file.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./erl_eval.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./orddict.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./file_io_server.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./erl_posix_msg.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === file:path_eval([".","/home/couchdb"],".erlang"): permission denied =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./couch.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: .. Function: list_dir. Process: application_controller. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./erl_scan.beam. Function: get_file. Process: code_server. =ERROR REPORT==== 14-Oct-2009::01:42:33 === File operation error: eacces. Target: ./erl_parse.beam. Function: get_file. Process: code_server. From caluml@REDACTED Wed Oct 14 21:11:52 2009 From: caluml@REDACTED (Calum) Date: Wed, 14 Oct 2009 20:11:52 +0100 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <474512.24919.qm@web31804.mail.mud.yahoo.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <474512.24919.qm@web31804.mail.mud.yahoo.com> Message-ID: <635498b70910141211t18210638l48657ab3d682b0c4@mail.gmail.com> On Wed, Oct 14, 2009 at 5:25 PM, Yogish Baliga wrote: > We are using Erlang across data center boundary. We are using proto_dist as > inet_ssl for communication between erlang nodes. ?Also using min and max > port setup provided by inet to limit the communication on a specific set of > ports. > On each physical box, we have setup iptable based firewall. > -- baliga Well, that would certainly make traversing unsecured networks fine - but what about asking people to share computing resources who don't necessarily trust each other with being able to run shell commands on each other's machines? I'm a little confused about the whole Pid conversation earlier though - surely if you're only able to spawn a single function on my box that I've defined, then Pids don't really come into it? From anders.nygren@REDACTED Wed Oct 14 21:24:29 2009 From: anders.nygren@REDACTED (Anders Nygren) Date: Wed, 14 Oct 2009 14:24:29 -0500 Subject: [erlang-questions] Erlang on EC2 - Filesystem errors? In-Reply-To: References: Message-ID: On Wed, Oct 14, 2009 at 1:50 PM, Paul Davis wrote: > Hey list, > > I've got a bit of a head scratcher. The basic premise is trying to run > Erlang on an EC2 sometimes results in errors like those pasted below. > I've seen this type of error from two different CouchDB users and the > only thing I can figure is that they're both using EC2. Granted lots > of other people use Erlang on EC2 just fine so I haven't the slightest > if that's even related. > > The only thing that's confusing me is how the VM even started if it's > unable to read these files. My first thought was that some rogue > backup process was changing file permissions as it did its thing but > both users reported nothing in cron or had knowledge of anything of > that nature running on their system. > > I've googled around looking for general filesystem issues on EC2 as > well as things related specifically to Erlang. The only thing that > looked remotely similar was very similar output that some rabbitmq > users have reported. The most interesting of which is this thread [1] > where the last post says that the guy could fix the errors just by > running a 'find | grep sth' which seems odd. > > Anyone else experienced this before? I have no useful information to provide except that I have seen this once on a Stratus ftServer running RedHat Enterprise Server. I don't remember what I checked when it happened but a reboot of the server fixed it. /Anders > > Thank you, > Paul J. Davis > > [1] http://www.ejabberd.im/node/3667 > > > {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation > error: eacces. Target: .. Function: read_file_info. Process: > code_server."} > {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation > error: eacces. Target: ./user_sup.beam. Function: get_file. Process: > code_server."} > {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation > error: eacces. Target: ./supervisor_bridge.beam. Function: get_file. > Process: code_server."} > {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation > error: eacces. Target: ./user.beam. Function: get_file. Process: > code_server."} > {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation > error: eacces. Target: ./kernel_config.beam. Function: get_file. > Process: code_server."} > {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation > error: eacces. Target: ./queue.beam. Function: get_file. Process: > code_server."} > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: .. Function: read_file_info. > Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./user_sup.beam. Function: > get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./supervisor_bridge.beam. > Function: get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./user.beam. Function: get_file. > Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./kernel_config.beam. Function: > get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./queue.beam. Function: > get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./error_logger_tty_h.beam. > Function: get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./io_lib.beam. Function: > get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./io_lib_format.beam. Function: > get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./io_lib_pretty.beam. Function: > get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./io.beam. Function: get_file. > Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./c.beam. Function: get_file. > Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./file.beam. Function: get_file. > Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./erl_eval.beam. Function: > get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./orddict.beam. Function: > get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./file_io_server.beam. Function: > get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./erl_posix_msg.beam. Function: > get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > file:path_eval([".","/home/couchdb"],".erlang"): permission denied > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./couch.beam. Function: > get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: .. Function: list_dir. Process: > application_controller. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./erl_scan.beam. Function: > get_file. Process: code_server. > > =ERROR REPORT==== 14-Oct-2009::01:42:33 === > File operation error: eacces. Target: ./erl_parse.beam. Function: > get_file. Process: code_server. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From bernie@REDACTED Wed Oct 14 23:43:55 2009 From: bernie@REDACTED (Bernard Duggan) Date: Thu, 15 Oct 2009 08:43:55 +1100 Subject: [erlang-questions] tcp connection with timeout In-Reply-To: References: Message-ID: <4AD6461B.8060200@m5net.com> kamiseq wrote: > hi, > is there a way to discard any message from peer after set timeout occurs? A > server should respond in given time or transmission should be repeated. > > I implemented small ex where one process is listening and responding in > random time (let say 1 to 3 sek). and it is always accepting exactly one > process during its life. > I'm not entirely sure what you mean by "accepting exactly one process". > other process is connecting via tcp and is sending msg. I set send_timeout > flag for socket and a timer on receive after to let say 500ms and 1000ms so > it should get timeout all the time. > send_timeout does not do what you seem to think it does. The acknowledgement of the send is generated by the far end's TCP stack before your code ever sees the packet. > when I receive timeout I send special msg that server should repeat last > response. all this is great but then I will receive two responses from > server with timeouted and then repeated msg. > > in fact Im receiving all "lost" messages that were sent to my client by > server. I could maybe add some ticket to the msg and take out any incoming > messages that are not matching but this is impossible right now (afac) as I > will break protocol. > I think your basic problem here is that you're using TCP, but trying to re-implement part of it (lost/timed-out message retransmission) on top of it. If you want to tinker with stuff at that level, use a UDP connection instead. Cheers, Bernard From kamiseq@REDACTED Thu Oct 15 00:14:51 2009 From: kamiseq@REDACTED (kamiseq) Date: Thu, 15 Oct 2009 00:14:51 +0200 Subject: [erlang-questions] tcp connection with timeout In-Reply-To: <4AD6461B.8060200@m5net.com> References: <4AD6461B.8060200@m5net.com> Message-ID: ok so how should I redesign my example? anyway the protocol I was given is like that pack eny user msg into 2, msg, 3, checksum ->server respond in 2 sek send 6 if something goes wrong send 21. all that numbers are from ascii table STX ETX ACK and NAK. so yes is is kind of reinventing the wheel but.... -- pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From fritchie@REDACTED Thu Oct 15 00:22:25 2009 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 14 Oct 2009 17:22:25 -0500 Subject: [erlang-questions] Erlang on EC2 - Filesystem errors? In-Reply-To: Message of "Wed, 14 Oct 2009 14:50:05 EDT." Message-ID: <94324.1255558945@snookles.snookles.com> Paul Davis wrote: pd> {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation pd> error: eacces. Target: .. Function: read_file_info. Process: pd> code_server."} Paul, the most common way I've run into this is to do the following as the superuser (and doesn't require EC2): mkdir -p /root/some-private-directory cd /root/some-private-directory chown root . chmod 700 . su some-non-privileged-user -c "erl blah blah blah..." The "some-non-privileged-user" user can't read anything in the current directory, which is still /root/some-private-directory. -Scott From kamiseq@REDACTED Thu Oct 15 00:31:25 2009 From: kamiseq@REDACTED (kamiseq) Date: Thu, 15 Oct 2009 00:31:25 +0200 Subject: [erlang-questions] tcp connection with timeout In-Reply-To: References: <4AD6461B.8060200@m5net.com> Message-ID: I think it is because the same protocol is for tcp and for rss connection -- pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From paul.joseph.davis@REDACTED Thu Oct 15 00:41:03 2009 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Wed, 14 Oct 2009 18:41:03 -0400 Subject: [erlang-questions] Erlang on EC2 - Filesystem errors? In-Reply-To: <94324.1255558945@snookles.snookles.com> References: <94324.1255558945@snookles.snookles.com> Message-ID: On Wed, Oct 14, 2009 at 6:22 PM, Scott Lystig Fritchie wrote: > Paul Davis wrote: > > pd> {error_logger,{{2009,10,14},{1,42,33}},std_error,"File operation > pd> error: eacces. Target: .. Function: read_file_info. Process: > pd> code_server."} > > Paul, the most common way I've run into this is to do the following as > the superuser (and doesn't require EC2): > > ? ?mkdir -p /root/some-private-directory > ? ?cd /root/some-private-directory > ? ?chown root . > ? ?chmod 700 . > ? ?su some-non-privileged-user -c "erl blah blah blah..." > > The "some-non-privileged-user" user can't read anything in the current > directory, which is still /root/some-private-directory. > > -Scott > Scott, I'm not sure I follow. You mean that running with a $CWD that the user does not have write permissions on can cause the code server to fail when reading beam files from $PREFIX/lib/erlang/* ? Or even from the user's home directory? I tried a simplistic erl session in a directory I don't have write permissions on, but nothing crazy happened: davisp@REDACTED:/root$ touch foo touch: cannot touch `foo': Permission denied davisp@REDACTED:/root$ erl Erlang (BEAM) emulator version 5.6.5 [source] [64-bit] [smp:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> io:format("How you doin' world?~n", []). How you doin' world? ok 2> Thanks again, Paul J. Davis From kevin@REDACTED Thu Oct 15 01:32:45 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Wed, 14 Oct 2009 19:32:45 -0400 Subject: [erlang-questions] CPU load of TCP server In-Reply-To: <20091014054002.073C13D0009@mail.pharos-avantgard.com> References: <20091014054002.073C13D0009@mail.pharos-avantgard.com> Message-ID: <1EF19894-4A38-4615-ABAB-C72BE9A6231B@hypotheticalabs.com> I wonder if using prim_inet:async_accept/2 would help reduce CPU usage. --Kevin On Oct 14, 2009, at 1:39 AM, Valentin Micic wrote: > Hi Andrey, > > From your code snippet I suspect that you've opened your socket with > {active, true} flag, which may explain excessive CPU usage -- it is > much > cheaper to keep excess messages in a TCP buffer, rather than as > bunch of > messages waiting on a process queue (for example, messages in the > TCP buffer > have no effect on selective receive). > > Thus, changing your code to: > > loop(Socket) -> > receive > {tcp, Socket, _Packet} -> > inet:setopts( Socket, [{active, once}] ), > loop(Socket); > {tcp_closed, Socket} -> > normal; > _ -> > loop(Socket) > after 500 -> > gen_tcp:send(Socket,[?PACKET]), > loop(Socket) > end. > > Should save some CPU cycles... don't forget to open this socket with > initial > {active, once}. > > As for performance difference between R12 and R13 -- not sure, but I > think > there is more tax to be paid for pushing bunch of messages around to a > scheduler with a right queue. > > V/ > > > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang- > questions@REDACTED] On > Behalf Of Andrey Tsirulev > Sent: 13 October 2009 08:34 PM > To: Rapsey; erlang-questions@REDACTED > Subject: Re: [erlang-questions] CPU load of TCP server > > Hi Sergej, > > Thanks for the hint. I moved timer to client. Now I have about 5.5% > of CPU > usage per each 1000 connections. I still expect it should be less.. > > Andrey > > > ----- Original Message ----- > From: "Rapsey" > To: > Sent: Tuesday, October 13, 2009 9:50 PM > Subject: Re: [erlang-questions] CPU load of TCP server > > >> Every time after gets executed a timer gets created (I presume). >> With 10k >> processes it probably makes a noticeable CPU impact. >> >> >> Sergej >> >> 2009/10/13 Andrey Tsirulev >> >>> Hello all, >>> >>> I'm exploring the possibility of using Erlang for my TCP service >>> application (actually the game server). I've prepared test server >>> and >>> client >>> applications. The test server application accepts client >>> connections and >>> sends 2 small (<1 Kb) packets per second to each client (and >>> receives >>> answers). >>> >>> I've met the following problems: >>> 1) Kernel polling doesn't give any benefit with R13B02-1. >>> 2) CPU load is too high. >>> >>> All the details are below. >>> >>> Here's my test server's `uname -a`: >>> Linux source 2.6.29-gentoo-r5 #1 SMP Tue Aug 18 01:15:17 MSD 2009 >>> x86_64 >>> AMD Sempron(tm) Dual Core Processor 2200 AuthenticAMD GNU/Linux >>> (I've made tests also with 2 other linux servers with different >>> kernel >>> versions and results were close). >>> >>> I've made server connection processes as simple and possible. I've >>> tried >>> up >>> to 10000 concurrent connections. >>> >>> Test results didn't not show any visible difference between using >>> multiple >>> remote machines for client connections, one remote machine or >>> localhost. >>> >>> I tried R13B02-1 and R12B-5 OTP versions. >>> >>> I found that memory usage grow is linear, as expected. But I came >>> to the >>> problem with CPU load. >>> >>> First of all, kernel polling didn't give any benefit for R13B02-1 >>> (while >>> erlang:system_info(kernel_poll) returned true and erl started with >>> message >>> [kernel-poll:true]). I've got about 55% of CPU usage with 4000 >>> connections >>> both with and without kernel polling enabled, while with R12B-5 I >>> have >>> about >>> 26% of CPU usage with +Ktrue. I suspect a bug either in OTP or in >>> gentoo >>> ebuild (of course it's also quite possible that I'm doing >>> something wrong > >>> or >>> missed something in docs). >>> >>> The following is about R12B-5. I get about 6-7% of CPU load per >>> every >>> 1000 >>> connections (about 60% CPU load for 10000 connections). I'm not >>> sure if I >>> should consider this as a good result or a bad one. Most of the >>> articles >>> on >>> the same subject say that CPU load is negligible in their tests >>> and they >>> are >>> fighting for memory only, so I expected I won't be CPU-limited >>> too, but >>> evidently I am. >>> >>> `top` says that about 50% of CPU load is userspace, 25% software >>> interrupts, 20% system and 5% hardware interrupts (that's by eye, >>> not >>> very >>> strict). >>> >>> I found that CPU load depends not as much on connection count but on >>> transmitted packet count (ok, that's obviously the number of system >>> calls). >>> Thus if I send 4 packets per second, not 2, I should decrease the >>> number >>> of >>> connections twice to preserve the same CPU load. >>> >>> CPU load does not depend on packet size. 1 byte or 1Kbyte - no >>> visible >>> difference. >>> >>> CPU usage is slightly less with active socket option enabled than >>> with >>> blocking recvs. >>> >>> CPU usage on the single windows client machine with 4000 connections >>> spawned is on the same level as with the linux server handling >>> these 4000 >>> connections (while I expected linux to perform better). >>> >>> Switching Nagle on and off had no effect. I also tried to tune TCP >>> stack >>> with sysctl using advises found here and there but almost without >>> any >>> effect >>> too. >>> >>> I've tried to trace with fprof and found that bottlenecks are 'send' >>> operations (but I'm a relative novice to erlang so I'm not sure my >>> usage >>> of >>> fprof was correct). Ok, that was expected too. I've read the 'why is >>> gen_tcp:send slow?' thread but none of advises given there helped >>> me. >>> >>> So the main question is: is the CPU usage of 7% per 1000 >>> connections (or >>> maybe better say 2000 packets per second) a good result? If no, >>> what is >>> the >>> expected result? How can I improve my test application? Or maybe >>> something >>> in my story looks strange? >>> >>> I know that the possible optimization is decreasing the number of >>> packets >>> and keep it in mind. >>> >>> Here's the server connection process loop: >>> >>> loop(Socket) -> >>> receive >>> {tcp, Socket, _Packet} -> >>> loop(Socket); >>> {tcp_closed, Socket} -> >>> normal; >>> _ -> >>> loop(Socket) >>> after 500 -> >>> gen_tcp:send(Socket,[?PACKET]), >>> loop(Socket) >>> end. >>> >>> Client loop has blocking recv and answers with send immediately. >>> >>> Thank you very much for your time. Sorry for too many words, I >>> tried to >>> provide all possible information. I will answer any question and >>> appreciate >>> any hint. >>> >>> Best regards, >>> Andrey >> > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From sapan.gcet@REDACTED Thu Oct 15 06:13:26 2009 From: sapan.gcet@REDACTED (sapan shah) Date: Thu, 15 Oct 2009 09:43:26 +0530 Subject: [erlang-questions] Calling a Function with Same argument twice In-Reply-To: <4AD5D2AD.2030604@erlang-consulting.com> References: <4AD5D2AD.2030604@erlang-consulting.com> Message-ID: > > sapan shah wrote: > >> Hi, >> >> see the code below. >> ... >> >> We call temp:start(1), the output is 4. The output should have been 1. >> > > No, 4 is right. > > Here, when start(1) is called, it intern calls >> {element(1, test(1,1), element(2, test(1,1)} >> > > test(2,2) results in two calls to test(1,1), > which each result in two calls to test(0,0), each > incrementing the 'test' object by 1. > > >> The first call of test(1,1) should recursively evaluate test(1,1) while >> the >> second call should already use the value returned by the first call.. Is >> this a right thinking??? >> > > I assume you're thinking that it would work like: > > test(0,0) -> > put(test, get(test) +1), > {0,0}; > test(X,Y) -> > Z = test(X-1, Y-1), > {element(1, Z), element(2, Z)}. > > Yes, I solved the original problem that I am working in this manner only... but here you make only one call and reuse the result. > If the test/2 function werer purely functional (i.e. no side- > effects), the two would be equivalent, but here you use get and > put, so it really makes a difference how many times you explicitly > call the function. > > So, the basic question is: If the test/2 function is side effect free (no put, no get or no io:format..), would erlang evaluater still call the test/2 function again, if it had called the same fuction with same arguments erlier??? Btw, since test/2 returns a two-tuple, the above is of course > equivalent to > > test(0,0) -> > put(test, get(test) +1), > {0,0}; > test(X,Y) -> > test(X-1, Y-1). > > ...which is almost equivalent to > > test(X,X) when is_integer(X) -> put(test, get(test) +1), {0,0}. > > (It differs only slightly in how it fails on erroneous input). > > Think twice before using get/put. Also, why wrap the return values > inside a 1-tuple? Remove the tuple, and you can remove some of the > calls to element/2. > > But of course it was a contrived example, so I assume the real > problem you're trying to solve is somewhat different? > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > -- Regards, Sapan Shah From markjwallis@REDACTED Thu Oct 15 07:39:14 2009 From: markjwallis@REDACTED (Mark Wallis) Date: Wed, 14 Oct 2009 22:39:14 -0700 Subject: asn1rt decode of 3GPP RANAP-PDU Message-ID: Dear Forum I have compiled the ASN.1 for 3GPP RANAP using the following options %% Generated by the Erlang ASN.1 BER-compiler version, utilizing bit-syntax:1.6.10 %% Purpose: encoder and decoder to the types in mod RANAP -module('RANAP'). -include("RANAP.hrl"). -define('RT_PER',asn1rt_per_bin). -asn1_info([{vsn,'1.6.10'}, {module,'RANAP'}, {options,[per_bin,undec_rest,{i,[46]}]}]). When I pass a RANAP-PDU through the deocder it decodes successfully but only prints 'ProtocolIE-Field',x and not the Protocol IE name (see below). Is this expected behaviour as I can see all the ids being declared in the .erl file? Regards Mark. 'RANAP':decode('RANAP-PDU', list_to_binary([0,19,64,74,0,0,6,0,3,64,1,0,0,15,64,6,0,50,244,149,0,4,0,58 ,64,8,0,50,244,149,0,4,0,4,0,16,64,24,23,5,8,0,50,244,149,0,133,79,8,57,1,87,0,0,0,4,40,51,3,79,24,160 ,0,79,64,3,48,0,15,0,86,64,5,50,244,149,0,1])). {ok,{initiatingMessage, {'InitiatingMessage',19,ignore, {'InitialUE-Message', [{'ProtocolIE-Field',3,ignore,'cs-domain'}, {'ProtocolIE-Field',15,ignore, {'LAI',[50,244,149],[0,4],asn1_NOVALUE}}, {'ProtocolIE-Field',58,ignore, {'SAI',[50,244,149],[0,4],[0,4],asn1_NOVALUE}}, {'ProtocolIE-Field',16,ignore, [5,8,0,50,244,149,0,133,79,8,57|...]}, {'ProtocolIE-Field',79,ignore,[0,0,1,1,0,0,0,0,0,0|...]}, {'ProtocolIE-Field',86,ignore, {'GlobalRNC-ID',[50,244,149],1}}], asn1_NOVALUE}}}, <<>>} From bengt.kleberg@REDACTED Thu Oct 15 07:53:27 2009 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 15 Oct 2009 07:53:27 +0200 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <635498b70910141211t18210638l48657ab3d682b0c4@mail.gmail.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <474512.24919.qm@web31804.mail.mud.yahoo.com> <635498b70910141211t18210638l48657ab3d682b0c4@mail.gmail.com> Message-ID: <1255586007.4802.6.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, The function you have allowed somebody to spawn on your machine can send messages to any Pid. bengt On Wed, 2009-10-14 at 20:11 +0100, Calum wrote: > On Wed, Oct 14, 2009 at 5:25 PM, Yogish Baliga wrote: > > We are using Erlang across data center boundary. We are using proto_dist as > > inet_ssl for communication between erlang nodes. Also using min and max > > port setup provided by inet to limit the communication on a specific set of > > ports. > > On each physical box, we have setup iptable based firewall. > > -- baliga > > Well, that would certainly make traversing unsecured networks fine - > but what about asking people to share computing resources who don't > necessarily trust each other with being able to run shell commands on > each other's machines? > > I'm a little confused about the whole Pid conversation earlier though > - surely if you're only able to spawn a single function on my box that > I've defined, then Pids don't really come into it? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From sapan.gcet@REDACTED Thu Oct 15 07:54:43 2009 From: sapan.gcet@REDACTED (sapan shah) Date: Thu, 15 Oct 2009 11:24:43 +0530 Subject: No Namespace for Macros Message-ID: Hi, Erlang does have the concept of namespace for the functions( we do lists:map(...)) Why does it not have this concept for macros? Consider this scenario: I have an application which deals with the value of some things. (lets say a house & a car) So, I define macros in two different files for the things. %%%%valueHouse.hrl -define(Value, 10). %%%%valueCar.hrl -define(Value, 2). Now, My application file could be something like this %%%application.erl -module(application). -include("valueHouse.hrl"). -include("valueCar.hrl"). %% This will not compile... ll give 'redefining macro error'... assume it is allowed in erlang star(X) -> case X of house -> ?valueHouse:Value; car -> ?valueCar:Value end. %%%%%%%%%%%%%%%%%% This is just a cooked up example. One may have this kind of need in a big application. For exapmle, Java supports this (public static final variables). Any Explanation????? Regards, Sapan Shah From anupam.kapoor@REDACTED Thu Oct 15 08:24:02 2009 From: anupam.kapoor@REDACTED (Anupam Kapoor) Date: Thu, 15 Oct 2009 11:54:02 +0530 Subject: [erlang-questions] No Namespace for Macros In-Reply-To: References: Message-ID: > For exapmle, Java supports this (public static final variables). > > Any Explanation????? erlang doesn't have any notion of objects. in java, each '.java' file contains a corresponding class-definition, and you can have multiple instances of those objects defined... kind regards anupam -- In the beginning was the lambda, and the lambda was with Emacs, and Emacs was the lambda. From bengt.kleberg@REDACTED Thu Oct 15 08:30:07 2009 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 15 Oct 2009 08:30:07 +0200 Subject: [erlang-questions] No Namespace for Macros In-Reply-To: References: Message-ID: <1255588207.4802.11.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, You might want to look into using the following for your example: house.erl: -module(house). -export([value/0]). value() -> 10. car.erl: -module(car). -export([value/0]). value() -> 2. bengt On Thu, 2009-10-15 at 11:24 +0530, sapan shah wrote: > Hi, > > Erlang does have the concept of namespace for the functions( we do > lists:map(...)) > Why does it not have this concept for macros? > > Consider this scenario: I have an application which deals with the value of > some things. (lets say a house & a car) > So, I define macros in two different files for the things. > %%%%valueHouse.hrl > -define(Value, 10). > %%%%valueCar.hrl > -define(Value, 2). > > Now, My application file could be something like this > %%%application.erl > -module(application). > -include("valueHouse.hrl"). > -include("valueCar.hrl"). %% This will not compile... ll give 'redefining > macro error'... assume it is allowed in erlang > > star(X) -> > case X of house -> ?valueHouse:Value; > car -> ?valueCar:Value > end. > %%%%%%%%%%%%%%%%%% > > This is just a cooked up example. One may have this kind of need in a big > application. > For exapmle, Java supports this (public static final variables). > > Any Explanation????? > > Regards, > Sapan Shah From anupam.kapoor@REDACTED Thu Oct 15 08:35:13 2009 From: anupam.kapoor@REDACTED (Anupam Kapoor) Date: Thu, 15 Oct 2009 12:05:13 +0530 Subject: [erlang-questions] No Namespace for Macros In-Reply-To: References: Message-ID: ofcourse another way to skin the cat is to return constants from functions in different modules... On Thu, Oct 15, 2009 at 11:54 AM, Anupam Kapoor wrote: >> For exapmle, Java supports this (public static final variables). >> >> Any Explanation????? > erlang doesn't have any notion of objects. in java, each '.java' file > contains a corresponding class-definition, and you can have multiple > instances of those objects defined... > > kind regards > anupam > > -- > In the beginning was the lambda, and the lambda was with Emacs, and > Emacs was the lambda. > -- In the beginning was the lambda, and the lambda was with Emacs, and Emacs was the lambda. From sapan.gcet@REDACTED Thu Oct 15 10:58:44 2009 From: sapan.gcet@REDACTED (sapan shah) Date: Thu, 15 Oct 2009 14:28:44 +0530 Subject: [erlang-questions] No Namespace for Macros In-Reply-To: References: Message-ID: As Anupag said it rightly & Bengt showed... there is a way around.. But my curiosity is, If it is for functions, why it is not for macros??? What would the developers of Erlang have thought & made it like that??? On Thu, Oct 15, 2009 at 12:05 PM, Anupam Kapoor wrote: > ofcourse another way to skin the cat is to return constants from > functions in different modules... > > > On Thu, Oct 15, 2009 at 11:54 AM, Anupam Kapoor > wrote: > >> For exapmle, Java supports this (public static final variables). > >> > >> Any Explanation????? > > erlang doesn't have any notion of objects. in java, each '.java' file > > contains a corresponding class-definition, and you can have multiple > > instances of those objects defined... > > > > kind regards > > anupam > > > > -- > > In the beginning was the lambda, and the lambda was with Emacs, and > > Emacs was the lambda. > > > > > > -- > In the beginning was the lambda, and the lambda was with Emacs, and > Emacs was the lambda. > -- Regards, Sapan Shah From valentin@REDACTED Thu Oct 15 09:25:34 2009 From: valentin@REDACTED (Valentin Micic) Date: Thu, 15 Oct 2009 09:25:34 +0200 Subject: [erlang-questions] A less trusting Erlang. References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <635498b70910121139s486f9b9cr5d5f7b2bf2794647@mail.gmail.com> <4AD38779.3040700@gmail.com> <4AD417B8.8000702@erlang-consulting.com> <4AD5EAAD.2000007@gmail.com> <4AD5EEB2.7030905@erlang-consulting.com> Message-ID: How about this: (scon2@REDACTED)2> {ok, Pid} = yourmodule:start(). (scon2@REDACTED)3> unregister( rex ). true (scon2@REDACTED)4> register( rex, Pid). true Original rex keeps running, but traffic is redirected to "yourmodule", which is impersonating rpc server (rex). Does not really turn the rpc server off, but the effects may be considered the same. V/ ----- Original Message ----- From: "Ulf Wiger" To: "Jacob Torrey" Cc: "Erlang Questions" Sent: Wednesday, October 14, 2009 5:30 PM Subject: Re: [erlang-questions] A less trusting Erlang. > Jacob Torrey wrote: >> Ulf, >> If rex a required process? Is there any way to turn it off, or replace >> it with a module of my choosing? Could you please point me to a >> reference on rex, all I can find is a passing reference in erl_call. >> >> Jacob > > Rex is the registered name of the rpc server. > > The one way I know of to turn it off is to start erlang with > erl -mode minimal > > > Not really sure where -mode minimal is documented, though, > but rpc is not all it turns off - you get no Distributed Erlang > at all. So I don't think that solves your problem. > > You should see this as an example of how difficult it is to make > Erlang safe if you allow remote processes to send messages. Since > any message can be sent to any process, 'rex' is just the most > obvious vulnerability to exploit. > > You are much better off setting up a narrow pipe and allowing > only a small set of messages. Since most communication is > wrapped inside function calls, it really isn't as limiting as > you might suspect at first. > > BR, > Ulf W > > >> >> Ulf Wiger wrote: >>> Jacob Torrey wrote: >>>> Could perhaps the pid generation in the VM be made more random, >>>> therefore you can only gain access to pids that you know the name to? >>>> This seems like an obvious (and pretty trivial) solution to obscure and >>>> increase the security of the Erlang system. >>> You don't even need to bother with the pids: >>> >>> {rex,Node} ! {'$gen_cast',{cast,M,F,A,user}} >>> >>> ...will allow you to execute any function on Node. >>> >>> BR, >>> Ulf W >> > > > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From icfp.publicity@REDACTED Thu Oct 15 12:00:13 2009 From: icfp.publicity@REDACTED (Wouter Swierstra) Date: Thu, 15 Oct 2009 12:00:13 +0200 Subject: ICFP10: Call for Workshop Proposals Message-ID: <53ff55480910150300of2e11eao689a68e88cf49ce1@mail.gmail.com> CALL FOR WORKSHOP AND CO-LOCATED EVENT PROPOSALS ICFP 2010 15th ACM SIGPLAN International Conference on Functional Programming September 27 - 29, 2010 Baltimore, Maryland http://www.icfpconference.org/icfp2010 The 15th ACM SIGPLAN International Conference on Functional Programming will be held in Baltimore, Maryland on September 27-29, 2010. ICFP provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. Proposals are invited for workshops (and other co-located events, such as tutorials) to be affiliated with ICFP 2010 and sponsored by SIGPLAN. These events should be more informal and focused than ICFP itself, include sessions that enable interaction among the attendees, and be fairly low-cost. The preference is for one-day events, but other schedules can also be considered. ---------------------------------------------------------------------- Submission details Deadline for submission: November 20, 2009 Notification of acceptance: December 18, 2009 Prospective organizers of workshops or other co-located events are invited to submit a completed workshop proposal form in plain text format to the ICFP 2010 workshop co-chairs (Derek Dreyer and Chris Stone), via email to icfp10-workshops at mpi-sws.org by November 20, 2009. (For proposals of co-located events other than workshops, please fill in the workshop proposal form and just leave blank any sections that do not apply.) Please note that this is a firm deadline. Organizers will be notified if their event proposal is accepted by December 18, 2009, and if successful, depending on the event, they will be asked to produce a final report after the event has taken place that is suitable for publication in SIGPLAN Notices. The proposal form is available at: http://www.icfpconference.org/icfp2010/icfp10-workshops-form.txt Further information about SIGPLAN sponsorship is available at: http://acm.org/sigplan/sigplan_workshop_proposal.htm ---------------------------------------------------------------------- Selection committee The proposals will be evaluated by a committee comprising the following members of the ICFP 2010 organizing committee, together with the members of the SIGPLAN executive committee. Workshop Co-Chair: Derek Dreyer (MPI-SWS) Workshop Co-Chair: Chris Stone (Harvey Mudd College) General Chair: Paul Hudak (Yale University) Program Chair: Stephanie Weirich (University of Pennsylvania) ---------------------------------------------------------------------- Further information Any queries should be addressed to the workshop co-chairs (Derek Dreyer and Chris Stone), via email to icfp10-workshops at mpi-sws.org. From kenji.rikitake@REDACTED Thu Oct 15 12:11:24 2009 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Thu, 15 Oct 2009 19:11:24 +0900 Subject: [erlang-questions] Sending messages between erlang nodes (across firewall boundary) In-Reply-To: <65871.49596.qm@web31807.mail.mud.yahoo.com> References: <65871.49596.qm@web31807.mail.mud.yahoo.com> Message-ID: <20091015101124.GA5124@k2r.org> If you are not demanding something versatile like the rpc module, the ssh module is worth trying. At least you can run a dedicated SSH server from Erlang shell, either in interactive or as a remote command, using SSH "exec" protocol. You can even write a dedicated SSH subsystem. inet_ssl_dist is non-trivial, but if your environment allows it, it's also worth trying, though it does NOT encrypt epmd communication. Kenji Rikitake In the message <65871.49596.qm@REDACTED> dated Tue, Oct 13, 2009 at 07:37:37PM -0700, Yogish Baliga writes: > I am working on setting up the Erlang nodes across multiple data centers. I do not want to send the data across the data centers in clear text. > > Within the data centers, I did a simple setup for 2 Erlang nodes on 2 physical boxes. Captured the tcp traffic while sending the messages between these 2 nodes. Raw TCP packets do not show the text being sent in clear text. Is it because by default the messages are encrypted using the cookie or something? Do I need to specify the proto_dist as inet_ssl in order to setup the Erlang node grid across data center? > > Ideally I don't want to setup the SSL communication between nodes. If that is what is absolutely required in-order to make sure that the communication is encrypted, I will do that. > > Thanx, > -- baliga From ulf.wiger@REDACTED Thu Oct 15 13:17:25 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 15 Oct 2009 13:17:25 +0200 Subject: [erlang-questions] Calling a Function with Same argument twice In-Reply-To: References: <4AD5D2AD.2030604@erlang-consulting.com> Message-ID: <4AD704C5.2070803@erlang-consulting.com> sapan shah wrote: > > So, the basic question is: If the test/2 function is side effect free > (no put, no get or no io:format..), would erlang evaluater still call > the test/2 function again, if it had called the same fuction with same > arguments erlier??? The evaluator will always do so. The compiler (which I assume was what you meant) is sometimes able to inline the function, if it is a local function, and simple enough. See the man page for the 'compile' module. Otherwise, it will always call the function each time. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From rob@REDACTED Thu Oct 15 13:16:45 2009 From: rob@REDACTED (Rob Charlton) Date: Thu, 15 Oct 2009 12:16:45 +0100 Subject: [erlang-questions] Calling a Function with Same argument twice In-Reply-To: References: <4AD5D2AD.2030604@erlang-consulting.com> Message-ID: <4AD7049D.1000403@erlang-consulting.com> sapan shah wrote: > So, the basic question is: If the test/2 function is side effect free (no > put, no get or no io:format..), would erlang evaluater still call the test/2 > function again, if it had called the same fuction with same arguments > erlier??? > Try this code and see: -module(side_effect_test). -compile(export_all). -include_lib("stdlib/include/ms_transform.hrl"). test() -> dbg:tracer(), dbg:p(all,call), dbg:tpl(?MODULE,dbg:fun2ms(fun(_)->return_trace() end)), %% and now the test test2(side_effect_free(2,2),side_effect_free(2,2)). test2(A,B) -> ok. side_effect_free(A,B) -> A + B. And the output: 1> side_effect_test:test(). (<0.31.0>) call side_effect_test:side_effect_free(2,2) (<0.31.0>) returned from side_effect_test:side_effect_free/2 -> 4 (<0.31.0>) call side_effect_test:side_effect_free(2,2) (<0.31.0>) returned from side_effect_test:side_effect_free/2 -> 4 (<0.31.0>) call side_effect_test:test2(4,4) (<0.31.0>) returned from side_effect_test:test2/2 -> ok So the answer is no - Erlang doesn't know that "side_effect_free" is free of side effects and that multiple calls with the same parameters can be optimized away, and so will evaluate it twice. Cheers Rob -- Erlang Training and Consulting Ltd www.erlang-consulting.com From caluml@REDACTED Thu Oct 15 14:16:43 2009 From: caluml@REDACTED (Calum) Date: Thu, 15 Oct 2009 13:16:43 +0100 Subject: [erlang-questions] Sending messages between erlang nodes (across firewall boundary) In-Reply-To: <65871.49596.qm@web31807.mail.mud.yahoo.com> References: <65871.49596.qm@web31807.mail.mud.yahoo.com> Message-ID: <635498b70910150516x76fd789dpac9afea45372809e@mail.gmail.com> On Wed, Oct 14, 2009 at 3:38 AM, Yogish Baliga wrote: > Hello all, > > ?I am working on setting up the Erlang nodes across multiple data centers. I do not want to send the data across the data centers in clear text. Perhaps it would be simpler to link the datacentre machines with a lightweight VPN solution like OpenVPN? I've used it extensively, and it's quick, simple, and lightweight to use. http://www.openvpn.net/index.php/open-source/downloads.html From raimo+erlang-questions@REDACTED Thu Oct 15 14:47:27 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 15 Oct 2009 14:47:27 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <200910141534121608248@its3.ch> References: <4AD5A5F2.3020407@erlang-consulting.com> <200910141534121608248@its3.ch> Message-ID: <20091015124727.GC24620@erix.ericsson.se> On Wed, Oct 14, 2009 at 03:34:12PM +0200, info wrote: > For me this function always returns {error,timeout} and I didn't find the reason even with the help of Raimo Niskanen ! Sorry about that, but your problem became a just too spooky Windows 2003 problem and that is not really my turf, and you seemed to have a possible workaround through inet_res. But it has been on my todo list to have a second look at the whole conversation when I could find the time... > > > At the end of my last mail you have the references. > Regards, > > Roberto Aloi > Erlang Training and Consulting Ltd. > http://www.erlang-consulting.com > http://aloiroberto.wordpress.com > > Robert Raschke wrote: > > This is part of the IP address spec (not sure where to look to find it, > > though). I've seen addresses specified like this on and off in various > > locations on a wide variety of OSes, for example in a /etc/hosts file you > > can write: > > > > localhost 127.1 > > > > Robby > > > > On Tue, Oct 13, 2009 at 8:06 PM, caio ariede wrote: > > > > > > > Interesting case. > > > > > > The result doesn't appear to be an issue, but a feature of the > > > gethostbyname > > > original implementation, in C code. > > > > > > You get the same result, testing with PHP: > > > > > > $ php -r 'var_dump(gethostbyname("12.27"));' > > > string(9) "12.0.0.27" > > > > > > And other interesting results: > > > > > > $ php -r 'var_dump(gethostbyname("255.2.256"));' > > > string(9) "255.2.1.0" > > > > > > But I can't see where it's really useful. > > > > > > Caio Ariede > > > http://caioariede.com/ > > > > > > > > > On Tue, Oct 13, 2009 at 3:15 PM, Garry Hodgson > > > > > > wrote: > > > > > > > > recently, a bug in my code caused us to pass a string > > > > representing a floating point number to inet:gethostbyname(). > > > > i would have expected it to return an error, but instead it > > > > returned an ip address, but one that made no sense to me: > > > > > > > > 1 > inet:gethostbyname( '12.27' ). > > > > {ok,{hostent,"12.27",[],inet,4,[{12,0,0,27}]}} > > > > > > > > so my question is, is this behavior correct, and if so, what > > > > exactly does it mean that a lookup of '12.27' maps to '12.0.0.27'? > > > > > > > > thanks > > > > > > > > -- > > > > Garry Hodgson > > > > Lead Member of Technical Staff > > > > AT&T Chief Security Office (CSO) > > > > > > > > "This e-mail and any files transmitted with it are AT&T property, are > > > > confidential, and are intended solely for the use of the individual or > > > > entity to whom this e-mail is addressed. If you are not one of the named > > > > recipient(s) or otherwise have reason to believe that you have received > > > > > > > this > > > > > > > message in error, please notify the sender and delete this message > > > > immediately from your computer. Any other use, retention, dissemination, > > > > forwarding, printing, or copying of this e-mail is strictly prohibited." > > > > > > > > ________________________________________________________________ > > > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > > > erlang-questions (at) erlang.org > > > > > > > > > > > > > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From info@REDACTED Thu Oct 15 15:02:40 2009 From: info@REDACTED (info) Date: Thu, 15 Oct 2009 15:02:40 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? References: <4AD5A5F2.3020407@erlang-consulting.com>, <200910141534121608248@its3.ch>, <20091015124727.GC24620@erix.ericsson.se> Message-ID: <200910151502401455839@its3.ch> Dear Raimo, I can resume: I discovered that gethostbyname call inet_gethost.c I don't understand inet_gethost.c :-( Where "goes" this problem for finding the information ? If we know where and what, we could perhaps find what is missing in my windows 2003 ... Hope to read you soon ! John On Wed, Oct 14, 2009 at 03:34:12PM +0200, info wrote: > For me this function always returns {error,timeout} and I didn't find the reason even with the help of Raimo Niskanen ! Sorry about that, but your problem became a just too spooky Windows 2003 problem and that is not really my turf, and you seemed to have a possible workaround through inet_res. But it has been on my todo list to have a second look at the whole conversation when I could find the time... > > > At the end of my last mail you have the references. > Regards, > > Roberto Aloi > Erlang Training and Consulting Ltd. > http://www.erlang-consulting.com > http://aloiroberto.wordpress.com > > Robert Raschke wrote: > > This is part of the IP address spec (not sure where to look to find it, > > though). I've seen addresses specified like this on and off in various > > locations on a wide variety of OSes, for example in a /etc/hosts file you > > can write: > > > > localhost 127.1 > > > > Robby > > > > On Tue, Oct 13, 2009 at 8:06 PM, caio ariede wrote: > > > > > > > Interesting case. > > > > > > The result doesn't appear to be an issue, but a feature of the > > > gethostbyname > > > original implementation, in C code. > > > > > > You get the same result, testing with PHP: > > > > > > $ php -r 'var_dump(gethostbyname("12.27"));' > > > string(9) "12.0.0.27" > > > > > > And other interesting results: > > > > > > $ php -r 'var_dump(gethostbyname("255.2.256"));' > > > string(9) "255.2.1.0" > > > > > > But I can't see where it's really useful. > > > > > > Caio Ariede > > > http://caioariede.com/ > > > > > > > > > On Tue, Oct 13, 2009 at 3:15 PM, Garry Hodgson > > > > > > wrote: > > > > > > > > recently, a bug in my code caused us to pass a string > > > > representing a floating point number to inet:gethostbyname(). > > > > i would have expected it to return an error, but instead it > > > > returned an ip address, but one that made no sense to me: > > > > > > > > 1 > inet:gethostbyname( '12.27' ). > > > > {ok,{hostent,"12.27",[],inet,4,[{12,0,0,27}]}} > > > > > > > > so my question is, is this behavior correct, and if so, what > > > > exactly does it mean that a lookup of '12.27' maps to '12.0.0.27'? > > > > > > > > thanks > > > > > > > > -- > > > > Garry Hodgson > > > > Lead Member of Technical Staff > > > > AT&T Chief Security Office (CSO) > > > > > > > > "This e-mail and any files transmitted with it are AT&T property, are > > > > confidential, and are intended solely for the use of the individual or > > > > entity to whom this e-mail is addressed. If you are not one of the named > > > > recipient(s) or otherwise have reason to believe that you have received > > > > > > > this > > > > > > > message in error, please notify the sender and delete this message > > > > immediately from your computer. Any other use, retention, dissemination, > > > > forwarding, printing, or copying of this e-mail is strictly prohibited." > > > > > > > > ________________________________________________________________ > > > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > > > erlang-questions (at) erlang.org > > > > > > > > > > > > > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From mevans@REDACTED Thu Oct 15 15:38:21 2009 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 15 Oct 2009 09:38:21 -0400 Subject: [erlang-questions] CPU load of TCP server In-Reply-To: References: <97619b170910131050g2632f794q46b45ad3cd48f831@mail.gmail.com> Message-ID: One thing I found was to play around with the settings for tcp recbuf and sendbuf (see setopts: http://www.erlang.org/doc/apps/kernel/index.html) I was experiencing high load with a http client implementation, and managed to cut it by a considerable amount by modifying the above values (espc. recbuf). It's worth a shot, but I might've been running into http client issues as opposed to basic tcp/gen_tcp problems. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Andrey Tsirulev Sent: Tuesday, October 13, 2009 2:34 PM To: Rapsey; erlang-questions@REDACTED Subject: Re: [erlang-questions] CPU load of TCP server Hi Sergej, Thanks for the hint. I moved timer to client. Now I have about 5.5% of CPU usage per each 1000 connections. I still expect it should be less.. Andrey ----- Original Message ----- From: "Rapsey" To: Sent: Tuesday, October 13, 2009 9:50 PM Subject: Re: [erlang-questions] CPU load of TCP server > Every time after gets executed a timer gets created (I presume). With 10k > processes it probably makes a noticeable CPU impact. > > > Sergej > > 2009/10/13 Andrey Tsirulev > >> Hello all, >> >> I'm exploring the possibility of using Erlang for my TCP service >> application (actually the game server). I've prepared test server and >> client >> applications. The test server application accepts client connections and >> sends 2 small (<1 Kb) packets per second to each client (and receives >> answers). >> >> I've met the following problems: >> 1) Kernel polling doesn't give any benefit with R13B02-1. >> 2) CPU load is too high. >> >> All the details are below. >> >> Here's my test server's `uname -a`: >> Linux source 2.6.29-gentoo-r5 #1 SMP Tue Aug 18 01:15:17 MSD 2009 x86_64 >> AMD Sempron(tm) Dual Core Processor 2200 AuthenticAMD GNU/Linux >> (I've made tests also with 2 other linux servers with different kernel >> versions and results were close). >> >> I've made server connection processes as simple and possible. I've tried >> up >> to 10000 concurrent connections. >> >> Test results didn't not show any visible difference between using >> multiple >> remote machines for client connections, one remote machine or localhost. >> >> I tried R13B02-1 and R12B-5 OTP versions. >> >> I found that memory usage grow is linear, as expected. But I came to the >> problem with CPU load. >> >> First of all, kernel polling didn't give any benefit for R13B02-1 (while >> erlang:system_info(kernel_poll) returned true and erl started with >> message >> [kernel-poll:true]). I've got about 55% of CPU usage with 4000 >> connections >> both with and without kernel polling enabled, while with R12B-5 I have >> about >> 26% of CPU usage with +Ktrue. I suspect a bug either in OTP or in gentoo >> ebuild (of course it's also quite possible that I'm doing something wrong >> or >> missed something in docs). >> >> The following is about R12B-5. I get about 6-7% of CPU load per every >> 1000 >> connections (about 60% CPU load for 10000 connections). I'm not sure if I >> should consider this as a good result or a bad one. Most of the articles >> on >> the same subject say that CPU load is negligible in their tests and they >> are >> fighting for memory only, so I expected I won't be CPU-limited too, but >> evidently I am. >> >> `top` says that about 50% of CPU load is userspace, 25% software >> interrupts, 20% system and 5% hardware interrupts (that's by eye, not >> very >> strict). >> >> I found that CPU load depends not as much on connection count but on >> transmitted packet count (ok, that's obviously the number of system >> calls). >> Thus if I send 4 packets per second, not 2, I should decrease the number >> of >> connections twice to preserve the same CPU load. >> >> CPU load does not depend on packet size. 1 byte or 1Kbyte - no visible >> difference. >> >> CPU usage is slightly less with active socket option enabled than with >> blocking recvs. >> >> CPU usage on the single windows client machine with 4000 connections >> spawned is on the same level as with the linux server handling these 4000 >> connections (while I expected linux to perform better). >> >> Switching Nagle on and off had no effect. I also tried to tune TCP stack >> with sysctl using advises found here and there but almost without any >> effect >> too. >> >> I've tried to trace with fprof and found that bottlenecks are 'send' >> operations (but I'm a relative novice to erlang so I'm not sure my usage >> of >> fprof was correct). Ok, that was expected too. I've read the 'why is >> gen_tcp:send slow?' thread but none of advises given there helped me. >> >> So the main question is: is the CPU usage of 7% per 1000 connections (or >> maybe better say 2000 packets per second) a good result? If no, what is >> the >> expected result? How can I improve my test application? Or maybe >> something >> in my story looks strange? >> >> I know that the possible optimization is decreasing the number of packets >> and keep it in mind. >> >> Here's the server connection process loop: >> >> loop(Socket) -> >> receive >> {tcp, Socket, _Packet} -> >> loop(Socket); >> {tcp_closed, Socket} -> >> normal; >> _ -> >> loop(Socket) >> after 500 -> >> gen_tcp:send(Socket,[?PACKET]), >> loop(Socket) >> end. >> >> Client loop has blocking recv and answers with send immediately. >> >> Thank you very much for your time. Sorry for too many words, I tried to >> provide all possible information. I will answer any question and >> appreciate >> any hint. >> >> Best regards, >> Andrey > ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From attila.r.nohl@REDACTED Thu Oct 15 16:08:47 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 15 Oct 2009 16:08:47 +0200 Subject: [erlang-questions] No Namespace for Macros In-Reply-To: References: Message-ID: <401d3ba30910150708y783eb9bbne66d454875570f33@mail.gmail.com> Macros (and records) can be defined outside modules, unlike functions. I think they are handled by a "preprocessor", not the compiler. Don't forget that macros in C++ are also outside classes and namespaces - the Erlang macro is modeled more like the C/C++ macros, not like Java variables. 2009/10/15, sapan shah : > As Anupag said it rightly & Bengt showed... there is a way around.. > But my curiosity is, > If it is for functions, why it is not for macros??? What would the > developers of Erlang have thought & made it like that??? > > On Thu, Oct 15, 2009 at 12:05 PM, Anupam Kapoor > wrote: > >> ofcourse another way to skin the cat is to return constants from >> functions in different modules... >> >> >> On Thu, Oct 15, 2009 at 11:54 AM, Anupam Kapoor >> wrote: >> >> For exapmle, Java supports this (public static final variables). >> >> >> >> Any Explanation????? >> > erlang doesn't have any notion of objects. in java, each '.java' file >> > contains a corresponding class-definition, and you can have multiple >> > instances of those objects defined... >> > >> > kind regards >> > anupam >> > >> > -- >> > In the beginning was the lambda, and the lambda was with Emacs, and >> > Emacs was the lambda. >> > >> >> >> >> -- >> In the beginning was the lambda, and the lambda was with Emacs, and >> Emacs was the lambda. >> > > > > -- > Regards, > Sapan Shah > From raimo+erlang-questions@REDACTED Thu Oct 15 17:28:45 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 15 Oct 2009 17:28:45 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <200910151502401455839@its3.ch> References: <20091015124727.GC24620@erix.ericsson.se> <200910151502401455839@its3.ch> Message-ID: <20091015152845.GA29367@erix.ericsson.se> On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > Dear Raimo, > I can resume: I discovered that gethostbyname call inet_gethost.c > I don't understand inet_gethost.c :-( > Where "goes" this problem for finding the information ? > If we know where and what, we could perhaps find what is missing in my windows 2003 ... > Hope to read you soon ! > John Well, there is not much more to say. inet_gethost.c calls struct hostent *gethostbyname(const char *name) in the winsock2 library: http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx It never returns. It is deprecated but that does not mean broken. I have a Windows 2003 server that it works on. We have not found the reason it differs between my server and your server. And that is about it. > > On Wed, Oct 14, 2009 at 03:34:12PM +0200, info wrote: > > For me this function always returns {error,timeout} and I didn't find the reason even with the help of Raimo Niskanen ! > > Sorry about that, but your problem became a just too spooky > Windows 2003 problem and that is not really my turf, and > you seemed to have a possible workaround through inet_res. > But it has been on my todo list to have a second look > at the whole conversation when I could find the time... > > > > > > > At the end of my last mail you have the references. > > Regards, > > > > Roberto Aloi > > Erlang Training and Consulting Ltd. > > http://www.erlang-consulting.com > > http://aloiroberto.wordpress.com > > > > Robert Raschke wrote: > > > This is part of the IP address spec (not sure where to look to find it, > > > though). I've seen addresses specified like this on and off in various > > > locations on a wide variety of OSes, for example in a /etc/hosts file you > > > can write: > > > > > > localhost 127.1 > > > > > > Robby > > > > > > On Tue, Oct 13, 2009 at 8:06 PM, caio ariede wrote: > > > > > > > > > > Interesting case. > > > > > > > > The result doesn't appear to be an issue, but a feature of the > > > > gethostbyname > > > > original implementation, in C code. > > > > > > > > You get the same result, testing with PHP: > > > > > > > > $ php -r 'var_dump(gethostbyname("12.27"));' > > > > string(9) "12.0.0.27" > > > > > > > > And other interesting results: > > > > > > > > $ php -r 'var_dump(gethostbyname("255.2.256"));' > > > > string(9) "255.2.1.0" > > > > > > > > But I can't see where it's really useful. > > > > > > > > Caio Ariede > > > > http://caioariede.com/ > > > > > > > > > > > > On Tue, Oct 13, 2009 at 3:15 PM, Garry Hodgson > > > > > > > > wrote: > > > > > > > > > > recently, a bug in my code caused us to pass a string > > > > > representing a floating point number to inet:gethostbyname(). > > > > > i would have expected it to return an error, but instead it > > > > > returned an ip address, but one that made no sense to me: > > > > > > > > > > 1 > inet:gethostbyname( '12.27' ). > > > > > {ok,{hostent,"12.27",[],inet,4,[{12,0,0,27}]}} > > > > > > > > > > so my question is, is this behavior correct, and if so, what > > > > > exactly does it mean that a lookup of '12.27' maps to '12.0.0.27'? > > > > > > > > > > thanks > > > > > > > > > > -- > > > > > Garry Hodgson > > > > > Lead Member of Technical Staff > > > > > AT&T Chief Security Office (CSO) > > > > > > > > > > "This e-mail and any files transmitted with it are AT&T property, are > > > > > confidential, and are intended solely for the use of the individual or > > > > > entity to whom this e-mail is addressed. If you are not one of the named > > > > > recipient(s) or otherwise have reason to believe that you have received > > > > > > > > > this > > > > > > > > > message in error, please notify the sender and delete this message > > > > > immediately from your computer. Any other use, retention, dissemination, > > > > > forwarding, printing, or copying of this e-mail is strictly prohibited." > > > > > > > > > > ________________________________________________________________ > > > > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > > > > erlang-questions (at) erlang.org > > > > > > > > > > > > > > > > > > > > > > > > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From rtrlists@REDACTED Thu Oct 15 17:48:38 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 15 Oct 2009 16:48:38 +0100 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <20091015152845.GA29367@erix.ericsson.se> References: <20091015124727.GC24620@erix.ericsson.se> <200910151502401455839@its3.ch> <20091015152845.GA29367@erix.ericsson.se> Message-ID: <6a3ae47e0910150848x2c6ddaa8i9edcc292b358657@mail.gmail.com> On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < raimo+erlang-questions@REDACTED > wrote: > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > Dear Raimo, > > I can resume: I discovered that gethostbyname call inet_gethost.c > > I don't understand inet_gethost.c :-( > > Where "goes" this problem for finding the information ? > > If we know where and what, we could perhaps find what is missing in my > windows 2003 ... > > Hope to read you soon ! > > John > > Well, there is not much more to say. inet_gethost.c calls > struct hostent *gethostbyname(const char *name) in the > winsock2 library: > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > It never returns. > It is deprecated but that does not mean broken. > I have a Windows 2003 server that it works on. > We have not found the reason it differs between > my server and your server. > And that is about it. > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server Version 4.0' installed. This is a specific add-on, nothing to do with Microsoft Firewall in your Network Properties. If I remove it, it works fine, if I add it it hangs. Apart from that the W2003 build is as vanilla as it gets. Nothing much to go on, but I would not be surprised if some system DLL is getting in the way. This is very hard to diagnose. One day I'll get around to compiling inet_gethost.exe in my setup, just to rule out poor linkage due to the VC++ setup used to build the shipping executables. If I compile up a trivial gethostbyname() example from MSDN, it works OK in all circumstances. There are a few strange dependencies in other shipped Windows files, for example beam.dll seems to depend on a particular MSVCR80.DLL with the version 8.0.50727.1433 . And if you have an older Windows machine, you get the completely unhelpful popup box telling you that your program could not be run. This has something to do with Windows Side-by-Side DLL deployment, and I completely fail to grasp what that's all about. Robby From kiszl@REDACTED Thu Oct 15 20:04:35 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Thu, 15 Oct 2009 20:04:35 +0200 Subject: [erlang-questions] No Namespace for Macros In-Reply-To: References: Message-ID: <4AD76433.5030108@tmit.bme.hu> Hello, You could stick to name your macros as modulename_macroname. %%%%valueHouse.hrl -define(valueHouse_Value, 10). %%%%valueCar.hrl -define(valueCar_Value, 2). star(X) -> case X of house -> ?valueHouse_Value; car -> ?valueCar_Value end. Regards, Zoltan. sapan shah wrote: > Hi, > > Erlang does have the concept of namespace for the functions( we do > lists:map(...)) > Why does it not have this concept for macros? > > Consider this scenario: I have an application which deals with the value of > some things. (lets say a house & a car) > So, I define macros in two different files for the things. > %%%%valueHouse.hrl > -define(Value, 10). > %%%%valueCar.hrl > -define(Value, 2). > > Now, My application file could be something like this > %%%application.erl > -module(application). > -include("valueHouse.hrl"). > -include("valueCar.hrl"). %% This will not compile... ll give 'redefining > macro error'... assume it is allowed in erlang > > star(X) -> > case X of house -> ?valueHouse:Value; > car -> ?valueCar:Value > end. > %%%%%%%%%%%%%%%%%% > > This is just a cooked up example. One may have this kind of need in a big > application. > For exapmle, Java supports this (public static final variables). > > Any Explanation????? > > Regards, > Sapan Shah > > From alfredo.fong@REDACTED Thu Oct 15 20:14:43 2009 From: alfredo.fong@REDACTED (Alfredo Fong Casas) Date: Thu, 15 Oct 2009 14:14:43 -0400 Subject: Flex+Erlang Message-ID: <1255630483.10386.9.camel@desarrollo10.scu.desoft.cu> Hi, some of you has communicate Erlang with Flex. Please a need an example of it and what i need for that. Thanks, Alfred From dave.smith.to@REDACTED Thu Oct 15 20:23:18 2009 From: dave.smith.to@REDACTED (Dave Smith) Date: Thu, 15 Oct 2009 14:23:18 -0400 Subject: [erlang-questions] Any way to correct the round off errors? In-Reply-To: References: <2a67d3ff0909200435q1ba72f0evc0f3ed544b9da344@mail.gmail.com> <4AB635EA.8070203@millstream.com> <1253460131.22238.51.camel@sredniczarny> <4AB65388.8050601@millstream.com> <603F5CF3-AD4B-4262-B4CD-5F096FC33EAE@cs.otago.ac.nz> Message-ID: <5ea453f90910151123y650ca599jeefd6cc040894ad6@mail.gmail.com> Sorry for reviving this old thread, but I missed it on the first pass and wanted to add my 2 cents. When most people talk of IEEE-754, they are referring to IEEE-754-1985 which which encompasses single and double precision floating points, which are both binary FP formats. Erlang's floating point type is the double precision binary. IEEE-754-2008 defines a much broader set of formats which includes 4 binary FP formats: binary16 (half precision) binary32 (single precision) binary64 (double precision) binary128 (quad precision) and 3 decimal FP formats: decimal32 decimal64 decimal 128 In addition there are the so-called interchange formats that are independent of machine architecture. The binary formats provide good precise values, but as has been pointed out, not all decimal values are represented in these formats. For scientific applications and applications where precise unrounded values are required, the binary formats are appropriate. For business applications where predictable rounded values are required, the decimal formats are appropriate. To get the results you are looking for you should use one of the decimal formats. Erlang does not support any decimal format natively, however, it can be supported in libraries. A couple of years ago I wrote a decimal linked-in driver that is backed by the IBM AlphaWorks DecNumber library. Calulations are done in decimal128 and the interface type is platform independent. Find it here: http://github.com/dsmith-to/decimal The library is very rough, but all the basic operations have been tested. I appologize, but the Makefile only works on mac-os-x. --DS 2009/9/21 Masklinn > On 21 Sep 2009, at 03:50 , Richard O'Keefe wrote: > >> >> The IEEE 754 standard is small and tolerably clear; >> there are drafts and summaries and review articles about it >> all over the web. >> > > First and foremost being Goldberg's "What Every Computer Scientist Should > Know About Floating-Point Arithmetic" ( > http://docs.sun.com/source/806-3568/ncg_goldberg.html) which is often > considered the most basic knowledge requirement before any discussion of > IEEE-754 floats. > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From max.lapshin@REDACTED Thu Oct 15 20:44:35 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 15 Oct 2009 22:44:35 +0400 Subject: [erlang-questions] Flex+Erlang In-Reply-To: <1255630483.10386.9.camel@desarrollo10.scu.desoft.cu> References: <1255630483.10386.9.camel@desarrollo10.scu.desoft.cu> Message-ID: Flex itself doesn't have any specific protocol. If you wish to use RTMP, you may try my fork of erlyvideo: http://github.com/maxlapshin/erlyvideo It will provide you two way function calling from flash to server and back. For example, it is the easiest way for cross-domain push. But much, much easier will be to got to old plain web-server from flash, however you will lack server-to-client channel. From matt.handler@REDACTED Thu Oct 15 22:51:37 2009 From: matt.handler@REDACTED (Matt Handler) Date: Thu, 15 Oct 2009 16:51:37 -0400 Subject: ets table creation in start script doesn't work Message-ID: <49427c610910151351v4dea7e0bmfa953c600d2298d9@mail.gmail.com> any clues why this doesn't work as expected: $ erl -eval "ets:new(some_table_name, [named_table])" when i get into the shell, it has no knowledge of the ets table "some_table_name" thanks, -matt From paul-trapexit@REDACTED Thu Oct 15 23:14:26 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Thu, 15 Oct 2009 14:14:26 -0700 (PDT) Subject: [erlang-questions] ets table creation in start script doesn't work In-Reply-To: <49427c610910151351v4dea7e0bmfa953c600d2298d9@mail.gmail.com> References: <49427c610910151351v4dea7e0bmfa953c600d2298d9@mail.gmail.com> Message-ID: The process that owns the ets table has died. Try this: % erl -eval 'spawn (fun () -> ets:new (some_table_name, [ named_table ]), receive after infinity -> ok end end)' Erlang (BEAM) emulator version 5.6.5 [source] [smp:2] [async-threads:0] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> ets:lookup (some_table_name, 1). [] -- p On Thu, 15 Oct 2009, Matt Handler wrote: > any clues why this doesn't work as expected: > > $ erl -eval "ets:new(some_table_name, [named_table])" > > when i get into the shell, it has no knowledge of the ets table > "some_table_name" > > thanks, > -matt > From rvirding@REDACTED Fri Oct 16 00:42:09 2009 From: rvirding@REDACTED (Robert Virding) Date: Fri, 16 Oct 2009 00:42:09 +0200 Subject: [erlang-questions] No Namespace for Macros In-Reply-To: <401d3ba30910150708y783eb9bbne66d454875570f33@mail.gmail.com> References: <401d3ba30910150708y783eb9bbne66d454875570f33@mail.gmail.com> Message-ID: <3dbc6d1c0910151542x71dc1427kb70f0d828c3d4b92@mail.gmail.com> Erlang macros are modeled directly on C macros, the version based on tokens and not on text. And as do C macros Erlang macros only exist within a preprocessor, they have no life of their own outside that. In this sense they are not like functions or variables from other languages. It is important to realise that an include file does not have a life of its own, it only exists within the file which includes it. So anything defined in it becomes part of the including file and in this respect it is perfectly logical that there is only one namespace. Anyway that is the reasoning behind the way they are. Robert 2009/10/15 Attila Rajmund Nohl > Macros (and records) can be defined outside modules, unlike functions. > I think they are handled by a "preprocessor", not the compiler. Don't > forget that macros in C++ are also outside classes and namespaces - > the Erlang macro is modeled more like the C/C++ macros, not like Java > variables. > > 2009/10/15, sapan shah : > > As Anupag said it rightly & Bengt showed... there is a way around.. > > But my curiosity is, > > If it is for functions, why it is not for macros??? What would the > > developers of Erlang have thought & made it like that??? > > > > On Thu, Oct 15, 2009 at 12:05 PM, Anupam Kapoor > > wrote: > > > >> ofcourse another way to skin the cat is to return constants from > >> functions in different modules... > >> > >> > >> On Thu, Oct 15, 2009 at 11:54 AM, Anupam Kapoor < > anupam.kapoor@REDACTED> > >> wrote: > >> >> For exapmle, Java supports this (public static final variables). > >> >> > >> >> Any Explanation????? > >> > erlang doesn't have any notion of objects. in java, each '.java' file > >> > contains a corresponding class-definition, and you can have multiple > >> > instances of those objects defined... > >> > > >> > kind regards > >> > anupam > >> > > >> > -- > >> > In the beginning was the lambda, and the lambda was with Emacs, and > >> > Emacs was the lambda. > >> > > >> > >> > >> > >> -- > >> In the beginning was the lambda, and the lambda was with Emacs, and > >> Emacs was the lambda. > >> > > > > > > > > -- > > Regards, > > Sapan Shah > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From rvirding@REDACTED Fri Oct 16 00:55:30 2009 From: rvirding@REDACTED (Robert Virding) Date: Fri, 16 Oct 2009 00:55:30 +0200 Subject: [erlang-patches] Epp Bugfix and macros overloading In-Reply-To: <4ACF4950.4070409@capflam.org> References: <4ACF4950.4070409@capflam.org> Message-ID: <3dbc6d1c0910151555r6f7c0209r3bc282a7bae53cec@mail.gmail.com> 2009/10/9 christopher faulet > Hi all, > > Here are two patches for the module epp in R13B02-1. > > * The second patch includes the first one and adds support of multiple > definitions for macros in the module epp (i.e. with the same name but > with different arities). This feature wouldn't break any code (I hope so > :) and might be usefull. > > Here is an simple example that uses it: > > ============================= > -module(test). > > -export([test/0]). > > -define(MY_MACRO(), foo). > -define(MY_MACRO, bar). > -define(MY_MACRO(X,Y), {X, Y}). > > > test() -> %% return [bar, foo, foo, {foo, bar}] > [ > ?MY_MACRO, %% use the 2nd def, replaced by bar > ?MY_MACRO(), %% use the 1st def, replaced by foo > ?MY_MACRO(foo), %% use the 2nd def, replaced by bar(foo) > ?MY_MACRO(foo, bar) %% use the 3rd def, replaced by {foo,bar} > ]. > > bar(X) -> > X. > ============================= > I rather like this extension, it is something I always meant to do but never got around to doing. I am not sure, though, that I completely agree with how you choose which definition to use, especially the third case here which I think should give an undefined macro error. My reasoning is that if you define a macro to have arguments, one or many definitions, and call it with arguments, even with an empty argument list then you should only try for the matching definition and not take the one without arguments. So if: -define(M, a). -define(M(), b). -define(M(X,Y), {X,Y}). then ?M - should use 1st def ?M() - should use 2nd def ?M(a,b) - should use 3rd def ?M(a) - should generate an error However if only: -define(M, a). then all calls will use this definition. Robert From ok@REDACTED Fri Oct 16 05:01:38 2009 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 16 Oct 2009 16:01:38 +1300 Subject: [erlang-questions] No Namespace for Macros In-Reply-To: References: Message-ID: <7C89B71C-D94F-4B50-B82D-F3D9D25CA856@cs.otago.ac.nz> On Oct 15, 2009, at 6:54 PM, sapan shah wrote: > Erlang does have the concept of namespace for the functions( we do > lists:map(...)) > Why does it not have this concept for macros? It does. It's just that you CAN refer to a function outside its namespace, but you CAN'T refer to a macro outside its namespace. > > > Consider this scenario: I have an application which deals with the > value of > some things. (lets say a house & a car) > So, I define macros in two different files for the things. > %%%%valueHouse.hrl > -define(Value, 10). > %%%%valueCar.hrl > -define(Value, 2). The obvious question is "why make them macros in the first place?" Another point is that these are include files, *NOT* modules. If you had % value-house.hrl value() -> 10. % value-car.hrl value() -> 2. neither of these definitions would be associated with anything other than the module that -included them. You would not expect ... -include('value-house.hrl'). -include('value-car.hrl'). ... to work. The namespace is the module, and here both functions would try to live inside the same module, but they can't. In the same way, if you do ... -include('valueHouse.hrl'). -include('valueCar.hrl'). ... you end up trying to provide to definitions for the same macro in the *SAME* name-space, namely the module. There is here no difference whatever between functions and macros. The only difference is that you can -export functions but you cannot -export macros, so the only way for a macro to be accessible in some name-space is for it to be defined in that name-space. > > This is just a cooked up example. One may have this kind of need in > a big > application. > For exapmle, Java supports this (public static final variables). Absolutely not. No way. public static final variables ARE NOT MACROS. Not even close. The Erlang equivalent of one of those Java thingies is a zero-argument exported function. (Admittedly such a thing cannot be used in pattern matching, but as you may have noticed, public static final variables can't be used in pattern matching in Java either.) Actually, for most purposes, the Erlang equivalent is *really* an atom. What you *really* want is abstract patterns. (I seem to say that a lot.) From sapan.gcet@REDACTED Fri Oct 16 06:03:00 2009 From: sapan.gcet@REDACTED (sapan shah) Date: Fri, 16 Oct 2009 09:33:00 +0530 Subject: [erlang-questions] No Namespace for Macros In-Reply-To: <3dbc6d1c0910151542x71dc1427kb70f0d828c3d4b92@mail.gmail.com> References: <401d3ba30910150708y783eb9bbne66d454875570f33@mail.gmail.com> <3dbc6d1c0910151542x71dc1427kb70f0d828c3d4b92@mail.gmail.com> Message-ID: On Fri, Oct 16, 2009 at 4:12 AM, Robert Virding wrote: > Erlang macros are modeled directly on C macros, the version based on tokens > and not on text. And as do C macros Erlang macros only exist within a > preprocessor, they have no life of their own outside that. In this sense > they are not like functions or variables from other languages. > > If we do the same thing in C, It surely gives 'redefining macro' as a warning (not as an error). & it considers the latest definition included in the file. > It is important to realise that an include file does not have a life of its > own, it only exists within the file which includes it. So anything defined > in it becomes part of the including file and in this respect it is > perfectly > logical that there is only one namespace. > > Anyway that is the reasoning behind the way they are. > > Robert > > 2009/10/15 Attila Rajmund Nohl > > > Macros (and records) can be defined outside modules, unlike functions. > > I think they are handled by a "preprocessor", not the compiler. Don't > > forget that macros in C++ are also outside classes and namespaces - > > the Erlang macro is modeled more like the C/C++ macros, not like Java > > variables. > > > > 2009/10/15, sapan shah : > > > As Anupag said it rightly & Bengt showed... there is a way around.. > > > But my curiosity is, > > > If it is for functions, why it is not for macros??? What would the > > > developers of Erlang have thought & made it like that??? > > > > > > On Thu, Oct 15, 2009 at 12:05 PM, Anupam Kapoor > > > wrote: > > > > > >> ofcourse another way to skin the cat is to return constants from > > >> functions in different modules... > > >> > > >> > > >> On Thu, Oct 15, 2009 at 11:54 AM, Anupam Kapoor < > > anupam.kapoor@REDACTED> > > >> wrote: > > >> >> For exapmle, Java supports this (public static final variables). > > >> >> > > >> >> Any Explanation????? > > >> > erlang doesn't have any notion of objects. in java, each '.java' > file > > >> > contains a corresponding class-definition, and you can have multiple > > >> > instances of those objects defined... > > >> > > > >> > kind regards > > >> > anupam > > >> > > > >> > -- > > >> > In the beginning was the lambda, and the lambda was with Emacs, and > > >> > Emacs was the lambda. > > >> > > > >> > > >> > > >> > > >> -- > > >> In the beginning was the lambda, and the lambda was with Emacs, and > > >> Emacs was the lambda. > > >> > > > > > > > > > > > > -- > > > Regards, > > > Sapan Shah > > > > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > > > > -- Regards, Sapan Shah From sapan.gcet@REDACTED Fri Oct 16 06:13:16 2009 From: sapan.gcet@REDACTED (sapan shah) Date: Fri, 16 Oct 2009 09:43:16 +0530 Subject: [erlang-questions] No Namespace for Macros In-Reply-To: <7C89B71C-D94F-4B50-B82D-F3D9D25CA856@cs.otago.ac.nz> References: <7C89B71C-D94F-4B50-B82D-F3D9D25CA856@cs.otago.ac.nz> Message-ID: On Fri, Oct 16, 2009 at 8:31 AM, Richard O'Keefe wrote: > > On Oct 15, 2009, at 6:54 PM, sapan shah wrote: > >> Erlang does have the concept of namespace for the functions( we do >> lists:map(...)) >> Why does it not have this concept for macros? >> > > It does. It's just that you CAN refer to a function outside its > namespace, but you CAN'T refer to a macro outside its namespace. > >> >> >> Consider this scenario: I have an application which deals with the value >> of >> some things. (lets say a house & a car) >> So, I define macros in two different files for the things. >> %%%%valueHouse.hrl >> -define(Value, 10). >> %%%%valueCar.hrl >> -define(Value, 2). >> > > The obvious question is "why make them macros in the first place?" > > Another point is that these are include files, *NOT* modules. > > If you had > > % value-house.hrl > value() -> 10. > > % value-car.hrl > value() -> 2. > > neither of these definitions would be associated with anything other > than the module that -included them. You would not expect > > ... > -include('value-house.hrl'). > -include('value-car.hrl'). > ... > to work. The namespace is the module, and here both functions would > try to live inside the same module, but they can't. > > In the same way, if you do > > ... > -include('valueHouse.hrl'). > -include('valueCar.hrl'). > ... > you end up trying to provide to definitions for the same macro > in the *SAME* name-space, namely the module. There is here no > difference whatever between functions and macros. > > The only difference is that you can -export functions but you > cannot -export macros, so the only way for a macro to be accessible > in some name-space is for it to be defined in that name-space. > Yes, I got this point. You can export functions but not macros... but, Why did the erlang developer not give -export functionality to macros, What was it that restricted them??? > >> > This is just a cooked up example. One may have this kind of need in a big >> application. >> For exapmle, Java supports this (public static final variables). >> > > > Absolutely not. No way. public static final variables ARE NOT MACROS. > Not even close. The Erlang equivalent of one of those Java thingies is > a zero-argument exported function. (Admittedly such a thing cannot be > used in pattern matching, but as you may have noticed, public static > final variables can't be used in pattern matching in Java either.) > > Sure, I missed this part. public static final variables are sort of consts not macros. > Actually, for most purposes, the Erlang equivalent is *really* an atom. > > What you *really* want is abstract patterns. > I did not get this part. It would be very nice if you can elaborate more on this. > (I seem to say that a lot.) > > > -- Regards, Sapan Shah From ok@REDACTED Fri Oct 16 06:43:41 2009 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 16 Oct 2009 17:43:41 +1300 Subject: [erlang-questions] No Namespace for Macros In-Reply-To: References: <7C89B71C-D94F-4B50-B82D-F3D9D25CA856@cs.otago.ac.nz> Message-ID: On Oct 16, 2009, at 5:13 PM, sapan shah wrote: > Yes, I got this point. You can export functions but not macros... but, > Why did the erlang developer not give -export functionality to > macros, What was it that restricted them??? Several of the Erlang developers had had experience with Lisp before developing Erlang. I have had enough experience with hot-loading Lisp code that used macros NEVER TO WANT TO GO THERE AGAIN. (In particular, a certain integrated development environment which preceded the adoption of the buzz-phrase IDE let you get yourself into a state in which there was *no* consistent load order.) It's one of the reasons why Quintus, who wanted macros themselves, resolutely ignored customer requests for macros in Prolog. The "feature" was mainly useful for shooting yourself in the throat. Macros are a feature that should never have been in Erlang in the first place. (With LFE, there's an argument for them. With plain Erlang, not.) You may have noticed that C doesn't export macros either. For the same reason: macros are a parse-time-ONLY construction; there simply isn't anything *to* export in the cpp model. Always remember that one of the major features for Erlang has always been its ability to dynamically load and unload modules, and if anything creates amazingly strong and amazingly nasty coupling between modules, macros do. In Erlang at the moment, we have *untracked* *indirect* coupling between modules that -include the same .hrl file. This is just one of many reasons why cpp-style macros passed their use-by date a decade or so before they were adopted. > What you *really* want is abstract patterns. > > I did not get this part. It would be very nice if you can elaborate > more on this. I have written about abstract patterns frequently in this mailing list; search the archives. There's an EEP for a simplified proposal, http://www.erlang.org/eeps/eep-0029.html Macros give you three things: A. The ability to define things that act like functions. For this purpose, it's better to define functions, and tell the compiler to inline them. (That's why C++ has 'inline'. Erlang has inline declarations too.) B. The ability to define things that act like constants and other patterns, and the ability to write quasi-functional things that can be used in guards. Abstract patterns do this. Abstract patterns are function-like things that would have a function-like representation in a .beam function, and could be imported and -exported. In your example, something like -module(house). -export([...,#value/0]). #value() -> 20. ... -module(car). -export([...,#value/0]). #value() -> 10. ... or whatever the magic numbers were, would do the job. You'd be able to use #house:value() and #car:value() in patterns as well as expressions. C. The ability to do things that warp the syntax of the language beyond recognition. Want ?BEGIN ... ?END with local variables? -define(BEGIN, (fun () ->). -define(END, end)()). zum(X) -> ?BEGIN Y = X+1, Y*2 ?END. It is an advantage not to be able to do this. From xavier@REDACTED Fri Oct 16 06:43:46 2009 From: xavier@REDACTED (Xavier Maillard) Date: Fri, 16 Oct 2009 06:43:46 +0200 Subject: erlang slang: porting it to slang2 ? Message-ID: <4AD7FA02.1090404@maillard.im> Hi, does anyone know what exact modifications should be done to the jungerl's lib slang in order to be built agains libslang2 ? Slang1 is pretty old now and it is hard to find a slang1 library that just work with recent kernels. My goal is to, eventually, make ermacs working and, if time permits it, to enhance it. This would be my first erlang development but I am really motivated in doing something with erlang :D Xavier From dj3vande@REDACTED Fri Oct 16 06:41:28 2009 From: dj3vande@REDACTED (Dave Vandervies) Date: Fri, 16 Oct 2009 00:41:28 -0400 Subject: [erlang-questions] No Namespace for Macros In-Reply-To: References: <401d3ba30910150708y783eb9bbne66d454875570f33@mail.gmail.com> <3dbc6d1c0910151542x71dc1427kb70f0d828c3d4b92@mail.gmail.com> Message-ID: <20091016044128.GA31623@goofy.davenet> Somebody claiming to be sapan shah wrote: > On Fri, Oct 16, 2009 at 4:12 AM, Robert Virding wrote: > > > Erlang macros are modeled directly on C macros, the version based on tokens > > and not on text. And as do C macros Erlang macros only exist within a > > preprocessor, they have no life of their own outside that. In this sense > > they are not like functions or variables from other languages. > > > > If we do the same thing in C, It surely gives 'redefining macro' as a > warning (not as an error). > & it considers the latest definition included in the file. According to the C standard (well, the public draft of it that I have a copy of): -------- 6.10.3 Macro replacement Constraints 1 Two replacement lists are identical if and only if the preprocessing tokens in both have the same number, ordering, spelling, and white-space separation, where all white-space separations are considered identical. 2 An identifier currently defined as an object-like macro shall not be redefined by another #define preprocessing directive unless the second definition is an object-like macro definition and the two replacement lists are identical. Likewise, an identifier currently defined as a function-like macro shall not be redefined by another #define preprocessing directive unless the second definition is a function-like macro definition that has the same number and spelling of parameters, and the two replacement lists are identical. [...] -------- Without getting into C language lawyering, the violation of a "shall" in a "constraints" section is the closest thing the C language (as opposed to a particular implementation) has to a fatal error. So, your compiler might be willing to accept it, but that doesn't really mean much (and if it claims to be a C compiler it's required to at the very least issue a warning). In any case, why is arbitrarily choosing the last (or first) definition instead of failing desireable? If the compiler accepts one definition, anything that depends on the meaning attached to the other definition(s) is now silently broken. A C compiler can get away with that, since C has a long and honorable tradition of giving programmers enough rope to hang themselves and then some, but even the authors of the C standard don't seem to think it SHOULD behave that way. So why should other languages do anything other than issue a diagnostic and refuse to complete the compilation? dave -- Dave Vandervies dj3vande@REDACTED Plan your future! Make God laugh! From zerthurd@REDACTED Fri Oct 16 11:47:06 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Fri, 16 Oct 2009 16:47:06 +0700 Subject: [erlang-bugs] erl -make returns ok status after compilation error Message-ID: Hello Make in erlang dose not handles errors in compilation. If some files in application broken, compilation with erlc returns 1, while erl -make returns 0 what indicates that compilation i successful. So, when we uses Makefile, where some targets should been compiled with erl -make, and when one target compilation is broken, make cannot catch this. Thank you -- Maxim Treskin From raimo+erlang-questions@REDACTED Fri Oct 16 12:13:22 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 16 Oct 2009 12:13:22 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <6a3ae47e0910150848x2c6ddaa8i9edcc292b358657@mail.gmail.com> References: <20091015124727.GC24620@erix.ericsson.se> <200910151502401455839@its3.ch> <20091015152845.GA29367@erix.ericsson.se> <6a3ae47e0910150848x2c6ddaa8i9edcc292b358657@mail.gmail.com> Message-ID: <20091016101322.GA4308@erix.ericsson.se> On Thu, Oct 15, 2009 at 04:48:38PM +0100, Robert Raschke wrote: > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > raimo+erlang-questions@REDACTED > > wrote: > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > Dear Raimo, > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > I don't understand inet_gethost.c :-( > > > Where "goes" this problem for finding the information ? > > > If we know where and what, we could perhaps find what is missing in my > > windows 2003 ... > > > Hope to read you soon ! > > > John > > > > Well, there is not much more to say. inet_gethost.c calls > > struct hostent *gethostbyname(const char *name) in the > > winsock2 library: > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > It never returns. > > It is deprecated but that does not mean broken. > > I have a Windows 2003 server that it works on. > > We have not found the reason it differs between > > my server and your server. > > And that is about it. > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > Version 4.0' installed. This is a specific add-on, nothing to do with > Microsoft Firewall in your Network Properties. > > If I remove it, it works fine, if I add it it hangs. Apart from that the > W2003 build is as vanilla as it gets. > This rings a bell for our Windows guys. Firewall software on Windows do nasty things. They e.g replace the Winsock API for other applications so things you had to do before suddenly breaks. Especially since inet_gethost.exe does calls from worker threads the API replacement may mess up. We have had hanging gethostbyname() for certain firewalls. We will install virtual a Windows 2003 server machine, the Microsoft Firewall Client for ISA Server seems to be downloadable http://www.microsoft.com/DownLoads/details.aspx?FamilyID=05c2c932-b15a-4990-b525-66380743da89&displaylang=en from Microsoft, but where to find, if we need, the Microsoft ISA Server I do not know... Then we will try to reproduce the problem. > Nothing much to go on, but I would not be surprised if some system DLL is > getting in the way. This is very hard to diagnose. > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > rule out poor linkage due to the VC++ setup used to build the shipping > executables. If I compile up a trivial gethostbyname() example from MSDN, it > works OK in all circumstances. > > There are a few strange dependencies in other shipped Windows files, for > example beam.dll seems to depend on a particular MSVCR80.DLL with the > version 8.0.50727.1433 . And if you have an older Windows machine, you get > the completely unhelpful popup box telling you that your program could not > be run. This has something to do with Windows Side-by-Side DLL deployment, > and I completely fail to grasp what that's all about. > > Robby -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From dennis.novikov@REDACTED Fri Oct 16 15:34:18 2009 From: dennis.novikov@REDACTED (Dennis Novikov) Date: Fri, 16 Oct 2009 16:34:18 +0300 Subject: [erlang-questions] [erlang-bugs] erl -make returns ok status after compilation error In-Reply-To: References: Message-ID: <8c412f0c0910160634n4535b01cv7d7140e5cc8a1fca@mail.gmail.com> On Fri, Oct 16, 2009 at 12:47 PM, Maxim Treskin wrote: > > Hello > > Make in erlang dose not handles errors in compilation. > > If some files in application broken, compilation with erlc returns 1, > while erl -make returns 0 what indicates that compilation i > successful. > So, when we uses Makefile, where some targets should been compiled > with erl -make, and when one target compilation is broken, make cannot > catch this. I had to use wrapper like this: -module(mk). -export([all/0]). all() -> ? ? case make:all() of ? ? ? up_to_date -> ? ? ? erlang:halt(0); ? ? _ -> ? ? ? erlang:halt(1) end. erl -noinput -noshell -s mk all From rtrlists@REDACTED Fri Oct 16 15:38:41 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 16 Oct 2009 14:38:41 +0100 Subject: [erlang-questions] [erlang-bugs] erl -make returns ok status after compilation error In-Reply-To: <8c412f0c0910160634n4535b01cv7d7140e5cc8a1fca@mail.gmail.com> References: <8c412f0c0910160634n4535b01cv7d7140e5cc8a1fca@mail.gmail.com> Message-ID: <6a3ae47e0910160638t2d5cb09al43ad65d5169df0ca@mail.gmail.com> On Fri, Oct 16, 2009 at 2:34 PM, Dennis Novikov wrote: > I had to use wrapper like this: > > -module(mk). > > -export([all/0]). > > all() -> > case make:all() of > up_to_date -> > erlang:halt(0); > _ -> > erlang:halt(1) > end. > > erl -noinput -noshell -s mk all > > You can also do this in one command without the need for a module: erl -noinput -eval "case make:all() of up_to_date -> halt(0); _ -> halt(1) end." Robby From info@REDACTED Fri Oct 16 16:24:53 2009 From: info@REDACTED (info) Date: Fri, 16 Oct 2009 16:24:53 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? References: <20091015124727.GC24620@erix.ericsson.se>, <200910151502401455839@its3.ch>, <20091015152845.GA29367@erix.ericsson.se> Message-ID: <200910161624530725939@its3.ch> My idea was to analyse the return parameters because gethostbyname doesn't return "timeout". Who and where decides to transform an error code in timeout code ? I hope to escalade the problem like this ... John On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > Dear Raimo, > I can resume: I discovered that gethostbyname call inet_gethost.c > I don't understand inet_gethost.c :-( > Where "goes" this problem for finding the information ? > If we know where and what, we could perhaps find what is missing in my windows 2003 ... > Hope to read you soon ! > John Well, there is not much more to say. inet_gethost.c calls struct hostent *gethostbyname(const char *name) in the winsock2 library: http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx It never returns. It is deprecated but that does not mean broken. I have a Windows 2003 server that it works on. We have not found the reason it differs between my server and your server. And that is about it. > > On Wed, Oct 14, 2009 at 03:34:12PM +0200, info wrote: > > For me this function always returns {error,timeout} and I didn't find the reason even with the help of Raimo Niskanen ! > > Sorry about that, but your problem became a just too spooky > Windows 2003 problem and that is not really my turf, and > you seemed to have a possible workaround through inet_res. > But it has been on my todo list to have a second look > at the whole conversation when I could find the time... > > > > > > > At the end of my last mail you have the references. > > Regards, > > > > Roberto Aloi > > Erlang Training and Consulting Ltd. > > http://www.erlang-consulting.com > > http://aloiroberto.wordpress.com > > > > Robert Raschke wrote: > > > This is part of the IP address spec (not sure where to look to find it, > > > though). I've seen addresses specified like this on and off in various > > > locations on a wide variety of OSes, for example in a /etc/hosts file you > > > can write: > > > > > > localhost 127.1 > > > > > > Robby > > > > > > On Tue, Oct 13, 2009 at 8:06 PM, caio ariede wrote: > > > > > > > > > > Interesting case. > > > > > > > > The result doesn't appear to be an issue, but a feature of the > > > > gethostbyname > > > > original implementation, in C code. > > > > > > > > You get the same result, testing with PHP: > > > > > > > > $ php -r 'var_dump(gethostbyname("12.27"));' > > > > string(9) "12.0.0.27" > > > > > > > > And other interesting results: > > > > > > > > $ php -r 'var_dump(gethostbyname("255.2.256"));' > > > > string(9) "255.2.1.0" > > > > > > > > But I can't see where it's really useful. > > > > > > > > Caio Ariede > > > > http://caioariede.com/ > > > > > > > > > > > > On Tue, Oct 13, 2009 at 3:15 PM, Garry Hodgson > > > > > > > > wrote: > > > > > > > > > > recently, a bug in my code caused us to pass a string > > > > > representing a floating point number to inet:gethostbyname(). > > > > > i would have expected it to return an error, but instead it > > > > > returned an ip address, but one that made no sense to me: > > > > > > > > > > 1 > inet:gethostbyname( '12.27' ). > > > > > {ok,{hostent,"12.27",[],inet,4,[{12,0,0,27}]}} > > > > > > > > > > so my question is, is this behavior correct, and if so, what > > > > > exactly does it mean that a lookup of '12.27' maps to '12.0.0.27'? > > > > > > > > > > thanks > > > > > > > > > > -- > > > > > Garry Hodgson > > > > > Lead Member of Technical Staff > > > > > AT&T Chief Security Office (CSO) > > > > > > > > > > "This e-mail and any files transmitted with it are AT&T property, are > > > > > confidential, and are intended solely for the use of the individual or > > > > > entity to whom this e-mail is addressed. If you are not one of the named > > > > > recipient(s) or otherwise have reason to believe that you have received > > > > > > > > > this > > > > > > > > > message in error, please notify the sender and delete this message > > > > > immediately from your computer. Any other use, retention, dissemination, > > > > > forwarding, printing, or copying of this e-mail is strictly prohibited." > > > > > > > > > > ________________________________________________________________ > > > > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > > > > erlang-questions (at) erlang.org > > > > > > > > > > > > > > > > > > > > > > > > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From info@REDACTED Fri Oct 16 16:29:20 2009 From: info@REDACTED (info) Date: Fri, 16 Oct 2009 16:29:20 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? References: <20091015124727.GC24620@erix.ericsson.se>, <200910151502401455839@its3.ch>, <20091015152845.GA29367@erix.ericsson.se>, <6a3ae47e0910150848x2c6ddaa8i9edcc292b358657@mail.gmail.com> Message-ID: <200910161629195074975@its3.ch> On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < raimo+erlang-questions@REDACTED > wrote: > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > Dear Raimo, > > I can resume: I discovered that gethostbyname call inet_gethost.c > > I don't understand inet_gethost.c :-( > > Where "goes" this problem for finding the information ? > > If we know where and what, we could perhaps find what is missing in my > windows 2003 ... > > Hope to read you soon ! > > John > > Well, there is not much more to say. inet_gethost.c calls > struct hostent *gethostbyname(const char *name) in the > winsock2 library: > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > It never returns. > It is deprecated but that does not mean broken. > I have a Windows 2003 server that it works on. > We have not found the reason it differs between > my server and your server. > And that is about it. > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server Version 4.0' installed. This is a specific add-on, nothing to do with Microsoft Firewall in your Network Properties. but I don't use ISA server !!! If I remove it, it works fine, if I add it it hangs. Apart from that the W2003 build is as vanilla as it gets. Nothing much to go on, but I would not be surprised if some system DLL is getting in the way. This is very hard to diagnose. One day I'll get around to compiling inet_gethost.exe in my setup, just to rule out poor linkage due to the VC++ setup used to build the shipping executables. If I compile up a trivial gethostbyname() example from MSDN, it works OK in all circumstances. There are a few strange dependencies in other shipped Windows files, for example beam.dll seems to depend on a particular MSVCR80.DLL with the version 8.0.50727.1433 . And if you have an older Windows machine, you get the completely unhelpful popup box telling you that your program could not be run. This has something to do with Windows Side-by-Side DLL deployment, and I completely fail to grasp what that's all about. but I don't have msvcr80.dll !!! Robby From zerthurd@REDACTED Fri Oct 16 16:29:42 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Fri, 16 Oct 2009 21:29:42 +0700 Subject: [erlang-questions] [erlang-bugs] erl -make returns ok status after compilation error In-Reply-To: <6a3ae47e0910160638t2d5cb09al43ad65d5169df0ca@mail.gmail.com> References: <8c412f0c0910160634n4535b01cv7d7140e5cc8a1fca@mail.gmail.com> <6a3ae47e0910160638t2d5cb09al43ad65d5169df0ca@mail.gmail.com> Message-ID: Thank you, Dennis and Robert! Yes, this is solution of issue, but it looks like some kind of tricks. It is more preferable to fix this behaviour in make.erl. -- Maxim Treskin From zerthurd@REDACTED Fri Oct 16 16:32:21 2009 From: zerthurd@REDACTED (Maxim Treskin) Date: Fri, 16 Oct 2009 21:32:21 +0700 Subject: [erlang-questions] [erlang-bugs] erl -make returns ok status after compilation error In-Reply-To: References: <8c412f0c0910160634n4535b01cv7d7140e5cc8a1fca@mail.gmail.com> <6a3ae47e0910160638t2d5cb09al43ad65d5169df0ca@mail.gmail.com> Message-ID: > It is more preferable to fix this behaviour in make.erl. Or in other place, where make:all/0 is called. -- Maxim Treskin From christopher.faulet@REDACTED Fri Oct 16 17:16:46 2009 From: christopher.faulet@REDACTED (christopher faulet) Date: Fri, 16 Oct 2009 17:16:46 +0200 Subject: [erlang-patches] Epp Bugfix and macros overloading In-Reply-To: <3dbc6d1c0910151555r6f7c0209r3bc282a7bae53cec@mail.gmail.com> References: <4ACF4950.4070409@capflam.org> <3dbc6d1c0910151555r6f7c0209r3bc282a7bae53cec@mail.gmail.com> Message-ID: <4AD88E5E.7060809@capflam.org> Robert Virding a ?crit : > I rather like this extension, it is something I always meant to do but never > got around to doing. > > I am not sure, though, that I completely agree with how you choose which > definition to use, especially the third case here which I think should give > an undefined macro error. My reasoning is that if you define a macro to have > arguments, one or many definitions, and call it with arguments, even with an > empty argument list then you should only try for the matching definition and > not take the one without arguments. So if: > > -define(M, a). > -define(M(), b). > -define(M(X,Y), {X,Y}). > > then > > ?M - should use 1st def > ?M() - should use 2nd def > ?M(a,b) - should use 3rd def > ?M(a) - should generate an error > > However if only: > > -define(M, a). > > then all calls will use this definition. > Hi Robert, Thanks for your comments. You are probably right. Your solution is less error prone and avoids ambiguities. I made a new patch that takes your suggestion into account. I also modified some errors to be more precise in diagnostics. -- Christopher Faulet -------------- next part -------------- A non-text attachment was scrubbed... Name: overload-epp-R13B2-1.patch Type: text/x-patch Size: 12576 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From raimo+erlang-questions@REDACTED Fri Oct 16 17:27:51 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 16 Oct 2009 17:27:51 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <200910161624530725939@its3.ch> References: <20091015152845.GA29367@erix.ericsson.se> <200910161624530725939@its3.ch> Message-ID: <20091016152751.GA15372@erix.ericsson.se> On Fri, Oct 16, 2009 at 04:24:53PM +0200, info wrote: > My idea was to analyse the return parameters because gethostbyname doesn't return "timeout". > Who and where decides to transform an error code in timeout code ? > I hope to escalade the problem like this ... > John The debug trick I showed earlier (inet_gethost_native:control({debug_level,99})) would show the return value, but as I understood it it never returns; it gets interrupted from inet_gethost.erl, even if you set the timeout ridiculously high, so we have no return value to go on... > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > Dear Raimo, > > I can resume: I discovered that gethostbyname call inet_gethost.c > > I don't understand inet_gethost.c :-( > > Where "goes" this problem for finding the information ? > > If we know where and what, we could perhaps find what is missing in my windows 2003 ... > > Hope to read you soon ! > > John > > Well, there is not much more to say. inet_gethost.c calls > struct hostent *gethostbyname(const char *name) in the > winsock2 library: > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > It never returns. > It is deprecated but that does not mean broken. > I have a Windows 2003 server that it works on. > We have not found the reason it differs between > my server and your server. > And that is about it. > > > > > > On Wed, Oct 14, 2009 at 03:34:12PM +0200, info wrote: > > > For me this function always returns {error,timeout} and I didn't find the reason even with the help of Raimo Niskanen ! > > > > Sorry about that, but your problem became a just too spooky > > Windows 2003 problem and that is not really my turf, and > > you seemed to have a possible workaround through inet_res. > > But it has been on my todo list to have a second look > > at the whole conversation when I could find the time... > > > > > > > > > > > At the end of my last mail you have the references. > > > Regards, > > > > > > Roberto Aloi > > > Erlang Training and Consulting Ltd. > > > http://www.erlang-consulting.com > > > http://aloiroberto.wordpress.com > > > > > > Robert Raschke wrote: > > > > This is part of the IP address spec (not sure where to look to find it, > > > > though). I've seen addresses specified like this on and off in various > > > > locations on a wide variety of OSes, for example in a /etc/hosts file you > > > > can write: > > > > > > > > localhost 127.1 > > > > > > > > Robby > > > > > > > > On Tue, Oct 13, 2009 at 8:06 PM, caio ariede wrote: > > > > > > > > > > > > > Interesting case. > > > > > > > > > > The result doesn't appear to be an issue, but a feature of the > > > > > gethostbyname > > > > > original implementation, in C code. > > > > > > > > > > You get the same result, testing with PHP: > > > > > > > > > > $ php -r 'var_dump(gethostbyname("12.27"));' > > > > > string(9) "12.0.0.27" > > > > > > > > > > And other interesting results: > > > > > > > > > > $ php -r 'var_dump(gethostbyname("255.2.256"));' > > > > > string(9) "255.2.1.0" > > > > > > > > > > But I can't see where it's really useful. > > > > > > > > > > Caio Ariede > > > > > http://caioariede.com/ > > > > > > > > > > > > > > > On Tue, Oct 13, 2009 at 3:15 PM, Garry Hodgson > > > > > > > > > > wrote: > > > > > > > > > > > > recently, a bug in my code caused us to pass a string > > > > > > representing a floating point number to inet:gethostbyname(). > > > > > > i would have expected it to return an error, but instead it > > > > > > returned an ip address, but one that made no sense to me: > > > > > > > > > > > > 1 > inet:gethostbyname( '12.27' ). > > > > > > {ok,{hostent,"12.27",[],inet,4,[{12,0,0,27}]}} > > > > > > > > > > > > so my question is, is this behavior correct, and if so, what > > > > > > exactly does it mean that a lookup of '12.27' maps to '12.0.0.27'? > > > > > > > > > > > > thanks > > > > > > > > > > > > -- > > > > > > Garry Hodgson > > > > > > Lead Member of Technical Staff > > > > > > AT&T Chief Security Office (CSO) > > > > > > > > > > > > "This e-mail and any files transmitted with it are AT&T property, are > > > > > > confidential, and are intended solely for the use of the individual or > > > > > > entity to whom this e-mail is addressed. If you are not one of the named > > > > > > recipient(s) or otherwise have reason to believe that you have received > > > > > > > > > > > this > > > > > > > > > > > message in error, please notify the sender and delete this message > > > > > > immediately from your computer. Any other use, retention, dissemination, > > > > > > forwarding, printing, or copying of this e-mail is strictly prohibited." > > > > > > > > > > > > ________________________________________________________________ > > > > > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > > > > > erlang-questions (at) erlang.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ________________________________________________________________ > > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > > erlang-questions (at) erlang.org > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > ________________________________________________________________ > > erlang-questions mailing list. See http://www.erlang.org/faq.html > > erlang-questions (at) erlang.org > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Fri Oct 16 17:33:33 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 16 Oct 2009 17:33:33 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <200910161629195074975@its3.ch> References: <6a3ae47e0910150848x2c6ddaa8i9edcc292b358657@mail.gmail.com> <200910161629195074975@its3.ch> Message-ID: <20091016153333.GA15561@erix.ericsson.se> On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: I guess Robby wrote: > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > raimo+erlang-questions@REDACTED > > wrote: > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > Dear Raimo, > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > I don't understand inet_gethost.c :-( > > > Where "goes" this problem for finding the information ? > > > If we know where and what, we could perhaps find what is missing in my > > windows 2003 ... > > > Hope to read you soon ! > > > John > > > > Well, there is not much more to say. inet_gethost.c calls > > struct hostent *gethostbyname(const char *name) in the > > winsock2 library: > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > It never returns. > > It is deprecated but that does not mean broken. > > I have a Windows 2003 server that it works on. > > We have not found the reason it differs between > > my server and your server. > > And that is about it. > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > Version 4.0' installed. This is a specific add-on, nothing to do with > Microsoft Firewall in your Network Properties. > I guess info wrote: > but I don't use ISA server !!! > No other interesting firewall software? I guess Robby wrote: > If I remove it, it works fine, if I add it it hangs. Apart from that the > W2003 build is as vanilla as it gets. > > Nothing much to go on, but I would not be surprised if some system DLL is > getting in the way. This is very hard to diagnose. > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > rule out poor linkage due to the VC++ setup used to build the shipping > executables. If I compile up a trivial gethostbyname() example from MSDN, it > works OK in all circumstances. > > There are a few strange dependencies in other shipped Windows files, for > example beam.dll seems to depend on a particular MSVCR80.DLL with the > version 8.0.50727.1433 . And if you have an older Windows machine, you get > the completely unhelpful popup box telling you that your program could not > be run. This has something to do with Windows Side-by-Side DLL deployment, > and I completely fail to grasp what that's all about. > I guess info wrote: > but I don't have msvcr80.dll !!! I guess you do, actually, since Erlang/OTP's installer is supposed to install that DLL, if it does not exist already. I guess Robby wrote: > > Robby -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From kamiseq@REDACTED Fri Oct 16 19:07:43 2009 From: kamiseq@REDACTED (kamiseq) Date: Fri, 16 Oct 2009 19:07:43 +0200 Subject: [erlang-questions] tcp connection with timeout In-Reply-To: <4AD6461B.8060200@m5net.com> References: <4AD6461B.8060200@m5net.com> Message-ID: hi, it is all true, but if I am working with the server that only listen to one client and is responding to its requests and if I am able to set send_timout to some value then after this time I expect that I will not get any response form the server no matter what so I can execute some logic, try other request or repeat action or shut down or whatever. 2009/10/14 Bernard Duggan > kamiseq wrote: > > hi, > > is there a way to discard any message from peer after set timeout occurs? > A > > server should respond in given time or transmission should be repeated. > > > > I implemented small ex where one process is listening and responding in > > random time (let say 1 to 3 sek). and it is always accepting exactly one > > process during its life. > > > I'm not entirely sure what you mean by "accepting exactly one process". > > other process is connecting via tcp and is sending msg. I set > send_timeout > > flag for socket and a timer on receive after to let say 500ms and 1000ms > so > > it should get timeout all the time. > > > send_timeout does not do what you seem to think it does. The > acknowledgement of the send is generated by the far end's TCP stack > before your code ever sees the packet. > > when I receive timeout I send special msg that server should repeat last > > response. all this is great but then I will receive two responses from > > server with timeouted and then repeated msg. > > > > in fact Im receiving all "lost" messages that were sent to my client by > > server. I could maybe add some ticket to the msg and take out any > incoming > > messages that are not matching but this is impossible right now (afac) as > I > > will break protocol. > > > I think your basic problem here is that you're using TCP, but trying to > re-implement part of it (lost/timed-out message retransmission) on top > of it. If you want to tinker with stuff at that level, use a UDP > connection instead. > > Cheers, > > Bernard > > -- pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From info@REDACTED Sat Oct 17 01:46:50 2009 From: info@REDACTED (info) Date: Sat, 17 Oct 2009 00:46:50 +0100 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? References: <6a3ae47e0910150848x2c6ddaa8i9edcc292b358657@mail.gmail.com>, <200910161629195074975@its3.ch>, <20091016153333.GA15561@erix.ericsson.se> Message-ID: <200910170046502888607@its3.ch> On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: I guess Robby wrote: > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > raimo+erlang-questions@REDACTED > > wrote: > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > Dear Raimo, > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > I don't understand inet_gethost.c :-( > > > Where "goes" this problem for finding the information ? > > > If we know where and what, we could perhaps find what is missing in my > > windows 2003 ... > > > Hope to read you soon ! > > > John > > > > Well, there is not much more to say. inet_gethost.c calls > > struct hostent *gethostbyname(const char *name) in the > > winsock2 library: > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > It never returns. > > It is deprecated but that does not mean broken. > > I have a Windows 2003 server that it works on. > > We have not found the reason it differs between > > my server and your server. > > And that is about it. > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > Version 4.0' installed. This is a specific add-on, nothing to do with > Microsoft Firewall in your Network Properties. > I guess info wrote: > but I don't use ISA server !!! > No other interesting firewall software? Yes: I am using Comodo. I guess Robby wrote: > If I remove it, it works fine, if I add it it hangs. Apart from that the > W2003 build is as vanilla as it gets. > > Nothing much to go on, but I would not be surprised if some system DLL is > getting in the way. This is very hard to diagnose. > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > rule out poor linkage due to the VC++ setup used to build the shipping > executables. If I compile up a trivial gethostbyname() example from MSDN, it > works OK in all circumstances. > > There are a few strange dependencies in other shipped Windows files, for > example beam.dll seems to depend on a particular MSVCR80.DLL with the > version 8.0.50727.1433 . And if you have an older Windows machine, you get > the completely unhelpful popup box telling you that your program could not > be run. This has something to do with Windows Side-by-Side DLL deployment, > and I completely fail to grasp what that's all about. > I guess info wrote: > but I don't have msvcr80.dll !!! I guess you do, actually, since Erlang/OTP's installer is supposed to install that DLL, if it does not exist already. I guess Robby wrote: > > Robby -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From info@REDACTED Sat Oct 17 11:19:57 2009 From: info@REDACTED (info) Date: Sat, 17 Oct 2009 11:19:57 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? References: <6a3ae47e0910150848x2c6ddaa8i9edcc292b358657@mail.gmail.com>, <200910161629195074975@its3.ch>, <20091016153333.GA15561@erix.ericsson.se> Message-ID: <200910171119566109737@its3.ch> On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: I guess Robby wrote: > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > raimo+erlang-questions@REDACTED > > wrote: > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > Dear Raimo, > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > I don't understand inet_gethost.c :-( > > > Where "goes" this problem for finding the information ? > > > If we know where and what, we could perhaps find what is missing in my > > windows 2003 ... > > > Hope to read you soon ! > > > John > > > > Well, there is not much more to say. inet_gethost.c calls > > struct hostent *gethostbyname(const char *name) in the > > winsock2 library: > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > It never returns. > > It is deprecated but that does not mean broken. > > I have a Windows 2003 server that it works on. > > We have not found the reason it differs between > > my server and your server. > > And that is about it. > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > Version 4.0' installed. This is a specific add-on, nothing to do with > Microsoft Firewall in your Network Properties. > I guess info wrote: > but I don't use ISA server !!! > No other interesting firewall software? I guess Robby wrote: > If I remove it, it works fine, if I add it it hangs. Apart from that the > W2003 build is as vanilla as it gets. > > Nothing much to go on, but I would not be surprised if some system DLL is > getting in the way. This is very hard to diagnose. > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > rule out poor linkage due to the VC++ setup used to build the shipping > executables. If I compile up a trivial gethostbyname() example from MSDN, it > works OK in all circumstances. > > There are a few strange dependencies in other shipped Windows files, for > example beam.dll seems to depend on a particular MSVCR80.DLL with the > version 8.0.50727.1433 . And if you have an older Windows machine, you get > the completely unhelpful popup box telling you that your program could not > be run. This has something to do with Windows Side-by-Side DLL deployment, > and I completely fail to grasp what that's all about. > I guess info wrote: > but I don't have msvcr80.dll !!! I guess you do, actually, since Erlang/OTP's installer is supposed to install that DLL, if it does not exist already. Unfortunately msvcr80.dll is neither in system32 nor in erlang directory !! (I found it with other applications but in their directory). I guess Robby wrote: > > Robby -- / Raimo Niskanen, Erlang/OTP, Ericsson AB ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From max.lapshin@REDACTED Sun Oct 18 15:07:00 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 18 Oct 2009 17:07:00 +0400 Subject: Chain two spawned processes Message-ID: Hi everyone. I cannot find solution for a problem: I need to spawn two processes (ffmpeg and x264) and chain their input and output. Shell command looks like: ffmpeg ... | x264 If I've made program in C, I would make pipe(), then two fork() and exec in each fork with reopening fds. It seems, that there is no way to do it in erlang, using existing functions. I haven't found any pipe() call in otp sources. From max.lapshin@REDACTED Sun Oct 18 15:15:21 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 18 Oct 2009 17:15:21 +0400 Subject: Chain two spawned processes In-Reply-To: References: Message-ID: Oh, it seems, that sys.c:1395 makes pipe for spawned process input-output, so what I need, is to get fd for output of first open_port and pass it as input for second open_port call. From rapsey@REDACTED Sun Oct 18 18:10:26 2009 From: rapsey@REDACTED (Rapsey) Date: Sun, 18 Oct 2009 18:10:26 +0200 Subject: [erlang-questions] Re: Chain two spawned processes In-Reply-To: References: Message-ID: <97619b170910180910w348af5c0sf55b751ee3b0320e@mail.gmail.com> Erlang ports are designed to work with stdout/stdin for programs. I know ffmpeg works just fine with stdout/stdin, don't know about x264 though. You open a port to ffmpeg and x264 read from one, send to other. Though I would be careful about doing it this way. I used to use open_port for spawning mencoder, but switched to using a simple c program for it. The reason was that my servers kept crashing every few days (with mencoder running 24/7) and it had something to do with that port I had open constantly (it made no difference how much data I was sending through, just that it was open). Erlang for some reason tried to reserve a large amount of memory, failed and crashed. Sergej On Sun, Oct 18, 2009 at 3:15 PM, Max Lapshin wrote: > Oh, it seems, that sys.c:1395 makes pipe for spawned process > input-output, so what I need, > is to get fd for output of first open_port and pass it as input for > second open_port call. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From max.lapshin@REDACTED Sun Oct 18 19:50:50 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 18 Oct 2009 21:50:50 +0400 Subject: [erlang-questions] Re: Chain two spawned processes In-Reply-To: <97619b170910180910w348af5c0sf55b751ee3b0320e@mail.gmail.com> References: <97619b170910180910w348af5c0sf55b751ee3b0320e@mail.gmail.com> Message-ID: On Sun, Oct 18, 2009 at 8:10 PM, Rapsey wrote: > Erlang ports are designed to work with stdout/stdin for programs. I know > ffmpeg works just fine with stdout/stdin, don't know about x264 though. You > open a port to ffmpeg and x264 read from one, send to other. Yes, it seems to be impossible to work without passing data through Erlang. In fact I need to take piped output of first ffmpeg and pass it as input to x264. But there is no such code in driver =( Maybe better just to spawn /bin/sh and look after its status? From mandarzzz@REDACTED Sun Oct 18 20:32:37 2009 From: mandarzzz@REDACTED (Armand Golpaygani) Date: Sun, 18 Oct 2009 20:32:37 +0200 Subject: Erlang in the clouds Message-ID: <76ff2e900910181132m67d86b77x4436341dfc5ce0d2@mail.gmail.com> Hello, I am working on a web project with Erlang and MongoDB. I am currently considering hosting providers. I know some of you successfully use Erlang on EC2. Do you know any other reliable service you would advise for a start-up ? Thanks Armand From rapsey@REDACTED Sun Oct 18 20:59:50 2009 From: rapsey@REDACTED (Rapsey) Date: Sun, 18 Oct 2009 20:59:50 +0200 Subject: [erlang-questions] Erlang in the clouds In-Reply-To: <76ff2e900910181132m67d86b77x4436341dfc5ce0d2@mail.gmail.com> References: <76ff2e900910181132m67d86b77x4436341dfc5ce0d2@mail.gmail.com> Message-ID: <97619b170910181159x3d324371yf6a0bfdaaeb9b42b@mail.gmail.com> hi, I've heard slicehost is very nice: http://www.slicehost.com/ I haven't used it myself, but there was some talk about it in the mongodb irc channel. What erlang driver are you going to be using? Sergej On Sun, Oct 18, 2009 at 8:32 PM, Armand Golpaygani wrote: > Hello, > > I am working on a web project with Erlang and MongoDB. I am currently > considering hosting providers. > I know some of you successfully use Erlang on EC2. Do you know any other > reliable service you would advise for a start-up ? > > Thanks > Armand > From corticalcomputer@REDACTED Sun Oct 18 21:05:31 2009 From: corticalcomputer@REDACTED (G.S.) Date: Sun, 18 Oct 2009 12:05:31 -0700 Subject: Any way to dynamically access Records? Message-ID: <2a67d3ff0910181205g444eee08pce356494efd8fc1a@mail.gmail.com> Is there a way to get this to work: Row#TableName.Column Where all are variables, dropped into the function. Or is there a way to get an equivalent? I need to read values I store in mnesia, but there rows are very long, and I have different tables, and need a way to access specific values at specific times, hence I have to use the setup above. Any suggestions would be welcomed, thanks. -Gene From mandarzzz@REDACTED Sun Oct 18 21:13:21 2009 From: mandarzzz@REDACTED (Armand Golpaygani) Date: Sun, 18 Oct 2009 21:13:21 +0200 Subject: [erlang-questions] Erlang in the clouds In-Reply-To: <97619b170910181159x3d324371yf6a0bfdaaeb9b42b@mail.gmail.com> References: <76ff2e900910181132m67d86b77x4436341dfc5ce0d2@mail.gmail.com> <97619b170910181159x3d324371yf6a0bfdaaeb9b42b@mail.gmail.com> Message-ID: <76ff2e900910181213s1f30f994r30c5e60794455a54@mail.gmail.com> Hey, Thanks for the link. I've started prototyping with Erlmongo :) Cheers Armand 2009/10/18 Rapsey > hi, > > I've heard slicehost is very nice: > http://www.slicehost.com/ > I haven't used it myself, but there was some talk about it in the mongodb > irc channel. > What erlang driver are you going to be using? > > Sergej > > On Sun, Oct 18, 2009 at 8:32 PM, Armand Golpaygani >wrote: > > > Hello, > > > > I am working on a web project with Erlang and MongoDB. I am currently > > considering hosting providers. > > I know some of you successfully use Erlang on EC2. Do you know any other > > reliable service you would advise for a start-up ? > > > > Thanks > > Armand > > > From max.lapshin@REDACTED Sun Oct 18 21:14:37 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 18 Oct 2009 23:14:37 +0400 Subject: [erlang-questions] Any way to dynamically access Records? In-Reply-To: <2a67d3ff0910181205g444eee08pce356494efd8fc1a@mail.gmail.com> References: <2a67d3ff0910181205g444eee08pce356494efd8fc1a@mail.gmail.com> Message-ID: Functions element/2, setelement/3 ? From nick@REDACTED Sun Oct 18 22:15:38 2009 From: nick@REDACTED (Nick Gerakines) Date: Sun, 18 Oct 2009 13:15:38 -0700 Subject: [erlang-questions] Erlang in the clouds In-Reply-To: <76ff2e900910181213s1f30f994r30c5e60794455a54@mail.gmail.com> References: <76ff2e900910181132m67d86b77x4436341dfc5ce0d2@mail.gmail.com> <97619b170910181159x3d324371yf6a0bfdaaeb9b42b@mail.gmail.com> <76ff2e900910181213s1f30f994r30c5e60794455a54@mail.gmail.com> Message-ID: I've been using Slicehost for about a year and a half for my Erlang projects. For a long time the I Play WoW Facebook app was on 5x256m slices using CouchDB/Memcached/MySQL. I have to say that I'm really happy with them because of the price as well as the service and support. I've still got 5 projects spread across 3x256m slices. I'm currently working on another application project on 2x256m slices using mochiweb and emongo and it works great. For this one I'm using Fedora 11 and building my own otp/erlang package, installing modules from source from GitHub and using the x86_64 mongodb tgz package. I've got the install commands used to get the environments up and running if you are interested. # Nick Gerakines On Sun, Oct 18, 2009 at 12:13 PM, Armand Golpaygani wrote: > Hey, > > Thanks for the link. > I've started prototyping with Erlmongo :) > > Cheers > Armand > > > 2009/10/18 Rapsey > >> hi, >> >> I've heard slicehost is very nice: >> http://www.slicehost.com/ >> I haven't used it myself, but there was some talk about it in the mongodb >> irc channel. >> What erlang driver are you going to be using? >> >> Sergej >> >> On Sun, Oct 18, 2009 at 8:32 PM, Armand Golpaygani > >wrote: >> >> > Hello, >> > >> > I am working on a web project with Erlang and MongoDB. I am currently >> > considering hosting providers. >> > I know some of you successfully use Erlang on EC2. Do you know any other >> > reliable service you would advise for a start-up ? >> > >> > Thanks >> > Armand >> > >> > From bernie@REDACTED Mon Oct 19 07:20:47 2009 From: bernie@REDACTED (Bernard Duggan) Date: Mon, 19 Oct 2009 16:20:47 +1100 Subject: [erlang-questions] tcp connection with timeout In-Reply-To: References: <4AD6461B.8060200@m5net.com> Message-ID: <4ADBF72F.9020308@m5net.com> kamiseq wrote: > if I am able to set send_timout to some value then after this time I > expect that I will not get any response form the server no matter what You expect wrong. As I said before, the send_timeout only deals with the time taken to receive an ACK from the server-side's TCP stack - it has nothing to do with the application-level response your server code may or may not decide to send back. Think about it: Erlang knows nothing about your application protocol, so how does it know that any particular incoming TCP message is an application-level ACK to the one you sent? If you want to implement a timeout for the receipt of an application-level message, you need to do that yourself using something like timer:send_after(), and make sure you discard any responses received after that timeout. B From jlprasantha@REDACTED Mon Oct 19 08:27:15 2009 From: jlprasantha@REDACTED (prasantha kumara) Date: Mon, 19 Oct 2009 11:57:15 +0530 Subject: process leak in http:post Message-ID: hi all , i am witing an application using erlang, there i am using http:post() methos. there it seems there is process leak there when i get he infomation about the process by using bt(PID) it gives following infomation 0xb4e54870 Return addr 0xb4cd9d10 (xml_message_deliverer:sendRequest/2 + 44) y(0) Catch 0xb4cd9bec (xml_message_deliverer:sendRequest/3 + 1980) y(1) [] y(2) [] y(3) 2 can any body help me to resolve this issue thanks prasantha From max.lapshin@REDACTED Mon Oct 19 09:41:18 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 19 Oct 2009 11:41:18 +0400 Subject: [erlang-questions] Re: Chain two spawned processes In-Reply-To: References: <97619b170910180910w348af5c0sf55b751ee3b0320e@mail.gmail.com> Message-ID: > I don't know if this applies to your problem, but when using {spawn, > Cmd} to open the port, the runtime actually starts a shell process to > evaluate Cmd, so you can use shell redirections to pipeline from one > command to another: Actually I wanted to distinguish their error streams (to detect status) and I was going to use open_port({spawn_executable, which does execle. I need just several API functions: 1) get output fd of spawned process (it is ofd[1]), 2) set 'passive' mode to spawned process so that it doesn't send any messages to controlled process 3) set output fd as input fd to next spawned process. 4) close this ofd[1] in master process Without this api, only spawning shell cmd is possible, but it is not enough =( From steven.charles.davis@REDACTED Mon Oct 19 13:03:25 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 19 Oct 2009 04:03:25 -0700 (PDT) Subject: Erlang in the clouds In-Reply-To: References: <76ff2e900910181132m67d86b77x4436341dfc5ce0d2@mail.gmail.com> <97619b170910181159x3d324371yf6a0bfdaaeb9b42b@mail.gmail.com> <76ff2e900910181213s1f30f994r30c5e60794455a54@mail.gmail.com> Message-ID: FWIW, Linode.com offers similar services to Slicehost... but can be cheaper: RAM/Storage/Transfer is 360/16/200 for 20 bucks vs 256/10/100. /s From max.lapshin@REDACTED Mon Oct 19 13:46:07 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 19 Oct 2009 15:46:07 +0400 Subject: [erlang-questions] Re: Chain two spawned processes In-Reply-To: References: <97619b170910180910w348af5c0sf55b751ee3b0320e@mail.gmail.com> Message-ID: Yes, it seems, just that Erlang unix layer lacks all these process control stuff. Look, what I really need is to connect two processes via one pipe so that no data is passed through beam, and grab stderr of second one to controll process of execution. sys.c cannot do this right now. Currently I will do exactly as You say: launch shell script. However, maybe will be reasonable to make patches to erlang unix/sys.c to add more control for process starting and spawning? From bengt.kleberg@REDACTED Mon Oct 19 14:35:45 2009 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 19 Oct 2009 14:35:45 +0200 Subject: [erlang-questions] Re: Chain two spawned processes In-Reply-To: References: <97619b170910180910w348af5c0sf55b751ee3b0320e@mail.gmail.com> Message-ID: <1255955745.4993.12.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, There is a well thought out design for controlling UNIX processes from Scheme here: http://www.faqs.org/faqs/unix-faq/shell/scsh-faq/ It is helpful when thinking about this kind of thing. bengt On Mon, 2009-10-19 at 15:46 +0400, Max Lapshin wrote: > Yes, it seems, just that Erlang unix layer lacks all these process > control stuff. > > Look, what I really need is to connect two processes via one pipe so > that no data is passed through beam, > and grab stderr of second one to controll process of execution. > > sys.c cannot do this right now. > > Currently I will do exactly as You say: launch shell script. However, > maybe will be reasonable to make patches to erlang unix/sys.c > to add more control for process starting and spawning? > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From roberto@REDACTED Mon Oct 19 15:10:38 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 19 Oct 2009 15:10:38 +0200 Subject: [erlang-questions] making http client close connections Message-ID: hi joel, did you happen to solve this particular issue? i'm starting a similar test and any input would be appreciated. cheers, r. From max.lapshin@REDACTED Mon Oct 19 15:51:49 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 19 Oct 2009 17:51:49 +0400 Subject: [erlang-questions] Re: Chain two spawned processes In-Reply-To: References: <97619b170910180910w348af5c0sf55b751ee3b0320e@mail.gmail.com> Message-ID: On Mon, Oct 19, 2009 at 5:47 PM, Samuel Rivas wrote: >> Look, what I really need is to connect two processes via one pipe so >> that no data is passed through beam, >> and grab stderr of second one to controll process of execution. > > I guess you also want to do something with the standard output of the > second process. Otherwise, you just redirect its stderr to stdout (and > redirect stdout to somewhere else) and you are done :) In this case I need only stderr of second process. However, I'd prefer full control after all these children. open_port API is not far from this. From james.hague@REDACTED Mon Oct 19 16:09:24 2009 From: james.hague@REDACTED (James Hague) Date: Mon, 19 Oct 2009 09:09:24 -0500 Subject: Tuple values returned in registers, without using heap space? Message-ID: I *thought* I remember reading about this in the release notes at one time, but I haven't been able to track it down. The gist was that if it could be determined that a non-exported function always returned a tuple of a fixed size, then those values were returned directly in BEAM registers without actually creating a tuple. Any pointers? From mark.peleus@REDACTED Mon Oct 19 17:01:14 2009 From: mark.peleus@REDACTED (mark peleus) Date: Mon, 19 Oct 2009 17:01:14 +0200 Subject: Dynamic OTP Children and simple_one_for_one Message-ID: <599dd3e0910190801r285969ffh4dec140d4683b75d@mail.gmail.com> Hi, My OTP application need to create a gen_server for every user when he connects to the server. I think the supervisor should be of type simple_one_for_one because all the children will be instances of the same gen_server. I don't understand what should be the myapp_sup:init/1 function. It suppose to return {ok, {SupervisorSpecification, ChildSpecificationList}} In my case I don't have connected users yet when starting the application so ChildSpecificationList is empty? When a user connects to the server I need to create a gen_server for him so I'll use: supervisor:start_child(SupervisorName, ChildSpec) SupervisorName is the pid or the registered supervisor name. Do I need to explicitly register the supervisor or is it registered automatically in myapp_sup:init/1 ? In case of simple_one_for_one ChildSpec should be a list of arguments and the ChildSpec defined in myapp_sup:init/1 is being used. If I didn't start a gen_server in the init and just returned an empty list of Children will it work? Do I need to create a dummy child for this to work or maybe use one_for_one instead of simple_one_for_one? Thanks From rvirding@REDACTED Mon Oct 19 17:06:33 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 19 Oct 2009 17:06:33 +0200 Subject: [erlang-questions] Tuple values returned in registers, without using heap space? In-Reply-To: References: Message-ID: <3dbc6d1c0910190806h6fc0990v213b4c0d8e5ef188@mail.gmail.com> 2009/10/19 James Hague > I *thought* I remember reading about this in the release notes at one time, > but I haven't been able to track it down. The gist was that if it could be > determined that a non-exported function always returned a tuple of a fixed > size, then those values were returned directly in BEAM registers without > actually creating a tuple. > > Any pointers? > Yes, this could be done in the compiler, the basic mechanisms are there but it is not done today. Except *within* functions when it is safe. To do so between non-exported functions in a module would be possible but require more analysis. Apart from checking that a function always returned a tuple of fixed size you would need to analyze the calls to such that function and fix the handling of the return tuple values. Not impossible of course but not done. Robert From rvirding@REDACTED Mon Oct 19 17:10:38 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 19 Oct 2009 17:10:38 +0200 Subject: [erlang-questions] Re: Chain two spawned processes In-Reply-To: References: <97619b170910180910w348af5c0sf55b751ee3b0320e@mail.gmail.com> Message-ID: <3dbc6d1c0910190810i7c9ef361w50f246e188a5ad2b@mail.gmail.com> Why not just write a port driver to that instead of baking in specifics into the run-time? Don't put things into the run-time that don't have to be there. Robert 2009/10/19 Max Lapshin > On Mon, Oct 19, 2009 at 5:47 PM, Samuel Rivas > wrote: > >> Look, what I really need is to connect two processes via one pipe so > >> that no data is passed through beam, > >> and grab stderr of second one to controll process of execution. > > > > I guess you also want to do something with the standard output of the > > second process. Otherwise, you just redirect its stderr to stdout (and > > redirect stdout to somewhere else) and you are done :) > > In this case I need only stderr of second process. However, I'd prefer > full control after all these children. > open_port API is not far from this. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From yu_c@REDACTED Mon Oct 19 17:13:15 2009 From: yu_c@REDACTED (Chih - Wei Yu [ MTN - Innovation Centre ]) Date: Mon, 19 Oct 2009 17:13:15 +0200 Subject: YAWS with SSL Connections Problem Message-ID: Hi all, We're running YAWS (1.75)(Erlang 12B-5) with SSL and there was a stage where we ran out of file descriptors. Now what seems to be is that ssl_esock process is using up all the descriptors. PID/FILES = 21651 / 1638 $ /usr/ucb/ps -auxww | grep 21651 user 12992 0.0 0.0 1352 1152 pts/1 S 10:59:23 0:00 grep 21651 user 21651 0.0 0.12024019184 ? S Sep 09 8235:25 /usr/local/erlang/R12B-5/lib/erlang/lib/ssl-3.10/priv/bin/ssl_esock $ netstat -an | grep IDLE |more *.* *.* 0 0 49152 0 IDLE *.* *.* 0 0 49152 0 IDLE *.* *.* 0 0 49152 0 IDLE *.* *.* 0 0 49152 0 IDLE $ netstat -an | grep IDLE | wc -l 1630 Another observation was that there were plenty of IDLE 'connections' when doing a netstat. Current max file descriptors is set to 4096 and thus it would eventually run out and the YAWS application would stop as it does not have descriptors to use. Two questions I would like to ask is - Under which circumstances does this happen and cause it not to clean up properly? - Has this been fixed in newer releases of YAWS? The following is also seen in the nohup.out =ERROR REPORT==== 19-Oct-2009::16:08:41 === SSL accept failed: enoproxysocket =ERROR REPORT==== 19-Oct-2009::16:09:29 === SSL accept failed: enoproxysocket Would also like to ask how the error enoproxysocket occurs. The OS we're running on is Solaris 10. Thanks Regards, Chih-Wei Yu ________________________________ NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/SUPPORT/LEGAL/Pages/EmailDisclaimer.aspx From max.lapshin@REDACTED Mon Oct 19 17:13:32 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 19 Oct 2009 19:13:32 +0400 Subject: [erlang-questions] Re: Chain two spawned processes In-Reply-To: <3dbc6d1c0910190810i7c9ef361w50f246e188a5ad2b@mail.gmail.com> References: <97619b170910180910w348af5c0sf55b751ee3b0320e@mail.gmail.com> <3dbc6d1c0910190810i7c9ef361w50f246e188a5ad2b@mail.gmail.com> Message-ID: On Mon, Oct 19, 2009 at 7:10 PM, Robert Virding wrote: > Why not just write a port driver to that instead of baking in specifics into > the run-time? Don't put things into the run-time that don't have to be > there. > Why don't have? open_port provide some preliminary support, but it is not logical. For example, it doesn't provide separate control for spawned process output and error stream, only mix them together. I will have to make port driver, but I don't know, how will it be supported under windows. From francesca.gangemi@REDACTED Mon Oct 19 17:42:06 2009 From: francesca.gangemi@REDACTED (Francesca Gangemi) Date: Mon, 19 Oct 2009 17:42:06 +0200 Subject: [erlang-questions] Dynamic OTP Children and simple_one_for_one In-Reply-To: <599dd3e0910190801r285969ffh4dec140d4683b75d@mail.gmail.com> References: <599dd3e0910190801r285969ffh4dec140d4683b75d@mail.gmail.com> Message-ID: <4ADC88CE.6090701@erlang-consulting.com> Hi Mark, mark peleus wrote: > Hi, > > My OTP application need to create a gen_server for every user when he > connects to the server. > I think the supervisor should be of type simple_one_for_one because all the > children will be instances of the same gen_server. > > I don't understand what should be the myapp_sup:init/1 function. > It suppose to return {ok, {SupervisorSpecification, ChildSpecificationList}} > In my case I don't have connected users yet when starting the application so > ChildSpecificationList is empty? > > Even though you don't have any connected users the child specification list should not be empty. Your init function should look like the following one init(_Args) -> Child_Spec = [{my_server,{my_server,start_link,[]}, temporary, 5000, worker, [my_server]}], {ok,{{simple_one_for_one,3,1}, Child_Spec}}. You can change same values as Restart (temporary) or Shutdown (5000) to better suit your application. When started, the supervisor will not start any child processes. > When a user connects to the server I need to create a gen_server for him so > I'll use: > supervisor:start_child(SupervisorName, ChildSpec) > SupervisorName is the pid or the registered supervisor name. > Do I need to explicitly register the supervisor or is it registered > automatically in myapp_sup:init/1 ? > > You don't need to register the supervisor. If the supervisor is started by calling start_link() -> supervisor:start_link({local, myapp_sup}, myapp_sup, []). it will be registered as myapp_sup and you can then add children dynamically by calling supervisor:start_child(myapp_sup, []). If you want you can also specify a list of arguments that will be passed to the start function specified in the child specification. Calling supervisor:start_child(myapp_sup, [A, B, C]). results in apply(my_server, start_link, [A,B,C]). Kind regards Francesca -- Francesca Gangemi, francesca@REDACTED Erlang Training and Consulting http://www.erlang-consulting.com/ From raimo+erlang-questions@REDACTED Mon Oct 19 17:46:14 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 19 Oct 2009 17:46:14 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <200910170046502888607@its3.ch> References: <20091016153333.GA15561@erix.ericsson.se> <200910170046502888607@its3.ch> Message-ID: <20091019154614.GA16576@erix.ericsson.se> On Sat, Oct 17, 2009 at 12:46:50AM +0100, info wrote: > On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: > I guess Robby wrote: > > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > > raimo+erlang-questions@REDACTED > > > wrote: > > > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > > Dear Raimo, > > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > > I don't understand inet_gethost.c :-( > > > > Where "goes" this problem for finding the information ? > > > > If we know where and what, we could perhaps find what is missing in my > > > windows 2003 ... > > > > Hope to read you soon ! > > > > John > > > > > > Well, there is not much more to say. inet_gethost.c calls > > > struct hostent *gethostbyname(const char *name) in the > > > winsock2 library: > > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > > It never returns. > > > It is deprecated but that does not mean broken. > > > I have a Windows 2003 server that it works on. > > > We have not found the reason it differs between > > > my server and your server. > > > And that is about it. > > > > > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > > Version 4.0' installed. This is a specific add-on, nothing to do with > > Microsoft Firewall in your Network Properties. > > > > I guess info wrote: > > but I don't use ISA server !!! > > > > No other interesting firewall software? > Yes: I am using Comodo. Can you try to uninstall Comodo and see if the problem disappears. If so we know where to look for the problem. (We have installed a 2003 server with Comodo and got no problems, other than the possibility to do configuration errors that would cause your symptoms) > > I guess Robby wrote: > > If I remove it, it works fine, if I add it it hangs. Apart from that the > > W2003 build is as vanilla as it gets. > > > > Nothing much to go on, but I would not be surprised if some system DLL is > > getting in the way. This is very hard to diagnose. > > > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > > rule out poor linkage due to the VC++ setup used to build the shipping > > executables. If I compile up a trivial gethostbyname() example from MSDN, it > > works OK in all circumstances. > > > > There are a few strange dependencies in other shipped Windows files, for > > example beam.dll seems to depend on a particular MSVCR80.DLL with the > > version 8.0.50727.1433 . And if you have an older Windows machine, you get > > the completely unhelpful popup box telling you that your program could not > > be run. This has something to do with Windows Side-by-Side DLL deployment, > > and I completely fail to grasp what that's all about. > > > > I guess info wrote: > > but I don't have msvcr80.dll !!! > > I guess you do, actually, since Erlang/OTP's installer is supposed > to install that DLL, if it does not exist already. > > I guess Robby wrote: > > > > Robby > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From pablo.platt@REDACTED Mon Oct 19 18:12:16 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 19 Oct 2009 09:12:16 -0700 (PDT) Subject: excute code in the supervisor when a child crashes? Message-ID: <665975.66110.qm@web112618.mail.gq1.yahoo.com> Hi, I have a supervisor with simple_one_for_one. When I start children with supervisor:start_child I save the child pid in an ETS table so I'll be able to exchange messages between all the child processes. When a child crashes I need to delete his pid from the ETS table or it'll grow constantly and eat all my memory. Is it possible to execute code when a child crashes or terminates? Thanks From dizzyd@REDACTED Mon Oct 19 19:22:41 2009 From: dizzyd@REDACTED (Dave Smith) Date: Mon, 19 Oct 2009 11:22:41 -0600 Subject: [erlang-questions] excute code in the supervisor when a child crashes? In-Reply-To: <665975.66110.qm@web112618.mail.gq1.yahoo.com> References: <665975.66110.qm@web112618.mail.gq1.yahoo.com> Message-ID: You won't be able to do this sort of accounting work in a supervisor. Typically this problem is solved by having a secondary process that acts as an entry point to the supervisor (i.e. calls supervisor:start_child) and maintains monitors on all processes. This secondary process then does the necessary accounting work when the child process exits. The rationale for this approach is supervisors are system-critical code and as such you really, really, REALLY don't want to introduce bugs into them. Processes and monitors are cheap (enough) in Erlang that a secondary accounting process is a reasonable solution and lets us avoid introducing new code into the supervision system. Hope that helps... D. On Mon, Oct 19, 2009 at 10:12 AM, Pablo Platt wrote: > Hi, > > I have a supervisor with simple_one_for_one. > When I start children with supervisor:start_child I save the child pid in an ETS table > so I'll be able to exchange messages between all the child processes. > > When a child crashes I need to delete his pid from the ETS table or it'll grow constantly and eat all my memory. > Is it possible to execute code when a child crashes or terminates? > > Thanks > > > From roberto.aloi@REDACTED Mon Oct 19 19:23:02 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Mon, 19 Oct 2009 18:23:02 +0100 Subject: [erlang-questions] excute code in the supervisor when a child crashes? In-Reply-To: <665975.66110.qm@web112618.mail.gq1.yahoo.com> References: <665975.66110.qm@web112618.mail.gq1.yahoo.com> Message-ID: <384B3581-80C6-4FFE-9EF3-0F281D3D19B7@erlang-consulting.com> Hi Pablo, what you actually need is an event handler. More information here: http://www1.erlang.org/doc/design_principles/events.html http://www.erlang.org/doc/man/gen_event.html One approach could be to append a handler process to your supervisor, managing the ets table whenever one of the children crash. Regards, Roberto Aloi roberto.aloi@REDACTED http://www.erlang-consulting.com --- On Oct 19, 2009, at 5:12 PM, Pablo Platt wrote: > Hi, > > I have a supervisor with simple_one_for_one. > When I start children with supervisor:start_child I save the child > pid in an ETS table > so I'll be able to exchange messages between all the child processes. > > When a child crashes I need to delete his pid from the ETS table or > it'll grow constantly and eat all my memory. > Is it possible to execute code when a child crashes or terminates? > > Thanks > > From dizzyd@REDACTED Mon Oct 19 19:44:30 2009 From: dizzyd@REDACTED (Dave Smith) Date: Mon, 19 Oct 2009 11:44:30 -0600 Subject: [erlang-questions] excute code in the supervisor when a child crashes? In-Reply-To: <878280.18543.qm@web112608.mail.gq1.yahoo.com> References: <665975.66110.qm@web112618.mail.gq1.yahoo.com> <878280.18543.qm@web112608.mail.gq1.yahoo.com> Message-ID: On Mon, Oct 19, 2009 at 11:39 AM, Pablo Platt wrote: > Sounds like a good solution. > I'll create a gen_server that will call supervisor:start_child > then it'll monitor the pid and store it in an ETS so other clients will be > able to send messags to the child. > If the child exists or crashes the monitoring gen_server will delete the ETS > record of this process. > > The complexity is if the monitoring process dies. > In this case it needs to scan the whole ETS table, re-monitor all live > process and delete records of all the dead processes. Well, I suppose it depends on what process owns that ETS table. If your gen_server owns the ETS table, that table will get deleted at crash time. In that situation, you could just use supervisor:which_children to rebuild the table, I suppose. Of course, it would be easier to just ensure that process never dies. ;) D. From pablo.platt@REDACTED Mon Oct 19 19:51:34 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 19 Oct 2009 10:51:34 -0700 (PDT) Subject: [erlang-questions] excute code in the supervisor when a child crashes? In-Reply-To: References: <665975.66110.qm@web112618.mail.gq1.yahoo.com> <878280.18543.qm@web112608.mail.gq1.yahoo.com> Message-ID: <987257.80171.qm@web112616.mail.gq1.yahoo.com> >In that situation, you could just use >supervisor:which_children to rebuild the table, I suppose. Sounds much better. >Of course, it would be easier to just ensure that process never dies. ;) If there are no bugs in the module it will never die? ________________________________ From: Dave Smith To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Mon, October 19, 2009 7:44:30 PM Subject: Re: [erlang-questions] excute code in the supervisor when a child crashes? On Mon, Oct 19, 2009 at 11:39 AM, Pablo Platt wrote: > Sounds like a good solution. > I'll create a gen_server that will call supervisor:start_child > then it'll monitor the pid and store it in an ETS so other clients will be > able to send messags to the child. > If the child exists or crashes the monitoring gen_server will delete the ETS > record of this process. > > The complexity is if the monitoring process dies. > In this case it needs to scan the whole ETS table, re-monitor all live > process and delete records of all the dead processes. Well, I suppose it depends on what process owns that ETS table. If your gen_server owns the ETS table, that table will get deleted at crash time. In that situation, you could just use supervisor:which_children to rebuild the table, I suppose. Of course, it would be easier to just ensure that process never dies. ;) D. From corticalcomputer@REDACTED Mon Oct 19 20:04:40 2009 From: corticalcomputer@REDACTED (G.S.) Date: Mon, 19 Oct 2009 11:04:40 -0700 Subject: [erlang-questions] Any way to dynamically access Records? In-Reply-To: References: <2a67d3ff0910181205g444eee08pce356494efd8fc1a@mail.gmail.com> Message-ID: <2a67d3ff0910191104o5cbecd9fv89d8b454436011fd@mail.gmail.com> That allows one to access it by index, I was talking about by Tag... but nm, I found an acceptable way around it. Thanks, -Gene On Sun, Oct 18, 2009 at 12:14 PM, Max Lapshin wrote: > Functions element/2, setelement/3 ? > From pablo.platt@REDACTED Mon Oct 19 19:40:55 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 19 Oct 2009 10:40:55 -0700 (PDT) Subject: [erlang-questions] excute code in the supervisor when a child crashes? In-Reply-To: <384B3581-80C6-4FFE-9EF3-0F281D3D19B7@erlang-consulting.com> References: <665975.66110.qm@web112618.mail.gq1.yahoo.com> <384B3581-80C6-4FFE-9EF3-0F281D3D19B7@erlang-consulting.com> Message-ID: <547450.31398.qm@web112616.mail.gq1.yahoo.com> Why having an even handler is better then just monitoring the child processes? How will the event manager be informed of crashed child processes? ________________________________ From: Roberto Aloi To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Mon, October 19, 2009 7:23:02 PM Subject: Re: [erlang-questions] excute code in the supervisor when a child crashes? Hi Pablo, what you actually need is an event handler. More information here: http://www1.erlang.org/doc/design_principles/events.html http://www.erlang.org/doc/man/gen_event.html One approach could be to append a handler process to your supervisor, managing the ets table whenever one of the children crash. Regards, Roberto Aloi roberto.aloi@REDACTED http://www.erlang-consulting.com --- On Oct 19, 2009, at 5:12 PM, Pablo Platt wrote: > Hi, > > I have a supervisor with simple_one_for_one. > When I start children with supervisor:start_child I save the child pid in an ETS table > so I'll be able to exchange messages between all the child processes. > > When a child crashes I need to delete his pid from the ETS table or it'll grow constantly and eat all my memory. > Is it possible to execute code when a child crashes or terminates? > > Thanks > > ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From pablo.platt@REDACTED Mon Oct 19 19:39:30 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 19 Oct 2009 10:39:30 -0700 (PDT) Subject: [erlang-questions] excute code in the supervisor when a child crashes? In-Reply-To: References: <665975.66110.qm@web112618.mail.gq1.yahoo.com> Message-ID: <878280.18543.qm@web112608.mail.gq1.yahoo.com> Sounds like a good solution. I'll create a gen_server that will call supervisor:start_child then it'll monitor the pid and store it in an ETS so other clients will be able to send messags to the child. If the child exists or crashes the monitoring gen_server will delete the ETS record of this process. The complexity is if the monitoring process dies. In this case it needs to scan the whole ETS table, re-monitor all live process and delete records of all the dead processes. ________________________________ From: Dave Smith To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Mon, October 19, 2009 7:22:41 PM Subject: Re: [erlang-questions] excute code in the supervisor when a child crashes? You won't be able to do this sort of accounting work in a supervisor. Typically this problem is solved by having a secondary process that acts as an entry point to the supervisor (i.e. calls supervisor:start_child) and maintains monitors on all processes. This secondary process then does the necessary accounting work when the child process exits. The rationale for this approach is supervisors are system-critical code and as such you really, really, REALLY don't want to introduce bugs into them. Processes and monitors are cheap (enough) in Erlang that a secondary accounting process is a reasonable solution and lets us avoid introducing new code into the supervision system. Hope that helps... D. On Mon, Oct 19, 2009 at 10:12 AM, Pablo Platt wrote: > Hi, > > I have a supervisor with simple_one_for_one. > When I start children with supervisor:start_child I save the child pid in an ETS table > so I'll be able to exchange messages between all the child processes. > > When a child crashes I need to delete his pid from the ETS table or it'll grow constantly and eat all my memory. > Is it possible to execute code when a child crashes or terminates? > > Thanks > > > From mihai@REDACTED Mon Oct 19 22:21:28 2009 From: mihai@REDACTED (Mihai Balea) Date: Mon, 19 Oct 2009 16:21:28 -0400 Subject: [erlang-questions] excute code in the supervisor when a child crashes? In-Reply-To: <878280.18543.qm@web112608.mail.gq1.yahoo.com> References: <665975.66110.qm@web112618.mail.gq1.yahoo.com> <878280.18543.qm@web112608.mail.gq1.yahoo.com> Message-ID: On Oct 19, 2009, at 1:39 PM, Pablo Platt wrote: > Sounds like a good solution. > I'll create a gen_server that will call supervisor:start_child > then it'll monitor the pid and store it in an ETS so other clients > will be able to send messags to the child. > If the child exists or crashes the monitoring gen_server will delete > the ETS record of this process. > > The complexity is if the monitoring process dies. > In this case it needs to scan the whole ETS table, re-monitor all > live process and delete records of all the dead processes. The problem with that is if the ETS table is owned by the monitoring process, it will go down with it. You might want to try an already existing solution, such as Ulf Wiger's proc/grpoc. It's basically the same idea, just that the code has already been in use for a while, so it should be relatively solid. In addition, gproc works in a clustered environment, if your requirements include that. Mihai From mandarzzz@REDACTED Mon Oct 19 22:33:35 2009 From: mandarzzz@REDACTED (Armand Golpaygani) Date: Mon, 19 Oct 2009 22:33:35 +0200 Subject: [erlang-questions] Re: Erlang in the clouds In-Reply-To: References: <76ff2e900910181132m67d86b77x4436341dfc5ce0d2@mail.gmail.com> <97619b170910181159x3d324371yf6a0bfdaaeb9b42b@mail.gmail.com> <76ff2e900910181213s1f30f994r30c5e60794455a54@mail.gmail.com> Message-ID: <76ff2e900910191333p5844ff9fs3690b366d5b3094c@mail.gmail.com> Thanks for all your suggestions. Armand 2009/10/19 Steve Davis > FWIW, Linode.com offers similar services to Slicehost... but can be > cheaper: > RAM/Storage/Transfer is 360/16/200 for 20 bucks vs 256/10/100. > /s > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From pablo.platt@REDACTED Mon Oct 19 23:08:16 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 19 Oct 2009 14:08:16 -0700 (PDT) Subject: [erlang-questions] excute code in the supervisor when a child crashes? In-Reply-To: References: <665975.66110.qm@web112618.mail.gq1.yahoo.com> <878280.18543.qm@web112608.mail.gq1.yahoo.com> Message-ID: <86242.42317.qm@web112603.mail.gq1.yahoo.com> Mihai, As you suggested I'm reading about gproc from: http://svn.ulf.wiger.net/gproc/branches/experimental-0906/gproc/doc/erlang07-wiger.pdf http://rklophaus.com/blog/2009/9/16/gproc-erlang-global-process-registry.html Is this the version I should use? http://svn.ulf.wiger.net/gproc/branches/experimental-0906/gproc/ I've read that some patches needed for the kernel and about a dependence on gen_leader. Is anything needed except compiling the grpoc app and putting it in the erlang path? Thanks ________________________________ From: Mihai Balea To: Pablo Platt Cc: erlang-questions Questions Sent: Mon, October 19, 2009 10:21:28 PM Subject: Re: [erlang-questions] excute code in the supervisor when a child crashes? On Oct 19, 2009, at 1:39 PM, Pablo Platt wrote: > Sounds like a good solution. > I'll create a gen_server that will call supervisor:start_child > then it'll monitor the pid and store it in an ETS so other clients will be able to send messags to the child. > If the child exists or crashes the monitoring gen_server will delete the ETS record of this process. > > The complexity is if the monitoring process dies. > In this case it needs to scan the whole ETS table, re-monitor all live process and delete records of all the dead processes. The problem with that is if the ETS table is owned by the monitoring process, it will go down with it. You might want to try an already existing solution, such as UlfWiger'sproc/grpoc. It's basically the same idea, just that the code has already been in use for a while, so it should be relatively solid. In addition, gproc works in a clustered environment, if your requirements include that. Mihai ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From rvirding@REDACTED Tue Oct 20 00:23:33 2009 From: rvirding@REDACTED (Robert Virding) Date: Tue, 20 Oct 2009 00:23:33 +0200 Subject: New version of LFE, Lisp Flavoured Erlang Message-ID: <3dbc6d1c0910191523v1225c5f5n3f05871e85034588@mail.gmail.com> I have just released LFE 0.5. It is, as usual, long overdue and contains some nice goodies: - Added macro list*. - Added a new shell command to set variables in the shell. - Cleaned up compiler options and made them more like the vanilla compiler. - Added unicode types to binaries: utf-8, utf-16 and utf-32. - Shell and compiler now print error data in LFE instead of vanilla. Shell error reporting still pretty basic, not as "beautiful" as vanilla. - Cleaned up i/o functions and added formatted output function. Still use vanilla command characters. - All i/o goes through then module lfe_io. The functions which return the corresponding printable string end in 1. So lfe_io:format prints formatted output while lfe_io:format1 returns the string. - Improved pretty-printing. - Many internal improvements. As usual it can be found on both trapexit.org and github : http://forum.trapexit.org/viewtopic.php?p=50027#50027 http://github.com/rvirding/lfe/tree On github v0.5 is tagged and on the master branch, generally any commit on the maser branch, at least the more recent, can be loaded as they should be consistent. If you take the develop branch you are on your own. Robert From agratton@REDACTED Tue Oct 20 02:27:00 2009 From: agratton@REDACTED (Angus Gratton) Date: Tue, 20 Oct 2009 11:27:00 +1100 Subject: [erlang-questions] Any way to dynamically access Records? In-Reply-To: <2a67d3ff0910181205g444eee08pce356494efd8fc1a@mail.gmail.com> References: <2a67d3ff0910181205g444eee08pce356494efd8fc1a@mail.gmail.com> Message-ID: <4ADD03D4.6030703@m5net.com> G.S. wrote: > Is there a way to get this to work: Row#TableName.Column > > Where all are variables, dropped into the function. FWIW, I (along with many others, it seems) have experienced similar problems and found ways around them. There are some interesting existing solutions out on the net, like these: http://www.trapexit.org/Match_Specifications_And_Records_(Dynamically!) http://chlorophil.blogspot.com/2007/04/dynamic-record-access-functions-with.html Also, if you're just doing mnesia index lookups, mnesia:index_read() will take an indexed field name and look it up at runtime (with the associated performance hit.) What we ended up doing is code generating a lot of our records. Subsequently, we also now code generate a module with functions to look up arbitrary field values in records, and also functions to generate an arbitrary matchspec from a list of tuples {fieldname, fieldvalue}. This is a similar approach to the first link I posted above, although it's a bit more lo-fi because all the functions get code generated, instead of inferred by the parse transform. Hth. - Angus From mihai@REDACTED Tue Oct 20 03:21:30 2009 From: mihai@REDACTED (Mihai Balea) Date: Mon, 19 Oct 2009 21:21:30 -0400 Subject: [erlang-questions] excute code in the supervisor when a child crashes? In-Reply-To: <86242.42317.qm@web112603.mail.gq1.yahoo.com> References: <665975.66110.qm@web112618.mail.gq1.yahoo.com> <878280.18543.qm@web112608.mail.gq1.yahoo.com> <86242.42317.qm@web112603.mail.gq1.yahoo.com> Message-ID: On Oct 19, 2009, at 5:08 PM, Pablo Platt wrote: > Mihai, > As you suggested I'm reading about gproc from: > http://svn.ulf.wiger.net/gproc/branches/experimental-0906/gproc/doc/erlang07-wiger.pdf > http://rklophaus.com/blog/2009/9/16/gproc-erlang-global-process-registry.html > > Is this the version I should use? http://svn.ulf.wiger.net/gproc/branches/experimental-0906/gproc/ > > I've read that some patches needed for the kernel and about a > dependence on gen_leader. > Is anything needed except compiling the grpoc app and putting it in > the erlang path? I don't have much hands on experience with gproc, but from what I hear, you only need the kernel patches if you want gproc to replace the built in process registry. If you use it as a standalone module, you only need gen_leader, which is indeed a dependency. However, if you only need a local registry, you might want to use Ulf's previous version, called proc. The code is relatively hard to find; one place to get it would be here: http://github.com/dizzyd/proc It all depends on whether your project requires a global or local registry. Mihai From tomas.abrahamsson@REDACTED Tue Oct 20 09:45:01 2009 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Tue, 20 Oct 2009 09:45:01 +0200 Subject: [erlang-questions] YAWS with SSL Connections Problem In-Reply-To: (Chih's message of "Mon, 19 Oct 2009 17:13:15 +0200") References: Message-ID: > We're running YAWS (1.75)(Erlang 12B-5) with SSL and there was a > stage where we ran out of file descriptors. Now what seems to be is > that ssl_esock process is using up all the descriptors. I don't remember exactly about reasons for usage of large number of file descriptors, though it is possible to raise the number of file descriptors a process can use, see the ulimit command (and on Solaris, also documentation for the /etc/system file, where you can raise the hard limits). It might possibly have something to do with your second question: > The following is also seen in the nohup.out > =ERROR REPORT==== 19-Oct-2009::16:08:41 === > SSL accept failed: enoproxysocket > The OS we're running on is Solaris 10. We've seen enoproxysocket on Solaris 10 with the old ssl implementation[1]. Not on Solaris 9 or 8 and not on Linux. IIRC, we've seen it under both light and heavy load. The high-level reason is that the Erlang side and the ssl_esock gets out of sync with each other. The low-level details are that the Erlang side opens a TCP connection to the ssl_esock, then sends a proxy_join command for that tcp-connection to the ssl_esock's stdin, but on Solaris 10, the ssl_esock discovers these two events in the opposite order, gets confused, and responds with the enoproxysocket error. The net result is that the SSL connection will be unusable, the Erlang side can't use it since it's received an error response from ssl_esock, and the ssl_esock doesn't know what to do with it. It seems this can happen both when accepting incoming SSL connections and when making an outgoing SSL connection. The ssl_esock code hasn't changed from R12B-4 to R13B02-1, it seems (except for copyright notices), so just upgrading to a newer Erlang version won't automatically solve your problem, unfortunately. ______ [1] The old ssl-implementation, i.e: an Erlang-side and the C-side ssl_esock, as opposed to the new_ssl which does all the SSL protocol handling in Erlang, see the ssl documentation for further details, espcially the {ssl_imp,new} option. BRs Tomas From ulf.wiger@REDACTED Tue Oct 20 09:49:01 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 20 Oct 2009 09:49:01 +0200 Subject: [erlang-questions] excute code in the supervisor when a child crashes? In-Reply-To: References: <665975.66110.qm@web112618.mail.gq1.yahoo.com> <878280.18543.qm@web112608.mail.gq1.yahoo.com> <86242.42317.qm@web112603.mail.gq1.yahoo.com> Message-ID: <4ADD6B6D.4090404@erlang-consulting.com> Mihai Balea wrote: > > I don't have much hands on experience with gproc, but from what I hear, > you only need the kernel patches if you want gproc to replace the built > in process registry. Correct. I should have been clearer about this from the start. The reason for patching some of the core OTP libraries was to demonstrate how gproc could be used if it were made a core part of the system. The things I demonstrated in my paper were e.g. listing all supervisors with their restart parameters, and listing all gen_servers in the system. It didn't replace the built-in registry, though. It could possibly replace 'global' in time, but you should mainly see it as a complement. > If you use it as a standalone module, you only need > gen_leader, which is indeed a dependency. The 'experimental' part of the experimental branch was that I split gen_leader into two - one local and one global part. This was trivial, since the API separates local and global context. The local part is pretty well tested and, I believe, also in actual use. I haven't finished the global part. The intention is to make use of the standard gen_leader, rather than the crummy patch that was in the initial gproc prototype. (The only reason for patching in the first place was that the demo inserted gproc in the OTP kernel application, in a way that it had to start before Distributed Erlang was up and running. This led to some unholy hacking of gen_leader. Splitting gproc in two solves this problem in a much better way. My only excuse for not thinking about it from the beginning is that I was in a hurry to finish the paper before the deadline). The place to get gen_leader nowadays is: http://wiki.github.com/KirinDave/gen_leader_revival -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From raimo+erlang-questions@REDACTED Tue Oct 20 10:09:06 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 20 Oct 2009 10:09:06 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <200910192353067661212@its3.ch> References: <20091019154614.GA16576@erix.ericsson.se> <200910192353067661212@its3.ch> Message-ID: <20091020080906.GA29778@erix.ericsson.se> Try searching from C:/WINDOWS. It may be under C:/WINDOWS/System32 or hide under C:/WINDOWS/WinSxS/*/ somewhere. It may also be so that the installer thought it did not need to install msvcr80.dll, and is wrong about that. Try re-installing Erlang and select the checkbox about installing Microsoft Runtime. That way it is certain Erlang uses the distributed runtime library. On Mon, Oct 19, 2009 at 11:53:07PM +0200, info wrote: > And what about msvcr80.dll which is missing in my server ? > > On Sat, Oct 17, 2009 at 12:46:50AM +0100, info wrote: > > On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: > > I guess Robby wrote: > > > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > > > raimo+erlang-questions@REDACTED > > > > wrote: > > > > > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > > > Dear Raimo, > > > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > > > I don't understand inet_gethost.c :-( > > > > > Where "goes" this problem for finding the information ? > > > > > If we know where and what, we could perhaps find what is missing in my > > > > windows 2003 ... > > > > > Hope to read you soon ! > > > > > John > > > > > > > > Well, there is not much more to say. inet_gethost.c calls > > > > struct hostent *gethostbyname(const char *name) in the > > > > winsock2 library: > > > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > > > It never returns. > > > > It is deprecated but that does not mean broken. > > > > I have a Windows 2003 server that it works on. > > > > We have not found the reason it differs between > > > > my server and your server. > > > > And that is about it. > > > > > > > > > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > > > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > > > Version 4.0' installed. This is a specific add-on, nothing to do with > > > Microsoft Firewall in your Network Properties. > > > > > > > I guess info wrote: > > > but I don't use ISA server !!! > > > > > > > No other interesting firewall software? > > Yes: I am using Comodo. > > Can you try to uninstall Comodo and see if the problem disappears. > If so we know where to look for the problem. > (We have installed a 2003 server with Comodo and got no problems, > other than the possibility to do configuration errors that > would cause your symptoms) > > > > > I guess Robby wrote: > > > If I remove it, it works fine, if I add it it hangs. Apart from that the > > > W2003 build is as vanilla as it gets. > > > > > > Nothing much to go on, but I would not be surprised if some system DLL is > > > getting in the way. This is very hard to diagnose. > > > > > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > > > rule out poor linkage due to the VC++ setup used to build the shipping > > > executables. If I compile up a trivial gethostbyname() example from MSDN, it > > > works OK in all circumstances. > > > > > > There are a few strange dependencies in other shipped Windows files, for > > > example beam.dll seems to depend on a particular MSVCR80.DLL with the > > > version 8.0.50727.1433 . And if you have an older Windows machine, you get > > > the completely unhelpful popup box telling you that your program could not > > > be run. This has something to do with Windows Side-by-Side DLL deployment, > > > and I completely fail to grasp what that's all about. > > > > > > > I guess info wrote: > > > but I don't have msvcr80.dll !!! > > > > I guess you do, actually, since Erlang/OTP's installer is supposed > > to install that DLL, if it does not exist already. > > > > I guess Robby wrote: > > > > > > Robby > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From yu_c@REDACTED Tue Oct 20 10:20:46 2009 From: yu_c@REDACTED (Chih - Wei Yu [ MTN - Innovation Centre ]) Date: Tue, 20 Oct 2009 10:20:46 +0200 Subject: [erlang-questions] YAWS with SSL Connections Problem In-Reply-To: References: Message-ID: Hi Tomas, Thank you for your response. But with the enoproxysocket error, will it cause a buildup of these IDLE 'connections' that we're seeing on the box? $ netstat -an | grep IDLE | more *.* *.* 0 0 49152 0 IDLE *.* *.* 0 0 49152 0 IDLE *.* *.* 0 0 49152 0 IDLE *.* *.* 0 0 49152 0 IDLE *.* *.* 0 0 49152 0 IDLE *.* *.* 0 0 49152 0 IDLE *.* *.* 0 0 49152 0 IDLE *.* *.* 0 0 49152 0 IDLE $ netstat -an | grep IDLE | wc -l 1699 Cause after we restart the runtime, the ssl_esock process is restarted and the above is cleared. The ulimit for file descriptors is set to 4096 but even if this is increased I think we will still have the same problem as the IDLEs continuous to build up. Is there any other solution perhaps that we can try? Regards, Chih-Wei Yu -----Original Message----- From: Tomas Abrahamsson [mailto:tomas.abrahamsson@REDACTED] Sent: Tuesday, October 20, 2009 9:45 AM To: Chih - Wei Yu [ MTN - Innovation Centre ] Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] YAWS with SSL Connections Problem > We're running YAWS (1.75)(Erlang 12B-5) with SSL and there was a > stage where we ran out of file descriptors. Now what seems to be is > that ssl_esock process is using up all the descriptors. I don't remember exactly about reasons for usage of large number of file descriptors, though it is possible to raise the number of file descriptors a process can use, see the ulimit command (and on Solaris, also documentation for the /etc/system file, where you can raise the hard limits). It might possibly have something to do with your second question: > The following is also seen in the nohup.out > =ERROR REPORT==== 19-Oct-2009::16:08:41 === > SSL accept failed: enoproxysocket > The OS we're running on is Solaris 10. We've seen enoproxysocket on Solaris 10 with the old ssl implementation[1]. Not on Solaris 9 or 8 and not on Linux. IIRC, we've seen it under both light and heavy load. The high-level reason is that the Erlang side and the ssl_esock gets out of sync with each other. The low-level details are that the Erlang side opens a TCP connection to the ssl_esock, then sends a proxy_join command for that tcp-connection to the ssl_esock's stdin, but on Solaris 10, the ssl_esock discovers these two events in the opposite order, gets confused, and responds with the enoproxysocket error. The net result is that the SSL connection will be unusable, the Erlang side can't use it since it's received an error response from ssl_esock, and the ssl_esock doesn't know what to do with it. It seems this can happen both when accepting incoming SSL connections and when making an outgoing SSL connection. The ssl_esock code hasn't changed from R12B-4 to R13B02-1, it seems (except for copyright notices), so just upgrading to a newer Erlang version won't automatically solve your problem, unfortunately. ______ [1] The old ssl-implementation, i.e: an Erlang-side and the C-side ssl_esock, as opposed to the new_ssl which does all the SSL protocol handling in Erlang, see the ssl documentation for further details, espcially the {ssl_imp,new} option. BRs Tomas NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/SUPPORT/LEGAL/Pages/EmailDisclaimer.aspx From tomas.abrahamsson@REDACTED Tue Oct 20 11:11:37 2009 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Tue, 20 Oct 2009 11:11:37 +0200 Subject: [erlang-questions] YAWS with SSL Connections Problem In-Reply-To: (Chih's message of "Tue, 20 Oct 2009 10:20:46 +0200") References: Message-ID: > Hi Tomas, > Thank you for your response. But with the enoproxysocket error, will > it cause a buildup of these IDLE 'connections' that we're seeing on > the box? It could be. Does the number of connections in IDLE match the number of enoproxysocket errors (approximately)? In a server situation, such as for Yaws, for each incoming SSL-connection to ssl_esock, the Erlang-side will open another (plaintext) connection to the ssl_esock (over the loopback interface). So shortly after an incoming SSL-connect, the ssl_esock will use up two file descriptors, one for the SSL-connection and one for the plaintext connection from the Erlang side. Then if there's an enoproxysocket error, I guess that after a while, the client will grow tired of waiting and disconnect, so then there'll be only the plaintext connection left. > Is there any other solution perhaps that we can try? I guess your main options are to either try to make yaws use the new ssl (and check whether the new ssl works well for you -- the documentation for new_ssl mentions "alpha version"), or to move away from Solaris 10 (I guess the same goes for Solaris 11, but I haven't tested, so I don't know). BRs Tomas From info@REDACTED Tue Oct 20 11:33:35 2009 From: info@REDACTED (info) Date: Tue, 20 Oct 2009 11:33:35 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? References: <20091016153333.GA15561@erix.ericsson.se>, <200910170046502888607@its3.ch>, <20091019154614.GA16576@erix.ericsson.se> Message-ID: <200910201133351502502@its3.ch> Dear Raimo, I uninstalled Comodo and ... the problem disappeared !!!!!!!!!!!!!!!!!!!!! What else now ? John I On Sat, Oct 17, 2009 at 12:46:50AM +0100, info wrote: > On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: > I guess Robby wrote: > > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > > raimo+erlang-questions@REDACTED > > > wrote: > > > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > > Dear Raimo, > > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > > I don't understand inet_gethost.c :-( > > > > Where "goes" this problem for finding the information ? > > > > If we know where and what, we could perhaps find what is missing in my > > > windows 2003 ... > > > > Hope to read you soon ! > > > > John > > > > > > Well, there is not much more to say. inet_gethost.c calls > > > struct hostent *gethostbyname(const char *name) in the > > > winsock2 library: > > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > > It never returns. > > > It is deprecated but that does not mean broken. > > > I have a Windows 2003 server that it works on. > > > We have not found the reason it differs between > > > my server and your server. > > > And that is about it. > > > > > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > > Version 4.0' installed. This is a specific add-on, nothing to do with > > Microsoft Firewall in your Network Properties. > > > > I guess info wrote: > > but I don't use ISA server !!! > > > > No other interesting firewall software? > Yes: I am using Comodo. Can you try to uninstall Comodo and see if the problem disappears. If so we know where to look for the problem. (We have installed a 2003 server with Comodo and got no problems, other than the possibility to do configuration errors that would cause your symptoms) > > I guess Robby wrote: > > If I remove it, it works fine, if I add it it hangs. Apart from that the > > W2003 build is as vanilla as it gets. > > > > Nothing much to go on, but I would not be surprised if some system DLL is > > getting in the way. This is very hard to diagnose. > > > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > > rule out poor linkage due to the VC++ setup used to build the shipping > > executables. If I compile up a trivial gethostbyname() example from MSDN, it > > works OK in all circumstances. > > > > There are a few strange dependencies in other shipped Windows files, for > > example beam.dll seems to depend on a particular MSVCR80.DLL with the > > version 8.0.50727.1433 . And if you have an older Windows machine, you get > > the completely unhelpful popup box telling you that your program could not > > be run. This has something to do with Windows Side-by-Side DLL deployment, > > and I completely fail to grasp what that's all about. > > > > I guess info wrote: > > but I don't have msvcr80.dll !!! > > I guess you do, actually, since Erlang/OTP's installer is supposed > to install that DLL, if it does not exist already. > > I guess Robby wrote: > > > > Robby > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From info@REDACTED Tue Oct 20 11:33:52 2009 From: info@REDACTED (info) Date: Tue, 20 Oct 2009 11:33:52 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? References: <20091019154614.GA16576@erix.ericsson.se>, <200910192353067661212@its3.ch>, <20091020080906.GA29778@erix.ericsson.se> Message-ID: <200910201133519008110@its3.ch> OK, I found several versions in WinSxS. Which one is the good one ? Shall I copy it in system32 ? John Try searching from C:/WINDOWS. It may be under C:/WINDOWS/System32 or hide under C:/WINDOWS/WinSxS/*/ somewhere. It may also be so that the installer thought it did not need to install msvcr80.dll, and is wrong about that. Try re-installing Erlang and select the checkbox about installing Microsoft Runtime. That way it is certain Erlang uses the distributed runtime library. On Mon, Oct 19, 2009 at 11:53:07PM +0200, info wrote: > And what about msvcr80.dll which is missing in my server ? > > On Sat, Oct 17, 2009 at 12:46:50AM +0100, info wrote: > > On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: > > I guess Robby wrote: > > > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > > > raimo+erlang-questions@REDACTED > > > > wrote: > > > > > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > > > Dear Raimo, > > > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > > > I don't understand inet_gethost.c :-( > > > > > Where "goes" this problem for finding the information ? > > > > > If we know where and what, we could perhaps find what is missing in my > > > > windows 2003 ... > > > > > Hope to read you soon ! > > > > > John > > > > > > > > Well, there is not much more to say. inet_gethost.c calls > > > > struct hostent *gethostbyname(const char *name) in the > > > > winsock2 library: > > > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > > > It never returns. > > > > It is deprecated but that does not mean broken. > > > > I have a Windows 2003 server that it works on. > > > > We have not found the reason it differs between > > > > my server and your server. > > > > And that is about it. > > > > > > > > > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > > > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > > > Version 4.0' installed. This is a specific add-on, nothing to do with > > > Microsoft Firewall in your Network Properties. > > > > > > > I guess info wrote: > > > but I don't use ISA server !!! > > > > > > > No other interesting firewall software? > > Yes: I am using Comodo. > > Can you try to uninstall Comodo and see if the problem disappears. > If so we know where to look for the problem. > (We have installed a 2003 server with Comodo and got no problems, > other than the possibility to do configuration errors that > would cause your symptoms) > > > > > I guess Robby wrote: > > > If I remove it, it works fine, if I add it it hangs. Apart from that the > > > W2003 build is as vanilla as it gets. > > > > > > Nothing much to go on, but I would not be surprised if some system DLL is > > > getting in the way. This is very hard to diagnose. > > > > > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > > > rule out poor linkage due to the VC++ setup used to build the shipping > > > executables. If I compile up a trivial gethostbyname() example from MSDN, it > > > works OK in all circumstances. > > > > > > There are a few strange dependencies in other shipped Windows files, for > > > example beam.dll seems to depend on a particular MSVCR80.DLL with the > > > version 8.0.50727.1433 . And if you have an older Windows machine, you get > > > the completely unhelpful popup box telling you that your program could not > > > be run. This has something to do with Windows Side-by-Side DLL deployment, > > > and I completely fail to grasp what that's all about. > > > > > > > I guess info wrote: > > > but I don't have msvcr80.dll !!! > > > > I guess you do, actually, since Erlang/OTP's installer is supposed > > to install that DLL, if it does not exist already. > > > > I guess Robby wrote: > > > > > > Robby > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From yu_c@REDACTED Tue Oct 20 11:50:39 2009 From: yu_c@REDACTED (Chih - Wei Yu [ MTN - Innovation Centre ]) Date: Tue, 20 Oct 2009 11:50:39 +0200 Subject: [erlang-questions] YAWS with SSL Connections Problem In-Reply-To: References: Message-ID: Hi Tomas, Thank you for your response once again. Regards, Chih-Wei Yu -----Original Message----- From: Tomas Abrahamsson [mailto:tomas.abrahamsson@REDACTED] Sent: Tuesday, October 20, 2009 11:12 AM To: Chih - Wei Yu [ MTN - Innovation Centre ] Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] YAWS with SSL Connections Problem > Hi Tomas, > Thank you for your response. But with the enoproxysocket error, will > it cause a buildup of these IDLE 'connections' that we're seeing on > the box? It could be. Does the number of connections in IDLE match the number of enoproxysocket errors (approximately)? In a server situation, such as for Yaws, for each incoming SSL-connection to ssl_esock, the Erlang-side will open another (plaintext) connection to the ssl_esock (over the loopback interface). So shortly after an incoming SSL-connect, the ssl_esock will use up two file descriptors, one for the SSL-connection and one for the plaintext connection from the Erlang side. Then if there's an enoproxysocket error, I guess that after a while, the client will grow tired of waiting and disconnect, so then there'll be only the plaintext connection left. > Is there any other solution perhaps that we can try? I guess your main options are to either try to make yaws use the new ssl (and check whether the new ssl works well for you -- the documentation for new_ssl mentions "alpha version"), or to move away from Solaris 10 (I guess the same goes for Solaris 11, but I haven't tested, so I don't know). BRs Tomas NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/SUPPORT/LEGAL/Pages/EmailDisclaimer.aspx From v@REDACTED Tue Oct 20 12:26:57 2009 From: v@REDACTED (Valentin Micic) Date: Tue, 20 Oct 2009 12:26:57 +0200 Subject: [erlang-questions] YAWS with SSL Connections Problem In-Reply-To: Message-ID: <20091020102716.D05A03D0D3C@mail.pharos-avantgard.com> I really don't see how we know more now then we did at the beginning of this thread. Surely there has to be an explanation as to what really causes the problem. Instead, I've learned stuff that I really didn't want to know, such as how insecure Erlang SSL support is. Considering that most of the fraud comes from within, e.g. people that may have an access to the machine how do I: 1) prevent someone from "snooping" the local loopback connection? 2) prevent someone from tracing ssl_esock process and see clear text sent and received via read and write system calls? I've always assumed that SSL support is tightly coupled with linked-in driver, hence not exposing anything via clear text (well other than user-level function call). I agree that such an implementation might have been more difficult to accomplish; however, one can at least avoid using a clear-text communication on a loopback connection by providing some form of pre-aranged symmetric-key encryption. Am I the only one seeing a *big* problem here? V/ -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Chih - Wei Yu [ MTN - Innovation Centre ] Sent: 20 October 2009 11:51 AM To: Tomas Abrahamsson Cc: erlang-questions@REDACTED Subject: RE: [erlang-questions] YAWS with SSL Connections Problem Hi Tomas, Thank you for your response once again. Regards, Chih-Wei Yu -----Original Message----- From: Tomas Abrahamsson [mailto:tomas.abrahamsson@REDACTED] Sent: Tuesday, October 20, 2009 11:12 AM To: Chih - Wei Yu [ MTN - Innovation Centre ] Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] YAWS with SSL Connections Problem > Hi Tomas, > Thank you for your response. But with the enoproxysocket error, will > it cause a buildup of these IDLE 'connections' that we're seeing on > the box? It could be. Does the number of connections in IDLE match the number of enoproxysocket errors (approximately)? In a server situation, such as for Yaws, for each incoming SSL-connection to ssl_esock, the Erlang-side will open another (plaintext) connection to the ssl_esock (over the loopback interface). So shortly after an incoming SSL-connect, the ssl_esock will use up two file descriptors, one for the SSL-connection and one for the plaintext connection from the Erlang side. Then if there's an enoproxysocket error, I guess that after a while, the client will grow tired of waiting and disconnect, so then there'll be only the plaintext connection left. > Is there any other solution perhaps that we can try? I guess your main options are to either try to make yaws use the new ssl (and check whether the new ssl works well for you -- the documentation for new_ssl mentions "alpha version"), or to move away from Solaris 10 (I guess the same goes for Solaris 11, but I haven't tested, so I don't know). BRs Tomas NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/SUPPORT/LEGAL/Pages/EmailDisclaimer.aspx ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From steven.charles.davis@REDACTED Tue Oct 20 14:35:44 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 20 Oct 2009 05:35:44 -0700 (PDT) Subject: Any way to dynamically access Records? In-Reply-To: <4ADD03D4.6030703@m5net.com> References: <2a67d3ff0910181205g444eee08pce356494efd8fc1a@mail.gmail.com> <4ADD03D4.6030703@m5net.com> Message-ID: In the absence of mnesia to retrieve record field definitions, I settled on a simple solution: namely adding the field names as the first field in a record, e.g.: -record(component, {fields=[id, class, body], id, class, body}). For the case I was dealing with, the memory overhead of doing this was not an issue, but YMMV. /s On Oct 19, 7:27?pm, Angus Gratton wrote: > G.S. wrote: > > Is there a way to get this to work: Row#TableName.Column > > > Where all are variables, dropped into the function. > > FWIW, I (along with many others, it seems) have experienced similar > problems and found ways around them. > > There are some interesting existing solutions out on the net, like these: > > http://www.trapexit.org/Match_Specifications_And_Records_(Dynamically!)http://chlorophil.blogspot.com/2007/04/dynamic-record-access-function... > > Also, if you're just doing mnesia index lookups, mnesia:index_read() > will take an indexed field name and look it up at runtime (with the > associated performance hit.) > > What we ended up doing is code generating a lot of our records. > Subsequently, we also now code generate a module with functions to look > up arbitrary field values in records, and also functions to generate an > arbitrary matchspec from a list of tuples {fieldname, fieldvalue}. This > is a similar approach to the first link I posted above, although it's a > bit more lo-fi because all the functions get code generated, instead of > inferred by the parse transform. > > Hth. > > - Angus > > ________________________________________________________________ > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html > erlang-questions (at) erlang.org From james.hague@REDACTED Tue Oct 20 15:42:18 2009 From: james.hague@REDACTED (James Hague) Date: Tue, 20 Oct 2009 08:42:18 -0500 Subject: [erlang-questions] Tuple values returned in registers, without using heap space? In-Reply-To: <3dbc6d1c0910190806h6fc0990v213b4c0d8e5ef188@mail.gmail.com> References: <3dbc6d1c0910190806h6fc0990v213b4c0d8e5ef188@mail.gmail.com> Message-ID: >Except *within* functions when it is safe. Ah right, that must be what I remember hearing about when it was added to the compiler. Thank you! James From vinoski@REDACTED Tue Oct 20 16:04:26 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 20 Oct 2009 10:04:26 -0400 Subject: [erlang-questions] Any way to dynamically access Records? In-Reply-To: <4ADD03D4.6030703@m5net.com> References: <2a67d3ff0910181205g444eee08pce356494efd8fc1a@mail.gmail.com> <4ADD03D4.6030703@m5net.com> Message-ID: <65b2728e0910200704h505fdbelc498225188a3a3c9@mail.gmail.com> On Mon, Oct 19, 2009 at 8:27 PM, Angus Gratton wrote: > G.S. wrote: > > Is there a way to get this to work: Row#TableName.Column > > > > Where all are variables, dropped into the function. > > What we ended up doing is code generating a lot of our records. > Perhaps Ulf's "exported records" might be of use: --steve From raimo+erlang-questions@REDACTED Tue Oct 20 17:29:22 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 20 Oct 2009 17:29:22 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <200910192353067661212@its3.ch> References: <20091019154614.GA16576@erix.ericsson.se> <200910192353067661212@its3.ch> Message-ID: <20091020152922.GA13836@erix.ericsson.se> On Tue, 20 Oct 2009 11:33:35 +0200 info wrote: > I uninstalled Comodo and ... the problem disappeared !!!!!!!!!!!!!!!!!!!!! > What else now ? > John Then we know it is firewall related. And we could not reproduce it with Comodo. Some more tiny detail seems to be needed to trigger the bug. Which version of Comodo do you have? Reinstall and see if the problem appears again, but first uninstall Erlang. Installation order may matter to trigger the bug. Uninstall both Erlang and Comodo and try installing Comodo first then Erlang. If that fails uninstall Erlang then Comodo (latest first), and install Erlang first then Comodo (the other order). Watch out the next paragraph about installing Erlang. On Tue, 20 Oct 2009 11:33:52 +0200 info wrote: > OK, I found several versions in WinSxS. > Which one is the good one ? > Shall I copy it in system32 ? No, uninstall Erlang, and reinstall it. Make sure to select the approx."Install Microsoft Runtime" in the installer. The installer installs it into a for Erlang unique location so Erlang will run exactly the runtime library it needs. Do not move it! We are now hunting something fishy (for Robert R) with IIS server and firewall client. Can you (info) tell us what happens when you have the symptoms, if you in a freshly started Erlang node only look up a short name e.g "localhost", because we now have a situation where it kind of works until we do a lookup containing a dot which hangs and after that everything hangs. Can you confirm this peculiar behaviour? / Raimo On Mon, Oct 19, 2009 at 11:53:07PM +0200, info wrote: > And what about msvcr80.dll which is missing in my server ? > > On Sat, Oct 17, 2009 at 12:46:50AM +0100, info wrote: > > On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: > > I guess Robby wrote: > > > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > > > raimo+erlang-questions@REDACTED > > > > wrote: > > > > > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > > > Dear Raimo, > > > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > > > I don't understand inet_gethost.c :-( > > > > > Where "goes" this problem for finding the information ? > > > > > If we know where and what, we could perhaps find what is missing in my > > > > windows 2003 ... > > > > > Hope to read you soon ! > > > > > John > > > > > > > > Well, there is not much more to say. inet_gethost.c calls > > > > struct hostent *gethostbyname(const char *name) in the > > > > winsock2 library: > > > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > > > It never returns. > > > > It is deprecated but that does not mean broken. > > > > I have a Windows 2003 server that it works on. > > > > We have not found the reason it differs between > > > > my server and your server. > > > > And that is about it. > > > > > > > > > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > > > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > > > Version 4.0' installed. This is a specific add-on, nothing to do with > > > Microsoft Firewall in your Network Properties. > > > > > > > I guess info wrote: > > > but I don't use ISA server !!! > > > > > > > No other interesting firewall software? > > Yes: I am using Comodo. > > Can you try to uninstall Comodo and see if the problem disappears. > If so we know where to look for the problem. > (We have installed a 2003 server with Comodo and got no problems, > other than the possibility to do configuration errors that > would cause your symptoms) > > > > > I guess Robby wrote: > > > If I remove it, it works fine, if I add it it hangs. Apart from that the > > > W2003 build is as vanilla as it gets. > > > > > > Nothing much to go on, but I would not be surprised if some system DLL is > > > getting in the way. This is very hard to diagnose. > > > > > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > > > rule out poor linkage due to the VC++ setup used to build the shipping > > > executables. If I compile up a trivial gethostbyname() example from MSDN, it > > > works OK in all circumstances. > > > > > > There are a few strange dependencies in other shipped Windows files, for > > > example beam.dll seems to depend on a particular MSVCR80.DLL with the > > > version 8.0.50727.1433 . And if you have an older Windows machine, you get > > > the completely unhelpful popup box telling you that your program could not > > > be run. This has something to do with Windows Side-by-Side DLL deployment, > > > and I completely fail to grasp what that's all about. > > > > > > > I guess info wrote: > > > but I don't have msvcr80.dll !!! > > > > I guess you do, actually, since Erlang/OTP's installer is supposed > > to install that DLL, if it does not exist already. > > > > I guess Robby wrote: > > > > > > Robby > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From info@REDACTED Tue Oct 20 18:45:25 2009 From: info@REDACTED (info) Date: Tue, 20 Oct 2009 18:45:25 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? References: <20091019154614.GA16576@erix.ericsson.se>, <200910192353067661212@its3.ch>, <20091020152922.GA13836@erix.ericsson.se> Message-ID: <200910201845249248802@its3.ch> On Tue, 20 Oct 2009 11:33:35 +0200 info wrote: > I uninstalled Comodo and ... the problem disappeared !!!!!!!!!!!!!!!!!!!!! > What else now ? > John Then we know it is firewall related. And we could not reproduce it with Comodo. Some more tiny detail seems to be needed to trigger the bug. Which version of Comodo do you have? I had the version 2.4.19.185 .. I had because I didn't keep the exe :-( and the version 3 is not compatible with windows 2003 I am "naked" since yesterday ! Reinstall and see if the problem appears again, but first uninstall Erlang. Installation order may matter to trigger the bug. Uninstall both Erlang and Comodo and try installing Comodo first then Erlang. If that fails uninstall Erlang then Comodo (latest first), and install Erlang first then Comodo (the other order). Watch out the next paragraph about installing Erlang. On Tue, 20 Oct 2009 11:33:52 +0200 info wrote: > OK, I found several versions in WinSxS. > Which one is the good one ? > Shall I copy it in system32 ? No, uninstall Erlang, and reinstall it. Make sure to select the approx."Install Microsoft Runtime" in the installer. The installer installs it into a for Erlang unique location so Erlang will run exactly the runtime library it needs. Do not move it! We are now hunting something fishy (for Robert R) with IIS server and firewall client. Can you (info) tell us what happens when you have the symptoms, if you in a freshly started Erlang node only look up a short name e.g "localhost", because we now have a situation where it kind of works until we do a lookup containing a dot which hangs and after that everything hangs. Can you confirm this peculiar behaviour? / Raimo On Mon, Oct 19, 2009 at 11:53:07PM +0200, info wrote: > And what about msvcr80.dll which is missing in my server ? > > On Sat, Oct 17, 2009 at 12:46:50AM +0100, info wrote: > > On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: > > I guess Robby wrote: > > > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > > > raimo+erlang-questions@REDACTED > > > > wrote: > > > > > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > > > Dear Raimo, > > > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > > > I don't understand inet_gethost.c :-( > > > > > Where "goes" this problem for finding the information ? > > > > > If we know where and what, we could perhaps find what is missing in my > > > > windows 2003 ... > > > > > Hope to read you soon ! > > > > > John > > > > > > > > Well, there is not much more to say. inet_gethost.c calls > > > > struct hostent *gethostbyname(const char *name) in the > > > > winsock2 library: > > > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > > > It never returns. > > > > It is deprecated but that does not mean broken. > > > > I have a Windows 2003 server that it works on. > > > > We have not found the reason it differs between > > > > my server and your server. > > > > And that is about it. > > > > > > > > > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > > > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > > > Version 4.0' installed. This is a specific add-on, nothing to do with > > > Microsoft Firewall in your Network Properties. > > > > > > > I guess info wrote: > > > but I don't use ISA server !!! > > > > > > > No other interesting firewall software? > > Yes: I am using Comodo. > > Can you try to uninstall Comodo and see if the problem disappears. > If so we know where to look for the problem. > (We have installed a 2003 server with Comodo and got no problems, > other than the possibility to do configuration errors that > would cause your symptoms) > > > > > I guess Robby wrote: > > > If I remove it, it works fine, if I add it it hangs. Apart from that the > > > W2003 build is as vanilla as it gets. > > > > > > Nothing much to go on, but I would not be surprised if some system DLL is > > > getting in the way. This is very hard to diagnose. > > > > > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > > > rule out poor linkage due to the VC++ setup used to build the shipping > > > executables. If I compile up a trivial gethostbyname() example from MSDN, it > > > works OK in all circumstances. > > > > > > There are a few strange dependencies in other shipped Windows files, for > > > example beam.dll seems to depend on a particular MSVCR80.DLL with the > > > version 8.0.50727.1433 . And if you have an older Windows machine, you get > > > the completely unhelpful popup box telling you that your program could not > > > be run. This has something to do with Windows Side-by-Side DLL deployment, > > > and I completely fail to grasp what that's all about. > > > > > > > I guess info wrote: > > > but I don't have msvcr80.dll !!! > > > > I guess you do, actually, since Erlang/OTP's installer is supposed > > to install that DLL, if it does not exist already. > > > > I guess Robby wrote: > > > > > > Robby > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From tomas.abrahamsson@REDACTED Tue Oct 20 22:02:30 2009 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Tue, 20 Oct 2009 22:02:30 +0200 Subject: [erlang-questions] YAWS with SSL Connections Problem In-Reply-To: <20091020102716.D05A03D0D3C@mail.pharos-avantgard.com> (Valentin Micic's message of "Tue, 20 Oct 2009 12:26:57 +0200") References: <20091020102716.D05A03D0D3C@mail.pharos-avantgard.com> Message-ID: > [...] Instead, I've learned stuff that I really didn't want to know, such > as how insecure Erlang SSL support is. I guess in case you're going to handle sensitive data in your system, you'd want to take this and many more factors into consideration when doing a risk analysis. If clear text communication between the erlang vm and the ssl_esock over the loopback interface is an issue, then I suppose you might also want to secure the erlang inter-node communication, if you run a distributed system, and maybe you would also want to take measures against someone reading the memory image of the erlang vm os-process, or against someone reading the swap. > Considering that most of the fraud comes from within, e.g. people that may > have an access to the machine how do I: > > 1) prevent someone from "snooping" the local loopback connection? > 2) prevent someone from tracing ssl_esock process and see clear text sent > and received via read and write system calls? Aside from being restrictive in granting system privileges, I'm afraid I don't know, but would of course be interested in learning more in case there happens to be a way. > I've always assumed that SSL support is tightly coupled with linked-in > driver, hence not exposing anything via clear text (well other than > user-level function call). From this point of view, the new ssl implementation has an advantage over the old: it doesn't use any ssl_esock, but handles the SSL protocol entirely within the erlang vm, as far as I've understood. Are there any users of the new ssl implementation, or if not, what are the major show-stoppers? I know Yaws says "%%{ssl_imp, new} - still doesn't work (R13B)" in yaws_server.erl, and I know the ssl_new man page gives the impression of it being a bit unpolished, saying "[t]he alfa version has a few limitations ... listed below", but what's the major show-stoppers? Is it the lack of diffie-hellman key exchange, the lack of CRL support (as listed) or something else? Interoperability issues? BRs Tomas From v@REDACTED Wed Oct 21 00:54:48 2009 From: v@REDACTED (Valentin Micic) Date: Wed, 21 Oct 2009 00:54:48 +0200 Subject: [erlang-questions] YAWS with SSL Connections Problem In-Reply-To: Message-ID: <20091020225508.76B453D0D3C@mail.pharos-avantgard.com> > If clear text communication between the erlang vm and > the ssl_esock over the loopback interface is an issue, > then I suppose you might also want to secure the erlang > inter-node communication, if you run a distributed > system, and maybe you would also want to take measures > against someone reading the memory image of the erlang > vm os-process, or against someone reading the swap. Let's not get carried away here -- it is one thing to run a snoop on a socket that carries HTTPS request as a clear text, and completely different to try to understand Erlang encoded traffic that is multiplexed over a single socket. Barrier to entry is much higher, and, as a consequence you can round-up fewer suspects :-) >> I've always assumed that SSL support is tightly coupled with linked-in >> driver, hence not exposing anything via clear text (well other than >> user-level function call). > From this point of view, the new ssl implementation > has an advantage over the old: it doesn't use any > ssl_esock, but handles the SSL protocol entirely within > the erlang vm, as far as I've understood. I am very glad to hear that (if this is indeed the case). > Are there any users of the new ssl implementation, or > if not, what are the major show-stoppers? Usually, it is very difficult to motivate a new implementation (especially if it is advertised as alpha) in a commercial environment -- auditors would consider it as a major risk. Issues are rarely technical; however, when a technical issue may be exploited to further some political goal, it invariably is. Regards V/ From kenneth.lundin@REDACTED Wed Oct 21 10:52:01 2009 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 21 Oct 2009 10:52:01 +0200 Subject: [erlang-questions] asn1rt decode of 3GPP RANAP-PDU In-Reply-To: References: Message-ID: On Thu, Oct 15, 2009 at 7:39 AM, Mark Wallis wrote: > Dear Forum > > I have compiled the ASN.1 for 3GPP RANAP using the following options > > %% Generated by the Erlang ASN.1 BER-compiler version, utilizing > bit-syntax:1.6.10 > %% Purpose: encoder and decoder to the types in mod RANAP > > -module('RANAP'). > -include("RANAP.hrl"). > -define('RT_PER',asn1rt_per_bin). > -asn1_info([{vsn,'1.6.10'}, > ? ? ? ? ? ?{module,'RANAP'}, > ? ? ? ? ? ?{options,[per_bin,undec_rest,{i,[46]}]}]). > > > When I pass a RANAP-PDU through the deocder it decodes successfully but only > prints 'ProtocolIE-Field',x and not the Protocol IE name (see below). Is > this expected behaviour as I can see all the ids being declared in the > .erl file? Yes this is the expected behaviour. The Id's of the Protocol-IEs are plain integers declared in the ASN.1 spec as e.g.: id-AreaIdentity INTEGER ::= 0 id-CN-BroadcastInformationPiece INTEGER ::= 1 These value declarations are represented in the generated .erl file as functions which return the integer value. You can use them when you are creating data on the Erlang side. When you decode data representing a Protocol-IE the id is just an integer which is represented as an integer in the Erlang internal representation as well. It is not possible to convert an integer with value 0 to the symbol id-AreaIdentity because there could be lots of other value definitions with integer value 0. For the symbolic representation to work you would need another type of declaration for the Id's and I don't know if that would be practical and anyway that is not used in the RANAp standard. I think something like the below example will do the trick and let you handle the ID's for IEs as atoms and it should be no change on the line (if you are encoding as BER). If PER is used it might have impact on the encoding. Protocol-IE-ID ::= INTEGER {arealIdentity(0), cN-BroadcastInformationPiece(1), ... id-AreaIdentity Protocol-IE-ID ::= arealIdentity id-CN-BroadcastInformationPiece Protocol-IE-ID cn-BroadcastInformationPiece /Kenneth Erlang/OTP , Ericsson > > Regards > > Mark. > > 'RANAP':decode('RANAP-PDU', > list_to_binary([0,19,64,74,0,0,6,0,3,64,1,0,0,15,64,6,0,50,244,149,0,4,0,58 > ,64,8,0,50,244,149,0,4,0,4,0,16,64,24,23,5,8,0,50,244,149,0,133,79,8,57,1,87,0,0,0,4,40,51,3,79,24,160 > ,0,79,64,3,48,0,15,0,86,64,5,50,244,149,0,1])). > {ok,{initiatingMessage, > ? ? ? ?{'InitiatingMessage',19,ignore, > ? ? ? ? ? ?{'InitialUE-Message', > ? ? ? ? ? ? ? ?[{'ProtocolIE-Field',3,ignore,'cs-domain'}, > ? ? ? ? ? ? ? ? {'ProtocolIE-Field',15,ignore, > ? ? ? ? ? ? ? ? ? ? {'LAI',[50,244,149],[0,4],asn1_NOVALUE}}, > ? ? ? ? ? ? ? ? {'ProtocolIE-Field',58,ignore, > ? ? ? ? ? ? ? ? ? ? {'SAI',[50,244,149],[0,4],[0,4],asn1_NOVALUE}}, > ? ? ? ? ? ? ? ? {'ProtocolIE-Field',16,ignore, > ? ? ? ? ? ? ? ? ? ? [5,8,0,50,244,149,0,133,79,8,57|...]}, > ? ? ? ? ? ? ? ? {'ProtocolIE-Field',79,ignore,[0,0,1,1,0,0,0,0,0,0|...]}, > ? ? ? ? ? ? ? ? {'ProtocolIE-Field',86,ignore, > ? ? ? ? ? ? ? ? ? ? {'GlobalRNC-ID',[50,244,149],1}}], > ? ? ? ? ? ? ? ?asn1_NOVALUE}}}, > ? ?<<>>} > From raimo+erlang-questions@REDACTED Wed Oct 21 10:38:03 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 21 Oct 2009 10:38:03 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <200910201845249248802@its3.ch> References: <20091020152922.GA13836@erix.ericsson.se> <200910201845249248802@its3.ch> Message-ID: <20091021083803.GA32147@erix.ericsson.se> On Tue, Oct 20, 2009 at 06:45:25PM +0200, info wrote: > > On Tue, 20 Oct 2009 11:33:35 +0200 info wrote: > > I uninstalled Comodo and ... the problem disappeared !!!!!!!!!!!!!!!!!!!!! > > What else now ? > > John > > Then we know it is firewall related. And we could not > reproduce it with Comodo. Some more tiny detail seems > to be needed to trigger the bug. Which version of Comodo > do you have? > > I had the version 2.4.19.185 .. > I had because I didn't keep the exe :-( and the version 3 is not compatible with windows 2003 > I am "naked" since yesterday ! We tried Comodo Firewall 3.12, and it seems to work just fine. I.e on Windows 2003 R2 Enterprise Edition with SP2. > > Reinstall and see if the problem appears again, but first > uninstall Erlang. Installation order may matter to trigger > the bug. Uninstall both Erlang and Comodo and try > installing Comodo first then Erlang. If that fails > uninstall Erlang then Comodo (latest first), and install Erlang > first then Comodo (the other order). Watch out the next > paragraph about installing Erlang. > > On Tue, 20 Oct 2009 11:33:52 +0200 info wrote: > > OK, I found several versions in WinSxS. > > Which one is the good one ? > > Shall I copy it in system32 ? > > No, uninstall Erlang, and reinstall it. Make sure to > select the approx."Install Microsoft Runtime" in the > installer. The installer installs it into a for > Erlang unique location so Erlang will run exactly > the runtime library it needs. Do not move it! > > > We are now hunting something fishy (for Robert R) with > IIS server and firewall client. Can you (info) tell us > what happens when you have the symptoms, if you in > a freshly started Erlang node only look up a > short name e.g "localhost", because we now have > a situation where it kind of works until we do > a lookup containing a dot which hangs and after > that everything hangs. Can you confirm this peculiar > behaviour? > > / Raimo > > > > On Mon, Oct 19, 2009 at 11:53:07PM +0200, info wrote: > > And what about msvcr80.dll which is missing in my server ? > > > > On Sat, Oct 17, 2009 at 12:46:50AM +0100, info wrote: > > > On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: > > > I guess Robby wrote: > > > > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > > > > raimo+erlang-questions@REDACTED > > > > > wrote: > > > > > > > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > > > > Dear Raimo, > > > > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > > > > I don't understand inet_gethost.c :-( > > > > > > Where "goes" this problem for finding the information ? > > > > > > If we know where and what, we could perhaps find what is missing in my > > > > > windows 2003 ... > > > > > > Hope to read you soon ! > > > > > > John > > > > > > > > > > Well, there is not much more to say. inet_gethost.c calls > > > > > struct hostent *gethostbyname(const char *name) in the > > > > > winsock2 library: > > > > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > > > > It never returns. > > > > > It is deprecated but that does not mean broken. > > > > > I have a Windows 2003 server that it works on. > > > > > We have not found the reason it differs between > > > > > my server and your server. > > > > > And that is about it. > > > > > > > > > > > > > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > > > > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > > > > Version 4.0' installed. This is a specific add-on, nothing to do with > > > > Microsoft Firewall in your Network Properties. > > > > > > > > > > I guess info wrote: > > > > but I don't use ISA server !!! > > > > > > > > > > No other interesting firewall software? > > > Yes: I am using Comodo. > > > > Can you try to uninstall Comodo and see if the problem disappears. > > If so we know where to look for the problem. > > (We have installed a 2003 server with Comodo and got no problems, > > other than the possibility to do configuration errors that > > would cause your symptoms) > > > > > > > > I guess Robby wrote: > > > > If I remove it, it works fine, if I add it it hangs. Apart from that the > > > > W2003 build is as vanilla as it gets. > > > > > > > > Nothing much to go on, but I would not be surprised if some system DLL is > > > > getting in the way. This is very hard to diagnose. > > > > > > > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > > > > rule out poor linkage due to the VC++ setup used to build the shipping > > > > executables. If I compile up a trivial gethostbyname() example from MSDN, it > > > > works OK in all circumstances. > > > > > > > > There are a few strange dependencies in other shipped Windows files, for > > > > example beam.dll seems to depend on a particular MSVCR80.DLL with the > > > > version 8.0.50727.1433 . And if you have an older Windows machine, you get > > > > the completely unhelpful popup box telling you that your program could not > > > > be run. This has something to do with Windows Side-by-Side DLL deployment, > > > > and I completely fail to grasp what that's all about. > > > > > > > > > > I guess info wrote: > > > > but I don't have msvcr80.dll !!! > > > > > > I guess you do, actually, since Erlang/OTP's installer is supposed > > > to install that DLL, if it does not exist already. > > > > > > I guess Robby wrote: > > > > > > > > Robby > > > > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From info@REDACTED Wed Oct 21 11:57:18 2009 From: info@REDACTED (info) Date: Wed, 21 Oct 2009 11:57:18 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? References: <20091020152922.GA13836@erix.ericsson.se>, <200910201845249248802@its3.ch>, <20091021083803.GA32147@erix.ericsson.se> Message-ID: <200910211157176241641@its3.ch> Dear Raimo, This night ... I am a little bit tired ... I tested according to your plan. I found again in my archive the comodo exe. Ouf ! This is the result: Test #1 uninstall Comodo restart inet:gethostbyname ok uninstall erlang install Comodo install erlang with windows runtime inet:gethostbyname ko Test #2 uninstall erlang uninstall Commodo restart install erlang with windows runtime inet:gethostbyname ok install Comodo restart inet:gethostbyname ko On Tue, Oct 20, 2009 at 06:45:25PM +0200, info wrote: > > On Tue, 20 Oct 2009 11:33:35 +0200 info wrote: > > I uninstalled Comodo and ... the problem disappeared !!!!!!!!!!!!!!!!!!!!! > > What else now ? > > John > > Then we know it is firewall related. And we could not > reproduce it with Comodo. Some more tiny detail seems > to be needed to trigger the bug. Which version of Comodo > do you have? > > I had the version 2.4.19.185 .. > I had because I didn't keep the exe :-( and the version 3 is not compatible with windows 2003 > I am "naked" since yesterday ! We tried Comodo Firewall 3.12, and it seems to work just fine. I.e on Windows 2003 R2 Enterprise Edition with SP2. The features of the product give a no support of windows 2003 !? I have windows 2003 sbs with sp2 > > Reinstall and see if the problem appears again, but first > uninstall Erlang. Installation order may matter to trigger > the bug. Uninstall both Erlang and Comodo and try > installing Comodo first then Erlang. If that fails > uninstall Erlang then Comodo (latest first), and install Erlang > first then Comodo (the other order). Watch out the next > paragraph about installing Erlang. > > On Tue, 20 Oct 2009 11:33:52 +0200 info wrote: > > OK, I found several versions in WinSxS. > > Which one is the good one ? > > Shall I copy it in system32 ? > > No, uninstall Erlang, and reinstall it. Make sure to > select the approx."Install Microsoft Runtime" in the > installer. The installer installs it into a for > Erlang unique location so Erlang will run exactly > the runtime library it needs. Do not move it! > > > We are now hunting something fishy (for Robert R) with > IIS server and firewall client. Can you (info) tell us > what happens when you have the symptoms, if you in > a freshly started Erlang node only look up a > short name e.g "localhost", because we now have > a situation where it kind of works until we do > a lookup containing a dot which hangs and after > that everything hangs. Can you confirm this peculiar > behaviour? > > / Raimo > > > > On Mon, Oct 19, 2009 at 11:53:07PM +0200, info wrote: > > And what about msvcr80.dll which is missing in my server ? > > > > On Sat, Oct 17, 2009 at 12:46:50AM +0100, info wrote: > > > On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: > > > I guess Robby wrote: > > > > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > > > > raimo+erlang-questions@REDACTED > > > > > wrote: > > > > > > > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > > > > Dear Raimo, > > > > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > > > > I don't understand inet_gethost.c :-( > > > > > > Where "goes" this problem for finding the information ? > > > > > > If we know where and what, we could perhaps find what is missing in my > > > > > windows 2003 ... > > > > > > Hope to read you soon ! > > > > > > John > > > > > > > > > > Well, there is not much more to say. inet_gethost.c calls > > > > > struct hostent *gethostbyname(const char *name) in the > > > > > winsock2 library: > > > > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > > > > It never returns. > > > > > It is deprecated but that does not mean broken. > > > > > I have a Windows 2003 server that it works on. > > > > > We have not found the reason it differs between > > > > > my server and your server. > > > > > And that is about it. > > > > > > > > > > > > > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > > > > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > > > > Version 4.0' installed. This is a specific add-on, nothing to do with > > > > Microsoft Firewall in your Network Properties. > > > > > > > > > > I guess info wrote: > > > > but I don't use ISA server !!! > > > > > > > > > > No other interesting firewall software? > > > Yes: I am using Comodo. > > > > Can you try to uninstall Comodo and see if the problem disappears. > > If so we know where to look for the problem. > > (We have installed a 2003 server with Comodo and got no problems, > > other than the possibility to do configuration errors that > > would cause your symptoms) > > > > > > > > I guess Robby wrote: > > > > If I remove it, it works fine, if I add it it hangs. Apart from that the > > > > W2003 build is as vanilla as it gets. > > > > > > > > Nothing much to go on, but I would not be surprised if some system DLL is > > > > getting in the way. This is very hard to diagnose. > > > > > > > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > > > > rule out poor linkage due to the VC++ setup used to build the shipping > > > > executables. If I compile up a trivial gethostbyname() example from MSDN, it > > > > works OK in all circumstances. > > > > > > > > There are a few strange dependencies in other shipped Windows files, for > > > > example beam.dll seems to depend on a particular MSVCR80.DLL with the > > > > version 8.0.50727.1433 . And if you have an older Windows machine, you get > > > > the completely unhelpful popup box telling you that your program could not > > > > be run. This has something to do with Windows Side-by-Side DLL deployment, > > > > and I completely fail to grasp what that's all about. > > > > > > > > > > I guess info wrote: > > > > but I don't have msvcr80.dll !!! > > > > > > I guess you do, actually, since Erlang/OTP's installer is supposed > > > to install that DLL, if it does not exist already. > > > > > > I guess Robby wrote: > > > > > > > > Robby > > > > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From max.lapshin@REDACTED Wed Oct 21 12:18:44 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 21 Oct 2009 14:18:44 +0400 Subject: [erlang-questions] Re: Chain two spawned processes In-Reply-To: References: <3dbc6d1c0910190810i7c9ef361w50f246e188a5ad2b@mail.gmail.com> <3dbc6d1c0910191435g38800e55o981a6ac6f39bb583@mail.gmail.com> Message-ID: This is first patch, required to catch stderr of spawned process (tools like x264 print their status to stderr). bif open_port/2 gets new flag: error (maybe bad name?). If you pass it, process will receive messages {error, ErrorData} Patch made for R13B02-1, not appliable for QNX because I even cannot imagine, how to test it. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-capture-stderr-of-spawned-process.patch Type: application/octet-stream Size: 16384 bytes Desc: not available URL: From raimo+erlang-questions@REDACTED Wed Oct 21 14:28:04 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 21 Oct 2009 14:28:04 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <200910211157176241641@its3.ch> References: <20091021083803.GA32147@erix.ericsson.se> <200910211157176241641@its3.ch> Message-ID: <20091021122804.GA4311@erix.ericsson.se> On Wed, Oct 21, 2009 at 11:57:18AM +0200, info wrote: > Dear Raimo, > This night ... I am a little bit tired ... I tested according to your plan. I found again in my archive the comodo exe. Ouf ! > This is the result: > Test #1 > uninstall Comodo > restart > inet:gethostbyname ok > uninstall erlang > install Comodo > install erlang with windows runtime > inet:gethostbyname ko > Test #2 > uninstall erlang > uninstall Commodo > restart > install erlang with windows runtime > inet:gethostbyname ok > install Comodo > restart > inet:gethostbyname ko And with "ko" you mean "knock out", I guess... > > On Tue, Oct 20, 2009 at 06:45:25PM +0200, info wrote: > > > > On Tue, 20 Oct 2009 11:33:35 +0200 info wrote: > > > I uninstalled Comodo and ... the problem disappeared !!!!!!!!!!!!!!!!!!!!! > > > What else now ? > > > John > > > > Then we know it is firewall related. And we could not > > reproduce it with Comodo. Some more tiny detail seems > > to be needed to trigger the bug. Which version of Comodo > > do you have? > > > > I had the version 2.4.19.185 .. > > I had because I didn't keep the exe :-( and the version 3 is not compatible with windows 2003 > > I am "naked" since yesterday ! > > We tried Comodo Firewall 3.12, and it seems to work just fine. > I.e on Windows 2003 R2 Enterprise Edition with SP2. > The features of the product give a no support of windows 2003 !? > I have windows 2003 sbs with sp2 I read in the Comodo forums that up to at least 3.10 it refused to install saying "OS not supported". If so you could suspect 3.12 kind of supports Windows 2003... I have located 2.4.19.184 on the Internet, we will try it and hope the difference to 2.4.19.185 does not matter... > > > > > Reinstall and see if the problem appears again, but first > > uninstall Erlang. Installation order may matter to trigger > > the bug. Uninstall both Erlang and Comodo and try > > installing Comodo first then Erlang. If that fails > > uninstall Erlang then Comodo (latest first), and install Erlang > > first then Comodo (the other order). Watch out the next > > paragraph about installing Erlang. > > > > On Tue, 20 Oct 2009 11:33:52 +0200 info wrote: > > > OK, I found several versions in WinSxS. > > > Which one is the good one ? > > > Shall I copy it in system32 ? > > > > No, uninstall Erlang, and reinstall it. Make sure to > > select the approx."Install Microsoft Runtime" in the > > installer. The installer installs it into a for > > Erlang unique location so Erlang will run exactly > > the runtime library it needs. Do not move it! > > > > > > We are now hunting something fishy (for Robert R) with > > IIS server and firewall client. Can you (info) tell us > > what happens when you have the symptoms, if you in > > a freshly started Erlang node only look up a > > short name e.g "localhost", because we now have > > a situation where it kind of works until we do > > a lookup containing a dot which hangs and after > > that everything hangs. Can you confirm this peculiar > > behaviour? > > > > / Raimo > > > > > > > > On Mon, Oct 19, 2009 at 11:53:07PM +0200, info wrote: > > > And what about msvcr80.dll which is missing in my server ? > > > > > > On Sat, Oct 17, 2009 at 12:46:50AM +0100, info wrote: > > > > On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: > > > > I guess Robby wrote: > > > > > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > > > > > raimo+erlang-questions@REDACTED > > > > > > wrote: > > > > > > > > > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > > > > > Dear Raimo, > > > > > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > > > > > I don't understand inet_gethost.c :-( > > > > > > > Where "goes" this problem for finding the information ? > > > > > > > If we know where and what, we could perhaps find what is missing in my > > > > > > windows 2003 ... > > > > > > > Hope to read you soon ! > > > > > > > John > > > > > > > > > > > > Well, there is not much more to say. inet_gethost.c calls > > > > > > struct hostent *gethostbyname(const char *name) in the > > > > > > winsock2 library: > > > > > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > > > > > It never returns. > > > > > > It is deprecated but that does not mean broken. > > > > > > I have a Windows 2003 server that it works on. > > > > > > We have not found the reason it differs between > > > > > > my server and your server. > > > > > > And that is about it. > > > > > > > > > > > > > > > > > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > > > > > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > > > > > Version 4.0' installed. This is a specific add-on, nothing to do with > > > > > Microsoft Firewall in your Network Properties. > > > > > > > > > > > > > I guess info wrote: > > > > > but I don't use ISA server !!! > > > > > > > > > > > > > No other interesting firewall software? > > > > Yes: I am using Comodo. > > > > > > Can you try to uninstall Comodo and see if the problem disappears. > > > If so we know where to look for the problem. > > > (We have installed a 2003 server with Comodo and got no problems, > > > other than the possibility to do configuration errors that > > > would cause your symptoms) > > > > > > > > > > > I guess Robby wrote: > > > > > If I remove it, it works fine, if I add it it hangs. Apart from that the > > > > > W2003 build is as vanilla as it gets. > > > > > > > > > > Nothing much to go on, but I would not be surprised if some system DLL is > > > > > getting in the way. This is very hard to diagnose. > > > > > > > > > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > > > > > rule out poor linkage due to the VC++ setup used to build the shipping > > > > > executables. If I compile up a trivial gethostbyname() example from MSDN, it > > > > > works OK in all circumstances. > > > > > > > > > > There are a few strange dependencies in other shipped Windows files, for > > > > > example beam.dll seems to depend on a particular MSVCR80.DLL with the > > > > > version 8.0.50727.1433 . And if you have an older Windows machine, you get > > > > > the completely unhelpful popup box telling you that your program could not > > > > > be run. This has something to do with Windows Side-by-Side DLL deployment, > > > > > and I completely fail to grasp what that's all about. > > > > > > > > > > > > > I guess info wrote: > > > > > but I don't have msvcr80.dll !!! > > > > > > > > I guess you do, actually, since Erlang/OTP's installer is supposed > > > > to install that DLL, if it does not exist already. > > > > > > > > I guess Robby wrote: > > > > > > > > > > Robby > > > > > > > > -- > > > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From robstewart57@REDACTED Wed Oct 21 16:18:28 2009 From: robstewart57@REDACTED (Rob Stewart) Date: Wed, 21 Oct 2009 15:18:28 +0100 Subject: MNesia Distribution Questions Message-ID: Hi there folks, Very quickly about me: - I'm doing a Masters in Software Engineering - I'm looking closely at Mnesia to involve it in my study OK, so here's where I'm at. It's quite early on in my report, but I'm getting a feel for what could be a useful investigation. I am briefly discussing cloud computing, and then in more detail at distributed computation and data storage (along with fault tolerance, high availability etc...). My supervisor and I have agreed that to illustrate the use of a distributed computation system, I am going to perform some large computation on a large dataset, probably using Pig (dataflow language atop of Hadoop). I have been given the university cluster to deploy Hadoop. (Bear with me....) So... now the big question for me right now is where to find a useful competitor (or rather a solution with similar goals). The easy option would be to compare Pig to another Hadoop interface, i.e. Hive, but those results would be pretty uninteresting). So instead, I'm looking into the realm of distributed databases. Now... as far as I'm concerned, the way in which Mnesia distributes the availability of data across nodes is similar to how Hadoop distributes data across the HDFS (Hadoop file system) across nodes). My issue here is, my lack of understanding on how a data query computation is distributed over a network of Mnesia nodes. I have a good understanding of how this is achieved with Hadoop (if there are 10 datanodes, then each will get a tenth of the work), but is there such a thing as parallel query processing with MNesia? Or... is MNesia just a way to very very quickly replicate the availability of data. I hope that you guru's can shed some light on this for me. I'm not aware of exactly how MNesia would deal with a data query where the MNesia network consists of say, 10 nodes? Does a user query just one of the 10, or does a user query the network? I'm really trying to think of a fair and interesting way to compare the concept of a distributed database (MNesia), against a distributed processing engine (Hadoop). There are other things I want to delve into also... For instance, I really need to know more about the difference between CouchDB and MNesia. So far, I can only establish that CouchDB is more useful for networks where nodes are likely to go offline at various times. (Not much knowledge!!). If, however, comparing a distributed database engine against a distributed processing engine is a non starter, let me know of that too !! Many thanks, I would really appreciate some feedback. Rob Stewart From roberto.aloi@REDACTED Wed Oct 21 16:49:46 2009 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Wed, 21 Oct 2009 15:49:46 +0100 Subject: [erlang-questions] MNesia Distribution Questions In-Reply-To: References: Message-ID: <930765BA-AFF7-4FE9-97BC-23CB16A6935C@erlang-consulting.com> Hi Rob, I guess the best I can do is pointing you to some useful resources: - http://www.erlang.se/publications/mnesia_overview.pdf - http://www.erlang.org/doc/apps/mnesia/Mnesia_chap1.html#1 - http://couchdb.apache.org/ - http://oreilly.com/catalog/9780596158163 - http://wiki.apache.org/couchdb/Frequently_asked_questions#why_no_mnesia I advise you to give a quick look to these references and, if you still are in doubt, ask the mailing list. Hope to be useful. Best regards, Roberto Aloi roberto.aloi@REDACTED http://www.erlang-consulting.com --- On Oct 21, 2009, at 3:18 PM, Rob Stewart wrote: > Hi there folks, > Very quickly about me: > - I'm doing a Masters in Software Engineering > - I'm looking closely at Mnesia to involve it in my study > > > OK, so here's where I'm at. It's quite early on in my report, but I'm > getting a feel for what could be a useful investigation. I am briefly > discussing cloud computing, and then in more detail at distributed > computation and data storage (along with fault tolerance, high > availability > etc...). My supervisor and I have agreed that to illustrate the use > of a > distributed computation system, I am going to perform some large > computation > on a large dataset, probably using Pig (dataflow language atop of > Hadoop). I > have been given the university cluster to deploy Hadoop. > > (Bear with me....) > > So... now the big question for me right now is where to find a > useful competitor (or rather a solution with similar goals). The > easy option > would be to compare Pig to another Hadoop interface, i.e. Hive, but > those > results would be pretty uninteresting). So instead, I'm looking into > the > realm of distributed databases. Now... as far as I'm concerned, the > way in > which Mnesia distributes the availability of data across nodes is > similar > to how Hadoop distributes data across the HDFS (Hadoop file system) > across > nodes). My issue here is, my lack of understanding on how a data query > computation is distributed over a network of Mnesia nodes. I have a > good > understanding of how this is achieved with Hadoop (if there are 10 > datanodes, then each will get a tenth of the work), but is there > such a > thing as parallel query processing with MNesia? Or... is MNesia just > a way > to very very quickly replicate the availability of data. > > I hope that you guru's can shed some light on this for me. I'm not > aware of > exactly how MNesia would deal with a data query where the MNesia > network > consists of say, 10 nodes? Does a user query just one of the 10, or > does a > user query the network? I'm really trying to think of a fair and > interesting > way to compare the concept of a distributed database (MNesia), > against a > distributed processing engine (Hadoop). > > There are other things I want to delve into also... For instance, I > really > need to know more about the difference between CouchDB and MNesia. > So far, I > can only establish that CouchDB is more useful for networks where > nodes are > likely to go offline at various times. (Not much knowledge!!). > > If, however, comparing a distributed database engine against a > distributed > processing engine is a non starter, let me know of that too !! > > Many thanks, I would really appreciate some feedback. > > > Rob Stewart From cureadvocate@REDACTED Wed Oct 21 18:00:53 2009 From: cureadvocate@REDACTED (Steven Edwards) Date: Wed, 21 Oct 2009 12:00:53 -0400 Subject: EUnit install fails Message-ID: I cloned the eunit trunk this morning (with git svn clone), cd'd to the eunit directory and ran make--then sudo make. Both return the following error: --- make[1]: Entering directory `/home/steven/eunit/src' erlc -pa ../ebin -W -I../include +warn_unused_vars +nowarn_shadow_vars +warn_unused_import -o../ebin eunit_autoexport.erl /home/steven/eunit/src/../ebin/eunit_autoexport.bea#: error writing file make[1]: *** [../ebin/eunit_autoexport.beam] Error 1 make[1]: Leaving directory `/home/steven/eunit/src' make: *** [subdirs] Error 2 --- Should I just try again in a couple days? :) Thanks, Steven From robstewart57@REDACTED Wed Oct 21 19:00:08 2009 From: robstewart57@REDACTED (Rob Stewart) Date: Wed, 21 Oct 2009 18:00:08 +0100 Subject: [erlang-questions] MNesia Distribution Questions In-Reply-To: <930765BA-AFF7-4FE9-97BC-23CB16A6935C@erlang-consulting.com> References: <930765BA-AFF7-4FE9-97BC-23CB16A6935C@erlang-consulting.com> Message-ID: Hi Roberto, Very useful links, thanks. >From that, I really just have two telling questions, that I have not, as yet, found a clear answer to. 1. And example - I have a compex query to apply to a dataset, of, say 10,000,000 rows in an MNesia database. This database is replicated over 10 nodes in a network. Will the query be split for equal computation to each of the 10 nodes, or will the query be executed on either one random or one selected MNesia node. 2. How, in a most interesting way, could a distributed database, like MNesia be compared (in detail) to a distributed processing engine like Hadoop or Dryad? Taking the example from part 1, I know that Hadoop will map/reduce the job across the 10 nodes for parallel processing. And either Pig or Hive or HBase provide a database-like interface to the Hadoop data. So does MNesia share any common goals with Hadoop and such like? Regards, Rob Stewart 2009/10/21 Roberto Aloi > Hi Rob, > > I guess the best I can do is pointing you to some useful resources: > > - http://www.erlang.se/publications/mnesia_overview.pdf > - http://www.erlang.org/doc/apps/mnesia/Mnesia_chap1.html#1 > - http://couchdb.apache.org/ > - http://oreilly.com/catalog/9780596158163 > - http://wiki.apache.org/couchdb/Frequently_asked_questions#why_no_mnesia > > I advise you to give a quick look to these references and, if you still are > in doubt, ask the mailing list. > Hope to be useful. > > Best regards, > > Roberto Aloi > roberto.aloi@REDACTED > http://www.erlang-consulting.com > --- > > > > > > On Oct 21, 2009, at 3:18 PM, Rob Stewart wrote: > > Hi there folks, >> Very quickly about me: >> - I'm doing a Masters in Software Engineering >> - I'm looking closely at Mnesia to involve it in my study >> >> >> OK, so here's where I'm at. It's quite early on in my report, but I'm >> getting a feel for what could be a useful investigation. I am briefly >> discussing cloud computing, and then in more detail at distributed >> computation and data storage (along with fault tolerance, high >> availability >> etc...). My supervisor and I have agreed that to illustrate the use of a >> distributed computation system, I am going to perform some large >> computation >> on a large dataset, probably using Pig (dataflow language atop of Hadoop). >> I >> have been given the university cluster to deploy Hadoop. >> >> (Bear with me....) >> >> So... now the big question for me right now is where to find a >> useful competitor (or rather a solution with similar goals). The easy >> option >> would be to compare Pig to another Hadoop interface, i.e. Hive, but those >> results would be pretty uninteresting). So instead, I'm looking into the >> realm of distributed databases. Now... as far as I'm concerned, the way in >> which Mnesia distributes the availability of data across nodes is similar >> to how Hadoop distributes data across the HDFS (Hadoop file system) across >> nodes). My issue here is, my lack of understanding on how a data query >> computation is distributed over a network of Mnesia nodes. I have a good >> understanding of how this is achieved with Hadoop (if there are 10 >> datanodes, then each will get a tenth of the work), but is there such a >> thing as parallel query processing with MNesia? Or... is MNesia just a way >> to very very quickly replicate the availability of data. >> >> I hope that you guru's can shed some light on this for me. I'm not aware >> of >> exactly how MNesia would deal with a data query where the MNesia network >> consists of say, 10 nodes? Does a user query just one of the 10, or does a >> user query the network? I'm really trying to think of a fair and >> interesting >> way to compare the concept of a distributed database (MNesia), against a >> distributed processing engine (Hadoop). >> >> There are other things I want to delve into also... For instance, I really >> need to know more about the difference between CouchDB and MNesia. So far, >> I >> can only establish that CouchDB is more useful for networks where nodes >> are >> likely to go offline at various times. (Not much knowledge!!). >> >> If, however, comparing a distributed database engine against a distributed >> processing engine is a non starter, let me know of that too !! >> >> Many thanks, I would really appreciate some feedback. >> >> >> Rob Stewart >> > > From ulf.wiger@REDACTED Wed Oct 21 19:06:20 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 21 Oct 2009 19:06:20 +0200 Subject: [erlang-questions] MNesia Distribution Questions In-Reply-To: References: <930765BA-AFF7-4FE9-97BC-23CB16A6935C@erlang-consulting.com> Message-ID: <4ADF3F8C.4030009@erlang-consulting.com> Rob Stewart wrote: > > 1. And example - I have a compex query to apply to a dataset, of, say > 10,000,000 rows in an MNesia database. This database is replicated over 10 > nodes in a network. Will the query be split for equal computation to each of > the 10 nodes, or will the query be executed on either one random or one > selected MNesia node. If it's one homogeneous table that is simply replicated across all nodes, the query will be executed on the originating node only. If the table is fragmented, a subset of fragments will be identified (if the whole key isn't bound, this will be all fragments), and the query will be executed on all fragments in parallel. The resulting sets will be merged, respecting the ordering semantics of the table (i.e. sorted if it's an ordered_set table, otherwise not). BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From rtrlists@REDACTED Wed Oct 21 19:15:23 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 21 Oct 2009 18:15:23 +0100 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <20091020152922.GA13836@erix.ericsson.se> References: <20091019154614.GA16576@erix.ericsson.se> <200910192353067661212@its3.ch> <20091020152922.GA13836@erix.ericsson.se> Message-ID: <6a3ae47e0910211015o3fb343fdo71ad84448dac340a@mail.gmail.com> On Tue, Oct 20, 2009 at 4:29 PM, Raimo Niskanen < raimo+erlang-questions@REDACTED > wrote: > > We are now hunting something fishy (for Robert R) with > IIS server and firewall client. Can you (info) tell us > what happens when you have the symptoms, if you in > a freshly started Erlang node only look up a > short name e.g "localhost", because we now have > a situation where it kind of works until we do > a lookup containing a dot which hangs and after > that everything hangs. Can you confirm this peculiar > behaviour? > > / Raimo > > Yes, an initial lookup of localhost worked, but looking up some other machine hung. But I cannot remember 100% if there was a difference between dotted and local network names. I will try this again sometime next week (when I have access to the machine again) to see if I can get a clearer picture of the behaviour. Robby From kevin@REDACTED Wed Oct 21 19:39:11 2009 From: kevin@REDACTED (Kevin A. Smith) Date: Wed, 21 Oct 2009 13:39:11 -0400 Subject: [erlang-questions] EUnit install fails In-Reply-To: References: Message-ID: <779FC0D9-B19D-4A44-8F9D-6AC19A1CDE06@hypotheticalabs.com> Did the build create an ebin directory in the path indicated? Whenever I get that error it means erlc is trying to write to a non-existent path. --Kevin On Oct 21, 2009, at 12:00 PM, Steven Edwards wrote: > I cloned the eunit trunk this morning (with git svn clone), cd'd to > the > eunit directory and ran make--then sudo make. Both return the > following > error: > > --- > make[1]: Entering directory `/home/steven/eunit/src' > erlc -pa ../ebin -W -I../include +warn_unused_vars +nowarn_shadow_vars > +warn_unused_import -o../ebin eunit_autoexport.erl > /home/steven/eunit/src/../ebin/eunit_autoexport.bea#: error writing > file > make[1]: *** [../ebin/eunit_autoexport.beam] Error 1 > make[1]: Leaving directory `/home/steven/eunit/src' > make: *** [subdirs] Error 2 > --- > > Should I just try again in a couple days? :) > > Thanks, > > Steven From cureadvocate@REDACTED Wed Oct 21 20:10:00 2009 From: cureadvocate@REDACTED (Steven Edwards) Date: Wed, 21 Oct 2009 14:10:00 -0400 Subject: [erlang-questions] EUnit install fails In-Reply-To: <4ADF3D2C.5020305@gmail.com> References: <4ADF3D2C.5020305@gmail.com> Message-ID: It didn't. I created the ebin directory and re-ran make. It got further, but now fails with this message: --- erlc -pa ../ebin -W -I../include +warn_unused_vars +nowarn_shadow_vars +warn_unused_import -o../ebin eunit_data.erl ./eunit_data.erl:29: can't find include lib "kernel/include/file.hrl" ./eunit_data.erl:337: record file_info undefined ./eunit_data.erl:611: record file_info undefined make[1]: *** [../ebin/eunit_data.beam] Error 1 make[1]: Leaving directory `/home/steven/eunit/src' make: *** [subdirs] Error 2 --- /usr/lib/erlang/lib contains kernel-2.12.5, but no kernel. ./kernel-2.12.5 lacks an include directory. find / -type f -name file.hrl returns nothing. Thanks for your help. Steven On Wed, Oct 21, 2009 at 12:56 PM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > Steven Edwards wrote: > >> I cloned the eunit trunk this morning (with git svn clone), cd'd to the >> eunit directory and ran make--then sudo make. Both return the following >> error: >> >> --- >> make[1]: Entering directory `/home/steven/eunit/src' >> erlc -pa ../ebin -W -I../include +warn_unused_vars +nowarn_shadow_vars >> +warn_unused_import -o../ebin eunit_autoexport.erl >> /home/steven/eunit/src/../ebin/eunit_autoexport.bea#: error writing file >> make[1]: *** [../ebin/eunit_autoexport.beam] Error 1 >> make[1]: Leaving directory `/home/steven/eunit/src' >> make: *** [subdirs] Error 2 >> --- >> >> Should I just try again in a couple days? :) >> > > Did the git svn clone include the empty ebin directory, and if so, > did you check the write permissions? > > /Richard > From cureadvocate@REDACTED Wed Oct 21 22:24:19 2009 From: cureadvocate@REDACTED (Steven Edwards) Date: Wed, 21 Oct 2009 16:24:19 -0400 Subject: [erlang-questions] EUnit install fails In-Reply-To: <4ADF6BFD.5090308@gmail.com> References: <4ADF3D2C.5020305@gmail.com> <4ADF6BFD.5090308@gmail.com> Message-ID: Just got it working--I had to install erlang-dev for make to finish successfully. Thanks again! Steven On Wed, Oct 21, 2009 at 4:15 PM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > Steven Edwards wrote: > >> --- >> erlc -pa ../ebin -W -I../include +warn_unused_vars +nowarn_shadow_vars >> +warn_unused_import -o../ebin eunit_data.erl >> ./eunit_data.erl:29: can't find include lib "kernel/include/file.hrl" >> ./eunit_data.erl:337: record file_info undefined >> ./eunit_data.erl:611: record file_info undefined >> make[1]: *** [../ebin/eunit_data.beam] Error 1 >> make[1]: Leaving directory `/home/steven/eunit/src' >> make: *** [subdirs] Error 2 >> --- >> >> /usr/lib/erlang/lib contains kernel-2.12.5, but no kernel. ./kernel-2.12.5 >> lacks an include directory. find / -type f -name file.hrl returns nothing. >> > > That looks like a broken installation to me. > > /Richard > > PS. The include_lib directive strips the version number > from the actual path when it is searching for files, > so "kernel/include" matches kernel-2.12.5/include. > From ngocdaothanh@REDACTED Thu Oct 22 07:04:02 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Thu, 22 Oct 2009 14:04:02 +0900 Subject: [erlang-questions] MNesia Distribution Questions In-Reply-To: <4ADF3F8C.4030009@erlang-consulting.com> References: <930765BA-AFF7-4FE9-97BC-23CB16A6935C@erlang-consulting.com> <4ADF3F8C.4030009@erlang-consulting.com> Message-ID: <5c493e530910212204u2e831ffdg3bee8bf7e7637d72@mail.gmail.com> Mnesia has 4 GB (or 2 GB?) limit per node. Is there a tutorial or doc about how to create fragmented Mnesia DB, so that a very big DB can be cut into pieces and saved on many nodes? Thanks. On Thu, Oct 22, 2009 at 2:06 AM, Ulf Wiger wrote: > Rob Stewart wrote: >> >> 1. And example - I have a compex query to apply to a dataset, of, say >> 10,000,000 rows in an MNesia database. This database is replicated over 10 >> nodes in a network. Will the query be split for equal computation to each >> of >> the 10 nodes, or will the query be executed on either one random or one >> selected MNesia node. > > If it's one homogeneous table that is simply replicated across all nodes, > the query will be executed on the originating node only. > > If the table is fragmented, a subset of fragments will be identified > (if the whole key isn't bound, this will be all fragments), and the > query will be executed on all fragments in parallel. The resulting > sets will be merged, respecting the ordering semantics of the table > (i.e. sorted if it's an ordered_set table, otherwise not). > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ulf.wiger@REDACTED Thu Oct 22 08:15:15 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 22 Oct 2009 08:15:15 +0200 Subject: [erlang-questions] MNesia Distribution Questions In-Reply-To: <5c493e530910212204u2e831ffdg3bee8bf7e7637d72@mail.gmail.com> References: <930765BA-AFF7-4FE9-97BC-23CB16A6935C@erlang-consulting.com> <4ADF3F8C.4030009@erlang-consulting.com> <5c493e530910212204u2e831ffdg3bee8bf7e7637d72@mail.gmail.com> Message-ID: <4ADFF873.7010608@erlang-consulting.com> Ngoc Dao wrote: > Mnesia has 4 GB (or 2 GB?) limit per node. Is there a tutorial or doc > about how to create fragmented Mnesia DB, so that a very big DB can be > cut into pieces and saved on many nodes? Strictly speaking, Mnesia has no such limit. - The 32-bit VM can only address 4 GB of RAM - Normal linux kernels give the application 3 GB to use. You can get past this using a largekernel. - Dets can address 2GB This means that for 32-bit Erlang: - The total amount of data residing in mnesia ram_copies or disc_copies on a given node cannot exceed 3GB or 4GB, depending on the OS. - The size of a disc_only copies table fragment* cannot exceed 2GB. For 64-bit Erlang, the restriction on disc_only_copies remains, but RAM tables can be much, much bigger. However, most data types will take up twice as much space in 64-bit as in 32-bit. Binaries are one notable exception. * Tables can be fragmented using mnesia_frag. If not fragmented, the 'table fragment' will be the entire table. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From info@REDACTED Thu Oct 22 16:52:51 2009 From: info@REDACTED (info) Date: Thu, 22 Oct 2009 16:52:51 +0200 Subject: managing the tcp/ip data flow and internal messages Message-ID: <200910221652508410332@its3.ch> I'm using gen_server to manage several socket connections. For each connection a process is running, listen a client, reading with gen_tcp:recv loop(S)-> case gen_tcp:recv(S,0)-> {ok,Data} -> %% handle data loop(S); {error,closed} -> exit( normal) end. Now I would like to complicate the situation ... I would like, from another gen_server, to send data to a specific client ! My first problem is to find the process "attached" to a client. Probably I must associate an ident (the client) to each process. My second problem is to not disturb the tcp/ip connection. I suppose I must use {active,once} and the instruction "receive". loop(S)-> inet:setopts(S,[{active,once}]), receive {tcp,S,Data} -> %%handle data {tcp_close,S}-> %%handle close {tcp_error,S,Reason}-> %%handle reason {message,Data}-> gen_tcp:send(S,Data), ... end. Is my approach coherent ? robust ? yes/no why ? John From kamiseq@REDACTED Thu Oct 22 19:01:07 2009 From: kamiseq@REDACTED (kamiseq) Date: Thu, 22 Oct 2009 19:01:07 +0200 Subject: [erlang-questions] managing the tcp/ip data flow and internal messages In-Reply-To: <200910221652508410332@its3.ch> References: <200910221652508410332@its3.ch> Message-ID: ok, first of all, you can have only one owner process of the socket, so you need to have one process listening/sending both for socket messages and for erlang messages. so gen_tcp:receive (as it is blocking) is not enough, and yes you need to use receive and {active, once}. but now you will face problems such that you can have many messages from inside of the system that will result in delayed response to socket clients. and what if your server get killed then you loose all messages in queue. you should keep message queue as short as possible. hope that may help -- pozdrawiam Pawe? Kami?ski kamiseq@REDACTED pkaminski.prv@REDACTED ______________________ From clist@REDACTED Thu Oct 22 20:17:34 2009 From: clist@REDACTED (Angel Alvarez) Date: Thu, 22 Oct 2009 20:17:34 +0200 Subject: FYI: Rewriting Playdar: C++ to Erlang, massive savings Message-ID: <200910222017.34993.clist@uah.es> Another erlang success history... http://www.metabrew.com/article/rewriting-playdar-c-to-erlang-massive-savings/ interesting fact is people misses having a tiny erlang distributable. I think erlang is more than de VM alone, what's the size of a comparable java setup (including OTP-like functionality)? another problems arises from the concept of having one erlang setup for each package we install on the system. "...Packaging an Erlang app for the desktop involves taking the installed VM directory structure and stripping out all the docs, source and parts of the Erlang stdlib we don?t use, then packaging it along with the compiled Playdar code. CouchDB does something like this too, and RabbitMQ ships the Erlang VM without stripping unneeded libs..." Well, erlang installs should detect prior erlang installations and boot from those setups without duplicating erlang dists whenever oyu install a new erlang app. Fortunaley (IMHO) these guys think the same after all: "...We?ll put together a Windows installer soon that?ll probably be around the same size. A 10MB download isn?t so bad nowadays, and I expect we can optimize the packaging process some more. Linux users will get a package that depends on the erlang VM in their package manager. Seems like shipping Erlang apps to the desktop isn?t so hard after all." /angel -- Agua para todo? No, Agua para Todos. ->>----------------------------------------------- Clist UAH a.k.a Angel ---------------------------------[www.uah.es]-<<-- Los dinosaurios murieron porque no ten?an un programa espacial. Larry Niven. From antoine.koener@REDACTED Fri Oct 23 11:07:52 2009 From: antoine.koener@REDACTED (Antoine Koener) Date: Fri, 23 Oct 2009 11:07:52 +0200 Subject: [erlang-questions] FYI: Rewriting Playdar: C++ to Erlang, massive savings In-Reply-To: <200910222017.34993.clist@uah.es> References: <200910222017.34993.clist@uah.es> Message-ID: <1c89b3a10910230207k6d0a75e5tfcc61b6a82890f11@mail.gmail.com> > "...We?ll put together a Windows installer soon that?ll probably be around > the same size. A 10MB download > isn?t so bad nowadays, and I expect we can optimize the packaging process > some more. Linux users will get > a package that depends on the erlang VM in their package manager. Seems > like shipping Erlang apps to the > desktop isn?t so hard after all." > > Thanks, interesting read I'm now building a windows installer to deploy some erlang application, and here's problems that takes a lot of time to fix. Erlang for windows (R13B) needs some versions of the VC++ 2005 Runtime. (msvcr80.dll) Unfortunately the only way the find which one is to read the last part of the file "erl.exe" for example and extract information related to its "manifest". There you will find that you need the VC++ 2005-SP1 runtime. (And I think that with R12B this was not the case) The crypto_drv.dll needs the openssl "ssleay32.dll" to be installed. If it's not the case you won't be able to use anything related to crypto... And thanks to windows dll Hell problem, you need to put this dll in \windows\system32 [ SystemFolder ], making the package uninstallation not really "nice" because you may remove this file, breaking other apps... Erlsrv is not really efficient, since your service may no longer be in place but still in the erlsrv registry key, you won't be able to reinstall it. Windows XP comes with tools like 'sc' that is able to set and configure a service. Erlsrv with "cmd.exe loves to break your double quotes" is absolutely not convenient. (I know that there's a new version of it, I didn't tested it for the moment, so maybe this part is no longer valid) There's two "bin" directories, which one is the "one" ? Same question with erl.ini. erl.ini: should contains good directory names but you can't use the single '\' because your path will not be valid :( While using the NSIS installer version is nice to install erlang for testing purpose, massively deploying it, is not possible using what windows admins are used to i.e. group policy and .msi files. (providing silent or "unattended" deployments) I'm using makemsi[1] to build a .msi package to deploy my windows app, while difficult to use, it works almost flawlessly. I'll be glad to share my experience with anyone. Finally my point is that under windows Erlang is difficult to repackage. May be we should take time to make it completely autonomous ? ( including every dependencies needed ) Then you may say, what about dependencies upgrades ? ( openssl vulnerability or else ) I will say that I've no good answers to this question :p I'm not pushing for massive adoption of Erlang but I think that if anyone is able to repackage its erlang app for windows and making it full automatic deployment compatible, erlang and erlang based apps may spread a lot quicker... [1] http://dennisbareis.com/makemsi.htm From vinubalaji@REDACTED Fri Oct 23 11:45:41 2009 From: vinubalaji@REDACTED (Vinubalaji Gopal) Date: Fri, 23 Oct 2009 02:45:41 -0700 Subject: otp genserver and port program Message-ID: <7d8fb81e0910230245n293c680dv40a2fee73fe2a8e0@mail.gmail.com> Hi all, I wrote a port program based on the design principles of http://www.trapexit.org/Writing_an_Erlang_Port_using_OTP_Principles The only difference is my program is a linked in driver and the linked in driver connects to a C program. The C program returns a tuple response. It works fine if only one client connects to the server (I get a proper response). But in case multiple client connects to the server I get a bad match error and the gen_server terminates :(. The debug message is not that useful and googling didn't give me anything useful. I am using port_command to send the message to the linked in driver (which is part of the handle_call). Is there any way to debug a gen_server crashing (I don't get anything other than the badmatch which I don't understand) and would gen_server handle_call not handle multiple calls that are received simultaneously? It works fine if I remove the port_command and just return random tuples. The error I get looks something like this: {{badmatch,^M {'$gen_call',^M {<0.189.0>,#Ref<0.0.0.1057>},^M {my_function,^M Msg}}} I would really appreciate any kind of help or pointers - have been trying to debug this for a while and can't get much information :(. -- Vinu In a world without fences who needs Gates? From nem@REDACTED Fri Oct 23 12:46:52 2009 From: nem@REDACTED (Geoff Cant) Date: Fri, 23 Oct 2009 12:46:52 +0200 Subject: [erlang-questions] otp genserver and port program In-Reply-To: <7d8fb81e0910230245n293c680dv40a2fee73fe2a8e0@mail.gmail.com> (Vinubalaji Gopal's message of "Fri, 23 Oct 2009 02:45:41 -0700") References: <7d8fb81e0910230245n293c680dv40a2fee73fe2a8e0@mail.gmail.com> Message-ID: Vinubalaji Gopal writes: > Hi all, > > I wrote a port program based on the design principles of > http://www.trapexit.org/Writing_an_Erlang_Port_using_OTP_Principles > > The only difference is my program is a linked in driver and the linked > in driver connects to a C program. The C program returns a tuple > response. It works fine if only one client connects to the server (I > get a proper response). But in case multiple client connects to the > server I get a bad match error and the gen_server terminates :(. The > debug message is not that useful and googling didn't give me anything > useful. I am using port_command to send the message to the linked in > driver (which is part of the handle_call). Is there any way to debug a > gen_server crashing (I don't get anything other than the badmatch > which I don't understand) and would gen_server handle_call not handle > multiple calls that are received simultaneously? It works fine if I > remove the port_command and just return random tuples. > > The error I get looks something like this: > {{badmatch,^M > {'$gen_call',^M > {<0.189.0>,#Ref<0.0.0.1057>},^M > {my_function,^M > Msg}}} > > I would really appreciate any kind of help or pointers - have been > trying to debug this for a while and can't get much information :(. Hi there, I'm going to attempt a little psychic debugging, so don't be surprised if I get it all wrong. :) I'm guessing you're receiving a message somewhere in your code that you expect to come from the port or the driver. I'm also guessing that your receive statement has a pattern that will match any message (certainly enough to match {'$gen_call', _, _}). You're probably sending a message to the driver and then waiting for the response, only the first message in the gen_server message queue is another call request and your code mis-interprets this as the reply from the driver. Matching the first message from the queue against something else would give you this badmatch error. If I've got this right, you probably just need to change your receive statement to match only the right messages to fix this problem. Cheers, -- Geoff Cant ps: if the above was nonsense, then we probably need to see a bit more of your code - particularly the handle_call function from your gen_server and the bit that does a send/receive to the linked in driver. From rtrlists@REDACTED Fri Oct 23 14:59:26 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 23 Oct 2009 13:59:26 +0100 Subject: [erlang-questions] FYI: Rewriting Playdar: C++ to Erlang, massive savings In-Reply-To: <1c89b3a10910230207k6d0a75e5tfcc61b6a82890f11@mail.gmail.com> References: <200910222017.34993.clist@uah.es> <1c89b3a10910230207k6d0a75e5tfcc61b6a82890f11@mail.gmail.com> Message-ID: <6a3ae47e0910230559x3a4189c9w7a97d1604627757c@mail.gmail.com> Lovely summary of the little gotchas when trying to create your own standalone packaged Erlang app for Windows. Thank you for sending this, Antoine. Robby On 10/23/09, Antoine Koener wrote: >> "...We?ll put together a Windows installer soon that?ll probably be around >> the same size. A 10MB download >> isn?t so bad nowadays, and I expect we can optimize the packaging process >> some more. Linux users will get >> a package that depends on the erlang VM in their package manager. Seems >> like shipping Erlang apps to the >> desktop isn?t so hard after all." >> >> > Thanks, interesting read > > > I'm now building a windows installer to deploy some erlang application, and > here's problems that takes a lot of time > to fix. > > Erlang for windows (R13B) needs some versions of the VC++ 2005 Runtime. > (msvcr80.dll) > Unfortunately the only way the find which one is to read the last part of > the file "erl.exe" for example and extract information related to its > "manifest". > There you will find that you need the VC++ 2005-SP1 runtime. > (And I think that with R12B this was not the case) > > > The crypto_drv.dll needs the openssl "ssleay32.dll" to be installed. If it's > not the case you won't be able to use anything > related to crypto... And thanks to windows dll Hell problem, you need to put > this dll in \windows\system32 [ SystemFolder ], > making the package uninstallation not really "nice" because you may remove > this file, breaking other apps... > > Erlsrv is not really efficient, since your service may no longer be in > place but still in the erlsrv registry key, you won't be able > to reinstall it. Windows XP comes with tools like 'sc' that is able to set > and configure a service. > Erlsrv with "cmd.exe loves to break your double quotes" is absolutely not > convenient. > (I know that there's a new version of it, I didn't tested it for the moment, > so maybe this part is no longer valid) > > There's two "bin" directories, which one is the "one" ? > Same question with erl.ini. > erl.ini: should contains good directory names but you can't use the single > '\' because your path will not be valid :( > > > While using the NSIS installer version is nice to install erlang for testing > purpose, massively deploying it, is not possible using > what windows admins are used to i.e. group policy and .msi files. (providing > silent or "unattended" deployments) > > I'm using makemsi[1] to build a .msi package to deploy my windows app, while > difficult to use, it works almost flawlessly. > I'll be glad to share my experience with anyone. > > Finally my point is that under windows Erlang is difficult to repackage. > May be we should take time to make it completely autonomous ? ( including > every dependencies needed ) > Then you may say, what about dependencies upgrades ? ( openssl > vulnerability or else ) > I will say that I've no good answers to this question :p > > I'm not pushing for massive adoption of Erlang but I think that if anyone is > able to repackage its erlang app for windows and > making it full automatic deployment compatible, erlang and erlang based apps > may spread a lot quicker... > > > > [1] http://dennisbareis.com/makemsi.htm > From james.hague@REDACTED Fri Oct 23 15:33:12 2009 From: james.hague@REDACTED (James Hague) Date: Fri, 23 Oct 2009 08:33:12 -0500 Subject: [erlang-questions] FYI: Rewriting Playdar: C++ to Erlang, massive savings In-Reply-To: <6a3ae47e0910230559x3a4189c9w7a97d1604627757c@mail.gmail.com> References: <200910222017.34993.clist@uah.es> <1c89b3a10910230207k6d0a75e5tfcc61b6a82890f11@mail.gmail.com> <6a3ae47e0910230559x3a4189c9w7a97d1604627757c@mail.gmail.com> Message-ID: It would also be good to completely avoid any use of the registry. The bit that is used right now is not needed at all, and Microsoft recommends not using the registry for application settings. werl.exe should use the proper location in the "Users" folder (under Vista--different location under XP), and there should be an option to not save settings at all. James From raimo+erlang-questions@REDACTED Fri Oct 23 15:56:03 2009 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 23 Oct 2009 15:56:03 +0200 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <20091021122804.GA4311@erix.ericsson.se> References: <20091021083803.GA32147@erix.ericsson.se> <200910211157176241641@its3.ch> <20091021122804.GA4311@erix.ericsson.se> Message-ID: <20091023135603.GA31042@erix.ericsson.se> After a week of bug hunting we can confirm this is really Microsoft's bug, but we have a workaround. (It may be Microsoft's and Comodo's bug, who knows...) It turns out if you mix overlapped I/O with non-overlapped (asyncronous with syncronous) on different pipes, in this case the pipe to inet_gethost.exe from erlang was opened by erlang as non-overlapped and inet_gethost.exe itself used overlapped on all its other sockets and pipes... ...alright if you mix ... I/O something gets not initialized in Microsoft's libs unless you do a gethostname() call before going into recv() of commands from erlang (we already found this out the hard way) but now also if you had certain firewalls installed it was not enough, you had to do gethostbyname() calls with longer domain names to work around Microsoft's bug. Since this felt like a way to shaky workaround we found another that seems much better: now erlang can open the pipe to this specific port program (inet_gethost.exe) for overlapped I/O so inet_gethost.exe now uses overlapped I/O only. This seems to avoid the bug. We reproduced the name lookup freeze with Comodo 2.4.19.185 and Microsoft's Client for ISA Server 4.0. Here is a snapshot build (without documentation) that does not freeze for neither of these firewalls. http://www.erlang.org/download/snapshots/otp_win32_R13B03_gethost_fix.exe Try it and see if it solves your problem. / Raimo Niskanen On Wed, Oct 21, 2009 at 02:28:04PM +0200, Raimo Niskanen wrote: > On Wed, Oct 21, 2009 at 11:57:18AM +0200, info wrote: > > Dear Raimo, > > This night ... I am a little bit tired ... I tested according to your plan. I found again in my archive the comodo exe. Ouf ! > > This is the result: > > Test #1 > > uninstall Comodo > > restart > > inet:gethostbyname ok > > uninstall erlang > > install Comodo > > install erlang with windows runtime > > inet:gethostbyname ko > > Test #2 > > uninstall erlang > > uninstall Commodo > > restart > > install erlang with windows runtime > > inet:gethostbyname ok > > install Comodo > > restart > > inet:gethostbyname ko > > And with "ko" you mean "knock out", I guess... > > > > > On Tue, Oct 20, 2009 at 06:45:25PM +0200, info wrote: > > > > > > On Tue, 20 Oct 2009 11:33:35 +0200 info wrote: > > > > I uninstalled Comodo and ... the problem disappeared !!!!!!!!!!!!!!!!!!!!! > > > > What else now ? > > > > John > > > > > > Then we know it is firewall related. And we could not > > > reproduce it with Comodo. Some more tiny detail seems > > > to be needed to trigger the bug. Which version of Comodo > > > do you have? > > > > > > I had the version 2.4.19.185 .. > > > I had because I didn't keep the exe :-( and the version 3 is not compatible with windows 2003 > > > I am "naked" since yesterday ! > > > > We tried Comodo Firewall 3.12, and it seems to work just fine. > > I.e on Windows 2003 R2 Enterprise Edition with SP2. > > The features of the product give a no support of windows 2003 !? > > I have windows 2003 sbs with sp2 > > I read in the Comodo forums that up to at least 3.10 it refused > to install saying "OS not supported". If so you could suspect > 3.12 kind of supports Windows 2003... > > I have located 2.4.19.184 on the Internet, we will > try it and hope the difference to 2.4.19.185 does > not matter... > > > > > > > > > Reinstall and see if the problem appears again, but first > > > uninstall Erlang. Installation order may matter to trigger > > > the bug. Uninstall both Erlang and Comodo and try > > > installing Comodo first then Erlang. If that fails > > > uninstall Erlang then Comodo (latest first), and install Erlang > > > first then Comodo (the other order). Watch out the next > > > paragraph about installing Erlang. > > > > > > On Tue, 20 Oct 2009 11:33:52 +0200 info wrote: > > > > OK, I found several versions in WinSxS. > > > > Which one is the good one ? > > > > Shall I copy it in system32 ? > > > > > > No, uninstall Erlang, and reinstall it. Make sure to > > > select the approx."Install Microsoft Runtime" in the > > > installer. The installer installs it into a for > > > Erlang unique location so Erlang will run exactly > > > the runtime library it needs. Do not move it! > > > > > > > > > We are now hunting something fishy (for Robert R) with > > > IIS server and firewall client. Can you (info) tell us > > > what happens when you have the symptoms, if you in > > > a freshly started Erlang node only look up a > > > short name e.g "localhost", because we now have > > > a situation where it kind of works until we do > > > a lookup containing a dot which hangs and after > > > that everything hangs. Can you confirm this peculiar > > > behaviour? > > > > > > / Raimo > > > > > > > > > > > > On Mon, Oct 19, 2009 at 11:53:07PM +0200, info wrote: > > > > And what about msvcr80.dll which is missing in my server ? > > > > > > > > On Sat, Oct 17, 2009 at 12:46:50AM +0100, info wrote: > > > > > On Fri, Oct 16, 2009 at 04:29:20PM +0200, info wrote: > > > > > I guess Robby wrote: > > > > > > On Thu, Oct 15, 2009 at 4:28 PM, Raimo Niskanen < > > > > > > raimo+erlang-questions@REDACTED > > > > > > > wrote: > > > > > > > > > > > > > On Thu, Oct 15, 2009 at 03:02:40PM +0200, info wrote: > > > > > > > > Dear Raimo, > > > > > > > > I can resume: I discovered that gethostbyname call inet_gethost.c > > > > > > > > I don't understand inet_gethost.c :-( > > > > > > > > Where "goes" this problem for finding the information ? > > > > > > > > If we know where and what, we could perhaps find what is missing in my > > > > > > > windows 2003 ... > > > > > > > > Hope to read you soon ! > > > > > > > > John > > > > > > > > > > > > > > Well, there is not much more to say. inet_gethost.c calls > > > > > > > struct hostent *gethostbyname(const char *name) in the > > > > > > > winsock2 library: > > > > > > > http://msdn.microsoft.com/en-us/library/ms738524(VS.85).aspx > > > > > > > It never returns. > > > > > > > It is deprecated but that does not mean broken. > > > > > > > I have a Windows 2003 server that it works on. > > > > > > > We have not found the reason it differs between > > > > > > > my server and your server. > > > > > > > And that is about it. > > > > > > > > > > > > > > > > > > > > > > > > > > > I can consistently reproduce the hanging gethostbyname() in inet_gethost.exe > > > > > > on Windows 2003 when I have the 'Microsoft Firewall Client for ISA Server > > > > > > Version 4.0' installed. This is a specific add-on, nothing to do with > > > > > > Microsoft Firewall in your Network Properties. > > > > > > > > > > > > > > > > I guess info wrote: > > > > > > but I don't use ISA server !!! > > > > > > > > > > > > > > > > No other interesting firewall software? > > > > > Yes: I am using Comodo. > > > > > > > > Can you try to uninstall Comodo and see if the problem disappears. > > > > If so we know where to look for the problem. > > > > (We have installed a 2003 server with Comodo and got no problems, > > > > other than the possibility to do configuration errors that > > > > would cause your symptoms) > > > > > > > > > > > > > > I guess Robby wrote: > > > > > > If I remove it, it works fine, if I add it it hangs. Apart from that the > > > > > > W2003 build is as vanilla as it gets. > > > > > > > > > > > > Nothing much to go on, but I would not be surprised if some system DLL is > > > > > > getting in the way. This is very hard to diagnose. > > > > > > > > > > > > One day I'll get around to compiling inet_gethost.exe in my setup, just to > > > > > > rule out poor linkage due to the VC++ setup used to build the shipping > > > > > > executables. If I compile up a trivial gethostbyname() example from MSDN, it > > > > > > works OK in all circumstances. > > > > > > > > > > > > There are a few strange dependencies in other shipped Windows files, for > > > > > > example beam.dll seems to depend on a particular MSVCR80.DLL with the > > > > > > version 8.0.50727.1433 . And if you have an older Windows machine, you get > > > > > > the completely unhelpful popup box telling you that your program could not > > > > > > be run. This has something to do with Windows Side-by-Side DLL deployment, > > > > > > and I completely fail to grasp what that's all about. > > > > > > > > > > > > > > > > I guess info wrote: > > > > > > but I don't have msvcr80.dll !!! > > > > > > > > > > I guess you do, actually, since Erlang/OTP's installer is supposed > > > > > to install that DLL, if it does not exist already. > > > > > > > > > > I guess Robby wrote: > > > > > > > > > > > > Robby > > > > > > > > > > -- > > > > > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > > > > > -- > > > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From vinubalaji@REDACTED Fri Oct 23 20:43:47 2009 From: vinubalaji@REDACTED (Vinubalaji Gopal) Date: Fri, 23 Oct 2009 11:43:47 -0700 Subject: [erlang-questions] otp genserver and port program In-Reply-To: References: <7d8fb81e0910230245n293c680dv40a2fee73fe2a8e0@mail.gmail.com> Message-ID: <7d8fb81e0910231143s7c037c27n42ade84d8b8912d4@mail.gmail.com> Hi Geoff, > I'm guessing you're receiving a message somewhere in your code that you > expect to come from the port or the driver. I'm also guessing that > your receive statement has a pattern that will match any message > (certainly enough to match {'$gen_call', _, _}). > Great that was the problem - I had a handle_call..... -> port_command(....) Receive Data -> %do a match on the data here in the handle_call and this was what was matching the {'$gen_server',... . I had no idea that the gen_server messages will be matched by this receive. Thanks a lot for clarifying that. -- Vinu From roger@REDACTED Fri Oct 23 23:39:46 2009 From: roger@REDACTED (Roger Price) Date: Fri, 23 Oct 2009 23:39:46 +0200 (CEST) Subject: Installing R13B02-1: Questions on curses and gcc Message-ID: Hi, Sorry if these are FAQs. I am trying to install R13B02-1 on opensuse 11.0. The following curses libraries are installed # locate ncurses /lib/libncurses.so.5 /lib/libncurses.so.5.6 /lib/libncursesw.so.5 /lib/libncursesw.so.5.6 /lib64/libncurses.so.5 /lib64/libncurses.so.5.6 /lib64/libncursesw.so.5 /lib64/libncursesw.so.5.6 but when I try to configure with the command script -c "./configure" configure.log the log ends with checking for kstat_open in -lkstat... no checking for tgetent in -lncurses... no checking for tgetent in -lcurses... no checking for tgetent in -ltermcap... no checking for tgetent in -ltermlib... no configure: error: No curses library functions found Question 1: Which curses library do I need for R13B02-1? I tried configuring without termcap: script -c "./configure --without-termcap" configure.log but now the log ends: checking size of long long... 8 checking if we should add -fno-tree-copyrename to CFLAGS for computed gotos to work properly... yes checking for broken gcc-4.3.0 compiler... yes configure: error: This gcc miscompiles the Erlang runtime system; please use a different version The README says "gcc-4.3.0 has a serious optimizer bug. It produces an Erlang emulator that will crash immediately. The bug is supposed to be fixed in gcc-4.3.1." My gcc is at version 4.3.1: # gcc --version gcc (SUSE Linux) 4.3.1 20080507 (prerelease) [gcc-4_3-branch revision 135036] Question2: Which gcc do I need to compile R13B02-1? Roger From erlangy@REDACTED Sat Oct 24 01:17:05 2009 From: erlangy@REDACTED (Michael McDaniel) Date: Fri, 23 Oct 2009 16:17:05 -0700 Subject: [erlang-questions] Installing R13B02-1: Questions on curses and gcc In-Reply-To: References: Message-ID: <20091023231705.GA3633@delora.autosys.us> On Fri, Oct 23, 2009 at 11:39:46PM +0200, Roger Price wrote: > Hi, Sorry if these are FAQs. I am trying to install R13B02-1 on opensuse > 11.0. > > The following curses libraries are installed > # locate ncurses > /lib/libncurses.so.5 > /lib/libncurses.so.5.6 > /lib/libncursesw.so.5 > /lib/libncursesw.so.5.6 > /lib64/libncurses.so.5 > /lib64/libncurses.so.5.6 > /lib64/libncursesw.so.5 > /lib64/libncursesw.so.5.6 > > but when I try to configure with the command > > script -c "./configure" configure.log > > the log ends with checking for kstat_open in -lkstat... no > checking for tgetent in -lncurses... no > checking for tgetent in -lcurses... no > checking for tgetent in -ltermcap... no > checking for tgetent in -ltermlib... no > configure: error: No curses library functions found > > Question 1: Which curses library do I need for R13B02-1? > > I tried configuring without termcap: > > script -c "./configure --without-termcap" configure.log > > but now the log ends: > > checking size of long long... 8 > checking if we should add -fno-tree-copyrename to CFLAGS for computed > gotos to work properly... yes > checking for broken gcc-4.3.0 compiler... yes > configure: error: This gcc miscompiles the Erlang runtime system; > please use a different version > > The README says "gcc-4.3.0 has a serious optimizer bug. It produces an > Erlang emulator that will crash immediately. The bug is supposed to be > fixed in gcc-4.3.1." > > My gcc is at version 4.3.1: > > # gcc --version > gcc (SUSE Linux) 4.3.1 20080507 (prerelease) [gcc-4_3-branch revision 135036] > > Question2: Which gcc do I need to compile R13B02-1? > > Roger > ________________________________________________________________ Perhaps you need to install libncurses5-dev Don't know about which gcc is required for R13B02-1 but for R13B01 I used $ gcc --version gcc (Ubuntu 4.3.3-5ubuntu4) 4.3.3 and it worked fine. ~Michael From wde@REDACTED Sat Oct 24 01:52:38 2009 From: wde@REDACTED (wde) Date: Sat, 24 Oct 2009 00:52:38 +0100 Subject: [erlang-questions] MNesia Distribution Questions Message-ID: <20091023225241.E2B7F80000A7@mwinf2819.orange.fr> Hello , I found an old post ( 3 years ago) with this comment : " Two major problems with dets as of today (and of yeasterday) 1. 64 bit indexing - really easy to fix 2. repair time - this is a bit harder to fix, but one solution could be along the following lines. a) keep the index and the freelist in a different file, b) have a file soley for the data. This file could be as easily repaired as a disklog file. One could just chunk through it, one term at a time, this way even large files will be fast to repair. c) when an object is put on the freelist, it needs also to be overwriten " What about dets implementation ? do you consider that there are always the major problems of dets implementation ? In more general, concerning Mnesia (dets include), on which part of the code we could work to improve this application ? Thank you ======= le 22/10/2009, 08:15:15 vous ?criviez: ======= >Ngoc Dao wrote: >> Mnesia has 4 GB (or 2 GB?) limit per node. Is there a tutorial or doc >> about how to create fragmented Mnesia DB, so that a very big DB can be >> cut into pieces and saved on many nodes? > >Strictly speaking, Mnesia has no such limit. > >- The 32-bit VM can only address 4 GB of RAM > >- Normal linux kernels give the application 3 GB to use. > You can get past this using a largekernel. > >- Dets can address 2GB > > >This means that for 32-bit Erlang: >- The total amount of data residing in mnesia > ram_copies or disc_copies on a given node cannot > exceed 3GB or 4GB, depending on the OS. > >- The size of a disc_only copies table fragment* cannot > exceed 2GB. > >For 64-bit Erlang, the restriction on disc_only_copies >remains, but RAM tables can be much, much bigger. >However, most data types will take up twice as much space >in 64-bit as in 32-bit. Binaries are one notable exception. > >* Tables can be fragmented using mnesia_frag. If not fragmented, >the 'table fragment' will be the entire table. > >BR, >Ulf W >-- >Ulf Wiger >CTO, Erlang Training & Consulting Ltd >http://www.erlang-consulting.com > >________________________________________________________________ >erlang-questions mailing list. See http://www.erlang.org/faq.html >erlang-questions (at) erlang.org > > = = = = = = = = = ========= = = = = = = = = = = wde wde@REDACTED 24/10/2009 From mikpe@REDACTED Sat Oct 24 12:24:08 2009 From: mikpe@REDACTED (Mikael Pettersson) Date: Sat, 24 Oct 2009 12:24:08 +0200 Subject: [erlang-questions] Installing R13B02-1: Questions on curses and gcc In-Reply-To: References: Message-ID: <19170.54728.631513.385505@pilspetsen.it.uu.se> Roger Price writes: > Hi, Sorry if these are FAQs. I am trying to install R13B02-1 on > opensuse 11.0. > > The following curses libraries are installed > # locate ncurses > /lib/libncurses.so.5 > /lib/libncurses.so.5.6 > /lib/libncursesw.so.5 > /lib/libncursesw.so.5.6 > /lib64/libncurses.so.5 > /lib64/libncurses.so.5.6 > /lib64/libncursesw.so.5 > /lib64/libncursesw.so.5.6 These are ncurses runtime libraries. > but when I try to configure with the command > > script -c "./configure" configure.log > > the log ends with > checking for kstat_open in -lkstat... no > checking for tgetent in -lncurses... no > checking for tgetent in -lcurses... no > checking for tgetent in -ltermcap... no > checking for tgetent in -ltermlib... no > configure: error: No curses library functions found > > Question 1: Which curses library do I need for R13B02-1? You need the ncurses development packages, which contain include files and sometimes additional library files. They would typically be called something like ncurses-dev or -devel. > I tried configuring without termcap: > > script -c "./configure --without-termcap" configure.log > > but now the log ends: > > checking size of long long... 8 > checking if we should add -fno-tree-copyrename to CFLAGS for > computed gotos to work properly... yes > checking for broken gcc-4.3.0 compiler... yes > configure: error: This gcc miscompiles the Erlang runtime system; > please use a different version > > The README says "gcc-4.3.0 has a serious optimizer bug. It produces > an Erlang emulator that will crash immediately. The bug is supposed > to be fixed in gcc-4.3.1." > > My gcc is at version 4.3.1: > > # gcc --version > gcc (SUSE Linux) 4.3.1 20080507 (prerelease) [gcc-4_3-branch revision 135036] Note that it says "prerelease". The bug in question (PR tree-optimization/36339) was fixed on the gcc-4_3-branch 2008-05-28 in revision 136101. You need a gcc update. > Question2: Which gcc do I need to compile R13B02-1? Most gcc versions work, so use whatever your Linux vendor tests and ships. If they ship broken crap like gcc-4.3.0 and don't offer updates, use gcc-4.3.4 or gcc-4.4.2. From yan.beijing.china@REDACTED Sat Oct 24 13:07:19 2009 From: yan.beijing.china@REDACTED (Yan Yan) Date: Sat, 24 Oct 2009 19:07:19 +0800 Subject: Erlang's process not killed by exit signal reasoned by "kill" Message-ID: <200910241907167657586@gmail.com> I am extremely interested in Erlang. However, there is a quite confusing question in Joe's textbook "Programming Erlang, Software for a Concurrent World". I cannot figure it out so I am now sending you an email to talk about it. (1)On Page 163, Quote If the reason is given as kill, then an untrappable exit signal will be sent. An untrappable exit signal will always kill the process it is sent to, even if it is a system process. End quote (2)On Page 169, Quote: 8> edemo1:start(true, {die,kill}). Process b received {'EXIT',<0.73.0>,kill} process b (<0.72.0>) is alive process c (<0.73.0>) is dead ok End quote. Here a and b are both system processes, while c is not. When c exits with the reason "kill" (not "killed"), it sends exit signal to b with the reason "kill". Therefore b should be killed and dead, but b is still alive here! (I had thought it was only a small typo in the book. But then I tested by myself and got the same result: b received the exit signal with the reason "kill" and b was not killed but alive.) Why is the system process b not killed by the exit signal with the reason "kill"? P.S. interesting is that if the code exit(kill) in process c is changed to exit(B, kill), b is killed by that signal, which is not conflict with quote on Page 163. I have also tested that both exit(kill) and exit(B, kill) send the same exit signal ({'EXIT', c's Pid, kill}) to B. Thanks a lot. It will be my honor if anyone can give me a reply and I am looking forward to it. Sincerely, Yan Yan From yan.beijing.china@REDACTED Sat Oct 24 13:22:28 2009 From: yan.beijing.china@REDACTED (Yan Yan) Date: Sat, 24 Oct 2009 19:22:28 +0800 Subject: Erlang's process not killed by the exit signal reasoned by kill Message-ID: <200910241922256251535@gmail.com> I am extremely interested in Erlang. However, there is a quite confusing question in Joe's textbook "Programming Erlang, Software for a Concurrent World". I cannot figure it out so I am now sending you an email to talk about it. (1)On Page 163, Quote If the reason is given as kill, then an untrappable exit signal will be sent. An untrappable exit signal will always kill the process it is sent to, even if it is a system process. End quote (2)On Page 169, Quote: 8> edemo1:start(true, {die,kill}). Process b received {'EXIT',<0.73.0>,kill} process b (<0.72.0>) is alive process c (<0.73.0>) is dead ok End quote. Here a and b are both system processes, while c is not. When c exits with the reason "kill" (not "killed"), it sends exit signal to b with the reason "kill". Therefore b should be killed and dead, but b is still alive here! (I had thought it was only a small typo in the book. But then I tested by myself and got the same result: b received the exit signal with the reason "kill" and b was not killed but alive.) Why is the system process b not killed by the exit signal with the reason "kill"? P.S. interesting is that if the code exit(kill) in process c is changed to exit(B, kill), b is killed by that signal, which is not conflict with quote on Page 163. I have also tested that both exit(kill) and exit(B, kill) send the same exit signal ({'EXIT', c's Pid, kill}) to B. Thanks a lot. It will be my honor if anyone can give me a reply and I am looking forward to it. Sincerely, Yan Yan From roger@REDACTED Sat Oct 24 13:54:40 2009 From: roger@REDACTED (Roger Price) Date: Sat, 24 Oct 2009 13:54:40 +0200 (CEST) Subject: [erlang-questions] Installing R13B02-1: Questions on curses and gcc In-Reply-To: <19170.54728.631513.385505@pilspetsen.it.uu.se> References: <19170.54728.631513.385505@pilspetsen.it.uu.se> Message-ID: Thanks Michael, Thanks Mikael, I installed ncurses-devel and used broken gcc 4.3.1 to compile gcc 4.3.2 which is acceptable to R13B02-1 configure. Roger On Sat, 24 Oct 2009, Mikael Pettersson wrote: > You need a gcc update. > > Question2: Which gcc do I need to compile R13B02-1? _______________ By the way, is top-posting acceptable in this list? From yan.beijing.china@REDACTED Sat Oct 24 17:42:24 2009 From: yan.beijing.china@REDACTED (=?utf-8?B?WWFuIFlhbg==?=) Date: Sat, 24 Oct 2009 23:42:24 +0800 Subject: =?utf-8?B?UmU6IFJlOiBbZXJsYW5nLXF1ZXN0aW9uc10gRXJsYW5nJ3MgcHJvY2VzcyBub3Qga2lsbGVkIGJ5IGV4aXQgc2lnbmFscmVhc29uZWQgYnkgImtpbGwi?= References: <200910241907167657586@gmail.com> Message-ID: <200910242342216873502@gmail.com> Thanks, Richard In fact, I had a very similiar thought with yours as I found b killed when b is not a system process but b not killed when b is a system process. However, even if exit(kill) acts the same as exit(self(), kill) does, there seems to be problems left. For example, if, in this setup, c does want ALL linked processes to be killed when c itself exits abnormally, we will want a simple code in c() just like exit(kill). In this way, we want all processes linked to c will be killed because of c's exit, even if there were any system processes among c's linked processes. Currently, if we write exit(kill) or exit(self(), kill) in c(), only c and c's linked non-system processes will be killed, but all system processed linked will survive, which is not correct. So far, there seems to be no other ways to let it happen correctly except: Code: c(......) -> blabla... %% P1, P2, ..., Pn are linked SYSTEM processes exit(P1, kill), exit(P2, kill), ... exit(Pn, kill), %%Here c is still alive, and we want c and its linked non-system processes also killed, so... exit(kill). %% or exit(self(), kill), or exit(anything).. End code. However, it still demands that programmers know ahead of time how many and which system processes are linked to c, which is not a good design pattern. Sincerely, Yan Yan Yan Yan 2009-10-24 From: Richard Carlsson Time: 2009-10-24 21:05:05 To: Yan Yan Fw: erlang-questions; erlang-bugs; Bj?n_Gustavsson Subject: Re: [erlang-questions] Erlang's process not killed by exit signalreasoned by "kill" Yan Yan wrote: > (2)On Page 169, (Page 161 in my copy of the book.) > Quote: > > 8> edemo1:start(true, {die,kill}). Process b received > {'EXIT',<0.73.0>,kill} process b (<0.72.0>) is alive process c > (<0.73.0>) is dead ok > > End quote. > > Here a and b are both system processes, while c is not. When c exits > with the reason "kill" (not "killed"), it sends exit signal to b with > the reason "kill". Therefore b should be killed and dead, but b is > still alive here! > > (I had thought it was only a small typo in the book. But then I > tested by myself and got the same result: b received the exit signal > with the reason "kill" and b was not killed but alive.) > > Why is the system process b not killed by the exit signal with the > reason "kill"? Interesting. There seems to be a difference in behaviour (probably not intentional) between `exit(kill)' and `exit(self(),kill)'. A summary of the setup: we have three processes, linked in a chain. 'a' is always trapping, 'b' may or may not be trapping, and 'c' is the one who dies by calling exit(Reason). 1. When 'b' is not trapping, and 'c' does exit(kill), we see a report from 'a' that 'b' died with reason 'killed'. 2. When 'b' is trapping, and 'c' does exit(kill), 'b' survives and reports that it sees the exit reason 'kill' from 'c' (not 'killed'). The question is: in case 2, why didn't 'b' die, when it apparently got a 'kill' message. This should be untrappable, which is why it is changed to 'killed' when it is propagated. But it was 'c' who died, so why wasn't the atom changed to 'killed'? If we change `exit(kill)' to `exit(self(),kill)' in 'c', we get the effect we expected: 'b' survives and reports that it sees 'killed' as the exit reason from 'c'. Then, a new question is why 'a' saw 'killed' in case 1 when 'b' is non-trapping. If 'c' dies in the same way in both, then doesn't 'b' get the same signal from 'c'? Apparently, it does, but since 'b' is not trapping, it doesn't matter what the atom is as long as it is something else than 'normal'. So 'b' dies due to an incoming 'kill', and this is then propagated as 'killed'. It seems that when a process does exit(kill) on itself, it causes a different outgoing signal than if it does exit(self(),kill). The former is not untrappable even though it has the reason 'kill', but if that in its turn causes another process to die, the atom will _then_ be rewritten to 'killed'. To me, it seems that this should be fixed so that exit(kill), even if it's an unusual case, should be propagated as 'killed'. /Richard From ajuttner.list@REDACTED Sat Oct 24 17:53:44 2009 From: ajuttner.list@REDACTED (Alpar Juttner) Date: Sat, 24 Oct 2009 17:53:44 +0200 Subject: systools:make_relup w/ vanilla erlang system Message-ID: <1256399624.5664.23.camel@piko.site> Hi, If I have an .appup file referring to other versions of the application, then systools:make_relup needs the corresponding .app files, too. Therefore those versions of the application should be installed to the erlang runtime system. Is is possible to directly specify those .app files for make_relup instead, so that I can use my vanilla erlang system for creating the relup file? Regards, Alpar From yan.beijing.china@REDACTED Sat Oct 24 18:00:27 2009 From: yan.beijing.china@REDACTED (=?utf-8?B?WWFuIFlhbg==?=) Date: Sun, 25 Oct 2009 00:00:27 +0800 Subject: =?utf-8?B?UmU6IFtlcmxhbmctcXVlc3Rpb25zXSBFcmxhbmcncyBwcm9jZXNzIG5vdCBraWxsZWQgYnkgZXhpdCBzaWduYWxyZWFzb25lZCBieSAia2lsbCI=?= References: <200910241907167657586@gmail.com> Message-ID: <200910250000242341407@gmail.com> Thanks, Richard In fact, I had a very similiar thought with yours as I found b killed when b is not a system process but b not killed when b is a system process. However, even if exit(kill) acts the same as exit(self(), kill) does, there seems to be problems left. For example, if, in this setup, c does want ALL linked processes to be killed when c itself exits abnormally, we will want a simple code in c() just like exit(kill). In this way, we want all processes linked to c will be killed because of c's exit, even if there were any system processes among c's linked processes. Currently, if we write exit(kill) or exit(self(), kill) in c(), only c and c's linked non-system processes will be killed, but all system processed linked will survive, which is not correct. So far, there seems to be no other ways to let it happen correctly except: Code: c(......) -> blabla... %% P1, P2, ..., Pn are linked SYSTEM processes exit(P1, kill), exit(P2, kill), ... exit(Pn, kill), %%Here c is still alive, and we want c and its linked non-system processes also killed, so... exit(kill). %% or exit(self(), kill), or exit(anything).. End code. However, it still demands that programmers know ahead of time how many and which system processes are linked to c, which is not a good design pattern. Sincerely, Yan Yan Yan Yan 2009-10-24 From: Richard Carlsson Time: 2009-10-24 21:05:05 To: Yan Yan Fw: erlang-questions; erlang-bugs; Bj?n_Gustavsson Subject: Re: [erlang-questions] Erlang's process not killed by exit signalreasoned by "kill" Yan Yan wrote: > (2)On Page 169, (Page 161 in my copy of the book.) > Quote: > > 8> edemo1:start(true, {die,kill}). Process b received > {'EXIT',<0.73.0>,kill} process b (<0.72.0>) is alive process c > (<0.73.0>) is dead ok > > End quote. > > Here a and b are both system processes, while c is not. When c exits > with the reason "kill" (not "killed"), it sends exit signal to b with > the reason "kill". Therefore b should be killed and dead, but b is > still alive here! > > (I had thought it was only a small typo in the book. But then I > tested by myself and got the same result: b received the exit signal > with the reason "kill" and b was not killed but alive.) > > Why is the system process b not killed by the exit signal with the > reason "kill"? Interesting. There seems to be a difference in behaviour (probably not intentional) between `exit(kill)' and `exit(self(),kill)'. A summary of the setup: we have three processes, linked in a chain. 'a' is always trapping, 'b' may or may not be trapping, and 'c' is the one who dies by calling exit(Reason). 1. When 'b' is not trapping, and 'c' does exit(kill), we see a report from 'a' that 'b' died with reason 'killed'. 2. When 'b' is trapping, and 'c' does exit(kill), 'b' survives and reports that it sees the exit reason 'kill' from 'c' (not 'killed'). The question is: in case 2, why didn't 'b' die, when it apparently got a 'kill' message. This should be untrappable, which is why it is changed to 'killed' when it is propagated. But it was 'c' who died, so why wasn't the atom changed to 'killed'? If we change `exit(kill)' to `exit(self(),kill)' in 'c', we get the effect we expected: 'b' survives and reports that it sees 'killed' as the exit reason from 'c'. Then, a new question is why 'a' saw 'killed' in case 1 when 'b' is non-trapping. If 'c' dies in the same way in both, then doesn't 'b' get the same signal from 'c'? Apparently, it does, but since 'b' is not trapping, it doesn't matter what the atom is as long as it is something else than 'normal'. So 'b' dies due to an incoming 'kill', and this is then propagated as 'killed'. It seems that when a process does exit(kill) on itself, it causes a different outgoing signal than if it does exit(self(),kill). The former is not untrappable even though it has the reason 'kill', but if that in its turn causes another process to die, the atom will _then_ be rewritten to 'killed'. To me, it seems that this should be fixed so that exit(kill), even if it's an unusual case, should be propagated as 'killed'. /Richard From rvirding@REDACTED Sat Oct 24 22:10:55 2009 From: rvirding@REDACTED (Robert Virding) Date: Sat, 24 Oct 2009 22:10:55 +0200 Subject: [erlang-questions] Erlang's process not killed by exit signalreasoned by "kill" In-Reply-To: <200910242342216873502@gmail.com> References: <200910241907167657586@gmail.com> <200910242342216873502@gmail.com> Message-ID: <3dbc6d1c0910241310p53a30c23m58ba263d271f99fe@mail.gmail.com> I just want to clear up one little point: there is in fact only one type of process, all processes behave in the same way with respect to dying and exit signals. So what Joe calls "system processes" are just processes started by the system, nothing more. That exit(kill) and exit(Self, kill) behave differently is *exactly* as it should be, they are *defined* to behave differently. exit/2 will *always* send an exit signal to the process even if the process itself! Its behaviour is consistent. When sending to itself the process should behave as if the exit signal came from "outside" the itself. This means that if you are trapping exits you should also trap the exits from exit/2 to yourself. Oth an exit/1 works "internally". It should behave in the same way as erlang:error/1. In fact originally they were the same and there was only exit/1. Internal errors also looked the same as exit/1. Things started to drift apart when internal errors got stack traces. Now stack traces are a Good Thing but it meant that the behaviour of errors and exit/1 diverged. Unfortunately, instead of fixing exit/1, which would have been the most sensible thing to do, the split was made permanent by adding error/1. So exit/1 and error/1 should behave in the same way, which is different to exit/2. 'kill' is a little special as we felt we needed something which wasn't trappable but at the same time we realised that if it spread in the same way as other exit signals then it would be uncontrollable. This is why it was decided that when a process received an exit 'kill' signal it should always unconditionally die but only resignal 'killed' to its linked processes. Whether doing exit(kill) should send a real exit 'kill' signal or only a 'killed' signal is an interesting question which I can't remember now what we decided. If it sends an exit 'kill' signal then it should behave as a real 'kill' signal and be non-trappable. This definitely something for the Erlang Rationale! Robert 2009/10/24 Yan Yan > Thanks, Richard > > In fact, I had a very similiar thought with yours as I found b killed when > b is not a system process but b not killed when b is a system process. > > However, even if exit(kill) acts the same as exit(self(), kill) does, there > seems to be problems left. For example, if, in this setup, c does want ALL > linked processes to be killed when c itself exits abnormally, we will want a > simple code in c() just like exit(kill). In this way, we want all processes > linked to c will be killed because of c's exit, even if there were any > system processes among c's linked processes. Currently, if we write > exit(kill) or exit(self(), kill) in c(), only c and c's linked non-system > processes will be killed, but all system processed linked will survive, > which is not correct. > > So far, there seems to be no other ways to let it happen correctly except: > > Code: > > c(......) -> > blabla... > > %% P1, P2, ..., Pn are linked SYSTEM processes > exit(P1, kill), > exit(P2, kill), > ... > exit(Pn, kill), > > %%Here c is still alive, and we want c and its linked non-system processes > also killed, so... > exit(kill). %% or exit(self(), kill), or exit(anything).. > > End code. > > However, it still demands that programmers know ahead of time how many and > which system processes are linked to c, which is not a good design pattern. > > Sincerely, > > Yan Yan > > > > > Yan Yan > 2009-10-24 > > > > From: Richard Carlsson > Time: 2009-10-24 21:05:05 > To: Yan Yan > Fw: erlang-questions; erlang-bugs; Bj?n_Gustavsson > Subject: Re: [erlang-questions] Erlang's process not killed by exit > signalreasoned by "kill" > > Yan Yan wrote: > > (2)On Page 169, > (Page 161 in my copy of the book.) > > Quote: > > > > 8> edemo1:start(true, {die,kill}). Process b received > > {'EXIT',<0.73.0>,kill} process b (<0.72.0>) is alive process c > > (<0.73.0>) is dead ok > > > > End quote. > > > > Here a and b are both system processes, while c is not. When c exits > > with the reason "kill" (not "killed"), it sends exit signal to b with > > the reason "kill". Therefore b should be killed and dead, but b is > > still alive here! > > > > (I had thought it was only a small typo in the book. But then I > > tested by myself and got the same result: b received the exit signal > > with the reason "kill" and b was not killed but alive.) > > > > Why is the system process b not killed by the exit signal with the > > reason "kill"? > Interesting. There seems to be a difference in behaviour (probably > not intentional) between `exit(kill)' and `exit(self(),kill)'. > A summary of the setup: we have three processes, linked in a chain. > 'a' is always trapping, 'b' may or may not be trapping, and 'c' is > the one who dies by calling exit(Reason). > 1. When 'b' is not trapping, and 'c' does exit(kill), we see a > report from 'a' that 'b' died with reason 'killed'. > 2. When 'b' is trapping, and 'c' does exit(kill), 'b' survives > and reports that it sees the exit reason 'kill' from 'c' > (not 'killed'). > The question is: in case 2, why didn't 'b' die, when it apparently > got a 'kill' message. This should be untrappable, which is why it is > changed to 'killed' when it is propagated. But it was 'c' who died, > so why wasn't the atom changed to 'killed'? > If we change `exit(kill)' to `exit(self(),kill)' in 'c', we get the > effect we expected: 'b' survives and reports that it sees 'killed' > as the exit reason from 'c'. > Then, a new question is why 'a' saw 'killed' in case 1 when 'b' is > non-trapping. If 'c' dies in the same way in both, then doesn't > 'b' get the same signal from 'c'? Apparently, it does, but since > 'b' is not trapping, it doesn't matter what the atom is as long > as it is something else than 'normal'. So 'b' dies due to an > incoming 'kill', and this is then propagated as 'killed'. > It seems that when a process does exit(kill) on itself, it causes > a different outgoing signal than if it does exit(self(),kill). > The former is not untrappable even though it has the reason 'kill', > but if that in its turn causes another process to die, the atom > will _then_ be rewritten to 'killed'. > To me, it seems that this should be fixed so that exit(kill), even > if it's an unusual case, should be propagated as 'killed'. > /Richard > From roberto@REDACTED Sun Oct 25 01:25:40 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 25 Oct 2009 01:25:40 +0200 Subject: set macro value depending on compilation parameters Message-ID: dear all, i need to define a macro to expand differently depending on a macro value passed at compilation time. for instance, consider the following macro declared inside a module 'test': -ifdef(foo). -define(FCTN, true). -endif. now, what i would like is that if i compile the module 'test' like this: 1>c(test, {d, foo, bar}). {ok, test} then inside module 'foo' the macro ?FCTN expands to true [as per the example above], while if i compile like this: 1>c(test, {d, foo, baz}). {ok, test} then inside module 'foo' the macro ?FCTN expands to false. the way i'm currently using to achieve this is: -ifdef(foo). -define(FCTN, case ?foo of bar -> true; baz -> false end). -endif. however, i do not want to have FCTN perform a case condition every time it is called, since it will ALWAYS return one of the two options once it is compiled. is erlc already optimizing this, ignoring the case condition? or is there another [and better] way to achieve this result? thank you for any suggestions you may have. cheers, r. From kiszl@REDACTED Sun Oct 25 02:00:08 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sun, 25 Oct 2009 02:00:08 +0200 Subject: [erlang-questions] set macro value depending on compilation parameters In-Reply-To: References: Message-ID: <4AE39508.8030002@tmit.bme.hu> Roberto Ostinelli wrote: > dear all, > > i need to define a macro to expand differently depending on a macro > value passed at compilation time. > > for instance, consider the following macro declared inside a module 'test': > > -ifdef(foo). > -define(FCTN, true). > -endif. > > now, what i would like is that if i compile the module 'test' like this: > > 1>c(test, {d, foo, bar}). > {ok, test} > > then inside module 'foo' the macro ?FCTN expands to true [as per the > example above], while if i compile like this: > > 1>c(test, {d, foo, baz}). > {ok, test} > > then inside module 'foo' the macro ?FCTN expands to false. > > the way i'm currently using to achieve this is: > > -ifdef(foo). > -define(FCTN, case ?foo of > bar -> > true; > baz -> > false > end). > -endif. > > however, i do not want to have FCTN perform a case condition every > time it is called, since it will ALWAYS return one of the two options > once it is compiled. is erlc already optimizing this, ignoring the > case condition? or is there another [and better] way to achieve this > result? > > thank you for any suggestions you may have. > > cheers, > > r. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > Hello, If you only need a binary solution, you can switch between the two by defining or not defining foo (with an arbitrary value): -ifdef(foo). -define(FCTN, true). -else. -define(FCTN, false). -endif. If you need multiple values I think your best bet currently is to pass the desired value with the definition: -ifdef(foo). -define(FCTN, ?foo). -else. -define(FCTN, false). -endif. Regards, Zoltan. From ulf.wiger@REDACTED Sun Oct 25 17:28:22 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 25 Oct 2009 17:28:22 +0100 Subject: sortable serialization format Message-ID: <4AE47CA6.90701@erlang-consulting.com> A while ago I started hacking on a serialization format that would have the same sorting properties as Erlang terms. I didn't quite get it to work (negative floats was the most difficult part), but when I returned to it today, I realized that it was only a very small problem. Once fixed, all my QuickCheck suites passed. http://svn.ulf.wiger.net/sext/trunk/sext/doc/index.html In many cases this format is slightly more compact than the External Term Format. The most notable exception is negative bignums, but large binaries also take up a little bit more space (asymptotically approaching 12.5% overhead for binaries larger than 36 bytes). (Strings are much bigger, since I don't try to detect that they are special cases of lists. This should perhaps be fixed.) You are welcome to give it a spin and report to me if you find good uses for it. The original idea was to make it easier to make use of external storage solutions with ordered set semantics, without foregoing the genericity and convenience of generic term serialization. BR, Ulf W PS Float comparisons are handled in the QuickCheck suite by coercing to IEEE 764 binary64 format. This is supposedly what Erlang uses internally, but I encountered lots of weird behaviour before doing this coercion. -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From ulf.wiger@REDACTED Sun Oct 25 23:26:12 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 25 Oct 2009 23:26:12 +0100 Subject: [erlang-questions] sortable serialization format In-Reply-To: References: <4AE47CA6.90701@erlang-consulting.com> Message-ID: <4AE4D084.4080208@erlang-consulting.com> Tony Finch wrote: > > How about using "consistent overhead byte stuffing" to encode binaries > more compactly? http://www.stuartcheshire.org/papers/COBSforToN.pdf > The Erlang reference manual doesn't say how binaries are ordered. Interesting, but wouldn't that break the sort order? Binaries and bit strings are sorted as left-aligned bit arrays. You are right, it isn't documented. It took me a while to wrap my brain around it, even though it's perfectly logical: 1> lists:sort([<<2#111:3>>, <<2#1111111:7>>, <<2#110111111:9>>, <<1:1>>]). [<<1:1>>,<<223,1:1>>,<<7:3>>,<<127:7>>] Looking back, I don't remember how I thought I would work... ...oh, but my email archive has better memory than I: 5> lists:sort([<<1:1>>,<<1:2>>,<<1:3>>,<<1:4>>, <<1:5>>,<<1:6>>,<<1:7>>,<<1:8>>, <<2:2>>,<<2:3>>,<<2:4>>,<<1>>, <<16>>, <<128>>, <<255>>]). [<<1>>, <<1>>, <<1:7>>, <<1:6>>, <<1:5>>, <<1:4>>, <<16>>, <<1:3>>, <<2:4>>, <<1:2>>, <<2:3>>, <<1:1>>, <<2:2>>, <<128>>, <<"?">>] A certain professor used the academic term "Gobsmacking!" to describe this, but the fallacy is that you fall into thinking that the sort order above should resemble the order of the integer values that we have encoded with different lengths. It makes sense only if you view them as bit streams: 6> lists:sort([{sext:pp(B),B} || B <- [<<1:1>>,<<1:2>>,<<1:3>>,<<1:4>>, <<1:5>>,<<1:6>>,<<1:7>>,<<1:8>>, <<2:2>>,<<2:3>>,<<2:4>>,<<1>>, <<16>>, <<128>>, <<255>>]]). [{"00000001",<<1>>}, {"00000001",<<1>>}, {"0000001",<<1:7>>}, {"000001",<<1:6>>}, {"00001",<<1:5>>}, {"0001",<<1:4>>}, {"00010000",<<16>>}, {"001",<<1:3>>}, {"0010",<<2:4>>}, {"01",<<1:2>>}, {"010",<<2:3>>}, {"1",<<1:1>>}, {"10",<<2:2>>}, {"10000000",<<128>>}, {"11111111",<<"?">>}] BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From marian.dan.alexandru@REDACTED Mon Oct 26 10:58:45 2009 From: marian.dan.alexandru@REDACTED (Marian Dan Alexandru) Date: Mon, 26 Oct 2009 11:58:45 +0200 Subject: Compiling Erlang using Intel's C++ Compiler 11 Message-ID: <5750205a0910260258mf4984e1va3a79fcc5c49b036@mail.gmail.com> If there is somebody interested I wrote a tutorial (http://marian-dan.com/wordpress/?p=311) about compiling Erlang with Intel's C++ Compiler 11 for enhancing memory transfer (using streaming and memory bounded threads) and floating point operations (more precision, less speed). From james.hague@REDACTED Mon Oct 26 17:31:59 2009 From: james.hague@REDACTED (James Hague) Date: Mon, 26 Oct 2009 11:31:59 -0500 Subject: potential compiler improvement? Message-ID: I've been looking a lot of BEAM code lately, so I can better understand how the VM works. I have generally been trying to avoid looking at generated code quality too much, because I can easily get obsessed with it (see http://prog21.dadgum.com/50.html). But I have noticed one general thing which might be worth investigating in the future. In a pattern match, all referenced values in tuples are loaded into BEAM registers. This happens right up front as part of the matching process. And it happens even if the values aren't used until some time later, and even if they're not used at all in a particular branch of code. Here's an example: test({A, B, C, D, E, F, G}) -> case A of this -> B + C + D; that -> E + F + G end. In this case, all seven tuple elements, A-G, are loaded into registers before the "case" is executed. This is even though three values are unneeded in each of the two possible branches. I'm no expert on how the compiler works, so I'm not going to hazard a guess at the difficulty of this. James From rapsey@REDACTED Mon Oct 26 19:24:48 2009 From: rapsey@REDACTED (Rapsey) Date: Mon, 26 Oct 2009 19:24:48 +0100 Subject: HAProxy on OpenBSD In-Reply-To: <249E65CE1FE3FE42B8E7D93EA3B1A9F323BC2F1B3B@hermod.andmore.com> References: <249E65CE1FE3FE42B8E7D93EA3B1A9F323BC2F1B3B@hermod.andmore.com> Message-ID: <97619b170910261124lb9b81bfkb4cc2f17501b41@mail.gmail.com> 4096 maxconn does not leave anything for the listen socket? Try setting kern.maxfiles to 16384 Sergej On Mon, Oct 26, 2009 at 7:18 PM, McCann, Brian wrote: > Hi all. I'm trying to setup HAProxy on OpenBSD and I'm having some > problems. I can't seem to get it to accept a maxconn value larger than > 1024. When I run it set to 4096 for example, I get the following: > > [ALERT] 298/135514 (16435) : parsing [/etc/haproxy/haproxy.cfg:5] : maxconn > value 4096 too high for this system. Limiting to 1024. Please use '-n' to > force the value. > > I don't quite get why...I checked kern.maxfiles, and that's set to 8192. > Anyone have any ideas? > > Thanks! > --Brian > > From attila.r.nohl@REDACTED Mon Oct 26 20:45:03 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 26 Oct 2009 20:45:03 +0100 Subject: [erlang-questions] potential compiler improvement? In-Reply-To: References: Message-ID: <401d3ba30910261245m747a333ei5deca2f93da3a2ca@mail.gmail.com> 2009/10/26, James Hague : [...] > In a pattern match, all referenced values in tuples are loaded into BEAM > registers. This happens right up front as part of the matching process. And > it happens even if the values aren't used until some time later, and even if > they're not used at all in a particular branch of code. Here's an example: > > test({A, B, C, D, E, F, G}) -> > case A of > this -> B + C + D; > that -> E + F + G > end. > > In this case, all seven tuple elements, A-G, are loaded into registers > before the "case" is executed. This is even though three values are unneeded > in each of the two possible branches. > > I'm no expert on how the compiler works, so I'm not going to hazard a guess > at the difficulty of this. I don't know much about the compiler, but I think it hasn't got the slightest idea of A being 'this', 'that' or something completely different, so it can't really optimize. Actually if A is not 'this' or 'that', you'll get a crash report which includes all parameters, so it is possible that all seven variables are used. From james.hague@REDACTED Mon Oct 26 21:18:14 2009 From: james.hague@REDACTED (James Hague) Date: Mon, 26 Oct 2009 15:18:14 -0500 Subject: [erlang-questions] potential compiler improvement? In-Reply-To: <401d3ba30910261245m747a333ei5deca2f93da3a2ca@mail.gmail.com> References: <401d3ba30910261245m747a333ei5deca2f93da3a2ca@mail.gmail.com> Message-ID: > > test({A, B, C, D, E, F, G}) -> > > ? ?case A of > > ? ? ? this -> B + C + D; > > ? ? ? that -> E + F + G > > ? ?end. > > I don't know much about the compiler, but I think it hasn't got the > slightest idea of A being 'this', 'that' or something completely > different, so it can't really optimize. You don't need to know the value of A ahead of time. You simply don't unpack B, C, D until you're in the 'this' branch (and once you've gone down that path then you know you don't need to bother unpacking E, F, G). From rvirding@REDACTED Mon Oct 26 22:02:35 2009 From: rvirding@REDACTED (Robert Virding) Date: Mon, 26 Oct 2009 22:02:35 +0100 Subject: [erlang-questions] potential compiler improvement? In-Reply-To: References: Message-ID: <3dbc6d1c0910261402u18f08f29l142682bee4b386cd@mail.gmail.com> 2009/10/26 James Hague > > In a pattern match, all referenced values in tuples are loaded into BEAM > registers. This happens right up front as part of the matching process. And > it happens even if the values aren't used until some time later, and even > if > they're not used at all in a particular branch of code. Here's an example: > > test({A, B, C, D, E, F, G}) -> > case A of > this -> B + C + D; > that -> E + F + G > end. > > In this case, all seven tuple elements, A-G, are loaded into registers > before the "case" is executed. This is even though three values are > unneeded > in each of the two possible branches. > Now the compiler just looks at which elements from the tuple may accessed and only extracts those. So if you only used A, B and C it would only extract the first 3 elements. It could save the whole argument and only extract the arguments when they are used but this would entail saving the whole tuple for the future as long as it may be used. In the general case it could maybe create more instructions, or maybe less, depending on what follows. The analysis would be more difficult, though probably not excessively so. Now the analysis and code generation is easier. It could also delay freeing the tuple which would mean more work the garbage collector. I have absolutely no idea of the results of these trade-offs. Robert From thomasl_erlang@REDACTED Mon Oct 26 22:57:59 2009 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 26 Oct 2009 14:57:59 -0700 (PDT) Subject: [erlang-questions] potential compiler improvement? In-Reply-To: References: Message-ID: <600275.90034.qm@web111415.mail.gq1.yahoo.com> ----- Original Message ---- > In this case, all seven tuple elements, A-G, are loaded into registers > before the "case" is executed. This is even though three values are unneeded > in each of the two possible branches. > > I'm no expert on how the compiler works, so I'm not going to hazard a guess > at the difficulty of this. A long time ago, I experimented with that in the old Ultrasparc Hipe compiler. The idea then was to sink loads to the places where it was used in all successor branches (or the same basic block), in order to avoid redundant loads. The dataflow analysis seemed pretty straightforward, though I didn't prove it correct or anything. The net effect, alas, was a slowdown due to increased load stalls, since the loads moved closer to their uses :-) Anyway, for BEAM on a modern architecture the net outcome may of course be quite different. Best, Thomas From garry@REDACTED Tue Oct 27 00:31:28 2009 From: garry@REDACTED (Garry Hodgson) Date: Mon, 26 Oct 2009 19:31:28 -0400 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <474512.24919.qm@web31804.mail.mud.yahoo.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> <474512.24919.qm@web31804.mail.mud.yahoo.com> Message-ID: <4AE63150.50202@research.att.com> Yogish Baliga wrote: > We are using Erlang across data center boundary. We are using proto_dist as inet_ssl for communication between erlang nodes. Also using min and max port setup provided by inet to limit the communication on a specific set of ports. > just curious. have you noticed any significant performance degradation going this route? -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From maruthavanan_s@REDACTED Tue Oct 27 05:19:52 2009 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Tue, 27 Oct 2009 00:19:52 -0400 Subject: Start A process of another node Message-ID: Hi, I need to start and stop a process on another node Say for e.g I have a main node in erlang. I have another sub node in erlang. I need to start some process in main node from sub node shell. I achieved this by using the below spawn('mainnode@REDACTED',sample,start,[]). but when I close the sub node the process is stopped. Is there any possibility that I can make the process live through out regardless of whether I close the sub node or not, the sample:start() would be running in main node? Thanks, Marutha From kagato@REDACTED Tue Oct 27 06:47:06 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Tue, 27 Oct 2009 00:47:06 -0500 Subject: [erlang-questions] Start A process of another node In-Reply-To: References: Message-ID: I'm unsure what to say here. Erlang doesn't do this to me. I started two nodes, one called main and one called sub. I loaded the process manager on the main node. I created a process on main using the following code: > spawn(main@REDACTED,fun() -> receive {never,Never} -> ok after 30000 - > > ok end end). It stayed around even after exiting the original node. That says to me that something that you process does depends on the original node. A few questions about your code: 1. Does it spawn_link(), spawn_opt() with the link option, or link() another process? 2. Does it do I/O? 3. Does it generate log messages? The second two (I/O / log messages) use the "group leader", I believe. In this case, the group leader would be on the original node, which might cause them to exit. Generally, I believe the group leader of a process should be init. To set the group leader to init on the new node, try: > group_leader( whereis(init), self() ) Note that this will affect logging, supervision trees, and IO. If you want to just start/stop a remotely supervised process, I recommend naming its supervisor globally, and adding/removing the childspec from the supervisor. On Oct 26, 2009, at 11:19 PM, maruthavanan s wrote: > > Hi, > > I need to start and stop a process on another node > > Say for e.g I have a main node in erlang. > > I have another sub node in erlang. I need to start some process in > main node from sub node shell. > > I achieved this by using the below > > spawn('mainnode@REDACTED',sample,start,[]). > > but when I close the sub node the process is stopped. Is there any > possibility that I can make the process live through out regardless > of whether I close the sub node or not, the sample:start() would be > running in main node? > > Thanks, > Marutha > -- Jayson Vantuyl kagato@REDACTED From mjtruog@REDACTED Tue Oct 27 07:04:23 2009 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 26 Oct 2009 23:04:23 -0700 Subject: [erlang-questions] Start A process of another node In-Reply-To: References: Message-ID: <4AE68D67.3070105@gmail.com> If you are using the slave module in the stdlib, since it tries to cleanup zombie nodes. Since the process is not spawned with a link, it shouldn't die. If your shell is attached to the node and you do abort (after Ctrl-C), that could take it down. Instead you should do Ctrl-C twice. maruthavanan s wrote: > Hi, > > I need to start and stop a process on another node > > Say for e.g I have a main node in erlang. > > I have another sub node in erlang. I need to start some process in main node from sub node shell. > > I achieved this by using the below > > spawn('mainnode@REDACTED',sample,start,[]). > > but when I close the sub node the process is stopped. Is there any possibility that I can make the process live through out regardless of whether I close the sub node or not, the sample:start() would be running in main node? > > Thanks, > Marutha > > From tony@REDACTED Tue Oct 27 07:14:14 2009 From: tony@REDACTED (Tony Arcieri) Date: Tue, 27 Oct 2009 00:14:14 -0600 Subject: [erlang-questions] A less trusting Erlang. In-Reply-To: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> References: <635498b70910111455gaf98388m5781ad6d9257f03d@mail.gmail.com> Message-ID: On Sun, Oct 11, 2009 at 3:55 PM, Calum wrote: > The big problem then is that Erlang is way too trusting for connecting > to random people in the default > if-you-know-my-cookie-you-can-do-whatever-you-like mode. > > The obvious thing (to me anyway) would be to limit the calls that > remote nodes can make - Erlang Untrusting, if you will, or Paranoid > Erlang. > I was too lazy to search the thread, so sorry if this was already mentioned. Are you looking for a capabilities model? It seems like the Erlang environment might be ideal for implementing one. Does one already exist? http://en.wikipedia.org/wiki/Object-capability_model -- Tony Arcieri Medioh/Nagravision From sapan.gcet@REDACTED Tue Oct 27 07:40:43 2009 From: sapan.gcet@REDACTED (sapan shah) Date: Tue, 27 Oct 2009 12:10:43 +0530 Subject: Serializing a list Message-ID: Hi, Lets assume that we have a list where, List -> integer | List; eg L1 = [1,[[2,3]],[4]]. when I perform binary_to_list(list_to_binary(L1)) The result is [1,2,3,4]. I loose the original structure of the list. How do then the serialization of the list is done? Is there any other way we can perform 'binary_to_list(list_to_binary(L))' functionality maintaining the structure of the list?? -- Regards, Sapan Shah From max.lapshin@REDACTED Tue Oct 27 07:43:48 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 27 Oct 2009 09:43:48 +0300 Subject: [erlang-questions] Serializing a list In-Reply-To: References: Message-ID: term_to_binary, binary_to_term From sapan.gcet@REDACTED Tue Oct 27 07:48:19 2009 From: sapan.gcet@REDACTED (sapan shah) Date: Tue, 27 Oct 2009 12:18:19 +0530 Subject: [erlang-questions] Serializing a list In-Reply-To: References: Message-ID: Thanks... missed this part... On Tue, Oct 27, 2009 at 12:13 PM, Max Lapshin wrote: > term_to_binary, binary_to_term > -- Regards, Sapan Shah From maruthavanan_s@REDACTED Tue Oct 27 08:58:05 2009 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Tue, 27 Oct 2009 03:58:05 -0400 Subject: [erlang-questions] Start A process of another node In-Reply-To: References: Message-ID: Thanks for your support, I would like to elaborate what I am doing I am using "Erlang (BEAM) emulator version 5.6.1 " with fedora core linux 9 I start one node of erlang as erl -sname main (main@REDACTED)1> now I start erl -sname sub (sub@REDACTED)1> now I spawn from sub spawn('mainnode@REDACTED',sample,start,[]). when I do erlang:registered() at main@REDACTED I can see the sample module registered at main node when I try quit sub@REDACTED I can still see sample module registered at main node, but when I try to send message using erlang:send({'main@REDACTED',sample},{"Message for Test"}) now the the registered module of sample vanishes at main node. Please help me. Below is the sample erlang code -module(sample). -compile(export_all). start()-> register(sample,self()), recv(). recv()-> receive Msg-> io:fwrite("Message Received ~p",[Msg]) end, recv(). Thanks and regards, Marutha > CC: erlang-questions@REDACTED > From: kagato@REDACTED > To: maruthavanan_s@REDACTED > Subject: Re: [erlang-questions] Start A process of another node > Date: Tue, 27 Oct 2009 00:47:06 -0500 > > I'm unsure what to say here. Erlang doesn't do this to me. > > I started two nodes, one called main and one called sub. I loaded the > process manager on the main node. > > I created a process on main using the following code: > > > spawn(main@REDACTED,fun() -> receive {never,Never} -> ok after 30000 - > > > ok end end). > > > It stayed around even after exiting the original node. > > That says to me that something that you process does depends on the > original node. A few questions about your code: > > 1. Does it spawn_link(), spawn_opt() with the link option, or link() > another process? > 2. Does it do I/O? > 3. Does it generate log messages? > > The second two (I/O / log messages) use the "group leader", I > believe. In this case, the group leader would be on the original > node, which might cause them to exit. Generally, I believe the group > leader of a process should be init. To set the group leader to init > on the new node, try: > > > group_leader( whereis(init), self() ) > > > Note that this will affect logging, supervision trees, and IO. If you > want to just start/stop a remotely supervised process, I recommend > naming its supervisor globally, and adding/removing the childspec from > the supervisor. > > On Oct 26, 2009, at 11:19 PM, maruthavanan s wrote: > > > > > Hi, > > > > I need to start and stop a process on another node > > > > Say for e.g I have a main node in erlang. > > > > I have another sub node in erlang. I need to start some process in > > main node from sub node shell. > > > > I achieved this by using the below > > > > spawn('mainnode@REDACTED',sample,start,[]). > > > > but when I close the sub node the process is stopped. Is there any > > possibility that I can make the process live through out regardless > > of whether I close the sub node or not, the sample:start() would be > > running in main node? > > > > Thanks, > > Marutha > > > > > > -- > Jayson Vantuyl > kagato@REDACTED > > > > > From mazen.harake@REDACTED Tue Oct 27 09:30:05 2009 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 27 Oct 2009 08:30:05 +0000 (GMT) Subject: [erlang-questions] Start A process of another node In-Reply-To: Message-ID: <27755766.233091256632205006.JavaMail.root@zimbra> 'main@REDACTED' is not the same as 'mainnode@REDACTED'... maybe that is your problem. Quoting the documentation for spawn/4: "Returns the pid of a new process started by the application of Module:Function to Args on Node. If Node does not exists, a useless pid is returned. Otherwise works like spawn/3." Hope this helps. /Mazen ----- Original Message ----- From: "maruthavanan s" To: kagato@REDACTED Cc: erlang-questions@REDACTED Sent: Tuesday, 27 October, 2009 09:58:05 (GMT+0200) Auto-Detected Subject: RE: [erlang-questions] Start A process of another node Thanks for your support, I would like to elaborate what I am doing I am using "Erlang (BEAM) emulator version 5.6.1 " with fedora core linux 9 I start one node of erlang as erl -sname main (main@REDACTED)1> now I start erl -sname sub (sub@REDACTED)1> now I spawn from sub spawn('mainnode@REDACTED',sample,start,[]). when I do erlang:registered() at main@REDACTED I can see the sample module registered at main node when I try quit sub@REDACTED I can still see sample module registered at main node, but when I try to send message using erlang:send({'main@REDACTED',sample},{"Message for Test"}) now the the registered module of sample vanishes at main node. Please help me. Below is the sample erlang code -module(sample). -compile(export_all). start()-> register(sample,self()), recv(). recv()-> receive Msg-> io:fwrite("Message Received ~p",[Msg]) end, recv(). Thanks and regards, Marutha > CC: erlang-questions@REDACTED > From: kagato@REDACTED > To: maruthavanan_s@REDACTED > Subject: Re: [erlang-questions] Start A process of another node > Date: Tue, 27 Oct 2009 00:47:06 -0500 > > I'm unsure what to say here. Erlang doesn't do this to me. > > I started two nodes, one called main and one called sub. I loaded the > process manager on the main node. > > I created a process on main using the following code: > > > spawn(main@REDACTED,fun() -> receive {never,Never} -> ok after 30000 - > > > ok end end). > > > It stayed around even after exiting the original node. > > That says to me that something that you process does depends on the > original node. A few questions about your code: > > 1. Does it spawn_link(), spawn_opt() with the link option, or link() > another process? > 2. Does it do I/O? > 3. Does it generate log messages? > > The second two (I/O / log messages) use the "group leader", I > believe. In this case, the group leader would be on the original > node, which might cause them to exit. Generally, I believe the group > leader of a process should be init. To set the group leader to init > on the new node, try: > > > group_leader( whereis(init), self() ) > > > Note that this will affect logging, supervision trees, and IO. If you > want to just start/stop a remotely supervised process, I recommend > naming its supervisor globally, and adding/removing the childspec from > the supervisor. > > On Oct 26, 2009, at 11:19 PM, maruthavanan s wrote: > > > > > Hi, > > > > I need to start and stop a process on another node > > > > Say for e.g I have a main node in erlang. > > > > I have another sub node in erlang. I need to start some process in > > main node from sub node shell. > > > > I achieved this by using the below > > > > spawn('mainnode@REDACTED',sample,start,[]). > > > > but when I close the sub node the process is stopped. Is there any > > possibility that I can make the process live through out regardless > > of whether I close the sub node or not, the sample:start() would be > > running in main node? > > > > Thanks, > > Marutha > > > > > > -- > Jayson Vantuyl > kagato@REDACTED > > > > > From kagato@REDACTED Tue Oct 27 09:46:20 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Tue, 27 Oct 2009 03:46:20 -0500 Subject: [erlang-questions] Start A process of another node In-Reply-To: References: Message-ID: <8115F212-5745-4793-B996-1DCB5934B720@souja.net> I think the problem is the io:fwrite call. I believe that IO goes through a process's "group leader", which is a sort of master process that handles things for groups of processes. Since the group leader is no longer around (after sub exits), I bet that the process exits with an error. Since errors are logged through the group leader, you probably don't see an error either. To fix this, after the process registers, change the group leader for your new process to the init process on the new node. To do so, change your start function like this: > start()-> > register(sample,self()), > group_leader( whereis(init), self() ), > recv(). Sent from my iPhone On Oct 27, 2009, at 2:58 AM, maruthavanan s wrote: > > Thanks for your support, I would like to elaborate what I am doing > > I am using "Erlang (BEAM) emulator version 5.6.1 " with fedora core > linux 9 > > I start one node of erlang as > > erl -sname main > (main@REDACTED)1> > > now I start > erl -sname sub > (sub@REDACTED)1> > > now I spawn from sub > spawn('mainnode@REDACTED',sample,start,[]). > > when I do erlang:registered() at main@REDACTED > I can see the sample module registered at main node > > when I try quit sub@REDACTED > I can still see sample module registered at main node, but when I > try to send message > using > > erlang:send({'main@REDACTED',sample},{"Message for Test"}) > > now the the registered module of sample vanishes at main node. > > Please help me. > > Below is the sample erlang code > -module(sample). > -compile(export_all). > > start()-> > register(sample,self()), > recv(). > > recv()-> > receive > Msg-> > io:fwrite("Message Received ~p",[Msg]) > end, > recv(). > > Thanks and regards, > Marutha > >> CC: erlang-questions@REDACTED >> From: kagato@REDACTED >> To: maruthavanan_s@REDACTED >> Subject: Re: [erlang-questions] Start A process of another node >> Date: Tue, 27 Oct 2009 00:47:06 -0500 >> >> I'm unsure what to say here. Erlang doesn't do this to me. >> >> I started two nodes, one called main and one called sub. I loaded >> the >> process manager on the main node. >> >> I created a process on main using the following code: >> >>> spawn(main@REDACTED,fun() -> receive {never,Never} -> ok after 30000 - >>>> ok end end). >> >> >> It stayed around even after exiting the original node. >> >> That says to me that something that you process does depends on the >> original node. A few questions about your code: >> >> 1. Does it spawn_link(), spawn_opt() with the link option, or link() >> another process? >> 2. Does it do I/O? >> 3. Does it generate log messages? >> >> The second two (I/O / log messages) use the "group leader", I >> believe. In this case, the group leader would be on the original >> node, which might cause them to exit. Generally, I believe the group >> leader of a process should be init. To set the group leader to init >> on the new node, try: >> >>> group_leader( whereis(init), self() ) >> >> >> Note that this will affect logging, supervision trees, and IO. If >> you >> want to just start/stop a remotely supervised process, I recommend >> naming its supervisor globally, and adding/removing the childspec >> from >> the supervisor. >> >> On Oct 26, 2009, at 11:19 PM, maruthavanan s wrote: >> >>> >>> Hi, >>> >>> I need to start and stop a process on another node >>> >>> Say for e.g I have a main node in erlang. >>> >>> I have another sub node in erlang. I need to start some process in >>> main node from sub node shell. >>> >>> I achieved this by using the below >>> >>> spawn('mainnode@REDACTED',sample,start,[]). >>> >>> but when I close the sub node the process is stopped. Is there any >>> possibility that I can make the process live through out regardless >>> of whether I close the sub node or not, the sample:start() would be >>> running in main node? >>> >>> Thanks, >>> Marutha >>> >> >> >> >> -- >> Jayson Vantuyl >> kagato@REDACTED >> >> >> >> >> > From kagato@REDACTED Tue Oct 27 10:19:05 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Tue, 27 Oct 2009 04:19:05 -0500 Subject: [erlang-questions] Start A process of another node In-Reply-To: <27755766.233091256632205006.JavaMail.root@zimbra> References: <27755766.233091256632205006.JavaMail.root@zimbra> Message-ID: <349C84CE-49C7-4C0A-B24D-8CB9B338C4A9@souja.net> Also, try using pman to monitor your processes. It's a bit nicer than watching for registered processes. To start pman on a node, just run pman:start() in the shell on that node. Sent from my iPhone From erlang@REDACTED Tue Oct 27 17:37:08 2009 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 27 Oct 2009 17:37:08 +0100 Subject: [erlang-questions] potential compiler improvement? In-Reply-To: <3dbc6d1c0910261402u18f08f29l142682bee4b386cd@mail.gmail.com> References: <3dbc6d1c0910261402u18f08f29l142682bee4b386cd@mail.gmail.com> Message-ID: <9b08084c0910270937o5f4f656bk824727949bcf3f6e@mail.gmail.com> I suspect unpacking 7 or 5 arguments takes more or less the same time. This involves accessing *p and *(p+1) etc. (in C) My understanding of things was once you had accessed *p accessing *(p+1) was free. Accessing *p pulls the data you need into the cache (which is potentially expensive) once in the cache accessing *(p+1) is free. What matters is cache misses. Aligning the start of the tuple on a cache line boundary should give a great win, also making sure that the entire tuple will fit in cache - this is very processor specific, so you'd need a JIT and deep magic to do this. Sticking stuff in different parts of memory and making sure that code does not span page boundaries will also help - don't have a tight loop spanning a page boundary. Erik Hagersten used to optimize C by physically moving C subroutines in a file - shuffle - recompile and measure. And the programs went faster ( a bit) we thought he was nuts, now he has a company that measures this :-) - It's the cache. Tricky stuff /Joe On Mon, Oct 26, 2009 at 10:02 PM, Robert Virding wrote: > 2009/10/26 James Hague > >> >> In a pattern match, all referenced values in tuples are loaded into BEAM >> registers. This happens right up front as part of the matching process. And >> it happens even if the values aren't used until some time later, and even >> if >> they're not used at all in a particular branch of code. ?Here's an example: >> >> test({A, B, C, D, E, F, G}) -> >> ? case A of >> ? ? ?this -> B + C + D; >> ? ? ?that -> E + F + G >> ? end. >> >> In this case, all seven tuple elements, A-G, are loaded into registers >> before the "case" is executed. This is even though three values are >> unneeded >> in each of the two possible branches. >> > > Now the compiler just looks at which elements from the tuple may accessed > and only extracts those. So if you only used A, B and C it would only > extract the first 3 elements. It could save the whole argument and only > extract the arguments when they are used but this would entail saving the > whole tuple for the future as long as it may be used. In the general case it > could maybe create more instructions, or maybe less, depending on what > follows. The analysis would be more difficult, though probably not > excessively so. Now the analysis and code generation is easier. It could > also delay freeing the tuple which would mean more work the garbage > collector. > > I have absolutely no idea of the results of these trade-offs. > > Robert > From james.hague@REDACTED Tue Oct 27 17:58:43 2009 From: james.hague@REDACTED (James Hague) Date: Tue, 27 Oct 2009 11:58:43 -0500 Subject: [erlang-questions] potential compiler improvement? In-Reply-To: <9b08084c0910270937o5f4f656bk824727949bcf3f6e@mail.gmail.com> References: <3dbc6d1c0910261402u18f08f29l142682bee4b386cd@mail.gmail.com> <9b08084c0910270937o5f4f656bk824727949bcf3f6e@mail.gmail.com> Message-ID: > I suspect unpacking 7 or 5 arguments takes more or less the same time. First off, good discussion! In C--and HiPE--perhaps the times are the same, but I was thinking about threaded BEAM code where unpacking the additional three values involves three more BEAM instructions (assuming those aren't collapsed into one super-instruction at load time). Also, in lots of functions a significant portion of the instructions are moves from temporary registers to parameter registers. (Typically there's a string of these right before a function call, one per parameter.) So it is *possible* that delaying the loads of tuple elements until right before needed may reduce the amount of register shuffling going on. Maybe. I completely agree with Joe that this is tricky stuff! James From bgustavsson@REDACTED Tue Oct 27 18:37:43 2009 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 27 Oct 2009 18:37:43 +0100 Subject: [erlang-questions] potential compiler improvement? In-Reply-To: References: <3dbc6d1c0910261402u18f08f29l142682bee4b386cd@mail.gmail.com> <9b08084c0910270937o5f4f656bk824727949bcf3f6e@mail.gmail.com> Message-ID: <6672d0160910271037m2c778391i921c9bd1d10d07d7@mail.gmail.com> On Tue, Oct 27, 2009 at 5:58 PM, James Hague wrote: > In C--and HiPE--perhaps the times are the same, but I was thinking > about threaded BEAM code where unpacking the additional three values > involves three more BEAM instructions (assuming those aren't collapsed > into one super-instruction at load time). They *are* collapsed into super-instructions. Not one, but three for your test program. You can see it if you do erts_debug:df(Mod) (The disassemlbly willl appear in Mod.dis) /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From marian.dan.alexandru@REDACTED Wed Oct 28 01:01:21 2009 From: marian.dan.alexandru@REDACTED (Marian Dan Alexandru) Date: Wed, 28 Oct 2009 02:01:21 +0200 Subject: Needed clarification - Compiling Erlang with Intel(R) C++ Compiler Message-ID: <5750205a0910271701s470833e0sf93baeb501ff2563@mail.gmail.com> In response to all the mails that I have received and the comments on reddit I decided to make some clarifications about what were my intentions. I personally believe that one of the benefits that an Open Source Project provides, is the ability to adapt the underlying code to your needs: find its weaknesses and try to improve them. I didn't touched a line of code, I just compiled it to use my underlying hardware. Every hardware component has its strengths and weaknesses, and the duty of a user is to make the most out of it. For further clarification http://marian-dan.com/wordpress/?p=324. From ok@REDACTED Wed Oct 28 01:10:22 2009 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 28 Oct 2009 13:10:22 +1300 Subject: [erlang-questions] Serializing a list In-Reply-To: References: Message-ID: <55AD087D-F94E-43B5-9C51-E6918432D09A@cs.otago.ac.nz> On Oct 27, 2009, at 7:40 PM, sapan shah wrote: > eg L1 = [1,[[2,3]],[4]]. > when I perform > binary_to_list(list_to_binary(L1)) > The result is [1,2,3,4]. I loose the original structure of the list. That's because list_to_binary/1 is defined to take an "iolist", which is a tree structure whose leaves are binaries and *bytes*, flatten it, and give you a binary. Had you tried a larger integer, such as list_to_binary([999]), you would have had even more trouble. What you want is term_to_binary/1 to do the encoding and binary_to_term/1 to do the decoding. Searching the 'erlang' manual page for 'to_binary' and 'binary_to' would have found these. From roberto@REDACTED Wed Oct 28 01:49:44 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 28 Oct 2009 01:49:44 +0100 Subject: mnesia vs ets Message-ID: dear all, i am populating a mnesia disc_copies table with 20,000 records like this: WriteFun = fun() -> [mnesia:dirty_write({test_table, K, {}}) || K <- lists:seq(1, 20000)] end, mnesia:activity(sync_dirty, WriteFun). then i try reading from the table using mnesia: [mnesia:dirty_read(test_table, K) || K <- lists:seq(1, 20000)]. and then using ets: [ets:lookup(test_table, K) || K <- lists:seq(1, 20000)]. benchmarking the two ways of reading gives the following results: MNESIA: 29473.89 rows / min. ETS: 93312.60 rows / min. this means that with ets reading speed is 3x than dirty_read with MNESIA. am i missing something here, or is this normal? r. From corticalcomputer@REDACTED Wed Oct 28 02:00:27 2009 From: corticalcomputer@REDACTED (G.S.) Date: Tue, 27 Oct 2009 18:00:27 -0700 Subject: [erlang-questions] mnesia vs ets In-Reply-To: References: Message-ID: <2a67d3ff0910271800i29ca3488s5e2ef9b6929c6bca@mail.gmail.com> It's normal. But you can if you wish, access the Mnesia's ets tables directly, which will give you extra speed in reading. Regards, -Gene On Tue, Oct 27, 2009 at 5:49 PM, Roberto Ostinelli wrote: > dear all, > > i am populating a mnesia disc_copies table with 20,000 records like this: > > WriteFun = fun() -> [mnesia:dirty_write({test_table, K, {}}) || K <- > lists:seq(1, 20000)] end, > mnesia:activity(sync_dirty, WriteFun). > > then i try reading from the table using mnesia: > > [mnesia:dirty_read(test_table, K) || K <- lists:seq(1, 20000)]. > > and then using ets: > > [ets:lookup(test_table, K) || K <- lists:seq(1, 20000)]. > > benchmarking the two ways of reading gives the following results: > > MNESIA: 29473.89 rows / min. > ETS: 93312.60 rows / min. > > this means that with ets reading speed is 3x than dirty_read with MNESIA. > > am i missing something here, or is this normal? > > r. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From roberto@REDACTED Wed Oct 28 02:25:58 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 28 Oct 2009 02:25:58 +0100 Subject: [erlang-questions] mnesia vs ets In-Reply-To: <2a67d3ff0910271759k6480bf9bm55145f193fea14c5@mail.gmail.com> References: <2a67d3ff0910271759k6480bf9bm55145f193fea14c5@mail.gmail.com> Message-ID: > If you wish you could directly access the ets tables within mnesia, that will give you the fastest read possible. isn't this what i'm doing with ets:lookup(test_table, K)? also, what's the point in doing a key/value mnesia:dirty_read on non-fragmented disc_copy tables instead of a 3x fasters ETS lookup? r. From rapsey@REDACTED Wed Oct 28 07:03:20 2009 From: rapsey@REDACTED (Rapsey) Date: Wed, 28 Oct 2009 07:03:20 +0100 Subject: [Erlyaws-list] Fall through for appmods? In-Reply-To: <36d886900910272013u1fd9c8c1xa0a4b05ac456553d@mail.gmail.com> References: <36d886900910272013u1fd9c8c1xa0a4b05ac456553d@mail.gmail.com> Message-ID: <97619b170910272303q5445ac4fsa59dfe3965bba298@mail.gmail.com> My workaround is to create an appmod like this: appmods = And modify the path in the reverse proxy I have listening on 80 (yaws is on a higher port). I use haproxy: reqrep ^GET\ /\ (.*) GET\ /d/\ \1 Sergej On Wed, Oct 28, 2009 at 4:13 AM, Nicklas Widlund Bjurman < lordmetroid@REDACTED> wrote: > I am trying to create a boostrap loader for webapps which is seemingly > integrated as the root folder so my config file looks like: > > ebin_dir = /home/dev/ebin > > > port = 8080 > listen = 0.0.0.0 > docroot = /home/dev/hp > appmods = > > > I want the conductor module to load the webapp specified modules and > so on but it would be quite inefficient I think to go through the > module everytime I want to reach an image file or whatever we have. > Actually at this point, I would be glad to reach anything but the > conductor module... > > Anyone know how to create a fallthrough either inside the conductor > module or by the means of Yaws configuration? > > Best regards > Nicklas W Bjurman > > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry(R) Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9 - 12, 2009. Register now! > http://p.sf.net/sfu/devconference > _______________________________________________ > Erlyaws-list mailing list > Erlyaws-list@REDACTED > https://lists.sourceforge.net/lists/listinfo/erlyaws-list > From ulf.wiger@REDACTED Wed Oct 28 08:58:00 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 28 Oct 2009 08:58:00 +0100 Subject: [erlang-questions] mnesia vs ets In-Reply-To: References: <2a67d3ff0910271759k6480bf9bm55145f193fea14c5@mail.gmail.com> Message-ID: <4AE7F988.1050102@erlang-consulting.com> Roberto Ostinelli wrote: > also, what's the point in doing a key/value mnesia:dirty_read on > non-fragmented disc_copy tables instead of a 3x fasters ETS lookup? Dirty read is location- and storage type-agnostic. This means that you will have greater flexibility to change the storage type or distribution pattern of the table without necessarily breaking your code. In some cases, this flexibility is not worth as much as the raw speed. Personally, I always start out using transactions, and always using mnesia:activity(Type, F). I have very rarely had to conclude that transactions weren't fast enough for my needs. BTW, dirty reads to _not_ support fragmented tables out of the box. Neither does the standard mnesia:transaction(F). You need to use mnesia:activity(Type, F, mnesia_frag). BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From rtrlists@REDACTED Wed Oct 28 12:35:34 2009 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 28 Oct 2009 11:35:34 +0000 Subject: [erlang-questions] is inet:gethostbyname( IP ) correct? In-Reply-To: <20091023135603.GA31042@erix.ericsson.se> References: <20091021083803.GA32147@erix.ericsson.se> <200910211157176241641@its3.ch> <20091021122804.GA4311@erix.ericsson.se> <20091023135603.GA31042@erix.ericsson.se> Message-ID: <6a3ae47e0910280435j2a7225ccqdf556aa107a58946@mail.gmail.com> Hi Raimo. On Fri, Oct 23, 2009 at 1:56 PM, Raimo Niskanen < raimo+erlang-questions@REDACTED > wrote: > After a week of bug hunting we can confirm this is really > Microsoft's bug, but we have a workaround. > (It may be Microsoft's and Comodo's bug, who knows...) > > It turns out if you mix overlapped I/O with non-overlapped > (asyncronous with syncronous) on different pipes, in this > case the pipe to inet_gethost.exe from erlang was opened > by erlang as non-overlapped and inet_gethost.exe itself > used overlapped on all its other sockets and pipes... > > ...alright if you mix ... I/O something gets not initialized > in Microsoft's libs unless you do a gethostname() call > before going into recv() of commands from erlang (we already > found this out the hard way) but now also if you had certain > firewalls installed it was not enough, you had to do > gethostbyname() calls with longer domain names to > work around Microsoft's bug. > > Since this felt like a way to shaky workaround we found > another that seems much better: now erlang can open the > pipe to this specific port program (inet_gethost.exe) > for overlapped I/O so inet_gethost.exe now uses overlapped > I/O only. This seems to avoid the bug. > > We reproduced the name lookup freeze with Comodo 2.4.19.185 > and Microsoft's Client for ISA Server 4.0. Here is > a snapshot build (without documentation) that > does not freeze for neither of these firewalls. > > http://www.erlang.org/download/snapshots/otp_win32_R13B03_gethost_fix.exe > > Try it and see if it solves your problem. > > / Raimo Niskanen > > > Congratulations! You have a winner. I have gone through the following tests: R13B02-1 on W2003, no Firewall Client for ISA inet:gethostbyname("localhost"). OK inet:gethostbyname("local machine name"). OK inet:gethostbyname("machinename.localdomain"). OK inet:gethostbyname("www.google.com"). OK inet:gethostbyname("www.cisco.com"). OK R13B02-1 on W2003, with Firewall Client for ISA inet:gethostbyname("localhost"). OK inet:gethostbyname("local machine name"). OK inet:gethostbyname("machinename.localdomain"). OK inet:gethostbyname("www.google.com"). timeout inet:gethostbyname("www.cisco.com"). timeout R13B03_gethost_fix on W2003, no Firewall Client for ISA inet:gethostbyname("localhost"). OK inet:gethostbyname("local machine name"). OK inet:gethostbyname("machinename.localdomain"). OK inet:gethostbyname("www.google.com"). OK inet:gethostbyname("www.cisco.com"). OK R13B03_gethost_fix on W2003, with Firewall Client for ISA inet:gethostbyname("localhost"). OK inet:gethostbyname("local machine name"). OK inet:gethostbyname("machinename.localdomain"). OK inet:gethostbyname("www.google.com"). OK inet:gethostbyname("www.cisco.com"). OK Brilliant work for tracking down this obscure behaviour. Robby From ngocdaothanh@REDACTED Wed Oct 28 12:46:41 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Wed, 28 Oct 2009 20:46:41 +0900 Subject: Process Dictionary vs Proplist in Web Frameworks Message-ID: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Hi, Web frameworks are normally designed as layers (web server -> middleware -> front controller -> controller -> action -> view -> etc.). Data needs to be passed from one layer to another. There are 2 ways to pass: 1. Proplist (environment variables) 2. Process dictionary The 2nd way: * Is Simple and natural in Erlang because normally one HTTP request is processed by one process. * Makes application code which uses the framework appear to be clean, because application developer does not have to manually pass an ugly proplist arround and arround. I want to ask about the (memory, CPU etc.) overhead of process dictionary, compared to proplist. Which way should be used in a web framework? Thanks. From max.lapshin@REDACTED Wed Oct 28 12:49:47 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 28 Oct 2009 14:49:47 +0300 Subject: [erlang-questions] Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Message-ID: Even such stateful languages like Ruby use explicit passing of all request information in hash table. There is absolutely no use in using some mutable dictionaries. From ngocdaothanh@REDACTED Wed Oct 28 13:01:32 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Wed, 28 Oct 2009 21:01:32 +0900 Subject: [erlang-questions] Process Dictionary vs Proplist in Web Frameworks In-Reply-To: References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Message-ID: <5c493e530910280501u5158203fk6a36ad93cb3f70a1@mail.gmail.com> In Rails you write in action: @my_var = ... Then later to take out the variable in view: <%= @my_var %> I think it is not explicit passing. On Wed, Oct 28, 2009 at 8:49 PM, Max Lapshin wrote: > Even such stateful languages like Ruby use explicit passing of all > request information in > hash table. There is absolutely no use in using some mutable dictionaries. > From attila.r.nohl@REDACTED Wed Oct 28 13:05:18 2009 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 28 Oct 2009 13:05:18 +0100 Subject: [erlang-questions] Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Message-ID: <401d3ba30910280505i42eef875v558f2eea0e538ce4@mail.gmail.com> 2009/10/28, Ngoc Dao : > Hi, > > Web frameworks are normally designed as layers (web server -> > middleware -> front controller -> controller -> action -> view -> > etc.). Data needs to be passed from one layer to another. There are 2 > ways to pass: > 1. Proplist (environment variables) > 2. Process dictionary > > The 2nd way: > * Is Simple and natural in Erlang because normally one HTTP request is > processed by one process. I don't have experience with web frameworks, but some erlang libraries use a number of processes "behind the scene", and callbacks might be executed in a quite different process context, so the process dictinary is less then useful in these cases. > * Makes application code which uses the framework appear to be clean, > because application developer does not have to manually pass an ugly > proplist arround and arround. It's the erlang way to carry that kind of stuff around, just think about the always-present State variable in gen_* callbacks. From rapsey@REDACTED Wed Oct 28 13:08:05 2009 From: rapsey@REDACTED (Rapsey) Date: Wed, 28 Oct 2009 13:08:05 +0100 Subject: [erlang-questions] Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <5c493e530910280501u5158203fk6a36ad93cb3f70a1@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <5c493e530910280501u5158203fk6a36ad93cb3f70a1@mail.gmail.com> Message-ID: <97619b170910280508u5b522e4fg7b3d9a920c70a58f@mail.gmail.com> What you wish to use is up to you. Process dictionary will definitely be faster, but it makes the code more difficult to manage and debug. Sergej On Wed, Oct 28, 2009 at 1:01 PM, Ngoc Dao wrote: > In Rails you write in action: > @my_var = ... > > Then later to take out the variable in view: > <%= @my_var %> > > I think it is not explicit passing. > > > On Wed, Oct 28, 2009 at 8:49 PM, Max Lapshin > wrote: > > Even such stateful languages like Ruby use explicit passing of all > > request information in > > hash table. There is absolutely no use in using some mutable > dictionaries. > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From nesrait@REDACTED Wed Oct 28 14:23:51 2009 From: nesrait@REDACTED (=?ISO-8859-1?Q?Davide_Marqu=EAs?=) Date: Wed, 28 Oct 2009 13:23:51 +0000 Subject: [erlang-questions] Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <97619b170910280508u5b522e4fg7b3d9a920c70a58f@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <5c493e530910280501u5158203fk6a36ad93cb3f70a1@mail.gmail.com> <97619b170910280508u5b522e4fg7b3d9a920c70a58f@mail.gmail.com> Message-ID: <523869a70910280623i37fdd30h2d4d7183502c4b52@mail.gmail.com> > > What you wish to use is up to you. Process dictionary will definitely be > faster, but it makes the code more difficult to manage and debug. > +1!! The process dictionary might seem appealing at first but if you use it you'll be unable to reason about the data flow just by looking at the connections across your functions/modules. Debugging stops being a matter of just looking at your code but and starts requiring that you reason about the side effects past code interactions might have produced in the process' current state. Good luck with that. ;) :Davide From james.hague@REDACTED Wed Oct 28 17:21:46 2009 From: james.hague@REDACTED (James Hague) Date: Wed, 28 Oct 2009 11:21:46 -0500 Subject: [erlang-questions] Process Dictionary vs Proplist in Web Frameworks In-Reply-To: References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Message-ID: > Even such stateful languages like Ruby use explicit passing of all > request information in hash table. There is absolutely no use in > using some mutable dictionaries. If the data remains constant once the request is parsed, then pass it around as a dictionary / gb_tree / etc. (or even just a list of {Key, Value} tuples, which has the fastest look-up time for shortish lists if you use lists:keyfind). Now in the case where you need to be making random reads AND WRITES to data like this, across a number of functions, then I can totally understand using the process dictionary. When I run into situations like that I wish I could just write a particular module in Python and be done with it. From rvirding@REDACTED Wed Oct 28 18:08:15 2009 From: rvirding@REDACTED (Robert Virding) Date: Wed, 28 Oct 2009 18:08:15 +0100 Subject: [erlang-questions] Erlang's process not killed by exit signalreasoned by "kill" In-Reply-To: <4AE36B43.6080300@gmail.com> References: <200910241907167657586@gmail.com> <200910242342216873502@gmail.com> <3dbc6d1c0910241310p53a30c23m58ba263d271f99fe@mail.gmail.com> <4AE36B43.6080300@gmail.com> Message-ID: <3dbc6d1c0910281008r553a9f2bxe69b59828bf19544@mail.gmail.com> 2009/10/24 Richard Carlsson > Robert Virding wrote: > >> That exit(kill) and exit(Self, kill) behave differently is *exactly* as it >> should be, they are *defined* to behave differently. >> [...] >> >> Whether doing exit(kill) should send a real exit 'kill' signal or only a >> 'killed' signal is an interesting question which I can't remember now what >> we decided. If it sends an exit 'kill' signal then >> it should behave as a real 'kill' signal and be non-trappable. >> > > The problem now was that it didn't do either: it sends off a signal > that says 'kill', but which isn't a real non-trappable kill signal. > So the behaviour is different where it shouldn't be. > Yes. Unfortunately, to my knowledge discussion of this has never come before. Robert P.S. This type of thing is a good reason to have a proper language spec. From pablo.platt@REDACTED Wed Oct 28 20:07:54 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Wed, 28 Oct 2009 12:07:54 -0700 (PDT) Subject: erlydtl and the template module Message-ID: <894183.64438.qm@web112608.mail.gq1.yahoo.com> Hi, I'm using erlydtl to compile templates for a website http://code.google.com/p/erlydtl/ The example on the first page shows how to compile a module: erlydtl:compile("/path/to/welcome.html", welcome_template), welcome_template:render([ {name, "Johnny"}, {friends, [<<"Frankie Lee">>, <<"Judas Priest">>]}, {primes, [1, 2, "3", <<"5">>]} ]). Is welcome_template a module? How long does it exists? until the process that called it terminates? If I have two processes handling two different requests at the same time will each process recreate the welcome_template ? Can I use the atom welcome_template for two different templates with two processes at the same time? Thanks From kiszl@REDACTED Wed Oct 28 20:17:39 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Wed, 28 Oct 2009 20:17:39 +0100 Subject: [erlang-questions] erlydtl and the template module In-Reply-To: <894183.64438.qm@web112608.mail.gq1.yahoo.com> References: <894183.64438.qm@web112608.mail.gq1.yahoo.com> Message-ID: <4AE898D3.6010408@tmit.bme.hu> Hi Pablo, Yes, welcome_template is a module. The call to compile creates a proper Erlang module from your template store. You can use it just as you would use any other module. It will be available as long as the Erlang VM is running. If you want it to persist, you can pass the outdir option to the compile call, and the module will also be saved as a .beam file. See http://code.google.com/p/erlydtl/wiki/Overview for details. Regards, Zoltan. Pablo Platt wrote: > Hi, > > I'm using erlydtl to compile templates for a website > http://code.google.com/p/erlydtl/ > > The example on the first page shows how to compile a module: > erlydtl:compile("/path/to/welcome.html", welcome_template), > welcome_template:render([ > {name, "Johnny"}, > {friends, [<<"Frankie Lee">>, <<"Judas Priest">>]}, > {primes, [1, 2, "3", <<"5">>]} > ]). > > Is welcome_template a module? > How long does it exists? until the process that called it terminates? > > If I have two processes handling two different requests at the same time > will each process recreate the welcome_template ? > > Can I use the atom welcome_template for two different templates with two processes at the same time? > > Thanks > > > > > From kiszl@REDACTED Wed Oct 28 20:41:50 2009 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Wed, 28 Oct 2009 20:41:50 +0100 Subject: [erlang-questions] erlydtl and the template module In-Reply-To: <195393.22894.qm@web112613.mail.gq1.yahoo.com> References: <894183.64438.qm@web112608.mail.gq1.yahoo.com> <4AE898D3.6010408@tmit.bme.hu> <195393.22894.qm@web112613.mail.gq1.yahoo.com> Message-ID: <4AE89E7E.9000501@tmit.bme.hu> Pablo, I believe that template compiling should only be done once, preferably when bootstrapping your server application. Alternatively you can compile the templates "by hand" to your ebin directory, and template modules will automatically be loaded when needed. The point in the behaviour of compile is that this way you can fix or update your templates without needing to restart the server. If you have templates that you want to use simultaneously, you should compile them into different modules (e.g. welcome and welcome2). Regards, Zoltan. Pablo Platt wrote: > Hi Zoltan, > > I've tried to compile in the console two different templates > welcome1.html and welcome2.html with the same atom: > erlydtl:compile("/path/to/welcome.html", welcome_template). > ok > erlydtl:compile("/path/to/welcome2.html", welcome_template). > ok > > If the welcome_template module already exists in memory I expected to > get an error. > Does erlydtl replace the welcome_template with a new one? Is this a bug? > > If erlydtl could check that the module already exists and use it > instead of recreating it, > it could increase performance like using output dir because you'll > need to compile templates only once. > > Thanks > > ------------------------------------------------------------------------ > *From:* Zoltan Lajos Kis > *To:* Pablo Platt > *Cc:* erlang-questions > *Sent:* Wed, October 28, 2009 9:17:39 PM > *Subject:* Re: [erlang-questions] erlydtl and the template module > > Hi Pablo, > > Yes, welcome_template is a module. The call to compile creates a > proper Erlang module from your template store. > You can use it just as you would use any other module. It will be > available as long as the Erlang VM is running. > If you want it to persist, you can pass the outdir option to the > compile call, and the module will also be saved as a .beam file. > See http://code.google.com/p/erlydtl/wiki/Overview for details. > > Regards, > Zoltan. > > > > Pablo Platt wrote: > > Hi, > > > > I'm using erlydtl to compile templates for a website > http://code.google.com/p/erlydtl/ > > > > The example on the first page shows how to compile a module: > > erlydtl:compile("/path/to/welcome.html", welcome_template), > > welcome_template:render([ > > {name, "Johnny"}, > > {friends, [<<"Frankie Lee">>, <<"Judas Priest">>]}, > > {primes, [1, 2, "3", <<"5">>]} > > ]). > > > > Is welcome_template a module? > > How long does it exists? until the process that called it terminates? > > > > If I have two processes handling two different requests at the same > time will each process recreate the welcome_template ? > > > > Can I use the atom welcome_template for two different templates with > two processes at the same time? > > > > Thanks > > > > > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From pablo.platt@REDACTED Wed Oct 28 20:28:14 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Wed, 28 Oct 2009 12:28:14 -0700 (PDT) Subject: [erlang-questions] erlydtl and the template module In-Reply-To: <4AE898D3.6010408@tmit.bme.hu> References: <894183.64438.qm@web112608.mail.gq1.yahoo.com> <4AE898D3.6010408@tmit.bme.hu> Message-ID: <195393.22894.qm@web112613.mail.gq1.yahoo.com> Hi Zoltan, I've tried to compile in the console two different templates welcome1.html and welcome2.html with the same atom: erlydtl:compile("/path/to/welcome.html", welcome_template). ok erlydtl:compile("/path/to/welcome2.html", welcome_template). ok If the welcome_template module already exists in memory I expected to get an error. Does erlydtl replace the welcome_template with a new one? Is this a bug? If erlydtl could check that the module already exists and use it instead of recreating it, it could increase performance like using output dir because you'll need to compile templates only once. Thanks ________________________________ From: Zoltan Lajos Kis To: Pablo Platt Cc: erlang-questions Sent: Wed, October 28, 2009 9:17:39 PM Subject: Re: [erlang-questions] erlydtl and the template module Hi Pablo, Yes, welcome_template is a module. The call to compile creates a proper Erlang module from your template store. You can use it just as you would use any other module. It will be available as long as the Erlang VM is running. If you want it to persist, you can pass the outdir option to the compile call, and the module will also be saved as a .beam file. See http://code.google.com/p/erlydtl/wiki/Overview for details. Regards, Zoltan. Pablo Platt wrote: > Hi, > > I'm using erlydtlto compile templates for a website http://code.google.com/p/erlydtl/ > > The example on the first page shows how to compile a module: > erlydtl:compile("/path/to/welcome.html", welcome_template), > welcome_template:render([ > {name, "Johnny"}, > {friends, [<<"Frankie Lee">>, <<"Judas Priest">>]}, > {primes, [1, 2, "3", <<"5">>]} > ]). > > Is welcome_template a module? > How long does it exists? until the process that called it terminates? > > If I have two processes handling two different requests at the same time will each process recreate the welcome_template ? > > Can I use the atom welcome_template for two different templates with two processes at the same time? > > Thanks > > > > ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From pablo.platt@REDACTED Wed Oct 28 20:45:52 2009 From: pablo.platt@REDACTED (Pablo Platt) Date: Wed, 28 Oct 2009 12:45:52 -0700 (PDT) Subject: [erlang-questions] erlydtl and the template module In-Reply-To: <4AE89E7E.9000501@tmit.bme.hu> References: <894183.64438.qm@web112608.mail.gq1.yahoo.com> <4AE898D3.6010408@tmit.bme.hu> <195393.22894.qm@web112613.mail.gq1.yahoo.com> <4AE89E7E.9000501@tmit.bme.hu> Message-ID: <627547.38876.qm@web112601.mail.gq1.yahoo.com> It is clear now. Maybe the erlydtl team can add this to the docs. Thanks ________________________________ From: Zoltan Lajos Kis To: Pablo Platt Cc: erlang-questions Sent: Wed, October 28, 2009 9:41:50 PM Subject: Re: [erlang-questions] erlydtl and the template module Pablo, I believe that template compiling should only be done once, preferably when bootstrapping your server application. Alternatively you can compile the templates "by hand" to your ebin directory, and template modules will automatically be loaded when needed. The point in the behaviour of compile is that this way you can fix or update your templates without needing to restart the server. If you have templates that you want to use simultaneously, you should compile them into different modules (e.g. welcome and welcome2). Regards, Zoltan. Pablo Platt wrote: > Hi Zoltan, > > I've tried to compile in the console two different templates welcome1.html and welcome2.html with the same atom: > erlydtl:compile("/path/to/welcome.html", welcome_template). > ok > erlydtl:compile("/path/to/welcome2.html", welcome_template). > ok > > If the welcome_template module already exists in memory I expected to get an error. > Does erlydtl replace the welcome_template with a new one? Is this a bug? > > If erlydtl could check that the module already exists and use it instead of recreating it, > it could increase performance like using output dir because you'll need to compile templates only once. > > Thanks > > ------------------------------------------------------------------------ > *From:* Zoltan Lajos Kis > *To:* Pablo Platt > *Cc:* erlang-questions > *Sent:* Wed, October 28, 2009 9:17:39 PM > *Subject:* Re: [erlang-questions] erlydtl and the template module > > Hi Pablo, > > Yes, welcome_template is a module. The call to compile creates a proper Erlang module from your template store. > You can use it just as you would use any other module. It will be available as long as the Erlang VM is running. > If you want it to persist, you can pass the outdir option to the compile call, and the module will also be saved as a .beam file. > See http://code.google.com/p/erlydtl/wiki/Overview for details. > > Regards, > Zoltan. > > > > Pablo Platt wrote: > > Hi, > > > > I'm using erlydtl to compile templates for a website http://code.google.com/p/erlydtl/ > > > > The example on the first page shows how to compile a module: > > erlydtl:compile("/path/to/welcome.html", welcome_template), > > welcome_template:render([ > > {name, "Johnny"}, > > {friends, [<<"Frankie Lee">>, <<"Judas Priest">>]}, > > {primes, [1, 2, "3", <<"5">>]} > > ]). > > > > Is welcome_template a module? > > How long does it exists? until the process that called it terminates? > > > > If I have two processes handling two different requests at the same time will each process recreate the welcome_template ? > > > > Can I use the atom welcome_template for two different templates with two processes at the same time? > > > > Thanks > > > > > > > > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From erlang@REDACTED Wed Oct 28 22:32:03 2009 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 28 Oct 2009 22:32:03 +0100 Subject: [erlang-questions] Erlang's process not killed by exit signalreasoned by "kill" In-Reply-To: <3dbc6d1c0910241310p53a30c23m58ba263d271f99fe@mail.gmail.com> References: <200910241907167657586@gmail.com> <200910242342216873502@gmail.com> <3dbc6d1c0910241310p53a30c23m58ba263d271f99fe@mail.gmail.com> Message-ID: <9b08084c0910281432m134e4f98p4154eb33ca224f52@mail.gmail.com> On Sat, Oct 24, 2009 at 9:10 PM, Robert Virding wrote: > I just want to clear up one little point: there is in fact only one type of > process, all processes behave in the same way with respect to dying and exit > signals. So what Joe calls "system processes" are just processes started by > the system, nothing more. In this context "system process" means "a process that has evaluated process_flag(trap_exit, true)" > > That exit(kill) and exit(Self, kill) behave differently is *exactly* as it > should be, they are *defined* to behave differently. exit/2 will *always* > send an exit signal to the process even if the process itself! Its behaviour > is consistent. When sending to itself the process should behave as if the > exit signal came from "outside" the itself. This means that if you are > trapping exits you should also trap the exits from exit/2 to yourself. > Yes - in the book and elsewhere we refer to "signals" and "messages" - but they are not the same thing. Signals get converted to messages and put into the inbox of a process if you have said process_flag(tap_exit, true). But the signal generated by exit(Pid, kill) *never* gets converted to a message. When c dies by evaluating exit(X) it broadcasts a signal to it's link set If b is linked to c it will convert this to the message {'EXIT',Pid,X} this is true for all X. If c says exit(Pic, kill) a *different kind* of unstoppable signal is sent and b must die. Signals are internal things and can never be printed, we can only print the messages that result form signals being converted to messages. The origonal reason for this was to be able to kill rogue processes. If A wants to kill B it evaluates exit(B, kill) and we are guaranted that B will die. If A just says exit(kill) it says I'm dieing with reason kill. The distinction between messages and signals is subtle. If you read section 9.4 subsection "system processes" of my book you'll see this behavior is described more or less as I have described it above. Now if you didn't like the above description we can try a different approach ---- a different way of describing the same thing The VM behaves as if there were 3 type of inter-process messages: (1, Data) tuples represent regular messages (2, Data) tuples represent signals 3 is the kill signal P ! X sends a type 1 message (1,X) exit(X) sends a type 2 message (2,X) exit(P, kill) sends a type 3 message exit(P, X) sends a type 2 message (2,X) on receiving a type 2 message if X != normal a normal process dies on receiving a type 2 message a system process converts this to a (1,X) message on receiving a type 3 message the receiving process dies unconditionally --- end This is reasonably clear - the *problem* is that to describe the behavior I've had to suddenly invent an new abstract machine and you have to guess the semantics of this machine. That's why we try very hard to use terms like signals and messages to describe the behavior in an abstract way and not in terms of a VM - because the description of the VM itself would be pretty large and for most purposes is not necessary. (( and anyway if I describe the VM in language in L1, what do I describe L1 in? L2? and what do I describe L2 in .....)) To make life simple we try to use meta-circular descriptions and describe Erlang behavior in terms of Erlang. Mostly this works - but for the case of signals and messages is not easy. > Oth an exit/1 works "internally". It should behave in the same way as > erlang:error/1. In fact originally they were the same and there was only > exit/1. Internal errors also looked the same as exit/1. Things started to > drift apart when internal errors got stack traces. Now stack traces are a > Good Thing but it meant that the behaviour of errors and exit/1 diverged. > Unfortunately, instead of fixing exit/1, which would have been the most > sensible thing to do, the split was made permanent by adding error/1. > > So exit/1 and error/1 should behave in the same way, which is different to > exit/2. > > 'kill' is a little special as we felt we needed something which wasn't > trappable but at the same time we realised that if it spread in the same way > as other exit signals then it would be uncontrollable. This is why it was > decided that when a process received an exit 'kill' signal it should always > unconditionally die but only resignal 'killed' to its linked processes. > Whether doing exit(kill) should send a real exit 'kill' signal or only a > 'killed' signal is an interesting question which I can't remember now what > we decided. If it sends an exit 'kill' signal then it should behave as a > real 'kill' signal and be non-trappable. > > This definitely something for the Erlang Rationale! Yup /Joe > > Robert > > 2009/10/24 Yan Yan > >> Thanks, Richard >> >> In fact, I had a very similiar thought with yours as I found b killed when >> b is not a system process but b not killed when b is a system process. >> >> However, even if exit(kill) acts the same as exit(self(), kill) does, there >> seems to be problems left. For example, if, in this setup, c does want ALL >> linked processes to be killed when c itself exits abnormally, we will want a >> simple code in c() just like exit(kill). In this way, we want all processes >> linked to c will be killed because of c's exit, even if there were any >> system processes among c's linked processes. Currently, if we write >> exit(kill) or exit(self(), kill) in c(), only c and c's linked non-system >> processes will be killed, but all system processed linked will survive, >> which is not correct. >> >> So far, there seems to be no other ways to let it happen correctly except: >> >> Code: >> >> c(......) -> >> blabla... >> >> %% P1, P2, ..., Pn are linked SYSTEM processes >> exit(P1, kill), >> exit(P2, kill), >> ... >> exit(Pn, kill), >> >> %%Here c is still alive, and we want c and its linked non-system processes >> also killed, so... >> exit(kill). %% or exit(self(), kill), or exit(anything).. >> >> End code. >> >> However, it still demands that programmers know ahead of time how many and >> which system processes are linked to c, which is not a good design pattern. >> >> Sincerely, >> >> Yan Yan >> >> >> >> >> Yan Yan >> 2009-10-24 >> >> >> >> From: Richard Carlsson >> Time: ?2009-10-24 21:05:05 >> To: Yan Yan >> Fw: erlang-questions; erlang-bugs; Bj?n_Gustavsson >> Subject: Re: [erlang-questions] Erlang's process not killed by exit >> signalreasoned by "kill" >> >> Yan Yan wrote: >> > (2)On Page 169, >> (Page 161 in my copy of the book.) >> > Quote: >> > >> > 8> edemo1:start(true, {die,kill}). ?Process b received >> > {'EXIT',<0.73.0>,kill} process b (<0.72.0>) is alive process c >> > (<0.73.0>) is dead ok >> > >> > End quote. >> > >> > Here a and b are both system processes, while c is not. When c exits >> > with the reason "kill" (not "killed"), it sends exit signal to b with >> > the reason "kill". Therefore b should be killed and dead, but b is >> > still alive here! >> > >> > (I had thought it was only a small typo in the book. But then I >> > tested by myself and got the same result: b received the exit signal >> > with the reason "kill" and b was not killed but alive.) >> > >> > Why is the system process b not killed by the exit signal with the >> > reason "kill"? >> Interesting. There seems to be a difference in behaviour (probably >> not intentional) between `exit(kill)' and `exit(self(),kill)'. >> A summary of the setup: we have three processes, linked in a chain. >> 'a' is always trapping, 'b' may or may not be trapping, and 'c' is >> the one who dies by calling exit(Reason). >> ?1. When 'b' is not trapping, and 'c' does exit(kill), we see a >> ? ? report from 'a' that 'b' died with reason 'killed'. >> ?2. When 'b' is trapping, and 'c' does exit(kill), 'b' survives >> ? ? and reports that it sees the exit reason 'kill' from 'c' >> ? ? (not 'killed'). >> The question is: in case 2, why didn't 'b' die, when it apparently >> got a 'kill' message. This should be untrappable, which is why it is >> changed to 'killed' when it is propagated. But it was 'c' who died, >> so why wasn't the atom changed to 'killed'? >> If we change `exit(kill)' to `exit(self(),kill)' in 'c', we get the >> effect we expected: 'b' survives and reports that it sees 'killed' >> as the exit reason from 'c'. >> Then, a new question is why 'a' saw 'killed' in case 1 when 'b' is >> non-trapping. If 'c' dies in the same way in both, then doesn't >> 'b' get the same signal from 'c'? Apparently, it does, but since >> 'b' is not trapping, it doesn't matter what the atom is as long >> as it is something else than 'normal'. So 'b' dies due to an >> incoming 'kill', and this is then propagated as 'killed'. >> It seems that when a process does exit(kill) on itself, it causes >> a different outgoing signal than if it does exit(self(),kill). >> The former is not untrappable even though it has the reason 'kill', >> but if that in its turn causes another process to die, the atom >> will _then_ be rewritten to 'killed'. >> To me, it seems that this should be fixed so that exit(kill), even >> if it's an unusual case, should be propagated as 'killed'. >> ? ? /Richard >> > From steven.charles.davis@REDACTED Wed Oct 28 23:18:21 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 28 Oct 2009 15:18:21 -0700 (PDT) Subject: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Message-ID: <6caf2fa5-dd5f-4544-b86f-dee142ee015d@g27g2000yqn.googlegroups.com> I believe the art of using the PD is to stick rigorously to "write once". /s From roberto@REDACTED Thu Oct 29 04:22:07 2009 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 29 Oct 2009 04:22:07 +0100 Subject: [erlang-questions] mnesia vs ets In-Reply-To: <4AE7F988.1050102@erlang-consulting.com> References: <2a67d3ff0910271759k6480bf9bm55145f193fea14c5@mail.gmail.com> <4AE7F988.1050102@erlang-consulting.com> Message-ID: > dirty_read adds 2 other things (two additional ets lookups) where > is table located (which node) and the stoarge type, ets or dets. > > So dirty_read abstracts away the location and storage type which you > implicit knows > when calling ets:lookup/2. thank you dan, that clears it up. 2009/10/28 Ulf Wiger : > In some cases, this flexibility is not worth as much as the raw > speed. Personally, I always start out using transactions, and > always using mnesia:activity(Type, F). I have very rarely had to > conclude that transactions weren't fast enough for my needs. > > BTW, dirty reads to _not_ support fragmented tables out of the > box. Neither does the standard mnesia:transaction(F). > You need to use mnesia:activity(Type, F, mnesia_frag). hi ulf, i do not get lost in premature optimization :) i have clear specifics that state a need for me to read 10,000 key/value pairs/sec. therefore, i'm benchmarking to see various speeds, so that i can then decide for the best speed/distribution/transaction tradeoff. i thought i might share the results with you: 1. mnesia table in 6 fragments distributed over 2 erlang nodes, created with: FragProps = [{node_pool, ['1@REDACTED', '2@REDACTED']}, {n_fragments, 6}, {n_disc_copies, 1}], mnesia:create_table(?TABLENAME, [{frag_properties, FragProps}, {attributes, record_info(fields, test)}]). - write [speed: 370.19 rows/min]: WriteFun = fun() -> [mnesia:dirty_write({?TABLENAME, K, {}}) || K <- lists:seq(1, Num)] end, mnesia:activity(sync_dirty, WriteFun, mnesia_frag). - read with transaction [speed: 361.10 rows/min]: ReadFun = fun(Key) -> mnesia:read({?TABLENAME, Key}) end, [mnesia:activity(transaction, ReadFun, [K], mnesia_frag) || K <- lists:seq(1, ?NUM)]. - read dirty [speed: 552.25 rows/min]: ReadFun = fun(Key) -> mnesia:read({?TABLENAME, Key}) end, [mnesia:activity(sync_dirty, ReadFun, [K], mnesia_frag) || K <- lists:seq(1, ?NUM)]. 2. mnesia table in 6 fragments over a single erlang node, created with: FragProps = [{node_pool, ['1@REDACTED']}, {n_fragments, 6}, {n_disc_copies, 1}], mnesia:create_table(?TABLENAME, [{frag_properties, FragProps}, {attributes, record_info(fields, test)}]). - write [speed: 3451.64 rows/min]: WriteFun = fun() -> [mnesia:dirty_write({?TABLENAME, K, {}}) || K <- lists:seq(1, Num)] end, mnesia:activity(sync_dirty, WriteFun). - read with transaction [speed: 947.47 rows/min]: ReadFun = fun(Key) -> mnesia:read({?TABLENAME, Key}) end, [mnesia:activity(transaction, ReadFun, [K], mnesia_frag) || K <- lists:seq(1, ?NUM)]. - read dirty [speed: 10900.37 rows/min rows/min]: ReadFun = fun(Key) -> mnesia:read({?TABLENAME, Key}) end, [mnesia:activity(sync_dirty, ReadFun, [K], mnesia_frag) || K <- lists:seq(1, ?NUM)]. 3. mnesia table no fragments on single node, created with: mnesia:create_table(?TABLENAME, [{disc_copies, ['1@REDACTED', '2@REDACTED']}, {attributes, record_info(fields, test)}]). - write [speed: 3545.29 rows/min]: WriteFun = fun() -> [mnesia:dirty_write({?TABLENAME, K, {}}) || K <- lists:seq(1, Num)] end, mnesia:activity(sync_dirty, WriteFun). - read with transaction [speed: 2148.44 rows/min]: [mnesia:transaction(fun(K) -> mnesia:read({?TABLENAME, K}) end) || K <- lists:seq(1, ?NUM)]. - read dirty [speed: 29384.40 rows/min]: [mnesia:dirty_read(?TABLENAME, K) || K <- lists:seq(1, ?NUM)]. 4. mnesia table with no fragments on single node. - read [speed: 91975.16 rows/min] interestingly enough: a. write on mnesia table no fragments on single node is same speed than with fragments. b. read [transaction/dirty] on mnesia table without fragments is 3x faster than with fragments. c. using ets to read a disc_copy not fragmented mnesia table is 3x faster than using mnesia dirty_write. does this meet your expectations? any comments always welcome :) r. From ngocdaothanh@REDACTED Thu Oct 29 06:26:25 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Thu, 29 Oct 2009 14:26:25 +0900 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <6caf2fa5-dd5f-4544-b86f-dee142ee015d@g27g2000yqn.googlegroups.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <6caf2fa5-dd5f-4544-b86f-dee142ee015d@g27g2000yqn.googlegroups.com> Message-ID: <5c493e530910282226l3770ed00ved9f19a56c2525bc@mail.gmail.com> Yes, because of "write once", I think the trade-off of using process dictionary for web is OK. For proplist, is there a trick (macro?) to add syntactic sugar to put and get? List2 = [{Key, Value} | List] and proplists:get_value(Key, List) are somewhat verbose. On Thu, Oct 29, 2009 at 7:18 AM, Steve Davis wrote: > I believe the art of using the PD is to stick rigorously to "write > once". > > /s From ulf.wiger@REDACTED Thu Oct 29 10:57:37 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 29 Oct 2009 10:57:37 +0100 Subject: [erlang-bugs] Re: Re: [erlang-questions] Erlang's process not killed by exit signalreasoned by "kill" In-Reply-To: <9b08084c0910281432m134e4f98p4154eb33ca224f52@mail.gmail.com> References: <200910241907167657586@gmail.com> <200910242342216873502@gmail.com> <3dbc6d1c0910241310p53a30c23m58ba263d271f99fe@mail.gmail.com> <9b08084c0910281432m134e4f98p4154eb33ca224f52@mail.gmail.com> Message-ID: <4AE96711.9020303@erlang-consulting.com> Joe Armstrong wrote: > > The distinction between messages and signals is subtle. One way to verify that it is indeed as Joe says: Eshell V5.6.5 (abort with ^G) 1> P = spawn(fun() -> timer:sleep(infinity) end). <0.32.0> 2> P ! {'EXIT',self(),foo}. {'EXIT',<0.30.0>,foo} 3> is_process_alive(P). true 4> exit(P,foo). true 5> is_process_alive(P). false 6> In other words, you cannot fake an exit signal by sending an {'EXIT',P,R} message. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From nem@REDACTED Thu Oct 29 12:39:59 2009 From: nem@REDACTED (Geoff Cant) Date: Thu, 29 Oct 2009 12:39:59 +0100 Subject: [erlang-questions] Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> (Ngoc Dao's message of "Wed, 28 Oct 2009 20:46:41 +0900") References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Message-ID: Ngoc Dao writes: > Hi, > > Web frameworks are normally designed as layers (web server -> > middleware -> front controller -> controller -> action -> view -> > etc.). Data needs to be passed from one layer to another. There are 2 > ways to pass: > 1. Proplist (environment variables) > 2. Process dictionary > > The 2nd way: > * Is Simple and natural in Erlang because normally one HTTP request is > processed by one process. > * Makes application code which uses the framework appear to be clean, > because application developer does not have to manually pass an ugly > proplist arround and arround. > I want to ask about the (memory, CPU etc.) overhead of process > dictionary, compared to proplist. Which way should be used in a web > framework? I would strongly advise against using the process dictionary to pass data between part of an erlang web framework.[1] You say that "normally one HTTP request is processed by one process" - by using the process dictionary you require that this is the case for all code that uses your framework. By making this decision, you are tying the hands of the users of your framework. They can no longer choose the process model that suits their problem and must use a one process per request design. In erlang it's quite common to hand requests off to other processes, possibly on other nodes, for execution to balance load, to move computation closer to needed resources, to turn synchronous tasks into asynchronous ones, to alter process memory profile, to isolate failures and so on. The use of the process dictionary precludes all these approaches. You also say that using the process dictionary "makes application code which uses the framework appear to be clean". From the point of view of the maintenance programmer, nothing could be further from the truth. Good erlang code does not use the process dictionary. Erlang programmers usually only have to think about the function body and arguments to work out what its going to do. Sprinkling 'get' and 'put' through the code means that an erlang programmer trying to understand your code now has to read all the code to figure out why something is happening. The order in which functions are called becomes important. The behaviour of functions in other modules becomes important because now there's a back-channel to propagate bugs, er, state between parts of the code. As a (curmudgeonly) future web framework user, I would almost certainly not choose a framework based on the use of non-erlangy features[2] such as the process dictionary firstly because the code would be more difficult to understand when someone would need to maintain it and secondly because the process dictionary would prevent me from using a different process model if I needed to. Using a proplist or 'dict' or some opaque datastructure and an API module is the natural, erlangy way to solve your problem. Good luck with your framework, -- Geoff Cant [1] More generally, I would strongly advise against using the process dictionary. [2] Ditto for parameterized modules and hierarchal module names[3] [3] I'm already guilty of this, but promise not to do it again. From rvirding@REDACTED Thu Oct 29 14:36:37 2009 From: rvirding@REDACTED (Robert Virding) Date: Thu, 29 Oct 2009 14:36:37 +0100 Subject: [erlang-questions] Erlang's process not killed by exit signalreasoned by "kill" In-Reply-To: <9b08084c0910281432m134e4f98p4154eb33ca224f52@mail.gmail.com> References: <200910241907167657586@gmail.com> <200910242342216873502@gmail.com> <3dbc6d1c0910241310p53a30c23m58ba263d271f99fe@mail.gmail.com> <9b08084c0910281432m134e4f98p4154eb33ca224f52@mail.gmail.com> Message-ID: <3dbc6d1c0910290636u557f51e2y51e067d4d7f117fe@mail.gmail.com> 2009/10/28 Joe Armstrong > > ---- a different way of describing the same thing > > The VM behaves as if there were 3 type of inter-process messages: > > (1, Data) tuples represent regular messages > (2, Data) tuples represent signals > 3 is the kill signal > > P ! X sends a type 1 message (1,X) > exit(X) sends a type 2 message (2,X) > exit(P, kill) sends a type 3 message > exit(P, X) sends a type 2 message (2,X) > > on receiving a type 2 message if X != normal a normal process dies > on receiving a type 2 message a system process converts this to a (1,X) > message > on receiving a type 3 message the receiving process dies unconditionally > > --- end > Yes, this is clear and understandable. In these terms I suppose you could describe the problem as: When I do exit(Self, kill) it sends a type 3 'kill' signal to myself and when I die I send type 2 'killed' signals to my linked processes. This is consistent irrespective to whom I send the kill signal. BUT when I do exit(kill) I send a type 2 signal with value 'kill' which doesn't behave as a type 3 'kill' signal though the name is the same. This is inconsistent, and I don't like inconsistencies. There are two different solutions: either send out real type 3 kill signals; or send out type 2 'killed' signals as if I had received a type 3 signal. Both are consistent, but which is best? This is how I have read the problem. Robert From ulf.wiger@REDACTED Thu Oct 29 15:04:24 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 29 Oct 2009 15:04:24 +0100 Subject: [erlang-bugs] Re: Re: [erlang-questions] Erlang's process not killed by exit signalreasoned by "kill" In-Reply-To: <3dbc6d1c0910290636u557f51e2y51e067d4d7f117fe@mail.gmail.com> References: <200910241907167657586@gmail.com> <200910242342216873502@gmail.com> <3dbc6d1c0910241310p53a30c23m58ba263d271f99fe@mail.gmail.com> <9b08084c0910281432m134e4f98p4154eb33ca224f52@mail.gmail.com> <3dbc6d1c0910290636u557f51e2y51e067d4d7f117fe@mail.gmail.com> Message-ID: <4AE9A0E8.7050101@erlang-consulting.com> Robert Virding wrote: > > BUT when I do exit(kill) I send a type 2 signal with value 'kill' which > doesn't behave as a type 3 'kill' signal though the name is the same. This > is inconsistent, and I don't like inconsistencies. There are two different > solutions: either send out real type 3 kill signals; or send out type 2 > 'killed' signals as if I had received a type 3 signal. Both are consistent, > but which is best? Why send out type 3 kill signals? What is that consistent with? I think the consistent behaviour would be to send out type 2 'killed' signals. BR, Ulf W -- Ulf Wiger CTO, Erlang Training & Consulting Ltd http://www.erlang-consulting.com From tony@REDACTED Thu Oct 29 14:15:44 2009 From: tony@REDACTED (Tony Rogvall) Date: Thu, 29 Oct 2009 14:15:44 +0100 Subject: [erlang-questions] Re: [erlang-bugs] Re: Re: [erlang-questions] Erlang's process not killed by exit signalreasoned by "kill" In-Reply-To: <4AE96711.9020303@erlang-consulting.com> References: <200910241907167657586@gmail.com> <200910242342216873502@gmail.com> <3dbc6d1c0910241310p53a30c23m58ba263d271f99fe@mail.gmail.com> <9b08084c0910281432m134e4f98p4154eb33ca224f52@mail.gmail.com> <4AE96711.9020303@erlang-consulting.com> Message-ID: <983FFB57-9C47-42AD-8192-D485663A992A@rogvall.se> Just to clarify some details about exit signals and trap_exit problems. You can fake exit signals if the process is trapping exits (Also the name of a web site ;-) In the code below this is illustrated by starting a process that trap exits. Then either do a exit(P, foo) in test1 or P ! {'EXIT',self(),foo} in test2. Also note that setting the process_flag(trap_exit,true) must be synchronized!!! (yes I stumbled right into it ;-) Trap exits should be avoided when not 100% designed into the solution. /Tony -module(trap). -compile(export_all). start() -> Main = self(), P = spawn(fun() -> process_flag(trap_exit, true), Main ! ok, receive {'EXIT',From,Reason} -> io:format("Exit: ~p ~p\n",[From,Reason]) end end), receive ok -> P end. test1() -> P = start(), exit(P, foo). test2() -> P = start(), P ! {'EXIT',self(),foo}. On 29 okt 2009, at 10.57, Ulf Wiger wrote: > Joe Armstrong wrote: >> >> The distinction between messages and signals is subtle. > > One way to verify that it is indeed as Joe says: > > Eshell V5.6.5 (abort with ^G) > 1> P = spawn(fun() -> timer:sleep(infinity) end). > <0.32.0> > 2> P ! {'EXIT',self(),foo}. > {'EXIT',<0.30.0>,foo} > 3> is_process_alive(P). > true > 4> exit(P,foo). > true > 5> is_process_alive(P). > false > 6> > > In other words, you cannot fake an exit signal by sending > an {'EXIT',P,R} message. > > BR, > Ulf W > -- > Ulf Wiger > CTO, Erlang Training & Consulting Ltd > http://www.erlang-consulting.com > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From kagato@REDACTED Thu Oct 29 21:50:40 2009 From: kagato@REDACTED (Jayson Vantuyl) Date: Thu, 29 Oct 2009 15:50:40 -0500 Subject: [erlang-questions] Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Message-ID: I'd say no, for a few reasons. 1. It makes it difficult if you ever do need to split stuff up into multiple processes. As soon as you need a "helper" process, all of your data is inaccessible. 2. It makes it difficult to debug via tracing. You can trace function calls, tracing changes to the process dictionary is a bit more hairy. 3. If you want to "hide" the proplist, just pass around some sort of "request" record. Then you don't see it unless you access the record. Or use macros and message a "request" process for the data. 4. If you use Mnesia, transactions can be retried, so you may end up with your mutations happening multiple times if anything happens inside of a transaction. I don't deny that Erlang needs some better syntax and conventions for passing data around (preferably via namespace magic like Ruby or Python), but the process dictionary can break a quite a few assumptions and break quite a few useful patterns. This could be fine if you're the only one writing the code, but you the process dictionary, in the wrong hands, can make many an Erlanger curse your name. On Oct 28, 2009, at 6:46 AM, Ngoc Dao wrote: > Hi, > > Web frameworks are normally designed as layers (web server -> > middleware -> front controller -> controller -> action -> view -> > etc.). Data needs to be passed from one layer to another. There are 2 > ways to pass: > 1. Proplist (environment variables) > 2. Process dictionary > > The 2nd way: > * Is Simple and natural in Erlang because normally one HTTP request is > processed by one process. > * Makes application code which uses the framework appear to be clean, > because application developer does not have to manually pass an ugly > proplist arround and arround. > > I want to ask about the (memory, CPU etc.) overhead of process > dictionary, compared to proplist. Which way should be used in a web > framework? > > Thanks. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > -- Jayson Vantuyl kagato@REDACTED From steven.charles.davis@REDACTED Thu Oct 29 22:13:31 2009 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 29 Oct 2009 14:13:31 -0700 (PDT) Subject: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Message-ID: On Oct 29, 6:39?am, Geoff Cant wrote: > Good erlang code does not use the process dictionary. Note that both gen_server and wx libraries make use of the PD. /s From vinoski@REDACTED Thu Oct 29 23:12:42 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Thu, 29 Oct 2009 18:12:42 -0400 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Message-ID: <65b2728e0910291512y1577168br4759540e2726879e@mail.gmail.com> On Thu, Oct 29, 2009 at 5:13 PM, Steve Davis wrote: > > > On Oct 29, 6:39 am, Geoff Cant wrote: > > > Good erlang code does not use the process dictionary. > > Note that both gen_server and wx libraries make use of the PD. As does Yaws. The fact that a new process spawned by a request handler process can't access the Yaws data stored in the request handler process's dictionary sometimes comes up as an issue, but all in all it's not a frequent problem. Still, Klacke or I will soon be adding a function to Yaws to copy the necessary data into the process dictionary of a new process to allow users to avoid this problem, but again, from what I've seen the problem doesn't come up all that often in practice. --steve From fedora789@REDACTED Thu Oct 29 23:12:45 2009 From: fedora789@REDACTED (joe lee) Date: Thu, 29 Oct 2009 15:12:45 -0700 Subject: Emacs Distel error "Module is not interpreted, can't set breakpoints." after interpreting In-Reply-To: <35eed8d00910291418h33713621o8cba41f8ba29f726@mail.gmail.com> References: <35eed8d00910291418h33713621o8cba41f8ba29f726@mail.gmail.com> Message-ID: <35eed8d00910291512s4336e745hc10f6a16117768ae@mail.gmail.com> Hi, I used instructions on http://bc.tech.coop/blog/070528.html to setup distel debugging. I kept getting error "Module is not interpreted, can't set breakpoints." after model interpreting. Here's the details: ======================================== Environment: GNU Emacs 23.1.1 (x86_64-redhat-linux-gnu, GTK+ Version 2.16.5) of 2009-08-03 on x86-4.fedora.phx.redhat.com Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] ( switching to Erlang R12 5.6.5 [source] [64-bit] also got the same error) Linux Fedora release 11: fc11.x86_64 Distel from Repository Root: http://distel.googlecode.com/svn Revision: 93 Last Changed Date: 2009-08-02 16:38:33 -0700 (Sun, 02 Aug 2009) ======================================== Steps: * compile with debug info (emacs@REDACTED)> c(test, debug_info). ======================================== * connect emacs to erlang node emacs@REDACTED ( C-c C-d n ) ======================================== * distel reload erlang module ( C-c C-d L) after this step I can see distel in the erlang node. (emacs@REDACTED)6> i(). ... ... <0.33.0> supervisor:kernel/1 233 57 0 kernel_safe_sup gen_server:loop/6 9 <0.36.0> erlang:apply/2 2584 40333 0 rpc:local_call/3 49 <0.46.0> inet_tcp_dist:do_accept/6 610 516 0 dist_util:con_loop/9 11 <0.50.0> distel:gl_proxy/1 233 1 0 'distel_gl_for_<8785. distel:gl_proxy/1 4 Total 28992 812442 0 286 ok ======================================== * Toggle debug interpreting of the module ( C-c C-d i) got messsage : Interpreting: test got distel:debug_subscriber* in i() output (emacs@REDACTED)7> i(). ... ... <0.46.0> inet_tcp_dist:do_accept/6 987 803 0 dist_util:con_loop/9 11 <0.50.0> distel:gl_proxy/1 233 1 0 'distel_gl_for_<8785. distel:gl_proxy/1 4 <0.53.0> distel:debug_subscriber_init/2 233 37 0 distel:debug_subscriber/1 5 <0.55.0> dbg_iserver:init/1 377 104 0 dbg_iserver gen_server:loop/6 9 Total 29945 901441 0 303 ======================================== * Toggle a breakpoint at current line ( C-c SPC) got messsage : Module is not interpreted, can't set breakpoints. When distel debugging work, i() should show two new process distel:attach_init/2, distel:attach_loop/1, but now it doesn't: (emacs@REDACTED)8> i(). ... ... <0.46.0> inet_tcp_dist:do_accept/6 987 1248 0 dist_util:con_loop/9 11 <0.50.0> distel:gl_proxy/1 233 1 0 'distel_gl_for_<8785. distel:gl_proxy/1 4 <0.53.0> distel:debug_subscriber_init/2 233 37 0 distel:debug_subscriber/1 5 <0.55.0> dbg_iserver:init/1 377 104 0 dbg_iserver gen_server:loop/6 9 Total 30322 967176 1 ~/.erlang.cookie has been set, erlang:get_cookie() returns correctly. I tried reload and interpret the module multiple times and also switched to erlang R12, still got the same result. I tried to connect to a erlang node outside of emacs, emacs showed a popup windows "Attach debugger to xyz@REDACTED instead of emacs@REDACTED" when interpreting, but still got the same error "Module is not interpreted, can't set breakpoints." Does anyone know how to get information to troubleshooting this problem ? Thanks a lot. John From nem@REDACTED Fri Oct 30 01:34:31 2009 From: nem@REDACTED (Geoff Cant) Date: Fri, 30 Oct 2009 01:34:31 +0100 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: (Steve Davis's message of "Thu, 29 Oct 2009 14:13:31 -0700 (PDT)") References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Message-ID: Steve Davis writes: > On Oct 29, 6:39?am, Geoff Cant wrote: > >> Good erlang code does not use the process dictionary. > > Note that both gen_server and wx libraries make use of the PD. > I think you'll find that gen_server makes almost no use of the process dictionary. The gen_server code itself makes only one reference to the process dictionary - to pass the pid of the parent process between the proc_lib:init stage and the gen_server:enter_loop stage through arbitrary intervening user code. proc_lib only uses the process dictionary to store the initial_call and parent pids of the process. The initial_call is only used in exit reports and the ancestor/parent pid information is used mainly in exit reports, though also occasionally by appmon to draw the process ancestry tree and httpc_manager to implement the is_inets_manager() function. I don't think the minimal use of the process dictionary in either of these cases undermines the case that good erlang code does not use the process dictionary. Occasionally you might have to, to work around historic code that you can't refactor for backwards compatibility reasons, or for some particular debugging cases. But that doesn't fall into my definition of good - just the best possible given the constraints. New erlang code should strive to avoid the process dictionary if at all possible. In the case of 'wx', the process dictionary is used to pass around a #wx_env{} structure - I'm not quite sure why this wasn't just a parameter instead. Maybe there's something else (like port ownership?) that makes tying the structure to the process in the process dict make sense? I'm kinda curious about this one - if there's a good argument to be made for using the process dictionary, I'll amend or retract my anti-process-dictionary bigotry :) Cheers, -- Geoff Cant From dionne@REDACTED Fri Oct 30 00:48:11 2009 From: dionne@REDACTED (Robert Dionne) Date: Thu, 29 Oct 2009 19:48:11 -0400 Subject: [erlang-questions] Emacs Distel error "Module is not interpreted, can't set breakpoints." after interpreting In-Reply-To: <35eed8d00910291512s4336e745hc10f6a16117768ae@mail.gmail.com> References: <35eed8d00910291418h33713621o8cba41f8ba29f726@mail.gmail.com> <35eed8d00910291512s4336e745hc10f6a16117768ae@mail.gmail.com> Message-ID: it can be finnicky. Some things to try: the version that is in github. I forked this one: http://github.com/kunley/distel use -sname when starting and specify that name when connecting does C-c C-d l work? It show a list of processes. who calls acts up (C-c C-d w) and I find the best way to make it work is to call the function once directly from the erl vm, .eg. distel:who_calls(my_module,my_fun,arity). I'm running R12B01 on OS X 10.6 Cheers, Bob On Oct 29, 2009, at 6:12 PM, joe lee wrote: > Hi, I used instructions on http://bc.tech.coop/blog/070528.html to > setup distel debugging. I kept getting error "Module is not > interpreted, can't set breakpoints." after model interpreting. Here's > the details: > > ======================================== > > Environment: > > GNU Emacs 23.1.1 (x86_64-redhat-linux-gnu, GTK+ Version 2.16.5) > of 2009-08-03 on x86-4.fedora.phx.redhat.com > > Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > ( switching to Erlang R12 5.6.5 [source] [64-bit] also got the same > error) > > Linux Fedora release 11: fc11.x86_64 > > Distel from Repository Root: http://distel.googlecode.com/svn > Revision: 93 > Last Changed Date: 2009-08-02 16:38:33 -0700 (Sun, 02 Aug 2009) > > ======================================== > > Steps: > > * compile with debug info > > (emacs@REDACTED)> c(test, debug_info). > > ======================================== > > * connect emacs to erlang node emacs@REDACTED ( C-c C-d n ) > > ======================================== > > > * distel reload erlang module ( C-c C-d L) > > after this step I can see distel in the erlang node. > > (emacs@REDACTED)6> i(). > ... ... > <0.33.0> supervisor:kernel/1 > 233 57 0 > kernel_safe_sup gen_server:loop/6 9 > <0.36.0> erlang:apply/2 2584 > 40333 0 > rpc:local_call/3 49 > <0.46.0> inet_tcp_dist:do_accept/6 > 610 516 0 > dist_util:con_loop/9 11 > <0.50.0> distel:gl_proxy/1 > 233 1 0 > 'distel_gl_for_<8785. distel:gl_proxy/1 4 > Total 28992 > 812442 0 > 286 > ok > > > ======================================== > > * Toggle debug interpreting of the module ( C-c C-d i) > got messsage : Interpreting: test > got distel:debug_subscriber* in i() output > > > (emacs@REDACTED)7> i(). > > ... ... > <0.46.0> inet_tcp_dist:do_accept/6 > 987 803 0 > dist_util:con_loop/9 11 > <0.50.0> distel:gl_proxy/1 > 233 1 0 > 'distel_gl_for_<8785. distel:gl_proxy/1 4 > <0.53.0> distel:debug_subscriber_init/2 > 233 37 0 > distel:debug_subscriber/1 5 > <0.55.0> dbg_iserver:init/1 > 377 104 0 > dbg_iserver gen_server:loop/6 9 > Total 29945 > 901441 0 > 303 > > ======================================== > > * Toggle a breakpoint at current line ( C-c SPC) > got messsage : Module is not interpreted, can't set breakpoints. > > When distel debugging work, i() should show two new process > distel:attach_init/2, > distel:attach_loop/1, > but now it doesn't: > > > (emacs@REDACTED)8> i(). > ... ... > > > <0.46.0> inet_tcp_dist:do_accept/6 987 > 1248 0 > dist_util:con_loop/9 11 > <0.50.0> distel:gl_proxy/1 > 233 1 0 > 'distel_gl_for_<8785. distel:gl_proxy/1 4 > <0.53.0> distel:debug_subscriber_init/2 > 233 37 0 > distel:debug_subscriber/1 5 > <0.55.0> dbg_iserver:init/1 > 377 104 0 > dbg_iserver gen_server:loop/6 9 > Total 30322 > 967176 1 > > > ~/.erlang.cookie has been set, erlang:get_cookie() returns correctly. > > I tried reload and interpret the module multiple times and also > switched to erlang R12, still got the same result. > > I tried to connect to a erlang node outside of emacs, emacs showed a > popup windows "Attach debugger to xyz@REDACTED instead of emacs@REDACTED" when > interpreting, but still got the same error "Module is not interpreted, > can't set breakpoints." > > Does anyone know how to get information to troubleshooting this > problem ? > > Thanks a lot. > > John > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > From ngocdaothanh@REDACTED Fri Oct 30 03:06:02 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 30 Oct 2009 11:06:02 +0900 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <65b2728e0910291512y1577168br4759540e2726879e@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <65b2728e0910291512y1577168br4759540e2726879e@mail.gmail.com> Message-ID: <5c493e530910291906m17e4d10aw371185015a6b5bb8@mail.gmail.com> Steve, So Yaws will stick with process dictionary? Actually I am creating this: http://github.com/ngocdaothanh/ale It only adds some routing rules and MVC conventions to make Yaws easier to use. A typical Ale application would have a front controller behind Yaws' back. Behind Yaws' back, either process dictionary or proplist can be used, and I want to decide which one to use. Ngoc. On Fri, Oct 30, 2009 at 7:12 AM, Steve Vinoski wrote: > As does Yaws. The fact that a new process spawned by a request handler > process can't access the Yaws data stored in the request handler process's > dictionary sometimes comes up as an issue, but all in all it's not a > frequent problem. Still, Klacke or I will soon be adding a function to Yaws > to copy the necessary data into the process dictionary of a new process to > allow users to avoid this problem, but again, from what I've seen the > problem doesn't come up all that often in practice. > > --steve From vinoski@REDACTED Fri Oct 30 04:23:26 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Thu, 29 Oct 2009 23:23:26 -0400 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <5c493e530910291906m17e4d10aw371185015a6b5bb8@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <65b2728e0910291512y1577168br4759540e2726879e@mail.gmail.com> <5c493e530910291906m17e4d10aw371185015a6b5bb8@mail.gmail.com> Message-ID: <65b2728e0910292023g4b6a23e9p3c4af9b27b22d42e@mail.gmail.com> On Thu, Oct 29, 2009 at 10:06 PM, Ngoc Dao wrote: > Steve, > > So Yaws will stick with process dictionary? > To the best of my knowledge, yes. > Actually I am creating this: > http://github.com/ngocdaothanh/ale > > It only adds some routing rules and MVC conventions to make Yaws > easier to use. What parts are difficult to use? > A typical Ale application would have a front controller > behind Yaws' back. Behind Yaws' back, either process dictionary or > proplist can be used, and I want to decide which one to use. > Klacke of course has the final say when it comes to Yaws, but I'm pretty certain there are no plans to move it away from using the process dictionary. --steve From ngocdaothanh@REDACTED Fri Oct 30 05:51:21 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 30 Oct 2009 13:51:21 +0900 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <65b2728e0910292023g4b6a23e9p3c4af9b27b22d42e@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <65b2728e0910291512y1577168br4759540e2726879e@mail.gmail.com> <5c493e530910291906m17e4d10aw371185015a6b5bb8@mail.gmail.com> <65b2728e0910292023g4b6a23e9p3c4af9b27b22d42e@mail.gmail.com> Message-ID: <5c493e530910292151g5d77fe97k854529aa376743e6@mail.gmail.com> > What parts are difficult to use? If Yaws aims at the application level, then it lacks the sense of web application framework like Sinatra, Merb, Rails etc. If Yaws aims at the web server level, then everything is OK: * The .yaws file part is like Apache CGI * The appmods part is like Java Servlet * Based on the powerful bare bone Yaws provides, higher level framework like Nitrogen can be *easily* constructed Ngoc On Fri, Oct 30, 2009 at 12:23 PM, Steve Vinoski wrote: > > On Thu, Oct 29, 2009 at 10:06 PM, Ngoc Dao wrote: >> >> Steve, >> >> So Yaws will stick with process dictionary? > > To the best of my knowledge, yes. > >> >> Actually I am creating this: >> http://github.com/ngocdaothanh/ale >> >> It only adds some routing rules and MVC conventions to make Yaws >> easier to use. > > What parts are difficult to use? > >> >> A typical Ale application would have a front controller >> behind Yaws' back. Behind Yaws' back, either process dictionary or >> proplist can be used, and I want to decide which one to use. > > Klacke of course has the final say when it comes to Yaws, but I'm pretty > certain there are no plans to move it away from using the process > dictionary. > --steve From jacob.vorreuter@REDACTED Fri Oct 30 06:29:00 2009 From: jacob.vorreuter@REDACTED (Jacob Vorreuter) Date: Thu, 29 Oct 2009 22:29:00 -0700 Subject: throttling the cpu/memory available to the VM Message-ID: Is it possible to specify a max memory usage and cpu usage for the VM when the runtime system starts? I would like to be able to run multiple VMs on the same system and ensure equal distribution of resources. From max.lapshin@REDACTED Fri Oct 30 06:39:04 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 30 Oct 2009 08:39:04 +0300 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <5c493e530910292151g5d77fe97k854529aa376743e6@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <65b2728e0910291512y1577168br4759540e2726879e@mail.gmail.com> <5c493e530910291906m17e4d10aw371185015a6b5bb8@mail.gmail.com> <65b2728e0910292023g4b6a23e9p3c4af9b27b22d42e@mail.gmail.com> <5c493e530910292151g5d77fe97k854529aa376743e6@mail.gmail.com> Message-ID: Damn, I've programming Rails for many years and I really don't see any reason to use global hash tables! Explicitly passed data is very, very convenient and predictive way to glue SEPARATED layers. Using of PD in gen_server is its own internal way to live, I haven't seen this PD outside and I'm not going to think about it. But using PD to pass variables from controller layer to view layer in erlang is a VERY, VERY bad way of programming. It is a sufficient reason not to use such software at all, because it is a sign of very bad quality of other code. Explicit passing of data generated in controller, to view is: a) clear to understand b) clear to hook and modify, cache, etc. c) testable d) separatable (you may move templates from erlang to other application server) PD is: a) unclear, what is required and what is passed to template b) unmodifieable and unhookable. Business logic migrates to templates c) very, very hard to test From rapsey@REDACTED Fri Oct 30 06:41:25 2009 From: rapsey@REDACTED (Rapsey) Date: Fri, 30 Oct 2009 06:41:25 +0100 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <5c493e530910292151g5d77fe97k854529aa376743e6@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <65b2728e0910291512y1577168br4759540e2726879e@mail.gmail.com> <5c493e530910291906m17e4d10aw371185015a6b5bb8@mail.gmail.com> <65b2728e0910292023g4b6a23e9p3c4af9b27b22d42e@mail.gmail.com> <5c493e530910292151g5d77fe97k854529aa376743e6@mail.gmail.com> Message-ID: <97619b170910292241s3a36586er7bb3630ed2e7f685@mail.gmail.com> Speaking of nitrogen, they use the process dictionary quite a bit. I think the PD is kind of like goto. If you use it wisely, you won't have any problems. If you abuse it, you're making a huge mess. Sergej On Fri, Oct 30, 2009 at 5:51 AM, Ngoc Dao wrote: > > What parts are difficult to use? > > If Yaws aims at the application level, then it lacks the sense of web > application framework like Sinatra, Merb, Rails etc. > > If Yaws aims at the web server level, then everything is OK: > * The .yaws file part is like Apache CGI > * The appmods part is like Java Servlet > * Based on the powerful bare bone Yaws provides, higher level > framework like Nitrogen can be *easily* constructed > > Ngoc > > > On Fri, Oct 30, 2009 at 12:23 PM, Steve Vinoski wrote: > > > > On Thu, Oct 29, 2009 at 10:06 PM, Ngoc Dao > wrote: > >> > >> Steve, > >> > >> So Yaws will stick with process dictionary? > > > > To the best of my knowledge, yes. > > > >> > >> Actually I am creating this: > >> http://github.com/ngocdaothanh/ale > >> > >> It only adds some routing rules and MVC conventions to make Yaws > >> easier to use. > > > > What parts are difficult to use? > > > >> > >> A typical Ale application would have a front controller > >> behind Yaws' back. Behind Yaws' back, either process dictionary or > >> proplist can be used, and I want to decide which one to use. > > > > Klacke of course has the final say when it comes to Yaws, but I'm pretty > > certain there are no plans to move it away from using the process > > dictionary. > > --steve > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ngocdaothanh@REDACTED Fri Oct 30 07:49:18 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 30 Oct 2009 15:49:18 +0900 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <65b2728e0910291512y1577168br4759540e2726879e@mail.gmail.com> <5c493e530910291906m17e4d10aw371185015a6b5bb8@mail.gmail.com> <65b2728e0910292023g4b6a23e9p3c4af9b27b22d42e@mail.gmail.com> <5c493e530910292151g5d77fe97k854529aa376743e6@mail.gmail.com> Message-ID: <5c493e530910292349o87a810cha79df70e59cbd47d@mail.gmail.com> Max, I agree that using PD arbitrarily is bad. But using PD restrictively is controllable. A web processing process has some unique properties: * It is normally short-lived, you want as high req/s as possible right? * PD are normally propagated one-way. If you want to spawn a new process, you can clone the PD. Take Ale for an example, see: http://github.com/ngocdaothanh/ale/blob/master/src/ale.erl * Things in PD are set only ONCE (well, app_add_head and app_add_js are accumalative). * Keys in PD are normally literal. You can easily track back where a value is set. * Keys in PD are namespaced, i.e: {app, title}. Things of Yaws, things of Ale, things of the app in PD are separated. * When you want to put a thing in PD from a controller, you use ale:app(Key, Value). When you want to get the thing out from a view, you use ale:app(Key). You don't spam the PD arbitrarily, you use the designated API. This way, PD is like environment variables of a Linux shell session. The environment is not for sharing mutable things, it is used for setting things. Ngoc On Fri, Oct 30, 2009 at 2:39 PM, Max Lapshin wrote: > Damn, I've programming Rails for many years and I really don't see any > reason to use global hash tables! > Explicitly passed data is very, very convenient and predictive way to > glue SEPARATED layers. > > Using of PD in gen_server is its own internal way to live, I haven't > seen this PD outside and I'm not going > to think about it. But using PD to pass variables from controller > layer to view layer in erlang is a VERY, VERY bad > way of programming. It is a sufficient reason not to use such software > at all, because it is a sign of very bad quality of other code. > > Explicit passing of data generated in controller, to view is: > a) clear to understand > b) clear to hook and modify, cache, etc. > c) testable > d) separatable (you may move templates from erlang to other application server) > > PD is: > a) unclear, what is required and what is passed to template > b) unmodifieable and unhookable. Business logic migrates to templates > c) very, very hard to test > From max.lapshin@REDACTED Fri Oct 30 07:51:42 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 30 Oct 2009 09:51:42 +0300 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <5c493e530910292349o87a810cha79df70e59cbd47d@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <65b2728e0910291512y1577168br4759540e2726879e@mail.gmail.com> <5c493e530910291906m17e4d10aw371185015a6b5bb8@mail.gmail.com> <65b2728e0910292023g4b6a23e9p3c4af9b27b22d42e@mail.gmail.com> <5c493e530910292151g5d77fe97k854529aa376743e6@mail.gmail.com> <5c493e530910292349o87a810cha79df70e59cbd47d@mail.gmail.com> Message-ID: This bad practice prevents from any clean way of injecting into chain of rendering HTML. When all data, required for rendering, is passed explicitly, it is possible to intercept it in filters and change. From ngocdaothanh@REDACTED Fri Oct 30 08:07:16 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 30 Oct 2009 16:07:16 +0900 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <65b2728e0910291512y1577168br4759540e2726879e@mail.gmail.com> <5c493e530910291906m17e4d10aw371185015a6b5bb8@mail.gmail.com> <65b2728e0910292023g4b6a23e9p3c4af9b27b22d42e@mail.gmail.com> <5c493e530910292151g5d77fe97k854529aa376743e6@mail.gmail.com> <5c493e530910292349o87a810cha79df70e59cbd47d@mail.gmail.com> Message-ID: <5c493e530910300007h5f99498id7435b838e59d14e@mail.gmail.com> In order to intercept, I think the explicit thing you need to know is the processing flow. For example: browser -> yaws -> front controller -> app-wide filter -> controller-wide filter -> action -> view -> layout -> front controller -> yaws -> browser The environment may be transfered implicitly inside the PD along the way. I don't know if this is relevant to this discussion, but I am creating this app which uses Ale which uses Yaws which uses PD: http://github.com/ngocdaothanh/khale Nitrogen is another example of the (successful?) use of PD. Ngoc On Fri, Oct 30, 2009 at 3:51 PM, Max Lapshin wrote: > This bad practice prevents from any clean way of injecting into chain > of rendering HTML. > When all data, required for rendering, is passed explicitly, it is > possible to intercept it in filters > and change. > From max.lapshin@REDACTED Fri Oct 30 08:09:25 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 30 Oct 2009 10:09:25 +0300 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <5c493e530910300007h5f99498id7435b838e59d14e@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <65b2728e0910291512y1577168br4759540e2726879e@mail.gmail.com> <5c493e530910291906m17e4d10aw371185015a6b5bb8@mail.gmail.com> <65b2728e0910292023g4b6a23e9p3c4af9b27b22d42e@mail.gmail.com> <5c493e530910292151g5d77fe97k854529aa376743e6@mail.gmail.com> <5c493e530910292349o87a810cha79df70e59cbd47d@mail.gmail.com> <5c493e530910300007h5f99498id7435b838e59d14e@mail.gmail.com> Message-ID: On Fri, Oct 30, 2009 at 10:07 AM, Ngoc Dao wrote: > In order to intercept, I think the explicit thing you need to know is > the processing flow. For example: > browser -> yaws -> front controller -> app-wide filter -> > controller-wide filter -> action -> view -> layout -> front controller > -> yaws -> browser Damn, You look at mutable Ruby and try to implement in immutable Erlang the same thing. after_filter in Rails can modify anything, including internal hidden instance variables. What can you do with set-once variables in Erlang? From dangud@REDACTED Fri Oct 30 08:14:45 2009 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 30 Oct 2009 08:14:45 +0100 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> Message-ID: <93df43b60910300014x1e1092c9ye1d6974000cc8dae@mail.gmail.com> On Fri, Oct 30, 2009 at 1:34 AM, Geoff Cant wrote: > > In the case of 'wx', the process dictionary is used to pass around a > #wx_env{} structure - I'm not quite sure why this wasn't just a > parameter instead. Maybe there's something else (like port ownership?) > that makes tying the structure to the process in the process dict make > sense? I'm kinda curious about this one - if there's a good argument to > be made for using the process dictionary, I'll amend or retract my > anti-process-dictionary bigotry :) My argument for having the 'env' in the process dictionary was that it is static, and that most of the calls in wx are prefixed with an 'TheObject' as well, so I didn't want the user to keep sending both the 'env' and 'This' in all calls, wxFrame:setTitle(Env, TheObject, "My window Title"), vs C++ TheObject->setTiltle("My Window Title"), I could have put the 'env' in each object reference, but that would have increased every object reference in erlang with two words, also for static calls which don't work on an object I would then have to add an 'env' parameter, which isn't there in the C++ lib. /Dan > > Cheers, > -- > Geoff Cant > > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From ngocdaothanh@REDACTED Fri Oct 30 08:32:00 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 30 Oct 2009 16:32:00 +0900 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <65b2728e0910291512y1577168br4759540e2726879e@mail.gmail.com> <5c493e530910291906m17e4d10aw371185015a6b5bb8@mail.gmail.com> <65b2728e0910292023g4b6a23e9p3c4af9b27b22d42e@mail.gmail.com> <5c493e530910292151g5d77fe97k854529aa376743e6@mail.gmail.com> <5c493e530910292349o87a810cha79df70e59cbd47d@mail.gmail.com> <5c493e530910300007h5f99498id7435b838e59d14e@mail.gmail.com> Message-ID: <5c493e530910300032l648ee605j66237c6ded734b73@mail.gmail.com> > What can you do with set-once variables in Erlang? For example: * In the controller you set-once an article in PD: http://github.com/ngocdaothanh/khale/blob/master/removable/article/c_article.erl * In the view you take out the article and render: http://github.com/ngocdaothanh/khale/blob/master/removable/article/v_article_show.erl The philosophy behind "set-once" is that you need to pass things in only one way, from a layer to layers behind it. For this purpose PD is a perfect medium, and its its implicitness would make the code less verbose. As a framework developer, I think it would make for app developers happy because their code would be less verbose. When you want to start Eshell, would you want to type "erl" or the full path to "erl"? Ngoc On Fri, Oct 30, 2009 at 4:09 PM, Max Lapshin wrote: > On Fri, Oct 30, 2009 at 10:07 AM, Ngoc Dao wrote: >> In order to intercept, I think the explicit thing you need to know is >> the processing flow. For example: >> browser -> yaws -> front controller -> app-wide filter -> >> controller-wide filter -> action -> view -> layout -> front controller >> -> yaws -> browser > > Damn, You look at mutable Ruby and try to implement in immutable > Erlang the same thing. > after_filter in Rails can modify anything, including internal hidden > instance variables. > What can you do with set-once variables in Erlang? From max.lapshin@REDACTED Fri Oct 30 08:38:36 2009 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 30 Oct 2009 10:38:36 +0300 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: <5c493e530910300032l648ee605j66237c6ded734b73@mail.gmail.com> References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <5c493e530910291906m17e4d10aw371185015a6b5bb8@mail.gmail.com> <65b2728e0910292023g4b6a23e9p3c4af9b27b22d42e@mail.gmail.com> <5c493e530910292151g5d77fe97k854529aa376743e6@mail.gmail.com> <5c493e530910292349o87a810cha79df70e59cbd47d@mail.gmail.com> <5c493e530910300007h5f99498id7435b838e59d14e@mail.gmail.com> <5c493e530910300032l648ee605j66237c6ded734b73@mail.gmail.com> Message-ID: You try not to hear me and not to answer my questions. HOW are you going to implement filters, that modify data, that is going to be rendered with PD? From ngocdaothanh@REDACTED Fri Oct 30 08:50:09 2009 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 30 Oct 2009 16:50:09 +0900 Subject: [erlang-questions] Re: Process Dictionary vs Proplist in Web Frameworks In-Reply-To: References: <5c493e530910280446n130bada1ice8fe090ba47ad0d@mail.gmail.com> <65b2728e0910292023g4b6a23e9p3c4af9b27b22d42e@mail.gmail.com> <5c493e530910292151g5d77fe97k854529aa376743e6@mail.gmail.com> <5c493e530910292349o87a810cha79df70e59cbd47d@mail.gmail.com> <5c493e530910300007h5f99498id7435b838e59d14e@mail.gmail.com> <5c493e530910300032l648ee605j66237c6ded734b73@mail.gmail.com> Message-ID: <5c493e530910300050i71c27c59p7422776d089552b1@mail.gmail.com> I have already answered: * See the flow * Use the designated API I feel that you want to implicitly say that you find no use in PD and it should be removed from the next version of Erlang. On Fri, Oct 30, 2009 at 4:38 PM, Max Lapshin wrote: > You try not to hear me and not to answer my questions. > HOW are you going to implement filters, that modify data, that is > going to be rendered with PD? From dmitriy.budashny@REDACTED Fri Oct 30 15:00:01 2009 From: dmitriy.budashny@REDACTED (Dmitriy Budashny) Date: Fri, 30 Oct 2009 16:00:01 +0200 Subject: application:start(snmp) failed Message-ID: <891161a10910300700t8ddbe35l721a27d93cd0b21c@mail.gmail.com> Hi I have no any problem with starting snmp with default mib_storage, but I got error while trying to use mnesia as a mib_storage. Below you can find error report message "Configuration error: [STANDARD-MIB]: configure failed: {missing_mandatory, sysObjectID}", but sysObjectID is defined in standard.conf, so I can't get is this my mistake or maybe a bug in snmp. Btw mnesia table snmpa_symbolic_store was created. snmp/snmp.config file: [{snmp, [ {agent, [ {mib_storage, {mnesia, []}}, {agent_verbosity, debug}, {config, [{dir, "agent/conf/"}]}, {db_dir, "cxDB/"}, {local_db, [{repair,true},{auto_save,5000},{verbosity,debug}]} ]}, {manager, [ {config, [{dir, "manager/conf/"}, {db_dir, "cxDB/"}]} ]} ]}]. snmp/agent/conf/standard.conf file: {sysName, "CX"}. {sysDescr, "Erlang/OTP Agent"}. {sysContact, "dmitriy.budashny@REDACTED"}. {sysLocation, "Somewhere in Ukraine"}. %%{sysObjectID, [1,3,6,1,4,1]}. {sysObjectID, [0,0]}. {sysServices, 72}. {snmpEnableAuthenTraps, enabled}. $ erl -sname cx@REDACTED -mnesia dir '"./snmp/cxDB"' -config snmp/ Erlang R13B02 (erts-5.7.3) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.3 (abort with ^G) (cx@REDACTED)1> mnesia:create_schema([node()]). ok (cx@REDACTED)2> mnesia:start(). ok (cx@REDACTED)3> snmp:start(). *** [2009:10:28 09:34:24 4994] SNMP A-LOCAL-DB LOG *** starting *** [2009:10:28 09:34:24 4995] SNMP A-LOCAL-DB DEBUG *** dets open done *** [2009:10:28 09:34:24 4995] SNMP A-LOCAL-DB DEBUG *** started *** [2009:10:28 09:34:24 4995] SNMP A-LOCAL-DB DEBUG *** started =ERROR REPORT==== 28-Oct-2009::11:34:25 === ** Configuration error: [STANDARD-MIB]: configure failed: {missing_mandatory, sysObjectID} *** [2009:10:28 09:34:25 413] SNMP A-LOCAL-DB LOG *** terminate: shutdown =INFO REPORT==== 28-Oct-2009::11:34:25 === application: snmp exited: {bad_return, {{snmp_app,start,[normal,[]]}, {'EXIT', {{badmatch, {error, {shutdown, {child,undefined,snmpa_supervisor, {snmpa_supervisor,start_link, [normal, [{mib_storage,{mnesia,[]}}, {agent_verbosity,debug}, {config,[{dir,"agent/conf/"}]}, {db_dir,"cxDB/"}, {local_db, [{repair,true},{auto_save,5000},{verbosity,debug}]}]]}, permanent,2000,supervisor, [snmpa_supervisor,supervisor]}}}}, [{snmp_app,start,2},{application_master,start_it_old,4}]}}}} type: temporary {error,{bad_return,{{snmp_app,start,[normal,[]]}, {'EXIT',{{badmatch,{error,{shutdown,{child,undefined, snmpa_supervisor, {snmpa_supervisor,start_link, [normal, [{mib_storage,{mnesia,[]}}, {agent_verbosity,debug}, {config,[...]}, {db_dir,...}, {...}]]}, permanent,2000,supervisor, [snmpa_supervisor,supervisor]}}}}, [{snmp_app,start,2}, {application_master,start_it_old,4}]}}}}} -- wbr, dym From dizzyd@REDACTED Fri Oct 30 16:43:57 2009 From: dizzyd@REDACTED (Dave Smith) Date: Fri, 30 Oct 2009 09:43:57 -0600 Subject: [erlang-questions] throttling the cpu/memory available to the VM In-Reply-To: References: Message-ID: >From what I know of the VM, Erlang does not provide either of these. Providing max memory usage is feasible, but I suspect would be somewhat difficult with the way the GC is structured. I've worked on a VM (Java) that tried to do strict CPU allocation -- that's a very hard problem to get right without introducing a lot of accounting overhead. Really, the operating system is the best place to enforce allocation of CPU, not the VM (or individual applications). Hope that helps, D. On Thu, Oct 29, 2009 at 11:29 PM, Jacob Vorreuter wrote: > Is it possible to specify a max memory usage and cpu usage for the VM > when the runtime system starts? ?I would like to be able to run > multiple VMs on the same system and ensure equal distribution of > resources. > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From gsmyth@REDACTED Fri Oct 30 17:25:55 2009 From: gsmyth@REDACTED (Greg Smyth) Date: Fri, 30 Oct 2009 16:25:55 +0000 Subject: Interesting style i haven't seen.... Message-ID: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> A friend just came up with this, and i thought it was pretty interesting(arguments against doing this kind of thing in Erlang aside...) as I'd never seen this type of thing done before... Anyone care to shed some light on why/how this works:- %% -module(dog). -export([new/1, woof/1]). new(Name) -> {dog, Name}. woof(Self) -> {dog, Name} = Self, io:format("~p says 'woof!'~n", [Name]). %% % 13> c(dog). % {ok,dog} % 14> Dog = dog:new("Fido"). % {dog,"Fido"} % 15> Dog:woof(). % "Fido" says 'woof!' % ok Many thanks, Greg From dizzyd@REDACTED Fri Oct 30 17:32:39 2009 From: dizzyd@REDACTED (Dave Smith) Date: Fri, 30 Oct 2009 10:32:39 -0600 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> References: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> Message-ID: Is this a trick question? I feel like I'm missing something here... :) If the confusion is re: the use of "new", remember that Erlang has no concept of objects or explicit allocation (like that anyways). D. On Fri, Oct 30, 2009 at 10:25 AM, Greg Smyth wrote: > A friend just came up with this, and i thought it was pretty > interesting(arguments against doing this kind of thing in Erlang > aside...) as I'd never seen this type of thing done before... > > Anyone care to shed some light on why/how this works:- > > %% > -module(dog). > -export([new/1, woof/1]). > > new(Name) -> {dog, Name}. > > woof(Self) -> > ?{dog, Name} = Self, > ?io:format("~p says 'woof!'~n", [Name]). > %% > > % 13> c(dog). > % {ok,dog} > > % 14> Dog = dog:new("Fido"). > % {dog,"Fido"} > > % 15> Dog:woof(). > % "Fido" says 'woof!' > % ok > > Many thanks, > Greg > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From adrianob@REDACTED Fri Oct 30 17:40:08 2009 From: adrianob@REDACTED (Adriano Bonat) Date: Fri, 30 Oct 2009 14:40:08 -0200 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: References: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> Message-ID: <6bb4c8cb0910300940r77804948mdc1da5fff2f80d69@mail.gmail.com> IIRC this is called parameterized modules :) On Fri, Oct 30, 2009 at 2:32 PM, Dave Smith wrote: > Is this a trick question? I feel like I'm missing something here... :) > > If the confusion is re: the use of "new", remember that Erlang has no > concept of objects or explicit allocation (like that anyways). > > D. > > On Fri, Oct 30, 2009 at 10:25 AM, Greg Smyth wrote: >> A friend just came up with this, and i thought it was pretty >> interesting(arguments against doing this kind of thing in Erlang >> aside...) as I'd never seen this type of thing done before... >> >> Anyone care to shed some light on why/how this works:- >> >> %% >> -module(dog). >> -export([new/1, woof/1]). >> >> new(Name) -> {dog, Name}. >> >> woof(Self) -> >> ?{dog, Name} = Self, >> ?io:format("~p says 'woof!'~n", [Name]). >> %% >> >> % 13> c(dog). >> % {ok,dog} >> >> % 14> Dog = dog:new("Fido"). >> % {dog,"Fido"} >> >> % 15> Dog:woof(). >> % "Fido" says 'woof!' >> % ok >> >> Many thanks, >> Greg >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From jim.mccoy@REDACTED Fri Oct 30 17:45:56 2009 From: jim.mccoy@REDACTED (Jim McCoy) Date: Fri, 30 Oct 2009 09:45:56 -0700 Subject: How to get a supervisor to tell me a child's startfunc? Message-ID: I need to discover the startfunc being used by a variety of processes started dynamically by a supervisor and I am having a hard time figuring out how to dig this information out of the system. Process A starts a bunch of children for supervisor S over its lifetime, process B needs to do some maintenance or checking of which of these children are running at various points in time, to do this B needs to specifically know the start function that A used when it passed the child spec for any particular child process in to S. This information does not seem to be available directly from the supervisor, so I am guessing that there is some other place I need to look but my various search queries seem to be missing the magic keyword that will not lead me back to the standard supervisor docs... Any hints? Thanks, jim From colm.dougan@REDACTED Fri Oct 30 17:45:42 2009 From: colm.dougan@REDACTED (Colm Dougan) Date: Fri, 30 Oct 2009 16:45:42 +0000 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> References: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> Message-ID: <24d4f39c0910300945n358b9b2at461dc97dc74646b6@mail.gmail.com> I think it works because Dog:woof() means the same thing as : dog:woof(Dog) .. by virtue of the fact that Dog is a tuple with a first element of "dog". I seem to recall this is a consequence of the parametrized modules feature. Colm On Fri, Oct 30, 2009 at 4:25 PM, Greg Smyth wrote: > A friend just came up with this, and i thought it was pretty > interesting(arguments against doing this kind of thing in Erlang > aside...) as I'd never seen this type of thing done before... > > Anyone care to shed some light on why/how this works:- > > %% > -module(dog). > -export([new/1, woof/1]). > > new(Name) -> {dog, Name}. > > woof(Self) -> > ?{dog, Name} = Self, > ?io:format("~p says 'woof!'~n", [Name]). > %% > > % 13> c(dog). > % {ok,dog} > > % 14> Dog = dog:new("Fido"). > % {dog,"Fido"} > > % 15> Dog:woof(). > % "Fido" says 'woof!' > % ok > > Many thanks, > Greg > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From adrianob@REDACTED Fri Oct 30 17:53:10 2009 From: adrianob@REDACTED (Adriano Bonat) Date: Fri, 30 Oct 2009 14:53:10 -0200 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: <24d4f39c0910300945n358b9b2at461dc97dc74646b6@mail.gmail.com> References: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> <24d4f39c0910300945n358b9b2at461dc97dc74646b6@mail.gmail.com> Message-ID: <6bb4c8cb0910300953g300b5bbex8fd1cc389a5cf8b2@mail.gmail.com> Exactly, 9> Blah = {modname, "arg1", "arg2", 3}. {modname,"arg1","arg2",3} 10> Blah:func(). ** exception error: undefined function modname:func/1 On Fri, Oct 30, 2009 at 2:45 PM, Colm Dougan wrote: > I think it works because > > Dog:woof() > > means the same thing as : > > dog:woof(Dog) > > .. by virtue of the fact that Dog is a tuple with a first element of > "dog". ?I seem to recall this is a consequence of the parametrized > modules feature. > > Colm > > On Fri, Oct 30, 2009 at 4:25 PM, Greg Smyth wrote: >> A friend just came up with this, and i thought it was pretty >> interesting(arguments against doing this kind of thing in Erlang >> aside...) as I'd never seen this type of thing done before... >> >> Anyone care to shed some light on why/how this works:- >> >> %% >> -module(dog). >> -export([new/1, woof/1]). >> >> new(Name) -> {dog, Name}. >> >> woof(Self) -> >> ?{dog, Name} = Self, >> ?io:format("~p says 'woof!'~n", [Name]). >> %% >> >> % 13> c(dog). >> % {ok,dog} >> >> % 14> Dog = dog:new("Fido"). >> % {dog,"Fido"} >> >> % 15> Dog:woof(). >> % "Fido" says 'woof!' >> % ok >> >> Many thanks, >> Greg >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From tony@REDACTED Fri Oct 30 18:15:04 2009 From: tony@REDACTED (Tony Arcieri) Date: Fri, 30 Oct 2009 11:15:04 -0600 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: <24d4f39c0910300945n358b9b2at461dc97dc74646b6@mail.gmail.com> References: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> <24d4f39c0910300945n358b9b2at461dc97dc74646b6@mail.gmail.com> Message-ID: On Fri, Oct 30, 2009 at 10:45 AM, Colm Dougan wrote: > .. by virtue of the fact that Dog is a tuple with a first element of > "dog". I seem to recall this is a consequence of the parametrized > modules feature. > Note that the paramaterized modules paper: http://www.erlang.se/workshop/2003/paper/p29-carlsson.pdf ...specifies the value representing a paramaterized module is opaque. This is a side-effect of the implementation. That said, very interesting. -- Tony Arcieri Medioh/Nagravision From dizzyd@REDACTED Fri Oct 30 18:22:50 2009 From: dizzyd@REDACTED (Dave Smith) Date: Fri, 30 Oct 2009 11:22:50 -0600 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: References: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> <24d4f39c0910300945n358b9b2at461dc97dc74646b6@mail.gmail.com> Message-ID: I may be wrong, but I don't think this has anything to do with parameterized modules, but is instead an old-school way of invoking functions. From the reference manual: "When applied to a number N of arguments, a tuple {Module,FunctionName} is interpreted as a fun, referring to the function FunctionName with arity N in the module Module. The function must be exported. This usage is deprecated. See Function Calls for an example." D. On Fri, Oct 30, 2009 at 11:15 AM, Tony Arcieri wrote: > On Fri, Oct 30, 2009 at 10:45 AM, Colm Dougan wrote: > >> .. by virtue of the fact that Dog is a tuple with a first element of >> "dog". ?I seem to recall this is a consequence of the parametrized >> modules feature. >> > > Note that the paramaterized modules paper: > > http://www.erlang.se/workshop/2003/paper/p29-carlsson.pdf > > ...specifies the value representing a paramaterized module is opaque. ?This > is a side-effect of the implementation. > > That said, very interesting. > > -- > Tony Arcieri > Medioh/Nagravision > From chris.tilt@REDACTED Fri Oct 30 17:36:06 2009 From: chris.tilt@REDACTED (Chris Tilt) Date: Fri, 30 Oct 2009 12:36:06 -0400 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: Message-ID: It's the use of Dog:woof() that throws me. Dog is a variable and I don't understand the semantics of "Variable:function()". Cheers, Chris On 10/30/09 9:32 AM, "Dave Smith" wrote: Is this a trick question? I feel like I'm missing something here... :) If the confusion is re: the use of "new", remember that Erlang has no concept of objects or explicit allocation (like that anyways). D. On Fri, Oct 30, 2009 at 10:25 AM, Greg Smyth wrote: > A friend just came up with this, and i thought it was pretty > interesting(arguments against doing this kind of thing in Erlang > aside...) as I'd never seen this type of thing done before... > > Anyone care to shed some light on why/how this works:- > > %% > -module(dog). > -export([new/1, woof/1]). > > new(Name) -> {dog, Name}. > > woof(Self) -> > {dog, Name} = Self, > io:format("~p says 'woof!'~n", [Name]). > %% > > % 13> c(dog). > % {ok,dog} > > % 14> Dog = dog:new("Fido"). > % {dog,"Fido"} > > % 15> Dog:woof(). > % "Fido" says 'woof!' > % ok > > Many thanks, > Greg > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > ________________________________________________________________ erlang-questions mailing list. See http://www.erlang.org/faq.html erlang-questions (at) erlang.org From bob@REDACTED Fri Oct 30 19:27:20 2009 From: bob@REDACTED (Bob Ippolito) Date: Fri, 30 Oct 2009 14:27:20 -0400 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: References: Message-ID: <6a36e7290910301127h9f820b8h4c3da1d2f23fa432@mail.gmail.com> That's how parameterized modules work. An instance of a parameterized module is a variable. This particular example is (ab)using an implementation detail of how parameterized modules are currently implemented. You'd get the same behavior with this module, which uses an actual parameterized module: ---- -module(dog, [Name]). -export([woof/0]). woof() -> io:format("~p says 'woof!'~n", [Name]). ---- 1> c(dog). {ok,dog} 2> Fido = dog:new(fido). {dog,fido} 3> Fido:woof(). fido says 'woof!' ok On Fri, Oct 30, 2009 at 12:36 PM, Chris Tilt wrote: > It's the use of Dog:woof() that throws me. Dog is a variable and I don't understand the semantics of "Variable:function()". > Cheers, Chris > > > On 10/30/09 9:32 AM, "Dave Smith" wrote: > > Is this a trick question? I feel like I'm missing something here... :) > > If the confusion is re: the use of "new", remember that Erlang has no > concept of objects or explicit allocation (like that anyways). > > D. > > On Fri, Oct 30, 2009 at 10:25 AM, Greg Smyth wrote: >> A friend just came up with this, and i thought it was pretty >> interesting(arguments against doing this kind of thing in Erlang >> aside...) as I'd never seen this type of thing done before... >> >> Anyone care to shed some light on why/how this works:- >> >> %% >> -module(dog). >> -export([new/1, woof/1]). >> >> new(Name) -> {dog, Name}. >> >> woof(Self) -> >> ?{dog, Name} = Self, >> ?io:format("~p says 'woof!'~n", [Name]). >> %% >> >> % 13> c(dog). >> % {ok,dog} >> >> % 14> Dog = dog:new("Fido"). >> % {dog,"Fido"} >> >> % 15> Dog:woof(). >> % "Fido" says 'woof!' >> % ok >> >> Many thanks, >> Greg >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From mihai@REDACTED Fri Oct 30 19:31:04 2009 From: mihai@REDACTED (Mihai Balea) Date: Fri, 30 Oct 2009 14:31:04 -0400 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: References: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> <24d4f39c0910300945n358b9b2at461dc97dc74646b6@mail.gmail.com> Message-ID: <02B9D339-3382-4EE1-A563-919AC8A0215A@hates.ms> On Oct 30, 2009, at 1:22 PM, Dave Smith wrote: > "When applied to a number N of arguments, a tuple > {Module,FunctionName} is interpreted as a fun, referring to the > function FunctionName with arity N in the module Module. The function > must be exported. This usage is deprecated. See Function Calls for an > example." I've always wondered why that particular syntax quirk is being deprecated. I think it's neat. Mihai From malcolm@REDACTED Fri Oct 30 19:33:18 2009 From: malcolm@REDACTED (Dowse, Malcolm) Date: Fri, 30 Oct 2009 18:33:18 -0000 Subject: module dependencies between applications when using .appup files Message-ID: <54E2F5E30BE5BC42AC4FF38C4545AAE1A352B9@UKLHREXCL02.activision.com> Hi, I have a question about the release handler and .appup files. I have two applications, app, and app_common, and a release which uses both. app_common is a dependency of app. So if app is running, app_common will be too.. but not the opposite. It is possible to force a module in app_common to load before one in app. You specify {load_module, app_module, [app_common_module]} in app.appup. This isn't very elegant, but it definitely works. The problem I'm having is that when upgrading the release, I want to load a module in app before I load one in app_common. Without a command like {load_module, app_module, [{reverse_dependency, app_common_module}]} there seems to be no way of doing this. Adding {load_module, app_common_module, [app_module]} to app_common.appup is out of the question. Can anyone tell me any clever tricks/hacks to get around this? Thanks in advance, Malcolm From gsmyth@REDACTED Fri Oct 30 17:40:42 2009 From: gsmyth@REDACTED (Greg Smyth) Date: Fri, 30 Oct 2009 16:40:42 +0000 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: References: Message-ID: <361b4e9d0910300940r68d43f94pf25d0b009da03044@mail.gmail.com> Yeah, It certainly wasn't a trick question, unless the trick is on me... :) I still dont get how:- Variable = {atom, Value}, Variable:function/0. then gets converted to a call to atom:function({atom, Value}) if that makes sense...? Cheers, Greg On Fri, Oct 30, 2009 at 4:36 PM, Chris Tilt wrote: > It?s the use of Dog:woof() that throws me. Dog is a variable and I don?t > understand the semantics of ?Variable:function()?. > Cheers, Chris > > > On 10/30/09 9:32 AM, "Dave Smith" wrote: > > Is this a trick question? I feel like I'm missing something here... :) > > If the confusion is re: the use of "new", remember that Erlang has no > concept of objects or explicit allocation (like that anyways). > > D. > > On Fri, Oct 30, 2009 at 10:25 AM, Greg Smyth wrote: >> A friend just came up with this, and i thought it was pretty >> interesting(arguments against doing this kind of thing in Erlang >> aside...) as I'd never seen this type of thing done before... >> >> Anyone care to shed some light on why/how this works:- >> >> %% >> -module(dog). >> -export([new/1, woof/1]). >> >> new(Name) -> {dog, Name}. >> >> woof(Self) -> >> ?{dog, Name} = Self, >> ?io:format("~p says 'woof!'~n", [Name]). >> %% >> >> % 13> c(dog). >> % {ok,dog} >> >> % 14> Dog = dog:new("Fido"). >> % {dog,"Fido"} >> >> % 15> Dog:woof(). >> % "Fido" says 'woof!' >> % ok >> >> Many thanks, >> Greg >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > > From gsmyth@REDACTED Fri Oct 30 17:51:18 2009 From: gsmyth@REDACTED (Greg Smyth) Date: Fri, 30 Oct 2009 16:51:18 +0000 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: References: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> Message-ID: <361b4e9d0910300951g28219d90t80ebf70bcb3741f@mail.gmail.com> Looking at your example, I don't think this is parameterised modules (although that is also interesting, thanks!). The confusing part (for me at least), is still the section:- > % 14> Dog = dog:new("Fido"). > % ok > > % 15> Dog:woof(). > % "Fido" says 'woof!' How does Value:function() - change into module:function(Value) when Value = {module, Something}) ? Cheers, Greg On Fri, Oct 30, 2009 at 4:42 PM, Dale Harvey wrote: > The syntax is a bit wrong, > -module(dog, [Name]). > -export([woof/1]). > woof(Self) -> > ?{dog, Name} = Self, > ?io:format("~p says 'woof!'~n", [Name]). > > % 13> c(dog). > % {ok,dog} > > % 14> Dog = dog:new("Fido"). > % ok > > % 15> Dog:woof(). > % "Fido" says 'woof!' > % ok > but erlang does support it, parameterised modules > http://www.clickcaster.com/channel/tag/modules?channel=diveintoerlang > 2009/10/30 Dave Smith >> >> Is this a trick question? I feel like I'm missing something here... :) >> >> If the confusion is re: the use of "new", remember that Erlang has no >> concept of objects or explicit allocation (like that anyways). >> >> D. >> >> On Fri, Oct 30, 2009 at 10:25 AM, Greg Smyth >> wrote: >> > A friend just came up with this, and i thought it was pretty >> > interesting(arguments against doing this kind of thing in Erlang >> > aside...) as I'd never seen this type of thing done before... >> > >> > Anyone care to shed some light on why/how this works:- >> > >> > %% >> > -module(dog). >> > -export([new/1, woof/1]). >> > >> > new(Name) -> {dog, Name}. >> > >> > woof(Self) -> >> > ?{dog, Name} = Self, >> > ?io:format("~p says 'woof!'~n", [Name]). >> > %% >> > >> > % 13> c(dog). >> > % {ok,dog} >> > >> > % 14> Dog = dog:new("Fido"). >> > % {dog,"Fido"} >> > >> > % 15> Dog:woof(). >> > % "Fido" says 'woof!' >> > % ok >> > >> > Many thanks, >> > Greg >> > >> > ________________________________________________________________ >> > erlang-questions mailing list. See http://www.erlang.org/faq.html >> > erlang-questions (at) erlang.org >> > >> > >> >> ________________________________________________________________ >> erlang-questions mailing list. See http://www.erlang.org/faq.html >> erlang-questions (at) erlang.org >> > > From nick@REDACTED Fri Oct 30 21:05:19 2009 From: nick@REDACTED (Nick Gerakines) Date: Fri, 30 Oct 2009 13:05:19 -0700 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: <361b4e9d0910300951g28219d90t80ebf70bcb3741f@mail.gmail.com> References: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> <361b4e9d0910300951g28219d90t80ebf70bcb3741f@mail.gmail.com> Message-ID: Yeah, not only can you have param'd modules but they can be extensions of each other. http://gist.github.com/149735 --- begin nick.erl -module(nick, [City, State]). -export([city/0, state/0]). state() -> io:format("nick:state/0 -> ~p~n", [State]). city() -> io:format("nick:city/0 -> ~p ~n", [City]). --- end --- begin vanessa.erl -module(vanessa, [City, State]). -extends(nick). -export([city/0]). city() -> io:format("child:city/0 -> ~p ~n", [City]). --- end $ erlc nick.erl && erlc vanessa.erl 1> Me = nick:new("Mountain View", "CA"). {nick,"Mountain View","CA"} 2> Me:city(). nick:city/0 -> "Mountain View" ok 3> Me:state(). nick:state/0 -> "CA" ok 4> Vanessa = vanessa:new("Mountain View", "CA"). {vanessa,{nick,"Mountain View","CA"},"Mountain View","CA"} 5> Vanessa:city(). child:city/0 -> "Mountain View" ok 6> Vanessa:state(). nick:state/0 -> "CA" ok 7> On Fri, Oct 30, 2009 at 9:51 AM, Greg Smyth wrote: > Looking at your example, I don't think this is parameterised modules > (although that is also interesting, thanks!). > > The confusing part (for me at least), is still the section:- > >> % 14> Dog = dog:new("Fido"). >> % ok >> >> % 15> Dog:woof(). >> % "Fido" says 'woof!' > > How does Value:function() ?- change into > > module:function(Value) > > when Value = {module, Something}) ? > > Cheers, > Greg > > On Fri, Oct 30, 2009 at 4:42 PM, Dale Harvey wrote: >> The syntax is a bit wrong, >> -module(dog, [Name]). >> -export([woof/1]). >> woof(Self) -> >> ?{dog, Name} = Self, >> ?io:format("~p says 'woof!'~n", [Name]). >> >> % 13> c(dog). >> % {ok,dog} >> >> % 14> Dog = dog:new("Fido"). >> % ok >> >> % 15> Dog:woof(). >> % "Fido" says 'woof!' >> % ok >> but erlang does support it, parameterised modules >> http://www.clickcaster.com/channel/tag/modules?channel=diveintoerlang >> 2009/10/30 Dave Smith >>> >>> Is this a trick question? I feel like I'm missing something here... :) >>> >>> If the confusion is re: the use of "new", remember that Erlang has no >>> concept of objects or explicit allocation (like that anyways). >>> >>> D. >>> >>> On Fri, Oct 30, 2009 at 10:25 AM, Greg Smyth >>> wrote: >>> > A friend just came up with this, and i thought it was pretty >>> > interesting(arguments against doing this kind of thing in Erlang >>> > aside...) as I'd never seen this type of thing done before... >>> > >>> > Anyone care to shed some light on why/how this works:- >>> > >>> > %% >>> > -module(dog). >>> > -export([new/1, woof/1]). >>> > >>> > new(Name) -> {dog, Name}. >>> > >>> > woof(Self) -> >>> > ?{dog, Name} = Self, >>> > ?io:format("~p says 'woof!'~n", [Name]). >>> > %% >>> > >>> > % 13> c(dog). >>> > % {ok,dog} >>> > >>> > % 14> Dog = dog:new("Fido"). >>> > % {dog,"Fido"} >>> > >>> > % 15> Dog:woof(). >>> > % "Fido" says 'woof!' >>> > % ok >>> > >>> > Many thanks, >>> > Greg >>> > >>> > ________________________________________________________________ >>> > erlang-questions mailing list. See http://www.erlang.org/faq.html >>> > erlang-questions (at) erlang.org >>> > >>> > >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From bob@REDACTED Fri Oct 30 21:30:59 2009 From: bob@REDACTED (Bob Ippolito) Date: Fri, 30 Oct 2009 16:30:59 -0400 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: <361b4e9d0910300951g28219d90t80ebf70bcb3741f@mail.gmail.com> References: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> <361b4e9d0910300951g28219d90t80ebf70bcb3741f@mail.gmail.com> Message-ID: <6a36e7290910301330r38378c21u7a0be0c5cb96496f@mail.gmail.com> It is parameterized modules, just not implemented correctly. The only reason it behaves that way is because of how they happen to be implemented in current versions of erlang's VM and compiler. In some future version of erlang the example at the beginning of this thread is expected to no longer work. On Friday, October 30, 2009, Greg Smyth wrote: > Looking at your example, I don't think this is parameterised modules > (although that is also interesting, thanks!). > > The confusing part (for me at least), is still the section:- > >> % 14> Dog = dog:new("Fido"). >> % ok >> >> % 15> Dog:woof(). >> % "Fido" says 'woof!' > > How does Value:function() ?- change into > > module:function(Value) > > when Value = {module, Something}) ? > > Cheers, > Greg > > On Fri, Oct 30, 2009 at 4:42 PM, Dale Harvey wrote: >> The syntax is a bit wrong, >> -module(dog, [Name]). >> -export([woof/1]). >> woof(Self) -> >> ?{dog, Name} = Self, >> ?io:format("~p says 'woof!'~n", [Name]). >> >> % 13> c(dog). >> % {ok,dog} >> >> % 14> Dog = dog:new("Fido"). >> % ok >> >> % 15> Dog:woof(). >> % "Fido" says 'woof!' >> % ok >> but erlang does support it, parameterised modules >> http://www.clickcaster.com/channel/tag/modules?channel=diveintoerlang >> 2009/10/30 Dave Smith >>> >>> Is this a trick question? I feel like I'm missing something here... :) >>> >>> If the confusion is re: the use of "new", remember that Erlang has no >>> concept of objects or explicit allocation (like that anyways). >>> >>> D. >>> >>> On Fri, Oct 30, 2009 at 10:25 AM, Greg Smyth >>> wrote: >>> > A friend just came up with this, and i thought it was pretty >>> > interesting(arguments against doing this kind of thing in Erlang >>> > aside...) as I'd never seen this type of thing done before... >>> > >>> > Anyone care to shed some light on why/how this works:- >>> > >>> > %% >>> > -module(dog). >>> > -export([new/1, woof/1]). >>> > >>> > new(Name) -> {dog, Name}. >>> > >>> > woof(Self) -> >>> > ?{dog, Name} = Self, >>> > ?io:format("~p says 'woof!'~n", [Name]). >>> > %% >>> > >>> > % 13> c(dog). >>> > % {ok,dog} >>> > >>> > % 14> Dog = dog:new("Fido"). >>> > % {dog,"Fido"} >>> > >>> > % 15> Dog:woof(). >>> > % "Fido" says 'woof!' >>> > % ok >>> > >>> > Many thanks, >>> > Greg >>> > >>> > ________________________________________________________________ >>> > erlang-questions mailing list. See http://www.erlang.org/faq.html >>> > erlang-questions (at) erlang.org >>> > >>> > >>> >>> ________________________________________________________________ >>> erlang-questions mailing list. See http://www.erlang.org/faq.html >>> erlang-questions (at) erlang.org >>> >> >> > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > > From rvirding@REDACTED Fri Oct 30 21:34:16 2009 From: rvirding@REDACTED (Robert Virding) Date: Fri, 30 Oct 2009 21:34:16 +0100 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: <02B9D339-3382-4EE1-A563-919AC8A0215A@hates.ms> References: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> <24d4f39c0910300945n358b9b2at461dc97dc74646b6@mail.gmail.com> <02B9D339-3382-4EE1-A563-919AC8A0215A@hates.ms> Message-ID: <3dbc6d1c0910301334m736630fev67d68aa1a8e04a3@mail.gmail.com> 2009/10/30 Mihai Balea > > On Oct 30, 2009, at 1:22 PM, Dave Smith wrote: > > "When applied to a number N of arguments, a tuple >> {Module,FunctionName} is interpreted as a fun, referring to the >> function FunctionName with arity N in the module Module. The function >> must be exported. This usage is deprecated. See Function Calls for an >> example." >> > > I've always wondered why that particular syntax quirk is being deprecated. > I think it's neat. They are a remenent from the bad old days when erlang was less functional than it is today and we didn't have funs This was a hack and has no reason for existence now that we do have funs. Robert From fedora789@REDACTED Sat Oct 31 02:27:26 2009 From: fedora789@REDACTED (fedora) Date: Fri, 30 Oct 2009 18:27:26 -0700 Subject: [erlang-questions] Emacs Distel error "Module is not interpreted, can't set breakpoints." after interpreting In-Reply-To: References: <35eed8d00910291418h33713621o8cba41f8ba29f726@mail.gmail.com> <35eed8d00910291512s4336e745hc10f6a16117768ae@mail.gmail.com> Message-ID: <35eed8d00910301827s290328bx98980eded98c63a1@mail.gmail.com> Thanks, Bob. I installed that distel fork. The process list ( C-c C-d l) works and distel:who_calls returns OK. But I still can't set breakpoint after interpreting due to error "Module is not interpreted, can't set breakpoints." I traced the error message to distel/elisp/edb.el and commented out the check of edb-module-interpreted-p . After this change, breakpoint setting works a couple times. But 50% of the cases the function with breakpoint hangs after I ran it. (defun edb-toggle-breakpoint (node module line) "Toggle a breakpoint on the current line." (interactive (list (erl-target-node) (edb-module) (edb-line-number))) ;; (unless (edb-module-interpreted-p module) ;; (error "Module is not interpreted, can't set breakpoints.")) (if edb-buffer-breakpoints-stale (edb-toggle-stale-breakpoint module line) (edb-toggle-real-breakpoint node module line))) I think there might be something wrong with rpc communication between emacs and the erlang node being debugged. So edb-module-interpreted-p() got the wrong status about which module has been interpreted. I am still trying to figure out how to debug the elisp code. -John On 10/29/09, Robert Dionne wrote: > it can be finnicky. Some things to try: > > the version that is in github. I forked this one: > http://github.com/kunley/distel > > use -sname when starting and specify that name when connecting > > does C-c C-d l work? It show a list of processes. > > who calls acts up (C-c C-d w) and I find the best way to make it work > is to call the function once directly from the erl vm, .eg. > distel:who_calls(my_module,my_fun,arity). > > I'm running R12B01 on OS X 10.6 > > Cheers, > > Bob > > > > On Oct 29, 2009, at 6:12 PM, johe wrote: > >> Hi, I used instructions on http://bc.tech.coop/blog/070528.html to >> setup distel debugging. I kept getting error "Module is not >> interpreted, can't set breakpoints." after model interpreting. Here's >> the details: >> >> ======================================== >> >> Environment: >> >> GNU Emacs 23.1.1 (x86_64-redhat-linux-gnu, GTK+ Version 2.16.5) >> of 2009-08-03 on x86-4.fedora.phx.redhat.com >> >> Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:2:2] [rq:2] >> [async-threads:0] [hipe] [kernel-poll:false] >> ( switching to Erlang R12 5.6.5 [source] [64-bit] also got the same >> error) >> >> Linux Fedora release 11: fc11.x86_64 >> >> Distel from Repository Root: http://distel.googlecode.com/svn >> Revision: 93 >> Last Changed Date: 2009-08-02 16:38:33 -0700 (Sun, 02 Aug 2009) >> >> ======================================== >> >> Steps: >> >> * compile with debug info >> >> (emacs@REDACTED)> c(test, debug_info). >> >> ======================================== >> >> * connect emacs to erlang node emacs@REDACTED ( C-c C-d n ) >> >> ======================================== >> >> >> * distel reload erlang module ( C-c C-d L) >> >> after this step I can see distel in the erlang node. >> >> (emacs@REDACTED)6> i(). >> ... ... >> <0.33.0> supervisor:kernel/1 >> 233 57 0 >> kernel_safe_sup gen_server:loop/6 9 >> <0.36.0> erlang:apply/2 2584 >> 40333 0 >> rpc:local_call/3 49 >> <0.46.0> inet_tcp_dist:do_accept/6 >> 610 516 0 >> dist_util:con_loop/9 11 >> <0.50.0> distel:gl_proxy/1 >> 233 1 0 >> 'distel_gl_for_<8785. distel:gl_proxy/1 4 >> Total 28992 >> 812442 0 >> 286 >> ok >> >> >> ======================================== >> >> * Toggle debug interpreting of the module ( C-c C-d i) >> got messsage : Interpreting: test >> got distel:debug_subscriber* in i() output >> >> >> (emacs@REDACTED)7> i(). >> >> ... ... >> <0.46.0> inet_tcp_dist:do_accept/6 >> 987 803 0 >> dist_util:con_loop/9 11 >> <0.50.0> distel:gl_proxy/1 >> 233 1 0 >> 'distel_gl_for_<8785. distel:gl_proxy/1 4 >> <0.53.0> distel:debug_subscriber_init/2 >> 233 37 0 >> distel:debug_subscriber/1 5 >> <0.55.0> dbg_iserver:init/1 >> 377 104 0 >> dbg_iserver gen_server:loop/6 9 >> Total 29945 >> 901441 0 >> 303 >> >> ======================================== >> >> * Toggle a breakpoint at current line ( C-c SPC) >> got messsage : Module is not interpreted, can't set breakpoints. >> >> When distel debugging work, i() should show two new process >> distel:attach_init/2, >> distel:attach_loop/1, >> but now it doesn't: >> >> >> (emacs@REDACTED)8> i(). >> ... ... >> >> >> <0.46.0> inet_tcp_dist:do_accept/6 987 >> 1248 0 >> dist_util:con_loop/9 11 >> <0.50.0> distel:gl_proxy/1 >> 233 1 0 >> 'distel_gl_for_<8785. distel:gl_proxy/1 4 >> <0.53.0> distel:debug_subscriber_init/2 >> 233 37 0 >> distel:debug_subscriber/1 5 >> <0.55.0> dbg_iserver:init/1 >> 377 104 0 >> dbg_iserver gen_server:loop/6 9 >> Total 30322 >> 967176 1 >> >> >> ~/.erlang.cookie has been set, erlang:get_cookie() returns correctly. >> >> I tried reload and interpret the module multiple times and also >> switched to erlang R12, still got the same result. >> >> I tried to connect to a erlang node outside of emacs, emacs showed a >> popup windows "Attach debugger to xyz@REDACTED instead of emacs@REDACTED" when >> interpreting, but still got the same error "Module is not interpreted, >> can't set breakpoints." >> >> Does anyone know how to get information to troubleshooting this >> problem ? >> >> Thanks a lot. >> >> John From ulf.wiger@REDACTED Sat Oct 31 12:17:15 2009 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 31 Oct 2009 12:17:15 +0100 Subject: sext and Tokyo Tyrant (Re: [erlang-questions] sortable serialization format) In-Reply-To: <4AE47CA6.90701@erlang-consulting.com> References: <4AE47CA6.90701@erlang-consulting.com> Message-ID: <4AEC1CBB.4020607@erlang-consulting.com> Ulf Wiger wrote: > > A while ago I started hacking on a serialization format that > would have the same sorting properties as Erlang terms. > > I didn't quite get it to work (negative floats was the most > difficult part), but when I returned to it today, I realized > that it was only a very small problem. Once fixed, all my > QuickCheck suites passed. I just had to try this on Tokyo Tyrant, so I wrote a small prototype for connecting to TT and encoding a few requests, using the sext library to encode terms before sending them. I realized that a new function was needed in sext: prefix(Term), which encodes a 'prefix' that will match similar terms, and allow some wildcarding. A prefix can't be decoded (at least, I didn't write any code for doing so). Some examples: Eshell V5.7.1 (abort with ^G) 1> sext:encode({1,2,3}). <<16,0,0,0,3,10,0,0,0,2,10,0,0,0,4,10,0,0,0,6>> 2> sext:prefix({1,'_','_'}). <<16,0,0,0,3,10,0,0,0,2>> 3> sext:encode([1,2,3]). <<17,10,0,0,0,2,10,0,0,0,4,10,0,0,0,6,0>> 4> sext:prefix([1,2|'_']). <<17,10,0,0,0,2,10,0,0,0,4>> Armed with this, I opened a B-tree table in Tokyo Tyrant, and connected to it with my prototype module. Eshell V5.7.1 (abort with ^G) 1> {ok,TT} = tt_proto:open(tt,[]). {ok,<0.35.0>} 2> tt_proto:put(TT,{1,a}, 1). ok 3> tt_proto:get(TT,{1,a}). {ok,1} 4> tt_proto:put(TT,{1,b}, 2). ok 5> tt_proto:put(TT,{1,c}, 3). ok 6> tt_proto:put(TT,{2,a}, 4). ok Now, for some prefix matching: 7> tt_proto:keys(TT,{1,'_'}). {ok,[{1,a},{1,b},{1,c}]} 8> tt_proto:keys(TT,{2,'_'}). {ok,[{2,a}]} 9> timer:tc(tt_proto,keys,[TT,{1,'_'}]). {279,{ok,[{1,a},{1,b},{1,c}]}} I made no real effort to optimize anything. The module starts a gen_server which keeps a connection open to ttserver. It handles only one query at a time, but looking at the TCP protocol, it's hard to see how it could to otherwise, as there is no tagging of requests. The round trip times are going to be fairly high for simple requests (compared to dets and mnesia on small data sets), but the main benefit of using TT in the first place ought to be either that the data set is uncomfortably large for mnesia and dets, or that one wants ordered_set semantics on disk-based storage. I put the tt_proto module in sext/examples/ There is some edoc for it too. http://svn.ulf.wiger.net/sext/trunk/sext/doc/index.html BR, Ulf W From tony@REDACTED Sat Oct 31 14:49:00 2009 From: tony@REDACTED (Tony Rogvall) Date: Sat, 31 Oct 2009 14:49:00 +0100 Subject: [erlang-questions] Interesting style i haven't seen.... In-Reply-To: <3dbc6d1c0910301334m736630fev67d68aa1a8e04a3@mail.gmail.com> References: <361b4e9d0910300925y29d9f2bbxd8c43e9beb24b3cb@mail.gmail.com> <24d4f39c0910300945n358b9b2at461dc97dc74646b6@mail.gmail.com> <02B9D339-3382-4EE1-A563-919AC8A0215A@hates.ms> <3dbc6d1c0910301334m736630fev67d68aa1a8e04a3@mail.gmail.com> Message-ID: As already pointed out, this is new stuff, that has to do with "abstract" modules (You can find it in beam_emu.c apply_last) The call corresponds to: apply({dog,"Fido"}, woof, []). As you can see it is not {dog,woof} ! This feature is just an implementation detail and may change. /Tony On 30 okt 2009, at 21.34, Robert Virding wrote: > 2009/10/30 Mihai Balea > >> >> On Oct 30, 2009, at 1:22 PM, Dave Smith wrote: >> >> "When applied to a number N of arguments, a tuple >>> {Module,FunctionName} is interpreted as a fun, referring to the >>> function FunctionName with arity N in the module Module. The >>> function >>> must be exported. This usage is deprecated. See Function Calls for >>> an >>> example." >>> >> >> I've always wondered why that particular syntax quirk is being >> deprecated. >> I think it's neat. > > > They are a remenent from the bad old days when erlang was less > functional > than it is today and we didn't have funs This was a hack and has no > reason > for existence now that we do have funs. > > Robert From paul-trapexit@REDACTED Sat Oct 31 17:18:36 2009 From: paul-trapexit@REDACTED (Paul Mineiro) Date: Sat, 31 Oct 2009 09:18:36 -0700 (PDT) Subject: [erlang-questions] sext and Tokyo Tyrant (Re: [erlang-questions] sortable serialization format) In-Reply-To: <4AEC1CBB.4020607@erlang-consulting.com> References: <4AE47CA6.90701@erlang-consulting.com> <4AEC1CBB.4020607@erlang-consulting.com> Message-ID: Awesome. That's 90% of tcerl; most of the complexity had to do with implementing erlang term order from encoded binaries, some of the rest with augmenting term_to_binary to allow for prefixes (there's no smallest or largest term in erlang), and the remainder with query planning. The query planning part is pure erlang and independent of the term encoding (http://code.google.com/p/tcerl/source/browse/trunk/tcerl/src/tcbdbmsutil.erl), so you could hopefully reuse it. In retrospect, clearly, I should have abandoned term_to_binary, if only because Erlang is easy and C is hard, so the C side should be just memcmp. -- p On Sat, 31 Oct 2009, Ulf Wiger wrote: > Ulf Wiger wrote: > > > > A while ago I started hacking on a serialization format that > > would have the same sorting properties as Erlang terms. > > > > I didn't quite get it to work (negative floats was the most > > difficult part), but when I returned to it today, I realized > > that it was only a very small problem. Once fixed, all my > > QuickCheck suites passed. > > I just had to try this on Tokyo Tyrant, so I wrote a small > prototype for connecting to TT and encoding a few requests, > using the sext library to encode terms before sending them. > > I realized that a new function was needed in sext: prefix(Term), > which encodes a 'prefix' that will match similar terms, and allow > some wildcarding. A prefix can't be decoded (at least, I didn't > write any code for doing so). > > Some examples: > > Eshell V5.7.1 (abort with ^G) > 1> sext:encode({1,2,3}). > <<16,0,0,0,3,10,0,0,0,2,10,0,0,0,4,10,0,0,0,6>> > 2> sext:prefix({1,'_','_'}). > <<16,0,0,0,3,10,0,0,0,2>> > 3> sext:encode([1,2,3]). > <<17,10,0,0,0,2,10,0,0,0,4,10,0,0,0,6,0>> > 4> sext:prefix([1,2|'_']). > <<17,10,0,0,0,2,10,0,0,0,4>> > > > Armed with this, I opened a B-tree table in Tokyo Tyrant, > and connected to it with my prototype module. > > Eshell V5.7.1 (abort with ^G) > 1> {ok,TT} = tt_proto:open(tt,[]). > {ok,<0.35.0>} > 2> tt_proto:put(TT,{1,a}, 1). > ok > 3> tt_proto:get(TT,{1,a}). > {ok,1} > 4> tt_proto:put(TT,{1,b}, 2). > ok > 5> tt_proto:put(TT,{1,c}, 3). > ok > 6> tt_proto:put(TT,{2,a}, 4). > ok > > Now, for some prefix matching: > > 7> tt_proto:keys(TT,{1,'_'}). > {ok,[{1,a},{1,b},{1,c}]} > 8> tt_proto:keys(TT,{2,'_'}). > {ok,[{2,a}]} > 9> timer:tc(tt_proto,keys,[TT,{1,'_'}]). > {279,{ok,[{1,a},{1,b},{1,c}]}} > > I made no real effort to optimize anything. The module starts > a gen_server which keeps a connection open to ttserver. It handles > only one query at a time, but looking at the TCP protocol, it's > hard to see how it could to otherwise, as there is no tagging of > requests. The round trip times are going to be fairly high for simple > requests (compared to dets and mnesia on small data sets), but the > main benefit of using TT in the first place ought to be either that > the data set is uncomfortably large for mnesia and dets, or that one > wants ordered_set semantics on disk-based storage. > > I put the tt_proto module in sext/examples/ > There is some edoc for it too. > > http://svn.ulf.wiger.net/sext/trunk/sext/doc/index.html > > BR, > Ulf W > > ________________________________________________________________ > erlang-questions mailing list. See http://www.erlang.org/faq.html > erlang-questions (at) erlang.org > >